@falsifian@www.falsifian.org Right I see. Yeah maybe we want to avoid that 🤣 I do kind of tend to agree with @xuu@txt.sour.is in another thread that there isn’t actually anything wrong with our use of Blake2 at all really, but we may want to consider all our options.

@xuu@txt.sour.is I don’t think this is a lextwt problem tbh. Just the Markdown aprser that yarnd currently uses. twtxt2html uses Goldmark and appears to behave better 🤣

@xuu@txt.sour.is Long while back, I experimented with using similarity algorithms to detect if two Twts were similar enough to be considered an “Edit”.

Right I see what you mean @xuu@txt.sour.is – Can you maybe come up with a fully fleshed out proposal for this? 🤔 This will help solve the problem of hash collision that result from the Twt/hash space growing larger over time without us having to change anything about the way we construct hashes in the first place. We just assume spec compliant clients will just dynamically handle this as the space grows.

@xuu@txt.sour.is I think we never progressed this idea further because we weren’t sure how to tell if a hash collision would occur in the first place right? In other words, how does Client A know to expand a hash vs. Client B in a 100% decentralised way? 🤔

Plus these so-called “LLM”(s) have a pretty good grasp of the “shape” of language, so they appear to be quite intelligent or produce intelligible response (when they’re actually quite stupid really).

@eldersnake@we.loveprivacy.club You don’t get left behind at all 🤣 It’s hyped up so much, it’s not even funny anymore. Basically at this point (so far at least) I’ve concluded that all this GenAI / LLM stuff is just a fancy auto-complete and indexing + search reinvented 🤣

Getting a little sick of AI this, AI that. Yes I’ll be left behind while everyone else jumps on the latest thing, but I’m not sure I care.

Oh. looks like its 4 chars. git show 64bf

@prologic@twtxt.net where was that idea?

i feel like we should isolate a subset of markdown that makes sense and built it into lextwt. it already has support for links and images. maybe basic formatting bold, italic. possibly block quote and bullet lists. no tables or footnotes

the stem matching is the same as how GIT does its branch hashes. i think you can stem it down to 2 or 3 sha bytes.

if a client sees someone in a yarn using a byte longer hash it can lengthen to match since it can assume that maybe the other client has a collision that it doesnt know about.

@prologic@twtxt.net the basic idea was to stem the hash.. so you have a hash abcdef0123456789... any sub string of that hash after the first 6 will match. so abcdef, abcdef012, abcdef0123456 all match the same. on the case of a collision i think we decided on matching the newest since we archive off older threads anyway. the third rule was about growing the minimum hash size after some threshold of collisions were detected.

@bender@twtxt.net This is the different Markdown parsers being used. Goldmark vs. gomarkdown. We need to switch to Goldmark 😅

@prologic@twtxt.net yes, like they show here: https://ferengi.one/#uebsf7a

@quark@ferengi.one i’m guessing the quotas text should’ve been emphasized?

@slashdot@feeds.twtxt.net NahahahahHa 🤣 So glad I don’t use LinkedIn 🤦‍♂️

@falsifian@www.falsifian.org No u don’t sorry. But I tend to agree with you and I think if we continue to use hashes we should keep the remainder in mind as we choose truncation values of N

@falsifian@www.falsifian.org Mostly because Git uses it 🤣 Known attacks that would affect our use? 🤔

@xuu@txt.sour.is I don’t recall where that discussion ended up being though?

@bender@twtxt.net wut da fuq?! 🤣

@xuu@txt.sour.is you mean my original idea of basically just automatically detecting Twt edits from the client side?

@xuu@txt.sour.is this is where you would need to prove that the editor delete request actually came from that feed author. Hence why integrity is much more important here.

@falsifian@www.falsifian.org without supporting dudes properly though you’re running into GDP issues and the right to forget. 🤣 we’ve had pretty lengthy discussions about this in the past years ago as well, but we never came to a conclusion. We’re all happy with.

@movq@www.uninformativ.de it would work, you are right, however, it has drawbacks, and I think in the long term would create a new set of problems that we would also then have to solve.

@david@collantes.us Hah 🤣

@david@collantes.us We’ll get there soon™ 🔜

@david@collantes.us Hah Welcome back! 😅

@david@collantes.us “Hello back” from the other corner of the world! 🫡

Incredibly upset—more than you could imagine—because I already made the first mistake, and corrected it (but twtxt.net got it on it’s cache, ugh!) :‘-( . Can’t wait for editing to become a reality!

Everything starts at a “hello world”. At least around these parts; the nerdy parts.

@movq@www.uninformativ.de going a little sideways on this, “*If twtxt/Yarn was to grow bigger, then this would become a concern again. But even Mastodon allows editing, so how much of a problem can it really be? 😅*”, wouldn’t it preparing for a potential (even if very, very, veeeeery remote) growth be a good thing? Mastodon signs all messages, keeps a history of edits, and it doesn’t break threads. It isn’t a problem there.😉 It is here.

I think keeping hashes is a must. If anything for that “feels good” feeling.

There is nothing wrong with how we currently run a diff to see what has been removed. if i build a merkle tree off all the twt hashes in a feed i can use that to verify a twt should be in a feed or not. and gossip that to my peers.

(Or maybe I’m talking nonsense. That’s known to happen. I’ll go to bed. 😂)

So.. basically a rehash of the email “unsend” requests? What if i was to make a (delete: 5vbi2ea) .. would it delete someone elses twt?

Brisbane is coming onboard. Roosters are “singing” all around @prologic@twtxt.net, and the dog is begging for the morning poo/pee walk. @prologic@twtxt.net throws a slipper at the dog, as he turns around, and hides under his comforter.

😂😂😂

@quark@ferengi.one Printing a version? I’ll think about it. 🤔

It would be easy to do for releases, but it’s a little hard to do for all the commits in between – jenny has no build process, so there’s no easy way to incorporate the output of git describe, for example.

isn’t the benefit of blake2b that it is a more efficient algo than sha1 and has the same or similar entropy to sha3? i thought we had partially solved this with some type of expanding hash size? additionally we could increase bit density by using base36 or base64/url-safe…

I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.

The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.

I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.

If twtxt/Yarn was to grow bigger, then this would become a concern again. But even Mastodon allows editing, so how much of a problem can it really be? 😅

I do have to “admit”, though, that hashes feel better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.

Hm.

I suspect that the (replyto:…) proposal would work just as well in practice.

Hey, @movq@www.uninformativ.de, a tiny thing to add to jenny, a -v switch. That way when you twtxt “That’s an older format that was used before jenny version v23.04”, I can go and run jenny -v, and “duh!” myself on the way to a git pull. :-D

@falsifian@www.falsifian.org @prologic@twtxt.net @lyse@lyse.isobeef.org

• editing, if you don’t care about message integrity
  

So that’s the big question, because that’s the only real difference between hashes and the (replyto:…) proposal.

Do we care about message integrity?

With (replyto:…), someone could write a twt, then I reply to it, like “you’re absolutely right!”, and then that person could change their twt to something malicious like “the earth is flat!” And then it would look like I’m a nutcase agreeing with that person. 😅

Hashes (in their current form) prevent that. The thread is broken and my reply clearly refers to something else. That’s good, right?

But now take into account that we want to allow editing anyway. Is there even a point to using hashes anymore? Isn’t message integrity ignored anyway now, at least in practice?

There’s no difference (in practice) between someone writing

2024-09-18T12:34Z    Brds are great!

and then editing it to either

2024-09-18T12:34Z    (original:#12379) Birds are great!  (Whoops, fixed a typo.)

or

2024-09-18T12:34Z    (original:#12379) The earth is flat!

The actual original message is (potentially) gone. The only thing that we can be sure of now is that the twt was edited in some way. Essentially, the actual twt message is no longer part of the hash, is it? What does #12379 refer to? The edited message or the original one? We want it to refer to the edited one, because we don’t want to break threads, so … what’s the point of using a hash?

Regarding jenny development: There have been enough changes in the last few weeks, imo. I want to let things settle for a while (potential bugfixes aside) and then I’m going to cut a new release.

And I guess the release after that is going to include all the threading/hashing stuff – if we can decide on one of the proposals. 😂

@falsifian@www.falsifian.org Yeah, delete requests feel very odd.

@prologic@twtxt.net I wish that was true! But I reckon there is still heaps of old stuff out there, that was created on a Windows machine. :-D And I wouldn’t be surprised if even today in that environment a new file does not make use of UTF-8.

@quark@ferengi.one I’m not convinced. :-D

@quark@ferengi.one @movq@www.uninformativ.de Yep, they’re all RFC3339. Obviously, +02:00 and +01:00 are best, because I use them! :-P In all seriousness, Z might be the best timezone, as it is shortest. And regarding privacy, it leaks the least information about the user’s rough location. But of course, one can just look at the activity and narrow down plausible regions, so that’s a weak argument.

@movq@www.uninformativ.de You’re right! switching from zsh to bash gave me the same result zq4fgq Thanks!

@prologic@twtxt.net I wouldn’t want my client to honour delete requests. I like my computer’s memory to be better than mine, not worse, so it would bug me if I remember seeing something and my computer can’t find it.

@aelaraji@aelaraji.com Looks like your shell didn’t turn the \n into actual newlines:

$ echo -n "https://twtxt.net/user/prologic/twtxt.txt\n2020-07-18T12:39:52Z\nHello World! 😊" | openssl dgst -blake2s256 -binary | base32 | tr -d '=' | tr 'A-Z' 'a-z' | tail -c 7
zq4fgq
$ printf "https://twtxt.net/user/prologic/twtxt.txt\\n2020-07-18T12:39:52Z\\nHello World! 😊" | openssl dgst -blake2s256 -binary | base32 | tr -d '=' | tr 'A-Z' 'a-z' | tail -c 7
p44j3q

@prologic@twtxt.net I ran the same command and got an even different result xD

~ » echo -n "https://twtxt.net/user/prologic/twtxt.txt\n2020-07-18T12:39:52Z\nHello World! 😊" | openssl dgst -blake2s256 -binary | base32 | tr -d '=' | tr 'A-Z' 'a-z' | tail -c 7
p44j3q