public inbox for nncp-devel@lists.stargrave.org Atom feed
* So I made a thing: nncp-echomail @ 2025-10-04 13:52 Eugene Medvedev 2025-10-04 16:56 ` gemtext (was: So I made a thing: nncp-echomail) Sergey Matveev 0 siblings, 1 reply; 9+ messages in thread From: Eugene Medvedev @ 2025-10-04 13:52 UTC (permalink / raw) To: nncp-devel I have mentioned previously that I've been working on a system using NNCP areas. So here it is, an echomail/forum-like system relying on NNCP areas for routing, deliberately designed to be as simple to code for as possible: https://github.com/Mihara/nncp-echomail This is a prototype. It will work, but there are very likely corner cases I didn't think of, things that are probably a bad idea someone will inevitably point out, and there's no mail editor yet. I plan to cook something up in the next few weeks, but maybe someone will beat me to it. As of today, this is by nature not resistant to sender spoofing: NNCP does not yet expose the actual sender ID to execs receiving packets from areas. Something that can hopefully be fixed, though. Please discuss. P.S. Yes, you can Usenet and email over NNCP, The plan is to get the areas some love, though, and rely on them for routing. -- Eugene Medvedev ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 13:52 So I made a thing: nncp-echomail Eugene Medvedev @ 2025-10-04 16:56 ` Sergey Matveev 2025-10-04 17:51 ` Eugene Medvedev 2025-10-15 20:46 ` Eugene Medvedev 0 siblings, 2 replies; 9+ messages in thread From: Sergey Matveev @ 2025-10-04 16:56 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 6961 bytes --] *** Eugene Medvedev [2025-10-04 16:52]: >So here it is, an echomail/forum-like system relying >on NNCP areas for routing [...] Years ago it was exactly that type of thing I wanted to exist in practice! I loved FidoNet's echomail and have never felt anything in Internet had the same feeling (although I have never used Usenet). Actually all that "area" terms in NNCP exactly referred to FTN world. >As of today, this is by nature not resistant to sender >spoofing: NNCP does not yet expose the actual >sender ID to execs receiving packets from areas. >Something that can hopefully be fixed, though. Yeah, that should be fixed easily, as that information actually exists in packets/NNCP-utilities. I liked the idea of <msgid>/<filename> links! ------------------------ >8 ------------------------ I just wanted to add my humble opinion about gemtext format. This is not a critique or suggestion, just an opinion and a thought. >While I will be the first to criticize Gemtext as a markup format, *(I love my inline italics!)* three important considerations exist: >1. It is *trivial* to parse. >2. A lot of open source software already exists to handle it, some of which can be badgered into being a full-fledged echomail editor with a little work. >3. The prospective audiences of Gemini and NNCP already intersect to a significant degree. Personally I dislike gemtext very much, because it is *not* human readable. It is machine-readable format. It suggests you to write long lines, that are not (conveniently) readable by humans. You have to use some utility to reformat them to shorter and readable ones. But that utility must be aware about preformatted text blocks and should not touch them. So you have to use something more advanced and complex than ordinary "fmt" utility. And all those "## headings" are not clearly visible, not very distinctive, compared to: ============ that heading ============ Both MUAs and FTN message editors generally just show the plain text they got from email or netmail message. That plain text is immediately can be read/consumed by the human. And I think that is the most important capability of text messages. gemtext (or Markdown) are some kind of the source code, machine readable format, requiring the renderer to view them *conveniently*. A while ago I created "SWG" format: http://www.git.stargrave.org/?p=swg.git;a=blob;f=README that virtually has no formatting at all. It is just a plain text, so format as you fish, make it consumable without any additional software. But we often want to insert links. And we often want to insert links to images, explicitly marking them as a link to image (even gopher:// protocol can do that, unlike gemtext, where you have to click on every link to (possible, you can not be 100% sure) an image). So in SWG you insert "=> URL[ optional text]" or "img URL[ optional text][ => URL-A]" lines, but end them with LF character (can be inserted in Vim with Ctrl-V + Enter, as an example). Ordinary text won't have a mix of CR and CRLF line endings. Ordinary cat/less/more will show those lines as an ordinary ones. But a machine can easily distinguish them from ordinary text ones. http://www.git.stargrave.org/?p=swg.git;a=blob;f=FORMAT That gives more flexible documents, even much more simpler parsing, ability to insert images (noone forces you to automatically load them, as you wish). Of course there is no difference between "flowed" (possible variable width) text and preformatted one -- it is just a ready to view plaintext "document". I think gemtext is a better choice than Markdown, that has huge specification, many features and a full complete 100% assurance that noone supports them all and your ".md" document will work only if it uses a tiny subset of the MD/CM. As many people say: there is no such format as MD, but a huge zoo of partly compatible set of implementations. And hardly anyone supports CM. That is why all projects, which maillists I am subscribed, does not even think about MD/CM as a possible choice for non-trivial (plaintext is enough for trivial one)documentation. But gemtext can not be used instead of a simple HTML (or even Gopher's menus!), containing internal links (anchors) and images. But SWG in my practice is pretty good at that, being much simpler thing. The only argument I heard about preformatted lines is that they look bad on smartphones. And because of those things existence we have to bury plain text formats and use machine readable ones, which require the renderer? So not an argument for a user of the normal personal computers. ------------------------ >8 ------------------------ By the way, just curious, why you used SHA-256 as a hash? There are so many much better (cryptographic) hash algorithms appeared for the last ~25 years! SHA-256 is literally the slowest one among any seriously used nowadays. SHA2-512 is much faster on 64-bit platforms, which are prevailing for our use-cases. Skein (one of the SHA3 finalists) is much faster and have considerable higher security margin. BLAKE2 (which is based on BLAKE -- another SHA3 finalists) is even much more faster than any of these, also having a great (much higher than SHA2) security margin. None of them have problems with length-extension attack, which you have to always remember about when using SHA1/2. Pure SHA3 (Keccak) may be a bit slower than SHA2, but it has less instructions it uses, it is simpler to implement, and "leaks" less on side-channel, due to lack of complex instructions. But both Keccak authors, and NIST itself, recommends to use SHAKE variant of SHA3 (if you are not forced to use pure SHA3), which is faster than SHA2, keeping all other advantages too. Not so much modern CPUs have SHA2 accelerated instructions, but Keccak is already included in more CPUs, making it even faster than BLAKE2. SHA2 is still secure (unlike SHA1), but for example BLAKE2 is available out-of-box in many "standard" libraries (Python added it a long time ago, as an example), SHA3 is also pretty widespreadly available. In virtually all modern cryptographic protocols/formats/algorithms where the hash is used, cryptographers choose either SHAKE (maybe SHA3), or BLAKE2 in majority of cases if they are not required to use NIST-approved hashers. I know, I know, that there are still many people using MD5, ignoring the fact that BLAKE2, especially BLAKE3 or accelerated Keccak a much more faster (even not mentioning the security), possibly there are many people using SHA1, but... SHA2 is another thing that should be replaced, at least because of the worst performance. Just my humble opinion of course. -- Sergey Matveev (http://www.stargrave.org/) LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 265 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 16:56 ` gemtext (was: So I made a thing: nncp-echomail) Sergey Matveev @ 2025-10-04 17:51 ` Eugene Medvedev 2025-10-04 18:20 ` Sergey Matveev 2025-10-15 20:46 ` Eugene Medvedev 1 sibling, 1 reply; 9+ messages in thread From: Eugene Medvedev @ 2025-10-04 17:51 UTC (permalink / raw) To: nncp-devel On Sat, 4 Oct 2025 at 19:58, Sergey Matveev <stargrave@stargrave•org> wrote: > I liked the idea of <msgid>/<filename> links! Initially, I actually wanted to send attachments separately. Further meditation on security eventually disabused me of this notion. :) I also considered using CBOR as the envelope format, but then realized it probably breaks the "implement in bash on a potato if you have to" constraint. I really don't want to be the only guy writing a mailer. :) > Personally I dislike gemtext very much, because it is *not* human > readable. It certainly has problems, most of them stemming from the fact that it was explicitly designed to be machine readable as easily as possible. It's a replacement for HTML, not Markdown, and not a very good one at that. Gemtext holy wars have been raging in the smol web since its inception. :) Still, I feel it was the best available alternative for this particular application. I could potentially introduce a header field specifying that the rest of the body is to be rendered as plain text? This might even work with the indexer as is, as long as text messages are then saved into ".txt" instead of ".gmi" (so that they get the text mime type) and the index links match... > Both MUAs and FTN message editors generally just show the plain text > they got from email or netmail message. That plain text is immediately > can be read/consumed by the human. ...But what about the cyrillic N? ;) > By the way, just curious, why you used SHA-256 as a hash? I am not a cryptographer. :) I needed something that fit those properties: 1. Available in as many language's standard libraries as feasible. 2. Sufficiently collision resistant so that people don't casually try to forge messages. 3. Not too long when represented as text, so that the "ReplyTo: " line in the header doesn't slide off screen. :) I don't think performance is a serious consideration: 1. Hash is only necessarily computed when receiving mail. There's hardly any reason to ever do it again. The average length of a message is not likely to ever exceed ~10 Kb, too. 2. If this project achieves 100+ messages a month anywhere, I'll be extremely happy and go buy a lottery ticket or twenty. SHA-512 was a bit too long (longer filenames, longer everything, and we get at least two layers - <group>/<msgid>...). SHA-256 seemed just right, even represented as hex, and then I stumbled on the idea of using base64 as per RFC4648, so this is what it uses now. I considered calling nncp-hash instead, since that is probably guaranteed to exist on the same machine the mailer is running on, but eventually decided that spawning processes left and right will introduce more room for error. It's not too late to pick something else. -- Eugene Medvedev ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 17:51 ` Eugene Medvedev @ 2025-10-04 18:20 ` Sergey Matveev 2025-10-04 18:39 ` Eugene Medvedev 0 siblings, 1 reply; 9+ messages in thread From: Sergey Matveev @ 2025-10-04 18:20 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 3944 bytes --] *** Eugene Medvedev [2025-10-04 20:51]: >Initially, I actually wanted to send attachments separately. >Further meditation on security eventually disabused me of this >notion. :) I also think that it will complicate things. >I also considered using CBOR as the envelope format, >but then realized it probably breaks the "implement in bash on a >potato if you have to" constraint. And again my humble opinion, that it is better to stick with MessagePack or maybe Netstrings (http://cr.yp.to/proto/netstrings.txt). But you already introduced separate index of attachments, that is also simple and does not require binary encoder. >Gemtext holy wars have been raging in the smol web since its >inception. :) Seems so. I am the person who dislike the whole technical side of gemini ecosystem. But I am still glad people trying to use *relatively* (oh that gemini:// TLS requirement!) simple solutions for communicating with each other. >Still, I feel it was the best available alternative for this >particular application. I could potentially introduce a header >field specifying that the rest of the body is to be rendered >as plain text? Well, it is your decision. I am afraid that those tiny steps could lead to reimplementation of the MIME. The most hard thing is to stop at good point of features and development :-) >...But what about the cyrillic N? ;) Actually I have never experienced problems with it, or seen someone has. But I get in FidoNet when it was already declining, at the beginning of 2000s, with much mature software :-) >1. Available in as many language's standard libraries as feasible. If BLAKE2 is not so widely available (Go's crypto library does not have it, for example, but golang.org/x/crypto had for a long time), then SHA3/SHAKE in my opinion is much better choice. Just recently I discovered that even BusyBox has "sha3" utility (but not SHAKE :-(). >2. Sufficiently collision resistant so that people don't casually try to >forge messages. That applies to all cryptographic hashes of course, except for found to be broken and weak ones (MD5 and SHA1). >3. Not too long when represented as text, so that the "ReplyTo: " line >in the header doesn't slide off screen. :) 256-bits are enough, no doubts. >1. Hash is only necessarily computed when receiving mail. There's >hardly any reason to ever do it again. The average length of a >message is not likely to ever exceed ~10 Kb, too. Agreed. It is not so critical in all those use-cases. >2. If this project achieves 100+ messages a month anywhere, I'll be >extremely happy and go buy a lottery ticket or twenty. Being the free software developer myself, understand you :-) >SHA-512 was a bit too long (longer filenames, longer everything, and >we get at least two layers - <group>/<msgid>...). You can safely shorten its output, as often done in practice. ZFS does that (one example immediately remembered). You get faster speeds on most computers people use, but still can use only 256-bits of its output. I also won't use full 512-bit output. >SHA-256 seemed just right Like BLAKE2b-256, BLAKE2s, SHA3-256, SHAKE128, or SHA2-512/2 :-) But SHA3 is available in most (all?) libraries already. Nearly all (completely all?) modern protocols which require NIST-approved hashes, moved from SHA2 (to SHAKE or SHA3). If NIST is not required, then BLAKE2 is very popular choice too, that is why it was added to Python's hashlib. >using base64 as per RFC4648, so this is what it uses now. Yeah, that is convenient decision. In most cases I prefer exactly that base64url too. In NNCP I choose Base32 because it looks nicer and still is relatively short enough, but only because it is visible on file system (however base64url is safe to use too). -- Sergey Matveev (http://www.stargrave.org/) LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 265 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 18:20 ` Sergey Matveev @ 2025-10-04 18:39 ` Eugene Medvedev 2025-10-04 19:23 ` Sergey Matveev 0 siblings, 1 reply; 9+ messages in thread From: Eugene Medvedev @ 2025-10-04 18:39 UTC (permalink / raw) To: nncp-devel On Sat, 4 Oct 2025 at 21:21, Sergey Matveev <stargrave@stargrave•org> wrote: > Well, it is your decision. I am afraid that those tiny steps could lead > to reimplementation of the MIME. The most hard thing is to stop at good > point of features and development :-) Yes, I suppose so. :) Well, let's see if anyone else around here has any input, I expect people will start peeking in with feature requests any day now... > Actually I have never experienced problems with it, or seen someone has. > But I get in FidoNet when it was already declining, at the beginning of > 2000s, with much mature software :-) I've been there for most of the 90s. FTN standards at the time specified the use of 0x8D as something equivalent to ­ in HTML, so it would simply disappear upon rendering everywhere. If you didn't tune your software to replace it with 0x48, everyone would call you a lamer. It did not do anything of the sort by default. :) Getting tripped up by that was something of a rite of passage. > You can safely shorten its output, as often done in practice. ZFS does > that (one example immediately remembered). Shorten as in what, just take the first 32 bytes? Are they all actually equally significant? -- Eugene Medvedev ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 18:39 ` Eugene Medvedev @ 2025-10-04 19:23 ` Sergey Matveev 2025-10-04 19:34 ` Eugene Medvedev 0 siblings, 1 reply; 9+ messages in thread From: Sergey Matveev @ 2025-10-04 19:23 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 1459 bytes --] *** Eugene Medvedev [2025-10-04 21:39]: >I've been there for most of the 90s. FTN standards at the time specified >the use of 0x8D as something equivalent to ­ in HTML, so it would >simply disappear upon rendering everywhere. If you didn't tune your >software to replace it with 0x48, everyone would call you a lamer. Yeah, I am aware about the etymology of that joke with "Н" :-) Seems that at my time all software was tend to be correctly configured. >> You can safely shorten its output, as often done in practice. ZFS does >> that (one example immediately remembered). > >Shorten as in what, just take the first 32 bytes? Are they all actually >equally significant? In practice, in case of SHA2 (and I believe in all other cryptographic hashes we use in practice), yes. They has uniform distribution. Ideally you would use SHA2-512/256 variant, where SHA2's state is initialised with different "magic numbers" to separate the context of ordinary SHA2-512 from SHA2-512/256, because if you happen to get SHA2-512, then you can "calculate" its stripped variant. With proper SHA2-512/256 hashes are not "interexchangable". But in most cases, especially in pretty trivial like nncp-echomail, we can ignore that paranoid context separation and just use 32 bytes of SHA2-512 (does not matter first half or second). -- Sergey Matveev (http://www.stargrave.org/) LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 265 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 19:23 ` Sergey Matveev @ 2025-10-04 19:34 ` Eugene Medvedev 0 siblings, 0 replies; 9+ messages in thread From: Eugene Medvedev @ 2025-10-04 19:34 UTC (permalink / raw) To: nncp-devel On Sat, 4 Oct 2025 at 22:25, Sergey Matveev <stargrave@stargrave•org> wrote: > Ideally > you would use SHA2-512/256 variant, where SHA2's state is initialised > with different "magic numbers" to separate the context of ordinary > SHA2-512 from SHA2-512/256, because if you happen to get SHA2-512, then > you can "calculate" its stripped variant. That, actually, was what I just changed it to upon reading up a bit on how this is done in ZFS. I figure this is a sufficiently happy medium for now. While there is no SHA-512/256 in coreutils, openssl dgst -sha512-256 should be available enough to use as the last resort... -- Eugene Medvedev ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: gemtext (was: So I made a thing: nncp-echomail) 2025-10-04 16:56 ` gemtext (was: So I made a thing: nncp-echomail) Sergey Matveev 2025-10-04 17:51 ` Eugene Medvedev @ 2025-10-15 20:46 ` Eugene Medvedev 2025-10-26 11:13 ` NNCP_PACKET and NNCP_ORIGIN Sergey Matveev 1 sibling, 1 reply; 9+ messages in thread From: Eugene Medvedev @ 2025-10-15 20:46 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 300 bytes --] On Sat, 4 Oct 2025 at 19:58, Sergey Matveev <stargrave@stargrave•org> wrote: > Yeah, that should be fixed easily, as that information actually exists > in packets/NNCP-utilities. So I took a stab at this, patch enclosed. Thoughts? I might have easily misunderstood something. -- Eugene Medvedev [-- Attachment #2: origin.patch --] [-- Type: text/x-patch, Size: 1248 bytes --] diff -r -u nncp-8.13.0/src/node.go nncp-8.13.0-patched/src/node.go --- nncp-8.13.0/src/node.go 1981-01-01 03:00:00.000000000 +0300 +++ nncp-8.13.0-patched/src/node.go 2025-10-15 22:56:13.944830866 +0300 @@ -57,6 +57,7 @@ OnlineDeadline time.Duration MaxOnlineTime time.Duration Calls []*Call + Origin *NodeId Busy bool sync.Mutex diff -r -u nncp-8.13.0/src/toss.go nncp-8.13.0-patched/src/toss.go --- nncp-8.13.0/src/toss.go 1981-01-01 03:00:00.000000000 +0300 +++ nncp-8.13.0-patched/src/toss.go 2025-10-15 23:06:32.799861865 +0300 @@ -185,10 +185,16 @@ } if !opts.DryRun { cmd := exec.Command(cmdline[0], append(cmdline[1:], args...)...) + origin := sender.Id.String() + if sender.Origin != nil { + origin = sender.Origin.String() + } cmd.Env = append( cmd.Env, "NNCP_SELF="+ctx.Self.Id.String(), "NNCP_SENDER="+sender.Id.String(), + "NNCP_PACKET="+pktName, + "NNCP_ORIGIN="+origin, "NNCP_NICE="+strconv.Itoa(int(pkt.Nice)), ) if pkt.Type == PktTypeExec { @@ -903,6 +909,7 @@ Name: area.Name, Incoming: area.Incoming, Exec: area.Exec, + Origin: pktEnc.Sender, } copy(areaNode.Id[:], area.Id[:]) pktName := fmt.Sprintf( ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: NNCP_PACKET and NNCP_ORIGIN 2025-10-15 20:46 ` Eugene Medvedev @ 2025-10-26 11:13 ` Sergey Matveev 0 siblings, 0 replies; 9+ messages in thread From: Sergey Matveev @ 2025-10-26 11:13 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 369 bytes --] Greetings! *** Eugene Medvedev [2025-10-15 23:46]: >So I took a stab at this, patch enclosed. >Thoughts? I might have easily misunderstood something. I did not extensively check your patch, but it seems good. Applied in "develop" branch. Thanks! -- Sergey Matveev (http://www.stargrave.org/) LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 265 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-10-26 11:13 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-10-04 13:52 So I made a thing: nncp-echomail Eugene Medvedev 2025-10-04 16:56 ` gemtext (was: So I made a thing: nncp-echomail) Sergey Matveev 2025-10-04 17:51 ` Eugene Medvedev 2025-10-04 18:20 ` Sergey Matveev 2025-10-04 18:39 ` Eugene Medvedev 2025-10-04 19:23 ` Sergey Matveev 2025-10-04 19:34 ` Eugene Medvedev 2025-10-15 20:46 ` Eugene Medvedev 2025-10-26 11:13 ` NNCP_PACKET and NNCP_ORIGIN Sergey Matveev