Subject: Re: Possible distant future cryptography and format changes

From: John Goerzen <[email-masked]>
Date: Tue, 07 Oct 2025 08:31:30 -0500

On Fri, Oct 03 2025, Sergey Matveev wrote:

> Greetings!
>
> *** John Goerzen [2025-10-02 19:11]:
>>That looks interesting!  I have been using MessagePack in a number of
>>contexts, such as Filespooler https://www.complete.org/filespooler/.
>
> I love MessagePack and it is the most preferred format for me in most
> cases. But it does not support streaming (field's length must be known

Very true.  I often have wrapped it in frames, with a leading u64 length
indicator.  Somewhat annoying, but it has generally been OK for me to
put the large payload, whatever it is, outside of the MessagePack and in
a frame.

> in advance) and does not force deterministic encoding (but in practice I
> won't be surprised that most implementations have strict behaviour),
> that was not perfect for embedded systems. So I just have not found good
> enough existing format. MessagePack won't be a problem for NNCP use-cases,
> but cryptographic messages (KEKS/CM) are already defined and Go
> implementation covers a big subset of them.

Can you send me a pointer to this definition and Go implementation?  I
haven't been able to dig it up.

>>With crypto code, that is a stronger bias since there is something of a
>>long history of (hopefully unintentional) weaknesses being discovered in
>>crypto code.  I know TLS folks are doing ML-KEM these days.
>
> Agreed. But I am biased against ML-KEM, because of DJB's:
> https://ntruprime.cr.yp.to/faq.html
> https://blog.cr.yp.to/20231003-countcorrectly.html
> NIST has pretty bad reputation (especially after Dual EC DRBG), so its
> decisions must be taken with high level of awareness. And, maybe most
> importantly, SNTRUP was used in practice a long before Kyber/ML-KEM,
> because of its inclusion in OpenSSH. It is just more mature and seems to
> have higher security level than ML-KEM. According to various maillists,
> many cryptographers are far from being satisfied with ML-KEM and they
> want to see at least FrodoKEM (which was chosen by German's BSI) and
> Classical McEliece more.

Thanks for the links & info!  I am not at all well-versed in the
post-quantum algorithms, so this is helpful.

>
>>I got the NNCP packet parsers and generators done [...]
>>However, the current format has some notable limitations, particularly
>>the 255-byte limits.
>>The Filespooler packet format is given here: [...]
>
> Yeah, I understand a will to implement all of that. Current NNCP's
> format can be treated like a fixed structures, nothing more. No
> flexibility.
>
> Honestly I lost my interest in developing NNCP. It just works for me (I
> use it extensively everyday). But I am already satisfied with only a part
> of its functionality. Many features I have not used maybe since their
> addition. I have got many items in my TODO list, but they are not
> important personally for me. Adding PQ-cryptography, as an
> counterexample :-), is an interesting task, so I will try to find time
> for it. Adding or even fixing of some other issues... it is boring,
> but of course they should be done. Maybe that is some kind of burnout.

That is totally understandable!  I've moved on from a number of my
earlier projects also, sometimes revisiting them years later.

I'm curious what other items are on your TODO list?

> NNCP get much attention (mainly by your, John, work) when my interest
> was already declining :-). I even have not updated FreeBSD's port for a
> long time. Recently I tried to update it, but it has some port-ecosystem
> related issues that I am not too enthusiastic to debug and fix.

Totally understandable.  Maybe this will finally get me to learn Go :-)

FWIW, I process around a thousand packets a day using NNCP for backups
(every ZFS dataset on every machine, every hour).  It is fantastic and
Just Works!

>>I would like to add a vote for compatibility [...]
>>Perhaps it could be accomplished by having upgraded nodes generate both
>>old-style keys and new-style ones. [...]
>
> Agreed. But... I am not sure that backward compatibility will stay.
>
> One of the annoying things in NNCP is its configuration format. It is
> not friendly for machine editing. Hjson can be converted to JSON and
> JSON can be edited by jq-like utilities, but that is not too convenient.
> I do not like the fact of having two completely different configuration
> formats: (h)json and "cfgdir" one, that has a huge hard-coded mapping

It's a bit janky, but it seems to work well enough for NNCPNET.  I
mostly use cfgdir for that.

> how to store each field of the configuration. So I am also thinking
> about replacing all of that with something different, especially when
> huge McEliece post-quantum keys will be definitely outside the
> configuration file. That is just a coincidence, but recently I wrote

Yep, that makes sense.

> "damn small configuration" tool, that some kind mimics OpenWRT's like
> UCI configuration tool experience, keeping all the data in a directories
> hierarchy, but also allowing exporting/importing all of that in a single
> text txtar-file.
> http://www.git.cypherpunks.su/?p=dsc.git;a=blob_plain;f=dsc;h=837942ba3c2e0c0c9511be721afb92c597f2e78b8bd0d0d56a99335b47fea072;hb=HEAD
> But maybe it is bad idea. Even does not have enough time to think about it.

There are a number of libraries for dealing with the old Windows .ini
format.  Python and Rust have one named configparser, for instance.
Most of these have support for both reading and writing.  It's a
reasonable format, better than all the YAML ones :-)

> But if configuration format will change, then newer NNCP will transform
> to something completely incompatible, starting from configuration file
> format, to different format of all kinds of packets and online protocol.
> Keeping an old and new implementations will be like keeping two
> different programs together. But I have no plans in changing nncp-*
> command's API or any other logic.

Just a comment - a different config file format doesn't necessarily mean
the other things have to change.  Though maybe it is the other changes
that would drive the different config file format.

Thanks!

- John