Subject: Re: Avoiding double writes

From: John Goerzen <[email-masked]>
Date: Mon, 01 Nov 2021 19:11:31 -0500

Hi Sergey!

On Mon, Nov 01 2021, Sergey Matveev wrote:

> *** John Goerzen [2021-11-01 12:47]:
>>Awhile back, we were discussing the temporary files that were needed for
>>reading stdin for nncp-exec or nncp-file.  I believe the reason for this is
>>that the header contains a signature of the data that follows, and it's not
>>practical to seek back and write that later.
>
> http://www.nncpgo.org/Encrypted.html
> It is not because of signature, but mainly because of SIZE field.
> Signed header contains everything you need to authenticate remote side
> and create encryption keys to process ENCRYPTED data. There is not
> signature over the whole packet. ENCRYPTED data contains encrypted size
> as a first short block, and then pile of 128KiB blocks. Each block is
> AEAD-encrypted, so it is authenticated. SIZE holds the payload size,
> that can be smaller than the whole packet, because of added junk to hide
> the actual payload size.

Got it.

So, from a glance at the code, this size is primarily used for:

- differentiating encrypted data from padding

- display output

> We can store some structure inside each block (single signalling byte),
> telling is it the payload one, or junk one. Now we can process it
> sequentially. But junk in that case has to be some real data we will
> really encrypt and authenticate. Junk can be zeroes, but AEAD-encryption
> is much more expensive than current junk-generator made as BLAKE3-XOF
> output. Much complication, more expensive (CPU). Ok, we know that if
> junk block is started, then we can be sure that everything after will be
> the junk, so we have to authenticate only the single block with the junk
> and then quickly generate it without real AEAD-processing. Except for
> the very last block with the size.

Is it even really necessary to store a size?  I was thinking of
something along these lines also, with the signaling byte (or, perhaps
more accurately, a blocksize indicator).  For instance, each block would
begin with a size of the block of encrypted data, and after all the
encrypted data, a block size of zero could be used.  There would be no
need to explicitly give a size because the stream of blocks would
contain all the needed information.

But I'm also confused about the signature - since that comes before the
encrypted blocks, isn't that also a problem?

Thanks for the conversation,

John