Subject: Re: NNCP at scale report

From: John Goerzen <[email-masked]>
Date: Sat, 02 Jan 2021 14:39:35 -0600

On Sat, Jan 02 2021, Sergey Matveev wrote:

> Greetings!
>
> *** John Goerzen [2021-01-01 21:09]:
>>I thought I would summarize how this is working for me!
>
> Thanks for sharing your interesting experience! I have got much 
> lower
> amounts of data transferred (but still many gigabytes) and less 
> nodes.
> All email is passing through NNCP for many years. "chunked" 
> transfers is
> done very often. But do not remember when I used nncp-bundle the 
> last
> time (but I like that creation :-))

Yes, nncp-bundle is a very nice idea.  I haven't used it much but 
it has definite potential.  Streaming formats are always great.

I'd be interested to hear more about your use case for chunked 
transfers.  I guess their main use would be if individual files 
are too large for sneakernet?  With the resumable transfers over 
the network, though, I haven't bothered.

>>- One of the machines is the spooler.  I want to use the same 
>>code  as
>>everything else, so I use nncp-file to send to self. 
>>Unfortunately nncp-toss
>>doesn't see that.  The workaround is to  call nncp-xfer 
>>/var/spool/nncptmp
>>twice.  It moves the files out  of tx, then into rx, and then it 
>>works :-)
>
> Or you can just "mv path/to/self/tx/* path/to/self/rx/".

I wondered, but then I wasn't sure if there would be race 
conditions with temporary files, half-written files, etc., and I 
didn't want to risk it.

>>- There are scenarios in which it may be helpful to redirect 
>>packets.
>
> Technically this is possible: just wrap the existing packet to
> transition wrapper and encrypt it. But current tools do not do 
> that.
> Think this is small niche, but it is not hard to to.
>
> Also there is ability to override "via" via "-via" command line 
> option,
> ignoring hard-coded configuration. But of course it is done only 
> for
> newly created packets.

My thought was this...  when I travel (hopefully we can do that 
again this year!) my laptop would, when on a good network, 
continue to send backups to the spooling server at my house. 
However, sometimes something happens - power outage, Internet 
outage, etc.  In that case I would rather have the backups -- 
including the ones previously generated -- go spool on my server 
where they could sit until the home network comes back up.

I also had the thought -- well I could probably just manually copy 
them over to the appropriate tx queue on the server.  Or if 
nncp-xfer or nncp-bundle would have a feature where they would not 
ignore, but instead ingest, packets whose next hop is any node 
that the machine knows about, I could always just send them down a 
nncp-bundle pipe.

> Well, I think that it is preferrable people to learn more of 
> Unix
> administration, because it will bring much more benefit. When I 
> have

As I think about this, you're probably right as far as whether 
it's worth it to spend precious dev time on.  I mean, it is very 
nice to have a self-contained "maintenance free" program, but 
people using NNCP are unlikely to be newbies at all this anyhow.

What might be useful would be a "maintenance" page in the manual, 
just listing things that a person might do: rotate the log file, 
look for old packets, look for old seen files.

Anyhow, if I knew Go I would totally be contributing patches at 
this point.

If I had a few more weeks of vacation, I might try to learn it 
just for this :-)

- John