Subject: Automatic routing: Thoughts, Questions, and Designs

From: John Goerzen <[email-masked]>
Date: Thu, 12 Oct 2023 11:54:33 -0500

Hi,

I am making up some plans for developing an automatic routing system for
larger-than-personal NNCP systems.

The basic idea is that nodes can use nncp-exec to automatically
distribute updates about the immediate peers they know of, and what
nodes those peers can route for, to their peers.  This would let any
given node compute the single next step for a packet (one entry in
-via), or, alternatively, the entire path to a destination (-via with
many items).

I hope to code this up before long and test it locally.

The ultimate idea is that NNCP users should be able to send email to
[email-masked] without having to have previous knowledge of it or
direct knowledge of how to route a packet there.  This is helped by the
current ability of NNCP to let you either specify an entire end-to-end
path in -via, or to specify a partial path (eg, just to the next hop,
and that hop calculates further hops).

Already without changes in NNCP, I can, therefore, support automatic
routing where the destination is known to us, even if many of the
intermediaries are not (subject to a caveat about route calculation
below).

Question: if a destination is not given in nncp.hjson, but its -via step
is, can we send a packet to it simply by giving its full node ID as the
destination?  That is, if the node ID is the public key, that should
theoretically work, right?

Now in order for this all to really work, at least in the "receive mail
from strangers" sense, there are two missing pieces.  The first is being
able to receive packets from unknown nodes.  Maybe a catch-all default
in the "neigh" section?  (This would be similar to the support that
already exists in the multicast area setup).

I almost wonder if we could work around this if everyone has the same
multicast area defined, with an empty subs, and then could run commands
like this:

nncp-file -via smarthost,desthost file.txt gloablarea:file.txt

The idea here is that nothing realizes it's an area packet except the
desthost.  I'm not sure if this actually works this way though.

The second missing piece is sort of an optional nice-to-have.  Having
this bolt-on tool is easy enough to use when generating a nncp-exec or
nncp-file command line.  Because we could just use -via to the first hop
and trust that hop to figure out the rest of the path, no mods to
nncp.hjson would be necessary on the originating node.

But at the next hop, there's the problem of there being no easy way to
drop in a custom routing decision.  We'd have to generate and maintain
many, many entries in nncp.hjson.

The workaround is to distribute deep pictures of the network from each
node to its peers, so the originating node would generate the entire
-via path, end to end.  That works, but is significantly less efficient
at a large scale.  But I could certainly prototype something along these
lines.

In short, there's a lot I can do within NNCP's existing system.  The
goal would be to expand on what I'm doing with quux, to make it more
distributed and replicable to other hub nodes -- maybe even publish a
Docker container or something for those that want to easily stand up a
NNCP public peer somewhere.

- John