Subject: Re: gvisor's 32-bit compatibility

From: Sergey Matveev <[email-masked]>
Date: Fri, 21 Jan 2022 11:24:17 +0300

Greetings!

*** John Goerzen [2022-01-20 15:45]:
>And according to the FAQ [1] , it is only supported on
>Linux on x86_64 (with "preliminary" support for arm64).

And indeed its latest commit even does not build on FreeBSD, so I used
one that was mentioned in golang.zx2c4.com/wireguard/tun/netstack.

>vendor/gvisor.dev/gvisor/pkg/tcpip/transport/tcp/snd.go:326:16: constant 9223372036854775807 overflows int
>
>That doesn't bode well..  It wouldn't surprise me at all if there are
>other issues that would keep it from working on 32-bit platforms, or
>non-Linux ones.

Probably and I won't be surprised too. But I would change constant in
snd.go from MaxInt64 to MaxInt32 and try to build it again. NNCP
requires relatively small part of gvisor's overall code, so I think
there is high probability that everything will work fine after.

Personally I do not try to support 32-bit systems too, as also many
other software. For example ZFS officially is not supported on non
64-bit systems. Even in Go 32-bit support is not so trivial thing to
reach: it is not only about "int" width expectations (as I thought some
years before), but for example sync/atomic clearly states that you have
to manually align fields in structure on 64-bit boundaries, otherwise
those atomic instructions may painfully fail. With 64-bit code you do
not care much about that alignment, because int, int64 and pointers are
all 64-bit and "automatically" aligned. It is just much easier to write
64-bit code. I try to make no expectations about "int" width and clearly
define int32/int64 where it is appropriate and important, but I do not
know how many other potential differences (like sync/atomic) there are.

>I don't know if there are other TCP implementations for Go that are
>usable.

I do not know too. I heard about gvisor from the comment in my blog :-)

>I do have NNCP 8.2.0 compiling on NetBSD, but whether it will work
>correctly, or keep working over time given the gvisor FAQ, is another
>question I guess (I don't have the environment to try that part yet).

I test in on FreeBSD. But pay attention that I use its some months older
version, mentioned in Wireguard (probably because of the same reasons).
Its latest one does not build on FreeBSD at all. But it does not make
its older version buggy on unable to work :-)

>I'd much rather
>have a uTP/TCP mismatch than sacrifice platform support in NNCP for
>sure.

I can add some kind of build switch (GO_CFLAGS="-tags XXX") and add
support of μTP as a replacement of TCP/IP. Yggdrasil-related code will
stay the same, but it will just use utp-code for Listen/Dial functions,
instead of gvisor's tcpip-code. Of course with returned drawback of
inability to deal with other nodes over TCP/IP, just only with the same
NNCP-μTP nodes.

I definitely do not wish to remove/revert tcpip, because it works faster
and honestly acts as real host with IPv6+TCP network stack. I do not
like utp-implementation at least because it depends on
github.com/anacrolix/sync library, having no information about its
acceptable usage, that actually has to be treated as non-free software.
It is pretty small and one can rewrite/replace its usage with something
clearly free, but someone has to do it (I do not want). And it is not
clear in what state is github.com/neilalexander/utp's code, that
explicitly notes that you should use libutp-wrapper library instead
(depending on github.com/anacrolix/sync again), that is also not an
option, because requires linking with C++ code, instead of pure-Go one.

When I searched for transport implementation libraries a year ago, I did
not find anything like simple TCP-like protocol implementation. Of
course one can write very simple transport protocol with trivial headers
and ACKing of every segment, but that will have awful performance. I did
not find anything with sliding window support, something like Zmodem at
least. With direct TCP connection between the nodes there are no
questions about transport protocol, because all of them are guaranteed
to be delivered in order, but although Yggdrasil uses TCP between the
peers, packets still can have various routes (can be reordered) without
actual delivery guarantees. Simple protocol without sliding window will
be slow. Probably I searched badly.

I met pure-Go QUIC implementation, but as far as I remember TLS can not
be turned off in it. I won't accept that TLS complication and huge
unneeded overhead.

So I hope replacing "s.sndSsthresh = math.MaxInt64" with MaxInt32 will
solve the issue.

-- 
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263  6422 AE1A 8109 E498 57EF