Subject: Re: NNCP in Debian/Ubuntu Update

From: John Goerzen <[email-masked]>
Date: Mon, 20 Sep 2021 15:10:41 -0500

On Mon, Sep 20 2021, Sergey Matveev wrote:

> Greetings!
>
> *** John Goerzen [2021-09-20 08:05]:
>>At long last, NNCP has been accepted into Debian unstable.
>
> Wonderful! Thank you!
>
> Just for the interest I looked at 
> https://salsa.debian.org/go-team/packages/nncp.git
> I have never dealt with Debian packages (well, 15+ years ago 
> played only
> with the one), so just my misunderstanding, but it seems that 
> NNCP is
> not built from the tarball? I see some dependencies on external
> Go-related packages and documentation building in debian/rules 
> file. I
> remind that NNCP's tarballs contain all Go dependencies, that 
> are
> statically linked in anyway. And they contain both Info and HTML
> documentation.

Yes.  Actually this is the reason it took me so long to get it in 
Debian; I had to learn about Go building and stuff.

So Debian has a very long-standing policy, going back decades, of 
not building dependencies from in-tree.  Even back in the C days, 
a lot of packages would require other libraries and include them 
in their tarballs.  That caused a number of problems at the 
distribution scale.  For instance, if there's a new OpenSSL vuln 
and various packages build from OpenSSL in-tarball, now instead of 
patching one thing, you first have to figure out where all it's 
embedded (a challenge itself) and then patch and rebuild all of 
those.  The same goes for bugs in OpenSSL, etc.  It's not 
particularly sustainable.

There is also another long-standing policy that every Debian 
package must be buildable using only what is in Debian, and 
without network access.  This is enforced at the CI level.  The 
rationale for this includes several aspects: 1) A build that 
downloads things from the Internet can break if those things go 
away; 2) it makes it easier to inject malware into the system; 3) 
it makes it harder to audit for license compliance; 4) it can lead 
to unintentional violations of the Debian Free Software Guidelines 
(if sources are downloaded but not distributed, this can also be a 
GPL violation).  So the Go and Rust ecosystems (plus others like 
Python) don't use the go/cargo/pip build mechanism to just 
download stuff off the Internet; they require that the 
dependencies already exist in Debian, and build using only those.

I did initially try to just use the tarball, but that was 
(somewhat predictably) rejected for these reasons.  So, I made 
separate packages for all the things that weren't already in 
Debian (blake3, balloon, etc) and got those into the archive. 
Then NNCP could go in. 
https://go-team.pages.debian.net/packaging.html talks about this.

The tarball is absolutely the right thing for users.  I like what 
you've done there.  It just wasn't the most suitable for a 
distribution (or, shall we say, a distribution with Debian's 
policies).

There is automated tooling to bring go stuff into Debian, since 
this is far from the only such case.  It is centered around the go 
convention of using git.  It mostly worked, here.  I had to patch 
up a few things but generally it worked well.  In fact, I do have 
a 7.7.0 release prepared -- the upgrade was one command -- which I 
will upload probably tomorrow once all of its dependencies have 
hit the mirrors.

The uilive thing is a bit of an odd one; it was still in-tree in 
the git repo and it survived review so I guess it works for now 
:-)

I also had to do a bit of hacking around the source tree; I don't 
quite understand it, but there was something relating to nncp/v7 
that the automated tooling didn't quite deal well with (you can 
see this in the debian/rules file).

So anyway, the long and short of it is, it is done, in the Debian 
way, and is working and will move ahead!

- John