Re: NNCP in Debian/Ubuntu Update

John Goerzen
2021-09-20T21:14:46Z

On Mon, Sep 20 2021, Sergey Matveev wrote:

> *** John Goerzen [2021-09-20 15:10]:
>>So Debian has a very long-standing policy, going back decades,
>>of not
>>building dependencies from in-tree.
>
> Hmm... I probably can understand that for dynamically linked
> executables, but that seems strange for statically linked ones,
> that Go does by default. Moreover, Go's go.mod/go.sum are tied
> not to some abstract version of the library, but literally to
> its
> exact commit/tag. And even in one Go program you can easily use
> different versions of the same library, that will force you to
> install golang-whatever-* of various versions. All of that seems
> very strange to me, but it is Debian's decision, ok :-)

Fundamentally I can't just change that on my own, so our
conversation here is discussing something that "is what it is",
regardless of whether we agree with it. But, hey, I like
conversation :-)

I'm not as familiar with the Go build, but if I could express this
in terms of rust:

In Rust, there is a Cargo.toml, which defines the general
requirements for dependencies (ie, package X, version 0.2.*) and
then a cargo.lock which records the precise version and hash of
the dependency used for a canonical build.

There are pros and cons to this...

The pro is, of course, that you can easily get a theoretically
identical build from anywhere.

And the con is that you may have dozens of different versions of
the same package in use on a system, making it difficult to patch
bugs and security issues.

So any person can come down on one side or the other of that, and
I totally understand that. Debian as a project has chosen where
it comes down, and carries the pros and cons with it. FreeBSD and
Arch probably come down differently, and live with the pros and
cons of their choices too.

One thing that flows out of these decisions is the possibility of
unattended-upgrades existing in Debian, but really not so much in
FreeBSD ports or Arch. unattended-upgrades basically can patch
your system for security bugs automatically from cron. That's not
for everyone, but I love it. It's possible primarily because
Debian backports security fixes to the version in stable. That
is, you get a narrowly-targeted patch that has a low risk of
introducing issues because it isn't pulling in major new versions
of software and such. But the choice here also helps, because if
the distro follows cargo.lock of the go equivalent, that can
preclude the possibility of narrowly-targeted patches and also
expand the work of the security team to unsustainable levels.

Tradeoffs abound. What's right for Debian isn't right for
everybody. Debian prizes stability -- that is, an installed
system keeps working in the same way for a long time. The
tradeoff is slower releases and older software at certain points
in a year compared to, say, Arch. On the other hand, compared to
Arch, a Debian system has fewer times where a breaking change is
likely to be introduced. So if you see what Debian's goals are,
maybe this sort of thing makes some sense respective to them.
Arch's goals are different and so different choices will flow out
of that. The world probably needs both, so I'm not trying to slam
Arch/BSD here.

Actually - I ran into the versioning issue while packaging nncp.
The blake3 package requires cpuid, and we had a newer cpuid in
Debian than what the current blake3 was ready for. cpuid changed
its interface a bit, so blake3 wouldn't compile. I patched blake3
in the Debian build to get it going (it was a 2-line fix, not
really worth introducing multiple cpuid versions for), and also
submitted the issue upstream. blake3 fixed the issue upstream as
well, so I dropped our local Debian patch and upgraded to that
version. So, you could pull in newer cpuid and blake3 into your
tarballs, if you like.

> And that frightens me, because if someone want to report a bug,
> then it
> have to report about dozen of dependencies and their exact
> versions,
> instead of reporting only of Go version and NNCP's one. Oh
> well...

Ahh, well we have a tool called reportbug that captures all of
that with a bug report (or at least enough for a Debian maintainer
to look it up in the project databases), and sends it to the
Debian bug-tracking system. Then it's supposed to be the job of
the Debian maintainer (me) to sort through that sort of thing
before submitting it upstream (you). In this case, if there was
any question, I would probably ask the submitter to try verifying
the problem with a tarball-built NNCP before passing a report on
to you.

We've got the full build log of every package in the distro, for
every architecture it may build on. For instance, here's the
build log for blake3:

https://buildd.debian.org/status/fetch.php?pkg=golang-lukechampine-blake3&arch=all&ver=1.1.6-1&stamp=1632155690&raw=0

And you can see the precise versions of go, cpuid, etc. that it
built against. Though again, normally this is the job of the
Debian maintainer to look at and sort through, not the upstream
author.

> personally I slowly moving more and more to GNU Stow type of
> packages
> management, manually building software :-)

Haven't heard of Stow, but I have heard of Gentoo. I tried it for
a spell.

>>The uilive thing is a bit of an odd one; it was still in-tree in
>>the git
>>repo and it survived review so I guess it works for now :-)
>
> As I remember I copied it and modified. It is not original
> version. Do
> not remember why, but I believe to strip off various complicated
> unneeded stuff.

Makes sense.

- John