public inbox for nncp-devel@lists.stargrave.org
Atom feed

* What is /hdr/ subdirectory?
@ 2026-03-10 23:28 Hadmut Danisch
  2026-03-11  2:50 ` John Goerzen
  2026-03-11  7:36 ` What is /hdr/ subdirectory? Sergey Matveev
  0 siblings, 2 replies; 11+ messages in thread
From: Hadmut Danisch @ 2026-03-10 23:28 UTC (permalink / raw)
  To: nncp-devel

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

Hi,


technical question because I'm hunting a bug:


In /var/spool/nncp/*/tx  there is a subdirectory  /hdr/ .

There is always a pair of files,  eg.


tx/7SM43TMJGSV3RGBZT4RPYK4XJN3HSMBGWPLEIBI2IMZPABREB4BQ
tx/hdr/7SM43TMJGSV3RGBZT4RPYK4XJN3HSMBGWPLEIBI2IMZPABREB4BQ


and I'm hunting the problem, why they do have different ownerships, the 
hdr file is owned by root and silently skipped by the daemon.


For some reason, the hdr file is generated significantly later (here: 
10min).


So my question is: What is the /hdr/ file? On what occasion is it generated?


regards

Hadmut


[-- Attachment #2: Type: text/html, Size: 1350 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

• * Re: What is /hdr/ subdirectory?
    2026-03-10 23:28 What is /hdr/ subdirectory? Hadmut Danisch
  @ 2026-03-11  2:50 ` John Goerzen
    2026-03-11 11:51   ` Permissions (Was: What is /hdr/ subdirectory?) Hadmut Danisch
    2026-03-11  7:36 ` What is /hdr/ subdirectory? Sergey Matveev
    1 sibling, 1 reply; 11+ messages in thread
  From: John Goerzen @ 2026-03-11  2:50 UTC (permalink / raw)
    To: Hadmut Danisch; +Cc: nncp-devel
  
  Hi Hadmut,
  
  Sergey has some information about it here:
  
  https://nncp.mirrors.quux.org/Spool.html#HdrFile
  
  You can also use a nohdr option in the config file or on the command
  line to suppress this.
  
  They can be safely removed also.
  
  - John
  
  On Wed, Mar 11 2026, Hadmut Danisch wrote:
  
  > Hi, 
  >
  > technical question because I'm hunting a bug:
  >
  > In /var/spool/nncp/*/tx  there is a subdirectory  /hdr/ . 
  >
  > There is always a pair of files,  eg. 
  >
  > tx/7SM43TMJGSV3RGBZT4RPYK4XJN3HSMBGWPLEIBI2IMZPABREB4BQ
  > tx/hdr/7SM43TMJGSV3RGBZT4RPYK4XJN3HSMBGWPLEIBI2IMZPABREB4BQ
  >
  > and I'm hunting the problem, why they do have different ownerships, the hdr file is
  > owned by root and silently skipped by the daemon. 
  >
  > For some reason, the hdr file is generated significantly later (here: 10min). 
  >
  > So my question is: What is the /hdr/ file? On what occasion is it generated?
  >
  > regards
  >
  > Hadmut
  
  ^ permalink raw reply	[flat|nested] 11+ messages in thread

• • * Permissions (Was: What is /hdr/ subdirectory?)
      2026-03-11  2:50 ` John Goerzen
    @ 2026-03-11 11:51   ` Hadmut Danisch
      2026-03-11 12:08     ` Permissions John Goerzen
      0 siblings, 1 reply; 11+ messages in thread
    From: Hadmut Danisch @ 2026-03-11 11:51 UTC (permalink / raw)
      To: John Goerzen; +Cc: nncp-devel
    
    Hi,
    
    I've found the problem.
    
    The debian/ubuntu nncp package comes with a flaw: In contrast to e.g. 
    uucp, the nncp package creates a nncp user and sets ownership of 
    /var/spool/nncp to nncp, but does not set the binaries setuid nncp, as, 
    e.g. uucp does.
    
    The binaries are, therefore, always run as the user who calls them.
    
    - it works, when run as nncp
    
    - it partly works when run as root, because all files can be written and 
    read as root, but  nncp-daemon and nncp-caller can't access them then.
    
    - it does not work at all when called by someone else, because the 
    program can't access /etc/nncp.yaml or /var/spool/nncp.
    
    
    To fix this, I have set _some_ of the binaries  setuid nncp. Which, on 
    the other hand, is a security flaw, because every user on the system can 
    rm nncp messages. But it worked.
    
    
    I hadn't set nncp-stat, because I thought, it only reads files.
    
    
    Every now and then I call nncp-stat as root to check everything is 
    working correct. This causes the /hdr/ files to be written, but with 
    ownership root. Once the /hdr/ file is owned by root, nncp-daemon 
    silently ignores them and the messages is never transmitted. Not even an 
    error message.  A
    
    chown -R nncp:nncp /var/spool/nncp
    
    fixes things temporarily, and  nncp-daemon can transmit them with next 
    run – until the admin calls nncp-stat again.
    
    
    So this is somewhat error prone. And it lacks a description about how 
    this – ownership and permissions – is supposed to work.
    
    
    nncp-daemon should issue an error message if it can't read the /hdr/ 
    file instead of silently ignoring the message.
    
    
    Proposal:
    
    - If the binaries are run as nncp or setuid nncp (i.e. same owner as 
    /etc/nncp.yaml and/or /var/spool/nncp):  If it can access /etc/nncp.yaml 
    and /var/spool/nncp  run as normal, otherwise abort with error message
    
    - if run as root, write files and create directories with the same 
    uid/gid as /var/spool/nncp , or at least with g+rw and /var/spool/nncp 
    set to g+s, to force all created files and directory to have nncp as 
    their group and group rw permissions.
    
    - otherwise, if run as a normal user,  use  ~/.config/nncp.yaml and 
    ~/.local/spool/nncp  instead of /etc/nncp.yaml and /var/spool/nncp
    
    
    regards
    
    Hadmut
    
    
    
    
    
    
    
    
    ^ permalink raw reply	[flat|nested] 11+ messages in thread

  • • * Re: Permissions
        2026-03-11 11:51   ` Permissions (Was: What is /hdr/ subdirectory?) Hadmut Danisch
      @ 2026-03-11 12:08     ` John Goerzen
        2026-03-11 12:32       ` Permissions Hadmut Danisch
        0 siblings, 1 reply; 11+ messages in thread
      From: John Goerzen @ 2026-03-11 12:08 UTC (permalink / raw)
        To: Hadmut Danisch; +Cc: nncp-devel
      
      On Wed, Mar 11 2026, Hadmut Danisch wrote:
      
      > The debian/ubuntu nncp package comes with a flaw: In contrast to e.g. uucp, the
      > nncp package creates a nncp user and sets ownership of /var/spool/nncp to nncp,
      > but does not set the binaries setuid nncp, as, e.g. uucp does.
      
      That's not a flaw; that's a feature.  setuid is a security risk, and
      makes the assumption that every user on the system wants to run the
      systemwide NNCP.  That's not really necessary with NNCP; it is entirely
      possible for every user to have their own NNCP instance that runs right
      out of their home directory, for instance.
      
      It also makes the assumption that you want every user on the system to
      be able to run the NNCP commands, which isn't an assumption I'd make.
      If you don't want that kind of access, then you'd have to additionally
      layer some sort of access check mechanism in NNCP itself, which is
      unnecessary complexity.
      
      You can use sudo to run NNCP commands as other users.  Note that if you
      are using nncp-file to send a file that the nncp user can't access,
      you'll need to pipe it to nncp-file instead.
      
      In my .bash_aliases, I have:
      
      for FILE in /usr/bin/nncp-*; do
         BASENAME="`basename "$FILE"`"
         alias "$BASENAME=sudo -Hu nncp $FILE"
      done
      
      https://www.complete.org/using-nncp-with-sudo/ has some more information
      on this.  It also describes how you can have a central nncp user on a
      system, and then use sudo to let it communicate with user-specific NNCP
      installations on it.
      
      > To fix this, I have set _some_ of the binaries  setuid nncp. Which, on the other
      > hand, is a security flaw, because every user on the system can rm nncp messages.
      > But it worked.
      
      That will be blown away when the NNCP package is upgraded; see
      dpkg-statoverride for a way to make it more permanent.  Though note that
      due to the change from multiple binaries to a single binary, even that
      won't quite be enough for the next upgrade only.
      
      > So this is somewhat error prone. And it lacks a description about how this –
      > ownership and permissions – is supposed to work.
      
      I wouldn't really recommend running NNCP commands as root (or PHP or
      whatever else).
      
      > nncp-daemon should issue an error message if it can't read the /hdr/ file
      > instead of silently ignoring the message.
      
      Probably a good idea for Sergey there.  Or it can just ignore the hdr file.
      
      > - otherwise, if run as a normal user,  use  ~/.config/nncp.yaml and
      >   ~/.local/spool/nncp  instead of /etc/nncp.yaml and /var/spool/nncp
      
      That's also a nice suggestion!
      
      - John
      
      ^ permalink raw reply	[flat|nested] 11+ messages in thread

    • • * Re: Permissions
          2026-03-11 12:08     ` Permissions John Goerzen
        @ 2026-03-11 12:32       ` Hadmut Danisch
          2026-03-11 16:27         ` Permissions John Goerzen
                                   ` (2 more replies)
          0 siblings, 3 replies; 11+ messages in thread
        From: Hadmut Danisch @ 2026-03-11 12:32 UTC (permalink / raw)
          To: John Goerzen; +Cc: nncp-devel
        
        
        Am 11.03.26 um 13:08 schrieb John Goerzen:
        > That's not a flaw; that's a feature.  setuid is a security risk, and
        > makes the assumption that every user on the system wants to run the
        > systemwide NNCP.  That's not really necessary with NNCP; it is entirely
        > possible for every user to have their own NNCP instance that runs right
        > out of their home directory, for instance.
        
        
        So what is the user nncp and /var/spool/nncp good for, if no one could 
        ever use it by design?
        
        Should every single process (postfix, databases,...) which is supposed 
        to send files, have it's own nncp configuration and daemon?
        
        I'm not talking about individual users. I'm talking about several 
        services which should send backups, transfer e-mails and so on. Should 
        they share a spool, a configuration, and a key, or should they have 
        their own one?
        
        
        regards
        
        Hadmut
        
        
        
        
        
        
        ^ permalink raw reply	[flat|nested] 11+ messages in thread

      • • * Re: Permissions
            2026-03-11 12:32       ` Permissions Hadmut Danisch
          @ 2026-03-11 16:27         ` John Goerzen
            2026-03-11 16:35           ` Permissions Hadmut Danisch
            2026-03-11 16:30         ` Permissions John Goerzen
            2026-03-15  8:17         ` Permissions Sergey Matveev
            2 siblings, 1 reply; 11+ messages in thread
          From: John Goerzen @ 2026-03-11 16:27 UTC (permalink / raw)
            To: Hadmut Danisch; +Cc: nncp-devel
          
          On Wed, Mar 11 2026, Hadmut Danisch wrote:
          
          > So what is the user nncp and /var/spool/nncp good for, if no one could ever use
          > it by design?
          >
          > Should every single process (postfix, databases,...) which is supposed to send
          > files, have it's own nncp configuration and daemon?
          
          That's really up to you.  As a distribution maintainer, I can't be
          making presumptions about all those things, especially in the direction
          of weakening security by default.
          
          In my own case, for nncpnet, I simply add "user = nncp" in the exim
          transport configuration.  
          
          For my Usenet server, I add the news user to the nncp group and make
          /var/spool/news setgid.  Perhaps not the best, as the news user can then
          read the nncp.hjson file, but I could as easily use sudo to call it
          (perhaps I should do that).  In any case, it's a single-use container so
          that makes reasonable sense.
          
          > I'm not talking about individual users. I'm talking about several services which
          > should send backups, transfer e-mails and so on. Should they share a spool, a
          > configuration, and a key, or should they have their own one?
          
          It is up to you.  They could:
          
          - All share the same configuration and be a member of the nncp group.
            Drawbacks is that they could then read the private keys.
          
          - Share the same configruation, but call nncp utilities via sudo.
            Benefits of that are restricting actions to specific nncp commands and
            isolating the private keys from the calling accounts.
          
          - Each have its own nncp installation, which could either directly
            interface with remote systems, or spool up to a local "hub"
            installation via nncp-xfer or regular call/daemon configuration,
            possibly via sudo
          
          UUCP was more prescriptive about system design, in part because it was
          from an earlier era where security was thought about differently, and in
          part because it required access to shared systemwide resources (modems)
          that often were restricted from user accounts.  TCP isn't single-user in
          the way a serial line is, so more flexible options are possible with
          NNCP.
          
          Incidentally, this is also why I ship example systemd service files for
          NNCP but do not activate them by default in the .debs.  There are too
          many possible permutations of configurations for me to accurately
          anticipate what the user is likely to need.
          
          One could argue that creating the nncp user and group falls into that
          category as well, and not without merit.  On the other hand, if somebody
          never uses the nncp user and group, it creates no actual change in
          system behavior, unlike, say, starting nncp-daemon, which opens a port.
          
          - John
          
          ^ permalink raw reply	[flat|nested] 11+ messages in thread

        • • * Re: Permissions
              2026-03-11 16:27         ` Permissions John Goerzen
            @ 2026-03-11 16:35           ` Hadmut Danisch
              2026-03-11 16:48             ` Permissions John Goerzen
              0 siblings, 1 reply; 11+ messages in thread
            From: Hadmut Danisch @ 2026-03-11 16:35 UTC (permalink / raw)
              To: John Goerzen; +Cc: nncp-devel
            
            
            Am 11.03.26 um 17:27 schrieb John Goerzen:
            > They could:
            >
            > - All share the same configuration and be a member of the nncp group.
            >    Drawbacks is that they could then read the private keys.
            >
            > - Share the same configruation, but call nncp utilities via sudo.
            >    Benefits of that are restricting actions to specific nncp commands and
            >    isolating the private keys from the calling accounts.
            >
            > - Each have its own nncp installation, which could either directly
            >    interface with remote systems, or spool up to a local "hub"
            >    installation via nncp-xfer or regular call/daemon configuration,
            >    possibly via sudo
            
            
            What about writing nncp so that it deals properly with the task and has 
            none of the drawbacks?
            
            
            
            
            ^ permalink raw reply	[flat|nested] 11+ messages in thread

          • • * Re: Permissions
                2026-03-11 16:35           ` Permissions Hadmut Danisch
              @ 2026-03-11 16:48             ` John Goerzen
                0 siblings, 0 replies; 11+ messages in thread
              From: John Goerzen @ 2026-03-11 16:48 UTC (permalink / raw)
                To: Hadmut Danisch; +Cc: nncp-devel
              
              On Wed, Mar 11 2026, Hadmut Danisch wrote:
              
              > What about writing nncp so that it deals properly with the task and has none of
              > the drawbacks?
              
              I'm not the author of nncp, but from my perspective it already pretty
              much is, modulo a couple of minor tweaks you pointed out.
              
              I mean, what do you propose?
              
              setuid/setgid brings along a whole host of problems.  Now any user on
              the system can invoke the tool.  There are very few setuid/setgid
              programs anymore on a modern system, and for good reason.  On my system,
              which has a fair amount of software installed, the setuid ones all seem
              to have system functions (passwd, mount, etc).  The only other one is
              procmail, and that has to do with writing into user directories from
              MTAs that aren't capable of running it as the user, I suppose.
              
              I think it's a good thing that we have more modern design patterns now.
              
              You can already set it up as a central daemon on the system where
              individual users connect to if you want, and I've documented how to do
              that also.  There's nothing stopping you from running a separate nncp
              for email, another for backups, etc. and they all just transfer files to
              the central daemon running as nncp.
              
              - John
              
              ^ permalink raw reply	[flat|nested] 11+ messages in thread

        • * Re: Permissions
            2026-03-11 12:32       ` Permissions Hadmut Danisch
            2026-03-11 16:27         ` Permissions John Goerzen
          @ 2026-03-11 16:30         ` John Goerzen
            2026-03-15  8:17         ` Permissions Sergey Matveev
            2 siblings, 0 replies; 11+ messages in thread
          From: John Goerzen @ 2026-03-11 16:30 UTC (permalink / raw)
            To: Hadmut Danisch; +Cc: nncp-devel
          
          On Wed, Mar 11 2026, Hadmut Danisch wrote:
          
          > So what is the user nncp and /var/spool/nncp good for, if no one could ever use
          > it by design?
          
          I guess I should say, I do use this on every installation of mine.  My
          typical laptop setup is to have the systemwide installation there, have
          it dump incoming files into a directory I have access to as my regular
          user, restrict nncp.hjson to just the nncp user, and use sudo from my
          regular user account to call nncp utilities, as documented before.
          
          For incoming nncp-exec packets, I use sudo in nncp.hjson to process them
          as whatever user is appropriate.
          
          - John
          
          ^ permalink raw reply	[flat|nested] 11+ messages in thread

        • * Re: Permissions
            2026-03-11 12:32       ` Permissions Hadmut Danisch
            2026-03-11 16:27         ` Permissions John Goerzen
            2026-03-11 16:30         ` Permissions John Goerzen
          @ 2026-03-15  8:17         ` Sergey Matveev
            2 siblings, 0 replies; 11+ messages in thread
          From: Sergey Matveev @ 2026-03-15  8:17 UTC (permalink / raw)
            To: nncp-devel
          
          [-- Attachment #1: Type: text/plain, Size: 760 bytes --]
          
          Greetings!
          
          *** Hadmut Danisch [2026-03-11 13:32]:
          >I'm not talking about individual users. I'm talking about several services
          >which should send backups, transfer e-mails and so on. Should they share a
          >spool, a configuration, and a key, or should they have their own one?
          
          I did not think about those questions, because there are many possible
          use-cases all of which with with own drawbacks and conveniences. It is
          up to end-users/maintainers. setuid solutions is questionable, as John
          pointed out. I do not want to say that it is not acceptable at all (I
          used it in other Go projects), but currently do not have any opinion
          on it.
          
          -- 
          Sergey Matveev (http://www.stargrave.org/)
          LibrePGP: 12AD 3268 9C66 0D42 6967  FD75 CB82 0563 2107 AD8A
          
          [-- Attachment #2: signature.asc --]
          [-- Type: application/pgp-signature, Size: 265 bytes --]
          
          ^ permalink raw reply	[flat|nested] 11+ messages in thread

• * Re: What is /hdr/ subdirectory?
    2026-03-10 23:28 What is /hdr/ subdirectory? Hadmut Danisch
    2026-03-11  2:50 ` John Goerzen
  @ 2026-03-11  7:36 ` Sergey Matveev
    1 sibling, 0 replies; 11+ messages in thread
  From: Sergey Matveev @ 2026-03-11  7:36 UTC (permalink / raw)
    To: nncp-devel
  
  [-- Attachment #1: Type: text/plain, Size: 1089 bytes --]
  
  Greetings!
  
  *** Hadmut Danisch [2026-03-11 00:28]:
  >and I'm hunting the problem, why they do have different ownerships, the hdr
  >file is owned by root and silently skipped by the daemon.
  
  Maybe different toss/daemon processes running at different times with
  different permissions/owners.
  
  >So my question is: What is the /hdr/ file? On what occasion is it generated?
  
  It is just a "cached" version of encrypted packet's header. Just its
  stripped part. There are options to disable its generation entirely.
  
  That feature was created to speed up process of listing/searching in big
  spools on ZFS. ZFS uses by default rather big records: 128KiB. So
  reading even a single byte of the packet, will lead to at least reading
  of the single block, which is up to 128KiB (before compression). That
  leads to big read-amplification during searching for packets in the
  spool with acceptable niceness. Those hdr files is just a hack to
  prevent that read-amplification.
  
  -- 
  Sergey Matveev (http://www.stargrave.org/)
  LibrePGP: 12AD 3268 9C66 0D42 6967  FD75 CB82 0563 2107 AD8A
  
  [-- Attachment #2: signature.asc --]
  [-- Type: application/pgp-signature, Size: 265 bytes --]
  
  ^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2026-03-15  8:18 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-03-10 23:28 What is /hdr/ subdirectory? Hadmut Danisch
2026-03-11  2:50 ` John Goerzen
2026-03-11 11:51   ` Permissions (Was: What is /hdr/ subdirectory?) Hadmut Danisch
2026-03-11 12:08     ` Permissions John Goerzen
2026-03-11 12:32       ` Permissions Hadmut Danisch
2026-03-11 16:27         ` Permissions John Goerzen
2026-03-11 16:35           ` Permissions Hadmut Danisch
2026-03-11 16:48             ` Permissions John Goerzen
2026-03-11 16:30         ` Permissions John Goerzen
2026-03-15  8:17         ` Permissions Sergey Matveev
2026-03-11  7:36 ` What is /hdr/ subdirectory? Sergey Matveev