Subject: Re: redo-ifchange hangs when called with many arguments

From: Sergey Matveev <[email-masked]>
Date: Tue, 19 Oct 2021 16:58:41 +0300

Greetings!

*** goredo [2021-10-16 15:58]:
>One of my projects resulted in an inadvertent stress test of how many arguments redo-ifchange can handle.

Well, basically it should depend solely on OS. Of course except for
maximal command line arguments number, that is the main limiting factor
of redo. I wonder why DJB decided to pass dependencies as arguments (as
tar), instead of reading them from stdin (as cpio/pax), that would not
artificially limit their number.

>The redo-ifchange process was just sleeping without making progress, like it was waiting for pipe input or something similar.

I do not clearly remember (possibly I am mistaken), but I think I met
that kind of behaviour last year when I run several thousand targets at
once in parallel. I do not remember how I debugged and traced it, but
the problem was in touching limits of opened file descriptors, that I
fixed in 1.2.0 release (it was non-optimal resources usage, but everything
got stuck at OS level limits). I also noticed that my FreeBSD has nearly
two million opened files limit, but some version of Ubuntu just only one
thousand.

>I also noticed before, that killing redo with C-c does not always terminate the process it spawned (through a .do file).

According to the code (and I checked it in practice), it should not
terminate any of its processes at all :-). I added "infanticide" to it
in 1.17.0 release.

-- 
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263  6422 AE1A 8109 E498 57EF