public inbox for goredo-devel@lists.stargrave.org
Atom feed
From: spacefrogg <spacefrogg-goredo@spacefrogg•net>
To: goredo-devel@lists.cypherpunks.ru
Subject: Several issues since 1.30
Date: Thu, 21 Sep 2023 19:12:21 +0000 [thread overview]
Message-ID: <3029e69b0d59a814b5bf23c0b4a8e4df@spacefrogg.net> (raw)
Hi,
I am experiencing several issues since updating to 1.30 (maybe present
one or two releases earlier, too).
1. I have a C project with a very flat hierarchy. About 90 C files are
to be compiled and linked into 5 static libraries. The compiler command
is pre-computed in a separate script and used (and redo-ifchange'd) in
the default.o target that compiles each C file. On a fresh run which
must establish the dependency the first time, I run into the following
warning issued by redo:
run.go:247: resource deadlock avoided
/home/user/project/out/.redo/cc.precompile.lock
N.B. The particular message shown above refers to the precompile script
that generates precompiled header files. It bears the same dependency
relation as the compile script: Each C file depends on it. Sometimes it
hits at that target, sometimes it hits the compiler script. This warning
is issued about 320 times. So, many times more than files depending on
it.
To be clear, everything works as expected, but it seems that redo has
become greedy of sorts and exhausts some of its resources. My toolchain
does not limit the amount of targets discharged to a `redo-ifchange`
call. So, I suspect the number of open files is an issue.
2. Sometimes, again for projects with many independent targets that can
run in parallel, redo *at least* prints out that it ran one of those
targets twice that all the targets depended upon (e.g. the compile
script). I suspect a serious issue here that might relate to no. 1.
Maybe, sometimes redo looses track of whether a target's dependency is
already handled due to the resource exhaustion. I could not verify
whether redo actually runs the target twice or just prints out the line
a second time. The two status lines are not always adjacent to each
other (some other status line(s) might be in between), which, to me,
hints to that it is actually executed multiple times.
3. The output of redo has become useless for larger amounts of targets
that can run in parallel. Note, that this effect is independent of
whether I run redo with a single job or multiple parallel jobs. What
happens, is, that the output of each target is printed one after another
without properly inserting the separators that conclude the run of each
target (e.g. redo . . yadda/yadda (yadda.do) (10.123s)). All the missing
separators are printed in a batch at some point. So, the general order
of things, i.e., log output first, separator line second, is preserved.
I suspect that some terminal flow control/buffering and stream handling
has changed internally and that this is a race condition between the log
output handler and the status printing (job manager?) handler. I also
noticed that sometimes, redo does not print any separators for quite a
while (while counting up the executed targets) and then printing many at
once at some point. I noticed this for targets that do not produce any
output. I suspect that this effect has the same cause.
4. In light of no. 3, I have a small feature request: The output of
targets can be very confusing, because it is unclear at first, what
target the output relates to. It just starts printing and only tells you
in the very last line, which target actually produced the output.
apenwarr redo does a better job, here, by printing two similar status
lines, once as a header and once as a footer. So the feature request is
as follows: As long as a target produces no output, do not print
anything. If it starts producing output, print a header and then its
output. Always print a footer after the end of a target. So, as long as
targets produce no output, the list of completed targets looks the same
as now, but targets that produce output are properly introduced by a
header line for better context.
Thanks for reading this far!
–Michael
next reply other threads:[~2023-09-21 19:39 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-21 19:12 spacefrogg [this message]
2023-09-23 19:16 ` resource deadlock avoided messages Sergey Matveev
2023-09-24 11:17 ` goredo
2023-09-25 16:14 ` Sergey Matveev
2023-09-23 19:16 ` Target runs multiple times in single session Sergey Matveev
2023-09-23 19:17 ` Slow/buffered output of targets finished messages Sergey Matveev
2023-09-23 19:17 ` Unclear what target the output relates to Sergey Matveev
2023-09-24 11:10 ` goredo
2023-09-25 16:20 ` Sergey Matveev
2023-09-28 13:55 ` spacefrogg
2023-09-30 16:57 ` Sergey Matveev
2023-10-16 13:24 ` Sergey Matveev