public inbox for goredo-devel@lists.stargrave.org
Atom feed

From: spacefrogg <spacefrogg-goredo@spacefrogg•net>
To: goredo-devel@lists.cypherpunks.ru
Subject: Re: Unclear what target the output relates to
Date: Thu, 28 Sep 2023 13:55:44 +0000	[thread overview]
Message-ID: <734040eacb1f05719287c0803c886a46@spacefrogg.net> (raw)
In-Reply-To: <ZRGzPybshEiavg9X@stargrave.org>

Hi!

I thought about the problem further and I've found the precise output, 
that is problematic for me.
Definitions:
target's output - the log messages sent to stderr. Example lines 
prefixed with (to)
status line - the (green, red) messages from goredo printed after a 
target finishes. Example lines prefixed (sl)

The problematic output appears, when a target A produces output and 
after that redo-ifchange's other targets B and C. It looks as follows:

(to) some log message of A
(to) some other message of A
(to) message that belongs to B
(sl) . B (B.do) (1.234s)
(sl) . C (C.do) (2.345s)
(sl) A (A.do) (5.678s)

Especially, when B or C redo-ifchange further targets, the distance from 
(to) lines to the matching (sl) line increases so much, that it is 
virtually impossible to reliably determine to which target the (to) 
lines belong. Also, output from A and B are concatenated because no 
other target has finished in the meantime.

I would prefer the following output:

(sl) A (A.do) ...
(to) some log message of A
(to) some other message of A
(sl) . B (B.do) ...
(to) message that belongs to B
(sl) . B (B.do) (1.234s)
(sl) . C (C.do) (2.345s)
(sl) A (A.do) (5.678s)

So, while C is not producing output, no header is produced and the 
output stays short and nice. The other targets get two (sl) lines, one 
header one footer. The output of A and B also cannot interfere because A 
is currently waiting on redo-ifchange of B and C and does not produce 
output.

I do realize, that without complete log linearization, there still 
remains the following case when running parallel jobs:

(sl) A (A.do) ...
(to) some log message of A
(to) some other message of A
(sl) . B (B.do) ...
(to) message that belongs to B
(sl) D (D.do) ...
(to) message of D
(to) late message that belongs to A
(sl) . B (B.do) (1.234s)
(sl) . C (C.do) (2.345s)
(sl) A (A.do) (5.678s)
(sl) D (D.do) (6.789s)

The last two (to) lines could belong to any of A, B or D. Proper 
linearization does incur real cost. But even if they do get mixed up, I 
maintain that the output as a whole is much more informative now with 
very little cost. The redo process knows which target it operates on and 
when it first starts to produce output. So, it could inject the (sl) 
header just before that. Especially when running with a single job, 
there should be no race conditions because targets A and D are executed 
one after another.

–Michael

next prev parent reply	other threads:[~2023-09-28 13:55 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-21 19:12 Several issues since 1.30 spacefrogg
2023-09-23 19:16 ` resource deadlock avoided messages Sergey Matveev
2023-09-24 11:17   ` goredo
2023-09-25 16:14     ` Sergey Matveev
2023-09-23 19:16 ` Target runs multiple times in single session Sergey Matveev
2023-09-23 19:17 ` Slow/buffered output of targets finished messages Sergey Matveev
2023-09-23 19:17 ` Unclear what target the output relates to Sergey Matveev
2023-09-24 11:10   ` goredo
2023-09-25 16:20     ` Sergey Matveev
2023-09-28 13:55       ` spacefrogg [this message]
2023-09-30 16:57         ` Sergey Matveev
2023-10-16 13:24         ` Sergey Matveev