Run ptests via stdbuf configured to line-buffering
As ptest-runner communicates with child processes via pipe2(), the corresponding channels are not attached to a pty. In that situation stdio facilities like printf() or fwrite() are fully buffered. If a ptest would use them, without bothering to fflush() the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled. If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of 'signs of life' from the child process.
stdbuf utility from coreutils adjusts the buffering to a line-buffered one, and so ptest-runner will get the lines as soon as they are written.
--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
utils.c +2 -5
@@ -243,16 +243,13 @@ filter_ptests(struct ptest_list *head, char **ptests, int ptest_num)static inline void run_child (char *run_ptest,int fd_stdout,int fd_stderr) { -char **argv =malloc (sizeof (char ) *2 ); +char * argv[] = {"stdbuf" ,"-oL" ,"-eL" ,"./run-ptest" ,NULL };chdir (dirname (strdup (run_ptest))); - argv[0 ] = run_ptest; - argv[1 ] =NULL ; -dup2 (fd_stdout,STDOUT_FILENO );// XXX: Redirect stderr to stdout to avoid buffer ordering problems. dup2 (fd_stdout,STDERR_FILENO ); -execv (run_ptest, argv); +execvp (argv[0 ], argv);exit (1 ); }
Signature · 1 line
2.17.1
Quoted 25 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(), the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled. If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of 'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line- buffered
one, and so ptest-runner will get the lines as soon as they are written.Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)utils.c +0
-0
utils.c filter_ptests(struct ptest_list *head, char · line 247
Quoted 6 lines
utils.c +1
-1@@ -243,16 +243,13 @@ filter_ptests(struct ptest_list *head, char **ptests, int ptest_num)static inline void run_child (char *run_ptest,int fd_stdout,int fd_stderr) { -char **argv =malloc (sizeof (char ) *2 ); +char * argv[] = {"stdbuf" ,"-oL" ,"-eL" ,"./run-ptest" ,NULL };
Should be run_ptest the from the function argument, not the string literal "./run-ptest" ?
Quoted 17 lines
utils.c +0
-3chdir (dirname (strdup (run_ptest))); - argv[0 ] = run_ptest; - argv[1 ] =NULL ; -dup2 (fd_stdout,STDOUT_FILENO );// XXX: Redirect stderr to stdout to avoid buffer ordering problems.
utils.c +1
-1dup2 (fd_stdout,STDERR_FILENO ); -execv (run_ptest, argv); +execvp (argv[0 ], argv);exit (1 ); }Signature · 1 line
2.17.1
Signature · 1 line
Joshua Watt <[email protected]>
Quoted 15 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(), the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled. If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of 'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line- buffered
one, and so ptest-runner will get the lines as soon as they are written.
Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
I'm a little torn on this. I noticed some of the run-ptest scripts use "| sed -u" whilst the one you were seeing problems with uses "| sed" without -u.
We may want to consider strongly recommending -u. I'm testing a patch with some tweaks like that in it...
Cheers,
Richard
Quoted 21 lines · Richard Purdie
Quoted 15 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(), the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled. If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of 'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line- buffered
one, and so ptest-runner will get the lines as soon as they are written.Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
Quoted 6 lines
I'm a little torn on this. I noticed some of the run-ptest scripts use "| sed -u" whilst the one you were seeing problems with uses "| sed" without -u.
We may want to consider strongly recommending -u. I'm testing a patch with some tweaks like that in it...
Please no. I'm running images without sed and using busybox sed instead, and that doesn't support -u. I'd rather be compatible with sed from busybox to keep changes to images minimal (e.g. install of additional packages) before executing ptests.
-Mikko
Quoted 33 lines · On Fri, 2019-04-05 at 06:16 +0000, [email protected] wrote:
Quoted 28 lines · Richard Purdie
Quoted 16 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(), the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled.
If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of 'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line- buffered
one, and so ptest-runner will get the lines as soon as they are written.Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)I'm a little torn on this. I noticed some of the run-ptest scripts use
"| sed -u" whilst the one you were seeing problems with uses "| sed"
without -u.Quoted 3 lines
We may want to consider strongly recommending -u. I'm testing a patch
with some tweaks like that in it...
Quoted 6 lines
Please no. I'm running images without sed and using busybox sed instead, and that
doesn't support -u. I'd rather be compatible with sed from busybox to keep changes
to images minimal (e.g. install of additional packages) before executing ptests.
The other alternative option being proposed is for ptest-runner to depend on coreutils which is worse?
I did test the -u option to sed in the openssh ptest runner and it did fix the problems we've been seeing.
I'm open to other alternatives but the -u option to sed is looking like the best one we have right now. These bugs are making our testing of ptests effectively useless and unpredictable so this is a serious issue...
Cheers,
Richard
Quoted 41 lines · On Fri, Apr 05, 2019 at 07:46:00AM +0100, [email protected]…
Quoted 33 lines · On Fri, 2019-04-05 at 06:16 +0000, [email protected] wrote:
Quoted 28 lines · Richard Purdie
Quoted 16 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(), the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled.
If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of 'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line- buffered
one, and so ptest-runner will get the lines as soon as they are written.Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)I'm a little torn on this. I noticed some of the run-ptest scripts use
"| sed -u" whilst the one you were seeing problems with uses "| sed"
without -u.Quoted 3 lines
We may want to consider strongly recommending -u. I'm testing a patch
with some tweaks like that in it...Quoted 6 lines
Please no. I'm running images without sed and using busybox sed instead, and that
doesn't support -u. I'd rather be compatible with sed from busybox to keep changes
to images minimal (e.g. install of additional packages) before executing ptests.
Quoted 2 lines
The other alternative option being proposed is for ptest-runner to depend on coreutils which is worse?
GNU sed does not come from coreutils but from sed recipe.
Your call in the end. I just provided my point of view.
Quoted 7 lines
I did test the -u option to sed in the openssh ptest runner and it did fix the problems we've been seeing.
I'm open to other alternatives but the -u option to sed is looking like the best one we have right now. These bugs are making our testing of ptests effectively useless and unpredictable so this is a serious issue...
Understood. I hope you could also add 'set -eux' to all ptest shell scripts. Many of them seem to be missing shell script error handling and failures like providing -u to busybox sed may go unnoticed.
-Mikko
Quoted 50 lines · On Fri, 2019-04-05 at 06:54 +0000, [email protected] wrote:
Quoted 48 lines · On Fri, Apr 05, 2019 at 07:46:00AM +0100, [email protected]…
Quoted 46 lines · On Fri, 2019-04-05 at 06:16 +0000, [email protected] wrote:
Quoted 34 lines · Richard Purdie
Quoted 21 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(),
the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully
buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled.
If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of
'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line-
buffered
one, and so ptest-runner will get the lines as soon as they are
written.Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)I'm a little torn on this. I noticed some of the run-ptest scripts
use
"| sed -u" whilst the one you were seeing problems with uses "| sed"
without -u.Quoted 3 lines
We may want to consider strongly recommending -u. I'm testing a patch
with some tweaks like that in it...Please no. I'm running images without sed and using busybox sed instead, and that
doesn't support -u. I'd rather be compatible with sed from busybox to
keep changes
to images minimal (e.g. install of additional packages) before executing ptests.
Quoted 6 lines
Quoted 2 lines
The other alternative option being proposed is for ptest-runner to depend on coreutils which is worse?
GNU sed does not come from coreutils but from sed recipe.
Your call in the end. I just provided my point of view.
The original patch in this thread from Alex needs stdbuf which would add a coreutils depends to ptest-runner for everything as I understand it.
Quoted 11 lines
Quoted 10 lines
I did test the -u option to sed in the openssh ptest runner and it did
fix the problems we've been seeing.I'm open to other alternatives but the -u option to sed is looking like
the best one we have right now. These bugs are making our testing of
ptests effectively useless and unpredictable so this is a serious issue...
Quoted 3 lines
Understood. I hope you could also add 'set -eux' to all ptest shell scripts. Many of them seem to be missing shell script error handling and failures like providing -u to busybox sed may go unnoticed.
Patches would be welcome to help clean up some of these scripts. I feel like I'm fighting a lonely battle trying to get any of this to work at times :(
The good news is the automated monitoring of the ptest results should catch problems like that as if a chunk of ptests "disappear", the new testing should quickly highlight that.
(We should improve the scripts, we just also have another way to spot problems)
Cheers,
Richard
Quoted 54 lines · On Fri, Apr 05, 2019 at 07:59:41AM +0100, [email protected]…
Quoted 53 lines · On Fri, 2019-04-05 at 06:54 +0000, [email protected] wrote:
Quoted 50 lines · On Fri, Apr 05, 2019 at 07:46:00AM +0100, [email protected]…
Quoted 39 lines · On Fri, 2019-04-05 at 06:16 +0000, [email protected] wrote:
Quoted 34 lines · Richard Purdie
Quoted 21 lines · Alexander Kanavin
As ptest-runner communicates with child processes via
pipe2(),
the corresponding channels are not attached to a pty. In that situation stdio facilities likeprintf()orfwrite()are fully
buffered. If a ptest would use them, without bothering tofflush()the output, ptest-runner will only receive what was written by the child ptest process after a buffer gets filled.
If the unit tests are proceeding slowly, this may mean that ptest-runner will erroneously timeout due to an apparent lack of
'signs of life' from the child process.stdbuf utility from coreutils adjusts the buffering to a line-
buffered
one, and so ptest-runner will get the lines as soon as they are
written.Quoted 4 lines
Signed-off-by Alexander Kanavin <[email protected]>--- utils.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)I'm a little torn on this. I noticed some of the run-ptest scripts
use
"| sed -u" whilst the one you were seeing problems with uses "| sed"
without -u.Quoted 3 lines
We may want to consider strongly recommending -u. I'm testing a patch
with some tweaks like that in it...Quoted 7 lines
Please no. I'm running images without sed and using busybox sed instead, and that
doesn't support -u. I'd rather be compatible with sed from busybox to
keep changes
to images minimal (e.g. install of additional packages) before executing ptests.The other alternative option being proposed is for ptest-runner to depend on coreutils which is worse?
Quoted 7 lines
Quoted 3 lines
GNU sed does not come from coreutils but from sed recipe.
Your call in the end. I just provided my point of view.
The original patch in this thread from Alex needs stdbuf which would add a coreutils depends to ptest-runner for everything as I understand it.
Ok, that too then. Then all hope is lost so lets include sed there as well :)
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
I did test the -u option to sed in the openssh ptest runner and it did
fix the problems we've been seeing.I'm open to other alternatives but the -u option to sed is looking like
the best one we have right now. These bugs are making our testing of
ptests effectively useless and unpredictable so this is a serious issue...
Quoted 7 lines
Quoted 3 lines
Understood. I hope you could also add 'set -eux' to all ptest shell scripts. Many of them seem to be missing shell script error handling and failures like providing -u to busybox sed may go unnoticed.
Patches would be welcome to help clean up some of these scripts. I feel like I'm fighting a lonely battle trying to get any of this to work at times :(
We're ramping up with ptests and will provide fixes back to upstream.
Quoted 6 lines
The good news is the automated monitoring of the ptest results should catch problems like that as if a chunk of ptests "disappear", the new testing should quickly highlight that.
(We should improve the scripts, we just also have another way to spot problems)
All this work is very much appreciated here in downstream! Thanks you very much!
-Mikko