Skip to contentSkip to conversation

iotests/testrunner: Flush after run_test()

PATCH Applied Hanna Reitz 3 participants 3 messages May 6, 2022 – May 12, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iotests/testrunner: Flush after run_test()

When stdout is not a terminal, the buffer may not be flushed at each end of line, so we should flush after each test is done. This is especially apparent when run by check-block, in two ways:

First, when running make check-block -jX with X > 1, progress indication was missing, even though testrunner.py does theoretically print each test's status once it has been run, even in multi-processing mode. Flushing after each test restores this progress indication.

Second, sometimes make check-block failed altogether, with an error message that "too few tests [were] run". I presume that's because one worker process in the job pool did not get to flush its stdout before the main process exited, and so meson did not get to see that worker's test results. In any case, by flushing at the end of run_test(), the problem has disappeared for me.

---
 tests/qemu-iotests/testrunner.py | 1 +
 1 file changed, 1 insertion(+)
tests/qemu-iotests/testrunner.py +1 -0
index aae70a8341..10d9e8ef27 100644
@@ -378,6 +378,7 @@ def run_test(self, test: str,
             else:
                 print(res.casenotrun)
 
+        sys.stdout.flush()
         return res
 
     def run_tests(self, tests: List[str], jobs: int = 1) -> bool:
Signature · 1 line
2.35.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 16 lines · Hanna Reitz

When stdout is not a terminal, the buffer may not be flushed at each end of line, so we should flush after each test is done. This is especially apparent when run by check-block, in two ways:

First, when running make check-block -jX with X > 1, progress indication was missing, even though testrunner.py does theoretically print each test's status once it has been run, even in multi-processing mode. Flushing after each test restores this progress indication.

Second, sometimes make check-block failed altogether, with an error message that "too few tests [were] run". I presume that's because one worker process in the job pool did not get to flush its stdout before the main process exited, and so meson did not get to see that worker's test results. In any case, by flushing at the end of run_test(), the problem has disappeared for me.

Quoted 4 lines
---
 tests/qemu-iotests/testrunner.py | 1 +
 1 file changed, 1 insertion(+)
Quoted 17 lines
tests/qemu-iotests/testrunner.py +1 -0
index aae70a8341..10d9e8ef27 100644
@@ -378,6 +378,7 @@ def run_test(self, test: str,
             else:
                 print(res.casenotrun)
 
+        sys.stdout.flush()
         return res
 
     def run_tests(self, tests: List[str], jobs: int = 1) -> bool:
Signature · 1 line
2.35.1
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
Applied
#2 lore raw
In reply to Hanna Reitz

Am 06.05.2022 um 15:42 hat Hanna Reitz geschrieben:

Quoted 9 lines

When stdout is not a terminal, the buffer may not be flushed at each end of line, so we should flush after each test is done. This is especially apparent when run by check-block, in two ways:

First, when running make check-block -jX with X > 1, progress indication was missing, even though testrunner.py does theoretically print each test's status once it has been run, even in multi-processing mode. Flushing after each test restores this progress indication.

Quoted 8 lines

Second, sometimes make check-block failed altogether, with an error message that "too few tests [were] run". I presume that's because one worker process in the job pool did not get to flush its stdout before the main process exited, and so meson did not get to see that worker's test results. In any case, by flushing at the end of run_test(), the problem has disappeared for me.

Thanks, applied to the block branch.

Kevin