oeqa/ptest: print a warning if ptests failed
PATCH 1/1 oeqa/ptest: print a warning if ptests failed
This allows spotting ptest regressions without having hard ptest failures (for that full ptest stability should be achieved).
--- meta/lib/oeqa/runtime/cases/ptest.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/oeqa/runtime/cases/ptest.py +1 -0
@@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase): failmsg = failmsg + "Failed ptests:\n%s" % pprint.pformat(failed_tests) if failmsg: + self.logger.warning("There were failing ptests.") self.fail(failmsg)
Signature · 1 line
2.29.2
Quoted 13 lines · Alexander Kanavin
This allows spotting ptest regressions without having hard ptest failures (for that full ptest stability should be achieved).
Signed-off-by Alexander Kanavin <[email protected]>--- meta/lib/oeqa/runtime/cases/ptest.py | 1 + 1 file changed, 1 insertion(+)meta/lib/oeqa/runtime/cases/ptest.py +0
-0
meta/lib/oeqa/runtime/cases/ptest.py class PtestRunnerTest(OERuntimeTestCase): · line 112
Quoted 5 lines
meta/lib/oeqa/runtime/cases/ptest.py +1
-0@@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase): failmsg = failmsg + "Failed ptests:\n%s" % pprint.pformat(failed_tests) if failmsg: + self.logger.warning("There were failing ptests.") self.fail(failmsg)
Have you looked at this on the arm worker? That seems to have four failing tests over three recipes:
{'libinput': ['libinput-test-suite'],
'strace': ['qual_fault-syscall.test', 'qual_fault.test'], 'valgrind': ['drd/tests/std_list']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1355
Not sure if these always fail or are intermittent?
Cheers,
Richard
Quoted 25 lines · Richard Purdie
Review
Quoted 9 lines · Alexander Kanavin
This allows spotting ptest regressions without having hard ptest failures (for that full ptest stability should be achieved).
Signed-off-by Alexander Kanavin <[email protected]>--- meta/lib/oeqa/runtime/cases/ptest.py | 1 + 1 file changed, 1 insertion(+)(quoted diff) +0
-0
b/meta/lib/oeqa/runtime/cases/ptest.pymeta/lib/oeqa/runtime/cases/ptest.py class PtestRunnerTest(OERuntimeTestCase): · line 108
Quoted 5 lines
index
a9572c81f0..0800f3c27f100644meta/lib/oeqa/runtime/cases/ptest.py +0
-0@@ -108,4 +108,5 @@ class PtestRunnerTest(OERuntimeTestCase): failmsg = failmsg + "Failed ptests:\n%s" %
pprint.pformat(failed_tests)meta/lib/oeqa/runtime/cases/ptest.py class PtestRunnerTest(OERuntimeTestCase): · line 112
Quoted 4 lines
meta/lib/oeqa/runtime/cases/ptest.py +1
-0if failmsg: + self.logger.warning("There were failing ptests.") self.fail(failmsg)Have you looked at this on the arm worker? That seems to have four failing tests over three recipes:
Quoted 8 lines
{'libinput': ['libinput-test-suite'],
'strace': ['qual_fault-syscall.test', 'qual_fault.test'], 'valgrind': ['drd/tests/std_list']}https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1355
Not sure if these always fail or are intermittent?
libinput failure should be addressed by Bruce's patch to enable UINPUT kernel option (I think I saw that in master-next, but now I don't).
The rest are intermittent, as latest master build only shows one different intermittent failure in glib (which I'm trying to reproduce with additional diagnostics):
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1356
Alex