runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run only when ptest-runner is availalble;
PATCH v3 1/3 runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run only when ptest-runner is availalble;
Previously the test would execute only when all available ptests for packages in the image were installed; some of those tests may be broken, never finish, take a very long time or simply irrelevant to the user who wants to check ptests of only a few specific packages, and does so by listing them explicitly via IMAGE_INSTALL_append or similar.
Presence of ptest-runner means there is at least one ptest package installed as they pull it in via a class dependency; ptest-runner is not generally installed otherwise.
--- meta/lib/oeqa/runtime/cases/ptest.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
meta/lib/oeqa/runtime/cases/ptest.py +4 -1
@@ -48,9 +48,12 @@ class PtestRunnerTest(OERuntimeTestCase): @OETestID(1600) @skipIfNotFeature('ptest', 'Test requires ptest to be in DISTRO_FEATURES') - @skipIfNotFeature('ptest-pkgs', 'Test requires ptest-pkgs to be in IMAGE_FEATURES') @OETestDepends(['ssh.SSHTest.test_ssh']) def test_ptestrunner(self): + status, output = self.target.run('which ptest-runner', 0) + if status != 0: + self.skipTest("No -ptest packages are installed in the image") + import datetime test_log_dir = self.td.get('TEST_LOG_DIR', '')
Signature · 1 line
2.15.1
PATCH v3 2/3 testimage.bbclass: add ptest to the list of runtime tests whenever possible
If no ptest packages are installed in the image, the test does nothing; if ptest packages are installed in the image, then they should be run without user having to enable that manually.
--- meta/classes/testimage.bbclass | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
meta/classes/testimage.bbclass +6 -6
@@ -44,16 +44,16 @@ DEVTESTSUITE = "gcc kernelmodule ldd" DEFAULT_TEST_SUITES = "${MINTESTSUITE} auto" DEFAULT_TEST_SUITES_pn-core-image-minimal = "${MINTESTSUITE}" DEFAULT_TEST_SUITES_pn-core-image-minimal-dev = "${MINTESTSUITE}" -DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE} perl python logrotate" +DEFAULT_TEST_SUITES_pn-core-image-full-cmdline = "${NETTESTSUITE} perl python logrotate ptest" DEFAULT_TEST_SUITES_pn-core-image-x11 = "${MINTESTSUITE}" -DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam parselogs ${RPMTESTSUITE}" +DEFAULT_TEST_SUITES_pn-core-image-lsb = "${NETTESTSUITE} pam parselogs ${RPMTESTSUITE} ptest" DEFAULT_TEST_SUITES_pn-core-image-sato = "${NETTESTSUITE} connman xorg parselogs ${RPMTESTSUITE} \ - ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)}" + ${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'python', '', d)} ptest" DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "${NETTESTSUITE} buildcpio buildlzip buildgalculator \ - connman ${DEVTESTSUITE} logrotate perl parselogs python ${RPMTESTSUITE} xorg" -DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam perl python parselogs ${RPMTESTSUITE}" + connman ${DEVTESTSUITE} logrotate perl parselogs python ${RPMTESTSUITE} xorg ptest" +DEFAULT_TEST_SUITES_pn-core-image-lsb-dev = "${NETTESTSUITE} pam perl python parselogs ${RPMTESTSUITE} ptest" DEFAULT_TEST_SUITES_pn-core-image-lsb-sdk = "${NETTESTSUITE} buildcpio buildlzip buildgalculator \ - connman ${DEVTESTSUITE} logrotate pam parselogs perl python ${RPMTESTSUITE}" + connman ${DEVTESTSUITE} logrotate pam parselogs perl python ${RPMTESTSUITE} ptest" DEFAULT_TEST_SUITES_pn-meta-toolchain = "auto" # aarch64 has no graphics
Signature · 1 line
2.15.1
PATCH v3 3/3 runtime/cases/ptest.py: fail when ptests fail on target
In reply to Alexander KanavinThat's the whole point isn't it? Previously this testcase succeeded even if some of the underlying on-target tests failed; the only way to find out if anything was wrong was to manually inspect the logs.
--- meta/lib/oeqa/runtime/cases/ptest.py | 8 ++++++++ 1 file changed, 8 insertions(+)
meta/lib/oeqa/runtime/cases/ptest.py +8 -0
@@ -83,3 +83,11 @@ class PtestRunnerTest(OERuntimeTestCase): # Remove the old link to create a new one os.remove(ptest_log_dir_link) os.symlink(os.path.basename(ptest_log_dir), ptest_log_dir_link) + + failed_tests = {} + for section in parse_result.result_dict: + failed_testcases = [ test for test, result in parse_result.result_dict[section] if result == 'fail' ] + if failed_testcases: + failed_tests[section] = failed_testcases + + self.assertFalse(failed_tests, msg = "Failed ptests: %s" %(str(failed_tests)))
Signature · 1 line
2.15.1
✗ patchtest: failure for "[v3] runtime/cases/ptest.py: d..." and 2 more
In reply to Alexander Kanavin== Series Details ==
Series: "[v3] runtime/cases/ptest.py: d..." and 2 more
Revision: 1
URL : https://patchwork.openembedded.org/series/10387/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:
* Patch [v3,1/3] runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run only when ptest-runner is availalble; Issue Commit shortlog is too long [test_shortlog_length] Suggested fix Edit shortlog so that it is 90 characters or less (currently 110 characters)
If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe