lttng-ust: add ptest support
PATCH 1/1 lttng-ust: add ptest support
Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.
The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.
Tested on qemuriscv64 with ptest-runner.
--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-kernel/lttng/lttng-ust/run-ptest | 31 ++++++++++ meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 58 ++++++++++++++++++- 3 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -64,6 +64,7 @@ PTESTS_FAST = "\ libxslt \ libyaml \ logrotate \ + lttng-ust \ lua \ lzo \ m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +31 -0
@@ -0,0 +1,31 @@ +#!/bin/sh + +PTEST_PATH=$(dirname "$0") +cd "$PTEST_PATH/tests" || exit 1 + +export UST_TESTS_SRCDIR="$PTEST_PATH/tests" +export UST_TESTS_BUILDDIR="$PTEST_PATH/tests" + +TESTS=" +unit/libringbuffer/test_shm +unit/gcc-weak-hidden/test_gcc_weak_hidden +unit/libcommon/test_get_cpu_mask_from_sysfs +unit/libcommon/test_get_max_cpuid_from_mask +unit/libcommon/test_get_max_cpuid_from_sysfs +unit/libcommon/test_get_possible_cpus_array_len +unit/libmsgpack/test_msgpack +unit/pthread_name/test_pthread_name +unit/snprintf/test_snprintf +unit/ust-elf/test_ust_elf +unit/ust-error/test_ust_error +unit/ust-utils/test_ust_utils +" + +for test in $TESTS; do + [ -x "$test" ] || continue + if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then + echo "PASS: $test" + else + echo "FAIL: $test" + fi +done
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +57 -1
73 lines not on this page · raw message
Signature · 1 line
2.43.0
Quoted 6 lines · Pratik Farkase
Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.
Quoted 8 lines
The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.
Tested on qemuriscv64 with ptest-runner.
Signed-off-by Pratik Farkase <[email protected]>---
Hi Pratik,
It looks like one of the tests is failing:
WARNING: core-image-ptest-lttng-ust-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
self.do_ptestrunner()
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 121, in do_ptestrunner
self.fail(failmsg)
AssertionError:
Failed ptests:
{'lttng-ust': ['unit/ust-error/test_ust_error']}
https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1469
https://autobuilder.yoctoproject.org/valkyrie/#/builders/110/builds/1491
Can you have a look at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 1/1 lttng-ust: add ptest support
In reply to Pratik FarkaseBuild and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.
The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.
Tested on qemuriscv64 with ptest-runner.
---
v2: Skip unit/ust-error on musl. It segfaults because lttng_ust_strerror()
negates its argument (undefined for the INT_MIN the test passes) and then reads its lookup table out of bounds; this only shows up on musl.
.../distro/include/ptest-packagelists.inc | 1 + meta/recipes-kernel/lttng/lttng-ust/run-ptest | 31 +++++++++ meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 63 ++++++++++++++++++- 3 files changed, 94 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -64,6 +64,7 @@ PTESTS_FAST = "\ libxslt \ libyaml \ logrotate \ + lttng-ust \ lua \ lzo \ m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +31 -0
@@ -0,0 +1,31 @@ +#!/bin/sh + +PTEST_PATH=$(dirname "$0") +cd "$PTEST_PATH/tests" || exit 1 + +export UST_TESTS_SRCDIR="$PTEST_PATH/tests" +export UST_TESTS_BUILDDIR="$PTEST_PATH/tests" + +TESTS=" +unit/libringbuffer/test_shm +unit/gcc-weak-hidden/test_gcc_weak_hidden +unit/libcommon/test_get_cpu_mask_from_sysfs +unit/libcommon/test_get_max_cpuid_from_mask +unit/libcommon/test_get_max_cpuid_from_sysfs +unit/libcommon/test_get_possible_cpus_array_len +unit/libmsgpack/test_msgpack +unit/pthread_name/test_pthread_name +unit/snprintf/test_snprintf +unit/ust-elf/test_ust_elf +unit/ust-error/test_ust_error +unit/ust-utils/test_ust_utils +" + +for test in $TESTS; do + [ -x "$test" ] || continue + if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then + echo "PASS: $test" + else + echo "FAIL: $test" + fi +done
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +59 -1
75 lines not on this page · raw message
Signature · 1 line
2.43.0
Patchtest results for [OE-core][PATCH v2] lttng-ust: add ptest support
Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:
---
Testing patch /home/patchtest/share/mboxes/v2-lttng-ust-add-ptest-support.patch
FAIL: test mbox format: Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am (test_mbox.TestMbox.test_mbox_format)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)
SKIP: pretest pylint: Python-unidiff parse error (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: Parse error Hunk is longer than expected (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: Parse error Hunk is longer than expected (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: Parse error Hunk is longer than expected (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: Python-unidiff parse error (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
---
Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!
PATCH v2 1/1 lttng-ust: add ptest support
In reply to Pratik FarkaseBuild and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.
The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.
Tested on qemuriscv64 with ptest-runner.
---
Notes:
v2: Skip unit/ust-error on musl. It segfaults because lttng_ust_strerror()
negates its argument (undefined for the INT_MIN the test passes) and then reads its lookup table out of bounds; this only shows up on musl.
.../distro/include/ptest-packagelists.inc | 1 + meta/recipes-kernel/lttng/lttng-ust/run-ptest | 31 ++++++++++ meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 60 ++++++++++++++++++- 3 files changed, 91 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -64,6 +64,7 @@ PTESTS_FAST = "\ libxslt \ libyaml \ logrotate \ + lttng-ust \ lua \ lzo \ m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +31 -0
@@ -0,0 +1,31 @@ +#!/bin/sh + +PTEST_PATH=$(dirname "$0") +cd "$PTEST_PATH/tests" || exit 1 + +export UST_TESTS_SRCDIR="$PTEST_PATH/tests" +export UST_TESTS_BUILDDIR="$PTEST_PATH/tests" + +TESTS=" +unit/libringbuffer/test_shm +unit/gcc-weak-hidden/test_gcc_weak_hidden +unit/libcommon/test_get_cpu_mask_from_sysfs +unit/libcommon/test_get_max_cpuid_from_mask +unit/libcommon/test_get_max_cpuid_from_sysfs +unit/libcommon/test_get_possible_cpus_array_len +unit/libmsgpack/test_msgpack +unit/pthread_name/test_pthread_name +unit/snprintf/test_snprintf +unit/ust-elf/test_ust_elf +unit/ust-error/test_ust_error +unit/ust-utils/test_ust_utils +" + +for test in $TESTS; do + [ -x "$test" ] || continue + if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then + echo "PASS: $test" + else + echo "FAIL: $test" + fi +done
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +59 -1
75 lines not on this page · raw message
Signature · 1 line
2.43.0
Quoted 4 lines · Pratik Farkase via openembedded-core
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +4
-0+TESTS=" +unit/libringbuffer/test_shm +unit/gcc-weak-hidden/test_gcc_weak_hidden +unit/libcommon/test_get_cpu_mask_from_sysfs
meta/recipes-kernel/lttng/lttng-ust/run-ptest line 22
Quoted 10 lines
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +10
-0+unit/libcommon/test_get_max_cpuid_from_mask +unit/libcommon/test_get_max_cpuid_from_sysfs +unit/libcommon/test_get_possible_cpus_array_len +unit/libmsgpack/test_msgpack +unit/pthread_name/test_pthread_name +unit/snprintf/test_snprintf +unit/ust-elf/test_ust_elf +unit/ust-error/test_ust_error +unit/ust-utils/test_ust_utils +"
Hello Pratik,
hardcoding a list of tests creates a maintainability problem. If upstream adds more tests, they will not be run. If upstream removes tests, this creates friction in version updates. Can you find ways to avoid it? One possibility is to ship the makefile, and issue 'make check' (or something similar) that would run tests from the list embedded in the makefile. Another option is to create a list on the fly with 'find'.
+LTTNG_UST_PTEST_DIRS = " \
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb do_install:append() { · line 86
Quoted 11 lines
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +10
-0+ unit/libringbuffer \ + unit/gcc-weak-hidden \ + unit/libcommon \ + unit/libmsgpack \ + unit/pthread_name \ + unit/snprintf \ + unit/ust-elf \ + unit/ust-error \ + unit/ust-utils \ +"
This has a similar problem. Rather than go over a hardcoded list of directories, can we just start from the top dir and visit everything? If too much is installed, then we can look into ways to trim it.
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb do_install:append() { · line 109
Quoted 13 lines
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +5
-0+ for d in ${LTTNG_UST_PTEST_DIRS}; do + mf=${B}/tests/$d/Makefile + install -d ${D}${PTEST_PATH}/tests/$d + for p in $(makefile-getvar $mf noinst_PROGRAMS); do + ${B}/libtool --mode=install install -m 0755 ${B}/tests/$d/$p${D}${PTEST_PATH}/tests/$d/
(quoted diff) +2
-0+ done + for s in $(makefile-getvar $mf dist_noinst_SCRIPTS
dist_check_SCRIPTS); dometa/recipes-kernel/lttng/lttng-ust_2.16.0.bb +3
-0+ install -m 0755 ${S}/tests/$d/$s ${D}${PTEST_PATH}/tests/$d/ + done + done
Here's perhaps the most important point. You've done a lot of ptest additions, but for autotools-driven projects this always results in hacky, difficult to maintain code. What we would really like to see is support for installable tests in the autotools upstream. Similar to how gnome project does it. E.g. extend the macros to add test installation targets, and making a script that would run them from the target. Then we could convert all the ptests to use that, a far cleaner, maintainable solution. We can even carry such a patch ourselves if autotools upstream rejects it (they can be notoriously stubborn in such matters).
Cheers,
Alex
PATCH v3 1/1 lttng-ust: add ptest support
In reply to Pratik FarkaseBuild and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.
The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.
Tested on qemuriscv64 with ptest-runner.
---
Notes:
v3: Derive the test list and directories dynamically from the autotools
TESTS variable (makefile-getvar) instead of hardcoding them, per review. This also picks up unit/ust-utils/test_ust_utils_cxx automatically. Drop the buildpaths INSANE_SKIP, which is no longer needed.
v2: Skip unit/ust-error on musl (lttng_ust_strerror() negates its argument,
undefined for INT_MIN, then reads its lookup table out of bounds).
.../distro/include/ptest-packagelists.inc | 1 + meta/recipes-kernel/lttng/lttng-ust/run-ptest | 16 +++++ meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 61 ++++++++++++++++++- 3 files changed, 77 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -64,6 +64,7 @@ PTESTS_FAST = "\ libxslt \ libyaml \ logrotate \ + lttng-ust \ lua \ lzo \ m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +16 -0
@@ -0,0 +1,16 @@ +#!/bin/sh + +PTEST_PATH=$(dirname "$0") +cd "$PTEST_PATH/tests" || exit 1 + +export UST_TESTS_SRCDIR="$PTEST_PATH/tests" +export UST_TESTS_BUILDDIR="$PTEST_PATH/tests" + +while read -r test; do + [ -x "$test" ] || continue + if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then + echo "PASS: $test" + else + echo "FAIL: $test" + fi +done < tests.list
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +60 -1
76 lines not on this page · raw message
Signature · 1 line
2.43.0