Skip to contentSkip to conversation

lttng-ust: add ptest support

PATCH Superseded Pratik Farkase 4 participants 7 messages Oct 29 – Nov 16

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lttng-ust: add ptest support

Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.

The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.

Tested on qemuriscv64 with ptest-runner.

---
 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-kernel/lttng/lttng-ust/run-ptest | 31 ++++++++++
 meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 58 ++++++++++++++++++-
 3 files changed, 89 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 74591a34f2..d5357d8732 100644
@@ -64,6 +64,7 @@ PTESTS_FAST = "\
     libxslt \
     libyaml \
     logrotate \
+    lttng-ust \
     lua \
     lzo \
     m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +31 -0
new file mode 100644
index 0000000000..468562eb70
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+PTEST_PATH=$(dirname "$0")
+cd "$PTEST_PATH/tests" || exit 1
+
+export UST_TESTS_SRCDIR="$PTEST_PATH/tests"
+export UST_TESTS_BUILDDIR="$PTEST_PATH/tests"
+
+TESTS="
+unit/libringbuffer/test_shm
+unit/gcc-weak-hidden/test_gcc_weak_hidden
+unit/libcommon/test_get_cpu_mask_from_sysfs
+unit/libcommon/test_get_max_cpuid_from_mask
+unit/libcommon/test_get_max_cpuid_from_sysfs
+unit/libcommon/test_get_possible_cpus_array_len
+unit/libmsgpack/test_msgpack
+unit/pthread_name/test_pthread_name
+unit/snprintf/test_snprintf
+unit/ust-elf/test_ust_elf
+unit/ust-error/test_ust_error
+unit/ust-utils/test_ust_utils
+"
+
+for test in $TESTS; do
+    [ -x "$test" ] || continue
+    if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then
+        echo "PASS: $test"
+    else
+        echo "FAIL: $test"
+    fi
+done
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +57 -1

73 lines not on this page · raw message

Signature · 1 line
2.43.0
#1 lore raw
Quoted 6 lines · Pratik Farkase

Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.

Quoted 8 lines

The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.

Tested on qemuriscv64 with ptest-runner.

---

Hi Pratik,

It looks like one of the tests is failing:

WARNING: core-image-ptest-lttng-ust-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
self.do_ptestrunner()
File "/srv/pokybuild/yocto-worker/qemuarm64-musl-ptest/build/layers/openembedded-core/meta/lib/oeqa/runtime/cases/ptest.py", line 121, in do_ptestrunner
self.fail(failmsg)
AssertionError:
Failed ptests:
{'lttng-ust': ['unit/ust-error/test_ust_error']}

https://autobuilder.yoctoproject.org/valkyrie/#/builders/109/builds/1469
https://autobuilder.yoctoproject.org/valkyrie/#/builders/110/builds/1491

Can you have a look at the issue?

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#2 lore raw

PATCH v2 1/1 lttng-ust: add ptest support

In reply to Pratik Farkase

Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.

The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.

Tested on qemuriscv64 with ptest-runner.

---
v2: Skip unit/ust-error on musl. It segfaults because lttng_ust_strerror()
negates its argument (undefined for the INT_MIN the test passes) and then reads its lookup table out of bounds; this only shows up on musl.

 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-kernel/lttng/lttng-ust/run-ptest | 31 +++++++++
 meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 63 ++++++++++++++++++-
 3 files changed, 94 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 74591a34f2..d5357d8732 100644
@@ -64,6 +64,7 @@ PTESTS_FAST = "\
     libxslt \
     libyaml \
     logrotate \
+    lttng-ust \
     lua \
     lzo \
     m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +31 -0
new file mode 100644
index 0000000000..468562eb70
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+PTEST_PATH=$(dirname "$0")
+cd "$PTEST_PATH/tests" || exit 1
+
+export UST_TESTS_SRCDIR="$PTEST_PATH/tests"
+export UST_TESTS_BUILDDIR="$PTEST_PATH/tests"
+
+TESTS="
+unit/libringbuffer/test_shm
+unit/gcc-weak-hidden/test_gcc_weak_hidden
+unit/libcommon/test_get_cpu_mask_from_sysfs
+unit/libcommon/test_get_max_cpuid_from_mask
+unit/libcommon/test_get_max_cpuid_from_sysfs
+unit/libcommon/test_get_possible_cpus_array_len
+unit/libmsgpack/test_msgpack
+unit/pthread_name/test_pthread_name
+unit/snprintf/test_snprintf
+unit/ust-elf/test_ust_elf
+unit/ust-error/test_ust_error
+unit/ust-utils/test_ust_utils
+"
+
+for test in $TESTS; do
+    [ -x "$test" ] || continue
+    if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then
+        echo "PASS: $test"
+    else
+        echo "FAIL: $test"
+    fi
+done
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +59 -1

75 lines not on this page · raw message

Signature · 1 line
2.43.0
#3 lore raw

Patchtest results for [OE-core][PATCH v2] lttng-ust: add ptest support

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/v2-lttng-ust-add-ptest-support.patch

FAIL: test mbox format: Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am (test_mbox.TestMbox.test_mbox_format)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: Python-unidiff parse error (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: Parse error Hunk is longer than expected (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: Parse error Hunk is longer than expected (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: Parse error Hunk is longer than expected (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: Python-unidiff parse error (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

#4 lore raw

PATCH v2 1/1 lttng-ust: add ptest support

In reply to Pratik Farkase

Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.

The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.

Tested on qemuriscv64 with ptest-runner.

---

Notes:
v2: Skip unit/ust-error on musl. It segfaults because lttng_ust_strerror()
negates its argument (undefined for the INT_MIN the test passes) and then reads its lookup table out of bounds; this only shows up on musl.

 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-kernel/lttng/lttng-ust/run-ptest | 31 ++++++++++
 meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 60 ++++++++++++++++++-
 3 files changed, 91 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 74591a34f2..d5357d8732 100644
@@ -64,6 +64,7 @@ PTESTS_FAST = "\
     libxslt \
     libyaml \
     logrotate \
+    lttng-ust \
     lua \
     lzo \
     m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +31 -0
new file mode 100644
index 0000000000..468562eb70
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+PTEST_PATH=$(dirname "$0")
+cd "$PTEST_PATH/tests" || exit 1
+
+export UST_TESTS_SRCDIR="$PTEST_PATH/tests"
+export UST_TESTS_BUILDDIR="$PTEST_PATH/tests"
+
+TESTS="
+unit/libringbuffer/test_shm
+unit/gcc-weak-hidden/test_gcc_weak_hidden
+unit/libcommon/test_get_cpu_mask_from_sysfs
+unit/libcommon/test_get_max_cpuid_from_mask
+unit/libcommon/test_get_max_cpuid_from_sysfs
+unit/libcommon/test_get_possible_cpus_array_len
+unit/libmsgpack/test_msgpack
+unit/pthread_name/test_pthread_name
+unit/snprintf/test_snprintf
+unit/ust-elf/test_ust_elf
+unit/ust-error/test_ust_error
+unit/ust-utils/test_ust_utils
+"
+
+for test in $TESTS; do
+    [ -x "$test" ] || continue
+    if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then
+        echo "PASS: $test"
+    else
+        echo "FAIL: $test"
+    fi
+done
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +59 -1

75 lines not on this page · raw message

Signature · 1 line
2.43.0
#5 lore raw
Quoted 4 lines · Pratik Farkase via openembedded-core
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +4 -0
+TESTS="
+unit/libringbuffer/test_shm
+unit/gcc-weak-hidden/test_gcc_weak_hidden
+unit/libcommon/test_get_cpu_mask_from_sysfs

meta/recipes-kernel/lttng/lttng-ust/run-ptest line 22

Quoted 10 lines
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +10 -0
+unit/libcommon/test_get_max_cpuid_from_mask
+unit/libcommon/test_get_max_cpuid_from_sysfs
+unit/libcommon/test_get_possible_cpus_array_len
+unit/libmsgpack/test_msgpack
+unit/pthread_name/test_pthread_name
+unit/snprintf/test_snprintf
+unit/ust-elf/test_ust_elf
+unit/ust-error/test_ust_error
+unit/ust-utils/test_ust_utils
+"

Hello Pratik,

hardcoding a list of tests creates a maintainability problem. If upstream adds more tests, they will not be run. If upstream removes tests, this creates friction in version updates. Can you find ways to avoid it? One possibility is to ship the makefile, and issue 'make check' (or something similar) that would run tests from the list embedded in the makefile. Another option is to create a list on the fly with 'find'.

+LTTNG_UST_PTEST_DIRS = " \

meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb do_install:append() { · line 86

Quoted 11 lines
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +10 -0
+    unit/libringbuffer \
+    unit/gcc-weak-hidden \
+    unit/libcommon \
+    unit/libmsgpack \
+    unit/pthread_name \
+    unit/snprintf \
+    unit/ust-elf \
+    unit/ust-error \
+    unit/ust-utils \
+"

This has a similar problem. Rather than go over a hardcoded list of directories, can we just start from the top dir and visit everything? If too much is installed, then we can look into ways to trim it.

meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb do_install:append() { · line 109

Quoted 13 lines
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +5 -0
+    for d in ${LTTNG_UST_PTEST_DIRS}; do
+        mf=${B}/tests/$d/Makefile
+        install -d ${D}${PTEST_PATH}/tests/$d
+        for p in $(makefile-getvar $mf noinst_PROGRAMS); do
+            ${B}/libtool --mode=install install -m 0755 ${B}/tests/$d/$p

${D}${PTEST_PATH}/tests/$d/

(quoted diff) +2 -0
+        done
+        for s in $(makefile-getvar $mf dist_noinst_SCRIPTS

dist_check_SCRIPTS); do

meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +3 -0
+            install -m 0755 ${S}/tests/$d/$s ${D}${PTEST_PATH}/tests/$d/
+        done
+    done

Here's perhaps the most important point. You've done a lot of ptest additions, but for autotools-driven projects this always results in hacky, difficult to maintain code. What we would really like to see is support for installable tests in the autotools upstream. Similar to how gnome project does it. E.g. extend the macros to add test installation targets, and making a script that would run them from the target. Then we could convert all the ptests to use that, a far cleaner, maintainable solution. We can even carry such a patch ourselves if autotools upstream rejects it (they can be notoriously stubborn in such matters).

Cheers,
Alex

#6 lore raw

PATCH v3 1/1 lttng-ust: add ptest support

In reply to Pratik Farkase

Build and run the lttng-ust unit test suite under ptest, modelled on the liburcu ptest support: the run-ptest script executes each TAP test from the tests/unit subdirectories and reports PASS/FAIL. The compile tests, benchmarks and the abi0-conflict regression test (unstable under QEMU) are excluded.

The ust-elf test ships prebuilt, intentionally foreign-architecture ELF fixtures, so they are kept from being stripped and the related QA checks are relaxed for the ptest package.

Tested on qemuriscv64 with ptest-runner.

---

Notes:
v3: Derive the test list and directories dynamically from the autotools
TESTS variable (makefile-getvar) instead of hardcoding them, per review. This also picks up unit/ust-utils/test_ust_utils_cxx automatically. Drop the buildpaths INSANE_SKIP, which is no longer needed.
v2: Skip unit/ust-error on musl (lttng_ust_strerror() negates its argument,
undefined for INT_MIN, then reads its lookup table out of bounds).

 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-kernel/lttng/lttng-ust/run-ptest | 16 +++++
 meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb | 61 ++++++++++++++++++-
 3 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-kernel/lttng/lttng-ust/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 74591a34f2..d5357d8732 100644
@@ -64,6 +64,7 @@ PTESTS_FAST = "\
     libxslt \
     libyaml \
     logrotate \
+    lttng-ust \
     lua \
     lzo \
     m4 \
meta/recipes-kernel/lttng/lttng-ust/run-ptest new file +16 -0
new file mode 100644
index 0000000000..b1f85cef81
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+PTEST_PATH=$(dirname "$0")
+cd "$PTEST_PATH/tests" || exit 1
+
+export UST_TESTS_SRCDIR="$PTEST_PATH/tests"
+export UST_TESTS_BUILDDIR="$PTEST_PATH/tests"
+
+while read -r test; do
+    [ -x "$test" ] || continue
+    if ( cd "$(dirname "$test")" && ./"$(basename "$test")" ); then
+        echo "PASS: $test"
+    else
+        echo "FAIL: $test"
+    fi
+done < tests.list
meta/recipes-kernel/lttng/lttng-ust_2.16.0.bb +60 -1

76 lines not on this page · raw message

Signature · 1 line
2.43.0