Skip to contentSkip to conversation

gcc-sanitizers: Add ptest support for AddressSanitizer tests

PATCH In review [email protected] 3 participants 4 messages Sep 2, 2024 – Sep 4, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 gcc-sanitizers: Add ptest support for AddressSanitizer tests

From: Deepthi Hemraj <[email protected]>

[YOCTO #15338]
Enabled ptest integration in gcc-sanitizers.inc to facilitate the proper testing and execution of address sanitizer (ASan)

Defined the do_check task to execute ASan tests with proper environment setup and updated do_install_ptest_base to copy ASan test executables to the ptest directory. Included the run-ptest script to automate test execution.

The run-ptest script iterates through executable files copied from the host, runs each on the target, and logs the outcomes as either 'PASS' or 'XFAIL'. XFAIL is because these tests are expected to fail on the host system as per dejagnu as { dg-shouldfail "asan" }

---
 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-devtools/gcc/gcc-14.2.inc        |  2 +
 meta/recipes-devtools/gcc/gcc-sanitizers.inc  | 43 ++++++++++++++++
 meta/recipes-devtools/gcc/gcc/asan-exp.patch  | 51 +++++++++++++++++++
 meta/recipes-devtools/gcc/gcc/run-ptest       | 28 ++++++++++
 5 files changed, 125 insertions(+)
 create mode 100644 meta/recipes-devtools/gcc/gcc/asan-exp.patch
 create mode 100755 meta/recipes-devtools/gcc/gcc/run-ptest
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 7b8418654a..13b15b8647 100644
@@ -85,6 +85,7 @@ PTESTS_FAST = "\
     wayland \
     xz \
     zlib \
+    gcc-sanitizers \
 "
 PTESTS_FAST:append:libc-glibc = " glibc-y2038-tests"
 PTESTS_PROBLEMS:remove:libc-glibc = "glibc-y2038-tests"
meta/recipes-devtools/gcc/gcc-14.2.inc +2 -0
index f05484cfc0..72cd93f4bd 100644
@@ -68,6 +68,8 @@ SRC_URI = "${BASEURI} \
            file://0023-Fix-install-path-of-linux64.h.patch \
            file://0024-Avoid-hardcoded-build-paths-into-ppc-libgcc.patch \
            file://0025-gcc-testsuite-tweaks-for-mips-OE.patch \
+	   file://asan-exp.patch \
+	   file://run-ptest \
 "
 
 S = "${TMPDIR}/work-shared/gcc-${PV}-${PR}/${SOURCEDIR}"
meta/recipes-devtools/gcc/gcc-sanitizers.inc +43 -0

46 lines not on this page · raw message

meta/recipes-devtools/gcc/gcc/asan-exp.patch new file +51 -0

51 lines not on this page · raw message

meta/recipes-devtools/gcc/gcc/run-ptest new file +28 -0
new file mode 100755
index 0000000000..b8dcd16e95
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+# This script is used to run sanitizers test suites
+
+ptestdir=$(dirname "$(readlink -f "$0")")
+export SRC_PATH=$ptestdir
+
+cd $ptestdir/
+files=$(ls tests/sanitizers-ptest/*.exe)
+total_testcases=$(ls tests/sanitizers-ptest/*.exe | wc -l)
+passed=0
+failed=0
+
+for f in $files; do
+    $f >/dev/null 2>&1
+    retcode=$?
+    if [ $retcode -eq 0 ]; then
+        echo "PASS" : $f
+        ((passed++))
+    else
+        echo "XFAIL" : $f
+        ((failed++))
+    fi
+done
+
+echo "Total Testcases: $total_testcases"
+echo "Passed: $passed"
+echo "Xfailed: $failed"
Signature · 1 line
2.43.0
#1 lore raw

Patchtest results for [PATCH] gcc-sanitizers: Add ptest support for AddressSanitizer tests

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/gcc-sanitizers-Add-ptest-support-for-AddressSanitizer-tests.patch

FAIL: test Signed-off-by presence: A patch file has been added without a Signed-off-by tag: 'asan-exp.patch' (test_patch.TestPatch.test_signed_off_by_presence)

PASS: test CVE tag format (test_patch.TestPatch.test_cve_tag_format)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test Upstream-Status presence (test_patch.TestPatch.test_upstream_status_presence_format)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

#2 lore raw
In reply to [email protected]
Quoted 16 lines · Deepthi via lists.openembedded.org

From: Deepthi Hemraj <[email protected]>

[YOCTO #15338]
Enabled ptest integration in gcc-sanitizers.inc to facilitate the proper testing and execution of address sanitizer (ASan)

Defined the do_check task to execute ASan tests with proper environment setup and updated do_install_ptest_base to copy ASan test executables to the ptest directory. Included the run-ptest script to automate test execution.

The run-ptest script iterates through executable files copied from the host, runs each on the target, and logs the outcomes as either 'PASS' or 'XFAIL'. XFAIL is because these tests are expected to fail on the host system as per dejagnu as { dg-shouldfail "asan" }

---
 .../distro/include/ptest-packagelists.inc     |  1 + meta/recipes-devtools/gcc/gcc-14.2.inc        |  2 + meta/recipes-devtools/gcc/gcc-sanitizers.inc  | 43 ++++++++++++++++ meta/recipes-devtools/gcc/gcc/asan-exp.patch  | 51 +++++++++++++++++++ meta/recipes-devtools/gcc/gcc/run-ptest       | 28 ++++++++++

Quoted 3 lines
 5 files changed, 125 insertions(+)

 create mode 100644 meta/recipes-devtools/gcc/gcc/asan-exp.patch create mode 100755 meta/recipes-devtools/gcc/gcc/run-ptest

Would it make more sense to run this as part of the other gcc testsuites?

It looks like there is a lot of duplication in these changes including a lot of ptest code.

I'm also quite concerned at the buildpaths exceptions, build paths would be pointless in target packages so I'm not sure why those are present.

Cheers,

Richard

#3 lore raw

Hi Richard,

Thanks for checking the patch and your comment
"Would it make more sense to run this as part of the other gcc testsuites?"

Definitely Yes. We can add the sanitizer tests as part of the existing gcc testsuites.

However, the sanitizer test was intentionally kept separate to avoid unnecessary building and testing of sanitizers that are not required for most users.

The current activity was taken up when checking the sanitizers support for all targets. This testsuite will just test the sanitizers support quickly within few minutes. Running the complete testsuite to check the sanitizers would require few hours. Hence, we had planned and implemented the ptest specifically for sanitizer support.

However, we will add it as part of the complete testsuite if that seems most profitable for everyone. Please let us know your views on the same so that we can work towards the same.