libc-test: add libc testsuite for musl
PATCH v2 1/1 libc-test: add libc testsuite for musl
libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project.
--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-core/musl/libc-test/run-ptest | 17 +++++++ meta/recipes-core/musl/libc-test_git.bb | 50 +++++++++++++++++++ 3 files changed, 68 insertions(+) create mode 100644 meta/recipes-core/musl/libc-test/run-ptest create mode 100644 meta/recipes-core/musl/libc-test_git.bb
meta/conf/distro/include/ptest-packagelists.inc +1 -0
@@ -105,6 +105,7 @@ PTESTS_SLOW = "\ PTESTS_SLOW:remove:riscv64 = "valgrind-ptest" PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest" +PTESTS_SLOW:append:libc-musl = " libc-test-ptest" # ruby-ptest \ # Timeout # lz4-ptest \ # Needs a rewrite
meta/recipes-core/musl/libc-test/run-ptest new file +17 -0
@@ -0,0 +1,17 @@ +#!/bin/sh + +cd /opt/libc-test +make cleanall +make run | tee libc-test.log + +echo "" +echo "--- ptest result ---" +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then + # Consolidate output format. + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" + grep '^FAIL src.*\.exe.*' libc-test.log \ + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' + exit 1 +else + echo "PASS: libc-test" +fi
meta/recipes-core/musl/libc-test_git.bb new file +50 -0
50 lines not on this page · raw message
Signature · 1 line
2.25.1
Quoted 31 lines · Chase Qi
libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project.
Signed-off-by Chase Qi <[email protected]>--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-core/musl/libc-test/run-ptest | 17 +++++++ meta/recipes-core/musl/libc-test_git.bb | 50 +++++++++++++++++++ 3 files changed, 68 insertions(+) create mode 100644 meta/recipes-core/musl/libc-test/run-ptest create mode 100644 meta/recipes-core/musl/libc-test_git.bbmeta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -105,6 +105,7 @@ PTESTS_SLOW = "\ PTESTS_SLOW:remove:riscv64 = "valgrind-ptest" PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest" +PTESTS_SLOW:append:libc-musl = " libc-test-ptest" # ruby-ptest \ # Timeout # lz4-ptest \ # Needs a rewritemeta/recipes-core/musl/libc-test/run-ptest new file +0
-0
meta/recipes-core/musl/libc-test/run-ptest line 2
Quoted 2 lines
meta/recipes-core/musl/libc-test/run-ptest new file +2
-0@@ -0,0 +1,17 @@ +#!/bin/sh +
Please use at least "set -e" to capture errors of failing commands.
meta/recipes-core/musl/libc-test/run-ptest line 5
Quoted 3 lines
meta/recipes-core/musl/libc-test/run-ptest new file +3
-0+cd /opt/libc-test +make cleanall +make run | tee libc-test.log
Depending how this handles errors, the return value from "make run" may need to be discarded, but I'd prefer this to be explicit if "set -e" is used.
meta/recipes-core/musl/libc-test/run-ptest line 16
Quoted 10 lines
meta/recipes-core/musl/libc-test/run-ptest new file +10
-0+echo "" +echo "--- ptest result ---" +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then + # Consolidate output format. + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main" + grep '^FAIL src.*\.exe.*' libc-test.log \ + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' + exit 1 +else + echo "PASS: libc-test"
Would be nice if something would check that tests were actually executed.
Cheers,
-Mikko
Hi Mikko,
Thanks for the review.
Quoted 25 lines · Mikko Rapeli
Quoted 23 lines · Chase Qi
libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project.
Signed-off-by Chase Qi <[email protected]>--- .../distro/include/ptest-packagelists.inc | 1 + meta/recipes-core/musl/libc-test/run-ptest | 17 +++++++ meta/recipes-core/musl/libc-test_git.bb | 50 +++++++++++++++++++ 3 files changed, 68 insertions(+) create mode 100644 meta/recipes-core/musl/libc-test/run-ptest create mode 100644 meta/recipes-core/musl/libc-test_git.bbmeta/conf/distro/include/ptest-packagelists.inc +1
-0@@ -105,6 +105,7 @@ PTESTS_SLOW = "\ PTESTS_SLOW:remove:riscv64 = "valgrind-ptest" PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest" +PTESTS_SLOW:append:libc-musl = " libc-test-ptest"
Quoted 14 lines
meta/recipes-core/musl/libc-test/run-ptest line 2
Quoted 10 lines
meta/conf/distro/include/ptest-packagelists.inc +0
-0# ruby-ptest \ # Timeout # lz4-ptest \ # Needs a rewritemeta/recipes-core/musl/libc-test/run-ptest new file +2
-0@@ -0,0 +1,17 @@ +#!/bin/sh +Please use at least "set -e" to capture errors of failing commands.
+1 Enabled in v3.
Quoted 8 lines
meta/recipes-core/musl/libc-test/run-ptest line 5
Quoted 3 lines
meta/recipes-core/musl/libc-test/run-ptest new file +3
-0+cd /opt/libc-test +make cleanall +make run | tee libc-test.logDepending how this handles errors, the return value from "make run" may need to be discarded, but I'd prefer this to be explicit if "set -e" is used.
Yes. In case the command return with non-zero, I switched to "make run || true" in v3 for the following log parsing.
Quoted 5 lines
Quoted 5 lines
meta/recipes-core/musl/libc-test/run-ptest new file +5
-0+echo "" +echo "--- ptest result ---" +if grep -q '^FAIL src.*\.exe.*' libc-test.log; then + # Consolidate output format. + # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main"
Quoted 8 lines
meta/recipes-core/musl/libc-test/run-ptest line 16
Quoted 5 lines
meta/recipes-core/musl/libc-test/run-ptest new file +5
-0+ grep '^FAIL src.*\.exe.*' libc-test.log \ + | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||' + exit 1 +else + echo "PASS: libc-test"Would be nice if something would check that tests were actually executed.
I think src/REPORT can be used to check if 'make run' is finished correctly. Per the logic here
http://nsz.repo.hu/git/?p=libc-test;a=blob;f=Makefile;h=cf2ab2b2c8ebef0a9eab8cb06efc7bb5d989cc83;hb=HEAD#l120,
I understand the file is generated after all sub-tests are finished. Would you please review v3?
Thanks,
Chase
Quoted 4 lines
Cheers,
-Mikko
meta/recipes-core/musl/libc-test_git.bb
Quoted 7 lines · Chase Qi via lists.openembedded.org
meta/recipes-core/musl/libc-test_git.bb new file +6
-0+SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" +SRC_URI = " \ + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ + file://run-ptest \ +" +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test”
The project’s web site at https://wiki.musl-libc.org/libc-test.html says the git URL is git://repo.or.cz/libc-test, so that’s the URL we should fetch from.
Also as this project doesn’t appear to have versions, set:
PV=“0+git${SRCPV}”.
This way the version has the git SHA in, and if they ever release a version it will be greater than 0 and sort correctly.
Ross
Hi Ross,
Thanks for the review.
Quoted 12 lines · Ross Burton
meta/recipes-core/musl/libc-test_git.bb
Quoted 7 lines · Chase Qi via lists.openembedded.org
meta/recipes-core/musl/libc-test_git.bb new file +6
-0+SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398" +SRC_URI = " \ + git://nsz.repo.hu:49100/repo/libc-test;branch=master \ + file://run-ptest \ +" +MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test”The project’s web site at https://wiki.musl-libc.org/libc-test.html says the git URL is
git://repo.or.cz/libc-test, so that’s the URL we should fetch from.
The summary page https://repo.or.cz/libc-test.git shows it is a mirror URL of git://nsz.repo.hu:49100/repo/libc-test. In v3, I removed the MIRRORS line. https://repo.or.cz/libc-test.git is used directly now. At least, there is a team maintaining the services.
Quoted 5 lines
Also as this project doesn’t appear to have versions, set:
PV=“0+git${SRCPV}”.
This way the version has the git SHA in, and if they ever release a version it will be greater than 0 and sort correctly.
Make sense, added in v3. Thanks for the suggestion.
- Chase
Quoted 2 lines
Ross