Skip to contentSkip to conversation

libc-test: add libc testsuite for musl

PATCH Superseded Chase Qi 3 participants 5 messages Nov 4, 2022 – Nov 8, 2022

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Chase Qi

author
#0 lore raw

PATCH v2 1/1 libc-test: add libc testsuite for musl

libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project.

---
 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-core/musl/libc-test/run-ptest    | 17 +++++++
 meta/recipes-core/musl/libc-test_git.bb       | 50 +++++++++++++++++++
 3 files changed, 68 insertions(+)
 create mode 100644 meta/recipes-core/musl/libc-test/run-ptest
 create mode 100644 meta/recipes-core/musl/libc-test_git.bb
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 32b0e5297a..3c9ff43bd9 100644
@@ -105,6 +105,7 @@ PTESTS_SLOW = "\
 
 PTESTS_SLOW:remove:riscv64 = "valgrind-ptest"
 PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest"
+PTESTS_SLOW:append:libc-musl = " libc-test-ptest"
 
 #    ruby-ptest \ # Timeout
 #    lz4-ptest \ # Needs a rewrite
meta/recipes-core/musl/libc-test/run-ptest new file +17 -0
new file mode 100644
index 0000000000..a30cee590e
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cd /opt/libc-test
+make cleanall
+make run | tee libc-test.log
+
+echo ""
+echo "--- ptest result ---"
+if grep -q '^FAIL src.*\.exe.*' libc-test.log; then
+    # Consolidate output format.
+    # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main"
+    grep '^FAIL src.*\.exe.*' libc-test.log \
+        | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||'
+    exit 1
+else
+    echo "PASS: libc-test"
+fi
meta/recipes-core/musl/libc-test_git.bb new file +50 -0

50 lines not on this page · raw message

Signature · 1 line
2.25.1
#1 lore raw
Quoted 31 lines · Chase Qi

libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project.

---
 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-core/musl/libc-test/run-ptest    | 17 +++++++
 meta/recipes-core/musl/libc-test_git.bb       | 50 +++++++++++++++++++
 3 files changed, 68 insertions(+)
 create mode 100644 meta/recipes-core/musl/libc-test/run-ptest
 create mode 100644 meta/recipes-core/musl/libc-test_git.bb
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 32b0e5297a..3c9ff43bd9 100644
@@ -105,6 +105,7 @@ PTESTS_SLOW = "\
 
 PTESTS_SLOW:remove:riscv64 = "valgrind-ptest"
 PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest"
+PTESTS_SLOW:append:libc-musl = " libc-test-ptest"
 
 #    ruby-ptest \ # Timeout
 #    lz4-ptest \ # Needs a rewrite
meta/recipes-core/musl/libc-test/run-ptest new file +0 -0
new file mode 100644
index 0000000000..a30cee590e

meta/recipes-core/musl/libc-test/run-ptest line 2

Quoted 2 lines
meta/recipes-core/musl/libc-test/run-ptest new file +2 -0
@@ -0,0 +1,17 @@
+#!/bin/sh
+

Please use at least "set -e" to capture errors of failing commands.

meta/recipes-core/musl/libc-test/run-ptest line 5

Quoted 3 lines
meta/recipes-core/musl/libc-test/run-ptest new file +3 -0
+cd /opt/libc-test
+make cleanall
+make run | tee libc-test.log

Depending how this handles errors, the return value from "make run" may need to be discarded, but I'd prefer this to be explicit if "set -e" is used.

meta/recipes-core/musl/libc-test/run-ptest line 16

Quoted 10 lines
meta/recipes-core/musl/libc-test/run-ptest new file +10 -0
+echo ""
+echo "--- ptest result ---"
+if grep -q '^FAIL src.*\.exe.*' libc-test.log; then
+    # Consolidate output format.
+    # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main"
+    grep '^FAIL src.*\.exe.*' libc-test.log \
+        | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||'
+    exit 1
+else
+    echo "PASS: libc-test"

Would be nice if something would check that tests were actually executed.

Cheers,

-Mikko

author
#2 lore raw

Hi Mikko,

Thanks for the review.

Quoted 25 lines · Mikko Rapeli
Quoted 23 lines · Chase Qi

libc-test is a collection of unit test to measure the correctness and robustness of a C/POSIX standard library implementation. It is developed as part of the musl project.

---
 .../distro/include/ptest-packagelists.inc     |  1 +
 meta/recipes-core/musl/libc-test/run-ptest    | 17 +++++++
 meta/recipes-core/musl/libc-test_git.bb       | 50 +++++++++++++++++++
 3 files changed, 68 insertions(+)
 create mode 100644 meta/recipes-core/musl/libc-test/run-ptest
 create mode 100644 meta/recipes-core/musl/libc-test_git.bb
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index 32b0e5297a..3c9ff43bd9 100644
@@ -105,6 +105,7 @@ PTESTS_SLOW = "\
 
 PTESTS_SLOW:remove:riscv64 = "valgrind-ptest"
 PTESTS_PROBLEMS:append:riscv64 = "valgrind-ptest"
+PTESTS_SLOW:append:libc-musl = " libc-test-ptest"
 
Quoted 14 lines

meta/recipes-core/musl/libc-test/run-ptest line 2

Quoted 10 lines
meta/conf/distro/include/ptest-packagelists.inc +0 -0
 #    ruby-ptest \ # Timeout
 #    lz4-ptest \ # Needs a rewrite
meta/recipes-core/musl/libc-test/run-ptest new file +2 -0
new file mode 100644
index 0000000000..a30cee590e
@@ -0,0 +1,17 @@
+#!/bin/sh
+

Please use at least "set -e" to capture errors of failing commands.

+1 Enabled in v3.

Quoted 8 lines

meta/recipes-core/musl/libc-test/run-ptest line 5

Quoted 3 lines
meta/recipes-core/musl/libc-test/run-ptest new file +3 -0
+cd /opt/libc-test
+make cleanall
+make run | tee libc-test.log

Depending how this handles errors, the return value from "make run" may need to be discarded, but I'd prefer this to be explicit if "set -e" is used.

Yes. In case the command return with non-zero, I switched to "make run || true" in v3 for the following log parsing.

Quoted 5 lines
Quoted 5 lines
meta/recipes-core/musl/libc-test/run-ptest new file +5 -0
+echo ""
+echo "--- ptest result ---"
+if grep -q '^FAIL src.*\.exe.*' libc-test.log; then
+    # Consolidate output format.
+    # e.g. "FAIL src/api/main.exe [status 1]" -> "FAIL: api_main"
Quoted 8 lines

meta/recipes-core/musl/libc-test/run-ptest line 16

Quoted 5 lines
meta/recipes-core/musl/libc-test/run-ptest new file +5 -0
+    grep '^FAIL src.*\.exe.*' libc-test.log \
+        | sed 's|^FAIL src/|FAIL: |;s|/|_|;s|\.exe.*\]||'
+    exit 1
+else
+    echo "PASS: libc-test"

Would be nice if something would check that tests were actually executed.

I think src/REPORT can be used to check if 'make run' is finished correctly. Per the logic here
http://nsz.repo.hu/git/?p=libc-test;a=blob;f=Makefile;h=cf2ab2b2c8ebef0a9eab8cb06efc7bb5d989cc83;hb=HEAD#l120,
I understand the file is generated after all sub-tests are finished. Would you please review v3?

Thanks,
Chase

Quoted 4 lines

Cheers,

-Mikko

#3 lore raw
In reply to Chase Qi

meta/recipes-core/musl/libc-test_git.bb

Quoted 7 lines · Chase Qi via lists.openembedded.org
meta/recipes-core/musl/libc-test_git.bb new file +6 -0
+SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398"
+SRC_URI = " \
+    git://nsz.repo.hu:49100/repo/libc-test;branch=master \
+    file://run-ptest \
+"
+MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test”

The project’s web site at https://wiki.musl-libc.org/libc-test.html says the git URL is git://repo.or.cz/libc-test, so that’s the URL we should fetch from.

Also as this project doesn’t appear to have versions, set:

PV=“0+git${SRCPV}”.

This way the version has the git SHA in, and if they ever release a version it will be greater than 0 and sort correctly.

Ross

author
#4 lore raw

Hi Ross,

Thanks for the review.

Quoted 12 lines · Ross Burton

meta/recipes-core/musl/libc-test_git.bb

Quoted 7 lines · Chase Qi via lists.openembedded.org
meta/recipes-core/musl/libc-test_git.bb new file +6 -0
+SRCREV = "18e28496adee3d84fefdda6efcb9c5b8996a2398"
+SRC_URI = " \
+    git://nsz.repo.hu:49100/repo/libc-test;branch=master \
+    file://run-ptest \
+"
+MIRRORS += "git://nsz.repo.hu:49100/repo/libc-test git://repo.or.cz/libc-test”

The project’s web site at https://wiki.musl-libc.org/libc-test.html says the git URL is git://repo.or.cz/libc-test, so that’s the URL we should fetch from.

The summary page https://repo.or.cz/libc-test.git shows it is a mirror URL of git://nsz.repo.hu:49100/repo/libc-test. In v3, I removed the MIRRORS line. https://repo.or.cz/libc-test.git is used directly now. At least, there is a team maintaining the services.

Quoted 5 lines

Also as this project doesn’t appear to have versions, set:

PV=“0+git${SRCPV}”.

This way the version has the git SHA in, and if they ever release a version it will be greater than 0 and sort correctly.

Make sense, added in v3. Thanks for the suggestion.

- Chase

Quoted 2 lines

Ross