openposix: Run the Open POSIX tests with kirk
cover openposix: Run the Open POSIX tests with kirk
Hi,
There are no runtest files for the Open POSIX tests, so kirk can not run them. Patch 1 installs the missing .args files.
Patch 2 generates runtest/openposix-* at install time. Each line calls a small wrapper that maps the Open POSIX exit codes to the LTP ones.
Tested with kirk on arm64 in QEMU: 1735 tests, 1 failure (pthread_rwlock_unlock_3-1).
Cheers,
Anders
Anders Roxell (2):
openposix: Install the .args files
openposix: Add runtest files for kirk
testcases/open_posix_testsuite/Makefile | 13 ++++++++- testcases/open_posix_testsuite/bin/Makefile | 6 +++- .../open_posix_testsuite/bin/openposix_run.sh | 29 +++++++++++++++++++ .../include/mk/config.mk.in | 3 ++ .../scripts/generate-makefiles.sh | 1 + .../scripts/generate-runtest.sh | 29 +++++++++++++++++++ 6 files changed, 79 insertions(+), 2 deletions(-) create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh
--
2.53.0
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 1/2 openposix: Install the .args files
run-tests.sh reads the arguments for a test from a .args file next to it, but make install does not copy the .args files. So on an installed system the two mqueues_multi_send_rev tests run without the number of threads and fail with a usage message.
Install the .args files together with the tests.
--- testcases/open_posix_testsuite/scripts/generate-makefiles.sh | 1 + 1 file changed, 1 insertion(+)
testcases/open_posix_testsuite/scripts/generate-makefiles.sh +1 -0
@@ -182,6 +182,7 @@ install: \$(INSTALL_DIR) run.sh \$(INSTALL_DIR)/\$\$file; \\ fi; \\ done + \$(if \$(wildcard \$(srcdir)/*.args),install -m 00644 \$(wildcard \$(srcdir)/*.args) \$(INSTALL_DIR)/) @if [ -d speculative ]; then \$(MAKE) -C speculative install; fi .PHONY: test
--
2.53.0
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] openposix: Install the .args files
Hi Anders,
Quoted 1 line · Anders Roxell
openposix: Install the .args files
--- [PATCH 2/2] ---
Review
Quoted 3 lines
+++
b/testcases/open_posix_testsuite/scripts/generate-runtest.sh(quoted diff) +2
-0+find "$suite" -name speculative -prune -o -type f \ + \( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \
Medium: On an in-place upgrade, this scan re-enables obsolete binaries such as pthread_detach_1-2, which was removed for testing undefined behavior and is absent from the current run.sh. Generate entries from the current build's install manifests rather than scanning all files left in the destination.
Verdict - Needs revision
---
Note:
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Regards,
LTP AI Reviewer
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] openposix: Install the .args files
Quoted 7 lines · On 2026-10-23 17:50, [email protected] wrote:
Hi Anders,
Quoted 1 line · Anders Roxell
openposix: Install the .args files
--- [PATCH 2/2] ---
Quoted 8 lines
Review
Quoted 3 lines
+++
b/testcases/open_posix_testsuite/scripts/generate-runtest.sh(quoted diff) +2
-0+find "$suite" -name speculative -prune -o -type f \ + \( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \Medium: On an in-place upgrade, this scan re-enables obsolete binaries such as pthread_detach_1-2, which was removed for testing undefined behavior and is absent from the current
run.sh. Generate entries from the current build's install manifests rather than scanning all files left in the destination.
This is about 2/2.
Yes, this looks like a real bug. make install never removes old files, so the find picks up anything that is left from an earlier install.
I send a v2 where the list comes from INSTALL_TARGETS in the generated Makefiles.
Cheers,
Anders
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH 2/2 openposix: Add runtest files for kirk
In reply to Anders RoxellLTP has no runtest files for the Open POSIX tests, so kirk can not run them. They are only run with run.sh, which prints its own summary.
Generate a runtest file for each suite at install time. The speculative tests are left out, like in run.sh, and the arguments from a test's .args file go on its runtest line. Each line calls a small wrapper, openposix_run.sh, which runs the test from its own directory like run-tests.sh does.
The wrapper also maps PTS_UNSUPPORTED and PTS_UNTESTED to TCONF. Without it kirk reports them as TWARN and TFAIL.
--- testcases/open_posix_testsuite/Makefile | 13 ++++++++- testcases/open_posix_testsuite/bin/Makefile | 6 +++- .../open_posix_testsuite/bin/openposix_run.sh | 29 +++++++++++++++++++ .../include/mk/config.mk.in | 3 ++ .../scripts/generate-runtest.sh | 29 +++++++++++++++++++ 5 files changed, 78 insertions(+), 2 deletions(-) create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh
testcases/open_posix_testsuite/Makefile +12 -1
@@ -74,7 +74,7 @@ generate-makefiles: distclean-makefiles $(top_srcdir)/scripts/generate-makefiles.sh .PHONY: install -install: bin-install conformance-install functional-install stress-install +install: bin-install conformance-install functional-install stress-install runtest-install .PHONY: test test: conformance-test functional-test stress-test
@@ -119,6 +119,17 @@ stress-test: @rm -f `if echo "$(LOGFILE)" | grep -q '^/'; then echo "$(LOGFILE)"; else echo "\`pwd\`/$(LOGFILE)"; fi`.$@ @$(TEST_MAKE) -C stress test +RUNTEST_SUITES= conformance functional stress + +.PHONY: runtest-install +runtest-install: $(addsuffix -install,$(RUNTEST_SUITES)) + @mkdir -p $(DESTDIR)/$(runtestdir) + @set -e; for suite in $(RUNTEST_SUITES); do \ + $(top_srcdir)/scripts/generate-runtest.sh \ + $(DESTDIR)/$(testdir) "$(testdir_rel)" $$suite \ + > $(DESTDIR)/$(runtestdir)/openposix-$$suite; \ + done + # Tools build and install targets. .PHONY: bin-install bin-install:
testcases/open_posix_testsuite/bin/Makefile +5 -1
@@ -16,7 +16,7 @@ clean: @rm -f t0.val .PHONY: install -install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin) +install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin) $(DESTDIR)/$(ltp_bindir) set -e; for file in $(INSTALL_BIN_TARGETS); do \ install -m 00755 $$file $(DESTDIR)/$(bindir)/$$file; \ done
@@ -27,9 +27,13 @@ install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin) install -m 00755 $$file $(DESTDIR)/$(testdir_bin)/$$file; \ done + install -m 00755 openposix_run.sh $(DESTDIR)/$(ltp_bindir)/ $(DESTDIR)/$(bindir): mkdir -p $@ $(DESTDIR)/$(testdir_bin): mkdir -p $@ + +$(DESTDIR)/$(ltp_bindir): + mkdir -p $@
testcases/open_posix_testsuite/bin/openposix_run.sh new file +29 -0
@@ -0,0 +1,29 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2026 Linaro Limited + +LTPROOT=${LTPROOT:-$(cd "$(dirname "$0")/../.." && pwd)} + +if [ $# -lt 1 ]; then + echo "Usage: $(basename "$0") <test path relative to LTPROOT> [args]" >&2 + exit 2 +fi + +test_path=$LTPROOT/$1 +test_file=${test_path##*/} +shift + +if [ ! -x "$test_path" ]; then + echo "$test_path is missing or not executable" >&2 + exit 2 +fi + +cd "${test_path%/*}" || exit 2 + +./"$test_file" "$@" +ret=$? + +case $ret in +4|5) exit 32 ;; +*) exit $ret ;; +esac
testcases/open_posix_testsuite/include/mk/config.mk.in +3 -0
@@ -14,4 +14,7 @@ testdir_rel := @testdir@ testdir := ${prefix}/${testdir_rel} testdir_bin := ${testdir}/bin +ltp_bindir := ${prefix}/testcases/bin +runtestdir := ${prefix}/runtest + CFLAGS += -std=c99 -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700 -Wall -Wextra
testcases/open_posix_testsuite/scripts/generate-runtest.sh new file +29 -0
@@ -0,0 +1,29 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2026 Linaro Limited + +if [ $# -ne 3 ]; then + echo "Usage: $(basename "$0") <installed testdir> <testdir relative to LTPROOT> <suite>" >&2 + exit 1 +fi + +installed_testdir=$1 +testdir_rel=$2 +suite=$3 + +cd "$installed_testdir" || exit 1 + +if [ ! -d "$suite" ]; then + echo "$installed_testdir/$suite does not exist" >&2 + exit 1 +fi + +find "$suite" -name speculative -prune -o -type f \ + \( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \ +while read -r test_path; do + test_name=${test_path##*/} + args_file=${test_path%/*}/${test_name%%.*}.args + args= + [ -f "$args_file" ] && read -r args < "$args_file" + echo "${test_name%.*} openposix_run.sh ${testdir_rel:+$testdir_rel/}$test_path${args:+ $args}" +done
--
2.53.0
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp