Skip to contentSkip to conversation

openposix: Run the Open POSIX tests with kirk

PATCH Superseded Anders Roxell 2 participants 5 messages Oct 23 – Oct 29

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover openposix: Run the Open POSIX tests with kirk

Hi,

There are no runtest files for the Open POSIX tests, so kirk can not run them. Patch 1 installs the missing .args files.

Patch 2 generates runtest/openposix-* at install time. Each line calls a small wrapper that maps the Open POSIX exit codes to the LTP ones.

Tested with kirk on arm64 in QEMU: 1735 tests, 1 failure (pthread_rwlock_unlock_3-1).

Cheers,
Anders

Anders Roxell (2):
openposix: Install the .args files
openposix: Add runtest files for kirk

 testcases/open_posix_testsuite/Makefile       | 13 ++++++++-
 testcases/open_posix_testsuite/bin/Makefile   |  6 +++-
 .../open_posix_testsuite/bin/openposix_run.sh | 29 +++++++++++++++++++
 .../include/mk/config.mk.in                   |  3 ++
 .../scripts/generate-makefiles.sh             |  1 +
 .../scripts/generate-runtest.sh               | 29 +++++++++++++++++++
 6 files changed, 79 insertions(+), 2 deletions(-)
 create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh
 create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh

--
2.53.0

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

PATCH 1/2 openposix: Install the .args files

run-tests.sh reads the arguments for a test from a .args file next to it, but make install does not copy the .args files. So on an installed system the two mqueues_multi_send_rev tests run without the number of threads and fail with a usage message.

Install the .args files together with the tests.

---
 testcases/open_posix_testsuite/scripts/generate-makefiles.sh | 1 +
 1 file changed, 1 insertion(+)
testcases/open_posix_testsuite/scripts/generate-makefiles.sh +1 -0
index 1c12a456cc41..19c781ba8054 100755
@@ -182,6 +182,7 @@ install: \$(INSTALL_DIR) run.sh
 				\$(INSTALL_DIR)/\$\$file;	\\
 		fi;						\\
 	done
+	\$(if \$(wildcard \$(srcdir)/*.args),install -m 00644 \$(wildcard \$(srcdir)/*.args) \$(INSTALL_DIR)/)
 	@if [ -d speculative ]; then \$(MAKE) -C speculative install; fi
 
 .PHONY: test

--
2.53.0

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

Re: [LTP] openposix: Install the .args files

Hi Anders,

Quoted 1 line · Anders Roxell

openposix: Install the .args files

--- [PATCH 2/2] ---

Review

Quoted 3 lines

+++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh

(quoted diff) +2 -0
+find "$suite" -name speculative -prune -o -type f \
+	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \

Medium: On an in-place upgrade, this scan re-enables obsolete binaries such as pthread_detach_1-2, which was removed for testing undefined behavior and is absent from the current run.sh. Generate entries from the current build's install manifests rather than scanning all files left in the destination.

Verdict - Needs revision

---
Note:

The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Regards,
LTP AI Reviewer

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Re: [LTP] openposix: Install the .args files

Quoted 7 lines · On 2026-10-23 17:50, [email protected] wrote:

Hi Anders,

Quoted 1 line · Anders Roxell

openposix: Install the .args files

--- [PATCH 2/2] ---

Quoted 8 lines

Review

Quoted 3 lines

+++ b/testcases/open_posix_testsuite/scripts/generate-runtest.sh

(quoted diff) +2 -0
+find "$suite" -name speculative -prune -o -type f \
+	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \

Medium: On an in-place upgrade, this scan re-enables obsolete binaries such as pthread_detach_1-2, which was removed for testing undefined behavior and is absent from the current run.sh. Generate entries from the current build's install manifests rather than scanning all files left in the destination.

This is about 2/2.

Yes, this looks like a real bug. make install never removes old files, so the find picks up anything that is left from an earlier install.

I send a v2 where the list comes from INSTALL_TARGETS in the generated Makefiles.

Cheers,
Anders

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

PATCH 2/2 openposix: Add runtest files for kirk

In reply to Anders Roxell

LTP has no runtest files for the Open POSIX tests, so kirk can not run them. They are only run with run.sh, which prints its own summary.

Generate a runtest file for each suite at install time. The speculative tests are left out, like in run.sh, and the arguments from a test's .args file go on its runtest line. Each line calls a small wrapper, openposix_run.sh, which runs the test from its own directory like run-tests.sh does.

The wrapper also maps PTS_UNSUPPORTED and PTS_UNTESTED to TCONF. Without it kirk reports them as TWARN and TFAIL.

---
 testcases/open_posix_testsuite/Makefile       | 13 ++++++++-
 testcases/open_posix_testsuite/bin/Makefile   |  6 +++-
 .../open_posix_testsuite/bin/openposix_run.sh | 29 +++++++++++++++++++
 .../include/mk/config.mk.in                   |  3 ++
 .../scripts/generate-runtest.sh               | 29 +++++++++++++++++++
 5 files changed, 78 insertions(+), 2 deletions(-)
 create mode 100755 testcases/open_posix_testsuite/bin/openposix_run.sh
 create mode 100755 testcases/open_posix_testsuite/scripts/generate-runtest.sh
testcases/open_posix_testsuite/Makefile +12 -1
index 4bead182170d..db141ba3cd8b 100644
@@ -74,7 +74,7 @@ generate-makefiles: distclean-makefiles
 		$(top_srcdir)/scripts/generate-makefiles.sh
 
 .PHONY: install
-install: bin-install conformance-install functional-install stress-install
+install: bin-install conformance-install functional-install stress-install runtest-install
 
 .PHONY: test
 test: conformance-test functional-test stress-test
@@ -119,6 +119,17 @@ stress-test:
 	@rm -f `if echo "$(LOGFILE)" | grep -q '^/'; then echo "$(LOGFILE)"; else echo "\`pwd\`/$(LOGFILE)"; fi`.$@
 	@$(TEST_MAKE) -C stress test
 
+RUNTEST_SUITES=		conformance functional stress
+
+.PHONY: runtest-install
+runtest-install: $(addsuffix -install,$(RUNTEST_SUITES))
+	@mkdir -p $(DESTDIR)/$(runtestdir)
+	@set -e; for suite in $(RUNTEST_SUITES); do \
+		$(top_srcdir)/scripts/generate-runtest.sh \
+			$(DESTDIR)/$(testdir) "$(testdir_rel)" $$suite \
+			> $(DESTDIR)/$(runtestdir)/openposix-$$suite; \
+	done
+
 # Tools build and install targets.
 .PHONY: bin-install
 bin-install:
testcases/open_posix_testsuite/bin/Makefile +5 -1
index 262a6567c5b6..5dfff99effcb 100644
@@ -16,7 +16,7 @@ clean:
 	@rm -f t0.val
 
 .PHONY: install
-install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin)
+install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin) $(DESTDIR)/$(ltp_bindir)
 	set -e; for file in $(INSTALL_BIN_TARGETS); do           \
 		install -m 00755 $$file $(DESTDIR)/$(bindir)/$$file; \
 	done
@@ -27,9 +27,13 @@ install: clean $(DESTDIR)/$(bindir) $(DESTDIR)/$(testdir_bin)
 		install -m 00755 $$file $(DESTDIR)/$(testdir_bin)/$$file; \
 	done
 
+	install -m 00755 openposix_run.sh $(DESTDIR)/$(ltp_bindir)/
 
 $(DESTDIR)/$(bindir):
 	mkdir -p $@
 
 $(DESTDIR)/$(testdir_bin):
 	mkdir -p $@
+
+$(DESTDIR)/$(ltp_bindir):
+	mkdir -p $@
testcases/open_posix_testsuite/bin/openposix_run.sh new file +29 -0
new file mode 100755
index 000000000000..43c52a38d956
@@ -0,0 +1,29 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linaro Limited
+
+LTPROOT=${LTPROOT:-$(cd "$(dirname "$0")/../.." && pwd)}
+
+if [ $# -lt 1 ]; then
+	echo "Usage: $(basename "$0") <test path relative to LTPROOT> [args]" >&2
+	exit 2
+fi
+
+test_path=$LTPROOT/$1
+test_file=${test_path##*/}
+shift
+
+if [ ! -x "$test_path" ]; then
+	echo "$test_path is missing or not executable" >&2
+	exit 2
+fi
+
+cd "${test_path%/*}" || exit 2
+
+./"$test_file" "$@"
+ret=$?
+
+case $ret in
+4|5) exit 32 ;;
+*) exit $ret ;;
+esac
testcases/open_posix_testsuite/include/mk/config.mk.in +3 -0
index 20422e0f1047..fae80116bbf7 100644
@@ -14,4 +14,7 @@ testdir_rel := @testdir@
 testdir     := ${prefix}/${testdir_rel}
 testdir_bin := ${testdir}/bin
 
+ltp_bindir  := ${prefix}/testcases/bin
+runtestdir  := ${prefix}/runtest
+
 CFLAGS  += -std=c99 -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700 -Wall -Wextra
testcases/open_posix_testsuite/scripts/generate-runtest.sh new file +29 -0
new file mode 100755
index 000000000000..e561a7adcf21
@@ -0,0 +1,29 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-or-later
+# Copyright (c) 2026 Linaro Limited
+
+if [ $# -ne 3 ]; then
+	echo "Usage: $(basename "$0") <installed testdir> <testdir relative to LTPROOT> <suite>" >&2
+	exit 1
+fi
+
+installed_testdir=$1
+testdir_rel=$2
+suite=$3
+
+cd "$installed_testdir" || exit 1
+
+if [ ! -d "$suite" ]; then
+	echo "$installed_testdir/$suite does not exist" >&2
+	exit 1
+fi
+
+find "$suite" -name speculative -prune -o -type f \
+	\( -name '*.run-test' -o -name '*.sh' ! -name run.sh \) -print | sort | \
+while read -r test_path; do
+	test_name=${test_path##*/}
+	args_file=${test_path%/*}/${test_name%%.*}.args
+	args=
+	[ -f "$args_file" ] && read -r args < "$args_file"
+	echo "${test_name%.*} openposix_run.sh ${testdir_rel:+$testdir_rel/}$test_path${args:+ $args}"
+done

--
2.53.0

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp