Skip to contentSkip to conversation

add coreutils ptest to oe-core

PATCH Superseded Trevor Gamblin 2 participants 5 messages Nov 23, 2019 – Nov 28, 2019

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover add coreutils ptest to oe-core

From: Trevor Gamblin <[email protected]>

Sample test results:

core-image-minimal:

MACHINE    | PASS | FAIL | SKIP | TOTAL | TIME (m) |
qemux86-64 |  483 |    0 |  132 |   615 |      3.8 |
qemuarm64  |  483 |    0 |  132 |   615 |       70 |

core-image-sato:

MACHINE    | PASS | FAIL | SKIP | TOTAL | Time (m) |
qemux86-64 |  484 |    0 |  131 |   615 |      3.9 |
qemuarm64  |  484 |    0 |  131 |   615 |       70 |

Trevor Gamblin (2):
coreutils: add ptest ptest-packagelists.inc: add coreutils to SLOW list

 .../distro/include/ptest-packagelists.inc     |  1 +
 .../coreutils/coreutils/run-ptest             | 11 +++++
 meta/recipes-core/coreutils/coreutils_8.31.bb | 40 +++++++++++++++++++
 3 files changed, 52 insertions(+)
 create mode 100755 meta/recipes-core/coreutils/coreutils/run-ptest
Signature · 1 line
2.23.0
#1 lore raw

PATCH v2 1/2 coreutils: add ptest

From: Trevor Gamblin <[email protected]>

coreutils has a large number of tests, (and potential RDEPENDS to support them), including tests added with the flags RUN_EXPENSIVE_TESTS and RUN_VERY_EXPENSIVE_TESTS that add significant run time for their coverage. The RUN_VERY_EXPENSIVE_TESTS option has been omitted from the run-ptest script to reduce some of this run time, even though this increases the SKIP count in the results. Also, the ptest directory for coreutils is given blanket permissions at runtime with chmod -R 777 to ensure that the user created for the tests will be able to run the test scripts and create the necessary files in the process.

There is still room to improve the results of this ptest without the aforementioned additions. Of the tests marked SKIP, there are 30 tests that are currently counted as SKIP because they require sudo permissions, and another 21 that require membership in multiple user groups. It is important to know that coreutils has tests for both root and nonroot users. Testing showed that 42 tests are skipped when running as root versus 30 when running as a non-root user, so the decision was made to run the suite as the latter.

Finally, note that the ptest suite for coreutils has a short runtime on x86-64/kvm (approximately 4.5 minutes), in contrast to the arm64 time (~70 minutes with RUN_EXPENSIVE_TESTS=yes enabled).

---
 .../coreutils/coreutils/run-ptest             | 11 +++++
 meta/recipes-core/coreutils/coreutils_8.31.bb | 40 +++++++++++++++++++
 2 files changed, 51 insertions(+)
 create mode 100755 meta/recipes-core/coreutils/coreutils/run-ptest
meta/recipes-core/coreutils/coreutils/run-ptest new file +11 -0
new file mode 100755
index 0000000000..6c53329e55
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+COREUTILSLIB=@libdir@/coreutils
+LOG="${COREUTILSLIB}/ptest/coreutils_ptest_$(date +%Y%m%d-%H%M%S).log"
+
+addgroup usergroup1
+adduser --ingroup usergroup1 tester
+
+su tester -c "cd ${COREUTILSLIB}/ptest && make check-TESTS RUN_EXPENSIVE_TESTS=yes top_srcdir=. srcdir=." 2>&1 | tee -a ${LOG}
+deluser tester 
+delgroup usergroup1
meta/recipes-core/coreutils/coreutils_8.31.bb +40 -0
index 57b2c1bdba..ce94dd0edb 100644
@@ -18,6 +18,7 @@ SRC_URI = "${GNU_MIRROR}/coreutils/${BP}.tar.xz \
            file://0001-uname-report-processor-and-hardware-correctly.patch \
            file://disable-ls-output-quoting.patch \
            file://0001-local.mk-fix-cross-compiling-problem.patch \
+           file://run-ptest \
           "
 
 SRC_URI_append_libc-musl = "file://strtod_fix_clash_with_strtold.patch"
@@ -143,3 +144,42 @@ python __anonymous() {
 }
 
 BBCLASSEXTEND = "native nativesdk"
+
+inherit ptest
+
+FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
+ 
+RDEPENDS_${PN}-ptest += "bash findutils gawk gdb libconvert-asn1-perl liberror-perl libmodule-build-perl libtimedate-perl liburi-perl make perl perl-module-file-stat python sed shadow strace"
+
+do_install_ptest () {
+	install -d ${D}${PTEST_PATH}/tests
+	cp -r ${S}/tests/* ${D}${PTEST_PATH}/tests
+	sed -i 's/ginstall/install/g'  `grep -R ginstall ${D}${PTEST_PATH}/tests | awk -F: '{print $1}' | uniq`
+	install -d ${D}${PTEST_PATH}/build-aux
+	install ${S}/build-aux/test-driver ${D}${PTEST_PATH}/build-aux/
+	cp ${B}/Makefile ${D}${PTEST_PATH}/
+	cp ${S}/init.cfg ${D}${PTEST_PATH}/
+	cp -r ${B}/src ${D}${PTEST_PATH}/
+	cp -r ${S}/src/*.c ${D}${PTEST_PATH}/src
+	cd ${D}${PTEST_PATH}
+	tar -czvf ${D}${PTEST_PATH}/src.tar.gz ./src
+	cd -
+	sed -i '/^VPATH/s/= .*$/= ./g' ${D}${PTEST_PATH}/Makefile
+	sed -i '/^PROGRAMS/s/^/#/g' ${D}${PTEST_PATH}/Makefile
+	sed -i '/^Makefile: /s/^.*$/Makefile:/g' ${D}${PTEST_PATH}/Makefile
+	sed -i '/^abs_srcdir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
+	sed -i '/^abs_top_builddir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
+	sed -i '/^abs_top_srcdir/s/= .*$/= \$\{PWD\}/g' ${D}${PTEST_PATH}/Makefile
+	sed -i '/^built_programs/s/ginstall/install/g' ${D}${PTEST_PATH}/Makefile
+    chmod -R 777 ${D}${PTEST_PATH}
+	# Disable subcase stty-pairs.sh, it will cause test framework hang
+	sed -i '/stty-pairs.sh/d' ${D}${PTEST_PATH}/Makefile
+	install ${B}/src/getlimits ${D}/${bindir}
+    # handle multilib
+    sed -i s:@libdir@:${libdir}:g ${D}${PTEST_PATH}/run-ptest
+}
+
+FILES_${PN}-ptest += "${bindir}/getlimits"
+
+INSANE_SKIP_${PN}-ptest += "ldflags"
+INSANE_SKIP_${PN}-ptest += "rpaths"
Signature · 1 line
2.23.0
#2 lore raw

PATCH v2 2/2 ptest-packagelists.inc: add coreutils to SLOW list

In reply to Trevor Gamblin

From: Trevor Gamblin <[email protected]>

---
 meta/conf/distro/include/ptest-packagelists.inc | 1 +
 1 file changed, 1 insertion(+)
meta/conf/distro/include/ptest-packagelists.inc +1 -0
index bb4724cf52..57ee92522d 100644
@@ -60,6 +60,7 @@ PTESTS_FAST = "\
 PTESTS_SLOW = "\
     bash-ptest \
     busybox-ptest \
+    coreutils-ptest \
     dbus-test-ptest \
     e2fsprogs-ptest \
     glib-2.0-ptest \
Signature · 1 line
2.23.0
#3 lore raw
In reply to Trevor Gamblin
Quoted 5 lines · Trevor Gamblin

From: Trevor Gamblin <[email protected]>

Sample test results:

core-image-minimal:

Quoted 8 lines
MACHINE    | PASS | FAIL | SKIP | TOTAL | TIME (m) |
qemux86-64 |  483 |    0 |  132 |   615 |      3.8 |
qemuarm64  |  483 |    0 |  132 |   615 |       70 |

core-image-sato:

MACHINE    | PASS | FAIL | SKIP | TOTAL | Time (m) |
qemux86-64 |  484 |    0 |  131 |   615 |      3.9 |
qemuarm64  |  484 |    0 |  131 |   615 |       70 |

I do like this patch but I think this is causing the reproducibile builds selftest to fail:

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/515

as coreutils ptest is in the list.

I think there are other patches in master-next contributing to this too but this one is an issue.

Its also causing a size overflow for genericx86-64:

https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/1298

but I believe Anuj and Ross are working through changes which would address this issue.

Cheers,

Richard

#4 lore raw
Quoted 24 lines · Richard Purdie
Quoted 5 lines · Trevor Gamblin

From: Trevor Gamblin <[email protected]>

Sample test results:

core-image-minimal:

Quoted 8 lines
MACHINE    | PASS | FAIL | SKIP | TOTAL | TIME (m) |
qemux86-64 |  483 |    0 |  132 |   615 |      3.8 |
qemuarm64  |  483 |    0 |  132 |   615 |       70 |

core-image-sato:

MACHINE    | PASS | FAIL | SKIP | TOTAL | Time (m) |
qemux86-64 |  484 |    0 |  131 |   615 |      3.9 |
qemuarm64  |  484 |    0 |  131 |   615 |       70 |

I do like this patch but I think this is causing the reproducibile builds selftest to fail:

https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/515

as coreutils ptest is in the list.

I think there are other patches in master-next contributing to this too but this one is an issue.

Quoted 6 lines

Its also causing a size overflow for genericx86-64:

https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/1298

but I believe Anuj and Ross are working through changes which would address this issue.

Right. I've made some modifications to a v3 that should also help. I still need to work on reproducibility though, so I'll hold off on sending it again for a while.

Quoted 5 lines

Cheers,

Richard