Kselftest integration into Kernel CI - Part 1
Subsystems (5)
Shuah Khan
maintainercover Kselftest integration into Kernel CI - Part 1
This patch series consists of first round of fixes to integrate Kselftest into Kernel CI.
You can find full list of problems in my announcement I sent out last week:
https://lkml.org/lkml/2020/2/27/2221
These fixes to android and seccomp tests address relocatable support. However, they will still leave the source directory dirty.
android test does headers_install in source directory. This is an easier problem to fix. seccomp on the other hand builds fixdep scripts under scripts/basic and installs headers in the source directory. It is linked to solving bpf relocatable build issue which I haven't given it a lot of thought for now.
There is no dependency on source directory for run-time which is what we want.
I will apply these kernelci topic branch for testing git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git
Shuah Khan (4):
selftests: Fix kselftest O=objdir build from cluttering top level
objdir
selftests: Fix seccomp to support relocatable build (O=objdir)
selftests: android: ion: Fix ionmap_test compile error
selftests: android: Fix custom install from skipping test progs
tools/testing/selftests/Makefile | 4 ++-- tools/testing/selftests/android/Makefile | 2 +- tools/testing/selftests/android/ion/Makefile | 2 +- tools/testing/selftests/seccomp/Makefile | 16 +++------------- 4 files changed, 7 insertions(+), 17 deletions(-)
Signature · 1 line
2.20.1
Shuah Khan
maintainerPATCH 1/4 selftests: Fix kselftest O=objdir build from cluttering top level objdir
make kselftest-all O=objdir builds create generated objects in objdir. This clutters the top level directory with kselftest objects. Fix it to create sub-directory under objdir for kselftest objects.
--- tools/testing/selftests/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/Makefile +2 -2
@@ -91,7 +91,7 @@ override LDFLAGS = override MAKEFLAGS = endif -# Append kselftest to KBUILD_OUTPUT to avoid cluttering +# Append kselftest to KBUILD_OUTPUT and O to avoid cluttering # KBUILD_OUTPUT with selftest objects and headers installed # by selftests Makefile or lib.mk. ifdef building_out_of_srctree
@@ -99,7 +99,7 @@ override LDFLAGS = endif ifneq ($(O),) - BUILD := $(O) + BUILD := $(O)/kselftest else ifneq ($(KBUILD_OUTPUT),) BUILD := $(KBUILD_OUTPUT)/kselftest
Signature · 1 line
2.20.1
Re: [PATCH 1/4] selftests: Fix kselftest O=objdir build from cluttering top level objdir
Quoted 5 lines · Shuah Khan
make kselftest-all O=objdir builds create generated objects in objdir. This clutters the top level directory with kselftest objects. Fix it to create sub-directory under objdir for kselftest objects.
Signed-off-by Shuah Khan <[email protected]>
Only somewhat related to this patch, another problem that wasn't in your doci is that the current O= support doesn't support relative paths.
For example, using O=/tmp/build-arm64 works, but O=build-arm64 doesn't. Try this:
$ make ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- O=build-arm64 defconfig $ make ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- O=build-arm64 kselftest-all make[1]: Entering directory '/work/kernel/linux/build-arm64' make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=arm64 -C ../../.. headers_install make[4]: ../scripts/Makefile.build: No such file or directory make[4]: *** No rule to make target '../scripts/Makefile.build'. Stop. Makefile:500: recipe for target 'scripts_basic' failed make[3]: *** [scripts_basic] Error 2 Makefile:151: recipe for target 'khdr' failed make[2]: *** [khdr] Error 2 /work/kernel/linux/Makefile:1220: recipe for target 'kselftest-all' failed make[1]: *** [kselftest-all] Error 2 make[1]: Leaving directory '/work/kernel/linux/build-arm64' Makefile:179: recipe for target 'sub-make' failed make: *** [sub-make] Error 2
Kevin
Quoted 28 lines
--- tools/testing/selftests/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)tools/testing/selftests/Makefile +2
-2@@ -91,7 +91,7 @@ override LDFLAGS = override MAKEFLAGS = endif -# Append kselftest to KBUILD_OUTPUT to avoid cluttering +# Append kselftest to KBUILD_OUTPUT and O to avoid cluttering # KBUILD_OUTPUT with selftest objects and headers installed # by selftests Makefile or lib.mk. ifdef building_out_of_srctree@@ -99,7 +99,7 @@ override LDFLAGS = endif ifneq ($(O),) - BUILD := $(O) + BUILD := $(O)/kselftest else ifneq ($(KBUILD_OUTPUT),) BUILD := $(KBUILD_OUTPUT)/kselftestSignature · 1 line
2.20.1
Shuah Khan
maintainerRe: [PATCH 1/4] selftests: Fix kselftest O=objdir build from cluttering top level objdir
Quoted 11 lines · Kevin Hilman
Quoted 5 lines · Shuah Khan
make kselftest-all O=objdir builds create generated objects in objdir. This clutters the top level directory with kselftest objects. Fix it to create sub-directory under objdir for kselftest objects.
Signed-off-by Shuah Khan <[email protected]>Only somewhat related to this patch, another problem that wasn't in your doci is that the current O= support doesn't support relative paths.
Yes I am aware of it and it is in the document as something that will be addressed later.
"Note: Relative paths don’t work - supporting relative paths breaks work-flows e.g:
powerpc. Explore fix. Compile work-flows. Not planning to support at the moment."
Quoted 20 lines
For example, using O=/tmp/build-arm64 works, but O=build-arm64 doesn't. Try this:
$ make ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- O=build-arm64 defconfig $ make ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- O=build-arm64 kselftest-all make[1]: Entering directory '/work/kernel/linux/build-arm64' make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=arm64 -C ../../.. headers_install make[4]: ../scripts/Makefile.build: No such file or directory make[4]: *** No rule to make target '../scripts/Makefile.build'. Stop. Makefile:500: recipe for target 'scripts_basic' failed make[3]: *** [scripts_basic] Error 2 Makefile:151: recipe for target 'khdr' failed make[2]: *** [khdr] Error 2 /work/kernel/linux/Makefile:1220: recipe for target 'kselftest-all' failed make[1]: *** [kselftest-all] Error 2 make[1]: Leaving directory '/work/kernel/linux/build-arm64' Makefile:179: recipe for target 'sub-make' failed make: *** [sub-make] Error 2
I am looking to address build and install issues first.
thanks,
-- Shuah
Re: [PATCH 1/4] selftests: Fix kselftest O=objdir build from cluttering top level objdir
Quoted 13 lines · Shuah Khan
Quoted 11 lines · Kevin Hilman
Quoted 5 lines · Shuah Khan
make kselftest-all O=objdir builds create generated objects in objdir. This clutters the top level directory with kselftest objects. Fix it to create sub-directory under objdir for kselftest objects.
Signed-off-by Shuah Khan <[email protected]>Only somewhat related to this patch, another problem that wasn't in your doci is that the current O= support doesn't support relative paths.
Quoted 5 lines
Yes I am aware of it and it is in the document as something that will be addressed later.
"Note: Relative paths don’t work - supporting relative paths breaks work-flows e.g:
powerpc. Explore fix. Compile work-flows. Not planning to support at the moment."
Hi,
Sorry to be dense but where is this documented (as quoted above)?
Thanks.
Quoted 25 lines
Quoted 20 lines
For example, using O=/tmp/build-arm64 works, but O=build-arm64 doesn't. Try this:
$ make ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- O=build-arm64 defconfig $ make ARCH=arm64 HOSTCC=gcc CROSS_COMPILE=aarch64-linux-gnu- O=build-arm64 kselftest-all make[1]: Entering directory '/work/kernel/linux/build-arm64' make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=arm64 -C ../../.. headers_install make[4]: ../scripts/Makefile.build: No such file or directory make[4]: *** No rule to make target '../scripts/Makefile.build'. Stop. Makefile:500: recipe for target 'scripts_basic' failed make[3]: *** [scripts_basic] Error 2 Makefile:151: recipe for target 'khdr' failed make[2]: *** [khdr] Error 2 /work/kernel/linux/Makefile:1220: recipe for target 'kselftest-all' failed make[1]: *** [kselftest-all] Error 2 make[1]: Leaving directory '/work/kernel/linux/build-arm64' Makefile:179: recipe for target 'sub-make' failed make: *** [sub-make] Error 2I am looking to address build and install issues first.
thanks,
-- Shuah
Signature · 1 line
~Randy
Shuah Khan
maintainerRe: [PATCH 1/4] selftests: Fix kselftest O=objdir build from cluttering top level objdir
Quoted 14 lines · Randy Dunlap
Quoted 13 lines · Shuah Khan
Quoted 11 lines · Kevin Hilman
Quoted 5 lines · Shuah Khan
make kselftest-all O=objdir builds create generated objects in objdir. This clutters the top level directory with kselftest objects. Fix it to create sub-directory under objdir for kselftest objects.
Signed-off-by Shuah Khan <[email protected]>Only somewhat related to this patch, another problem that wasn't in your doci is that the current O= support doesn't support relative paths.
Quoted 6 lines
Quoted 5 lines
Yes I am aware of it and it is in the document as something that will be addressed later.
"Note: Relative paths don’t work - supporting relative paths breaks work-flows e.g:
powerpc. Explore fix. Compile work-flows. Not planning to support at the moment."
I attempted to fix it once and had to revert the patch. This relative problem needs to be fixed and being tracked as a bug.
Quoted 3 lines
Quoted 3 lines
I am looking to address build and install issues first.
As I said above, there were higher priority test build and install failures in cross-build and native build use-cases when I made this call to prioritize fixing them first. I fixed all the known issues.
As of Linux 5.7-rc5 all of the know issues related test build and install failures have been fixed.
With tha done, it is time to work on the relative path fix. I have my reverted patch to start with and get it to work with the workflows so it doesn't break them.
I also have the following patch you gave me as reference for fixing the relative path problem:
https://lore.kernel.org/lkml/158351957799.3363.15269768530697526765.stgit@devnote2/
In any case, based on my previous experience fixing this problem, I plan to take the fix in rc1 to allow enough time to fix any problems that show up. It will be fixed for sure.
Hope this helps addresses any concerns you may have on whether or not this problem will be fixed.
thanks,
-- Shuah
Shuah Khan
maintainerPATCH 2/4 selftests: Fix seccomp to support relocatable build (O=objdir)
In reply to Shuah KhanFix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable TEST_GEN_PROGS for objects to leverage lib.mk common framework for relocatable builds.
--- tools/testing/selftests/seccomp/Makefile | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)
tools/testing/selftests/seccomp/Makefile +3 -13
@@ -1,17 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -all: - -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthread -seccomp_bpf: seccomp_bpf.c ../kselftest_harness.h - $(CC) $(CFLAGS) $(LDFLAGS) $< -lpthread -o $@ - -TEST_PROGS += $(BINARIES) -EXTRA_CLEAN := $(BINARIES) +TEST_GEN_PROGS := seccomp_bpf seccomp_benchmark -all: $(BINARIES) +include ../lib.mk
Signature · 1 line
2.20.1
Re: [PATCH 2/4] selftests: Fix seccomp to support relocatable build (O=objdir)
Quoted 16 lines · Shuah Khan
Fix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable
TEST_GEN_PROGSfor objects to leverage lib.mk common framework for relocatable builds.Signed-off-by Shuah Khan <[email protected]>--- tools/testing/selftests/seccomp/Makefile | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)tools/testing/selftests/seccomp/Makefile +0
-1@@ -1,17 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -all:
tools/testing/selftests/seccomp/Makefile line 11
Quoted 10 lines
tools/testing/selftests/seccomp/Makefile +1
-7- -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthread -seccomp_bpf: seccomp_bpf.c ../kselftest_harness.h
How is the ../kselftest_harness.h dependency detected in the resulting build rules?
Otherwise, looks good.
-Kees
Quoted 11 lines
tools/testing/selftests/seccomp/Makefile +2
-5- $(CC) $(CFLAGS) $(LDFLAGS) $< -lpthread -o $@ - -TEST_PROGS += $(BINARIES) -EXTRA_CLEAN := $(BINARIES) +TEST_GEN_PROGS := seccomp_bpf seccomp_benchmark -all: $(BINARIES) +include ../lib.mkSignature · 1 line
2.20.1
Signature · 1 line
Kees Cook
Shuah Khan
maintainerRe: [PATCH 2/4] selftests: Fix seccomp to support relocatable build (O=objdir)
Quoted 26 lines · Kees Cook
Quoted 25 lines · Shuah Khan
Fix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable
TEST_GEN_PROGSfor objects to leverage lib.mk common framework for relocatable builds.Signed-off-by Shuah Khan <[email protected]>--- tools/testing/selftests/seccomp/Makefile | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)tools/testing/selftests/seccomp/Makefile +1
-7@@ -1,17 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -all: - -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthread
Quoted 6 lines
tools/testing/selftests/seccomp/Makefile line 11
Quoted 1 line
tools/testing/selftests/seccomp/Makefile +0
-1-seccomp_bpf: seccomp_bpf.c ../kselftest_harness.hHow is the
../kselftest_harness.hdependency detected in the resulting build rules?Otherwise, looks good.
Didn't see any problems. I will look into adding the dependency.
thanks,
-- Shuah
Re: [PATCH 2/4] selftests: Fix seccomp to support relocatable build (O=objdir)
Quoted 27 lines · Shuah Khan
Quoted 26 lines · Kees Cook
Quoted 25 lines · Shuah Khan
Fix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable
TEST_GEN_PROGSfor objects to leverage lib.mk common framework for relocatable builds.Signed-off-by Shuah Khan <[email protected]>--- tools/testing/selftests/seccomp/Makefile | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)tools/testing/selftests/seccomp/Makefile +1
-7@@ -1,17 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -all: - -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthread
Quoted 8 lines
Quoted 6 lines
tools/testing/selftests/seccomp/Makefile line 11
Quoted 1 line
tools/testing/selftests/seccomp/Makefile +0
-1-seccomp_bpf: seccomp_bpf.c ../kselftest_harness.hHow is the
../kselftest_harness.hdependency detected in the resulting build rules?Otherwise, looks good.
Didn't see any problems. I will look into adding the dependency.
Before:
$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp
make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \
ARCH=powerpc -C ../../.. headers_install
INSTALL /home/michael/build/adhoc/kselftest/usr/include
gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
gcc -Wl,-no-as-needed -Wall seccomp_benchmark.c -o seccomp_benchmark$ touch tools/testing/selftests/kselftest_harness.h
$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp
make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \
ARCH=powerpc -C ../../.. headers_install
INSTALL /home/michael/build/adhoc/kselftest/usr/include
gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
$Note that touching the header causes it to rebuild seccomp_bpf.
With this patch applied:
$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp
make -s --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \
ARCH=powerpc -C ../../.. headers_install
gcc -Wl,-no-as-needed -Wall -lpthread seccomp_bpf.c -o /home/michael/build/adhoc/kselftest/seccomp/seccomp_bpf
gcc -Wl,-no-as-needed -Wall -lpthread seccomp_benchmark.c -o /home/michael/build/adhoc/kselftest/seccomp/seccomp_benchmark$ touch tools/testing/selftests/kselftest_harness.h
$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp
make -s --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \
ARCH=powerpc -C ../../.. headers_install
make[1]: Nothing to be done for 'all'.
$So yeah it still needs:
seccomp_bpf: ../kselftest_harness.h
cheers
Shuah Khan
maintainerRe: [PATCH 2/4] selftests: Fix seccomp to support relocatable build (O=objdir)
Quoted 65 lines · Michael Ellerman
Quoted 27 lines · Shuah Khan
Quoted 26 lines · Kees Cook
Quoted 25 lines · Shuah Khan
Fix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable
TEST_GEN_PROGSfor objects to leverage lib.mk common framework for relocatable builds.Signed-off-by Shuah Khan <[email protected]>--- tools/testing/selftests/seccomp/Makefile | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)tools/testing/selftests/seccomp/Makefile +1
-7@@ -1,17 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -all: - -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthreadQuoted 8 lines
Quoted 6 lines
tools/testing/selftests/seccomp/Makefile line 11
Quoted 1 line
tools/testing/selftests/seccomp/Makefile +0
-1-seccomp_bpf: seccomp_bpf.c ../kselftest_harness.hHow is the
../kselftest_harness.hdependency detected in the resulting build rules?Otherwise, looks good.
Didn't see any problems. I will look into adding the dependency.
Before:
$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=powerpc -C ../../.. headers_install INSTALL /home/michael/build/adhoc/kselftest/usr/include gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf gcc -Wl,-no-as-needed -Wall seccomp_benchmark.c -o seccomp_benchmark$ touch tools/testing/selftests/kselftest_harness.h$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=powerpc -C ../../.. headers_install INSTALL /home/michael/build/adhoc/kselftest/usr/include gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf $Note that touching the header causes it to rebuild
seccomp_bpf.With this patch applied:
$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp make -s --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=powerpc -C ../../.. headers_install gcc -Wl,-no-as-needed -Wall -lpthread seccomp_bpf.c -o /home/michael/build/adhoc/kselftest/seccomp/seccomp_bpf gcc -Wl,-no-as-needed -Wall -lpthread seccomp_benchmark.c -o /home/michael/build/adhoc/kselftest/seccomp/seccomp_benchmark
Quoted 9 lines
$ touch tools/testing/selftests/kselftest_harness.h$ make --no-print-directory -C tools/testing/selftests/ TARGETS=seccomp make -s --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=powerpc -C ../../.. headers_install make[1]: Nothing to be done for 'all'. $
Thanks. I realized I overlooked header dependency case.
Quoted 5 lines
So yeah it still needs:
seccomp_bpf:../kselftest_harness.h
Yes v2 coming up. It also has to handle OUTPUT relocation.
thanks,
-- Shuah
Shuah Khan
maintainerPATCH 3/4 selftests: android: ion: Fix ionmap_test compile error
In reply to Shuah Khanionmap_test compile rule is missing ipcsocket.c dependency. Add it to fix the following compile errors:
..android/ion/ionutils.c:221: undefined reference to sendtosocket' ..android/ion/ionutils.c:243: undefined reference to receivefromsocket'
--- tools/testing/selftests/android/ion/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/android/ion/Makefile +1 -1
@@ -17,4 +17,4 @@ include ../../lib.mk $(OUTPUT)/ionapp_export: ionapp_export.c ipcsocket.c ionutils.c $(OUTPUT)/ionapp_import: ionapp_import.c ipcsocket.c ionutils.c -$(OUTPUT)/ionmap_test: ionmap_test.c ionutils.c +$(OUTPUT)/ionmap_test: ionmap_test.c ionutils.c ipcsocket.c
Signature · 1 line
2.20.1
Shuah Khan
maintainerPATCH 4/4 selftests: android: Fix custom install from skipping test progs
In reply to Shuah KhanUpdate custom install rule to install all generated test programs. This fixes android/ion tests to be installed correctly.
--- tools/testing/selftests/android/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/android/Makefile +1 -1
@@ -21,7 +21,7 @@ all: override define INSTALL_RULE mkdir -p $(INSTALL_PATH) - install -t $(INSTALL_PATH) $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) +install -t $(INSTALL_PATH) $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES) $(TEST_GEN_PROGS) $(TEST_CUSTOM_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) @for SUBDIR in $(SUBDIRS); do \ BUILD_TARGET=$(OUTPUT)/$$SUBDIR; \
Signature · 1 line
2.20.1
Shuah Khan
maintainerQuoted 7 lines · Shuah Khan
This patch series consists of first round of fixes to integrate Kselftest into Kernel CI.
You can find full list of problems in my announcement I sent out last week:
Here is the lore link:
https://lore.kernel.org/lkml/[email protected]/
thanks,
-- Shuah