selftests: bpf: fix compilation with out-of-tree $(OUTPUT)
Subsystems (6)
PATCH 1/2 selftests: bpf: fix compilation with out-of-tree $(OUTPUT)
A bunch of related changes lumped together:
* Create prog_tests and verifier output directories; these don't exist with out-of-tree $(OUTPUT)
* Add missing -I (via separate TEST_{PROGS,VERIFIER}_CFLAGS) for the main tree ($(PWD) != $(OUTPUT) for out-of-tree)
* Add libbpf.a dependency for test_progs_32 (parallel make fails otherwise)
* Add missing "; \" after "cd" when generating test.h headers
Tested by:
$ alias m="make -s -j$(nproc)" $ m -C tools/testing/selftests/bpf/ clean $ m -C tools/lib/bpf/ clean $ rm -rf xxx; mkdir xxx; m -C tools/testing/selftests/bpf/ OUTPUT=$PWD/xxx $ m -C tools/testing/selftests/bpf/
--- tools/testing/selftests/bpf/Makefile | 33 +++++++++++++++++++--------- 1 file changed, 23 insertions(+), 10 deletions(-)
tools/testing/selftests/bpf/Makefile +23 -10
@@ -153,6 +153,9 @@ endif endif endif +TEST_PROGS_CFLAGS := -I. -I$(OUTPUT) +TEST_VERIFIER_CFLAGS := -I. -I$(OUTPUT) -Iverifier + ifneq ($(SUBREG_CODEGEN),) ALU32_BUILD_DIR = $(OUTPUT)/alu32 TEST_CUSTOM_PROGS += $(ALU32_BUILD_DIR)/test_progs_32
@@ -162,13 +165,15 @@ TEST_CUSTOM_PROGS += $(ALU32_BUILD_DIR)/test_progs_32 $(ALU32_BUILD_DIR)/urandom_read: $(OUTPUT)/urandom_read cp $< $@ -$(ALU32_BUILD_DIR)/test_progs_32: test_progs.c $(ALU32_BUILD_DIR) \ +$(ALU32_BUILD_DIR)/test_progs_32: test_progs.c $(OUTPUT)/libbpf.a\ + $(ALU32_BUILD_DIR) \ $(ALU32_BUILD_DIR)/urandom_read - $(CC) $(CFLAGS) -o $(ALU32_BUILD_DIR)/test_progs_32 $< \ - trace_helpers.c prog_tests/*.c $(OUTPUT)/libbpf.a $(LDLIBS) + $(CC) $(TEST_PROGS_CFLAGS) $(CFLAGS) \ + -o $(ALU32_BUILD_DIR)/test_progs_32 \ + test_progs.c trace_helpers.c prog_tests/*.c \ + $(OUTPUT)/libbpf.a $(LDLIBS) $(ALU32_BUILD_DIR)/test_progs_32: $(PROG_TESTS_H) -$(ALU32_BUILD_DIR)/test_progs_32: CFLAGS += -I$(OUTPUT) $(ALU32_BUILD_DIR)/test_progs_32: prog_tests/*.c $(ALU32_BUILD_DIR)/%.o: progs/%.c $(ALU32_BUILD_DIR) \
@@ -202,12 +207,16 @@ endif PROG_TESTS_H := $(OUTPUT)/prog_tests/tests.h $(OUTPUT)/test_progs: $(PROG_TESTS_H) -$(OUTPUT)/test_progs: CFLAGS += -I$(OUTPUT) +$(OUTPUT)/test_progs: CFLAGS += $(TEST_PROGS_CFLAGS) $(OUTPUT)/test_progs: prog_tests/*.c +PROG_TESTS_DIR = $(OUTPUT)/prog_tests +$(PROG_TESTS_DIR): + mkdir -p $@ + PROG_TESTS_FILES := $(wildcard prog_tests/*.c) -$(PROG_TESTS_H): $(PROG_TESTS_FILES) - $(shell ( cd prog_tests/ +$(PROG_TESTS_H): $(PROG_TESTS_DIR) $(PROG_TESTS_FILES) + $(shell ( cd prog_tests/; \ echo '/* Generated header, do not edit */'; \ echo '#ifdef DECLARE'; \ ls *.c 2> /dev/null | \
@@ -221,11 +230,15 @@ $(PROG_TESTS_H): $(PROG_TESTS_FILES) VERIFIER_TESTS_H := $(OUTPUT)/verifier/tests.h $(OUTPUT)/test_verifier: $(VERIFIER_TESTS_H) -$(OUTPUT)/test_verifier: CFLAGS += -I$(OUTPUT) +$(OUTPUT)/test_verifier: CFLAGS += $(TEST_VERIFIER_CFLAGS) + +VERIFIER_TESTS_DIR = $(OUTPUT)/verifier +$(VERIFIER_TESTS_DIR): + mkdir -p $@ VERIFIER_TEST_FILES := $(wildcard verifier/*.c) -$(OUTPUT)/verifier/tests.h: $(VERIFIER_TEST_FILES) - $(shell ( cd verifier/ +$(OUTPUT)/verifier/tests.h: $(VERIFIER_TESTS_DIR) $(VERIFIER_TEST_FILES) + $(shell ( cd verifier/; \ echo '/* Generated header, do not edit */'; \ echo '#ifdef FILL_ARRAY'; \ ls *.c 2> /dev/null | \
Signature · 1 line
2.21.0.352.gf09ad66450-goog
PATCH 2/2 libbpf: force fixdep compilation at the start of the build
libbpf targets don't explicitly depend on fixdep target, so when we do 'make -j$(nproc)', there is a high probability, that some objects will be built before fixdep binary is available.
Fix this by running sub-make; this makes sure that fixdep dependency is properly accounted for.
For the same issue in perf, see commit abb26210a395 ("perf tools: Force fixdep compilation at the start of the build").
Before:
$ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/
Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]
HOSTCC /tmp/bld/fixdep.o CC /tmp/bld/libbpf.o CC /tmp/bld/bpf.o CC /tmp/bld/btf.o CC /tmp/bld/nlattr.o CC /tmp/bld/libbpf_errno.o CC /tmp/bld/str_error.o CC /tmp/bld/netlink.o CC /tmp/bld/bpf_prog_linfo.o CC /tmp/bld/libbpf_probes.o CC /tmp/bld/xsk.o HOSTLD /tmp/bld/fixdep-in.o LINK /tmp/bld/fixdep LD /tmp/bld/libbpf-in.o LINK /tmp/bld/libbpf.a LINK /tmp/bld/libbpf.so LINK /tmp/bld/test_libbpf
$ head /tmp/bld/.libbpf.o.cmd # cannot find fixdep (/usr/local/google/home/sdf/src/linux/xxx//fixdep) # using basic dep data
/tmp/bld/libbpf.o: libbpf.c /usr/include/stdc-predef.h \
/usr/include/stdlib.h /usr/include/features.h \
/usr/include/x86_64-linux-gnu/sys/cdefs.h \
/usr/include/x86_64-linux-gnu/bits/wordsize.h \
/usr/include/x86_64-linux-gnu/gnu/stubs.h \
/usr/include/x86_64-linux-gnu/gnu/stubs-64.h \
/usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h \
After:
$ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/
Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]
HOSTCC /tmp/bld/fixdep.o HOSTLD /tmp/bld/fixdep-in.o LINK /tmp/bld/fixdep CC /tmp/bld/libbpf.o CC /tmp/bld/bpf.o CC /tmp/bld/nlattr.o CC /tmp/bld/btf.o CC /tmp/bld/libbpf_errno.o CC /tmp/bld/str_error.o CC /tmp/bld/netlink.o CC /tmp/bld/bpf_prog_linfo.o CC /tmp/bld/libbpf_probes.o CC /tmp/bld/xsk.o LD /tmp/bld/libbpf-in.o LINK /tmp/bld/libbpf.a LINK /tmp/bld/libbpf.so LINK /tmp/bld/test_libbpf
$ head /tmp/bld/.libbpf.o.cmd cmd_/tmp/bld/libbpf.o := gcc -Wp,-MD,/tmp/bld/.libbpf.o.d -Wp,-MT,/tmp/bld/libbpf.o -g -Wall -DHAVE_LIBELF_MMAP_SUPPORT -DCOMPAT_NEED_REALLOCARRAY -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -Wstrict-aliasing=3 -Werror -Wall -fPIC -I. -I/usr/local/google/home/sdf/src/linux/tools/include -I/usr/local/google/home/sdf/src/linux/tools/arch/x86/include/uapi -I/usr/local/google/home/sdf/src/linux/tools/include/uapi -fvisibility=hidden -D"BUILD_STR(s)=$(pound)s" -c -o /tmp/bld/libbpf.o libbpf.c
source_/tmp/bld/libbpf.o := libbpf.c
deps_/tmp/bld/libbpf.o := \
/usr/include/stdc-predef.h \
/usr/include/stdlib.h \
/usr/include/features.h \
/usr/include/x86_64-linux-gnu/sys/cdefs.h \
/usr/include/x86_64-linux-gnu/bits/wordsize.h \
--- tools/lib/bpf/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/lib/bpf/Makefile +2 -1
@@ -147,7 +147,8 @@ endif TARGETS = $(CMD_TARGETS) -all: fixdep all_cmd +all: fixdep + $(Q)$(MAKE) all_cmd all_cmd: $(CMD_TARGETS) check
Signature · 1 line
2.21.0.352.gf09ad66450-goog
Quoted 16 lines · Stanislav Fomichev
A bunch of related changes lumped together:
* Createprog_testsand verifier output directories; these don't exist with out-of-tree $(OUTPUT)
* Add missing -I (via separate TEST_{PROGS,VERIFIER}_CFLAGS) for the main tree ($(PWD) != $(OUTPUT) for out-of-tree)
* Add libbpf.a dependency fortest_progs_32(parallel make fails otherwise)
* Add missing "; \" after "cd" when generatingtest.hheadersTested by:
$ alias m="make -s -j$(nproc)" $ m -C tools/testing/selftests/bpf/ clean $ m -C tools/lib/bpf/ clean $ rm -rf xxx; mkdir xxx; m -C tools/testing/selftests/bpf/ OUTPUT=$PWD/xxx $ m -C tools/testing/selftests/bpf/
Quoted 5 lines
Fixes 3f30658830f3 ("selftests: bpf: break up test_progs - preparations")Fixes 2dfb40121ee8 ("selftests: bpf: prepare for break up of verifier tests")Fixes 3ef84346c561 ("selftests: bpf: makefile support sub-register code-gen test mode")Signed-off-by Stanislav Fomichev <[email protected]>
Re: [PATCH bpf 2/2] libbpf: force fixdep compilation at the start of the build
In reply to Stanislav FomichevQuoted 87 lines · Stanislav Fomichev
libbpf targets don't explicitly depend on fixdep target, so when we do 'make -j$(nproc)', there is a high probability, that some objects will be built before fixdep binary is available.
Fix this by running sub-make; this makes sure that fixdep dependency is properly accounted for.
For the same issue in perf, see commit
abb26210a395("perf tools: Force fixdep compilation at the start of the build").Before:
$ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]HOSTCC /tmp/bld/fixdep.o CC /tmp/bld/libbpf.o CC /tmp/bld/bpf.o CC /tmp/bld/btf.o CC /tmp/bld/nlattr.o CC /tmp/bld/libbpf_errno.o CC /tmp/bld/str_error.o CC /tmp/bld/netlink.o CC /tmp/bld/bpf_prog_linfo.o CC /tmp/bld/libbpf_probes.o CC /tmp/bld/xsk.o HOSTLD /tmp/bld/fixdep-in.o LINK /tmp/bld/fixdep LD /tmp/bld/libbpf-in.o LINK /tmp/bld/libbpf.a LINK /tmp/bld/libbpf.so LINK /tmp/bld/test_libbpf$ head /tmp/bld/.libbpf.o.cmd # cannot find fixdep (/usr/local/google/home/sdf/src/linux/xxx//fixdep) # using basic dep data
/tmp/bld/libbpf.o:libbpf.c/usr/include/stdc-predef.h\
/usr/include/stdlib.h/usr/include/features.h\
/usr/include/x86_64-linux-gnu/sys/cdefs.h\
/usr/include/x86_64-linux-gnu/bits/wordsize.h\
/usr/include/x86_64-linux-gnu/gnu/stubs.h\
/usr/include/x86_64-linux-gnu/gnu/stubs-64.h\
/usr/lib/gcc/x86_64-linux-gnu/7/include/stddef.h\After:
$ rm -rf /tmp/bld; mkdir /tmp/bld; make -j$(nproc) O=/tmp/bld -C tools/lib/bpf/Auto-detecting system features:
... libelf: [ on ]
... bpf: [ on ]HOSTCC /tmp/bld/fixdep.o HOSTLD /tmp/bld/fixdep-in.o LINK /tmp/bld/fixdep CC /tmp/bld/libbpf.o CC /tmp/bld/bpf.o CC /tmp/bld/nlattr.o CC /tmp/bld/btf.o CC /tmp/bld/libbpf_errno.o CC /tmp/bld/str_error.o CC /tmp/bld/netlink.o CC /tmp/bld/bpf_prog_linfo.o CC /tmp/bld/libbpf_probes.o CC /tmp/bld/xsk.o LD /tmp/bld/libbpf-in.o LINK /tmp/bld/libbpf.a LINK /tmp/bld/libbpf.so LINK /tmp/bld/test_libbpf$ head /tmp/bld/.libbpf.o.cmd cmd_/tmp/bld/libbpf.o := gcc -Wp,-MD,/tmp/bld/.libbpf.o.d -Wp,-MT,/tmp/bld/libbpf.o -g -Wall -DHAVE_LIBELF_MMAP_SUPPORT -DCOMPAT_NEED_REALLOCARRAY -Wbad-function-cast -Wdeclaration-after-statement -Wformat-security -Wformat-y2k -Winit-self -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wno-system-headers -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-default -Wswitch-enum -Wundef -Wwrite-strings -Wformat -Wstrict-aliasing=3 -Werror -Wall -fPIC -I. -I/usr/local/google/home/sdf/src/linux/tools/include -I/usr/local/google/home/sdf/src/linux/tools/arch/x86/include/uapi -I/usr/local/google/home/sdf/src/linux/tools/include/uapi -fvisibility=hidden -D"BUILD_STR(s)=$(pound)s" -c -o /tmp/bld/libbpf.o libbpf.c
source_/tmp/bld/libbpf.o:=libbpf.c
deps_/tmp/bld/libbpf.o:= \
/usr/include/stdc-predef.h\
/usr/include/stdlib.h\
/usr/include/features.h\
/usr/include/x86_64-linux-gnu/sys/cdefs.h\
/usr/include/x86_64-linux-gnu/bits/wordsize.h\
Quoted 4 lines
Reported-by Eric Dumazet <[email protected]>Fixes 7c422f557266 ("tools build: Build fixdep helper from perf and basic libs")Signed-off-by Stanislav Fomichev <[email protected]>
Quoted 15 lines · Stanislav Fomichev
A bunch of related changes lumped together:
* Createprog_testsand verifier output directories; these don't exist with out-of-tree $(OUTPUT)
* Add missing -I (via separate TEST_{PROGS,VERIFIER}_CFLAGS) for the main tree ($(PWD) != $(OUTPUT) for out-of-tree)
* Add libbpf.a dependency fortest_progs_32(parallel make fails otherwise)
* Add missing "; \" after "cd" when generatingtest.hheadersTested by:
$ alias m="make -s -j$(nproc)" $ m -C tools/testing/selftests/bpf/ clean $ m -C tools/lib/bpf/ clean $ rm -rf xxx; mkdir xxx; m -C tools/testing/selftests/bpf/ OUTPUT=$PWD/xxx $ m -C tools/testing/selftests/bpf/
Quoted 4 lines
Fixes 3f30658830f3 ("selftests: bpf: break up test_progs - preparations")Fixes 2dfb40121ee8 ("selftests: bpf: prepare for break up of verifier tests")Fixes 3ef84346c561 ("selftests: bpf: makefile support sub-register code-gen test mode")
Btw, please no newline between these tags, I've fixed it up.
Quoted 1 line
Signed-off-by Stanislav Fomichev <[email protected]>
Applied, thanks!