Fix, clean up, and revamp selftests/bpf Makefile
Subsystems (5)
v3 cover Fix, clean up, and revamp selftests/bpf Makefile
This patch set extensively revamps selftests/bpf's Makefile to generalize test runner concept and apply it uniformly to test_maps and test_progs test runners, along with test_progs' few build "flavors", exercising various ways to build BPF programs.
As we do that, we fix dependencies between various phases of test runners, and simplify some one-off rules and dependencies currently present in Makefile. test_progs' flavors are now built into root $(OUTPUT) directory and can be run without any extra steps right from there. E.g., test_progs-alu32 is built and is supposed to be run from $(OUTPUT). It will cd into alu32/ subdirectory to load correct set of BPF object files (which are different from the ones built for test_progs).
Outline:
- patch #1 teaches test_progs about flavor sub-directories;
- patch #2 fixes one of CO-RE tests to not depend strictly on process name;
- patch #3 changes test_maps's usage of map_tests/tests.h to be the same as test_progs' one;
- patch #4 adds convenient short make test_progs-like targets to build only individual tests, if necessary;
- patch #5 is a main patch in the series; it uses a bunch of make magic (mainly $(call) and $(eval)) to define test runner "skeleton" and apply it to 4 different test runners, lots more details in corresponding commit description;
- patch #6 does a bit of post-clean up for test_queue_map and test_stack_map BPF programs;
- patch #7 cleans up test_libbpf.sh/test_libbpf_open superseded by test_progs.
v2->v3:
- drop test_xdp.o mixed compilation mode, remove test_libbpf.sh (Alexei);
v1->v2:
- drop test_progs-native causing compilation failures due to __builtin_preserve_field_access, add back test_xdp.o override, which will now emit rule re-definition warning.
Andrii Nakryiko (7):
selftests/bpf: teach test_progs to cd into subdir selftests/bpf: make CO-RE reloc test impartial to test_progs flavor selftests/bpf: switch test_maps to test_progs' test.h format selftests/bpf: add simple per-test targets to Makefile selftests/bpf: replace test_progs and test_maps w/ general rule selftests/bpf: move test_queue_stack_map.h into progs/ where it
belongs
selftest/bpf: remove test_libbpf.sh and test_libbpf_open
tools/testing/selftests/bpf/.gitignore | 7 +-
tools/testing/selftests/bpf/Makefile | 338 ++++++++++--------
.../selftests/bpf/prog_tests/core_reloc.c | 4 +-
.../selftests/bpf/progs/core_reloc_types.h | 2 +-
.../bpf/progs/test_core_reloc_kernel.c | 3 +-
.../bpf/{ => progs}/test_queue_stack_map.h | 0
tools/testing/selftests/bpf/test_libbpf | Bin 0 -> 384568 bytes
tools/testing/selftests/bpf/test_libbpf.sh | 43 ---
tools/testing/selftests/bpf/test_libbpf_open | Bin 0 -> 396096 bytes
tools/testing/selftests/bpf/test_maps.c | 8 +-
tools/testing/selftests/bpf/test_progs.c | 33 +-
11 files changed, 234 insertions(+), 204 deletions(-)
rename tools/testing/selftests/bpf/{ => progs}/test_queue_stack_map.h (100%)
create mode 100755 tools/testing/selftests/bpf/test_libbpf
delete mode 100755 tools/testing/selftests/bpf/test_libbpf.sh
create mode 100755 tools/testing/selftests/bpf/test_libbpf_openSignature · 1 line
2.17.1
PATCH v3 1/7 selftests/bpf: teach test_progs to cd into subdir
We are building a bunch of "flavors" of test_progs, e.g., w/ alu32 flag for Clang when building BPF object. test_progs setup is relying on having all the BPF object files and extra resources to be available in current working directory, though. But we actually build all these files into a separate sub-directory. Next set of patches establishes convention of naming "flavored" test_progs (and test runner binaries in general) as test_progs-flavor (e.g., test_progs-alu32), for each such extra flavor. This patch teaches test_progs binary to automatically detect its own extra flavor based on its argv[0], and if present, to change current directory to a flavor-specific subdirectory.
--- tools/testing/selftests/bpf/test_progs.c | 33 +++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/test_progs.c +32 -1
@@ -306,7 +306,7 @@ void *spin_lock_thread(void *arg) }/* extern declarations for test funcs */ -#define DEFINE_TEST (name) extern void test_##name(); +#define DEFINE_TEST (name) extern void test_##name(void);#include <prog_tests/tests.h> #undef DEFINE_TEST
@@ -518,6 +518,33 @@ static void stdio_restore(void)#endif } +/* +* Determine if test_progs is running as a "flavored" test runner and switch +* into corresponding sub-directory to load correct BPF objects. +* +* This is done by looking at executable name. If it contains "-flavor" +* suffix, then we are running as a flavored test runner. +*/ +int cd_flavor_subdir (const char *exec_name) +{ +/* General form of argv[0] passed here is: +* some/path/to/test_progs[-flavor], where -flavor part is optional. +* First cut out "test_progs[-flavor]" part, then extract "flavor" +* part, if it's there. +*/ +const char *flavor =strrchr (exec_name,'/' ); + +if (!flavor) +return 0 ; + flavor++; + flavor =strrchr (flavor,'-' ); +if (!flavor) +return 0 ; + flavor++; +printf ("Switching to flavor '%s' subdirectory...\n" , flavor); +return chdir (flavor); +} +int main (int argc,char **argv) {static const struct argp argp = {
@@ -531,6 +558,10 @@ int main(int argc, char **argv)if (err)return err; + err =cd_flavor_subdir (argv[0 ]); +if (err) +return err; +libbpf_set_print (libbpf_print_fn);srand (time (NULL ));
Signature · 1 line
2.17.1
PATCH v3 2/7 selftests/bpf: make CO-RE reloc test impartial to test_progs flavor
In reply to Andrii Nakryikotest_core_reloc_kernel test captures its own process name and validates it as part of the test. Given extra "flavors" of test_progs, this break for anything by default test_progs binary. Fix the test to cut out flavor part of the process name.
--- tools/testing/selftests/bpf/prog_tests/core_reloc.c | 4 ++-- tools/testing/selftests/bpf/progs/core_reloc_types.h | 2 +- tools/testing/selftests/bpf/progs/test_core_reloc_kernel.c | 3 ++- 3 files changed, 5 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/prog_tests/core_reloc.c +2 -2
@@ -211,8 +211,8 @@ static struct core_reloc_test_case test_cases[] = { .input_len = 0, .output = STRUCT_TO_CHAR_PTR(core_reloc_kernel_output) { .valid = { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, }, - .comm ="test_progs\0\0\0\0\0" , - .comm_len = 11, + .comm ="test_progs" , + .comm_len = sizeof("test_progs"), }, .output_len = sizeof(struct core_reloc_kernel_output), },
tools/testing/selftests/bpf/progs/core_reloc_types.h +1 -1
@@ -6,7 +6,7 @@struct core_reloc_kernel_output {int valid[10 ]; -char comm[16 ]; +char comm[sizeof ("test_progs" )];int comm_len; };
tools/testing/selftests/bpf/progs/test_core_reloc_kernel.c +2 -1
@@ -15,7 +15,8 @@ static volatile struct data {struct core_reloc_kernel_output {int valid[10 ]; -char comm[16 ]; +/* we have test_progs[-flavor], so cut flavor part */ +char comm[sizeof ("test_progs" )];int comm_len; };
Signature · 1 line
2.17.1
PATCH v3 3/7 selftests/bpf: switch test_maps to test_progs' test.h format
In reply to Andrii NakryikoMake test_maps use tests.h header format consistent with the one used by test_progs, to facilitate unification.
--- tools/testing/selftests/bpf/Makefile | 8 +------- tools/testing/selftests/bpf/test_maps.c | 8 ++++---- 2 files changed, 5 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/Makefile +1 -7
@@ -249,14 +249,8 @@ $(OUTPUT)/test_maps: test_maps.c $(MAP_TESTS_FILES) | $(MAP_TESTS_H) $(MAP_TESTS_H): $(MAP_TESTS_FILES) | $(MAP_TESTS_DIR) $(shell ( cd map_tests/; \ echo '/* Generated header, do not edit */'; \ - echo '#ifdef DECLARE'; \ ls *.c 2> /dev/null | \ - sed -e 's@\([^\.]*\)\.c@extern void test_\1(void);@'; \ - echo '#endif'; \ - echo '#ifdef CALL'; \ - ls *.c 2> /dev/null | \ - sed -e 's@\([^\.]*\)\.c@test_\1();@'; \ - echo '#endif' \ + sed -e 's@\([^\.]*\)\.c@DEFINE_TEST(\1)@'; \ ) > $(MAP_TESTS_H)) VERIFIER_TESTS_DIR = $(OUTPUT)/verifier
tools/testing/selftests/bpf/test_maps.c +4 -4
@@ -1717,9 +1717,9 @@ static void run_all_tests(void)test_map_in_map (); } -#define DECLARE +#define DEFINE_TEST (name) extern void test_##name(void);#include <map_tests/tests.h> -#undef DECLARE +#undef DEFINE_TESTint main (void ) {
@@ -1731,9 +1731,9 @@ int main(void) map_flags =BPF_F_NO_PREALLOC ;run_all_tests (); -#define CALL +#define DEFINE_TEST (name) test_##name();#include <map_tests/tests.h> -#undef CALL +#undef DEFINE_TESTprintf ("test_maps: OK, %d SKIPPED\n" , skips);return 0 ;
Signature · 1 line
2.17.1
PATCH v3 4/7 selftests/bpf: add simple per-test targets to Makefile
In reply to Andrii NakryikoCurrently it's impossible to do make test_progs and have only test_progs be built, because all the binary targets are defined in terms of $(OUTPUT)/<binary>, and $(OUTPUT) is absolute path to current directory (or whatever gets overridden to by user).
This patch adds simple re-directing targets for all test targets making it possible to do simple and nice make test_progs (and any other target).
--- tools/testing/selftests/bpf/Makefile | 10 ++++++++++ 1 file changed, 10 insertions(+)
tools/testing/selftests/bpf/Makefile +10 -0
@@ -84,6 +84,16 @@ TEST_GEN_PROGS_EXTENDED = test_libbpf_open test_sock_addr test_skb_cgroup_id_use include ../lib.mk +# Define simple and short `make test_progs`, `make test_sysctl`, etc targets +# to build individual tests. +# NOTE: Semicolon at the end is critical to override lib.mk's default static +# rule for binaries. +$(notdir $(TEST_GEN_PROGS) \ + $(TEST_PROGS) \ + $(TEST_PROGS_EXTENDED) \ + $(TEST_GEN_PROGS_EXTENDED) \ + $(TEST_CUSTOM_PROGS)): %: $(OUTPUT)/% ; + # NOTE: $(OUTPUT) won't get default value if used before lib.mk TEST_CUSTOM_PROGS = $(OUTPUT)/urandom_read all: $(TEST_CUSTOM_PROGS)
Signature · 1 line
2.17.1
PATCH v3 5/7 selftests/bpf: replace test_progs and test_maps w/ general rule
In reply to Andrii NakryikoDefine test runner generation meta-rule that codifies dependencies between test runner, its tests, and its dependent BPF programs. Use that for defining test_progs and test_maps test-runners. Also additionally define 2 flavors of test_progs:
- alu32, which builds BPF programs with 32-bit registers codegen;
- bpf_gcc, which build BPF programs using GCC, if it supports BPF target.
Overall, this is accomplished through $(eval)'ing a set of generic rules, which defines Makefile targets dynamically at runtime. See comments explaining the need for 2 $(evals), though.
For each test runner we have (test_maps and test_progs, currently), and, optionally, their flavors, the logic of build process is modeled as follows (using test_progs as an example):
- all BPF objects are in progs/:
- BPF object's .o file is built into output directory from
corresponding progs/.c file;
- all BPF objects in progs/*.c depend on all progs/*.h headers;
- all BPF objects depend on bpf_*.h helpers from libbpf (but not libbpf archive). There is an extra rule to trigger bpf_helper_defs.h (re-)build, if it's not present/outdated);
- build recipe for BPF object can be re-defined per test runner/flavor;
- test files are built from prog_tests/*.c:
- all such test file objects are built on individual file basis;
- currently, every single test file depends on all BPF object files; this might be improved in follow up patches to do 1-to-1 dependency, but allowing to customize this per each individual test;
- each test runner definition can specify a list of extra .c and .h files to be built along test files and test runner binary; all such headers are becoming automatic dependency of each test .c file;
- due to test files sometimes embedding (using .incbin assembly directive) contents of some BPF objects at compilation time, which are expected to be in CWD of compiler, compilation for test file object does cd into test runner's output directory; to support this mode all the include paths are turned into absolute paths using $(abspath) make function;
- prog_tests/test.h is automatically (re-)generated with an entry for each .c file in prog_tests/;
- final test runner binary is linked together from test object files and extra object files, linking together libbpf's archive as well;
- it's possible to specify extra "resource" files/targets, which will be copied into test runner output directory, if it differes from Makefile-wide $(OUTPUT). This is used to ensure btf_dump test cases and urandom_read binary is put into a test runner's CWD for tests to find them in runtime.
For flavored test runners, their output directory is a subdirectory of common Makefile-wide $(OUTPUT) directory with flavor name used as subdirectory name.
BPF objects targets might be reused between different test runners, so extra checks are employed to not double-define them. Similarly, we have redefinition guards for output directories and test headers.
test_verifier follows slightly different patterns and is simple enough to not justify generalizing TEST_RUNNER_DEFINE/TEST_RUNNER_DEFINE_RULES further to accomodate these differences. Instead, rules for test_verifier are minimized and simplified, while preserving correctness of dependencies.
--- tools/testing/selftests/bpf/.gitignore | 5 +- tools/testing/selftests/bpf/Makefile | 316 ++++++++++++++----------- 2 files changed, 182 insertions(+), 139 deletions(-)
tools/testing/selftests/bpf/.gitignore +3 -2
@@ -7,7 +7,7 @@ FEATURE-DUMP.libbpf fixdep test_align test_dev_cgroup -test_progs +/test_progs* test_tcpbpf_user test_verifier_log feature
@@ -33,9 +33,10 @@ test_tcpnotify_user test_libbpf test_tcp_check_syncookie_user test_sysctl -alu32 libbpf.pc libbpf.so.* test_hashmap test_btf_dump xdping +/alu32 +/bpf_gcc
tools/testing/selftests/bpf/Makefile +179 -137
399 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 6/7 selftests/bpf: move test_queue_stack_map.h into progs/ where it belongs
In reply to Andrii Nakryikotest_queue_stack_map.h is used only from BPF programs. Thus it should be part of progs/ subdir. An added benefit of moving it there is that new TEST_RUNNER_DEFINE_RULES macro-rule will properly capture dependency on this header for all BPF objects and trigger re-build, if it changes.
---
tools/testing/selftests/bpf/Makefile | 3 ---
tools/testing/selftests/bpf/{ => progs}/test_queue_stack_map.h | 0
2 files changed, 3 deletions(-)
rename tools/testing/selftests/bpf/{ => progs}/test_queue_stack_map.h (100%)tools/testing/selftests/bpf/Makefile +0 -3
@@ -153,9 +153,6 @@ CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \ $(OUTPUT)/test_l4lb_noinline.o: BPF_CFLAGS += -fno-inline $(OUTPUT)/test_xdp_noinline.o: BPF_CFLAGS += -fno-inline -$(OUTPUT)/test_queue_map.o: test_queue_stack_map.h -$(OUTPUT)/test_stack_map.o: test_queue_stack_map.h - $(OUTPUT)/flow_dissector_load.o: flow_dissector_load.h # Build BPF object using Clang
tools/testing/selftests/bpf/progs/test_queue_stack_map.h renamed from tools/testing/selftests/bpf/test_queue_stack_map.h +0 -0
Signature · 1 line
2.17.1
PATCH v3 7/7 selftest/bpf: remove test_libbpf.sh and test_libbpf_open
In reply to Andrii Nakryikotest_progs is much more sophisticated superset of tests compared to test_libbpf.sh and test_libbpf_open. Remove test_libbpf.sh and test_libbpf_open.
--- tools/testing/selftests/bpf/.gitignore | 2 - tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/test_libbpf | Bin 0 -> 384568 bytes tools/testing/selftests/bpf/test_libbpf.sh | 43 ------------------- tools/testing/selftests/bpf/test_libbpf_open | Bin 0 -> 396096 bytes 5 files changed, 1 insertion(+), 47 deletions(-) create mode 100755 tools/testing/selftests/bpf/test_libbpf delete mode 100755 tools/testing/selftests/bpf/test_libbpf.sh create mode 100755 tools/testing/selftests/bpf/test_libbpf_open
tools/testing/selftests/bpf/.gitignore +0 -2
@@ -11,7 +11,6 @@ test_dev_cgroup test_tcpbpf_user test_verifier_log feature -test_libbpf_open test_sock test_sock_addr test_sock_fields
@@ -30,7 +29,6 @@ flow_dissector_load test_netcnt test_section_names test_tcpnotify_user -test_libbpf test_tcp_check_syncookie_user test_sysctl libbpf.pc
tools/testing/selftests/bpf/Makefile +1 -2
@@ -52,7 +52,6 @@ TEST_FILES = # Order correspond to 'make run_tests' order TEST_PROGS := test_kmod.sh \ - test_libbpf.sh \ test_xdp_redirect.sh \ test_xdp_meta.sh \ test_xdp_veth.sh \
@@ -79,7 +78,7 @@ TEST_PROGS_EXTENDED := with_addr.sh \ test_xdp_vlan.sh # Compile but not part of 'make run_tests' -TEST_GEN_PROGS_EXTENDED = test_libbpf_open test_sock_addr test_skb_cgroup_id_user \ +TEST_GEN_PROGS_EXTENDED = test_sock_addr test_skb_cgroup_id_user \ flow_dissector_load test_flow_dissector test_tcp_check_syncookie_user \ test_lirc_mode2_user
tools/testing/selftests/bpf/test_libbpf new file binary +0 -0
tools/testing/selftests/bpf/test_libbpf.sh deleted +0 -43
43 lines not on this page · raw message
tools/testing/selftests/bpf/test_libbpf_open new file binary +0 -0
Signature · 1 line
2.17.1
Re: [PATCH v3 bpf-next 7/7] selftest/bpf: remove test_libbpf.sh and test_libbpf_open
Quoted 5 lines · Andrii Nakryiko
test_progsis much more sophisticated superset of tests compared totest_libbpf.shandtest_libbpf_open. Removetest_libbpf.shandtest_libbpf_open.
Quoted 7 lines
Signed-off-by Andrii Nakryiko <[email protected]>--- tools/testing/selftests/bpf/.gitignore | 2 - tools/testing/selftests/bpf/Makefile | 3 +- tools/testing/selftests/bpf/test_libbpf | Bin 0 -> 384568 bytes tools/testing/selftests/bpf/test_libbpf.sh | 43 ------------------- tools/testing/selftests/bpf/test_libbpf_open | Bin 0 -> 396096 bytes
well this was certainly not intended, sorry about that
tools/testing/selftests/bpf/test_libbpf
Quoted 52 lines
5 files changed, 1 insertion(+), 47 deletions(-) create mode 100755 tools/testing/selftests/bpf/test_libbpf delete mode 100755 tools/testing/selftests/bpf/test_libbpf.sh create mode 100755 tools/testing/selftests/bpf/test_libbpf_opentools/testing/selftests/bpf/.gitignore +0
-2@@ -11,7 +11,6 @@ test_dev_cgroup test_tcpbpf_user test_verifier_log feature -test_libbpf_open test_sock test_sock_addr test_sock_fields@@ -30,7 +29,6 @@ flow_dissector_load test_netcnt test_section_names test_tcpnotify_user -test_libbpf test_tcp_check_syncookie_user test_sysctl libbpf.pctools/testing/selftests/bpf/Makefile +1
-2@@ -52,7 +52,6 @@ TEST_FILES = # Order correspond to 'make run_tests' order TEST_PROGS := test_kmod.sh \ - test_libbpf.sh \ test_xdp_redirect.sh \ test_xdp_meta.sh \ test_xdp_veth.sh \@@ -79,7 +78,7 @@ TEST_PROGS_EXTENDED := with_addr.sh \ test_xdp_vlan.sh # Compile but not part of 'make run_tests' -TEST_GEN_PROGS_EXTENDED = test_libbpf_open test_sock_addr test_skb_cgroup_id_user \ +TEST_GEN_PROGS_EXTENDED = test_sock_addr test_skb_cgroup_id_user \ flow_dissector_load test_flow_dissector test_tcp_check_syncookie_user \ test_lirc_mode2_usertools/testing/selftests/bpf/test_libbpf new file binary +0
-0
oops