bpftool, selftests: Add tests for C dump and fix a dropped type
Subsystems (4)
Ihor Solodrai
reviewerv2 cover bpftool, selftests: Add tests for C dump and fix a dropped type
"bpftool btf dump format c" generates the vmlinux.h that BPF programs are built against, and it has no direct test coverage. Building against the header is not a substitute for testing it, nothing a compiler accepts would break the build.
Add selftests targeting C dump in particular.
While we are here, refactor btf_dump selftests to use a new compare_text_to_expected() helper and become more efficient.
Fix a bug in bpftool dump that dropped a type when sorting.
---
v1->v2:
* new patches (#1-#3) introducing compare_text_to_expected() helper and refactoring btf_dump selftests
* bpftool dump selftest compares the output to the expected files exactly (Eduard)
* use VALIDATE_RAW_BTF macro in the new test (Eduard)
* drop v1 patch #4 ("selftests/bpf: Check that sorting preserves types in bpftool dump") as it became unnecessary
* nit in the bpftool fix (Andrii)
v1: https://lore.kernel.org/bpf/[email protected]/
---
Ihor Solodrai (6):
selftests/bpf: Add compare_text_to_expected() helper selftests/bpf: Modernize btf_dump test scaffolding selftests/bpf: Compare btf_dump expected output in-process selftests/bpf: NUL-terminate bpftool command output selftests/bpf: Add tests for bpftool btf dump format c
bpftool: Don't drop a type in the sorted C dump
tools/bpf/bpftool/btf.c | 6 +- tools/testing/selftests/bpf/Makefile | 8 +- .../bpf/bpftool_btf_dump_sorted.expected | 48 +++++ .../bpf/bpftool_btf_dump_unsorted.expected | 48 +++++ tools/testing/selftests/bpf/bpftool_helpers.c | 7 +- .../bpf/prog_tests/bpftool_btf_dump.c | 178 ++++++++++++++++++ .../selftests/bpf/prog_tests/btf_dump.c | 125 ++++++++---- .../bpf/prog_tests/prog_tests_framework.c | 23 +++ tools/testing/selftests/bpf/testing_helpers.c | 43 +++++ tools/testing/selftests/bpf/testing_helpers.h | 3 + 10 files changed, 446 insertions(+), 43 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_unsorted.expected create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_btf_dump.c
base-commit: cd35e1b10182c42b4ae31ee49119463b17d8ba7f
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH v2 1/6 selftests/bpf: Add compare_text_to_expected() helper
Some selftests generate text (such as BTF dump) and check it against an expectation committed nearby.
There is no shared way to do that. prog_tests/btf_dump.c assembles an "awk ... | diff -u" pipeline and hands it to system(). Any other test wanting the same behaviour needs to reproduce both the comparison and the reporting of a mismatch.
test_progs captures per-subtest output by pointing the stdout and stderr FILE * globals at a memstream. But a child process inherits descriptors, not the globals. So the "| diff -u" goes to the console instead of the subtest log and is absent from the failure report.
Add a helper that compares two strings and runs diff(1) on mismatch, properly relaying the output to stdout. Add tests for the helper.
--- .../bpf/prog_tests/prog_tests_framework.c | 23 ++++++++++ tools/testing/selftests/bpf/testing_helpers.c | 43 +++++++++++++++++++ tools/testing/selftests/bpf/testing_helpers.h | 3 ++ 3 files changed, 69 insertions(+)
tools/testing/selftests/bpf/prog_tests/prog_tests_framework.c +23 -0
@@ -179,3 +179,26 @@ void test_prog_tests_framework_expected_msgs(void) } } } + +void test_prog_tests_framework_compare_text (void ) +{ +int err; + +if (test__start_subtest ("compare_text_match" )) { + err =compare_text_to_expected ("same\n" ,"same\n" ); +ASSERT_EQ (err,0 ,"match_rc" ); +test__end_subtest (); + } + +if (test__start_subtest ("compare_text_mismatch" )) { + err =compare_text_to_expected ("line two\n" ,"line one\n" ); +fflush (stdout); + +ASSERT_EQ (err,-1 ,"mismatch_rc" ); +ASSERT_HAS_SUBSTR (env.subtest_state->log_buf,"-line one" , +"diff_has_expected" ); +ASSERT_HAS_SUBSTR (env.subtest_state->log_buf,"+line two" , +"diff_has_actual" ); +test__end_subtest (); + } +}
tools/testing/selftests/bpf/testing_helpers.c +43 -0
46 lines not on this page · raw message
tools/testing/selftests/bpf/testing_helpers.h +3 -0
@@ -61,4 +61,7 @@ int testing_prog_flags(void);bool is_jit_enabled (void );int stack_mprotect (void ); +/* Runs diff(1) on mismatch */ +int compare_text_to_expected (const char *actual,const char *expected); +#endif /* __TESTING_HELPERS_H */
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH v2 2/6 selftests/bpf: Modernize btf_dump test scaffolding
In reply to Ihor SolodraiRefactor test_btf_dump_case() in order to:
* use newer ASSERT_* macros instead of CHECK
* drop the file-scope "duration" variable CHECK required
--- .../selftests/bpf/prog_tests/btf_dump.c | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +8 -11
@@ -2,8 +2,6 @@#include <test_progs.h> #include <bpf/btf.h> -static int duration =0 ; -void btf_dump_printf (void *ctx,const char *fmt,va_list args) {vfprintf (ctx, fmt, args);
@@ -69,8 +67,9 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)if (!t->known_ptr_sz) {btf__set_pointer_size (btf,8 ); }else { -CHECK (btf__pointer_size (btf) !=8 ,"ptr_sz" ,"exp %d, got %zu\n" , -8 ,btf__pointer_size (btf)); +size_t ptr_sz =btf__pointer_size (btf); + +ASSERT_EQ (ptr_sz, (size_t )8 ,"ptr_sz" ); }snprintf (out_file,sizeof (out_file),"/tmp/%s.output.XXXXXX" , t->file);
@@ -80,8 +79,7 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t) goto done; } f =fdopen (fd,"w" ); -if (CHECK (f ==NULL ,"open_tmp" ,"failed to open file: %s(%d)\n" , -strerror (errno), errno)) { +if (!ASSERT_OK_PTR (f,"open_tmp" )) {close (fd); goto done; }
@@ -89,9 +87,8 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t) err =btf_dump_all_types (btf, f);fclose (f);close (fd); -if (CHECK (err,"btf_dump" ,"failure during C dumping: %d\n" , err)) { +if (!ASSERT_OK (err,"btf_dump" )) goto done; - }snprintf (test_file,sizeof (test_file),"progs/%s.c" , t->file);if (access (test_file,R_OK ) ==-1 )
@@ -114,10 +111,10 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)"out {sub(/^[ \\t]*\\*/, \"\"); print}' '%s' | diff -u - '%s'" , test_file, out_file); err =system (diff_cmd); -if (CHECK (err,"diff" , -"differing test output, output=%s, err=%d, diff cmd:\n%s\n" , - out_file, err, diff_cmd)) +if (!ASSERT_OK (err,"diff" )) { +fprintf (stdout,"output=%s, diff cmd:\n%s\n" , out_file, diff_cmd); goto done; + }remove (out_file);
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH v2 3/6 selftests/bpf: Compare btf_dump expected output in-process
In reply to Ihor Solodraitest_btf_dump_case() writes the dumped types to a temporary file and then runs a command against the expectation:
system("awk '...' 'progs/x.c' | diff -u - '/tmp/x.output.XXXXXX'");This creates three processes per test case: a shell, awk and diff. The awk cannot be deferred to the failure path, because its output is the expected text.
In a VM with a 9p root (used on BPF CI) process creation dominates the cost. A bare fork+exec measures around 110ms, so the btf_dump test cases spend most of their runtime forking.
Replace the awk pipeline with a simple marker parser. Buffer the dump in memory, and use compare_text_to_expected(), which only runs diff(1) on mismatch.
Measured with "time ./test_progs -t btf_dump" in the VM:
real 2.751s -> 0.702s, sys 1.901s -> 0.193s
This also drops the temporary file setup.
--- .../selftests/bpf/prog_tests/btf_dump.c | 112 +++++++++++++----- 1 file changed, 82 insertions(+), 30 deletions(-)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +82 -30
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #include <test_progs.h> #include <bpf/btf.h> +#include "testing_helpers.h" void btf_dump_printf (void *ctx,const char *fmt,va_list args) {
@@ -43,12 +44,75 @@ static int btf_dump_all_types(const struct btf *btf, void *ctx)return err; } +/* +* Expected output is embedded in the test case source, between +* START-EXPECTED-OUTPUT and END-EXPECTED-OUTPUT markers. A region is either +* plain C, where a declaration doubles as its own expectation, or C wrapped in +* a block comment, where the rendered form differs from the source. All +* regions of a file concatenate into one expectation, compared against one +* whole-file dump. +* +* Returns a malloc'd buffer for the caller to free, or NULL on failure. +*/ +static char *read_expected_output (const char *path) +{ +size_t out_sz =0 , line_cap =0 ; +char *out =NULL , *line =NULL ; +bool in_region = false; +FILE *f, *out_file; + + f =fopen (path,"r" ); +if (!f) +return NULL ; + + out_file =open_memstream (&out, &out_sz); +if (!out_file) { +fclose (f); +return NULL ; + } + +while (getline (&line, &line_cap, f) >0 ) { +const char *p; + +if (strstr (line,"START-EXPECTED-OUTPUT" )) { + in_region = true; +continue ; + } +if (strstr (line,"END-EXPECTED-OUTPUT" )) + in_region = false; +if (!in_region) +continue ; + + p = line +strspn (line," \t" ); + +/* opening or closing line of a commented out region */ +if (!strncmp (p,"/*" ,2 ) || !strncmp (p,"*/" ,2 )) +continue ; + +/* +* Only a '*' directly after the indentation is a comment +* prefix. Without one the line is taken as it is, leading +* whitespace included. +*/ + p = *p =='*' ? p +1 : line; + +fputs (p, out_file); + } + +free (line); +fclose (f); +fclose (out_file); +return out; +} +static int test_btf_dump_case (int n,struct btf_dump_test_case *t) { -char test_file[256 ], out_file[256 ], diff_cmd[1024 ]; +char *dump =NULL , *expected =NULL ;struct btf *btf =NULL ; -int err =0 , fd =-1 ; -FILE *f =NULL ; +char test_file[256 ]; +size_t dump_sz =0 ; +int err =0 ; +FILE *f;snprintf (test_file,sizeof (test_file),"%s.bpf.o" , t->file);
@@ -72,21 +136,14 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)ASSERT_EQ (ptr_sz, (size_t )8 ,"ptr_sz" ); } -snprintf (out_file,sizeof (out_file),"/tmp/%s.output.XXXXXX" , t->file); - fd =mkstemp (out_file); -if (!ASSERT_GE (fd,0 ,"create_tmp" )) { - err = fd; - goto done; - } - f =fdopen (fd,"w" ); -if (!ASSERT_OK_PTR (f,"open_tmp" )) { -close (fd); + f =open_memstream (&dump, &dump_sz); +if (!ASSERT_OK_PTR (f,"open_memstream" )) { + err = -errno; goto done; } err =btf_dump_all_types (btf, f);fclose (f); -close (fd);if (!ASSERT_OK (err,"btf_dump" )) goto done;
@@ -97,28 +154,23 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)* without preserving the directory structure. */ snprintf (test_file,sizeof (test_file),"%s.c" , t->file); -/* -* Diff test output and expected test output, contained between -* START-EXPECTED-OUTPUT and END-EXPECTED-OUTPUT lines in test case. -* For expected output lines, everything before '*' is stripped out. -* Also lines containing comment start and comment end markers are -* ignored. -*/ -snprintf (diff_cmd,sizeof (diff_cmd), -"awk '/START-EXPECTED-OUTPUT/{out=1;next} " -"/END-EXPECTED-OUTPUT/{out=0} " -"/\\/\\*|\\*\\//{next} " /* ignore comment start/end lines */ -"out {sub(/^[ \\t]*\\*/, \"\"); print}' '%s' | diff -u - '%s'" , - test_file, out_file); - err =system (diff_cmd); -if (!ASSERT_OK (err,"diff" )) { -fprintf (stdout,"output=%s, diff cmd:\n%s\n" , out_file, diff_cmd); + + expected =read_expected_output (test_file); +if (!ASSERT_OK_PTR (expected,"read_expected_output" )) { + err = -errno; goto done; } -remove (out_file); +/* +* The mismatch has already been reported, so this only has to +* register the failure. ASSERT_OK() would append a stale errno to it. +*/ + err =compare_text_to_expected (dump, expected); +ASSERT_EQ (err,0 ,"compare_text_to_expected" ); done: +free (expected); +free (dump);btf__free (btf);return err; }
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH v2 4/6 selftests/bpf: NUL-terminate bpftool command output
In reply to Ihor Solodrairun_command() writes a bpftool command's output into a caller-supplied buffer, and every caller treats that buffer as a C string.
However fread() reports a byte count and doesn't terminate the string. The helper does not terminate either, so callers have to zero the buffer first. prog_tests/bpftool_metadata.c does not, for example.
Read one byte less and terminate in the helper.
--- tools/testing/selftests/bpf/bpftool_helpers.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/bpftool_helpers.c +5 -2
@@ -53,6 +53,7 @@ static int run_command(char *args, char *output_buf, size_t output_max_len)static char bpftool_path[PATH_MAX ] = {};bool suppress_output = !(output_buf && output_max_len);char command[BPFTOOL_FULL_CMD_MAX_LEN ]; +size_t n;FILE *f;int ret;
@@ -68,8 +69,10 @@ static int run_command(char *args, char *output_buf, size_t output_max_len)if (!f)return 1 ; -if (!suppress_output) -fread (output_buf,1 , output_max_len, f); +if (!suppress_output) { + n =fread (output_buf,1 , output_max_len -1 , f); + output_buf[n] ='\0' ; + } ret =pclose (f);return ret;
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH v2 5/6 selftests/bpf: Add tests for bpftool btf dump format c
In reply to Ihor Solodrai"bpftool btf dump format c" generates the vmlinux.h that BPF programs are built against, and it has no test coverage at all. The only in-tree consumers are build systems.
Building against the header only catches what a compiler rejects, which is insufficient.
Introduce a bpftool_btf_dump selftest. Dump a small hand-built BTF and compare the output against committed expectations.
The BTF is assembled with btf__add_*() rather than compiled from BPF C because the fixture needs a 4-byte "long int". That is what makes bpftool render the hole in struct holey as a pair of "long: 32;" bitfields. A BPF target is always 64-bit, so a compiled fixture could not supply it.
Comparing the whole dump makes every change to a generated header show up in a patch.
--- tools/testing/selftests/bpf/Makefile | 8 +- .../bpf/bpftool_btf_dump_sorted.expected | 44 +++++ .../bpf/bpftool_btf_dump_unsorted.expected | 48 +++++ .../bpf/prog_tests/bpftool_btf_dump.c | 178 ++++++++++++++++++ 4 files changed, 276 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_unsorted.expected create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_btf_dump.c
tools/testing/selftests/bpf/Makefile +6 -2
@@ -119,7 +119,9 @@ TEST_GEN_PROGS += test_progs-cpuv4 TEST_INST_SUBDIRS += cpuv4 endif -TEST_FILES = xsk_prereqs.sh $(wildcard progs/btf_dump_test_case_*.c) +TEST_FILES = xsk_prereqs.sh $(wildcard progs/btf_dump_test_case_*.c) \ + bpftool_btf_dump_sorted.expected \ + bpftool_btf_dump_unsorted.expected # Order correspond to 'make run_tests' order TEST_PROGS := test_kmod.sh \
@@ -887,7 +889,9 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \ ima_setup.sh \ $(VERIFY_SIG_SETUP) \ $(wildcard progs/btf_dump_test_case_*.c) \ - $(wildcard progs/*.bpf.o) + $(wildcard progs/*.bpf.o) \ + bpftool_btf_dump_sorted.expected \ + bpftool_btf_dump_unsorted.expected TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS) -DENABLE_ATOMICS_TESTS $(eval $(call DEFINE_TEST_RUNNER,test_progs))
tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected new file +44 -0
44 lines not on this page · raw message
tools/testing/selftests/bpf/bpftool_btf_dump_unsorted.expected new file +48 -0
48 lines not on this page · raw message
tools/testing/selftests/bpf/prog_tests/bpftool_btf_dump.c new file +178 -0
178 lines not on this page · raw message
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH v2 6/6 bpftool: Don't drop a type in the sorted C dump
In reply to Ihor SolodraiThe C dump sorts types by default, so that generated headers are diffable. The sorted dump emits one type fewer than the unsorted dump of the same BTF.
dump_btf_c() starts its loop at index 1 to skip the void type at BTF type ID 0. That holds for the unsorted dump, where the array index is the type ID, but not after qsort(): position 0 is then the lowest ranked type, and btf_type_rank() ranks an anonymous enum 0 while void takes the default rank of 10. So the enum is skipped, and void is emitted instead as a no-op.
--- tools/bpf/bpftool/btf.c | 6 +++++- .../testing/selftests/bpf/bpftool_btf_dump_sorted.expected | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-)
tools/bpf/bpftool/btf.c +5 -1
@@ -805,9 +805,13 @@ static int dump_btf_c(const struct btf *btf,if (sort_dump) datums =sort_btf_c (btf); -for (i =1 ; i < cnt; i++) { +for (i =0 ; i < cnt; i++) {int idx = datums ? datums[i].index : i; +/* type ID 0 is void, skip it */ +if (idx ==0 ) +continue ; + err =btf_dump__dump_type (d, idx);if (err) goto done;
tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected +4 -0
@@ -21,6 +21,10 @@ #endif #endif +enum { + E0 = 1, +}; + struct holey { int c; long: 32;
Signature · 1 line
2.55.0
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Fri, 28 Aug 2026 14:52:01 -0700 you wrote:
"bpftool btf dump format c" generates the
vmlinux.hthat BPF programs are built against, and it has no direct test coverage. Building against the header is not a substitute for testing it, nothing a compiler accepts would break the build.Add selftests targeting C dump in particular.
[...]
Here is the summary with links:
- [bpf-next,v2,1/6] selftests/bpf: Add compare_text_to_expected() helper
https://git.kernel.org/bpf/bpf-next/c/bd71904697c6 - [bpf-next,v2,2/6] selftests/bpf: Modernize btf_dump test scaffolding https://git.kernel.org/bpf/bpf-next/c/c0722dc3288a - [bpf-next,v2,3/6] selftests/bpf: Compare btf_dump expected output in-process https://git.kernel.org/bpf/bpf-next/c/122255fb1933 - [bpf-next,v2,4/6] selftests/bpf: NUL-terminate bpftool command output https://git.kernel.org/bpf/bpf-next/c/4f8a4c76f670 - [bpf-next,v2,5/6] selftests/bpf: Add tests for bpftool btf dump format c https://git.kernel.org/bpf/bpf-next/c/438147f7f601 - [bpf-next,v2,6/6] bpftool: Don't drop a type in the sorted C dump https://git.kernel.org/bpf/bpf-next/c/ff01f88a9136
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html