Skip to contentSkip to conversation

bpftool, selftests: Add tests for C dump and fix a dropped type

PATCH Applied Ihor Solodrai 2 participants 8 messages Aug 28 – Aug 30

Subsystems (4)
lore
Share & tools
lore

Ihor Solodrai

author reviewer
#0 lore raw

v2 cover bpftool, selftests: Add tests for C dump and fix a dropped type

"bpftool btf dump format c" generates the vmlinux.h that BPF programs are built against, and it has no direct test coverage. Building against the header is not a substitute for testing it, nothing a compiler accepts would break the build.

Add selftests targeting C dump in particular.

While we are here, refactor btf_dump selftests to use a new compare_text_to_expected() helper and become more efficient.

Fix a bug in bpftool dump that dropped a type when sorting.

---

v1->v2:
* new patches (#1-#3) introducing compare_text_to_expected() helper and refactoring btf_dump selftests
* bpftool dump selftest compares the output to the expected files exactly (Eduard)
* use VALIDATE_RAW_BTF macro in the new test (Eduard)
* drop v1 patch #4 ("selftests/bpf: Check that sorting preserves types in bpftool dump") as it became unnecessary
* nit in the bpftool fix (Andrii)

v1: https://lore.kernel.org/bpf/[email protected]/

---

Ihor Solodrai (6):
selftests/bpf: Add compare_text_to_expected() helper selftests/bpf: Modernize btf_dump test scaffolding selftests/bpf: Compare btf_dump expected output in-process selftests/bpf: NUL-terminate bpftool command output selftests/bpf: Add tests for bpftool btf dump format c
bpftool: Don't drop a type in the sorted C dump

 tools/bpf/bpftool/btf.c                       |   6 +-
 tools/testing/selftests/bpf/Makefile          |   8 +-
 .../bpf/bpftool_btf_dump_sorted.expected      |  48 +++++
 .../bpf/bpftool_btf_dump_unsorted.expected    |  48 +++++
 tools/testing/selftests/bpf/bpftool_helpers.c |   7 +-
 .../bpf/prog_tests/bpftool_btf_dump.c         | 178 ++++++++++++++++++
 .../selftests/bpf/prog_tests/btf_dump.c       | 125 ++++++++----
 .../bpf/prog_tests/prog_tests_framework.c     |  23 +++
 tools/testing/selftests/bpf/testing_helpers.c |  43 +++++
 tools/testing/selftests/bpf/testing_helpers.h |   3 +
 10 files changed, 446 insertions(+), 43 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected
 create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_unsorted.expected
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_btf_dump.c

base-commit: cd35e1b10182c42b4ae31ee49119463b17d8ba7f

Signature · 1 line
2.55.0
author reviewer
#1 lore raw

PATCH v2 1/6 selftests/bpf: Add compare_text_to_expected() helper

Some selftests generate text (such as BTF dump) and check it against an expectation committed nearby.

There is no shared way to do that. prog_tests/btf_dump.c assembles an "awk ... | diff -u" pipeline and hands it to system(). Any other test wanting the same behaviour needs to reproduce both the comparison and the reporting of a mismatch.

test_progs captures per-subtest output by pointing the stdout and stderr FILE * globals at a memstream. But a child process inherits descriptors, not the globals. So the "| diff -u" goes to the console instead of the subtest log and is absent from the failure report.

Add a helper that compares two strings and runs diff(1) on mismatch, properly relaying the output to stdout. Add tests for the helper.

---
 .../bpf/prog_tests/prog_tests_framework.c     | 23 ++++++++++
 tools/testing/selftests/bpf/testing_helpers.c | 43 +++++++++++++++++++
 tools/testing/selftests/bpf/testing_helpers.h |  3 ++
 3 files changed, 69 insertions(+)
tools/testing/selftests/bpf/prog_tests/prog_tests_framework.c +23 -0
index 7607cfc2408c..d111fe105447 100644
@@ -179,3 +179,26 @@ void test_prog_tests_framework_expected_msgs(void)
 		}
 	}
 }
+
+void test_prog_tests_framework_compare_text(void)
+{
+	int err;
+
+	if (test__start_subtest("compare_text_match")) {
+		err = compare_text_to_expected("same\n", "same\n");
+		ASSERT_EQ(err, 0, "match_rc");
+		test__end_subtest();
+	}
+
+	if (test__start_subtest("compare_text_mismatch")) {
+		err = compare_text_to_expected("line two\n", "line one\n");
+		fflush(stdout);
+
+		ASSERT_EQ(err, -1, "mismatch_rc");
+		ASSERT_HAS_SUBSTR(env.subtest_state->log_buf, "-line one",
+				  "diff_has_expected");
+		ASSERT_HAS_SUBSTR(env.subtest_state->log_buf, "+line two",
+				  "diff_has_actual");
+		test__end_subtest();
+	}
+}
tools/testing/selftests/bpf/testing_helpers.c +43 -0

46 lines not on this page · raw message

tools/testing/selftests/bpf/testing_helpers.h +3 -0
index 2edc6fb7fc52..1c58a2f08b64 100644
@@ -61,4 +61,7 @@ int testing_prog_flags(void);
 bool is_jit_enabled(void);
 int stack_mprotect(void);
 
+/* Runs diff(1) on mismatch */
+int compare_text_to_expected(const char *actual, const char *expected);
+
 #endif /* __TESTING_HELPERS_H */
Signature · 1 line
2.55.0
author reviewer
#2 lore raw

PATCH v2 2/6 selftests/bpf: Modernize btf_dump test scaffolding

In reply to Ihor Solodrai

Refactor test_btf_dump_case() in order to:
* use newer ASSERT_* macros instead of CHECK
* drop the file-scope "duration" variable CHECK required

---
 .../selftests/bpf/prog_tests/btf_dump.c       | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +8 -11
index 9f1b50e07a29..e9d2d4c26509 100644
@@ -2,8 +2,6 @@
 #include <test_progs.h>
 #include <bpf/btf.h>
 
-static int duration = 0;
-
 void btf_dump_printf(void *ctx, const char *fmt, va_list args)
 {
 	vfprintf(ctx, fmt, args);
@@ -69,8 +67,9 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 	if (!t->known_ptr_sz) {
 		btf__set_pointer_size(btf, 8);
 	} else {
-		CHECK(btf__pointer_size(btf) != 8, "ptr_sz", "exp %d, got %zu\n",
-		      8, btf__pointer_size(btf));
+		size_t ptr_sz = btf__pointer_size(btf);
+
+		ASSERT_EQ(ptr_sz, (size_t)8, "ptr_sz");
 	}
 
 	snprintf(out_file, sizeof(out_file), "/tmp/%s.output.XXXXXX", t->file);
@@ -80,8 +79,7 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 		goto done;
 	}
 	f = fdopen(fd, "w");
-	if (CHECK(f == NULL, "open_tmp",  "failed to open file: %s(%d)\n",
-		  strerror(errno), errno)) {
+	if (!ASSERT_OK_PTR(f, "open_tmp")) {
 		close(fd);
 		goto done;
 	}
@@ -89,9 +87,8 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 	err = btf_dump_all_types(btf, f);
 	fclose(f);
 	close(fd);
-	if (CHECK(err, "btf_dump", "failure during C dumping: %d\n", err)) {
+	if (!ASSERT_OK(err, "btf_dump"))
 		goto done;
-	}
 
 	snprintf(test_file, sizeof(test_file), "progs/%s.c", t->file);
 	if (access(test_file, R_OK) == -1)
@@ -114,10 +111,10 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 		 "out {sub(/^[ \\t]*\\*/, \"\"); print}' '%s' | diff -u - '%s'",
 		 test_file, out_file);
 	err = system(diff_cmd);
-	if (CHECK(err, "diff",
-		  "differing test output, output=%s, err=%d, diff cmd:\n%s\n",
-		  out_file, err, diff_cmd))
+	if (!ASSERT_OK(err, "diff")) {
+		fprintf(stdout, "output=%s, diff cmd:\n%s\n", out_file, diff_cmd);
 		goto done;
+	}
 
 	remove(out_file);
 
Signature · 1 line
2.55.0
author reviewer
#3 lore raw

PATCH v2 3/6 selftests/bpf: Compare btf_dump expected output in-process

In reply to Ihor Solodrai

test_btf_dump_case() writes the dumped types to a temporary file and then runs a command against the expectation:

	system("awk '...' 'progs/x.c' | diff -u - '/tmp/x.output.XXXXXX'");

This creates three processes per test case: a shell, awk and diff. The awk cannot be deferred to the failure path, because its output is the expected text.

In a VM with a 9p root (used on BPF CI) process creation dominates the cost. A bare fork+exec measures around 110ms, so the btf_dump test cases spend most of their runtime forking.

Replace the awk pipeline with a simple marker parser. Buffer the dump in memory, and use compare_text_to_expected(), which only runs diff(1) on mismatch.

Measured with "time ./test_progs -t btf_dump" in the VM:

	real 2.751s -> 0.702s, sys 1.901s -> 0.193s

This also drops the temporary file setup.

---
 .../selftests/bpf/prog_tests/btf_dump.c       | 112 +++++++++++++-----
 1 file changed, 82 insertions(+), 30 deletions(-)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +82 -30
index e9d2d4c26509..fe04a955d46c 100644
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <test_progs.h>
 #include <bpf/btf.h>
+#include "testing_helpers.h"
 
 void btf_dump_printf(void *ctx, const char *fmt, va_list args)
 {
@@ -43,12 +44,75 @@ static int btf_dump_all_types(const struct btf *btf, void *ctx)
 	return err;
 }
 
+/*
+ * Expected output is embedded in the test case source, between
+ * START-EXPECTED-OUTPUT and END-EXPECTED-OUTPUT markers. A region is either
+ * plain C, where a declaration doubles as its own expectation, or C wrapped in
+ * a block comment, where the rendered form differs from the source. All
+ * regions of a file concatenate into one expectation, compared against one
+ * whole-file dump.
+ *
+ * Returns a malloc'd buffer for the caller to free, or NULL on failure.
+ */
+static char *read_expected_output(const char *path)
+{
+	size_t out_sz = 0, line_cap = 0;
+	char *out = NULL, *line = NULL;
+	bool in_region = false;
+	FILE *f, *out_file;
+
+	f = fopen(path, "r");
+	if (!f)
+		return NULL;
+
+	out_file = open_memstream(&out, &out_sz);
+	if (!out_file) {
+		fclose(f);
+		return NULL;
+	}
+
+	while (getline(&line, &line_cap, f) > 0) {
+		const char *p;
+
+		if (strstr(line, "START-EXPECTED-OUTPUT")) {
+			in_region = true;
+			continue;
+		}
+		if (strstr(line, "END-EXPECTED-OUTPUT"))
+			in_region = false;
+		if (!in_region)
+			continue;
+
+		p = line + strspn(line, " \t");
+
+		/* opening or closing line of a commented out region */
+		if (!strncmp(p, "/*", 2) || !strncmp(p, "*/", 2))
+			continue;
+
+		/*
+		 * Only a '*' directly after the indentation is a comment
+		 * prefix. Without one the line is taken as it is, leading
+		 * whitespace included.
+		 */
+		p = *p == '*' ? p + 1 : line;
+
+		fputs(p, out_file);
+	}
+
+	free(line);
+	fclose(f);
+	fclose(out_file);
+	return out;
+}
+
 static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 {
-	char test_file[256], out_file[256], diff_cmd[1024];
+	char *dump = NULL, *expected = NULL;
 	struct btf *btf = NULL;
-	int err = 0, fd = -1;
-	FILE *f = NULL;
+	char test_file[256];
+	size_t dump_sz = 0;
+	int err = 0;
+	FILE *f;
 
 	snprintf(test_file, sizeof(test_file), "%s.bpf.o", t->file);
 
@@ -72,21 +136,14 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 		ASSERT_EQ(ptr_sz, (size_t)8, "ptr_sz");
 	}
 
-	snprintf(out_file, sizeof(out_file), "/tmp/%s.output.XXXXXX", t->file);
-	fd = mkstemp(out_file);
-	if (!ASSERT_GE(fd, 0, "create_tmp")) {
-		err = fd;
-		goto done;
-	}
-	f = fdopen(fd, "w");
-	if (!ASSERT_OK_PTR(f, "open_tmp")) {
-		close(fd);
+	f = open_memstream(&dump, &dump_sz);
+	if (!ASSERT_OK_PTR(f, "open_memstream")) {
+		err = -errno;
 		goto done;
 	}
 
 	err = btf_dump_all_types(btf, f);
 	fclose(f);
-	close(fd);
 	if (!ASSERT_OK(err, "btf_dump"))
 		goto done;
 
@@ -97,28 +154,23 @@ static int test_btf_dump_case(int n, struct btf_dump_test_case *t)
 		 * without preserving the directory structure.
 		 */
 		snprintf(test_file, sizeof(test_file), "%s.c", t->file);
-	/*
-	 * Diff test output and expected test output, contained between
-	 * START-EXPECTED-OUTPUT and END-EXPECTED-OUTPUT lines in test case.
-	 * For expected output lines, everything before '*' is stripped out.
-	 * Also lines containing comment start and comment end markers are
-	 * ignored. 
-	 */
-	snprintf(diff_cmd, sizeof(diff_cmd),
-		 "awk '/START-EXPECTED-OUTPUT/{out=1;next} "
-		 "/END-EXPECTED-OUTPUT/{out=0} "
-		 "/\\/\\*|\\*\\//{next} " /* ignore comment start/end lines */
-		 "out {sub(/^[ \\t]*\\*/, \"\"); print}' '%s' | diff -u - '%s'",
-		 test_file, out_file);
-	err = system(diff_cmd);
-	if (!ASSERT_OK(err, "diff")) {
-		fprintf(stdout, "output=%s, diff cmd:\n%s\n", out_file, diff_cmd);
+
+	expected = read_expected_output(test_file);
+	if (!ASSERT_OK_PTR(expected, "read_expected_output")) {
+		err = -errno;
 		goto done;
 	}
 
-	remove(out_file);
+	/*
+	 * The mismatch has already been reported, so this only has to
+	 * register the failure. ASSERT_OK() would append a stale errno to it.
+	 */
+	err = compare_text_to_expected(dump, expected);
+	ASSERT_EQ(err, 0, "compare_text_to_expected");
 
 done:
+	free(expected);
+	free(dump);
 	btf__free(btf);
 	return err;
 }
Signature · 1 line
2.55.0
author reviewer
#4 lore raw

PATCH v2 4/6 selftests/bpf: NUL-terminate bpftool command output

In reply to Ihor Solodrai

run_command() writes a bpftool command's output into a caller-supplied buffer, and every caller treats that buffer as a C string.

However fread() reports a byte count and doesn't terminate the string. The helper does not terminate either, so callers have to zero the buffer first. prog_tests/bpftool_metadata.c does not, for example.

Read one byte less and terminate in the helper.

Fixes f21fae577446 ("selftests/bpf: Add a few helpers for bpftool testing")
---
 tools/testing/selftests/bpf/bpftool_helpers.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/bpftool_helpers.c +5 -2
index 0a2a4f0a2794..c49fdd90eb03 100644
@@ -53,6 +53,7 @@ static int run_command(char *args, char *output_buf, size_t output_max_len)
 	static char bpftool_path[PATH_MAX] = {};
 	bool suppress_output = !(output_buf && output_max_len);
 	char command[BPFTOOL_FULL_CMD_MAX_LEN];
+	size_t n;
 	FILE *f;
 	int ret;
 
@@ -68,8 +69,10 @@ static int run_command(char *args, char *output_buf, size_t output_max_len)
 	if (!f)
 		return 1;
 
-	if (!suppress_output)
-		fread(output_buf, 1, output_max_len, f);
+	if (!suppress_output) {
+		n = fread(output_buf, 1, output_max_len - 1, f);
+		output_buf[n] = '\0';
+	}
 	ret = pclose(f);
 
 	return ret;
Signature · 1 line
2.55.0
author reviewer
#5 lore raw

PATCH v2 5/6 selftests/bpf: Add tests for bpftool btf dump format c

In reply to Ihor Solodrai

"bpftool btf dump format c" generates the vmlinux.h that BPF programs are built against, and it has no test coverage at all. The only in-tree consumers are build systems.

Building against the header only catches what a compiler rejects, which is insufficient.

Introduce a bpftool_btf_dump selftest. Dump a small hand-built BTF and compare the output against committed expectations.

The BTF is assembled with btf__add_*() rather than compiled from BPF C because the fixture needs a 4-byte "long int". That is what makes bpftool render the hole in struct holey as a pair of "long: 32;" bitfields. A BPF target is always 64-bit, so a compiled fixture could not supply it.

Comparing the whole dump makes every change to a generated header show up in a patch.

---
 tools/testing/selftests/bpf/Makefile          |   8 +-
 .../bpf/bpftool_btf_dump_sorted.expected      |  44 +++++
 .../bpf/bpftool_btf_dump_unsorted.expected    |  48 +++++
 .../bpf/prog_tests/bpftool_btf_dump.c         | 178 ++++++++++++++++++
 4 files changed, 276 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected
 create mode 100644 tools/testing/selftests/bpf/bpftool_btf_dump_unsorted.expected
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_btf_dump.c
tools/testing/selftests/bpf/Makefile +6 -2
index 5f1a3bfc0569..d99581257e54 100644
@@ -119,7 +119,9 @@ TEST_GEN_PROGS += test_progs-cpuv4
 TEST_INST_SUBDIRS += cpuv4
 endif
 
-TEST_FILES = xsk_prereqs.sh $(wildcard progs/btf_dump_test_case_*.c)
+TEST_FILES = xsk_prereqs.sh $(wildcard progs/btf_dump_test_case_*.c)	\
+	     bpftool_btf_dump_sorted.expected					\
+	     bpftool_btf_dump_unsorted.expected
 
 # Order correspond to 'make run_tests' order
 TEST_PROGS := test_kmod.sh \
@@ -887,7 +889,9 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read				\
 		       ima_setup.sh 					\
 		       $(VERIFY_SIG_SETUP)				\
 		       $(wildcard progs/btf_dump_test_case_*.c)		\
-		       $(wildcard progs/*.bpf.o)
+		       $(wildcard progs/*.bpf.o)			\
+		       bpftool_btf_dump_sorted.expected			\
+		       bpftool_btf_dump_unsorted.expected
 TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
 TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS) -DENABLE_ATOMICS_TESTS
 $(eval $(call DEFINE_TEST_RUNNER,test_progs))
tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected new file +44 -0

44 lines not on this page · raw message

tools/testing/selftests/bpf/bpftool_btf_dump_unsorted.expected new file +48 -0

48 lines not on this page · raw message

tools/testing/selftests/bpf/prog_tests/bpftool_btf_dump.c new file +178 -0

178 lines not on this page · raw message

Signature · 1 line
2.55.0
author reviewer
#6 lore raw

PATCH v2 6/6 bpftool: Don't drop a type in the sorted C dump

In reply to Ihor Solodrai

The C dump sorts types by default, so that generated headers are diffable. The sorted dump emits one type fewer than the unsorted dump of the same BTF.

dump_btf_c() starts its loop at index 1 to skip the void type at BTF type ID 0. That holds for the unsorted dump, where the array index is the type ID, but not after qsort(): position 0 is then the lowest ranked type, and btf_type_rank() ranks an anonymous enum 0 while void takes the default rank of 10. So the enum is skipped, and void is emitted instead as a no-op.

Fixes 94133cf24bb3 ("bpftool: Introduce btf c dump sorting")
---
 tools/bpf/bpftool/btf.c                                     | 6 +++++-
 .../testing/selftests/bpf/bpftool_btf_dump_sorted.expected  | 4 ++++
 2 files changed, 9 insertions(+), 1 deletion(-)
tools/bpf/bpftool/btf.c +5 -1
index c9589026da8d..bca0a3982f09 100644
@@ -805,9 +805,13 @@ static int dump_btf_c(const struct btf *btf,
 
 		if (sort_dump)
 			datums = sort_btf_c(btf);
-		for (i = 1; i < cnt; i++) {
+		for (i = 0; i < cnt; i++) {
 			int idx = datums ? datums[i].index : i;
 
+			/* type ID 0 is void, skip it */
+			if (idx == 0)
+				continue;
+
 			err = btf_dump__dump_type(d, idx);
 			if (err)
 				goto done;
tools/testing/selftests/bpf/bpftool_btf_dump_sorted.expected +4 -0
index a9c6b688ffd7..5470b2b43229 100644
@@ -21,6 +21,10 @@
 #endif
 #endif
 
+enum {
+	E0 = 1,
+};
+
 struct holey {
 	int c;
 	long: 32;
Signature · 1 line
2.55.0
#7 lore raw
In reply to Ihor Solodrai
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 8 lines · On Fri, 28 Aug 2026 14:52:01 -0700 you wrote:

"bpftool btf dump format c" generates the vmlinux.h that BPF programs are built against, and it has no direct test coverage. Building against the header is not a substitute for testing it, nothing a compiler accepts would break the build.

Add selftests targeting C dump in particular.

[...]

Here is the summary with links:
- [bpf-next,v2,1/6] selftests/bpf: Add compare_text_to_expected() helper

    https://git.kernel.org/bpf/bpf-next/c/bd71904697c6
  - [bpf-next,v2,2/6] selftests/bpf: Modernize btf_dump test scaffolding
    https://git.kernel.org/bpf/bpf-next/c/c0722dc3288a
  - [bpf-next,v2,3/6] selftests/bpf: Compare btf_dump expected output in-process
    https://git.kernel.org/bpf/bpf-next/c/122255fb1933
  - [bpf-next,v2,4/6] selftests/bpf: NUL-terminate bpftool command output
    https://git.kernel.org/bpf/bpf-next/c/4f8a4c76f670
  - [bpf-next,v2,5/6] selftests/bpf: Add tests for bpftool btf dump format c
    https://git.kernel.org/bpf/bpf-next/c/438147f7f601
  - [bpf-next,v2,6/6] bpftool: Don't drop a type in the sorted C dump
    https://git.kernel.org/bpf/bpf-next/c/ff01f88a9136

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html