Skip to contentSkip to conversation

selftests/bpf: migrate a few bpftool testing scripts

PATCH Superseded Alexis Lothoré (eBPF Foundation) 2 participants 6 messages Jan 21 – Jan 22

Subsystems (3)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover selftests/bpf: migrate a few bpftool testing scripts

Hello,
this is the v3 for some bpftool tests conversion. The new tests are being integrated in test_progs so that they can be executed on each CI run.

- First commit introduces a few dedicated helpers to execute bpftool commands, with or without retrieving the generated stdout output
- Second commit integrates test_bpftool_metadata.sh into test_progs
- Third commit integrates test_bpftool_map.sh into test_progs

---
Changes in v3:
- Drop commit reordering objects in Makefile
- Rebased series on ci/bpf-next_base to fix conflict
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- drop standalone runner in favor of test_progs
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Alexis Lothoré (eBPF Foundation) (3):
bpf/selftests: add a few helpers for bpftool testing selftests/bpf: convert test_bpftool_metadata.sh into test_progs framework selftests/bpf: convert test_bpftool_map_access.sh into test_progs framework

 tools/testing/selftests/bpf/Makefile               |   5 +-
 tools/testing/selftests/bpf/bpftool_helpers.c      |  74 ++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  11 +
 .../selftests/bpf/prog_tests/bpftool_maps_access.c | 371 +++++++++++++++++++
 .../selftests/bpf/prog_tests/bpftool_metadata.c    | 134 +++++++
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 7 files changed, 592 insertions(+), 486 deletions(-)

---
base-commit: a1726111c973a6b6f9a46d79db80bcce89cdda7c
change-id: 20251212-bpftool-tests-bb165c4cceb8

Best regards,

Signature · 1 line
Alexis Lothoré (eBPF Foundation) <[email protected]>
#1 lore raw

PATCH v3 1/3 bpf/selftests: add a few helpers for bpftool testing

In order to integrate some bpftool tests into test_progs, define a few specific helpers that allow to execute bpftool commands, while possibly retrieving the command output. Those helpers most notably set the path to the bpftool binary under test. This version checks different possible paths relative to the directories where the different test_progs runners are executed, as we want to make sure not to accidentally use a bootstrap version of the binary.

---
Changes in v2:
- Drop the new runner from commit, keep only the new bpftool-specific helpers
- replace hardcoded path with a detected (and cached) path

---
 tools/testing/selftests/bpf/Makefile          |  3 +-
 tools/testing/selftests/bpf/bpftool_helpers.c | 74 +++++++++++++++++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.h | 11 ++++
 3 files changed, 87 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/Makefile +2 -1
index 9488d076c740..2cc559ecc723 100644
@@ -747,7 +747,8 @@ TRUNNER_EXTRA_SOURCES := test_progs.c		\
 			 json_writer.c 		\
 			 $(VERIFY_SIG_HDR)		\
 			 flow_dissector_load.h	\
-			 ip_check_defrag_frags.h
+			 ip_check_defrag_frags.h	\
+			 bpftool_helpers.c
 TRUNNER_LIB_SOURCES := find_bit.c
 TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read				\
 		       $(OUTPUT)/liburandom_read.so			\
tools/testing/selftests/bpf/bpftool_helpers.c new file +74 -0

74 lines not on this page · raw message

tools/testing/selftests/bpf/bpftool_helpers.h new file +11 -0
new file mode 100644
index 000000000000..dec1ba201410
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdbool.h>
+
+#define MAX_BPFTOOL_CMD_LEN	(256)
+
+int run_bpftool_command(char *args);
+int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len);
Signature · 1 line
2.52.0
#2 lore raw

PATCH v3 2/3 selftests/bpf: convert test_bpftool_metadata.sh into test_progs framework

In reply to Alexis Lothoré (eBPF Foundation)

The test_bpftool_metadata.sh script validates that bpftool properly returns in its ouptput any metadata generated by bpf programs through some .rodata sections.

Port this test to the test_progs framework so that it can be executed automatically in CI. The new test, similarly to the former script, checks that valid data appears both for textual output and json output, as well as for both data not used at all and used data. For the json check part, the expected json string is hardcoded to avoid bringing a new external dependency (eg: a json deserializer) for test_progs. As the test is now converted into test_progs, remove the former script.

The newly converted test brings two new subtests:

  #37/1    bpftool_metadata/metadata_unused:OK
  #37/2    bpftool_metadata/metadata_used:OK
  #37      bpftool_metadata:OK

Summary: 1/2 PASSED, 0 SKIPPED, 0 FAILED

---
Changes in v2:
- move the new test in prog_tests directory
- use power of 2 for bpftool output buffer size
- check snprintf return when building commands
- target program by pin path rather than prog name
- remove a few blank lines

---
 tools/testing/selftests/bpf/Makefile               |   1 -
 .../selftests/bpf/prog_tests/bpftool_metadata.c    | 134 +++++++++++++++++++++
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -------------
 3 files changed, 134 insertions(+), 86 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -1
index 2cc559ecc723..1bb7db1ed6ea 100644
@@ -109,7 +109,6 @@ TEST_PROGS := test_kmod.sh \
 	test_bpftool_build.sh \
 	test_bpftool.sh \
 	test_bpftool_map.sh \
-	test_bpftool_metadata.sh \
 	test_doc_build.sh \
 	test_xsk.sh \
 	test_xdp_features.sh
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +134 -0

134 lines not on this page · raw message

tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -85

85 lines not on this page · raw message

Signature · 1 line
2.52.0
#3 lore raw

Re: [PATCH bpf-next v3 2/3] selftests/bpf: convert test_bpftool_metadata.sh into test_progs framework

The original shell script also verified the map listing output:
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +0 -0
new file mode 100644
index 000000000000..b5598741a01c

[ ... ]

Quoted 30 lines
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +30 -0
+static void run_test(struct test_desc *test)
+{
+	int ret;
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog load %s %s",
+			test->bpf_prog, test->bpffs_path);
+	if (!ASSERT_GT(ret, 0, "format prog insert command"))
+		return;
+	ret = run_bpftool_command(cmd);
+	if (!ASSERT_OK(ret, "load program"))
+		return;
+
+	/* Check output with default format */
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog show pinned %s",
+		       test->bpffs_path);
+	if (!ASSERT_GT(ret, 0, "format pinned prog check command"))
+		return;
+	ret = get_bpftool_command_output(cmd, output,
+			MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info")) {
+		ret = check_metadata(output, test->expected_output,
+				ARRAY_SIZE(test->expected_output));
+		ASSERT_OK(ret, "find metadata");
+	}
+
+	/* Check output with json format */
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog -j show pinned %s",
+		       test->bpffs_path);
+	if (!ASSERT_GT(ret, 0, "format pinned prog check command in json"))

tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c line 92

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +10 -0
+		return;
+	ret = get_bpftool_command_output(cmd, output,
+					 MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info in json")) {
+		ret = check_metadata(output, test->expected_output_json,
+				ARRAY_SIZE(test->expected_output_json));
+		ASSERT_OK(ret, "find metadata in json");
+	}
+
+}

The original shell script also verified the map listing output:

Quoted 5 lines
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -0
deleted file mode 100755
index b5520692f41b..000000000000

[ ... ]

Quoted 3 lines
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -3
-bpftool prog --json | grep '"metadata":{"a":"foo","b":1}' > /dev/null
-
-bpftool map | grep 'metadata.rodata' > /dev/null

[ ... ]

tools/testing/selftests/bpf/test_bpftool_metadata.sh line 81

Quoted 3 lines
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -3
-bpftool prog --json | grep '"metadata":{"a":"bar","b":2}' > /dev/null
-
-bpftool map | grep 'metadata.rodata' > /dev/null

Was the "bpftool map | grep 'metadata.rodata'" check intentionally removed from the converted test? The original script verified that the metadata map appeared in bpftool's map listing, which tests a different code path than the "prog show" commands. If this was dropped on purpose, it might be worth a comment explaining why; otherwise this appears to be a test coverage regression.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/21225032840

#4 lore raw

Re: [PATCH bpf-next v3 2/3] selftests/bpf: convert test_bpftool_metadata.sh into test_progs framework

Quoted 77 lines · bot+bpf-ci
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +0 -0
new file mode 100644
index 000000000000..b5598741a01c

[ ... ]

Quoted 30 lines
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +30 -0
+static void run_test(struct test_desc *test)
+{
+	int ret;
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog load %s %s",
+			test->bpf_prog, test->bpffs_path);
+	if (!ASSERT_GT(ret, 0, "format prog insert command"))
+		return;
+	ret = run_bpftool_command(cmd);
+	if (!ASSERT_OK(ret, "load program"))
+		return;
+
+	/* Check output with default format */
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog show pinned %s",
+		       test->bpffs_path);
+	if (!ASSERT_GT(ret, 0, "format pinned prog check command"))
+		return;
+	ret = get_bpftool_command_output(cmd, output,
+			MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info")) {
+		ret = check_metadata(output, test->expected_output,
+				ARRAY_SIZE(test->expected_output));
+		ASSERT_OK(ret, "find metadata");
+	}
+
+	/* Check output with json format */
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog -j show pinned %s",
+		       test->bpffs_path);
+	if (!ASSERT_GT(ret, 0, "format pinned prog check command in json"))

tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c line 92

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c new file +10 -0
+		return;
+	ret = get_bpftool_command_output(cmd, output,
+					 MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info in json")) {
+		ret = check_metadata(output, test->expected_output_json,
+				ARRAY_SIZE(test->expected_output_json));
+		ASSERT_OK(ret, "find metadata in json");
+	}
+
+}

The original shell script also verified the map listing output:

Quoted 5 lines
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -0
deleted file mode 100755
index b5520692f41b..000000000000

[ ... ]

Quoted 3 lines
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -3
-bpftool prog --json | grep '"metadata":{"a":"foo","b":1}' > /dev/null
-
-bpftool map | grep 'metadata.rodata' > /dev/null

[ ... ]

tools/testing/selftests/bpf/test_bpftool_metadata.sh line 81

Quoted 3 lines
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -3
-bpftool prog --json | grep '"metadata":{"a":"bar","b":2}' > /dev/null
-
-bpftool map | grep 'metadata.rodata' > /dev/null

Was the "bpftool map | grep 'metadata.rodata'" check intentionally removed from the converted test? The original script verified that the metadata map appeared in bpftool's map listing, which tests a different code path than the "prog show" commands. If this was dropped on purpose, it might be worth a comment explaining why; otherwise this appears to be a test coverage regression.

Quoted 5 lines

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/21225032840

The bot is right, I forgot to port this specific check. That will be added in the next revision.

Alexis

Signature · 3 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#5 lore raw

PATCH v3 3/3 selftests/bpf: convert test_bpftool_map_access.sh into test_progs framework

In reply to Alexis Lothoré (eBPF Foundation)

The test_bpftool_map.sh script tests that maps read/write accesses are being properly allowed/refused by the kernel depending on a specific fmod_ret program being attached on security_bpf_map function.

Rewrite this test to integrate it in the test_progs. The new test spawns a few subtests:

  #36/1    bpftool_maps_access/unprotected_unpinned:OK
  #36/2    bpftool_maps_access/unprotected_pinned:OK
  #36/3    bpftool_maps_access/protected_unpinned:OK
  #36/4    bpftool_maps_access/protected_pinned:OK
  #36/5    bpftool_maps_access/nested_maps:OK
  #36/6    bpftool_maps_access/btf_list:OK
  #36      bpftool_maps_access:OK

Summary: 1/6 PASSED, 0 SKIPPED, 0 FAILED

---
Changes in v2:
- move the new test in prog_tests directory
- collect Quentin's AB

---
 tools/testing/selftests/bpf/Makefile               |   1 -
 .../selftests/bpf/prog_tests/bpftool_maps_access.c | 371 +++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 3 files changed, 371 insertions(+), 399 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -1
index 1bb7db1ed6ea..2c2f68a171ed 100644
@@ -108,7 +108,6 @@ TEST_PROGS := test_kmod.sh \
 	test_xdping.sh \
 	test_bpftool_build.sh \
 	test_bpftool.sh \
-	test_bpftool_map.sh \
 	test_doc_build.sh \
 	test_xsk.sh \
 	test_xdp_features.sh
tools/testing/selftests/bpf/prog_tests/bpftool_maps_access.c new file +371 -0

371 lines not on this page · raw message

tools/testing/selftests/bpf/test_bpftool_map.sh deleted +0 -398

398 lines not on this page · raw message

Signature · 1 line
2.52.0