Skip to contentSkip to conversation

selftests/bpf: add a new runner for bpftool tests

PATCH Superseded Alexis Lothoré (eBPF Foundation) 4 participants 13 messages Jan 14 – Jan 16

Subsystems (3)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover selftests/bpf: add a new runner for bpftool tests

Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.

Some of those tests are validating bpftool behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...) and so they do not integrate well in test_progs. This series proposes to introduce a new runner to support those tests. This new runner is heavily inspired from test_progs, with slightly less features.

- First commit move the assert macros exposed by test_progs into a shared header so that they can be used by the new runner
- Second commit introduces the new runner, as well as a first test
- Third commit adds a second test
- Fourth commit drops the legacy scripts corresponding to those new tests

The series only adds the runner, but it is not enough to have it running in CI. The github actions need to be updated both in [1] and [2], and I am not sure how it si supposed to be contributed (the part needed in [1] must likely be integrated in the commit that is systematically brought on top of the bpf-next_base branch). Still, an example of test_bpftool run can be found in [3], in which I have forked and updated the needed Github actions ([4])

[1] https://github.com/kernel-patches/bpf.git
[2] https://github.com/libbpf/ci
[3] https://github.com/kernel-patches/bpf/pull/10711
[4] https://github.com/Tropicao/libbpf-ci

---
Alexis Lothoré (eBPF Foundation) (4):
bpf/selftests: move assert macros into a dedicated header bpf/selftests: introduce bptool test runner and a first test selftests/bpf: add bpftool map manipulations tests selftests/bpf: remove converted bpftool test scripts

 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  16 +-
 tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
 13 files changed, 1041 insertions(+), 711 deletions(-)

---
base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
change-id: 20251212-bpftool-tests-bb165c4cceb8

Best regards,

Signature · 1 line
Alexis Lothoré (eBPF Foundation) <[email protected]>
#1 lore raw

PATCH 1/4 bpf/selftests: move assert macros into a dedicated header

The test_progs runner defines a large set of convenient assert macros to perform all the tests. Writing a new runner involves rewriting some macros if we want some basic testing features like standardized failure log.

Export those assert macros from test_progs into a dedicated header so that we can use those in any test_runner. The sole requirement to be able to use those macros is to define a test__fail function in the runner that will be called whenever an assert fails.

---
 tools/testing/selftests/bpf/assert_helpers.h | 231 +++++++++++++++++++++++++++
 tools/testing/selftests/bpf/test_progs.h     | 226 +-------------------------
 2 files changed, 232 insertions(+), 225 deletions(-)
tools/testing/selftests/bpf/assert_helpers.h new file +231 -0
new file mode 100644
index 000000000000..93ab5bf39431
@@ -0,0 +1,231 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+#include <stdio.h>
+#include <errno.h>
+#include <stdbool.h>
+
+#define _CHECK(condition, tag, duration, format...) ({			\
+	int __ret = !!(condition);					\
+	int __save_errno = errno;					\
+	if (__ret) {							\
+		test__fail();						\
+		fprintf(stdout, "%s:FAIL:%s ", __func__, tag);		\
+		fprintf(stdout, ##format);				\
+	} else {							\
+		fprintf(stdout, "%s:PASS:%s %d nsec\n",			\
+		       __func__, tag, duration);			\
+	}								\
+	errno = __save_errno;						\
+	__ret;								\
+})
+
+#define CHECK_FAIL(condition) ({					\
+	int __ret = !!(condition);					\
+	int __save_errno = errno;					\
+	if (__ret) {							\
+		test__fail();						\
+		fprintf(stdout, "%s:FAIL:%d\n", __func__, __LINE__);	\
+	}								\
+	errno = __save_errno;						\
+	__ret;								\
+})
+
+#define CHECK(condition, tag, format...) \
+	_CHECK(condition, tag, duration, format)
+#define CHECK_ATTR(condition, tag, format...) \
+	_CHECK(condition, tag, tattr.duration, format)
+
+#define ASSERT_FAIL(fmt, args...) ({					\
+	static int duration;						\
+	CHECK(false, "", fmt"\n", ##args);				\
+	false;								\
+})
+
+#define ASSERT_TRUE(actual, name) ({					\
+	static int duration;						\
+	bool ___ok = (actual);						\
+	CHECK(!___ok, (name), "unexpected %s: got FALSE\n", (name));	\
+	___ok;								\
+})
+
+#define ASSERT_FALSE(actual, name) ({					\
+	static int duration;						\
+	bool ___ok = !(actual);						\
+	CHECK(!___ok, (name), "unexpected %s: got TRUE\n", (name));	\
+	___ok;								\
+})
+
+#define ASSERT_EQ(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act == ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld != expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_NEQ(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act != ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld == expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_LT(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act < ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld >= expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_LE(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act <= ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld > expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_GT(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act > ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld <= expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_GE(actual, expected, name) ({				\
+	static int duration;						\
+	typeof(actual) ___act = (actual);				\
+	typeof(expected) ___exp = (expected);				\
+	bool ___ok = ___act >= ___exp;					\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual %lld < expected %lld\n",		\
+	      (name), (long long)(___act), (long long)(___exp));	\
+	___ok;								\
+})
+
+#define ASSERT_STREQ(actual, expected, name) ({				\
+	static int duration;						\
+	const char *___act = actual;					\
+	const char *___exp = expected;					\
+	bool ___ok = strcmp(___act, ___exp) == 0;			\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual '%s' != expected '%s'\n",		\
+	      (name), ___act, ___exp);					\
+	___ok;								\
+})
+
+#define ASSERT_STRNEQ(actual, expected, len, name) ({			\
+	static int duration;						\
+	const char *___act = actual;					\
+	const char *___exp = expected;					\
+	int ___len = len;						\
+	bool ___ok = strncmp(___act, ___exp, ___len) == 0;		\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: actual '%.*s' != expected '%.*s'\n",	\
+	      (name), ___len, ___act, ___len, ___exp);			\
+	___ok;								\
+})
+
+#define ASSERT_HAS_SUBSTR(str, substr, name) ({				\
+	static int duration;						\
+	const char *___str = str;					\
+	const char *___substr = substr;					\
+	bool ___ok = strstr(___str, ___substr) != NULL;			\
+	CHECK(!___ok, (name),						\
+	      "unexpected %s: '%s' is not a substring of '%s'\n",	\
+	      (name), ___substr, ___str);				\
+	___ok;								\
+})
+
+#define ASSERT_MEMEQ(actual, expected, len, name) ({			\
+	static int duration;						\
+	const void *__act = actual;					\
+	const void *__exp = expected;					\
+	int __len = len;						\
+	bool ___ok = memcmp(__act, __exp, __len) == 0;			\
+	CHECK(!___ok, (name), "unexpected memory mismatch\n");		\
+	fprintf(stdout, "actual:\n");					\
+	hexdump("\t", __act, __len);					\
+	fprintf(stdout, "expected:\n");					\
+	hexdump("\t", __exp, __len);					\
+	___ok;								\
+})
+
+#define ASSERT_OK(res, name) ({						\
+	static int duration;						\
+	long long ___res = (res);					\
+	bool ___ok = ___res == 0;					\
+	CHECK(!___ok, (name), "unexpected error: %lld (errno %d)\n",	\
+	      ___res, errno);						\
+	___ok;								\
+})
+
+#define ASSERT_ERR(res, name) ({					\
+	static int duration;						\
+	long long ___res = (res);					\
+	bool ___ok = ___res < 0;					\
+	CHECK(!___ok, (name), "unexpected success: %lld\n", ___res);	\
+	___ok;								\
+})
+
+#define ASSERT_NULL(ptr, name) ({					\
+	static int duration;						\
+	const void *___res = (ptr);					\
+	bool ___ok = !___res;						\
+	CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res);	\
+	___ok;								\
+})
+
+#define ASSERT_OK_PTR(ptr, name) ({					\
+	static int duration;						\
+	const void *___res = (ptr);					\
+	int ___err = libbpf_get_error(___res);				\
+	bool ___ok = ___err == 0;					\
+	CHECK(!___ok, (name), "unexpected error: %d\n", ___err);	\
+	___ok;								\
+})
+
+#define ASSERT_ERR_PTR(ptr, name) ({					\
+	static int duration;						\
+	const void *___res = (ptr);					\
+	int ___err = libbpf_get_error(___res);				\
+	bool ___ok = ___err != 0;					\
+	CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res);	\
+	___ok;								\
+})
+
+#define ASSERT_OK_FD(fd, name) ({					\
+	static int duration;						\
+	int ___fd = (fd);						\
+	bool ___ok = ___fd >= 0;					\
+	CHECK(!___ok, (name), "unexpected fd: %d (errno %d)\n",		\
+	      ___fd, errno);						\
+	___ok;								\
+})
+
+#define ASSERT_ERR_FD(fd, name) ({					\
+	static int duration;						\
+	int ___fd = (fd);						\
+	bool ___ok = ___fd < 0;						\
+	CHECK(!___ok, (name), "unexpected fd: %d\n", ___fd);		\
+	___ok;								\
+})
+
tools/testing/selftests/bpf/test_progs.h +1 -225

238 lines not on this page · raw message

Signature · 1 line
2.52.0
Acked-by on v1 1/4
#2 lore raw

Re: [PATCH bpf-next 1/4] bpf/selftests: move assert macros into a dedicated header

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>

Quoted 11 lines

The test_progs runner defines a large set of convenient assert macros to perform all the tests. Writing a new runner involves rewriting some macros if we want some basic testing features like standardized failure log.

Export those assert macros from test_progs into a dedicated header so that we can use those in any test_runner. The sole requirement to be able to use those macros is to define a test__fail function in the runner that will be called whenever an assert fails.

#3 lore raw

PATCH 2/4 bpf/selftests: introduce bptool test runner and a first test

In reply to Alexis Lothoré (eBPF Foundation)

The tools/testing/selftests/bpf directory contains multiple scripts (shell, python, c code, etc) that aim to test some specific features from bpftool. Those isolated tests are currently not executed by any CI automation. Create a dedicated runner for any bpftool-related test that can then be added to the list of executed runners in bpf CI automation. This new runner (and the corresponding Makefile tooling) is highly inspired from test_progs, but kept a bit simpler. This version supports the following features:

- autodetection of bpftool test stored in the in bpftool_tests directory
- bpftool binary under test is passed as runner argument
- a few helpers to allow to easily run abpftool commands while possibly collecting the output
- usage of assert macros shared with test_progs
- basic sub-tests management
- logs collection, logs being dumped only for failed tests
- exit code reflecting whether all tests have passed or not

As this runner needs at least one test to be implemented to properly compile, also bring bpftool_metadata, which is the conversion of test_bpftool_metadata.sh: this test validates that the output of some basic prog/map listings done with bpftool properly returns the metadata collected from the .rodata section of eBPF programs.

This new runner gives an output similar to the one generated by test_progs:

#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 1 PASSED, 0 FAILED

---
 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  14 ++-
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +++
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 ++++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++++++
 8 files changed, 439 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/.gitignore +1 -0
index b8bf51b7a0b0..9498cc11de97 100644
@@ -2,6 +2,7 @@
 bpftool
 bpf-helpers*
 bpf-syscall*
+test_bpftool
 test_verifier
 test_maps
 test_lru_map
tools/testing/selftests/bpf/Makefile +13 -1
index fd42b7193d4e..a1fe94efa53c 100644
@@ -76,7 +76,8 @@ endif
 TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_progs \
 	test_sockmap \
 	test_tcpnotify_user \
-	test_progs-no_alu32
+	test_progs-no_alu32 \
+	test_bpftool
 TEST_INST_SUBDIRS := no_alu32
 
 # Also test bpf-gcc, if present
@@ -791,6 +792,17 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built)
 TRUNNER_BPF_CFLAGS :=
 $(eval $(call DEFINE_TEST_RUNNER,test_maps))
 
+# Define bpftool test runner.
+TRUNNER_TESTS_DIR := bpftool_tests
+TRUNNER_BPF_PROGS_DIR := progs
+TRUNNER_EXTRA_SOURCES := test_bpftool.c \
+			 bpftool_helpers.c
+TRUNNER_LIB_SOURCES :=
+TRUNNER_EXTRA_FILES :=
+TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
+TRUNNER_BPF_CFLAGS :=
+$(eval $(call DEFINE_TEST_RUNNER,test_bpftool))
+
 # Define test_verifier test runner.
 # It is much simpler than test_maps/test_progs and sufficiently different from
 # them (e.g., test.h is using completely pattern), that it's worth just
tools/testing/selftests/bpf/bpftool_helpers.c new file +114 -0

114 lines not on this page · raw message

tools/testing/selftests/bpf/bpftool_helpers.h new file +19 -0
new file mode 100644
index 000000000000..1eacec7936ba
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdbool.h>
+
+#define MAX_BPFTOOL_CMD_LEN	(256)
+
+#ifndef ARRAY_SIZE
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#endif
+
+int run_bpftool_command(char *args);
+int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len);
+void test__start_subtest(const char *subtests_name);
+void test__end_subtest(void);
+void hijack_stdio(void);
+void restore_stdio(void);
tools/testing/selftests/bpf/bpftool_tests/.gitignore new file +2 -0
new file mode 100644
index 000000000000..89c4a3d37544
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+tests.h
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +128 -0

128 lines not on this page · raw message

tools/testing/selftests/bpf/test_bpftool.c new file +126 -0

126 lines not on this page · raw message

tools/testing/selftests/bpf/test_bpftool.h new file +36 -0
new file mode 100644
index 000000000000..a78659eeaf2b
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdio.h>
+#include <stdbool.h>
+
+extern struct bpftool_runner_env env;
+
+void test__fail(void);
+
+struct test_state {
+	char *name;
+	char *log;
+	size_t log_size;
+	bool failed;
+	int subtests_count;
+	int subtests_failures;
+	FILE *saved_stdout;
+	FILE *saved_stderr;
+};
+
+struct subtest_state {
+	char *name;
+	char *log;
+	size_t log_size;
+	bool failed;
+};
+struct bpftool_runner_env {
+	char *bpftool_path;
+	int failure_cnt;
+	FILE *saved_stdout;
+	FILE *saved_stderr;
+	struct test_state *current_test;
+	struct subtest_state *current_subtest;
+	struct subtest_state *subtest_states;
+};
Signature · 1 line
2.52.0
#4 lore raw

Re: [PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>

Quoted 10 lines

The tools/testing/selftests/bpf directory contains multiple scripts (shell, python, c code, etc) that aim to test some specific features from bpftool. Those isolated tests are currently not executed by any CI automation. Create a dedicated runner for any bpftool-related test that can then be added to the list of executed runners in bpf CI automation. This new runner (and the corresponding Makefile tooling) is highly inspired from test_progs, but kept a bit simpler. This version supports the following features:

- autodetection of bpftool test stored in the in bpftool_tests

(Typo: "in the in")

Quoted 3 lines

directory
- bpftool binary under test is passed as runner argument
- a few helpers to allow to easily run abpftool commands while possibly

(Typo: abpftool)

Quoted 15 lines

collecting the output
- usage of assert macros shared with test_progs
- basic sub-tests management
- logs collection, logs being dumped only for failed tests
- exit code reflecting whether all tests have passed or not

As this runner needs at least one test to be implemented to properly compile, also bring bpftool_metadata, which is the conversion of test_bpftool_metadata.sh: this test validates that the output of some basic prog/map listings done with bpftool properly returns the metadata collected from the .rodata section of eBPF programs.

This new runner gives an output similar to the one generated by test_progs:

Quoted 6 lines

#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 1 PASSED, 0 FAILED

Thanks a lot for this! It looks good, I only have a few minor comments.

Quoted 118 lines
---
 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  14 ++-
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +++
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 ++++++++++++++++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++++++
 8 files changed, 439 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/.gitignore +1 -0
index b8bf51b7a0b0..9498cc11de97 100644
@@ -2,6 +2,7 @@
 bpftool
 bpf-helpers*
 bpf-syscall*
+test_bpftool
 test_verifier
 test_maps
 test_lru_map
tools/testing/selftests/bpf/Makefile +13 -1
index fd42b7193d4e..a1fe94efa53c 100644
@@ -76,7 +76,8 @@ endif
 TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_progs \
 	test_sockmap \
 	test_tcpnotify_user \
-	test_progs-no_alu32
+	test_progs-no_alu32 \
+	test_bpftool
 TEST_INST_SUBDIRS := no_alu32
 
 # Also test bpf-gcc, if present
@@ -791,6 +792,17 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built)
 TRUNNER_BPF_CFLAGS :=
 $(eval $(call DEFINE_TEST_RUNNER,test_maps))
 
+# Define bpftool test runner.
+TRUNNER_TESTS_DIR := bpftool_tests
+TRUNNER_BPF_PROGS_DIR := progs
+TRUNNER_EXTRA_SOURCES := test_bpftool.c \
+			 bpftool_helpers.c
+TRUNNER_LIB_SOURCES :=
+TRUNNER_EXTRA_FILES :=
+TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
+TRUNNER_BPF_CFLAGS :=
+$(eval $(call DEFINE_TEST_RUNNER,test_bpftool))
+
 # Define test_verifier test runner.
 # It is much simpler than test_maps/test_progs and sufficiently different from
 # them (e.g., test.h is using completely pattern), that it's worth just
tools/testing/selftests/bpf/bpftool_helpers.c new file +57 -0
new file mode 100644
index 000000000000..ff8084d9a121
@@ -0,0 +1,114 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include "bpftool_helpers.h"
+#include "test_bpftool.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdbool.h>
+
+#define BPFTOOL_PATH		"./tools/sbin/bpftool"
+#define BPFTOOL_CMD_MAX_LEN	256
+
+static int run_command(char *command, bool get_output, char *output_buf, size_t output_max_len)
+{
+	FILE *f;
+	int ret;
+
+	f = popen(command, "r");
+	if (!f)
+		return 1;
+
+	if (get_output)
+		fread(output_buf, 1, output_max_len, f);
+	ret = pclose(f);
+
+	return ret;
+}
+
+int run_bpftool_command(char *args)
+{
+	char cmd[BPFTOOL_CMD_MAX_LEN];
+	int ret;
+
+	ret = snprintf(cmd, BPFTOOL_CMD_MAX_LEN, "%s %s > /dev/null 2>&1",
+		       env.bpftool_path, args);
+	if (ret !=
+	    strlen(env.bpftool_path) + 1 + strlen(args) + strlen(" > /dev/null 2>&1")) {
+		fprintf(stderr, "Failed to generate bpftool command\n");
+		return 1;
+	}
+
+	return run_command(cmd, false, NULL, 0);
+}
+
+int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len)
+{
+	int ret;
+	char cmd[BPFTOOL_CMD_MAX_LEN];
+
+	ret = snprintf(cmd, BPFTOOL_CMD_MAX_LEN, "%s %s", env.bpftool_path,
+		       args);
+	if (ret != strlen(args) + strlen(env.bpftool_path) + 1) {
+		fprintf(stderr, "Failed to generate bpftool command");
+		return 1;
+	}
+
+	return run_command(cmd, true, output_buf, output_max_len);
+}

tools/testing/selftests/bpf/bpftool_helpers.c line 67

Quoted 10 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +10 -0
+
+void hijack_stdio(void)
+{
+	fflush(stdout);
+	fflush(stderr);
+	if (env.current_subtest) {
+		env.current_test->saved_stdout = stdout;
+		env.current_test->saved_stderr = stderr;
+		stdout = open_memstream(&env.current_subtest->log,
+					&env.current_subtest->log_size);

Missing check on open_memstream() return value? Same below.

tools/testing/selftests/bpf/bpftool_helpers.c line 68

Quoted 1 line
tools/testing/selftests/bpf/bpftool_helpers.c new file +1 -0
+

Nit: blank line

Quoted 6 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +6 -0
+	} else {
+		env.saved_stdout = stdout;
+		env.saved_stderr = stderr;
+		stdout = open_memstream(&env.current_test->log,
+					&env.current_test->log_size);
+	}

tools/testing/selftests/bpf/bpftool_helpers.c line 84

Quoted 10 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +10 -0
+	stderr = stdout;
+}
+
+void restore_stdio(void)
+{
+	fclose(stdout);
+	if (env.current_subtest) {
+		stdout = env.current_test->saved_stdout;
+		stderr = env.current_test->saved_stderr;
+

Nit: blank line

tools/testing/selftests/bpf/bpftool_helpers.c line 89

Quoted 5 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +5 -0
+	} else {
+		stdout = env.saved_stdout;
+		stderr = env.saved_stderr;
+	}
+

Nit: blank line again

tools/testing/selftests/bpf/bpftool_helpers.c line 98

Quoted 9 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +9 -0
+}
+
+void test__start_subtest(const char *subtest_name)
+{
+	test__end_subtest();
+	env.current_test->subtests_count++;
+	env.subtest_states = realloc(env.subtest_states,
+				     env.current_test->subtests_count *
+					     sizeof(struct subtest_state));

Missing check on realloc()'s return value?

Quoted 64 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +16 -0
+	env.current_subtest =
+		&env.subtest_states[env.current_test->subtests_count - 1];
+	memset(env.current_subtest, 0, sizeof(struct subtest_state));
+	env.current_subtest->name = strdup(subtest_name);
+
+	hijack_stdio();
+}
+
+void test__end_subtest(void)
+{
+	if (env.current_subtest) {
+		restore_stdio();
+		env.current_subtest = NULL;
+	}
+}
+
tools/testing/selftests/bpf/bpftool_helpers.h new file +19 -0
new file mode 100644
index 000000000000..1eacec7936ba
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdbool.h>
+
+#define MAX_BPFTOOL_CMD_LEN	(256)
+
+#ifndef ARRAY_SIZE
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+#endif
+
+int run_bpftool_command(char *args);
+int get_bpftool_command_output(char *args, char *output_buf, size_t output_max_len);
+void test__start_subtest(const char *subtests_name);
+void test__end_subtest(void);
+void hijack_stdio(void);
+void restore_stdio(void);
tools/testing/selftests/bpf/bpftool_tests/.gitignore new file +2 -0
new file mode 100644
index 000000000000..89c4a3d37544
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+tests.h
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +9 -0
new file mode 100644
index 000000000000..e7146b26f298
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <bpftool_helpers.h>
+#include <test_bpftool.h>
+#include <assert_helpers.h>
+#include <linux/bpf.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <sys/stat.h>

tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 19

Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10 -0
+#include <stdbool.h>
+
+#define BPFFS_DIR	"/sys/fs/bpf/test_metadata"
+#define BPFFS_USED	BPFFS_DIR "/used"
+#define BPFFS_UNUSED	BPFFS_DIR "/unused"
+
+#define BPF_FILE_USED	"metadata_used.bpf.o"
+#define BPF_FILE_UNUSED "metadata_unused.bpf.o"
+
+#define MAX_BPFTOOL_OUTPUT_LEN	(100*1000)

Maybe a power of 2 instead, 64k?

Quoted 37 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +37 -0
+
+#define MAX_TOKENS_TO_CHECK	3
+static char output[MAX_BPFTOOL_OUTPUT_LEN];
+
+struct test_desc {
+	char *name;
+	char *bpf_prog;
+	char *bpffs_path;
+	char *expected_output[MAX_TOKENS_TO_CHECK];
+	char *expected_output_json[MAX_TOKENS_TO_CHECK];
+};
+
+static int setup(struct test_desc *test)
+{
+	return mkdir(BPFFS_DIR, 0700);
+}
+
+static void cleanup(struct test_desc *test)
+{
+	unlink(test->bpffs_path);
+	rmdir(BPFFS_DIR);
+}
+
+static int check_metadata(char *buf, char * const *tokens, int count)
+{
+	int i;
+
+	for (i = 0; i < count && tokens[i]; i++)
+		if (!strstr(buf, tokens[i]))
+			return 1;
+
+	return 0;
+}
+
+static void run_test(struct test_desc *test)
+{
+	int ret;

tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 66

Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10 -0
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+
+	snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "prog load %s %s",
+			test->bpf_prog, test->bpffs_path);
+	ret = run_bpftool_command(cmd);
+	if (!ASSERT_OK(ret, "load program"))
+		return;
+
+	/* Check output with default format */
+	ret = get_bpftool_command_output("prog show name prog", output,

I'd probably use "pinned" + pinned path rather than "name prog", especially because the name for the program does not otherwise appear in this file (and same below).

Quoted 6 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +6 -0
+			MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info")) {
+		ret = check_metadata(output, test->expected_output,
+				ARRAY_SIZE(test->expected_output));
+		ASSERT_OK(ret, "find metadata");
+	}

tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 82

Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10 -0
+
+	/* Check output with json format */
+	ret = get_bpftool_command_output("prog -j show name prog", output,
+					 MAX_BPFTOOL_OUTPUT_LEN);
+	if (ASSERT_OK(ret, "get program info in json")) {
+		ret = check_metadata(output, test->expected_output_json,
+				ARRAY_SIZE(test->expected_output_json));
+		ASSERT_OK(ret, "find metadata in json");
+	}
+

Nit: blank line

Quoted 34 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +34 -0
+}
+
+struct test_desc tests[] = {
+	{
+		.name = "metadata_unused",
+		.bpf_prog = BPF_FILE_UNUSED,
+		.bpffs_path = BPFFS_UNUSED,
+		.expected_output = {
+			"a = \"foo\"",
+			"b = 1"
+		},
+		.expected_output_json = {
+			"\"metadata\":{\"a\":\"foo\",\"b\":1}"
+		}
+	},
+	{
+		.name = "metadata_used",
+		.bpf_prog = BPF_FILE_USED,
+		.bpffs_path = BPFFS_USED,
+		.expected_output = {
+			"a = \"bar\"",
+			"b = 2"
+		},
+		.expected_output_json = {
+			"\"metadata\":{\"a\":\"bar\",\"b\":2}"
+		}
+	}
+};
+
+static const int tests_count = ARRAY_SIZE(tests);
+
+void test_metadata(void)
+{
+	int i, ret;

tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 126

Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10 -0
+
+	for (i = 0; i < tests_count; i++) {
+		test__start_subtest(tests[i].name);
+		ret = setup(&tests[i]);
+		if (!ASSERT_OK(ret, "setup bpffs pin dir"))
+			continue;
+		run_test(&tests[i]);
+		cleanup(&tests[i]);
+	}
+

Nit: blank line

Quoted 47 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +2 -0
+}
+
tools/testing/selftests/bpf/test_bpftool.c new file +39 -0
new file mode 100644
index 000000000000..b5fb17d5ea2d
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <test_bpftool.h>
+#include <bpftool_helpers.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdbool.h>
+#include <string.h>
+#include <unistd.h>
+
+struct bpftool_runner_env env = {0};
+
+#define DEFINE_TEST(name) extern void test_##name(void);
+#include <bpftool_tests/tests.h>
+#undef DEFINE_TEST
+
+struct prog_test_def {
+	char *test_name;
+	void (*run_test)(void);
+};
+
+static struct prog_test_def prog_test_defs[] = {
+#define DEFINE_TEST(name) {			\
+	.test_name = #name,			\
+	.run_test = &test_##name,		\
+},
+#include <bpftool_tests/tests.h>
+#undef DEFINE_TEST
+};
+
+
+static const int tests_count = ARRAY_SIZE(prog_test_defs);
+
+/* Needed method for the assert macros exposed by assert_helpers.h */
+void test__fail(void)
+{
+	if (env.current_subtest)
+		env.current_subtest->failed = true;
+	if (!env.current_test->failed)
+		env.failure_cnt++;

tools/testing/selftests/bpf/test_bpftool.c line 49

Quoted 10 lines
tools/testing/selftests/bpf/test_bpftool.c new file +10 -0
+	env.current_test->failed = true;
+}
+
+static void test_setup(struct test_state *test, char *name)
+{
+	env.current_test = test;
+	env.current_test->name = strdup(name);
+}
+
+static void dump_results(struct test_state *test, int test_index)

Would you mind renaming this function? It also frees the logs and names and subtests states. Maybe "dump_results_and_cleanup"? Or move the all the clean-up to test_teardown(). Even if we loop again over the subtests, the overhead should be minimal, and I'd find it easier to follow.

tools/testing/selftests/bpf/test_bpftool.c line 51

Quoted 2 lines
tools/testing/selftests/bpf/test_bpftool.c new file +2 -0
+{
+	int j;

Let's use "i" rather than "j"

Quoted 54 lines
tools/testing/selftests/bpf/test_bpftool.c new file +54 -0
+
+	if (test->failed)
+		fprintf(stdout, "%s\n", test->log);
+	free(test->log);
+	for (j = 0; j < test->subtests_count; j++) {
+		if (env.subtest_states[j].failed)
+			fprintf(stdout, "%s\n", env.subtest_states[j].log);
+		free(env.subtest_states[j].log);
+		fprintf(stdout, "#%d/%d\t%s/%s: %s\n", test_index+1, j+1,
+				env.current_test->name,
+				env.subtest_states[j].name,
+				env.subtest_states[j].failed ? "KO" : "OK");
+		free(env.subtest_states[j].name);
+	}
+	if (env.current_test->subtests_count) {
+		free(env.subtest_states);
+		env.subtest_states = NULL;
+	}
+	fprintf(stdout, "#%d\t%s: %s\n", test_index + 1, test->name,
+		test->failed ? "KO" : "OK");
+}
+
+static void test_teardown(struct test_state *test, int test_index)
+{
+	dump_results(test, test_index);
+	free(env.current_test->name);
+	env.current_test = NULL;
+}
+
+static int parse_args(int argc, char *argv[])
+{
+	if (argc != 2)
+		return 1;
+	if (access(argv[1], R_OK|X_OK))
+		return 1;
+	env.bpftool_path = argv[1];
+
+	return 0;
+}
+
+static void usage(char *prog)
+{
+	fprintf(stdout, "Usage: %s <bpftool_path>\n", prog);
+	fprintf(stdout, "\t<bpftool_path>: path to the bpftool binary to test\n");
+}
+
+int main(int argc, char *argv[])
+{
+	struct test_state *ctx = NULL;
+	int i;
+
+	if (parse_args(argc, argv)) {
+		fprintf(stderr, "Invalid arguments\n");
+		usage(argv[0]);

tools/testing/selftests/bpf/test_bpftool.c line 115

Quoted 10 lines
tools/testing/selftests/bpf/test_bpftool.c new file +10 -0
+		exit(EXIT_FAILURE);
+	}
+
+	ctx = calloc(tests_count, sizeof(struct test_state));
+	if (!ctx)
+		exit(EXIT_FAILURE);
+
+	for (i = 0; i < tests_count; i++) {
+		test_setup(&ctx[i], prog_test_defs[i].test_name);
+		hijack_stdio();

Nit: Move hijack_stdio()/restore_stdio() to test_setup()/test_teardown()?

Quoted 54 lines
tools/testing/selftests/bpf/test_bpftool.c new file +11 -0
+		prog_test_defs[i].run_test();
+		test__end_subtest();
+		restore_stdio();
+		test_teardown(&ctx[i], i);
+	}
+
+	fprintf(stdout, "Summary: %d PASSED, %d FAILED\n",
+		tests_count - env.failure_cnt, env.failure_cnt);
+	free(ctx);
+	return env.failure_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
+}
tools/testing/selftests/bpf/test_bpftool.h new file +36 -0
new file mode 100644
index 000000000000..a78659eeaf2b
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#pragma once
+
+#include <stdio.h>
+#include <stdbool.h>
+
+extern struct bpftool_runner_env env;
+
+void test__fail(void);
+
+struct test_state {
+	char *name;
+	char *log;
+	size_t log_size;
+	bool failed;
+	int subtests_count;
+	int subtests_failures;
+	FILE *saved_stdout;
+	FILE *saved_stderr;
+};
+
+struct subtest_state {
+	char *name;
+	char *log;
+	size_t log_size;
+	bool failed;
+};
+struct bpftool_runner_env {
+	char *bpftool_path;
+	int failure_cnt;
+	FILE *saved_stdout;
+	FILE *saved_stderr;
+	struct test_state *current_test;
+	struct subtest_state *current_subtest;
+	struct subtest_state *subtest_states;
+};
#5 lore raw

Re: [PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test

Hi Quentin, thanks for the review !

I agree with all your comments. Andrii raised some concerns about bringing a new runner, I'll wait for this new runner question to be settled before bringing those changes, as it could make this series change quite a lot.

Thanks,

Alexis

Signature · 3 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

PATCH 3/4 selftests/bpf: add bpftool map manipulations tests

In reply to Alexis Lothoré (eBPF Foundation)

The test_bpftool_map.sh script tests that maps read/write accesses are being properly allowed/refused by the kernel depending on a specific fmod_ret program being attached on security_bpf_map function.

Rewrite this test to integrate it in the new test_bpftool runner. The new test spawns a few subtests:

#1/1 maps_access/unprotected_unpinned: OK
#1/2 maps_access/unprotected_pinned: OK
#1/3 maps_access/protected_unpinned: OK
#1/4 maps_access/protected_pinned: OK
#1/5 maps_access/nested_maps: OK
#1/6 maps_access/btf_list: OK
#1 maps_access: OK
#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 2 PASSED, 0 FAILED

---
 .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++++
 1 file changed, 370 insertions(+)
tools/testing/selftests/bpf/bpftool_tests/bpftool_maps_access.c new file +370 -0
new file mode 100644
index 000000000000..da4277c79e22
@@ -0,0 +1,370 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <bpf/libbpf.h>
+#include <bpftool_helpers.h>
+#include <stdlib.h>
+#include <test_bpftool.h>
+#include <assert_helpers.h>
+#include <linux/bpf.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <stdint.h>
+#include <sys/stat.h>
+#include <stdbool.h>
+#include <bpf/bpf.h>
+#include "security_bpf_map.skel.h"
+
+#define PROTECTED_MAP_NAME	"prot_map"
+#define UNPROTECTED_MAP_NAME	"not_prot_map"
+#define BPF_ITER_FILE		"bpf_iter_map_elem.bpf.o"
+#define BPFFS_PIN_DIR		"/sys/fs/bpf/test_bpftool_map"
+#define INNER_MAP_NAME		"inner_map_tt"
+#define OUTER_MAP_NAME		"outer_map_tt"
+
+#define MAP_NAME_MAX_LEN	64
+#define PATH_MAX_LEN		128
+
+enum map_protection {
+	PROTECTED,
+	UNPROTECTED
+};
+
+struct test_desc {
+	char *name;
+	enum map_protection protection;
+	struct bpf_map *map;
+	char *map_name;
+	bool pinned;
+	char pin_path[PATH_MAX_LEN];
+	bool write_must_fail;
+};
+
+static struct security_bpf_map *general_setup(void)
+{
+	struct security_bpf_map *skel;
+	uint32_t key, value;
+	int ret, i;
+
+	skel = security_bpf_map__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "open and load skeleton"))
+		goto end;
+
+	struct bpf_map *maps[] = {skel->maps.prot_map, skel->maps.not_prot_map};
+
+	ret = security_bpf_map__attach(skel);
+	if (!ASSERT_OK(ret, "attach maps security programs"))
+		goto end_destroy;
+
+	for (i = 0; i < sizeof(maps)/sizeof(struct bpf_map *); i++) {
+		for (key = 0; key < 2; key++) {
+			int ret = bpf_map__update_elem(maps[i], &key,
+					sizeof(key), &key, sizeof(key),
+					0);
+			if (!ASSERT_OK(ret, "set initial map value"))
+				goto end_destroy;
+		}
+	}
+
+	key = 0;
+	value = 1;
+	ret = bpf_map__update_elem(skel->maps.prot_status_map, &key,
+			sizeof(key), &value, sizeof(value), 0);
+	if (!ASSERT_OK(ret, "configure map protection"))
+		goto end_destroy;
+
+	if (!ASSERT_OK(mkdir(BPFFS_PIN_DIR, S_IFDIR), "create bpffs pin dir"))
+		goto end_destroy;
+
+	return skel;
+end_destroy:
+	security_bpf_map__destroy(skel);
+end:
+	return NULL;
+}
+
+static void general_cleanup(struct security_bpf_map *skel)
+{
+	rmdir(BPFFS_PIN_DIR);
+	security_bpf_map__destroy(skel);
+}
+
+static void update_test_desc(struct security_bpf_map *skel,
+			      struct test_desc *test)
+{
+	/* Now that the skeleton is loaded, update all missing fields to
+	 * have the subtest properly configured
+	 */
+	if (test->protection == PROTECTED) {
+		test->map = skel->maps.prot_map;
+		test->map_name = PROTECTED_MAP_NAME;
+	} else {
+		test->map = skel->maps.not_prot_map;
+		test->map_name = UNPROTECTED_MAP_NAME;
+	}
+}
+
+static int test_setup(struct security_bpf_map *skel, struct test_desc *desc)
+{
+	int ret;
+
+	update_test_desc(skel, desc);
+
+	if (desc->pinned) {
+		ret = snprintf(desc->pin_path, PATH_MAX_LEN, "%s/%s", BPFFS_PIN_DIR,
+				desc->name);
+		if (!ASSERT_GT(ret, 0, "format pin path"))
+			return 1;
+		ret = bpf_map__pin(desc->map, desc->pin_path);
+		if (!ASSERT_OK(ret, "pin map"))
+			return 1;
+	}
+
+	return 0;
+}
+
+static void test_cleanup(struct test_desc *desc)
+{
+	if (desc->pinned)
+		bpf_map__unpin(desc->map, NULL);
+}
+
+static int lookup_map_value(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "map lookup %s key 0 0 0 0",
+			map_handle);
+	if (!ASSERT_GT(ret, 0, "format map lookup cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int read_map_btf_data(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "btf dump map %s",
+			map_handle);
+	if (!ASSERT_GT(ret, 0, "format map btf dump cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int write_map_value(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN,
+		       "map update %s key 0 0 0 0 value 1 1 1 1", map_handle);
+	if (!ASSERT_GT(ret, 0, "format value write cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int delete_map_value(char *map_handle)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN,
+		       "map delete %s key 0 0 0 0", map_handle);
+	if (!ASSERT_GT(ret, 0, "format value deletion cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int iterate_on_map_values(char *map_handle, char *iter_pin_path)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+
+	ret = snprintf(cmd, MAX_BPFTOOL_CMD_LEN, "iter pin %s %s map %s",
+		       BPF_ITER_FILE, iter_pin_path, map_handle);
+	if (!ASSERT_GT(ret, 0, "format iterator cration cmd"))
+		return 1;
+	ret = run_bpftool_command(cmd);
+	if (ret)
+		return ret;
+	ret = snprintf(cmd, MAP_NAME_MAX_LEN, "cat %s", iter_pin_path);
+	if (ret < 0)
+		goto cleanup;
+	ret = system(cmd);
+
+cleanup:
+	unlink(iter_pin_path);
+	return ret;
+}
+
+static int create_inner_map(void)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(
+		cmd, MAX_BPFTOOL_CMD_LEN,
+		"map create %s/%s type array key 4 value 4 entries 4 name %s",
+		BPFFS_PIN_DIR, INNER_MAP_NAME, INNER_MAP_NAME);
+	if (!ASSERT_GT(ret, 0, "format inner map create cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static int create_outer_map(void)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(
+		cmd, MAX_BPFTOOL_CMD_LEN,
+		"map create %s/%s type hash_of_maps key 4 value 4 entries 2 name %s inner_map name %s",
+		BPFFS_PIN_DIR, OUTER_MAP_NAME, OUTER_MAP_NAME, INNER_MAP_NAME);
+	if (!ASSERT_GT(ret, 0, "format outer map create cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static void delete_pinned_map(char *map_name)
+{
+	char pin_path[PATH_MAX_LEN];
+	int ret;
+
+	ret = snprintf(pin_path, PATH_MAX_LEN, "%s/%s", BPFFS_PIN_DIR,
+		       map_name);
+	if (ret >= 0)
+		unlink(pin_path);
+}
+
+static int add_outer_map_entry(int key)
+{
+	char cmd[MAX_BPFTOOL_CMD_LEN];
+	int ret = 0;
+
+	ret = snprintf(
+		cmd, MAX_BPFTOOL_CMD_LEN,
+		"map update pinned %s/%s key %d 0 0 0 value name %s",
+		BPFFS_PIN_DIR, OUTER_MAP_NAME, key, INNER_MAP_NAME);
+	if (!ASSERT_GT(ret, 0, "format outer map value addition cmd"))
+		return 1;
+	return run_bpftool_command(cmd);
+}
+
+static void test_basic_access(struct test_desc *desc)
+{
+	char map_handle[MAP_NAME_MAX_LEN];
+	char iter_pin_path[PATH_MAX_LEN];
+	int ret;
+
+	if (desc->pinned)
+		ret = snprintf(map_handle, MAP_NAME_MAX_LEN, "pinned %s",
+			       desc->pin_path);
+	else
+		ret = snprintf(map_handle, MAP_NAME_MAX_LEN, "name %s",
+			       desc->map_name);
+	if (!ASSERT_GT(ret, 0, "format map handle"))
+		return;
+
+	ret = lookup_map_value(map_handle);
+	ASSERT_OK(ret, "read map value");
+
+	ret = read_map_btf_data(map_handle);
+	ASSERT_OK(ret, "read map btf data");
+
+	ret = write_map_value(map_handle);
+	ASSERT_OK(desc->write_must_fail ? !ret : ret, "write map value");
+
+	ret = delete_map_value(map_handle);
+	ASSERT_OK(desc->write_must_fail ? !ret : ret, "delete map value");
+	/* Restore deleted value */
+	if (!ret)
+		write_map_value(map_handle);
+
+	ret = snprintf(iter_pin_path, PATH_MAX_LEN, "%s/iter", BPFFS_PIN_DIR);
+	if (ASSERT_GT(ret, 0, "format iter pin path")) {
+		ret = iterate_on_map_values(map_handle, iter_pin_path);
+		ASSERT_OK(ret, "iterate on map values");
+	}
+}
+
+static void test_create_nested_maps(void)
+{
+	if (!ASSERT_OK(create_inner_map(), "create inner map"))
+		return;
+	if (!ASSERT_OK(create_outer_map(), "create outer map"))
+		goto end_cleanup_inner;
+	ASSERT_OK(add_outer_map_entry(0), "add a first entry in outer map");
+	ASSERT_OK(add_outer_map_entry(1), "add a second entry in outer map");
+	ASSERT_NEQ(add_outer_map_entry(2), 0, "add a third entry in outer map");
+
+	delete_pinned_map(OUTER_MAP_NAME);
+end_cleanup_inner:
+	delete_pinned_map(INNER_MAP_NAME);
+}
+
+static void test_btf_list(void)
+{
+	ASSERT_OK(run_bpftool_command("btf list"), "list btf data");
+}
+
+static struct test_desc tests[] = {
+	{
+		.name = "unprotected_unpinned",
+		.protection = UNPROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = false,
+		.write_must_fail = false,
+	},
+	{
+		.name = "unprotected_pinned",
+		.protection = UNPROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = true,
+		.write_must_fail = false,
+	},
+	{
+		.name = "protected_unpinned",
+		.protection = PROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = false,
+		.write_must_fail = true,
+	},
+	{
+		.name = "protected_pinned",
+		.protection = PROTECTED,
+		.map_name = UNPROTECTED_MAP_NAME,
+		.pinned = true,
+		.write_must_fail = true,
+	}
+};
+
+static const size_t tests_count = ARRAY_SIZE(tests);
+
+void test_maps_access(void)
+{
+	struct security_bpf_map *skel;
+	struct test_desc *current;
+	int i;
+
+	skel = general_setup();
+	if (!ASSERT_OK_PTR(skel, "prepare programs"))
+		goto cleanup;
+
+	for (i = 0; i < tests_count; i++) {
+		current = &tests[i];
+		test__start_subtest(current->name);
+		if (ASSERT_OK(test_setup(skel, current), "subtest setup"))
+			test_basic_access(current);
+		test_cleanup(current);
+	}
+	test__start_subtest("nested_maps");
+	test_create_nested_maps();
+	test__start_subtest("btf_list");
+	test_btf_list();
+
+cleanup:
+	general_cleanup(skel);
+}
+
Signature · 1 line
2.52.0
Acked-by on v1 3/4
#7 lore raw

Re: [PATCH bpf-next 3/4] selftests/bpf: add bpftool map manipulations tests

2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>

Quoted 19 lines

The test_bpftool_map.sh script tests that maps read/write accesses are being properly allowed/refused by the kernel depending on a specific fmod_ret program being attached on security_bpf_map function.

Rewrite this test to integrate it in the new test_bpftool runner. The new test spawns a few subtests:

#1/1 maps_access/unprotected_unpinned: OK
#1/2 maps_access/unprotected_pinned: OK
#1/3 maps_access/protected_unpinned: OK
#1/4 maps_access/protected_pinned: OK
#1/5 maps_access/nested_maps: OK
#1/6 maps_access/btf_list: OK
#1 maps_access: OK
#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 2 PASSED, 0 FAILED

Quoted 1 line
#8 lore raw

PATCH 4/4 selftests/bpf: remove converted bpftool test scripts

In reply to Alexis Lothoré (eBPF Foundation)

Now that test_bpftool_map.sh and test_bpftool_metadata.sh are integrated into test_bpftool runner, remove those.

---
 tools/testing/selftests/bpf/Makefile               |   2 -
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 3 files changed, 485 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -2
index a1fe94efa53c..daa25d6c8fbb 100644
@@ -109,8 +109,6 @@ TEST_PROGS := test_kmod.sh \
 	test_xdping.sh \
 	test_bpftool_build.sh \
 	test_bpftool.sh \
-	test_bpftool_map.sh \
-	test_bpftool_metadata.sh \
 	test_doc_build.sh \
 	test_xsk.sh \
 	test_xdp_features.sh
tools/testing/selftests/bpf/test_bpftool_map.sh deleted +0 -398

398 lines not on this page · raw message

tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -85

85 lines not on this page · raw message

Signature · 1 line
2.52.0
#10 lore raw
In reply to Alexis Lothoré (eBPF Foundation)
Quoted 9 lines · Alexis Lothoré (eBPF Foundation)

Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.

Some of those tests are validating bpftool behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...) and so they do not integrate well in test_progs. This series proposes to

Can you elaborate why they do not integrate well? In my mind, test_progs should be the only runner into which we invest effort (parallel tests, all the different filtering, etc; why would we have to reimplement subsets of this). The fact that we have test_maps and test_verifier is historical and if we had enough time we'd merge all of them into test_progs.

What exactly in test_progs would prevent us from implementing bpftool test runner?

Quoted 53 lines

introduce a new runner to support those tests. This new runner is heavily inspired from test_progs, with slightly less features.

- First commit move the assert macros exposed by test_progs into a shared header so that they can be used by the new runner
- Second commit introduces the new runner, as well as a first test
- Third commit adds a second test
- Fourth commit drops the legacy scripts corresponding to those new tests

The series only adds the runner, but it is not enough to have it running in CI. The github actions need to be updated both in [1] and [2], and I am not sure how it si supposed to be contributed (the part needed in [1] must likely be integrated in the commit that is systematically brought on top of the bpf-next_base branch). Still, an example of test_bpftool run can be found in [3], in which I have forked and updated the needed Github actions ([4])

[1] https://github.com/kernel-patches/bpf.git
[2] https://github.com/libbpf/ci
[3] https://github.com/kernel-patches/bpf/pull/10711
[4] https://github.com/Tropicao/libbpf-ci

---
Alexis Lothoré (eBPF Foundation) (4):
bpf/selftests: move assert macros into a dedicated header bpf/selftests: introduce bptool test runner and a first test selftests/bpf: add bpftool map manipulations tests selftests/bpf: remove converted bpftool test scripts

 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  16 +-
 tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
 13 files changed, 1041 insertions(+), 711 deletions(-)

---
base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
change-id: 20251212-bpftool-tests-bb165c4cceb8

Best regards,
--
Alexis Lothoré (eBPF Foundation) <[email protected]>

#11 lore raw

Hi Andrii,

Quoted 19 lines · Andrii Nakryiko
Quoted 9 lines · Alexis Lothoré (eBPF Foundation)

Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.

Some of those tests are validating bpftool behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...) and so they do not integrate well in test_progs. This series proposes to

Can you elaborate why they do not integrate well? In my mind, test_progs should be the only runner into which we invest effort (parallel tests, all the different filtering, etc; why would we have to reimplement subsets of this). The fact that we have test_maps and test_verifier is historical and if we had enough time we'd merge all of them into test_progs.

Quoted 2 lines

What exactly in test_progs would prevent us from implementing bpftool test runner?

I don't think there is any strong technical blocker preventing from integrating those tests directly into test_progs. That's rather about the fact that test_progs tests depends (almost) exclusively on libbpf/skeletons. Those bpftool tests rather need to directly execute bpftool and parse its stdout output, so I thought that it made sense to have a dedicated runner for this. If I'm wrong and so if those tests should rather be moved in the test_progs runner (eg to avoid duplicating the runner features), I'm fine with it. Any additional opinion on this is welcome.

Thanks,

Alexis

Signature · 3 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#12 lore raw
Quoted 23 lines · Alexis Lothoré

Hi Andrii,

Quoted 19 lines · Andrii Nakryiko
Quoted 9 lines · Alexis Lothoré (eBPF Foundation)

Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.

Some of those tests are validating bpftool behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...) and so they do not integrate well in test_progs. This series proposes to

Can you elaborate why they do not integrate well? In my mind, test_progs should be the only runner into which we invest effort (parallel tests, all the different filtering, etc; why would we have to reimplement subsets of this). The fact that we have test_maps and test_verifier is historical and if we had enough time we'd merge all of them into test_progs.

Quoted 7 lines
Quoted 2 lines

What exactly in test_progs would prevent us from implementing bpftool test runner?

I don't think there is any strong technical blocker preventing from integrating those tests directly into test_progs. That's rather about the fact that test_progs tests depends (almost) exclusively on libbpf/skeletons. Those bpftool tests rather need to directly execute

There are actually plenty of test in test_progs that do networking setups, calling system() to launch various binaries, etc. So it never was purely for libbpf/skeletons/whatnot.

So yeah, I think bpftool testing should still be implemented as one (or many) test_progs tests (and maybe subtests), utilizing test_progs's generic multi-process testing setup, filtering, reporting, etc infrastructure. No need to add extra runners.

Quoted 14 lines

bpftool and parse its stdout output, so I thought that it made sense to have a dedicated runner for this. If I'm wrong and so if those tests should rather be moved in the test_progs runner (eg to avoid duplicating the runner features), I'm fine with it. Any additional opinion on this is welcome.

Thanks,

Alexis
--
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com