selftests/bpf: add a new runner for bpftool tests
Subsystems (3)
cover selftests/bpf: add a new runner for bpftool tests
Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.
Some of those tests are validating bpftool behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...) and so they do not integrate well in test_progs. This series proposes to introduce a new runner to support those tests. This new runner is heavily inspired from test_progs, with slightly less features.
- First commit move the assert macros exposed by test_progs into a shared header so that they can be used by the new runner
- Second commit introduces the new runner, as well as a first test
- Third commit adds a second test
- Fourth commit drops the legacy scripts corresponding to those new tests
The series only adds the runner, but it is not enough to have it running in CI. The github actions need to be updated both in [1] and [2], and I am not sure how it si supposed to be contributed (the part needed in [1] must likely be integrated in the commit that is systematically brought on top of the bpf-next_base branch). Still, an example of test_bpftool run can be found in [3], in which I have forked and updated the needed Github actions ([4])
[1] https://github.com/kernel-patches/bpf.git
[2] https://github.com/libbpf/ci
[3] https://github.com/kernel-patches/bpf/pull/10711
[4] https://github.com/Tropicao/libbpf-ci
---
Alexis Lothoré (eBPF Foundation) (4):
bpf/selftests: move assert macros into a dedicated header bpf/selftests: introduce bptool test runner and a first test selftests/bpf: add bpftool map manipulations tests selftests/bpf: remove converted bpftool test scripts
tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 16 +- tools/testing/selftests/bpf/assert_helpers.h | 231 ++++++++++++ tools/testing/selftests/bpf/bpftool_helpers.c | 114 ++++++ tools/testing/selftests/bpf/bpftool_helpers.h | 19 + .../testing/selftests/bpf/bpftool_tests/.gitignore | 2 + .../bpf/bpftool_tests/bpftool_maps_access.c | 370 +++++++++++++++++++ .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++ tools/testing/selftests/bpf/test_bpftool.c | 126 +++++++ tools/testing/selftests/bpf/test_bpftool.h | 36 ++ tools/testing/selftests/bpf/test_bpftool_map.sh | 398 --------------------- .../testing/selftests/bpf/test_bpftool_metadata.sh | 85 ----- tools/testing/selftests/bpf/test_progs.h | 226 +----------- 13 files changed, 1041 insertions(+), 711 deletions(-)
---
base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
change-id: 20251212-bpftool-tests-bb165c4cceb8
Best regards,
Signature · 1 line
Alexis Lothoré (eBPF Foundation) <[email protected]>
PATCH 1/4 bpf/selftests: move assert macros into a dedicated header
The test_progs runner defines a large set of convenient assert macros to perform all the tests. Writing a new runner involves rewriting some macros if we want some basic testing features like standardized failure log.
Export those assert macros from test_progs into a dedicated header so that we can use those in any test_runner. The sole requirement to be able to use those macros is to define a test__fail function in the runner that will be called whenever an assert fails.
--- tools/testing/selftests/bpf/assert_helpers.h | 231 +++++++++++++++++++++++++++ tools/testing/selftests/bpf/test_progs.h | 226 +------------------------- 2 files changed, 232 insertions(+), 225 deletions(-)
tools/testing/selftests/bpf/assert_helpers.h new file +231 -0
@@ -0,0 +1,231 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#pragma once +#include <stdio.h> +#include <errno.h> +#include <stdbool.h> + +#define _CHECK(condition, tag, duration, format...) ({ \ + int __ret = !!(condition); \ + int __save_errno = errno; \ + if (__ret) { \ + test__fail(); \ + fprintf(stdout, "%s:FAIL:%s ", __func__, tag); \ + fprintf(stdout, ##format); \ + } else { \ + fprintf(stdout, "%s:PASS:%s %d nsec\n", \ + __func__, tag, duration); \ + } \ + errno = __save_errno; \ + __ret; \ +}) + +#define CHECK_FAIL (condition) ({ \ + int __ret = !!(condition); \ + int __save_errno = errno; \ + if (__ret) { \ + test__fail(); \ + fprintf(stdout, "%s:FAIL:%d\n", __func__, __LINE__); \ + } \ + errno = __save_errno; \ + __ret; \ +}) + +#define CHECK(condition, tag, format...) \ + _CHECK(condition, tag, duration, format) +#define CHECK_ATTR(condition, tag, format...) \ + _CHECK(condition, tag, tattr.duration, format) + +#define ASSERT_FAIL(fmt, args...) ({ \ + static int duration; \ + CHECK(false, "", fmt"\n", ##args); \ + false; \ +}) + +#define ASSERT_TRUE (actual, name) ({ \ + static int duration; \ + bool ___ok = (actual); \ + CHECK(!___ok, (name), "unexpected %s: got FALSE\n", (name)); \ + ___ok; \ +}) + +#define ASSERT_FALSE (actual, name) ({ \ + static int duration; \ + bool ___ok = !(actual); \ + CHECK(!___ok, (name), "unexpected %s: got TRUE\n", (name)); \ + ___ok; \ +}) + +#define ASSERT_EQ (actual, expected, name) ({ \ + static int duration; \ + typeof(actual) ___act = (actual); \ + typeof(expected) ___exp = (expected); \ + bool ___ok = ___act == ___exp; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual %lld != expected %lld\n", \ + (name), (long long)(___act), (long long)(___exp)); \ + ___ok; \ +}) + +#define ASSERT_NEQ (actual, expected, name) ({ \ + static int duration; \ + typeof(actual) ___act = (actual); \ + typeof(expected) ___exp = (expected); \ + bool ___ok = ___act != ___exp; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual %lld == expected %lld\n", \ + (name), (long long)(___act), (long long)(___exp)); \ + ___ok; \ +}) + +#define ASSERT_LT (actual, expected, name) ({ \ + static int duration; \ + typeof(actual) ___act = (actual); \ + typeof(expected) ___exp = (expected); \ + bool ___ok = ___act < ___exp; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual %lld >= expected %lld\n", \ + (name), (long long)(___act), (long long)(___exp)); \ + ___ok; \ +}) + +#define ASSERT_LE (actual, expected, name) ({ \ + static int duration; \ + typeof(actual) ___act = (actual); \ + typeof(expected) ___exp = (expected); \ + bool ___ok = ___act <= ___exp; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual %lld > expected %lld\n", \ + (name), (long long)(___act), (long long)(___exp)); \ + ___ok; \ +}) + +#define ASSERT_GT (actual, expected, name) ({ \ + static int duration; \ + typeof(actual) ___act = (actual); \ + typeof(expected) ___exp = (expected); \ + bool ___ok = ___act > ___exp; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual %lld <= expected %lld\n", \ + (name), (long long)(___act), (long long)(___exp)); \ + ___ok; \ +}) + +#define ASSERT_GE (actual, expected, name) ({ \ + static int duration; \ + typeof(actual) ___act = (actual); \ + typeof(expected) ___exp = (expected); \ + bool ___ok = ___act >= ___exp; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual %lld < expected %lld\n", \ + (name), (long long)(___act), (long long)(___exp)); \ + ___ok; \ +}) + +#define ASSERT_STREQ (actual, expected, name) ({ \ + static int duration; \ + const char *___act = actual; \ + const char *___exp = expected; \ + bool ___ok = strcmp(___act, ___exp) == 0; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual '%s' != expected '%s'\n", \ + (name), ___act, ___exp); \ + ___ok; \ +}) + +#define ASSERT_STRNEQ (actual, expected, len, name) ({ \ + static int duration; \ + const char *___act = actual; \ + const char *___exp = expected; \ + int ___len = len; \ + bool ___ok = strncmp(___act, ___exp, ___len) == 0; \ + CHECK(!___ok, (name), \ + "unexpected %s: actual '%.*s' != expected '%.*s'\n", \ + (name), ___len, ___act, ___len, ___exp); \ + ___ok; \ +}) + +#define ASSERT_HAS_SUBSTR (str, substr, name) ({ \ + static int duration; \ + const char *___str = str; \ + const char *___substr = substr; \ + bool ___ok = strstr(___str, ___substr) != NULL; \ + CHECK(!___ok, (name), \ + "unexpected %s: '%s' is not a substring of '%s'\n", \ + (name), ___substr, ___str); \ + ___ok; \ +}) + +#define ASSERT_MEMEQ (actual, expected, len, name) ({ \ + static int duration; \ + const void *__act = actual; \ + const void *__exp = expected; \ + int __len = len; \ + bool ___ok = memcmp(__act, __exp, __len) == 0; \ + CHECK(!___ok, (name), "unexpected memory mismatch\n"); \ + fprintf(stdout, "actual:\n"); \ + hexdump("\t", __act, __len); \ + fprintf(stdout, "expected:\n"); \ + hexdump("\t", __exp, __len); \ + ___ok; \ +}) + +#define ASSERT_OK (res, name) ({ \ + static int duration; \ + long long ___res = (res); \ + bool ___ok = ___res == 0; \ + CHECK(!___ok, (name), "unexpected error: %lld (errno %d)\n", \ + ___res, errno); \ + ___ok; \ +}) + +#define ASSERT_ERR (res, name) ({ \ + static int duration; \ + long long ___res = (res); \ + bool ___ok = ___res < 0; \ + CHECK(!___ok, (name), "unexpected success: %lld\n", ___res); \ + ___ok; \ +}) + +#define ASSERT_NULL (ptr, name) ({ \ + static int duration; \ + const void *___res = (ptr); \ + bool ___ok = !___res; \ + CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res); \ + ___ok; \ +}) + +#define ASSERT_OK_PTR (ptr, name) ({ \ + static int duration; \ + const void *___res = (ptr); \ + int ___err = libbpf_get_error(___res); \ + bool ___ok = ___err == 0; \ + CHECK(!___ok, (name), "unexpected error: %d\n", ___err); \ + ___ok; \ +}) + +#define ASSERT_ERR_PTR (ptr, name) ({ \ + static int duration; \ + const void *___res = (ptr); \ + int ___err = libbpf_get_error(___res); \ + bool ___ok = ___err != 0; \ + CHECK(!___ok, (name), "unexpected pointer: %p\n", ___res); \ + ___ok; \ +}) + +#define ASSERT_OK_FD (fd, name) ({ \ + static int duration; \ + int ___fd = (fd); \ + bool ___ok = ___fd >= 0; \ + CHECK(!___ok, (name), "unexpected fd: %d (errno %d)\n", \ + ___fd, errno); \ + ___ok; \ +}) + +#define ASSERT_ERR_FD (fd, name) ({ \ + static int duration; \ + int ___fd = (fd); \ + bool ___ok = ___fd < 0; \ + CHECK(!___ok, (name), "unexpected fd: %d\n", ___fd); \ + ___ok; \ +}) +
tools/testing/selftests/bpf/test_progs.h +1 -225
238 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 1/4] bpf/selftests: move assert macros into a dedicated header
2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>
Quoted 11 lines
The
test_progsrunner defines a large set of convenient assert macros to perform all the tests. Writing a new runner involves rewriting some macros if we want some basic testing features like standardized failure log.Export those assert macros from
test_progsinto a dedicated header so that we can use those in anytest_runner. The sole requirement to be able to use those macros is to define atest__failfunction in the runner that will be called whenever an assert fails.Signed-off-by Alexis Lothoré (eBPF Foundation) <[email protected]>
PATCH 2/4 bpf/selftests: introduce bptool test runner and a first test
In reply to Alexis Lothoré (eBPF Foundation)The tools/testing/selftests/bpf directory contains multiple scripts (shell, python, c code, etc) that aim to test some specific features from bpftool. Those isolated tests are currently not executed by any CI automation. Create a dedicated runner for any bpftool-related test that can then be added to the list of executed runners in bpf CI automation. This new runner (and the corresponding Makefile tooling) is highly inspired from test_progs, but kept a bit simpler. This version supports the following features:
- autodetection of bpftool test stored in the in bpftool_tests directory
- bpftool binary under test is passed as runner argument
- a few helpers to allow to easily run abpftool commands while possibly collecting the output
- usage of assert macros shared with test_progs
- basic sub-tests management
- logs collection, logs being dumped only for failed tests
- exit code reflecting whether all tests have passed or not
As this runner needs at least one test to be implemented to properly compile, also bring bpftool_metadata, which is the conversion of test_bpftool_metadata.sh: this test validates that the output of some basic prog/map listings done with bpftool properly returns the metadata collected from the .rodata section of eBPF programs.
This new runner gives an output similar to the one generated by test_progs:
#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 1 PASSED, 0 FAILED
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 14 ++- tools/testing/selftests/bpf/bpftool_helpers.c | 114 ++++++++++++++++++ tools/testing/selftests/bpf/bpftool_helpers.h | 19 +++ .../testing/selftests/bpf/bpftool_tests/.gitignore | 2 + .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++++++++++++++++ tools/testing/selftests/bpf/test_bpftool.c | 126 ++++++++++++++++++++ tools/testing/selftests/bpf/test_bpftool.h | 36 ++++++ 8 files changed, 439 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -2,6 +2,7 @@ bpftool bpf-helpers* bpf-syscall* +test_bpftool test_verifier test_maps test_lru_map
tools/testing/selftests/bpf/Makefile +13 -1
@@ -76,7 +76,8 @@ endif TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_progs \ test_sockmap \ test_tcpnotify_user \ - test_progs-no_alu32 + test_progs-no_alu32 \ + test_bpftool TEST_INST_SUBDIRS := no_alu32 # Also test bpf-gcc, if present
@@ -791,6 +792,17 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built) TRUNNER_BPF_CFLAGS := $(eval $(call DEFINE_TEST_RUNNER,test_maps)) +# Define bpftool test runner. +TRUNNER_TESTS_DIR := bpftool_tests +TRUNNER_BPF_PROGS_DIR := progs +TRUNNER_EXTRA_SOURCES := test_bpftool.c \ + bpftool_helpers.c +TRUNNER_LIB_SOURCES := +TRUNNER_EXTRA_FILES := +TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE +TRUNNER_BPF_CFLAGS := +$(eval $(call DEFINE_TEST_RUNNER,test_bpftool)) + # Define test_verifier test runner. # It is much simpler than test_maps/test_progs and sufficiently different from # them (e.g., test.h is using completely pattern), that it's worth just
tools/testing/selftests/bpf/bpftool_helpers.c new file +114 -0
114 lines not on this page · raw message
tools/testing/selftests/bpf/bpftool_helpers.h new file +19 -0
@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#pragma once + +#include <stdlib.h> +#include <stdio.h> +#include <stdbool.h> + +#define MAX_BPFTOOL_CMD_LEN (256) + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE (arr) (sizeof(arr) / sizeof((arr)[0])) +#endif + +int run_bpftool_command (char *args); +int get_bpftool_command_output (char *args,char *output_buf,size_t output_max_len); +void test__start_subtest (const char *subtests_name); +void test__end_subtest (void ); +void hijack_stdio (void ); +void restore_stdio (void );
tools/testing/selftests/bpf/bpftool_tests/.gitignore new file +2 -0
@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +tests.h
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +128 -0
128 lines not on this page · raw message
tools/testing/selftests/bpf/test_bpftool.c new file +126 -0
126 lines not on this page · raw message
tools/testing/selftests/bpf/test_bpftool.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#pragma once + +#include <stdio.h> +#include <stdbool.h> + +extern struct bpftool_runner_env env; + +void test__fail (void ); + +struct test_state { +char *name; +char *log; +size_t log_size; +bool failed; +int subtests_count; +int subtests_failures; +FILE *saved_stdout; +FILE *saved_stderr; +}; + +struct subtest_state { +char *name; +char *log; +size_t log_size; +bool failed; +}; +struct bpftool_runner_env { +char *bpftool_path; +int failure_cnt; +FILE *saved_stdout; +FILE *saved_stderr; +struct test_state *current_test; +struct subtest_state *current_subtest; +struct subtest_state *subtest_states; +};
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test
2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>
Quoted 10 lines
The
tools/testing/selftests/bpfdirectory contains multiple scripts (shell, python, c code, etc) that aim to test some specific features from bpftool. Those isolated tests are currently not executed by any CI automation. Create a dedicated runner for any bpftool-related test that can then be added to the list of executed runners in bpf CI automation. This new runner (and the correspondingMakefiletooling) is highly inspired fromtest_progs, but kept a bit simpler. This version supports the following features:- autodetection of bpftool test stored in the in
bpftool_tests
(Typo: "in the in")
Quoted 3 lines
directory
- bpftool binary under test is passed as runner argument
- a few helpers to allow to easily run abpftool commands while possibly
(Typo: abpftool)
Quoted 15 lines
collecting the output
- usage of assert macros shared withtest_progs
- basic sub-tests management
- logs collection, logs being dumped only for failed tests
- exit code reflecting whether all tests have passed or notAs this runner needs at least one test to be implemented to properly compile, also bring
bpftool_metadata, which is the conversion oftest_bpftool_metadata.sh: this test validates that the output of some basic prog/map listings done with bpftool properly returns the metadata collected from the .rodata section of eBPF programs.This new runner gives an output similar to the one generated by
test_progs:
Quoted 6 lines
#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 1 PASSED, 0 FAILEDSigned-off-by Alexis Lothoré (eBPF Foundation) <[email protected]>
Thanks a lot for this! It looks good, I only have a few minor comments.
Quoted 118 lines
--- tools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 14 ++- tools/testing/selftests/bpf/bpftool_helpers.c | 114 ++++++++++++++++++ tools/testing/selftests/bpf/bpftool_helpers.h | 19 +++ .../testing/selftests/bpf/bpftool_tests/.gitignore | 2 + .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++++++++++++++++ tools/testing/selftests/bpf/test_bpftool.c | 126 ++++++++++++++++++++ tools/testing/selftests/bpf/test_bpftool.h | 36 ++++++ 8 files changed, 439 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/.gitignore +1
-0@@ -2,6 +2,7 @@ bpftool bpf-helpers* bpf-syscall* +test_bpftool test_verifier test_maps test_lru_maptools/testing/selftests/bpf/Makefile +13
-1@@ -76,7 +76,8 @@ endif TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_progs \ test_sockmap \ test_tcpnotify_user \ - test_progs-no_alu32 + test_progs-no_alu32 \ + test_bpftool TEST_INST_SUBDIRS := no_alu32 # Also test bpf-gcc, if present@@ -791,6 +792,17 @@ TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built) TRUNNER_BPF_CFLAGS := $(eval $(call DEFINE_TEST_RUNNER,test_maps)) +# Define bpftool test runner. +TRUNNER_TESTS_DIR := bpftool_tests +TRUNNER_BPF_PROGS_DIR := progs +TRUNNER_EXTRA_SOURCES := test_bpftool.c \ + bpftool_helpers.c +TRUNNER_LIB_SOURCES := +TRUNNER_EXTRA_FILES := +TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE +TRUNNER_BPF_CFLAGS := +$(eval $(call DEFINE_TEST_RUNNER,test_bpftool)) + # Define test_verifier test runner. # It is much simpler than test_maps/test_progs and sufficiently different from # them (e.g., test.h is using completely pattern), that it's worth justtools/testing/selftests/bpf/bpftool_helpers.c new file +57
-0@@ -0,0 +1,114 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include "bpftool_helpers.h" +#include "test_bpftool.h" +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <stdbool.h> + +#define BPFTOOL_PATH "./tools/sbin/bpftool" +#define BPFTOOL_CMD_MAX_LEN 256 + +static int run_command (char *command,bool get_output,char *output_buf,size_t output_max_len) +{ +FILE *f; +int ret; + + f =popen (command,"r" ); +if (!f) +return 1 ; + +if (get_output) +fread (output_buf,1 , output_max_len, f); + ret =pclose (f); + +return ret; +} + +int run_bpftool_command (char *args) +{ +char cmd[BPFTOOL_CMD_MAX_LEN ]; +int ret; + + ret =snprintf (cmd,BPFTOOL_CMD_MAX_LEN ,"%s %s > /dev/null 2>&1" , + env.bpftool_path, args); +if (ret != +strlen (env.bpftool_path) +1 +strlen (args) +strlen (" > /dev/null 2>&1" )) { +fprintf (stderr,"Failed to generate bpftool command\n" ); +return 1 ; + } + +return run_command (cmd, false,NULL ,0 ); +} + +int get_bpftool_command_output (char *args,char *output_buf,size_t output_max_len) +{ +int ret; +char cmd[BPFTOOL_CMD_MAX_LEN ]; + + ret =snprintf (cmd,BPFTOOL_CMD_MAX_LEN ,"%s %s" , env.bpftool_path, + args); +if (ret !=strlen (args) +strlen (env.bpftool_path) +1 ) { +fprintf (stderr,"Failed to generate bpftool command" ); +return 1 ; + } + +return run_command (cmd, true, output_buf, output_max_len); +}
tools/testing/selftests/bpf/bpftool_helpers.c line 67
Quoted 10 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +10
-0+ +void hijack_stdio (void ) +{ +fflush (stdout); +fflush (stderr); +if (env.current_subtest) { + env.current_test->saved_stdout = stdout; + env.current_test->saved_stderr = stderr; + stdout =open_memstream (&env.current_subtest->log, + &env.current_subtest->log_size);
Missing check on open_memstream() return value? Same below.
tools/testing/selftests/bpf/bpftool_helpers.c line 68
Quoted 1 line
tools/testing/selftests/bpf/bpftool_helpers.c new file +1
-0+
Nit: blank line
Quoted 6 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +6
-0+ }else { + env.saved_stdout = stdout; + env.saved_stderr = stderr; + stdout =open_memstream (&env.current_test->log, + &env.current_test->log_size); + }
tools/testing/selftests/bpf/bpftool_helpers.c line 84
Quoted 10 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +10
-0+ stderr = stdout; +} + +void restore_stdio (void ) +{ +fclose (stdout); +if (env.current_subtest) { + stdout = env.current_test->saved_stdout; + stderr = env.current_test->saved_stderr; +
Nit: blank line
tools/testing/selftests/bpf/bpftool_helpers.c line 89
Quoted 5 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +5
-0+ }else { + stdout = env.saved_stdout; + stderr = env.saved_stderr; + } +
Nit: blank line again
tools/testing/selftests/bpf/bpftool_helpers.c line 98
Quoted 9 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +9
-0+} + +void test__start_subtest (const char *subtest_name) +{ +test__end_subtest (); + env.current_test->subtests_count++; + env.subtest_states =realloc (env.subtest_states, + env.current_test->subtests_count * +sizeof (struct subtest_state ));
Missing check on realloc()'s return value?
Quoted 64 lines
tools/testing/selftests/bpf/bpftool_helpers.c new file +16
-0+ env.current_subtest = + &env.subtest_states[env.current_test->subtests_count -1 ]; +memset (env.current_subtest,0 ,sizeof (struct subtest_state )); + env.current_subtest->name =strdup (subtest_name); + +hijack_stdio (); +} + +void test__end_subtest (void ) +{ +if (env.current_subtest) { +restore_stdio (); + env.current_subtest =NULL ; + } +} +tools/testing/selftests/bpf/bpftool_helpers.h new file +19
-0@@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#pragma once + +#include <stdlib.h> +#include <stdio.h> +#include <stdbool.h> + +#define MAX_BPFTOOL_CMD_LEN (256) + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE (arr) (sizeof(arr) / sizeof((arr)[0])) +#endif + +int run_bpftool_command (char *args); +int get_bpftool_command_output (char *args,char *output_buf,size_t output_max_len); +void test__start_subtest (const char *subtests_name); +void test__end_subtest (void ); +void hijack_stdio (void ); +void restore_stdio (void );tools/testing/selftests/bpf/bpftool_tests/.gitignore new file +2
-0@@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +tests.htools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +9
-0@@ -0,0 +1,128 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <bpftool_helpers.h> +#include <test_bpftool.h> +#include <assert_helpers.h> +#include <linux/bpf.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> +#include <sys/stat.h>
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 19
Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10
-0+#include <stdbool.h> + +#define BPFFS_DIR "/sys/fs/bpf/test_metadata" +#define BPFFS_USED BPFFS_DIR "/used" +#define BPFFS_UNUSED BPFFS_DIR "/unused" + +#define BPF_FILE_USED "metadata_used.bpf.o" +#define BPF_FILE_UNUSED "metadata_unused.bpf.o" + +#define MAX_BPFTOOL_OUTPUT_LEN (100*1000)
Maybe a power of 2 instead, 64k?
Quoted 37 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +37
-0+ +#define MAX_TOKENS_TO_CHECK 3 +static char output[MAX_BPFTOOL_OUTPUT_LEN ]; + +struct test_desc { +char *name; +char *bpf_prog; +char *bpffs_path; +char *expected_output[MAX_TOKENS_TO_CHECK ]; +char *expected_output_json[MAX_TOKENS_TO_CHECK ]; +}; + +static int setup (struct test_desc *test) +{ +return mkdir (BPFFS_DIR ,0700 ); +} + +static void cleanup (struct test_desc *test) +{ +unlink (test->bpffs_path); +rmdir (BPFFS_DIR ); +} + +static int check_metadata (char *buf,char *const *tokens,int count) +{ +int i; + +for (i =0 ; i < count && tokens[i]; i++) +if (!strstr (buf, tokens[i])) +return 1 ; + +return 0 ; +} + +static void run_test (struct test_desc *test) +{ +int ret;
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 66
Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10
-0+char cmd[MAX_BPFTOOL_CMD_LEN ]; + +snprintf (cmd,MAX_BPFTOOL_CMD_LEN ,"prog load %s %s" , + test->bpf_prog, test->bpffs_path); + ret =run_bpftool_command (cmd); +if (!ASSERT_OK (ret,"load program" )) +return ; + +/* Check output with default format */ + ret = get_bpftool_command_output("prog show name prog", output,
I'd probably use "pinned" + pinned path rather than "name prog", especially because the name for the program does not otherwise appear in this file (and same below).
Quoted 6 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +6
-0+ MAX_BPFTOOL_OUTPUT_LEN); +if (ASSERT_OK (ret,"get program info" )) { + ret =check_metadata (output, test->expected_output, +ARRAY_SIZE (test->expected_output)); +ASSERT_OK (ret,"find metadata" ); + }
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 82
Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10
-0+ +/* Check output with json format */ + ret =get_bpftool_command_output ("prog -j show name prog" , output, +MAX_BPFTOOL_OUTPUT_LEN ); +if (ASSERT_OK (ret,"get program info in json" )) { + ret =check_metadata (output, test->expected_output_json, +ARRAY_SIZE (test->expected_output_json)); +ASSERT_OK (ret,"find metadata in json" ); + } +
Nit: blank line
Quoted 34 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +34
-0+} + +struct test_desc tests[] = { + { + .name ="metadata_unused" , + .bpf_prog =BPF_FILE_UNUSED , + .bpffs_path =BPFFS_UNUSED , + .expected_output = { +"a = \"foo\"" , +"b = 1" + }, + .expected_output_json = { +"\"metadata\":{\"a\":\"foo\",\"b\":1}" + } + }, + { + .name ="metadata_used" , + .bpf_prog =BPF_FILE_USED , + .bpffs_path =BPFFS_USED , + .expected_output = { +"a = \"bar\"" , +"b = 2" + }, + .expected_output_json = { +"\"metadata\":{\"a\":\"bar\",\"b\":2}" + } + } +}; + +static const int tests_count =ARRAY_SIZE (tests); + +void test_metadata (void ) +{ +int i, ret;
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c line 126
Quoted 10 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +10
-0+ +for (i =0 ; i < tests_count; i++) { +test__start_subtest (tests[i].name); + ret =setup (&tests[i]); +if (!ASSERT_OK (ret,"setup bpffs pin dir" )) +continue ; +run_test (&tests[i]); +cleanup (&tests[i]); + } +
Nit: blank line
Quoted 47 lines
tools/testing/selftests/bpf/bpftool_tests/bpftool_metadata.c new file +2
-0+} +tools/testing/selftests/bpf/test_bpftool.c new file +39
-0@@ -0,0 +1,126 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <test_bpftool.h> +#include <bpftool_helpers.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdbool.h> +#include <string.h> +#include <unistd.h> + +struct bpftool_runner_env env = {0 }; + +#define DEFINE_TEST (name) extern void test_##name(void); +#include <bpftool_tests/tests.h> +#undef DEFINE_TEST + +struct prog_test_def { +char *test_name; +void (*run_test)(void ); +}; + +static struct prog_test_def prog_test_defs[] = { +#define DEFINE_TEST(name) { \ + .test_name = #name, \ + .run_test = &test_##name, \ +}, +#include <bpftool_tests/tests.h> +#undef DEFINE_TEST +}; + + +static const int tests_count =ARRAY_SIZE (prog_test_defs); + +/* Needed method for the assert macros exposed by assert_helpers.h */ +void test__fail (void ) +{ +if (env.current_subtest) + env.current_subtest->failed = true; +if (!env.current_test->failed) + env.failure_cnt++;
tools/testing/selftests/bpf/test_bpftool.c line 49
Quoted 10 lines
tools/testing/selftests/bpf/test_bpftool.c new file +10
-0+ env.current_test->failed = true; +} + +static void test_setup (struct test_state *test,char *name) +{ + env.current_test = test; + env.current_test->name =strdup (name); +} + +static void dump_results (struct test_state *test,int test_index)
Would you mind renaming this function? It also frees the logs and names and subtests states. Maybe "dump_results_and_cleanup"? Or move the all the clean-up to test_teardown(). Even if we loop again over the subtests, the overhead should be minimal, and I'd find it easier to follow.
tools/testing/selftests/bpf/test_bpftool.c line 51
Quoted 2 lines
tools/testing/selftests/bpf/test_bpftool.c new file +2
-0+{ +int j;
Let's use "i" rather than "j"
Quoted 54 lines
tools/testing/selftests/bpf/test_bpftool.c new file +54
-0+ +if (test->failed) +fprintf (stdout,"%s\n" , test->log); +free (test->log); +for (j =0 ; j < test->subtests_count; j++) { +if (env.subtest_states[j].failed) +fprintf (stdout,"%s\n" , env.subtest_states[j].log); +free (env.subtest_states[j].log); +fprintf (stdout,"#%d/%d\t%s/%s: %s\n" , test_index+1 , j+1 , + env.current_test->name, + env.subtest_states[j].name, + env.subtest_states[j].failed ?"KO" :"OK" ); +free (env.subtest_states[j].name); + } +if (env.current_test->subtests_count) { +free (env.subtest_states); + env.subtest_states =NULL ; + } +fprintf (stdout,"#%d\t%s: %s\n" , test_index +1 , test->name, + test->failed ?"KO" :"OK" ); +} + +static void test_teardown (struct test_state *test,int test_index) +{ +dump_results (test, test_index); +free (env.current_test->name); + env.current_test =NULL ; +} + +static int parse_args (int argc,char *argv[]) +{ +if (argc !=2 ) +return 1 ; +if (access (argv[1 ],R_OK |X_OK )) +return 1 ; + env.bpftool_path = argv[1 ]; + +return 0 ; +} + +static void usage (char *prog) +{ +fprintf (stdout,"Usage: %s <bpftool_path>\n" , prog); +fprintf (stdout,"\t<bpftool_path>: path to the bpftool binary to test\n" ); +} + +int main (int argc,char *argv[]) +{ +struct test_state *ctx =NULL ; +int i; + +if (parse_args (argc, argv)) { +fprintf (stderr,"Invalid arguments\n" ); +usage (argv[0 ]);
tools/testing/selftests/bpf/test_bpftool.c line 115
Quoted 10 lines
tools/testing/selftests/bpf/test_bpftool.c new file +10
-0+exit (EXIT_FAILURE ); + } + + ctx =calloc (tests_count,sizeof (struct test_state )); +if (!ctx) +exit (EXIT_FAILURE ); + +for (i =0 ; i < tests_count; i++) { +test_setup (&ctx[i], prog_test_defs[i].test_name); +hijack_stdio ();
Nit: Move hijack_stdio()/restore_stdio() to test_setup()/test_teardown()?
Quoted 54 lines
tools/testing/selftests/bpf/test_bpftool.c new file +11
-0+ prog_test_defs[i].run_test (); +test__end_subtest (); +restore_stdio (); +test_teardown (&ctx[i], i); + } + +fprintf (stdout,"Summary: %d PASSED, %d FAILED\n" , + tests_count - env.failure_cnt, env.failure_cnt); +free (ctx); +return env.failure_cnt ?EXIT_FAILURE :EXIT_SUCCESS ; +}tools/testing/selftests/bpf/test_bpftool.h new file +36
-0@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#pragma once + +#include <stdio.h> +#include <stdbool.h> + +extern struct bpftool_runner_env env; + +void test__fail (void ); + +struct test_state { +char *name; +char *log; +size_t log_size; +bool failed; +int subtests_count; +int subtests_failures; +FILE *saved_stdout; +FILE *saved_stderr; +}; + +struct subtest_state { +char *name; +char *log; +size_t log_size; +bool failed; +}; +struct bpftool_runner_env { +char *bpftool_path; +int failure_cnt; +FILE *saved_stdout; +FILE *saved_stderr; +struct test_state *current_test; +struct subtest_state *current_subtest; +struct subtest_state *subtest_states; +};
Re: [PATCH bpf-next 2/4] bpf/selftests: introduce bptool test runner and a first test
Hi Quentin, thanks for the review !
I agree with all your comments. Andrii raised some concerns about bringing a new runner, I'll wait for this new runner question to be settled before bringing those changes, as it could make this series change quite a lot.
Thanks,
Alexis
Signature · 3 lines
Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 3/4 selftests/bpf: add bpftool map manipulations tests
In reply to Alexis Lothoré (eBPF Foundation)The test_bpftool_map.sh script tests that maps read/write accesses are being properly allowed/refused by the kernel depending on a specific fmod_ret program being attached on security_bpf_map function.
Rewrite this test to integrate it in the new test_bpftool runner. The new test spawns a few subtests:
#1/1 maps_access/unprotected_unpinned: OK
#1/2 maps_access/unprotected_pinned: OK
#1/3 maps_access/protected_unpinned: OK
#1/4 maps_access/protected_pinned: OK
#1/5 maps_access/nested_maps: OK
#1/6 maps_access/btf_list: OK
#1 maps_access: OK
#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 2 PASSED, 0 FAILED
--- .../bpf/bpftool_tests/bpftool_maps_access.c | 370 +++++++++++++++++++++ 1 file changed, 370 insertions(+)
tools/testing/selftests/bpf/bpftool_tests/bpftool_maps_access.c new file +370 -0
@@ -0,0 +1,370 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include <bpf/libbpf.h> +#include <bpftool_helpers.h> +#include <stdlib.h> +#include <test_bpftool.h> +#include <assert_helpers.h> +#include <linux/bpf.h> +#include <unistd.h> +#include <fcntl.h> +#include <stdint.h> +#include <sys/stat.h> +#include <stdbool.h> +#include <bpf/bpf.h> +#include "security_bpf_map.skel.h" + +#define PROTECTED_MAP_NAME "prot_map" +#define UNPROTECTED_MAP_NAME "not_prot_map" +#define BPF_ITER_FILE "bpf_iter_map_elem.bpf.o" +#define BPFFS_PIN_DIR "/sys/fs/bpf/test_bpftool_map" +#define INNER_MAP_NAME "inner_map_tt" +#define OUTER_MAP_NAME "outer_map_tt" + +#define MAP_NAME_MAX_LEN 64 +#define PATH_MAX_LEN 128 + +enum map_protection { +PROTECTED , +UNPROTECTED +}; + +struct test_desc { +char *name; +enum map_protection protection; +struct bpf_map *map; +char *map_name; +bool pinned; +char pin_path[PATH_MAX_LEN ]; +bool write_must_fail; +}; + +static struct security_bpf_map *general_setup (void ) +{ +struct security_bpf_map *skel; +uint32_t key, value; +int ret, i; + + skel =security_bpf_map__open_and_load (); +if (!ASSERT_OK_PTR (skel,"open and load skeleton" )) + goto end; + +struct bpf_map *maps[] = {skel->maps.prot_map, skel->maps.not_prot_map}; + + ret =security_bpf_map__attach (skel); +if (!ASSERT_OK (ret,"attach maps security programs" )) + goto end_destroy; + +for (i =0 ; i <sizeof (maps)/sizeof (struct bpf_map *); i++) { +for (key =0 ; key <2 ; key++) { +int ret =bpf_map__update_elem (maps[i], &key, +sizeof (key), &key,sizeof (key), +0 ); +if (!ASSERT_OK (ret,"set initial map value" )) + goto end_destroy; + } + } + + key =0 ; + value =1 ; + ret =bpf_map__update_elem (skel->maps.prot_status_map, &key, +sizeof (key), &value,sizeof (value),0 ); +if (!ASSERT_OK (ret,"configure map protection" )) + goto end_destroy; + +if (!ASSERT_OK (mkdir (BPFFS_PIN_DIR ,S_IFDIR ),"create bpffs pin dir" )) + goto end_destroy; + +return skel; +end_destroy: +security_bpf_map__destroy (skel); +end: +return NULL ; +} + +static void general_cleanup (struct security_bpf_map *skel) +{ +rmdir (BPFFS_PIN_DIR ); +security_bpf_map__destroy (skel); +} + +static void update_test_desc (struct security_bpf_map *skel, +struct test_desc *test) +{ +/* Now that the skeleton is loaded, update all missing fields to +* have the subtest properly configured +*/ +if (test->protection ==PROTECTED ) { + test->map = skel->maps.prot_map; + test->map_name =PROTECTED_MAP_NAME ; + }else { + test->map = skel->maps.not_prot_map; + test->map_name =UNPROTECTED_MAP_NAME ; + } +} + +static int test_setup (struct security_bpf_map *skel,struct test_desc *desc) +{ +int ret; + +update_test_desc (skel, desc); + +if (desc->pinned) { + ret =snprintf (desc->pin_path,PATH_MAX_LEN ,"%s/%s" ,BPFFS_PIN_DIR , + desc->name); +if (!ASSERT_GT (ret,0 ,"format pin path" )) +return 1 ; + ret =bpf_map__pin (desc->map, desc->pin_path); +if (!ASSERT_OK (ret,"pin map" )) +return 1 ; + } + +return 0 ; +} + +static void test_cleanup (struct test_desc *desc) +{ +if (desc->pinned) +bpf_map__unpin (desc->map,NULL ); +} + +static int lookup_map_value (char *map_handle) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf (cmd,MAX_BPFTOOL_CMD_LEN ,"map lookup %s key 0 0 0 0" , + map_handle); +if (!ASSERT_GT (ret,0 ,"format map lookup cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static int read_map_btf_data (char *map_handle) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf (cmd,MAX_BPFTOOL_CMD_LEN ,"btf dump map %s" , + map_handle); +if (!ASSERT_GT (ret,0 ,"format map btf dump cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static int write_map_value (char *map_handle) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf (cmd,MAX_BPFTOOL_CMD_LEN , +"map update %s key 0 0 0 0 value 1 1 1 1" , map_handle); +if (!ASSERT_GT (ret,0 ,"format value write cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static int delete_map_value (char *map_handle) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf (cmd,MAX_BPFTOOL_CMD_LEN , +"map delete %s key 0 0 0 0" , map_handle); +if (!ASSERT_GT (ret,0 ,"format value deletion cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static int iterate_on_map_values (char *map_handle,char *iter_pin_path) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + + ret =snprintf (cmd,MAX_BPFTOOL_CMD_LEN ,"iter pin %s %s map %s" , +BPF_ITER_FILE , iter_pin_path, map_handle); +if (!ASSERT_GT (ret,0 ,"format iterator cration cmd" )) +return 1 ; + ret =run_bpftool_command (cmd); +if (ret) +return ret; + ret =snprintf (cmd,MAP_NAME_MAX_LEN ,"cat %s" , iter_pin_path); +if (ret <0 ) + goto cleanup; + ret =system (cmd); + +cleanup: +unlink (iter_pin_path); +return ret; +} + +static int create_inner_map (void ) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf ( + cmd,MAX_BPFTOOL_CMD_LEN , +"map create %s/%s type array key 4 value 4 entries 4 name %s" , +BPFFS_PIN_DIR ,INNER_MAP_NAME ,INNER_MAP_NAME ); +if (!ASSERT_GT (ret,0 ,"format inner map create cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static int create_outer_map (void ) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf ( + cmd,MAX_BPFTOOL_CMD_LEN , +"map create %s/%s type hash_of_maps key 4 value 4 entries 2 name %s inner_map name %s" , +BPFFS_PIN_DIR ,OUTER_MAP_NAME ,OUTER_MAP_NAME ,INNER_MAP_NAME ); +if (!ASSERT_GT (ret,0 ,"format outer map create cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static void delete_pinned_map (char *map_name) +{ +char pin_path[PATH_MAX_LEN ]; +int ret; + + ret =snprintf (pin_path,PATH_MAX_LEN ,"%s/%s" ,BPFFS_PIN_DIR , + map_name); +if (ret >=0 ) +unlink (pin_path); +} + +static int add_outer_map_entry (int key) +{ +char cmd[MAX_BPFTOOL_CMD_LEN ]; +int ret =0 ; + + ret =snprintf ( + cmd,MAX_BPFTOOL_CMD_LEN , +"map update pinned %s/%s key %d 0 0 0 value name %s" , +BPFFS_PIN_DIR ,OUTER_MAP_NAME , key,INNER_MAP_NAME ); +if (!ASSERT_GT (ret,0 ,"format outer map value addition cmd" )) +return 1 ; +return run_bpftool_command (cmd); +} + +static void test_basic_access (struct test_desc *desc) +{ +char map_handle[MAP_NAME_MAX_LEN ]; +char iter_pin_path[PATH_MAX_LEN ]; +int ret; + +if (desc->pinned) + ret =snprintf (map_handle,MAP_NAME_MAX_LEN ,"pinned %s" , + desc->pin_path); +else + ret =snprintf (map_handle,MAP_NAME_MAX_LEN ,"name %s" , + desc->map_name); +if (!ASSERT_GT (ret,0 ,"format map handle" )) +return ; + + ret =lookup_map_value (map_handle); +ASSERT_OK (ret,"read map value" ); + + ret =read_map_btf_data (map_handle); +ASSERT_OK (ret,"read map btf data" ); + + ret =write_map_value (map_handle); +ASSERT_OK (desc->write_must_fail ? !ret : ret,"write map value" ); + + ret =delete_map_value (map_handle); +ASSERT_OK (desc->write_must_fail ? !ret : ret,"delete map value" ); +/* Restore deleted value */ +if (!ret) +write_map_value (map_handle); + + ret =snprintf (iter_pin_path,PATH_MAX_LEN ,"%s/iter" ,BPFFS_PIN_DIR ); +if (ASSERT_GT (ret,0 ,"format iter pin path" )) { + ret =iterate_on_map_values (map_handle, iter_pin_path); +ASSERT_OK (ret,"iterate on map values" ); + } +} + +static void test_create_nested_maps (void ) +{ +if (!ASSERT_OK (create_inner_map (),"create inner map" )) +return ; +if (!ASSERT_OK (create_outer_map (),"create outer map" )) + goto end_cleanup_inner; +ASSERT_OK (add_outer_map_entry (0 ),"add a first entry in outer map" ); +ASSERT_OK (add_outer_map_entry (1 ),"add a second entry in outer map" ); +ASSERT_NEQ (add_outer_map_entry (2 ),0 ,"add a third entry in outer map" ); + +delete_pinned_map (OUTER_MAP_NAME ); +end_cleanup_inner: +delete_pinned_map (INNER_MAP_NAME ); +} + +static void test_btf_list (void ) +{ +ASSERT_OK (run_bpftool_command ("btf list" ),"list btf data" ); +} + +static struct test_desc tests[] = { + { + .name ="unprotected_unpinned" , + .protection =UNPROTECTED , + .map_name =UNPROTECTED_MAP_NAME , + .pinned = false, + .write_must_fail = false, + }, + { + .name ="unprotected_pinned" , + .protection =UNPROTECTED , + .map_name =UNPROTECTED_MAP_NAME , + .pinned = true, + .write_must_fail = false, + }, + { + .name ="protected_unpinned" , + .protection =PROTECTED , + .map_name =UNPROTECTED_MAP_NAME , + .pinned = false, + .write_must_fail = true, + }, + { + .name ="protected_pinned" , + .protection =PROTECTED , + .map_name =UNPROTECTED_MAP_NAME , + .pinned = true, + .write_must_fail = true, + } +}; + +static const size_t tests_count =ARRAY_SIZE (tests); + +void test_maps_access (void ) +{ +struct security_bpf_map *skel; +struct test_desc *current; +int i; + + skel =general_setup (); +if (!ASSERT_OK_PTR (skel,"prepare programs" )) + goto cleanup; + +for (i =0 ; i < tests_count; i++) { + current = &tests[i]; +test__start_subtest (current->name); +if (ASSERT_OK (test_setup (skel, current),"subtest setup" )) +test_basic_access (current); +test_cleanup (current); + } +test__start_subtest ("nested_maps" ); +test_create_nested_maps (); +test__start_subtest ("btf_list" ); +test_btf_list (); + +cleanup: +general_cleanup (skel); +} +
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 3/4] selftests/bpf: add bpftool map manipulations tests
2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>
Quoted 19 lines
The
test_bpftool_map.shscript tests that maps read/write accesses are being properly allowed/refused by the kernel depending on a specificfmod_retprogram being attached onsecurity_bpf_mapfunction.Rewrite this test to integrate it in the new
test_bpftoolrunner. The new test spawns a few subtests:#1/1 maps_access/unprotected_unpinned: OK
#1/2 maps_access/unprotected_pinned: OK
#1/3 maps_access/protected_unpinned: OK
#1/4 maps_access/protected_pinned: OK
#1/5 maps_access/nested_maps: OK
#1/6 maps_access/btf_list: OK
#1maps_access: OK
#2/1 metadata/metadata_unused: OK
#2/2 metadata/metadata_used: OK
#2 metadata: OK
Summary: 2 PASSED, 0 FAILED
Quoted 1 line
Signed-off-by Alexis Lothoré (eBPF Foundation) <[email protected]>
PATCH 4/4 selftests/bpf: remove converted bpftool test scripts
In reply to Alexis Lothoré (eBPF Foundation)Now that test_bpftool_map.sh and test_bpftool_metadata.sh are integrated into test_bpftool runner, remove those.
--- tools/testing/selftests/bpf/Makefile | 2 - tools/testing/selftests/bpf/test_bpftool_map.sh | 398 --------------------- .../testing/selftests/bpf/test_bpftool_metadata.sh | 85 ----- 3 files changed, 485 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -2
@@ -109,8 +109,6 @@ TEST_PROGS := test_kmod.sh \ test_xdping.sh \ test_bpftool_build.sh \ test_bpftool.sh \ - test_bpftool_map.sh \ - test_bpftool_metadata.sh \ test_doc_build.sh \ test_xsk.sh \ test_xdp_features.sh
tools/testing/selftests/bpf/test_bpftool_map.sh deleted +0 -398
398 lines not on this page · raw message
tools/testing/selftests/bpf/test_bpftool_metadata.sh deleted +0 -85
85 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH bpf-next 4/4] selftests/bpf: remove converted bpftool test scripts
2026-01-14 09:59 UTC+0100 ~ "Alexis Lothoré (eBPF Foundation)" <[email protected]>
Quoted 4 lines
Now that
test_bpftool_map.shandtest_bpftool_metadata.share integrated intotest_bpftoolrunner, remove those.Signed-off-by Alexis Lothoré (eBPF Foundation) <[email protected]>
Thanks a lot!
Quoted 9 lines · Alexis Lothoré (eBPF Foundation)
Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.Some of those tests are validating bpftool
behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)and so they do not integrate well intest_progs. This series proposes to
Can you elaborate why they do not integrate well? In my mind, test_progs should be the only runner into which we invest effort (parallel tests, all the different filtering, etc; why would we have to reimplement subsets of this). The fact that we have test_maps and test_verifier is historical and if we had enough time we'd merge all of them into test_progs.
What exactly in test_progs would prevent us from implementing bpftool test runner?
Quoted 53 lines
introduce a new runner to support those tests. This new runner is heavily inspired from
test_progs, with slightly less features.- First commit move the assert macros exposed by
test_progsinto a shared header so that they can be used by the new runner
- Second commit introduces the new runner, as well as a first test
- Third commit adds a second test
- Fourth commit drops the legacy scripts corresponding to those new testsThe series only adds the runner, but it is not enough to have it running in CI. The github actions need to be updated both in [1] and [2], and I am not sure how it si supposed to be contributed (the part needed in [1] must likely be integrated in the commit that is systematically brought on top of the bpf-next_base branch). Still, an example of
test_bpftoolrun can be found in [3], in which I have forked and updated the needed Github actions ([4])[1] https://github.com/kernel-patches/bpf.git
[2] https://github.com/libbpf/ci
[3] https://github.com/kernel-patches/bpf/pull/10711
[4] https://github.com/Tropicao/libbpf-ciSigned-off-by Alexis Lothoré (eBPF Foundation) <[email protected]>---
Alexis Lothoré (eBPF Foundation) (4):
bpf/selftests: move assert macros into a dedicated header bpf/selftests: introduce bptool test runner and a first test selftests/bpf: add bpftool map manipulations tests selftests/bpf: remove converted bpftool test scriptstools/testing/selftests/bpf/.gitignore | 1 + tools/testing/selftests/bpf/Makefile | 16 +- tools/testing/selftests/bpf/assert_helpers.h | 231 ++++++++++++ tools/testing/selftests/bpf/bpftool_helpers.c | 114 ++++++ tools/testing/selftests/bpf/bpftool_helpers.h | 19 + .../testing/selftests/bpf/bpftool_tests/.gitignore | 2 + .../bpf/bpftool_tests/bpftool_maps_access.c | 370 +++++++++++++++++++ .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++ tools/testing/selftests/bpf/test_bpftool.c | 126 +++++++ tools/testing/selftests/bpf/test_bpftool.h | 36 ++ tools/testing/selftests/bpf/test_bpftool_map.sh | 398 --------------------- .../testing/selftests/bpf/test_bpftool_metadata.sh | 85 ----- tools/testing/selftests/bpf/test_progs.h | 226 +----------- 13 files changed, 1041 insertions(+), 711 deletions(-)---
base-commit:807cd0dc688b0e7314f53dc3a594f247bb3b665b
change-id: 20251212-bpftool-tests-bb165c4cceb8Best regards,
--
Alexis Lothoré (eBPF Foundation) <[email protected]>
Hi Andrii,
Quoted 19 lines · Andrii Nakryiko
Quoted 9 lines · Alexis Lothoré (eBPF Foundation)
Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.Some of those tests are validating bpftool
behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)and so they do not integrate well intest_progs. This series proposes toCan you elaborate why they do not integrate well? In my mind,
test_progsshould be the only runner into which we invest effort (parallel tests, all the different filtering, etc; why would we have to reimplement subsets of this). The fact that we havetest_mapsandtest_verifieris historical and if we had enough time we'd merge all of them intotest_progs.
Quoted 2 lines
What exactly in
test_progswould prevent us from implementing bpftool test runner?
I don't think there is any strong technical blocker preventing from integrating those tests directly into test_progs. That's rather about the fact that test_progs tests depends (almost) exclusively on libbpf/skeletons. Those bpftool tests rather need to directly execute bpftool and parse its stdout output, so I thought that it made sense to have a dedicated runner for this. If I'm wrong and so if those tests should rather be moved in the test_progs runner (eg to avoid duplicating the runner features), I'm fine with it. Any additional opinion on this is welcome.
Thanks,
Alexis
Signature · 3 lines
Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 23 lines · Alexis Lothoré
Hi Andrii,
Quoted 19 lines · Andrii Nakryiko
Quoted 9 lines · Alexis Lothoré (eBPF Foundation)
Hello,
this series is part of the larger effort aiming to convert all standalone tests to the CI runners so that they are properly executed on patches submission.Some of those tests are validating bpftool
behavior(test_bpftool_map.sh, test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)and so they do not integrate well intest_progs. This series proposes toCan you elaborate why they do not integrate well? In my mind,
test_progsshould be the only runner into which we invest effort (parallel tests, all the different filtering, etc; why would we have to reimplement subsets of this). The fact that we havetest_mapsandtest_verifieris historical and if we had enough time we'd merge all of them intotest_progs.
Quoted 7 lines
Quoted 2 lines
What exactly in
test_progswould prevent us from implementing bpftool test runner?I don't think there is any strong technical blocker preventing from integrating those tests directly into
test_progs. That's rather about the fact thattest_progstests depends (almost) exclusively on libbpf/skeletons. Those bpftool tests rather need to directly execute
There are actually plenty of test in test_progs that do networking setups, calling system() to launch various binaries, etc. So it never was purely for libbpf/skeletons/whatnot.
So yeah, I think bpftool testing should still be implemented as one (or many) test_progs tests (and maybe subtests), utilizing test_progs's generic multi-process testing setup, filtering, reporting, etc infrastructure. No need to add extra runners.
Quoted 14 lines
bpftool and parse its stdout output, so I thought that it made sense to have a dedicated runner for this. If I'm wrong and so if those tests should rather be moved in the
test_progsrunner (eg to avoid duplicating the runner features), I'm fine with it. Any additional opinion on this is welcome.Thanks,
Alexis
--
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com