Skip to contentSkip to conversation

Fix verifier test failures in verbose mode

PATCH Applied Gregory Bell 3 participants 5 messages May 12, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Fix verifier test failures in verbose mode

This patch series fixes two issues that cause false failures in the BPF verifier test suite when run with verbose output (-v).

The following tests fail only when running the test_verifier in verbose.

#458/p ld_dw: xor semi-random 64 bit imms, test 5 FAIL #494/p precise: test 1 FAIL
#495/p precise: test 2 FAIL
#497/p precise: ST zero to stack insn is supported FAIL #498/p precise: STX insn causing spi > allocated_stack FAIL #501/p scale: scale test 1 FAIL
#502/p scale: scale test 2 FAIL

This leads to inconsistent results across verbose and non-verbose runs.

Patch 1 addresses an issue where the verbose flag (-v) unintentionally overrides the opts.log_level, leading to incorrect contents when checking bpf_vlog in tests with expected_ret == VERBOSE_ACCEPT. This occurs when running verbose with -v but not -vv

Patch 2 increases the size of the bpf_vlog[] buffer to prevent truncation of large verifier logs, which was causing failures in several scale and 64-bit immediate tests.

Before patches:
./test_verifier | grep FAIL
Summary: 790 PASSED, 0 SKIPPED, 0 FAILED

./test_verifier -v | grep FAIL
#115/p BPF_ST_MEM stack imm sign FAIL
#458/p ld_dw: xor semi-random 64 bit imms, test 5 FAIL #494/p precise: test 1 FAIL
#495/p precise: test 2 FAIL
#497/p precise: ST zero to stack insn is supported FAIL #498/p precise: STX insn causing spi > allocated_stack FAIL #501/p scale: scale test 1 FAIL
#502/p scale: scale test 2 FAIL
Summary: 782 PASSED, 0 SKIPPED, 8 FAILED

./test_verifier -vv | grep FAIL
#458/p ld_dw: xor semi-random 64 bit imms, test 5 FAIL #501/p scale: scale test 1 FAIL
#502/p scale: scale test 2 FAIL
Summary: 787 PASSED, 0 SKIPPED, 3 FAILED

After patches:
./test_verifier -v | grep FAIL
Summary: 790 PASSED, 0 SKIPPED, 0 FAILED
./test_verifier -vv | grep FAIL
Summary: 790 PASSED, 0 SKIPPED, 0 FAILED

These fixes improve test reliability and ensure consistent behavior across verbose and non-verbose runs.

Gregory Bell (2):
selftests/bpf: test_verifier verbose causes erroneous failures selftests/bpf: test_verifier verbose log overflows

 tools/testing/selftests/bpf/test_verifier.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
Signature · 1 line
2.49.0
#1 lore raw

PATCH 1/2 selftests/bpf: test_verifier verbose causes erroneous failures

When running test_verifier with the -v flag and a test with expected_ret==VERBOSE_ACCEPT, the opts.log_level is unintentionally overwritten because the verbose flag takes precedence. This leads to a mismatch in the expected and actual contents of bpf_vlog, causing tests to fail incorrectly.

Reorder the conditional logic that sets opts.log_level to preserve the expected log level and prevent it from being overridden by -v.

---
 tools/testing/selftests/bpf/test_verifier.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/test_verifier.c +3 -3
index 447b68509d76..2d13e862b078 100644
@@ -1559,10 +1559,10 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
 		       test->errstr_unpriv : test->errstr;
 
 	opts.expected_attach_type = test->expected_attach_type;
-	if (verbose)
-		opts.log_level = verif_log_level | 4; /* force stats */
-	else if (expected_ret == VERBOSE_ACCEPT)
+	if (expected_ret == VERBOSE_ACCEPT)
 		opts.log_level = 2;
+	else if (verbose)
+		opts.log_level = verif_log_level | 4; /* force stats */
 	else
 		opts.log_level = DEFAULT_LIBBPF_LOG_LEVEL;
 	opts.prog_flags = pflags;
Signature · 1 line
2.49.0
#2 lore raw

PATCH 2/2 selftests/bpf: test_verifier verbose log overflows

In reply to Gregory Bell

Tests:
- 458/p ld_dw: xor semi-random 64-bit imms, test 5
- 501/p scale: scale test 1
- 502/p scale: scale test 2

fail in verbose mode due to bpf_vlog[] overflowing. These tests generate large verifier logs that exceed the current buffer size, causing them to fail to load.

Increase the size of the bpf_vlog[] buffer to accommodate larger logs and prevent false failures during test runs with verbose output.

---
 tools/testing/selftests/bpf/test_verifier.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/test_verifier.c +1 -1
index 2d13e862b078..27db34ecf3f5 100644
@@ -734,7 +734,7 @@ static __u32 btf_raw_types[] = {
 	BTF_MEMBER_ENC(71, 13, 128), /* struct prog_test_member __kptr *ptr; */
 };
 
-static char bpf_vlog[UINT_MAX >> 8];
+static char bpf_vlog[UINT_MAX >> 5];
 
 static int load_btf_spec(__u32 *types, int types_len,
 			 const char *strings, int strings_len)
Signature · 1 line
2.49.0
maintainer Tested-by on the v1 series
#3 lore raw
In reply to Gregory Bell
Quoted 22 lines · Gregory Bell

This patch series fixes two issues that cause false failures in the BPF verifier test suite when run with verbose output (-v).

The following tests fail only when running the test_verifier in verbose.

#458/p ld_dw: xor semi-random 64 bit imms, test 5 FAIL #494/p precise: test 1 FAIL
#495/p precise: test 2 FAIL
#497/p precise: ST zero to stack insn is supported FAIL #498/p precise: STX insn causing spi > allocated_stack FAIL #501/p scale: scale test 1 FAIL
#502/p scale: scale test 2 FAIL

This leads to inconsistent results across verbose and non-verbose runs.

Patch 1 addresses an issue where the verbose flag (-v) unintentionally overrides the opts.log_level, leading to incorrect contents when checking bpf_vlog in tests with expected_ret == VERBOSE_ACCEPT. This occurs when running verbose with -v but not -vv

Quoted 8 lines

Patch 2 increases the size of the bpf_vlog[] buffer to prevent truncation of large verifier logs, which was causing failures in several scale and 64-bit immediate tests.

Before patches:
./test_verifier | grep FAIL
Summary: 790 PASSED, 0 SKIPPED, 0 FAILED

Can reproduce the issue with -v option, the series fixes failures I see.

#4 lore raw
In reply to Gregory Bell
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 14 lines · On Mon, 12 May 2025 10:04:11 -0400 you wrote:

This patch series fixes two issues that cause false failures in the BPF verifier test suite when run with verbose output (-v).

The following tests fail only when running the test_verifier in verbose.

#458/p ld_dw: xor semi-random 64 bit imms, test 5 FAIL #494/p precise: test 1 FAIL
#495/p precise: test 2 FAIL
#497/p precise: ST zero to stack insn is supported FAIL #498/p precise: STX insn causing spi > allocated_stack FAIL #501/p scale: scale test 1 FAIL
#502/p scale: scale test 2 FAIL

Quoted 1 line

[...]

Here is the summary with links:
- [bpf-next,1/2] selftests/bpf: test_verifier verbose causes erroneous failures

    https://git.kernel.org/bpf/bpf-next/c/c5bcc8c78127
  - [bpf-next,2/2] selftests/bpf: test_verifier verbose log overflows
    https://git.kernel.org/bpf/bpf-next/c/af8a5125a04c

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html