Skip to contentSkip to conversation

selftests/bpf: allow requesting log level 2 in test_verifier

PATCH Applied Andrii Nakryiko 4 participants 6 messages Oct 5, 2022 – Oct 6, 2022

Subsystems (3)
lore
Share & tools
lore

Andrii Nakryiko

author maintainer
#0 lore raw

PATCH 1/3 selftests/bpf: allow requesting log level 2 in test_verifier

Log level 1 on successfully verified programs are basically equivalent to log level 4 (stats-only), so it's useful to be able to request more verbose logs at log level 2. Teach test_verifier to recognize -vv as "very verbose" mode switch and use log level 2 in such mode.

Also force verifier stats regradless of -v or -vv, they are very minimal and useful to be always emitted in verbose mode.

---
 tools/testing/selftests/bpf/test_verifier.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/test_verifier.c +10 -3
index 2dbcbf363c18..9c7091100b7f 100644
@@ -68,7 +68,6 @@
 #define SKIP_INSNS()	BPF_RAW_INSN(0xde, 0xa, 0xd, 0xbeef, 0xdeadbeef)
 
 #define DEFAULT_LIBBPF_LOG_LEVEL	4
-#define VERBOSE_LIBBPF_LOG_LEVEL	1
 
 #define F_NEEDS_EFFICIENT_UNALIGNED_ACCESS	(1 << 0)
 #define F_LOAD_WITH_STRICT_ALIGNMENT		(1 << 1)
@@ -81,6 +80,7 @@
 static bool unpriv_disabled = false;
 static int skips;
 static bool verbose = false;
+static int verif_log_level = 0;
 
 struct kfunc_btf_id_pair {
 	const char *kfunc;
@@ -759,7 +759,7 @@ static int load_btf_spec(__u32 *types, int types_len,
 		    .log_buf = bpf_vlog,
 		    .log_size = sizeof(bpf_vlog),
 		    .log_level = (verbose
-				  ? VERBOSE_LIBBPF_LOG_LEVEL
+				  ? verif_log_level
 				  : DEFAULT_LIBBPF_LOG_LEVEL),
 	);
 
@@ -1491,7 +1491,7 @@ static void do_test_single(struct bpf_test *test, bool unpriv,
 
 	opts.expected_attach_type = test->expected_attach_type;
 	if (verbose)
-		opts.log_level = VERBOSE_LIBBPF_LOG_LEVEL;
+		opts.log_level = verif_log_level | 4; /* force stats */
 	else if (expected_ret == VERBOSE_ACCEPT)
 		opts.log_level = 2;
 	else
@@ -1746,6 +1746,13 @@ int main(int argc, char **argv)
 	if (argc > 1 && strcmp(argv[1], "-v") == 0) {
 		arg++;
 		verbose = true;
+		verif_log_level = 1;
+		argc--;
+	}
+	if (argc > 1 && strcmp(argv[1], "-vv") == 0) {
+		arg++;
+		verbose = true;
+		verif_log_level = 2;
 		argc--;
 	}
 
Signature · 1 line
2.30.2
author maintainer
#1 lore raw

PATCH 2/3 selftests/bpf: avoid reporting +100% difference in veristat for actual 0%

In special case when both base and comparison values are 0, veristat currently reports "+0 (+100%)" difference, which is quite confusing. Fix it up to be "+0 (+0%)".

---
 tools/testing/selftests/bpf/veristat.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/veristat.c +8 -4
index b0d83a28e348..38f678122a7d 100644
@@ -1104,17 +1104,21 @@ static void output_comp_stats(const struct verif_stats *base, const struct verif
 			else
 				snprintf(diff_buf, sizeof(diff_buf), "%s", "MISMATCH");
 		} else {
+			double p = 0.0;
+
 			snprintf(base_buf, sizeof(base_buf), "%ld", base_val);
 			snprintf(comp_buf, sizeof(comp_buf), "%ld", comp_val);
 
 			diff_val = comp_val - base_val;
 			if (base == &fallback_stats || comp == &fallback_stats || base_val == 0) {
-				snprintf(diff_buf, sizeof(diff_buf), "%+ld (%+.2lf%%)",
-					 diff_val, comp_val < base_val ? -100.0 : 100.0);
+				if (comp_val == base_val)
+					p = 0.0; /* avoid +0 (+100%) case */
+				else
+					p = comp_val < base_val ? -100.0 : 100.0;
 			} else {
-				snprintf(diff_buf, sizeof(diff_buf), "%+ld (%+.2lf%%)",
-					 diff_val, diff_val * 100.0 / base_val);
+				 p = diff_val * 100.0 / base_val;
 			}
+			snprintf(diff_buf, sizeof(diff_buf), "%+ld (%+.2lf%%)", diff_val, p);
 		}
 
 		switch (fmt) {
Signature · 1 line
2.30.2
author maintainer
#2 lore raw

PATCH 3/3 selftests/bpf: add BPF object fixup step to veristat

In reply to Andrii Nakryiko

Add a step to attempt to "fix up" BPF object file to make it possible to successfully load it. E.g., set non-zero size for BPF maps that expect max_entries set, but BPF object file itself doesn't have declarative max_entries values specified.

Another issue was with automatic map pinning. Pinning has no effect on BPF verification process itself but can interfere when validating multiple related programs and object files, so veristat disabled all the pinning explicitly.

In the future more such fix up heuristics could be added to accommodate common patterns encountered in practice.

---
 tools/testing/selftests/bpf/veristat.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
tools/testing/selftests/bpf/veristat.c +25 -0
index 38f678122a7d..973cbf6af323 100644
@@ -509,6 +509,28 @@ static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats *
 	return 0;
 }
 
+static void fixup_obj(struct bpf_object *obj)
+{
+	struct bpf_map *map;
+
+	bpf_object__for_each_map(map, obj) {
+		/* disable pinning */
+		bpf_map__set_pin_path(map, NULL);
+
+		/* fix up map size, if necessary */
+		switch (bpf_map__type(map)) {
+		case BPF_MAP_TYPE_SK_STORAGE:
+		case BPF_MAP_TYPE_TASK_STORAGE:
+		case BPF_MAP_TYPE_INODE_STORAGE:
+		case BPF_MAP_TYPE_CGROUP_STORAGE:
+			break;
+		default:
+			if (bpf_map__max_entries(map) == 0)
+				bpf_map__set_max_entries(map, 1);
+		}
+	}
+}
+
 static int process_prog(const char *filename, struct bpf_object *obj, struct bpf_program *prog)
 {
 	const char *prog_name = bpf_program__name(prog);
@@ -543,6 +565,9 @@ static int process_prog(const char *filename, struct bpf_object *obj, struct bpf
 	}
 	verif_log_buf[0] = '\0';
 
+	/* increase chances of successful BPF object loading */
+	fixup_obj(obj);
+
 	err = bpf_object__load(obj);
 	env.progs_processed++;
 
Signature · 1 line
2.30.2
#3 lore raw

Re: [PATCH bpf-next 3/3] selftests/bpf: add BPF object fixup step to veristat

Quoted 35 lines · Andrii Nakryiko

Add a step to attempt to "fix up" BPF object file to make it possible to successfully load it. E.g., set non-zero size for BPF maps that expect max_entries set, but BPF object file itself doesn't have declarative max_entries values specified.

Another issue was with automatic map pinning. Pinning has no effect on BPF verification process itself but can interfere when validating multiple related programs and object files, so veristat disabled all the pinning explicitly.

In the future more such fix up heuristics could be added to accommodate common patterns encountered in practice.

---
 tools/testing/selftests/bpf/veristat.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
tools/testing/selftests/bpf/veristat.c +8 -0
index 38f678122a7d..973cbf6af323 100644
@@ -509,6 +509,28 @@ static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats *
        return 0;
 }
 
+static void fixup_obj(struct bpf_object *obj)
+{
+       struct bpf_map *map;
+
+       bpf_object__for_each_map(map, obj) {
+               /* disable pinning */
+               bpf_map__set_pin_path(map, NULL);
+

tools/testing/selftests/bpf/veristat.c static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats * · line 529

Quoted 10 lines
tools/testing/selftests/bpf/veristat.c +10 -0
+               /* fix up map size, if necessary */
+               switch (bpf_map__type(map)) {
+               case BPF_MAP_TYPE_SK_STORAGE:
+               case BPF_MAP_TYPE_TASK_STORAGE:
+               case BPF_MAP_TYPE_INODE_STORAGE:
+               case BPF_MAP_TYPE_CGROUP_STORAGE:
+                       break;
+               default:
+                       if (bpf_map__max_entries(map) == 0)
+                               bpf_map__set_max_entries(map, 1);

Should we drop if (==0) check and set max_entries=1 unconditionally to save memory and reduce map creation time ?
since max_entries doesn't affect verifiability.

Applied the set for now.

#4 lore raw

Re: [PATCH bpf-next 3/3] selftests/bpf: add BPF object fixup step to veristat

Quoted 43 lines · Alexei Starovoitov
Quoted 41 lines · Andrii Nakryiko

Add a step to attempt to "fix up" BPF object file to make it possible to successfully load it. E.g., set non-zero size for BPF maps that expect max_entries set, but BPF object file itself doesn't have declarative max_entries values specified.

Another issue was with automatic map pinning. Pinning has no effect on BPF verification process itself but can interfere when validating multiple related programs and object files, so veristat disabled all the pinning explicitly.

In the future more such fix up heuristics could be added to accommodate common patterns encountered in practice.

---
 tools/testing/selftests/bpf/veristat.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)
tools/testing/selftests/bpf/veristat.c +14 -0
index 38f678122a7d..973cbf6af323 100644
@@ -509,6 +509,28 @@ static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats *
        return 0;
 }
 
+static void fixup_obj(struct bpf_object *obj)
+{
+       struct bpf_map *map;
+
+       bpf_object__for_each_map(map, obj) {
+               /* disable pinning */
+               bpf_map__set_pin_path(map, NULL);
+
+               /* fix up map size, if necessary */
+               switch (bpf_map__type(map)) {
+               case BPF_MAP_TYPE_SK_STORAGE:
+               case BPF_MAP_TYPE_TASK_STORAGE:
+               case BPF_MAP_TYPE_INODE_STORAGE:
+               case BPF_MAP_TYPE_CGROUP_STORAGE:
Quoted 8 lines

tools/testing/selftests/bpf/veristat.c static int parse_verif_log(char * const buf, size_t buf_sz, struct verif_stats * · line 529

Quoted 4 lines
tools/testing/selftests/bpf/veristat.c +4 -0
+                       break;
+               default:
+                       if (bpf_map__max_entries(map) == 0)
+                               bpf_map__set_max_entries(map, 1);

Should we drop if (==0) check and set max_entries=1 unconditionally to save memory and reduce map creation time ?
since max_entries doesn't affect verifiability.

This might break the map-in-map case, I think? I see xsk_map_meta_equal() takes into account max_entries, and array_map_meta_equal() also check max_entries equality unless BPF_F_INNER_MAP is specified. So in some cases valid apps won't load correctly.

Given veristat loads one object at a time, hopefully memory usage won't be a big issue in practice. It seems safer and simpler to keep it as is.

Quoted 2 lines

Applied the set for now.

#5 lore raw
In reply to Andrii Nakryiko
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 9 lines · On Wed, 5 Oct 2022 09:14:48 -0700 you wrote:

Log level 1 on successfully verified programs are basically equivalent to log level 4 (stats-only), so it's useful to be able to request more verbose logs at log level 2. Teach test_verifier to recognize -vv as "very verbose" mode switch and use log level 2 in such mode.

Also force verifier stats regradless of -v or -vv, they are very minimal and useful to be always emitted in verbose mode.

[...]

Here is the summary with links:
- [bpf-next,1/3] selftests/bpf: allow requesting log level 2 in test_verifier

    https://git.kernel.org/bpf/bpf-next/c/2a72f5951ac6
  - [bpf-next,2/3] selftests/bpf: avoid reporting +100% difference in veristat for actual 0%
    https://git.kernel.org/bpf/bpf-next/c/6df2eb45e378
  - [bpf-next,3/3] selftests/bpf: add BPF object fixup step to veristat
    https://git.kernel.org/bpf/bpf-next/c/60df8c4d32d4

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html