Skip to contentSkip to conversation

bpf: add diagnostics for rejected memory and map accesses

PATCH In review Suchit Karunakaran 3 participants 4 messages Sep 27 – Sep 28

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 bpf: add diagnostics for rejected memory and map accesses

Emit structured verifier diagnostics when BPF programs are rejected for unsupported or prohibited memory operations. Cover sign-extending arena loads without JIT support, read/write access restrictions on maps, writes through read-only pointers, direct packet writes, and modifying helpers used with BPF_F_RDONLY_PROG.

---
Changes since v1:
- Modified the diagnostics messages to better explain the issues to the user

---
 kernel/bpf/fixups.c   |  6 ++++++
 kernel/bpf/verifier.c | 24 ++++++++++++++++++++++++
 2 files changed, 30 insertions(+)
kernel/bpf/fixups.c +6 -0
index e9c2d6c06218..4216b359bf28 100644
@@ -10,6 +10,7 @@
 #include <linux/perf_event.h>
 #include <linux/sched/signal.h>
 #include <net/xdp.h>
+#include "diagnostics.h"
 #include "disasm.h"
 
 #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
@@ -1055,6 +1056,11 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
 			if (BPF_MODE(insn->code) == BPF_MEMSX) {
 				if (!bpf_jit_supports_insn(insn, true)) {
 					verbose(env, "sign extending loads from arena are not supported yet\n");
+					bpf_diag_policy(
+						env, i + delta,
+						"sign-extending arena load",
+						"this JIT backend doesn't support it yet",
+						"Recompile the BPF program with Clang's -mcpu=v3 to avoid generating sign-extending load instructions.");
 					return -EOPNOTSUPP;
 				}
 				insn->code = BPF_CLASS(insn->code) | BPF_PROBE_MEM32SX | BPF_SIZE(insn->code);
kernel/bpf/verifier.c +24 -0
index 5c8626215ee6..bbe88c259fd0 100644
@@ -4472,12 +4472,22 @@ static int check_map_access_type(struct bpf_verifier_env *env, struct bpf_reg_st
 	if (type == BPF_WRITE && !(cap & BPF_MAP_CAN_WRITE)) {
 		verbose(env, "write into map forbidden, value_size=%d off=%lld size=%d\n",
 			map->value_size, reg_smin(reg) + off, size);
+		bpf_diag_policy(env, env->insn_idx,
+				bpf_diag_fmt(env, "write to map '%s'",
+					map->name[0] ? map->name : "unnamed"),
+				"this map was created with BPF_F_RDONLY_PROG, which allows BPF programs to only read it",
+				"Remove the write, or create the map without BPF_F_RDONLY_PROG if BPF programs need to write to it.");
 		return -EACCES;
 	}
 
 	if (type == BPF_READ && !(cap & BPF_MAP_CAN_READ)) {
 		verbose(env, "read from map forbidden, value_size=%d off=%lld size=%d\n",
 			map->value_size, reg_smin(reg) + off, size);
+		bpf_diag_policy(env, env->insn_idx,
+				bpf_diag_fmt(env, "read from map '%s'",
+					map->name[0] ? map->name : "unnamed"),
+				"this map was created with BPF_F_WRONLY_PROG, which allows BPF programs to only write it",
+				"Remove the read, or create the map without BPF_F_WRONLY_PROG if BPF programs need to read from it.");
 		return -EACCES;
 	}
 
@@ -6944,6 +6954,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
 		if (t == BPF_WRITE && rdonly_mem) {
 			verbose(env, "%s cannot write into %s\n",
 				reg_arg_name(env, argno), reg_type_str(env, reg->type));
+			bpf_diag_policy(env, insn_idx, "memory write",
+					bpf_diag_fmt(env, "%s points to read-only memory",
+						reg_arg_name(env, argno)),
+					"Use a writable destination, or copy the data into a writable buffer before modifying it.");
 			return -EACCES;
 		}
 
@@ -7032,6 +7046,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
 	} else if (reg_is_pkt_pointer(reg)) {
 		if (t == BPF_WRITE && !may_access_direct_pkt_data(env, NULL, t)) {
 			verbose(env, "cannot write into packet\n");
+			bpf_diag_policy(env, insn_idx,
+					"direct packet write",
+					"packet data is read-only through direct access for this program type",
+					"Remove the direct write, or perform the modification in a program type and hook that support packet writes.");
 			return -EACCES;
 		}
 		if (t == BPF_WRITE && value_regno >= 0 &&
@@ -11805,6 +11823,12 @@ record_func_map(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
 	     func_id == BPF_FUNC_map_push_elem ||
 	     func_id == BPF_FUNC_map_pop_elem)) {
 		verbose(env, "write into map forbidden\n");
+		bpf_diag_policy(env, insn_idx,
+				bpf_diag_fmt(env, "call to %s on map '%s'",
+					func_id_name(func_id),
+					map->name[0] ? map->name : "unnamed"),
+				"the map was created with BPF_F_RDONLY_PROG, which blocks modifications from BPF programs",
+				"Remove the modifying helper call, or use a map created without BPF_F_RDONLY_PROG if BPF programs need to modify it.");
 		return -EACCES;
 	}
 
Signature · 1 line
2.55.0
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] bpf: missing diagnostics for some read-only memory writes --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] bpf: missing diagnostics for some read-only memory writes
--

commit 184c7fe51457fa3d0ba8959d5aae6f21fcf5effc
Author: Suchit Karunakaran <[email protected]>

bpf: add diagnostics for rejected memory and map accesses

Emit structured verifier diagnostics when BPF programs are rejected for unsupported or prohibited memory operations. Cover sign-extending arena loads without JIT support, read/write access restrictions on maps, writes through read-only pointers, direct packet writes, and modifying helpers used with BPF_F_RDONLY_PROG.

Quoted 13 lines
kernel/bpf/verifier.c +4 -0
@@ -6449,6 +6459,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
 		if (t == BPF_WRITE && rdonly_mem) {
 			verbose(env, "%s cannot write into %s\n",
 				reg_arg_name(env, argno), reg_type_str(env, reg->type));
+			bpf_diag_policy(env, insn_idx, "memory write",
+					bpf_diag_fmt(env, "%s points to read-only memory",
+						reg_arg_name(env, argno)),
+					"Use a writable destination, or copy the data into a writable buffer before modifying it.");
 			return -EACCES;
 		}

[ ... ]

kernel/bpf/verifier.c static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b · line 6962

Quoted 6 lines
kernel/bpf/verifier.c +0 -0
 	} else if (type_is_sk_pointer(reg->type)) {
 		if (t == BPF_WRITE) {
 			verbose(env, "%s cannot write into %s\n",
 				reg_arg_name(env, argno), reg_type_str(env, reg->type));
 			return -EACCES;
 		}

[Severity: Medium]
Since the commit message mentions adding diagnostics for writes through read-only pointers, should bpf_diag_policy() also be called here for direct writes to read-only socket pointers in check_mem_access()?

[ ... ]

kernel/bpf/verifier.c

Quoted 11 lines
kernel/bpf/verifier.c +0 -0
 	} else if (base_type(reg->type) == PTR_TO_BUF &&
 		   !type_may_be_null(reg->type)) {
 		bool rdonly_mem = type_is_rdonly_mem(reg->type);
 		u32 *max_access;
 
 		if (rdonly_mem) {
 			if (t == BPF_WRITE) {
 				verbose(env, "%s cannot write into %s\n",
 					reg_arg_name(env, argno), reg_type_str(env, reg->type));
 				return -EACCES;
 			}

[Severity: Medium]
Does this direct write to a read-only PTR_TO_BUF also need the new bpf_diag_policy() diagnostic, similar to how it was added for PTR_TO_MEM earlier in check_mem_access()?

[ ... ]

Quoted 1 line
kernel/bpf/verifier.c +0 -0
@@ -7061,6 +7061,15 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,

[ ... ]

kernel/bpf/verifier.c static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b

Quoted 8 lines
kernel/bpf/verifier.c +0 -0
 	case PTR_TO_MEM:
 		if (type_is_rdonly_mem(reg->type)) {
 			if (access_type == BPF_WRITE) {
 				verbose(env, "%s cannot write into %s\n",
 					reg_arg_name(env, argno), reg_type_str(env, reg->type));
 				return -EACCES;
 			}
 		}

[Severity: Medium]
If a BPF program passes a read-only PTR_TO_MEM to a helper function that expects a writable buffer, the access is rejected here in check_helper_mem_access(). Should this path also emit the structured diagnostic?

kernel/bpf/verifier.c

Quoted 10 lines
kernel/bpf/verifier.c +0 -0
 		return check_mem_region_access(env, reg, argno, 0,
 					       access_size, reg->mem_size,
 					       zero_size_allowed);
 	case PTR_TO_BUF:
 		if (type_is_rdonly_mem(reg->type)) {
 			if (access_type == BPF_WRITE) {
 				verbose(env, "%s cannot write into %s\n",
 					reg_arg_name(env, argno), reg_type_str(env, reg->type));
 				return -EACCES;
 			}

[Severity: Medium]
Similarly, does the rejection of helper function writes to read-only PTR_TO_BUF pointers in check_helper_mem_access() need a bpf_diag_policy() call?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#2 lore raw
In reply to Suchit Karunakaran

kernel/bpf/verifier.c static int check_map_access_type(struct bpf_verifier_env *env, struct bpf_reg_st · line 4479

Quoted 9 lines · Suchit Karunakaran
kernel/bpf/verifier.c +5 -0
@@ -4472,12 +4472,22 @@ static int check_map_access_type(struct bpf_verifier_env *env, struct bpf_reg_st
 	if (type == BPF_WRITE && !(cap & BPF_MAP_CAN_WRITE)) {
 		verbose(env, "write into map forbidden, value_size=%d off=%lld size=%d\n",
 			map->value_size, reg_smin(reg) + off, size);
+		bpf_diag_policy(env, env->insn_idx,
+				bpf_diag_fmt(env, "write to map '%s'",
+					map->name[0] ? map->name : "unnamed"),
+				"this map was created with BPF_F_RDONLY_PROG, which allows BPF programs to only read it",
+				"Remove the write, or create the map without BPF_F_RDONLY_PROG if BPF programs need to write to it.");

That's not true.
dev_map_init_map() and insn_array_alloc() set BPF_F_RDONLY_PROG in the kernel for every devmap and insn_array.
libbpf sets it for .rodata when the prog has a const global. The user didn't create the map with that flag and cannot create it without.
Same in record_func_map().

[...]

Quoted 2 lines
kernel/bpf/verifier.c +1 -0
@@ -6944,6 +6954,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
+					"Use a writable destination, or copy the data into a writable buffer before modifying it.");

[...]

kernel/bpf/verifier.c static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b · line 7046

Quoted 2 lines
kernel/bpf/verifier.c +1 -0
@@ -7032,6 +7046,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
+					"Remove the direct write, or perform the modification in a program type and hook that support packet writes.");

These two are the same as in v1 and don't tell the user anything.

Pls focus your tokens elsewhere. I don't feel we will converge here.

pw-bot: cr

#3 lore raw

Hi Alexei. Sorry for the inconvenience, I didn't mean to ignore your feedback. Please let me know if the following suggestions align with your expectations. Of course, these are loose ideas, just to help me understand how to frame the suggestions as I'm a bit clueless.

Quoted 12 lines · Alexei Starovoitov

kernel/bpf/verifier.c static int check_map_access_type(struct bpf_verifier_env *env, struct bpf_reg_st · line 4479

Quoted 9 lines · Suchit Karunakaran
kernel/bpf/verifier.c +5 -0
@@ -4472,12 +4472,22 @@ static int check_map_access_type(struct bpf_verifier_env *env, struct bpf_reg_st
      if (type == BPF_WRITE && !(cap & BPF_MAP_CAN_WRITE)) {
              verbose(env, "write into map forbidden, value_size=%d off=%lld size=%d\n",
                      map->value_size, reg_smin(reg) + off, size);
+             bpf_diag_policy(env, env->insn_idx,
+                             bpf_diag_fmt(env, "write to map '%s'",
+                                     map->name[0] ? map->name : "unnamed"),
+                             "this map was created with BPF_F_RDONLY_PROG, which allows BPF programs to only read it",
+                             "Remove the write, or create the map without BPF_F_RDONLY_PROG if BPF programs need to write to it.");
Quoted 8 lines

That's not true.
dev_map_init_map() and insn_array_alloc() set BPF_F_RDONLY_PROG in the kernel for every devmap and insn_array.
libbpf sets it for .rodata when the prog has a const global. The user didn't create the map with that flag and cannot create it without.
Same in record_func_map().

I'm sorry I wasn't aware of this. I wrote the suggestion based on the fact that BPF_F_RDONLY_PROG and BPF_F_WRONLY_PROG cannot coexist. How about something like below?
"Move the map updates and deletions to userspace using bpf_map_update_elem() and bpf_map_delete_elem() if the map and program types support it". Or maybe I'll try to write suggestions based on the map type.

Quoted 7 lines

[...]

Quoted 2 lines
kernel/bpf/verifier.c +1 -0
@@ -6944,6 +6954,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
+                                     "Use a writable destination, or copy the data into a writable buffer before modifying it.");

[...]

Quoted 7 lines

kernel/bpf/verifier.c static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b · line 7046

Quoted 2 lines
kernel/bpf/verifier.c +1 -0
@@ -7032,6 +7046,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, struct b
+                                     "Remove the direct write, or perform the modification in a program type and hook that support packet writes.");

These two are the same as in v1 and don't tell the user anything.

Pls focus your tokens elsewhere. I don't feel we will converge here.

For direct packet writes: "Move packet modification to a TC classifier (BPF_PROG_TYPE_SCHED_CLS) attached at ingress or egress or an XDP program attached at ingress. For lightweight tunnels, use the transmit hook '(BPF_PROG_TYPE_LWT_XMIT)'"
For read-only memory access: "Copy the data into a local stack variable and perform modifications on the new variable." Does this direction feel right, or would you like the suggestions to be even more specific?

I’ll think harder before sending the next revision. I’ll also split the patch so that each commit adds diagnostics for related issues and I missed some places, as the AI suggested.