Skip to contentSkip to conversation

Misc BPF gotox fixes

PATCH Superseded Daniel Borkmann 4 participants 29 messages Sep 14 – Sep 15

Subsystems (4)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Daniel Borkmann

author maintainer
#0 lore raw

v2 cover Misc BPF gotox fixes

Address various open issues around gotox instruction, see individual commits for details. Also BPF selftests are added to assert that these are addressed.

v1 -> v2:
- Remove jt_spans_subprogs and reject right away (Eduard)
- Rework jump table to no longer keep it in insn_aux_data (Eduard)
- Fix up comments in bpf_compute_scc (Eduard)
- Fix up commit message nit wrt signed_loader (Anton)
- Added 3 more fixes related to gotox along with BPF selftests

Daniel Borkmann (11):
bpf: Avoid quadratic successor rescan in bpf_compute_scc
bpf: Bound the number of indirect jump edges in a program
bpf: Cache the jump table of a subprogram during CFG discovery
bpf: Reject indirect jumps that leave their subprogram
bpf: Don't remove indirect jump targets in the nop removal pass
bpf: Clear insn array jitted pointers on map reuse
bpf: Retarget indirect jump targets across prologue prepends selftests/bpf: Add tests for the indirect jump edge limit selftests/bpf: Add tests for indirect jumps across subprograms selftests/bpf: Add tests for gotox nop targets and insn array reuse selftests/bpf: Add tests for gotox targets across prologue prepends

 include/linux/bpf.h                           |    2 +-
 include/linux/bpf_verifier.h                  |    6 +-
 kernel/bpf/bpf_insn_array.c                   |   16 +-
 kernel/bpf/cfg.c                              |  208 ++--
 kernel/bpf/fixups.c                           |   63 +-
 kernel/bpf/liveness.c                         |   18 +-
 kernel/bpf/verifier.c                         |   58 +-
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 1061 ++++++++++++++++-
 .../selftests/bpf/progs/verifier_gotox.c      |    2 +-
 9 files changed, 1276 insertions(+), 158 deletions(-)
Signature · 1 line
2.43.0
author maintainer
#1 lore raw

PATCH v2 1/11 bpf: Avoid quadratic successor rescan in bpf_compute_scc

The iterative Tarjan DFS in bpf_compute_scc() emulates recursion with an explicit 'dfs' stack: when a successor has not been visited yet, the successor is pushed and the walk restarts at the top of the loop. On the way back to a vertex the successor scan starts over at index zero, so a vertex with k successors rescans up to k successors on each of its up to k descents, i.e. O(k^2) work.

For ordinary instructions k <= 2 and this is irrelevant. For a gotox the successors are the jump table of the containing subprogram, whose size is bounded only by the max_entries of the insn_array map, so k can reach the 1M instruction complexity limit. Loading such a program keeps a CPU busy in the loop for a very long time before verification even begins.

Record in 'dfs_pos' the successor index each frame stopped at and resume the scan there. Each edge is therefore examined a bounded number of times and the walk becomes linear in the number of edges.

Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
---
 kernel/bpf/cfg.c | 26 ++++++++++++++++++++------
 1 file changed, 20 insertions(+), 6 deletions(-)
kernel/bpf/cfg.c +20 -6
index 842c7d1eabcc..5cfd02c42afd 100644
@@ -749,7 +749,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	struct bpf_insn_aux_data *aux = env->insn_aux_data;
 	const u32 insn_cnt = env->prog->len;
 	int stack_sz, dfs_sz, err = 0;
-	u32 *stack, *pre, *low, *dfs;
+	u32 *stack, *pre, *low, *dfs, *dfs_pos;
 	u32 i, j, t, w;
 	u32 next_preorder_num;
 	u32 next_scc_id;
@@ -762,13 +762,16 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	 * - 'stack' accumulates vertices in DFS order, see invariant comment below;
 	 * - 'pre[t] == p' => preorder number of vertex 't' is 'p';
 	 * - 'low[t] == n' => smallest preorder number of the vertex reachable from 't' is 'n';
-	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion.
+	 * - 'dfs' DFS traversal stack, used to emulate explicit recursion;
+	 * - 'dfs_pos[k] == j' => the frame 'dfs[k]' resumes visiting its
+	 *   successors at index 'j'.
 	 */
 	stack = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	pre = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	low = kvcalloc(insn_cnt, sizeof(int), GFP_KERNEL_ACCOUNT);
 	dfs = kvcalloc(insn_cnt, sizeof(*dfs), GFP_KERNEL_ACCOUNT);
-	if (!stack || !pre || !low || !dfs) {
+	dfs_pos = kvcalloc(insn_cnt, sizeof(*dfs_pos), GFP_KERNEL_ACCOUNT);
+	if (!stack || !pre || !low || !dfs || !dfs_pos) {
 		err = -ENOMEM;
 		goto exit;
 	}
@@ -851,6 +854,7 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 		stack_sz = 0;
 		dfs_sz = 1;
 		dfs[0] = i;
+		dfs_pos[0] = 0;
 dfs_continue:
 		while (dfs_sz) {
 			w = dfs[dfs_sz - 1];
@@ -860,13 +864,22 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 				next_preorder_num++;
 				stack[stack_sz++] = w;
 			}
-			/* Visit 'w' successors */
+			/* Visit remaining 'w' successors */
 			succ = bpf_insn_successors(env, w);
-			for (j = 0; j < succ->cnt; ++j) {
+			for (j = dfs_pos[dfs_sz - 1]; j < succ->cnt; ++j) {
 				if (pre[succ->items[j]]) {
 					low[w] = min(low[w], low[succ->items[j]]);
 				} else {
-					dfs[dfs_sz++] = succ->items[j];
+					/*
+					 * Once DFS for succ->items[j] is complete,
+					 * pre[succ->items[j]] is non-zero, hence
+					 * resuming at 'j' allows to follow the
+					 * low[w] = min(...) update branch above.
+					 */
+					dfs_pos[dfs_sz - 1] = j;
+					dfs_pos[dfs_sz] = 0;
+					dfs[dfs_sz] = succ->items[j];
+					dfs_sz++;
 					goto dfs_continue;
 				}
 			}
@@ -916,5 +929,6 @@ int bpf_compute_scc(struct bpf_verifier_env *env)
 	kvfree(pre);
 	kvfree(low);
 	kvfree(dfs);
+	kvfree(dfs_pos);
 	return err;
 }
Signature · 1 line
2.43.0
author maintainer
#2 lore raw

PATCH v2 2/11 bpf: Bound the number of indirect jump edges in a program

In reply to Daniel Borkmann

Every gotox instruction gets its own copy of the jump table of the subprog containing it, and each distinct target in that table is a CFG successor of the instruction. The number of such edges is therefore the number of gotox instructions times the number of distinct targets, and neither factor is bounded by anything except the instruction limit.

What is expensive is a BPF prog whose gotox instructions are themselves the targets, which makes the edge count quadratic. 1024 such gotox are already ~1e6 edges and about 4s of CPU to load.

Bound the total across the program at BPF_COMPLEXITY_LIMIT_INSNS, aka the limit as the number of instructions the verifier processes. Progs with real switch statements are orders of magnitude below this.

Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
---
 include/linux/bpf_verifier.h |  1 +
 kernel/bpf/cfg.c             | 14 ++++++++++++++
 2 files changed, 15 insertions(+)
include/linux/bpf_verifier.h +1 -0
index 36b65797877d..04bb8f71cabe 100644
@@ -977,6 +977,7 @@ struct bpf_verifier_env {
 		int cur_stack;
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
+		u32 gotox_edges;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
kernel/bpf/cfg.c +14 -0
index 5cfd02c42afd..df4513e71c80 100644
@@ -9,6 +9,8 @@
 
 #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
 
+#define BPF_MAX_GOTOX_EDGES	BPF_COMPLEXITY_LIMIT_INSNS
+
 /* non-recursive DFS pseudo code
  * 1  procedure DFS-iterative(G,v):
  * 2      label v as discovered
@@ -388,6 +390,18 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 			return PTR_ERR(jt);
 
 		env->insn_aux_data[t].jt = jt;
+
+		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+				       &env->cfg.gotox_edges) ||
+		    env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+			verbose(env, "number of indirect jump edges in the program exceeds %u\n",
+				BPF_MAX_GOTOX_EDGES);
+			bpf_diag_program_structure(env, t, "too many indirect jump edges",
+				"Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+				"The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+				BPF_MAX_GOTOX_EDGES);
+			return -E2BIG;
+		}
 	}
 
 	mark_prune_point(env, t);
Signature · 1 line
2.43.0
#3 lore raw

Re: [PATCH bpf v2 02/11] bpf: Bound the number of indirect jump edges in a program

Does this bound prevent the load-time DoS the commit message describes?
Quoted 34 lines
include/linux/bpf_verifier.h +1 -0
index 36b65797877d0..04bb8f71cabe3 100644
@@ -977,6 +977,7 @@ struct bpf_verifier_env {
 		int cur_stack;
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
+		u32 gotox_edges;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
kernel/bpf/cfg.c +7 -0
index 5cfd02c42afd5..df4513e71c80a 100644
@@ -9,6 +9,8 @@
 
 #define verbose(env, fmt, args...) bpf_verifier_log_write(env, fmt, ##args)
 
+#define BPF_MAX_GOTOX_EDGES	BPF_COMPLEXITY_LIMIT_INSNS
+
 /* non-recursive DFS pseudo code
  * 1  procedure DFS-iterative(G,v):
  * 2      label v as discovered
@@ -388,6 +390,18 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
 			return PTR_ERR(jt);
 
 		env->insn_aux_data[t].jt = jt;
+
+		if (check_add_overflow(env->cfg.gotox_edges, jt->cnt,
+				       &env->cfg.gotox_edges) ||
+		    env->cfg.gotox_edges > BPF_MAX_GOTOX_EDGES) {
+			verbose(env, "number of indirect jump edges in the program exceeds %u\n",

kernel/bpf/cfg.c static int visit_gotox_insn(int t, struct bpf_verifier_env *env) · line 407

Quoted 10 lines
kernel/bpf/cfg.c +7 -0
+				BPF_MAX_GOTOX_EDGES);
+			bpf_diag_program_structure(env, t, "too many indirect jump edges",
+				"Reduce the number of indirect jumps, or the number of distinct targets they can reach.",
+				"The program has more than %u indirect jump edges in total, counted over every gotox instruction.",
+				BPF_MAX_GOTOX_EDGES);
+			return -E2BIG;
+		}
 	}
 
 	mark_prune_point(env, t);

Does this bound prevent the load-time DoS the commit message describes?

The accounting itself is correct: the counter increments only inside the if (!jt) block, so each gotox contributes jt->cnt exactly once, and the counter is program-wide across both walk_cfg passes.

However, gotox_edges counts the CFG edges added, not the work that check_cfg() performs. The expensive operations happen before this check:

visit_gotox_insn() -> create_jt() -> jt_from_subprog()

jt_from_subprog() loops over every insn_array map in the program, and for each map it allocates map->max_entries u32s and calls bpf_copy_insn_array_uniq(), which does one map_lookup_elem() per entry and then sorts them. This runs once per gotox instruction.

The edges added per gotox are only the distinct targets that survive the sort and dedup, which can be 1 or 2 no matter how large the maps are.

So the cost is O(gotox_count * total_map_entries * log) while the limit constrains gotox_count * distinct_targets. A program can sit at a small fraction of BPF_MAX_GOTOX_EDGES and still take arbitrarily long to reject.

The commit message states "1024 such gotox are already ~1e6 edges and about 4s of CPU to load" and "Progs with real switch statements are orders of magnitude below this", which reads as if the 1e6-edge bound caps load time in the seconds range. It does not.

This is confirmed by the next patch in the series, b94c1f057586 ("bpf: Cache the jump table of a subprogram during CFG discovery"), which measures with this bound already applied: "A program of 4003 instructions with 2000 gotox and one 500k entry map holding two distinct targets has 4000 indirect jump edges, 0.4% of the limit, and takes 343s to be rejected. [...] At the insn limit, with a single 1M entry map, the same shape extrapolates to 41 hours." That patch carries the edge accounting over verbatim and instead builds each subprogram's jump table once, which makes check_cfg linear.

For the unprivileged load path, bpf_check() holds bpf_verifier_lock, so a single such program stalls all other non-privileged program loads for the duration.

Given the Fixes: tag and Reported-by:, should the commit message clarify that this patch bounds the edge count only, and that the per-gotox jump-table construction cost is addressed separately in the next patch? Otherwise the series might be read as closing the DoS at this commit.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

author maintainer
#4 lore raw

PATCH v2 3/11 bpf: Cache the jump table of a subprogram during CFG discovery

In reply to Daniel Borkmann

create_jt() builds the jump table of the subprogram containing a gotox by copying out and sorting every insn_array map of the program, and it does so once per gotox instruction. The cost is therefore the number of gotox instructions times the number of entries in all of the maps. A program of 4003 instructions with 2000 gotox and one 500k entry map holding two distinct targets has 4000 indirect jump edges, 0.4% of the limit, and takes 343s to be rejected. The map costs next to nothing to prepare, as an unset entry is already a valid target. At the insn limit, with a single 1M entry map, the same shape extrapolates to 41 hours.

All gotox instructions of a subprogram share the same jump table, so build the table of every subprogram in a single pass over the maps and let every gotox use it in place: bpf_insn_successors() resolves a gotox through its containing subprogram, and the tables are kept until the verifier environment is torn down, so nothing is copied into the instruction aux data. The edge accounting in visit_gotox_insn() moves to the first visit of the gotox, marked by the BRANCH bit of its CFG state.

The only user left of insn_aux_data[].jt is then the two entry table which visit_abnormal_return_insn() allocates for tail_call and ld_{abs,ind} insns, so that bpf_insn_successors() reports the hidden exit from their subprogram. Both are recognisable by their opcode, so derive that edge in bpf_insn_successors() from the exit_idx of the containing subprogram instead and drop the field along with bpf_clear_insn_aux_data(). The instruction aux data then owns no allocation, thus nothing needs to be freed when insns are removed or the verifier environment is torn down.

check_cfg() is then linear in the number of map entries plus the number of indirect jump edges, so what still scales now with the program is what BPF_MAX_GOTOX_EDGES bounds:

gotox map entries edges before after
----------------------------------------------

    500       250000   1000     35.34s     0.07s
   1000       250000   2000     82.00s     0.07s
   2000       250000   4000    148.62s     0.07s
   2000       125000   4000     74.54s     0.05s
   2000       500000   4000    342.71s     0.19s
Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
---
 include/linux/bpf_verifier.h |   5 +-
 kernel/bpf/cfg.c             | 143 ++++++++++++++++-------------------
 kernel/bpf/fixups.c          |  19 -----
 kernel/bpf/liveness.c        |  18 ++++-
 kernel/bpf/verifier.c        |   3 +-
 5 files changed, 85 insertions(+), 103 deletions(-)
include/linux/bpf_verifier.h +3 -2
index 04bb8f71cabe..754e24d41f47 100644
@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
 		/* remember the offset of node field within type to rewrite */
 		u64 insert_off;
 	};
-	struct bpf_iarray *jt;	/* jump table for gotox or bpf_tailcall call instruction */
 	struct btf_struct_meta *kptr_struct_meta;
 	u64 map_key_state; /* constant (32 bit) key tracking for maps */
 	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */
@@ -805,6 +804,7 @@ struct bpf_subprog_info {
 	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
 	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
 	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
 	u16 stack_depth; /* max. stack depth used by this function */
 	u16 stack_extra;
 	u32 insns_total;
@@ -978,6 +978,7 @@ struct bpf_verifier_env {
 		/* current position in the insn_postorder vector */
 		int cur_postorder;
 		u32 gotox_edges;
+		bool subprog_jts_ready;
 	} cfg;
 	struct backtrack_state bt;
 	struct bpf_jmp_history_entry *cur_hist_ent;
@@ -1502,6 +1503,7 @@ bool bpf_is_throw_kfunc(struct bpf_insn *insn);
 int bpf_compute_const_regs(struct bpf_verifier_env *env);
 int bpf_prune_dead_branches(struct bpf_verifier_env *env);
 int bpf_check_cfg(struct bpf_verifier_env *env);
+void bpf_free_subprog_jts(struct bpf_verifier_env *env);
 int bpf_compute_postorder(struct bpf_verifier_env *env);
 int bpf_compute_scc(struct bpf_verifier_env *env);
 
@@ -1703,7 +1705,6 @@ struct bpf_kfunc_desc_tab {
 };
 
 /* Functions exported from verifier.c, used by fixups.c */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len);
 void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog);
 bool bpf_allow_tail_call_in_subprogs(struct bpf_verifier_env *env);
 bool bpf_verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm);
kernel/bpf/cfg.c +66 -77

211 lines not on this page · raw message

kernel/bpf/fixups.c +0 -19
index 52d3cec33672..f3cf66b8147a 100644
@@ -500,23 +500,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
 	return 0;
 }
 
-/*
- * Clean up dynamically allocated fields of aux data for instructions [start, ...]
- */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
-{
-	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
-	int end = start + len;
-	int i;
-
-	for (i = start; i < end; i++) {
-		if (aux_data[i].jt) {
-			kvfree(aux_data[i].jt);
-			aux_data[i].jt = NULL;
-		}
-	}
-}
-
 static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 {
 	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
@@ -526,8 +509,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 	if (bpf_prog_is_offloaded(env->prog->aux))
 		bpf_prog_offload_remove_insns(env, off, cnt);
 
-	bpf_clear_insn_aux_data(env, off, cnt);
-
 	err = bpf_remove_insns(env->prog, off, cnt);
 	if (err)
 		return err;
kernel/bpf/liveness.c +15 -3
index 301fc60bddc4..08592fdb5510 100644
@@ -248,9 +248,12 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
 	struct bpf_iarray *succ, *jt;
 	int insn_sz;
 
-	jt = env->insn_aux_data[idx].jt;
-	if (unlikely(jt))
-		return jt;
+	/* All gotox of a subprogram share its jump table, see compute_subprog_jts() */
+	if (unlikely(insn_is_gotox(insn))) {
+		jt = bpf_find_containing_subprog(env, idx)->jt;
+		if (jt)
+			return jt;
+	}
 
 	/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */
 	succ = env->succ;
@@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
 	if (opcode_info->can_jump)
 		succ->items[succ->cnt++] = idx + bpf_jmp_offset(insn) + 1;
 
+	/*
+	 * tail_call upon success and ld_{abs,ind} upon load failure return
+	 * from the subprogram, which is a hidden edge to its exit.
+	 */
+	if (unlikely((bpf_helper_call(insn) && insn->imm == BPF_FUNC_tail_call) ||
+		     (BPF_CLASS(insn->code) == BPF_LD &&
+		      (BPF_MODE(insn->code) == BPF_ABS || BPF_MODE(insn->code) == BPF_IND))))
+		succ->items[succ->cnt++] = bpf_find_containing_subprog(env, idx)->exit_idx;
+
 	return succ;
 }
 
kernel/bpf/verifier.c +1 -2
index ddba53eaa333..32157d145a60 100644
@@ -21477,8 +21477,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
 	release_maps(env);
 	release_btfs(env);
 err_free_env:
-	if (env->insn_aux_data)
-		bpf_clear_insn_aux_data(env, 0, env->insn_aux_data_len);
+	bpf_free_subprog_jts(env);
 	vfree(env->insn_aux_data);
 	kvfree(env->fd_array);
 	bpf_stack_liveness_free(env);
Signature · 1 line
2.43.0
#5 lore raw

Re: [PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

The patch moves jt ownership into bpf_subprog_info but doesn't update the code that removes instructions. When verifier_remove_insns() deletes a whole subprogram…
Quoted 13 lines
include/linux/bpf_verifier.h +0 -1
index 04bb8f71cabe3..754e24d41f47f 100644
@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
 		/* remember the offset of node field within type to rewrite */
 		u64 insert_off;
 	};
-	struct bpf_iarray *jt;	/* jump table for gotox or bpf_tailcall call instruction */
 	struct btf_struct_meta *kptr_struct_meta;
 	u64 map_key_state; /* constant (32 bit) key tracking for maps */
 	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */

include/linux/bpf_verifier.h struct bpf_subprog_info { · line 807

Quoted 4 lines
include/linux/bpf_verifier.h +1 -0
@@ -805,6 +804,7 @@ struct bpf_subprog_info {
 	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
 	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
 	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
                                ^^^^

The patch moves jt ownership into bpf_subprog_info but doesn't update the code that removes instructions. When verifier_remove_insns() deletes a whole subprogram, adjust_subprog_starts_after_remove() in kernel/bpf/fixups.c compacts the array with memmove:

    move = env->subprog_cnt + 1 - j;
    memmove(env->subprog_info + i,
            env->subprog_info + j,
            sizeof(*env->subprog_info) * move);
    env->subprog_cnt -= j - i;

If subprogram 3 owns a jump table and subprograms 1 and 2 are removed (i=1, j=3), slots 3 and 4 are never rewritten, so subprog_info[1].jt and subprog_info[3].jt both point to the same kvmalloc'd buffer.

Does bpf_free_subprog_jts() then double-free that buffer? Its loop walks the entire fixed-size array rather than stopping at env->subprog_cnt:

    for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++)
        kvfree(env->subprog_info[i].jt);

so both the moved copy at index 1 and the stale alias at index 3 would be freed.

Also, when verifier_remove_insns() overwrites subprog_info[i..j), doesn't that leak the jump table of any subprogram in that range?

Before this patch, verifier_remove_insns() called bpf_clear_insn_aux_data(env, off, cnt) to free the jt of every removed instruction and then memmoved the aux array. The new owner needs the same
handling: kvfree the tables of removed subprograms and clear the stale tail slots left by the memmove, or bound bpf_free_subprog_jts() to env->subprog_cnt and explicitly zero the tail.

[ ... ]

Quoted 4 lines
kernel/bpf/cfg.c +0 -0
index df4513e71c80a..c38d33f0b0ba5 100644

[ ... ]

kernel/bpf/cfg.c static struct bpf_iarray *jt_from_subprog(struct bpf_verifier_env *env, · line 369

Quoted 10 lines
kernel/bpf/cfg.c +10 -0
+void bpf_free_subprog_jts(struct bpf_verifier_env *env)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(env->subprog_info); i++) {
+		kvfree(env->subprog_info[i].jt);
+		env->subprog_info[i].jt = NULL;
+	}
+	env->cfg.subprog_jts_ready = false;
+}

This iterates over all BPF_MAX_SUBPROGS + 2 slots (258 entries), but adjust_subprog_starts_after_remove() only updates env->subprog_cnt when subprograms are removed. Slots beyond that count can hold stale pointers duplicated by the memmove. Is there a risk of freeing the same allocation twice, once from its new index and once from the old stale slot?

[ ... ]

Quoted 29 lines
kernel/bpf/fixups.c +0 -17
index 52d3cec336727..f3cf66b8147a4 100644
@@ -500,23 +500,6 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
 	return 0;
 }
 
-/*
- * Clean up dynamically allocated fields of aux data for instructions [start, ...]
- */
-void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
-{
-	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
-	int end = start + len;
-	int i;
-
-	for (i = start; i < end; i++) {
-		if (aux_data[i].jt) {
-			kvfree(aux_data[i].jt);
-			aux_data[i].jt = NULL;
-		}
-	}
-}
-
 static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 {
 	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;

kernel/bpf/fixups.c static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt) · line 530

Quoted 5 lines
kernel/bpf/fixups.c +0 -2
@@ -526,8 +509,6 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 	if (bpf_prog_is_offloaded(env->prog->aux))
 		bpf_prog_offload_remove_insns(env, off, cnt);
 
-	bpf_clear_insn_aux_data(env, off, cnt);
-
                          ^^^^

Removing bpf_clear_insn_aux_data() makes sense for insn_aux_data, since it no longer owns the jt allocation. But subprog_info[].jt is still an owned pointer, and adjust_subprog_starts_after_remove() will memmove those structs when entire subprograms disappear.

Does verifier_remove_insns() need equivalent cleanup for the new owner? When j - i removed subprograms are dropped, the memmove copies slots [j, subprog_cnt] down to [i, ...] and decrements subprog_cnt by j - i, but it doesn't kvfree the jt pointers being overwritten in [i, j) or NULL the duplicates left in the tail slots.

Quoted 3 lines
kernel/bpf/fixups.c +0 -0
 	err = bpf_remove_insns(env->prog, off, cnt);
 	if (err)
 		return err;

[ ... ]

Quoted 5 lines
kernel/bpf/verifier.c +0 -0
index ddba53eaa3331..32157d145a609 100644

kernel/bpf/verifier.c int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, · line 21483

Quoted 9 lines
kernel/bpf/verifier.c +1 -2
@@ -21477,8 +21477,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
 	release_maps(env);
 	release_btfs(env);
 err_free_env:
-	if (env->insn_aux_data)
-		bpf_clear_insn_aux_data(env, 0, env->insn_aux_data_len);
+	bpf_free_subprog_jts(env);
 	vfree(env->insn_aux_data);
 	kvfree(env->fd_array);
 	bpf_stack_liveness_free(env);

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

author maintainer
#6 lore raw

Re: [PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

Quoted 18 lines · On 9/14/26 8:19 PM, [email protected] wrote:
Quoted 13 lines
include/linux/bpf_verifier.h +0 -1
index 04bb8f71cabe3..754e24d41f47f 100644
@@ -670,7 +670,6 @@ struct bpf_insn_aux_data {
  		/* remember the offset of node field within type to rewrite */
  		u64 insert_off;
  	};
-	struct bpf_iarray *jt;	/* jump table for gotox or bpf_tailcall call instruction */
  	struct btf_struct_meta *kptr_struct_meta;
  	u64 map_key_state; /* constant (32 bit) key tracking for maps */
  	int ctx_field_size; /* the ctx field size for load insn, maybe 0 */

include/linux/bpf_verifier.h struct bpf_subprog_info { · line 807

Quoted 4 lines
include/linux/bpf_verifier.h +1 -0
@@ -805,6 +804,7 @@ struct bpf_subprog_info {
  	u32 linfo_idx; /* The idx to the main_prog->aux->linfo */
  	u32 postorder_start; /* The idx to the env->cfg.insn_postorder */
  	u32 exit_idx; /* Index of one of the BPF_EXIT instructions in this subprogram */
+	struct bpf_iarray *jt; /* jump table shared by all gotox of this subprogram */
                                 ^^^^
Quoted 4 lines

The patch moves jt ownership into bpf_subprog_info but doesn't update the code that removes instructions. When verifier_remove_insns() deletes a whole subprogram, adjust_subprog_starts_after_remove() in kernel/bpf/fixups.c compacts the array with memmove:

Great catch! I'll fix this up in a v3 along with a selftest, and look into either reordering patch 2 & 3 or tweaking the commit msg a bit.

maintainer
#7 lore raw

Re: [PATCH bpf v2 03/11] bpf: Cache the jump table of a subprogram during CFG discovery

In reply to Daniel Borkmann

On Mon, 2026-09-14 at 19:03 +0200, Daniel Borkmann wrote:

Hi Daniel,

I think this is a good approach.

...

Quoted 1 line
kernel/bpf/cfg.c +0 -0

...

Quoted 13 lines
kernel/bpf/cfg.c +0 -0
@@ -383,13 +409,12 @@ static int visit_gotox_insn(int t, struct bpf_verifier_env *env)
  	struct bpf_iarray *jt;
  	int i, w;

kernel/bpf/cfg.c +3 -5
-	jt = env->insn_aux_data[t].jt;
-	if (!jt) {
-		jt = create_jt(t, env);
-		if (IS_ERR(jt))
-			return PTR_ERR(jt);
+	jt = subprog_jt(t, env);
+	if (IS_ERR(jt))
+		return PTR_ERR(jt);

kernel/bpf/cfg.c static int visit_gotox_insn(int t, struct bpf_verifier_env *env) · line 417

Quoted 3 lines
kernel/bpf/cfg.c +2 -1
-		env->insn_aux_data[t].jt = jt;
+	if (!(insn_state[t] & BRANCH)) {
+		insn_state[t] |= BRANCH;

I think it is possible to exit early here if BRANCH is already set.

Quoted 3 lines


  if (check_add_overflow(env->cfg.gotox_edges, jt->cnt, &env->cfg.gotox_edges) ||

...

Quoted 1 line
kernel/bpf/liveness.c +0 -0

...

kernel/bpf/liveness.c bpf_insn_successors(struct bpf_verifier_env *env, u32 idx) · line 277

Quoted 12 lines
kernel/bpf/liveness.c +0 -0
@@ -264,6 +267,15 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
  	if (opcode_info->can_jump)
  		succ->items[succ->cnt++] = idx + bpf_jmp_offset(insn) + 1;

kernel/bpf/liveness.c +8 -0
+	/*
+	 * tail_call upon success and ld_{abs,ind} upon load failure return
+	 * from the subprogram, which is a hidden edge to its exit.
+	 */
+	if (unlikely((bpf_helper_call(insn) && insn->imm == BPF_FUNC_tail_call) ||
+		     (BPF_CLASS(insn->code) == BPF_LD &&
+		      (BPF_MODE(insn->code) == BPF_ABS || BPF_MODE(insn->code) == BPF_IND))))
+		succ->items[succ->cnt++] = bpf_find_containing_subprog(env, idx)->exit_idx;

It appears that check_subprogs() assigns exit_idx but does not guarantee it to be set. Meaning that for the following progam exit_idx would remain zero:

0: r0 = 0
1: r0 = *(u8 *)skb[0]
2: goto -2

Hence the above would add a spurious edge.
I think check_subprogs() should be updated to demand each subprogram to have at-least one exit.

Quoted 4 lines

+
  return succ;
 }

...

author maintainer
#8 lore raw

PATCH v2 4/11 bpf: Reject indirect jumps that leave their subprogram

In reply to Daniel Borkmann

The jump table of a subprog is collected in compute_subprog_jts() from the insn_array maps of the program, and a map is attributed to the subprog that contains its first entry. check_indirect_jump() instead resolves the targets from the map the gotox register actually points to, bounded only by the index range of that register, and never relates them back to the subprog of the gotox.

The two disagree, so bpf_insn_successors() reports a subset of the edges the BPF program can take and a gotox can enter a subprog the CFG never walked.

Close both ends: reject a map that reaches past the subprog holding its first entry right when it is collected in compute_subprog_jts(), so that every map is confined to one subprog, and confine the resolved targets of a gotox to its subprog in check_indirect_jump(). A target inside the subprog is then always in the jump table the CFG walked, which is the invariant that actually has to hold, and is kept as a verifier_bug_if().

Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
---
 kernel/bpf/cfg.c                              | 39 +++++--------
 kernel/bpf/verifier.c                         | 55 +++++++++++++++++++
 .../selftests/bpf/progs/verifier_gotox.c      |  2 +-
 3 files changed, 69 insertions(+), 27 deletions(-)
kernel/bpf/cfg.c +13 -26
index c38d33f0b0ba..5a91d73d3e3f 100644
@@ -287,7 +287,7 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
 
 /*
  * Collect the jump table of every subprogram that has one, as the combined
- * table of all maps whose first target lands inside that subprogram. All gotox
+ * table of all maps whose targets land inside that subprogram. All gotox
  * instructions of a subprogram share the same table, so this is done in a
  * single pass over the maps rather than once per gotox.
  */
@@ -310,15 +310,21 @@ static int compute_subprog_jts(struct bpf_verifier_env *env)
 		if (IS_ERR(jt_cur))
 			return PTR_ERR(jt_cur);
 
-		/*
-		 * This is enough to check one element. The full table is
-		 * checked to fit inside the subprog later in subprog_jt()
-		 */
 		subprog = bpf_find_containing_subprog(env, jt_cur->items[0]);
 		if (!subprog) {
 			kvfree(jt_cur);
 			continue;
 		}
+		if (jt_cur->items[jt_cur->cnt - 1] >= (subprog + 1)->start) {
+			verbose(env, "jump table of subprog starting at %u spans multiple subprogs\n",
+				subprog->start);
+			bpf_diag_program_structure(env, subprog->start, "jump table spans subprograms",
+				"Keep every entry of a jump table inside one subprogram.",
+				"A jump table found for the subprogram that starts at instruction %u reaches past its end at instruction %u.",
+				subprog->start, (subprog + 1)->start);
+			kvfree(jt_cur);
+			return -EINVAL;
+		}
 
 		old_cnt = subprog->jt ? subprog->jt->cnt : 0;
 		jt = bpf_iarray_realloc(subprog->jt, old_cnt + jt_cur->cnt);
@@ -358,9 +364,7 @@ static struct bpf_iarray *
 subprog_jt(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_subprog_info *subprog;
-	int subprog_start, subprog_end;
-	struct bpf_iarray *jt;
-	int i, err;
+	int subprog_start, err;
 
 	if (!env->cfg.subprog_jts_ready) {
 		err = compute_subprog_jts(env);
@@ -370,7 +374,6 @@ subprog_jt(int t, struct bpf_verifier_env *env)
 
 	subprog = bpf_find_containing_subprog(env, t);
 	subprog_start = subprog->start;
-	subprog_end = (subprog + 1)->start;
 
 	if (!subprog->jt) {
 		verbose(env, "no jump tables found for subprog starting at %u\n", subprog_start);
@@ -381,23 +384,7 @@ subprog_jt(int t, struct bpf_verifier_env *env)
 		return ERR_PTR(-EINVAL);
 	}
 
-	jt = subprog->jt;
-
-	/* Check that the every element of the jump table fits within the given subprogram */
-	for (i = 0; i < jt->cnt; i++) {
-		if (jt->items[i] < subprog_start || jt->items[i] >= subprog_end) {
-			verbose(env, "jump table for insn %d points outside of the subprog [%u,%u]\n",
-					t, subprog_start, subprog_end);
-			bpf_diag_program_structure(
-				env, t, "jump table target out of range",
-				"Keep every jump-table target inside the same subprogram.",
-				"The jump table for instruction %d points outside subprogram range [%u,%u).",
-				t, subprog_start, subprog_end);
-			return ERR_PTR(-EINVAL);
-		}
-	}
-
-	return jt;
+	return subprog->jt;
 }
 
 /* "conditional jump with N edges" */
kernel/bpf/verifier.c +55 -0

72 lines not on this page · raw message

tools/testing/selftests/bpf/progs/verifier_gotox.c +1 -1
index 0e27c2c79c57..4931a77ba0b2 100644
@@ -366,7 +366,7 @@ __used static int test_subprog(void)
 }
 
 SEC("socket")
-__failure __msg("jump table for insn 4 points outside of the subprog [0,10]")
+__failure __msg("jump table of subprog starting at 0 spans multiple subprogs")
 __naked void jump_table_outside_subprog(void)
 {
 	asm volatile ("						\
Signature · 1 line
2.43.0
maintainer Acked-by on v2 4/11
#9 lore raw

Re: [PATCH bpf v2 04/11] bpf: Reject indirect jumps that leave their subprogram

Quoted 17 lines · Daniel Borkmann

The jump table of a subprog is collected in compute_subprog_jts() from the insn_array maps of the program, and a map is attributed to the subprog that contains its first entry. check_indirect_jump() instead resolves the targets from the map the gotox register actually points to, bounded only by the index range of that register, and never relates them back to the subprog of the gotox.

The two disagree, so bpf_insn_successors() reports a subset of the edges the BPF program can take and a gotox can enter a subprog the CFG never walked.

Close both ends: reject a map that reaches past the subprog holding its first entry right when it is collected in compute_subprog_jts(), so that every map is confined to one subprog, and confine the resolved targets of a gotox to its subprog in check_indirect_jump(). A target inside the subprog is then always in the jump table the CFG walked, which is the invariant that actually has to hold, and is kept as a verifier_bug_if().

Quoted 6 lines
Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")

---

...

Quoted 15 lines
kernel/bpf/verifier.c +0 -0
@@ -18238,6 +18270,29 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
  		return -EINVAL;
  	}

kernel/bpf/verifier.c +11 -0
+	subprog = bpf_find_containing_subprog(env, env->insn_idx);
+	if (verifier_bug_if(!subprog, env, "no subprog contains insn %d", env->insn_idx))
+		return -EFAULT;
+	subprog_start = subprog->start;
+	subprog_end = (subprog + 1)->start;
+
+	jt = subprog->jt;
+	if (verifier_bug_if(!jt, env, "no jump table for insn %d", env->insn_idx))
+		return -EFAULT;
+
+	for (i = 0; i < n; i++) {

kernel/bpf/verifier.c static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in · line 18293

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+		u32 target = env->gotox_tmp_buf->items[i];
+
+		if (target < subprog_start || target >= subprog_end)
+			return reject_gotox_out_of_subprog(env, target, subprog_start,
+							   subprog_end);
+		/* Maps are confined to a subprog, see compute_subprog_jts() */
+		if (verifier_bug_if(!jt_contains(jt, target), env,
+				    "insn %d target %u is not in the jump table of its subprog",
+				    env->insn_idx, target))
+			return -EFAULT;

Nit: I think this check is redundant, it can be dropped together with
jt_contains(), moving the above subprog_{start,end} checks out of the loop.

Quoted 5 lines
(quoted diff) +2 -0
+	}
+

  for (i = 0; i < n - 1; i++) {
  mark_indirect_target(env, env->gotox_tmp_buf->items[i]); other_branch = push_stack(env, env->gotox_tmp_buf->items[i],

...

author maintainer
#10 lore raw

PATCH v2 5/11 bpf: Don't remove indirect jump targets in the nop removal pass

In reply to Daniel Borkmann

bpf_opt_remove_nops() drops every ja +0 and may_goto +0 unconditionally. A gotox target may itself be such a transparent nop: it is a verified entry of an insn array jump table that is reachable at run time through the indirect jump. bpf_insn_array_adjust_after_remove() marks it as INSN_DELETED instead, and bpf_insn_array_ready() skips INSN_DELETED entries, so the program loads with a hole in its jump table. The x86 JIT, for example, lowers gotox to a raw indirect jump, so executing that index jumps to the zeroed ips[] slot, that is, to address 0. Fix by keeping an indirect jump target in place.

Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
---
 kernel/bpf/fixups.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/bpf/fixups.c +3 -0
index f3cf66b8147a..aa3fb7b87fb0 100644
@@ -599,6 +599,7 @@ int bpf_opt_remove_dead_code(struct bpf_verifier_env *env)
 
 int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 {
+	struct bpf_insn_aux_data *aux = env->insn_aux_data;
 	struct bpf_insn *insn = env->prog->insnsi;
 	int insn_cnt = env->prog->len;
 	bool is_may_goto_0, is_ja;
@@ -610,6 +611,8 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 
 		if (!is_may_goto_0 && !is_ja)
 			continue;
+		if (aux[i].indirect_target)
+			continue;
 
 		err = verifier_remove_insns(env, i, 1);
 		if (err)
Signature · 1 line
2.43.0
maintainer Acked-by on v2 5/11
#11 lore raw

Re: [PATCH bpf v2 05/11] bpf: Don't remove indirect jump targets in the nop removal pass

Quoted 10 lines · Daniel Borkmann

bpf_opt_remove_nops() drops every ja +0 and may_goto +0 unconditionally. A gotox target may itself be such a transparent nop: it is a verified entry of an insn array jump table that is reachable at run time through the indirect jump. bpf_insn_array_adjust_after_remove() marks it as INSN_DELETED instead, and bpf_insn_array_ready() skips INSN_DELETED entries, so the program loads with a hole in its jump table. The x86 JIT, for example, lowers gotox to a raw indirect jump, so executing that index jumps to the zeroed ips[] slot, that is, to address 0. Fix by keeping an indirect jump target in place.

Quoted 6 lines
Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")

---

...

author maintainer
#12 lore raw

PATCH v2 6/11 bpf: Clear insn array jitted pointers on map reuse

In reply to Daniel Borkmann

An insn array records, per entry, the xlated offset, the jitted offset and a jitted target pointer in ips[]. bpf_insn_array_init() runs when the map is bound to a program and resets only xlated_off; the jitted offset and the ips[] pointer are left as the previous owner set them. A program can be verified and JITed and only then fail to load (e.g. bpf_check_tail_call()). By that point bpf_prog_update_insn_ptrs() has already filled ips[] with pointers into that program's JIT image. Thus, the map is handed back for reuse with the stale pointers intact. If the next program reuses the map, the stale pointer survived the now active program's jump table, pointing into the previous, freed image. Fix by resetting the jitted offset and the ips[] pointer alongside the xlated offset.

Fixes b4ce5923e780 ("bpf, x86: add new map type: instructions array")
---
 kernel/bpf/bpf_insn_array.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
kernel/bpf/bpf_insn_array.c +8 -4
index a2f84afe6f7c..5874abd1f743 100644
@@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
 		return -EBUSY;
 
 	/*
-	 * Reset all the map indexes to the original values.  This is needed,
-	 * e.g., when a replay of verification with different log level should
-	 * be performed.
+	 * Reset the map to its pre-verification state. The xlated and jitted
+	 * offsets and the jitted target pointers are recomputed by the verifier
+	 * and the JIT for this program, so any values left by a previous owner
+	 * must be cleared here.
 	 */
-	for (i = 0; i < map->max_entries; i++)
+	for (i = 0; i < map->max_entries; i++) {
 		values[i].xlated_off = values[i].orig_off;
+		values[i].jitted_off = 0;
+		insn_array->ips[i] = 0;
+	}
 
 	return 0;
 }
Signature · 1 line
2.43.0
#13 lore raw

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing release semantics in bpf_insn_array_release can allow stale JIT pointers to overwrite the re-initialized array due to CPU store reordering. --

commit 9b6ea61f193d20b96c453fda13952db5c1dc1506
Author: Daniel Borkmann <[email protected]>

bpf: Clear insn array jitted pointers on map reuse

This commit fixes a bug where stale JIT pointers and jitted offsets are left in an instruction array map when it is reused. It clears these fields during map initialization in bpf_insn_array_init().

Quoted 15 lines
kernel/bpf/bpf_insn_array.c +4 -3
index a2f84afe6f7c6..5874abd1f7438 100644
@@ -199,12 +199,16 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
 		return -EBUSY;
 
 	/*
-	 * Reset all the map indexes to the original values.  This is needed,
-	 * e.g., when a replay of verification with different log level should
-	 * be performed.
+	 * Reset the map to its pre-verification state. The xlated and jitted
+	 * offsets and the jitted target pointers are recomputed by the verifier
+	 * and the JIT for this program, so any values left by a previous owner
+	 * must be cleared here.

kernel/bpf/bpf_insn_array.c int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog) · line 214

Quoted 10 lines
kernel/bpf/bpf_insn_array.c +4 -1
 	 */
-	for (i = 0; i < map->max_entries; i++)
+	for (i = 0; i < map->max_entries; i++) {
 		values[i].xlated_off = values[i].orig_off;
+		values[i].jitted_off = 0;
+		insn_array->ips[i] = 0;
+	}
 
 	return 0;
 }

[Severity: High]
This isn't a bug introduced by this patch, but while looking at how the map is reused, I noticed a potential missing release semantics issue in bpf_insn_array_release():

kernel/bpf/bpf_insn_array.c:bpf_insn_array_release() {
	struct bpf_insn_array *insn_array = cast_insn_array(map);

	atomic_set(&insn_array->used, 0);
}

Is it possible that without release semantics (like atomic_set_release), prior stores to the map can be reordered after this flag is set?

If two threads concurrently load a BPF program using the same BPF_MAP_TYPE_INSN_ARRAY map:

Thread 1 JITs the program and writes to ips via bpf_prog_update_insn_ptrs(), but fails a late verification check like bpf_check_tail_call(). It then releases the map by setting used = 0 via the relaxed atomic_set().

Thread 2 acquires the map using atomic_xchg() and zeroes ips during initialization, as added in this patch.

Could CPU reordering on weakly-ordered architectures allow Thread 1's delayed JIT stores to overwrite the zeros with stale pointers, causing Thread 2 to subsequently use those stale pointers?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author maintainer
#14 lore raw

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

Quoted 5 lines · On 9/14/26 7:29 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Missing release semantics in bpf_insn_array_release can allow stale JIT pointers to overwrite the re-initialized array due to CPU store reordering. --

[...]

Quoted 21 lines

[Severity: High]
This isn't a bug introduced by this patch, but while looking at how the map is reused, I noticed a potential missing release semantics issue in bpf_insn_array_release():

kernel/bpf/bpf_insn_array.c:bpf_insn_array_release() {
	struct bpf_insn_array *insn_array = cast_insn_array(map);

	atomic_set(&insn_array->used, 0);
}

Is it possible that without release semantics (like atomic_set_release), prior stores to the map can be reordered after this flag is set?

If two threads concurrently load a BPF program using the same BPF_MAP_TYPE_INSN_ARRAY map:

Thread 1 JITs the program and writes to ips via bpf_prog_update_insn_ptrs(), but fails a late verification check like bpf_check_tail_call(). It then releases the map by setting used = 0 via the relaxed atomic_set().

Quoted 6 lines

Thread 2 acquires the map using atomic_xchg() and zeroes ips during initialization, as added in this patch.

Could CPU reordering on weakly-ordered architectures allow Thread 1's delayed JIT stores to overwrite the zeros with stale pointers, causing Thread 2 to subsequently use those stale pointers?

I'll look into that one separately.

#15 lore raw

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

In reply to Daniel Borkmann
The sentence "the stale pointer survived the now active program's jump table, pointing into the previous, freed image" is hard to parse. 'Survived' takes the jump table as a direct object, so it…
Quoted 13 lines

bpf: Clear insn array jitted pointers on map reuse

An insn array records, per entry, the xlated offset, the jitted offset and a jitted target pointer in ips[]. bpf_insn_array_init() runs when the map is bound to a program and resets only xlated_off; the jitted offset and the ips[] pointer are left as the previous owner set them. A program can be verified and JITed and only then fail to load (e.g. bpf_check_tail_call()). By that point bpf_prog_update_insn_ptrs() has already filled ips[] with pointers into that program's JIT image. Thus, the map is handed back for reuse with the stale pointers intact. If the next program reuses the map, the stale pointer survived the now active program's jump table, pointing into the previous, freed image. Fix by resetting the jitted offset and the ips[] pointer alongside the xlated offset.

The sentence "the stale pointer survived the now active program's jump table, pointing into the previous, freed image" is hard to parse. 'Survived' takes the jump table as a direct object, so it reads as if the pointer outlived the table rather than persisting inside it. Would "the stale pointer survives in the now active program's jump table" be clearer? This is the sentence that describes the actual security impact, so the wording matters for anyone reading the changelog to judge severity.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

maintainer Acked-by on v2 6/11
#16 lore raw

Re: [PATCH bpf v2 06/11] bpf: Clear insn array jitted pointers on map reuse

In reply to Daniel Borkmann
Quoted 12 lines · Daniel Borkmann

An insn array records, per entry, the xlated offset, the jitted offset and a jitted target pointer in ips[]. bpf_insn_array_init() runs when the map is bound to a program and resets only xlated_off; the jitted offset and the ips[] pointer are left as the previous owner set them. A program can be verified and JITed and only then fail to load (e.g. bpf_check_tail_call()). By that point bpf_prog_update_insn_ptrs() has already filled ips[] with pointers into that program's JIT image. Thus, the map is handed back for reuse with the stale pointers intact. If the next program reuses the map, the stale pointer survived the now active program's jump table, pointing into the previous, freed image. Fix by resetting the jitted offset and the ips[] pointer alongside the xlated offset.

Quoted 6 lines
Fixes b4ce5923e780 ("bpf, x86: add new map type: instructions array")

---

...

author maintainer
#17 lore raw

PATCH v2 7/11 bpf: Retarget indirect jump targets across prologue prepends

In reply to Daniel Borkmann

A few patches do not expand in place, they prepend: gen_epilogue and gen_prologue splice their instructions in front of insn 0 in bpf_convert_ctx_accesses(), and bpf_do_misc_fixups() prepends the may_goto counter init at every subprog start.

They copy the original instruction of 'tgt_idx' into the last slot of the patch buffer, so it now lives at tgt_idx + delta, and call adjust_jmp_off() to move direct branches from tgt_idx to tgt_idx + delta.

Nothing does the same for indirect branches, so a BPF_MAP_TYPE_INSN_ARRAY slot that named tgt_idx keeps naming tgt_idx, which is now the first prepended instruction, and insn_aux_data[tgt_idx].indirect_target makes the JIT emit the landing pad there. Add a __bpf_patch_insn_data() variant and pass the BPF_PREPEND at the affected call-sites to fix up the delta.

Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Fixes 07ae6c130b46 ("bpf: Add helper to detect indirect jump targets")
---
 include/linux/bpf.h         |  2 +-
 kernel/bpf/bpf_insn_array.c |  4 ++--
 kernel/bpf/fixups.c         | 41 ++++++++++++++++++++++++++-----------
 3 files changed, 32 insertions(+), 15 deletions(-)
include/linux/bpf.h +1 -1
index e57af902560c..8115f307444a 100644
@@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);
 int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
 int bpf_insn_array_ready(struct bpf_map *map);
 void bpf_insn_array_release(struct bpf_map *map);
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);
 void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);
 
 #ifdef CONFIG_BPF_SYSCALL
kernel/bpf/bpf_insn_array.c +2 -2
index 5874abd1f743..350ecea8f1e8 100644
@@ -235,7 +235,7 @@ void bpf_insn_array_release(struct bpf_map *map)
 	atomic_set(&insn_array->used, 0);
 }
 
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len)
 {
 	struct bpf_insn_array *insn_array = cast_insn_array(map);
 	int i;
@@ -244,7 +244,7 @@ void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
 		return;
 
 	for (i = 0; i < map->max_entries; i++) {
-		if (insn_array->values[i].xlated_off <= off)
+		if (insn_array->values[i].xlated_off < first)
 			continue;
 		if (insn_array->values[i].xlated_off == INSN_DELETED)
 			continue;
kernel/bpf/fixups.c +29 -12

106 lines not on this page · raw message

Signature · 1 line
2.43.0
#18 lore raw

Re: [PATCH bpf v2 07/11] bpf: Retarget indirect jump targets across prologue prepends

This isn't a bug, but since BPF_APPEND also covers buffers that prepend to the original insn (like the BPF_ST_NOSPEC sites where the original insn is copied to the end of the buffer), would names…
Quoted 11 lines
include/linux/bpf.h +1 -1
index e57af902560c3..8115f307444ad 100644
@@ -4165,7 +4165,7 @@ struct bpf_prog *bpf_prog_find_from_stack(void);
 int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
 int bpf_insn_array_ready(struct bpf_map *map);
 void bpf_insn_array_release(struct bpf_map *map);
-void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
+void bpf_insn_array_adjust(struct bpf_map *map, u32 first, u32 len);
 void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);

[ ... ]

kernel/bpf/fixups.c static int get_callee_stack_depth(struct bpf_verifier_env *env, · line 220

Quoted 13 lines
kernel/bpf/fixups.c +5 -0
index aa3fb7b87fb06..e132bd0600c7e 100644
@@ -213,12 +213,18 @@ static int get_callee_stack_depth(struct bpf_verifier_env *env,
 }
 #endif
 
+enum bpf_patch_mode {
+	BPF_APPEND,
+	BPF_PREPEND,
+};
+

This isn't a bug, but since BPF_APPEND also covers buffers that prepend to the original insn (like the BPF_ST_NOSPEC sites where the original insn is copied to the end of the buffer), would names describing whether the target index moves be clearer here?

The mode really selects whether the original instruction stays at 'off' or moves to 'off + len - 1', which is why the BPF_PREPEND sites coincide exactly with the two adjust_jmp_off() call sites. Something like BPF_PATCH_KEEP_TARGET vs BPF_PATCH_MOVE_TARGET, or at least BPF_PATCH_APPEND / BPF_PATCH_PREPEND to match the usual stem pairing in this subsystem, might read less contradicted by neighbouring APPEND call sites.

Quoted 9 lines
kernel/bpf/fixups.c +2 -1
 /* single env->prog->insni[off] instruction was replaced with the range
  * insni[off, off + cnt).  Adjust corresponding insn_aux_data by copying
  * [0, off) and [off, end) to new locations, so the patched range stays zero
  */
 static void adjust_insn_aux_data(struct bpf_verifier_env *env,
-				 struct bpf_prog *new_prog, u32 off, u32 cnt)
+				 struct bpf_prog *new_prog, u32 off, u32 cnt,
+				 enum bpf_patch_mode mode)
 {

[ ... ]

Quoted 19 lines
kernel/bpf/fixups.c +4 -2
@@ -755,7 +769,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
 			insn_buf[cnt++] = BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1,
 						      -subprogs[0].stack_depth);
 			insn_buf[cnt++] = env->prog->insnsi[0];
-			new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+			new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+							 BPF_PREPEND);
 			if (!new_prog)
 				return -ENOMEM;
 			env->prog = new_prog;
@@ -778,7 +793,8 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env)
 			verifier_bug(env, "prologue is too long");
 			return -EFAULT;
 		} else if (cnt) {
-			new_prog = bpf_patch_insn_data(env, 0, insn_buf, cnt);
+			new_prog = __bpf_patch_insn_data(env, 0, insn_buf, cnt,
+							 BPF_PREPEND);
 			if (!new_prog)
 				return -ENOMEM;

[ ... ]

Quoted 6 lines

bpf: Retarget indirect jump targets across prologue prepends

A few patches do not expand in place, they prepend: gen_epilogue and gen_prologue splice their instructions in front of insn 0 in bpf_convert_ctx_accesses(), and bpf_do_misc_fixups() prepends the may_goto counter init at every subprog start.

This isn't a bug, but would the changelog read more precisely if it said that for the gen_epilogue case, the ctx-save insn is prepended at insn 0, and that gen_prologue is the one that splices its own instructions there?

In bpf_convert_ctx_accesses() above, when ops->gen_epilogue is present, the epilogue body is patched in at the BPF_EXIT insn (still BPF_APPEND). What is actually prepended at insn 0 is the single ctx-saving 'BPF_STX_MEM(BPF_DW, BPF_REG_FP, BPF_REG_1, ...)' plus the copied insnsi[0]. A reader trying to match the three converted call sites to the sentence "gen_epilogue and gen_prologue splice their instructions in front of insn 0" has to re-derive that distinction.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

maintainer Acked-by on v2 7/11
#19 lore raw

Re: [PATCH bpf v2 07/11] bpf: Retarget indirect jump targets across prologue prepends

In reply to Daniel Borkmann
Quoted 15 lines · Daniel Borkmann

A few patches do not expand in place, they prepend: gen_epilogue and gen_prologue splice their instructions in front of insn 0 in bpf_convert_ctx_accesses(), and bpf_do_misc_fixups() prepends the may_goto counter init at every subprog start.

They copy the original instruction of 'tgt_idx' into the last slot of the patch buffer, so it now lives at tgt_idx + delta, and call adjust_jmp_off() to move direct branches from tgt_idx to tgt_idx + delta.

Nothing does the same for indirect branches, so a BPF_MAP_TYPE_INSN_ARRAY slot that named tgt_idx keeps naming tgt_idx, which is now the first prepended instruction, and insn_aux_data[tgt_idx].indirect_target makes the JIT emit the landing pad there. Add a __bpf_patch_insn_data() variant and pass the BPF_PREPEND at the affected call-sites to fix up the delta.

Quoted 7 lines
Fixes 493d9e0d6083 ("bpf, x86: add support for indirect jumps")
Fixes 07ae6c130b46 ("bpf: Add helper to detect indirect jump targets")

---

...

author maintainer
#20 lore raw

PATCH v2 8/11 selftests/bpf: Add tests for the indirect jump edge limit

In reply to Daniel Borkmann

Add corner case tests for programs whose gotox instructions are their own jump table targets.

# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array [...]

  #24/10   bpf_insn_array/too-many-gotox-edges:OK
  #24/11   bpf_insn_array/gotox-edges-at-limit:OK
  #24/12   bpf_insn_array/gotox-edges-across-subprogs:OK
  #24      bpf_insn_array:OK

Summary: 1/12 PASSED, 0 SKIPPED, 0/0 FAILED

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 266 ++++++++++++++++++
 1 file changed, 266 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +266 -0
index 0222a9a5d076..08f5d4f3d64f 100644
@@ -453,6 +453,263 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
+#define GOTOX_CNT_AT_LIMIT	1000
+#define GOTOX_LOG_SZ		(256 * 1024)
+
+static const char gotox_limit_msg[] =
+	"number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+	/* the run of gotox itself, plus the insn right after it */
+	const __u32 jt_cnt = gotox_cnt + 1;
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < jt_cnt; i++) {
+		val.orig_off = first_gotox + i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int prog_fd;
+
+	log[0] = 0;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = fd_array_cnt;
+	opts.log_buf = log;
+	opts.log_size = GOTOX_LOG_SZ;
+	opts.log_level = 1;
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+	if (prog_fd >= 0) {
+		close(prog_fd);
+		return 0;
+	}
+	return prog_fd;
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	__u32 i;
+
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+	for (i = 1; i <= gotox_cnt; i++)
+		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+	const __u32 insn_cnt = gotox_cnt + 3;
+	struct bpf_insn *insns;
+	bool attempted = false;
+	int map_fd;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		return false;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd = gotox_jt_create(1, gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+	attempted = true;
+free_insns:
+	free(insns);
+	return attempted;
+}
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	if (try_load_gotox_prog(gotox_cnt, log, &err))
+		check_gotox_limit_hit(log, err);
+
+	free(log);
+}
+
+/*
+ * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
+ * table holds the starts of the blocks that follow plus the exit block,
+ * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
+ * exactly at the limit and must still load.
+ */
+#define GOTOX_BLOCK_SZ		4
+
+static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	struct bpf_insn *at;
+	__u32 k;
+
+	for (k = 0; k < gotox_cnt; k++) {
+		at = insns + k * GOTOX_BLOCK_SZ;
+
+		/* r1 = &jt[0], by index 0 into fd_array */
+		at[0] = (struct bpf_insn) {
+			.code = BPF_LD | BPF_DW | BPF_IMM,
+			.dst_reg = BPF_REG_1,
+			.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+			.imm = 0,
+		};
+		at[1] = (struct bpf_insn) { .imm = 0 };
+		at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
+		at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	}
+
+	insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
+}
+
+static int gotox_chain_jt_create(__u32 gotox_cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < gotox_cnt; i++) {
+		val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static void check_gotox_edges_at_limit(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
+	struct bpf_insn *insns;
+	char *log;
+	int map_fd, err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_chain_fill(insns, gotox_cnt);
+
+	map_fd = gotox_chain_jt_create(gotox_cnt);
+	if (map_fd < 0)
+		goto free_insns;
+
+	err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+
+	if (!ASSERT_OK(err, "program at the edge limit should load"))
+		fprintf(stderr, "verifier log: %s\n", log);
+
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
+static void check_gotox_edges_across_subprogs(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
+	const __u32 sub_start = gotox_cnt + 3;
+	const __u32 insn_cnt = 2 * (gotox_cnt + 3);
+	int map_fd[2] = { -1, -1 };
+	struct bpf_insn *insns;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		goto free_log;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
+					    BPF_PSEUDO_CALL, 0,
+					    sub_start - (gotox_cnt + 1) - 1);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	gotox_run_fill(insns + sub_start, gotox_cnt);
+	insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd[0] = gotox_jt_create(1, gotox_cnt);
+	if (map_fd[0] < 0)
+		goto free_insns;
+	map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
+	check_gotox_limit_hit(log, err);
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_insns:
+	free(insns);
+free_log:
+	free(log);
+}
+
 static void check_bpf_side(void)
 {
 	check_bpf_no_lookup();
@@ -490,6 +747,15 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
+
+	if (test__start_subtest("too-many-gotox-edges"))
+		check_too_many_gotox_edges();
+
+	if (test__start_subtest("gotox-edges-at-limit"))
+		check_gotox_edges_at_limit();
+
+	if (test__start_subtest("gotox-edges-across-subprogs"))
+		check_gotox_edges_across_subprogs();
 }
 #else
 static void __test_bpf_insn_array(void)
Signature · 1 line
2.43.0
#21 lore raw

Re: [PATCH bpf v2 08/11] selftests/bpf: Add tests for the indirect jump edge limit

This isn't a bug, but could try_load_gotox_prog() be inlined or simplified? It has a single caller in check_too_many_gotox_edges(), and the bool return plus int *err out-parameter split makes that…
Quoted 95 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +87 -0
index 0222a9a5d0761..08f5d4f3d64fb 100644
@@ -453,6 +453,263 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
+#define GOTOX_CNT_AT_LIMIT	1000
+#define GOTOX_LOG_SZ		(256 * 1024)
+
+static const char gotox_limit_msg[] =
+	"number of indirect jump edges in the program exceeds";
+
+static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
+{
+	/* the run of gotox itself, plus the insn right after it */
+	const __u32 jt_cnt = gotox_cnt + 1;
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < jt_cnt; i++) {
+		val.orig_off = first_gotox + i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int prog_fd;
+
+	log[0] = 0;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = fd_array_cnt;
+	opts.log_buf = log;
+	opts.log_size = GOTOX_LOG_SZ;
+	opts.log_level = 1;
+
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+	if (prog_fd >= 0) {
+		close(prog_fd);
+		return 0;
+	}
+	return prog_fd;
+}
+
+/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
+static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
+{
+	__u32 i;
+
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
+	for (i = 1; i <= gotox_cnt; i++)
+		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+}
+
+static void check_gotox_limit_hit(const char *log, int err)
+{
+	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
+}
+
+static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
+{
+	const __u32 insn_cnt = gotox_cnt + 3;
+	struct bpf_insn *insns;
+	bool attempted = false;
+	int map_fd;
+
+	insns = calloc(insn_cnt, sizeof(*insns));
+	if (!ASSERT_OK_PTR(insns, "calloc insns"))
+		return false;
+
+	gotox_run_fill(insns, gotox_cnt);
+	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
+
+	map_fd = gotox_jt_create(1, gotox_cnt);

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_bpf_no_lookup(void) · line 552

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+	if (map_fd < 0)
+		goto free_insns;
+
+	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
+	close(map_fd);
+	attempted = true;
+free_insns:
+	free(insns);
+	return attempted;
+}

This isn't a bug, but could try_load_gotox_prog() be inlined or simplified? It has a single caller in check_too_many_gotox_edges(), and the bool return plus int *err out-parameter split makes that test read differently from check_gotox_edges_at_limit() and check_gotox_edges_across_subprogs(), which open-code the identical scaffolding (calloc log, calloc insns, fill insns, create map, load, close map, free) inline.

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+
+/*
+ * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
+ * + 1) edges, hence the program is over the limit by gotox_cnt edges.
+ */
+static void check_too_many_gotox_edges(void)
+{
+	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
+	char *log;
+	int err;

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_bpf_no_lookup(void) · line 572

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	if (try_load_gotox_prog(gotox_cnt, log, &err))
+		check_gotox_limit_hit(log, err);
+
+	free(log);
+}

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

maintainer
#22 lore raw

Re: [PATCH bpf v2 08/11] selftests/bpf: Add tests for the indirect jump edge limit

In reply to Daniel Borkmann
Quoted 11 lines · Daniel Borkmann

Add corner case tests for programs whose gotox instructions are their own jump table targets.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array [...]
  #24/10   bpf_insn_array/too-many-gotox-edges:OK
  #24/11   bpf_insn_array/gotox-edges-at-limit:OK
  #24/12   bpf_insn_array/gotox-edges-across-subprogs:OK
  #24      bpf_insn_array:OK
  Summary: 1/12 PASSED, 0 SKIPPED, 0/0 FAILED

Quoted 3 lines

---

I think it's a bit of an overkill of a test.
Codex generated transformation to inline assembly as attached. Object file size is 100k, but I think it is significantly easier to understand in this form. Wdyt?

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +0 -240
index f102435675c2..70150b54bdaf 100644
@@ -454,40 +454,8 @@ static void check_bpf_no_lookup(void)
 	close(map_fd);
 }
 
-#define GOTOX_CNT_AT_LIMIT	1000
 #define GOTOX_LOG_SZ		(256 * 1024)
 
-static const char gotox_limit_msg[] =
-	"number of indirect jump edges in the program exceeds";
-
-static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
-{
-	/* the run of gotox itself, plus the insn right after it */
-	const __u32 jt_cnt = gotox_cnt + 1;
-	struct bpf_insn_array_value val = {};
-	int map_fd;
-	__u32 i;
-
-	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, jt_cnt);
-	if (!ASSERT_GE(map_fd, 0, "map_create"))
-		return map_fd;
-
-	for (i = 0; i < jt_cnt; i++) {
-		val.orig_off = first_gotox + i;
-		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
-			       "bpf_map_update_elem"))
-			goto err;
-	}
-
-	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
-		goto err;
-
-	return map_fd;
-err:
-	close(map_fd);
-	return -1;
-}
-
 static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
 				 int *fd_array, __u32 fd_array_cnt, char *log,
 				 int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
@@ -523,208 +491,6 @@ static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
 				     -1, NULL, 0);
 }
 
-/* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
-static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
-{
-	__u32 i;
-
-	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 0);
-	for (i = 1; i <= gotox_cnt; i++)
-		insns[i] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
-}
-
-static void check_gotox_limit_hit(const char *log, int err)
-{
-	ASSERT_EQ(err, -E2BIG, "program should have been rejected");
-	ASSERT_HAS_SUBSTR(log, gotox_limit_msg, "verifier log");
-}
-
-static bool try_load_gotox_prog(__u32 gotox_cnt, char *log, int *err)
-{
-	const __u32 insn_cnt = gotox_cnt + 3;
-	struct bpf_insn *insns;
-	bool attempted = false;
-	int map_fd;
-
-	insns = calloc(insn_cnt, sizeof(*insns));
-	if (!ASSERT_OK_PTR(insns, "calloc insns"))
-		return false;
-
-	gotox_run_fill(insns, gotox_cnt);
-	insns[gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
-	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
-
-	map_fd = gotox_jt_create(1, gotox_cnt);
-	if (map_fd < 0)
-		goto free_insns;
-
-	*err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
-	close(map_fd);
-	attempted = true;
-free_insns:
-	free(insns);
-	return attempted;
-}
-
-/*
- * The extra exit target in the jump table makes for gotox_cnt * (gotox_cnt
- * + 1) edges, hence the program is over the limit by gotox_cnt edges.
- */
-static void check_too_many_gotox_edges(void)
-{
-	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
-	char *log;
-	int err;
-
-	log = calloc(1, GOTOX_LOG_SZ);
-	if (!ASSERT_OK_PTR(log, "calloc log"))
-		return;
-
-	if (try_load_gotox_prog(gotox_cnt, log, &err))
-		check_gotox_limit_hit(log, err);
-
-	free(log);
-}
-
-/*
- * A chain of blocks, where block k loads jt[k] and jumps to it. The jump
- * table holds the starts of the blocks that follow plus the exit block,
- * which is gotox_cnt targets for gotox_cnt gotox, so the program sits
- * exactly at the limit and must still load.
- */
-#define GOTOX_BLOCK_SZ		4
-
-static void gotox_chain_fill(struct bpf_insn *insns, __u32 gotox_cnt)
-{
-	struct bpf_insn *at;
-	__u32 k;
-
-	for (k = 0; k < gotox_cnt; k++) {
-		at = insns + k * GOTOX_BLOCK_SZ;
-
-		/* r1 = &jt[0], by index 0 into fd_array */
-		at[0] = (struct bpf_insn) {
-			.code = BPF_LD | BPF_DW | BPF_IMM,
-			.dst_reg = BPF_REG_1,
-			.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
-			.imm = 0,
-		};
-		at[1] = (struct bpf_insn) { .imm = 0 };
-		at[2] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, k * 8);
-		at[3] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
-	}
-
-	insns[gotox_cnt * GOTOX_BLOCK_SZ] = BPF_MOV64_IMM(BPF_REG_0, 0);
-	insns[gotox_cnt * GOTOX_BLOCK_SZ + 1] = BPF_EXIT_INSN();
-}
-
-static int gotox_chain_jt_create(__u32 gotox_cnt)
-{
-	struct bpf_insn_array_value val = {};
-	int map_fd;
-	__u32 i;
-
-	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, gotox_cnt);
-	if (!ASSERT_GE(map_fd, 0, "map_create"))
-		return map_fd;
-
-	for (i = 0; i < gotox_cnt; i++) {
-		val.orig_off = (i + 1) * GOTOX_BLOCK_SZ;
-		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
-			       "bpf_map_update_elem"))
-			goto err;
-	}
-
-	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
-		goto err;
-
-	return map_fd;
-err:
-	close(map_fd);
-	return -1;
-}
-
-static void check_gotox_edges_at_limit(void)
-{
-	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT;
-	const __u32 insn_cnt = gotox_cnt * GOTOX_BLOCK_SZ + 2;
-	struct bpf_insn *insns;
-	char *log;
-	int map_fd, err;
-
-	log = calloc(1, GOTOX_LOG_SZ);
-	if (!ASSERT_OK_PTR(log, "calloc log"))
-		return;
-
-	insns = calloc(insn_cnt, sizeof(*insns));
-	if (!ASSERT_OK_PTR(insns, "calloc insns"))
-		goto free_log;
-
-	gotox_chain_fill(insns, gotox_cnt);
-
-	map_fd = gotox_chain_jt_create(gotox_cnt);
-	if (map_fd < 0)
-		goto free_insns;
-
-	err = gotox_prog_load(insns, insn_cnt, &map_fd, 1, log);
-	close(map_fd);
-
-	if (!ASSERT_OK(err, "program at the edge limit should load"))
-		fprintf(stderr, "verifier log: %s\n", log);
-
-free_insns:
-	free(insns);
-free_log:
-	free(log);
-}
-
-static void check_gotox_edges_across_subprogs(void)
-{
-	const __u32 gotox_cnt = GOTOX_CNT_AT_LIMIT * 3 / 4;
-	const __u32 sub_start = gotox_cnt + 3;
-	const __u32 insn_cnt = 2 * (gotox_cnt + 3);
-	int map_fd[2] = { -1, -1 };
-	struct bpf_insn *insns;
-	char *log;
-	int err;
-
-	log = calloc(1, GOTOX_LOG_SZ);
-	if (!ASSERT_OK_PTR(log, "calloc log"))
-		return;
-
-	insns = calloc(insn_cnt, sizeof(*insns));
-	if (!ASSERT_OK_PTR(insns, "calloc insns"))
-		goto free_log;
-
-	gotox_run_fill(insns, gotox_cnt);
-	insns[gotox_cnt + 1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0,
-					    BPF_PSEUDO_CALL, 0,
-					    sub_start - (gotox_cnt + 1) - 1);
-	insns[gotox_cnt + 2] = BPF_EXIT_INSN();
-
-	gotox_run_fill(insns + sub_start, gotox_cnt);
-	insns[sub_start + gotox_cnt + 1] = BPF_MOV64_IMM(BPF_REG_0, 0);
-	insns[sub_start + gotox_cnt + 2] = BPF_EXIT_INSN();
-
-	map_fd[0] = gotox_jt_create(1, gotox_cnt);
-	if (map_fd[0] < 0)
-		goto free_insns;
-	map_fd[1] = gotox_jt_create(sub_start + 1, gotox_cnt);
-	if (map_fd[1] < 0)
-		goto close_maps;
-
-	err = gotox_prog_load(insns, insn_cnt, map_fd, 2, log);
-	check_gotox_limit_hit(log, err);
-
-close_maps:
-	close(map_fd[0]);
-	close(map_fd[1]);
-free_insns:
-	free(insns);
-free_log:
-	free(log);
-}
-
 static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
 {
 	struct bpf_insn_array_value val = {};
@@ -1501,12 +1267,6 @@ static void __test_bpf_insn_array(void)
 		check_no_map_reuse();
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
-	if (test__start_subtest("too-many-gotox-edges"))
-		check_too_many_gotox_edges();
-	if (test__start_subtest("gotox-edges-at-limit"))
-		check_gotox_edges_at_limit();
-	if (test__start_subtest("gotox-edges-across-subprogs"))
-		check_gotox_edges_across_subprogs();
 	if (test__start_subtest("gotox-jt-spans-subprogs"))
 		check_gotox_jt_spans_subprogs();
 	if (test__start_subtest("gotox-jt-spans-with-own-table"))
tools/testing/selftests/bpf/progs/verifier_gotox.c +94 -0

100 lines not on this page · raw message

author maintainer
#23 lore raw

PATCH v2 9/11 selftests/bpf: Add tests for indirect jumps across subprograms

In reply to Daniel Borkmann

Add various gotox corner case tests for indirect jumps across subprograms.

# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array,verifier_gotox [...]

  #24/13   bpf_insn_array/gotox-jt-spans-subprogs:OK
  #24/14   bpf_insn_array/gotox-jt-spans-with-own-table:OK
  #24/15   bpf_insn_array/gotox-target-other-subprog:OK
  #24/16   bpf_insn_array/gotox-jt-per-subprog:OK
  #24/17   bpf_insn_array/gotox-target-subprog-from-main:OK
  #24/18   bpf_insn_array/gotox-index-slice-other-subprog:OK
  #24/19   bpf_insn_array/gotox-target-other-global-subprog:OK
  #24/20   bpf_insn_array/gotox-callback-leaves-subprog:OK
  #24      bpf_insn_array:OK

[...]
#618 verifier_gotox:OK
Summary: 2/49 PASSED, 0 SKIPPED, 0/0 FAILED

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 555 +++++++++++++++++-
 1 file changed, 553 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +553 -2
index 08f5d4f3d64f..d6ebb4c37aea 100644
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 
 #include <bpf/bpf.h>
+#include <bpf/btf.h>
 #include <test_progs.h>
 
 #if defined(__x86_64__) || defined(__powerpc__) || defined(__aarch64__)
@@ -487,8 +488,9 @@ static int gotox_jt_create(__u32 first_gotox, __u32 gotox_cnt)
 	return -1;
 }
 
-static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
-			   int *fd_array, __u32 fd_array_cnt, char *log)
+static int gotox_prog_load_funcs(struct bpf_insn *insns, __u32 insn_cnt,
+				 int *fd_array, __u32 fd_array_cnt, char *log,
+				 int btf_fd, struct bpf_func_info *fi, __u32 fi_cnt)
 {
 	LIBBPF_OPTS(bpf_prog_load_opts, opts);
 	int prog_fd;
@@ -499,6 +501,12 @@ static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
 	opts.log_buf = log;
 	opts.log_size = GOTOX_LOG_SZ;
 	opts.log_level = 1;
+	if (fi_cnt) {
+		opts.prog_btf_fd = btf_fd;
+		opts.func_info = fi;
+		opts.func_info_cnt = fi_cnt;
+		opts.func_info_rec_size = sizeof(*fi);
+	}
 
 	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
 	if (prog_fd >= 0) {
@@ -508,6 +516,13 @@ static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
 	return prog_fd;
 }
 
+static int gotox_prog_load(struct bpf_insn *insns, __u32 insn_cnt,
+			   int *fd_array, __u32 fd_array_cnt, char *log)
+{
+	return gotox_prog_load_funcs(insns, insn_cnt, fd_array, fd_array_cnt, log,
+				     -1, NULL, 0);
+}
+
 /* Fill in 'r1 = 0; gotox_cnt x gotox r1' at 'insns'. */
 static void gotox_run_fill(struct bpf_insn *insns, __u32 gotox_cnt)
 {
@@ -710,6 +725,518 @@ static void check_gotox_edges_across_subprogs(void)
 	free(log);
 }
 
+static int gotox_jt_create_offs(const __u32 *offs, __u32 cnt)
+{
+	struct bpf_insn_array_value val = {};
+	int map_fd;
+	__u32 i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return map_fd;
+
+	for (i = 0; i < cnt; i++) {
+		val.orig_off = offs[i];
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0,
+			       "bpf_map_update_elem"))
+			goto err;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto err;
+
+	return map_fd;
+err:
+	close(map_fd);
+	return -1;
+}
+
+#define GOTOX_SUB_START		4
+#define GOTOX_MAIN_TGT		2
+#define GOTOX_SUB_TGT		8
+#define GOTOX_TWO_INSN_CNT	10
+
+static void gotox_two_subprogs_fill(struct bpf_insn *insns, __u32 jt_idx, __u32 jt_off)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SUB_START - 1 - 1);
+	insns[GOTOX_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_EXIT_INSN();
+
+	/* r1 = &jt[0], by index 'jt_idx' into fd_array */
+	insns[GOTOX_SUB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = jt_idx,
+	};
+	insns[GOTOX_SUB_START + 1] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, jt_off * 8);
+	insns[7] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[9] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_other_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static void check_gotox_jt_per_subprog(void)
+{
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 1, 0);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, 0, "bpf(BPF_PROG_LOAD)");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_FWD_GOTOX		11
+#define GOTOX_FWD_OWN_TGT	12
+#define GOTOX_FWD_SUB_START	14
+#define GOTOX_FWD_INSN_CNT	16
+
+static void gotox_from_main_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_REG(BPF_REG_6, BPF_REG_1);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_FWD_SUB_START - 1 - 1);
+	insns[2] = BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6,
+			       offsetof(struct xdp_md, ingress_ifindex));
+	insns[3] = BPF_JMP_IMM(BPF_JEQ, BPF_REG_2, 0, 4);
+
+	/* r1 = &jt_leaves[0], by index 1 into fd_array */
+	insns[4] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 1,
+	};
+	insns[5] = (struct bpf_insn) { .imm = 0 };
+	insns[6] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[7] = BPF_JMP_A(3);
+
+	/* r1 = &jt_own[0], by index 0 into fd_array */
+	insns[8] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_FWD_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_FWD_OWN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+	insns[GOTOX_FWD_SUB_START] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[15] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_target_subprog_from_main(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 14 leaves the subprog [0,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The only map of the subprog holding the gotox reaches past that subprog, so
+ * the subprog is left without a jump table at all.
+ */
+static void check_gotox_jt_spans_subprogs(void)
+{
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	const __u32 jt_leaves[] = { GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_leaves, ARRAY_SIZE(jt_leaves));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+/*
+ * The subprog holding the gotox has a well formed jump table of its own and
+ * also collects a map that reaches past its end. The spanning map is still
+ * rejected, even though the subprog is not left without a table.
+ */
+static void check_gotox_jt_spans_with_own_table(void)
+{
+	const __u32 jt_own[] = { GOTOX_FWD_OWN_TGT };
+	const __u32 jt_span[] = { GOTOX_FWD_OWN_TGT, GOTOX_FWD_SUB_START };
+	struct bpf_insn insns[GOTOX_FWD_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_from_main_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_own, ARRAY_SIZE(jt_own));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_span, ARRAY_SIZE(jt_span));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "jump table of subprog starting at 0 spans multiple subprogs",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+#define GOTOX_SLICE_SUB_START	6
+#define GOTOX_SLICE_GOTOX	14
+#define GOTOX_SLICE_SUB_TGT	15
+#define GOTOX_SLICE_INSN_CNT	17
+
+static void gotox_slice_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[1] = BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_CALL, 0,
+				GOTOX_SLICE_SUB_START - 1 - 1);
+	insns[2] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[3] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[5] = BPF_EXIT_INSN();
+
+	insns[GOTOX_SLICE_SUB_START] =
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1,
+			    offsetof(struct xdp_md, ingress_ifindex));
+	insns[7] = BPF_ALU64_IMM(BPF_AND, BPF_REG_2, 1);
+	insns[8] = BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, 1);
+	insns[9] = BPF_ALU64_IMM(BPF_LSH, BPF_REG_2, 3);
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[10] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[11] = (struct bpf_insn) { .imm = 0 };
+	insns[12] = BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2);
+	insns[13] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+
+	insns[GOTOX_SLICE_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_SLICE_SUB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 1);
+	insns[16] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_index_slice_other_subprog(void)
+{
+	const __u32 jt_main[] = { 2, 3, 4 };
+	const __u32 jt_sub[] = { GOTOX_SLICE_SUB_TGT };
+	struct bpf_insn insns[GOTOX_SLICE_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_slice_fill(insns);
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_log;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load(insns, ARRAY_SIZE(insns), map_fd, 2, log);
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 14 to 3 leaves the subprog [6,17)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_log:
+	free(log);
+}
+
+static int gotox_btf_create(const __u32 *starts, const __u8 *linkage, __u32 cnt,
+			    struct bpf_func_info *fi, struct btf **pbtf)
+{
+	int int_id, proto_id, id;
+	struct btf *btf;
+	char name[24];
+	__u32 i;
+
+	btf = btf__new_empty();
+	if (!ASSERT_OK_PTR(btf, "btf__new_empty"))
+		return -1;
+
+	int_id = btf__add_int(btf, "int", 4, BTF_INT_SIGNED);
+	if (!ASSERT_GT(int_id, 0, "btf__add_int"))
+		goto err;
+
+	proto_id = btf__add_func_proto(btf, int_id);
+	if (!ASSERT_GT(proto_id, 0, "btf__add_func_proto"))
+		goto err;
+
+	for (i = 0; i < cnt; i++) {
+		snprintf(name, sizeof(name), "gotox_f%u", i);
+		id = btf__add_func(btf, name, linkage[i], proto_id);
+		if (!ASSERT_GT(id, 0, "btf__add_func"))
+			goto err;
+		fi[i].insn_off = starts[i];
+		fi[i].type_id = id;
+	}
+
+	if (!ASSERT_OK(btf__load_into_kernel(btf), "btf__load_into_kernel"))
+		goto err;
+
+	*pbtf = btf;
+	return btf__fd(btf);
+err:
+	btf__free(btf);
+	return -1;
+}
+
+static void check_gotox_target_other_global_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_SUB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_GLOBAL };
+	const __u32 jt_main[] = { GOTOX_MAIN_TGT };
+	const __u32 jt_sub[] = { GOTOX_SUB_TGT };
+	struct bpf_insn insns[GOTOX_TWO_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_two_subprogs_fill(insns, 0, 0);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_sub, ARRAY_SIZE(jt_sub));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 7 to 2 leaves the subprog [4,10)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
+#define GOTOX_CB_MAIN_TGT	6
+#define GOTOX_CB_START		8
+#define GOTOX_CB_GOTOX		11
+#define GOTOX_CB_TGT		12
+#define GOTOX_CB_INSN_CNT	14
+
+static void gotox_callback_fill(struct bpf_insn *insns)
+{
+	insns[0] = BPF_MOV64_IMM(BPF_REG_1, 1);
+	/* r2 = &callback */
+	insns[1] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_2,
+		.src_reg = BPF_PSEUDO_FUNC,
+		.imm = GOTOX_CB_START - 1 - 1,
+	};
+	insns[2] = (struct bpf_insn) { .imm = 0 };
+	insns[3] = BPF_MOV64_IMM(BPF_REG_3, 0);
+	insns[4] = BPF_MOV64_IMM(BPF_REG_4, 0);
+	insns[5] = BPF_EMIT_CALL(BPF_FUNC_loop);
+	insns[GOTOX_CB_MAIN_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[7] = BPF_EXIT_INSN();
+
+	/* r1 = &jt_main[0], by index 0 into fd_array */
+	insns[GOTOX_CB_START] = (struct bpf_insn) {
+		.code = BPF_LD | BPF_DW | BPF_IMM,
+		.dst_reg = BPF_REG_1,
+		.src_reg = BPF_PSEUDO_MAP_IDX_VALUE,
+		.imm = 0,
+	};
+	insns[9] = (struct bpf_insn) { .imm = 0 };
+	insns[10] = BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0);
+	insns[GOTOX_CB_GOTOX] = BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0);
+	insns[GOTOX_CB_TGT] = BPF_MOV64_IMM(BPF_REG_0, 0);
+	insns[13] = BPF_EXIT_INSN();
+}
+
+static void check_gotox_callback_leaves_subprog(void)
+{
+	const __u32 starts[] = { 0, GOTOX_CB_START };
+	const __u8 linkage[] = { BTF_FUNC_GLOBAL, BTF_FUNC_STATIC };
+	const __u32 jt_main[] = { GOTOX_CB_MAIN_TGT };
+	const __u32 jt_cb[] = { GOTOX_CB_TGT };
+	struct bpf_insn insns[GOTOX_CB_INSN_CNT];
+	int map_fd[2] = { -1, -1 };
+	struct bpf_func_info fi[2];
+	struct btf *btf = NULL;
+	int btf_fd;
+	char *log;
+	int err;
+
+	log = calloc(1, GOTOX_LOG_SZ);
+	if (!ASSERT_OK_PTR(log, "calloc log"))
+		return;
+
+	gotox_callback_fill(insns);
+
+	btf_fd = gotox_btf_create(starts, linkage, ARRAY_SIZE(starts), fi, &btf);
+	if (btf_fd < 0)
+		goto free_log;
+
+	map_fd[0] = gotox_jt_create_offs(jt_main, ARRAY_SIZE(jt_main));
+	if (map_fd[0] < 0)
+		goto free_btf;
+	map_fd[1] = gotox_jt_create_offs(jt_cb, ARRAY_SIZE(jt_cb));
+	if (map_fd[1] < 0)
+		goto close_maps;
+
+	err = gotox_prog_load_funcs(insns, ARRAY_SIZE(insns), map_fd, 2, log,
+				    btf_fd, fi, ARRAY_SIZE(fi));
+	ASSERT_EQ(err, -EINVAL, "program should have been rejected");
+	ASSERT_HAS_SUBSTR(log, "indirect jump from insn 11 to 6 leaves the subprog [8,14)",
+			  "verifier log");
+
+close_maps:
+	close(map_fd[0]);
+	close(map_fd[1]);
+free_btf:
+	btf__free(btf);
+free_log:
+	free(log);
+}
+
 static void check_bpf_side(void)
 {
 	check_bpf_no_lookup();
@@ -756,6 +1283,30 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("gotox-edges-across-subprogs"))
 		check_gotox_edges_across_subprogs();
+
+	if (test__start_subtest("gotox-jt-spans-subprogs"))
+		check_gotox_jt_spans_subprogs();
+
+	if (test__start_subtest("gotox-jt-spans-with-own-table"))
+		check_gotox_jt_spans_with_own_table();
+
+	if (test__start_subtest("gotox-target-other-subprog"))
+		check_gotox_target_other_subprog();
+
+	if (test__start_subtest("gotox-jt-per-subprog"))
+		check_gotox_jt_per_subprog();
+
+	if (test__start_subtest("gotox-target-subprog-from-main"))
+		check_gotox_target_subprog_from_main();
+
+	if (test__start_subtest("gotox-index-slice-other-subprog"))
+		check_gotox_index_slice_other_subprog();
+
+	if (test__start_subtest("gotox-target-other-global-subprog"))
+		check_gotox_target_other_global_subprog();
+
+	if (test__start_subtest("gotox-callback-leaves-subprog"))
+		check_gotox_callback_leaves_subprog();
 }
 #else
 static void __test_bpf_insn_array(void)
Signature · 1 line
2.43.0
author maintainer
#24 lore raw

PATCH v2 10/11 selftests/bpf: Add tests for gotox nop targets and insn array reuse

In reply to Daniel Borkmann

Add two tests to the bpf_insn_array suite which excercise gotox target which is a ja +0, and the insn array reuse which initially fails the post-JIT stage.

# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array [...]

  #24/21   bpf_insn_array/gotox-target-nop:OK
  #24/22   bpf_insn_array/insn-array-stale-reuse:OK
  #24      bpf_insn_array:OK

Summary: 1/22 PASSED, 0 SKIPPED, 0/0 FAILED

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 140 ++++++++++++++++++
 1 file changed, 140 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +140 -0
index d6ebb4c37aea..9908e538c022 100644
@@ -1242,6 +1242,140 @@ static void check_bpf_side(void)
 	check_bpf_no_lookup();
 }
 
+static void check_gotox_target_nop(void)
+{
+	struct bpf_insn insns[] = {
+		/* r1 = &jt[0] */
+		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: the nop target */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 4;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target must not be INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target must have a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
+static void check_insn_array_stale_reuse(void)
+{
+	/* helper of a different prog type, to set the prog array's owner */
+	struct bpf_insn sf[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	/*
+	 * prog1 binds the insn array (entry points at a real insn, so the JIT
+	 * fills it in) and tail calls a prog array whose owner is now a
+	 * different type, which fails in the post-JIT compatibility check.
+	 */
+	struct bpf_insn prog1[] = {
+		BPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */
+		BPF_MOV64_IMM(BPF_REG_3, 0),			   /* r3 = index */
+		BPF_EMIT_CALL(BPF_FUNC_tail_call),
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),		   /* insn 4: real entry */
+		BPF_EXIT_INSN(),
+	};
+	/*
+	 * prog2 reuses the insn array as a tracker. Its entry points at a nop
+	 * that the nop pass removes, so the entry becomes INSN_DELETED and the
+	 * JIT skips it -- exactly the slot a stale pointer would linger in.
+	 */
+	struct bpf_insn prog2[] = {
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: tracked nop */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;
+	struct bpf_insn_array_value val = {};
+	int fd_array[2];
+	__u32 key = 0;
+
+	jt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(jt_fd, 0, "insn_array create"))
+		return;
+	val.orig_off = 4;
+	if (!ASSERT_EQ(bpf_map_update_elem(jt_fd, &key, &val, 0), 0, "insn_array update"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, "insn_array freeze"))
+		goto cleanup;
+
+	pa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, "pa", 4, 4, 1, NULL);
+	if (!ASSERT_GE(pa_fd, 0, "prog_array create"))
+		goto cleanup;
+	sf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", sf, ARRAY_SIZE(sf), NULL);
+	if (!ASSERT_GE(sf_fd, 0, "owner prog load"))
+		goto cleanup;
+	/* insert a non-XDP prog to pin the prog array owner to another type */
+	if (!ASSERT_EQ(bpf_map_update_elem(pa_fd, &key, &sf_fd, 0), 0, "prog_array update"))
+		goto cleanup;
+
+	fd_array[0] = jt_fd;
+	fd_array[1] = pa_fd;
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = 2;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog1, ARRAY_SIZE(prog1), &opts);
+	if (!ASSERT_LT(prog_fd, 0, "prog1 must fail the post-JIT tail call check"))
+		goto cleanup;
+
+	/* prog1 reached the JIT before failing, so the entry is now populated */
+	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog1"))
+		goto cleanup;
+	if (!ASSERT_NEQ(val.jitted_off, 0, "prog1 should have filled the jitted address"))
+		goto cleanup;
+
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog2, ARRAY_SIZE(prog2), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "prog2 reuse load"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog2"))
+		goto cleanup;
+	ASSERT_EQ(val.xlated_off, (__u32)-1, "reused entry should be INSN_DELETED");
+	ASSERT_EQ(val.jitted_off, 0, "stale jitted address must be cleared on reuse");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	if (sf_fd >= 0)
+		close(sf_fd);
+	if (pa_fd >= 0)
+		close(pa_fd);
+	if (jt_fd >= 0)
+		close(jt_fd);
+}
+
 static void __test_bpf_insn_array(void)
 {
 	/* Test if offsets are adjusted properly */
@@ -1307,6 +1441,12 @@ static void __test_bpf_insn_array(void)
 
 	if (test__start_subtest("gotox-callback-leaves-subprog"))
 		check_gotox_callback_leaves_subprog();
+
+	if (test__start_subtest("gotox-target-nop"))
+		check_gotox_target_nop();
+
+	if (test__start_subtest("insn-array-stale-reuse"))
+		check_insn_array_stale_reuse();
 }
 #else
 static void __test_bpf_insn_array(void)
Signature · 1 line
2.43.0
#25 lore raw

Re: [PATCH bpf v2 10/11] selftests/bpf: Add tests for gotox nop targets and insn array reuse

This isn't a bug, but could this setup reuse gotox_jt_create_offs() from earlier in the file? The create/update/freeze sequence here is what gotox_jt_create_offs() already does, so map_fd =…
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +0 -0
index d6ebb4c37aea7..9908e538c022b 100644

[ ... ]

Quoted 15 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +15 -0
+static void check_gotox_target_nop(void)
+{
+	struct bpf_insn insns[] = {
+		/* r1 = &jt[0] */
+		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0),
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0), /* gotox r1 */
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: the nop target */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_bpf_side(void) · line 1269

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 4;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;

This isn't a bug, but could this setup reuse gotox_jt_create_offs() from earlier in the file? The create/update/freeze sequence here is what gotox_jt_create_offs() already does, so map_fd = gotox_jt_create_offs((const __u32[]){ 4 }, 1); would express it in one line.

Quoted 16 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +16 -0
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target must not be INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target must have a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}

[ ... ]

Quoted 37 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +37 -0
+static void check_insn_array_stale_reuse(void)
+{
+	/* helper of a different prog type, to set the prog array's owner */
+	struct bpf_insn sf[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	/*
+	 * prog1 binds the insn array (entry points at a real insn, so the JIT
+	 * fills it in) and tail calls a prog array whose owner is now a
+	 * different type, which fails in the post-JIT compatibility check.
+	 */
+	struct bpf_insn prog1[] = {
+		BPF_LD_IMM64_RAW(BPF_REG_2, BPF_PSEUDO_MAP_IDX, 1), /* r2 = prog array */
+		BPF_MOV64_IMM(BPF_REG_3, 0),			   /* r3 = index */
+		BPF_EMIT_CALL(BPF_FUNC_tail_call),
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),		   /* insn 4: real entry */
+		BPF_EXIT_INSN(),
+	};
+	/*
+	 * prog2 reuses the insn array as a tracker. Its entry points at a nop
+	 * that the nop pass removes, so the entry becomes INSN_DELETED and the
+	 * JIT skips it -- exactly the slot a stale pointer would linger in.
+	 */
+	struct bpf_insn prog2[] = {
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* nop */
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0),		/* insn 4: tracked nop */
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS),
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	int jt_fd = -1, pa_fd = -1, sf_fd = -1, prog_fd = -1;
+	struct bpf_insn_array_value val = {};
+	int fd_array[2];

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_bpf_side(void) · line 1333

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+	__u32 key = 0;
+
+	jt_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(jt_fd, 0, "insn_array create"))
+		return;
+	val.orig_off = 4;
+	if (!ASSERT_EQ(bpf_map_update_elem(jt_fd, &key, &val, 0), 0, "insn_array update"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(jt_fd), 0, "insn_array freeze"))
+		goto cleanup;

This isn't a bug, but could this setup also reuse gotox_jt_create_offs()? The same create/update/freeze sequence appears here as well.

Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +14 -0
+
+	pa_fd = bpf_map_create(BPF_MAP_TYPE_PROG_ARRAY, "pa", 4, 4, 1, NULL);
+	if (!ASSERT_GE(pa_fd, 0, "prog_array create"))
+		goto cleanup;
+	sf_fd = bpf_prog_load(BPF_PROG_TYPE_SOCKET_FILTER, NULL, "GPL", sf, ARRAY_SIZE(sf), NULL);
+	if (!ASSERT_GE(sf_fd, 0, "owner prog load"))
+		goto cleanup;
+	/* insert a non-XDP prog to pin the prog array owner to another type */
+	if (!ASSERT_EQ(bpf_map_update_elem(pa_fd, &key, &sf_fd, 0), 0, "prog_array update"))
+		goto cleanup;
+
+	fd_array[0] = jt_fd;
+	fd_array[1] = pa_fd;
+	opts.fd_array = fd_array;

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_bpf_side(void) · line 1357

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+	opts.fd_array_cnt = 2;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog1, ARRAY_SIZE(prog1), &opts);
+	if (!ASSERT_LT(prog_fd, 0, "prog1 must fail the post-JIT tail call check"))
+		goto cleanup;
+
+	/* prog1 reached the JIT before failing, so the entry is now populated */
+	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog1"))
+		goto cleanup;
+	if (!ASSERT_NEQ(val.jitted_off, 0, "prog1 should have filled the jitted address"))
+		goto cleanup;

A subsystem pattern flags this as potentially concerning: the negative load of prog1 is only checked with ASSERT_LT(prog_fd, 0), so the test accepts any failure.

The assertion immediately after it (ASSERT_NEQ(val.jitted_off, 0)) is only meaningful if prog1 failed specifically in the post-JIT bpf_check_tail_call() stage in __bpf_prog_select_runtime(). Any earlier failure also satisfies ASSERT_LT, and the test then reports "prog1 should have filled the jitted address" instead of the real cause.

For example, if the JIT is off, prog1 hits -ENOTSUPP from:

kernel/bpf/core.c:__bpf_prog_select_runtime() {
    ...
    if (!fp->jited && jit_needed)
        return -ENOTSUPP;
    ...
}

Or if a future change moves the prog_array owner-type check into check_map_prog_compatibility(), the verifier would reject prog1 before it reaches the JIT.

Every other negative case in this file pins the code instead. In check_gotox_target_other_subprog():

ASSERT_EQ(err, -EINVAL, "program should have been rejected")

And in check_no_map_reuse():

ASSERT_EQ(prog_fd, -EINVAL, ...)

Would asserting prog_fd == -EINVAL here keep the test self-diagnosing?

On the tree as of this commit, prog1 does fail exactly at bpf_check_tail_call(), so the test is correct today. The question is whether it remains self-diagnosing if the kernel behavior changes.

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", prog2, ARRAY_SIZE(prog2), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "prog2 reuse load"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(jt_fd, &key, &val), 0, "lookup after prog2"))
+		goto cleanup;
+	ASSERT_EQ(val.xlated_off, (__u32)-1, "reused entry should be INSN_DELETED");
+	ASSERT_EQ(val.jitted_off, 0, "stale jitted address must be cleared on reuse");

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_bpf_side(void) · line 1377

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	if (sf_fd >= 0)
+		close(sf_fd);
+	if (pa_fd >= 0)
+		close(pa_fd);
+	if (jt_fd >= 0)
+		close(jt_fd);
+}

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886

maintainer
#26 lore raw

Re: [PATCH bpf v2 10/11] selftests/bpf: Add tests for gotox nop targets and insn array reuse

In reply to Daniel Borkmann
Quoted 11 lines · Daniel Borkmann

Add two tests to the bpf_insn_array suite which excercise gotox target which is a ja +0, and the insn array reuse which initially fails the post-JIT stage.

  # LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array [...]
  #24/21   bpf_insn_array/gotox-target-nop:OK
  #24/22   bpf_insn_array/insn-array-stale-reuse:OK
  #24      bpf_insn_array:OK
  Summary: 1/22 PASSED, 0 SKIPPED, 0/0 FAILED

Quoted 3 lines

---

Tbh, having a full stale ips issure repro seem to be an overkill. I'd move the first test to inline assembly with __xlated or __jited matcher and drop the insn-array-stale-reuse. ¯\_(ツ)_/¯

...

author maintainer
#27 lore raw

PATCH v2 11/11 selftests/bpf: Add tests for gotox targets across prologue prepends

In reply to Daniel Borkmann

Add tests to the bpf_insn_array suite for a gotox whose jump table points at the first instruction of a subprogram.

# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t bpf_insn_array [...]

  #24/23   bpf_insn_array/gotox-target-prologue-shift:OK
  #24/24   bpf_insn_array/gotox-target-ctx-prologue-shift:OK
  #24      bpf_insn_array:OK

Summary: 1/24 PASSED, 0 SKIPPED, 0/0 FAILED

As small nit, cleanup the extra whitespaces between subtests.

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 130 ++++++++++++++----
 1 file changed, 107 insertions(+), 23 deletions(-)
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +107 -23
index 9908e538c022..f102435675c2 100644
@@ -1376,77 +1376,161 @@ static void check_insn_array_stale_reuse(void)
 		close(jt_fd);
 }
 
+static void check_gotox_target_prologue_shift(void)
+{
+	struct bpf_insn insns[] = {
+		/* insn 0: gotox target and subprog start */
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		/* may_goto +4 -> exit block, bounds the loop */
+		BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),
+		/* r1 = &jt[0] (insns 2 and 3) */
+		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */
+		/* insn 5: gotox r1 -> insn 0 */
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 0;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
+static void check_gotox_target_ctx_prologue_shift(void)
+{
+	struct bpf_insn insns[] = {
+		/* insn 0: gotox target and subprog start */
+		BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),
+		BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */
+		BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),
+		BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),
+		BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),
+		BPF_MOV64_REG(BPF_REG_4, BPF_REG_2),
+		BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),
+		BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),
+		/* insn 9: direct packet write -> tc unclone prologue prepended at insn 0 */
+		BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),
+		/* r5 = &jt[0] (insns 10 and 11) */
+		BPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */
+		/* insn 13: gotox r5 -> insn 0 */
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),
+		BPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 0;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns,
+				ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
 static void __test_bpf_insn_array(void)
 {
 	/* Test if offsets are adjusted properly */
-
 	if (test__start_subtest("one2one"))
 		check_one_to_one_mapping();
-
 	if (test__start_subtest("simple"))
 		check_simple();
-
 	if (test__start_subtest("deletions"))
 		check_deletions();
-
 	if (test__start_subtest("deletions-with-functions"))
 		check_deletions_with_functions();
-
 	if (test__start_subtest("blindness"))
 		check_blindness();
-
 	/* Check all kinds of operations and related restrictions */
-
 	if (test__start_subtest("incorrect-index"))
 		check_incorrect_index();
-
 	if (test__start_subtest("load-unfrozen-map"))
 		check_load_unfrozen_map();
-
 	if (test__start_subtest("no-map-reuse"))
 		check_no_map_reuse();
-
 	if (test__start_subtest("bpf-side-ops"))
 		check_bpf_side();
-
 	if (test__start_subtest("too-many-gotox-edges"))
 		check_too_many_gotox_edges();
-
 	if (test__start_subtest("gotox-edges-at-limit"))
 		check_gotox_edges_at_limit();
-
 	if (test__start_subtest("gotox-edges-across-subprogs"))
 		check_gotox_edges_across_subprogs();
-
 	if (test__start_subtest("gotox-jt-spans-subprogs"))
 		check_gotox_jt_spans_subprogs();
-
 	if (test__start_subtest("gotox-jt-spans-with-own-table"))
 		check_gotox_jt_spans_with_own_table();
-
 	if (test__start_subtest("gotox-target-other-subprog"))
 		check_gotox_target_other_subprog();
-
 	if (test__start_subtest("gotox-jt-per-subprog"))
 		check_gotox_jt_per_subprog();
-
 	if (test__start_subtest("gotox-target-subprog-from-main"))
 		check_gotox_target_subprog_from_main();
-
 	if (test__start_subtest("gotox-index-slice-other-subprog"))
 		check_gotox_index_slice_other_subprog();
-
 	if (test__start_subtest("gotox-target-other-global-subprog"))
 		check_gotox_target_other_global_subprog();
-
 	if (test__start_subtest("gotox-callback-leaves-subprog"))
 		check_gotox_callback_leaves_subprog();
-
 	if (test__start_subtest("gotox-target-nop"))
 		check_gotox_target_nop();
-
 	if (test__start_subtest("insn-array-stale-reuse"))
 		check_insn_array_stale_reuse();
+	if (test__start_subtest("gotox-target-prologue-shift"))
+		check_gotox_target_prologue_shift();
+	if (test__start_subtest("gotox-target-ctx-prologue-shift"))
+		check_gotox_target_ctx_prologue_shift();
 }
 #else
 static void __test_bpf_insn_array(void)
Signature · 1 line
2.43.0
#28 lore raw

Re: [PATCH bpf v2 11/11] selftests/bpf: Add tests for gotox targets across prologue prepends

This isn't a bug, but the ~30-line body after the insns[] array in check_gotox_target_ctx_prologue_shift() is a verbatim copy of the same block in check_gotox_target_prologue_shift() above. The only…
Quoted 100 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +92 -0
index 9908e538c022b..f102435675c2b 100644
@@ -1376,77 +1376,161 @@ static void check_insn_array_stale_reuse(void)
 		close(jt_fd);
 }
 
+static void check_gotox_target_prologue_shift(void)
+{
+	struct bpf_insn insns[] = {
+		/* insn 0: gotox target and subprog start */
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		/* may_goto +4 -> exit block, bounds the loop */
+		BPF_RAW_INSN(BPF_JMP | BPF_JCOND, 0, 0, 4, 0),
+		/* r1 = &jt[0] (insns 2 and 3) */
+		BPF_LD_IMM64_RAW(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_1, 0), /* r1 = ips[0] */
+		/* insn 5: gotox r1 -> insn 0 */
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_1, 0, 0, 0),
+		BPF_MOV64_IMM(BPF_REG_0, XDP_PASS), /* insn 6: exit block */
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 0;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}
+
+static void check_gotox_target_ctx_prologue_shift(void)
+{
+	struct bpf_insn insns[] = {
+		/* insn 0: gotox target and subprog start */
+		BPF_LDX_MEM(BPF_DW, BPF_REG_7, BPF_REG_10, -8),
+		BPF_JMP_IMM(BPF_JEQ, BPF_REG_7, 0x5a5a, 12), /* second visit exits */
+		BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0x5a5a),
+		BPF_MOV64_REG(BPF_REG_6, BPF_REG_1), /* r6 = ctx */
+		BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_6, offsetof(struct __sk_buff, data)),
+		BPF_LDX_MEM(BPF_W, BPF_REG_3, BPF_REG_6, offsetof(struct __sk_buff, data_end)),
+		BPF_MOV64_REG(BPF_REG_4, BPF_REG_2),
+		BPF_ALU64_IMM(BPF_ADD, BPF_REG_4, 1),
+		BPF_JMP_REG(BPF_JGT, BPF_REG_4, BPF_REG_3, 5),
+		/* insn 9: direct packet write -> tc unclone prologue prepended at insn 0 */
+		BPF_ST_MEM(BPF_B, BPF_REG_2, 0, 0),
+		/* r5 = &jt[0] (insns 10 and 11) */
+		BPF_LD_IMM64_RAW(BPF_REG_5, BPF_PSEUDO_MAP_IDX_VALUE, 0),
+		BPF_LDX_MEM(BPF_DW, BPF_REG_5, BPF_REG_5, 0), /* r5 = ips[0] */
+		/* insn 13: gotox r5 -> insn 0 */
+		BPF_RAW_INSN(BPF_JMP | BPF_JA | BPF_X, BPF_REG_5, 0, 0, 0),
+		BPF_MOV64_IMM(BPF_REG_0, 0), /* insn 14: exit block */
+		BPF_EXIT_INSN(),
+	};
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd;
+	__u32 key = 0;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	val.orig_off = 0;
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	opts.fd_array = &map_fd;
+	opts.fd_array_cnt = 1;
+	prog_fd = bpf_prog_load(BPF_PROG_TYPE_SCHED_CLS, NULL, "GPL", insns,
+				ARRAY_SIZE(insns), &opts);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c static void check_insn_array_stale_reuse(void) · line 1480

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +10 -0
+	if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &key, &val), 0, "bpf_map_lookup_elem"))
+		goto cleanup;
+	ASSERT_NEQ(val.xlated_off, 0, "gotox target retargeted past prepend");
+	ASSERT_NEQ(val.xlated_off, (__u32)-1, "gotox target not INSN_DELETED");
+	ASSERT_NEQ(val.jitted_off, 0, "gotox target has a jitted address");
+cleanup:
+	if (prog_fd >= 0)
+		close(prog_fd);
+	close(map_fd);
+}

This isn't a bug, but the ~30-line body after the insns[] array in check_gotox_target_ctx_prologue_shift() is a verbatim copy of the same block in check_gotox_target_prologue_shift() above. The only differences are BPF_PROG_TYPE_SCHED_CLS vs BPF_PROG_TYPE_XDP and the wrapping of the bpf_prog_load() call.

The file's earlier check_gotox_target_nop() test (added in the same series) is a third near-copy of the same map-create/update/freeze/load/lookup/assert/cleanup sequence, differing only in orig_off and the assertion labels. The file otherwise tends to factor this kind of setup out with helpers like map_create(), prog_load(), gotox_jt_create(), gotox_prog_load(), and __check_success().

Could these three tests share a small helper - something like check_gotox_target(prog_type, insns, insn_cnt, orig_off) - instead of repeating the map create/freeze/load/lookup/cleanup sequence a third time?

Quoted 24 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +0 -7
 static void __test_bpf_insn_array(void)
 {
 	/* Test if offsets are adjusted properly */
-
 	if (test__start_subtest("one2one"))
 		check_one_to_one_mapping();
-
 	if (test__start_subtest("simple"))
 		check_simple();
-
 	if (test__start_subtest("deletions"))
 		check_deletions();
-
 	if (test__start_subtest("deletions-with-functions"))
 		check_deletions_with_functions();
-
 	if (test__start_subtest("blindness"))
 		check_blindness();
-
 	/* Check all kinds of operations and related restrictions */
-
 	if (test__start_subtest("incorrect-index"))
 		check_incorrect_index();

[ ... ]

Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +4 -1
-
 	if (test__start_subtest("insn-array-stale-reuse"))
 		check_insn_array_stale_reuse();
+	if (test__start_subtest("gotox-target-prologue-shift"))
+		check_gotox_target_prologue_shift();
+	if (test__start_subtest("gotox-target-ctx-prologue-shift"))
+		check_gotox_target_ctx_prologue_shift();

}

This isn't a bug, but besides adding the two new subtests, the patch drops the blank line between every existing subtest in __test_bpf_insn_array(), which accounts for 44 of the 130 changed lines. It is a pure reflow with no bearing on the new tests, and it also removes the blank line that used to set the two grouping comments apart, so '/* Test if offsets are adjusted properly */' and '/* Check all kinds of operations and related restrictions */' now abut the surrounding calls and no longer read as section headers.

Would it be worth splitting the blank-line cleanup in __test_bpf_insn_array() into its own patch, so the diff for the two new subtests stays small - and keeping a blank line before the two grouping comments so they still stand out?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34875126886