Skip to contentSkip to conversation

bpf: infer zext_dst based on static register liveness analysis

PATCH Superseded Eduard Zingerman 3 participants 14 messages Jul 31

Subsystems (5)
Kselftest BPF BPF selftests BPF core +1 more subsystems

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Eduard Zingerman

author maintainer
#0 lore raw

cover bpf: infer zext_dst based on static register liveness analysis

Min-gyu Kim reported a bug in 32-bit operations zero extension handling [1]. The bug was introduced by the commit [2]. The tl;dr of the bug is that the verifier does not carry zero extension marks across pruning points. The detailed mechanism is described in patch #3.

Fix this by reworking zero extension logic to avoid main-path based subreg_def tracking, and instead extend the live registers analysis to track upper register halves' liveness.

As noted in the commit message for patch #3:

There is one notable drop in precision: whenever a BPF subprogram is called, all 64 bits of parameter registers are presumed to be used. The assumption is that such a drop in precision would not inflict noticeable performance penalty.

Ilya, could you please help with testing this conjecture? Min-gyu, could you please test the proposed fix against your reproducer?

Side note:
Patch #1 is a small refactoring for disasm.c, as patch #3 adds a new location where bpf_verbose_insn() is called and there as well I have to wrangle with the newline added by the disasm code.

[1] https://lore.kernel.org/bpf/CAGKGUv=sOuqQtA1Ub-5JXfA4FPosJFYKAQE4B79cK+P1erxqtg@mail.gmail.com/
[2] commit 107e16979905 ("bpf: disable and remove registers chain based liveness")

---
Eduard Zingerman (5):
bpf: do not print a newline after disassembly in bpf_verbose_insn()
bpf: track upper 32-bit register halves' liveness in compute_live_registers()
bpf: infer zext_dst based on static register liveness analysis
bpf: simplify the bpf_is_reg64() signature
selftests/bpf: verify zext_dst annotations for various instructions

 include/linux/bpf_verifier.h                      |  10 +-
 kernel/bpf/backtrack.c                            |   1 +
 kernel/bpf/disasm.c                               |  68 +++---
 kernel/bpf/fixups.c                               |  19 +-
 kernel/bpf/liveness.c                             | 139 +++++++++---
 kernel/bpf/verifier.c                             | 150 ++-----------
 tools/bpf/bpftool/xlated_dumper.c                 |  10 +-
 tools/testing/selftests/bpf/disasm_helpers.c      |   3 +-
 tools/testing/selftests/bpf/prog_tests/verifier.c |   2 +
 tools/testing/selftests/bpf/progs/verifier_zext.c | 260 ++++++++++++++++++++++
 10 files changed, 433 insertions(+), 229 deletions(-)

---
base-commit: fdec474c65fd35d5a6e1497ed50a9f98c07192f0
change-id: 20260731-static-zext-938cd128273c

author maintainer
#1 lore raw

PATCH 1/5 bpf: do not print a newline after disassembly in bpf_verbose_insn()

At the moment there are more callsites that want bpf_verbose_insn() to not print a newline after the instruction, than callsites that want a newline. Drop '\n' from disasm.c. Non-functional change.

The changes in bpftool are verified by writing a bpf program using a variety of instructions and comparing prog dump xlated output in the following modes: plain, opcodes, visual, visual opcodes. The output before and after the changes is identical.

---
 kernel/bpf/backtrack.c                       |  1 +
 kernel/bpf/disasm.c                          | 68 ++++++++++++++--------------
 kernel/bpf/liveness.c                        |  6 +--
 kernel/bpf/verifier.c                        |  1 +
 tools/bpf/bpftool/xlated_dumper.c            | 10 ++--
 tools/testing/selftests/bpf/disasm_helpers.c |  3 +-
 6 files changed, 42 insertions(+), 47 deletions(-)
kernel/bpf/backtrack.c +1 -0
index 2e4ae0ef0860..7c4599e65ece 100644
@@ -285,6 +285,7 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 		verbose(env, "stack=%s before ", env->tmp_str_buf);
 		verbose(env, "%d: ", idx);
 		bpf_verbose_insn(env, insn);
+		verbose(env, "\n");
 	}
 
 	/* If there is a history record that some registers gained range at this insn,
kernel/bpf/disasm.c +34 -34

219 lines not on this page · raw message

kernel/bpf/liveness.c +2 -4
index 0aadfbae0acc..ff1e68cc4bd1 100644
@@ -497,7 +497,6 @@ static void print_instance(struct bpf_verifier_env *env, struct func_instance *i
 		pos = env->log.end_pos;
 		verbose(env, "%3d: ", insn_idx);
 		bpf_verbose_insn(env, &insns[insn_idx]);
-		bpf_vlog_reset(&env->log, env->log.end_pos - 1); /* remove \n */
 		insn_pos = env->log.end_pos;
 		verbose(env, "%*c;", bpf_vlog_alignment(insn_pos - pos), ' ');
 		pos = env->log.end_pos;
@@ -1043,7 +1042,6 @@ static void arg_track_log(struct bpf_verifier_env *env, struct bpf_insn *insn, i
 		if (!printed) {
 			verbose(env, "%3d: ", idx);
 			bpf_verbose_insn(env, insn);
-			bpf_vlog_reset(&env->log, env->log.end_pos - 1);
 			printed = true;
 		}
 		verbose(env, "\tr%d: ", i); verbose_arg_track(env, &at_in[i]);
@@ -1058,7 +1056,6 @@ static void arg_track_log(struct bpf_verifier_env *env, struct bpf_insn *insn, i
 		if (!printed) {
 			verbose(env, "%3d: ", idx);
 			bpf_verbose_insn(env, insn);
-			bpf_vlog_reset(&env->log, env->log.end_pos - 1);
 			printed = true;
 		}
 		verbose(env, "\tsa%d: ", i); verbose_arg_track(env, &at_in[ai]);
@@ -1070,7 +1067,6 @@ static void arg_track_log(struct bpf_verifier_env *env, struct bpf_insn *insn, i
 		if (!printed) {
 			verbose(env, "%3d: ", idx);
 			bpf_verbose_insn(env, insn);
-			bpf_vlog_reset(&env->log, env->log.end_pos - 1);
 			printed = true;
 		}
 		verbose(env, "\tfp%+d: ", -(i + 1) * 8); verbose_arg_track(env, &at_stack_in[i]);
@@ -1545,6 +1541,7 @@ static void print_subprog_arg_access(struct bpf_verifier_env *env,
 
 		verbose(env, "%3d: ", idx);
 		bpf_verbose_insn(env, &insns[idx]);
+		verbose(env, "\n");
 
 		/* Collect what needs printing */
 		if (is_ldx_stx_call &&
@@ -2285,6 +2282,7 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 					verbose(env, ".");
 			verbose(env, " ");
 			bpf_verbose_insn(env, &insns[i]);
+			verbose(env, "\n");
 			if (bpf_is_ldimm64(&insns[i]))
 				i++;
 		}
kernel/bpf/verifier.c +1 -0
index e6f35f4e715b..6123e7e4a320 100644
@@ -17475,6 +17475,7 @@ static int do_check(struct bpf_verifier_env *env)
 			env->prev_log_pos = env->log.end_pos;
 			verbose(env, "%d: ", env->insn_idx);
 			bpf_verbose_insn(env, insn);
+			verbose(env, "\n");
 			env->prev_insn_print_pos = env->log.end_pos - env->prev_log_pos;
 			env->prev_log_pos = env->log.end_pos;
 		}
tools/bpf/bpftool/xlated_dumper.c +3 -7
index 5e7cb8b36fef..7b33d847d800 100644
@@ -129,16 +129,10 @@ print_insn_for_graph(void *private_data, const char *fmt, ...)
 static void __printf(2, 3)
 print_insn_json(void *private_data, const char *fmt, ...)
 {
-	unsigned int l = strlen(fmt);
-	char chomped_fmt[l];
 	va_list args;
 
 	va_start(args, fmt);
-	if (l > 0) {
-		strncpy(chomped_fmt, fmt, l - 1);
-		chomped_fmt[l - 1] = '\0';
-	}
-	jsonw_vprintf_enquote(json_wtr, chomped_fmt, args);
+	jsonw_vprintf_enquote(json_wtr, fmt, args);
 	va_end(args);
 }
 
@@ -351,6 +345,7 @@ void dump_xlated_plain(struct dump_data *dd, void *buf, unsigned int len,
 
 		printf("%4u: ", i);
 		print_bpf_insn(&cbs, insn + i, true);
+		printf("\n");
 
 		if (opcodes) {
 			printf("       ");
@@ -417,6 +412,7 @@ void dump_xlated_for_graph(struct dump_data *dd, void *buf_start, void *buf_end,
 
 		printf("%u: ", insn_off);
 		print_bpf_insn(&cbs, cur, true);
+		printf("\\l\\\n");
 
 		if (opcodes) {
 			printf("\\ \\ \\ \\ ");
tools/testing/selftests/bpf/disasm_helpers.c +1 -2
index f529f1c8c171..30221352568d 100644
@@ -55,10 +55,9 @@ struct bpf_insn *disasm_insn(struct bpf_insn *insn, char *buf, size_t buf_sz)
 	 * for each instruction (FF stands for instruction `code` byte).
 	 * Remove the prefix inplace, and also simplify call instructions.
 	 * E.g.: "(85) call foo#10" -> "call foo".
-	 * Also remove newline in the end (the 'max(strlen(buf) - 1, 0)' thing).
 	 */
 	pfx_end = buf + 5;
-	sfx_start = buf + max((int)strlen(buf) - 1, 0);
+	sfx_start = buf + (int)strlen(buf);
 	if (strncmp(pfx_end, "call ", 5) == 0 && (tmp = strrchr(buf, '#')))
 		sfx_start = tmp;
 	len = sfx_start - pfx_end;
Signature · 1 line
2.55.0
#2 lore raw

Re: [PATCH bpf-next 1/5] bpf: do not print a newline after disassembly in bpf_verbose_insn()

This isn't a bug, but now that no disasm.c format string carries a newline and dump_xlated_for_graph() emits "\\l\\\n" itself, is the '\n' arm of print_insn_for_graph() still reachable?
Quoted 4 lines
tools/bpf/bpftool/xlated_dumper.c +0 -0
index 5e7cb8b36fef2..7b33d847d800c 100644

[ ... ]

tools/bpf/bpftool/xlated_dumper.c void dump_xlated_for_graph(struct dump_data *dd, void *buf_start, void *buf_end, · line 418

Quoted 8 lines
tools/bpf/bpftool/xlated_dumper.c +1 -0
@@ -417,6 +412,7 @@ void dump_xlated_for_graph(struct dump_data *dd, void *buf_start, void *buf_end,
 
 		printf("%u: ", insn_off);
 		print_bpf_insn(&cbs, cur, true);
+		printf("\\l\\\n");
 
 		if (opcodes) {
 			printf("\\ \\ \\ \\ ");

This isn't a bug, but now that no disasm.c format string carries a newline and dump_xlated_for_graph() emits "\\l\\\n" itself, is the '\n' arm of print_insn_for_graph() still reachable?

Looking at print_insn_for_graph():

static void __printf(2, 3)
print_insn_for_graph(void *private_data, const char *fmt, ...)

{
	...
	p = buf;
	while (*p != '\0') {
		if (*p == '\n') {
			memmove(p + 3, p, strlen(buf) + 1 - (p - buf));
			/* Align each instruction dump row left. */
			*p++ = '\\';
			*p++ = 'l';
			/* Output multiline concatenation. */
			*p++ = '\\';
		} else if (*p == '<' || *p == '>' || *p == '|' || *p == '&') {
	...
}

It looks like the if (*p == '\n') arm no longer has any producer, since this patch drops '\n' from all 34 verbose() format strings in kernel/bpf/disasm.c and the interpolated values (symbol names parsed with sscanf "%s") cannot contain whitespace either. Could this arm be dropped so only the '<'/'>'/'|'/'&' escaping remains?

Note this would also retire the latent memmove(p + 3, ...) expansion that had no headroom check against the 64-byte buf.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/30658911996

author maintainer
#3 lore raw

Re: [PATCH bpf-next 1/5] bpf: do not print a newline after disassembly in bpf_verbose_insn()

Quoted 13 lines · On Fri, 2026-07-31 at 20:24 +0000, [email protected] wrote:
Quoted 4 lines
tools/bpf/bpftool/xlated_dumper.c +0 -0
index 5e7cb8b36fef2..7b33d847d800c 100644

[ ... ]

Quoted 6 lines
tools/bpf/bpftool/xlated_dumper.c +1 -0
@@ -417,6 +412,7 @@ void dump_xlated_for_graph(struct dump_data *dd, void *buf_start, void *buf_end,
 
  		printf("%u: ", insn_off);
  		print_bpf_insn(&cbs, cur, true);
+		printf("\\l\\\n");
 
Quoted 6 lines

tools/bpf/bpftool/xlated_dumper.c void dump_xlated_for_graph(struct dump_data *dd, void *buf_start, void *buf_end, · line 418

Quoted 2 lines
tools/bpf/bpftool/xlated_dumper.c +0 -0
  		if (opcodes) {
  			printf("\\ \\ \\ \\ ");

This isn't a bug, but now that no disasm.c format string carries a newline and dump_xlated_for_graph() emits "\\l\\\n" itself, is the '\n' arm of print_insn_for_graph() still reachable?

It is a dead code and it should be removed.

...

author maintainer
#4 lore raw

PATCH 2/5 bpf: track upper 32-bit register halves' liveness in compute_live_registers()

In reply to Eduard Zingerman

Extend compute_live_registers() to track upper and lower register halves' liveness separately. This is mostly straightforward:
- use/def masks are extended to track 2 bits per register;
- compute_insn_live_regs() is updated to properly track these 2 bits according to the instruction semantics.

The only quirk is kfunc call processing logic, where we follow the verifier.c:check_kfunc_call() and verifier.c:mark_btf_func_reg_size() and infer whether the upper half of the parameter register is used by the call based on the parameter's BTF type size.

---
 include/linux/bpf_verifier.h |   2 +
 kernel/bpf/liveness.c        | 119 +++++++++++++++++++++++++++++++------------
 kernel/bpf/verifier.c        |   5 +-
 3 files changed, 93 insertions(+), 33 deletions(-)
include/linux/bpf_verifier.h +2 -0
index 682c2cd3b844..d4bffed94e0a 100644
@@ -1050,6 +1050,8 @@ static inline struct bpf_subprog_info *subprog_info(struct bpf_verifier_env *env
 }
 
 struct bpf_call_summary {
+	const struct btf *btf;
+	const struct btf_type *func_proto;
 	u8 num_params;
 	bool is_void;
 	bool fastcall;
kernel/bpf/liveness.c +87 -32

206 lines not on this page · raw message

kernel/bpf/verifier.c +4 -1
index 6123e7e4a320..8304452edc0f 100644
@@ -16730,10 +16730,11 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
 	int i;
 
 	if (bpf_helper_call(call)) {
-
 		if (bpf_get_helper_proto(env, call->imm, &fn) < 0)
 			/* error would be reported later */
 			return false;
+		cs->btf = NULL;
+		cs->func_proto = NULL;
 		cs->fastcall = fn->allow_fastcall &&
 			       (bpf_verifier_inlines_helper_call(env, call->imm) ||
 				bpf_jit_inlines_helper_call(call->imm));
@@ -16754,6 +16755,8 @@ bool bpf_get_call_summary(struct bpf_verifier_env *env, struct bpf_insn *call,
 		if (err < 0)
 			/* error would be reported later */
 			return false;
+		cs->btf = meta.btf;
+		cs->func_proto = meta.func_proto;
 		cs->num_params = btf_type_vlen(meta.func_proto);
 		cs->fastcall = meta.kfunc_flags & KF_FASTCALL;
 		cs->is_void = btf_type_is_void(btf_type_by_id(meta.btf, meta.func_proto->type));
Signature · 1 line
2.55.0
#5 lore raw

Re: [PATCH bpf-next 2/5] bpf: track upper 32-bit register halves' liveness in compute_live_registers()

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Critical] Critical verifier bug in conditional jump liveness tracking causing missing zero-extensions.…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] Critical verifier bug in conditional jump liveness tracking causing missing zero-extensions.
- [Low] Suboptimal liveness tracking for 32-bit conditional jumps causes unnecessary zero-extensions. --

--- Patch [2]: [PATCH bpf-next 2/5] bpf: track upper 32-bit register halves' liveness in compute_live_registers() --- commit 977deff7b46d8e36a957595add75da7e58a91c59
Author: Eduard Zingerman <[email protected]>

bpf: track upper 32-bit register halves' liveness in compute_live_registers()

This commit extends compute_live_registers() to track upper and lower register halves' liveness separately. It uses 2 bits per register and updates compute_insn_live_regs() and kfunc call processing to infer usage based on BTF type sizes.

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
index ff1e68cc4bd1c..c996b8a03e792 100644

[ ... ]

Quoted 1 line
kernel/bpf/liveness.c +0 -0
@@ -2174,17 +2228,15 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,

[ ... ]

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2237

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
 		default:
 			def = 0;
 			if (BPF_SRC(insn->code) == BPF_K)
 				use = dst;

[Severity: Low]
Does this unconditionally apply the 64-bit mask for 32-bit jumps?

Because dst is defined as the 64-bit mask, using it for BPF_JMP32 conditional jumps incorrectly assumes the upper 32 bits are alive.

Could this trigger unnecessary zero-extension instructions in the JIT?

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2239

Quoted 3 lines
kernel/bpf/liveness.c +1 -1
 			else
-				use = dst | src;
+				use = dst | (code == BPF_JMP32 ? src : src32);

[Severity: Critical]
Are the conditions and logic for tracking 64-bit jumps correct here?

The variable code is derived using BPF_OP(), which masks out the class bits. Because of this, it will never equal the instruction class BPF_JMP32, making the condition always false.

Additionally, if this were checking the class instead of code, it appears the branches of the ternary operator are reversed. As written, 64-bit jumps always evaluate to src32, marking only the lower 32 bits of the source register as live.

Could this cause the verifier to incorrectly omit the zero-extension flag for the preceding instruction that produced the source register, potentially leaving garbage data in the upper 32 bits during 64-bit comparisons?

Quoted 3 lines
kernel/bpf/liveness.c +0 -0
 		}
 		break;
 	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author maintainer
#6 lore raw

Re: [PATCH bpf-next 2/5] bpf: track upper 32-bit register halves' liveness in compute_live_registers()

Quoted 47 lines · On Fri, 2026-07-31 at 19:18 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] Critical verifier bug in conditional jump liveness tracking causing missing zero-extensions.
- [Low] Suboptimal liveness tracking for 32-bit conditional jumps causes unnecessary zero-extensions. --

--- Patch [2]: [PATCH bpf-next 2/5] bpf: track upper 32-bit register halves' liveness in compute_live_registers() --- commit 977deff7b46d8e36a957595add75da7e58a91c59
Author: Eduard Zingerman <[email protected]>

bpf: track upper 32-bit register halves' liveness in compute_live_registers()

This commit extends compute_live_registers() to track upper and lower register halves' liveness separately. It uses 2 bits per register and updates compute_insn_live_regs() and kfunc call processing to infer usage based on BTF type sizes.

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
index ff1e68cc4bd1c..c996b8a03e792 100644

[ ... ]

Quoted 1 line
kernel/bpf/liveness.c +0 -0
@@ -2174,17 +2228,15 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,

[ ... ]

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2237

Quoted 4 lines
kernel/bpf/liveness.c +0 -0
 		default:
 			def = 0;
 			if (BPF_SRC(insn->code) == BPF_K)
 				use = dst;

[Severity: Low]
Does this unconditionally apply the 64-bit mask for 32-bit jumps?

Because dst is defined as the 64-bit mask, using it for BPF_JMP32 conditional jumps incorrectly assumes the upper 32 bits are alive.

Could this trigger unnecessary zero-extension instructions in the JIT?

kernel/bpf/liveness.c static void compute_insn_live_regs(struct bpf_verifier_env *env, · line 2239

Quoted 3 lines
kernel/bpf/liveness.c +1 -1
 			else
-				use = dst | src;
+				use = dst | (code == BPF_JMP32 ? src : src32);

[Severity: Critical]
Are the conditions and logic for tracking 64-bit jumps correct here?

The variable code is derived using BPF_OP(), which masks out the class bits. Because of this, it will never equal the instruction class BPF_JMP32, making the condition always false.

Quoted 8 lines

Additionally, if this were checking the class instead of code, it appears the branches of the ternary operator are reversed. As written, 64-bit jumps always evaluate to src32, marking only the lower 32 bits of the source register as live.

Could this cause the verifier to incorrectly omit the zero-extension flag for the preceding instruction that produced the source register, potentially leaving garbage data in the upper 32 bits during 64-bit comparisons?

These two issues are the same. So much for testing, need to fix this.

...

author maintainer
#7 lore raw

PATCH 3/5 bpf: infer zext_dst based on static register liveness analysis

In reply to Eduard Zingerman

As reported in the thread [1], the verifier's 32-bit operations zero extension logic is broken. This logic is responsible for correct semantics of 32-bit operations on s390 architecture. According to BPF semantics, operation w1 += 1 is supposed to zero extend the upper half of the register r1. On s390 the JIT relies on the verifier emitting explicit zero extension before such operations.

The verifier attempts to minimize the amount of zero extensions inserted by tracking whether upper halves of the 64-bit registers are ever used. Previously such tracking worked as follows:
- bpf_reg_state->subreg_def field was set by do_check_insn() for each operation defining lower but not the upper halves of the register.
- Whenever an operation reading the whole register was verified, the verifier checked register's subreg_def and set bpf_insn_aux_data->zext_dst flag as true via a call to mark_insn_zext() function.
- After the verification was complete, a special pass bpf_opt_subreg_zext_lo32_rnd_hi32() extended 32-bit operations with bpf_insn_aux_data->zext_dst set as true by adding explicit zero extension.

Note that the logic above relies on bpf_reg_state->subreg_def, which is a property of a current verifier state.
Before the commit [2] two additional steps happened:
- The verifier tracked upper and lower register halves' liveness as flags REG_LIVE_READ{32,64} in bpf_reg_state->live.
- The function propagate_liveness() called mark_insn_zext() in order to transfer the knowledge about which registers have their upper halves alive (and thus might require zero extension).

The commit [2] removed the two steps described above, hence making possible a situation like below:
- The register's upper half is set and is used on some verification path P1 and the register happens not to be marked as precise.
- The checkpoint C is created while processing some instruction between register initialization and usage.
- On some other verification path P2 the register's upper half is not initialized and that path ends hitting the checkpoint C.
- In such a case the register's initialization on path P2 would lack zext_dst mark, making it possible for the program to inject an arbitrary value in the register's upper half.

This commit replaces subreg_def based logic with computing zext_dst statically, as a part of the bpf_compute_live_registers() analysis:
- The analysis now tracks usage of upper and lower halves of the registers separately.
- If some instruction defines a 32-bit subregister, but not the whole register, *and* the upper half of the register is alive after that instruction, the instruction is marked as zext_dst.

There is one notable drop in precision: whenever a BPF subprogram is called, all 64 bits of parameter registers are presumed to be used. The assumption is that such a drop in precision would not inflict a noticeable performance penalty.

[1] https://lore.kernel.org/bpf/CAGKGUv=sOuqQtA1Ub-5JXfA4FPosJFYKAQE4B79cK+P1erxqtg@mail.gmail.com/
[2] commit 107e16979905 ("bpf: disable and remove registers chain based liveness")

---
 include/linux/bpf_verifier.h |   6 +--
 kernel/bpf/fixups.c          |  17 ++++---
 kernel/bpf/liveness.c        |  14 ++++++
 kernel/bpf/verifier.c        | 105 +------------------------------------------
 4 files changed, 27 insertions(+), 115 deletions(-)
include/linux/bpf_verifier.h +1 -5
index d4bffed94e0a..66f506535cbd 100644
@@ -162,11 +162,6 @@ struct bpf_reg_state {
 	 * pointing to bpf_func_state.
 	 */
 	u32 frameno;
-	/* Tracks subreg definition. The stored value is the insn_idx of the
-	 * writing insn. This is safe because subreg_def is used before any insn
-	 * patching which only happens after main verification finished.
-	 */
-	s32 subreg_def;
 	/* if (!precise && SCALAR_VALUE) min/max/tnum don't affect safety */
 	bool precise;
 };
@@ -1657,5 +1652,6 @@ int bpf_convert_ctx_accesses(struct bpf_verifier_env *env);
 int bpf_jit_subprogs(struct bpf_verifier_env *env);
 int bpf_fixup_call_args(struct bpf_verifier_env *env);
 int bpf_do_misc_fixups(struct bpf_verifier_env *env);
+int bpf_insn_def32(struct bpf_insn *insn);
 
 #endif /* _LINUX_BPF_VERIFIER_H */
kernel/bpf/fixups.c +10 -7
index a0bddada7964..6125598d16a8 100644
@@ -44,15 +44,18 @@ static int insn_def_regno(const struct bpf_insn *insn)
 	}
 }
 
-/* Return TRUE if INSN has defined any 32-bit value explicitly. */
-static bool insn_has_def32(struct bpf_insn *insn)
+/*
+ * Return the 32-bit subregister defined by INSN, or -1 if INSN does not
+ * explicitly define a 32-bit value.
+ */
+int bpf_insn_def32(struct bpf_insn *insn)
 {
 	int dst_reg = insn_def_regno(insn);
 
-	if (dst_reg == -1)
-		return false;
+	if (dst_reg < 0 || bpf_is_reg64(insn, dst_reg, NULL, DST_OP))
+		return -1;
 
-	return !bpf_is_reg64(insn, dst_reg, NULL, DST_OP);
+	return dst_reg;
 }
 
 static int kfunc_desc_cmp_by_imm_off(const void *a, const void *b)
@@ -169,7 +172,7 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
 	 * (cnt == 1) is taken or not. There is no guarantee INSN at OFF is the
 	 * original insn at old prog.
 	 */
-	data[off].zext_dst = insn_has_def32(insn + off + cnt - 1);
+	data[off].zext_dst = bpf_insn_def32(insn + off + cnt - 1) >= 0;
 
 	if (cnt == 1)
 		return;
@@ -181,7 +184,7 @@ static void adjust_insn_aux_data(struct bpf_verifier_env *env,
 	for (i = off; i < off + cnt - 1; i++) {
 		/* Expand insni[off]'s seen count to the patched range. */
 		data[i].seen = old_seen;
-		data[i].zext_dst = insn_has_def32(insn + i);
+		data[i].zext_dst = bpf_insn_def32(insn + i) >= 0;
 	}
 
 	/*
kernel/bpf/liveness.c +14 -0
index c996b8a03e79..ea002550671a 100644
@@ -2258,6 +2258,7 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	struct bpf_insn *insns = env->prog->insnsi;
 	struct insn_live_regs *state;
 	int insn_cnt = env->prog->len;
+	u64 pos, insn_pos;
 	int err = 0, i, j;
 	bool changed;
 
@@ -2317,9 +2318,18 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	}
 
 	for (i = 0; i < insn_cnt; ++i) {
+		int def32 = bpf_insn_def32(&insns[i]);
+		u32 out = state[i].out;
 		u32 in = state[i].in;
 
 		insn_aux[i].live_regs_before = mask_lo(in) | mask_hi(in);
+		/*
+		 * On architectures where 32-bit operations do not reset upper halves
+		 * of the registers, the verifier needs to zero extend a destination
+		 * register if an instruction defines a 32-bit subregister and the
+		 * upper half of that register is alive after the instruction.
+		 */
+		insn_aux[i].zext_dst = def32 >= 0 && (mask_hi(out) & BIT(def32));
 	}
 
 	if (env->log.level & BPF_LOG_LEVEL2) {
@@ -2336,7 +2346,11 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 				else
 					verbose(env, ".");
 			verbose(env, " ");
+			pos = env->log.end_pos;
 			bpf_verbose_insn(env, &insns[i]);
+			insn_pos = env->log.end_pos;
+			if (insn_aux[i].zext_dst)
+				verbose(env, "%*c; zext", bpf_vlog_alignment(insn_pos - pos), ' ');
 			verbose(env, "\n");
 			if (bpf_is_ldimm64(&insns[i]))
 				i++;
kernel/bpf/verifier.c +2 -103

289 lines not on this page · raw message

Signature · 1 line
2.55.0
#8 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Critical] Incorrect static liveness tracking for 64-bit conditional jumps leads to missing…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] Incorrect static liveness tracking for 64-bit conditional jumps leads to missing zero-extensions (zext_dst), resulting in verifier/runtime state divergence and potential privilege escalation. --

--- Patch [3]: [PATCH bpf-next 3/5] bpf: infer zext_dst based on static register liveness analysis --- commit ab8592d6a434dbfed0cbb0bd067d9032b308db8d
Author: Eduard Zingerman <[email protected]>

bpf: infer zext_dst based on static register liveness analysis

This commit replaces the dynamic subreg_def based zero-extension tracking with static liveness analysis to correctly infer zext_dst and prevent missing zero-extensions on architectures like s390x.

Quoted 10 lines
kernel/bpf/liveness.c +2 -0
index c996b8a03e792..ea002550671a8 100644
@@ -2317,9 +2318,18 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	}
 
 	for (i = 0; i < insn_cnt; ++i) {
+		int def32 = bpf_insn_def32(&insns[i]);
+		u32 out = state[i].out;

kernel/bpf/liveness.c int bpf_compute_live_registers(struct bpf_verifier_env *env) · line 2332

Quoted 10 lines
kernel/bpf/liveness.c +7 -0
 		u32 in = state[i].in;
 
 		insn_aux[i].live_regs_before = mask_lo(in) | mask_hi(in);
+		/*
+		 * On architectures where 32-bit operations do not reset upper halves
+		 * of the registers, the verifier needs to zero extend a destination
+		 * register if an instruction defines a 32-bit subregister and the
+		 * upper half of that register is alive after the instruction.
+		 */
+		insn_aux[i].zext_dst = def32 >= 0 && (mask_hi(out) & BIT(def32));

[Severity: Critical]
Does this code correctly evaluate liveness for 64-bit conditional jumps?

The new zext_dst logic relies on mask_hi(out) accurately reflecting if the upper half of a register is alive.

However, looking at compute_insn_live_regs() in kernel/bpf/liveness.c for conditional jumps:

    const u8 class = BPF_CLASS(insn->code);
    const u8 code = BPF_OP(insn->code);
    ...
    case BPF_JMP:
    case BPF_JMP32:
        switch (code) {
        ...
        default:
            def = 0;
            if (BPF_SRC(insn->code) == BPF_K)
                use = dst;
            else
                use = dst | (code == BPF_JMP32 ? src : src32);
        }

Can the code variable ever equal BPF_JMP32?

BPF_OP() extracts the operation code (which is a multiple of 16), while BPF_JMP32 is a class (0x06). This means the check code == BPF_JMP32 evaluates to false, and the source register usage always falls back to src32, even for 64-bit jumps (BPF_JMP).

If a 32-bit operation defines a register, and it is followed by a 64-bit conditional jump using that register as a source with no other uses of the upper 32 bits, the upper half won't be marked as alive.

As a result, zext_dst wouldn't be set, which could lead to the JIT skipping the mandatory zero extension. This would cause the 64-bit jump to compare against uninitialized garbage in the upper bits, potentially allowing out-of-bounds access.

Should this check use class == BPF_JMP32 instead?

Quoted 1 line
kernel/bpf/liveness.c +0 -0
 	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author maintainer
#9 lore raw
Quoted 72 lines · On Fri, 2026-07-31 at 19:26 +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] Incorrect static liveness tracking for 64-bit conditional jumps leads to missing zero-extensions (zext_dst), resulting in verifier/runtime state divergence and potential privilege escalation. --

--- Patch [3]: [PATCH bpf-next 3/5] bpf: infer zext_dst based on static register liveness analysis --- commit ab8592d6a434dbfed0cbb0bd067d9032b308db8d
Author: Eduard Zingerman <[email protected]>

bpf: infer zext_dst based on static register liveness analysis

This commit replaces the dynamic subreg_def based zero-extension tracking with static liveness analysis to correctly infer zext_dst and prevent missing zero-extensions on architectures like s390x.

Quoted 10 lines
kernel/bpf/liveness.c +2 -0
index c996b8a03e792..ea002550671a8 100644
@@ -2317,9 +2318,18 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	}
 
 	for (i = 0; i < insn_cnt; ++i) {
+		int def32 = bpf_insn_def32(&insns[i]);
+		u32 out = state[i].out;

kernel/bpf/liveness.c int bpf_compute_live_registers(struct bpf_verifier_env *env) · line 2332

Quoted 10 lines
kernel/bpf/liveness.c +7 -0
 		u32 in = state[i].in;
 
 		insn_aux[i].live_regs_before = mask_lo(in) | mask_hi(in);
+		/*
+		 * On architectures where 32-bit operations do not reset upper halves
+		 * of the registers, the verifier needs to zero extend a destination
+		 * register if an instruction defines a 32-bit subregister and the
+		 * upper half of that register is alive after the instruction.
+		 */
+		insn_aux[i].zext_dst = def32 >= 0 && (mask_hi(out) & BIT(def32));

[Severity: Critical]
Does this code correctly evaluate liveness for 64-bit conditional jumps?

The new zext_dst logic relies on mask_hi(out) accurately reflecting if the upper half of a register is alive.

However, looking at compute_insn_live_regs() in kernel/bpf/liveness.c for conditional jumps:

    const u8 class = BPF_CLASS(insn->code);
    const u8 code = BPF_OP(insn->code);
    ...
    case BPF_JMP:
    case BPF_JMP32:
        switch (code) {
        ...
        default:
            def = 0;
            if (BPF_SRC(insn->code) == BPF_K)
                use = dst;
            else
                use = dst | (code == BPF_JMP32 ? src : src32);
        }

Can the code variable ever equal BPF_JMP32?

BPF_OP() extracts the operation code (which is a multiple of 16), while BPF_JMP32 is a class (0x06). This means the check code == BPF_JMP32 evaluates to false, and the source register usage always falls back to src32, even for 64-bit jumps (BPF_JMP).

If a 32-bit operation defines a register, and it is followed by a 64-bit conditional jump using that register as a source with no other uses of the upper 32 bits, the upper half won't be marked as alive.

Quoted 8 lines

As a result, zext_dst wouldn't be set, which could lead to the JIT skipping the mandatory zero extension. This would cause the 64-bit jump to compare against uninitialized garbage in the upper bits, potentially allowing out-of-bounds access.

Should this check use class == BPF_JMP32 instead?

Quoted 1 line
kernel/bpf/liveness.c +0 -0
 	}

This is the same (real) issue as reported for the previous patch.

author maintainer
#10 lore raw

PATCH 4/5 bpf: simplify the bpf_is_reg64() signature

In reply to Eduard Zingerman

After the previous commit this function is only used in a context where destination register's property is queried. Hence, simplify the function by removing the 'regno' and 't' parameters.

---
 include/linux/bpf_verifier.h |  2 +-
 kernel/bpf/fixups.c          |  4 ++--
 kernel/bpf/verifier.c        | 39 +++++++--------------------------------
 3 files changed, 10 insertions(+), 35 deletions(-)
include/linux/bpf_verifier.h +1 -1
index 66f506535cbd..321ebd308d38 100644
@@ -1628,7 +1628,7 @@ struct bpf_kfunc_desc_tab {
 };
 
 /* Functions exported from verifier.c, used by fixups.c */
-bool bpf_is_reg64(struct bpf_insn *insn, u32 regno, struct bpf_reg_state *reg, enum bpf_reg_arg_type t);
+bool bpf_is_reg64(struct bpf_insn *insn);
 void bpf_clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len);
 void bpf_mark_subprog_exc_cb(struct bpf_verifier_env *env, int subprog);
 bool bpf_allow_tail_call_in_subprogs(struct bpf_verifier_env *env);
kernel/bpf/fixups.c +2 -2
index 6125598d16a8..73ab95573f60 100644
@@ -52,7 +52,7 @@ int bpf_insn_def32(struct bpf_insn *insn)
 {
 	int dst_reg = insn_def_regno(insn);
 
-	if (dst_reg < 0 || bpf_is_reg64(insn, dst_reg, NULL, DST_OP))
+	if (dst_reg < 0 || bpf_is_reg64(insn))
 		return -1;
 
 	return dst_reg;
@@ -623,7 +623,7 @@ int bpf_opt_subreg_zext_lo32_rnd_hi32(struct bpf_verifier_env *env,
 			 *       BPF_STX + SRC_OP, so it is safe to pass NULL
 			 *       here.
 			 */
-			if (bpf_is_reg64(&insn, load_reg, NULL, DST_OP)) {
+			if (bpf_is_reg64(&insn)) {
 				if (class == BPF_LD &&
 				    BPF_MODE(code) == BPF_IMM)
 					i++;
kernel/bpf/verifier.c +7 -32
index 0709dded013e..d5b6fef1ad89 100644
@@ -3028,11 +3028,10 @@ static void mark_stack_slots_scratched(struct bpf_verifier_env *env,
 }
 
 /* This function is supposed to be used by the following 32-bit optimization
- * code only. It returns TRUE if the source or destination register operates
- * on 64-bit, otherwise return FALSE.
+ * code only. It returns TRUE if the destination register operates on 64-bit,
+ * otherwise return FALSE.
  */
-bool bpf_is_reg64(struct bpf_insn *insn,
-	      u32 regno, struct bpf_reg_state *reg, enum bpf_reg_arg_type t)
+bool bpf_is_reg64(struct bpf_insn *insn)
 {
 	u8 code, class, op;
 
@@ -3053,11 +3052,6 @@ bool bpf_is_reg64(struct bpf_insn *insn,
 			 */
 			if (insn->src_reg == BPF_PSEUDO_CALL)
 				return false;
-			/* Helper call will reach here because of arg type
-			 * check, conservatively return TRUE.
-			 */
-			if (t == SRC_OP)
-				return true;
 
 			return false;
 		}
@@ -3073,22 +3067,11 @@ bool bpf_is_reg64(struct bpf_insn *insn,
 	if (class == BPF_ALU || class == BPF_JMP32)
 		return false;
 
-	if (class == BPF_LDX) {
-		if (t != SRC_OP)
-			return BPF_SIZE(code) == BPF_DW || BPF_MODE(code) == BPF_MEMSX;
-		/* LDX source must be ptr. */
-		return true;
-	}
+	if (class == BPF_LDX)
+		return BPF_SIZE(code) == BPF_DW || BPF_MODE(code) == BPF_MEMSX;
 
-	if (class == BPF_STX) {
-		/* BPF_STX (including atomic variants) has one or more source
-		 * operands, one of which is a ptr. Check whether the caller is
-		 * asking about it.
-		 */
-		if (t == SRC_OP && reg->type != SCALAR_VALUE)
-			return true;
+	if (class == BPF_STX)
 		return BPF_SIZE(code) == BPF_DW;
-	}
 
 	if (class == BPF_LD) {
 		u8 mode = BPF_MODE(code);
@@ -3098,15 +3081,7 @@ bool bpf_is_reg64(struct bpf_insn *insn,
 			return true;
 
 		/* Both LD_IND and LD_ABS return 32-bit data. */
-		if (t != SRC_OP)
-			return  false;
-
-		/* Implicit ctx ptr. */
-		if (regno == BPF_REG_6)
-			return true;
-
-		/* Explicit source could be any width. */
-		return true;
+		return false;
 	}
 
 	if (class == BPF_ST)
Signature · 1 line
2.55.0
#11 lore raw

Re: [PATCH bpf-next 4/5] bpf: simplify the bpf_is_reg64() signature

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Low] A comment explicitly referring to removed arguments of bpf_is_reg64() was left intact. - [Low] The…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Low] A comment explicitly referring to removed arguments of bpf_is_reg64() was left intact.
- [Low] The handling of BPF_JMP, BPF_JMP32, and BPF_ST classes inside bpf_is_reg64() is now completely dead code and contains misleading comments. --

--- Patch [4]: [PATCH bpf-next 4/5] bpf: simplify the bpf_is_reg64() signature --- commit ed53634b5ab05a53b4f5cd2fc3aac98c66a8ccd4
Author: Eduard Zingerman <[email protected]>

bpf: simplify the bpf_is_reg64() signature

After the previous commit this function is only used in a context where destination register's property is queried. Hence, simplify the function by removing the 'regno' and 't' parameters.

Quoted 4 lines
kernel/bpf/fixups.c +0 -0
index 6125598d16a84..73ab95573f609 100644

[ ... ]

kernel/bpf/fixups.c int bpf_opt_subreg_zext_lo32_rnd_hi32(struct bpf_verifier_env *env, · line 626

Quoted 6 lines
kernel/bpf/fixups.c +1 -1
@@ -623,7 +623,7 @@ int bpf_opt_subreg_zext_lo32_rnd_hi32(struct bpf_verifier_env *env,
 			 *       BPF_STX + SRC_OP, so it is safe to pass NULL
 			 *       here.
 			 */
-			if (bpf_is_reg64(&insn, load_reg, NULL, DST_OP)) {
+			if (bpf_is_reg64(&insn)) {

[Severity: Low]
Since the parameters have been removed from bpf_is_reg64(), should we also remove or update this comment referring to the "fourth one"?

Quoted 9 lines
kernel/bpf/fixups.c +0 -0
 				if (class == BPF_LD &&
 				    BPF_MODE(code) == BPF_IMM)
 					i++;
kernel/bpf/verifier.c +0 -0
index 0709dded013ea..d5b6fef1ad897 100644
@@ -3028,11 +3028,10 @@ static void mark_stack_slots_scratched(struct bpf_verifier_env *env,
 }

kernel/bpf/verifier.c static void mark_stack_slots_scratched(struct bpf_verifier_env *env, · line 3034

Quoted 10 lines
kernel/bpf/verifier.c +3 -4
 
 /* This function is supposed to be used by the following 32-bit optimization
- * code only. It returns TRUE if the source or destination register operates
- * on 64-bit, otherwise return FALSE.
+ * code only. It returns TRUE if the destination register operates on 64-bit,
+ * otherwise return FALSE.
  */
-bool bpf_is_reg64(struct bpf_insn *insn,
-	      u32 regno, struct bpf_reg_state *reg, enum bpf_reg_arg_type t)
+bool bpf_is_reg64(struct bpf_insn *insn)

[Severity: Low]
Are the branches handling BPF_JMP, BPF_JMP32, and BPF_ST inside this function now dead code?

The callers in bpf_insn_def32() and bpf_opt_subreg_zext_lo32_rnd_hi32() only evaluate bpf_is_reg64() if insn_def_regno() returns a value >= 0.

Since insn_def_regno() returns -1 for BPF_JMP, BPF_JMP32, and BPF_ST, execution should never reach those branches here anymore. This leaves behind unreachable code and makes the corresponding comments (like "BPF_EXIT for 'main' will reach here") inaccurate.

Quoted 3 lines
kernel/bpf/verifier.c +0 -0
 {
 	u8 code, class, op;
 
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#12 lore raw

Re: [PATCH bpf-next 4/5] bpf: simplify the bpf_is_reg64() signature

In reply to Eduard Zingerman
The changelog appears to undercount the removed parameters. Looking at the actual signature change:
Quoted 6 lines

bpf: simplify the bpf_is_reg64() signature

After the previous commit this function is only used in a context where destination register's property is queried. Hence, simplify the function by removing the 'regno' and 't' parameters.

The changelog appears to undercount the removed parameters. Looking at the actual signature change:

include/linux/bpf_verifier.h:
-bool bpf_is_reg64(struct bpf_insn *insn, u32 regno, struct bpf_reg_state *reg, enum bpf_reg_arg_type t); +bool bpf_is_reg64(struct bpf_insn *insn);

Three parameters are dropped: regno, reg, and t. The message mentions only 'regno' and 't', omitting 'struct bpf_reg_state *reg'.

The reg parameter isn't incidental: it was the only pointer argument, both call sites passed NULL for it, and its removal retires the contract that the BPF_STX SRC_OP arm relied on. The fixups.c comment at line 622 explicitly documented this:

kernel/bpf/fixups.c:

    /* NOTE: arg "reg" (the fourth one) is only used for
     *       BPF_STX + SRC_OP, so it is safe to pass NULL
     *       here.
     */
    if (bpf_is_reg64(&insn)) {

Could the message say 'removing the regno, reg and t parameters' (or 'removing all parameters except insn') so readers following the series can account for all three removals?

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/30658911996

author maintainer
#13 lore raw

PATCH 5/5 selftests/bpf: verify zext_dst annotations for various instructions

In reply to Eduard Zingerman

Includes the following test cases:
- a test showing that zero extension flags do not propagate through state pruning in the unpatched kernel.
- a 32-bit ALU result read as 64-bit (zext) and read only as 32-bit (no zext);
- a 64-bit MOV (never zext'ed);
- a narrow (32-bit) BPF_LDX load whose result is read as 64-bit;
- 32-bit atomic fetch_add and cmpxchg whose result is read as 64-bit;
- a CFG case where a 32-bit definition's upper half is used only on one of two branches;
- no zext for dead registers;
- LD_ABS defines only lower 32 bits, hence needs zext when the result is used as 64-bits;
- Helper and subprogram parameters are considered to use full 64 bits;
- kfunc parameters are read according to their BTF type width.

---
 tools/testing/selftests/bpf/prog_tests/verifier.c |   2 +
 tools/testing/selftests/bpf/progs/verifier_zext.c | 260 ++++++++++++++++++++++
 2 files changed, 262 insertions(+)
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index be97f6887f0e..fba562c81969 100644
@@ -124,6 +124,7 @@
 #include "verifier_jit_inline.skel.h"
 #include "irq.skel.h"
 #include "verifier_ctx_ptr_param.skel.h"
+#include "verifier_zext.skel.h"
 
 #define MAX_ENTRIES 11
 
@@ -277,6 +278,7 @@ void test_irq(void)			      { RUN(irq); }
 void test_verifier_mtu(void)		      { RUN(verifier_mtu); }
 void test_verifier_jit_inline(void)               { RUN(verifier_jit_inline); }
 void test_verifier_ctx_ptr_param(void)       { RUN(verifier_ctx_ptr_param); }
+void test_verifier_zext(void)                 { RUN_TESTS(verifier_zext); }
 
 static int init_test_val_map(struct bpf_object *obj, char *map_name)
 {
tools/testing/selftests/bpf/progs/verifier_zext.c new file +260 -0

260 lines not on this page · raw message

Signature · 1 line
2.55.0