bpf: Disallow interpreter fallback for interpreter-unsupported insns
Subsystems (6)
cover bpf: Disallow interpreter fallback for interpreter-unsupported insns
Sashiko reported two potential issues about interpreter fallback [1] [2].
After verifying them by patch #7, I think they are real issues. With LLM assistance, the interpreter does not support the internal BPF_PROBE_ATOMIC insn and the gotox insn (used for indirect jumps), either.
1) the user BPF_ADDR_SPACE_CAST insn
the interpreter just ignores it.
2) the arena ST/STX/LDX insn
the interpreter could hit the BUG_ON() in ___bpf_prog_run().
3) the BPF_MOV64_PERCPU_REG insn
the interpreter could hit page fault, due to loading memory from invalid __percpu pointer.
4) the internal BPF_PROBE_ATOMIC insn
the interpreter could hit the BUG_ON() in ___bpf_prog_run().
5) the gotox insn used for indirect jumps
the interpreter could hit the BUG_ON() in ___bpf_prog_run(), too.
Reject these insns on interpreter fallback path in __bpf_prog_select_runtime().
This series is built on
"bpf: Fix unaligned interpreter panic on JIT fallback path" [3]. The patch #7 is also able to verify the issue of un-JITed helper.
However, The patch #7 aims to verify the issues. I think it is not proper to be applied to upstream, because it adds a stub 'bpf_jit_test_fail_task' to bpf_prog_jit_compile() for the tests.
I'd like to drop the patch #7 in the next revision.
Link:
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
Leon Hwang (6):
bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
bpf: Disallow interpreter fallback for arena insn
bpf: Disallow interpreter fallback for BPF_MOV64_PERCPU_REG insn
bpf: Disallow interpreter fallback for internal BPF_PROBE_ATOMIC insn
bpf: Disallow interpreter fallback for gotox insn lib/test_bpf: Add interpreter-fallback tests
include/linux/bpf.h | 1 + include/linux/filter.h | 4 + kernel/bpf/core.c | 69 +- lib/test_bpf.c | 800 ++++++++++++++++++++++- tools/lib/bpf/skel_internal.h | 2 + tools/testing/selftests/bpf/test_kmod.sh | 39 +- 6 files changed, 903 insertions(+), 12 deletions(-)
--
2.54.0
RFC PATCH 1/6 bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
The interpreter is unable to handle the user BPF_ADDR_SPACE_CAST insn, whose '->off' is 1:
static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
{
ALU64_MOV_X:
switch (OFF) {
case 0:
DST = SRC;
break;
case 8:
DST = (s8) SRC;
break;
case 16:
DST = (s16) SRC;
break;
case 32:
DST = (s32) SRC;
break;
}
CONT;
}On the fallback path from JIT in __bpf_prog_select_runtime(), reject the insn to avoid being ignored by interpreter.
--- kernel/bpf/core.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)
kernel/bpf/core.c +23 -6
@@ -2608,23 +2608,37 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, strucreturn prog; } +static bool bpf_insn_requires_jit (struct bpf_insn *insn) +{ +if (insn_is_cast_user (insn)) +return true; + +return false; +} +/* Fix up helper call offsets on JIT fallback path. */ -static void bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp) +static int bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp) {struct bpf_insn *insn = fp->insnsi;const struct bpf_func_proto *fn;int i; -if (!env || !env->ops->get_func_proto) -return ; +if (!env) +return 0 ;for (i =0 ; i < fp->len; i++, insn++) { -if (bpf_helper_call (insn) &&bpf_jit_inlines_helper_call (insn->imm)) { +if (env->ops->get_func_proto &&bpf_helper_call (insn) && +bpf_jit_inlines_helper_call (insn->imm)) { fn = env->ops->get_func_proto (insn->imm, env->prog);if (fn && fn->func) insn->imm = fn->func - __bpf_call_base; } + +if (bpf_insn_requires_jit (insn)) +return -EOPNOTSUPP ; } + +return 0 ; } struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
@@ -2663,8 +2677,11 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, structreturn fp; } -if (!fp->jited) -bpf_fixup_fallback_helpers (env, fp); +if (!fp->jited) { + *err =bpf_fixup_fallback_helpers (env, fp); +if (*err) +return fp; + } }else { *err =bpf_prog_offload_compile (fp);if (*err)
Signature · 1 line
2.54.0
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 66 lines · Leon Hwang
The interpreter is unable to handle the user
BPF_ADDR_SPACE_CASTinsn, whose '->off' is 1:static u64
___bpf_prog_run(u64 *regs, const struct bpf_insn *insn){ ALU64_MOV_X: switch (OFF) { case 0: DST = SRC; break; case 8: DST = (s8) SRC; break; case 16: DST = (s16) SRC; break; case 32: DST = (s32) SRC; break; } CONT; }On the fallback path from JIT in
__bpf_prog_select_runtime(), reject the insn to avoid being ignored by interpreter.Fixes 142fd4d2dcf5 ("bpf: Add x86-64 JIT support for bpf_addr_space_cast instruction.")Signed-off-by Leon Hwang <[email protected]>---
kernel/bpf/core.c | 29 +++++++++++++++++++++++------1 file changed, 23 insertions(+), 6 deletions(-)kernel/bpf/core.c +0
-0@@ -2608,23 +2608,37 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env,struc
return prog;
}kernel/bpf/core.c +8
-0+static bool bpf_insn_requires_jit (struct bpf_insn *insn) +{ +if (insn_is_cast_user (insn)) +return true; + +return false; +} +/* Fix up helper call offsets on JIT fallback path. */
kernel/bpf/core.c +1
-1-static void bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp) +static int bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp){
structbpf_insn*insn =fp->insnsi;
const structbpf_func_proto*fn;
int i;kernel/bpf/core.c +2
-2-if (!env || !env->ops->get_func_proto) -return ; +if (!env) +return 0 ;
for (i = 0; i <fp->len; i++, insn++) {kernel/bpf/core.c +2
-1-if (bpf_helper_call (insn) &&bpf_jit_inlines_helper_call (insn->imm)) { +if (env->ops->get_func_proto &&bpf_helper_call (insn) && + bpf_jit_inlines_helper_call(insn->imm)) {
Quoted 3 lines
fn =
env->ops->get_func_proto(insn->imm, env->prog); if (fn &&fn->func)
insn->imm=fn->func-__bpf_call_base;
It might be better to use the BPF_CALL_IMM macro. insn->imm = BPF_CALL_IMM(fn->func);
Quoted 25 lines
}
kernel/bpf/core.c +3
-0+ +if (bpf_insn_requires_jit (insn)) +return -EOPNOTSUPP ;}
kernel/bpf/core.c +2
-0+ +return 0 ;}
structbpf_prog*__bpf_prog_select_runtime(structbpf_verifier_env*env, structbpf_prog*fp,kernel/bpf/core.c +0
-0@@ -2663,8 +2677,11 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env,struct
return fp;
}kernel/bpf/core.c +5
-2-if (!fp->jited) -bpf_fixup_fallback_helpers (env, fp); +if (!fp->jited) { + *err =bpf_fixup_fallback_helpers (env, fp); +if (*err) +return fp; + }} else {
*err =bpf_prog_offload_compile(fp); if (*err)
Signature · 2 lines
Thanks, KaFai
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 1 line · KaFai Wan
On Fri, 2026-06-26 at 23:43 +0800, Leon Hwang wrote:
[...]
Quoted 13 lines
Quoted 13 lines
/* Fix up helper call offsets on JIT fallback path. */
kernel/bpf/core.c +1
-1-static void bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp) +static int bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp){
structbpf_insn*insn =fp->insnsi;
const structbpf_func_proto*fn;
int i;kernel/bpf/core.c +2
-2-if (!env || !env->ops->get_func_proto) -return ; +if (!env) +return 0 ;
Quoted 8 lines
Quoted 7 lines
for (i = 0; i <
fp->len; i++, insn++) {kernel/bpf/core.c +2
-1-if (bpf_helper_call (insn) &&bpf_jit_inlines_helper_call (insn->imm)) { +if (env->ops->get_func_proto &&bpf_helper_call (insn) && + bpf_jit_inlines_helper_call(insn->imm)) {fn =
env->ops->get_func_proto(insn->imm, env->prog); if (fn &&fn->func)
insn->imm=fn->func-__bpf_call_base;It might be better to use the
BPF_CALL_IMMmacro.insn->imm=BPF_CALL_IMM(fn->func);
Makes sense.
This might be applied to Tiezhu's patch.
Thanks,
Leon
Quoted 6 lines
Quoted 6 lines
}
kernel/bpf/core.c +3
-0+ +if (bpf_insn_requires_jit (insn)) +return -EOPNOTSUPP ;}
[...]
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 26 lines · Leon Hwang
Quoted 1 line · KaFai Wan
On Fri, 2026-06-26 at 23:43 +0800, Leon Hwang wrote:
[...]
Quoted 13 lines
Quoted 13 lines
/* Fix up helper call offsets on JIT fallback path. */
kernel/bpf/core.c +3
-3-static void bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp) +static int bpf_fixup_fallback_helpers (struct bpf_verifier_env *env,struct bpf_prog *fp) { struct bpf_insn *insn = fp->insnsi; const struct bpf_func_proto *fn; int i; -if (!env || !env->ops->get_func_proto) -return ; +if (!env) +return 0 ;Quoted 8 lines
kernel/bpf/core.c static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struc · line 2634
Quoted 7 lines
for (i = 0; i <
fp->len; i++, insn++) {kernel/bpf/core.c +2
-1-if (bpf_helper_call (insn) &&bpf_jit_inlines_helper_call (insn->imm)) { +if (env->ops->get_func_proto &&bpf_helper_call (insn) && + bpf_jit_inlines_helper_call(insn->imm)) { fn = env->ops->get_func_proto(insn->imm, env->prog); if (fn && fn->func) insn->imm = fn->func - __bpf_call_base;It might be better to use the
BPF_CALL_IMMmacro.insn->imm=BPF_CALL_IMM(fn->func);
Quoted 3 lines
Makes sense.
This might be applied to Tiezhu's patch.
Hi all,
Alexei said in another thread like this:
``` > [3] https://lore.kernel.org/bpf/[email protected]/ I don't think we need such fallback in patch [3]. ```
https://lore.kernel.org/bpf/[email protected]/
Should I just drop my previous patch? To be honest, I'm not sure how to solve the issue mentioned in it. Is that a real problem, and does it need to be fixed? What is the next step?
Thanks,
Tiezhu
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
On 1/7/26 09:26, Tiezhu Yang wrote:
[...]
Quoted 11 lines
Hi all,
Alexei said in another thread like this:
``` > [3] https://lore.kernel.org/bpf/20260615025316.24429-1- [email protected]/ I don't think we need such fallback in patch [3]. ```
Quoted 6 lines
https://lore.kernel.org/bpf/[email protected]/
Should I just drop my previous patch? To be honest, I'm not sure how to solve the issue mentioned in it. Is that a real problem, and does it need to be fixed? What is the next step?
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Thanks,
Leon
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 20 lines · Leon Hwang
On 1/7/26 09:26, Tiezhu Yang wrote:
[...]Quoted 11 lines
Hi all,
Alexei said in another thread like this:
``` > [3] https://lore.kernel.org/bpf/20260615025316.24429-1- [email protected]/ I don't think we need such fallback in patch [3]. ```Quoted 6 lines
https://lore.kernel.org/bpf/[email protected]/
Should I just drop my previous patch? To be honest, I'm not sure how to solve the issue mentioned in it. Is that a real problem, and does it need to be fixed? What is the next step?
Quoted 4 lines
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Hi Alexei and Leon,
What about the following: (not tested yet, if it is good, I can test it)
----->8-----
kernel/bpf/core.c +13 -0
@@ -2608,6 +2608,19 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, strucreturn prog; } +static bool bpf_prog_has_inline_helpers (const struct bpf_prog *fp) +{ +struct bpf_insn *insn = fp->insnsi; +int i; + +for (i =0 ; i < fp->len; i++, insn++) { +if (bpf_helper_call (insn) &&bpf_jit_inlines_helper_call (insn->imm)) +return true; + } + +return false; +} + struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env
*env, struct bpf_prog *fp,
int *err)
{
@@ -2620,7 +2633,8 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
goto finalize;
if (IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON) ||
- bpf_prog_has_kfunc_call(fp))
+ bpf_prog_has_kfunc_call(fp) ||
+ bpf_prog_has_inline_helpers(fp))
jit_needed = true;
if (!bpf_prog_select_interpreter(fp))----->8-----
If you are OK with the above changes, I will test it again and send a new version later.
Thanks,
Tiezhu
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
On 1/7/26 14:49, Tiezhu Yang wrote:
[...]
Quoted 6 lines
Quoted 5 lines
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Quoted 9 lines
Hi Alexei and Leon,
What about the following: (not tested yet, if it is good, I can test it)
[...]
If you are OK with the above changes, I will test it again and send a new version later.
If you don't mind, I think I can disallow the interpreter fallback by the below diff in the next revision of this series.
The 'aux->jit_required' is introduced by following Alexei's suggestion.
Thanks,
Leon
---
kernel/bpf/fixups.c +3 -1
@@ -1841,8 +1841,10 @@ int bpf_do_misc_fixups(struct bpf_verifier_env *env) }/* Skip inlining the helper call if the JIT does it. */ -if (bpf_jit_inlines_helper_call (insn->imm)) +if (bpf_jit_inlines_helper_call (insn->imm)) { + prog->aux->jit_required = true; goto next_insn; + }if (insn->imm == BPF_FUNC_get_route_realm) prog->dst_needed =1 ;
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 14 lines · Leon Hwang
On 1/7/26 14:49, Tiezhu Yang wrote:
[...]Quoted 12 lines
Quoted 5 lines
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Hi Alexei and Leon,
What about the following: (not tested yet, if it is good, I can test it)
[...]
Quoted 8 lines
Quoted 3 lines
If you are OK with the above changes, I will test it again and send a new version later.
If you don't mind, I think I can disallow the interpreter fallback by the below diff in the next revision of this series.
The '
aux->jit_required' is introduced by following Alexei's suggestion.
I am not sure if introducing a new aux->jit_required member is necessary, as we already have a local jit_needed flag in the current logic.
Additionally, the issue reported in my patch [1] is fundamentally tied to bpf_jit_inlines_helper_call(). Maybe we can simply reuse jit_needed by checking bpf_prog_has_inline_helpers() right before JITing, and then reject fallback with -ENOTSUPP if JIT compilation fails.
Let us wait for Alexei's comments on the overall direction before respinning.
[1] bpf: Fix unaligned interpreter panic on JIT fallback path
https://lore.kernel.org/bpf/[email protected]/
Thanks,
Tiezhu
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 19 lines · Tiezhu Yang
Quoted 18 lines · Leon Hwang
On 1/7/26 14:49, Tiezhu Yang wrote:
[...]Quoted 15 lines
Quoted 5 lines
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Hi Alexei and Leon,
What about the following: (not tested yet, if it is good, I can test it)
[...]
If you are OK with the above changes, I will test it again and send a new version later.
Quoted 7 lines
Quoted 4 lines
If you don't mind, I think I can disallow the interpreter fallback by the below diff in the next revision of this series.
The '
aux->jit_required' is introduced by following Alexei's suggestion.I am not sure if introducing a new
aux->jit_requiredmember is necessary, as we already have a localjit_neededflag in the current logic.
A 'must_jit'-like member was suggested by Alexei [1].
Probably, the commits in [2] could help to understand 'aux->jit_required'.
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://github.com/Asphaltt/bpf/commits/bpf/fix-interpreter-fallback/v2/
Quoted 5 lines
Additionally, the issue reported in my patch [1] is fundamentally tied to
bpf_jit_inlines_helper_call(). Maybe we can simply reusejit_neededby checkingbpf_prog_has_inline_helpers()right before JITing, and then reject fallback with-ENOTSUPPif JIT compilation fails.
Correct. Reusing jit_needed is preferred.
Thanks,
Leon
Quoted 9 lines
Let us wait for Alexei's comments on the overall direction before respinning.
[1] bpf: Fix unaligned interpreter panic on JIT fallback path
https://lore.kernel.org/bpf/[email protected]/Thanks,
Tiezhu
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 36 lines · Leon Huang Fu
Quoted 19 lines · Tiezhu Yang
Quoted 18 lines · Leon Hwang
On 1/7/26 14:49, Tiezhu Yang wrote:
[...]Quoted 15 lines
Quoted 5 lines
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Hi Alexei and Leon,
What about the following: (not tested yet, if it is good, I can test it)
[...]
If you are OK with the above changes, I will test it again and send a new version later.
Quoted 7 lines
Quoted 4 lines
If you don't mind, I think I can disallow the interpreter fallback by the below diff in the next revision of this series.
The '
aux->jit_required' is introduced by following Alexei's suggestion.I am not sure if introducing a new
aux->jit_requiredmember is necessary, as we already have a localjit_neededflag in the current logic.A 'must_jit'-like member was suggested by Alexei [1].
Probably, the commits in [2] could help to understand '
aux->jit_required'.[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://github.com/Asphaltt/bpf/commits/bpf/fix-interpreter-fallback/v2/Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Additionally, the issue reported in my patch [1] is fundamentally tied to
bpf_jit_inlines_helper_call(). Maybe we can simply reusejit_neededby checkingbpf_prog_has_inline_helpers()right before JITing, and then reject fallback with-ENOTSUPPif JIT compilation fails.Correct. Reusing
jit_neededis preferred.
I agree.
To avoid bloating struct bpf_prog_aux and to keep the refactoring minimal, I suggest that you submit a patch to introduce a centralized bpf_prog_requires_jit() helper in kernel/bpf/core.c.
This will cleanly bundle bpf_prog_has_kfunc_call() and other JIT-only instructions (like user cast, arena, etc.) together, like this:
kernel/bpf/core.c +18 -0
@@ -2608,6 +2608,24 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, strucreturn prog; } +static bool bpf_prog_requires_jit (const struct bpf_prog *fp) +{ +struct bpf_insn *insn = fp->insnsi; +int i; + +if (bpf_prog_has_kfunc_call (fp)) +return true; + +for (i =0 ; i < fp->len; i++, insn++) { +if (insn_is_cast_user (insn) || +insn_is_arena_load_store (insn) || +insn_is_indirect_jump (insn)) +return true; + } + +return false; +} + struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env
*env, struct bpf_prog *fp,
int *err)
{
@@ -2620,7 +2638,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
goto finalize;
if (IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON) ||
- bpf_prog_has_kfunc_call(fp))
+ bpf_prog_requires_jit(fp))
jit_needed = true;
if (!bpf_prog_select_interpreter(fp))Once that infrastructure is in place, I can simply spin my next revision by adding the bpf_jit_inlines_helper_call() check into this centralized function, like so:
kernel/bpf/core.c +3 -0
@@ -2617,6 +2617,9 @@ static bool bpf_prog_requires_jit(const struct bpf_prog *fp)return true;for (i =0 ; i < fp->len; i++, insn++) { +if (bpf_helper_call (insn) &&bpf_jit_inlines_helper_call (insn->imm)) +return true; +if (insn_is_cast_user (insn) ||insn_is_arena_load_store (insn) ||insn_is_indirect_jump (insn))
This keeps the core runtime selection logic completely clean.
Thanks,
Tiezhu
Re: [RFC PATCH bpf 1/6] bpf: Disallow interpreter fallback for user BPF_ADDR_SPACE_CAST insn
Quoted 69 lines · Tiezhu Yang
Quoted 36 lines · Leon Huang Fu
Quoted 19 lines · Tiezhu Yang
Quoted 18 lines · Leon Hwang
On 1/7/26 14:49, Tiezhu Yang wrote:
[...]Quoted 15 lines
Quoted 5 lines
It is a real issue.
Instead of fixing up helper calls, it seems better to prevent interpreter fallback if the prog has any JIT-inlineable helper call.
Hi Alexei and Leon,
What about the following: (not tested yet, if it is good, I can test it)
[...]
If you are OK with the above changes, I will test it again and send a new version later.
Quoted 7 lines
Quoted 4 lines
If you don't mind, I think I can disallow the interpreter fallback by the below diff in the next revision of this series.
The '
aux->jit_required' is introduced by following Alexei's suggestion.I am not sure if introducing a new
aux->jit_requiredmember is necessary, as we already have a localjit_neededflag in the current logic.A 'must_jit'-like member was suggested by Alexei [1].
Probably, the commits in [2] could help to understand '
aux->jit_required'.[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://github.com/Asphaltt/bpf/commits/bpf/fix-interpreter-fallback/v2/Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Additionally, the issue reported in my patch [1] is fundamentally tied to
bpf_jit_inlines_helper_call(). Maybe we can simply reusejit_neededby checkingbpf_prog_has_inline_helpers()right before JITing, and then reject fallback with-ENOTSUPPif JIT compilation fails.Correct. Reusing
jit_neededis preferred.I agree.
To avoid bloating struct
bpf_prog_auxand to keep the refactoring minimal, I suggest that you submit a patch to introduce a centralizedbpf_prog_requires_jit()helper inkernel/bpf/core.c.This will cleanly bundle
bpf_prog_has_kfunc_call()and other JIT-only instructions (like user cast, arena, etc.) together, like this:kernel/bpf/core.c +7
-0@@ -2608,6 +2608,24 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struc return prog; } +static bool bpf_prog_requires_jit(const struct bpf_prog *fp) +{ + struct bpf_insn *insn = fp->insnsi; + int i; + + if (bpf_prog_has_kfunc_call(fp)) + return true;
kernel/bpf/core.c static struct bpf_prog · line 2628
Quoted 10 lines
kernel/bpf/core.c +10
-0+ +for (i =0 ; i < fp->len; i++, insn++) { +if (insn_is_cast_user (insn) || +insn_is_arena_load_store (insn) || +insn_is_indirect_jump (insn)) +return true; + } + +return false; +}
No. I already explained that this forking of the logic is not ok.
kernel/bpf/core.c
Quoted 12 lines
kernel/bpf/core.c +1
-0+ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,int *err)
{kernel/bpf/core.c +0
-0@@ -2620,7 +2638,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct
bpf_verifier_env*env, structgoto finalize; if (IS_ENABLED(CONFIG_BPF_JIT_ALWAYS_ON) ||kernel/bpf/core.c +1
-1- bpf_prog_has_kfunc_call(fp)) + bpf_prog_requires_jit(fp))
This has to be fp->jit_required and
the verifier has to set it when necessary.
RFC PATCH 2/6 bpf: Disallow interpreter fallback for arena insn
In reply to Leon HwangThe interpreter does not recognize the arena ST/LDX/STX insns. Thereafter, it would hit the BUG_ON() in ___bpf_prog_run() at run time.
[ 2.579196] BPF interpreter: unknown opcode a2 (imm: 0x0) [ 2.579998] ------------[ cut here ]------------ [ 2.580652] kernel BUG at kernel/bpf/core.c:2349! [ 2.581314] Oops: invalid opcode: 0000 [#1] SMP PTI
On the fallback path from JIT in __bpf_prog_select_runtime(), reject the BPF_PROBE_MEM32/BPF_PROBE_MEM32SX insn to avoid the BUG.
--- kernel/bpf/core.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
kernel/bpf/core.c +16 -0
@@ -2613,6 +2613,22 @@ static bool bpf_insn_requires_jit(struct bpf_insn *insn)if (insn_is_cast_user (insn))return true; +switch (BPF_CLASS (insn->code)) { +case BPF_ST : +case BPF_LDX : +case BPF_STX : +/* arena ST/LDX/STX insns */ +if (BPF_MODE (insn->code) ==BPF_PROBE_MEM32 ) +return true; +if (BPF_MODE (insn->code) ==BPF_PROBE_MEM32SX && +BPF_CLASS (insn->code) ==BPF_LDX && +BPF_SIZE (insn->code) !=BPF_DW ) +return true; +break ; +default : +return false; + } +return false; }
Signature · 1 line
2.54.0
RFC PATCH 3/6 bpf: Disallow interpreter fallback for BPF_MOV64_PERCPU_REG insn
In reply to Leon HwangThe BPF_MOV64_PERCPU_REG insn requires JIT to emit native code to for 'dst_reg = src_reg + <percpu_base_off>'.
However, the interpreter ignores the 'off' at its ALU64_MOV_K label. The 'off' indicates the insn is BPF_MOV64_PERCPU_REG insn. Then, when the interpreter loads memory from the register, it will hit a page fault.
[ 2.545572] BUG: unable to handle page fault for address: ffffffffacaaf034 [ 2.546485] #PF: supervisor read access in kernel mode [ 2.547167] #PF: error_code(0x0000) - not-present page [ 2.547850] PGD 134e63067 P4D 134e63067 PUD 134e64063 PMD 10021c063 PTE 800ffffeca550062 [ 2.548912] Oops: Oops: 0000 [#1] SMP PTI
On the fallback path from JIT in __bpf_prog_select_runtime(), reject the BPF_MOV64_PERCPU_REG insn to avoid the page fault.
--- kernel/bpf/core.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/bpf/core.c +3 -0
@@ -2610,6 +2610,9 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, strucstatic bool bpf_insn_requires_jit (struct bpf_insn *insn) { +if (insn_is_mov_percpu_addr (insn)) +return true; +if (insn_is_cast_user (insn))return true;
Signature · 1 line
2.54.0
RFC PATCH 4/6 bpf: Disallow interpreter fallback for internal BPF_PROBE_ATOMIC insn
In reply to Leon HwangThe interpreter does not recognize the BPF_PROBE_ATOMIC insn. Thereafter, it would hit the BUG_ON() in ___bpf_prog_run() at run time.
[ 2.617849] BPF interpreter: unknown opcode e3 (imm: 0x110) [ 2.618643] ------------[ cut here ]------------ [ 2.619277] kernel BUG at kernel/bpf/core.c:2349! [ 2.619928] Oops: invalid opcode: 0000 [#1] SMP PTI
On the fallback path from JIT in __bpf_prog_select_runtime(), reject the BPF_PROBE_ATOMIC insn to avoid the BUG.
--- kernel/bpf/core.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/bpf/core.c +3 -0
@@ -2627,6 +2627,9 @@ static bool bpf_insn_requires_jit(struct bpf_insn *insn) BPF_CLASS(insn->code) == BPF_LDX && BPF_SIZE(insn->code) != BPF_DW)return true; +/* BPF_LOAD_ACQ, BPF_STORE_REL insns */ +if (BPF_MODE (insn->code) ==BPF_PROBE_ATOMIC ) +return true;break ;default :return false;
Signature · 1 line
2.54.0
RFC PATCH 5/6 bpf: Disallow interpreter fallback for gotox insn
In reply to Leon HwangThe interpreter does not recognize the BPF_JMP|BPF_JA|BPF_X insn, which is used for insn_array map. Thereafter, it would hit the BUG_ON() in ___bpf_prog_run() at run time.
[ 2.563726] BPF interpreter: unknown opcode 0d (imm: 0x0) [ 2.564557] ------------[ cut here ]------------ [ 2.565206] kernel BUG at kernel/bpf/core.c:2349! [ 2.565882] Oops: invalid opcode: 0000 [#1] SMP PTI
On the fallback path from JIT in __bpf_prog_select_runtime(), reject the BPF_JMP|BPF_JA|BPF_X insn to avoid the BUG.
--- kernel/bpf/core.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/bpf/core.c +3 -0
@@ -2613,6 +2613,9 @@ static bool bpf_insn_requires_jit(struct bpf_insn *insn)if (insn_is_mov_percpu_addr (insn))return true; +if (insn_is_gotox (insn)) +return true; +if (insn_is_cast_user (insn))return true;
Signature · 1 line
2.54.0
RFC PATCH 6/6 lib/test_bpf: Add interpreter-fallback tests
In reply to Leon HwangAdd following tests to verify interpreter-fallback issues:
* arena-related insns, such as user ADDR_SPACE_CAST insn, ST/STX/LDX insns, are not recognized by the interpreter.
* the internal ADDR_PERCPU insn is not recognized by the interpreter.
* the internal PROBE_ATOMIC insn is not recognized by the interpreter, too.
* the gotox insn used for indirect jumps is not recognized by the interpreter.
With the fixes, all of them are rejected by returning -EOPNOTSUPP.
--- include/linux/bpf.h | 1 + include/linux/filter.h | 4 + kernel/bpf/core.c | 15 + lib/test_bpf.c | 800 ++++++++++++++++++++++- tools/lib/bpf/skel_internal.h | 2 + tools/testing/selftests/bpf/test_kmod.sh | 39 +- 6 files changed, 855 insertions(+), 6 deletions(-)
include/linux/bpf.h +1 -0
@@ -2699,6 +2699,7 @@ extern const struct bpf_prog_ops bpf_offload_prog_ops;extern const struct bpf_verifier_ops tc_cls_act_analyzer_ops;extern const struct bpf_verifier_ops xdp_analyzer_ops; +int kern_sys_bpf (int cmd,union bpf_attr *attr,unsigned int size);struct bpf_prog *bpf_prog_get (u32 ufd);struct bpf_prog *bpf_prog_get_type_dev (u32 ufd,enum bpf_prog_type type,bool attach_drv);
include/linux/filter.h +4 -0
@@ -1312,6 +1312,10 @@ extern int bpf_jit_kallsyms;extern long bpf_jit_limit;extern long bpf_jit_limit_max; +#if IS_ENABLED (CONFIG_TEST_BPF ) +void bpf_jit_set_test_force_fail (bool force); +#endif +typedef void (*bpf_jit_fill_hole_t )(void *area,unsigned int size);void bpf_jit_fill_hole_with_zero (void *area,unsigned int size);
kernel/bpf/core.c +15 -0
@@ -550,6 +550,16 @@ int bpf_jit_harden __read_mostly;long bpf_jit_limit__read_mostly ;long bpf_jit_limit_max__read_mostly ; +#if IS_ENABLED (CONFIG_TEST_BPF ) +static struct task_struct *bpf_jit_test_fail_task; + +void bpf_jit_set_test_force_fail (bool force) +{ +WRITE_ONCE (bpf_jit_test_fail_task, force ? current :NULL ); +} +EXPORT_SYMBOL_GPL (bpf_jit_set_test_force_fail); +#endif +static void bpf_prog_ksym_set_addr (struct bpf_prog *prog) {
@@ -2567,6 +2577,11 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, strucstruct bpf_prog *orig_prog;struct bpf_insn_aux_data *orig_insn_aux; +#if IS_ENABLED (CONFIG_TEST_BPF ) +if (unlikely (READ_ONCE (bpf_jit_test_fail_task) == current)) +return prog; +#endif +if (!bpf_prog_need_blind (prog))return bpf_int_jit_compile (env, prog);
lib/test_bpf.c +797 -3
841 lines not on this page · raw message
tools/lib/bpf/skel_internal.h +2 -0
@@ -76,7 +76,9 @@ struct bpf_load_and_run_opts {__u32 excl_prog_hash_sz; }; +#ifndef __KERNEL__long kern_sys_bpf (__u32 cmd,void *attr,__u32 attr_size); +#endif static inline int skel_sys_bpf (enum bpf_cmd cmd,union bpf_attr *attr,unsigned int size)
tools/testing/selftests/bpf/test_kmod.sh +36 -3
@@ -47,14 +47,14 @@ test_run() rc=1 fi fi - rmmod test_bpf 2> /dev/null + rmmod test_bpf 2> /dev/null dmesg | grep FAIL } test_save() { - JE=`sysctl -n net.core.bpf_jit_enable` - JH=`sysctl -n net.core.bpf_jit_harden` + JE=$(sysctl -n net.core.bpf_jit_enable) + JH=$(sysctl -n net.core.bpf_jit_harden) } test_restore()
@@ -63,11 +63,44 @@ test_restore() sysctl -w net.core.bpf_jit_harden=$JH 2>&1 > /dev/null } +test_interpreter_fallback() +{ + if ! sysctl -w net.core.bpf_jit_enable=1 >/dev/null 2>&1 || + ! sysctl -w net.core.bpf_jit_harden=0 >/dev/null 2>&1; then + echo "[ interpreter fallback: SKIP ]" + return + fi + + echo "[ interpreter fallback ]" + dmesg -C + if [ -f ${OUTPUT}/lib/test_bpf.ko ]; then + insmod ${OUTPUT}/lib/test_bpf.ko \ + test_suite=test_interpreter_fallback 2>/dev/null + if [ $? -ne 0 ]; then + rc=1 + fi + else + if ! /sbin/modprobe -q -n test_bpf \ + test_suite=test_interpreter_fallback; then + echo "test_bpf (test_suite=test_interpreter_fallback): [SKIP]" + elif /sbin/modprobe -q test_bpf \ + test_suite=test_interpreter_fallback; then + echo "test_bpf (test_suite=test_interpreter_fallback): ok" + else + echo "test_bpf (test_suite=test_interpreter_fallback): [FAIL]" + rc=1 + fi + fi + rmmod test_bpf 2>/dev/null + dmesg | grep FAIL +} + rc=0 test_save test_run 0 0 "$@" test_run 1 0 "$@" test_run 1 1 "$@" test_run 1 2 "$@" +test_interpreter_fallback test_restore exit $rc
Signature · 1 line
2.54.0
Quoted 4 lines · Leon Hwang
Sashiko reported two potential issues about interpreter fallback [1] [2].
After verifying them by patch #7, I think they are real issues. With
Sorry, it should be the patch #6.
Quoted 20 lines
LLM assistance, the interpreter does not support the internal
BPF_PROBE_ATOMICinsn and the gotox insn (used for indirect jumps), either.1) the user
BPF_ADDR_SPACE_CASTinsn
the interpreter just ignores it.2) the arena ST/STX/LDX insn
the interpreter could hit theBUG_ON()in___bpf_prog_run().3) the
BPF_MOV64_PERCPU_REGinsn
the interpreter could hit page fault, due to loading memory from invalid__percpupointer.4) the internal
BPF_PROBE_ATOMICinsn
the interpreter could hit theBUG_ON()in___bpf_prog_run().5) the gotox insn used for indirect jumps
the interpreter could hit theBUG_ON()in___bpf_prog_run(), too.
Quoted 6 lines
Reject these insns on interpreter fallback path in
__bpf_prog_select_runtime().This series is built on
"bpf: Fix unaligned interpreter panic on JIT fallback path" [3]. The patch #7 is also able to verify the issue of un-JITed helper.
^ patch #6
Quoted 2 lines
However, The patch #7 aims to verify the issues. I think it is not
^ patch #6
Quoted 4 lines
proper to be applied to upstream, because it adds a stub '
bpf_jit_test_fail_task' tobpf_prog_jit_compile()for the tests.I'd like to drop the patch #7 in the next revision.
^ patch #6
Thanks,
Leon
Quoted 24 lines
Link:
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/Leon Hwang (6):
bpf: Disallow interpreter fallback for userBPF_ADDR_SPACE_CASTinsn
bpf: Disallow interpreter fallback for arena insn
bpf: Disallow interpreter fallback forBPF_MOV64_PERCPU_REGinsn
bpf: Disallow interpreter fallback for internalBPF_PROBE_ATOMICinsn
bpf: Disallow interpreter fallback for gotox insnlib/test_bpf: Add interpreter-fallback testsinclude/linux/bpf.h | 1 + include/linux/filter.h | 4 + kernel/bpf/core.c | 69 +- lib/test_bpf.c | 800 ++++++++++++++++++++++- tools/lib/bpf/skel_internal.h | 2 + tools/testing/selftests/bpf/test_kmod.sh | 39 +- 6 files changed, 903 insertions(+), 12 deletions(-)--
2.54.0
Quoted 35 lines · Leon Hwang
Sashiko reported two potential issues about interpreter fallback [1] [2].
After verifying them by patch #7, I think they are real issues. With LLM assistance, the interpreter does not support the internal
BPF_PROBE_ATOMICinsn and the gotox insn (used for indirect jumps), either.1) the user
BPF_ADDR_SPACE_CASTinsn
the interpreter just ignores it.2) the arena ST/STX/LDX insn
the interpreter could hit theBUG_ON()in___bpf_prog_run().3) the
BPF_MOV64_PERCPU_REGinsn
the interpreter could hit page fault, due to loading memory from invalid__percpupointer.4) the internal
BPF_PROBE_ATOMICinsn
the interpreter could hit theBUG_ON()in___bpf_prog_run().5) the gotox insn used for indirect jumps
the interpreter could hit theBUG_ON()in___bpf_prog_run(), too.Reject these insns on interpreter fallback path in
__bpf_prog_select_runtime().This series is built on
"bpf: Fix unaligned interpreter panic on JIT fallback path" [3]. The patch #7 is also able to verify the issue of un-JITed helper.However, The patch #7 aims to verify the issues. I think it is not proper to be applied to upstream, because it adds a stub '
bpf_jit_test_fail_task' tobpf_prog_jit_compile()for the tests.
Quoted 6 lines
I'd like to drop the patch #7 in the next revision.
Link:
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
I don't think we need such fallback in patch [3].
And approach taken by this set also doesn't scale, since it splits the verifier/JIT logic into core.c which will be hard to keep consistent. I think we need another bit like jit_requested in prog like 'must_jit' that the verifier set for addr_space_cast, kfuncs and the rest.
Then the core.c change will be:
kernel/bpf/core.c +1 -1
@@ -2620,7 +2620,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct goto finalize;if (IS_ENABLED (CONFIG_BPF_JIT_ALWAYS_ON ) || -bpf_prog_has_kfunc_call (fp)) + fp->must_jit) jit_needed = true;
Quoted 1 line · Alexei Starovoitov
On Fri Jun 26, 2026 at 8:43 AM PDT, Leon Hwang wrote:
[...]
Quoted 2 lines
I don't think we need such fallback in patch [3].
Do you mean we can add the internal BPF_ADDR_PERCPU insn support to the interpreter?
See my reply in "PROBLEM: BPF interpreter fallback after JIT compilation of BPF_ADDR_PERCPU leads to kernel panic" thread [1].
[1]
https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
And approach taken by this set also doesn't scale, since it splits the verifier/JIT logic into
core.cwhich will be hard to keep consistent. I think we need another bit likejit_requestedin prog like 'must_jit' that the verifier set foraddr_space_cast, kfuncs and the rest.Then the
core.cchange will be:kernel/bpf/core.c +0
-0
kernel/bpf/core.c struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct · line 2625
Quoted 7 lines
kernel/bpf/core.c +1
-1@@ -2620,7 +2620,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct goto finalize;if (IS_ENABLED (CONFIG_BPF_JIT_ALWAYS_ON ) || -bpf_prog_has_kfunc_call (fp)) + fp->must_jit) jit_needed = true;
Got it. Will follow your suggestion.
Thanks,
Leon
Quoted 5 lines · Leon Hwang
Quoted 1 line · Alexei Starovoitov
On Fri Jun 26, 2026 at 8:43 AM PDT, Leon Hwang wrote:
[...]
Quoted 2 lines
I don't think we need such fallback in patch [3].
Sorry, it seems I misunderstood the 'patch [3]' as the patch #3 of this series.
Will discuss the fallback further in the sibling thread:
https://lore.kernel.org/bpf/[email protected]/.
Thanks,
Leon
Quoted 35 lines
Do you mean we can add the internal
BPF_ADDR_PERCPUinsn support to the interpreter?See my reply in "PROBLEM: BPF interpreter fallback after JIT compilation of
BPF_ADDR_PERCPUleads to kernel panic" thread [1].[1]
https://lore.kernel.org/bpf/[email protected]/Quoted 12 lines
And approach taken by this set also doesn't scale, since it splits the verifier/JIT logic into
core.cwhich will be hard to keep consistent. I think we need another bit likejit_requestedin prog like 'must_jit' that the verifier set foraddr_space_cast, kfuncs and the rest.Then the
core.cchange will be:kernel/bpf/core.c +0
-0kernel/bpf/core.c struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct · line 2625
Quoted 7 lines
kernel/bpf/core.c +1
-1@@ -2620,7 +2620,7 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct goto finalize;if (IS_ENABLED (CONFIG_BPF_JIT_ALWAYS_ON ) || -bpf_prog_has_kfunc_call (fp)) + fp->must_jit) jit_needed = true;Got it. Will follow your suggestion.
Thanks,
Leon
Quoted 8 lines · Leon Hwang
Quoted 1 line · Alexei Starovoitov
On Fri Jun 26, 2026 at 8:43 AM PDT, Leon Hwang wrote:
[...]
Quoted 2 lines
I don't think we need such fallback in patch [3].
Do you mean we can add the internal
BPF_ADDR_PERCPUinsn support to the interpreter?
No. See why CONFIG_BPF_JIT_ALWAYS_ON was introduced.
The interpreter is a security hole.
It has to be _compiled out_ of the kernel.
Hence all patches that improve fallback to interpreter are misguided. The interpreter has to be removed completely... sooner or later.
Quoted 5 lines · Alexei Starovoitov
Quoted 4 lines · Leon Hwang
Quoted 1 line · Alexei Starovoitov
On Fri Jun 26, 2026 at 8:43 AM PDT, Leon Hwang wrote:
[...]
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
I don't think we need such fallback in patch [3].
Do you mean we can add the internal
BPF_ADDR_PERCPUinsn support to the interpreter?No. See why
CONFIG_BPF_JIT_ALWAYS_ONwas introduced. The interpreter is a security hole.
Got it. The commit message of290af86629b2 ("bpf: introduce BPF_JIT_ALWAYS_ON config") has explained the security concern.
Quoted 3 lines
It has to be
_compiledout_of the kernel.
Hence all patches that improve fallback to interpreter are misguided. The interpreter has to be removed completely... sooner or later.
Will disallow interpreter fallback for the internal BPF_ADDR_PERCPU insn.
Thanks,
Leon