bpf, arm64: fix fp initialization for exception boundary
Subsystems (3)
Puranjay Mohan
maintainercover bpf, arm64: fix fp initialization for exception boundary
In the ARM64 BPF JIT when prog->aux->exception_boundary is set for a BPF program, find_used_callee_regs() is not called because for a program acting as exception boundary, all callee saved registers are saved. find_used_callee_regs() sets ctx->fp_used = true; when it sees FP being used in any of the instructions.
For programs acting as exception boundary, ctx->fp_used always remains false and therefore, BPF frame pointer is never set-up for such programs in the prologue.
This can cause crashes like:
With the following BPF program loaded and attached:
static __noinline int static_func(u64 i)
{
bpf_throw(0);
return i;
}
SEC("fentry/do_unlinkat")
int BPF_PROG(do_unlinkat, int dfd, struct filename *name)
{
volatile u64 a[2] = {0};
a[1] = __sync_fetch_and_add(&a[0], 1);
static_func(23);
return 0;
}Triggering it causes a page fault because the FP register is not initialised.
[root@localhost ~]# touch test
[root@localhost ~]# rm test
Unable to handle kernel paging request at virtual address fffffffffffffff0 Mem abort info: ESR = 0x0000000096000006 EC = 0x25: DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 FSC = 0x06: level 2 translation fault Data abort info: ISV = 0, ISS = 0x00000006, ISS2 = 0x00000000 CM = 0, WnR = 0, TnD = 0, TagAccess = 0 GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000043783000 [fffffffffffffff0] pgd=0000000000000000, p4d=00000000450a0403, pud=00000000450a1403, pmd=0000000000000000 Internal error: Oops: 0000000096000006 [#1] SMP Modules linked in: CPU: 12 UID: 0 PID: 487 Comm: rm Not tainted 6.16.0-rc6-00212-g7abc678e3084 #7 PREEMPT Hardware name: linux,dummy-virt (DT) pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : bpf_prog_5148f2d6554f3ab3_do_unlinkat+0x48/0x90 lr : bpf_trampoline_6442562433+0x68/0x168 sp : ffff80008c8b3d10 x29: ffff80008c8b3d80 x28: ffff0000d66d0000 x27: 0000000000000000 x26: ffff80008c8b3d70 x25: 0000000000000000 x24: 0000000000000000 x23: 0000000060001000 x22: 0000ffffaea95b0c x21: 00000000ffffffff x20: 0000000000000001 x19: ffff80008c1bd000 x18: 0000000000000000 x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 x14: 0000000000000000 x13: 0000000000020000 x12: 0000000000000015 x11: 0000000000000000 x10: fffffffffffffff0 x9 : ffff8000871d65e8 x8 : ffff800084eb1d30 x7 : 0000000000000000 x6 : 0000000000000001 x5 : 00000000c3affd79 x4 : 000000001055937d x3 : ffff801809627000 x2 : ffff0000d66d0000 x1 : 0000000100000000 x0 : 0000000000000001 Call trace: bpf_prog_5148f2d6554f3ab3_do_unlinkat+0x48/0x90 (P) bpf_trampoline_6442562433+0x68/0x168 do_unlinkat+0x8/0x290 __arm64_sys_unlinkat+0x44/0x90 invoke_syscall+0x50/0x120 el0_svc_common.constprop.0+0xc8/0xf0 do_el0_svc+0x24/0x38 el0_svc+0x48/0xf0 el0t_64_sync_handler+0xc8/0xd0 el0t_64_sync+0x198/0x1a0 Code: f90007e0 f90003e0 d2800020 d100432a (f8e00140) ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Oops: Fatal exception SMP: stopping secondary CPUs Kernel Offset: disabled CPU features: 0x2000,000081c0,02004ca1,04407a0b Memory Limit: none ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---
Please take it into the bpf tree if needed, I sent it to bpf-next as this bug is multiple months old.
Puranjay Mohan (1):
bpf, arm64: fix fp initialization for exception boundary
arch/arm64/net/bpf_jit_comp.c | 1 + 1 file changed, 1 insertion(+)
Signature · 1 line
2.47.1
Puranjay Mohan
maintainerPATCH 1/1 bpf, arm64: fix fp initialization for exception boundary
In the ARM64 BPF JIT when prog->aux->exception_boundary is set for a BPF program, find_used_callee_regs() is not called because for a program acting as exception boundary, all callee saved registers are saved. find_used_callee_regs() sets ctx->fp_used = true; when it sees FP being used in any of the instructions.
For programs acting as exception boundary, ctx->fp_used remains false even if frame pointer is used by the program and therefore, FP is not set-up for such programs in the prologue. This can cause the kernel to crash due to a pagefault.
Fix it by setting ctx->fp_used = true for exception boundary programs as fp is always saved in such programs.
--- arch/arm64/net/bpf_jit_comp.c | 1 + 1 file changed, 1 insertion(+)
arch/arm64/net/bpf_jit_comp.c +1 -0
@@ -412,6 +412,7 @@ static void push_callee_regs(struct jit_ctx *ctx)emit (A64_PUSH (A64_R (23 ),A64_R (24 ),A64_SP ), ctx);emit (A64_PUSH (A64_R (25 ),A64_R (26 ),A64_SP ), ctx);emit (A64_PUSH (A64_R (27 ),A64_R (28 ),A64_SP ), ctx); + ctx->fp_used = true; }else {find_used_callee_regs (ctx);for (i =0 ; i +1 < ctx->nr_used_callee_reg; i +=2 ) {
Signature · 1 line
2.47.1
Quoted 25 lines · Puranjay Mohan
In the ARM64 BPF JIT when
prog->aux->exception_boundaryis set for a BPF program,find_used_callee_regs()is not called because for a program acting as exception boundary, all callee saved registers are saved.find_used_callee_regs()setsctx->fp_used = true;when it sees FP being used in any of the instructions.For programs acting as exception boundary,
ctx->fp_usedremains false even if frame pointer is used by the program and therefore, FP is not set-up for such programs in the prologue. This can cause the kernel to crash due to a pagefault.Fix it by setting
ctx->fp_used= true for exception boundary programs as fp is always saved in such programs.Fixes 5d4fa9ec5643 ("bpf, arm64: Avoid blindly saving/restoring all callee-saved registers")Signed-off-by Puranjay Mohan <[email protected]>--- arch/arm64/net/bpf_jit_comp.c | 1 + 1 file changed, 1 insertion(+)arch/arm64/net/bpf_jit_comp.c +0
-0
arch/arm64/net/bpf_jit_comp.c static void push_callee_regs(struct jit_ctx *ctx) · line 418
Quoted 7 lines
arch/arm64/net/bpf_jit_comp.c +1
-0@@ -412,6 +412,7 @@ static void push_callee_regs(struct jit_ctx *ctx)emit (A64_PUSH (A64_R (23 ),A64_R (24 ),A64_SP ), ctx);emit (A64_PUSH (A64_R (25 ),A64_R (26 ),A64_SP ), ctx);emit (A64_PUSH (A64_R (27 ),A64_R (28 ),A64_SP ), ctx); + ctx->fp_used = true; }else {find_used_callee_regs (ctx);for (i =0 ; i +1 < ctx->nr_used_callee_reg; i +=2 ) {
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Quoted 10 lines · On Tue, 22 Jul 2025 13:34:08 +0000 you wrote:
In the ARM64 BPF JIT when
prog->aux->exception_boundaryis set for a BPF program,find_used_callee_regs()is not called because for a program acting as exception boundary, all callee saved registers are saved.find_used_callee_regs()setsctx->fp_used = true;when it sees FP being used in any of the instructions.
For programs acting as exception boundary,ctx->fp_usedalways remains false and therefore, BPF frame pointer is never set-up for such programs in the prologue.[...]
Here is the summary with links:
- [bpf-next,1/1] bpf, arm64: fix fp initialization for exception boundary
https://git.kernel.org/bpf/bpf-next/c/b114fcee766d
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html