LoongArch: BPF: Fix off by one error in build_prologue()
Subsystems (3)
Hengqi Chen
reviewerPATCH 1/1 LoongArch: BPF: Fix off by one error in build_prologue()
Vincent reported that running BPF progs with tailcalls on LoongArch causes kernel hard lockup. Debugging the issues shows that the JITed image missing a jirl instruction at the end of the epilogue.
There are two passes in JIT compile, the first pass set the flags and the second pass generates JIT code based on those flags. With BPF progs mixing bpf2bpf and tailcalls, build_prologue() generates N insns in the first pass and then generates N+1 insns in the second pass. This makes epilogue_offset off by one and we will jump to some unexpected insn and cause lockup. Fix this by inserting a nop insn.
--- arch/loongarch/include/asm/inst.h | 5 +++++ arch/loongarch/net/bpf_jit.c | 2 ++ 2 files changed, 7 insertions(+)
arch/loongarch/include/asm/inst.h +5 -0
@@ -695,6 +695,11 @@ static inline void emit_jirl(union loongarch_instruction *insn, insn->reg2i16_format.rj = rj; } +static inline void emit_nop (union loongarch_instruction *insn) +{ + insn->word =INSN_NOP ; +} +#define DEF_EMIT_REG2BSTRD_FORMAT (NAME ,OP ) \ static inline void emit_##NAME(union loongarch_instruction *insn, \ enum loongarch_gpr rd, \
arch/loongarch/net/bpf_jit.c +2 -0
@@ -142,6 +142,8 @@ static void build_prologue(struct jit_ctx *ctx)*/ if (seen_tail_call (ctx) &&seen_call (ctx))move_reg (ctx,TCC_SAVED ,REG_TCC ); +else +emit_insn (ctx, nop); ctx->stack_size = stack_adjust; }
Signature · 1 line
2.43.5
Quoted 53 lines · Hengqi Chen
Vincent reported that running BPF progs with tailcalls on LoongArch causes kernel hard lockup. Debugging the issues shows that the JITed image missing a jirl instruction at the end of the epilogue.
There are two passes in JIT compile, the first pass set the flags and the second pass generates JIT code based on those flags. With BPF progs mixing bpf2bpf and tailcalls,
build_prologue()generates N insns in the first pass and then generates N+1 insns in the second pass. This makesepilogue_offsetoff by one and we will jump to some unexpected insn and cause lockup. Fix this by inserting a nop insn.Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Fixes bb035ef0cc91 ("LoongArch: BPF: Support mixing bpf2bpf and tailcalls")Reported-by Vincent Li <[email protected]>Closes https://lore.kernel.org/loongarch/CAK3+h2w6WESdBN3UCr3WKHByD7D6Q_Ve1EDAjotVrnx6Or_c8g@mail.gmail.com/ open hereCloses https://lore.kernel.org/bpf/CAK3+h2woEjG_N=-XzqEGaAeCmgu2eTCUc7p6bP4u8Q+DFHm-7g@mail.gmail.com/ open hereSigned-off-by Hengqi Chen <[email protected]>--- arch/loongarch/include/asm/inst.h | 5 +++++ arch/loongarch/net/bpf_jit.c | 2 ++ 2 files changed, 7 insertions(+)arch/loongarch/include/asm/inst.h +5
-0@@ -695,6 +695,11 @@ static inline void emit_jirl(union loongarch_instruction *insn, insn->reg2i16_format.rj = rj; } +static inline void emit_nop (union loongarch_instruction *insn) +{ + insn->word =INSN_NOP ; +} +#define DEF_EMIT_REG2BSTRD_FORMAT (NAME ,OP ) \ static inline void emit_##NAME(union loongarch_instruction *insn, \ enum loongarch_gpr rd, \arch/loongarch/net/bpf_jit.c +2
-1@@ -142,6 +142,8 @@ static void build_prologue(struct jit_ctx *ctx)*/ if (seen_tail_call (ctx) &&seen_call (ctx))move_reg (ctx,TCC_SAVED ,REG_TCC ); +else +emit_insn (ctx, nop); ctx->stack_size = stack_adjust; } --
Quoted 2 lines
2.43.5
Thanks Hengqi for the patch, I tested the patch on the LoongArch platform with the loxilb tc BPF program, it does not lockup up the kernel anymore.
attach loxilb tc BPF program to loongfire firewall green0 interface
[root@loongfire ~]# loxilb --whitelist="green0"
loxilb start
...
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook1 prog tc_packet_func
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook2 prog tc_packet_func_slow
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook3 prog tc_packet_func_fw
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook4 prog tc_csum_func1
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook5 prog tc_csum_func2
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook6 prog tc_slow_unp_func
14:43:13 DEBUG common_libbpf.c:161: tc: autoload sec tc_packet_hook7 prog tc_packet_func_masq
libbpf: Error in bpf_create_map_xattr(nh_map):Invalid argument(-22). Retrying without BTF.
libbpf: Error in bpf_create_map_xattr(xctk):Operation not supported(-95). Retrying without BTF.
2025-03-15 14:43:14 Get xsync()
14:43:15 INFO common_libbpf.c:204: tc: bpf attach OK for green0 <===============
14:43:15 DEBUG loxilb_libdp.c:3311: llb_link_prop_add: IF-green0 added idx 1 type 2
2025-03-15 14:43:15 ebpf intfmap added - 5 vlan 0 -> 1 2025-03-15 14:43:15 ebpf txintfmap added - 1 -> 5
Above log shows the program loaded and attached to the green0 interface, no lockup.
One strange thing is that bpftool net does not show the tc BPF program attached to green0 interface, attached to virtual interface llb0 ok, I guess this should be irrelevant to kernel lockup issue.
[root@loongfire ~]# bpftool net
xdp:
llb0(7) generic id 55
tc:
llb0(7) clsact/ingress tc_packet_func_:[59] id 59
flow_dissector:
netfilter:
Applied, thanks.
Huacai
Quoted 56 lines · Hengqi Chen
Vincent reported that running BPF progs with tailcalls on LoongArch causes kernel hard lockup. Debugging the issues shows that the JITed image missing a jirl instruction at the end of the epilogue.
There are two passes in JIT compile, the first pass set the flags and the second pass generates JIT code based on those flags. With BPF progs mixing bpf2bpf and tailcalls,
build_prologue()generates N insns in the first pass and then generates N+1 insns in the second pass. This makesepilogue_offsetoff by one and we will jump to some unexpected insn and cause lockup. Fix this by inserting a nop insn.Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Fixes bb035ef0cc91 ("LoongArch: BPF: Support mixing bpf2bpf and tailcalls")Reported-by Vincent Li <[email protected]>Closes https://lore.kernel.org/loongarch/CAK3+h2w6WESdBN3UCr3WKHByD7D6Q_Ve1EDAjotVrnx6Or_c8g@mail.gmail.com/ open hereCloses https://lore.kernel.org/bpf/CAK3+h2woEjG_N=-XzqEGaAeCmgu2eTCUc7p6bP4u8Q+DFHm-7g@mail.gmail.com/ open hereSigned-off-by Hengqi Chen <[email protected]>--- arch/loongarch/include/asm/inst.h | 5 +++++ arch/loongarch/net/bpf_jit.c | 2 ++ 2 files changed, 7 insertions(+)arch/loongarch/include/asm/inst.h +5
-0@@ -695,6 +695,11 @@ static inline void emit_jirl(union loongarch_instruction *insn, insn->reg2i16_format.rj = rj; } +static inline void emit_nop (union loongarch_instruction *insn) +{ + insn->word =INSN_NOP ; +} +#define DEF_EMIT_REG2BSTRD_FORMAT (NAME ,OP ) \ static inline void emit_##NAME(union loongarch_instruction *insn, \ enum loongarch_gpr rd, \arch/loongarch/net/bpf_jit.c +2
-1@@ -142,6 +142,8 @@ static void build_prologue(struct jit_ctx *ctx)*/ if (seen_tail_call (ctx) &&seen_call (ctx))move_reg (ctx,TCC_SAVED ,REG_TCC ); +else +emit_insn (ctx, nop); ctx->stack_size = stack_adjust; } --2.43.5