LoongArch: BPF: Use 4 instructions for function address in JIT
Subsystems (3)
Hengqi Chen
reviewerPATCH v2 1/1 LoongArch: BPF: Use 4 instructions for function address in JIT
This patch fixes the following issue of function calls in JIT, like:
[ 29.346981] multi-func JIT bug 105 != 103
The issus can be reproduced by running the "inline simple bpf_loop call" verifier test.
This is because we are emiting 2-4 instructions for 64-bit immediate moves. During the first pass of JIT, the placeholder address is zero, emiting two instructions for it. In the extra pass, the function address is in XKVRANGE, emiting four instructions for it. This change the instruction index in JIT context. Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
--- arch/loongarch/net/bpf_jit.c | 2 +- arch/loongarch/net/bpf_jit.h | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-)
arch/loongarch/net/bpf_jit.c +1 -1
@@ -841,7 +841,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extif (ret <0 )return ret; -move_imm (ctx, t1, func_addr, is32); +move_addr (ctx, t1, func_addr);emit_insn (ctx, jirl, t1,LOONGARCH_GPR_RA ,0 );move_reg (ctx, regmap[BPF_REG_0 ],LOONGARCH_GPR_A0 );break ;
arch/loongarch/net/bpf_jit.h +21 -0
@@ -82,6 +82,27 @@ static inline void emit_sext_32(struct jit_ctx *ctx, enum loongarch_gpr reg, booemit_insn (ctx, addiw, reg, reg,0 ); } +static inline void move_addr (struct jit_ctx *ctx,enum loongarch_gpr rd,u64 addr) +{ +u64 imm_11_0, imm_31_12, imm_51_32, imm_63_52; + +/* lu12iw rd, imm_31_12 */ + imm_31_12 = (addr >>12 ) &0xfffff ; +emit_insn (ctx, lu12iw, rd, imm_31_12); + +/* ori rd, rd, imm_11_0 */ + imm_11_0 = addr &0xfff ; +emit_insn (ctx, ori, rd, rd, imm_11_0); + +/* lu32id rd, imm_51_32 */ + imm_51_32 = (addr >>32 ) &0xfffff ; +emit_insn (ctx, lu32id, rd, imm_51_32); + +/* lu52id rd, rd, imm_63_52 */ + imm_63_52 = (addr >>52 ) &0xfff ; +emit_insn (ctx, lu52id, rd, rd, imm_63_52); +} +static inline void move_imm (struct jit_ctx *ctx,enum loongarch_gpr rd,long imm,bool is32) {long imm_11_0, imm_31_12, imm_51_32, imm_63_52, imm_51_0, imm_51_31;
--
2.31.1
Quoted 15 lines · Hengqi Chen
This patch fixes the following issue of function calls in JIT, like:
[ 29.346981] multi-func JIT bug 105 != 103The issus can be reproduced by running the "inline simple
bpf_loopcall" verifier test.This is because we are emiting 2-4 instructions for 64-bit immediate moves. During the first pass of JIT, the placeholder address is zero, emiting two instructions for it. In the extra pass, the function address is in XKVRANGE, emiting four instructions for it. This change the instruction index in JIT context. Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
Quoted 2 lines
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by Hengqi Chen <[email protected]>
Hi, Hengqi
Quoted 21 lines · Tiezhu Yang
Quoted 18 lines · Hengqi Chen
This patch fixes the following issue of function calls in JIT, like:
[ 29.346981] multi-func JIT bug 105 != 103The issus can be reproduced by running the "inline simple
bpf_loopcall" verifier test.This is because we are emiting 2-4 instructions for 64-bit immediate moves.
During the first pass of JIT, the placeholder address is zero, emiting two
instructions for it. In the extra pass, the function address is in XKVRANGE,
emiting four instructions for it. This change the instruction index in JIT context. Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
Quoted 5 lines
Quoted 2 lines
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by Hengqi Chen <[email protected]>Tested-by Tiezhu Yang <[email protected]>
In RISCV, I looked at the 'case BPF_JMP | BPF_CALL' handling, emit_call
emit_jump_and_link
The number of instructions is controlled by the 'force_jalr' parameter in emit_jump_and_link().
force_jalr = true: auipc/jalr (fixed-length)force_jalr = false: jal or auipc/jalr (variable-length)
"force_jalr" comes from "fixed" in bpf_jit_get_func_addr().
- In LoongArch, is it possible to do a similar operation by using 'func_addr_fixed' to control whether it is variable, instead of using a fixed-length unconditionally through move_addr()?
- Commit information can refer to b54b6003612a ("riscv, bpf: Emit fixed-length instructions for BPF_PSEUDO_FUNC").
- In 'case BPF_LD | BPF_IMM | BPF_DW', is it also necessary to add BPF_PSEUDO_FUNC processing?
Youling.
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Quoted 8 lines · On Tue, 14 Feb 2023 15:26:33 +0000 you wrote:
This patch fixes the following issue of function calls in JIT, like:
[ 29.346981] multi-func JIT bug 105 != 103The issus can be reproduced by running the "inline simple
bpf_loopcall" verifier test.[...]
Here is the summary with links:
- [v2] LoongArch: BPF: Use 4 instructions for function address in JIT
https://git.kernel.org/bpf/bpf-next/c/64f50f657572
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html