Allow mixing bpf2bpf calls with tailcalls on LoongArch
Subsystems (3)
Hengqi Chen
reviewercover Allow mixing bpf2bpf calls with tailcalls on LoongArch
This patchset enables mixing bpf2bpf calls with tailcalls on LoongArch.
The first patch fixes JIT for function calls, like:
[ 29.346981] multi-func JIT bug 105 != 103
This is because we are emiting variable instructions for 64-bit immediate moves. During the first pass of JIT, the placeholder address is just zero, emiting two instructions for it. In the extra pass, the function address is in XKVRANGE, emiting four instructions for it. This change the instruction index in JIT context. Fix it by using a fixed 4-instruction sequence.
The second patch enables mixing bpf2bpf calls with tailcalls on LoongArch.
Hengqi Chen (2):
LoongArch: BPF: Treat function address as 64-bit value
LoongArch: BPF: Support mixing bpf2bpf and tailcalls
arch/loongarch/net/bpf_jit.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-)
--
2.31.1
Hengqi Chen
reviewerPATCH 1/2 LoongArch: BPF: Treat function address as 64-bit value
Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
--- arch/loongarch/net/bpf_jit.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
arch/loongarch/net/bpf_jit.c +22 -1
@@ -446,6 +446,27 @@ static int add_exception_handler(const struct bpf_insn *insn,return 0 ; } +static inline void emit_addr_move (struct jit_ctx *ctx,enum loongarch_gpr rd,u64 addr) +{ +u64 imm_11_0, imm_31_12, imm_51_32, imm_63_52; + +/* lu12iw rd, imm_31_12 */ + imm_31_12 = (addr >>12 ) &0xfffff ; +emit_insn (ctx, lu12iw, rd, imm_31_12); + +/* ori rd, rd, imm_11_0 */ + imm_11_0 = addr &0xfff ; +emit_insn (ctx, ori, rd, rd, imm_11_0); + +/* lu32id rd, imm_51_32 */ + imm_51_32 = (addr >>32 ) &0xfffff ; +emit_insn (ctx, lu32id, rd, imm_51_32); + +/* lu52id rd, rd, imm_63_52 */ + imm_63_52 = (addr >>52 ) &0xfff ; +emit_insn (ctx, lu52id, rd, rd, imm_63_52); +} +static int build_insn (const struct bpf_insn *insn,struct jit_ctx *ctx,bool extra_pass) {u8 tm =-1 ;
@@ -841,7 +862,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extif (ret <0 )return ret; -move_imm (ctx, t1, func_addr, is32); +emit_addr_move (ctx, t1, func_addr);emit_insn (ctx, jirl, t1,LOONGARCH_GPR_RA ,0 );move_reg (ctx, regmap[BPF_REG_0 ],LOONGARCH_GPR_A0 );break ;
Signature · 1 line
2.31.1
Tiezhu Yang
maintainerRe: [PATCH 1/2] LoongArch: BPF: Treat function address as 64-bit value
Hi Hengqi,
Quoted 44 lines · Hengqi Chen
Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by Hengqi Chen <[email protected]>--- arch/loongarch/net/bpf_jit.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)arch/loongarch/net/bpf_jit.c +21
-0@@ -446,6 +446,27 @@ static int add_exception_handler(const struct bpf_insn *insn,return 0 ; } +static inline void emit_addr_move (struct jit_ctx *ctx,enum loongarch_gpr rd,u64 addr) +{ +u64 imm_11_0, imm_31_12, imm_51_32, imm_63_52; + +/* lu12iw rd, imm_31_12 */ + imm_31_12 = (addr >>12 ) &0xfffff ; +emit_insn (ctx, lu12iw, rd, imm_31_12); + +/* ori rd, rd, imm_11_0 */ + imm_11_0 = addr &0xfff ; +emit_insn (ctx, ori, rd, rd, imm_11_0); + +/* lu32id rd, imm_51_32 */ + imm_51_32 = (addr >>32 ) &0xfffff ; +emit_insn (ctx, lu32id, rd, imm_51_32); + +/* lu52id rd, rd, imm_63_52 */ + imm_63_52 = (addr >>52 ) &0xfff ; +emit_insn (ctx, lu52id, rd, rd, imm_63_52); +} +static int build_insn (const struct bpf_insn *insn,struct jit_ctx *ctx,bool extra_pass) {u8 tm =-1 ;
arch/loongarch/net/bpf_jit.c static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext · line 868
Quoted 8 lines
arch/loongarch/net/bpf_jit.c +1
-1@@ -841,7 +862,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extif (ret <0 )return ret; -move_imm (ctx, t1, func_addr, is32); +emit_addr_move (ctx, t1, func_addr);emit_insn (ctx, jirl, t1,LOONGARCH_GPR_RA ,0 );move_reg (ctx, regmap[BPF_REG_0 ],LOONGARCH_GPR_A0 );break ;
The code itself looks good to me.
Could you please give more detailed info in the commit message? For example, description of problem, steps to reproduce, ... I think the descriptions in the cover letter are useful, it is better to record them in the commit message.
Additionally, emit_addr_move() is similar with move_imm(), it is better to define emit_addr_move() before move_imm() in bpf_jit.h.
Thanks,
Tiezhu
Tiezhu Yang
maintainerRe: [PATCH 1/2] LoongArch: BPF: Treat function address as 64-bit value
Quoted 22 lines · Tiezhu Yang
Hi Hengqi,
Quoted 19 lines · Hengqi Chen
Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by Hengqi Chen <[email protected]>--- arch/loongarch/net/bpf_jit.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)arch/loongarch/net/bpf_jit.c +0
-0@@ -446,6 +446,27 @@ static int add_exception_handler(const structbpf_insn *insn, return 0; }
Quoted 2 lines
arch/loongarch/net/bpf_jit.c static int add_exception_handler(const struct · line 451
Quoted 2 lines
arch/loongarch/net/bpf_jit.c +1
-0+static inline void emit_addr_move (struct jit_ctx *ctx,enum loongarch_gpr rd,u64 addr)
Small nit:
Maybe use move_addr() ( like move_imm() ) is better than emit_addr_move()?
Thanks,
Tiezhu
Hengqi Chen
reviewerRe: [PATCH 1/2] LoongArch: BPF: Treat function address as 64-bit value
In reply to Tiezhu YangHi Tiezhu,
Quoted 64 lines · Tiezhu Yang
Hi Hengqi,
Quoted 44 lines · Hengqi Chen
Let's always use 4 instructions for function address in JIT. So that the instruction sequences don't change between the first pass and the extra pass for function calls.
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by Hengqi Chen <[email protected]>--- arch/loongarch/net/bpf_jit.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)arch/loongarch/net/bpf_jit.c +21
-0@@ -446,6 +446,27 @@ static int add_exception_handler(const struct bpf_insn *insn,return 0 ; } +static inline void emit_addr_move (struct jit_ctx *ctx,enum loongarch_gpr rd,u64 addr) +{ +u64 imm_11_0, imm_31_12, imm_51_32, imm_63_52; + +/* lu12iw rd, imm_31_12 */ + imm_31_12 = (addr >>12 ) &0xfffff ; +emit_insn (ctx, lu12iw, rd, imm_31_12); + +/* ori rd, rd, imm_11_0 */ + imm_11_0 = addr &0xfff ; +emit_insn (ctx, ori, rd, rd, imm_11_0); + +/* lu32id rd, imm_51_32 */ + imm_51_32 = (addr >>32 ) &0xfffff ; +emit_insn (ctx, lu32id, rd, imm_51_32); + +/* lu52id rd, rd, imm_63_52 */ + imm_63_52 = (addr >>52 ) &0xfff ; +emit_insn (ctx, lu52id, rd, rd, imm_63_52); +} +static int build_insn (const struct bpf_insn *insn,struct jit_ctx *ctx,bool extra_pass) {u8 tm =-1 ;arch/loongarch/net/bpf_jit.c static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext · line 868
Quoted 8 lines
arch/loongarch/net/bpf_jit.c +1
-1@@ -841,7 +862,7 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extif (ret <0 )return ret; -move_imm (ctx, t1, func_addr, is32); +emit_addr_move (ctx, t1, func_addr);emit_insn (ctx, jirl, t1,LOONGARCH_GPR_RA ,0 );move_reg (ctx, regmap[BPF_REG_0 ],LOONGARCH_GPR_A0 );break ;The code itself looks good to me.
Could you please give more detailed info in the commit message? For example, description of problem, steps to reproduce, ... I think the descriptions in the cover letter are useful, it is better to record them in the commit message.
Quoted 6 lines
Additionally,
emit_addr_move()is similar withmove_imm(), it is better to defineemit_addr_move()beforemove_imm()inbpf_jit.h.Thanks,
Tiezhu
I've addressed your comments and send a v2 for review. The second patch is dropped, as it is incomplete.
Cheers,
---
Hengqi
Hengqi Chen
reviewerPATCH 2/2 LoongArch: BPF: Support mixing bpf2bpf and tailcalls
In reply to Hengqi ChenThe current implementation already allow such mixing. Let's enable it in JIT.
--- arch/loongarch/net/bpf_jit.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/loongarch/net/bpf_jit.c +6 -0
@@ -1269,3 +1269,9 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)return prog; } + +/* Indicate the JIT backend supports mixing bpf2bpf and tailcalls. */ +bool bpf_jit_supports_subprog_tailcalls (void ) +{ +return true; +}
Signature · 1 line
2.31.1