bpf, mips: Add BPF_MOVSX support to the JITs
Subsystems (3)
cover bpf, mips: Add BPF_MOVSX support to the JITs
bpf-next is closed for the merge window, so I am sending this for review. I plan to post this series after it reopens.
The MIPS32 and MIPS64 JITs lower register BPF_MOVSX as an ordinary move because their register-move paths do not interpret insn->off. Negative low-width values therefore retain incorrect upper bits.
First factor ordinary register moves into backend-local 32- and 64-bit helpers. Then handle the BPF-defined MOVSX widths from insn->off in those helpers. This keeps the verifier-inserted ALU32 zero extension on its existing path and preserves the MIPS32 register-pair handling. The implementation uses shift sequences rather than the R2+ seb/seh instructions so it also works on base MIPS32.
MOVSX can be applied before or after the in-flight SDIV/SMOD series. Both must precede MEMSX (yet to be sent). The ordering discussion is here:
Patch 1 did not change any test_bpf result. On the four little-endian profiles where the complete suite was run at each boundary, patch 2 fixed the ALU32 byte and halfword MOVSX cases and the ALU64 byte, halfword, and word cases. The 32-bit totals moved from 1031 passes and 31 failures to 1036 passes and 26 failures; the 64-bit totals moved from 1030 passes and 31 failures to 1035 passes and 26 failures. Patch 2 did not change any other test result. Ordinary MOV and zero-extension controls passed at each boundary.
Selector-focused QEMU testing covered MIPS32 base, R2, and R6 plus pre-R6 MIPS64, each in big- and little-endian configurations. No physical MIPS hardware was tested.
Nicholas Dudar (2):
bpf, mips: Factor register moves into helpers bpf, mips: Add support for BPF_MOVSX in the JITs
arch/mips/net/bpf_jit_comp32.c | 54 +++++++++++++++++++++++++++++++--- arch/mips/net/bpf_jit_comp64.c | 48 ++++++++++++++++++++++++++++-- 2 files changed, 95 insertions(+), 7 deletions(-)
base-commit: 6655c409707ec8ce9ce0850ffe4fe02331fd4d9c
RFC 1/2 bpf, mips: Factor register moves into helpers
Both MIPS JITs currently lower register MOV instructions directly in build_insn(), duplicating backend-specific handling for ALU32 and ALU64 destinations.
Introduce 32-bit and 64-bit register-move helpers in each backend and route the existing MOV paths through them. Keep the verifier-inserted zero-extension marker on its dedicated path, so this is a behavior-preserving refactor.
--- arch/mips/net/bpf_jit_comp32.c | 22 ++++++++++++++++++---- arch/mips/net/bpf_jit_comp64.c | 18 +++++++++++++++--- 2 files changed, 33 insertions(+), 7 deletions(-)
arch/mips/net/bpf_jit_comp32.c +18 -4
@@ -190,6 +190,22 @@ static void emit_zext_ver(struct jit_context *ctx, const u8 dst[]) } } +/* Register move operation (32-bit) */ +static void emit_mov_r32 (struct jit_context *ctx,const u8 dst[], +const u8 src[]) +{ +emit_mov_r (ctx,lo (dst),lo (src)); +emit_zext_ver (ctx, dst); +} + +/* Register move operation (64-bit) */ +static void emit_mov_r64 (struct jit_context *ctx,const u8 dst[], +const u8 src[]) +{ +emit_mov_r (ctx,lo (dst),lo (src)); +emit_mov_r (ctx,hi (dst),hi (src)); +} +/* Load delay slot, if ISA mandates it */ static void emit_load_delay (struct jit_context *ctx) {
@@ -1485,8 +1501,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)/* Special mov32 for zext */ emit_mov_i (ctx,hi (dst),0 ); }else { -emit_mov_r (ctx,lo (dst),lo (src)); -emit_zext_ver (ctx, dst); +emit_mov_r32 (ctx, dst, src); }break ;/* dst = -dst */
@@ -1555,8 +1570,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)break ;/* dst = src (64-bit) */ case BPF_ALU64 |BPF_MOV |BPF_X : -emit_mov_r (ctx,lo (dst),lo (src)); -emit_mov_r (ctx,hi (dst),hi (src)); +emit_mov_r64 (ctx, dst, src);break ;/* dst = -dst (64-bit) */ case BPF_ALU64 |BPF_NEG :
arch/mips/net/bpf_jit_comp64.c +15 -3
@@ -120,6 +120,19 @@ static void emit_zext_ver(struct jit_context *ctx, u8 dst)emit_zext (ctx, dst); } +/* Register move operation (32-bit) */ +static void emit_mov_r32 (struct jit_context *ctx,u8 dst,u8 src) +{ +emit_mov_r (ctx, dst, src); +emit_zext_ver (ctx, dst); +} + +/* Register move operation (64-bit) */ +static void emit_mov_r64 (struct jit_context *ctx,u8 dst,u8 src) +{ +emit_mov_r (ctx, dst, src); +} +/* dst = imm (64-bit) */ static void emit_mov_i64 (struct jit_context *ctx,u8 dst,u64 imm64) {
@@ -656,8 +669,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)/* Special mov32 for zext */ emit_zext (ctx, dst); }else { -emit_mov_r (ctx, dst, src); -emit_zext_ver (ctx, dst); +emit_mov_r32 (ctx, dst, src); }break ;/* dst = -dst */
@@ -742,7 +754,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)break ;/* dst = src (64-bit) */ case BPF_ALU64 |BPF_MOV |BPF_X : -emit_mov_r (ctx, dst, src); +emit_mov_r64 (ctx, dst, src);break ;/* dst = -dst (64-bit) */ case BPF_ALU64 |BPF_NEG :
[email protected]
AI reviewRe: [RFC bpf-next 1/2] bpf, mips: Factor register moves into helpers
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Critical] The MIPS BPF JIT silently miscompiles BPF_MOVSX (move with sign extension) instructions as standard zero-extended or 64-bit copies. --
--- Patch [1]: [RFC bpf-next 1/2] bpf, mips: Factor register moves into helpers --- commit a341e9c2445aa10a781c4264cc9b57117f7c9711
Author: Nicholas Dudar <[email protected]>
bpf, mips: Factor register moves into helpers
This patch refactors the MIPS BPF JIT by introducing 32-bit and 64-bit register-move helpers to remove duplicated backend-specific handling for MOV instructions. It preserves the verifier-inserted zero-extension marker behavior.
Quoted 4 lines
arch/mips/net/bpf_jit_comp32.c +0
-0
[ ... ]
arch/mips/net/bpf_jit_comp32.c int build_insn(const struct bpf_insn *insn, struct jit_context *ctx) · line 1504
Quoted 7 lines
arch/mips/net/bpf_jit_comp32.c +1
-2@@ -1485,8 +1501,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)/* Special mov32 for zext */ emit_mov_i (ctx,hi (dst),0 ); }else { -emit_mov_r (ctx,lo (dst),lo (src)); -emit_zext_ver (ctx, dst); +emit_mov_r32 (ctx, dst, src);
[Severity: Critical]
This is a pre-existing issue, but does this switch case in build_insn() silently miscompile BPF_MOVSX (move with sign extension) instructions?
When the BPF program uses a move with sign extension (insn->off != 0), this code generation appears to perform a standard zero-extended copy instead. Since the verifier tracks bounds assuming the value was sign-extended, could this discrepancy allow a program to bypass verifier bounds checks and perform out-of-bounds memory accesses?
Since you are refactoring the MOV handlers here, should emit_mov_r32() be updated to take insn->off and implement the proper sign extension logic?
Quoted 3 lines
arch/mips/net/bpf_jit_comp32.c +0
-0}break ;/* dst = -dst */
[ ... ]
Quoted 4 lines
arch/mips/net/bpf_jit_comp64.c +0
-0
[ ... ]
arch/mips/net/bpf_jit_comp64.c int build_insn(const struct bpf_insn *insn, struct jit_context *ctx) · line 757
Quoted 6 lines
arch/mips/net/bpf_jit_comp64.c +1
-1@@ -742,7 +754,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)break ;/* dst = src (64-bit) */ case BPF_ALU64 |BPF_MOV |BPF_X : -emit_mov_r (ctx, dst, src); +emit_mov_r64 (ctx, dst, src);
[Severity: Critical]
This is also a pre-existing issue, but does the 64-bit MOV handler similarly ignore insn->off in build_insn()?
Like the 32-bit case, if a program uses a BPF_MOVSX instruction, does this result in a standard 64-bit copy instead of the required sign extension?
Quoted 3 lines
arch/mips/net/bpf_jit_comp64.c +0
-0break ;/* dst = -dst (64-bit) */ case BPF_ALU64 |BPF_NEG :
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [RFC bpf-next 1/2] bpf, mips: Factor register moves into helpers
Thanks for the review.
Quoted 2 lines
This is a pre-existing issue, but does this switch case in
build_insn()silently miscompileBPF_MOVSX(move with sign extension) instructions?
Yes. Patch 1/2 is a behavior-preserving helper extraction, so that intermediate retains the existing behavior.
Quoted 2 lines
Since you are refactoring the MOV handlers here, should
emit_mov_r32()be updated to takeinsn->offand implement the proper sign extension logic?
Patch 2/2 does that for both emit_mov_r32() and emit_mov_r64(). It passes insn->off to the move helpers. They handle these verifier-valid widths:
8 and 16 bits for ALU32, and 8, 16, and 32 bits for ALU64. An offset of zero remains an ordinary move. The verifier-inserted zero-extension move stays on its separate path.
RFC 2/2 bpf, mips: Add support for BPF_MOVSX in the JITs
In reply to Nicholas DudarThe MIPS32 and MIPS64 JITs lower register BPF_MOVSX instructions as ordinary moves because their register-move paths do not interpret insn->off. Negative low-width values therefore retain incorrect upper bits.
Teach the register-move helpers to sign-extend the BPF-defined MOVSX
widths: 8 and 16 for ALU32, and 8, 16, and 32 for ALU64. Propagate the sign into the MIPS32 high word for ALU64, while retaining the existing verifier-managed zero extension for ALU32.
Keep the verifier-inserted zero-extension move on its dedicated path and interpret the raw offset within the MOV helpers.
--- arch/mips/net/bpf_jit_comp32.c | 50 ++++++++++++++++++++++++++++------ arch/mips/net/bpf_jit_comp64.c | 46 +++++++++++++++++++++++++------ 2 files changed, 79 insertions(+), 17 deletions(-)
arch/mips/net/bpf_jit_comp32.c +41 -9
@@ -190,20 +190,52 @@ static void emit_zext_ver(struct jit_context *ctx, const u8 dst[]) } } -/* Register move operation (32-bit) */ +/* Register move operation (32-bit), optionally with sign extension */ static void emit_mov_r32 (struct jit_context *ctx,const u8 dst[], -const u8 src[]) +const u8 src[],s16 off) { -emit_mov_r (ctx,lo (dst),lo (src)); +switch (off) { +case 8 : +emit (ctx, sll,lo (dst),lo (src),24 ); +emit (ctx, sra,lo (dst),lo (dst),24 ); +clobber_reg (ctx,lo (dst)); +break ; +case 16 : +emit (ctx, sll,lo (dst),lo (src),16 ); +emit (ctx, sra,lo (dst),lo (dst),16 ); +clobber_reg (ctx,lo (dst)); +break ; +default : +emit_mov_r (ctx,lo (dst),lo (src)); +break ; + }emit_zext_ver (ctx, dst); } -/* Register move operation (64-bit) */ +/* Register move operation (64-bit), optionally with sign extension */ static void emit_mov_r64 (struct jit_context *ctx,const u8 dst[], -const u8 src[]) +const u8 src[],s16 off) { -emit_mov_r (ctx,lo (dst),lo (src)); -emit_mov_r (ctx,hi (dst),hi (src)); +switch (off) { +case 8 : +emit (ctx, sll,lo (dst),lo (src),24 ); +emit (ctx, sra,lo (dst),lo (dst),24 ); +break ; +case 16 : +emit (ctx, sll,lo (dst),lo (src),16 ); +emit (ctx, sra,lo (dst),lo (dst),16 ); +break ; +case 32 : +emit (ctx, move,lo (dst),lo (src)); +break ; +default : +emit_mov_r (ctx,lo (dst),lo (src)); +emit_mov_r (ctx,hi (dst),hi (src)); +return ; + } +clobber_reg (ctx,lo (dst)); +emit (ctx, sra,hi (dst),lo (dst),31 ); +clobber_reg (ctx,hi (dst)); }/* Load delay slot, if ISA mandates it */
@@ -1501,7 +1533,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)/* Special mov32 for zext */ emit_mov_i (ctx,hi (dst),0 ); }else { -emit_mov_r32 (ctx, dst, src); +emit_mov_r32 (ctx, dst, src, off); }break ;/* dst = -dst */
@@ -1570,7 +1602,7 @@ int build_insn(const struct bpf_insn *insn, struct jit_context *ctx)break ;/* dst = src (64-bit) */ case BPF_ALU64 |BPF_MOV |BPF_X : -emit_mov_r64 (ctx, dst, src); +emit_mov_r64 (ctx, dst, src, off);break ;/* dst = -dst (64-bit) */ case BPF_ALU64 |BPF_NEG :
arch/mips/net/bpf_jit_comp64.c +38 -8
69 lines not on this page · raw message
Johan Almbladh
maintainerRe: [RFC bpf-next 2/2] bpf, mips: Add support for BPF_MOVSX in the JITs
Hi Nicholas,
The MOV/MOVSX structure looks good in general. Some remarks on the implementation of individual emittor functions.
Quoted 13 lines
arch/mips/net/bpf_jit_comp32.c +9
-2+/* Register move operation (32-bit), optionally with sign extension */ static void emit_mov_r32 (struct jit_context *ctx,const u8 dst[], -const u8 src[]) +const u8 src[],s16 off) { -emit_mov_r (ctx,lo (dst),lo (src)); +switch (off) { +case 8 : +emit (ctx, sll,lo (dst),lo (src),24 ); +emit (ctx, sra,lo (dst),lo (dst),24 ); +clobber_reg (ctx,lo (dst)); +break ; +case 16 :
arch/mips/net/bpf_jit_comp32.c static void emit_zext_ver(struct jit_context *ctx, const u8 dst[]) · line 213
Quoted 10 lines
arch/mips/net/bpf_jit_comp32.c +8
-0+emit (ctx, sll,lo (dst),lo (src),16 ); +emit (ctx, sra,lo (dst),lo (dst),16 ); +clobber_reg (ctx,lo (dst)); +break ; +default : +emit_mov_r (ctx,lo (dst),lo (src)); +break ; + }emit_zext_ver (ctx, dst); }
I would move the clobber marker to the end to reduce code duplication. That is the pattern used in other places in the jit. Also consider collapsing the cases off=8 and off=16 into one, with the shift argument computed as 32 - off. That would parameterize the shift logic instead of duplicating it.
The above applies to all four variants of the mov emittor.
Quoted 18 lines
arch/mips/net/bpf_jit_comp32.c +13
-3+/* Register move operation (64-bit), optionally with sign extension */ static void emit_mov_r64 (struct jit_context *ctx,const u8 dst[], -const u8 src[]) +const u8 src[],s16 off) { -emit_mov_r (ctx,lo (dst),lo (src)); -emit_mov_r (ctx,hi (dst),hi (src)); +switch (off) { +case 8 : +emit (ctx, sll,lo (dst),lo (src),24 ); +emit (ctx, sra,lo (dst),lo (dst),24 ); +break ; +case 16 : +emit (ctx, sll,lo (dst),lo (src),16 ); +emit (ctx, sra,lo (dst),lo (dst),16 ); +break ; +case 32 : +emit (ctx, move,lo (dst),lo (src));
arch/mips/net/bpf_jit_comp32.c static void emit_zext_ver(struct jit_context *ctx, const u8 dst[]) · line 239
Quoted 10 lines
arch/mips/net/bpf_jit_comp32.c +9
-0+break ; +default : +emit_mov_r (ctx,lo (dst),lo (src)); +emit_mov_r (ctx,hi (dst),hi (src)); +return ; + } +clobber_reg (ctx,lo (dst)); +emit (ctx, sra,hi (dst),lo (dst),31 ); +clobber_reg (ctx,hi (dst)); }
I have tried to avoid returns in the middle like this. If you combine cases 8 and 16 as suggested, you can put the sra sign extension into cases 8/16 and 32, and get rid of the return in the default case.
Quoted 10 lines
arch/mips/net/bpf_jit_comp64.c +7
-2-static void emit_mov_r64 (struct jit_context *ctx,u8 dst,u8 src) +/* Register move operation (64-bit), optionally with sign extension */ +static void emit_mov_r64 (struct jit_context *ctx,u8 dst,u8 src,s16 off) { -emit_mov_r (ctx, dst, src); +switch (off) { +case 8 : +emit (ctx, dsll32, dst, src,24 ); +emit (ctx, dsra32, dst, dst,24 ); +break ;
arch/mips/net/bpf_jit_comp64.c static void emit_zext_ver(struct jit_context *ctx, u8 dst) · line 161
Quoted 10 lines
arch/mips/net/bpf_jit_comp64.c +10
-0+case 16 : +emit (ctx, dsll32, dst, src,16 ); +emit (ctx, dsra32, dst, dst,16 ); +break ; +case 32 : +emit (ctx, sll, dst, src,0 ); +break ; +default : +emit_mov_r (ctx, dst, src); +return ;
This return is not needed. Replace with break. Consider using the emit_sext helper instead of raw sll to b better convey intent.
Thanks,
Johan
Re: [RFC bpf-next 2/2] bpf, mips: Add support for BPF_MOVSX in the JITs
Hi Johan,
Quoted 2 lines · Johan Almbladh
The MOV/MOVSX structure looks good in general. Some remarks on the implementation of individual emittor functions.
Thanks for the review. I incorporated the suggestions into v2: moved clobber bookkeeping to the common emitter tails, grouped the 8- and 16-bit cases using 32 - off, removed the middle returns, and used emit_sext() for the MIPS64 MOVSX32 case.
I'll send v2 shortly.
Thanks,
Nicholas