LoongArch: BPF: Fix the offset of stptr.w and stptr.d stores
Subsystems (4)
PATCH v2 1/1 LoongArch: BPF: Fix the offset of stptr.w and stptr.d stores
From: George Guo <[email protected]>
When testing bpf selftests "jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:
$ sudo ./test_progs -t jit_insns/ldst_offsets | grep FAIL | head -1 store_offsets:FAIL:stx size=0 base=r0 src=r1: byte at offset -16384
The JIT incorrectly treats the signed 14-bit immediate of stptr.w and stptr.d as a byte offset. For offsets outside the signed 12-bit range of st.w and st.d, it checks the byte offset with is_signed_imm14() and encodes it unchanged. However, the hardware scales the encoded immediate by 4 when calculating the memory address. For example, a store at offset -4096 writes at offset -16384 instead, outside the memory range checked by the verifier.
Pass the offset in units of 4 bytes when emitting stptr.w and stptr.d for both immediate and register stores. Use these instructions only for offsets that are multiples of 4 in the range [-32768, 32764], and keep the register-offset fallback for other offsets that do not fit the signed 12-bit immediate of st.w or st.d.
Before:
$ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1 Summary: 0/0 PASSED, 0 SKIPPED, 2/2 FAILED
After:
$ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1 Summary: 2/2 PASSED, 0 SKIPPED, 0/0 FAILED
---
v2:
- Rename is_ptr_off() to is_aligned_signed_imm16() and drop its comment (Tiezhu)
- Remove the now unused is_signed_imm14() (Tiezhu)
- Reword the commit message
v1: https://lore.kernel.org/all/[email protected]/
arch/loongarch/net/bpf_jit.c | 16 ++++++++-------- arch/loongarch/net/bpf_jit.h | 6 +++++- 2 files changed, 13 insertions(+), 9 deletions(-)
arch/loongarch/net/bpf_jit.c +8 -8
@@ -1400,8 +1400,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extmove_imm (ctx, t1, imm);if (is_signed_imm12 (off)) {emit_insn (ctx, stw, t1, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrw, t1, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrw, t1, dst, off >>2 ); }else {move_imm (ctx, t2, off);emit_insn (ctx, stxw, t1, dst, t2);
@@ -1411,8 +1411,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extmove_imm (ctx, t1, imm);if (is_signed_imm12 (off)) {emit_insn (ctx, std, t1, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrd, t1, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrd, t1, dst, off >>2 ); }else {move_imm (ctx, t2, off);emit_insn (ctx, stxd, t1, dst, t2);
@@ -1460,8 +1460,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extcase BPF_W :if (is_signed_imm12 (off)) {emit_insn (ctx, stw, src, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrw, src, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrw, src, dst, off >>2 ); }else {move_imm (ctx, t1, off);emit_insn (ctx, stxw, src, dst, t1);
@@ -1470,8 +1470,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extcase BPF_DW :if (is_signed_imm12 (off)) {emit_insn (ctx, std, src, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrd, src, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrd, src, dst, off >>2 ); }else {move_imm (ctx, t1, off);emit_insn (ctx, stxd, src, dst, t1);
arch/loongarch/net/bpf_jit.h +5 -1
@@ -45,13 +45,17 @@ do { \ } while (0)#define is_signed_imm12 (val) signed_imm_check(val, 12) -#define is_signed_imm14 (val) signed_imm_check(val, 14)#define is_signed_imm16 (val) signed_imm_check(val, 16)#define is_signed_imm26 (val) signed_imm_check(val, 26)#define is_signed_imm32 (val) signed_imm_check(val, 32)#define is_signed_imm52 (val) signed_imm_check(val, 52)#define is_unsigned_imm12 (val) unsigned_imm_check(val, 12) +static inline bool is_aligned_signed_imm16 (int off) +{ +return !(off &3 ) &&is_signed_imm16 (off); +} +static inline int bpf2la_offset (int bpf_insn,int off,const struct jit_ctx *ctx) {/* BPF JMP offset is relative to the next instruction */
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
Signature · 1 line
2.53.0
Quoted 4 lines · George Guo
From: George Guo <[email protected]>
When testing bpf selftests "
jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:
...
Quoted 3 lines
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by George Guo <[email protected]>
Thanks,
Tiezhu
Huacai Chen
maintainerQuoted 98 lines · George Guo
From: George Guo <[email protected]>
When testing bpf selftests "
jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:$ sudo ./test_progs -t jit_insns/ldst_offsets | grep FAIL | head -1 store_offsets:FAIL:stx size=0 base=r0 src=r1: byte at offset -16384The JIT incorrectly treats the signed 14-bit immediate of stptr.w and stptr.d as a byte offset. For offsets outside the signed 12-bit range of st.w and st.d, it checks the byte offset with
is_signed_imm14()and encodes it unchanged. However, the hardware scales the encoded immediate by 4 when calculating the memory address. For example, a store at offset -4096 writes at offset -16384 instead, outside the memory range checked by the verifier.Pass the offset in units of 4 bytes when emitting stptr.w and stptr.d for both immediate and register stores. Use these instructions only for offsets that are multiples of 4 in the range [-32768, 32764], and keep the register-offset fallback for other offsets that do not fit the signed 12-bit immediate of st.w or st.d.
Before:
$ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1 Summary: 0/0 PASSED, 0 SKIPPED, 2/2 FAILEDAfter:
$ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1 Summary: 2/2 PASSED, 0 SKIPPED, 0/0 FAILEDFixes 5dc615520c4d ("LoongArch: Add BPF JIT support")Signed-off-by George Guo <[email protected]>---
v2:
- Renameis_ptr_off()tois_aligned_signed_imm16()and drop its comment (Tiezhu)
- Remove the now unusedis_signed_imm14()(Tiezhu)
- Reword the commit message
v1: https://lore.kernel.org/all/[email protected]/arch/loongarch/net/bpf_jit.c | 16 ++++++++-------- arch/loongarch/net/bpf_jit.h | 6 +++++- 2 files changed, 13 insertions(+), 9 deletions(-)arch/loongarch/net/bpf_jit.c +8
-8@@ -1400,8 +1400,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extmove_imm (ctx, t1, imm);if (is_signed_imm12 (off)) {emit_insn (ctx, stw, t1, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrw, t1, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrw, t1, dst, off >>2 ); }else {move_imm (ctx, t2, off);emit_insn (ctx, stxw, t1, dst, t2);@@ -1411,8 +1411,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extmove_imm (ctx, t1, imm);if (is_signed_imm12 (off)) {emit_insn (ctx, std, t1, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrd, t1, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrd, t1, dst, off >>2 ); }else {move_imm (ctx, t2, off);emit_insn (ctx, stxd, t1, dst, t2);@@ -1460,8 +1460,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extcase BPF_W :if (is_signed_imm12 (off)) {emit_insn (ctx, stw, src, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrw, src, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrw, src, dst, off >>2 ); }else {move_imm (ctx, t1, off);emit_insn (ctx, stxw, src, dst, t1);@@ -1470,8 +1470,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool extcase BPF_DW :if (is_signed_imm12 (off)) {emit_insn (ctx, std, src, dst, off); - }else if (is_signed_imm14 (off)) { -emit_insn (ctx, stptrd, src, dst, off); + }else if (is_aligned_signed_imm16 (off)) { +emit_insn (ctx, stptrd, src, dst, off >>2 ); }else {move_imm (ctx, t1, off);emit_insn (ctx, stxd, src, dst, t1);arch/loongarch/net/bpf_jit.h +0
-0
arch/loongarch/net/bpf_jit.h do { \ · line 48
Quoted 4 lines
arch/loongarch/net/bpf_jit.h +0
-1@@ -45,13 +45,17 @@ do { \ } while (0)#define is_signed_imm12 (val) signed_imm_check(val, 12) -#define is_signed_imm14 (val) signed_imm_check(val, 14)
Is there instructions really take a 14bit imm (not something like ldptr.d/stptr.d)?
If yes, I suggest keeping is_signed_imm14() because maybe we need it in future.
Huacai
Quoted 19 lines
arch/loongarch/net/bpf_jit.h +5
-0#define is_signed_imm16 (val) signed_imm_check(val, 16)#define is_signed_imm26 (val) signed_imm_check(val, 26)#define is_signed_imm32 (val) signed_imm_check(val, 32)#define is_signed_imm52 (val) signed_imm_check(val, 52)#define is_unsigned_imm12 (val) unsigned_imm_check(val, 12) +static inline bool is_aligned_signed_imm16 (int off) +{ +return !(off &3 ) &&is_signed_imm16 (off); +} +static inline int bpf2la_offset (int bpf_insn,int off,const struct jit_ctx *ctx) {/* BPF JMP offset is relative to the next instruction */ base-commit:
aebb7274476403040d582c61478df8f4a7bf9cd3--
2.53.0
Tiezhu Yang
maintainerQuoted 6 lines · Huacai Chen
Quoted 5 lines · George Guo
From: George Guo <[email protected]>
When testing bpf selftests "
jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:
...
Quoted 5 lines
Quoted 2 lines
#define
is_signed_imm12(val)signed_imm_check(val, 12)
-#defineis_signed_imm14(val)signed_imm_check(val, 14)Is there instructions really take a 14bit imm (not something like ldptr.d/stptr.d)?
If yes, I suggest keepingis_signed_imm14()because maybe we need it in future.
According to the LoongArch Reference Manual, there are 8 instructions in total that take a 14-bit signed immediate (si14):
ldptr.{w,d}, stptr.{w,d}, ll.{w,d}, and sc.{w,d}.
All of them implicitly shift si14 left by 2 bits when calculating the virtual address (vaddr = GR[rj] + SignExtend({si14, 2'b0}, GRLEN)), so none of them take an unscaled 14-bit immediate directly.
In the JIT, byte offsets for all these instructions should be validated with is_aligned_signed_imm16() and encoded as (off >> 2). Removing the unused is_signed_imm14() macro also helps prevent similar misuse in the future.
So there is no need to keep is_signed_imm14(), and v2 should be good to go.
Thanks,
Tiezhu
Huacai Chen
maintainerQuoted 25 lines · Tiezhu Yang
Quoted 6 lines · Huacai Chen
Quoted 5 lines · George Guo
From: George Guo <[email protected]>
When testing bpf selftests "
jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:...
Quoted 5 lines
Quoted 2 lines
#define
is_signed_imm12(val)signed_imm_check(val, 12)
-#defineis_signed_imm14(val)signed_imm_check(val, 14)Is there instructions really take a 14bit imm (not something like ldptr.d/stptr.d)?
If yes, I suggest keepingis_signed_imm14()because maybe we need it in future.According to the LoongArch Reference Manual, there are 8 instructions in total that take a 14-bit signed immediate (si14):
ldptr.{w,d}, stptr.{w,d}, ll.{w,d}, and sc.{w,d}.All of them implicitly shift si14 left by 2 bits when calculating the virtual address (vaddr = GR[rj] +
SignExtend({si14, 2'b0}, GRLEN)), so none of them take an unscaled 14-bit immediate directly.
Quoted 7 lines
In the JIT, byte offsets for all these instructions should be validated with
is_aligned_signed_imm16()and encoded as (off >> 2). Removing the unusedis_signed_imm14()macro also helps prevent similar misuse in the future.So there is no need to keep
is_signed_imm14(), and v2 should be good to go.
OK, make sense, queued.
Huacai
Quoted 4 lines
Thanks,
Tiezhu