Skip to contentSkip to conversation

LoongArch: BPF: Fix the offset of stptr.w and stptr.d stores

PATCH In review George Guo 3 participants 5 messages Nov 18 – Nov 19

Subsystems (4)
lore
Share & tools
lore

George Guo

author
#0 lore raw

PATCH v2 1/1 LoongArch: BPF: Fix the offset of stptr.w and stptr.d stores

From: George Guo <[email protected]>

When testing bpf selftests "jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:

  $ sudo ./test_progs -t jit_insns/ldst_offsets | grep FAIL | head -1
  store_offsets:FAIL:stx size=0 base=r0 src=r1: byte at offset -16384

The JIT incorrectly treats the signed 14-bit immediate of stptr.w and stptr.d as a byte offset. For offsets outside the signed 12-bit range of st.w and st.d, it checks the byte offset with is_signed_imm14() and encodes it unchanged. However, the hardware scales the encoded immediate by 4 when calculating the memory address. For example, a store at offset -4096 writes at offset -16384 instead, outside the memory range checked by the verifier.

Pass the offset in units of 4 bytes when emitting stptr.w and stptr.d for both immediate and register stores. Use these instructions only for offsets that are multiples of 4 in the range [-32768, 32764], and keep the register-offset fallback for other offsets that do not fit the signed 12-bit immediate of st.w or st.d.

Before:

  $ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1
  Summary: 0/0 PASSED, 0 SKIPPED, 2/2 FAILED

After:

  $ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1
  Summary: 2/2 PASSED, 0 SKIPPED, 0/0 FAILED
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")

---
v2:
- Rename is_ptr_off() to is_aligned_signed_imm16() and drop its comment (Tiezhu)
- Remove the now unused is_signed_imm14() (Tiezhu)
- Reword the commit message
v1: https://lore.kernel.org/all/[email protected]/

 arch/loongarch/net/bpf_jit.c | 16 ++++++++--------
 arch/loongarch/net/bpf_jit.h |  6 +++++-
 2 files changed, 13 insertions(+), 9 deletions(-)
arch/loongarch/net/bpf_jit.c +8 -8
index 15af1abb5cb6..906f34f420f1 100644
@@ -1400,8 +1400,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 			move_imm(ctx, t1, imm);
 			if (is_signed_imm12(off)) {
 				emit_insn(ctx, stw, t1, dst, off);
-			} else if (is_signed_imm14(off)) {
-				emit_insn(ctx, stptrw, t1, dst, off);
+			} else if (is_aligned_signed_imm16(off)) {
+				emit_insn(ctx, stptrw, t1, dst, off >> 2);
 			} else {
 				move_imm(ctx, t2, off);
 				emit_insn(ctx, stxw, t1, dst, t2);
@@ -1411,8 +1411,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 			move_imm(ctx, t1, imm);
 			if (is_signed_imm12(off)) {
 				emit_insn(ctx, std, t1, dst, off);
-			} else if (is_signed_imm14(off)) {
-				emit_insn(ctx, stptrd, t1, dst, off);
+			} else if (is_aligned_signed_imm16(off)) {
+				emit_insn(ctx, stptrd, t1, dst, off >> 2);
 			} else {
 				move_imm(ctx, t2, off);
 				emit_insn(ctx, stxd, t1, dst, t2);
@@ -1460,8 +1460,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 		case BPF_W:
 			if (is_signed_imm12(off)) {
 				emit_insn(ctx, stw, src, dst, off);
-			} else if (is_signed_imm14(off)) {
-				emit_insn(ctx, stptrw, src, dst, off);
+			} else if (is_aligned_signed_imm16(off)) {
+				emit_insn(ctx, stptrw, src, dst, off >> 2);
 			} else {
 				move_imm(ctx, t1, off);
 				emit_insn(ctx, stxw, src, dst, t1);
@@ -1470,8 +1470,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
 		case BPF_DW:
 			if (is_signed_imm12(off)) {
 				emit_insn(ctx, std, src, dst, off);
-			} else if (is_signed_imm14(off)) {
-				emit_insn(ctx, stptrd, src, dst, off);
+			} else if (is_aligned_signed_imm16(off)) {
+				emit_insn(ctx, stptrd, src, dst, off >> 2);
 			} else {
 				move_imm(ctx, t1, off);
 				emit_insn(ctx, stxd, src, dst, t1);
arch/loongarch/net/bpf_jit.h +5 -1
index 2bd0dc5b911c..777c76112d95 100644
@@ -45,13 +45,17 @@ do {										\
 } while (0)
 
 #define is_signed_imm12(val)	signed_imm_check(val, 12)
-#define is_signed_imm14(val)	signed_imm_check(val, 14)
 #define is_signed_imm16(val)	signed_imm_check(val, 16)
 #define is_signed_imm26(val)	signed_imm_check(val, 26)
 #define is_signed_imm32(val)	signed_imm_check(val, 32)
 #define is_signed_imm52(val)	signed_imm_check(val, 52)
 #define is_unsigned_imm12(val)	unsigned_imm_check(val, 12)
 
+static inline bool is_aligned_signed_imm16(int off)
+{
+	return !(off & 3) && is_signed_imm16(off);
+}
+
 static inline int bpf2la_offset(int bpf_insn, int off, const struct jit_ctx *ctx)
 {
 	/* BPF JMP offset is relative to the next instruction */

base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3

Signature · 1 line
2.53.0
maintainer Acked-by Tested-by on v2 1/1
#1 lore raw
Quoted 4 lines · George Guo

From: George Guo <[email protected]>

When testing bpf selftests "jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:

...

Quoted 3 lines
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")

Thanks,
Tiezhu

maintainer
#2 lore raw
In reply to George Guo
Quoted 98 lines · George Guo

From: George Guo <[email protected]>

When testing bpf selftests "jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:

  $ sudo ./test_progs -t jit_insns/ldst_offsets | grep FAIL | head -1
  store_offsets:FAIL:stx size=0 base=r0 src=r1: byte at offset -16384

The JIT incorrectly treats the signed 14-bit immediate of stptr.w and stptr.d as a byte offset. For offsets outside the signed 12-bit range of st.w and st.d, it checks the byte offset with is_signed_imm14() and encodes it unchanged. However, the hardware scales the encoded immediate by 4 when calculating the memory address. For example, a store at offset -4096 writes at offset -16384 instead, outside the memory range checked by the verifier.

Pass the offset in units of 4 bytes when emitting stptr.w and stptr.d for both immediate and register stores. Use these instructions only for offsets that are multiples of 4 in the range [-32768, 32764], and keep the register-offset fallback for other offsets that do not fit the signed 12-bit immediate of st.w or st.d.

Before:

  $ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1
  Summary: 0/0 PASSED, 0 SKIPPED, 2/2 FAILED

After:

  $ sudo ./test_progs -t jit_insns/ldst_offsets | tail -1
  Summary: 2/2 PASSED, 0 SKIPPED, 0/0 FAILED
Fixes 5dc615520c4d ("LoongArch: Add BPF JIT support")

---
v2:
- Rename is_ptr_off() to is_aligned_signed_imm16() and drop its comment (Tiezhu)
- Remove the now unused is_signed_imm14() (Tiezhu)
- Reword the commit message
v1: https://lore.kernel.org/all/[email protected]/

 arch/loongarch/net/bpf_jit.c | 16 ++++++++--------
 arch/loongarch/net/bpf_jit.h |  6 +++++-
 2 files changed, 13 insertions(+), 9 deletions(-)
arch/loongarch/net/bpf_jit.c +8 -8
index 15af1abb5cb6..906f34f420f1 100644
@@ -1400,8 +1400,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
                        move_imm(ctx, t1, imm);
                        if (is_signed_imm12(off)) {
                                emit_insn(ctx, stw, t1, dst, off);
-                       } else if (is_signed_imm14(off)) {
-                               emit_insn(ctx, stptrw, t1, dst, off);
+                       } else if (is_aligned_signed_imm16(off)) {
+                               emit_insn(ctx, stptrw, t1, dst, off >> 2);
                        } else {
                                move_imm(ctx, t2, off);
                                emit_insn(ctx, stxw, t1, dst, t2);
@@ -1411,8 +1411,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
                        move_imm(ctx, t1, imm);
                        if (is_signed_imm12(off)) {
                                emit_insn(ctx, std, t1, dst, off);
-                       } else if (is_signed_imm14(off)) {
-                               emit_insn(ctx, stptrd, t1, dst, off);
+                       } else if (is_aligned_signed_imm16(off)) {
+                               emit_insn(ctx, stptrd, t1, dst, off >> 2);
                        } else {
                                move_imm(ctx, t2, off);
                                emit_insn(ctx, stxd, t1, dst, t2);
@@ -1460,8 +1460,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
                case BPF_W:
                        if (is_signed_imm12(off)) {
                                emit_insn(ctx, stw, src, dst, off);
-                       } else if (is_signed_imm14(off)) {
-                               emit_insn(ctx, stptrw, src, dst, off);
+                       } else if (is_aligned_signed_imm16(off)) {
+                               emit_insn(ctx, stptrw, src, dst, off >> 2);
                        } else {
                                move_imm(ctx, t1, off);
                                emit_insn(ctx, stxw, src, dst, t1);
@@ -1470,8 +1470,8 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx, bool ext
                case BPF_DW:
                        if (is_signed_imm12(off)) {
                                emit_insn(ctx, std, src, dst, off);
-                       } else if (is_signed_imm14(off)) {
-                               emit_insn(ctx, stptrd, src, dst, off);
+                       } else if (is_aligned_signed_imm16(off)) {
+                               emit_insn(ctx, stptrd, src, dst, off >> 2);
                        } else {
                                move_imm(ctx, t1, off);
                                emit_insn(ctx, stxd, src, dst, t1);
arch/loongarch/net/bpf_jit.h +0 -0
index 2bd0dc5b911c..777c76112d95 100644

arch/loongarch/net/bpf_jit.h do { \ · line 48

Quoted 4 lines
arch/loongarch/net/bpf_jit.h +0 -1
@@ -45,13 +45,17 @@ do {                                                                                \
 } while (0)
 
 #define is_signed_imm12(val)   signed_imm_check(val, 12)
-#define is_signed_imm14(val)   signed_imm_check(val, 14)

Is there instructions really take a 14bit imm (not something like ldptr.d/stptr.d)?
If yes, I suggest keeping is_signed_imm14() because maybe we need it in future.

Huacai

Quoted 19 lines
arch/loongarch/net/bpf_jit.h +5 -0
 #define is_signed_imm16(val)   signed_imm_check(val, 16)
 #define is_signed_imm26(val)   signed_imm_check(val, 26)
 #define is_signed_imm32(val)   signed_imm_check(val, 32)
 #define is_signed_imm52(val)   signed_imm_check(val, 52)
 #define is_unsigned_imm12(val) unsigned_imm_check(val, 12)
 
+static inline bool is_aligned_signed_imm16(int off)
+{
+       return !(off & 3) && is_signed_imm16(off);
+}
+
 static inline int bpf2la_offset(int bpf_insn, int off, const struct jit_ctx *ctx)
 {
        /* BPF JMP offset is relative to the next instruction */

base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3 --
2.53.0

maintainer
#3 lore raw
Quoted 6 lines · Huacai Chen
Quoted 5 lines · George Guo

From: George Guo <[email protected]>

When testing bpf selftests "jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:

...

Quoted 5 lines
Quoted 2 lines

#define is_signed_imm12(val) signed_imm_check(val, 12)
-#define is_signed_imm14(val) signed_imm_check(val, 14)

Is there instructions really take a 14bit imm (not something like ldptr.d/stptr.d)?
If yes, I suggest keeping is_signed_imm14() because maybe we need it in future.

According to the LoongArch Reference Manual, there are 8 instructions in total that take a 14-bit signed immediate (si14):
ldptr.{w,d}, stptr.{w,d}, ll.{w,d}, and sc.{w,d}.

All of them implicitly shift si14 left by 2 bits when calculating the virtual address (vaddr = GR[rj] + SignExtend({si14, 2'b0}, GRLEN)), so none of them take an unscaled 14-bit immediate directly.

In the JIT, byte offsets for all these instructions should be validated with is_aligned_signed_imm16() and encoded as (off >> 2). Removing the unused is_signed_imm14() macro also helps prevent similar misuse in the future.

So there is no need to keep is_signed_imm14(), and v2 should be good to go.

Thanks,
Tiezhu

maintainer
#4 lore raw
Quoted 25 lines · Tiezhu Yang
Quoted 6 lines · Huacai Chen
Quoted 5 lines · George Guo

From: George Guo <[email protected]>

When testing bpf selftests "jit_insns" and "jit_insns_blinded", the "ldst_offsets" subtest fails for word and doubleword stores:

...

Quoted 5 lines
Quoted 2 lines

#define is_signed_imm12(val) signed_imm_check(val, 12)
-#define is_signed_imm14(val) signed_imm_check(val, 14)

Is there instructions really take a 14bit imm (not something like ldptr.d/stptr.d)?
If yes, I suggest keeping is_signed_imm14() because maybe we need it in future.

According to the LoongArch Reference Manual, there are 8 instructions in total that take a 14-bit signed immediate (si14):
ldptr.{w,d}, stptr.{w,d}, ll.{w,d}, and sc.{w,d}.

All of them implicitly shift si14 left by 2 bits when calculating the virtual address (vaddr = GR[rj] + SignExtend({si14, 2'b0}, GRLEN)), so none of them take an unscaled 14-bit immediate directly.

Quoted 7 lines

In the JIT, byte offsets for all these instructions should be validated with is_aligned_signed_imm16() and encoded as (off >> 2). Removing the unused is_signed_imm14() macro also helps prevent similar misuse in the future.

So there is no need to keep is_signed_imm14(), and v2 should be good to go.

OK, make sense, queued.

Huacai

Quoted 4 lines

Thanks,
Tiezhu