Skip to contentSkip to conversation

bpf: Support new insns from cpu v4

PATCH Superseded Yonghong Song 7 participants 51 messages Jul 13, 2023 – Jul 24, 2023

Subsystems (9)
Documentation x86 Kselftest BPF +5 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover bpf: Support new insns from cpu v4

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:
. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

This patch set added kernel support for insns proposed in [1] except BPF_ST which already has full kernel support. Beside the above proposed insns, LLVM will generate BPF_ST insn as well under -mcpu=v4 ([2]).

The patchset implements interpreter and jit support for these new insns as well as necessary verifier support.

To test this patch set, you need to have latest llvm from 'main' branch of llvm-project repo and apply [2] on top of it.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://reviews.llvm.org/D144829

Changelogs:
RFCv1 -> v2:
. add more verifier supports for signed extend load and mov insns. . rename some insn names to be more consistent with intel practice. . add cpuv4 test runner for test progs.
. add more unit and C tests.
. add documentation.

Yonghong Song (15):
bpf: Support new sign-extension load insns
bpf: Fix sign-extension ctx member accesses
bpf: Support new sign-extension mov insns
bpf: Support new unconditional bswap instruction
bpf: Support new signed div/mod instructions.
bpf: Fix jit blinding with new sdiv/smov insns
bpf: Support new 32bit offset jmp instruction selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing selftests/bpf: Add unit tests for new sign-extension load insns selftests/bpf: Add unit tests for new sign-extension mov insns selftests/bpf: Add unit tests for new bswap insns selftests/bpf: Add unit tests for new sdiv/smod insns selftests/bpf: Add unit tests for new gotol insn selftests/bpf: Test ldsx with more complex cases docs/bpf: Add documentation for new instructions

 Documentation/bpf/bpf_design_QA.rst           |   5 -
 .../bpf/standardization/instruction-set.rst   | 100 ++-
 arch/x86/net/bpf_jit_comp.c                   | 131 ++-
 include/linux/filter.h                        |  14 +-
 include/uapi/linux/bpf.h                      |   1 +
 kernel/bpf/cgroup.c                           |  14 +-
 kernel/bpf/core.c                             | 174 +++-
 kernel/bpf/verifier.c                         | 315 ++++++--
 tools/include/uapi/linux/bpf.h                |   1 +
 tools/testing/selftests/bpf/.gitignore        |   2 +
 tools/testing/selftests/bpf/Makefile          |  18 +-
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |   9 +-
 .../selftests/bpf/prog_tests/test_ldsx_insn.c |  88 ++
 .../selftests/bpf/prog_tests/verifier.c       |  10 +
 .../selftests/bpf/progs/test_ldsx_insn.c      |  75 ++
 .../selftests/bpf/progs/verifier_bswap.c      |  45 ++
 .../selftests/bpf/progs/verifier_gotol.c      |  30 +
 .../selftests/bpf/progs/verifier_ldsx.c       | 115 +++
 .../selftests/bpf/progs/verifier_movsx.c      | 177 ++++
 .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
 20 files changed, 1929 insertions(+), 158 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_ldsx_insn.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_ldsx_insn.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bswap.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotol.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_ldsx.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_movsx.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH v2 1/15 bpf: Support new sign-extension load insns

Add interpreter/jit support for new sign-extension load insns which adds a new mode (BPF_MEMSX).
Also add verifier support to recognize these insns and to do proper verification with new insns. In verifier, besides to deduce proper bounds for the dst_reg, probed memory access is handled by remembering insn mode in insn->imm field so later on proper jit insns can be emitted.

---
 arch/x86/net/bpf_jit_comp.c    |  32 ++++++++-
 include/uapi/linux/bpf.h       |   1 +
 kernel/bpf/core.c              |  13 ++++
 kernel/bpf/verifier.c          | 125 +++++++++++++++++++++++++++------
 tools/include/uapi/linux/bpf.h |   1 +
 5 files changed, 151 insertions(+), 21 deletions(-)
arch/x86/net/bpf_jit_comp.c +31 -1
index 438adb695daa..addeea95f397 100644
@@ -779,6 +779,29 @@ static void emit_ldx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 	*pprog = prog;
 }
 
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_ldsx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{
+	u8 *prog = *pprog;
+
+	switch (size) {
+	case BPF_B:
+		/* Emit 'movsx rax, byte ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBE);
+		break;
+	case BPF_H:
+		/* Emit 'movsx rax, word ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBF);
+		break;
+	case BPF_W:
+		/* Emit 'movsx rax, dword ptr [rax+0x14]' */
+		EMIT2(add_2mod(0x48, src_reg, dst_reg), 0x63);
+		break;
+	}
+	emit_insn_suffix(&prog, src_reg, dst_reg, off);
+	*pprog = prog;
+}
+
 /* STX: *(u8*)(dst_reg + off) = src_reg */
 static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 {
@@ -1370,6 +1393,9 @@ st:			if (is_imm8(insn->off))
 		case BPF_LDX | BPF_PROBE_MEM | BPF_W:
 		case BPF_LDX | BPF_MEM | BPF_DW:
 		case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
+		case BPF_LDX | BPF_MEMSX | BPF_B:
+		case BPF_LDX | BPF_MEMSX | BPF_H:
+		case BPF_LDX | BPF_MEMSX | BPF_W:
 			insn_off = insn->off;
 
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
@@ -1415,7 +1441,11 @@ st:			if (is_imm8(insn->off))
 				start_of_ldx = prog;
 				end_of_jmp[-1] = start_of_ldx - end_of_jmp;
 			}
-			emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			if ((BPF_MODE(insn->code) == BPF_PROBE_MEM && insn->imm == BPF_MEMSX) ||
+			    BPF_MODE(insn->code) == BPF_MEMSX)
+				emit_ldsx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			else
+				emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
 				struct exception_table_entry *ex;
 				u8 *_insn = image + proglen + (start_of_ldx - temp);
include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
kernel/bpf/core.c +13 -0
index dc85240a0134..8a1cc658789e 100644
@@ -1610,6 +1610,9 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(LDX, MEM, H),			\
 	INSN_3(LDX, MEM, W),			\
 	INSN_3(LDX, MEM, DW),			\
+	INSN_3(LDX, MEMSX, B),			\
+	INSN_3(LDX, MEMSX, H),			\
+	INSN_3(LDX, MEMSX, W),			\
 	/*   Immediate based. */		\
 	INSN_3(LD, IMM, DW)
 
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	LDST(DW, u64)
 #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\
+	LDX_MEMSX_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS
+
 #define ATOMIC_ALU_OP(BOP, KOP)						\
 		case BOP:						\
 			if (BPF_SIZE(insn->code) == BPF_W)		\
kernel/bpf/verifier.c +105 -20

239 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH bpf-next v2 01/15] bpf: Support new sign-extension load insns

kernel/bpf/core.c static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn) · line 1948

Quoted 6 lines · Yonghong Song
kernel/bpf/core.c +1 -0
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	LDST(DW, u64)
 #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\

LDSX ?

Quoted 8 lines
kernel/bpf/core.c +8 -0
+	LDX_MEMSX_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS

...

Quoted 13 lines
kernel/bpf/verifier.c +4 -1
@@ -17503,7 +17580,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		if (insn->code == (BPF_LDX | BPF_MEM | BPF_B) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_H) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_W) ||
-		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW)) {
+		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_B) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_H) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_W)) {
 			type = BPF_READ;
 		} else if (insn->code == (BPF_STX | BPF_MEM | BPF_B) ||
 			   insn->code == (BPF_STX | BPF_MEM | BPF_H) ||

kernel/bpf/verifier.c static int convert_ctx_accesses(struct bpf_verifier_env *env) · line 17649

Quoted 8 lines
kernel/bpf/verifier.c +5 -0
@@ -17562,6 +17642,11 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		 */
 		case PTR_TO_BTF_ID | MEM_ALLOC | PTR_UNTRUSTED:
 			if (type == BPF_READ) {
+				/* it is hard to differentiate that the
+				 * BPF_PROBE_MEM is for BPF_MEM or BPF_MEMSX,
+				 * let us use insn->imm to remember it.
+				 */
+				insn->imm = BPF_MODE(insn->code);

That's a fragile approach.
And the evidence is in this patch.
This part of interpreter:

        LDX_PROBE_MEM_##SIZEOP:                                         \
                bpf_probe_read_kernel(&DST, sizeof(SIZE),               \
                                      (const void *)(long) (SRC + insn->off));  \
                DST = *((SIZE *)&DST);                                  \

wasn't updated to handle sign extension.

How about
#define BPF_PROBE_MEMSX 0x40 /* same as BPF_IND */

and handle it in JITs and interpreter.
We need a selftest for BTF style access to signed fields to make sure both interpreter and JIT handling of BPF_PROBE_MEMSX is tested.

#3 lore raw

Re: [PATCH bpf-next v2 01/15] bpf: Support new sign-extension load insns

Quoted 9 lines · Alexei Starovoitov

kernel/bpf/core.c static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn) · line 1948

Quoted 6 lines · Yonghong Song
kernel/bpf/core.c +1 -0
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
  	LDST(DW, u64)
  #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\

LDSX ?

Ack.

Quoted 37 lines
Quoted 9 lines
kernel/bpf/core.c +8 -0
+	LDX_MEMSX_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS

...

Quoted 13 lines
kernel/bpf/verifier.c +4 -1
@@ -17503,7 +17580,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
  		if (insn->code == (BPF_LDX | BPF_MEM | BPF_B) ||
  		    insn->code == (BPF_LDX | BPF_MEM | BPF_H) ||
  		    insn->code == (BPF_LDX | BPF_MEM | BPF_W) ||
-		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW)) {
+		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_B) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_H) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_W)) {
  			type = BPF_READ;
  		} else if (insn->code == (BPF_STX | BPF_MEM | BPF_B) ||
  			   insn->code == (BPF_STX | BPF_MEM | BPF_H) ||

kernel/bpf/verifier.c static int convert_ctx_accesses(struct bpf_verifier_env *env) · line 17649

Quoted 8 lines
kernel/bpf/verifier.c +5 -0
@@ -17562,6 +17642,11 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
  		 */
  		case PTR_TO_BTF_ID | MEM_ALLOC | PTR_UNTRUSTED:
  			if (type == BPF_READ) {
+				/* it is hard to differentiate that the
+				 * BPF_PROBE_MEM is for BPF_MEM or BPF_MEMSX,
+				 * let us use insn->imm to remember it.
+				 */
+				insn->imm = BPF_MODE(insn->code);

That's a fragile approach.
And the evidence is in this patch.
This part of interpreter:

Quoted 6 lines
         LDX_PROBE_MEM_##SIZEOP:                                         \
                 bpf_probe_read_kernel(&DST, sizeof(SIZE),               \
                                       (const void *)(long) (SRC + insn->off));  \
                 DST = *((SIZE *)&DST);                                  \

wasn't updated to handle sign extension.

Thanks for catching this!

Quoted 5 lines

How about
#define BPF_PROBE_MEMSX 0x40 /* same as BPF_IND */

and handle it in JITs and interpreter.

Good idea. Will do.

Quoted 2 lines

We need a selftest for BTF style access to signed fields to make sure both interpreter and JIT handling of BPF_PROBE_MEMSX is tested.

Will do.

maintainer
#4 lore raw

Re: [PATCH bpf-next v2 01/15] bpf: Support new sign-extension load insns

In reply to Yonghong Song
Quoted 354 lines · Yonghong Song
Quoted 354 lines

Add interpreter/jit support for new sign-extension load insns which adds a new mode (BPF_MEMSX).
Also add verifier support to recognize these insns and to do proper verification with new insns. In verifier, besides to deduce proper bounds for the dst_reg, probed memory access is handled by remembering insn mode in insn->imm field so later on proper jit insns can be emitted.

---
 arch/x86/net/bpf_jit_comp.c    |  32 ++++++++-
 include/uapi/linux/bpf.h       |   1 +
 kernel/bpf/core.c              |  13 ++++
 kernel/bpf/verifier.c          | 125 +++++++++++++++++++++++++++------
 tools/include/uapi/linux/bpf.h |   1 +
 5 files changed, 151 insertions(+), 21 deletions(-)
arch/x86/net/bpf_jit_comp.c +31 -1
index 438adb695daa..addeea95f397 100644
@@ -779,6 +779,29 @@ static void emit_ldx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 	*pprog = prog;
 }
 
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_ldsx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{
+	u8 *prog = *pprog;
+
+	switch (size) {
+	case BPF_B:
+		/* Emit 'movsx rax, byte ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBE);
+		break;
+	case BPF_H:
+		/* Emit 'movsx rax, word ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBF);
+		break;
+	case BPF_W:
+		/* Emit 'movsx rax, dword ptr [rax+0x14]' */
+		EMIT2(add_2mod(0x48, src_reg, dst_reg), 0x63);
+		break;
+	}
+	emit_insn_suffix(&prog, src_reg, dst_reg, off);
+	*pprog = prog;
+}
+
 /* STX: *(u8*)(dst_reg + off) = src_reg */
 static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 {
@@ -1370,6 +1393,9 @@ st:			if (is_imm8(insn->off))
 		case BPF_LDX | BPF_PROBE_MEM | BPF_W:
 		case BPF_LDX | BPF_MEM | BPF_DW:
 		case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
+		case BPF_LDX | BPF_MEMSX | BPF_B:
+		case BPF_LDX | BPF_MEMSX | BPF_H:
+		case BPF_LDX | BPF_MEMSX | BPF_W:
 			insn_off = insn->off;
 
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
@@ -1415,7 +1441,11 @@ st:			if (is_imm8(insn->off))
 				start_of_ldx = prog;
 				end_of_jmp[-1] = start_of_ldx - end_of_jmp;
 			}
-			emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			if ((BPF_MODE(insn->code) == BPF_PROBE_MEM && insn->imm == BPF_MEMSX) ||
+			    BPF_MODE(insn->code) == BPF_MEMSX)
+				emit_ldsx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			else
+				emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
 				struct exception_table_entry *ex;
 				u8 *_insn = image + proglen + (start_of_ldx - temp);
include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
kernel/bpf/core.c +13 -0
index dc85240a0134..8a1cc658789e 100644
@@ -1610,6 +1610,9 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(LDX, MEM, H),			\
 	INSN_3(LDX, MEM, W),			\
 	INSN_3(LDX, MEM, DW),			\
+	INSN_3(LDX, MEMSX, B),			\
+	INSN_3(LDX, MEMSX, H),			\
+	INSN_3(LDX, MEMSX, W),			\
 	/*   Immediate based. */		\
 	INSN_3(LD, IMM, DW)
 
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	LDST(DW, u64)
 #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\
+	LDX_MEMSX_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS
+
 #define ATOMIC_ALU_OP(BOP, KOP)						\
 		case BOP:						\
 			if (BPF_SIZE(insn->code) == BPF_W)		\
kernel/bpf/verifier.c +96 -19
index 81a93eeac7a0..fbe4ca72d4c1 100644
@@ -5795,6 +5795,77 @@ static void coerce_reg_to_size(struct bpf_reg_state *reg, int size)
 	__reg_combine_64_into_32(reg);
 }
 
+static void set_sext64_default_val(struct bpf_reg_state *reg, int size)
+{
+	if (size == 1) {
+		reg->smin_value = reg->s32_min_value = S8_MIN;
+		reg->smax_value = reg->s32_max_value = S8_MAX;
+	} else if (size == 2) {
+		reg->smin_value = reg->s32_min_value = S16_MIN;
+		reg->smax_value = reg->s32_max_value = S16_MAX;
+	} else {
+		/* size == 4 */
+		reg->smin_value = reg->s32_min_value = S32_MIN;
+		reg->smax_value = reg->s32_max_value = S32_MAX;
+	}
+	reg->umin_value = reg->u32_min_value = 0;
+	reg->umax_value = U64_MAX;
+	reg->u32_max_value = U32_MAX;
+	reg->var_off = tnum_unknown;
+}
+
+static void coerce_reg_to_size_sx(struct bpf_reg_state *reg, int size)
+{
+	u64 top_smax_value, top_smin_value;
+	s64 init_s64_max, init_s64_min, s64_max, s64_min;
+	u64 num_bits = size * 8;
+
+	top_smax_value = ((u64)reg->smax_value >> num_bits) << num_bits;
+	top_smin_value = ((u64)reg->smin_value >> num_bits) << num_bits;
+
+	if (top_smax_value != top_smin_value)
+		goto out;
+
+	/* find the s64_min and s64_min after sign extension */
+	if (size == 1) {
+		init_s64_max = (s8)reg->smax_value;
+		init_s64_min = (s8)reg->smin_value;
+	} else if (size == 2) {
+		init_s64_max = (s16)reg->smax_value;
+		init_s64_min = (s16)reg->smin_value;
+	} else {
+		/* size == 4 */
+		init_s64_max = (s32)reg->smax_value;
+		init_s64_min = (s32)reg->smin_value;
+	}
+
+	s64_max = max(init_s64_max, init_s64_min);
+	s64_min = min(init_s64_max, init_s64_min);
+
+	if (s64_max >= 0 && s64_min >= 0) {
+		reg->smin_value = reg->s32_min_value = s64_min;
+		reg->smax_value = reg->s32_max_value = s64_max;
+		reg->umin_value = reg->u32_min_value = s64_min;
+		reg->umax_value = reg->u32_max_value = s64_max;
+		reg->var_off = tnum_range(s64_min, s64_max);
+		return;
+	}
+
+	if (s64_min < 0 && s64_max < 0) {
+		reg->smin_value = reg->s32_min_value = s64_min;
+		reg->smax_value = reg->s32_max_value = s64_max;
+		reg->umin_value = (u64)s64_max;
+		reg->umax_value = (u64)s64_min;
+		reg->u32_min_value = (u32)s64_max;
+		reg->u32_max_value = (u32)s64_min;
+		reg->var_off = tnum_range((u64)s64_max, (u64)s64_min);
+		return;
+	}
+
+out:
+	set_sext64_default_val(reg, size);
+}
+
 static bool bpf_map_is_rdonly(const struct bpf_map *map)
 {
 	/* A map is considered read-only if the following condition are true:
@@ -5815,7 +5886,8 @@ static bool bpf_map_is_rdonly(const struct bpf_map *map)
 	       !bpf_map_write_active(map);
 }
 
-static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val)
+static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val,
+			       bool is_ldsx)
 {
 	void *ptr;
 	u64 addr;
@@ -5828,13 +5900,13 @@ static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val)
 
 	switch (size) {
 	case sizeof(u8):
-		*val = (u64)*(u8 *)ptr;
+		*val = is_ldsx ? (s64)*(s8 *)ptr : (u64)*(u8 *)ptr;
 		break;
 	case sizeof(u16):
-		*val = (u64)*(u16 *)ptr;
+		*val = is_ldsx ? (s64)*(s16 *)ptr : (u64)*(u16 *)ptr;
 		break;
 	case sizeof(u32):
-		*val = (u64)*(u32 *)ptr;
+		*val = is_ldsx ? (s64)*(s32 *)ptr : (u64)*(u32 *)ptr;
 		break;
 	case sizeof(u64):
 		*val = *(u64 *)ptr;
@@ -6248,7 +6320,7 @@ static int check_stack_access_within_bounds(
  */
 static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regno,
 			    int off, int bpf_size, enum bpf_access_type t,
-			    int value_regno, bool strict_alignment_once)
+			    int value_regno, bool strict_alignment_once, bool is_ldsx)
 {
 	struct bpf_reg_state *regs = cur_regs(env);
 	struct bpf_reg_state *reg = regs + regno;
@@ -6309,7 +6381,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 				u64 val = 0;
 
 				err = bpf_map_direct_read(map, map_off, size,
-							  &val);
+							  &val, is_ldsx);
 				if (err)
 					return err;
 
@@ -6479,8 +6551,11 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 
 	if (!err && size < BPF_REG_SIZE && value_regno >= 0 && t == BPF_READ &&
 	    regs[value_regno].type == SCALAR_VALUE) {
-		/* b/h/w load zero-extends, mark upper bits as known 0 */
-		coerce_reg_to_size(&regs[value_regno], size);
+		if (!is_ldsx)
+			/* b/h/w load zero-extends, mark upper bits as known 0 */
+			coerce_reg_to_size(&regs[value_regno], size);
+		else
+			coerce_reg_to_size_sx(&regs[value_regno], size);
 	}
 	return err;
 }
@@ -6572,17 +6647,17 @@ static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_i
 	 * case to simulate the register fill.
 	 */
 	err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
-			       BPF_SIZE(insn->code), BPF_READ, -1, true);
+			       BPF_SIZE(insn->code), BPF_READ, -1, true, false);
 	if (!err && load_reg >= 0)
 		err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
 				       BPF_SIZE(insn->code), BPF_READ, load_reg,
-				       true);
+				       true, false);
 	if (err)
 		return err;
 
 	/* Check whether we can write into the same memory. */
 	err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
-			       BPF_SIZE(insn->code), BPF_WRITE, -1, true);
+			       BPF_SIZE(insn->code), BPF_WRITE, -1, true, false);
 	if (err)
 		return err;
 
@@ -6828,7 +6903,7 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
 				return zero_size_allowed ? 0 : -EACCES;
 
 			return check_mem_access(env, env->insn_idx, regno, offset, BPF_B,
-						atype, -1, false);
+						atype, -1, false, false);
 		}
 
 		fallthrough;
@@ -7200,7 +7275,7 @@ static int process_dynptr_func(struct bpf_verifier_env *env, int regno, int insn
 		/* we write BPF_DW bits (8 bytes) at a time */
 		for (i = 0; i < BPF_DYNPTR_SIZE; i += 8) {
 			err = check_mem_access(env, insn_idx, regno,
-					       i, BPF_DW, BPF_WRITE, -1, false);
+					       i, BPF_DW, BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 		}
@@ -7293,7 +7368,7 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id
 
 		for (i = 0; i < nr_slots * 8; i += BPF_REG_SIZE) {
 			err = check_mem_access(env, insn_idx, regno,
-					       i, BPF_DW, BPF_WRITE, -1, false);
+					       i, BPF_DW, BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 		}
@@ -9437,7 +9512,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 	 */
 	for (i = 0; i < meta.access_size; i++) {
 		err = check_mem_access(env, insn_idx, meta.regno, i, BPF_B,
-				       BPF_WRITE, -1, false);
+				       BPF_WRITE, -1, false, false);
 		if (err)
 			return err;
 	}
@@ -16315,7 +16390,8 @@ static int do_check(struct bpf_verifier_env *env)
 			 */
 			err = check_mem_access(env, env->insn_idx, insn->src_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_READ, insn->dst_reg, false);
+					       BPF_READ, insn->dst_reg, false,
+					       BPF_MODE(insn->code) == BPF_MEMSX);
 			if (err)
 				return err;
 
@@ -16352,7 +16428,7 @@ static int do_check(struct bpf_verifier_env *env)
 			/* check that memory (dst_reg + off) is writeable */
 			err = check_mem_access(env, env->insn_idx, insn->dst_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_WRITE, insn->src_reg, false);
+					       BPF_WRITE, insn->src_reg, false, false);
 			if (err)
 				return err;
 
@@ -16377,7 +16453,7 @@ static int do_check(struct bpf_verifier_env *env)
 			/* check that memory (dst_reg + off) is writeable */
 			err = check_mem_access(env, env->insn_idx, insn->dst_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_WRITE, -1, false);
+					       BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 
@@ -16805,7 +16881,8 @@ static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env)
 
 	for (i = 0; i < insn_cnt; i++, insn++) {
 		if (BPF_CLASS(insn->code) == BPF_LDX &&
-		    (BPF_MODE(insn->code) != BPF_MEM || insn->imm != 0)) {
+		    ((BPF_MODE(insn->code) != BPF_MEM && BPF_MODE(insn->code) != BPF_MEMSX) ||
+		    insn->imm != 0)) {
 			verbose(env, "BPF_LDX uses reserved fields\n");
 			return -EINVAL;
 		}
@@ -17503,7 +17580,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
Quoted 8 lines

kernel/bpf/verifier.c static int convert_ctx_accesses(struct bpf_verifier_env *env) · line 17586

Quoted 8 lines
kernel/bpf/verifier.c +4 -1
 		if (insn->code == (BPF_LDX | BPF_MEM | BPF_B) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_H) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_W) ||
-		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW)) {
+		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_B) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_H) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_W)) {

Later in this function there is a code that deals with is_narrower_load condition (line 17785 in my case).
This code handles the case when e.g. 1 byte is read from a 4 byte field. It does so by first converting such load to 4 byte load and than adding BPF_RSH and BPF_AND instructions.
It appears to me that this code should handle sign extension as well.

Quoted 27 lines
Quoted 27 lines
kernel/bpf/verifier.c +5 -0
 			type = BPF_READ;
 		} else if (insn->code == (BPF_STX | BPF_MEM | BPF_B) ||
 			   insn->code == (BPF_STX | BPF_MEM | BPF_H) ||
@@ -17562,6 +17642,11 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		 */
 		case PTR_TO_BTF_ID | MEM_ALLOC | PTR_UNTRUSTED:
 			if (type == BPF_READ) {
+				/* it is hard to differentiate that the
+				 * BPF_PROBE_MEM is for BPF_MEM or BPF_MEMSX,
+				 * let us use insn->imm to remember it.
+				 */
+				insn->imm = BPF_MODE(insn->code);
 				insn->code = BPF_LDX | BPF_PROBE_MEM |
 					BPF_SIZE((insn)->code);
 				env->prog->aux->num_exentries++;
tools/include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
maintainer
#5 lore raw

Re: [PATCH bpf-next v2 01/15] bpf: Support new sign-extension load insns

In reply to Yonghong Song
Quoted 176 lines · Yonghong Song

Add interpreter/jit support for new sign-extension load insns which adds a new mode (BPF_MEMSX).
Also add verifier support to recognize these insns and to do proper verification with new insns. In verifier, besides to deduce proper bounds for the dst_reg, probed memory access is handled by remembering insn mode in insn->imm field so later on proper jit insns can be emitted.

---
 arch/x86/net/bpf_jit_comp.c    |  32 ++++++++-
 include/uapi/linux/bpf.h       |   1 +
 kernel/bpf/core.c              |  13 ++++
 kernel/bpf/verifier.c          | 125 +++++++++++++++++++++++++++------
 tools/include/uapi/linux/bpf.h |   1 +
 5 files changed, 151 insertions(+), 21 deletions(-)
arch/x86/net/bpf_jit_comp.c +31 -1
index 438adb695daa..addeea95f397 100644
@@ -779,6 +779,29 @@ static void emit_ldx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 	*pprog = prog;
 }
 
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_ldsx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{
+	u8 *prog = *pprog;
+
+	switch (size) {
+	case BPF_B:
+		/* Emit 'movsx rax, byte ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBE);
+		break;
+	case BPF_H:
+		/* Emit 'movsx rax, word ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBF);
+		break;
+	case BPF_W:
+		/* Emit 'movsx rax, dword ptr [rax+0x14]' */
+		EMIT2(add_2mod(0x48, src_reg, dst_reg), 0x63);
+		break;
+	}
+	emit_insn_suffix(&prog, src_reg, dst_reg, off);
+	*pprog = prog;
+}
+
 /* STX: *(u8*)(dst_reg + off) = src_reg */
 static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 {
@@ -1370,6 +1393,9 @@ st:			if (is_imm8(insn->off))
 		case BPF_LDX | BPF_PROBE_MEM | BPF_W:
 		case BPF_LDX | BPF_MEM | BPF_DW:
 		case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
+		case BPF_LDX | BPF_MEMSX | BPF_B:
+		case BPF_LDX | BPF_MEMSX | BPF_H:
+		case BPF_LDX | BPF_MEMSX | BPF_W:
 			insn_off = insn->off;
 
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
@@ -1415,7 +1441,11 @@ st:			if (is_imm8(insn->off))
 				start_of_ldx = prog;
 				end_of_jmp[-1] = start_of_ldx - end_of_jmp;
 			}
-			emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			if ((BPF_MODE(insn->code) == BPF_PROBE_MEM && insn->imm == BPF_MEMSX) ||
+			    BPF_MODE(insn->code) == BPF_MEMSX)
+				emit_ldsx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			else
+				emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
 				struct exception_table_entry *ex;
 				u8 *_insn = image + proglen + (start_of_ldx - temp);
include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
kernel/bpf/core.c +13 -0
index dc85240a0134..8a1cc658789e 100644
@@ -1610,6 +1610,9 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(LDX, MEM, H),			\
 	INSN_3(LDX, MEM, W),			\
 	INSN_3(LDX, MEM, DW),			\
+	INSN_3(LDX, MEMSX, B),			\
+	INSN_3(LDX, MEMSX, H),			\
+	INSN_3(LDX, MEMSX, W),			\
 	/*   Immediate based. */		\
 	INSN_3(LD, IMM, DW)
 
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	LDST(DW, u64)
 #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\
+	LDX_MEMSX_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS
+
 #define ATOMIC_ALU_OP(BOP, KOP)						\
 		case BOP:						\
 			if (BPF_SIZE(insn->code) == BPF_W)		\
kernel/bpf/verifier.c +51 -0
index 81a93eeac7a0..fbe4ca72d4c1 100644
@@ -5795,6 +5795,77 @@ static void coerce_reg_to_size(struct bpf_reg_state *reg, int size)
 	__reg_combine_64_into_32(reg);
 }
 
+static void set_sext64_default_val(struct bpf_reg_state *reg, int size)
+{
+	if (size == 1) {
+		reg->smin_value = reg->s32_min_value = S8_MIN;
+		reg->smax_value = reg->s32_max_value = S8_MAX;
+	} else if (size == 2) {
+		reg->smin_value = reg->s32_min_value = S16_MIN;
+		reg->smax_value = reg->s32_max_value = S16_MAX;
+	} else {
+		/* size == 4 */
+		reg->smin_value = reg->s32_min_value = S32_MIN;
+		reg->smax_value = reg->s32_max_value = S32_MAX;
+	}
+	reg->umin_value = reg->u32_min_value = 0;
+	reg->umax_value = U64_MAX;
+	reg->u32_max_value = U32_MAX;
+	reg->var_off = tnum_unknown;
+}
+
+static void coerce_reg_to_size_sx(struct bpf_reg_state *reg, int size)
+{
+	u64 top_smax_value, top_smin_value;
+	s64 init_s64_max, init_s64_min, s64_max, s64_min;
+	u64 num_bits = size * 8;
+
+	top_smax_value = ((u64)reg->smax_value >> num_bits) << num_bits;
+	top_smin_value = ((u64)reg->smin_value >> num_bits) << num_bits;
+
+	if (top_smax_value != top_smin_value)
+		goto out;
+
+	/* find the s64_min and s64_min after sign extension */
+	if (size == 1) {
+		init_s64_max = (s8)reg->smax_value;
+		init_s64_min = (s8)reg->smin_value;
+	} else if (size == 2) {
+		init_s64_max = (s16)reg->smax_value;
+		init_s64_min = (s16)reg->smin_value;
+	} else {
+		/* size == 4 */
+		init_s64_max = (s32)reg->smax_value;
+		init_s64_min = (s32)reg->smin_value;
+	}
+
+	s64_max = max(init_s64_max, init_s64_min);
+	s64_min = min(init_s64_max, init_s64_min);
+
+	if (s64_max >= 0 && s64_min >= 0) {
+		reg->smin_value = reg->s32_min_value = s64_min;
+		reg->smax_value = reg->s32_max_value = s64_max;
+		reg->umin_value = reg->u32_min_value = s64_min;

kernel/bpf/verifier.c static void coerce_reg_to_size(struct bpf_reg_state *reg, int size) · line 5858

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+		reg->umax_value = reg->u32_max_value = s64_max;
+		reg->var_off = tnum_range(s64_min, s64_max);
+		return;
+	}
+
+	if (s64_min < 0 && s64_max < 0) {
+		reg->smin_value = reg->s32_min_value = s64_min;
+		reg->smax_value = reg->s32_max_value = s64_max;
+		reg->umin_value = (u64)s64_max;
+		reg->umax_value = (u64)s64_min;

I think the last two assignments are not correct for the following example:

{
	"testtesttest",
	.insns = {
		BPF_EMIT_CALL(BPF_FUNC_get_prandom_u32),
		BPF_JMP_IMM(BPF_JLT, BPF_REG_0, 0xff80, 2),
		BPF_JMP_IMM(BPF_JGT, BPF_REG_0, 0xffff, 1),
		{
			.code  = BPF_ALU64 | BPF_MOV | BPF_X,
			.dst_reg = BPF_REG_0,
			.src_reg = BPF_REG_0,
			.off   = 8,
			.imm   = 0,
		},
		BPF_EXIT_INSN(),
	},
	.result = ACCEPT,
	.retval = 0,
},

Here is execution log:

0: R1=ctx(off=0,imm=0) R10=fp0
0: (85) call bpf_get_prandom_u32#7 ; R0_w=Pscalar()

1: (a5) if r0 < 0xff80 goto pc+2   ; R0_w=Pscalar(umin=65408)
2: (25) if r0 > 0xffff goto pc+1   ; R0_w=Pscalar(umin=65408,umax=65535,var_off=(0xff80; 0x7f))
3: (bf) r0 = r0                    ; R0_w=Pscalar
                                      (smin=-128,smax=-1,
                                       umin=18'446'744'073'709'551'615,
                                       umax=18'446'744'073'709'551'488,
                                       var_off=(0xffffffffffffff80; 0x7f),
                                       u32_min=-1,u32_max=-128)

4: (95) exit

Note that umax < umin, which should not happen.
In this case the assignments in question are:

    reg->umin_value = (u64)s64_max; // == -1   == 0xffffffffffffffff
    reg->umax_value = (u64)s64_min; // == -128 == 0xffffffffffffff80
Quoted 203 lines
kernel/bpf/verifier.c +44 -20
+		reg->u32_min_value = (u32)s64_max;
+		reg->u32_max_value = (u32)s64_min;
+		reg->var_off = tnum_range((u64)s64_max, (u64)s64_min);
+		return;
+	}
+
+out:
+	set_sext64_default_val(reg, size);
+}
+
 static bool bpf_map_is_rdonly(const struct bpf_map *map)
 {
 	/* A map is considered read-only if the following condition are true:
@@ -5815,7 +5886,8 @@ static bool bpf_map_is_rdonly(const struct bpf_map *map)
 	       !bpf_map_write_active(map);
 }
 
-static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val)
+static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val,
+			       bool is_ldsx)
 {
 	void *ptr;
 	u64 addr;
@@ -5828,13 +5900,13 @@ static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val)
 
 	switch (size) {
 	case sizeof(u8):
-		*val = (u64)*(u8 *)ptr;
+		*val = is_ldsx ? (s64)*(s8 *)ptr : (u64)*(u8 *)ptr;
 		break;
 	case sizeof(u16):
-		*val = (u64)*(u16 *)ptr;
+		*val = is_ldsx ? (s64)*(s16 *)ptr : (u64)*(u16 *)ptr;
 		break;
 	case sizeof(u32):
-		*val = (u64)*(u32 *)ptr;
+		*val = is_ldsx ? (s64)*(s32 *)ptr : (u64)*(u32 *)ptr;
 		break;
 	case sizeof(u64):
 		*val = *(u64 *)ptr;
@@ -6248,7 +6320,7 @@ static int check_stack_access_within_bounds(
  */
 static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regno,
 			    int off, int bpf_size, enum bpf_access_type t,
-			    int value_regno, bool strict_alignment_once)
+			    int value_regno, bool strict_alignment_once, bool is_ldsx)
 {
 	struct bpf_reg_state *regs = cur_regs(env);
 	struct bpf_reg_state *reg = regs + regno;
@@ -6309,7 +6381,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 				u64 val = 0;
 
 				err = bpf_map_direct_read(map, map_off, size,
-							  &val);
+							  &val, is_ldsx);
 				if (err)
 					return err;
 
@@ -6479,8 +6551,11 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 
 	if (!err && size < BPF_REG_SIZE && value_regno >= 0 && t == BPF_READ &&
 	    regs[value_regno].type == SCALAR_VALUE) {
-		/* b/h/w load zero-extends, mark upper bits as known 0 */
-		coerce_reg_to_size(&regs[value_regno], size);
+		if (!is_ldsx)
+			/* b/h/w load zero-extends, mark upper bits as known 0 */
+			coerce_reg_to_size(&regs[value_regno], size);
+		else
+			coerce_reg_to_size_sx(&regs[value_regno], size);
 	}
 	return err;
 }
@@ -6572,17 +6647,17 @@ static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_i
 	 * case to simulate the register fill.
 	 */
 	err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
-			       BPF_SIZE(insn->code), BPF_READ, -1, true);
+			       BPF_SIZE(insn->code), BPF_READ, -1, true, false);
 	if (!err && load_reg >= 0)
 		err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
 				       BPF_SIZE(insn->code), BPF_READ, load_reg,
-				       true);
+				       true, false);
 	if (err)
 		return err;
 
 	/* Check whether we can write into the same memory. */
 	err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
-			       BPF_SIZE(insn->code), BPF_WRITE, -1, true);
+			       BPF_SIZE(insn->code), BPF_WRITE, -1, true, false);
 	if (err)
 		return err;
 
@@ -6828,7 +6903,7 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
 				return zero_size_allowed ? 0 : -EACCES;
 
 			return check_mem_access(env, env->insn_idx, regno, offset, BPF_B,
-						atype, -1, false);
+						atype, -1, false, false);
 		}
 
 		fallthrough;
@@ -7200,7 +7275,7 @@ static int process_dynptr_func(struct bpf_verifier_env *env, int regno, int insn
 		/* we write BPF_DW bits (8 bytes) at a time */
 		for (i = 0; i < BPF_DYNPTR_SIZE; i += 8) {
 			err = check_mem_access(env, insn_idx, regno,
-					       i, BPF_DW, BPF_WRITE, -1, false);
+					       i, BPF_DW, BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 		}
@@ -7293,7 +7368,7 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id
 
 		for (i = 0; i < nr_slots * 8; i += BPF_REG_SIZE) {
 			err = check_mem_access(env, insn_idx, regno,
-					       i, BPF_DW, BPF_WRITE, -1, false);
+					       i, BPF_DW, BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 		}
@@ -9437,7 +9512,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 	 */
 	for (i = 0; i < meta.access_size; i++) {
 		err = check_mem_access(env, insn_idx, meta.regno, i, BPF_B,
-				       BPF_WRITE, -1, false);
+				       BPF_WRITE, -1, false, false);
 		if (err)
 			return err;
 	}
@@ -16315,7 +16390,8 @@ static int do_check(struct bpf_verifier_env *env)
 			 */
 			err = check_mem_access(env, env->insn_idx, insn->src_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_READ, insn->dst_reg, false);
+					       BPF_READ, insn->dst_reg, false,
+					       BPF_MODE(insn->code) == BPF_MEMSX);
 			if (err)
 				return err;
 
@@ -16352,7 +16428,7 @@ static int do_check(struct bpf_verifier_env *env)
 			/* check that memory (dst_reg + off) is writeable */
 			err = check_mem_access(env, env->insn_idx, insn->dst_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_WRITE, insn->src_reg, false);
+					       BPF_WRITE, insn->src_reg, false, false);
 			if (err)
 				return err;
 
@@ -16377,7 +16453,7 @@ static int do_check(struct bpf_verifier_env *env)
 			/* check that memory (dst_reg + off) is writeable */
 			err = check_mem_access(env, env->insn_idx, insn->dst_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_WRITE, -1, false);
+					       BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 
@@ -16805,7 +16881,8 @@ static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env)
 
 	for (i = 0; i < insn_cnt; i++, insn++) {
 		if (BPF_CLASS(insn->code) == BPF_LDX &&
-		    (BPF_MODE(insn->code) != BPF_MEM || insn->imm != 0)) {
+		    ((BPF_MODE(insn->code) != BPF_MEM && BPF_MODE(insn->code) != BPF_MEMSX) ||
+		    insn->imm != 0)) {
 			verbose(env, "BPF_LDX uses reserved fields\n");
 			return -EINVAL;
 		}
@@ -17503,7 +17580,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		if (insn->code == (BPF_LDX | BPF_MEM | BPF_B) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_H) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_W) ||
-		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW)) {
+		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_B) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_H) ||
+		    insn->code == (BPF_LDX | BPF_MEMSX | BPF_W)) {
 			type = BPF_READ;
 		} else if (insn->code == (BPF_STX | BPF_MEM | BPF_B) ||
 			   insn->code == (BPF_STX | BPF_MEM | BPF_H) ||
@@ -17562,6 +17642,11 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		 */
 		case PTR_TO_BTF_ID | MEM_ALLOC | PTR_UNTRUSTED:
 			if (type == BPF_READ) {
+				/* it is hard to differentiate that the
+				 * BPF_PROBE_MEM is for BPF_MEM or BPF_MEMSX,
+				 * let us use insn->imm to remember it.
+				 */
+				insn->imm = BPF_MODE(insn->code);
 				insn->code = BPF_LDX | BPF_PROBE_MEM |
 					BPF_SIZE((insn)->code);
 				env->prog->aux->num_exentries++;
tools/include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
#6 lore raw

Re: [PATCH bpf-next v2 01/15] bpf: Support new sign-extension load insns

Quoted 221 lines · Eduard Zingerman
Quoted 176 lines · Yonghong Song

Add interpreter/jit support for new sign-extension load insns which adds a new mode (BPF_MEMSX).
Also add verifier support to recognize these insns and to do proper verification with new insns. In verifier, besides to deduce proper bounds for the dst_reg, probed memory access is handled by remembering insn mode in insn->imm field so later on proper jit insns can be emitted.

---
  arch/x86/net/bpf_jit_comp.c    |  32 ++++++++-
  include/uapi/linux/bpf.h       |   1 +
  kernel/bpf/core.c              |  13 ++++
  kernel/bpf/verifier.c          | 125 +++++++++++++++++++++++++++------
  tools/include/uapi/linux/bpf.h |   1 +
  5 files changed, 151 insertions(+), 21 deletions(-)
arch/x86/net/bpf_jit_comp.c +31 -1
index 438adb695daa..addeea95f397 100644
@@ -779,6 +779,29 @@ static void emit_ldx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
  	*pprog = prog;
  }
 
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_ldsx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{
+	u8 *prog = *pprog;
+
+	switch (size) {
+	case BPF_B:
+		/* Emit 'movsx rax, byte ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBE);
+		break;
+	case BPF_H:
+		/* Emit 'movsx rax, word ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBF);
+		break;
+	case BPF_W:
+		/* Emit 'movsx rax, dword ptr [rax+0x14]' */
+		EMIT2(add_2mod(0x48, src_reg, dst_reg), 0x63);
+		break;
+	}
+	emit_insn_suffix(&prog, src_reg, dst_reg, off);
+	*pprog = prog;
+}
+
  /* STX: *(u8*)(dst_reg + off) = src_reg */
  static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
  {
@@ -1370,6 +1393,9 @@ st:			if (is_imm8(insn->off))
  		case BPF_LDX | BPF_PROBE_MEM | BPF_W:
  		case BPF_LDX | BPF_MEM | BPF_DW:
  		case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
+		case BPF_LDX | BPF_MEMSX | BPF_B:
+		case BPF_LDX | BPF_MEMSX | BPF_H:
+		case BPF_LDX | BPF_MEMSX | BPF_W:
  			insn_off = insn->off;
 
  			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
@@ -1415,7 +1441,11 @@ st:			if (is_imm8(insn->off))
  				start_of_ldx = prog;
  				end_of_jmp[-1] = start_of_ldx - end_of_jmp;
  			}
-			emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			if ((BPF_MODE(insn->code) == BPF_PROBE_MEM && insn->imm == BPF_MEMSX) ||
+			    BPF_MODE(insn->code) == BPF_MEMSX)
+				emit_ldsx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			else
+				emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
  			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
  				struct exception_table_entry *ex;
  				u8 *_insn = image + proglen + (start_of_ldx - temp);
include/uapi/linux/bpf.h +1 -0
index 600d0caebbd8..c7196302d1eb 100644
@@ -19,6 +19,7 @@
 
  /* ld/ldx fields */
  #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMSX	0x80	/* load with sign extension */
  #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
  #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
kernel/bpf/core.c +13 -0
index dc85240a0134..8a1cc658789e 100644
@@ -1610,6 +1610,9 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
  	INSN_3(LDX, MEM, H),			\
  	INSN_3(LDX, MEM, W),			\
  	INSN_3(LDX, MEM, DW),			\
+	INSN_3(LDX, MEMSX, B),			\
+	INSN_3(LDX, MEMSX, H),			\
+	INSN_3(LDX, MEMSX, W),			\
  	/*   Immediate based. */		\
  	INSN_3(LD, IMM, DW)
 
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
  	LDST(DW, u64)
  #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\
+	LDX_MEMSX_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS
+
  #define ATOMIC_ALU_OP(BOP, KOP)						\
  		case BOP:						\
  			if (BPF_SIZE(insn->code) == BPF_W)		\
kernel/bpf/verifier.c +51 -0
index 81a93eeac7a0..fbe4ca72d4c1 100644
@@ -5795,6 +5795,77 @@ static void coerce_reg_to_size(struct bpf_reg_state *reg, int size)
  	__reg_combine_64_into_32(reg);
  }
 
+static void set_sext64_default_val(struct bpf_reg_state *reg, int size)
+{
+	if (size == 1) {
+		reg->smin_value = reg->s32_min_value = S8_MIN;
+		reg->smax_value = reg->s32_max_value = S8_MAX;
+	} else if (size == 2) {
+		reg->smin_value = reg->s32_min_value = S16_MIN;
+		reg->smax_value = reg->s32_max_value = S16_MAX;
+	} else {
+		/* size == 4 */
+		reg->smin_value = reg->s32_min_value = S32_MIN;
+		reg->smax_value = reg->s32_max_value = S32_MAX;
+	}
+	reg->umin_value = reg->u32_min_value = 0;
+	reg->umax_value = U64_MAX;
+	reg->u32_max_value = U32_MAX;
+	reg->var_off = tnum_unknown;
+}
+
+static void coerce_reg_to_size_sx(struct bpf_reg_state *reg, int size)
+{
+	u64 top_smax_value, top_smin_value;
+	s64 init_s64_max, init_s64_min, s64_max, s64_min;
+	u64 num_bits = size * 8;
+
+	top_smax_value = ((u64)reg->smax_value >> num_bits) << num_bits;
+	top_smin_value = ((u64)reg->smin_value >> num_bits) << num_bits;
+
+	if (top_smax_value != top_smin_value)
+		goto out;
+
+	/* find the s64_min and s64_min after sign extension */
+	if (size == 1) {
+		init_s64_max = (s8)reg->smax_value;
+		init_s64_min = (s8)reg->smin_value;
+	} else if (size == 2) {
+		init_s64_max = (s16)reg->smax_value;
+		init_s64_min = (s16)reg->smin_value;
+	} else {
+		/* size == 4 */
+		init_s64_max = (s32)reg->smax_value;
+		init_s64_min = (s32)reg->smin_value;
+	}
+
+	s64_max = max(init_s64_max, init_s64_min);
+	s64_min = min(init_s64_max, init_s64_min);
+
+	if (s64_max >= 0 && s64_min >= 0) {
+		reg->smin_value = reg->s32_min_value = s64_min;
+		reg->smax_value = reg->s32_max_value = s64_max;
+		reg->umin_value = reg->u32_min_value = s64_min;

kernel/bpf/verifier.c static void coerce_reg_to_size(struct bpf_reg_state *reg, int size) · line 5858

Quoted 10 lines
kernel/bpf/verifier.c +10 -0
+		reg->umax_value = reg->u32_max_value = s64_max;
+		reg->var_off = tnum_range(s64_min, s64_max);
+		return;
+	}
+
+	if (s64_min < 0 && s64_max < 0) {
+		reg->smin_value = reg->s32_min_value = s64_min;
+		reg->smax_value = reg->s32_max_value = s64_max;
+		reg->umin_value = (u64)s64_max;
+		reg->umax_value = (u64)s64_min;

I think the last two assignments are not correct for the following example:

{
	"testtesttest",
	.insns = {
		BPF_EMIT_CALL(BPF_FUNC_get_prandom_u32),
		BPF_JMP_IMM(BPF_JLT, BPF_REG_0, 0xff80, 2),
		BPF_JMP_IMM(BPF_JGT, BPF_REG_0, 0xffff, 1),
		{
			.code  = BPF_ALU64 | BPF_MOV | BPF_X,
			.dst_reg = BPF_REG_0,
			.src_reg = BPF_REG_0,
			.off   = 8,
			.imm   = 0,
		},
		BPF_EXIT_INSN(),
	},
	.result = ACCEPT,
	.retval = 0,
},

Here is execution log:

0: R1=ctx(off=0,imm=0) R10=fp0
0: (85) call bpf_get_prandom_u32#7 ; R0_w=Pscalar()

1: (a5) if r0 < 0xff80 goto pc+2   ; R0_w=Pscalar(umin=65408)
2: (25) if r0 > 0xffff goto pc+1   ; R0_w=Pscalar(umin=65408,umax=65535,var_off=(0xff80; 0x7f))
3: (bf) r0 = r0                    ; R0_w=Pscalar
                                       (smin=-128,smax=-1,
                                        umin=18'446'744'073'709'551'615,
                                        umax=18'446'744'073'709'551'488,
                                        var_off=(0xffffffffffffff80; 0x7f),
                                        u32_min=-1,u32_max=-128)
Quoted 7 lines

4: (95) exit

Note that umax < umin, which should not happen.
In this case the assignments in question are:

     reg->umin_value = (u64)s64_max; // == -1   == 0xffffffffffffffff
     reg->umax_value = (u64)s64_min; // == -128 == 0xffffffffffffff80

Thanks for pointing out. Yes, the assignment is incorrect and they are mismatched. Will fix the issue and add a test for this.

Quoted 13 lines
Quoted 12 lines
kernel/bpf/verifier.c +10 -0
+		reg->u32_min_value = (u32)s64_max;
+		reg->u32_max_value = (u32)s64_min;
+		reg->var_off = tnum_range((u64)s64_max, (u64)s64_min);
+		return;
+	}
+
+out:
+	set_sext64_default_val(reg, size);
+}
+

[...]

#7 lore raw

PATCH v2 2/15 bpf: Fix sign-extension ctx member accesses

In reply to Yonghong Song

In uapi bpf.h, there are two ctx structures which contain signed members. Without cpu v4, such signed members will be loaded with unsigned load and the compiler will generate proper left and right shifts to get the proper final value.

With sign-extension load, however, left/right shifts are gone, we need to ensure these signed members are properly handled, with signed loads or other means. The following are list of signed ctx members and how they are handled.

(1).

  struct bpf_sock {
     ...
     __s32 rx_queue_mapping;

}

The corresponding kernel fields are

  struct sock_common {
     ...
     unsigned short          skc_rx_queue_mapping;
     ...

}

Current ctx rewriter uses unsigned load for the kernel field which is correct and does not need further handling.

(2).

  struct bpf_sockopt {
     ...
     __s32   level;
     __s32   optname;
     __s32   optlen;
     __s32   retval;
  }
The level/optname/optlen are from struct bpf_sockopt_kern
  struct bpf_sockopt_kern {
     ...
     s32             level;
     s32             optname;
     s32             optlen;
     ...
  }
and the 'retval' is from struct bpf_cg_run_ctx
  struct bpf_cg_run_ctx {
     ...
     int retval;

}
Current the above four fields are loaded with unsigned load. Let us modify the read macro for bpf_sockopt which use the same signedness for the original insn.

---
 kernel/bpf/cgroup.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
kernel/bpf/cgroup.c +8 -6
index 5b2741aa0d9b..29e3606ff6f4 100644
@@ -2397,9 +2397,10 @@ static bool cg_sockopt_is_valid_access(int off, int size,
 }
 
 #define CG_SOCKOPT_READ_FIELD(F)					\
-	BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F),	\
+	BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F) |	\
+		      BPF_MODE(si->code) | BPF_CLASS(si->code)),	\
 		    si->dst_reg, si->src_reg,				\
-		    offsetof(struct bpf_sockopt_kern, F))
+		    offsetof(struct bpf_sockopt_kern, F), si->imm)
 
 #define CG_SOCKOPT_WRITE_FIELD(F)					\
 	BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F) |	\
@@ -2456,7 +2457,7 @@ static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
 			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct task_struct, bpf_ctx),
 					      treg, treg,
 					      offsetof(struct task_struct, bpf_ctx));
-			*insn++ = BPF_RAW_INSN(BPF_CLASS(si->code) | BPF_MEM |
+			*insn++ = BPF_RAW_INSN(BPF_CLASS(si->code) | BPF_MODE(si->code) |
 					       BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval),
 					       treg, si->src_reg,
 					       offsetof(struct bpf_cg_run_ctx, retval),
@@ -2470,9 +2471,10 @@ static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
 			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct task_struct, bpf_ctx),
 					      si->dst_reg, si->dst_reg,
 					      offsetof(struct task_struct, bpf_ctx));
-			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval),
-					      si->dst_reg, si->dst_reg,
-					      offsetof(struct bpf_cg_run_ctx, retval));
+			*insn++ = BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval) |
+						BPF_MODE(si->code) | BPF_CLASS(si->code)),
+					       si->dst_reg, si->dst_reg,
+					       offsetof(struct bpf_cg_run_ctx, retval), si->imm);
 		}
 		break;
 	case offsetof(struct bpf_sockopt, optval):
Signature · 1 line
2.34.1
maintainer
#8 lore raw

Re: [PATCH bpf-next v2 02/15] bpf: Fix sign-extension ctx member accesses

Quoted 9 lines · Yonghong Song
Quoted 9 lines

In uapi bpf.h, there are two ctx structures which contain signed members. Without cpu v4, such signed members will be loaded with unsigned load and the compiler will generate proper left and right shifts to get the proper final value.

With sign-extension load, however, left/right shifts are gone, we need to ensure these signed members are properly handled, with signed loads or other means. The following are list of signed ctx members and how they are handled.

This is not a generic approach, in theory any field could be cast as a signed integer. Do we want to support this? If so, then it should be handled in convert_ctx_access() by generating additional sign extension instructions.

Quoted 87 lines
Quoted 87 lines

(1).

  struct bpf_sock {
     ...
     __s32 rx_queue_mapping;

}

The corresponding kernel fields are

  struct sock_common {
     ...
     unsigned short          skc_rx_queue_mapping;
     ...

}

Current ctx rewriter uses unsigned load for the kernel field which is correct and does not need further handling.

(2).

  struct bpf_sockopt {
     ...
     __s32   level;
     __s32   optname;
     __s32   optlen;
     __s32   retval;
  }
The level/optname/optlen are from struct bpf_sockopt_kern
  struct bpf_sockopt_kern {
     ...
     s32             level;
     s32             optname;
     s32             optlen;
     ...
  }
and the 'retval' is from struct bpf_cg_run_ctx
  struct bpf_cg_run_ctx {
     ...
     int retval;

}
Current the above four fields are loaded with unsigned load. Let us modify the read macro for bpf_sockopt which use the same signedness for the original insn.

---
 kernel/bpf/cgroup.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
kernel/bpf/cgroup.c +8 -6
index 5b2741aa0d9b..29e3606ff6f4 100644
@@ -2397,9 +2397,10 @@ static bool cg_sockopt_is_valid_access(int off, int size,
 }
 
 #define CG_SOCKOPT_READ_FIELD(F)					\
-	BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F),	\
+	BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F) |	\
+		      BPF_MODE(si->code) | BPF_CLASS(si->code)),	\
 		    si->dst_reg, si->src_reg,				\
-		    offsetof(struct bpf_sockopt_kern, F))
+		    offsetof(struct bpf_sockopt_kern, F), si->imm)
 
 #define CG_SOCKOPT_WRITE_FIELD(F)					\
 	BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F) |	\
@@ -2456,7 +2457,7 @@ static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
 			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct task_struct, bpf_ctx),
 					      treg, treg,
 					      offsetof(struct task_struct, bpf_ctx));
-			*insn++ = BPF_RAW_INSN(BPF_CLASS(si->code) | BPF_MEM |
+			*insn++ = BPF_RAW_INSN(BPF_CLASS(si->code) | BPF_MODE(si->code) |
 					       BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval),
 					       treg, si->src_reg,
 					       offsetof(struct bpf_cg_run_ctx, retval),
@@ -2470,9 +2471,10 @@ static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
 			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct task_struct, bpf_ctx),
 					      si->dst_reg, si->dst_reg,
 					      offsetof(struct task_struct, bpf_ctx));
-			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval),
-					      si->dst_reg, si->dst_reg,
-					      offsetof(struct bpf_cg_run_ctx, retval));
+			*insn++ = BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval) |
+						BPF_MODE(si->code) | BPF_CLASS(si->code)),
+					       si->dst_reg, si->dst_reg,
+					       offsetof(struct bpf_cg_run_ctx, retval), si->imm);
 		}
 		break;
 	case offsetof(struct bpf_sockopt, optval):
#9 lore raw

Re: [PATCH bpf-next v2 02/15] bpf: Fix sign-extension ctx member accesses

Quoted 11 lines · Eduard Zingerman
Quoted 9 lines · Yonghong Song
Quoted 9 lines

In uapi bpf.h, there are two ctx structures which contain signed members. Without cpu v4, such signed members will be loaded with unsigned load and the compiler will generate proper left and right shifts to get the proper final value.

With sign-extension load, however, left/right shifts are gone, we need to ensure these signed members are properly handled, with signed loads or other means. The following are list of signed ctx members and how they are handled.

Quoted 4 lines

This is not a generic approach, in theory any field could be cast as a signed integer. Do we want to support this? If so, then it should be handled in convert_ctx_access() by generating additional sign extension instructions.

Good point. I will make change in verifier.c which is more generic and cover more cases.

Quoted 89 lines
Quoted 88 lines
Quoted 88 lines

(1).

   struct bpf_sock {
      ...
      __s32 rx_queue_mapping;

}

The corresponding kernel fields are

   struct sock_common {
      ...
      unsigned short          skc_rx_queue_mapping;
      ...

}

Current ctx rewriter uses unsigned load for the kernel field which is correct and does not need further handling.

(2).

   struct bpf_sockopt {
      ...
      __s32   level;
      __s32   optname;
      __s32   optlen;
      __s32   retval;
   }
The level/optname/optlen are from struct bpf_sockopt_kern
   struct bpf_sockopt_kern {
      ...
      s32             level;
      s32             optname;
      s32             optlen;
      ...
   }
and the 'retval' is from struct bpf_cg_run_ctx
   struct bpf_cg_run_ctx {
      ...
      int retval;

}
Current the above four fields are loaded with unsigned load. Let us modify the read macro for bpf_sockopt which use the same signedness for the original insn.

---
  kernel/bpf/cgroup.c | 14 ++++++++------
  1 file changed, 8 insertions(+), 6 deletions(-)
kernel/bpf/cgroup.c +8 -6
index 5b2741aa0d9b..29e3606ff6f4 100644
@@ -2397,9 +2397,10 @@ static bool cg_sockopt_is_valid_access(int off, int size,
  }
 
  #define CG_SOCKOPT_READ_FIELD(F)					\
-	BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F),	\
+	BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F) |	\
+		      BPF_MODE(si->code) | BPF_CLASS(si->code)),	\
  		    si->dst_reg, si->src_reg,				\
-		    offsetof(struct bpf_sockopt_kern, F))
+		    offsetof(struct bpf_sockopt_kern, F), si->imm)
 
  #define CG_SOCKOPT_WRITE_FIELD(F)					\
  	BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_sockopt_kern, F) |	\
@@ -2456,7 +2457,7 @@ static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
  			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct task_struct, bpf_ctx),
  					      treg, treg,
  					      offsetof(struct task_struct, bpf_ctx));
-			*insn++ = BPF_RAW_INSN(BPF_CLASS(si->code) | BPF_MEM |
+			*insn++ = BPF_RAW_INSN(BPF_CLASS(si->code) | BPF_MODE(si->code) |
  					       BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval),
  					       treg, si->src_reg,
  					       offsetof(struct bpf_cg_run_ctx, retval),
@@ -2470,9 +2471,10 @@ static u32 cg_sockopt_convert_ctx_access(enum bpf_access_type type,
  			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct task_struct, bpf_ctx),
  					      si->dst_reg, si->dst_reg,
  					      offsetof(struct task_struct, bpf_ctx));
-			*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval),
-					      si->dst_reg, si->dst_reg,
-					      offsetof(struct bpf_cg_run_ctx, retval));
+			*insn++ = BPF_RAW_INSN((BPF_FIELD_SIZEOF(struct bpf_cg_run_ctx, retval) |
+						BPF_MODE(si->code) | BPF_CLASS(si->code)),
+					       si->dst_reg, si->dst_reg,
+					       offsetof(struct bpf_cg_run_ctx, retval), si->imm);
  		}
  		break;
  	case offsetof(struct bpf_sockopt, optval):
#10 lore raw

PATCH v2 3/15 bpf: Support new sign-extension mov insns

In reply to Yonghong Song

Add interpreter/jit support for new sign-extension mov insns. The original 'MOV' insn is extended to support signed version for both ALU and ALU64 operations. For ALU mode, the insn->off value of 8 or 16 indicates sign-extension from 8- or 16-bit value to 32-bit value. For ALU64 mode, the insn->off value of 8/16/32 indicates sign-extension from 8-, 16- or 32-bit value to 64-bit value.

---
 arch/x86/net/bpf_jit_comp.c |  43 ++++++++++-
 kernel/bpf/core.c           |  28 ++++++-
 kernel/bpf/verifier.c       | 150 +++++++++++++++++++++++++++++-------
 3 files changed, 190 insertions(+), 31 deletions(-)
arch/x86/net/bpf_jit_comp.c +40 -3
index addeea95f397..a740a1a6e71d 100644
@@ -701,6 +701,38 @@ static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg)
 	*pprog = prog;
 }
 
+static void emit_movsx_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+			   u32 src_reg)
+{
+	u8 *prog = *pprog;
+
+	if (is64) {
+		/* movs[b,w,l]q dst, src */
+		if (num_bits == 8)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 16)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 32)
+			EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+			      add_2reg(0xC0, src_reg, dst_reg));
+	} else {
+		/* movs[b,w]l dst, src */
+		if (num_bits == 8) {
+			EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		} else if (num_bits == 16) {
+			if (is_ereg(dst_reg) || is_ereg(src_reg))
+				EMIT1(add_2mod(0x40, src_reg, dst_reg));
+			EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		}
+	}
+
+	*pprog = prog;
+}
+
 /* Emit the suffix (ModR/M etc) for addressing *(ptr_reg + off) and val_reg */
 static void emit_insn_suffix(u8 **pprog, u32 ptr_reg, u32 val_reg, int off)
 {
@@ -1051,9 +1083,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 
 		case BPF_ALU64 | BPF_MOV | BPF_X:
 		case BPF_ALU | BPF_MOV | BPF_X:
-			emit_mov_reg(&prog,
-				     BPF_CLASS(insn->code) == BPF_ALU64,
-				     dst_reg, src_reg);
+			if (insn->off == 0)
+				emit_mov_reg(&prog,
+					     BPF_CLASS(insn->code) == BPF_ALU64,
+					     dst_reg, src_reg);
+			else
+				emit_movsx_reg(&prog, insn->off,
+					       BPF_CLASS(insn->code) == BPF_ALU64,
+					       dst_reg, src_reg);
 			break;
 
 			/* neg dst */
kernel/bpf/core.c +26 -2
index 8a1cc658789e..fe648a158c9e 100644
@@ -61,6 +61,7 @@
 #define AX	regs[BPF_REG_AX]
 #define ARG1	regs[BPF_REG_ARG1]
 #define CTX	regs[BPF_REG_CTX]
+#define OFF	insn->off
 #define IMM	insn->imm
 
 struct bpf_mem_alloc bpf_global_ma;
@@ -1736,13 +1737,36 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 		DST = -DST;
 		CONT;
 	ALU_MOV_X:
-		DST = (u32) SRC;
+		switch (OFF) {
+		case 0:
+			DST = (u32) SRC;
+			break;
+		case 8:
+			DST = (u32)(s8) SRC;
+			break;
+		case 16:
+			DST = (u32)(s16) SRC;
+			break;
+		}
 		CONT;
 	ALU_MOV_K:
 		DST = (u32) IMM;
 		CONT;
 	ALU64_MOV_X:
-		DST = SRC;
+		switch (OFF) {
+		case 0:
+			DST = SRC;
+			break;
+		case 8:
+			DST = (s8) SRC;
+			break;
+		case 16:
+			DST = (s16) SRC;
+			break;
+		case 32:
+			DST = (s32) SRC;
+			break;
+		}
 		CONT;
 	ALU64_MOV_K:
 		DST = IMM;
kernel/bpf/verifier.c +124 -26

185 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#11 lore raw

Re: [PATCH bpf-next v2 03/15] bpf: Support new sign-extension mov insns

Quoted 36 lines · Yonghong Song
Quoted 36 lines

Add interpreter/jit support for new sign-extension mov insns. The original 'MOV' insn is extended to support signed version for both ALU and ALU64 operations. For ALU mode, the insn->off value of 8 or 16 indicates sign-extension from 8- or 16-bit value to 32-bit value. For ALU64 mode, the insn->off value of 8/16/32 indicates sign-extension from 8-, 16- or 32-bit value to 64-bit value.

---
 arch/x86/net/bpf_jit_comp.c |  43 ++++++++++-
 kernel/bpf/core.c           |  28 ++++++-
 kernel/bpf/verifier.c       | 150 +++++++++++++++++++++++++++++-------
 3 files changed, 190 insertions(+), 31 deletions(-)
arch/x86/net/bpf_jit_comp.c +13 -0
index addeea95f397..a740a1a6e71d 100644
@@ -701,6 +701,38 @@ static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg)
 	*pprog = prog;
 }
 
+static void emit_movsx_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+			   u32 src_reg)
+{
+	u8 *prog = *pprog;
+
+	if (is64) {
+		/* movs[b,w,l]q dst, src */
+		if (num_bits == 8)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 16)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
Quoted 8 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 724

Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +8 -0
+		else if (num_bits == 32)
+			EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+			      add_2reg(0xC0, src_reg, dst_reg));
+	} else {
+		/* movs[b,w]l dst, src */
+		if (num_bits == 8) {
+			EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));

Nit: As far as I understand 4-126 Vol. 2B of [1]

     the 0x40 prefix (REX prefix) is optional here
     (same as implemented below for num_bits == 16).

[1] https://cdrdv2.intel.com/v1/dl/getContent/671200

Quoted 4 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 728

Quoted 4 lines
arch/x86/net/bpf_jit_comp.c +4 -0
+		} else if (num_bits == 16) {
+			if (is_ereg(dst_reg) || is_ereg(src_reg))
+				EMIT1(add_2mod(0x40, src_reg, dst_reg));
+			EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,

Nit: Basing on the same manual I don't understand why
add_2mod(0x0f, src_reg, dst_reg) is used, '0xf' should suffice (but I tried it both ways and it works...).

Quoted 261 lines
Quoted 261 lines
arch/x86/net/bpf_jit_comp.c +15 -3
+			      add_2reg(0xC0, src_reg, dst_reg));
+		}
+	}
+
+	*pprog = prog;
+}
+
 /* Emit the suffix (ModR/M etc) for addressing *(ptr_reg + off) and val_reg */
 static void emit_insn_suffix(u8 **pprog, u32 ptr_reg, u32 val_reg, int off)
 {
@@ -1051,9 +1083,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 
 		case BPF_ALU64 | BPF_MOV | BPF_X:
 		case BPF_ALU | BPF_MOV | BPF_X:
-			emit_mov_reg(&prog,
-				     BPF_CLASS(insn->code) == BPF_ALU64,
-				     dst_reg, src_reg);
+			if (insn->off == 0)
+				emit_mov_reg(&prog,
+					     BPF_CLASS(insn->code) == BPF_ALU64,
+					     dst_reg, src_reg);
+			else
+				emit_movsx_reg(&prog, insn->off,
+					       BPF_CLASS(insn->code) == BPF_ALU64,
+					       dst_reg, src_reg);
 			break;
 
 			/* neg dst */
kernel/bpf/core.c +26 -2
index 8a1cc658789e..fe648a158c9e 100644
@@ -61,6 +61,7 @@
 #define AX	regs[BPF_REG_AX]
 #define ARG1	regs[BPF_REG_ARG1]
 #define CTX	regs[BPF_REG_CTX]
+#define OFF	insn->off
 #define IMM	insn->imm
 
 struct bpf_mem_alloc bpf_global_ma;
@@ -1736,13 +1737,36 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 		DST = -DST;
 		CONT;
 	ALU_MOV_X:
-		DST = (u32) SRC;
+		switch (OFF) {
+		case 0:
+			DST = (u32) SRC;
+			break;
+		case 8:
+			DST = (u32)(s8) SRC;
+			break;
+		case 16:
+			DST = (u32)(s16) SRC;
+			break;
+		}
 		CONT;
 	ALU_MOV_K:
 		DST = (u32) IMM;
 		CONT;
 	ALU64_MOV_X:
-		DST = SRC;
+		switch (OFF) {
+		case 0:
+			DST = SRC;
+			break;
+		case 8:
+			DST = (s8) SRC;
+			break;
+		case 16:
+			DST = (s16) SRC;
+			break;
+		case 32:
+			DST = (s32) SRC;
+			break;
+		}
 		CONT;
 	ALU64_MOV_K:
 		DST = IMM;
kernel/bpf/verifier.c +115 -26
index fbe4ca72d4c1..5fee9f24cb5e 100644
@@ -3421,7 +3421,7 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
 			return 0;
 		if (opcode == BPF_MOV) {
 			if (BPF_SRC(insn->code) == BPF_X) {
-				/* dreg = sreg
+				/* dreg = sreg or dreg = (s8, s16, s32)sreg
 				 * dreg needs precision after this insn
 				 * sreg needs precision before this insn
 				 */
@@ -5866,6 +5866,64 @@ static void coerce_reg_to_size_sx(struct bpf_reg_state *reg, int size)
 	set_sext64_default_val(reg, size);
 }
 
+static void set_sext32_default_val(struct bpf_reg_state *reg, int size)
+{
+	if (size == 1) {
+		reg->s32_min_value = S8_MIN;
+		reg->s32_max_value = S8_MAX;
+	} else {
+		/* size == 2 */
+		reg->s32_min_value = S16_MIN;
+		reg->s32_max_value = S16_MAX;
+	}
+	reg->u32_min_value = 0;
+	reg->u32_max_value = U32_MAX;
+}
+
+static void coerce_subreg_to_size_sx(struct bpf_reg_state *reg, int size)
+{
+	s32 init_s32_max, init_s32_min, s32_max, s32_min;
+	u32 top_smax_value, top_smin_value;
+	u32 num_bits = size * 8;
+
+	top_smax_value = ((u32)reg->s32_max_value >> num_bits) << num_bits;
+	top_smin_value = ((u32)reg->s32_min_value >> num_bits) << num_bits;
+
+	if (top_smax_value != top_smin_value)
+		goto out;
+
+	/* find the s32_min and s32_min after sign extension */
+	if (size == 1) {
+		init_s32_max = (s8)reg->s32_max_value;
+		init_s32_min = (s8)reg->s32_min_value;
+	} else {
+		/* size == 2 */
+		init_s32_max = (s16)reg->s32_max_value;
+		init_s32_min = (s16)reg->s32_min_value;
+	}
+	s32_max = max(init_s32_max, init_s32_min);
+	s32_min = min(init_s32_max, init_s32_min);
+
+	if (s32_min >= 0 && s32_max >= 0) {
+		reg->s32_min_value = s32_min;
+		reg->s32_max_value = s32_max;
+		reg->u32_min_value = 0;
+		reg->u32_max_value = U32_MAX;
+		return;
+	}
+
+	if (s32_min < 0 && s32_max < 0) {
+		reg->s32_min_value = s32_min;
+		reg->s32_max_value = s32_max;
+		reg->u32_min_value = (u32)s32_max;
+		reg->u32_max_value = (u32)s32_min;
+		return;
+	}
+
+out:
+	set_sext32_default_val(reg, size);
+}
+
 static bool bpf_map_is_rdonly(const struct bpf_map *map)
 {
 	/* A map is considered read-only if the following condition are true:
@@ -13003,11 +13061,23 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 	} else if (opcode == BPF_MOV) {
 
 		if (BPF_SRC(insn->code) == BPF_X) {
-			if (insn->imm != 0 || insn->off != 0) {
+			if (insn->imm != 0) {
 				verbose(env, "BPF_MOV uses reserved fields\n");
 				return -EINVAL;
 			}
 
+			if (BPF_CLASS(insn->code) == BPF_ALU) {
+				if (insn->off != 0 && insn->off != 8 && insn->off != 16) {
+					verbose(env, "BPF_MOV uses reserved fields\n");
+					return -EINVAL;
+				}
+			} else {
+				if (insn->off != 0 && insn->off != 8 && insn->off != 16 && insn->off != 32) {
+					verbose(env, "BPF_MOV uses reserved fields\n");
+					return -EINVAL;
+				}
+			}
+
 			/* check src operand */
 			err = check_reg_arg(env, insn->src_reg, SRC_OP);
 			if (err)
@@ -13031,18 +13101,32 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 				       !tnum_is_const(src_reg->var_off);
 
 			if (BPF_CLASS(insn->code) == BPF_ALU64) {
-				/* case: R1 = R2
-				 * copy register state to dest reg
-				 */
-				if (need_id)
-					/* Assign src and dst registers the same ID
-					 * that will be used by find_equal_scalars()
-					 * to propagate min/max range.
+				if (insn->off == 0) {
+					/* case: R1 = R2
+					 * copy register state to dest reg
 					 */
-					src_reg->id = ++env->id_gen;
-				copy_register_state(dst_reg, src_reg);
-				dst_reg->live |= REG_LIVE_WRITTEN;
-				dst_reg->subreg_def = DEF_NOT_SUBREG;
+					if (need_id)
+						/* Assign src and dst registers the same ID
+						 * that will be used by find_equal_scalars()
+						 * to propagate min/max range.
+						 */
+						src_reg->id = ++env->id_gen;
+					copy_register_state(dst_reg, src_reg);
+					dst_reg->live |= REG_LIVE_WRITTEN;
+					dst_reg->subreg_def = DEF_NOT_SUBREG;
+				} else {
+					/* case: R1 = (s8, s16 s32)R2 */
+					bool no_sext = src_reg->umax_value < (1ULL << (insn->off - 1));
+
+					if (no_sext && need_id)
+						src_reg->id = ++env->id_gen;
+					copy_register_state(dst_reg, src_reg);
+					if (!no_sext)
+						dst_reg->id = 0;
+					coerce_reg_to_size_sx(dst_reg, insn->off >> 3);
+					dst_reg->live |= REG_LIVE_WRITTEN;
+					dst_reg->subreg_def = DEF_NOT_SUBREG;
+				}
 			} else {
 				/* R1 = (u32) R2 */
 				if (is_pointer_value(env, insn->src_reg)) {
@@ -13051,19 +13135,33 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 						insn->src_reg);
 					return -EACCES;
 				} else if (src_reg->type == SCALAR_VALUE) {
-					bool is_src_reg_u32 = src_reg->umax_value <= U32_MAX;
-
-					if (is_src_reg_u32 && need_id)
-						src_reg->id = ++env->id_gen;
-					copy_register_state(dst_reg, src_reg);
-					/* Make sure ID is cleared if src_reg is not in u32 range otherwise
-					 * dst_reg min/max could be incorrectly
-					 * propagated into src_reg by find_equal_scalars()
-					 */
-					if (!is_src_reg_u32)
-						dst_reg->id = 0;
-					dst_reg->live |= REG_LIVE_WRITTEN;
-					dst_reg->subreg_def = env->insn_idx + 1;
+					if (insn->off == 0) {
+						bool is_src_reg_u32 = src_reg->umax_value <= U32_MAX;
+
+						if (is_src_reg_u32 && need_id)
+							src_reg->id = ++env->id_gen;
+						copy_register_state(dst_reg, src_reg);
+						/* Make sure ID is cleared if src_reg is not in u32 range otherwise
+						 * dst_reg min/max could be incorrectly
+						 * propagated into src_reg by find_equal_scalars()
+						 */
+						if (!is_src_reg_u32)
+							dst_reg->id = 0;
+						dst_reg->live |= REG_LIVE_WRITTEN;
+						dst_reg->subreg_def = env->insn_idx + 1;
+					} else {
+						/* case: W1 = (s8, s16)W2 */
+						bool no_sext = src_reg->umax_value < (1ULL << (insn->off - 1));
+
Quoted 8 lines

kernel/bpf/verifier.c static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn) · line 13163

Quoted 8 lines
kernel/bpf/verifier.c +8 -0
+						if (no_sext && need_id)
+							src_reg->id = ++env->id_gen;
+						copy_register_state(dst_reg, src_reg);
+						if (!no_sext)
+							dst_reg->id = 0;
+						dst_reg->live |= REG_LIVE_WRITTEN;
+						dst_reg->subreg_def = env->insn_idx + 1;
+						coerce_subreg_to_size_sx(dst_reg, insn->off >> 3);

I tried the following test program:

{
"testtesttest",
.insns = {
BPF_MOV64_IMM(BPF_REG_7, 0xffff),
{
.code = BPF_ALU | BPF_MOV | BPF_X,
.dst_reg = BPF_REG_0,
.src_reg = BPF_REG_7,
.off = 16,
.imm = 0,
},
BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
.result = ACCEPT,
.retval = 0,
},

And it produces verification log as below:

0: R1=ctx(off=0,imm=0) R10=fp0

 0: (b7) r7 = 65535    ; R7_w=P65535
 1: (bc) w0 = w7       ; R0_w=P65535 R7_w=P65535
 2: (77) r0 >>= 1      ; R0_w=P32767

3: (95) exit
...
FAIL retval 2147483647 != 0 (run 1/1)

Note that verifier considers R0 to be 0x7FFF at 3, while actual value during execution is 0x7FFF'FFFF.

Quoted 4 lines
Quoted 4 lines
kernel/bpf/verifier.c +1 -0
+					}
 				} else {
 					mark_reg_unknown(env, regs,
 							 insn->dst_reg);
#12 lore raw

Re: [PATCH bpf-next v2 03/15] bpf: Support new sign-extension mov insns

Quoted 47 lines · Eduard Zingerman
Quoted 36 lines · Yonghong Song
Quoted 36 lines

Add interpreter/jit support for new sign-extension mov insns. The original 'MOV' insn is extended to support signed version for both ALU and ALU64 operations. For ALU mode, the insn->off value of 8 or 16 indicates sign-extension from 8- or 16-bit value to 32-bit value. For ALU64 mode, the insn->off value of 8/16/32 indicates sign-extension from 8-, 16- or 32-bit value to 64-bit value.

---
  arch/x86/net/bpf_jit_comp.c |  43 ++++++++++-
  kernel/bpf/core.c           |  28 ++++++-
  kernel/bpf/verifier.c       | 150 +++++++++++++++++++++++++++++-------
  3 files changed, 190 insertions(+), 31 deletions(-)
arch/x86/net/bpf_jit_comp.c +13 -0
index addeea95f397..a740a1a6e71d 100644
@@ -701,6 +701,38 @@ static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg)
  	*pprog = prog;
  }
 
+static void emit_movsx_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+			   u32 src_reg)
+{
+	u8 *prog = *pprog;
+
+	if (is64) {
+		/* movs[b,w,l]q dst, src */
+		if (num_bits == 8)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 16)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
Quoted 8 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 724

Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +8 -0
+		else if (num_bits == 32)
+			EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+			      add_2reg(0xC0, src_reg, dst_reg));
+	} else {
+		/* movs[b,w]l dst, src */
+		if (num_bits == 8) {
+			EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));

Nit: As far as I understand 4-126 Vol. 2B of [1]

Quoted 2 lines
      the 0x40 prefix (REX prefix) is optional here
      (same as implemented below for num_bits == 16).

I think 0x40 prefix at least neededif register is from R8 - R15? I use this website to do asm/disasm experiments and did try various combinations with first 8 and later 8 registers and it seems correct results are generated.

Quoted 4 lines

[1] https://cdrdv2.intel.com/v1/dl/getContent/671200

Quoted 8 lines
Quoted 4 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 728

Quoted 4 lines
arch/x86/net/bpf_jit_comp.c +4 -0
+		} else if (num_bits == 16) {
+			if (is_ereg(dst_reg) || is_ereg(src_reg))
+				EMIT1(add_2mod(0x40, src_reg, dst_reg));
+			EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,

Nit: Basing on the same manual I don't understand why
add_2mod(0x0f, src_reg, dst_reg) is used, '0xf' should suffice (but I tried it both ways and it works...).

From the above online assembler website.

But I will check the doc to see whether it can be simplified.

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
arch/x86/net/bpf_jit_comp.c +15 -3
+			      add_2reg(0xC0, src_reg, dst_reg));
+		}
+	}
+
+	*pprog = prog;
+}
+
  /* Emit the suffix (ModR/M etc) for addressing *(ptr_reg + off) and val_reg */
  static void emit_insn_suffix(u8 **pprog, u32 ptr_reg, u32 val_reg, int off)
  {
@@ -1051,9 +1083,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 
  		case BPF_ALU64 | BPF_MOV | BPF_X:
  		case BPF_ALU | BPF_MOV | BPF_X:
-			emit_mov_reg(&prog,
-				     BPF_CLASS(insn->code) == BPF_ALU64,
-				     dst_reg, src_reg);
+			if (insn->off == 0)
+				emit_mov_reg(&prog,
+					     BPF_CLASS(insn->code) == BPF_ALU64,
+					     dst_reg, src_reg);
+			else
+				emit_movsx_reg(&prog, insn->off,
+					       BPF_CLASS(insn->code) == BPF_ALU64,
+					       dst_reg, src_reg);
  			break;
 
  			/* neg dst */
kernel/bpf/core.c +1 -0
index 8a1cc658789e..fe648a158c9e 100644
@@ -61,6 +61,7 @@
  #define AX	regs[BPF_REG_AX]
  #define ARG1	regs[BPF_REG_ARG1]
  #define CTX	regs[BPF_REG_CTX]
+#define OFF	insn->off
  #define IMM	insn->imm
 

[...]

Quoted 217 lines
Quoted 182 lines
Quoted 182 lines
kernel/bpf/verifier.c +115 -26
index fbe4ca72d4c1..5fee9f24cb5e 100644
@@ -3421,7 +3421,7 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, int subseq_idx,
  			return 0;
  		if (opcode == BPF_MOV) {
  			if (BPF_SRC(insn->code) == BPF_X) {
-				/* dreg = sreg
+				/* dreg = sreg or dreg = (s8, s16, s32)sreg
  				 * dreg needs precision after this insn
  				 * sreg needs precision before this insn
  				 */
@@ -5866,6 +5866,64 @@ static void coerce_reg_to_size_sx(struct bpf_reg_state *reg, int size)
  	set_sext64_default_val(reg, size);
  }
 
+static void set_sext32_default_val(struct bpf_reg_state *reg, int size)
+{
+	if (size == 1) {
+		reg->s32_min_value = S8_MIN;
+		reg->s32_max_value = S8_MAX;
+	} else {
+		/* size == 2 */
+		reg->s32_min_value = S16_MIN;
+		reg->s32_max_value = S16_MAX;
+	}
+	reg->u32_min_value = 0;
+	reg->u32_max_value = U32_MAX;
+}
+
+static void coerce_subreg_to_size_sx(struct bpf_reg_state *reg, int size)
+{
+	s32 init_s32_max, init_s32_min, s32_max, s32_min;
+	u32 top_smax_value, top_smin_value;
+	u32 num_bits = size * 8;
+
+	top_smax_value = ((u32)reg->s32_max_value >> num_bits) << num_bits;
+	top_smin_value = ((u32)reg->s32_min_value >> num_bits) << num_bits;
+
+	if (top_smax_value != top_smin_value)
+		goto out;
+
+	/* find the s32_min and s32_min after sign extension */
+	if (size == 1) {
+		init_s32_max = (s8)reg->s32_max_value;
+		init_s32_min = (s8)reg->s32_min_value;
+	} else {
+		/* size == 2 */
+		init_s32_max = (s16)reg->s32_max_value;
+		init_s32_min = (s16)reg->s32_min_value;
+	}
+	s32_max = max(init_s32_max, init_s32_min);
+	s32_min = min(init_s32_max, init_s32_min);
+
+	if (s32_min >= 0 && s32_max >= 0) {
+		reg->s32_min_value = s32_min;
+		reg->s32_max_value = s32_max;
+		reg->u32_min_value = 0;
+		reg->u32_max_value = U32_MAX;
+		return;
+	}
+
+	if (s32_min < 0 && s32_max < 0) {
+		reg->s32_min_value = s32_min;
+		reg->s32_max_value = s32_max;
+		reg->u32_min_value = (u32)s32_max;
+		reg->u32_max_value = (u32)s32_min;
+		return;
+	}
+
+out:
+	set_sext32_default_val(reg, size);
+}
+
  static bool bpf_map_is_rdonly(const struct bpf_map *map)
  {
  	/* A map is considered read-only if the following condition are true:
@@ -13003,11 +13061,23 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
  	} else if (opcode == BPF_MOV) {
 
  		if (BPF_SRC(insn->code) == BPF_X) {
-			if (insn->imm != 0 || insn->off != 0) {
+			if (insn->imm != 0) {
  				verbose(env, "BPF_MOV uses reserved fields\n");
  				return -EINVAL;
  			}
 
+			if (BPF_CLASS(insn->code) == BPF_ALU) {
+				if (insn->off != 0 && insn->off != 8 && insn->off != 16) {
+					verbose(env, "BPF_MOV uses reserved fields\n");
+					return -EINVAL;
+				}
+			} else {
+				if (insn->off != 0 && insn->off != 8 && insn->off != 16 && insn->off != 32) {
+					verbose(env, "BPF_MOV uses reserved fields\n");
+					return -EINVAL;
+				}
+			}
+
  			/* check src operand */
  			err = check_reg_arg(env, insn->src_reg, SRC_OP);
  			if (err)
@@ -13031,18 +13101,32 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
  				       !tnum_is_const(src_reg->var_off);
 
  			if (BPF_CLASS(insn->code) == BPF_ALU64) {
-				/* case: R1 = R2
-				 * copy register state to dest reg
-				 */
-				if (need_id)
-					/* Assign src and dst registers the same ID
-					 * that will be used by find_equal_scalars()
-					 * to propagate min/max range.
+				if (insn->off == 0) {
+					/* case: R1 = R2
+					 * copy register state to dest reg
  					 */
-					src_reg->id = ++env->id_gen;
-				copy_register_state(dst_reg, src_reg);
-				dst_reg->live |= REG_LIVE_WRITTEN;
-				dst_reg->subreg_def = DEF_NOT_SUBREG;
+					if (need_id)
+						/* Assign src and dst registers the same ID
+						 * that will be used by find_equal_scalars()
+						 * to propagate min/max range.
+						 */
+						src_reg->id = ++env->id_gen;
+					copy_register_state(dst_reg, src_reg);
+					dst_reg->live |= REG_LIVE_WRITTEN;
+					dst_reg->subreg_def = DEF_NOT_SUBREG;
+				} else {
+					/* case: R1 = (s8, s16 s32)R2 */
+					bool no_sext = src_reg->umax_value < (1ULL << (insn->off - 1));
+
+					if (no_sext && need_id)
+						src_reg->id = ++env->id_gen;
+					copy_register_state(dst_reg, src_reg);
+					if (!no_sext)
+						dst_reg->id = 0;
+					coerce_reg_to_size_sx(dst_reg, insn->off >> 3);
+					dst_reg->live |= REG_LIVE_WRITTEN;
+					dst_reg->subreg_def = DEF_NOT_SUBREG;
+				}
  			} else {
  				/* R1 = (u32) R2 */
  				if (is_pointer_value(env, insn->src_reg)) {
@@ -13051,19 +13135,33 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
  						insn->src_reg);
  					return -EACCES;
  				} else if (src_reg->type == SCALAR_VALUE) {
-					bool is_src_reg_u32 = src_reg->umax_value <= U32_MAX;
-
-					if (is_src_reg_u32 && need_id)
-						src_reg->id = ++env->id_gen;
-					copy_register_state(dst_reg, src_reg);
-					/* Make sure ID is cleared if src_reg is not in u32 range otherwise
-					 * dst_reg min/max could be incorrectly
-					 * propagated into src_reg by find_equal_scalars()
-					 */
-					if (!is_src_reg_u32)
-						dst_reg->id = 0;
-					dst_reg->live |= REG_LIVE_WRITTEN;
-					dst_reg->subreg_def = env->insn_idx + 1;
+					if (insn->off == 0) {
+						bool is_src_reg_u32 = src_reg->umax_value <= U32_MAX;
+
+						if (is_src_reg_u32 && need_id)
+							src_reg->id = ++env->id_gen;
+						copy_register_state(dst_reg, src_reg);
+						/* Make sure ID is cleared if src_reg is not in u32 range otherwise
+						 * dst_reg min/max could be incorrectly
+						 * propagated into src_reg by find_equal_scalars()
+						 */
+						if (!is_src_reg_u32)
+							dst_reg->id = 0;
+						dst_reg->live |= REG_LIVE_WRITTEN;
+						dst_reg->subreg_def = env->insn_idx + 1;
+					} else {
+						/* case: W1 = (s8, s16)W2 */
+						bool no_sext = src_reg->umax_value < (1ULL << (insn->off - 1));
+
Quoted 8 lines

kernel/bpf/verifier.c static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn) · line 13163

Quoted 8 lines
kernel/bpf/verifier.c +8 -0
+						if (no_sext && need_id)
+							src_reg->id = ++env->id_gen;
+						copy_register_state(dst_reg, src_reg);
+						if (!no_sext)
+							dst_reg->id = 0;
+						dst_reg->live |= REG_LIVE_WRITTEN;
+						dst_reg->subreg_def = env->insn_idx + 1;
+						coerce_subreg_to_size_sx(dst_reg, insn->off >> 3);

I tried the following test program:

{
"testtesttest",
.insns = {
BPF_MOV64_IMM(BPF_REG_7, 0xffff),
{
.code = BPF_ALU | BPF_MOV | BPF_X,
.dst_reg = BPF_REG_0,
.src_reg = BPF_REG_7,
.off = 16,
.imm = 0,
},
BPF_ALU64_IMM(BPF_RSH, BPF_REG_0, 1),
BPF_EXIT_INSN(),
},
.result = ACCEPT,
.retval = 0,
},

And it produces verification log as below:

0: R1=ctx(off=0,imm=0) R10=fp0

  0: (b7) r7 = 65535    ; R7_w=P65535
  1: (bc) w0 = w7       ; R0_w=P65535 R7_w=P65535
  2: (77) r0 >>= 1      ; R0_w=P32767
Quoted 6 lines

3: (95) exit
...
FAIL retval 2147483647 != 0 (run 1/1)

Note that verifier considers R0 to be 0x7FFF at 3, while actual value during execution is 0x7FFF'FFFF.

This is a verifier issue. Will fix.

Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
kernel/bpf/verifier.c +1 -0
+					}
  				} else {
  					mark_reg_unknown(env, regs,
  							 insn->dst_reg);
maintainer
#13 lore raw

Re: [PATCH bpf-next v2 03/15] bpf: Support new sign-extension mov insns

On Tue, 2023-07-18 at 18:17 -0700, Yonghong Song wrote:
[...]

Quoted 26 lines
Quoted 23 lines
Quoted 13 lines
Quoted 13 lines
arch/x86/net/bpf_jit_comp.c +13 -0
+static void emit_movsx_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+			   u32 src_reg)
+{
+	u8 *prog = *pprog;
+
+	if (is64) {
+		/* movs[b,w,l]q dst, src */
+		if (num_bits == 8)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 16)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
Quoted 8 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 724

Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +8 -0
+		else if (num_bits == 32)
+			EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+			      add_2reg(0xC0, src_reg, dst_reg));
+	} else {
+		/* movs[b,w]l dst, src */
+		if (num_bits == 8) {
+			EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));

Nit: As far as I understand 4-126 Vol. 2B of [1]

Quoted 2 lines
      the 0x40 prefix (REX prefix) is optional here
      (same as implemented below for num_bits == 16).
Quoted 1 line

I think 0x40 prefix at least neededif register is from R8 - R15?

Yes, please see below.

Quoted 3 lines

I use this website to do asm/disasm experiments and did try various combinations with first 8 and later 8 registers and it seems correct results are generated.

It seems all roads lead to that web-site, I used it as well :) Today I learned that the following could be used:

echo 'movsx rax,ax' | as -o /dev/null -aln -msyntax=intel -mnaked-reg

Which opens a road to scripting experiments.

Quoted 13 lines
Quoted 4 lines

[1] https://cdrdv2.intel.com/v1/dl/getContent/671200

Quoted 8 lines
Quoted 4 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 728

Quoted 4 lines
arch/x86/net/bpf_jit_comp.c +4 -0
+		} else if (num_bits == 16) {
+			if (is_ereg(dst_reg) || is_ereg(src_reg))
+				EMIT1(add_2mod(0x40, src_reg, dst_reg));
+			EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,

Nit: Basing on the same manual I don't understand why
add_2mod(0x0f, src_reg, dst_reg) is used, '0xf' should suffice (but I tried it both ways and it works...).

Quoted 3 lines

From the above online assembler website.

But I will check the doc to see whether it can be simplified.

I tried all combinations of r0..r9 for 64/32-bit destinations, 32/16/8 sources [1]:
- 0x40 based prefix is generated if any of the following is true:
- dst is 64 bit
- dst is ereg
- src is ereg
- dst is 32-bit and src is 'sil' (part of 'rsi', used for r2) (!) This one is surprising and web-site shows the same results. For example movsx eax,sil is encoded as 40 0F BE C6, disassembling 0F BE C6 (w/o prefix) gives movsx eax,dh.
- opcodes:

  - 63      64-bit dst, 32-bit src
  - 0F BF   64-bit dst, 16-bit src
  - 0F BE   64-bit dst,  8-bit src
  - 0F BF   32-bit dst, 16-bit src (same as 64-bit dst)
  - 0F BE   32-bit dst,  8-bit src (same as 64-bit dst)

Script is at [2] (it is not particularly interesting, but in case if you want to tweak it).

[1] https://gist.github.com/eddyz87/94b35fd89f023c43dd2480e196b28ea1
[2] https://gist.github.com/eddyz87/60991379c547df11d30fa91901862227

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/net/bpf_jit_comp.c +6 -0
+			      add_2reg(0xC0, src_reg, dst_reg));
+		}
+	}
+
+	*pprog = prog;
+}

[...]

#14 lore raw

Re: [PATCH bpf-next v2 03/15] bpf: Support new sign-extension mov insns

Quoted 37 lines · Eduard Zingerman

On Tue, 2023-07-18 at 18:17 -0700, Yonghong Song wrote:
[...]

Quoted 26 lines
Quoted 23 lines
Quoted 13 lines
Quoted 13 lines
arch/x86/net/bpf_jit_comp.c +13 -0
+static void emit_movsx_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+                          u32 src_reg)
+{
+       u8 *prog = *pprog;
+
+       if (is64) {
+               /* movs[b,w,l]q dst, src */
+               if (num_bits == 8)
+                       EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+                             add_2reg(0xC0, src_reg, dst_reg));
+               else if (num_bits == 16)
+                       EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+                             add_2reg(0xC0, src_reg, dst_reg));
Quoted 8 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 724

Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +8 -0
+               else if (num_bits == 32)
+                       EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+                             add_2reg(0xC0, src_reg, dst_reg));
+       } else {
+               /* movs[b,w]l dst, src */
+               if (num_bits == 8) {
+                       EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+                             add_2reg(0xC0, src_reg, dst_reg));

Nit: As far as I understand 4-126 Vol. 2B of [1]

Quoted 2 lines
      the 0x40 prefix (REX prefix) is optional here
      (same as implemented below for num_bits == 16).
Quoted 1 line

I think 0x40 prefix at least neededif register is from R8 - R15?

Yes, please see below.

Quoted 3 lines

I use this website to do asm/disasm experiments and did try various combinations with first 8 and later 8 registers and it seems correct results are generated.

Quoted 6 lines

It seems all roads lead to that web-site, I used it as well :) Today I learned that the following could be used:

echo 'movsx rax,ax' | as -o /dev/null -aln -msyntax=intel -mnaked-reg

Which opens a road to scripting experiments.

This internal tool from llvm-project may also be useful:)

llvm-mc -triple=x86_64 -show-inst -x86-asm-syntax=intel -output-asm-variant=1 <<< 'movsx rax, ax'

Quoted 47 lines
Quoted 9 lines
Quoted 9 lines

[1] https://cdrdv2.intel.com/v1/dl/getContent/671200

Quoted 4 lines
Quoted 4 lines
arch/x86/net/bpf_jit_comp.c +4 -0
+               } else if (num_bits == 16) {
+                       if (is_ereg(dst_reg) || is_ereg(src_reg))
+                               EMIT1(add_2mod(0x40, src_reg, dst_reg));
+                       EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,
Quoted 7 lines
Quoted 3 lines

Nit: Basing on the same manual I don't understand why
add_2mod(0x0f, src_reg, dst_reg) is used, '0xf' should suffice (but I tried it both ways and it works...).

From the above online assembler website.

But I will check the doc to see whether it can be simplified.

I tried all combinations of r0..r9 for 64/32-bit destinations, 32/16/8 sources [1]:
- 0x40 based prefix is generated if any of the following is true:
- dst is 64 bit
- dst is ereg
- src is ereg
- dst is 32-bit and src is 'sil' (part of 'rsi', used for r2) (!) This one is surprising and web-site shows the same results. For example movsx eax,sil is encoded as 40 0F BE C6, disassembling 0F BE C6 (w/o prefix) gives movsx eax,dh.
- opcodes:

  - 63      64-bit dst, 32-bit src
  - 0F BF   64-bit dst, 16-bit src
  - 0F BE   64-bit dst,  8-bit src
  - 0F BF   32-bit dst, 16-bit src (same as 64-bit dst)
  - 0F BE   32-bit dst,  8-bit src (same as 64-bit dst)

Script is at [2] (it is not particularly interesting, but in case if you want to tweak it).

[1] https://gist.github.com/eddyz87/94b35fd89f023c43dd2480e196b28ea1
[2] https://gist.github.com/eddyz87/60991379c547df11d30fa91901862227

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/net/bpf_jit_comp.c +6 -0
+                             add_2reg(0xC0, src_reg, dst_reg));
+               }
+       }
+
+       *pprog = prog;
+}

[...]

Signature · 1 line
宋方睿
maintainer
#15 lore raw

Re: [PATCH bpf-next v2 03/15] bpf: Support new sign-extension mov insns

Quoted 41 lines · Fangrui Song
Quoted 40 lines · Eduard Zingerman

On Tue, 2023-07-18 at 18:17 -0700, Yonghong Song wrote:
[...]

Quoted 26 lines
Quoted 23 lines
Quoted 13 lines
Quoted 13 lines
arch/x86/net/bpf_jit_comp.c +13 -0
+static void emit_movsx_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+                          u32 src_reg)
+{
+       u8 *prog = *pprog;
+
+       if (is64) {
+               /* movs[b,w,l]q dst, src */
+               if (num_bits == 8)
+                       EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+                             add_2reg(0xC0, src_reg, dst_reg));
+               else if (num_bits == 16)
+                       EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+                             add_2reg(0xC0, src_reg, dst_reg));
Quoted 8 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 724

Quoted 8 lines
arch/x86/net/bpf_jit_comp.c +8 -0
+               else if (num_bits == 32)
+                       EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+                             add_2reg(0xC0, src_reg, dst_reg));
+       } else {
+               /* movs[b,w]l dst, src */
+               if (num_bits == 8) {
+                       EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+                             add_2reg(0xC0, src_reg, dst_reg));

Nit: As far as I understand 4-126 Vol. 2B of [1]

Quoted 2 lines
      the 0x40 prefix (REX prefix) is optional here
      (same as implemented below for num_bits == 16).
Quoted 1 line

I think 0x40 prefix at least neededif register is from R8 - R15?

Yes, please see below.

Quoted 3 lines

I use this website to do asm/disasm experiments and did try various combinations with first 8 and later 8 registers and it seems correct results are generated.

It seems all roads lead to that web-site, I used it as well :) Today I learned that the following could be used:

Quoted 8 lines
Quoted 3 lines

echo 'movsx rax,ax' | as -o /dev/null -aln -msyntax=intel -mnaked-reg

Which opens a road to scripting experiments.

This internal tool from llvm-project may also be useful:)

llvm-mc -triple=x86_64 -show-inst -x86-asm-syntax=intel -output-asm-variant=1 <<< 'movsx rax, ax'

Thank you, this works (with -show-encoding).

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines

[1] https://cdrdv2.intel.com/v1/dl/getContent/671200

Quoted 8 lines
Quoted 4 lines

arch/x86/net/bpf_jit_comp.c static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg) · line 728

Quoted 4 lines
arch/x86/net/bpf_jit_comp.c +4 -0
+               } else if (num_bits == 16) {
+                       if (is_ereg(dst_reg) || is_ereg(src_reg))
+                               EMIT1(add_2mod(0x40, src_reg, dst_reg));
+                       EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,

Nit: Basing on the same manual I don't understand why
add_2mod(0x0f, src_reg, dst_reg) is used, '0xf' should suffice (but I tried it both ways and it works...).

From the above online assembler website.

But I will check the doc to see whether it can be simplified.

Quoted 10 lines
Quoted 10 lines

I tried all combinations of r0..r9 for 64/32-bit destinations, 32/16/8 sources [1]:
- 0x40 based prefix is generated if any of the following is true:
- dst is 64 bit
- dst is ereg
- src is ereg
- dst is 32-bit and src is 'sil' (part of 'rsi', used for r2) (!) This one is surprising and web-site shows the same results. For example movsx eax,sil is encoded as 40 0F BE C6, disassembling 0F BE C6 (w/o prefix) gives movsx eax,dh.

I think I found the place in the manual that explains situation:

3.7.2.1 Register Operands in 64-Bit Mode
Register operands in 64-bit mode can be any of the following:
- ...
- 8-bit general-purpose registers: AL, BL, CL, DL, SIL, DIL, SPL, BPL, and R8B-R15B are available using REX prefixes;
AL, BL, CL, DL, AH, BH, CH, DH are available without using REX prefixes.
- ...

Vol. 1, page 3-21
https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-vol-1-manual.pdf

Quoted 23 lines
Quoted 20 lines

- opcodes:

  - 63      64-bit dst, 32-bit src
  - 0F BF   64-bit dst, 16-bit src
  - 0F BE   64-bit dst,  8-bit src
  - 0F BF   32-bit dst, 16-bit src (same as 64-bit dst)
  - 0F BE   32-bit dst,  8-bit src (same as 64-bit dst)

Script is at [2] (it is not particularly interesting, but in case if you want to tweak it).

[1] https://gist.github.com/eddyz87/94b35fd89f023c43dd2480e196b28ea1
[2] https://gist.github.com/eddyz87/60991379c547df11d30fa91901862227

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/net/bpf_jit_comp.c +6 -0
+                             add_2reg(0xC0, src_reg, dst_reg));
+               }
+       }
+
+       *pprog = prog;
+}

[...]

#16 lore raw

PATCH v2 4/15 bpf: Support new unconditional bswap instruction

In reply to Yonghong Song

The existing 'be' and 'le' insns will do conditional bswap depends on host endianness. This patch implements unconditional bswap insns.

---
 arch/x86/net/bpf_jit_comp.c |  1 +
 kernel/bpf/core.c           | 14 ++++++++++++++
 kernel/bpf/verifier.c       |  2 +-
 3 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/net/bpf_jit_comp.c +1 -0
index a740a1a6e71d..adda5e7626b4 100644
@@ -1322,6 +1322,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 			break;
 
 		case BPF_ALU | BPF_END | BPF_FROM_BE:
+		case BPF_ALU64 | BPF_END | BPF_FROM_LE:
 			switch (imm32) {
 			case 16:
 				/* Emit 'ror %ax, 8' to swap lower 2 bytes */
kernel/bpf/core.c +14 -0
index fe648a158c9e..86bb412fee39 100644
@@ -1524,6 +1524,7 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(ALU64, DIV,  X),			\
 	INSN_3(ALU64, MOD,  X),			\
 	INSN_2(ALU64, NEG),			\
+	INSN_3(ALU64, END, TO_LE),		\
 	/*   Immediate based. */		\
 	INSN_3(ALU64, ADD,  K),			\
 	INSN_3(ALU64, SUB,  K),			\
@@ -1845,6 +1846,19 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 			break;
 		}
 		CONT;
+	ALU64_END_TO_LE:
+		switch (IMM) {
+		case 16:
+			DST = (__force u16) __swab16(DST);
+			break;
+		case 32:
+			DST = (__force u32) __swab32(DST);
+			break;
+		case 64:
+			DST = (__force u64) __swab64(DST);
+			break;
+		}
+		CONT;
 
 	/* CALL */
 	JMP_CALL:
kernel/bpf/verifier.c +1 -1
index 5fee9f24cb5e..22ba0744547b 100644
@@ -13036,7 +13036,7 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 		} else {
 			if (insn->src_reg != BPF_REG_0 || insn->off != 0 ||
 			    (insn->imm != 16 && insn->imm != 32 && insn->imm != 64) ||
-			    BPF_CLASS(insn->code) == BPF_ALU64) {
+			    (BPF_CLASS(insn->code) == BPF_ALU64 && BPF_SRC(insn->code) != BPF_K)) {
 				verbose(env, "BPF_END uses reserved fields\n");
 				return -EINVAL;
 			}
Signature · 1 line
2.34.1
maintainer
#17 lore raw

Re: [PATCH bpf-next v2 04/15] bpf: Support new unconditional bswap instruction

Quoted 5 lines · Yonghong Song
Quoted 5 lines

The existing 'be' and 'le' insns will do conditional bswap depends on host endianness. This patch implements unconditional bswap insns.

Note sure if this is important, but here is an observation:
function is_reg64() has the following code:

...
if (class == BPF_ALU64 || class == BPF_JMP || /* BPF_END always use BPF_ALU class. */
(class == BPF_ALU && op == BPF_END && insn->imm == 64)) return true;
...

It probably has to be updated but I'm not sure how:
- either check insn->imm == 64 for ALU64 as well;
- or just update the comment, given that instruction always sets all 64-bits.

Quoted 63 lines
Quoted 63 lines
---
 arch/x86/net/bpf_jit_comp.c |  1 +
 kernel/bpf/core.c           | 14 ++++++++++++++
 kernel/bpf/verifier.c       |  2 +-
 3 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/net/bpf_jit_comp.c +1 -0
index a740a1a6e71d..adda5e7626b4 100644
@@ -1322,6 +1322,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 			break;
 
 		case BPF_ALU | BPF_END | BPF_FROM_BE:
+		case BPF_ALU64 | BPF_END | BPF_FROM_LE:
 			switch (imm32) {
 			case 16:
 				/* Emit 'ror %ax, 8' to swap lower 2 bytes */
kernel/bpf/core.c +14 -0
index fe648a158c9e..86bb412fee39 100644
@@ -1524,6 +1524,7 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(ALU64, DIV,  X),			\
 	INSN_3(ALU64, MOD,  X),			\
 	INSN_2(ALU64, NEG),			\
+	INSN_3(ALU64, END, TO_LE),		\
 	/*   Immediate based. */		\
 	INSN_3(ALU64, ADD,  K),			\
 	INSN_3(ALU64, SUB,  K),			\
@@ -1845,6 +1846,19 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 			break;
 		}
 		CONT;
+	ALU64_END_TO_LE:
+		switch (IMM) {
+		case 16:
+			DST = (__force u16) __swab16(DST);
+			break;
+		case 32:
+			DST = (__force u32) __swab32(DST);
+			break;
+		case 64:
+			DST = (__force u64) __swab64(DST);
+			break;
+		}
+		CONT;
 
 	/* CALL */
 	JMP_CALL:
kernel/bpf/verifier.c +1 -1
index 5fee9f24cb5e..22ba0744547b 100644
@@ -13036,7 +13036,7 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 		} else {
 			if (insn->src_reg != BPF_REG_0 || insn->off != 0 ||
 			    (insn->imm != 16 && insn->imm != 32 && insn->imm != 64) ||
-			    BPF_CLASS(insn->code) == BPF_ALU64) {
+			    (BPF_CLASS(insn->code) == BPF_ALU64 && BPF_SRC(insn->code) != BPF_K)) {
 				verbose(env, "BPF_END uses reserved fields\n");
 				return -EINVAL;
 			}
#18 lore raw

Re: [PATCH bpf-next v2 04/15] bpf: Support new unconditional bswap instruction

Quoted 17 lines · Eduard Zingerman
Quoted 5 lines · Yonghong Song
Quoted 5 lines

The existing 'be' and 'le' insns will do conditional bswap depends on host endianness. This patch implements unconditional bswap insns.

Note sure if this is important, but here is an observation:
function is_reg64() has the following code:

  ...
  if (class == BPF_ALU64 || class == BPF_JMP ||
  /* BPF_END always use BPF_ALU class. */
  (class == BPF_ALU && op == BPF_END && insn->imm == 64))
  return true;
  ...
Quoted 3 lines

It probably has to be updated but I'm not sure how:
- either check insn->imm == 64 for ALU64 as well;
- or just update the comment, given that instruction always sets all 64-bits.

I think we can remove insn->imm == 64 from condition. But I need to double check.

Quoted 65 lines
Quoted 64 lines
Quoted 64 lines
---
  arch/x86/net/bpf_jit_comp.c |  1 +
  kernel/bpf/core.c           | 14 ++++++++++++++
  kernel/bpf/verifier.c       |  2 +-
  3 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/net/bpf_jit_comp.c +1 -0
index a740a1a6e71d..adda5e7626b4 100644
@@ -1322,6 +1322,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
  			break;
 
  		case BPF_ALU | BPF_END | BPF_FROM_BE:
+		case BPF_ALU64 | BPF_END | BPF_FROM_LE:
  			switch (imm32) {
  			case 16:
  				/* Emit 'ror %ax, 8' to swap lower 2 bytes */
kernel/bpf/core.c +14 -0
index fe648a158c9e..86bb412fee39 100644
@@ -1524,6 +1524,7 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
  	INSN_3(ALU64, DIV,  X),			\
  	INSN_3(ALU64, MOD,  X),			\
  	INSN_2(ALU64, NEG),			\
+	INSN_3(ALU64, END, TO_LE),		\
  	/*   Immediate based. */		\
  	INSN_3(ALU64, ADD,  K),			\
  	INSN_3(ALU64, SUB,  K),			\
@@ -1845,6 +1846,19 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
  			break;
  		}
  		CONT;
+	ALU64_END_TO_LE:
+		switch (IMM) {
+		case 16:
+			DST = (__force u16) __swab16(DST);
+			break;
+		case 32:
+			DST = (__force u32) __swab32(DST);
+			break;
+		case 64:
+			DST = (__force u64) __swab64(DST);
+			break;
+		}
+		CONT;
 
  	/* CALL */
  	JMP_CALL:
kernel/bpf/verifier.c +1 -1
index 5fee9f24cb5e..22ba0744547b 100644
@@ -13036,7 +13036,7 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
  		} else {
  			if (insn->src_reg != BPF_REG_0 || insn->off != 0 ||
  			    (insn->imm != 16 && insn->imm != 32 && insn->imm != 64) ||
-			    BPF_CLASS(insn->code) == BPF_ALU64) {
+			    (BPF_CLASS(insn->code) == BPF_ALU64 && BPF_SRC(insn->code) != BPF_K)) {
  				verbose(env, "BPF_END uses reserved fields\n");
  				return -EINVAL;
  			}
#19 lore raw

PATCH v2 5/15 bpf: Support new signed div/mod instructions.

In reply to Yonghong Song

Add interpreter/jit support for new signed div/mod insns. The new signed div/mod instructions are encoded with unsigned div/mod instructions plus insn->off == 1. Also add basic verifier support to ensure new insns get accepted.

---
 arch/x86/net/bpf_jit_comp.c | 27 +++++++----
 kernel/bpf/core.c           | 96 ++++++++++++++++++++++++++++++-------
 kernel/bpf/verifier.c       |  6 ++-
 3 files changed, 103 insertions(+), 26 deletions(-)
arch/x86/net/bpf_jit_comp.c +19 -8
index adda5e7626b4..3176b60d25c7 100644
@@ -1194,15 +1194,26 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 				/* mov rax, dst_reg */
 				emit_mov_reg(&prog, is64, BPF_REG_0, dst_reg);
 
-			/*
-			 * xor edx, edx
-			 * equivalent to 'xor rdx, rdx', but one byte less
-			 */
-			EMIT2(0x31, 0xd2);
+			if (insn->off == 0) {
+				/*
+				 * xor edx, edx
+				 * equivalent to 'xor rdx, rdx', but one byte less
+				 */
+				EMIT2(0x31, 0xd2);
 
-			/* div src_reg */
-			maybe_emit_1mod(&prog, src_reg, is64);
-			EMIT2(0xF7, add_1reg(0xF0, src_reg));
+				/* div src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF0, src_reg));
+			} else {
+				if (BPF_CLASS(insn->code) == BPF_ALU)
+					EMIT1(0x99); /* cltd */
+				else
+					EMIT2(0x48, 0x99); /* cqto */
+
+				/* idiv src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF8, src_reg));
+			}
 
 			if (BPF_OP(insn->code) == BPF_MOD &&
 			    dst_reg != BPF_REG_3)
kernel/bpf/core.c +80 -16

116 lines not on this page · raw message

kernel/bpf/verifier.c +4 -2
index 22ba0744547b..b606c8ed5470 100644
@@ -13192,7 +13192,8 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 	} else {	/* all other ALU ops: and, sub, xor, add, ... */
 
 		if (BPF_SRC(insn->code) == BPF_X) {
-			if (insn->imm != 0 || insn->off != 0) {
+			if (insn->imm != 0 || insn->off > 1 ||
+			    (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
 				verbose(env, "BPF_ALU uses reserved fields\n");
 				return -EINVAL;
 			}
@@ -13201,7 +13202,8 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 			if (err)
 				return err;
 		} else {
-			if (insn->src_reg != BPF_REG_0 || insn->off != 0) {
+			if (insn->src_reg != BPF_REG_0 || insn->off > 1 ||
+			    (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
 				verbose(env, "BPF_ALU uses reserved fields\n");
 				return -EINVAL;
 			}
Signature · 1 line
2.34.1
maintainer
#20 lore raw

Re: [PATCH bpf-next v2 05/15] bpf: Support new signed div/mod instructions.

Quoted 34 lines · Yonghong Song

Add interpreter/jit support for new signed div/mod insns. The new signed div/mod instructions are encoded with unsigned div/mod instructions plus insn->off == 1. Also add basic verifier support to ensure new insns get accepted.

---
 arch/x86/net/bpf_jit_comp.c | 27 +++++++----
 kernel/bpf/core.c           | 96 ++++++++++++++++++++++++++++++-------
 kernel/bpf/verifier.c       |  6 ++-
 3 files changed, 103 insertions(+), 26 deletions(-)
arch/x86/net/bpf_jit_comp.c +6 -6
index adda5e7626b4..3176b60d25c7 100644
@@ -1194,15 +1194,26 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 				/* mov rax, dst_reg */
 				emit_mov_reg(&prog, is64, BPF_REG_0, dst_reg);
 
-			/*
-			 * xor edx, edx
-			 * equivalent to 'xor rdx, rdx', but one byte less
-			 */
-			EMIT2(0x31, 0xd2);
+			if (insn->off == 0) {
+				/*
+				 * xor edx, edx
+				 * equivalent to 'xor rdx, rdx', but one byte less
+				 */
+				EMIT2(0x31, 0xd2);
 
-			/* div src_reg */

arch/x86/net/bpf_jit_comp.c static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image · line 1211

Quoted 10 lines
arch/x86/net/bpf_jit_comp.c +8 -2
-			maybe_emit_1mod(&prog, src_reg, is64);
-			EMIT2(0xF7, add_1reg(0xF0, src_reg));
+				/* div src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF0, src_reg));
+			} else {
+				if (BPF_CLASS(insn->code) == BPF_ALU)
+					EMIT1(0x99); /* cltd */
+				else
+					EMIT2(0x48, 0x99); /* cqto */

Nitpick: I can't find names cltd/cqto in the Intel instruction manual,
instead it uses names cdq/cqo for these instructions. (See Vol. 2A pages 3-315 and 3-497)

Quoted 153 lines
arch/x86/net/bpf_jit_comp.c +5 -0
+
+				/* idiv src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF8, src_reg));
+			}
 
 			if (BPF_OP(insn->code) == BPF_MOD &&
 			    dst_reg != BPF_REG_3)
kernel/bpf/core.c +80 -16
index 86bb412fee39..6f7134657935 100644
@@ -1789,36 +1789,100 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 		(*(s64 *) &DST) >>= IMM;
 		CONT;
 	ALU64_MOD_X:
-		div64_u64_rem(DST, SRC, &AX);
-		DST = AX;
+		switch (OFF) {
+		case 0:
+			div64_u64_rem(DST, SRC, &AX);
+			DST = AX;
+			break;
+		case 1:
+			AX = div64_s64(DST, SRC);
+			DST = DST - AX * SRC;
+			break;
+		}
 		CONT;
 	ALU_MOD_X:
-		AX = (u32) DST;
-		DST = do_div(AX, (u32) SRC);
+		switch (OFF) {
+		case 0:
+			AX = (u32) DST;
+			DST = do_div(AX, (u32) SRC);
+			break;
+		case 1:
+			AX = (s32) DST;
+			DST = (u32)do_div(AX, (s32) SRC);
+			break;
+		}
 		CONT;
 	ALU64_MOD_K:
-		div64_u64_rem(DST, IMM, &AX);
-		DST = AX;
+		switch (OFF) {
+		case 0:
+			div64_u64_rem(DST, IMM, &AX);
+			DST = AX;
+			break;
+		case 1:
+			AX = div64_s64(DST, IMM);
+			DST = DST - AX * IMM;
+			break;
+		}
 		CONT;
 	ALU_MOD_K:
-		AX = (u32) DST;
-		DST = do_div(AX, (u32) IMM);
+		switch (OFF) {
+		case 0:
+			AX = (u32) DST;
+			DST = do_div(AX, (u32) IMM);
+			break;
+		case 1:
+			AX = (s32) DST;
+			DST = (u32)do_div(AX, (s32) IMM);
+			break;
+		}
 		CONT;
 	ALU64_DIV_X:
-		DST = div64_u64(DST, SRC);
+		switch (OFF) {
+		case 0:
+			DST = div64_u64(DST, SRC);
+			break;
+		case 1:
+			DST = div64_s64(DST, SRC);
+			break;
+		}
 		CONT;
 	ALU_DIV_X:
-		AX = (u32) DST;
-		do_div(AX, (u32) SRC);
-		DST = (u32) AX;
+		switch (OFF) {
+		case 0:
+			AX = (u32) DST;
+			do_div(AX, (u32) SRC);
+			DST = (u32) AX;
+			break;
+		case 1:
+			AX = (s32) DST;
+			do_div(AX, (s32) SRC);
+			DST = (u32) AX;
+			break;
+		}
 		CONT;
 	ALU64_DIV_K:
-		DST = div64_u64(DST, IMM);
+		switch (OFF) {
+		case 0:
+			DST = div64_u64(DST, IMM);
+			break;
+		case 1:
+			DST = div64_s64(DST, IMM);
+			break;
+		}
 		CONT;
 	ALU_DIV_K:
-		AX = (u32) DST;
-		do_div(AX, (u32) IMM);
-		DST = (u32) AX;
+		switch (OFF) {
+		case 0:
+			AX = (u32) DST;
+			do_div(AX, (u32) IMM);
+			DST = (u32) AX;
+			break;
+		case 1:
+			AX = (s32) DST;
+			do_div(AX, (s32) IMM);
+			DST = (u32) AX;
+			break;
+		}
 		CONT;
 	ALU_END_TO_BE:
 		switch (IMM) {
kernel/bpf/verifier.c +4 -2
index 22ba0744547b..b606c8ed5470 100644
@@ -13192,7 +13192,8 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 	} else {	/* all other ALU ops: and, sub, xor, add, ... */
 
 		if (BPF_SRC(insn->code) == BPF_X) {
-			if (insn->imm != 0 || insn->off != 0) {
+			if (insn->imm != 0 || insn->off > 1 ||
+			    (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
 				verbose(env, "BPF_ALU uses reserved fields\n");
 				return -EINVAL;
 			}
@@ -13201,7 +13202,8 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 			if (err)
 				return err;
 		} else {
-			if (insn->src_reg != BPF_REG_0 || insn->off != 0) {
+			if (insn->src_reg != BPF_REG_0 || insn->off > 1 ||
+			    (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
 				verbose(env, "BPF_ALU uses reserved fields\n");
 				return -EINVAL;
 			}
#21 lore raw

Re: [PATCH bpf-next v2 05/15] bpf: Support new signed div/mod instructions.

Quoted 34 lines · Eduard Zingerman
Quoted 33 lines · Yonghong Song

Add interpreter/jit support for new signed div/mod insns. The new signed div/mod instructions are encoded with unsigned div/mod instructions plus insn->off == 1. Also add basic verifier support to ensure new insns get accepted.

---
  arch/x86/net/bpf_jit_comp.c | 27 +++++++----
  kernel/bpf/core.c           | 96 ++++++++++++++++++++++++++++++-------
  kernel/bpf/verifier.c       |  6 ++-
  3 files changed, 103 insertions(+), 26 deletions(-)
arch/x86/net/bpf_jit_comp.c +6 -5
index adda5e7626b4..3176b60d25c7 100644
@@ -1194,15 +1194,26 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
  				/* mov rax, dst_reg */
  				emit_mov_reg(&prog, is64, BPF_REG_0, dst_reg);
 
-			/*
-			 * xor edx, edx
-			 * equivalent to 'xor rdx, rdx', but one byte less
-			 */
-			EMIT2(0x31, 0xd2);
+			if (insn->off == 0) {
+				/*
+				 * xor edx, edx
+				 * equivalent to 'xor rdx, rdx', but one byte less
+				 */
+				EMIT2(0x31, 0xd2);
 
Quoted 15 lines

arch/x86/net/bpf_jit_comp.c static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image · line 1211

Quoted 11 lines
arch/x86/net/bpf_jit_comp.c +8 -3
-			/* div src_reg */
-			maybe_emit_1mod(&prog, src_reg, is64);
-			EMIT2(0xF7, add_1reg(0xF0, src_reg));
+				/* div src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF0, src_reg));
+			} else {
+				if (BPF_CLASS(insn->code) == BPF_ALU)
+					EMIT1(0x99); /* cltd */
+				else
+					EMIT2(0x48, 0x99); /* cqto */

Nitpick: I can't find names cltd/cqto in the Intel instruction manual,
instead it uses names cdq/cqo for these instructions. (See Vol. 2A pages 3-315 and 3-497)

I got these asm names from
https://defuse.ca/online-x86-assembler.htm
I will check the Intel insn manual and make the change accordingly.

Quoted 9 lines
Quoted 9 lines
arch/x86/net/bpf_jit_comp.c +5 -0
+
+				/* idiv src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF8, src_reg));
+			}
 
  			if (BPF_OP(insn->code) == BPF_MOD &&
  			    dst_reg != BPF_REG_3)

[...]

#22 lore raw

Re: [PATCH bpf-next v2 05/15] bpf: Support new signed div/mod instructions.

Quoted 54 lines · Yonghong Song
Quoted 34 lines · Eduard Zingerman
Quoted 33 lines · Yonghong Song

Add interpreter/jit support for new signed div/mod insns. The new signed div/mod instructions are encoded with unsigned div/mod instructions plus insn->off == 1. Also add basic verifier support to ensure new insns get accepted.

---
  arch/x86/net/bpf_jit_comp.c | 27 +++++++----
  kernel/bpf/core.c           | 96 ++++++++++++++++++++++++++++++-------
  kernel/bpf/verifier.c       |  6 ++-
  3 files changed, 103 insertions(+), 26 deletions(-)
arch/x86/net/bpf_jit_comp.c +6 -5
index adda5e7626b4..3176b60d25c7 100644
@@ -1194,15 +1194,26 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
                             /* mov rax, dst_reg */
                             emit_mov_reg(&prog, is64, BPF_REG_0, dst_reg);
 
-                    /*
-                     * xor edx, edx
-                     * equivalent to 'xor rdx, rdx', but one byte less
-                     */
-                    EMIT2(0x31, 0xd2);
+                    if (insn->off == 0) {
+                            /*
+                             * xor edx, edx
+                             * equivalent to 'xor rdx, rdx', but one byte less
+                             */
+                            EMIT2(0x31, 0xd2);
 
Quoted 15 lines

arch/x86/net/bpf_jit_comp.c static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image · line 1211

Quoted 11 lines
arch/x86/net/bpf_jit_comp.c +8 -3
-                    /* div src_reg */
-                    maybe_emit_1mod(&prog, src_reg, is64);
-                    EMIT2(0xF7, add_1reg(0xF0, src_reg));
+                            /* div src_reg */
+                            maybe_emit_1mod(&prog, src_reg, is64);
+                            EMIT2(0xF7, add_1reg(0xF0, src_reg));
+                    } else {
+                            if (BPF_CLASS(insn->code) == BPF_ALU)
+                                    EMIT1(0x99); /* cltd */
+                            else
+                                    EMIT2(0x48, 0x99); /* cqto */

Nitpick: I can't find names cltd/cqto in the Intel instruction manual,
instead it uses names cdq/cqo for these instructions. (See Vol. 2A pages 3-315 and 3-497)

Quoted 4 lines

I got these asm names from
https://defuse.ca/online-x86-assembler.htm
I will check the Intel insn manual and make the change accordingly.

Heh. I've been using the same.
Most of the comments in the x86 JIT code are from there :)

and it actually returns 99 -> cdq, 4899 -> cqo

cltd/cqto must be aliases ?

#23 lore raw

Re: [PATCH bpf-next v2 05/15] bpf: Support new signed div/mod instructions.

Quoted 60 lines · Alexei Starovoitov
Quoted 54 lines · Yonghong Song
Quoted 34 lines · Eduard Zingerman
Quoted 33 lines · Yonghong Song

Add interpreter/jit support for new signed div/mod insns. The new signed div/mod instructions are encoded with unsigned div/mod instructions plus insn->off == 1. Also add basic verifier support to ensure new insns get accepted.

---
   arch/x86/net/bpf_jit_comp.c | 27 +++++++----
   kernel/bpf/core.c           | 96 ++++++++++++++++++++++++++++++-------
   kernel/bpf/verifier.c       |  6 ++-
   3 files changed, 103 insertions(+), 26 deletions(-)
arch/x86/net/bpf_jit_comp.c +6 -5
index adda5e7626b4..3176b60d25c7 100644
@@ -1194,15 +1194,26 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
                              /* mov rax, dst_reg */
                              emit_mov_reg(&prog, is64, BPF_REG_0, dst_reg);
 
-                    /*
-                     * xor edx, edx
-                     * equivalent to 'xor rdx, rdx', but one byte less
-                     */
-                    EMIT2(0x31, 0xd2);
+                    if (insn->off == 0) {
+                            /*
+                             * xor edx, edx
+                             * equivalent to 'xor rdx, rdx', but one byte less
+                             */
+                            EMIT2(0x31, 0xd2);
 
Quoted 15 lines

arch/x86/net/bpf_jit_comp.c static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image · line 1211

Quoted 11 lines
arch/x86/net/bpf_jit_comp.c +8 -3
-                    /* div src_reg */
-                    maybe_emit_1mod(&prog, src_reg, is64);
-                    EMIT2(0xF7, add_1reg(0xF0, src_reg));
+                            /* div src_reg */
+                            maybe_emit_1mod(&prog, src_reg, is64);
+                            EMIT2(0xF7, add_1reg(0xF0, src_reg));
+                    } else {
+                            if (BPF_CLASS(insn->code) == BPF_ALU)
+                                    EMIT1(0x99); /* cltd */
+                            else
+                                    EMIT2(0x48, 0x99); /* cqto */

Nitpick: I can't find names cltd/cqto in the Intel instruction manual,
instead it uses names cdq/cqo for these instructions. (See Vol. 2A pages 3-315 and 3-497)

Quoted 4 lines

I got these asm names from
https://defuse.ca/online-x86-assembler.htm
I will check the Intel insn manual and make the change accordingly.

Quoted 6 lines

Heh. I've been using the same.
Most of the comments in the x86 JIT code are from there :)

and it actually returns 99 -> cdq, 4899 -> cqo

cltd/cqto must be aliases ?

Yes, cltd/cqto are aliases.

From llvm-project repo:

[~/work/llvm-project/llvm/lib/Target/X86 (ni23)]$ egrep -r cltd
egrep: warning: egrep is obsolescent; using grep -E
README.txt: cltd
X86InstrAsmAlias.td:def : MnemonicAlias<"cdq", "cltd", "att">;
X86InstrExtension.td: "{cltd|cdq}", []>, OpSize32, Sched<[WriteALU]>;

[~/work/llvm-project/llvm/lib/Target/X86 (ni23)]$ egrep -r cqto
egrep: warning: egrep is obsolescent; using grep -E X86InstrAsmAlias.td:def : MnemonicAlias<"cqo", "cqto", "att">;
X86InstrExtension.td: "{cqto|cqo}", []>, Sched<[WriteALU]>, Requires<[In64BitMode]>;

I will change to use cdq and cgo.

#24 lore raw

PATCH v2 6/15 bpf: Fix jit blinding with new sdiv/smov insns

In reply to Yonghong Song

Handle new insns properly in bpf_jit_blind_insn() function.

---
 include/linux/filter.h | 14 ++++++++++----
 kernel/bpf/core.c      |  4 ++--
 2 files changed, 12 insertions(+), 6 deletions(-)
include/linux/filter.h +10 -4
index f69114083ec7..7a0144d66dfb 100644
@@ -90,22 +90,28 @@ struct ctl_table_header;
 
 /* ALU ops on registers, bpf_add|sub|...: dst_reg += src_reg */
 
-#define BPF_ALU64_REG(OP, DST, SRC)				\
+#define BPF_ALU64_REG_OFF(OP, DST, SRC, OFF)			\
 	((struct bpf_insn) {					\
 		.code  = BPF_ALU64 | BPF_OP(OP) | BPF_X,	\
 		.dst_reg = DST,					\
 		.src_reg = SRC,					\
-		.off   = 0,					\
+		.off   = OFF,					\
 		.imm   = 0 })
 
-#define BPF_ALU32_REG(OP, DST, SRC)				\
+#define BPF_ALU64_REG(OP, DST, SRC)				\
+	BPF_ALU64_REG_OFF(OP, DST, SRC, 0)
+
+#define BPF_ALU32_REG_OFF(OP, DST, SRC, OFF)			\
 	((struct bpf_insn) {					\
 		.code  = BPF_ALU | BPF_OP(OP) | BPF_X,		\
 		.dst_reg = DST,					\
 		.src_reg = SRC,					\
-		.off   = 0,					\
+		.off   = OFF,					\
 		.imm   = 0 })
 
+#define BPF_ALU32_REG(OP, DST, SRC)				\
+	BPF_ALU32_REG_OFF(OP, DST, SRC, 0)
+
 /* ALU ops on immediates, bpf_add|sub|...: dst_reg += imm32 */
 
 #define BPF_ALU64_IMM(OP, DST, IMM)				\
kernel/bpf/core.c +2 -2
index 6f7134657935..c75391ac575e 100644
@@ -1272,7 +1272,7 @@ static int bpf_jit_blind_insn(const struct bpf_insn *from,
 	case BPF_ALU | BPF_MOD | BPF_K:
 		*to++ = BPF_ALU32_IMM(BPF_MOV, BPF_REG_AX, imm_rnd ^ from->imm);
 		*to++ = BPF_ALU32_IMM(BPF_XOR, BPF_REG_AX, imm_rnd);
-		*to++ = BPF_ALU32_REG(from->code, from->dst_reg, BPF_REG_AX);
+		*to++ = BPF_ALU32_REG_OFF(from->code, from->dst_reg, BPF_REG_AX, from->off);
 		break;
 
 	case BPF_ALU64 | BPF_ADD | BPF_K:
@@ -1286,7 +1286,7 @@ static int bpf_jit_blind_insn(const struct bpf_insn *from,
 	case BPF_ALU64 | BPF_MOD | BPF_K:
 		*to++ = BPF_ALU64_IMM(BPF_MOV, BPF_REG_AX, imm_rnd ^ from->imm);
 		*to++ = BPF_ALU64_IMM(BPF_XOR, BPF_REG_AX, imm_rnd);
-		*to++ = BPF_ALU64_REG(from->code, from->dst_reg, BPF_REG_AX);
+		*to++ = BPF_ALU64_REG_OFF(from->code, from->dst_reg, BPF_REG_AX, from->off);
 		break;
 
 	case BPF_JMP | BPF_JEQ  | BPF_K:
Signature · 1 line
2.34.1
#25 lore raw

PATCH v2 7/15 bpf: Support new 32bit offset jmp instruction

In reply to Yonghong Song

Add interpreter/jit/verifier support for 32bit offset jmp instruction. If a conditional jmp instruction needs more than 16bit offset, it can be simulated with a conditional jmp + a 32bit jmp insn.

---
 arch/x86/net/bpf_jit_comp.c | 28 ++++++++++++++++++----------
 kernel/bpf/core.c           | 19 ++++++++++++++++---
 kernel/bpf/verifier.c       | 32 ++++++++++++++++++++++----------
 3 files changed, 56 insertions(+), 23 deletions(-)
arch/x86/net/bpf_jit_comp.c +18 -10
index 3176b60d25c7..db2791f3b003 100644
@@ -1809,16 +1809,24 @@ st:			if (is_imm8(insn->off))
 			break;
 
 		case BPF_JMP | BPF_JA:
-			if (insn->off == -1)
-				/* -1 jmp instructions will always jump
-				 * backwards two bytes. Explicitly handling
-				 * this case avoids wasting too many passes
-				 * when there are long sequences of replaced
-				 * dead code.
-				 */
-				jmp_offset = -2;
-			else
-				jmp_offset = addrs[i + insn->off] - addrs[i];
+		case BPF_JMP32 | BPF_JA:
+			if (BPF_CLASS(insn->code) == BPF_JMP) {
+				if (insn->off == -1)
+					/* -1 jmp instructions will always jump
+					 * backwards two bytes. Explicitly handling
+					 * this case avoids wasting too many passes
+					 * when there are long sequences of replaced
+					 * dead code.
+					 */
+					jmp_offset = -2;
+				else
+					jmp_offset = addrs[i + insn->off] - addrs[i];
+			} else {
+				if (insn->imm == -1)
+					jmp_offset = -2;
+				else
+					jmp_offset = addrs[i + insn->imm] - addrs[i];
+			}
 
 			if (!jmp_offset) {
 				/*
kernel/bpf/core.c +16 -3
index c75391ac575e..019d186ab206 100644
@@ -373,7 +373,12 @@ static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
 {
 	const s32 off_min = S16_MIN, off_max = S16_MAX;
 	s32 delta = end_new - end_old;
-	s32 off = insn->off;
+	s32 off;
+
+	if (insn->code == (BPF_JMP32 | BPF_JA))
+		off = insn->imm;
+	else
+		off = insn->off;
 
 	if (curr < pos && curr + off + 1 >= end_old)
 		off += delta;
@@ -381,8 +386,12 @@ static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
 		off -= delta;
 	if (off < off_min || off > off_max)
 		return -ERANGE;
-	if (!probe_pass)
-		insn->off = off;
+	if (!probe_pass) {
+		if (insn->code == (BPF_JMP32 | BPF_JA))
+			insn->imm = off;
+		else
+			insn->off = off;
+	}
 	return 0;
 }
 
@@ -1593,6 +1602,7 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(JMP, JSLE, K),			\
 	INSN_3(JMP, JSET, K),			\
 	INSN_2(JMP, JA),			\
+	INSN_2(JMP32, JA),			\
 	/* Store instructions. */		\
 	/*   Register based. */			\
 	INSN_3(STX, MEM,  B),			\
@@ -1972,6 +1982,9 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	JMP_JA:
 		insn += insn->off;
 		CONT;
+	JMP32_JA:
+		insn += insn->imm;
+		CONT;
 	JMP_EXIT:
 		return BPF_R0;
 	/* JMP */
kernel/bpf/verifier.c +22 -10
index b606c8ed5470..eaf95dfad80a 100644
@@ -2855,7 +2855,10 @@ static int check_subprogs(struct bpf_verifier_env *env)
 			goto next;
 		if (BPF_OP(code) == BPF_EXIT || BPF_OP(code) == BPF_CALL)
 			goto next;
-		off = i + insn[i].off + 1;
+		if (code == (BPF_JMP32 | BPF_JA))
+			off = i + insn[i].imm + 1;
+		else
+			off = i + insn[i].off + 1;
 		if (off < subprog_start || off >= subprog_end) {
 			verbose(env, "jump out of range from insn %d to %d\n", i, off);
 			return -EINVAL;
@@ -2867,6 +2870,7 @@ static int check_subprogs(struct bpf_verifier_env *env)
 			 * or unconditional jump back
 			 */
 			if (code != (BPF_JMP | BPF_EXIT) &&
+			    code != (BPF_JMP32 | BPF_JA) &&
 			    code != (BPF_JMP | BPF_JA)) {
 				verbose(env, "last insn is not an exit or jmp\n");
 				return -EINVAL;
@@ -14747,7 +14751,7 @@ static int visit_func_call_insn(int t, struct bpf_insn *insns,
 static int visit_insn(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_insn *insns = env->prog->insnsi, *insn = &insns[t];
-	int ret;
+	int ret, off;
 
 	if (bpf_pseudo_func(insn))
 		return visit_func_call_insn(t, insns, env, true);
@@ -14795,14 +14799,19 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
 		if (BPF_SRC(insn->code) != BPF_K)
 			return -EINVAL;
 
+		if (BPF_CLASS(insn->code) == BPF_JMP)
+			off = insn->off;
+		else
+			off = insn->imm;
+
 		/* unconditional jump with single edge */
-		ret = push_insn(t, t + insn->off + 1, FALLTHROUGH, env,
+		ret = push_insn(t, t + off + 1, FALLTHROUGH, env,
 				true);
 		if (ret)
 			return ret;
 
-		mark_prune_point(env, t + insn->off + 1);
-		mark_jmp_point(env, t + insn->off + 1);
+		mark_prune_point(env, t + off + 1);
+		mark_jmp_point(env, t + off + 1);
 
 		return ret;
 
@@ -16598,15 +16607,18 @@ static int do_check(struct bpf_verifier_env *env)
 				mark_reg_scratched(env, BPF_REG_0);
 			} else if (opcode == BPF_JA) {
 				if (BPF_SRC(insn->code) != BPF_K ||
-				    insn->imm != 0 ||
 				    insn->src_reg != BPF_REG_0 ||
 				    insn->dst_reg != BPF_REG_0 ||
-				    class == BPF_JMP32) {
+				    (class == BPF_JMP && insn->imm != 0) ||
+				    (class == BPF_JMP32 && insn->off != 0)) {
 					verbose(env, "BPF_JA uses reserved fields\n");
 					return -EINVAL;
 				}
 
-				env->insn_idx += insn->off + 1;
+				if (class == BPF_JMP)
+					env->insn_idx += insn->off + 1;
+				else
+					env->insn_idx += insn->imm + 1;
 				continue;
 
 			} else if (opcode == BPF_EXIT) {
@@ -17453,13 +17465,13 @@ static bool insn_is_cond_jump(u8 code)
 {
 	u8 op;
 
+	op = BPF_OP(code);
 	if (BPF_CLASS(code) == BPF_JMP32)
-		return true;
+		return op != BPF_JA;
 
 	if (BPF_CLASS(code) != BPF_JMP)
 		return false;
 
-	op = BPF_OP(code);
 	return op != BPF_JA && op != BPF_EXIT && op != BPF_CALL;
 }
 
Signature · 1 line
2.34.1
#26 lore raw

PATCH v2 8/15 selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing

In reply to Yonghong Song

Similar to no-alu32 runner, a cpuv4 runner is created to test bpf programs compiled with -mcpu=v4.

The following are some num-of-insn statistics for each newer instructions, excluding naked asm (verifier_*) tests:

   insn pattern                # of instructions
   reg = (s8)reg               4
   reg = (s16)reg              2
   reg = (s32)reg              26
   reg = *(s8 *)(reg + off)    11
   reg = *(s16 *)(reg + off)   14
   reg = *(s32 *)(reg + off)   15214
   reg = bswap16 reg           133
   reg = bswap32 reg           38
   reg = bswap64 reg           14
   reg s/= reg                 0
   reg s%= reg                 0
   gotol <offset>              58

Note that in llvm -mcpu=v4 implementation, the compiler is a little bit conservative about generating 'gotol' insn (32-bit branch offset) as it didn't precise count the number of insns (e.g., some insns are debug insns, etc.). Compared to old 'goto' insn, newer 'gotol' insn should have comparable verification states to 'goto' insn.

I did not collect verifier stats now since I have not really started to do proper range bound estimation with these instructions.

With current patch set, all selftests passed with -mcpu=v4 when running test_progs-cpuv4 binary.

---
 tools/testing/selftests/bpf/.gitignore |  2 ++
 tools/testing/selftests/bpf/Makefile   | 18 ++++++++++++++----
 2 files changed, 16 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/.gitignore +2 -0
index 116fecf80ca1..110518ba4804 100644
@@ -13,6 +13,7 @@ test_dev_cgroup
 /test_progs
 /test_progs-no_alu32
 /test_progs-bpf_gcc
+/test_progs-cpuv4
 test_verifier_log
 feature
 test_sock
@@ -36,6 +37,7 @@ test_cpp
 *.lskel.h
 /no_alu32
 /bpf_gcc
+/cpuv4
 /host-tools
 /tools
 /runqslower
tools/testing/selftests/bpf/Makefile +14 -4
index 882be03b179f..4b2cf5d40120 100644
@@ -44,7 +44,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
 	test_sock test_sockmap get_cgroup_id_user \
 	test_cgroup_storage \
 	test_tcpnotify_user test_sysctl \
-	test_progs-no_alu32
+	test_progs-no_alu32 test_progs-cpuv4
 
 # Also test bpf-gcc, if present
 ifneq ($(BPF_GCC),)
@@ -383,6 +383,11 @@ define CLANG_NOALU32_BPF_BUILD_RULE
 	$(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
 	$(Q)$(CLANG) $3 -O2 --target=bpf -c $1 -mcpu=v2 -o $2
 endef
+# Similar to CLANG_BPF_BUILD_RULE, but with cpu-v4
+define CLANG_CPUV4_BPF_BUILD_RULE
+	$(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
+	$(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v4 -o $2
+endef
 # Build BPF object using GCC
 define GCC_BPF_BUILD_RULE
 	$(call msg,GCC-BPF,$(TRUNNER_BINARY),$2)
@@ -425,7 +430,7 @@ LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.c,$(foreach skel,$(LINKED_SKELS),$($(ske
 # $eval()) and pass control to DEFINE_TEST_RUNNER_RULES.
 # Parameters:
 # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
 define DEFINE_TEST_RUNNER
 
 TRUNNER_OUTPUT := $(OUTPUT)$(if $2,/)$2
@@ -453,7 +458,7 @@ endef
 # Using TRUNNER_XXX variables, provided by callers of DEFINE_TEST_RUNNER and
 # set up by DEFINE_TEST_RUNNER itself, create test runner build rules with:
 # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
 define DEFINE_TEST_RUNNER_RULES
 
 ifeq ($($(TRUNNER_OUTPUT)-dir),)
@@ -584,6 +589,11 @@ TRUNNER_BPF_BUILD_RULE := CLANG_NOALU32_BPF_BUILD_RULE
 TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
 $(eval $(call DEFINE_TEST_RUNNER,test_progs,no_alu32))
 
+# Define test_progs-cpuv4 test runner.
+TRUNNER_BPF_BUILD_RULE := CLANG_CPUV4_BPF_BUILD_RULE
+TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
+$(eval $(call DEFINE_TEST_RUNNER,test_progs,cpuv4))
+
 # Define test_progs BPF-GCC-flavored test runner.
 ifneq ($(BPF_GCC),)
 TRUNNER_BPF_BUILD_RULE := GCC_BPF_BUILD_RULE
@@ -681,7 +691,7 @@ EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)	\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
 	feature bpftool							\
 	$(addprefix $(OUTPUT)/,*.o *.skel.h *.lskel.h *.subskel.h	\
-			       no_alu32 bpf_gcc bpf_testmod.ko		\
+			       no_alu32 cpuv4 bpf_gcc bpf_testmod.ko	\
 			       liburandom_read.so)
 
 .PHONY: docs docs-clean
Signature · 1 line
2.34.1
#27 lore raw

Re: [PATCH bpf-next v2 08/15] selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing

Quoted 73 lines · Yonghong Song

Similar to no-alu32 runner, a cpuv4 runner is created to test bpf programs compiled with -mcpu=v4.

The following are some num-of-insn statistics for each newer instructions, excluding naked asm (verifier_*) tests:

   insn pattern                # of instructions
   reg = (s8)reg               4
   reg = (s16)reg              2
   reg = (s32)reg              26
   reg = *(s8 *)(reg + off)    11
   reg = *(s16 *)(reg + off)   14
   reg = *(s32 *)(reg + off)   15214
   reg = bswap16 reg           133
   reg = bswap32 reg           38
   reg = bswap64 reg           14
   reg s/= reg                 0
   reg s%= reg                 0
   gotol <offset>              58

Note that in llvm -mcpu=v4 implementation, the compiler is a little bit conservative about generating 'gotol' insn (32-bit branch offset) as it didn't precise count the number of insns (e.g., some insns are debug insns, etc.). Compared to old 'goto' insn, newer 'gotol' insn should have comparable verification states to 'goto' insn.

I did not collect verifier stats now since I have not really started to do proper range bound estimation with these instructions.

With current patch set, all selftests passed with -mcpu=v4 when running test_progs-cpuv4 binary.

---
 tools/testing/selftests/bpf/.gitignore |  2 ++
 tools/testing/selftests/bpf/Makefile   | 18 ++++++++++++++----
 2 files changed, 16 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/.gitignore +2 -0
index 116fecf80ca1..110518ba4804 100644
@@ -13,6 +13,7 @@ test_dev_cgroup
 /test_progs
 /test_progs-no_alu32
 /test_progs-bpf_gcc
+/test_progs-cpuv4
 test_verifier_log
 feature
 test_sock
@@ -36,6 +37,7 @@ test_cpp
 *.lskel.h
 /no_alu32
 /bpf_gcc
+/cpuv4
 /host-tools
 /tools
 /runqslower
tools/testing/selftests/bpf/Makefile +1 -1
index 882be03b179f..4b2cf5d40120 100644
@@ -44,7 +44,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
        test_sock test_sockmap get_cgroup_id_user \
        test_cgroup_storage \
        test_tcpnotify_user test_sysctl \
-       test_progs-no_alu32
+       test_progs-no_alu32 test_progs-cpuv4
 
 # Also test bpf-gcc, if present
 ifneq ($(BPF_GCC),)

tools/testing/selftests/bpf/Makefile define CLANG_NOALU32_BPF_BUILD_RULE · line 390

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +5 -0
@@ -383,6 +383,11 @@ define CLANG_NOALU32_BPF_BUILD_RULE
        $(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
        $(Q)$(CLANG) $3 -O2 --target=bpf -c $1 -mcpu=v2 -o $2
 endef
+# Similar to CLANG_BPF_BUILD_RULE, but with cpu-v4
+define CLANG_CPUV4_BPF_BUILD_RULE
+       $(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
+       $(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v4 -o $2
+endef

Use --target= (Clang 3.4 preferred form) for new code :)

Quoted 45 lines
tools/testing/selftests/bpf/Makefile +8 -4
 # Build BPF object using GCC
 define GCC_BPF_BUILD_RULE
        $(call msg,GCC-BPF,$(TRUNNER_BINARY),$2)
@@ -425,7 +430,7 @@ LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.c,$(foreach skel,$(LINKED_SKELS),$($(ske
 # $eval()) and pass control to DEFINE_TEST_RUNNER_RULES.
 # Parameters:
 # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
 define DEFINE_TEST_RUNNER
 
 TRUNNER_OUTPUT := $(OUTPUT)$(if $2,/)$2
@@ -453,7 +458,7 @@ endef
 # Using TRUNNER_XXX variables, provided by callers of DEFINE_TEST_RUNNER and
 # set up by DEFINE_TEST_RUNNER itself, create test runner build rules with:
 # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
 define DEFINE_TEST_RUNNER_RULES
 
 ifeq ($($(TRUNNER_OUTPUT)-dir),)
@@ -584,6 +589,11 @@ TRUNNER_BPF_BUILD_RULE := CLANG_NOALU32_BPF_BUILD_RULE
 TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
 $(eval $(call DEFINE_TEST_RUNNER,test_progs,no_alu32))
 
+# Define test_progs-cpuv4 test runner.
+TRUNNER_BPF_BUILD_RULE := CLANG_CPUV4_BPF_BUILD_RULE
+TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
+$(eval $(call DEFINE_TEST_RUNNER,test_progs,cpuv4))
+
 # Define test_progs BPF-GCC-flavored test runner.
 ifneq ($(BPF_GCC),)
 TRUNNER_BPF_BUILD_RULE := GCC_BPF_BUILD_RULE
@@ -681,7 +691,7 @@ EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)      \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool                                                 \
        $(addprefix $(OUTPUT)/,*.o *.skel.h *.lskel.h *.subskel.h       \
-                              no_alu32 bpf_gcc bpf_testmod.ko          \
+                              no_alu32 cpuv4 bpf_gcc bpf_testmod.ko    \
                               liburandom_read.so)
 
 .PHONY: docs docs-clean
--

2.34.1

Signature · 1 line
宋方睿
#28 lore raw

Re: [PATCH bpf-next v2 08/15] selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing

Quoted 71 lines · Fangrui Song
Quoted 70 lines · Yonghong Song

Similar to no-alu32 runner, a cpuv4 runner is created to test bpf programs compiled with -mcpu=v4.

The following are some num-of-insn statistics for each newer instructions, excluding naked asm (verifier_*) tests:

    insn pattern                # of instructions
    reg = (s8)reg               4
    reg = (s16)reg              2
    reg = (s32)reg              26
    reg = *(s8 *)(reg + off)    11
    reg = *(s16 *)(reg + off)   14
    reg = *(s32 *)(reg + off)   15214
    reg = bswap16 reg           133
    reg = bswap32 reg           38
    reg = bswap64 reg           14
    reg s/= reg                 0
    reg s%= reg                 0
    gotol <offset>              58

Note that in llvm -mcpu=v4 implementation, the compiler is a little bit conservative about generating 'gotol' insn (32-bit branch offset) as it didn't precise count the number of insns (e.g., some insns are debug insns, etc.). Compared to old 'goto' insn, newer 'gotol' insn should have comparable verification states to 'goto' insn.

I did not collect verifier stats now since I have not really started to do proper range bound estimation with these instructions.

With current patch set, all selftests passed with -mcpu=v4 when running test_progs-cpuv4 binary.

---
  tools/testing/selftests/bpf/.gitignore |  2 ++
  tools/testing/selftests/bpf/Makefile   | 18 ++++++++++++++----
  2 files changed, 16 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/.gitignore +2 -0
index 116fecf80ca1..110518ba4804 100644
@@ -13,6 +13,7 @@ test_dev_cgroup
  /test_progs
  /test_progs-no_alu32
  /test_progs-bpf_gcc
+/test_progs-cpuv4
  test_verifier_log
  feature
  test_sock
@@ -36,6 +37,7 @@ test_cpp
  *.lskel.h
  /no_alu32
  /bpf_gcc
+/cpuv4
  /host-tools
  /tools
  /runqslower
tools/testing/selftests/bpf/Makefile +1 -1
index 882be03b179f..4b2cf5d40120 100644
@@ -44,7 +44,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
         test_sock test_sockmap get_cgroup_id_user \
         test_cgroup_storage \
         test_tcpnotify_user test_sysctl \
-       test_progs-no_alu32
+       test_progs-no_alu32 test_progs-cpuv4
 
Quoted 13 lines

tools/testing/selftests/bpf/Makefile define CLANG_NOALU32_BPF_BUILD_RULE · line 390

Quoted 11 lines
tools/testing/selftests/bpf/Makefile +5 -0
  # Also test bpf-gcc, if present
  ifneq ($(BPF_GCC),)
@@ -383,6 +383,11 @@ define CLANG_NOALU32_BPF_BUILD_RULE
         $(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
         $(Q)$(CLANG) $3 -O2 --target=bpf -c $1 -mcpu=v2 -o $2
  endef
+# Similar to CLANG_BPF_BUILD_RULE, but with cpu-v4
+define CLANG_CPUV4_BPF_BUILD_RULE
+       $(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
+       $(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v4 -o $2
+endef

Use --target= (Clang 3.4 preferred form) for new code :)

Thanks for reminding me this. Will fix in the next revision. Using '-target bpf' for too long so forgot '--target=bpf' although just learned it a few weeks ago.

Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/bpf/Makefile +1 -1
  # Build BPF object using GCC
  define GCC_BPF_BUILD_RULE
         $(call msg,GCC-BPF,$(TRUNNER_BINARY),$2)
@@ -425,7 +430,7 @@ LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.c,$(foreach skel,$(LINKED_SKELS),$($(ske
  # $eval()) and pass control to DEFINE_TEST_RUNNER_RULES.
  # Parameters:
  # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
  define DEFINE_TEST_RUNNER

[...]

#29 lore raw

PATCH v2 9/15 selftests/bpf: Add unit tests for new sign-extension load insns

In reply to Yonghong Song

Add unit tests for new ldsx insns. The test includes sign-extension with a single value or with a value range.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_ldsx.c       | 115 ++++++++++++++++++
 2 files changed, 117 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_ldsx.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index c375e59ff28d..6eec6a9463c8 100644
@@ -31,6 +31,7 @@
 #include "verifier_int_ptr.skel.h"
 #include "verifier_jeq_infer_not_null.skel.h"
 #include "verifier_ld_ind.skel.h"
+#include "verifier_ldsx.skel.h"
 #include "verifier_leak_ptr.skel.h"
 #include "verifier_loops1.skel.h"
 #include "verifier_lwt.skel.h"
@@ -133,6 +134,7 @@ void test_verifier_helper_value_access(void)  { RUN(verifier_helper_value_access
 void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }
 void test_verifier_jeq_infer_not_null(void)   { RUN(verifier_jeq_infer_not_null); }
 void test_verifier_ld_ind(void)               { RUN(verifier_ld_ind); }
+void test_verifier_ldsx(void)                  { RUN(verifier_ldsx); }
 void test_verifier_leak_ptr(void)             { RUN(verifier_leak_ptr); }
 void test_verifier_loops1(void)               { RUN(verifier_loops1); }
 void test_verifier_lwt(void)                  { RUN(verifier_lwt); }
tools/testing/selftests/bpf/progs/verifier_ldsx.c new file +115 -0

115 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#30 lore raw

Re: [PATCH bpf-next v2 09/15] selftests/bpf: Add unit tests for new sign-extension load insns

Quoted 70 lines · Yonghong Song

Add unit tests for new ldsx insns. The test includes sign-extension with a single value or with a value range.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_ldsx.c       | 115 ++++++++++++++++++
 2 files changed, 117 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_ldsx.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index c375e59ff28d..6eec6a9463c8 100644
@@ -31,6 +31,7 @@
 #include "verifier_int_ptr.skel.h"
 #include "verifier_jeq_infer_not_null.skel.h"
 #include "verifier_ld_ind.skel.h"
+#include "verifier_ldsx.skel.h"
 #include "verifier_leak_ptr.skel.h"
 #include "verifier_loops1.skel.h"
 #include "verifier_lwt.skel.h"
@@ -133,6 +134,7 @@ void test_verifier_helper_value_access(void)  { RUN(verifier_helper_value_access
 void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }
 void test_verifier_jeq_infer_not_null(void)   { RUN(verifier_jeq_infer_not_null); }
 void test_verifier_ld_ind(void)               { RUN(verifier_ld_ind); }
+void test_verifier_ldsx(void)                  { RUN(verifier_ldsx); }
 void test_verifier_leak_ptr(void)             { RUN(verifier_leak_ptr); }
 void test_verifier_loops1(void)               { RUN(verifier_loops1); }
 void test_verifier_lwt(void)                  { RUN(verifier_lwt); }
tools/testing/selftests/bpf/progs/verifier_ldsx.c new file +34 -0
new file mode 100644
index 000000000000..cd90913583b9
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+SEC("socket")
+__description("LDSX, S8")
+__success __success_unpriv __retval(-2)
+__naked void ldsx_s8(void)
+{
+	asm volatile ("					\
+	r1 = 0x3fe;					\
+	*(u64 *)(r10 - 8) = r1;				\
+	r0 = *(s8 *)(r10 - 8);				\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("LDSX, S16")
+__success __success_unpriv __retval(-2)
+__naked void ldsx_s16(void)
+{
+	asm volatile ("					\
+	r1 = 0x3fffe;					\
+	*(u64 *)(r10 - 8) = r1;				\
+	r0 = *(s16 *)(r10 - 8);				\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("LDSX, S32")

tools/testing/selftests/bpf/progs/verifier_ldsx.c line 44

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c new file +10 -0
+__success __success_unpriv __retval(-2)
+__naked void ldsx_s32(void)
+{
+	asm volatile ("					\
+	r1 = 0xfffffffe;				\
+	*(u64 *)(r10 - 8) = r1;				\
+	r0 = *(s32 *)(r10 - 8);				\
+	exit;						\
+"	::: __clobber_all);
+}

Note that __retval is a 32-bit value, so if I change
r0 = *(s32 *)(r10 - 8); to

  r0 = *(u32 *)(r10 - 8);
         ^

__retval would be the same, probably better to add some shift right as a last instruction or something like that.
(Also applies to a similar test in the movsx tests).

tools/testing/selftests/bpf/progs/verifier_ldsx.c line 49

Quoted 5 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c new file +5 -0
+
+SEC("socket")
+__description("LDSX, S8 range checking")
+__success __success_unpriv __retval(1)
+__naked void ldsx_s8_range(void)

Maybe add __log_level(2) and a few
__msg("... R1_w=scalar(smin=-128,smax=127)") ?
So that verifier range logic is checked as well.

Quoted 66 lines
tools/testing/selftests/bpf/progs/verifier_ldsx.c new file +66 -0
+{
+	asm volatile ("					\
+	call %[bpf_get_prandom_u32];			\
+	*(u64 *)(r10 - 8) = r0;				\
+	r1 = *(s8 *)(r10 - 8);				\
+	/* r1 with s8 range */				\
+	if r1 s> 0x7f goto l0_%=;			\
+	if r1 s< -0x80 goto l0_%=;			\
+	r0 = 1;						\
+l1_%=:							\
+	exit;						\
+l0_%=:							\
+	r0 = 2;						\
+	goto l1_%=;					\
+"	:
+	: __imm(bpf_get_prandom_u32)
+	: __clobber_all);
+}
+
+SEC("socket")
+__description("LDSX, S16 range checking")
+__success __success_unpriv __retval(1)
+__naked void ldsx_s16_range(void)
+{
+	asm volatile ("					\
+	call %[bpf_get_prandom_u32];			\
+	*(u64 *)(r10 - 8) = r0;				\
+	r1 = *(s16 *)(r10 - 8);				\
+	/* r1 with s16 range */				\
+	if r1 s> 0x7fff goto l0_%=;			\
+	if r1 s< -0x8000 goto l0_%=;			\
+	r0 = 1;						\
+l1_%=:							\
+	exit;						\
+l0_%=:							\
+	r0 = 2;						\
+	goto l1_%=;					\
+"	:
+	: __imm(bpf_get_prandom_u32)
+	: __clobber_all);
+}
+
+SEC("socket")
+__description("LDSX, S32 range checking")
+__success __success_unpriv __retval(1)
+__naked void ldsx_s32_range(void)
+{
+	asm volatile ("					\
+	call %[bpf_get_prandom_u32];			\
+	*(u64 *)(r10 - 8) = r0;				\
+	r1 = *(s32 *)(r10 - 8);				\
+	/* r1 with s16 range */				\
+	if r1 s> 0x7fffFFFF goto l0_%=;			\
+	if r1 s< -0x80000000 goto l0_%=;		\
+	r0 = 1;						\
+l1_%=:							\
+	exit;						\
+l0_%=:							\
+	r0 = 2;						\
+	goto l1_%=;					\
+"	:
+	: __imm(bpf_get_prandom_u32)
+	: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
#31 lore raw

PATCH v2 10/15 selftests/bpf: Add unit tests for new sign-extension mov insns

In reply to Yonghong Song

Add unit tests for movsx insns.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_movsx.c      | 177 ++++++++++++++++++
 2 files changed, 179 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_movsx.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 6eec6a9463c8..037af7799cdf 100644
@@ -41,6 +41,7 @@
 #include "verifier_map_ret_val.skel.h"
 #include "verifier_masking.skel.h"
 #include "verifier_meta_access.skel.h"
+#include "verifier_movsx.skel.h"
 #include "verifier_netfilter_ctx.skel.h"
 #include "verifier_netfilter_retcode.skel.h"
 #include "verifier_prevent_map_lookup.skel.h"
@@ -144,6 +145,7 @@ void test_verifier_map_ptr_mixing(void)       { RUN(verifier_map_ptr_mixing); }
 void test_verifier_map_ret_val(void)          { RUN(verifier_map_ret_val); }
 void test_verifier_masking(void)              { RUN(verifier_masking); }
 void test_verifier_meta_access(void)          { RUN(verifier_meta_access); }
+void test_verifier_movsx(void)                 { RUN(verifier_movsx); }
 void test_verifier_netfilter_ctx(void)        { RUN(verifier_netfilter_ctx); }
 void test_verifier_netfilter_retcode(void)    { RUN(verifier_netfilter_retcode); }
 void test_verifier_prevent_map_lookup(void)   { RUN(verifier_prevent_map_lookup); }
tools/testing/selftests/bpf/progs/verifier_movsx.c new file +177 -0

177 lines not on this page · raw message

Signature · 1 line
2.34.1
#32 lore raw

PATCH v2 11/15 selftests/bpf: Add unit tests for new bswap insns

In reply to Yonghong Song

Add unit tests for bswap insns.

---
 .../selftests/bpf/prog_tests/verifier.c       |  2 +
 .../selftests/bpf/progs/verifier_bswap.c      | 45 +++++++++++++++++++
 2 files changed, 47 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bswap.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 037af7799cdf..885532540bc3 100644
@@ -11,6 +11,7 @@
 #include "verifier_bounds_deduction_non_const.skel.h"
 #include "verifier_bounds_mix_sign_unsign.skel.h"
 #include "verifier_bpf_get_stack.skel.h"
+#include "verifier_bswap.skel.h"
 #include "verifier_btf_ctx_access.skel.h"
 #include "verifier_cfg.skel.h"
 #include "verifier_cgroup_inv_retcode.skel.h"
@@ -115,6 +116,7 @@ void test_verifier_bounds_deduction(void)     { RUN(verifier_bounds_deduction);
 void test_verifier_bounds_deduction_non_const(void)     { RUN(verifier_bounds_deduction_non_const); }
 void test_verifier_bounds_mix_sign_unsign(void) { RUN(verifier_bounds_mix_sign_unsign); }
 void test_verifier_bpf_get_stack(void)        { RUN(verifier_bpf_get_stack); }
+void test_verifier_bswap(void)                { RUN(verifier_bswap); }
 void test_verifier_btf_ctx_access(void)       { RUN(verifier_btf_ctx_access); }
 void test_verifier_cfg(void)                  { RUN(verifier_cfg); }
 void test_verifier_cgroup_inv_retcode(void)   { RUN(verifier_cgroup_inv_retcode); }
tools/testing/selftests/bpf/progs/verifier_bswap.c new file +45 -0

45 lines not on this page · raw message

Signature · 1 line
2.34.1
#33 lore raw

PATCH v2 12/15 selftests/bpf: Add unit tests for new sdiv/smod insns

In reply to Yonghong Song

Add unit tests for sdiv/smod insns.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
 2 files changed, 765 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 885532540bc3..a591d7b673f1 100644
@@ -54,6 +54,7 @@
 #include "verifier_ringbuf.skel.h"
 #include "verifier_runtime_jit.skel.h"
 #include "verifier_scalar_ids.skel.h"
+#include "verifier_sdiv.skel.h"
 #include "verifier_search_pruning.skel.h"
 #include "verifier_sock.skel.h"
 #include "verifier_spill_fill.skel.h"
@@ -159,6 +160,7 @@ void test_verifier_regalloc(void)             { RUN(verifier_regalloc); }
 void test_verifier_ringbuf(void)              { RUN(verifier_ringbuf); }
 void test_verifier_runtime_jit(void)          { RUN(verifier_runtime_jit); }
 void test_verifier_scalar_ids(void)           { RUN(verifier_scalar_ids); }
+void test_verifier_sdiv(void)                 { RUN(verifier_sdiv); }
 void test_verifier_search_pruning(void)       { RUN(verifier_search_pruning); }
 void test_verifier_sock(void)                 { RUN(verifier_sock); }
 void test_verifier_spill_fill(void)           { RUN(verifier_spill_fill); }
tools/testing/selftests/bpf/progs/verifier_sdiv.c new file +763 -0

763 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#34 lore raw

Re: [PATCH bpf-next v2 12/15] selftests/bpf: Add unit tests for new sdiv/smod insns

Quoted 744 lines · Yonghong Song

Add unit tests for sdiv/smod insns.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
 2 files changed, 765 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 885532540bc3..a591d7b673f1 100644
@@ -54,6 +54,7 @@
 #include "verifier_ringbuf.skel.h"
 #include "verifier_runtime_jit.skel.h"
 #include "verifier_scalar_ids.skel.h"
+#include "verifier_sdiv.skel.h"
 #include "verifier_search_pruning.skel.h"
 #include "verifier_sock.skel.h"
 #include "verifier_spill_fill.skel.h"
@@ -159,6 +160,7 @@ void test_verifier_regalloc(void)             { RUN(verifier_regalloc); }
 void test_verifier_ringbuf(void)              { RUN(verifier_ringbuf); }
 void test_verifier_runtime_jit(void)          { RUN(verifier_runtime_jit); }
 void test_verifier_scalar_ids(void)           { RUN(verifier_scalar_ids); }
+void test_verifier_sdiv(void)                 { RUN(verifier_sdiv); }
 void test_verifier_search_pruning(void)       { RUN(verifier_search_pruning); }
 void test_verifier_sock(void)                 { RUN(verifier_sock); }
 void test_verifier_spill_fill(void)           { RUN(verifier_spill_fill); }
tools/testing/selftests/bpf/progs/verifier_sdiv.c new file +709 -0
new file mode 100644
index 000000000000..d92098d670ef
@@ -0,0 +1,763 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 1")
+__success __success_unpriv __retval(-20)
+__naked void sdiv32_non_zero_imm_1(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w0 s/= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 2")
+__success __success_unpriv __retval(-20)
+__naked void sdiv32_non_zero_imm_2(void)
+{
+	asm volatile ("					\
+	w0 = 41;					\
+	w0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 3")
+__success __success_unpriv __retval(20)
+__naked void sdiv32_non_zero_imm_3(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 4")
+__success __success_unpriv __retval(-21)
+__naked void sdiv32_non_zero_imm_4(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w0 s/= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 5")
+__success __success_unpriv __retval(-21)
+__naked void sdiv32_non_zero_imm_5(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 6")
+__success __success_unpriv __retval(21)
+__naked void sdiv32_non_zero_imm_6(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 7")
+__success __success_unpriv __retval(21)
+__naked void sdiv32_non_zero_imm_7(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w0 s/= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero imm divisor, check 8")
+__success __success_unpriv __retval(20)
+__naked void sdiv32_non_zero_imm_8(void)
+{
+	asm volatile ("					\
+	w0 = 41;					\
+	w0 s/= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 1")
+__success __success_unpriv __retval(-20)
+__naked void sdiv32_non_zero_reg_1(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w1 = 2;						\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 2")
+__success __success_unpriv __retval(-20)
+__naked void sdiv32_non_zero_reg_2(void)
+{
+	asm volatile ("					\
+	w0 = 41;					\
+	w1 = -2;					\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 3")
+__success __success_unpriv __retval(20)
+__naked void sdiv32_non_zero_reg_3(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w1 = -2;					\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 4")
+__success __success_unpriv __retval(-21)
+__naked void sdiv32_non_zero_reg_4(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w1 = 2;						\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 5")
+__success __success_unpriv __retval(-21)
+__naked void sdiv32_non_zero_reg_5(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w1 = -2;					\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 6")
+__success __success_unpriv __retval(21)
+__naked void sdiv32_non_zero_reg_6(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w1 = -2;					\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 7")
+__success __success_unpriv __retval(21)
+__naked void sdiv32_non_zero_reg_7(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w1 = 2;						\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, non-zero reg divisor, check 8")
+__success __success_unpriv __retval(20)
+__naked void sdiv32_non_zero_reg_8(void)
+{
+	asm volatile ("					\
+	w0 = 41;					\
+	w1 = 2;						\
+	w0 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero imm divisor, check 1")
+__success __success_unpriv __retval(-20)
+__naked void sdiv64_non_zero_imm_1(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r0 s/= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero imm divisor, check 2")
+__success __success_unpriv __retval(-20)
+__naked void sdiv64_non_zero_imm_2(void)
+{
+	asm volatile ("					\
+	r0 = 41;					\
+	r0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero imm divisor, check 3")
+__success __success_unpriv __retval(20)
+__naked void sdiv64_non_zero_imm_3(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero imm divisor, check 4")
+__success __success_unpriv __retval(-21)
+__naked void sdiv64_non_zero_imm_4(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r0 s/= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero imm divisor, check 5")
+__success __success_unpriv __retval(-21)
+__naked void sdiv64_non_zero_imm_5(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero imm divisor, check 6")
+__success __success_unpriv __retval(21)
+__naked void sdiv64_non_zero_imm_6(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r0 s/= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero reg divisor, check 1")
+__success __success_unpriv __retval(-20)
+__naked void sdiv64_non_zero_reg_1(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r1 = 2;						\
+	r0 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero reg divisor, check 2")
+__success __success_unpriv __retval(-20)
+__naked void sdiv64_non_zero_reg_2(void)
+{
+	asm volatile ("					\
+	r0 = 41;					\
+	r1 = -2;					\
+	r0 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero reg divisor, check 3")
+__success __success_unpriv __retval(20)
+__naked void sdiv64_non_zero_reg_3(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r1 = -2;					\
+	r0 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero reg divisor, check 4")
+__success __success_unpriv __retval(-21)
+__naked void sdiv64_non_zero_reg_4(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r1 = 2;						\
+	r0 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero reg divisor, check 5")
+__success __success_unpriv __retval(-21)
+__naked void sdiv64_non_zero_reg_5(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r1 = -2;					\
+	r0 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV64, non-zero reg divisor, check 6")
+__success __success_unpriv __retval(21)
+__naked void sdiv64_non_zero_reg_6(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r1 = -2;					\
+	r0 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero imm divisor, check 1")
+__success __success_unpriv __retval(-1)
+__naked void smod32_non_zero_imm_1(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w0 s%%= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero imm divisor, check 2")
+__success __success_unpriv __retval(1)
+__naked void smod32_non_zero_imm_2(void)
+{
+	asm volatile ("					\
+	w0 = 41;					\
+	w0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero imm divisor, check 3")
+__success __success_unpriv __retval(-1)
+__naked void smod32_non_zero_imm_3(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero imm divisor, check 4")
+__success __success_unpriv __retval(0)
+__naked void smod32_non_zero_imm_4(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w0 s%%= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero imm divisor, check 5")
+__success __success_unpriv __retval(0)
+__naked void smod32_non_zero_imm_5(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero imm divisor, check 6")
+__success __success_unpriv __retval(0)
+__naked void smod32_non_zero_imm_6(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero reg divisor, check 1")
+__success __success_unpriv __retval(-1)
+__naked void smod32_non_zero_reg_1(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w1 = 2;						\
+	w0 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero reg divisor, check 2")
+__success __success_unpriv __retval(1)
+__naked void smod32_non_zero_reg_2(void)
+{
+	asm volatile ("					\
+	w0 = 41;					\
+	w1 = -2;					\
+	w0 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero reg divisor, check 3")
+__success __success_unpriv __retval(-1)
+__naked void smod32_non_zero_reg_3(void)
+{
+	asm volatile ("					\
+	w0 = -41;					\
+	w1 = -2;					\
+	w0 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero reg divisor, check 4")
+__success __success_unpriv __retval(0)
+__naked void smod32_non_zero_reg_4(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w1 = 2;						\
+	w0 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero reg divisor, check 5")
+__success __success_unpriv __retval(0)
+__naked void smod32_non_zero_reg_5(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w1 = -2;					\
+	w0 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, non-zero reg divisor, check 6")
+__success __success_unpriv __retval(0)
+__naked void smod32_non_zero_reg_6(void)
+{
+	asm volatile ("					\
+	w0 = -42;					\
+	w1 = -2;					\
+	w0 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 1")
+__success __success_unpriv __retval(-1)
+__naked void smod64_non_zero_imm_1(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r0 s%%= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 2")
+__success __success_unpriv __retval(1)
+__naked void smod64_non_zero_imm_2(void)
+{
+	asm volatile ("					\
+	r0 = 41;					\
+	r0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 3")
+__success __success_unpriv __retval(-1)
+__naked void smod64_non_zero_imm_3(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 4")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_imm_4(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r0 s%%= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 5")
+__success __success_unpriv __retval(-0)
+__naked void smod64_non_zero_imm_5(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 6")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_imm_6(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r0 s%%= -2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 7")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_imm_7(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r0 s%%= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero imm divisor, check 8")
+__success __success_unpriv __retval(1)
+__naked void smod64_non_zero_imm_8(void)
+{
+	asm volatile ("					\
+	r0 = 41;					\
+	r0 s%%= 2;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 1")
+__success __success_unpriv __retval(-1)
+__naked void smod64_non_zero_reg_1(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r1 = 2;						\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 2")
+__success __success_unpriv __retval(1)
+__naked void smod64_non_zero_reg_2(void)
+{
+	asm volatile ("					\
+	r0 = 41;					\
+	r1 = -2;					\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 3")
+__success __success_unpriv __retval(-1)
+__naked void smod64_non_zero_reg_3(void)
+{
+	asm volatile ("					\
+	r0 = -41;					\
+	r1 = -2;					\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 4")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_reg_4(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r1 = 2;						\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 5")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_reg_5(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r1 = -2;					\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 6")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_reg_6(void)
+{
+	asm volatile ("					\
+	r0 = -42;					\
+	r1 = -2;					\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 7")
+__success __success_unpriv __retval(0)
+__naked void smod64_non_zero_reg_7(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r1 = 2;						\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, non-zero reg divisor, check 8")
+__success __success_unpriv __retval(1)
+__naked void smod64_non_zero_reg_8(void)
+{
+	asm volatile ("					\
+	r0 = 41;					\
+	r1 = 2;						\
+	r0 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SDIV32, zero divisor")
+__success __success_unpriv __retval(42)

tools/testing/selftests/bpf/progs/verifier_sdiv.c line 719

Quoted 10 lines
tools/testing/selftests/bpf/progs/verifier_sdiv.c new file +10 -0
+__naked void sdiv32_zero_divisor(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w1 = 0;						\
+	w2 = -1;					\
+	w2 s/= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}

Maybe add "w0 = w2" in the end to check how division by zero modifies the DST register?

Quoted 44 lines
tools/testing/selftests/bpf/progs/verifier_sdiv.c new file +44 -0
+
+SEC("socket")
+__description("SDIV64, zero divisor")
+__success __success_unpriv __retval(42)
+__naked void sdiv64_zero_divisor(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r1 = 0;						\
+	r2 = -1;					\
+	r2 s/= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD32, zero divisor")
+__success __success_unpriv __retval(42)
+__naked void smod32_zero_divisor(void)
+{
+	asm volatile ("					\
+	w0 = 42;					\
+	w1 = 0;						\
+	w2 = -1;					\
+	w2 s%%= w1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+SEC("socket")
+__description("SMOD64, zero divisor")
+__success __success_unpriv __retval(42)
+__naked void smod64_zero_divisor(void)
+{
+	asm volatile ("					\
+	r0 = 42;					\
+	r1 = 0;						\
+	r2 = -1;					\
+	r2 s%%= r1;					\
+	exit;						\
+"	::: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
#35 lore raw

PATCH v2 13/15 selftests/bpf: Add unit tests for new gotol insn

In reply to Yonghong Song

Add unit tests for gotol insn.

---
 .../selftests/bpf/prog_tests/verifier.c       |  2 ++
 .../selftests/bpf/progs/verifier_gotol.c      | 30 +++++++++++++++++++
 2 files changed, 32 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotol.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index a591d7b673f1..e3e68c97b40c 100644
@@ -25,6 +25,7 @@
 #include "verifier_direct_stack_access_wraparound.skel.h"
 #include "verifier_div0.skel.h"
 #include "verifier_div_overflow.skel.h"
+#include "verifier_gotol.skel.h"
 #include "verifier_helper_access_var_len.skel.h"
 #include "verifier_helper_packet_access.skel.h"
 #include "verifier_helper_restricted.skel.h"
@@ -131,6 +132,7 @@ void test_verifier_direct_packet_access(void) { RUN(verifier_direct_packet_acces
 void test_verifier_direct_stack_access_wraparound(void) { RUN(verifier_direct_stack_access_wraparound); }
 void test_verifier_div0(void)                 { RUN(verifier_div0); }
 void test_verifier_div_overflow(void)         { RUN(verifier_div_overflow); }
+void test_verifier_gotol(void)                { RUN(verifier_gotol); }
 void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_len); }
 void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
 void test_verifier_helper_restricted(void)    { RUN(verifier_helper_restricted); }
tools/testing/selftests/bpf/progs/verifier_gotol.c new file +30 -0
new file mode 100644
index 000000000000..78870ea4d468
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+SEC("socket")
+__description("gotol, small_imm")
+__success __success_unpriv __retval(1)
+__naked void gotol_small_imm(void)
+{
+	asm volatile ("					\
+	call %[bpf_ktime_get_ns];			\
+	if r0 == 0 goto l0_%=;				\
+	gotol l1_%=;					\
+l2_%=:							\
+	gotol l3_%=;					\
+l1_%=:							\
+	r0 = 1;						\
+	gotol l2_%=;					\
+l0_%=:							\
+	r0 = 2;						\
+l3_%=:							\
+	exit;						\
+"	:
+	: __imm(bpf_ktime_get_ns)
+	: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
Signature · 1 line
2.34.1
#36 lore raw

PATCH v2 14/15 selftests/bpf: Test ldsx with more complex cases

In reply to Yonghong Song

Test reading signed rdonly map value, read/write map value, probed memory and ctx field. Without proper verifier/git handling, the test will fail.

---
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  9 +-
 .../selftests/bpf/prog_tests/test_ldsx_insn.c | 88 +++++++++++++++++++
 .../selftests/bpf/progs/test_ldsx_insn.c      | 75 ++++++++++++++++
 3 files changed, 171 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_ldsx_insn.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_ldsx_insn.c
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +8 -1
index aaf6ef1201c7..e14c9c6f69eb 100644
@@ -75,6 +75,12 @@ bpf_testmod_test_struct_arg_6(struct bpf_testmod_struct_arg_3 *a) {
 	return bpf_testmod_test_struct_arg_result;
 }
 
+noinline int
+bpf_testmod_test_arg_ptr_to_struct(struct bpf_testmod_struct_arg_1 *a) {
+	bpf_testmod_test_struct_arg_result = a->a;
+	return bpf_testmod_test_struct_arg_result;
+}
+
 __bpf_kfunc void
 bpf_testmod_test_mod_kfunc(int i)
 {
@@ -203,7 +209,7 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj,
 		.off = off,
 		.len = len,
 	};
-	struct bpf_testmod_struct_arg_1 struct_arg1 = {10};
+	struct bpf_testmod_struct_arg_1 struct_arg1 = {10}, struct_arg1_2 = {-1};
 	struct bpf_testmod_struct_arg_2 struct_arg2 = {2, 3};
 	struct bpf_testmod_struct_arg_3 *struct_arg3;
 	int i = 1;
@@ -216,6 +222,7 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj,
 	(void)bpf_testmod_test_struct_arg_3(1, 4, struct_arg2);
 	(void)bpf_testmod_test_struct_arg_4(struct_arg1, 1, 2, 3, struct_arg2);
 	(void)bpf_testmod_test_struct_arg_5();
+	(void)bpf_testmod_test_arg_ptr_to_struct(&struct_arg1_2);
 
 	struct_arg3 = kmalloc((sizeof(struct bpf_testmod_struct_arg_3) +
 				sizeof(int)), GFP_KERNEL);
tools/testing/selftests/bpf/prog_tests/test_ldsx_insn.c new file +88 -0

88 lines not on this page · raw message

tools/testing/selftests/bpf/progs/test_ldsx_insn.c new file +75 -0

75 lines not on this page · raw message

Signature · 1 line
2.34.1
#37 lore raw

PATCH v2 15/15 docs/bpf: Add documentation for new instructions

In reply to Yonghong Song

Add documentation in instruction-set.rst for new instruction encoding and their corresponding operations. Also removed the question related to 'no BPF_SDIV' in bpf_design_QA.rst since we have BPF_SDIV insn now.

---
 Documentation/bpf/bpf_design_QA.rst           |   5 -
 .../bpf/standardization/instruction-set.rst   | 100 ++++++++++++------
 2 files changed, 66 insertions(+), 39 deletions(-)
Documentation/bpf/bpf_design_QA.rst +0 -5
index 38372a956d65..eb19c945f4d5 100644
@@ -140,11 +140,6 @@ A: Because if we picked one-to-one relationship to x64 it would have made
 it more complicated to support on arm64 and other archs. Also it
 needs div-by-zero runtime check.
 
-Q: Why there is no BPF_SDIV for signed divide operation?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A: Because it would be rarely used. llvm errors in such case and
-prints a suggestion to use unsigned divide instead.
-
 Q: Why BPF has implicit prologue and epilogue?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 A: Because architectures like sparc have register windows and in general
Documentation/bpf/standardization/instruction-set.rst +66 -34

153 lines not on this page · raw message

Signature · 1 line
2.34.1
#38 lore raw

Re: [PATCH bpf-next v2 15/15] docs/bpf: Add documentation for new instructions

Quoted 18 lines · Yonghong Song
Documentation/bpf/standardization/instruction-set.rst +0 -10
index 751e657973f0..367f426d09a1 100644
@@ -154,24 +154,27 @@ otherwise identical operations.
 The 'code' field encodes the operation as below, where 'src' and 'dst' refer
 to the values of the source and destination registers, respectively.
 
-========  =====  ==========================================================
-code      value  description
-========  =====  ==========================================================
-BPF_ADD   0x00   dst += src
-BPF_SUB   0x10   dst -= src
-BPF_MUL   0x20   dst \*= src
-BPF_DIV   0x30   dst = (src != 0) ? (dst / src) : 0
-BPF_OR    0x40   dst \|= src
-BPF_AND   0x50   dst &= src
-BPF_LSH   0x60   dst <<= (src & mask)

Documentation/bpf/standardization/instruction-set.rst otherwise identical operations. · line 158

Quoted 10 lines
Documentation/bpf/standardization/instruction-set.rst +2 -8
-BPF_RSH   0x70   dst >>= (src & mask)
-BPF_NEG   0x80   dst = -src
-BPF_MOD   0x90   dst = (src != 0) ? (dst % src) : dst
-BPF_XOR   0xa0   dst ^= src
-BPF_MOV   0xb0   dst = src
-BPF_ARSH  0xc0   sign extending dst >>= (src & mask)
-BPF_END   0xd0   byte swap operations (see `Byte swap instructions`_ below)
-========  =====  ==========================================================
+========  =====  ============  ==========================================================
+code      value  offset value  description

How about just 'offset' ?

Quoted 30 lines
Documentation/bpf/standardization/instruction-set.rst +19 -4
+========  =====  ============  ==========================================================
+BPF_ADD   0x00   0             dst += src
+BPF_SUB   0x10   0             dst -= src
+BPF_MUL   0x20   0             dst \*= src
+BPF_DIV   0x30   0             dst = (src != 0) ? (dst / src) : 0
+BPF_SDIV  0x30   1             dst = (src != 0) ? (dst s/ src) : 0
+BPF_OR    0x40   0             dst \|= src
+BPF_AND   0x50   0             dst &= src
+BPF_LSH   0x60   0             dst <<= (src & mask)
+BPF_RSH   0x70   0             dst >>= (src & mask)
+BPF_NEG   0x80   0             dst = -src
+BPF_MOD   0x90   0             dst = (src != 0) ? (dst % src) : dst
+BPF_SMOD  0x90   1             dst = (src != 0) ? (dst s% src) : dst
+BPF_XOR   0xa0   0             dst ^= src
+BPF_MOV   0xb0   0             dst = src
+BPF_MOVSX 0xb0   8/16/32       dst = (s8,16,s32)src
+BPF_ARSH  0xc0   0             sign extending dst >>= (src & mask)
+BPF_END   0xd0   0             byte swap operations (see `Byte swap instructions`_ below)
+========  =====  ============  ==========================================================
 
 Underflow and overflow are allowed during arithmetic operations, meaning
 the 64-bit or 32-bit value will wrap. If eBPF program execution would
@@ -198,11 +201,19 @@ where '(u32)' indicates that the upper 32 bits are zeroed.
 
   dst = dst ^ imm32
 
-Also note that the division and modulo operations are unsigned. Thus, for
-``BPF_ALU``, 'imm' is first interpreted as an unsigned 32-bit value, whereas
-for ``BPF_ALU64``, 'imm' is first sign extended to 64 bits and the result
-interpreted as an unsigned 64-bit value. There are no instructions for

Documentation/bpf/standardization/instruction-set.rst where '(u32)' indicates that the upper 32 bits are zeroed. · line 212

Quoted 10 lines
Documentation/bpf/standardization/instruction-set.rst +9 -1
-signed division or modulo.
+Note that most instructions have instruction offset of 0. But three instructions
+(BPF_SDIV, BPF_SMOD, BPF_MOVSX) have non-zero offset.
+
+The devision and modulo operations support both unsigned and signed flavors.
+For unsigned operation (BPF_DIV and BPF_MOD), for ``BPF_ALU``, 'imm' is first
+interpreted as an unsigned 32-bit value, whereas for ``BPF_ALU64``, 'imm' is
+first sign extended to 64 bits and the result interpreted as an unsigned 64-bit
+value.  For signed operation (BPF_SDIV and BPF_SMOD), for both ``BPF_ALU`` and
+``BPF_ALU64``, 'imm' is interpreted as a signed value.

Probably worth clarifying that in case of S[DIV|MOD] | ALU64 the imm is sign extended from 32 to 64 and interpreted as signed 64-bit.

Documentation/bpf/standardization/instruction-set.rst where '(u32)' indicates that the upper 32 bits are zeroed. · line 216

Quoted 4 lines
Documentation/bpf/standardization/instruction-set.rst +4 -0
+
+Instruction BPF_MOVSX does move operation with sign extension. For ``BPF_ALU``
+mode, 8-bit and 16-bit sign extensions to 32-bit are supported. For ``BPF_ALU64``,
+8-bit, 16-bit and 32-bit sign extenstions to 64-bit are supported.

How about:

Instruction BPF_MOVSX does move operation with sign extension.
BPF_ALU | MOVSX sign extendes 8-bit and 16-bit into 32-bit and upper 32-bit are zeroed. BPF_ALU64 | MOVSX sign extends 8-bit, 16-bit and 32-bit into 64-bit.

Documentation/bpf/standardization/instruction-set.rst Examples: · line 257

Quoted 4 lines
Documentation/bpf/standardization/instruction-set.rst +3 -0
+``BPF_ALU64 | BPF_TO_LE | BPF_END`` with imm = 16 means::
+
+  dst = bswap16(dst)

Worth spelling out imm 32 and 64 too ?

Documentation/bpf/standardization/instruction-set.rst instructions that transfer data between a register and memory. · line 386

Quoted 8 lines
Documentation/bpf/standardization/instruction-set.rst +7 -0
+The ``BPF_MEMSX`` mode modifier is used to encode sign-extension load
+instructions that transfer data between a register and memory.
+
+``BPF_MEMSX | <size> | BPF_LDX`` means::
+
+  dst = *(sign-extension size *) (src + offset)
+

How about:

``BPF_MEM | <size> | BPF_LDX`` means::

dst = *(unsigned size *) (src + offset)

``BPF_MEMSX | <size> | BPF_LDX`` means::

dst = *(signed size *) (src + offset)

#39 lore raw

Re: [PATCH bpf-next v2 15/15] docs/bpf: Add documentation for new instructions

Quoted 23 lines · Alexei Starovoitov
Quoted 22 lines · Yonghong Song
Documentation/bpf/standardization/instruction-set.rst +0 -14
index 751e657973f0..367f426d09a1 100644
@@ -154,24 +154,27 @@ otherwise identical operations.
  The 'code' field encodes the operation as below, where 'src' and 'dst' refer
  to the values of the source and destination registers, respectively.
 
-========  =====  ==========================================================
-code      value  description
-========  =====  ==========================================================
-BPF_ADD   0x00   dst += src
-BPF_SUB   0x10   dst -= src
-BPF_MUL   0x20   dst \*= src
-BPF_DIV   0x30   dst = (src != 0) ? (dst / src) : 0
-BPF_OR    0x40   dst \|= src
-BPF_AND   0x50   dst &= src
-BPF_LSH   0x60   dst <<= (src & mask)
-BPF_RSH   0x70   dst >>= (src & mask)
-BPF_NEG   0x80   dst = -src
-BPF_MOD   0x90   dst = (src != 0) ? (dst % src) : dst
-BPF_XOR   0xa0   dst ^= src
Quoted 8 lines

Documentation/bpf/standardization/instruction-set.rst otherwise identical operations. · line 158

Quoted 6 lines
Documentation/bpf/standardization/instruction-set.rst +2 -4
-BPF_MOV   0xb0   dst = src
-BPF_ARSH  0xc0   sign extending dst >>= (src & mask)
-BPF_END   0xd0   byte swap operations (see `Byte swap instructions`_ below)
-========  =====  ==========================================================
+========  =====  ============  ==========================================================
+code      value  offset value  description

How about just 'offset' ?

Ack.

Quoted 36 lines
Quoted 36 lines
Documentation/bpf/standardization/instruction-set.rst +23 -5
+========  =====  ============  ==========================================================
+BPF_ADD   0x00   0             dst += src
+BPF_SUB   0x10   0             dst -= src
+BPF_MUL   0x20   0             dst \*= src
+BPF_DIV   0x30   0             dst = (src != 0) ? (dst / src) : 0
+BPF_SDIV  0x30   1             dst = (src != 0) ? (dst s/ src) : 0
+BPF_OR    0x40   0             dst \|= src
+BPF_AND   0x50   0             dst &= src
+BPF_LSH   0x60   0             dst <<= (src & mask)
+BPF_RSH   0x70   0             dst >>= (src & mask)
+BPF_NEG   0x80   0             dst = -src
+BPF_MOD   0x90   0             dst = (src != 0) ? (dst % src) : dst
+BPF_SMOD  0x90   1             dst = (src != 0) ? (dst s% src) : dst
+BPF_XOR   0xa0   0             dst ^= src
+BPF_MOV   0xb0   0             dst = src
+BPF_MOVSX 0xb0   8/16/32       dst = (s8,16,s32)src
+BPF_ARSH  0xc0   0             sign extending dst >>= (src & mask)
+BPF_END   0xd0   0             byte swap operations (see `Byte swap instructions`_ below)
+========  =====  ============  ==========================================================
 
  Underflow and overflow are allowed during arithmetic operations, meaning
  the 64-bit or 32-bit value will wrap. If eBPF program execution would
@@ -198,11 +201,19 @@ where '(u32)' indicates that the upper 32 bits are zeroed.
 
    dst = dst ^ imm32
 
-Also note that the division and modulo operations are unsigned. Thus, for
-``BPF_ALU``, 'imm' is first interpreted as an unsigned 32-bit value, whereas
-for ``BPF_ALU64``, 'imm' is first sign extended to 64 bits and the result
-interpreted as an unsigned 64-bit value. There are no instructions for
-signed division or modulo.
+Note that most instructions have instruction offset of 0. But three instructions
+(BPF_SDIV, BPF_SMOD, BPF_MOVSX) have non-zero offset.
+
+The devision and modulo operations support both unsigned and signed flavors.
Quoted 8 lines

Documentation/bpf/standardization/instruction-set.rst where '(u32)' indicates that the upper 32 bits are zeroed. · line 212

Quoted 5 lines
Documentation/bpf/standardization/instruction-set.rst +5 -0
+For unsigned operation (BPF_DIV and BPF_MOD), for ``BPF_ALU``, 'imm' is first
+interpreted as an unsigned 32-bit value, whereas for ``BPF_ALU64``, 'imm' is
+first sign extended to 64 bits and the result interpreted as an unsigned 64-bit
+value.  For signed operation (BPF_SDIV and BPF_SMOD), for both ``BPF_ALU`` and
+``BPF_ALU64``, 'imm' is interpreted as a signed value.

Probably worth clarifying that in case of S[DIV|MOD] | ALU64 the imm is sign extended from 32 to 64 and interpreted as signed 64-bit.

I thought this before but think "'imm' is interpreted as a signed value"
might be good enough since in this case BPF_ALU and BPF_ALU64 should have the same signed 'imm' value. But what you suggested will make it more clearer. Will do.

Quoted 11 lines

Documentation/bpf/standardization/instruction-set.rst where '(u32)' indicates that the upper 32 bits are zeroed. · line 216

Quoted 5 lines
Documentation/bpf/standardization/instruction-set.rst +4 -0
+
+Instruction BPF_MOVSX does move operation with sign extension. For ``BPF_ALU``
+mode, 8-bit and 16-bit sign extensions to 32-bit are supported. For ``BPF_ALU64``,
+8-bit, 16-bit and 32-bit sign extenstions to 64-bit are supported.

How about:

Instruction BPF_MOVSX does move operation with sign extension.
BPF_ALU | MOVSX sign extendes 8-bit and 16-bit into 32-bit and upper 32-bit are zeroed. BPF_ALU64 | MOVSX sign extends 8-bit, 16-bit and 32-bit into 64-bit.

In previous doc, we already says that BPF_ALU operation has upper 32-bit zeroed. But since this sign extension is special, agree it is worthwhile to mention upper 32-bit zeroed explictly.

Quoted 7 lines

Documentation/bpf/standardization/instruction-set.rst Examples: · line 257

Quoted 5 lines
Documentation/bpf/standardization/instruction-set.rst +3 -0
+``BPF_ALU64 | BPF_TO_LE | BPF_END`` with imm = 16 means::
+
+  dst = bswap16(dst)

Worth spelling out imm 32 and 64 too ?

Will do.

Quoted 12 lines

Documentation/bpf/standardization/instruction-set.rst instructions that transfer data between a register and memory. · line 386

Quoted 9 lines
Documentation/bpf/standardization/instruction-set.rst +7 -0
+The ``BPF_MEMSX`` mode modifier is used to encode sign-extension load
+instructions that transfer data between a register and memory.
+
+``BPF_MEMSX | <size> | BPF_LDX`` means::
+
+  dst = *(sign-extension size *) (src + offset)
+

How about:

Quoted 7 lines

``BPF_MEM | <size> | BPF_LDX`` means::

dst = *(unsigned size *) (src + offset)

``BPF_MEMSX | <size> | BPF_LDX`` means::

dst = *(signed size *) (src + offset)

Will do.

Quoted 1 line
#40 lore raw

RE: [PATCH bpf-next v2 15/15] docs/bpf: Add documentation for new instructions

In reply to Yonghong Song
Quoted 3 lines · Yonghong Song

Add documentation in instruction-set.rst for new instruction encoding and their corresponding operations. Also removed the question related to 'no BPF_SDIV' in bpf_design_QA.rst since we have BPF_SDIV insn now.

Why did you choose to differentiate the instruction by offset instead of using a separate opcode value? I don't think there's any other instructions that do so, and there's spare opcode values as far as I can see.

Using a separate offset works but would end up requiring another column in the IANA registry assuming we have one. So why the extra complexity and inconsistency introduced now?

Dave

#41 lore raw

Re: [PATCH bpf-next v2 15/15] docs/bpf: Add documentation for new instructions

Quoted 6 lines · Dave Thaler
Quoted 3 lines · Yonghong Song

Add documentation in instruction-set.rst for new instruction encoding and their corresponding operations. Also removed the question related to 'no BPF_SDIV' in bpf_design_QA.rst since we have BPF_SDIV insn now.

Quoted 7 lines

Why did you choose to differentiate the instruction by offset instead of using a separate opcode value? I don't think there's any other instructions that do so, and there's spare opcode values as far as I can see.

Using a separate offset works but would end up requiring another column in the IANA registry assuming we have one. So why the extra complexity and inconsistency introduced now?

"another column in IANA" is the last thing to worry about.

#42 lore raw

RE: [PATCH bpf-next v2 15/15] docs/bpf: Add documentation for new instructions

In reply to Yonghong Song
Quoted 233 lines

-----Original Message-----
From: Yonghong Song <[email protected]>
Sent: Wednesday, July 12, 2023 11:09 PM
To: [email protected]
Cc: Alexei Starovoitov <[email protected]>; Andrii Nakryiko <[email protected]>; Daniel Borkmann <[email protected]>; Fangrui Song <[email protected]>; [email protected]
Subject: [PATCH bpf-next v2 15/15] docs/bpf: Add documentation for new instructions

Add documentation in instruction-set.rst for new instruction encoding and their corresponding operations. Also removed the question related to 'no BPF_SDIV' in bpf_design_QA.rst since we have BPF_SDIV insn now.

---
 Documentation/bpf/bpf_design_QA.rst           |   5 -
 .../bpf/standardization/instruction-set.rst   | 100 ++++++++++++------
 2 files changed, 66 insertions(+), 39 deletions(-)
Documentation/bpf/bpf_design_QA.rst +0 -0
index 38372a956d65..eb19c945f4d5 100644
@@ -140,11 +140,6 @@ A: Because if we picked one-to-one relationship to

x64 it would have made it more complicated to support on arm64 and other archs. Also it needs div-by-zero runtime check.

Documentation/bpf/bpf_design_QA.rst +0 -3
-Q: Why there is no BPF_SDIV for signed divide operation?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A: Because it would be rarely used. llvm errors in such case and -prints a

suggestion to use unsigned divide instead.
-
Q: Why BPF has implicit prologue and epilogue?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A: Because architectures like sparc have register windows and in general diff
--git a/Documentation/bpf/standardization/instruction-set.rst b/Documentation/bpf/standardization/instruction-set.rst
index 751e657973f0..367f426d09a1 100644

Documentation/bpf/standardization/instruction-set.rst +0 -0
@@ -154,24 +154,27 @@ otherwise identical operations.
 The 'code' field encodes the operation as below, where 'src' and 'dst' refer  to

the values of the source and destination registers, respectively.

-======== =====
==========================================================
-code value description
-======== =====
==========================================================

Documentation/bpf/standardization/instruction-set.rst +0 -14
-BPF_ADD   0x00   dst += src
-BPF_SUB   0x10   dst -= src
-BPF_MUL   0x20   dst \*= src
-BPF_DIV   0x30   dst = (src != 0) ? (dst / src) : 0
-BPF_OR    0x40   dst \|= src
-BPF_AND   0x50   dst &= src
-BPF_LSH   0x60   dst <<= (src & mask)
-BPF_RSH   0x70   dst >>= (src & mask)
-BPF_NEG   0x80   dst = -src
-BPF_MOD   0x90   dst = (src != 0) ? (dst % src) : dst
-BPF_XOR   0xa0   dst ^= src
-BPF_MOV   0xb0   dst = src
-BPF_ARSH  0xc0   sign extending dst >>= (src & mask)
-BPF_END   0xd0   byte swap operations (see `Byte swap instructions`_

below)
-======== =====
==========================================================
+======== ===== ============
==========================================================

+code      value  offset value  description
+========  =====  ============

==========================================================

Documentation/bpf/standardization/instruction-set.rst +17 -0
+BPF_ADD   0x00   0             dst += src
+BPF_SUB   0x10   0             dst -= src
+BPF_MUL   0x20   0             dst \*= src
+BPF_DIV   0x30   0             dst = (src != 0) ? (dst / src) : 0
+BPF_SDIV  0x30   1             dst = (src != 0) ? (dst s/ src) : 0
+BPF_OR    0x40   0             dst \|= src
+BPF_AND   0x50   0             dst &= src
+BPF_LSH   0x60   0             dst <<= (src & mask)
+BPF_RSH   0x70   0             dst >>= (src & mask)
+BPF_NEG   0x80   0             dst = -src
+BPF_MOD   0x90   0             dst = (src != 0) ? (dst % src) : dst
+BPF_SMOD  0x90   1             dst = (src != 0) ? (dst s% src) : dst
+BPF_XOR   0xa0   0             dst ^= src
+BPF_MOV   0xb0   0             dst = src
+BPF_MOVSX 0xb0   8/16/32       dst = (s8,16,s32)src
+BPF_ARSH  0xc0   0             sign extending dst >>= (src & mask)
+BPF_END   0xd0   0             byte swap operations (see `Byte swap

instructions`_ below)

+========  =====  ============
+==========================================================

Underflow and overflow are allowed during arithmetic operations, meaning
the 64-bit or 32-bit value will wrap. If eBPF program execution would @@ - 198,11 +201,19 @@ where '(u32)' indicates that the upper 32 bits are zeroed.

dst = dst ^ imm32

-Also note that the division and modulo operations are unsigned. Thus, for - ``BPF_ALU``, 'imm' is first interpreted as an unsigned 32-bit value, whereas - for ``BPF_ALU64``, 'imm' is first sign extended to 64 bits and the result - interpreted as an unsigned 64-bit value. There are no instructions for -signed division or modulo.

Documentation/bpf/standardization/instruction-set.rst +4 -0
+Note that most instructions have instruction offset of 0. But three
+instructions (BPF_SDIV, BPF_SMOD, BPF_MOVSX) have non-zero offset.
+
+The devision and modulo operations support both unsigned and signed

flavors.

Documentation/bpf/standardization/instruction-set.rst +5 -0
+For unsigned operation (BPF_DIV and BPF_MOD), for ``BPF_ALU``, 'imm' is
+first interpreted as an unsigned 32-bit value, whereas for
+``BPF_ALU64``, 'imm' is first sign extended to 64 bits and the result
+interpreted as an unsigned 64-bit value.  For signed operation
+(BPF_SDIV and BPF_SMOD), for both ``BPF_ALU`` and ``BPF_ALU64``, 'imm'

is interpreted as a signed value.

Documentation/bpf/standardization/instruction-set.rst +4 -0
+
+Instruction BPF_MOVSX does move operation with sign extension. For
+``BPF_ALU`` mode, 8-bit and 16-bit sign extensions to 32-bit are
+supported. For ``BPF_ALU64``, 8-bit, 16-bit and 32-bit sign extenstions to

64-bit are supported.

Shift operations use a mask of 0x3F (63) for 64-bit operations and 0x1F (31)
for 32-bit operations.

Documentation/bpf/standardization/instruction-set.rst +2 -2
@@ -210,21 +221,23 @@ for 32-bit operations.
 Byte swap instructions
 ~~~~~~~~~~~~~~~~~~~~~~
 
-The byte swap instructions use an instruction class of ``BPF_ALU`` and a 4-bit
-'code' field of ``BPF_END``.
+The byte swap instructions use instruction classes of ``BPF_ALU`` and
+``BPF_ALU64`` and a 4-bit 'code' field of ``BPF_END``.
 
 The byte swap instructions operate on the destination register  only and do

not use a separate source register or immediate value.

-The 1-bit source operand field in the opcode is used to select what byte - order the operation convert from or to:

Documentation/bpf/standardization/instruction-set.rst +3 -1
+For ``BPF_ALU``, the 1-bit source operand field in the opcode is used
+to select what byte order the operation convert from or to. For
+``BPF_ALU64``, the 1-bit source operand field in the opcode is not used.
 
-=========  =====

=================================================
-source value description
-========= =====
=================================================

Documentation/bpf/standardization/instruction-set.rst +0 -3
-BPF_TO_LE  0x00   convert between host byte order and little endian
-BPF_TO_BE  0x08   convert between host byte order and big endian
-=========  =====

=================================================
+========= ========= =====
=================================================

+class      source     value  description
+=========  =========  =====

=================================================
+BPF_ALU BPF_TO_LE 0x00 convert between host byte order and little
endian
+BPF_ALU BPF_TO_BE 0x08 convert between host byte order and big
endian

+BPF_ALU64  BPF_TO_LE  0x00   do byte swap unconditionally
+=========  =========  =====

+=================================================

The 'imm' field encodes the width of the swap operations. The following
widths are supported: 16, 32 and 64.

Documentation/bpf/standardization/instruction-set.rst +4 -0
@@ -239,6 +252,10 @@ Examples:
 
   dst = htobe64(dst)
 
+``BPF_ALU64 | BPF_TO_LE | BPF_END`` with imm = 16 means::
+
+  dst = bswap16(dst)
+
 Jump instructions
 -----------------
 
@@ -249,7 +266,8 @@ The 'code' field encodes the operation as below:
 ========  =====  ===  ===========================================

=========================================

 code      value  src  description                                  notes
 ========  =====  ===  ===========================================

=========================================

Documentation/bpf/standardization/instruction-set.rst +7 -1
-BPF_JA    0x0    0x0  PC += offset                                 BPF_JMP only
+BPF_JA    0x0    0x0  PC += offset                                 BPF_JMP class
+BPF_JA    0x0    0x0  PC += imm                                    BPF_JMP32 class
 BPF_JEQ   0x1    any  PC += offset if dst == src
 BPF_JGT   0x2    any  PC += offset if dst > src                    unsigned
 BPF_JGE   0x3    any  PC += offset if dst >= src                   unsigned
@@ -278,6 +296,10 @@ Example:
 
 where 's>=' indicates a signed '>=' comparison.
 
+Note there are two flavors of BPF_JA instrions. BPF_JMP class permits
+16-bit jump offset while
+BPF_JMP32 permits 32-bit jump offset. A >16bit conditional jmp can be
+converted to a <16bit conditional jmp plus a 32-bit unconditional jump.
+
 Helper functions
 ~~~~~~~~~~~~~~~~
@@ -320,6 +342,7 @@ The mode modifier is one of:
   BPF_ABS        0x20   legacy BPF packet access (absolute)   `Legacy BPF

Packet access instructions`_
BPF_IND 0x40 legacy BPF packet access (indirect) `Legacy BPF Packet
access instructions`_
BPF_MEM 0x60 regular load and store operations `Regular load and
store operations`_
+ BPF_MEMSX 0x80 sign-extension load operations `Sign-extension
load operations`_

   BPF_ATOMIC     0xc0   atomic operations                     `Atomic operations`_
   =============  =====  ====================================

=============

Documentation/bpf/standardization/instruction-set.rst +0 -0
@@ -354,6 +377,15 @@ instructions that transfer data between a register

and memory.

Where size is one of: ``BPF_B``, ``BPF_H``, ``BPF_W``, or ``BPF_DW``.

Documentation/bpf/standardization/instruction-set.rst +9 -1
+The ``BPF_MEMSX`` mode modifier is used to encode sign-extension load
+instructions that transfer data between a register and memory.
+
+``BPF_MEMSX | <size> | BPF_LDX`` means::
+
+  dst = *(sign-extension size *) (src + offset)
+
+Where size is one of: ``BPF_B``, ``BPF_H`` or ``BPF_W``.
+
 Atomic operations
 -----------------
 
--
Quoted 2 lines

2.34.1

Adding [email protected] to the To line since this is proposing a chance to the ISA draft.

Dave

maintainer
#43 lore raw
In reply to Yonghong Song
Quoted 11 lines · Yonghong Song
Quoted 11 lines

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:
. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

Quoted 6 lines
Quoted 6 lines

This patch set added kernel support for insns proposed in [1] except BPF_ST which already has full kernel support. Beside the above proposed insns, LLVM will generate BPF_ST insn as well under -mcpu=v4 ([2]).

The patchset implements interpreter and jit support for these new insns as well as necessary verifier support.

Hi Yonghong,

Sorry for delayed response, I'm still in the middle of the series. I've tested this patch-set using updated LLVM version and have a few notes:
- kernel/bpf/disasm.c needs an update to handle new instructions;
- test_verifier test 'invalid 64-bit BPF_END' from basic_instr.c is failing because '.code = BPF_ALU64 | BPF_END | BPF_TO_LE' is now valid;
- I've looked through the usage of BPF_LDX and found that there is a function seccomp.c:seccomp_check_filter(), that directly checks possible CLASS / CODE combinations. Should this function be updated to handle new instructions?

Thanks,
Eduard

Quoted 61 lines
Quoted 61 lines

To test this patch set, you need to have latest llvm from 'main' branch of llvm-project repo and apply [2] on top of it.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://reviews.llvm.org/D144829

Changelogs:
RFCv1 -> v2:
. add more verifier supports for signed extend load and mov insns. . rename some insn names to be more consistent with intel practice. . add cpuv4 test runner for test progs.
. add more unit and C tests.
. add documentation.

Yonghong Song (15):
bpf: Support new sign-extension load insns
bpf: Fix sign-extension ctx member accesses
bpf: Support new sign-extension mov insns
bpf: Support new unconditional bswap instruction
bpf: Support new signed div/mod instructions.
bpf: Fix jit blinding with new sdiv/smov insns
bpf: Support new 32bit offset jmp instruction selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing selftests/bpf: Add unit tests for new sign-extension load insns selftests/bpf: Add unit tests for new sign-extension mov insns selftests/bpf: Add unit tests for new bswap insns selftests/bpf: Add unit tests for new sdiv/smod insns selftests/bpf: Add unit tests for new gotol insn selftests/bpf: Test ldsx with more complex cases docs/bpf: Add documentation for new instructions

 Documentation/bpf/bpf_design_QA.rst           |   5 -
 .../bpf/standardization/instruction-set.rst   | 100 ++-
 arch/x86/net/bpf_jit_comp.c                   | 131 ++-
 include/linux/filter.h                        |  14 +-
 include/uapi/linux/bpf.h                      |   1 +
 kernel/bpf/cgroup.c                           |  14 +-
 kernel/bpf/core.c                             | 174 +++-
 kernel/bpf/verifier.c                         | 315 ++++++--
 tools/include/uapi/linux/bpf.h                |   1 +
 tools/testing/selftests/bpf/.gitignore        |   2 +
 tools/testing/selftests/bpf/Makefile          |  18 +-
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |   9 +-
 .../selftests/bpf/prog_tests/test_ldsx_insn.c |  88 ++
 .../selftests/bpf/prog_tests/verifier.c       |  10 +
 .../selftests/bpf/progs/test_ldsx_insn.c      |  75 ++
 .../selftests/bpf/progs/verifier_bswap.c      |  45 ++
 .../selftests/bpf/progs/verifier_gotol.c      |  30 +
 .../selftests/bpf/progs/verifier_ldsx.c       | 115 +++
 .../selftests/bpf/progs/verifier_movsx.c      | 177 ++++
 .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
 20 files changed, 1929 insertions(+), 158 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_ldsx_insn.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_ldsx_insn.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bswap.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotol.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_ldsx.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_movsx.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c
#44 lore raw
Quoted 4 lines · Eduard Zingerman

- I've looked through the usage of BPF_LDX and found that there is a function seccomp.c:seccomp_check_filter(), that directly checks possible CLASS / CODE combinations. Should this function be updated to handle new instructions?

This is classic bpf. Why would it change? What is the concern?

maintainer
#45 lore raw
Quoted 7 lines · Alexei Starovoitov
Quoted 4 lines · Eduard Zingerman

- I've looked through the usage of BPF_LDX and found that there is a function seccomp.c:seccomp_check_filter(), that directly checks possible CLASS / CODE combinations. Should this function be updated to handle new instructions?

This is classic bpf. Why would it change? What is the concern?

Sorry, I missed the call to bpf_check_classic() in filter.c:bpf_prepare_filter().

#46 lore raw
In reply to Eduard Zingerman
Quoted 19 lines · Eduard Zingerman
Quoted 11 lines · Yonghong Song
Quoted 11 lines

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:
. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

Quoted 6 lines
Quoted 6 lines

This patch set added kernel support for insns proposed in [1] except BPF_ST which already has full kernel support. Beside the above proposed insns, LLVM will generate BPF_ST insn as well under -mcpu=v4 ([2]).

The patchset implements interpreter and jit support for these new insns as well as necessary verifier support.

Quoted 6 lines

Hi Yonghong,

Sorry for delayed response, I'm still in the middle of the series. I've tested this patch-set using updated LLVM version and have a few notes:
- kernel/bpf/disasm.c needs an update to handle new instructions;

Thanks for pointing this out. My bad. I have disasm.c patch in RFCv1, but somehow during cherry-pick process, I missed it in v2. Will add it back to next revision.

Quoted 3 lines

- test_verifier test 'invalid 64-bit BPF_END' from basic_instr.c is failing because '.code = BPF_ALU64 | BPF_END | BPF_TO_LE' is now valid;

I didn't run 'test_verifier'. Thanks for pointing it out. Will run it and fix the problem in the next revision.

Quoted 70 lines

- I've looked through the usage of BPF_LDX and found that there is a function seccomp.c:seccomp_check_filter(), that directly checks possible CLASS / CODE combinations. Should this function be updated to handle new instructions?

Thanks,
Eduard

Quoted 61 lines
Quoted 61 lines

To test this patch set, you need to have latest llvm from 'main' branch of llvm-project repo and apply [2] on top of it.

   [1] https://lore.kernel.org/bpf/[email protected]/
   [2] https://reviews.llvm.org/D144829

Changelogs:
RFCv1 -> v2:
. add more verifier supports for signed extend load and mov insns. . rename some insn names to be more consistent with intel practice. . add cpuv4 test runner for test progs.
. add more unit and C tests.
. add documentation.

Yonghong Song (15):
bpf: Support new sign-extension load insns
bpf: Fix sign-extension ctx member accesses
bpf: Support new sign-extension mov insns
bpf: Support new unconditional bswap instruction
bpf: Support new signed div/mod instructions.
bpf: Fix jit blinding with new sdiv/smov insns
bpf: Support new 32bit offset jmp instruction selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing selftests/bpf: Add unit tests for new sign-extension load insns selftests/bpf: Add unit tests for new sign-extension mov insns selftests/bpf: Add unit tests for new bswap insns selftests/bpf: Add unit tests for new sdiv/smod insns selftests/bpf: Add unit tests for new gotol insn selftests/bpf: Test ldsx with more complex cases docs/bpf: Add documentation for new instructions

  Documentation/bpf/bpf_design_QA.rst           |   5 -
  .../bpf/standardization/instruction-set.rst   | 100 ++-
  arch/x86/net/bpf_jit_comp.c                   | 131 ++-
  include/linux/filter.h                        |  14 +-
  include/uapi/linux/bpf.h                      |   1 +
  kernel/bpf/cgroup.c                           |  14 +-
  kernel/bpf/core.c                             | 174 +++-
  kernel/bpf/verifier.c                         | 315 ++++++--
  tools/include/uapi/linux/bpf.h                |   1 +
  tools/testing/selftests/bpf/.gitignore        |   2 +
  tools/testing/selftests/bpf/Makefile          |  18 +-
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |   9 +-
  .../selftests/bpf/prog_tests/test_ldsx_insn.c |  88 ++
  .../selftests/bpf/prog_tests/verifier.c       |  10 +
  .../selftests/bpf/progs/test_ldsx_insn.c      |  75 ++
  .../selftests/bpf/progs/verifier_bswap.c      |  45 ++
  .../selftests/bpf/progs/verifier_gotol.c      |  30 +
  .../selftests/bpf/progs/verifier_ldsx.c       | 115 +++
  .../selftests/bpf/progs/verifier_movsx.c      | 177 ++++
  .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
  20 files changed, 1929 insertions(+), 158 deletions(-)

create mode 100644 tools/testing/selftests/bpf/prog_tests/test_ldsx_insn.c create mode 100644 tools/testing/selftests/bpf/progs/test_ldsx_insn.c create mode 100644 tools/testing/selftests/bpf/progs/verifier_bswap.c create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotol.c create mode 100644 tools/testing/selftests/bpf/progs/verifier_ldsx.c create mode 100644 tools/testing/selftests/bpf/progs/verifier_movsx.c create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c

#47 lore raw

Hi Yonghong.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

I am adding the V4 BPF instructions to binutils. Where is the precise "pseudo-c" syntax used by the new instructions documented?

For ALU sdiv/smod we are using:

rd s/= rs
rd s%= rs

For ALU32 sdiv/smod we are using:

wd s/= ws
wd s%= ws

For ALU movs instruction I just made up:

rd s= (i8) rs
rd s= (i16) rs
rd s= (i32) rs

For ALU32 movs I just made up:

wd s= (i8) ws
wd s= (i16) ws
wd s= (i32) ws

Thanks!

#48 lore raw
Quoted 11 lines

Hi Yonghong.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

I am adding the V4 BPF instructions to binutils. Where is the precise "pseudo-c" syntax used by the new instructions documented?

I looked at the tests in https://reviews.llvm.org/D144829 and:

Quoted 4 lines

For ALU sdiv/smod we are using:

rd s/= rs
rd s%= rs

Looks like I chose wisely, just by chance 8-)

Quoted 5 lines

For ALU movs instruction I just made up:

   rd s= (i8) rs
   rd s= (i16) rs
   rd s= (i32) rs

Just changed that in binutils [1] to

rd = (s8) rs
rd = (s16) rs
rd = (s32) rs

Quoted 5 lines

For ALU32 movs I just made up:

   wd s= (i8) ws
   wd s= (i16) ws
   wd s= (i32) ws

Just changed that in binutils [1] to

wd = (s8) ws
wd = (s16) ws
wd = (s32) ws

[1] https://sourceware.org/pipermail/binutils/2023-July/128544.html

#49 lore raw
Quoted 39 lines
Quoted 11 lines

Hi Yonghong.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

I am adding the V4 BPF instructions to binutils. Where is the precise "pseudo-c" syntax used by the new instructions documented?

I looked at the tests in https://reviews.llvm.org/D144829 and:

Quoted 4 lines

For ALU sdiv/smod we are using:

rd s/= rs
rd s%= rs

Looks like I chose wisely, just by chance 8-)

Quoted 5 lines

For ALU movs instruction I just made up:

   rd s= (i8) rs
   rd s= (i16) rs
   rd s= (i32) rs

Just changed that in binutils [1] to

rd = (s8) rs
rd = (s16) rs
rd = (s32) rs

Quoted 5 lines

For ALU32 movs I just made up:

   wd s= (i8) ws
   wd s= (i16) ws
   wd s= (i32) ws
Quoted 7 lines

Just changed that in binutils [1] to

wd = (s8) ws
wd = (s16) ws
wd = (s32) ws

[1] https://sourceware.org/pipermail/binutils/2023-July/128544.html

And finally for byte swap instructions:

rd = bswap16 rd
rd = bswap32 rd
rd = bswap64 rd

https://sourceware.org/pipermail/binutils/2023-July/128546.html

So, at this point we should have support for all the new BPF V4 instructions in the binutils opcodes, assembler and disassembler.

We are working now in getting GCC making good use of them. Salud!

#50 lore raw
Quoted 53 lines · Jose E. Marchesi
Quoted 39 lines
Quoted 11 lines

Hi Yonghong.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

I am adding the V4 BPF instructions to binutils. Where is the precise "pseudo-c" syntax used by the new instructions documented?

I looked at the tests in https://reviews.llvm.org/D144829 and:

Quoted 4 lines

For ALU sdiv/smod we are using:

rd s/= rs
rd s%= rs

Looks like I chose wisely, just by chance 8-)

Quoted 5 lines

For ALU movs instruction I just made up:

    rd s= (i8) rs
    rd s= (i16) rs
    rd s= (i32) rs

Just changed that in binutils [1] to

   rd = (s8) rs
   rd = (s16) rs
   rd = (s32) rs
Quoted 5 lines

For ALU32 movs I just made up:

    wd s= (i8) ws
    wd s= (i16) ws
    wd s= (i32) ws
Quoted 7 lines

Just changed that in binutils [1] to

   wd = (s8) ws
   wd = (s16) ws
   wd = (s32) ws

[1] https://sourceware.org/pipermail/binutils/2023-July/128544.html

And finally for byte swap instructions:

rd = bswap16 rd
rd = bswap32 rd
rd = bswap64 rd

Quoted 6 lines

https://sourceware.org/pipermail/binutils/2023-July/128546.html

So, at this point we should have support for all the new BPF V4 instructions in the binutils opcodes, assembler and disassembler.

We are working now in getting GCC making good use of them.

Sounds great. Thanks!
I plan to push cpu v4 to llvm18 which should be available in the next week.

Quoted 1 line

Salud!