Skip to contentSkip to conversation

bpf: Support new insns from cpu v4

RFC PATCH Superseded Yonghong Song 5 participants 22 messages Jun 29, 2023 – Jul 3, 2023 1 message not in the archive

Subsystems (7)
x86 Kselftest BPF BPF selftests +3 more subsystems

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover bpf: Support new insns from cpu v4

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:
. sign extended load
. sign extended mov
. bswap
. signed div/mod
. ja with 32-bit offset

This patch set added kernel support for insns proposed in [1] except BPF_ST which already has full kernel support. Beside the above proposed insns, LLVM will generate BPF_ST insn as well under -mcpu=v4 ([2]).

The patchset implements interpreter and jit support for these new insns. It has minimum verifier support in order to pass bpf selftests. More work will be required to cover verification and other aspects (e.g. blinding, etc.).

I send the patch set earlier and want to get some early feedbacks before I dig into verifier details. Patch 3 (sign-extension mov insns) prompts me to think whether we should have unsign-extension mov insns as well (see Patch 3 commit message for details). The last patch contains some statistics of how many new insns are actually generated in selftests. Please see each individual patch for details.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://reviews.llvm.org/D144829

Yonghong Song (13):
bpf: Support new sign-extension load insns
bpf: Add verifier support for sign-extension load insns
bpf: Support new sign-extension mov insns
bpf: Support new unconditional bswap instruction
bpf: Support new signed div/mod instructions.
bpf: Support new 32bit offset jmp instruction
bpf: Add kernel/bpftool asm support for new instructions selftests/bpf: Add unit tests for new sign-extension load insns selftests/bpf: Add unit tests for new sign-extension mov insns selftests/bpf: Add unit tests for new bswap insns selftests/bpf: Add unit tests for new sdiv/smod insns selftests/bpf: Add unit tests for new gotol insn selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing

 arch/x86/net/bpf_jit_comp.c                   | 130 ++-
 include/uapi/linux/bpf.h                      |   1 +
 kernel/bpf/core.c                             | 170 +++-
 kernel/bpf/disasm.c                           |  57 +-
 kernel/bpf/verifier.c                         | 100 ++-
 tools/include/uapi/linux/bpf.h                |   1 +
 tools/testing/selftests/bpf/Makefile          |  18 +-
 .../selftests/bpf/prog_tests/verifier.c       |  10 +
 .../selftests/bpf/progs/verifier_bswap.c      |  45 ++
 .../selftests/bpf/progs/verifier_gotol.c      |  30 +
 .../selftests/bpf/progs/verifier_lds.c        |  46 ++
 .../selftests/bpf/progs/verifier_movs.c       |  67 ++
 .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
 13 files changed, 1357 insertions(+), 81 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bswap.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotol.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_lds.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_movs.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c
Signature · 1 line
2.34.1
#1 lore raw

RFC PATCH 1/13 bpf: Support new sign-extension load insns

Add interpreter/jit support for new sign-extension load insns. Also add basic verifier support so newer instruction can be accepted by verifier.

---
 arch/x86/net/bpf_jit_comp.c    | 31 ++++++++++++++++++++++++++++++-
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/core.c              | 13 +++++++++++++
 kernel/bpf/verifier.c          |  8 ++++++--
 tools/include/uapi/linux/bpf.h |  1 +
 5 files changed, 51 insertions(+), 3 deletions(-)
arch/x86/net/bpf_jit_comp.c +30 -1
index 438adb695daa..70d6a2c289ec 100644
@@ -779,6 +779,29 @@ static void emit_ldx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 	*pprog = prog;
 }
 
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_lds(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{
+	u8 *prog = *pprog;
+
+	switch (size) {
+	case BPF_B:
+		/* Emit 'movsx rax, byte ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBE);
+		break;
+	case BPF_H:
+		/* Emit 'movsx rax, word ptr [rax + off]' */
+		EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x0F, 0xBF);
+		break;
+	case BPF_W:
+		/* Emit 'movsx rax, dword ptr [rax+0x14]' */
+		EMIT2(add_2mod(0x48, src_reg, dst_reg), 0x63);
+		break;
+	}
+	emit_insn_suffix(&prog, src_reg, dst_reg, off);
+	*pprog = prog;
+}
+
 /* STX: *(u8*)(dst_reg + off) = src_reg */
 static void emit_stx(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
 {
@@ -1370,6 +1393,9 @@ st:			if (is_imm8(insn->off))
 		case BPF_LDX | BPF_PROBE_MEM | BPF_W:
 		case BPF_LDX | BPF_MEM | BPF_DW:
 		case BPF_LDX | BPF_PROBE_MEM | BPF_DW:
+		case BPF_LDX | BPF_MEMS | BPF_B:
+		case BPF_LDX | BPF_MEMS | BPF_H:
+		case BPF_LDX | BPF_MEMS | BPF_W:
 			insn_off = insn->off;
 
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
@@ -1415,7 +1441,10 @@ st:			if (is_imm8(insn->off))
 				start_of_ldx = prog;
 				end_of_jmp[-1] = start_of_ldx - end_of_jmp;
 			}
-			emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			if (BPF_MODE(insn->code) == BPF_MEMS)
+				emit_lds(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
+			else
+				emit_ldx(&prog, BPF_SIZE(insn->code), dst_reg, src_reg, insn_off);
 			if (BPF_MODE(insn->code) == BPF_PROBE_MEM) {
 				struct exception_table_entry *ex;
 				u8 *_insn = image + proglen + (start_of_ldx - temp);
include/uapi/linux/bpf.h +1 -0
index 60a9d59beeab..b28109bc5c54 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMS	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
kernel/bpf/core.c +13 -0
index dc85240a0134..acb7abd2eba0 100644
@@ -1610,6 +1610,9 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(LDX, MEM, H),			\
 	INSN_3(LDX, MEM, W),			\
 	INSN_3(LDX, MEM, DW),			\
+	INSN_3(LDX, MEMS, B),			\
+	INSN_3(LDX, MEMS, H),			\
+	INSN_3(LDX, MEMS, W),			\
 	/*   Immediate based. */		\
 	INSN_3(LD, IMM, DW)
 
@@ -1942,6 +1945,16 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	LDST(DW, u64)
 #undef LDST
 
+#define LDS(SIZEOP, SIZE)						\
+	LDX_MEMS_##SIZEOP:						\
+		DST = *(SIZE *)(unsigned long) (SRC + insn->off);	\
+		CONT;
+
+	LDS(B,   s8)
+	LDS(H,  s16)
+	LDS(W,  s32)
+#undef LDS
+
 #define ATOMIC_ALU_OP(BOP, KOP)						\
 		case BOP:						\
 			if (BPF_SIZE(insn->code) == BPF_W)		\
kernel/bpf/verifier.c +6 -2
index 11e54dd8b6dd..212c367e2f46 100644
@@ -16790,7 +16790,8 @@ static int resolve_pseudo_ldimm64(struct bpf_verifier_env *env)
 
 	for (i = 0; i < insn_cnt; i++, insn++) {
 		if (BPF_CLASS(insn->code) == BPF_LDX &&
-		    (BPF_MODE(insn->code) != BPF_MEM || insn->imm != 0)) {
+		    ((BPF_MODE(insn->code) != BPF_MEM && BPF_MODE(insn->code) != BPF_MEMS) ||
+		    insn->imm != 0)) {
 			verbose(env, "BPF_LDX uses reserved fields\n");
 			return -EINVAL;
 		}
@@ -17488,7 +17489,10 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)
 		if (insn->code == (BPF_LDX | BPF_MEM | BPF_B) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_H) ||
 		    insn->code == (BPF_LDX | BPF_MEM | BPF_W) ||
-		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW)) {
+		    insn->code == (BPF_LDX | BPF_MEM | BPF_DW) ||
+		    insn->code == (BPF_LDX | BPF_MEMS | BPF_B) ||
+		    insn->code == (BPF_LDX | BPF_MEMS | BPF_H) ||
+		    insn->code == (BPF_LDX | BPF_MEMS | BPF_W)) {
 			type = BPF_READ;
 		} else if (insn->code == (BPF_STX | BPF_MEM | BPF_B) ||
 			   insn->code == (BPF_STX | BPF_MEM | BPF_H) ||
tools/include/uapi/linux/bpf.h +1 -0
index 60a9d59beeab..b28109bc5c54 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMS	0x80	/* load with sign extension */
 #define BPF_ATOMIC	0xc0	/* atomic memory ops - op type in immediate */
 #define BPF_XADD	0xc0	/* exclusive add - legacy name */
 
Signature · 1 line
2.34.1
#2 lore raw

Re: [RFC PATCH bpf-next 01/13] bpf: Support new sign-extension load insns

Quoted 4 lines · Yonghong Song
arch/x86/net/bpf_jit_comp.c +3 -0
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_lds(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{

...

include/uapi/linux/bpf.h line 22

Quoted 9 lines
include/uapi/linux/bpf.h +1 -0
index 60a9d59beeab..b28109bc5c54 100644
@@ -19,6 +19,7 @@
 
 /* ld/ldx fields */
 #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMS	0x80	/* load with sign extension */

Intel assembly instruction to do sign extending mov is called 'movsx'. Let's adopt SX suffix here and in other patches ?

s/BPF_MEMS/BPF_MEMSX/ here.
s/emit_lds/emit_ldsx/ above.

s/emit_movs_reg/emit_movsx_reg/ in patch 3.

s/bpf_movs_string/bpf_movsx_string/ in patch 7
s/bpf_lds_string/bpf_ldsx_string/ in patch 7.
s/is_movs/is_movsx/ in patch 7.

sdiv/smod can stay as-is.

Naming is hard, of course.

#3 lore raw

Re: [RFC PATCH bpf-next 01/13] bpf: Support new sign-extension load insns

Quoted 22 lines · Alexei Starovoitov
Quoted 4 lines · Yonghong Song
arch/x86/net/bpf_jit_comp.c +3 -0
+/* LDX: dst_reg = *(s8*)(src_reg + off) */
+static void emit_lds(u8 **pprog, u32 size, u32 dst_reg, u32 src_reg, int off)
+{

...

include/uapi/linux/bpf.h line 22

Quoted 9 lines
include/uapi/linux/bpf.h +1 -0
index 60a9d59beeab..b28109bc5c54 100644
@@ -19,6 +19,7 @@
 
  /* ld/ldx fields */
  #define BPF_DW		0x18	/* double word (64-bit) */
+#define BPF_MEMS	0x80	/* load with sign extension */

Intel assembly instruction to do sign extending mov is called 'movsx'. Let's adopt SX suffix here and in other patches ?

s/BPF_MEMS/BPF_MEMSX/ here.
s/emit_lds/emit_ldsx/ above.

Quoted 7 lines

s/emit_movs_reg/emit_movsx_reg/ in patch 3.

s/bpf_movs_string/bpf_movsx_string/ in patch 7
s/bpf_lds_string/bpf_ldsx_string/ in patch 7.
s/is_movs/is_movsx/ in patch 7.

sdiv/smod can stay as-is.

Sounds good to me! Will use the above suggested names in the next revision.

Quoted 2 lines

Naming is hard, of course.

#4 lore raw

RFC PATCH 2/13 bpf: Add verifier support for sign-extension load insns

In reply to Yonghong Song

Add sign-extension load support for map values. This is a minimum change to make selftests passing.

---
 kernel/bpf/verifier.c | 38 ++++++++++++++++++++++++++------------
 1 file changed, 26 insertions(+), 12 deletions(-)
kernel/bpf/verifier.c +26 -12
index 212c367e2f46..6845504d42a5 100644
@@ -6247,7 +6247,7 @@ static int check_stack_access_within_bounds(
  */
 static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regno,
 			    int off, int bpf_size, enum bpf_access_type t,
-			    int value_regno, bool strict_alignment_once)
+			    int value_regno, bool strict_alignment_once, bool sign_ext_ld)
 {
 	struct bpf_reg_state *regs = cur_regs(env);
 	struct bpf_reg_state *reg = regs + regno;
@@ -6316,6 +6316,19 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 				__mark_reg_known(&regs[value_regno], val);
 			} else {
 				mark_reg_unknown(env, regs, value_regno);
+
+				if (sign_ext_ld) {
+					if (size == 1) {
+						regs[value_regno].smax_value = (char)INT_MAX;
+						regs[value_regno].smin_value = (char)INT_MIN;
+					} else if (size == 2) {
+						regs[value_regno].smax_value = (short)INT_MAX;
+						regs[value_regno].smin_value = (short)INT_MIN;
+					} else if (size == 4) {
+						regs[value_regno].smax_value = INT_MAX;
+						regs[value_regno].smin_value = INT_MIN;
+					}
+				}
 			}
 		}
 	} else if (base_type(reg->type) == PTR_TO_MEM) {
@@ -6477,7 +6490,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 	}
 
 	if (!err && size < BPF_REG_SIZE && value_regno >= 0 && t == BPF_READ &&
-	    regs[value_regno].type == SCALAR_VALUE) {
+	    regs[value_regno].type == SCALAR_VALUE && !sign_ext_ld) {
 		/* b/h/w load zero-extends, mark upper bits as known 0 */
 		coerce_reg_to_size(&regs[value_regno], size);
 	}
@@ -6571,17 +6584,17 @@ static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_i
 	 * case to simulate the register fill.
 	 */
 	err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
-			       BPF_SIZE(insn->code), BPF_READ, -1, true);
+			       BPF_SIZE(insn->code), BPF_READ, -1, true, false);
 	if (!err && load_reg >= 0)
 		err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
 				       BPF_SIZE(insn->code), BPF_READ, load_reg,
-				       true);
+				       true, false);
 	if (err)
 		return err;
 
 	/* Check whether we can write into the same memory. */
 	err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off,
-			       BPF_SIZE(insn->code), BPF_WRITE, -1, true);
+			       BPF_SIZE(insn->code), BPF_WRITE, -1, true, false);
 	if (err)
 		return err;
 
@@ -6827,7 +6840,7 @@ static int check_helper_mem_access(struct bpf_verifier_env *env, int regno,
 				return zero_size_allowed ? 0 : -EACCES;
 
 			return check_mem_access(env, env->insn_idx, regno, offset, BPF_B,
-						atype, -1, false);
+						atype, -1, false, false);
 		}
 
 		fallthrough;
@@ -7199,7 +7212,7 @@ static int process_dynptr_func(struct bpf_verifier_env *env, int regno, int insn
 		/* we write BPF_DW bits (8 bytes) at a time */
 		for (i = 0; i < BPF_DYNPTR_SIZE; i += 8) {
 			err = check_mem_access(env, insn_idx, regno,
-					       i, BPF_DW, BPF_WRITE, -1, false);
+					       i, BPF_DW, BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 		}
@@ -7292,7 +7305,7 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id
 
 		for (i = 0; i < nr_slots * 8; i += BPF_REG_SIZE) {
 			err = check_mem_access(env, insn_idx, regno,
-					       i, BPF_DW, BPF_WRITE, -1, false);
+					       i, BPF_DW, BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 		}
@@ -9422,7 +9435,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
 	 */
 	for (i = 0; i < meta.access_size; i++) {
 		err = check_mem_access(env, insn_idx, meta.regno, i, BPF_B,
-				       BPF_WRITE, -1, false);
+				       BPF_WRITE, -1, false, false);
 		if (err)
 			return err;
 	}
@@ -16300,7 +16313,8 @@ static int do_check(struct bpf_verifier_env *env)
 			 */
 			err = check_mem_access(env, env->insn_idx, insn->src_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_READ, insn->dst_reg, false);
+					       BPF_READ, insn->dst_reg, false,
+					       BPF_MODE(insn->code) == BPF_MEMS);
 			if (err)
 				return err;
 
@@ -16337,7 +16351,7 @@ static int do_check(struct bpf_verifier_env *env)
 			/* check that memory (dst_reg + off) is writeable */
 			err = check_mem_access(env, env->insn_idx, insn->dst_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_WRITE, insn->src_reg, false);
+					       BPF_WRITE, insn->src_reg, false, false);
 			if (err)
 				return err;
 
@@ -16362,7 +16376,7 @@ static int do_check(struct bpf_verifier_env *env)
 			/* check that memory (dst_reg + off) is writeable */
 			err = check_mem_access(env, env->insn_idx, insn->dst_reg,
 					       insn->off, BPF_SIZE(insn->code),
-					       BPF_WRITE, -1, false);
+					       BPF_WRITE, -1, false, false);
 			if (err)
 				return err;
 
Signature · 1 line
2.34.1
#5 lore raw

Re: [RFC PATCH bpf-next 02/13] bpf: Add verifier support for sign-extension load insns

kernel test robot noticed the following build warnings:

Hi Yonghong,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Yonghong-Song/bpf-Support-new-sign-extension-load-insns/20230629-144251
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20230629063726.1649316-1-yhs%40fb.com
patch subject: [RFC PATCH bpf-next 02/13] bpf: Add verifier support for sign-extension load insns
config: x86_64-randconfig-x061-20230702 (https://download.01.org/0day-ci/archive/20230702/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230702/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)
kernel/bpf/verifier.c:18560:38: sparse: sparse: subtraction of functions? Share your drugs

>> kernel/bpf/verifier.c:6322:86: sparse: sparse: cast truncates bits from constant value (7fffffff becomes ff)
>> kernel/bpf/verifier.c:6323:86: sparse: sparse: cast truncates bits from constant value (80000000 becomes 0)
>> kernel/bpf/verifier.c:6325:87: sparse: sparse: cast truncates bits from constant value (7fffffff becomes ffff)

kernel/bpf/verifier.c:6326:87: sparse: sparse: cast truncates bits from constant value (80000000 becomes 0) kernel/bpf/verifier.c: note: in included file (through include/linux/bpf.h, include/linux/bpf-cgroup.h):
include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar

vim +6322 kernel/bpf/verifier.c

6241
6242 /* check whether memory at (regno + off) is accessible for t = (read | write)
6243 * if t==write, value_regno is a register which value is stored into memory
6244 * if t==read, value_regno is a register which will receive the value from memory
6245 * if t==write && value_regno==-1, some unknown value is stored into memory
6246 * if t==read && value_regno==-1, don't care what we read from memory
6247 */
6248 static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regno,
6249 int off, int bpf_size, enum bpf_access_type t,
6250 int value_regno, bool strict_alignment_once, bool sign_ext_ld)
6251 {
6252 struct bpf_reg_state *regs = cur_regs(env);
6253 struct bpf_reg_state *reg = regs + regno;
6254 struct bpf_func_state *state;
6255 int size, err = 0;
6256
6257 size = bpf_size_to_bytes(bpf_size);
6258 if (size < 0)
6259 return size;
6260
6261 /* alignment checks will add in reg->off themselves */
6262 err = check_ptr_alignment(env, reg, off, size, strict_alignment_once);
6263 if (err)
6264 return err;
6265
6266 /* for access checks, reg->off is just part of off */
6267 off += reg->off;
6268
6269 if (reg->type == PTR_TO_MAP_KEY) {
6270 if (t == BPF_WRITE) {
6271 verbose(env, "write to change key R%d not allowed\n", regno);
6272 return -EACCES;
6273 }
6274
6275 err = check_mem_region_access(env, regno, off, size,
6276 reg->map_ptr->key_size, false);
6277 if (err)
6278 return err;
6279 if (value_regno >= 0)
6280 mark_reg_unknown(env, regs, value_regno);
6281 } else if (reg->type == PTR_TO_MAP_VALUE) {
6282 struct btf_field *kptr_field = NULL;
6283
6284 if (t == BPF_WRITE && value_regno >= 0 &&
6285 is_pointer_value(env, value_regno)) {
6286 verbose(env, "R%d leaks addr into map\n", value_regno);
6287 return -EACCES;
6288 }
6289 err = check_map_access_type(env, regno, off, size, t);
6290 if (err)
6291 return err;
6292 err = check_map_access(env, regno, off, size, false, ACCESS_DIRECT);
6293 if (err)
6294 return err;
6295 if (tnum_is_const(reg->var_off))
6296 kptr_field = btf_record_find(reg->map_ptr->record,
6297 off + reg->var_off.value, BPF_KPTR);
6298 if (kptr_field) {
6299 err = check_map_kptr_access(env, regno, value_regno, insn_idx, kptr_field);
6300 } else if (t == BPF_READ && value_regno >= 0) {
6301 struct bpf_map *map = reg->map_ptr;
6302
6303 /* if map is read-only, track its contents as scalars */
6304 if (tnum_is_const(reg->var_off) &&
6305 bpf_map_is_rdonly(map) &&
6306 map->ops->map_direct_value_addr) {
6307 int map_off = off + reg->var_off.value;
6308 u64 val = 0;
6309
6310 err = bpf_map_direct_read(map, map_off, size,
6311 &val);
6312 if (err)
6313 return err;
6314
6315 regs[value_regno].type = SCALAR_VALUE;
6316 __mark_reg_known(&regs[value_regno], val);
6317 } else {
6318 mark_reg_unknown(env, regs, value_regno);
6319
6320 if (sign_ext_ld) {
6321 if (size == 1) {

> 6322							regs[value_regno].smax_value = (char)INT_MAX;
> 6323							regs[value_regno].smin_value = (char)INT_MIN;
  6324						} else if (size == 2) {
> 6325							regs[value_regno].smax_value = (short)INT_MAX;

6326 regs[value_regno].smin_value = (short)INT_MIN;
6327 } else if (size == 4) {
6328 regs[value_regno].smax_value = INT_MAX;
6329 regs[value_regno].smin_value = INT_MIN;
6330 }
6331 }
6332 }
6333 }
6334 } else if (base_type(reg->type) == PTR_TO_MEM) {
6335 bool rdonly_mem = type_is_rdonly_mem(reg->type);
6336
6337 if (type_may_be_null(reg->type)) {
6338 verbose(env, "R%d invalid mem access '%s'\n", regno,
6339 reg_type_str(env, reg->type));
6340 return -EACCES;
6341 }
6342
6343 if (t == BPF_WRITE && rdonly_mem) {
6344 verbose(env, "R%d cannot write into %s\n",
6345 regno, reg_type_str(env, reg->type));
6346 return -EACCES;
6347 }
6348
6349 if (t == BPF_WRITE && value_regno >= 0 &&
6350 is_pointer_value(env, value_regno)) {
6351 verbose(env, "R%d leaks addr into mem\n", value_regno);
6352 return -EACCES;
6353 }
6354
6355 err = check_mem_region_access(env, regno, off, size,
6356 reg->mem_size, false);
6357 if (!err && value_regno >= 0 && (t == BPF_READ || rdonly_mem))
6358 mark_reg_unknown(env, regs, value_regno);
6359 } else if (reg->type == PTR_TO_CTX) {
6360 enum bpf_reg_type reg_type = SCALAR_VALUE;
6361 struct btf *btf = NULL;
6362 u32 btf_id = 0;
6363
6364 if (t == BPF_WRITE && value_regno >= 0 &&
6365 is_pointer_value(env, value_regno)) {
6366 verbose(env, "R%d leaks addr into ctx\n", value_regno);
6367 return -EACCES;
6368 }
6369
6370 err = check_ptr_off_reg(env, reg, regno);
6371 if (err < 0)
6372 return err;
6373
6374 err = check_ctx_access(env, insn_idx, off, size, t, &reg_type, &btf,
6375 &btf_id);
6376 if (err)
6377 verbose_linfo(env, insn_idx, "; ");
6378 if (!err && t == BPF_READ && value_regno >= 0) {
6379 /* ctx access returns either a scalar, or a
6380 * PTR_TO_PACKET[_META,_END]. In the latter
6381 * case, we know the offset is zero.
6382 */
6383 if (reg_type == SCALAR_VALUE) {
6384 mark_reg_unknown(env, regs, value_regno);
6385 } else {
6386 mark_reg_known_zero(env, regs,
6387 value_regno);
6388 if (type_may_be_null(reg_type))
6389 regs[value_regno].id = ++env->id_gen;
6390 /* A load of ctx field could have different
6391 * actual load size with the one encoded in the
6392 * insn. When the dst is PTR, it is for sure not
6393 * a sub-register.
6394 */
6395 regs[value_regno].subreg_def = DEF_NOT_SUBREG;
6396 if (base_type(reg_type) == PTR_TO_BTF_ID) {
6397 regs[value_regno].btf = btf;
6398 regs[value_regno].btf_id = btf_id;
6399 }
6400 }
6401 regs[value_regno].type = reg_type;
6402 }
6403
6404 } else if (reg->type == PTR_TO_STACK) {
6405 /* Basic bounds checks. */
6406 err = check_stack_access_within_bounds(env, regno, off, size, ACCESS_DIRECT, t);
6407 if (err)
6408 return err;
6409
6410 state = func(env, reg);
6411 err = update_stack_depth(env, state, off);
6412 if (err)
6413 return err;
6414
6415 if (t == BPF_READ)
6416 err = check_stack_read(env, regno, off, size,
6417 value_regno);
6418 else
6419 err = check_stack_write(env, regno, off, size,
6420 value_regno, insn_idx);
6421 } else if (reg_is_pkt_pointer(reg)) {
6422 if (t == BPF_WRITE && !may_access_direct_pkt_data(env, NULL, t)) {
6423 verbose(env, "cannot write into packet\n");
6424 return -EACCES;
6425 }
6426 if (t == BPF_WRITE && value_regno >= 0 &&
6427 is_pointer_value(env, value_regno)) {
6428 verbose(env, "R%d leaks addr into packet\n",
6429 value_regno);
6430 return -EACCES;
6431 }
6432 err = check_packet_access(env, regno, off, size, false);
6433 if (!err && t == BPF_READ && value_regno >= 0)
6434 mark_reg_unknown(env, regs, value_regno);
6435 } else if (reg->type == PTR_TO_FLOW_KEYS) {
6436 if (t == BPF_WRITE && value_regno >= 0 &&
6437 is_pointer_value(env, value_regno)) {
6438 verbose(env, "R%d leaks addr into flow keys\n",
6439 value_regno);
6440 return -EACCES;
6441 }
6442
6443 err = check_flow_keys_access(env, off, size);
6444 if (!err && t == BPF_READ && value_regno >= 0)
6445 mark_reg_unknown(env, regs, value_regno);
6446 } else if (type_is_sk_pointer(reg->type)) {
6447 if (t == BPF_WRITE) {
6448 verbose(env, "R%d cannot write into %s\n",
6449 regno, reg_type_str(env, reg->type));
6450 return -EACCES;
6451 }
6452 err = check_sock_access(env, insn_idx, regno, off, size, t);
6453 if (!err && value_regno >= 0)
6454 mark_reg_unknown(env, regs, value_regno);
6455 } else if (reg->type == PTR_TO_TP_BUFFER) {
6456 err = check_tp_buffer_access(env, reg, regno, off, size);
6457 if (!err && t == BPF_READ && value_regno >= 0)
6458 mark_reg_unknown(env, regs, value_regno);
6459 } else if (base_type(reg->type) == PTR_TO_BTF_ID &&
6460 !type_may_be_null(reg->type)) {
6461 err = check_ptr_to_btf_access(env, regs, regno, off, size, t,
6462 value_regno);
6463 } else if (reg->type == CONST_PTR_TO_MAP) {
6464 err = check_ptr_to_map_access(env, regs, regno, off, size, t,
6465 value_regno);
6466 } else if (base_type(reg->type) == PTR_TO_BUF) {
6467 bool rdonly_mem = type_is_rdonly_mem(reg->type);
6468 u32 *max_access;
6469
6470 if (rdonly_mem) {
6471 if (t == BPF_WRITE) {
6472 verbose(env, "R%d cannot write into %s\n",
6473 regno, reg_type_str(env, reg->type));
6474 return -EACCES;
6475 }
6476 max_access = &env->prog->aux->max_rdonly_access;
6477 } else {
6478 max_access = &env->prog->aux->max_rdwr_access;
6479 }
6480
6481 err = check_buffer_access(env, reg, regno, off, size, false,
6482 max_access);
6483
6484 if (!err && value_regno >= 0 && (rdonly_mem || t == BPF_READ))
6485 mark_reg_unknown(env, regs, value_regno);
6486 } else {
6487 verbose(env, "R%d invalid mem access '%s'\n", regno,
6488 reg_type_str(env, reg->type));
6489 return -EACCES;
6490 }
6491
6492 if (!err && size < BPF_REG_SIZE && value_regno >= 0 && t == BPF_READ &&
6493 regs[value_regno].type == SCALAR_VALUE && !sign_ext_ld) {
6494 /* b/h/w load zero-extends, mark upper bits as known 0 */
6495 coerce_reg_to_size(&regs[value_regno], size);
6496 }
6497 return err;
6498 }
6499

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#6 lore raw

Re: [RFC PATCH bpf-next 02/13] bpf: Add verifier support for sign-extension load insns

Quoted 119 lines · kernel test robot

Hi Yonghong,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Yonghong-Song/bpf-Support-new-sign-extension-load-insns/20230629-144251
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20230629063726.1649316-1-yhs%40fb.com
patch subject: [RFC PATCH bpf-next 02/13] bpf: Add verifier support for sign-extension load insns
config: x86_64-randconfig-x061-20230702 (https://download.01.org/0day-ci/archive/20230702/[email protected]/config )
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230702/[email protected]/reproduce )

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)
kernel/bpf/verifier.c:18560:38: sparse: sparse: subtraction of functions? Share your drugs

Quoted 3 lines
Quoted 3 lines

kernel/bpf/verifier.c:6322:86: sparse: sparse: cast truncates bits from constant value (7fffffff becomes ff) kernel/bpf/verifier.c:6323:86: sparse: sparse: cast truncates bits from constant value (80000000 becomes 0) kernel/bpf/verifier.c:6325:87: sparse: sparse: cast truncates bits from constant value (7fffffff becomes ffff)

kernel/bpf/verifier.c:6326:87: sparse: sparse: cast truncates bits from constant value (80000000 becomes 0) kernel/bpf/verifier.c: note: in included file (through include/linux/bpf.h, include/linux/bpf-cgroup.h):
include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast to non-scalar include/linux/bpfptr.h:65:40: sparse: sparse: cast from non-scalar

vim +6322 kernel/bpf/verifier.c

   6241	
   6242	/* check whether memory at (regno + off) is accessible for t = (read | write)
   6243	 * if t==write, value_regno is a register which value is stored into memory
   6244	 * if t==read, value_regno is a register which will receive the value from memory
   6245	 * if t==write && value_regno==-1, some unknown value is stored into memory
   6246	 * if t==read && value_regno==-1, don't care what we read from memory
   6247	 */
   6248	static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regno,
   6249				    int off, int bpf_size, enum bpf_access_type t,
   6250				    int value_regno, bool strict_alignment_once, bool sign_ext_ld)
   6251	{
   6252		struct bpf_reg_state *regs = cur_regs(env);
   6253		struct bpf_reg_state *reg = regs + regno;
   6254		struct bpf_func_state *state;
   6255		int size, err = 0;
   6256	
   6257		size = bpf_size_to_bytes(bpf_size);
   6258		if (size < 0)
   6259			return size;
   6260	
   6261		/* alignment checks will add in reg->off themselves */
   6262		err = check_ptr_alignment(env, reg, off, size, strict_alignment_once);
   6263		if (err)
   6264			return err;
   6265	
   6266		/* for access checks, reg->off is just part of off */
   6267		off += reg->off;
   6268	
   6269		if (reg->type == PTR_TO_MAP_KEY) {
   6270			if (t == BPF_WRITE) {
   6271				verbose(env, "write to change key R%d not allowed\n", regno);
   6272				return -EACCES;
   6273			}
   6274	
   6275			err = check_mem_region_access(env, regno, off, size,
   6276						      reg->map_ptr->key_size, false);
   6277			if (err)
   6278				return err;
   6279			if (value_regno >= 0)
   6280				mark_reg_unknown(env, regs, value_regno);
   6281		} else if (reg->type == PTR_TO_MAP_VALUE) {
   6282			struct btf_field *kptr_field = NULL;
   6283	
   6284			if (t == BPF_WRITE && value_regno >= 0 &&
   6285			    is_pointer_value(env, value_regno)) {
   6286				verbose(env, "R%d leaks addr into map\n", value_regno);
   6287				return -EACCES;
   6288			}
   6289			err = check_map_access_type(env, regno, off, size, t);
   6290			if (err)
   6291				return err;
   6292			err = check_map_access(env, regno, off, size, false, ACCESS_DIRECT);
   6293			if (err)
   6294				return err;
   6295			if (tnum_is_const(reg->var_off))
   6296				kptr_field = btf_record_find(reg->map_ptr->record,
   6297							     off + reg->var_off.value, BPF_KPTR);
   6298			if (kptr_field) {
   6299				err = check_map_kptr_access(env, regno, value_regno, insn_idx, kptr_field);
   6300			} else if (t == BPF_READ && value_regno >= 0) {
   6301				struct bpf_map *map = reg->map_ptr;
   6302	
   6303				/* if map is read-only, track its contents as scalars */
   6304				if (tnum_is_const(reg->var_off) &&
   6305				    bpf_map_is_rdonly(map) &&
   6306				    map->ops->map_direct_value_addr) {
   6307					int map_off = off + reg->var_off.value;
   6308					u64 val = 0;
   6309	
   6310					err = bpf_map_direct_read(map, map_off, size,
   6311								  &val);
   6312					if (err)
   6313						return err;
   6314	
   6315					regs[value_regno].type = SCALAR_VALUE;
   6316					__mark_reg_known(&regs[value_regno], val);
   6317				} else {
   6318					mark_reg_unknown(env, regs, value_regno);
   6319	
   6320					if (sign_ext_ld) {
   6321						if (size == 1) {
Quoted 5 lines
Quoted 2 lines

6322 regs[value_regno].smax_value = (char)INT_MAX;
6323 regs[value_regno].smin_value = (char)INT_MIN;

6324 } else if (size == 2) {

Quoted 1 line

6325 regs[value_regno].smax_value = (short)INT_MAX;

6326 regs[value_regno].smin_value = (short)INT_MIN;

The above codes are intentional. if this caused compilation warning, I will try to rewrite the silence the warnings.
Thanks!

Quoted 7 lines
   6327						} else if (size == 4) {
   6328							regs[value_regno].smax_value = INT_MAX;
   6329							regs[value_regno].smin_value = INT_MIN;
   6330						}
   6331					}
   6332				}
   6333			}

[...]

#7 lore raw

RFC PATCH 3/13 bpf: Support new sign-extension mov insns

In reply to Yonghong Song

Add interpreter/jit support for new sign-extension mov insns. The verifier support is basic and the better register range calculation will be done in the future revision.

NOTE: currently new insns support:
ALU:

      dst = (s8)src
      dst = (s16)src
  ALU64:
      dst = (s8)src
      dst = (s16)src
      dst = (s32)src
while unsigned mov insns support:
  ALU:
      dst = (u32)src

ALU64:
dst = src
Should we support more unsigned-extension mov insns like below?
ALU:

      dst = (u8)src
      dst = (u16)src
  ALU64:
      dst = (u8)src
      dst = (u16)src
      dst = (u32)src
---
 arch/x86/net/bpf_jit_comp.c | 43 ++++++++++++++++++++++++++++++++++---
 kernel/bpf/core.c           | 28 ++++++++++++++++++++++--
 kernel/bpf/verifier.c       | 14 +++++++++++-
 3 files changed, 79 insertions(+), 6 deletions(-)
arch/x86/net/bpf_jit_comp.c +40 -3
index 70d6a2c289ec..7c85d1b01931 100644
@@ -701,6 +701,38 @@ static void emit_mov_reg(u8 **pprog, bool is64, u32 dst_reg, u32 src_reg)
 	*pprog = prog;
 }
 
+static void emit_movs_reg(u8 **pprog, int num_bits, bool is64, u32 dst_reg,
+			  u32 src_reg)
+{
+	u8 *prog = *pprog;
+
+	if (is64) {
+		/* movs[b,w,l]q dst, src */
+		if (num_bits == 8)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 16)
+			EMIT4(add_2mod(0x48, src_reg, dst_reg), 0x0f, 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		else if (num_bits == 32)
+			EMIT3(add_2mod(0x48, src_reg, dst_reg), 0x63,
+			      add_2reg(0xC0, src_reg, dst_reg));
+	} else {
+		/* movs[b,w]l dst, src */
+		if (num_bits == 8) {
+			EMIT4(add_2mod(0x40, src_reg, dst_reg), 0x0f, 0xbe,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		} else if (num_bits == 16) {
+			if (is_ereg(dst_reg) || is_ereg(src_reg))
+				EMIT1(add_2mod(0x40, src_reg, dst_reg));
+			EMIT3(add_2mod(0x0f, src_reg, dst_reg), 0xbf,
+			      add_2reg(0xC0, src_reg, dst_reg));
+		}
+	}
+
+	*pprog = prog;
+}
+
 /* Emit the suffix (ModR/M etc) for addressing *(ptr_reg + off) and val_reg */
 static void emit_insn_suffix(u8 **pprog, u32 ptr_reg, u32 val_reg, int off)
 {
@@ -1051,9 +1083,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 
 		case BPF_ALU64 | BPF_MOV | BPF_X:
 		case BPF_ALU | BPF_MOV | BPF_X:
-			emit_mov_reg(&prog,
-				     BPF_CLASS(insn->code) == BPF_ALU64,
-				     dst_reg, src_reg);
+			if (insn->off == 0)
+				emit_mov_reg(&prog,
+					     BPF_CLASS(insn->code) == BPF_ALU64,
+					     dst_reg, src_reg);
+			else
+				emit_movs_reg(&prog, insn->off,
+					      BPF_CLASS(insn->code) == BPF_ALU64,
+					      dst_reg, src_reg);
 			break;
 
 			/* neg dst */
kernel/bpf/core.c +26 -2
index acb7abd2eba0..72ee246ac3af 100644
@@ -61,6 +61,7 @@
 #define AX	regs[BPF_REG_AX]
 #define ARG1	regs[BPF_REG_ARG1]
 #define CTX	regs[BPF_REG_CTX]
+#define OFF	insn->off
 #define IMM	insn->imm
 
 struct bpf_mem_alloc bpf_global_ma;
@@ -1736,13 +1737,36 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 		DST = -DST;
 		CONT;
 	ALU_MOV_X:
-		DST = (u32) SRC;
+		switch (OFF) {
+		case 0:
+			DST = (u32) SRC;
+			break;
+		case 8:
+			DST = (s8) SRC;
+			break;
+		case 16:
+			DST = (s16) SRC;
+			break;
+		}
 		CONT;
 	ALU_MOV_K:
 		DST = (u32) IMM;
 		CONT;
 	ALU64_MOV_X:
-		DST = SRC;
+		switch (OFF) {
+		case 0:
+			DST = SRC;
+			break;
+		case 8:
+			DST = (s8) SRC;
+			break;
+		case 16:
+			DST = (s16) SRC;
+			break;
+		case 32:
+			DST = (s32) SRC;
+			break;
+		}
 		CONT;
 	ALU64_MOV_K:
 		DST = IMM;
kernel/bpf/verifier.c +13 -1
index 6845504d42a5..5c5b37b6b39a 100644
@@ -12926,11 +12926,23 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 	} else if (opcode == BPF_MOV) {
 
 		if (BPF_SRC(insn->code) == BPF_X) {
-			if (insn->imm != 0 || insn->off != 0) {
+			if (insn->imm != 0) {
 				verbose(env, "BPF_MOV uses reserved fields\n");
 				return -EINVAL;
 			}
 
+			if (BPF_CLASS(insn->code) == BPF_ALU) {
+				if (insn->off != 0 && insn->off != 8 && insn->off != 16) {
+					verbose(env, "BPF_MOV uses reserved fields\n");
+					return -EINVAL;
+				}
+			} else {
+				if (insn->off != 0 && insn->off != 8 && insn->off != 16 && insn->off != 32) {
+					verbose(env, "BPF_MOV uses reserved fields\n");
+					return -EINVAL;
+				}
+			}
+
 			/* check src operand */
 			err = check_reg_arg(env, insn->src_reg, SRC_OP);
 			if (err)
Signature · 1 line
2.34.1
#8 lore raw

RFC PATCH 4/13 bpf: Support new unconditional bswap instruction

In reply to Yonghong Song

The existing 'be' and 'le' insns will do conditional bswap depends on host endianness. This patch implements unconditional bswap insns.

---
 arch/x86/net/bpf_jit_comp.c |  1 +
 kernel/bpf/core.c           | 14 ++++++++++++++
 kernel/bpf/verifier.c       |  2 +-
 3 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/net/bpf_jit_comp.c +1 -0
index 7c85d1b01931..6385a8d740b0 100644
@@ -1322,6 +1322,7 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 			break;
 
 		case BPF_ALU | BPF_END | BPF_FROM_BE:
+		case BPF_ALU64 | BPF_END | BPF_FROM_LE:
 			switch (imm32) {
 			case 16:
 				/* Emit 'ror %ax, 8' to swap lower 2 bytes */
kernel/bpf/core.c +14 -0
index 72ee246ac3af..b59b41a3d07c 100644
@@ -1524,6 +1524,7 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(ALU64, DIV,  X),			\
 	INSN_3(ALU64, MOD,  X),			\
 	INSN_2(ALU64, NEG),			\
+	INSN_3(ALU64, END, TO_LE),		\
 	/*   Immediate based. */		\
 	INSN_3(ALU64, ADD,  K),			\
 	INSN_3(ALU64, SUB,  K),			\
@@ -1845,6 +1846,19 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 			break;
 		}
 		CONT;
+	ALU64_END_TO_LE:
+		switch (IMM) {
+		case 16:
+			DST = (__force u16) __swab16(DST);
+			break;
+		case 32:
+			DST = (__force u32) __swab32(DST);
+			break;
+		case 64:
+			DST = (__force u64) __swab64(DST);
+			break;
+		}
+		CONT;
 
 	/* CALL */
 	JMP_CALL:
kernel/bpf/verifier.c +1 -1
index 5c5b37b6b39a..d515bfee1a8f 100644
@@ -12901,7 +12901,7 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 		} else {
 			if (insn->src_reg != BPF_REG_0 || insn->off != 0 ||
 			    (insn->imm != 16 && insn->imm != 32 && insn->imm != 64) ||
-			    BPF_CLASS(insn->code) == BPF_ALU64) {
+			    (BPF_CLASS(insn->code) == BPF_ALU64 && BPF_SRC(insn->code) != BPF_K)) {
 				verbose(env, "BPF_END uses reserved fields\n");
 				return -EINVAL;
 			}
Signature · 1 line
2.34.1
#9 lore raw

RFC PATCH 5/13 bpf: Support new signed div/mod instructions.

In reply to Yonghong Song

Add interpreter/jit support for new signed div/mod instructions. Also add basic verifier support to ensure new insns get accepted.

---
 arch/x86/net/bpf_jit_comp.c | 27 +++++++----
 kernel/bpf/core.c           | 96 ++++++++++++++++++++++++++++++-------
 kernel/bpf/verifier.c       |  6 ++-
 3 files changed, 103 insertions(+), 26 deletions(-)
arch/x86/net/bpf_jit_comp.c +19 -8
index 6385a8d740b0..0c8d881f3ada 100644
@@ -1194,15 +1194,26 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, u8 *rw_image
 				/* mov rax, dst_reg */
 				emit_mov_reg(&prog, is64, BPF_REG_0, dst_reg);
 
-			/*
-			 * xor edx, edx
-			 * equivalent to 'xor rdx, rdx', but one byte less
-			 */
-			EMIT2(0x31, 0xd2);
+			if (insn->off == 0) {
+				/*
+				 * xor edx, edx
+				 * equivalent to 'xor rdx, rdx', but one byte less
+				 */
+				EMIT2(0x31, 0xd2);
 
-			/* div src_reg */
-			maybe_emit_1mod(&prog, src_reg, is64);
-			EMIT2(0xF7, add_1reg(0xF0, src_reg));
+				/* div src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF0, src_reg));
+			} else {
+				if (BPF_CLASS(insn->code) == BPF_ALU)
+					EMIT1(0x99); /* cltd */
+				else
+					EMIT2(0x48, 0x99); /* cqto */
+
+				/* idiv src_reg */
+				maybe_emit_1mod(&prog, src_reg, is64);
+				EMIT2(0xF7, add_1reg(0xF8, src_reg));
+			}
 
 			if (BPF_OP(insn->code) == BPF_MOD &&
 			    dst_reg != BPF_REG_3)
kernel/bpf/core.c +80 -16

116 lines not on this page · raw message

kernel/bpf/verifier.c +4 -2
index d515bfee1a8f..cc14d2ac3c5a 100644
@@ -13029,7 +13029,8 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 	} else {	/* all other ALU ops: and, sub, xor, add, ... */
 
 		if (BPF_SRC(insn->code) == BPF_X) {
-			if (insn->imm != 0 || insn->off != 0) {
+			if (insn->imm != 0 || insn->off > 1 ||
+			    (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
 				verbose(env, "BPF_ALU uses reserved fields\n");
 				return -EINVAL;
 			}
@@ -13038,7 +13039,8 @@ static int check_alu_op(struct bpf_verifier_env *env, struct bpf_insn *insn)
 			if (err)
 				return err;
 		} else {
-			if (insn->src_reg != BPF_REG_0 || insn->off != 0) {
+			if (insn->src_reg != BPF_REG_0 || insn->off > 1 ||
+			    (insn->off == 1 && opcode != BPF_MOD && opcode != BPF_DIV)) {
 				verbose(env, "BPF_ALU uses reserved fields\n");
 				return -EINVAL;
 			}
Signature · 1 line
2.34.1
#10 lore raw

RFC PATCH 6/13 bpf: Support new 32bit offset jmp instruction

In reply to Yonghong Song

Add interpreter/jit/verifier support for 32bit offset jmp instruction.

---
 arch/x86/net/bpf_jit_comp.c | 28 ++++++++++++++++++----------
 kernel/bpf/core.c           | 19 ++++++++++++++++---
 kernel/bpf/verifier.c       | 32 ++++++++++++++++++++++----------
 3 files changed, 56 insertions(+), 23 deletions(-)
arch/x86/net/bpf_jit_comp.c +18 -10
index 0c8d881f3ada..885820dcaf0e 100644
@@ -1808,16 +1808,24 @@ st:			if (is_imm8(insn->off))
 			break;
 
 		case BPF_JMP | BPF_JA:
-			if (insn->off == -1)
-				/* -1 jmp instructions will always jump
-				 * backwards two bytes. Explicitly handling
-				 * this case avoids wasting too many passes
-				 * when there are long sequences of replaced
-				 * dead code.
-				 */
-				jmp_offset = -2;
-			else
-				jmp_offset = addrs[i + insn->off] - addrs[i];
+		case BPF_JMP32 | BPF_JA:
+			if (BPF_CLASS(insn->code) == BPF_JMP) {
+				if (insn->off == -1)
+					/* -1 jmp instructions will always jump
+					 * backwards two bytes. Explicitly handling
+					 * this case avoids wasting too many passes
+					 * when there are long sequences of replaced
+					 * dead code.
+					 */
+					jmp_offset = -2;
+				else
+					jmp_offset = addrs[i + insn->off] - addrs[i];
+			} else {
+				if (insn->imm == -1)
+					jmp_offset = -2;
+				else
+					jmp_offset = addrs[i + insn->imm] - addrs[i];
+			}
 
 			if (!jmp_offset) {
 				/*
kernel/bpf/core.c +16 -3
index 279d095ca1d1..288e49129018 100644
@@ -373,7 +373,12 @@ static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
 {
 	const s32 off_min = S16_MIN, off_max = S16_MAX;
 	s32 delta = end_new - end_old;
-	s32 off = insn->off;
+	s32 off;
+
+	if (insn->code == (BPF_JMP32 | BPF_JA))
+		off = insn->imm;
+	else
+		off = insn->off;
 
 	if (curr < pos && curr + off + 1 >= end_old)
 		off += delta;
@@ -381,8 +386,12 @@ static int bpf_adj_delta_to_off(struct bpf_insn *insn, u32 pos, s32 end_old,
 		off -= delta;
 	if (off < off_min || off > off_max)
 		return -ERANGE;
-	if (!probe_pass)
-		insn->off = off;
+	if (!probe_pass) {
+		if (insn->code == (BPF_JMP32 | BPF_JA))
+			insn->imm = off;
+		else
+			insn->off = off;
+	}
 	return 0;
 }
 
@@ -1593,6 +1602,7 @@ EXPORT_SYMBOL_GPL(__bpf_call_base);
 	INSN_3(JMP, JSLE, K),			\
 	INSN_3(JMP, JSET, K),			\
 	INSN_2(JMP, JA),			\
+	INSN_2(JMP32, JA),			\
 	/* Store instructions. */		\
 	/*   Register based. */			\
 	INSN_3(STX, MEM,  B),			\
@@ -1972,6 +1982,9 @@ static u64 ___bpf_prog_run(u64 *regs, const struct bpf_insn *insn)
 	JMP_JA:
 		insn += insn->off;
 		CONT;
+	JMP32_JA:
+		insn += insn->imm;
+		CONT;
 	JMP_EXIT:
 		return BPF_R0;
 	/* JMP */
kernel/bpf/verifier.c +22 -10
index cc14d2ac3c5a..28832ae9e6ca 100644
@@ -2854,7 +2854,10 @@ static int check_subprogs(struct bpf_verifier_env *env)
 			goto next;
 		if (BPF_OP(code) == BPF_EXIT || BPF_OP(code) == BPF_CALL)
 			goto next;
-		off = i + insn[i].off + 1;
+		if (code == (BPF_JMP32 | BPF_JA))
+			off = i + insn[i].imm + 1;
+		else
+			off = i + insn[i].off + 1;
 		if (off < subprog_start || off >= subprog_end) {
 			verbose(env, "jump out of range from insn %d to %d\n", i, off);
 			return -EINVAL;
@@ -2866,6 +2869,7 @@ static int check_subprogs(struct bpf_verifier_env *env)
 			 * or unconditional jump back
 			 */
 			if (code != (BPF_JMP | BPF_EXIT) &&
+			    code != (BPF_JMP32 | BPF_JA) &&
 			    code != (BPF_JMP | BPF_JA)) {
 				verbose(env, "last insn is not an exit or jmp\n");
 				return -EINVAL;
@@ -14584,7 +14588,7 @@ static int visit_func_call_insn(int t, struct bpf_insn *insns,
 static int visit_insn(int t, struct bpf_verifier_env *env)
 {
 	struct bpf_insn *insns = env->prog->insnsi, *insn = &insns[t];
-	int ret;
+	int ret, off;
 
 	if (bpf_pseudo_func(insn))
 		return visit_func_call_insn(t, insns, env, true);
@@ -14632,14 +14636,19 @@ static int visit_insn(int t, struct bpf_verifier_env *env)
 		if (BPF_SRC(insn->code) != BPF_K)
 			return -EINVAL;
 
+		if (BPF_CLASS(insn->code) == BPF_JMP)
+			off = insn->off;
+		else
+			off = insn->imm;
+
 		/* unconditional jump with single edge */
-		ret = push_insn(t, t + insn->off + 1, FALLTHROUGH, env,
+		ret = push_insn(t, t + off + 1, FALLTHROUGH, env,
 				true);
 		if (ret)
 			return ret;
 
-		mark_prune_point(env, t + insn->off + 1);
-		mark_jmp_point(env, t + insn->off + 1);
+		mark_prune_point(env, t + off + 1);
+		mark_jmp_point(env, t + off + 1);
 
 		return ret;
 
@@ -16435,15 +16444,18 @@ static int do_check(struct bpf_verifier_env *env)
 				mark_reg_scratched(env, BPF_REG_0);
 			} else if (opcode == BPF_JA) {
 				if (BPF_SRC(insn->code) != BPF_K ||
-				    insn->imm != 0 ||
 				    insn->src_reg != BPF_REG_0 ||
 				    insn->dst_reg != BPF_REG_0 ||
-				    class == BPF_JMP32) {
+				    (class == BPF_JMP && insn->imm != 0) ||
+				    (class == BPF_JMP32 && insn->off != 0)) {
 					verbose(env, "BPF_JA uses reserved fields\n");
 					return -EINVAL;
 				}
 
-				env->insn_idx += insn->off + 1;
+				if (class == BPF_JMP)
+					env->insn_idx += insn->off + 1;
+				else
+					env->insn_idx += insn->imm + 1;
 				continue;
 
 			} else if (opcode == BPF_EXIT) {
@@ -17290,13 +17302,13 @@ static bool insn_is_cond_jump(u8 code)
 {
 	u8 op;
 
+	op = BPF_OP(code);
 	if (BPF_CLASS(code) == BPF_JMP32)
-		return true;
+		return op != BPF_JA;
 
 	if (BPF_CLASS(code) != BPF_JMP)
 		return false;
 
-	op = BPF_OP(code);
 	return op != BPF_JA && op != BPF_EXIT && op != BPF_CALL;
 }
 
Signature · 1 line
2.34.1
#11 lore raw

RFC PATCH 7/13 bpf: Add kernel/bpftool asm support for new instructions

In reply to Yonghong Song

Add asm support for new instructions so kernel verifier and bpftool xlated insn dumps can have proper asm syntax for new instructions.

---
 kernel/bpf/disasm.c | 57 ++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 51 insertions(+), 6 deletions(-)
kernel/bpf/disasm.c +51 -6
index 7b4afb7d96db..cc46e708df5e 100644
@@ -87,6 +87,17 @@ const char *const bpf_alu_string[16] = {
 	[BPF_END >> 4]  = "endian",
 };
 
+const char *const bpf_alu_sign_string[16] = {
+	[BPF_DIV >> 4]  = "s/=",
+	[BPF_MOD >> 4]  = "s%=",
+};
+
+const char *const bpf_movs_string[4] = {
+	[0] = "(s8)",
+	[1] = "(s16)",
+	[3] = "(s32)",
+};
+
 static const char *const bpf_atomic_alu_string[16] = {
 	[BPF_ADD >> 4]  = "add",
 	[BPF_AND >> 4]  = "and",
@@ -101,6 +112,12 @@ static const char *const bpf_ldst_string[] = {
 	[BPF_DW >> 3] = "u64",
 };
 
+static const char *const bpf_lds_string[] = {
+	[BPF_W >> 3]  = "s32",
+	[BPF_H >> 3]  = "s16",
+	[BPF_B >> 3]  = "s8",
+};
+
 static const char *const bpf_jmp_string[16] = {
 	[BPF_JA >> 4]   = "jmp",
 	[BPF_JEQ >> 4]  = "==",
@@ -128,6 +145,26 @@ static void print_bpf_end_insn(bpf_insn_print_t verbose,
 		insn->imm, insn->dst_reg);
 }
 
+static void print_bpf_bswap_insn(bpf_insn_print_t verbose,
+			       void *private_data,
+			       const struct bpf_insn *insn)
+{
+	verbose(private_data, "(%02x) r%d = bswap%d r%d\n",
+		insn->code, insn->dst_reg,
+		insn->imm, insn->dst_reg);
+}
+
+static bool is_sdiv_smod(const struct bpf_insn *insn) {
+	u8 op = BPF_OP(insn->code);
+	u16 off = insn->off;
+
+	return (op == BPF_DIV || op == BPF_MOD) && off == 1;
+}
+
+static bool is_movs(const struct bpf_insn *insn) {
+	return BPF_OP(insn->code) == BPF_MOV && insn->off != 0;
+}
+
 void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 		    const struct bpf_insn *insn,
 		    bool allow_ptr_leaks)
@@ -138,7 +175,7 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 	if (class == BPF_ALU || class == BPF_ALU64) {
 		if (BPF_OP(insn->code) == BPF_END) {
 			if (class == BPF_ALU64)
-				verbose(cbs->private_data, "BUG_alu64_%02x\n", insn->code);
+				print_bpf_bswap_insn(verbose, cbs->private_data, insn);
 			else
 				print_bpf_end_insn(verbose, cbs->private_data, insn);
 		} else if (BPF_OP(insn->code) == BPF_NEG) {
@@ -147,17 +184,20 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 				insn->dst_reg, class == BPF_ALU ? 'w' : 'r',
 				insn->dst_reg);
 		} else if (BPF_SRC(insn->code) == BPF_X) {
-			verbose(cbs->private_data, "(%02x) %c%d %s %c%d\n",
+			verbose(cbs->private_data, "(%02x) %c%d %s %s%c%d\n",
 				insn->code, class == BPF_ALU ? 'w' : 'r',
 				insn->dst_reg,
-				bpf_alu_string[BPF_OP(insn->code) >> 4],
+				is_sdiv_smod(insn) ? bpf_alu_sign_string[BPF_OP(insn->code) >> 4]
+						   : bpf_alu_string[BPF_OP(insn->code) >> 4],
+				is_movs(insn) ? bpf_movs_string[(insn->off >> 3) - 1] : "",
 				class == BPF_ALU ? 'w' : 'r',
 				insn->src_reg);
 		} else {
 			verbose(cbs->private_data, "(%02x) %c%d %s %d\n",
 				insn->code, class == BPF_ALU ? 'w' : 'r',
 				insn->dst_reg,
-				bpf_alu_string[BPF_OP(insn->code) >> 4],
+				is_sdiv_smod(insn) ? bpf_alu_sign_string[BPF_OP(insn->code) >> 4]
+						   : bpf_alu_string[BPF_OP(insn->code) >> 4],
 				insn->imm);
 		}
 	} else if (class == BPF_STX) {
@@ -218,13 +258,15 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 			verbose(cbs->private_data, "BUG_st_%02x\n", insn->code);
 		}
 	} else if (class == BPF_LDX) {
-		if (BPF_MODE(insn->code) != BPF_MEM) {
+		if (BPF_MODE(insn->code) != BPF_MEM && BPF_MODE(insn->code) != BPF_MEMS) {
 			verbose(cbs->private_data, "BUG_ldx_%02x\n", insn->code);
 			return;
 		}
 		verbose(cbs->private_data, "(%02x) r%d = *(%s *)(r%d %+d)\n",
 			insn->code, insn->dst_reg,
-			bpf_ldst_string[BPF_SIZE(insn->code) >> 3],
+			BPF_MODE(insn->code) == BPF_MEM ?
+				 bpf_ldst_string[BPF_SIZE(insn->code) >> 3] :
+				 bpf_lds_string[BPF_SIZE(insn->code) >> 3],
 			insn->src_reg, insn->off);
 	} else if (class == BPF_LD) {
 		if (BPF_MODE(insn->code) == BPF_ABS) {
@@ -279,6 +321,9 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 		} else if (insn->code == (BPF_JMP | BPF_JA)) {
 			verbose(cbs->private_data, "(%02x) goto pc%+d\n",
 				insn->code, insn->off);
+		} else if (insn->code == (BPF_JMP32 | BPF_JA)) {
+			verbose(cbs->private_data, "(%02x) gotol pc%+d\n",
+				insn->code, insn->imm);
 		} else if (insn->code == (BPF_JMP | BPF_EXIT)) {
 			verbose(cbs->private_data, "(%02x) exit\n", insn->code);
 		} else if (BPF_SRC(insn->code) == BPF_X) {
Signature · 1 line
2.34.1
#12 lore raw

Re: [RFC PATCH bpf-next 07/13] bpf: Add kernel/bpftool asm support for new instructions

kernel test robot noticed the following build warnings:

Hi Yonghong,

[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:

[auto build test WARNING on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Yonghong-Song/bpf-Support-new-sign-extension-load-insns/20230629-144251
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20230629063756.1652245-1-yhs%40fb.com
patch subject: [RFC PATCH bpf-next 07/13] bpf: Add kernel/bpftool asm support for new instructions
config: x86_64-randconfig-x061-20230702 (https://download.01.org/0day-ci/archive/20230703/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230703/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> kernel/bpf/disasm.c:90:12: sparse: sparse: symbol 'bpf_alu_sign_string' was not declared. Should it be static?
>> kernel/bpf/disasm.c:95:12: sparse: sparse: symbol 'bpf_movs_string' was not declared. Should it be static?
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#13 lore raw

RFC PATCH 8/13 selftests/bpf: Add unit tests for new sign-extension load insns

In reply to Yonghong Song
---
 .../selftests/bpf/prog_tests/verifier.c       |  2 +
 .../selftests/bpf/progs/verifier_lds.c        | 46 +++++++++++++++++++
 2 files changed, 48 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_lds.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 070a13833c3f..bc05391174c7 100644
@@ -31,6 +31,7 @@
 #include "verifier_int_ptr.skel.h"
 #include "verifier_jeq_infer_not_null.skel.h"
 #include "verifier_ld_ind.skel.h"
+#include "verifier_lds.skel.h"
 #include "verifier_leak_ptr.skel.h"
 #include "verifier_loops1.skel.h"
 #include "verifier_lwt.skel.h"
@@ -132,6 +133,7 @@ void test_verifier_helper_value_access(void)  { RUN(verifier_helper_value_access
 void test_verifier_int_ptr(void)              { RUN(verifier_int_ptr); }
 void test_verifier_jeq_infer_not_null(void)   { RUN(verifier_jeq_infer_not_null); }
 void test_verifier_ld_ind(void)               { RUN(verifier_ld_ind); }
+void test_verifier_lds(void)                  { RUN(verifier_lds); }
 void test_verifier_leak_ptr(void)             { RUN(verifier_leak_ptr); }
 void test_verifier_loops1(void)               { RUN(verifier_loops1); }
 void test_verifier_lwt(void)                  { RUN(verifier_lwt); }
tools/testing/selftests/bpf/progs/verifier_lds.c new file +46 -0

46 lines not on this page · raw message

Signature · 1 line
2.34.1
#14 lore raw

RFC PATCH 9/13 selftests/bpf: Add unit tests for new sign-extension mov insns

In reply to Yonghong Song
---
 .../selftests/bpf/prog_tests/verifier.c       |  2 +
 .../selftests/bpf/progs/verifier_movs.c       | 67 +++++++++++++++++++
 2 files changed, 69 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_movs.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index bc05391174c7..a4d9fc57aff2 100644
@@ -41,6 +41,7 @@
 #include "verifier_map_ret_val.skel.h"
 #include "verifier_masking.skel.h"
 #include "verifier_meta_access.skel.h"
+#include "verifier_movs.skel.h"
 #include "verifier_netfilter_ctx.skel.h"
 #include "verifier_netfilter_retcode.skel.h"
 #include "verifier_prevent_map_lookup.skel.h"
@@ -143,6 +144,7 @@ void test_verifier_map_ptr_mixing(void)       { RUN(verifier_map_ptr_mixing); }
 void test_verifier_map_ret_val(void)          { RUN(verifier_map_ret_val); }
 void test_verifier_masking(void)              { RUN(verifier_masking); }
 void test_verifier_meta_access(void)          { RUN(verifier_meta_access); }
+void test_verifier_movs(void)                 { RUN(verifier_movs); }
 void test_verifier_netfilter_ctx(void)        { RUN(verifier_netfilter_ctx); }
 void test_verifier_netfilter_retcode(void)    { RUN(verifier_netfilter_retcode); }
 void test_verifier_prevent_map_lookup(void)   { RUN(verifier_prevent_map_lookup); }
tools/testing/selftests/bpf/progs/verifier_movs.c new file +67 -0

67 lines not on this page · raw message

Signature · 1 line
2.34.1
#15 lore raw

RFC PATCH 10/13 selftests/bpf: Add unit tests for new bswap insns

In reply to Yonghong Song
---
 .../selftests/bpf/prog_tests/verifier.c       |  2 +
 .../selftests/bpf/progs/verifier_bswap.c      | 45 +++++++++++++++++++
 2 files changed, 47 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_bswap.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index a4d9fc57aff2..63112b3755cb 100644
@@ -11,6 +11,7 @@
 #include "verifier_bounds_deduction_non_const.skel.h"
 #include "verifier_bounds_mix_sign_unsign.skel.h"
 #include "verifier_bpf_get_stack.skel.h"
+#include "verifier_bswap.skel.h"
 #include "verifier_btf_ctx_access.skel.h"
 #include "verifier_cfg.skel.h"
 #include "verifier_cgroup_inv_retcode.skel.h"
@@ -114,6 +115,7 @@ void test_verifier_bounds_deduction(void)     { RUN(verifier_bounds_deduction);
 void test_verifier_bounds_deduction_non_const(void)     { RUN(verifier_bounds_deduction_non_const); }
 void test_verifier_bounds_mix_sign_unsign(void) { RUN(verifier_bounds_mix_sign_unsign); }
 void test_verifier_bpf_get_stack(void)        { RUN(verifier_bpf_get_stack); }
+void test_verifier_bswap(void)                { RUN(verifier_bswap); }
 void test_verifier_btf_ctx_access(void)       { RUN(verifier_btf_ctx_access); }
 void test_verifier_cfg(void)                  { RUN(verifier_cfg); }
 void test_verifier_cgroup_inv_retcode(void)   { RUN(verifier_cgroup_inv_retcode); }
tools/testing/selftests/bpf/progs/verifier_bswap.c new file +45 -0

45 lines not on this page · raw message

Signature · 1 line
2.34.1
#16 lore raw

RFC PATCH 11/13 selftests/bpf: Add unit tests for new sdiv/smod insns

In reply to Yonghong Song
---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_sdiv.c       | 763 ++++++++++++++++++
 2 files changed, 765 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_sdiv.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index 63112b3755cb..b2c041a4bc70 100644
@@ -54,6 +54,7 @@
 #include "verifier_ringbuf.skel.h"
 #include "verifier_runtime_jit.skel.h"
 #include "verifier_scalar_ids.skel.h"
+#include "verifier_sdiv.skel.h"
 #include "verifier_search_pruning.skel.h"
 #include "verifier_sock.skel.h"
 #include "verifier_spill_fill.skel.h"
@@ -158,6 +159,7 @@ void test_verifier_regalloc(void)             { RUN(verifier_regalloc); }
 void test_verifier_ringbuf(void)              { RUN(verifier_ringbuf); }
 void test_verifier_runtime_jit(void)          { RUN(verifier_runtime_jit); }
 void test_verifier_scalar_ids(void)           { RUN(verifier_scalar_ids); }
+void test_verifier_sdiv(void)                 { RUN(verifier_sdiv); }
 void test_verifier_search_pruning(void)       { RUN(verifier_search_pruning); }
 void test_verifier_sock(void)                 { RUN(verifier_sock); }
 void test_verifier_spill_fill(void)           { RUN(verifier_spill_fill); }
tools/testing/selftests/bpf/progs/verifier_sdiv.c new file +763 -0

763 lines not on this page · raw message

Signature · 1 line
2.34.1
#17 lore raw

RFC PATCH 12/13 selftests/bpf: Add unit tests for new gotol insn

In reply to Yonghong Song
---
 .../selftests/bpf/prog_tests/verifier.c       |  2 ++
 .../selftests/bpf/progs/verifier_gotol.c      | 30 +++++++++++++++++++
 2 files changed, 32 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotol.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index b2c041a4bc70..d07251560ce4 100644
@@ -25,6 +25,7 @@
 #include "verifier_direct_stack_access_wraparound.skel.h"
 #include "verifier_div0.skel.h"
 #include "verifier_div_overflow.skel.h"
+#include "verifier_gotol.skel.h"
 #include "verifier_helper_access_var_len.skel.h"
 #include "verifier_helper_packet_access.skel.h"
 #include "verifier_helper_restricted.skel.h"
@@ -130,6 +131,7 @@ void test_verifier_direct_packet_access(void) { RUN(verifier_direct_packet_acces
 void test_verifier_direct_stack_access_wraparound(void) { RUN(verifier_direct_stack_access_wraparound); }
 void test_verifier_div0(void)                 { RUN(verifier_div0); }
 void test_verifier_div_overflow(void)         { RUN(verifier_div_overflow); }
+void test_verifier_gotol(void)                { RUN(verifier_gotol); }
 void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_len); }
 void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
 void test_verifier_helper_restricted(void)    { RUN(verifier_helper_restricted); }
tools/testing/selftests/bpf/progs/verifier_gotol.c new file +30 -0
new file mode 100644
index 000000000000..78870ea4d468
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+SEC("socket")
+__description("gotol, small_imm")
+__success __success_unpriv __retval(1)
+__naked void gotol_small_imm(void)
+{
+	asm volatile ("					\
+	call %[bpf_ktime_get_ns];			\
+	if r0 == 0 goto l0_%=;				\
+	gotol l1_%=;					\
+l2_%=:							\
+	gotol l3_%=;					\
+l1_%=:							\
+	r0 = 1;						\
+	gotol l2_%=;					\
+l0_%=:							\
+	r0 = 2;						\
+l3_%=:							\
+	exit;						\
+"	:
+	: __imm(bpf_ktime_get_ns)
+	: __clobber_all);
+}
+
+char _license[] SEC("license") = "GPL";
Signature · 1 line
2.34.1
#18 lore raw

RFC PATCH 13/13 selftests/bpf: Add a cpuv4 test runner for cpu=v4 testing

In reply to Yonghong Song

Similar to no-alu32 runner, a cpuv4 runner is created to test bpf programs compiled with -mcpu=v4.

The following are some num-of-insn statistics for each newer instructions, excluding naked asm (verifier_*) tests:

   insn pattern                # of instructions
   reg = (s8)reg               4
   reg = (s16)reg              2
   reg = (s32)reg              26
   reg = *(s8 *)(reg + off)    11
   reg = *(s16 *)(reg + off)   14
   reg = *(s32 *)(reg + off)   15214
   reg = bswap16 reg           133
   reg = bswap32 reg           38
   reg = bswap64 reg           14
   reg s/= reg                 0
   reg s%= reg                 0
   gotol <offset>              58

Note that in llvm -mcpu=v4 implementation, the compiler is a little bit conservative about generating 'gotol' insn (32-bit branch offset) as it didn't precise count the number of insns (e.g., some insns are debug insns, etc.). Compared to old 'goto' insn, newer 'gotol' insn should have comparable verification states to 'goto' insn.

I did not collect verifier stats now since I have not really started to do proper range bound estimation with these instructions.

With current patch set, all selftests passed with -mcpu=v4 when running test_progs-cpuv4 binary.

---
 tools/testing/selftests/bpf/Makefile | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/Makefile +14 -4
index 538df8fb8c42..5f901cb0df59 100644
@@ -40,7 +40,7 @@ TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test
 	test_sock test_sockmap get_cgroup_id_user \
 	test_cgroup_storage \
 	test_tcpnotify_user test_sysctl \
-	test_progs-no_alu32
+	test_progs-no_alu32 test_progs-cpuv4
 
 # Also test bpf-gcc, if present
 ifneq ($(BPF_GCC),)
@@ -379,6 +379,11 @@ define CLANG_NOALU32_BPF_BUILD_RULE
 	$(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
 	$(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v2 -o $2
 endef
+# Similar to CLANG_BPF_BUILD_RULE, but with cpu-v4
+define CLANG_CPUV4_BPF_BUILD_RULE
+	$(call msg,CLNG-BPF,$(TRUNNER_BINARY),$2)
+	$(Q)$(CLANG) $3 -O2 -target bpf -c $1 -mcpu=v4 -o $2
+endef
 # Build BPF object using GCC
 define GCC_BPF_BUILD_RULE
 	$(call msg,GCC-BPF,$(TRUNNER_BINARY),$2)
@@ -421,7 +426,7 @@ LINKED_BPF_SRCS := $(patsubst %.bpf.o,%.c,$(foreach skel,$(LINKED_SKELS),$($(ske
 # $eval()) and pass control to DEFINE_TEST_RUNNER_RULES.
 # Parameters:
 # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
 define DEFINE_TEST_RUNNER
 
 TRUNNER_OUTPUT := $(OUTPUT)$(if $2,/)$2
@@ -449,7 +454,7 @@ endef
 # Using TRUNNER_XXX variables, provided by callers of DEFINE_TEST_RUNNER and
 # set up by DEFINE_TEST_RUNNER itself, create test runner build rules with:
 # $1 - test runner base binary name (e.g., test_progs)
-# $2 - test runner extra "flavor" (e.g., no_alu32, gcc-bpf, etc)
+# $2 - test runner extra "flavor" (e.g., no_alu32, cpuv4, gcc-bpf, etc)
 define DEFINE_TEST_RUNNER_RULES
 
 ifeq ($($(TRUNNER_OUTPUT)-dir),)
@@ -580,6 +585,11 @@ TRUNNER_BPF_BUILD_RULE := CLANG_NOALU32_BPF_BUILD_RULE
 TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
 $(eval $(call DEFINE_TEST_RUNNER,test_progs,no_alu32))
 
+# Define test_progs-cpuv4 test runner.
+TRUNNER_BPF_BUILD_RULE := CLANG_CPUV4_BPF_BUILD_RULE
+TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
+$(eval $(call DEFINE_TEST_RUNNER,test_progs,cpuv4))
+
 # Define test_progs BPF-GCC-flavored test runner.
 ifneq ($(BPF_GCC),)
 TRUNNER_BPF_BUILD_RULE := GCC_BPF_BUILD_RULE
@@ -674,7 +684,7 @@ EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)	\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
 	feature bpftool							\
 	$(addprefix $(OUTPUT)/,*.o *.skel.h *.lskel.h *.subskel.h	\
-			       no_alu32 bpf_gcc bpf_testmod.ko		\
+			       no_alu32 cpuv4 bpf_gcc bpf_testmod.ko	\
 			       liburandom_read.so)
 
 .PHONY: docs docs-clean
Signature · 1 line
2.34.1

A message that is not in the archive (sent off-list or not yet received): PH7PR21MB38786422B9929D253E279810A325A@PH7PR21MB3878.namprd21.prod.outlook.com

#20 lore raw
Quoted 9 lines · Dave Thaler
Quoted 5 lines · Yonghong Song

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:

[...]

What about also updating instruction-set.rst?

Will update doc in the next revision.

Quoted 4 lines

Dave

#21 lore raw

Hi Yonghong,

Quoted 9 lines · Yonghong Song
Quoted 6 lines · Dave Thaler
Quoted 5 lines · Yonghong Song

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:

Quoted 10 lines
Quoted 3 lines

[...]

What about also updating instruction-set.rst?

Will update doc in the next revision.

Quoted 3 lines

Dave

I think bpf_design_QA.rst needs to be updated as well:

Q: Why there is no BPF_SDIV for signed divide operation? [..]

Thanks,
Daniel

#22 lore raw
Quoted 19 lines · Daniel Xu

Hi Yonghong,

Quoted 16 lines · Yonghong Song
Quoted 9 lines · Dave Thaler
Quoted 5 lines · Yonghong Song

In previous discussion ([1]), it is agreed that we should introduce cpu version 4 (llvm flag -mcpu=v4) which contains some instructions which can simplify code, make code easier to understand, fix the existing problem, or simply for feature completeness. More specifically, the following new insns are proposed:

[...]

What about also updating instruction-set.rst?

Will update doc in the next revision.

Quoted 1 line
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines

Dave

I think bpf_design_QA.rst needs to be updated as well:

Q: Why there is no BPF_SDIV for signed divide operation? [..]

Sure. Will do!

Quoted 3 lines

Thanks,
Daniel