Skip to contentSkip to conversation

BPF indirect jumps

PATCH Superseded Anton Protopopov 7 participants 32 messages Nov 2, 2025 – Nov 5, 2025

Subsystems (8)
x86 Kselftest BPF BPF selftests +4 more subsystems

v10 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v10 cover BPF indirect jumps

This patchset implements a new type of map, instruction set, and uses it to build support for indirect branches in BPF (on x86). (The same map will be later used to provide support for indirect calls and static keys.) See [1], [2] for more context.

Short table of contents:

* Patches 1-4 implement the new map of type BPF_MAP_TYPE_INSN_SET and corresponding selftests. This map can be used to track the "original -> xlated -> jitted mapping" for a given program. Patches 1,2 implement the main functionality, patches 3,4 add support for the "blinded" variant.

* Patches 5-7 implement the support for indirect jumps on x86

* Patches 8-11 add libbpf support for LLVM-compiled programs containing indirect jumps, and selftests.

The jump table support was merged to LLVM and now can be enabled with -mcpu=v4, see [3]. The __BPF_FEATURE_GOTOX macros can be used to check if the compiler supports the feature or not.

See individual patches for more details on the implementation details.

v9 -> v10 (this series):

* Three bugs were noticed by AI in v9 (two old, one introduced by v9):
* [new] insn_array_alloc_size could overflow u32, switched to u64 (AI)
* map_ptr should be compared in regsafe for PTR_TO_INSN (AI)
* duplicate elements were copied in jt_from_map (AI)

* added a selftest in verifier_gotox with a jump table containing non-unique entries

v8 -> v9 (https://lore.kernel.org/bpf/[email protected]/T/#t):

* instruction arrays:
* remove the size restriction of 256 elements
* add a comments about addrs usage, old and new (Alexei)

* libbpf:
* properly prefix warnings (Andrii)
* cast j[t] to long long for printf and some other minor cleanups (Andrii)

* selftests:
* use __BPF_FEATURE_GOTOX in selftests and skip tests if it's not set (Eduard)
* fix a typo in a selftest assembly (AI)

v7 -> v8 (https://lore.kernel.org/bpf/[email protected]/T/#u):

* instruction arrays:
* simplify the bpf_prog_update_insn_ptrs function (Eduard)
* remove a semicolon after a function definition (AI)

* libbpf:
* add a proper error path in libbpf patch (AI)
* re-re-factor the create_jt_map & find_subprog_idx (Eduard)

* selftests:
* verifier_gotox: add a test for a jump table pointing to outside of a subprog (Eduard)
* used test__skip instead of just running an empty test
* split tests in bpf_gotox into subtests for convenience

* random:
* drop the docs commit for now

v6 -> v7 (https://lore.kernel.org/bpf/[email protected]/T/#t):

* rebased and dropped already merged commits

* instruction arrays
* use jit_data to find mappings from insn to jit (Alexei)
* alloc ips as part of the main allocation (Eduard)
* the jitted_ip member wasn't actually used (Eduard)
* remove the bpf_insn_ptr structure, which is not needed for this patch

* indirect jumps, kernel:
* fix a memory leak in create_jt (AI)
* use proper reg+8*ereg in its_static_thunk (AI)
* some minor cleanups (Eduard)

* indirect jumps, libbpf:
* refactor the jt_adjust_off() piece (Edurad)
* move "JUMPTABLES_SEC" into libbpf_internal.h (Eduard)
* remove an unnecessary if (Eduard)

* verifier_gotox: add tests to verify that gotox rX works with all registers

v5 -> v6 (https://lore.kernel.org/bpf/[email protected]/T/#u):

* instruction arrays:
* better document struct bpf_insn_array_value (Eduard)
* remove a condition in bpf_insn_array_adjust_after_remove (Eduard)
* make userspace see original, xlated, and jitted indexes (+original) (Eduard)

* indirect jumps, kernel:
* reject writes to the map
* reject unaligned ops
* add a check what w is not outside the program in check_config for gotox (Eduard)
* do not introduce unneeded bpf_find_containing_subprog_idx
* simplify error processing for bpf_find_containing_subprog (Eduard)
* add insn_state |= DISCOVERED when it's discovered (Eduard)
* support SUB operations on PTR_TO_INSN (Eduard)
* make gotox_tmp_buf a bpf_iarray and use helper to relocate it (Eduard)
* rename fields of bpf_iarray to more generic (Eduard)
* re-implement visit_gotox_insn in a loop (Eduard)
* some minor cleanups (Eduard)

* libbpf:
* struct reloc_desc: add a comment about union (Eduard)
* rename parameters of (and one other place in code) {create,add}_jt_map to sym_off (Eduard)
* create_jt_map: check that size/off are 8-byte aligned (Eduard)

* Selftests:
* instruction array selftests:
* only run tests on x86_64
* write a more generic function to test things to reduce code (Eduard)
* errno wasn't used in checks, so don't reset it (Eduard)
* print i, xlated_off and map_out[i] here (Eduard)
* added verifier_gotox selftests which do not depend on LLVM:
* disabled bpf_gotox tests by default

* other changes:
* remove an extra function in bpf disasm (Eduard)
* some minor cleanups in the insn_successors patch (Eduard)
* update documentation in Documentation/bpf/linux-notes.html about jumps, now it is supported :)

v3 -> v4 -> v5 (https://lore.kernel.org/bpf/[email protected]/):

* [v4 -> v5] rebased on top of the last bpf-next/master

* instruction arrays:
* add copyright (Alexei)
* remove mutexes, add frozen back (Alexei)
* setup 1:1 prog-map correspondence using atomic_xchg
* do not copy/paste array_map_get_next_key, add a common helper (Alexei)
* misc minor code cleanups (Alexei)

* indirect jumps, kernel side:
* remove jt_allocated, just check if insn is gotox (Eduard)
* use copy_register_state instead of individual copies (Eduard)
* in push_stack is_speculative should be inherited (Eduard)
* a few cleanups for insn_successors, including omitting error path (Eduard)
* check if reserved fields are used when considering gotox instruction (Eduard)
* read size and alignment of read from insn_array should be 8 (Eduard)
* put buffer for sorting in subfun info and realloc to grow as needed (Eduard)
* properly do jump_point / prune_point from push_gotox_edge (Eduard)
* use range_within to check states (Eduard)
* some minor cleanups and fix commit message (Eduard)

* indirect jumps, libbpf side:
* close map_fd in some error paths in create_jt_map (Andrii)
* maps for jump tables are actually not closed at all, fix this (Andrii)
* rename map from jt to .jumptables (Andrii)
* use errstr in an error message (Andrii)
* rephrase error message to look more standard (Andrii)
* misc other minor renames and cleanups (Andrii)

* selftests:
* add the frozen selftest back
* add a selftest for two jumps loading same table

* some other changes:
* rebase and split insn_successor changes into separate patch
* use PTR_ERR_OR_ZERO in the push stack patch (Eduard)
* indirect jumps on x86: properly re-read *pprog (Eduard)

v2 -> v3 (https://lore.kernel.org/bpf/[email protected]/):

* fix build failure when CONFIG_BPF_SYSCALL is not set (kbuild-bot)
* reformat bpftool help messages (Quentin)

v1 -> v2 (https://lore.kernel.org/bpf/[email protected]/):

* push_stack changes:
* sanitize_speculative_path should just return int (Eduard)
* return code from sanitize_speculative_path, not EFAULT (Eduard)
* when BPF_COMPLEXITY_LIMIT_JMP_SEQ is reached, return E2BIG (Eduard)

* indirect jumps:
* omit support for .imm=fd in gotox, as we're not using it for now (Eduard)
* struct jt -> struct bpf_iarray (Eduard)
* insn_successors: rewrite the interface to just return a pointer (Eduard)
* remove min_index/max_index, use umin_value/umax_value instead (Alexei, Eduard)
* move emit_indirect_jump args change to the previous patch (Eduard)
* add a comment to map_mem_size() (Eduard)
* use verifier_bug for some error cases in check_indirect_jump (Eduard)
* clear_insn_aux_data: use start,len instead of start,end (Eduard)
* make regs[insn->dst_reg].type = PTR_TO_INSN part of check_mem_access (Eduard)

* constant blinding changes:
* make subprog_start adjustment better readable (Eduard)
* do not set subprog len, it is already set (Eduard)

* libbpf:
* remove check that relocations from .rodata are ok (Anton)
* do not freeze the map, it is not necessary anymore (Anton)
* rename the goto_x -> gotox everywhere (Anton)
* use u64 when parsing LLVM jump tables (Eduard)
* split patch in two due to spaces->tabs change (Eduard)
* split bpftool changes to bpftool patch (Andrii)
* make sym_size it a union with ext_idx (Andrii)
* properly copy/free the jumptables_data section from elf (Andrii)
* a few cosmetic changes around create_jt_map (Andrii)
* fix some comments + rewrite patch description (Andrii)
* inline bpf_prog__append_subprog_offsets (Andrii)
* subprog_sec_offst -> subprog_sec_off (Andrii)
* !strcmp -> strcmp() == 0 (Andrii)
* make some function names more readable (Andrii)
* allocate table of subfunc offsets via libbpf_reallocarray (Andrii)

* selftests:
* squash insn_array* tests together (Anton)

* fixed build warnings (kernel test robot)

RFC -> v1 (https://lore.kernel.org/bpf/[email protected]/):

* I've tried to address all the comments provided by Alexei and Eduard in RFC. Will try to list the most important of them below.
* One big change: move from older LLVM version [5] to newer [4]. Now LLVM generates jump tables as symbols in the new special section ".jumptables". Another part of this change is that libbpf now doesn't try to link map load and goto *rX, as
1) this is absolutely not reliable 2) for some use cases this is impossible (namely, when more than one jump table can be used in the same gotox instruction).
* Added insn_successors() support (Alexei, Eduard). This includes getting rid of the ugly bpf_insn_set_iter_xlated_offset() interface (Eduard).
* Removed hack for the unreachable instruction, as new LLVM thank to Eduard doesn't generate it.
* Set mem_size for direct map access properly instead of hacking. Remove off>0 check. (Alexei)
* Do not allocate new memory for min_index/max_index (Alexei, Eduard)
* Information required during check_cfg is now cached to be reused later (Alexei + general logic for supporting multiple JT per jump)
* Properly compare registers in regsafe (Alexei, Eduard)
* Remove support for JMP32 (Eduard)
* Better checks in adjust_ptr_min_max_vals (Eduard)
* More selftests were added (but still there's room for more) which directly use gotox (Alexei)
* More checks and verbose messages added
* "unique pointers" are no more in the map

Links:
1. https://lpc.events/event/18/contributions/1941/
2. https://lwn.net/Articles/1017439/
3. https://github.com/llvm/llvm-project/pull/149715
4. https://github.com/llvm/llvm-project/pull/149715#issuecomment-3274833753
6. rfc: https://lore.kernel.org/bpf/[email protected]/

Anton Protopopov (11):
bpf, x86: add new map type: instructions array selftests/bpf: add selftests for new insn_array map
bpf: support instructions arrays with constants blinding selftests/bpf: test instructions arrays with blinding bpf, x86: allow indirect jumps to r8...r15
bpf, x86: add support for indirect jumps
bpf: disasm: add support for BPF_JMP|BPF_JA|BPF_X
libbpf: support llvm-generated indirect jumps
bpftool: Recognize insn_array map type selftests/bpf: add new verifier_gotox test selftests/bpf: add C-level selftests for indirect jumps

 arch/x86/net/bpf_jit_comp.c                   |  42 +-
 include/linux/bpf.h                           |  16 +
 include/linux/bpf_types.h                     |   1 +
 include/linux/bpf_verifier.h                  |  11 +
 include/uapi/linux/bpf.h                      |  21 +
 kernel/bpf/Makefile                           |   2 +-
 kernel/bpf/bpf_insn_array.c                   | 301 +++++++++++
 kernel/bpf/core.c                             |  21 +
 kernel/bpf/disasm.c                           |   3 +
 kernel/bpf/liveness.c                         |   3 +
 kernel/bpf/log.c                              |   1 +
 kernel/bpf/syscall.c                          |  22 +
 kernel/bpf/verifier.c                         | 428 ++++++++++++++-
 .../bpf/bpftool/Documentation/bpftool-map.rst |   3 +-
 tools/bpf/bpftool/map.c                       |   3 +-
 tools/include/uapi/linux/bpf.h                |  21 +
 tools/lib/bpf/libbpf.c                        | 248 ++++++++-
 tools/lib/bpf/libbpf_internal.h               |   2 +
 tools/lib/bpf/libbpf_probes.c                 |   4 +
 tools/lib/bpf/linker.c                        |   9 +-
 tools/testing/selftests/bpf/Makefile          |   4 +-
 .../selftests/bpf/prog_tests/bpf_gotox.c      | 272 ++++++++++
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 504 ++++++++++++++++++
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 tools/testing/selftests/bpf/progs/bpf_gotox.c | 437 +++++++++++++++
 .../selftests/bpf/progs/verifier_gotox.c      | 389 ++++++++++++++
 26 files changed, 2748 insertions(+), 22 deletions(-)
 create mode 100644 kernel/bpf/bpf_insn_array.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_gotox.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_gotox.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotox.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH v10 1/11 bpf, x86: add new map type: instructions array

On bpf(BPF_PROG_LOAD) syscall user-supplied BPF programs are translated by the verifier into "xlated" BPF programs. During this process the original instructions offsets might be adjusted and/or individual instructions might be replaced by new sets of instructions, or deleted.

Add a new BPF map type which is aimed to keep track of how, for a given program, the original instructions were relocated during the verification. Also, besides keeping track of the original -> xlated mapping, make x86 JIT to build the xlated -> jitted mapping for every instruction listed in an instruction array. This is required for every future application of instruction arrays: static keys, indirect jumps and indirect calls.

A map of the BPF_MAP_TYPE_INSN_ARRAY type must be created with a u32 keys and value of size 8. The values have different semantics for userspace and for BPF space. For userspace a value consists of two u32 values – xlated and jitted offsets. For BPF side the value is a real pointer to a jitted instruction.

On map creation/initialization, before loading the program, each element of the map should be initialized to point to an instruction offset within the program. Before the program load such maps should be made frozen. After the program verification xlated and jitted offsets can be read via the bpf(2) syscall.

If a tracked instruction is removed by the verifier, then the xlated offset is set to (u32)-1 which is considered to be too big for a valid BPF program offset.

One such a map can, obviously, be used to track one and only one BPF program. If the verification process was unsuccessful, then the same map can be re-used to verify the program with a different log level. However, if the program was loaded fine, then such a map, being frozen in any case, can't be reused by other programs even after the program release.

Example. Consider the following original and xlated programs:

    Original prog:                      Xlated prog:

     0:  r1 = 0x0                        0: r1 = 0
     1:  *(u32 *)(r10 - 0x4) = r1        1: *(u32 *)(r10 -4) = r1
     2:  r2 = r10                        2: r2 = r10
     3:  r2 += -0x4                      3: r2 += -4
     4:  r1 = 0x0 ll                     4: r1 = map[id:88]
     6:  call 0x1                        6: r1 += 272
                                         7: r0 = *(u32 *)(r2 +0)
                                         8: if r0 >= 0x1 goto pc+3
                                         9: r0 <<= 3
                                        10: r0 += r1
                                        11: goto pc+1
                                        12: r0 = 0
     7:  r6 = r0                        13: r6 = r0
     8:  if r6 == 0x0 goto +0x2         14: if r6 == 0x0 goto pc+4
     9:  call 0x76                      15: r0 = 0xffffffff8d2079c0
                                        17: r0 = *(u64 *)(r0 +0)
    10:  *(u64 *)(r6 + 0x0) = r0        18: *(u64 *)(r6 +0) = r0
    11:  r0 = 0x0                       19: r0 = 0x0
    12:  exit                           20: exit

An instruction array map, containing, e.g., instructions [0,4,7,12] will be translated by the verifier to [0,4,13,20]. A map with index 5 (the middle of 16-byte instruction) or indexes greater than 12 (outside the program boundaries) would be rejected.

The functionality provided by this patch will be extended in consequent patches to implement BPF Static Keys, indirect jumps, and indirect calls.

---
 arch/x86/net/bpf_jit_comp.c    |   9 ++
 include/linux/bpf.h            |  15 ++
 include/linux/bpf_types.h      |   1 +
 include/linux/bpf_verifier.h   |   2 +
 include/uapi/linux/bpf.h       |  21 +++
 kernel/bpf/Makefile            |   2 +-
 kernel/bpf/bpf_insn_array.c    | 286 +++++++++++++++++++++++++++++++++
 kernel/bpf/syscall.c           |  22 +++
 kernel/bpf/verifier.c          |  43 +++++
 tools/include/uapi/linux/bpf.h |  21 +++
 10 files changed, 421 insertions(+), 1 deletion(-)
 create mode 100644 kernel/bpf/bpf_insn_array.c
arch/x86/net/bpf_jit_comp.c +9 -0
index d4c93d9e73e4..6e3574033473 100644
@@ -3827,6 +3827,15 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
 			jit_data->header = header;
 			jit_data->rw_header = rw_header;
 		}
+
+		/*
+		 * The bpf_prog_update_insn_ptrs function expects addrs to
+		 * point to the first byte of the jitted instruction (unlike
+		 * the bpf_prog_fill_jited_linfo below, which, for historical
+		 * reasons, expects to point to the next instruction)
+		 */
+		bpf_prog_update_insn_ptrs(prog, addrs, image);
+
 		/*
 		 * ctx.prog_offset is used when CFI preambles put code *before*
 		 * the function. See emit_cfi(). For FineIBT specifically this code
include/linux/bpf.h +15 -0
index a47d67db3be5..9d41a6affcef 100644
@@ -3797,4 +3797,19 @@ int bpf_prog_get_file_line(struct bpf_prog *prog, unsigned long ip, const char *
 			   const char **linep, int *nump);
 struct bpf_prog *bpf_prog_find_from_stack(void);
 
+int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
+int bpf_insn_array_ready(struct bpf_map *map);
+void bpf_insn_array_release(struct bpf_map *map);
+void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
+void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);
+
+#ifdef CONFIG_BPF_SYSCALL
+void bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image);
+#else
+static inline void
+bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image)
+{
+}
+#endif
+
 #endif /* _LINUX_BPF_H */
include/linux/bpf_types.h +1 -0
index fa78f49d4a9a..b13de31e163f 100644
@@ -133,6 +133,7 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_RINGBUF, ringbuf_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_BLOOM_FILTER, bloom_filter_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_USER_RINGBUF, user_ringbuf_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_ARENA, arena_map_ops)
+BPF_MAP_TYPE(BPF_MAP_TYPE_INSN_ARRAY, insn_array_map_ops)
 
 BPF_LINK_TYPE(BPF_LINK_TYPE_RAW_TRACEPOINT, raw_tracepoint)
 BPF_LINK_TYPE(BPF_LINK_TYPE_TRACING, tracing)
include/linux/bpf_verifier.h +2 -0
index c6eb68b6389c..6b820d8d77af 100644
@@ -754,8 +754,10 @@ struct bpf_verifier_env {
 	struct list_head free_list;	/* list of struct bpf_verifier_state_list */
 	struct bpf_map *used_maps[MAX_USED_MAPS]; /* array of map's used by eBPF program */
 	struct btf_mod_pair used_btfs[MAX_USED_BTFS]; /* array of BTF's used by BPF program */
+	struct bpf_map *insn_array_maps[MAX_USED_MAPS]; /* array of INSN_ARRAY map's to be relocated */
 	u32 used_map_cnt;		/* number of used maps */
 	u32 used_btf_cnt;		/* number of used BTF objects */
+	u32 insn_array_map_cnt;		/* number of used maps of type BPF_MAP_TYPE_INSN_ARRAY */
 	u32 id_gen;			/* used to generate unique reg IDs */
 	u32 hidden_subprog_cnt;		/* number of hidden subprogs */
 	int exception_callback_subprog;
include/uapi/linux/bpf.h +21 -0
index 1d73f165394d..f5713f59ac10 100644
@@ -1026,6 +1026,7 @@ enum bpf_map_type {
 	BPF_MAP_TYPE_USER_RINGBUF,
 	BPF_MAP_TYPE_CGRP_STORAGE,
 	BPF_MAP_TYPE_ARENA,
+	BPF_MAP_TYPE_INSN_ARRAY,
 	__MAX_BPF_MAP_TYPE
 };
 
@@ -7649,4 +7650,24 @@ enum bpf_kfunc_flags {
 	BPF_F_PAD_ZEROS = (1ULL << 0),
 };
 
+/*
+ * Values of a BPF_MAP_TYPE_INSN_ARRAY entry must be of this type.
+ *
+ * Before the map is used the orig_off field should point to an
+ * instruction inside the program being loaded. The other fields
+ * must be set to 0.
+ *
+ * After the program is loaded, the xlated_off will be adjusted
+ * by the verifier to point to the index of the original instruction
+ * in the xlated program. If the instruction is deleted, it will
+ * be set to (u32)-1. The jitted_off will be set to the corresponding
+ * offset in the jitted image of the program.
+ */
+struct bpf_insn_array_value {
+	__u32 orig_off;
+	__u32 xlated_off;
+	__u32 jitted_off;
+	__u32 :32;
+};
+
 #endif /* _UAPI__LINUX_BPF_H__ */
kernel/bpf/Makefile +1 -1
index 7fd0badfacb1..232cbc97434d 100644
@@ -9,7 +9,7 @@ CFLAGS_core.o += -Wno-override-init $(cflags-nogcse-yy)
 obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o log.o token.o liveness.o
 obj-$(CONFIG_BPF_SYSCALL) += bpf_iter.o map_iter.o task_iter.o prog_iter.o link_iter.o
 obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o bloom_filter.o
-obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o
+obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o bpf_insn_array.o
 obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o
 obj-${CONFIG_BPF_LSM}	  += bpf_inode_storage.o
 obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o
kernel/bpf/bpf_insn_array.c new file +286 -0

286 lines not on this page · raw message

kernel/bpf/syscall.c +22 -0
index 8a129746bd6c..f62d61b6730a 100644
@@ -1493,6 +1493,7 @@ static int map_create(union bpf_attr *attr, bpfptr_t uattr)
 	case BPF_MAP_TYPE_STRUCT_OPS:
 	case BPF_MAP_TYPE_CPUMAP:
 	case BPF_MAP_TYPE_ARENA:
+	case BPF_MAP_TYPE_INSN_ARRAY:
 		if (!bpf_token_capable(token, CAP_BPF))
 			goto put_token;
 		break;
@@ -2853,6 +2854,23 @@ static int bpf_prog_verify_signature(struct bpf_prog *prog, union bpf_attr *attr
 	return err;
 }
 
+static int bpf_prog_mark_insn_arrays_ready(struct bpf_prog *prog)
+{
+	int err;
+	int i;
+
+	for (i = 0; i < prog->aux->used_map_cnt; i++) {
+		if (prog->aux->used_maps[i]->map_type != BPF_MAP_TYPE_INSN_ARRAY)
+			continue;
+
+		err = bpf_insn_array_ready(prog->aux->used_maps[i]);
+		if (err)
+			return err;
+	}
+
+	return 0;
+}
+
 /* last field in 'union bpf_attr' used by this command */
 #define BPF_PROG_LOAD_LAST_FIELD keyring_id
 
@@ -3082,6 +3100,10 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
 	if (err < 0)
 		goto free_used_maps;
 
+	err = bpf_prog_mark_insn_arrays_ready(prog);
+	if (err < 0)
+		goto free_used_maps;
+
 	err = bpf_prog_alloc_id(prog);
 	if (err)
 		goto free_used_maps;
kernel/bpf/verifier.c +43 -0

79 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +21 -0
index 1d73f165394d..f5713f59ac10 100644
@@ -1026,6 +1026,7 @@ enum bpf_map_type {
 	BPF_MAP_TYPE_USER_RINGBUF,
 	BPF_MAP_TYPE_CGRP_STORAGE,
 	BPF_MAP_TYPE_ARENA,
+	BPF_MAP_TYPE_INSN_ARRAY,
 	__MAX_BPF_MAP_TYPE
 };
 
@@ -7649,4 +7650,24 @@ enum bpf_kfunc_flags {
 	BPF_F_PAD_ZEROS = (1ULL << 0),
 };
 
+/*
+ * Values of a BPF_MAP_TYPE_INSN_ARRAY entry must be of this type.
+ *
+ * Before the map is used the orig_off field should point to an
+ * instruction inside the program being loaded. The other fields
+ * must be set to 0.
+ *
+ * After the program is loaded, the xlated_off will be adjusted
+ * by the verifier to point to the index of the original instruction
+ * in the xlated program. If the instruction is deleted, it will
+ * be set to (u32)-1. The jitted_off will be set to the corresponding
+ * offset in the jitted image of the program.
+ */
+struct bpf_insn_array_value {
+	__u32 orig_off;
+	__u32 xlated_off;
+	__u32 jitted_off;
+	__u32 :32;
+};
+
 #endif /* _UAPI__LINUX_BPF_H__ */
Signature · 1 line
2.34.1
#2 lore raw

PATCH v10 2/11 selftests/bpf: add selftests for new insn_array map

In reply to Anton Protopopov

Add the following selftests for new insn_array map:

* Incorrect instruction indexes are rejected
* Two programs can't use the same map
* BPF progs can't operate the map
* no changes to code => map is the same
* expected changes when instructions are added
* expected changes when instructions are deleted
* expected changes when multiple functions are present

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 409 ++++++++++++++++++
 1 file changed, 409 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +409 -0
new file mode 100644
index 000000000000..96ee9c9984f1
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <bpf/bpf.h>
+#include <test_progs.h>
+
+#ifdef __x86_64__
+static int map_create(__u32 map_type, __u32 max_entries)
+{
+	const char *map_name = "insn_array";
+	__u32 key_size = 4;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+
+	return bpf_map_create(map_type, map_name, key_size, value_size, max_entries, NULL);
+}
+
+static int prog_load(struct bpf_insn *insns, __u32 insn_cnt, int *fd_array, __u32 fd_array_cnt)
+{
+	LIBBPF_OPTS(bpf_prog_load_opts, opts);
+
+	opts.fd_array = fd_array;
+	opts.fd_array_cnt = fd_array_cnt;
+
+	return bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+}
+
+static void __check_success(struct bpf_insn *insns, __u32 insn_cnt, __u32 *map_in, __u32 *map_out)
+{
+	struct bpf_insn_array_value val = {};
+	int prog_fd = -1, map_fd, i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, insn_cnt);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	for (i = 0; i < insn_cnt; i++) {
+		val.orig_off = map_in[i];
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+			goto cleanup;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	prog_fd = prog_load(insns, insn_cnt, &map_fd, 1);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	for (i = 0; i < insn_cnt; i++) {
+		char buf[64];
+
+		if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+			goto cleanup;
+
+		snprintf(buf, sizeof(buf), "val.xlated_off should be equal map_out[%d]", i);
+		ASSERT_EQ(val.xlated_off, map_out[i], buf);
+	}
+
+cleanup:
+	close(prog_fd);
+	close(map_fd);
+}
+
+/*
+ * Load a program, which will not be anyhow mangled by the verifier.  Add an
+ * insn_array map pointing to every instruction. Check that it hasn't changed
+ * after the program load.
+ */
+static void check_one_to_one_mapping(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 4),
+		BPF_MOV64_IMM(BPF_REG_0, 3),
+		BPF_MOV64_IMM(BPF_REG_0, 2),
+		BPF_MOV64_IMM(BPF_REG_0, 1),
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	__u32 map_in[] = {0, 1, 2, 3, 4, 5};
+	__u32 map_out[] = {0, 1, 2, 3, 4, 5};
+
+	__check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Load a program with two patches (get jiffies, for simplicity). Add an
+ * insn_array map pointing to every instruction. Check how it was changed
+ * after the program load.
+ */
+static void check_simple(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 2),
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+		BPF_MOV64_IMM(BPF_REG_0, 1),
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	__u32 map_in[] = {0, 1, 2, 3, 4, 5};
+	__u32 map_out[] = {0, 1, 4, 5, 8, 9};
+
+	__check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Verifier can delete code in two cases: nops & dead code. From insn
+ * array's point of view, the two cases are the same, so test using
+ * the simplest method: by loading some nops
+ */
+static void check_deletions(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 2),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+		BPF_MOV64_IMM(BPF_REG_0, 1),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	__u32 map_in[] = {0, 1, 2, 3, 4, 5};
+	__u32 map_out[] = {0, -1, 1, -1, 2, 3};
+
+	__check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Same test as check_deletions, but also add code which adds instructions
+ */
+static void check_deletions_with_functions(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 2),
+		BPF_MOV64_IMM(BPF_REG_0, 1),
+		BPF_EXIT_INSN(),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+		BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+		BPF_MOV64_IMM(BPF_REG_0, 2),
+		BPF_EXIT_INSN(),
+	};
+	__u32 map_in[] =  { 0, 1,  2, 3, 4, 5, /* func */  6, 7,  8, 9, 10};
+	__u32 map_out[] = {-1, 0, -1, 3, 4, 5, /* func */ -1, 6, -1, 9, 10};
+
+	__check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Try to load a program with a map which points to outside of the program
+ */
+static void check_out_of_bounds_index(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 4),
+		BPF_MOV64_IMM(BPF_REG_0, 3),
+		BPF_MOV64_IMM(BPF_REG_0, 2),
+		BPF_MOV64_IMM(BPF_REG_0, 1),
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd, map_fd;
+	struct bpf_insn_array_value val = {};
+	int key;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	key = 0;
+	val.orig_off = ARRAY_SIZE(insns); /* too big */
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_EQ(prog_fd, -EINVAL, "program should have been rejected (prog_fd != -EINVAL)")) {
+		close(prog_fd);
+		goto cleanup;
+	}
+
+cleanup:
+	close(map_fd);
+}
+
+/*
+ * Try to load a program with a map which points to the middle of 16-bit insn
+ */
+static void check_mid_insn_index(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_LD_IMM64(BPF_REG_0, 0), /* 2 x 8 */
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd, map_fd;
+	struct bpf_insn_array_value val = {};
+	int key;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	key = 0;
+	val.orig_off = 1; /* middle of 16-byte instruction */
+	if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &key, &val, 0), 0, "bpf_map_update_elem"))
+		goto cleanup;
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_EQ(prog_fd, -EINVAL, "program should have been rejected (prog_fd != -EINVAL)")) {
+		close(prog_fd);
+		goto cleanup;
+	}
+
+cleanup:
+	close(map_fd);
+}
+
+static void check_incorrect_index(void)
+{
+	check_out_of_bounds_index();
+	check_mid_insn_index();
+}
+
+/* Once map was initialized, it should be frozen */
+static void check_load_unfrozen_map(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd = -1, map_fd;
+	struct bpf_insn_array_value val = {};
+	int i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, ARRAY_SIZE(insns));
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(insns); i++) {
+		val.orig_off = i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+			goto cleanup;
+	}
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_EQ(prog_fd, -EINVAL, "program should have been rejected (prog_fd != -EINVAL)"))
+		goto cleanup;
+
+	/* correctness: now freeze the map, the program should load fine */
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	for (i = 0; i < ARRAY_SIZE(insns); i++) {
+		if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+			goto cleanup;
+
+		ASSERT_EQ(val.xlated_off, i, "val should be equal i");
+	}
+
+cleanup:
+	close(prog_fd);
+	close(map_fd);
+}
+
+/* Map can be used only by one BPF program */
+static void check_no_map_reuse(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 0),
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd = -1, map_fd, extra_fd = -1;
+	struct bpf_insn_array_value val = {};
+	int i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, ARRAY_SIZE(insns));
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(insns); i++) {
+		val.orig_off = i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+			goto cleanup;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	for (i = 0; i < ARRAY_SIZE(insns); i++) {
+		if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+			goto cleanup;
+
+		ASSERT_EQ(val.xlated_off, i, "val should be equal i");
+	}
+
+	extra_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_EQ(extra_fd, -EBUSY, "program should have been rejected (extra_fd != -EBUSY)"))
+		goto cleanup;
+
+	/* correctness: check that prog is still loadable without fd_array */
+	extra_fd = prog_load(insns, ARRAY_SIZE(insns), NULL, 0);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD): expected no error"))
+		goto cleanup;
+
+cleanup:
+	close(extra_fd);
+	close(prog_fd);
+	close(map_fd);
+}
+
+static void check_bpf_no_lookup(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_LD_MAP_FD(BPF_REG_1, 0),
+		BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+		BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+		BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+		BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd = -1, map_fd;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, 1);
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	insns[0].imm = map_fd;
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), NULL, 0);
+	if (!ASSERT_EQ(prog_fd, -EINVAL, "program should have been rejected (prog_fd != -EINVAL)"))
+		goto cleanup;
+
+	/* correctness: check that prog is still loadable with normal map */
+	close(map_fd);
+	map_fd = map_create(BPF_MAP_TYPE_ARRAY, 1);
+	insns[0].imm = map_fd;
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), NULL, 0);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+cleanup:
+	close(prog_fd);
+	close(map_fd);
+}
+
+static void check_bpf_side(void)
+{
+	check_bpf_no_lookup();
+}
+
+static void __test_bpf_insn_array(void)
+{
+	/* Test if offsets are adjusted properly */
+
+	if (test__start_subtest("one2one"))
+		check_one_to_one_mapping();
+
+	if (test__start_subtest("simple"))
+		check_simple();
+
+	if (test__start_subtest("deletions"))
+		check_deletions();
+
+	if (test__start_subtest("deletions-with-functions"))
+		check_deletions_with_functions();
+
+	/* Check all kinds of operations and related restrictions */
+
+	if (test__start_subtest("incorrect-index"))
+		check_incorrect_index();
+
+	if (test__start_subtest("load-unfrozen-map"))
+		check_load_unfrozen_map();
+
+	if (test__start_subtest("no-map-reuse"))
+		check_no_map_reuse();
+
+	if (test__start_subtest("bpf-side-ops"))
+		check_bpf_side();
+}
+#else
+static void __test_bpf_insn_array(void)
+{
+	test__skip();
+}
+#endif
+
+void test_bpf_insn_array(void)
+{
+	__test_bpf_insn_array();
+}
Signature · 1 line
2.34.1
#3 lore raw

Re: [PATCH v10 bpf-next 02/11] selftests/bpf: add selftests for new insn_array map

Quoted 162 lines · Anton Protopopov

Add the following selftests for new insn_array map:

* Incorrect instruction indexes are rejected
* Two programs can't use the same map
* BPF progs can't operate the map
* no changes to code => map is the same
* expected changes when instructions are added
* expected changes when instructions are deleted
* expected changes when multiple functions are present

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 409 ++++++++++++++++++
 1 file changed, 409 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +138 -0
new file mode 100644
index 000000000000..96ee9c9984f1
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <bpf/bpf.h>
+#include <test_progs.h>
+
+#ifdef __x86_64__
+static int map_create(__u32 map_type, __u32 max_entries)
+{
+       const char *map_name = "insn_array";
+       __u32 key_size = 4;
+       __u32 value_size = sizeof(struct bpf_insn_array_value);
+
+       return bpf_map_create(map_type, map_name, key_size, value_size, max_entries, NULL);
+}
+
+static int prog_load(struct bpf_insn *insns, __u32 insn_cnt, int *fd_array, __u32 fd_array_cnt)
+{
+       LIBBPF_OPTS(bpf_prog_load_opts, opts);
+
+       opts.fd_array = fd_array;
+       opts.fd_array_cnt = fd_array_cnt;
+
+       return bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+}
+
+static void __check_success(struct bpf_insn *insns, __u32 insn_cnt, __u32 *map_in, __u32 *map_out)
+{
+       struct bpf_insn_array_value val = {};
+       int prog_fd = -1, map_fd, i;
+
+       map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, insn_cnt);
+       if (!ASSERT_GE(map_fd, 0, "map_create"))
+               return;
+
+       for (i = 0; i < insn_cnt; i++) {
+               val.orig_off = map_in[i];
+               if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+                       goto cleanup;
+       }
+
+       if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+               goto cleanup;
+
+       prog_fd = prog_load(insns, insn_cnt, &map_fd, 1);
+       if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+               goto cleanup;
+
+       for (i = 0; i < insn_cnt; i++) {
+               char buf[64];
+
+               if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+                       goto cleanup;
+
+               snprintf(buf, sizeof(buf), "val.xlated_off should be equal map_out[%d]", i);
+               ASSERT_EQ(val.xlated_off, map_out[i], buf);
+       }
+
+cleanup:
+       close(prog_fd);
+       close(map_fd);
+}
+
+/*
+ * Load a program, which will not be anyhow mangled by the verifier.  Add an
+ * insn_array map pointing to every instruction. Check that it hasn't changed
+ * after the program load.
+ */
+static void check_one_to_one_mapping(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 4),
+               BPF_MOV64_IMM(BPF_REG_0, 3),
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 2, 3, 4, 5};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Load a program with two patches (get jiffies, for simplicity). Add an
+ * insn_array map pointing to every instruction. Check how it was changed
+ * after the program load.
+ */
+static void check_simple(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 4, 5, 8, 9};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Verifier can delete code in two cases: nops & dead code. From insn
+ * array's point of view, the two cases are the same, so test using
+ * the simplest method: by loading some nops
+ */
+static void check_deletions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, -1, 1, -1, 2, 3};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Same test as check_deletions, but also add code which adds instructions
+ */
+static void check_deletions_with_functions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_EXIT_INSN(),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c line 148

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +10 -0
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] =  { 0, 1,  2, 3, 4, 5, /* func */  6, 7,  8, 9, 10};
+       __u32 map_out[] = {-1, 0, -1, 3, 4, 5, /* func */ -1, 6, -1, 9, 10};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}

I was thinking of taking the first 5 patches, but this one fails:
./test_progs -t bpf_insn_array
...

#19/4    bpf_insn_array/deletions-with-functions:FAIL
#19/5    bpf_insn_array/blindness:OK
#19/6    bpf_insn_array/incorrect-index:OK
#19/7    bpf_insn_array/load-unfrozen-map:OK
#19/8    bpf_insn_array/no-map-reuse:OK
#19/9    bpf_insn_array/bpf-side-ops:OK
#19      bpf_insn_array:FAIL

I don't see what you're changing later in the patches to make it pass, but the failure highlights the issue with bisectability. Pls take a look.

This one also fails:
#170/3 libbpf_str/bpf_map_type_str:FAIL
#170 libbpf_str:FAIL

I was thinking of hacking it as an extra patch (without full support in patch 8), but gave up when I saw deletions-with-functions failing.

Maybe also split the main libbpf patch into prep patch with basic introduction of insn_array ?
Or keep it as-is, if respin comes soon.

deletions-with-functions is the most suspicious.

#4 lore raw

Re: [PATCH v10 bpf-next 02/11] selftests/bpf: add selftests for new insn_array map

Quoted 186 lines · Alexei Starovoitov
Quoted 162 lines · Anton Protopopov

Add the following selftests for new insn_array map:

* Incorrect instruction indexes are rejected
* Two programs can't use the same map
* BPF progs can't operate the map
* no changes to code => map is the same
* expected changes when instructions are added
* expected changes when instructions are deleted
* expected changes when multiple functions are present

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 409 ++++++++++++++++++
 1 file changed, 409 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +138 -0
new file mode 100644
index 000000000000..96ee9c9984f1
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <bpf/bpf.h>
+#include <test_progs.h>
+
+#ifdef __x86_64__
+static int map_create(__u32 map_type, __u32 max_entries)
+{
+       const char *map_name = "insn_array";
+       __u32 key_size = 4;
+       __u32 value_size = sizeof(struct bpf_insn_array_value);
+
+       return bpf_map_create(map_type, map_name, key_size, value_size, max_entries, NULL);
+}
+
+static int prog_load(struct bpf_insn *insns, __u32 insn_cnt, int *fd_array, __u32 fd_array_cnt)
+{
+       LIBBPF_OPTS(bpf_prog_load_opts, opts);
+
+       opts.fd_array = fd_array;
+       opts.fd_array_cnt = fd_array_cnt;
+
+       return bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+}
+
+static void __check_success(struct bpf_insn *insns, __u32 insn_cnt, __u32 *map_in, __u32 *map_out)
+{
+       struct bpf_insn_array_value val = {};
+       int prog_fd = -1, map_fd, i;
+
+       map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, insn_cnt);
+       if (!ASSERT_GE(map_fd, 0, "map_create"))
+               return;
+
+       for (i = 0; i < insn_cnt; i++) {
+               val.orig_off = map_in[i];
+               if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+                       goto cleanup;
+       }
+
+       if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+               goto cleanup;
+
+       prog_fd = prog_load(insns, insn_cnt, &map_fd, 1);
+       if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+               goto cleanup;
+
+       for (i = 0; i < insn_cnt; i++) {
+               char buf[64];
+
+               if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+                       goto cleanup;
+
+               snprintf(buf, sizeof(buf), "val.xlated_off should be equal map_out[%d]", i);
+               ASSERT_EQ(val.xlated_off, map_out[i], buf);
+       }
+
+cleanup:
+       close(prog_fd);
+       close(map_fd);
+}
+
+/*
+ * Load a program, which will not be anyhow mangled by the verifier.  Add an
+ * insn_array map pointing to every instruction. Check that it hasn't changed
+ * after the program load.
+ */
+static void check_one_to_one_mapping(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 4),
+               BPF_MOV64_IMM(BPF_REG_0, 3),
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 2, 3, 4, 5};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Load a program with two patches (get jiffies, for simplicity). Add an
+ * insn_array map pointing to every instruction. Check how it was changed
+ * after the program load.
+ */
+static void check_simple(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 4, 5, 8, 9};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Verifier can delete code in two cases: nops & dead code. From insn
+ * array's point of view, the two cases are the same, so test using
+ * the simplest method: by loading some nops
+ */
+static void check_deletions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, -1, 1, -1, 2, 3};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Same test as check_deletions, but also add code which adds instructions
+ */
+static void check_deletions_with_functions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_EXIT_INSN(),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c line 148

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +10 -0
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] =  { 0, 1,  2, 3, 4, 5, /* func */  6, 7,  8, 9, 10};
+       __u32 map_out[] = {-1, 0, -1, 3, 4, 5, /* func */ -1, 6, -1, 9, 10};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}

I was thinking of taking the first 5 patches, but this one fails:
./test_progs -t bpf_insn_array
...

#19/4    bpf_insn_array/deletions-with-functions:FAIL
#19/5    bpf_insn_array/blindness:OK
#19/6    bpf_insn_array/incorrect-index:OK
#19/7    bpf_insn_array/load-unfrozen-map:OK
#19/8    bpf_insn_array/no-map-reuse:OK
#19/9    bpf_insn_array/bpf-side-ops:OK
#19      bpf_insn_array:FAIL
Quoted 3 lines

I don't see what you're changing later in the patches to make it pass, but the failure highlights the issue with bisectability. Pls take a look.

Thanks! I've found the chunk, it was

@@ -21664,2 +21705,4 @@ static int jit_subprogs(struct bpf_verifier_env *env)

                func[i]->aux->arena = prog->aux->arena;
+               func[i]->aux->used_maps = env->used_maps;
+               func[i]->aux->used_map_cnt = env->used_map_cnt;
                num_exentries = 0;
Quoted 10 lines

This one also fails:
#170/3 libbpf_str/bpf_map_type_str:FAIL
#170 libbpf_str:FAIL

I was thinking of hacking it as an extra patch (without full support in patch 8), but gave up when I saw deletions-with-functions failing.

Maybe also split the main libbpf patch into prep patch with basic introduction of insn_array ?

I've split a commit that teaches libbpf about insn_array + moved the bpftool commit lower. All the tests pass now.

Quoted 1 line

Or keep it as-is, if respin comes soon.

I can send the first chunk separately today,
or the whole thing a few days later.

Quoted 1 line

[...]

#5 lore raw

Re: [PATCH v10 bpf-next 02/11] selftests/bpf: add selftests for new insn_array map

Quoted 195 lines · Anton Protopopov
Quoted 186 lines · Alexei Starovoitov
Quoted 162 lines · Anton Protopopov

Add the following selftests for new insn_array map:

* Incorrect instruction indexes are rejected
* Two programs can't use the same map
* BPF progs can't operate the map
* no changes to code => map is the same
* expected changes when instructions are added
* expected changes when instructions are deleted
* expected changes when multiple functions are present

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 409 ++++++++++++++++++
 1 file changed, 409 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +138 -0
new file mode 100644
index 000000000000..96ee9c9984f1
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <bpf/bpf.h>
+#include <test_progs.h>
+
+#ifdef __x86_64__
+static int map_create(__u32 map_type, __u32 max_entries)
+{
+       const char *map_name = "insn_array";
+       __u32 key_size = 4;
+       __u32 value_size = sizeof(struct bpf_insn_array_value);
+
+       return bpf_map_create(map_type, map_name, key_size, value_size, max_entries, NULL);
+}
+
+static int prog_load(struct bpf_insn *insns, __u32 insn_cnt, int *fd_array, __u32 fd_array_cnt)
+{
+       LIBBPF_OPTS(bpf_prog_load_opts, opts);
+
+       opts.fd_array = fd_array;
+       opts.fd_array_cnt = fd_array_cnt;
+
+       return bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+}
+
+static void __check_success(struct bpf_insn *insns, __u32 insn_cnt, __u32 *map_in, __u32 *map_out)
+{
+       struct bpf_insn_array_value val = {};
+       int prog_fd = -1, map_fd, i;
+
+       map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, insn_cnt);
+       if (!ASSERT_GE(map_fd, 0, "map_create"))
+               return;
+
+       for (i = 0; i < insn_cnt; i++) {
+               val.orig_off = map_in[i];
+               if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+                       goto cleanup;
+       }
+
+       if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+               goto cleanup;
+
+       prog_fd = prog_load(insns, insn_cnt, &map_fd, 1);
+       if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+               goto cleanup;
+
+       for (i = 0; i < insn_cnt; i++) {
+               char buf[64];
+
+               if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+                       goto cleanup;
+
+               snprintf(buf, sizeof(buf), "val.xlated_off should be equal map_out[%d]", i);
+               ASSERT_EQ(val.xlated_off, map_out[i], buf);
+       }
+
+cleanup:
+       close(prog_fd);
+       close(map_fd);
+}
+
+/*
+ * Load a program, which will not be anyhow mangled by the verifier.  Add an
+ * insn_array map pointing to every instruction. Check that it hasn't changed
+ * after the program load.
+ */
+static void check_one_to_one_mapping(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 4),
+               BPF_MOV64_IMM(BPF_REG_0, 3),
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 2, 3, 4, 5};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Load a program with two patches (get jiffies, for simplicity). Add an
+ * insn_array map pointing to every instruction. Check how it was changed
+ * after the program load.
+ */
+static void check_simple(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 4, 5, 8, 9};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Verifier can delete code in two cases: nops & dead code. From insn
+ * array's point of view, the two cases are the same, so test using
+ * the simplest method: by loading some nops
+ */
+static void check_deletions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, -1, 1, -1, 2, 3};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Same test as check_deletions, but also add code which adds instructions
+ */
+static void check_deletions_with_functions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_EXIT_INSN(),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c line 148

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +10 -0
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] =  { 0, 1,  2, 3, 4, 5, /* func */  6, 7,  8, 9, 10};
+       __u32 map_out[] = {-1, 0, -1, 3, 4, 5, /* func */ -1, 6, -1, 9, 10};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}

I was thinking of taking the first 5 patches, but this one fails:
./test_progs -t bpf_insn_array
...

#19/4    bpf_insn_array/deletions-with-functions:FAIL
#19/5    bpf_insn_array/blindness:OK
#19/6    bpf_insn_array/incorrect-index:OK
#19/7    bpf_insn_array/load-unfrozen-map:OK
#19/8    bpf_insn_array/no-map-reuse:OK
#19/9    bpf_insn_array/bpf-side-ops:OK
#19      bpf_insn_array:FAIL
Quoted 3 lines

I don't see what you're changing later in the patches to make it pass, but the failure highlights the issue with bisectability. Pls take a look.

Thanks! I've found the chunk, it was

Review static int jit_subprogs(struct bpf_verifier_env *env) · line 21708

Quoted 4 lines
(quoted diff) +2 -0
@@ -21664,2 +21705,4 @@ static int jit_subprogs(struct bpf_verifier_env *env)
                func[i]->aux->arena = prog->aux->arena;
+               func[i]->aux->used_maps = env->used_maps;
+               func[i]->aux->used_map_cnt = env->used_map_cnt;
                num_exentries = 0;

argh. No need for this copy. Pls use prog->aux->main_prog_aux instead.

Quoted 11 lines
Quoted 10 lines

This one also fails:
#170/3 libbpf_str/bpf_map_type_str:FAIL
#170 libbpf_str:FAIL

I was thinking of hacking it as an extra patch (without full support in patch 8), but gave up when I saw deletions-with-functions failing.

Maybe also split the main libbpf patch into prep patch with basic introduction of insn_array ?

Quoted 7 lines

I've split a commit that teaches libbpf about insn_array + moved the bpftool commit lower. All the tests pass now.

Quoted 1 line

Or keep it as-is, if respin comes soon.

I can send the first chunk separately today,
or the whole thing a few days later.

Up to you. Smaller chunks are easier to review.

#6 lore raw

Re: [PATCH v10 bpf-next 02/11] selftests/bpf: add selftests for new insn_array map

Quoted 196 lines · Alexei Starovoitov
Quoted 194 lines · Anton Protopopov
Quoted 186 lines · Alexei Starovoitov
Quoted 162 lines · Anton Protopopov

Add the following selftests for new insn_array map:

* Incorrect instruction indexes are rejected
* Two programs can't use the same map
* BPF progs can't operate the map
* no changes to code => map is the same
* expected changes when instructions are added
* expected changes when instructions are deleted
* expected changes when multiple functions are present

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 409 ++++++++++++++++++
 1 file changed, 409 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +138 -0
new file mode 100644
index 000000000000..96ee9c9984f1
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <bpf/bpf.h>
+#include <test_progs.h>
+
+#ifdef __x86_64__
+static int map_create(__u32 map_type, __u32 max_entries)
+{
+       const char *map_name = "insn_array";
+       __u32 key_size = 4;
+       __u32 value_size = sizeof(struct bpf_insn_array_value);
+
+       return bpf_map_create(map_type, map_name, key_size, value_size, max_entries, NULL);
+}
+
+static int prog_load(struct bpf_insn *insns, __u32 insn_cnt, int *fd_array, __u32 fd_array_cnt)
+{
+       LIBBPF_OPTS(bpf_prog_load_opts, opts);
+
+       opts.fd_array = fd_array;
+       opts.fd_array_cnt = fd_array_cnt;
+
+       return bpf_prog_load(BPF_PROG_TYPE_XDP, NULL, "GPL", insns, insn_cnt, &opts);
+}
+
+static void __check_success(struct bpf_insn *insns, __u32 insn_cnt, __u32 *map_in, __u32 *map_out)
+{
+       struct bpf_insn_array_value val = {};
+       int prog_fd = -1, map_fd, i;
+
+       map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, insn_cnt);
+       if (!ASSERT_GE(map_fd, 0, "map_create"))
+               return;
+
+       for (i = 0; i < insn_cnt; i++) {
+               val.orig_off = map_in[i];
+               if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+                       goto cleanup;
+       }
+
+       if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+               goto cleanup;
+
+       prog_fd = prog_load(insns, insn_cnt, &map_fd, 1);
+       if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+               goto cleanup;
+
+       for (i = 0; i < insn_cnt; i++) {
+               char buf[64];
+
+               if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+                       goto cleanup;
+
+               snprintf(buf, sizeof(buf), "val.xlated_off should be equal map_out[%d]", i);
+               ASSERT_EQ(val.xlated_off, map_out[i], buf);
+       }
+
+cleanup:
+       close(prog_fd);
+       close(map_fd);
+}
+
+/*
+ * Load a program, which will not be anyhow mangled by the verifier.  Add an
+ * insn_array map pointing to every instruction. Check that it hasn't changed
+ * after the program load.
+ */
+static void check_one_to_one_mapping(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 4),
+               BPF_MOV64_IMM(BPF_REG_0, 3),
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 2, 3, 4, 5};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Load a program with two patches (get jiffies, for simplicity). Add an
+ * insn_array map pointing to every instruction. Check how it was changed
+ * after the program load.
+ */
+static void check_simple(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, 1, 4, 5, 8, 9};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Verifier can delete code in two cases: nops & dead code. From insn
+ * array's point of view, the two cases are the same, so test using
+ * the simplest method: by loading some nops
+ */
+static void check_deletions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 0),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] = {0, 1, 2, 3, 4, 5};
+       __u32 map_out[] = {0, -1, 1, -1, 2, 3};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}
+
+/*
+ * Same test as check_deletions, but also add code which adds instructions
+ */
+static void check_deletions_with_functions(void)
+{
+       struct bpf_insn insns[] = {
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 2),
+               BPF_MOV64_IMM(BPF_REG_0, 1),
+               BPF_EXIT_INSN(),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */

tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c line 148

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c new file +10 -0
+               BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_jiffies64),
+               BPF_JMP_IMM(BPF_JA, 0, 0, 0), /* nop */
+               BPF_MOV64_IMM(BPF_REG_0, 2),
+               BPF_EXIT_INSN(),
+       };
+       __u32 map_in[] =  { 0, 1,  2, 3, 4, 5, /* func */  6, 7,  8, 9, 10};
+       __u32 map_out[] = {-1, 0, -1, 3, 4, 5, /* func */ -1, 6, -1, 9, 10};
+
+       __check_success(insns, ARRAY_SIZE(insns), map_in, map_out);
+}

I was thinking of taking the first 5 patches, but this one fails:
./test_progs -t bpf_insn_array
...

#19/4    bpf_insn_array/deletions-with-functions:FAIL
#19/5    bpf_insn_array/blindness:OK
#19/6    bpf_insn_array/incorrect-index:OK
#19/7    bpf_insn_array/load-unfrozen-map:OK
#19/8    bpf_insn_array/no-map-reuse:OK
#19/9    bpf_insn_array/bpf-side-ops:OK
#19      bpf_insn_array:FAIL
Quoted 3 lines

I don't see what you're changing later in the patches to make it pass, but the failure highlights the issue with bisectability. Pls take a look.

Thanks! I've found the chunk, it was

Quoted 7 lines

Review static int jit_subprogs(struct bpf_verifier_env *env) · line 21708

Quoted 5 lines
(quoted diff) +2 -0
@@ -21664,2 +21705,4 @@ static int jit_subprogs(struct bpf_verifier_env *env)
                func[i]->aux->arena = prog->aux->arena;
+               func[i]->aux->used_maps = env->used_maps;
+               func[i]->aux->used_map_cnt = env->used_map_cnt;
                num_exentries = 0;

argh. No need for this copy. Pls use prog->aux->main_prog_aux instead.

It might be called before the used_maps are copied into aux...

Quoted 20 lines
Quoted 11 lines
Quoted 10 lines

This one also fails:
#170/3 libbpf_str/bpf_map_type_str:FAIL
#170 libbpf_str:FAIL

I was thinking of hacking it as an extra patch (without full support in patch 8), but gave up when I saw deletions-with-functions failing.

Maybe also split the main libbpf patch into prep patch with basic introduction of insn_array ?

Quoted 7 lines

I've split a commit that teaches libbpf about insn_array + moved the bpftool commit lower. All the tests pass now.

Quoted 1 line

Or keep it as-is, if respin comes soon.

I can send the first chunk separately today,
or the whole thing a few days later.

Up to you. Smaller chunks are easier to review.

#7 lore raw

PATCH v10 3/11 bpf: support instructions arrays with constants blinding

In reply to Anton Protopopov

When bpf_jit_harden is enabled, all constants in the BPF code are blinded to prevent JIT spraying attacks. This happens during JIT phase. Adjust all the related instruction arrays accordingly.

---
 kernel/bpf/core.c     | 20 ++++++++++++++++++++
 kernel/bpf/verifier.c | 11 ++++++++++-
 2 files changed, 30 insertions(+), 1 deletion(-)
kernel/bpf/core.c +20 -0
index d595fe512498..4b62a03d6df5 100644
@@ -1450,6 +1450,23 @@ void bpf_jit_prog_release_other(struct bpf_prog *fp, struct bpf_prog *fp_other)
 	bpf_prog_clone_free(fp_other);
 }
 
+static void adjust_insn_arrays(struct bpf_prog *prog, u32 off, u32 len)
+{
+#ifdef CONFIG_BPF_SYSCALL
+	struct bpf_map *map;
+	int i;
+
+	if (len <= 1)
+		return;
+
+	for (i = 0; i < prog->aux->used_map_cnt; i++) {
+		map = prog->aux->used_maps[i];
+		if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY)
+			bpf_insn_array_adjust(map, off, len);
+	}
+#endif
+}
+
 struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
 {
 	struct bpf_insn insn_buff[16], aux[2];
@@ -1505,6 +1522,9 @@ struct bpf_prog *bpf_jit_blind_constants(struct bpf_prog *prog)
 		clone = tmp;
 		insn_delta = rewritten - 1;
 
+		/* Instructions arrays must be updated using absolute xlated offsets */
+		adjust_insn_arrays(clone, prog->aux->subprog_start + i, rewritten);
+
 		/* Walk new program and skip insns we just inserted. */
 		insn = clone->insnsi + i + insn_delta;
 		insn_cnt += insn_delta;
kernel/bpf/verifier.c +10 -1
index dd597ea80d99..2b771e2bf35a 100644
@@ -21601,6 +21601,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 	struct bpf_insn *insn;
 	void *old_bpf_func;
 	int err, num_exentries;
+	int old_len, subprog_start_adjustment = 0;
 
 	if (env->subprog_cnt <= 1)
 		return 0;
@@ -21675,7 +21676,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 		func[i]->aux->func_idx = i;
 		/* Below members will be freed only at prog->aux */
 		func[i]->aux->btf = prog->aux->btf;
-		func[i]->aux->subprog_start = subprog_start;
+		func[i]->aux->subprog_start = subprog_start + subprog_start_adjustment;
 		func[i]->aux->func_info = prog->aux->func_info;
 		func[i]->aux->func_info_cnt = prog->aux->func_info_cnt;
 		func[i]->aux->poke_tab = prog->aux->poke_tab;
@@ -21729,7 +21730,15 @@ static int jit_subprogs(struct bpf_verifier_env *env)
 		func[i]->aux->might_sleep = env->subprog_info[i].might_sleep;
 		if (!i)
 			func[i]->aux->exception_boundary = env->seen_exception;
+
+		/*
+		 * To properly pass the absolute subprog start to jit
+		 * all instruction adjustments should be accumulated
+		 */
+		old_len = func[i]->len;
 		func[i] = bpf_int_jit_compile(func[i]);
+		subprog_start_adjustment += func[i]->len - old_len;
+
 		if (!func[i]->jited) {
 			err = -ENOTSUPP;
 			goto out_free;
Signature · 1 line
2.34.1
#8 lore raw

PATCH v10 4/11 selftests/bpf: test instructions arrays with blinding

In reply to Anton Protopopov

Add a specific test for instructions arrays with blinding enabled.

---
 .../selftests/bpf/prog_tests/bpf_insn_array.c | 95 +++++++++++++++++++
 1 file changed, 95 insertions(+)
tools/testing/selftests/bpf/prog_tests/bpf_insn_array.c +95 -0
index 96ee9c9984f1..cf852318eeb2 100644
@@ -227,6 +227,98 @@ static void check_incorrect_index(void)
 	check_mid_insn_index();
 }
 
+static int set_bpf_jit_harden(char *level)
+{
+	char old_level;
+	int err = -1;
+	int fd = -1;
+
+	fd = open("/proc/sys/net/core/bpf_jit_harden", O_RDWR | O_NONBLOCK);
+	if (fd < 0) {
+		ASSERT_FAIL("open .../bpf_jit_harden returned %d (errno=%d)", fd, errno);
+		return -1;
+	}
+
+	err = read(fd, &old_level, 1);
+	if (err != 1) {
+		ASSERT_FAIL("read from .../bpf_jit_harden returned %d (errno=%d)", err, errno);
+		err = -1;
+		goto end;
+	}
+
+	lseek(fd, 0, SEEK_SET);
+
+	err = write(fd, level, 1);
+	if (err != 1) {
+		ASSERT_FAIL("write to .../bpf_jit_harden returned %d (errno=%d)", err, errno);
+		err = -1;
+		goto end;
+	}
+
+	err = 0;
+	*level = old_level;
+end:
+	if (fd >= 0)
+		close(fd);
+	return err;
+}
+
+static void check_blindness(void)
+{
+	struct bpf_insn insns[] = {
+		BPF_MOV64_IMM(BPF_REG_0, 4),
+		BPF_MOV64_IMM(BPF_REG_0, 3),
+		BPF_MOV64_IMM(BPF_REG_0, 2),
+		BPF_MOV64_IMM(BPF_REG_0, 1),
+		BPF_EXIT_INSN(),
+	};
+	int prog_fd = -1, map_fd;
+	struct bpf_insn_array_value val = {};
+	char bpf_jit_harden = '@'; /* non-exizsting value */
+	int i;
+
+	map_fd = map_create(BPF_MAP_TYPE_INSN_ARRAY, ARRAY_SIZE(insns));
+	if (!ASSERT_GE(map_fd, 0, "map_create"))
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(insns); i++) {
+		val.orig_off = i;
+		if (!ASSERT_EQ(bpf_map_update_elem(map_fd, &i, &val, 0), 0, "bpf_map_update_elem"))
+			goto cleanup;
+	}
+
+	if (!ASSERT_EQ(bpf_map_freeze(map_fd), 0, "bpf_map_freeze"))
+		goto cleanup;
+
+	bpf_jit_harden = '2';
+	if (set_bpf_jit_harden(&bpf_jit_harden)) {
+		bpf_jit_harden = '@'; /* open, read or write failed => no write was done */
+		goto cleanup;
+	}
+
+	prog_fd = prog_load(insns, ARRAY_SIZE(insns), &map_fd, 1);
+	if (!ASSERT_GE(prog_fd, 0, "bpf(BPF_PROG_LOAD)"))
+		goto cleanup;
+
+	for (i = 0; i < ARRAY_SIZE(insns); i++) {
+		char fmt[32];
+
+		if (!ASSERT_EQ(bpf_map_lookup_elem(map_fd, &i, &val), 0, "bpf_map_lookup_elem"))
+			goto cleanup;
+
+		snprintf(fmt, sizeof(fmt), "val should be equal 3*%d", i);
+		ASSERT_EQ(val.xlated_off, i * 3, fmt);
+	}
+
+cleanup:
+	/* restore the old one */
+	if (bpf_jit_harden != '@')
+		set_bpf_jit_harden(&bpf_jit_harden);
+
+	close(prog_fd);
+	close(map_fd);
+}
+
 /* Once map was initialized, it should be frozen */
 static void check_load_unfrozen_map(void)
 {
@@ -382,6 +474,9 @@ static void __test_bpf_insn_array(void)
 	if (test__start_subtest("deletions-with-functions"))
 		check_deletions_with_functions();
 
+	if (test__start_subtest("blindness"))
+		check_blindness();
+
 	/* Check all kinds of operations and related restrictions */
 
 	if (test__start_subtest("incorrect-index"))
Signature · 1 line
2.34.1
#9 lore raw

PATCH v10 5/11 bpf, x86: allow indirect jumps to r8...r15

In reply to Anton Protopopov

Currently the emit_indirect_jump() function only accepts one of the RAX, RCX, ..., RBP registers as the destination. Make it to accept R8, R9, ..., R15 as well, and make callers to pass BPF registers, not native registers. This is required to enable indirect jumps support in eBPF.

---
 arch/x86/net/bpf_jit_comp.c | 30 ++++++++++++++++++++++--------
 1 file changed, 22 insertions(+), 8 deletions(-)
arch/x86/net/bpf_jit_comp.c +22 -8
index 6e3574033473..e7123f0f2e66 100644
@@ -660,24 +660,38 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t,
 
 #define EMIT_LFENCE()	EMIT3(0x0F, 0xAE, 0xE8)
 
-static void emit_indirect_jump(u8 **pprog, int reg, u8 *ip)
+static void __emit_indirect_jump(u8 **pprog, int reg, bool ereg)
 {
 	u8 *prog = *pprog;
 
+	if (ereg)
+		EMIT1(0x41);
+
+	EMIT2(0xFF, 0xE0 + reg);
+
+	*pprog = prog;
+}
+
+static void emit_indirect_jump(u8 **pprog, int bpf_reg, u8 *ip)
+{
+	u8 *prog = *pprog;
+	int reg = reg2hex[bpf_reg];
+	bool ereg = is_ereg(bpf_reg);
+
 	if (cpu_feature_enabled(X86_FEATURE_INDIRECT_THUNK_ITS)) {
 		OPTIMIZER_HIDE_VAR(reg);
-		emit_jump(&prog, its_static_thunk(reg), ip);
+		emit_jump(&prog, its_static_thunk(reg + 8*ereg), ip);
 	} else if (cpu_feature_enabled(X86_FEATURE_RETPOLINE_LFENCE)) {
 		EMIT_LFENCE();
-		EMIT2(0xFF, 0xE0 + reg);
+		__emit_indirect_jump(&prog, reg, ereg);
 	} else if (cpu_feature_enabled(X86_FEATURE_RETPOLINE)) {
 		OPTIMIZER_HIDE_VAR(reg);
 		if (cpu_feature_enabled(X86_FEATURE_CALL_DEPTH))
-			emit_jump(&prog, &__x86_indirect_jump_thunk_array[reg], ip);
+			emit_jump(&prog, &__x86_indirect_jump_thunk_array[reg + 8*ereg], ip);
 		else
-			emit_jump(&prog, &__x86_indirect_thunk_array[reg], ip);
+			emit_jump(&prog, &__x86_indirect_thunk_array[reg + 8*ereg], ip);
 	} else {
-		EMIT2(0xFF, 0xE0 + reg);	/* jmp *%\reg */
+		__emit_indirect_jump(&prog, reg, ereg);
 		if (IS_ENABLED(CONFIG_MITIGATION_RETPOLINE) || IS_ENABLED(CONFIG_MITIGATION_SLS))
 			EMIT1(0xCC);		/* int3 */
 	}
@@ -797,7 +811,7 @@ static void emit_bpf_tail_call_indirect(struct bpf_prog *bpf_prog,
 	 * rdi == ctx (1st arg)
 	 * rcx == prog->bpf_func + X86_TAIL_CALL_OFFSET
 	 */
-	emit_indirect_jump(&prog, 1 /* rcx */, ip + (prog - start));
+	emit_indirect_jump(&prog, BPF_REG_4 /* R4 -> rcx */, ip + (prog - start));
 
 	/* out: */
 	ctx->tail_call_indirect_label = prog - start;
@@ -3543,7 +3557,7 @@ static int emit_bpf_dispatcher(u8 **pprog, int a, int b, s64 *progs, u8 *image,
 		if (err)
 			return err;
 
-		emit_indirect_jump(&prog, 2 /* rdx */, image + (prog - buf));
+		emit_indirect_jump(&prog, BPF_REG_3 /* R3 -> rdx */, image + (prog - buf));
 
 		*pprog = prog;
 		return 0;
Signature · 1 line
2.34.1
#10 lore raw

PATCH v10 6/11 bpf, x86: add support for indirect jumps

In reply to Anton Protopopov

Add support for a new instruction

    BPF_JMP|BPF_X|BPF_JA, SRC=0, DST=Rx, off=0, imm=0

which does an indirect jump to a location stored in Rx. The register Rx should have type PTR_TO_INSN. This new type assures that the Rx register contains a value (or a range of values) loaded from a correct jump table – map of type instruction array.

For example, for a C switch LLVM will generate the following code:

    0:   r3 = r1                    # "switch (r3)"
    1:   if r3 > 0x13 goto +0x666   # check r3 boundaries
    2:   r3 <<= 0x3                 # adjust to an index in array of addresses
    3:   r1 = 0xbeef ll             # r1 is PTR_TO_MAP_VALUE, r1->map_ptr=M
    5:   r1 += r3                   # r1 inherits boundaries from r3
    6:   r1 = *(u64 *)(r1 + 0x0)    # r1 now has type INSN_TO_PTR
    7:   gotox r1                   # jit will generate proper code

Here the gotox instruction corresponds to one particular map. This is possible however to have a gotox instruction which can be loaded from different maps, e.g.

    0:   r1 &= 0x1
    1:   r2 <<= 0x3
    2:   r3 = 0x0 ll                # load from map M_1
    4:   r3 += r2
    5:   if r1 == 0x0 goto +0x4
    6:   r1 <<= 0x3
    7:   r3 = 0x0 ll                # load from map M_2
    9:   r3 += r1
    A:   r1 = *(u64 *)(r3 + 0x0)
    B:   gotox r1                   # jump to target loaded from M_1 or M_2

During check_cfg stage the verifier will collect all the maps which point to inside the subprog being verified. When building the config, the high 16 bytes of the insn_state are used, so this patch (theoretically) supports jump tables of up to 2^16 slots.

During the later stage, in check_indirect_jump, it is checked that the register Rx was loaded from a particular instruction array.

---
 arch/x86/net/bpf_jit_comp.c  |   3 +
 include/linux/bpf.h          |   1 +
 include/linux/bpf_verifier.h |   9 +
 kernel/bpf/bpf_insn_array.c  |  15 ++
 kernel/bpf/core.c            |   1 +
 kernel/bpf/liveness.c        |   3 +
 kernel/bpf/log.c             |   1 +
 kernel/bpf/verifier.c        | 374 ++++++++++++++++++++++++++++++++++-
 8 files changed, 401 insertions(+), 6 deletions(-)
arch/x86/net/bpf_jit_comp.c +3 -0
index e7123f0f2e66..c4d076b26397 100644
@@ -2628,6 +2628,9 @@ st:			if (is_imm8(insn->off))
 
 			break;
 
+		case BPF_JMP | BPF_JA | BPF_X:
+			emit_indirect_jump(&prog, insn->dst_reg, image + addrs[i - 1]);
+			break;
 		case BPF_JMP | BPF_JA:
 		case BPF_JMP32 | BPF_JA:
 			if (BPF_CLASS(insn->code) == BPF_JMP) {
include/linux/bpf.h +1 -0
index 9d41a6affcef..09d5dc541d1c 100644
@@ -1001,6 +1001,7 @@ enum bpf_reg_type {
 	PTR_TO_ARENA,
 	PTR_TO_BUF,		 /* reg points to a read/write buffer */
 	PTR_TO_FUNC,		 /* reg points to a bpf program function */
+	PTR_TO_INSN,		 /* reg points to a bpf program instruction */
 	CONST_PTR_TO_DYNPTR,	 /* reg points to a const struct bpf_dynptr */
 	__BPF_REG_TYPE_MAX,
 
include/linux/bpf_verifier.h +9 -0
index 6b820d8d77af..5441341f1ab9 100644
@@ -527,6 +527,7 @@ struct bpf_insn_aux_data {
 		struct {
 			u32 map_index;		/* index into used_maps[] */
 			u32 map_off;		/* offset from value base address */
+			struct bpf_iarray *jt;	/* jump table for gotox instruction */
 		};
 		struct {
 			enum bpf_reg_type reg_type;	/* type of pseudo_btf_id */
@@ -840,6 +841,7 @@ struct bpf_verifier_env {
 	struct bpf_scc_info **scc_info;
 	u32 scc_cnt;
 	struct bpf_iarray *succ;
+	struct bpf_iarray *gotox_tmp_buf;
 };
 
 static inline struct bpf_func_info_aux *subprog_aux(struct bpf_verifier_env *env, int subprog)
@@ -1050,6 +1052,13 @@ static inline bool bpf_stack_narrow_access_ok(int off, int fill_size, int spill_
 	return !(off % BPF_REG_SIZE);
 }
 
+static inline bool insn_is_gotox(struct bpf_insn *insn)
+{
+	return BPF_CLASS(insn->code) == BPF_JMP &&
+	       BPF_OP(insn->code) == BPF_JA &&
+	       BPF_SRC(insn->code) == BPF_X;
+}
+
 const char *reg_type_str(struct bpf_verifier_env *env, enum bpf_reg_type type);
 const char *dynptr_type_str(enum bpf_dynptr_type type);
 const char *iter_type_str(const struct btf *btf, u32 btf_id);
kernel/bpf/bpf_insn_array.c +15 -0
index 2053fda377bb..61ce52882632 100644
@@ -114,6 +114,20 @@ static u64 insn_array_mem_usage(const struct bpf_map *map)
 	return insn_array_alloc_size(map->max_entries);
 }
 
+static int insn_array_map_direct_value_addr(const struct bpf_map *map, u64 *imm, u32 off)
+{
+	struct bpf_insn_array *insn_array = cast_insn_array(map);
+
+	if ((off % sizeof(long)) != 0 ||
+	    (off / sizeof(long)) >= map->max_entries)
+		return -EINVAL;
+
+	/* from BPF's point of view, this map is a jump table */
+	*imm = (unsigned long)insn_array->ips + off;
+
+	return 0;
+}
+
 BTF_ID_LIST_SINGLE(insn_array_btf_ids, struct, bpf_insn_array)
 
 const struct bpf_map_ops insn_array_map_ops = {
@@ -126,6 +140,7 @@ const struct bpf_map_ops insn_array_map_ops = {
 	.map_delete_elem = insn_array_delete_elem,
 	.map_check_btf = insn_array_check_btf,
 	.map_mem_usage = insn_array_mem_usage,
+	.map_direct_value_addr = insn_array_map_direct_value_addr,
 	.map_btf_id = &insn_array_btf_ids[0],
 };
 
kernel/bpf/core.c +1 -0
index 4b62a03d6df5..ef4448f18aad 100644
@@ -1708,6 +1708,7 @@ bool bpf_opcode_in_insntable(u8 code)
 		[BPF_LD | BPF_IND | BPF_B] = true,
 		[BPF_LD | BPF_IND | BPF_H] = true,
 		[BPF_LD | BPF_IND | BPF_W] = true,
+		[BPF_JMP | BPF_JA | BPF_X] = true,
 		[BPF_JMP | BPF_JCOND] = true,
 	};
 #undef BPF_INSN_3_TBL
kernel/bpf/liveness.c +3 -0
index bffb495bc933..a7240013fd9d 100644
@@ -485,6 +485,9 @@ bpf_insn_successors(struct bpf_verifier_env *env, u32 idx)
 	struct bpf_iarray *succ;
 	int insn_sz;
 
+	if (unlikely(insn_is_gotox(insn)))
+		return env->insn_aux_data[idx].jt;
+
 	/* pre-allocated array of size up to 2; reset cnt, as it may have been used already */
 	succ = env->succ;
 	succ->cnt = 0;
kernel/bpf/log.c +1 -0
index 70221aafc35c..a0c3b35de2ce 100644
@@ -461,6 +461,7 @@ const char *reg_type_str(struct bpf_verifier_env *env, enum bpf_reg_type type)
 		[PTR_TO_ARENA]		= "arena",
 		[PTR_TO_BUF]		= "buf",
 		[PTR_TO_FUNC]		= "func",
+		[PTR_TO_INSN]		= "insn",
 		[PTR_TO_MAP_KEY]	= "map_key",
 		[CONST_PTR_TO_DYNPTR]	= "dynptr_ptr",
 	};
kernel/bpf/verifier.c +368 -6

485 lines not on this page · raw message

Signature · 1 line
2.34.1
#11 lore raw

Re: [PATCH v10 bpf-next 06/11] bpf, x86: add support for indirect jumps

AI reviewed your patch. Please fix the bug or email reply why it's not a bug. See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
```
commit 6cb8001495faae09541d55309638b3c1920bae06
Author: Anton Protopopov <[email protected]>

bpf, x86: add support for indirect jumps

This patch adds support for indirect jumps in BPF through a new
BPF_JMP|BPF_X|BPF_JA instruction. It introduces the PTR_TO_INSN register
type for instruction pointers loaded from INSN_ARRAY maps, along with
verifier support for tracking jump tables and x86 JIT implementation.

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c

[ ... ]

> @@ -21374,6 +21374,28 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
>  	return 0;
>  }
>
> +/*
> + * Clean up dynamically allocated fields of aux data for instructions [start, ...]
> + */
> +static void clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
> +{
> +	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
> +	struct bpf_insn *insns = env->prog->insnsi;
> +	int end = start + len;
> +	int i;
> +
> +	for (i = start; i < end; i++) {
> +		if (insn_is_gotox(&insns[i])) {
> +			kvfree(aux_data[i].jt);
> +			aux_data[i].jt = NULL;
> +		}
> +
> +		if (bpf_is_ldimm64(&insns[i]))
> +			i++;
> +	}
> +}
> +
>  static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
>  {
>  	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
> @@ -21393,6 +21415,8 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt
>
>  	adjust_insn_arrays_after_remove(env, off, cnt);
>
> +	clear_insn_aux_data(env, off, cnt);
> +
>  	memmove(aux_data + off,	aux_data + off + cnt,
>  		sizeof(*aux_data) * (orig_prog_len - off - cnt));

Can clear_insn_aux_data() correctly identify which aux_data entries need
cleanup when it's called after bpf_remove_insns()? After instructions are
removed, the insns[] array has been modified - insns[off] now contains what
was previously insns[off+cnt]. But aux_data hasn't been shifted yet, so
clear_insn_aux_data() checks the wrong instructions when deciding which
aux_data entries to free.

For example, if instructions [1,2] are removed where insn 2 is a gotox:
- bpf_remove_insns() removes insns[1] and insns[2], shifts later insns down
- insns[1] now contains what was insns[3]
- clear_insn_aux_data() checks insn_is_gotox(&insns[1]) (now insn 3!)
- It might free aux_data[1].jt if insn 3 is a gotox, even though insn 3
  isn't being removed
- It never checks the original insns[2] which was the actual gotox that
  needed cleanup

Should clear_insn_aux_data() be called before bpf_remove_insns() instead?


```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: bpf, x86: add support for indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

#12 lore raw

Re: [PATCH v10 bpf-next 06/11] bpf, x86: add support for indirect jumps

Quoted 69 lines · On 25/11/02 09:20PM, [email protected] wrote:

```
commit 6cb8001495faae09541d55309638b3c1920bae06
Author: Anton Protopopov <[email protected]>

bpf, x86: add support for indirect jumps

This patch adds support for indirect jumps in BPF through a new BPF_JMP|BPF_X|BPF_JA instruction. It introduces the PTR_TO_INSN register type for instruction pointers loaded from INSN_ARRAY maps, along with verifier support for tracking jump tables and x86 JIT implementation.

Quoted 1 line
kernel/bpf/verifier.c +0 -0

[ ... ]

Quoted 29 lines
kernel/bpf/verifier.c +21 -0
@@ -21374,6 +21374,28 @@ static int bpf_adj_linfo_after_remove(struct bpf_verifier_env *env, u32 off,
 	return 0;
 }
 
+/*
+ * Clean up dynamically allocated fields of aux data for instructions [start, ...]
+ */
+static void clear_insn_aux_data(struct bpf_verifier_env *env, int start, int len)
+{
+	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;
+	struct bpf_insn *insns = env->prog->insnsi;
+	int end = start + len;
+	int i;
+
+	for (i = start; i < end; i++) {
+		if (insn_is_gotox(&insns[i])) {
+			kvfree(aux_data[i].jt);
+			aux_data[i].jt = NULL;
+		}
+
+		if (bpf_is_ldimm64(&insns[i]))
+			i++;
+	}
+}
+
 static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt)
 {
 	struct bpf_insn_aux_data *aux_data = env->insn_aux_data;

kernel/bpf/verifier.c static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt · line 21421

Quoted 7 lines
kernel/bpf/verifier.c +2 -0
@@ -21393,6 +21415,8 @@ static int verifier_remove_insns(struct bpf_verifier_env *env, u32 off, u32 cnt
 
 	adjust_insn_arrays_after_remove(env, off, cnt);
 
+	clear_insn_aux_data(env, off, cnt);
+
 	memmove(aux_data + off,	aux_data + off + cnt,
 		sizeof(*aux_data) * (orig_prog_len - off - cnt));

Can clear_insn_aux_data() correctly identify which aux_data entries need cleanup when it's called after bpf_remove_insns()? After instructions are removed, the insns[] array has been modified - insns[off] now contains what was previously insns[off+cnt]. But aux_data hasn't been shifted yet, so clear_insn_aux_data() checks the wrong instructions when deciding which aux_data entries to free.

For example, if instructions [1,2] are removed where insn 2 is a gotox:
- bpf_remove_insns() removes insns[1] and insns[2], shifts later insns down
- insns[1] now contains what was insns[3]
- clear_insn_aux_data() checks insn_is_gotox(&insns[1]) (now insn 3!)
- It might free aux_data[1].jt if insn 3 is a gotox, even though insn 3 isn't being removed
- It never checks the original insns[2] which was the actual gotox that needed cleanup

Quoted 1 line

Should clear_insn_aux_data() be called before bpf_remove_insns() instead?

Looks plausible. I will move it upper and add a comment on the call order.

Quoted 9 lines

```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: bpf, x86: add support for indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

#13 lore raw

PATCH v10 7/11 bpf: disasm: add support for BPF_JMP|BPF_JA|BPF_X

In reply to Anton Protopopov

Add support for indirect jump instruction.

Example output from bpftool:

0: (79) r3 = *(u64 *)(r1 +0)
1: (25) if r3 > 0x4 goto pc+666
2: (67) r3 <<= 3
3: (18) r1 = 0xffffbeefspameggs
5: (0f) r1 += r3
6: (79) r1 = *(u64 *)(r1 +0)
7: (0d) gotox r1

---
 kernel/bpf/disasm.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/bpf/disasm.c +3 -0
index 20883c6b1546..f8a3c7eb451e 100644
@@ -358,6 +358,9 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,
 		} else if (insn->code == (BPF_JMP | BPF_JA)) {
 			verbose(cbs->private_data, "(%02x) goto pc%+d\n",
 				insn->code, insn->off);
+		} else if (insn->code == (BPF_JMP | BPF_JA | BPF_X)) {
+			verbose(cbs->private_data, "(%02x) gotox r%d\n",
+				insn->code, insn->dst_reg);
 		} else if (insn->code == (BPF_JMP | BPF_JCOND) &&
 			   insn->src_reg == BPF_MAY_GOTO) {
 			verbose(cbs->private_data, "(%02x) may_goto pc%+d\n",
Signature · 1 line
2.34.1
#14 lore raw

PATCH v10 8/11 libbpf: support llvm-generated indirect jumps

In reply to Anton Protopopov

For v4 instruction set LLVM is allowed to generate indirect jumps for switch statements and for 'goto *rX' assembly. Every such a jump will be accompanied by necessary metadata, e.g. (llvm-objdump -Sr ...):

       0:       r2 = 0x0 ll
                0000000000000030:  R_BPF_64_64  BPF.JT.0.0

Here BPF.JT.1.0 is a symbol residing in the .jumptables section:

    Symbol table:
       4: 0000000000000000   240 OBJECT  GLOBAL DEFAULT     4 BPF.JT.0.0

The -bpf-min-jump-table-entries llvm option may be used to control the minimal size of a switch which will be converted to an indirect jumps.

---
 tools/lib/bpf/libbpf.c          | 248 +++++++++++++++++++++++++++++++-
 tools/lib/bpf/libbpf_internal.h |   2 +
 tools/lib/bpf/libbpf_probes.c   |   4 +
 tools/lib/bpf/linker.c          |   9 +-
 4 files changed, 260 insertions(+), 3 deletions(-)
tools/lib/bpf/libbpf.c +247 -1
index fbe74686c97d..ed14090a9e13 100644
@@ -190,6 +190,7 @@ static const char * const map_type_name[] = {
 	[BPF_MAP_TYPE_USER_RINGBUF]             = "user_ringbuf",
 	[BPF_MAP_TYPE_CGRP_STORAGE]		= "cgrp_storage",
 	[BPF_MAP_TYPE_ARENA]			= "arena",
+	[BPF_MAP_TYPE_INSN_ARRAY]		= "insn_array",
 };
 
 static const char * const prog_type_name[] = {
@@ -369,6 +370,7 @@ enum reloc_type {
 	RELO_EXTERN_CALL,
 	RELO_SUBPROG_ADDR,
 	RELO_CORE,
+	RELO_INSN_ARRAY,
 };
 
 struct reloc_desc {
@@ -379,7 +381,16 @@ struct reloc_desc {
 		struct {
 			int map_idx;
 			int sym_off;
-			int ext_idx;
+			/*
+			 * The following two fields can be unionized, as the
+			 * ext_idx field is used for extern symbols, and the
+			 * sym_size is used for jump tables, which are never
+			 * extern
+			 */
+			union {
+				int ext_idx;
+				int sym_size;
+			};
 		};
 	};
 };
@@ -421,6 +432,11 @@ struct bpf_sec_def {
 	libbpf_prog_attach_fn_t prog_attach_fn;
 };
 
+struct bpf_light_subprog {
+	__u32 sec_insn_off;
+	__u32 sub_insn_off;
+};
+
 /*
  * bpf_prog should be a better name but it has been used in
  * linux/filter.h.
@@ -494,6 +510,9 @@ struct bpf_program {
 	__u32 line_info_cnt;
 	__u32 prog_flags;
 	__u8  hash[SHA256_DIGEST_LENGTH];
+
+	struct bpf_light_subprog *subprogs;
+	__u32 subprog_cnt;
 };
 
 struct bpf_struct_ops {
@@ -667,6 +686,7 @@ struct elf_state {
 	int symbols_shndx;
 	bool has_st_ops;
 	int arena_data_shndx;
+	int jumptables_data_shndx;
 };
 
 struct usdt_manager;
@@ -738,6 +758,16 @@ struct bpf_object {
 	void *arena_data;
 	size_t arena_data_sz;
 
+	void *jumptables_data;
+	size_t jumptables_data_sz;
+
+	struct {
+		struct bpf_program *prog;
+		int sym_off;
+		int fd;
+	} *jumptable_maps;
+	size_t jumptable_map_cnt;
+
 	struct kern_feature_cache *feat_cache;
 	char *token_path;
 	int token_fd;
@@ -764,6 +794,7 @@ void bpf_program__unload(struct bpf_program *prog)
 
 	zfree(&prog->func_info);
 	zfree(&prog->line_info);
+	zfree(&prog->subprogs);
 }
 
 static void bpf_program__exit(struct bpf_program *prog)
@@ -3942,6 +3973,13 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
 			} else if (strcmp(name, ARENA_SEC) == 0) {
 				obj->efile.arena_data = data;
 				obj->efile.arena_data_shndx = idx;
+			} else if (strcmp(name, JUMPTABLES_SEC) == 0) {
+				obj->jumptables_data = malloc(data->d_size);
+				if (!obj->jumptables_data)
+					return -ENOMEM;
+				memcpy(obj->jumptables_data, data->d_buf, data->d_size);
+				obj->jumptables_data_sz = data->d_size;
+				obj->efile.jumptables_data_shndx = idx;
 			} else {
 				pr_info("elf: skipping unrecognized data section(%d) %s\n",
 					idx, name);
@@ -4634,6 +4672,16 @@ static int bpf_program__record_reloc(struct bpf_program *prog,
 		return 0;
 	}
 
+	/* jump table data relocation */
+	if (shdr_idx == obj->efile.jumptables_data_shndx) {
+		reloc_desc->type = RELO_INSN_ARRAY;
+		reloc_desc->insn_idx = insn_idx;
+		reloc_desc->map_idx = -1;
+		reloc_desc->sym_off = sym->st_value;
+		reloc_desc->sym_size = sym->st_size;
+		return 0;
+	}
+
 	/* generic map reference relocation */
 	if (type == LIBBPF_MAP_UNSPEC) {
 		if (!bpf_object__shndx_is_maps(obj, shdr_idx)) {
@@ -6144,6 +6192,157 @@ static void poison_kfunc_call(struct bpf_program *prog, int relo_idx,
 	insn->imm = POISON_CALL_KFUNC_BASE + ext_idx;
 }
 
+static int find_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off)
+{
+	size_t i;
+
+	for (i = 0; i < obj->jumptable_map_cnt; i++) {
+		/*
+		 * This might happen that same offset is used for two different
+		 * programs (as jump tables can be the same). However, for
+		 * different programs different maps should be created.
+		 */
+		if (obj->jumptable_maps[i].sym_off == sym_off &&
+		    obj->jumptable_maps[i].prog == prog)
+			return obj->jumptable_maps[i].fd;
+	}
+
+	return -ENOENT;
+}
+
+static int add_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off, int map_fd)
+{
+	size_t cnt = obj->jumptable_map_cnt;
+	size_t size = sizeof(obj->jumptable_maps[0]);
+	void *tmp;
+
+	tmp = libbpf_reallocarray(obj->jumptable_maps, cnt + 1, size);
+	if (!tmp)
+		return -ENOMEM;
+
+	obj->jumptable_maps = tmp;
+	obj->jumptable_maps[cnt].prog = prog;
+	obj->jumptable_maps[cnt].sym_off = sym_off;
+	obj->jumptable_maps[cnt].fd = map_fd;
+	obj->jumptable_map_cnt++;
+
+	return 0;
+}
+
+static int find_subprog_idx(struct bpf_program *prog, int insn_idx)
+{
+	int i;
+
+	for (i = prog->subprog_cnt - 1; i >= 0; i--) {
+		if (insn_idx >= prog->subprogs[i].sub_insn_off)
+			return i;
+	}
+
+	return -1;
+}
+
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;
+		} else {
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}
+
+	err = bpf_map_freeze(map_fd);
+	if (err)
+		goto err_close;
+
+	err = add_jt_map(obj, prog, sym_off, map_fd);
+	if (err)
+		goto err_close;
+
+	return map_fd;
+
+err_close:
+	close(map_fd);
+	return err;
+}
+
 /* Relocate data references within program code:
  *  - map references;
  *  - global variable references;
@@ -6235,6 +6434,20 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog)
 		case RELO_CORE:
 			/* will be handled by bpf_program_record_relos() */
 			break;
+		case RELO_INSN_ARRAY: {
+			int map_fd;
+
+			map_fd = create_jt_map(obj, prog, relo);
+			if (map_fd < 0) {
+				pr_warn("prog '%s': relo #%d: can't create jump table: sym_off %u\n",
+					prog->name, i, relo->sym_off);
+				return map_fd;
+			}
+			insn[0].src_reg = BPF_PSEUDO_MAP_VALUE;
+			insn->imm = map_fd;
+			insn->off = 0;
+		}
+			break;
 		default:
 			pr_warn("prog '%s': relo #%d: bad relo type %d\n",
 				prog->name, i, relo->type);
@@ -6432,6 +6645,24 @@ static int append_subprog_relos(struct bpf_program *main_prog, struct bpf_progra
 	return 0;
 }
 
+static int save_subprog_offsets(struct bpf_program *main_prog, struct bpf_program *subprog)
+{
+	size_t size = sizeof(main_prog->subprogs[0]);
+	int cnt = main_prog->subprog_cnt;
+	void *tmp;
+
+	tmp = libbpf_reallocarray(main_prog->subprogs, cnt + 1, size);
+	if (!tmp)
+		return -ENOMEM;
+
+	main_prog->subprogs = tmp;
+	main_prog->subprogs[cnt].sec_insn_off = subprog->sec_insn_off;
+	main_prog->subprogs[cnt].sub_insn_off = subprog->sub_insn_off;
+	main_prog->subprog_cnt++;
+
+	return 0;
+}
+
 static int
 bpf_object__append_subprog_code(struct bpf_object *obj, struct bpf_program *main_prog,
 				struct bpf_program *subprog)
@@ -6461,6 +6692,14 @@ bpf_object__append_subprog_code(struct bpf_object *obj, struct bpf_program *main
 	err = append_subprog_relos(main_prog, subprog);
 	if (err)
 		return err;
+
+	err = save_subprog_offsets(main_prog, subprog);
+	if (err) {
+		pr_warn("prog '%s': failed to add subprog offsets: %s\n",
+			main_prog->name, errstr(err));
+		return err;
+	}
+
 	return 0;
 }
 
@@ -9228,6 +9467,13 @@ void bpf_object__close(struct bpf_object *obj)
 
 	zfree(&obj->arena_data);
 
+	zfree(&obj->jumptables_data);
+	obj->jumptables_data_sz = 0;
+
+	for (i = 0; i < obj->jumptable_map_cnt; i++)
+		close(obj->jumptable_maps[i].fd);
+	zfree(&obj->jumptable_maps);
+
 	free(obj);
 }
 
tools/lib/bpf/libbpf_internal.h +2 -0
index 35b2527bedec..fc59b21b51b5 100644
@@ -74,6 +74,8 @@
 #define ELF64_ST_VISIBILITY(o) ((o) & 0x03)
 #endif
 
+#define JUMPTABLES_SEC ".jumptables"
+
 #define BTF_INFO_ENC(kind, kind_flag, vlen) \
 	((!!(kind_flag) << 31) | ((kind) << 24) | ((vlen) & BTF_MAX_VLEN))
 #define BTF_TYPE_ENC(name, info, size_or_type) (name), (info), (size_or_type)
tools/lib/bpf/libbpf_probes.c +4 -0
index 9dfbe7750f56..bccf4bb747e1 100644
@@ -364,6 +364,10 @@ static int probe_map_create(enum bpf_map_type map_type)
 	case BPF_MAP_TYPE_SOCKHASH:
 	case BPF_MAP_TYPE_REUSEPORT_SOCKARRAY:
 		break;
+	case BPF_MAP_TYPE_INSN_ARRAY:
+		key_size	= sizeof(__u32);
+		value_size	= sizeof(struct bpf_insn_array_value);
+		break;
 	case BPF_MAP_TYPE_UNSPEC:
 	default:
 		return -EOPNOTSUPP;
tools/lib/bpf/linker.c +7 -2
index 56ae77047bc3..f6ca3b23b17a 100644
@@ -27,6 +27,7 @@
 #include "strset.h"
 
 #define BTF_EXTERN_SEC ".extern"
+#define JUMPTABLES_REL_SEC ".rel.jumptables"
 
 struct src_sec {
 	const char *sec_name;
@@ -2025,6 +2026,9 @@ static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
 			obj->sym_map[src_sym_idx] = dst_sec->sec_sym_idx;
 			return 0;
 		}
+
+		if (strcmp(src_sec->sec_name, JUMPTABLES_SEC) == 0)
+			goto add_sym;
 	}
 
 	if (sym_bind == STB_LOCAL)
@@ -2271,8 +2275,9 @@ static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *ob
 						insn->imm += sec->dst_off / sizeof(struct bpf_insn);
 					else
 						insn->imm += sec->dst_off;
-				} else {
-					pr_warn("relocation against STT_SECTION in non-exec section is not supported!\n");
+				} else if (strcmp(src_sec->sec_name, JUMPTABLES_REL_SEC) != 0) {
+					pr_warn("relocation against STT_SECTION in section %s is not supported!\n",
+						src_sec->sec_name);
 					return -EINVAL;
 				}
 			}
Signature · 1 line
2.34.1
#15 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

AI reviewed your patch. Please fix the bug or email reply why it's not a bug. See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
```
commit b44690c09995d34f7665c7d687b359d41a6ab79f
Author: Anton Protopopov <[email protected]>

libbpf: support llvm-generated indirect jumps

This commit adds support for LLVM-generated indirect jumps in BPF v4
instruction set. It handles jump table metadata from the .jumptables
ELF section and creates BPF_MAP_TYPE_INSN_ARRAY maps for them.

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index fbe74686c..ed14090a9 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c

[ ... ]

> @@ -6144,6 +6192,157 @@ static void poison_kfunc_call(struct bpf_program *prog, int relo_idx,
>  	insn->imm = POISON_CALL_KFUNC_BASE + ext_idx;
>  }
>
> +static int find_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off)
> +{

[ ... ]

> +static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
> +{
> +	const __u32 jt_entry_size = 8;
> +	int sym_off = relo->sym_off;
> +	int jt_size = relo->sym_size;
> +	__u32 max_entries = jt_size / jt_entry_size;
> +	__u32 value_size = sizeof(struct bpf_insn_array_value);
> +	struct bpf_insn_array_value val = {};
> +	int subprog_idx;
> +	int map_fd, err;
> +	__u64 insn_off;
> +	__u64 *jt;
> +	__u32 i;
> +
> +	map_fd = find_jt_map(obj, prog, sym_off);
> +	if (map_fd >= 0)
> +		return map_fd;
> +
> +	if (sym_off % jt_entry_size) {
> +		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
> +			sym_off, jt_entry_size);
> +		return -EINVAL;
> +	}
> +
> +	if (jt_size % jt_entry_size) {
> +		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
> +			jt_size, jt_entry_size);
> +		return -EINVAL;
> +	}
> +
> +	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
> +				4, value_size, max_entries, NULL);
> +	if (map_fd < 0)
> +		return map_fd;
> +
> +	if (!obj->jumptables_data) {
> +		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
> +		err = -EINVAL;
> +		goto err_close;
> +	}
> +	if (sym_off + jt_size > obj->jumptables_data_sz) {
> +		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
> +			obj->jumptables_data_sz, sym_off + jt_size);
> +		err = -EINVAL;
> +		goto err_close;
> +	}
> +
> +	subprog_idx = -1; /* main program */
> +	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
> +		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
> +		err = -EINVAL;
> +		goto err_close;
> +	}
> +	if (prog->subprogs)
> +		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
> +
> +	jt = (__u64 *)(obj->jumptables_data + sym_off);
> +	for (i = 0; i < max_entries; i++) {
> +		/*
> +		 * The offset should be made to be relative to the beginning of
> +		 * the main function, not the subfunction.
> +		 */
> +		insn_off = jt[i]/sizeof(struct bpf_insn);
> +		if (subprog_idx >= 0) {
> +			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
> +			insn_off += prog->subprogs[subprog_idx].sub_insn_off;
> +		} else {
> +			insn_off -= prog->sec_insn_off;
> +		}
> +
> +		/*
> +		 * LLVM-generated jump tables contain u64 records, however
> +		 * should contain values that fit in u32.
> +		 */
> +		if (insn_off > UINT32_MAX) {
> +			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
> +				(long long)jt[i], sym_off + i);
                                                      ^^^^^^^^^^^^

Does this error message print the correct offset? Since jt is a pointer
to __u64, the array access jt[i] is at byte offset "sym_off + i * 8",
not "sym_off + i". All the other error messages in create_jt_map report
byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this
one should probably be "sym_off + i * jt_entry_size" for consistency.

> +			err = -EINVAL;
> +			goto err_close;
> +		}
> +
> +		val.orig_off = insn_off;
> +		err = bpf_map_update_elem(map_fd, &i, &val, 0);
> +		if (err)
> +			goto err_close;
> +	}
> +
> +	err = bpf_map_freeze(map_fd);
> +	if (err)
> +		goto err_close;
> +
> +	err = add_jt_map(obj, prog, sym_off, map_fd);
> +	if (err)
> +		goto err_close;
> +
> +	return map_fd;
> +
> +err_close:
> +	close(map_fd);
> +	return err;
> +}

[ ... ]


```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: libbpf: support llvm-generated indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

#16 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

Quoted 106 lines · On 25/11/02 09:13PM, [email protected] wrote:

```
commit b44690c09995d34f7665c7d687b359d41a6ab79f
Author: Anton Protopopov <[email protected]>

libbpf: support llvm-generated indirect jumps

This commit adds support for LLVM-generated indirect jumps in BPF v4 instruction set. It handles jump table metadata from the .jumptables ELF section and creates BPF_MAP_TYPE_INSN_ARRAY maps for them.

Quoted 4 lines
tools/lib/bpf/libbpf.c +0 -0
index fbe74686c..ed14090a9 100644

[ ... ]

Quoted 6 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -6144,6 +6192,157 @@ static void poison_kfunc_call(struct bpf_program *prog, int relo_idx,
 	insn->imm = POISON_CALL_KFUNC_BASE + ext_idx;
 }
 
+static int find_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off)
+{

[ ... ]

Quoted 67 lines
tools/lib/bpf/libbpf.c +67 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;
+		} else {

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6320

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
                                                      ^^^^^^^^^^^^
Quoted 5 lines

Does this error message print the correct offset? Since jt is a pointer to __u64, the array access jt[i] is at byte offset "sym_off + i * 8", not "sym_off + i". All the other error messages in create_jt_map report byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this one should probably be "sym_off + i * jt_entry_size" for consistency.

Yes, you're right. This can be fixed as a follow up or in a respin.

Quoted 36 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14 -0
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}
+
+	err = bpf_map_freeze(map_fd);
+	if (err)
+		goto err_close;
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6344

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	err = add_jt_map(obj, prog, sym_off, map_fd);
+	if (err)
+		goto err_close;
+
+	return map_fd;
+
+err_close:
+	close(map_fd);
+	return err;
+}

[ ... ]

```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: libbpf: support llvm-generated indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

#17 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

In reply to [email protected]
Quoted 106 lines · On 25/11/02 09:13PM, [email protected] wrote:

```
commit b44690c09995d34f7665c7d687b359d41a6ab79f
Author: Anton Protopopov <[email protected]>

libbpf: support llvm-generated indirect jumps

This commit adds support for LLVM-generated indirect jumps in BPF v4 instruction set. It handles jump table metadata from the .jumptables ELF section and creates BPF_MAP_TYPE_INSN_ARRAY maps for them.

Quoted 4 lines
tools/lib/bpf/libbpf.c +0 -0
index fbe74686c..ed14090a9 100644

[ ... ]

Quoted 6 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -6144,6 +6192,157 @@ static void poison_kfunc_call(struct bpf_program *prog, int relo_idx,
 	insn->imm = POISON_CALL_KFUNC_BASE + ext_idx;
 }
 
+static int find_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off)
+{

[ ... ]

Quoted 67 lines
tools/lib/bpf/libbpf.c +67 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;
+		} else {

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6320

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
                                                      ^^^^^^^^^^^^
Quoted 5 lines

Does this error message print the correct offset? Since jt is a pointer to __u64, the array access jt[i] is at byte offset "sym_off + i * 8", not "sym_off + i". All the other error messages in create_jt_map report byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this one should probably be "sym_off + i * jt_entry_size" for consistency.

Is there a way to run this AI as part of any PR to kernel-patches/bpf, not only those coming from the mailing list? Maybe for a selected commit?

Also, how deterministinc it is? Will it generate different comments for a given patch for different runs?

Quoted 36 lines
Quoted 14 lines
tools/lib/bpf/libbpf.c +14 -0
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}
+
+	err = bpf_map_freeze(map_fd);
+	if (err)
+		goto err_close;
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6344

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	err = add_jt_map(obj, prog, sym_off, map_fd);
+	if (err)
+		goto err_close;
+
+	return map_fd;
+
+err_close:
+	close(map_fd);
+	return err;
+}

[ ... ]

```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: libbpf: support llvm-generated indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

reviewer
#18 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

Quoted 113 lines · Anton Protopopov
Quoted 106 lines · On 25/11/02 09:13PM, [email protected] wrote:

```
commit b44690c09995d34f7665c7d687b359d41a6ab79f
Author: Anton Protopopov <[email protected]>

libbpf: support llvm-generated indirect jumps

This commit adds support for LLVM-generated indirect jumps in BPF v4 instruction set. It handles jump table metadata from the .jumptables ELF section and creates BPF_MAP_TYPE_INSN_ARRAY maps for them.

Quoted 4 lines
tools/lib/bpf/libbpf.c +0 -0
index fbe74686c..ed14090a9 100644

[ ... ]

Quoted 6 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -6144,6 +6192,157 @@ static void poison_kfunc_call(struct bpf_program *prog, int relo_idx,
 	insn->imm = POISON_CALL_KFUNC_BASE + ext_idx;
 }
 
+static int find_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off)
+{

[ ... ]

Quoted 67 lines
tools/lib/bpf/libbpf.c +67 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;
+		} else {

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6320

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
                                                      ^^^^^^^^^^^^
Quoted 5 lines

Does this error message print the correct offset? Since jt is a pointer to __u64, the array access jt[i] is at byte offset "sym_off + i * 8", not "sym_off + i". All the other error messages in create_jt_map report byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this one should probably be "sym_off + i * jt_entry_size" for consistency.

Quoted 3 lines

Is there a way to run this AI as part of any PR to kernel-patches/bpf, not only those coming from the mailing list? Maybe for a selected commit?

Hi Anton,

If you have access to an "agentic" AI coding tool that runs locally, such as Claude Code, you can use our prompts repository [1] with a trigger prompt like this:

Current directory is the root of a Linux Kernel git repository. Using the prompt review/review-core.md and the prompt directory review do a code review of the top commit in the Linux repository.

The prompts expect the "agent" to be able to read and write files, and execute basic commands such as grep, find, awk and similar.

In principle it's possible to enable the review CI job for arbitrary pull requests, but the tokens are not free so we haven't considered that yet.

[1] https://github.com/masoncl/review-prompts

Quoted 3 lines

Also, how deterministinc it is? Will it generate different comments for a given patch for different runs?

The short answer is no, the answers are not deterministic.

However for typical/obvious bugs you might often get a comment about the same issue worded differently.

Quoted 38 lines
Quoted 37 lines
Quoted 15 lines
tools/lib/bpf/libbpf.c +14 -0
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}
+
+	err = bpf_map_freeze(map_fd);
+	if (err)
+		goto err_close;
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6344

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	err = add_jt_map(obj, prog, sym_off, map_fd);
+	if (err)
+		goto err_close;
+
+	return map_fd;
+
+err_close:
+	close(map_fd);
+	return err;
+}

[ ... ]

```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: libbpf: support llvm-generated indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

#19 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

Quoted 4 lines · Ihor Solodrai
Quoted 1 line · Anton Protopopov

On 25/11/02 09:13PM, [email protected] wrote:

                                                     ^^^^^^^^^^^^
Quoted 21 lines
Quoted 10 lines
Quoted 6 lines

Does this error message print the correct offset? Since jt is a pointer to __u64, the array access jt[i] is at byte offset "sym_off + i * 8", not "sym_off + i". All the other error messages in create_jt_map report byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this one should probably be "sym_off + i * jt_entry_size" for consistency.

Is there a way to run this AI as part of any PR to kernel-patches/bpf, not only those coming from the mailing list? Maybe for a selected commit?

Hi Anton,

If you have access to an "agentic" AI coding tool that runs locally, such as Claude Code, you can use our prompts repository [1] with a trigger prompt like this:

Current directory is the root of a Linux Kernel git repository. Using the prompt review/review-core.md and the prompt directory review do a code review of the top commit in the Linux repository.

Quoted 6 lines

The prompts expect the "agent" to be able to read and write files, and execute basic commands such as grep, find, awk and similar.

In principle it's possible to enable the review CI job for arbitrary pull requests, but the tokens are not free so we haven't considered that yet.

At least for me, it really helps having the reviews on the list. It gives me the chance to see what kinds of bugs AI is flagging correctly and where the false positives are. I do try and fix all the bad reviews that people flag, so the comments here are really helpful.

This isn't meant to discourage people from running reviews locally, I'm happy to help get you setup. But I also don't want to add a barrier to contributing code.

Quoted 11 lines

[1] https://github.com/masoncl/review-prompts

Quoted 3 lines

Also, how deterministinc it is? Will it generate different comments for a given patch for different runs?

The short answer is no, the answers are not deterministic.

However for typical/obvious bugs you might often get a comment about the same issue worded differently.

Yeah, recent changes to the prompts have made it better, but for some bugs it still wanders off without flagging a percentage of the time.

Also, sometimes it'll get excited about finding a bug (even if a false positive) and skip to the end of the review, so if a patch has multiple problems, we might need multiple submissions to see them. I've been working on this as well, but there's still room for improvement.

-chris

#20 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

Quoted 34 lines · Chris Mason
Quoted 4 lines · Ihor Solodrai
Quoted 1 line · Anton Protopopov

On 25/11/02 09:13PM, [email protected] wrote:

                                                     ^^^^^^^^^^^^
Quoted 21 lines
Quoted 10 lines
Quoted 6 lines

Does this error message print the correct offset? Since jt is a pointer to __u64, the array access jt[i] is at byte offset "sym_off + i * 8", not "sym_off + i". All the other error messages in create_jt_map report byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this one should probably be "sym_off + i * jt_entry_size" for consistency.

Is there a way to run this AI as part of any PR to kernel-patches/bpf, not only those coming from the mailing list? Maybe for a selected commit?

Hi Anton,

If you have access to an "agentic" AI coding tool that runs locally, such as Claude Code, you can use our prompts repository [1] with a trigger prompt like this:

Current directory is the root of a Linux Kernel git repository. Using the prompt review/review-core.md and the prompt directory review do a code review of the top commit in the Linux repository.

Quoted 6 lines

The prompts expect the "agent" to be able to read and write files, and execute basic commands such as grep, find, awk and similar.

In principle it's possible to enable the review CI job for arbitrary pull requests, but the tokens are not free so we haven't considered that yet.

Quoted 8 lines

At least for me, it really helps having the reviews on the list. It gives me the chance to see what kinds of bugs AI is flagging correctly and where the false positives are. I do try and fix all the bad reviews that people flag, so the comments here are really helpful.

This isn't meant to discourage people from running reviews locally, I'm happy to help get you setup. But I also don't want to add a barrier to contributing code.

Thanks. In my case AI, for the most part, finds real bugs, so it looks to be helpful and its reviews make sense on the list. If only it could dump more in one pass, not squeese them one by one :)

Quoted 10 lines
Quoted 10 lines

[1] https://github.com/masoncl/review-prompts

Quoted 3 lines

Also, how deterministinc it is? Will it generate different comments for a given patch for different runs?

The short answer is no, the answers are not deterministic.

However for typical/obvious bugs you might often get a comment about the same issue worded differently.

Quoted 7 lines

Yeah, recent changes to the prompts have made it better, but for some bugs it still wanders off without flagging a percentage of the time.

Also, sometimes it'll get excited about finding a bug (even if a false positive) and skip to the end of the review, so if a patch has multiple problems, we might need multiple submissions to see them. I've been working on this as well, but there's still room for improvement.

After all, it just mimics how a real reviewer would behave (excited => skip)

Quoted 2 lines

-chris

#21 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

In reply to Ihor Solodrai
Quoted 130 lines · Ihor Solodrai
Quoted 113 lines · Anton Protopopov
Quoted 106 lines · On 25/11/02 09:13PM, [email protected] wrote:

```
commit b44690c09995d34f7665c7d687b359d41a6ab79f
Author: Anton Protopopov <[email protected]>

libbpf: support llvm-generated indirect jumps

This commit adds support for LLVM-generated indirect jumps in BPF v4 instruction set. It handles jump table metadata from the .jumptables ELF section and creates BPF_MAP_TYPE_INSN_ARRAY maps for them.

Quoted 4 lines
tools/lib/bpf/libbpf.c +0 -0
index fbe74686c..ed14090a9 100644

[ ... ]

Quoted 6 lines
tools/lib/bpf/libbpf.c +2 -0
@@ -6144,6 +6192,157 @@ static void poison_kfunc_call(struct bpf_program *prog, int relo_idx,
 	insn->imm = POISON_CALL_KFUNC_BASE + ext_idx;
 }
 
+static int find_jt_map(struct bpf_object *obj, struct bpf_program *prog, int sym_off)
+{

[ ... ]

Quoted 67 lines
tools/lib/bpf/libbpf.c +67 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;
+		} else {

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6320

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
                                                      ^^^^^^^^^^^^
Quoted 5 lines

Does this error message print the correct offset? Since jt is a pointer to __u64, the array access jt[i] is at byte offset "sym_off + i * 8", not "sym_off + i". All the other error messages in create_jt_map report byte offsets and sizes (sym_off, jt_size, sym_off + jt_size), so this one should probably be "sym_off + i * jt_entry_size" for consistency.

Quoted 3 lines

Is there a way to run this AI as part of any PR to kernel-patches/bpf, not only those coming from the mailing list? Maybe for a selected commit?

Hi Anton,

If you have access to an "agentic" AI coding tool that runs locally, such as Claude Code, you can use our prompts repository [1] with a trigger prompt like this:

Current directory is the root of a Linux Kernel git repository. Using the prompt review/review-core.md and the prompt directory review do a code review of the top commit in the Linux repository.

Quoted 8 lines

The prompts expect the "agent" to be able to read and write files, and execute basic commands such as grep, find, awk and similar.

In principle it's possible to enable the review CI job for arbitrary pull requests, but the tokens are not free so we haven't considered that yet.

[1] https://github.com/masoncl/review-prompts

Nice, thanks!

Quoted 8 lines
Quoted 3 lines

Also, how deterministinc it is? Will it generate different comments for a given patch for different runs?

The short answer is no, the answers are not deterministic.

However for typical/obvious bugs you might often get a comment about the same issue worded differently.

Ok, makes sense

Quoted 39 lines
Quoted 38 lines
Quoted 37 lines
Quoted 15 lines
tools/lib/bpf/libbpf.c +14 -0
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}
+
+	err = bpf_map_freeze(map_fd);
+	if (err)
+		goto err_close;
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6344

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	err = add_jt_map(obj, prog, sym_off, map_fd);
+	if (err)
+		goto err_close;
+
+	return map_fd;
+
+err_close:
+	close(map_fd);
+	return err;
+}

[ ... ]

```

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: libbpf: support llvm-generated indirect jumps CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19018051915

maintainer
#22 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

In reply to Anton Protopopov

On Sun, 2025-11-02 at 20:57 +0000, Anton Protopopov wrote:

[...]

Quoted 56 lines
tools/lib/bpf/libbpf.c +56 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6309

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;

I'd like to reiterate my point about relocation related warnings [1]:

Quoted 7 lines

I'm seeing the following messages when rebuilding bpf_gotox using llvm main, where Yonghong added __BPF_FEATURE_GOTOX.

CLNG-BPF [test_progs-cpuv4] bpf_gotox.bpf.o
GEN-SKEL [test_progs-cpuv4] bpf_gotox.skel.h
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables

In the context of Yonghong's reply [2].

I inserted some debug prints and confirm that these relocations are generated for basic block labels, e.g.:

.S file corresponding to shortened bpf_gotox.c:

      ...
             gotox r1
     .Ltmp25:
     .Ltmp26:
     .Ltmp27:                                # Block address taken
     LBB0_5:                                 # %l1
             #DEBUG_LABEL: one_jump_two_maps:l1
             .loc    0 36 10 is_stmt 1               # progs/bpf_gotox.c:36:10
     .Ltmp28:
             w1 = *(u32 *)(r10 - 4)
      ...
             .section        .jumptables,"",@progbits
    BPF.JT.0.0:
            .quad   LBB0_5

objdump --symbols, corresponding to same shortened bpf_gotox.c:

Symbol table '.symtab' contains 18 entries:
Num: Value Size Type Bind Vis Ndx Name
...
2: 0000000000000000 0 SECTION LOCAL DEFAULT 3 syscall

objdump --relocations, corresponding to same shortened bpf_gotox.c:

    Relocation section '.rel.jumptables' at offset 0xde8 contains 4 entries:
         Offset             Info             Type               Symbol's Value  Symbol's Name
     0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
     ...

Here the first entry corresponds to LBB0_5 symbol, specifically:

                         Relocation type (R_BPF_64_ABS64).
                             vvvvvvvv
   0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
   ^^^^^^^^^^^^^^^^  ^^^^^^^^                                ^^^^^^^^^^^^^^^^
   Offset at which   Section                                 Given that relocation type is
   to apply the      index                                   R_BPF_64_ABS64, this is the value
   relocation,       for 'syscall'.                          which has to be written at offset.
   first jumptables                                          (See [3]).

record.

Given above, I conclude that:

- [to Anton] libbpf has to apply the relocations from .rel.jumptables in order to assign correct the sec_insn_off for records in the jump table. Right now we imply that each record in the table corresponds to a section where jump table is referenced from, but that is not true.

- [to Yonghong] LLVM should generate a different relocation kind, or a different "Symbol's Value", otherwise applying relocations as instructed in [3] will lead to zeroes in the jump table:

Quoted 5 lines

In another case, R_BPF_64_ABS64 relocation type is used for normal 64-bit data. The actual to-be-relocated data is stored at r_offset and the read/write data bitsize is 64 (8 bytes). The relocation can be resolved with the symbol value plus implicit addend.

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://docs.kernel.org/bpf/llvm_reloc.html

Quoted 20 lines
tools/lib/bpf/libbpf.c +20 -0
+		} else {
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}

[...]

maintainer
#23 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

Quoted 146 lines · Eduard Zingerman

On Sun, 2025-11-02 at 20:57 +0000, Anton Protopopov wrote:

[...]

Quoted 56 lines
tools/lib/bpf/libbpf.c +56 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6309

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;

I'd like to reiterate my point about relocation related warnings [1]:

Quoted 7 lines

I'm seeing the following messages when rebuilding bpf_gotox using llvm main, where Yonghong added __BPF_FEATURE_GOTOX.

CLNG-BPF [test_progs-cpuv4] bpf_gotox.bpf.o
GEN-SKEL [test_progs-cpuv4] bpf_gotox.skel.h
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables

In the context of Yonghong's reply [2].

I inserted some debug prints and confirm that these relocations are generated for basic block labels, e.g.:

.S file corresponding to shortened bpf_gotox.c:

      ...
             gotox r1
     .Ltmp25:
     .Ltmp26:
     .Ltmp27:                                # Block address taken
     LBB0_5:                                 # %l1
             #DEBUG_LABEL: one_jump_two_maps:l1
             .loc    0 36 10 is_stmt 1               # progs/bpf_gotox.c:36:10
     .Ltmp28:
             w1 = *(u32 *)(r10 - 4)
      ...
             .section        .jumptables,"",@progbits
    BPF.JT.0.0:
            .quad   LBB0_5

objdump --symbols, corresponding to same shortened bpf_gotox.c:

Symbol table '.symtab' contains 18 entries:
Num: Value Size Type Bind Vis Ndx Name
...
2: 0000000000000000 0 SECTION LOCAL DEFAULT 3 syscall

objdump --relocations, corresponding to same shortened bpf_gotox.c:

    Relocation section '.rel.jumptables' at offset 0xde8 contains 4 entries:
         Offset             Info             Type               Symbol's Value  Symbol's Name
     0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
     ...

Here the first entry corresponds to LBB0_5 symbol, specifically:

                         Relocation type (R_BPF_64_ABS64).
                             vvvvvvvv
   0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
   ^^^^^^^^^^^^^^^^  ^^^^^^^^                                ^^^^^^^^^^^^^^^^
   Offset at which   Section                                 Given that relocation type is
   to apply the      index                                   R_BPF_64_ABS64, this is the value
   relocation,       for 'syscall'.                          which has to be written at offset.
   first jumptables                                          (See [3]).

record.

Given above, I conclude that:

- [to Anton] libbpf has to apply the relocations from .rel.jumptables in order to assign correct the sec_insn_off for records in the jump table. Right now we imply that each record in the table corresponds to a section where jump table is referenced from, but that is not true.

- [to Yonghong] LLVM should generate a different relocation kind, or a different "Symbol's Value", otherwise applying relocations as instructed in [3] will lead to zeroes in the jump table:

Quoted 5 lines

In another case, R_BPF_64_ABS64 relocation type is used for normal 64-bit data. The actual to-be-relocated data is stored at r_offset and the read/write data bitsize is 64 (8 bytes). The relocation can be resolved with the symbol value plus implicit addend.

Quoted 3 lines

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://docs.kernel.org/bpf/llvm_reloc.html

Ok, according to [4], the "implicit addend" means the value in the location to be modified. Meaning that final calculation should be:

<jump-table value> + <'syscall' symbol value>

So, we either need to assume that section symbol values are always zero, or do the computation on the libbpf side. (Or skip generating such relocations in LLVM, if we know that symbol value is zero).

[4] System V Application Binary Interface, Edition 4.1,

    Section 4-28, Page 72.

[...]

Quoted 22 lines
Quoted 20 lines
tools/lib/bpf/libbpf.c +20 -0
+		} else {
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}

[...]

reviewer
#24 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

In reply to Eduard Zingerman
Quoted 135 lines · Eduard Zingerman

On Sun, 2025-11-02 at 20:57 +0000, Anton Protopopov wrote:

[...]

Quoted 56 lines
tools/lib/bpf/libbpf.c +56 -0
+static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struct reloc_desc *relo)
+{
+	const __u32 jt_entry_size = 8;
+	int sym_off = relo->sym_off;
+	int jt_size = relo->sym_size;
+	__u32 max_entries = jt_size / jt_entry_size;
+	__u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	int subprog_idx;
+	int map_fd, err;
+	__u64 insn_off;
+	__u64 *jt;
+	__u32 i;
+
+	map_fd = find_jt_map(obj, prog, sym_off);
+	if (map_fd >= 0)
+		return map_fd;
+
+	if (sym_off % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable start %d should be multiple of %u\n",
+			sym_off, jt_entry_size);
+		return -EINVAL;
+	}
+
+	if (jt_size % jt_entry_size) {
+		pr_warn("map '.jumptables': jumptable size %d should be multiple of %u\n",
+			jt_size, jt_entry_size);
+		return -EINVAL;
+	}
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".jumptables",
+				4, value_size, max_entries, NULL);
+	if (map_fd < 0)
+		return map_fd;
+
+	if (!obj->jumptables_data) {
+		pr_warn("map '.jumptables': ELF file is missing jump table data\n");
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (sym_off + jt_size > obj->jumptables_data_sz) {
+		pr_warn("map '.jumptables': jumptables_data size is %zd, trying to access %d\n",
+			obj->jumptables_data_sz, sym_off + jt_size);
+		err = -EINVAL;
+		goto err_close;
+	}
+
+	subprog_idx = -1; /* main program */
+	if (relo->insn_idx < 0 || relo->insn_idx >= prog->insns_cnt) {
+		pr_warn("map '.jumptables': invalid instruction index %d\n", relo->insn_idx);
+		err = -EINVAL;
+		goto err_close;
+	}
+	if (prog->subprogs)
+		subprog_idx = find_subprog_idx(prog, relo->insn_idx);
+

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6309

Quoted 10 lines
tools/lib/bpf/libbpf.c +10 -0
+	jt = (__u64 *)(obj->jumptables_data + sym_off);
+	for (i = 0; i < max_entries; i++) {
+		/*
+		 * The offset should be made to be relative to the beginning of
+		 * the main function, not the subfunction.
+		 */
+		insn_off = jt[i]/sizeof(struct bpf_insn);
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;

I'd like to reiterate my point about relocation related warnings [1]:

Quoted 7 lines

I'm seeing the following messages when rebuilding bpf_gotox using llvm main, where Yonghong added __BPF_FEATURE_GOTOX.

CLNG-BPF [test_progs-cpuv4] bpf_gotox.bpf.o
GEN-SKEL [test_progs-cpuv4] bpf_gotox.skel.h
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables

In the context of Yonghong's reply [2].

I inserted some debug prints and confirm that these relocations are generated for basic block labels, e.g.:

   .S file corresponding to shortened bpf_gotox.c:
       ...
              gotox r1
      .Ltmp25:
      .Ltmp26:
      .Ltmp27:                                # Block address taken
      LBB0_5:                                 # %l1
              #DEBUG_LABEL: one_jump_two_maps:l1
              .loc    0 36 10 is_stmt 1               # progs/bpf_gotox.c:36:10
      .Ltmp28:
              w1 = *(u32 *)(r10 - 4)
       ...
              .section        .jumptables,"",@progbits
     BPF.JT.0.0:
             .quad   LBB0_5

   objdump --symbols, corresponding to same shortened bpf_gotox.c:

     Symbol table '.symtab' contains 18 entries:
        Num:    Value          Size Type    Bind   Vis       Ndx Name
          ...
          2: 0000000000000000     0 SECTION LOCAL  DEFAULT     3 syscall

   objdump --relocations, corresponding to same shortened bpf_gotox.c:

     Relocation section '.rel.jumptables' at offset 0xde8 contains 4 entries:
          Offset             Info             Type               Symbol's Value  Symbol's Name
      0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
      ...

Here the first entry corresponds to LBB0_5 symbol, specifically:

Relocation type (R_BPF_64_ABS64).
vvvvvvvv
0000000000000000 0000000200000002 R_BPF_64_ABS64 0000000000000000 syscall
^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^^^^^^^^
Offset at which Section Given that relocation type is
to apply the index R_BPF_64_ABS64, this is the value
relocation, for 'syscall'. which has to be written at offset.
first jumptables (See [3]).
record.

Given above, I conclude that:

- [to Anton] libbpf has to apply the relocations from .rel.jumptables in order to assign correct the sec_insn_off for records in the jump table. Right now we imply that each record in the table corresponds to a section where jump table is referenced from, but that is not true.

Quoted 9 lines

- [to Yonghong] LLVM should generate a different relocation kind, or a different "Symbol's Value", otherwise applying relocations as instructed in [3] will lead to zeroes in the jump table:

Quoted 5 lines

In another case, R_BPF_64_ABS64 relocation type is used for normal 64-bit data. The actual to-be-relocated data is stored at r_offset and the read/write data bitsize is 64 (8 bytes). The relocation can be resolved with the symbol value plus implicit addend.

I think the following llvm patch
https://github.com/llvm/llvm-project/pull/166301
should avoid jumptable relocation. The idea is originated from one of Eduard's idea which put the label difference in the jump table entry. Please give a try. I think it should work.

Quoted 26 lines

[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://docs.kernel.org/bpf/llvm_reloc.html

Quoted 20 lines
tools/lib/bpf/libbpf.c +20 -0
+		} else {
+			insn_off -= prog->sec_insn_off;
+		}
+
+		/*
+		 * LLVM-generated jump tables contain u64 records, however
+		 * should contain values that fit in u32.
+		 */
+		if (insn_off > UINT32_MAX) {
+			pr_warn("map '.jumptables': invalid jump table value 0x%llx at offset %d\n",
+				(long long)jt[i], sym_off + i);
+			err = -EINVAL;
+			goto err_close;
+		}
+
+		val.orig_off = insn_off;
+		err = bpf_map_update_elem(map_fd, &i, &val, 0);
+		if (err)
+			goto err_close;
+	}

[...]

maintainer
#25 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

On Mon, 2025-11-03 at 21:26 -0800, Yonghong Song wrote:

[...]

Quoted 78 lines
Quoted 68 lines

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6309

Quoted 3 lines
tools/lib/bpf/libbpf.c +3 -0
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;

I'd like to reiterate my point about relocation related warnings [1]:

Quoted 7 lines

I'm seeing the following messages when rebuilding bpf_gotox using llvm main, where Yonghong added __BPF_FEATURE_GOTOX.

CLNG-BPF [test_progs-cpuv4] bpf_gotox.bpf.o
GEN-SKEL [test_progs-cpuv4] bpf_gotox.skel.h
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables

In the context of Yonghong's reply [2].

I inserted some debug prints and confirm that these relocations are generated for basic block labels, e.g.:

   .S file corresponding to shortened bpf_gotox.c:
       ...
              gotox r1
      .Ltmp25:
      .Ltmp26:
      .Ltmp27:                                # Block address taken
      LBB0_5:                                 # %l1
              #DEBUG_LABEL: one_jump_two_maps:l1
              .loc    0 36 10 is_stmt 1               # progs/bpf_gotox.c:36:10
      .Ltmp28:
              w1 = *(u32 *)(r10 - 4)
       ...
              .section        .jumptables,"",@progbits
     BPF.JT.0.0:
             .quad   LBB0_5

   objdump --symbols, corresponding to same shortened bpf_gotox.c:

     Symbol table '.symtab' contains 18 entries:
        Num:    Value          Size Type    Bind   Vis       Ndx Name
          ...
          2: 0000000000000000     0 SECTION LOCAL  DEFAULT     3 syscall

   objdump --relocations, corresponding to same shortened bpf_gotox.c:

     Relocation section '.rel.jumptables' at offset 0xde8 contains 4 entries:
          Offset             Info             Type               Symbol's Value  Symbol's Name
      0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
      ...

Here the first entry corresponds to LBB0_5 symbol, specifically:

Relocation type (R_BPF_64_ABS64).
vvvvvvvv
0000000000000000 0000000200000002 R_BPF_64_ABS64 0000000000000000 syscall
^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^^^^^^^^
Offset at which Section Given that relocation type is
to apply the index R_BPF_64_ABS64, this is the value
relocation, for 'syscall'. which has to be written at offset.
first jumptables (See [3]).
record.

Given above, I conclude that:

- [to Anton] libbpf has to apply the relocations from .rel.jumptables in order to assign correct the sec_insn_off for records in the jump table. Right now we imply that each record in the table corresponds to a section where jump table is referenced from, but that is not true.

Quoted 9 lines

- [to Yonghong] LLVM should generate a different relocation kind, or a different "Symbol's Value", otherwise applying relocations as instructed in [3] will lead to zeroes in the jump table:

Quoted 5 lines

In another case, R_BPF_64_ABS64 relocation type is used for normal 64-bit data. The actual to-be-relocated data is stored at r_offset and the read/write data bitsize is 64 (8 bytes). The relocation can be resolved with the symbol value plus implicit addend.

Quoted 5 lines

I think the following llvm patch

    https://github.com/llvm/llvm-project/pull/166301

should avoid jumptable relocation. The idea is originated from one of Eduard's idea which put the label difference in the jump table entry. Please give a try. I think it should work.

I tried Yonghong's llvm change, and it solves the issue. Relocations are no longer generated for .jumptables section. So, nothing to change on the libbpf side.

[...]

#26 lore raw

Re: [PATCH v10 bpf-next 08/11] libbpf: support llvm-generated indirect jumps

Quoted 88 lines · Eduard Zingerman

On Mon, 2025-11-03 at 21:26 -0800, Yonghong Song wrote:

[...]

Quoted 78 lines
Quoted 68 lines

tools/lib/bpf/libbpf.c static void poison_kfunc_call(struct bpf_program *prog, int relo_idx, · line 6309

Quoted 3 lines
tools/lib/bpf/libbpf.c +3 -0
+		if (subprog_idx >= 0) {
+			insn_off -= prog->subprogs[subprog_idx].sec_insn_off;
+			insn_off += prog->subprogs[subprog_idx].sub_insn_off;

I'd like to reiterate my point about relocation related warnings [1]:

Quoted 7 lines

I'm seeing the following messages when rebuilding bpf_gotox using llvm main, where Yonghong added __BPF_FEATURE_GOTOX.

CLNG-BPF [test_progs-cpuv4] bpf_gotox.bpf.o
GEN-SKEL [test_progs-cpuv4] bpf_gotox.skel.h
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables
libbpf: elf: skipping relo section(13) .rel.jumptables for section(6) .jumptables

In the context of Yonghong's reply [2].

I inserted some debug prints and confirm that these relocations are generated for basic block labels, e.g.:

   .S file corresponding to shortened bpf_gotox.c:
       ...
              gotox r1
      .Ltmp25:
      .Ltmp26:
      .Ltmp27:                                # Block address taken
      LBB0_5:                                 # %l1
              #DEBUG_LABEL: one_jump_two_maps:l1
              .loc    0 36 10 is_stmt 1               # progs/bpf_gotox.c:36:10
      .Ltmp28:
              w1 = *(u32 *)(r10 - 4)
       ...
              .section        .jumptables,"",@progbits
     BPF.JT.0.0:
             .quad   LBB0_5

   objdump --symbols, corresponding to same shortened bpf_gotox.c:

     Symbol table '.symtab' contains 18 entries:
        Num:    Value          Size Type    Bind   Vis       Ndx Name
          ...
          2: 0000000000000000     0 SECTION LOCAL  DEFAULT     3 syscall

   objdump --relocations, corresponding to same shortened bpf_gotox.c:

     Relocation section '.rel.jumptables' at offset 0xde8 contains 4 entries:
          Offset             Info             Type               Symbol's Value  Symbol's Name
      0000000000000000  0000000200000002 R_BPF_64_ABS64         0000000000000000 syscall
      ...

Here the first entry corresponds to LBB0_5 symbol, specifically:

Relocation type (R_BPF_64_ABS64).
vvvvvvvv
0000000000000000 0000000200000002 R_BPF_64_ABS64 0000000000000000 syscall
^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^^^^^^^^
Offset at which Section Given that relocation type is
to apply the index R_BPF_64_ABS64, this is the value
relocation, for 'syscall'. which has to be written at offset.
first jumptables (See [3]).
record.

Given above, I conclude that:

- [to Anton] libbpf has to apply the relocations from .rel.jumptables in order to assign correct the sec_insn_off for records in the jump table. Right now we imply that each record in the table corresponds to a section where jump table is referenced from, but that is not true.

Quoted 9 lines

- [to Yonghong] LLVM should generate a different relocation kind, or a different "Symbol's Value", otherwise applying relocations as instructed in [3] will lead to zeroes in the jump table:

Quoted 5 lines

In another case, R_BPF_64_ABS64 relocation type is used for normal 64-bit data. The actual to-be-relocated data is stored at r_offset and the read/write data bitsize is 64 (8 bytes). The relocation can be resolved with the symbol value plus implicit addend.

Quoted 5 lines

I think the following llvm patch

    https://github.com/llvm/llvm-project/pull/166301

should avoid jumptable relocation. The idea is originated from one of Eduard's idea which put the label difference in the jump table entry. Please give a try. I think it should work.

Quoted 3 lines

I tried Yonghong's llvm change, and it solves the issue. Relocations are no longer generated for .jumptables section. So, nothing to change on the libbpf side.

Thanks Eduard and Yonghong! It does work, I've removed the check from libbpf.

One additional change I had to do is to fix verifier_gotox not to generate .jumptables.rel:

        SEC("socket")                                                   \
        OUTCOME                                                         \
        __naked void jump_table_ ## NAME(void)                          \
        {                                                               \
                asm volatile ("                                         \
                .pushsection .jumptables,\"\",@progbits;                \
        jt0_%=:                                                         \
-               .quad ret0_%=;                                          \
-               .quad ret1_%=;                                          \
+               .quad ret0_%= - socket;                                 \
+               .quad ret1_%= - socket;                                 \
                .size jt0_%=, 16;                                       \
                .global jt0_%=;                                         \
                .popsection;                                            \

(And alike in other subtests, of course.)

Quoted 1 line

[...]

#27 lore raw

PATCH v10 9/11 bpftool: Recognize insn_array map type

In reply to Anton Protopopov

Teach bpftool to recognize instruction array map type.

---
 tools/bpf/bpftool/Documentation/bpftool-map.rst | 3 ++-
 tools/bpf/bpftool/map.c                         | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
tools/bpf/bpftool/Documentation/bpftool-map.rst +2 -1
index 252e4c538edb..1af3305ea2b2 100644
@@ -55,7 +55,8 @@ MAP COMMANDS
 |     | **devmap** | **devmap_hash** | **sockmap** | **cpumap** | **xskmap** | **sockhash**
 |     | **cgroup_storage** | **reuseport_sockarray** | **percpu_cgroup_storage**
 |     | **queue** | **stack** | **sk_storage** | **struct_ops** | **ringbuf** | **inode_storage**
-|     | **task_storage** | **bloom_filter** | **user_ringbuf** | **cgrp_storage** | **arena** }
+|     | **task_storage** | **bloom_filter** | **user_ringbuf** | **cgrp_storage** | **arena**
+|     | **insn_array** }
 
 DESCRIPTION
 ===========
tools/bpf/bpftool/map.c +2 -1
index c9de44a45778..7ebf7dbcfba4 100644
@@ -1477,7 +1477,8 @@ static int do_help(int argc, char **argv)
 		"                 devmap | devmap_hash | sockmap | cpumap | xskmap | sockhash |\n"
 		"                 cgroup_storage | reuseport_sockarray | percpu_cgroup_storage |\n"
 		"                 queue | stack | sk_storage | struct_ops | ringbuf | inode_storage |\n"
-		"                 task_storage | bloom_filter | user_ringbuf | cgrp_storage | arena }\n"
+		"                 task_storage | bloom_filter | user_ringbuf | cgrp_storage | arena |\n"
+		"                 insn_array }\n"
 		"       " HELP_SPEC_OPTIONS " |\n"
 		"                    {-f|--bpffs} | {-n|--nomount} }\n"
 		"",
Signature · 1 line
2.34.1
#28 lore raw

PATCH v10 10/11 selftests/bpf: add new verifier_gotox test

In reply to Anton Protopopov

Add a set of tests to validate core gotox functionality without need to rely on compilers.

---
 .../selftests/bpf/prog_tests/verifier.c       |   2 +
 .../selftests/bpf/progs/verifier_gotox.c      | 389 ++++++++++++++++++
 2 files changed, 391 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_gotox.c
tools/testing/selftests/bpf/prog_tests/verifier.c +2 -0
index c0e8ffdaa484..4b4b081b46cc 100644
@@ -35,6 +35,7 @@
 #include "verifier_global_subprogs.skel.h"
 #include "verifier_global_ptr_args.skel.h"
 #include "verifier_gotol.skel.h"
+#include "verifier_gotox.skel.h"
 #include "verifier_helper_access_var_len.skel.h"
 #include "verifier_helper_packet_access.skel.h"
 #include "verifier_helper_restricted.skel.h"
@@ -173,6 +174,7 @@ void test_verifier_div_overflow(void)         { RUN(verifier_div_overflow); }
 void test_verifier_global_subprogs(void)      { RUN(verifier_global_subprogs); }
 void test_verifier_global_ptr_args(void)      { RUN(verifier_global_ptr_args); }
 void test_verifier_gotol(void)                { RUN(verifier_gotol); }
+void test_verifier_gotox(void)                { RUN(verifier_gotox); }
 void test_verifier_helper_access_var_len(void) { RUN(verifier_helper_access_var_len); }
 void test_verifier_helper_packet_access(void) { RUN(verifier_helper_packet_access); }
 void test_verifier_helper_restricted(void)    { RUN(verifier_helper_restricted); }
tools/testing/selftests/bpf/progs/verifier_gotox.c new file +389 -0

389 lines not on this page · raw message

Signature · 1 line
2.34.1
#29 lore raw

PATCH v10 11/11 selftests/bpf: add C-level selftests for indirect jumps

In reply to Anton Protopopov

Add C-level selftests for indirect jumps to validate LLVM and libbpf functionality. The tests are intentionally disabled, to be run locally by developers, but will not make the CI red.

---
 tools/testing/selftests/bpf/Makefile          |   4 +-
 .../selftests/bpf/prog_tests/bpf_gotox.c      | 272 +++++++++++
 tools/testing/selftests/bpf/progs/bpf_gotox.c | 437 ++++++++++++++++++
 3 files changed, 712 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_gotox.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_gotox.c
tools/testing/selftests/bpf/Makefile +3 -1
index 93dbafa050c9..34ea23c63bd5 100644
@@ -453,7 +453,9 @@ BPF_CFLAGS = -g -Wall -Werror -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN)	\
 	     -I$(abspath $(OUTPUT)/../usr/include)			\
 	     -std=gnu11		 					\
 	     -fno-strict-aliasing 					\
-	     -Wno-compare-distinct-pointer-types
+	     -Wno-compare-distinct-pointer-types			\
+	     -Wno-initializer-overrides					\
+	     #
 # TODO: enable me -Wsign-compare
 
 CLANG_CFLAGS = $(CLANG_SYS_INCLUDES)
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +272 -0

272 lines not on this page · raw message

tools/testing/selftests/bpf/progs/bpf_gotox.c new file +437 -0

437 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#30 lore raw

Re: [PATCH v10 bpf-next 11/11] selftests/bpf: add C-level selftests for indirect jumps

On Sun, 2025-11-02 at 20:57 +0000, Anton Protopopov wrote:

[...]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +0 -0
new file mode 100644
index 000000000000..2a55fa91e1fa
@@ -0,0 +1,272 @@

[...]

Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +14 -0
+static void check_simple_fentry(struct bpf_gotox *skel,
+				struct bpf_program *prog,
+				__u64 ctx_in,
+				__u64 expected)
+{
+	skel->bss->in_user = ctx_in;
+	skel->bss->ret_user = 0;
+
+	/* trigger */
+	usleep(1);
+
+	if (!ASSERT_EQ(skel->bss->ret_user, expected, "skel->bss->ret_user"))
+		return;
+}

[...]

tools/testing/selftests/bpf/prog_tests/bpf_gotox.c line 201

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +10 -0
+static void check_other_sec(struct bpf_gotox *skel)
+{
+	__u64 in[]   = {0, 1, 2, 3, 4,  5, 77};
+	__u64 out[]  = {2, 3, 4, 5, 7, 19, 19};
+	int i;
+
+	bpf_program__attach(skel->progs.simple_test_other_sec);
+	for (i = 0; i < ARRAY_SIZE(in); i++)
+		check_simple_fentry(skel, skel->progs.simple_test_other_sec, in[i], out[i]);
+}

The above means that fentry programs are accumulated for 'sys_nanosleep', right? In all 3 test cases that use check_simple_fentry() the identical 'out' values are used. Should the programs be detached here to avoid possible masking of a misbehaving program?

[...]

tools/testing/selftests/bpf/progs/bpf_gotox.c

Quoted 5 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +0 -0
new file mode 100644
index 000000000000..8a84f4b225b2

+SEC("fentry/" SYS_PREFIX "sys_nanosleep")
+int use_nonstatic_global_other_sec(void *ctx)
+{
+ return __nonstatic_global(in_user);
+}

Should this check for target process pid?

[...]

Quoted 9 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +9 -0
+#define SKIP_TEST(TEST_NAME)				\
+	SEC("syscall") int TEST_NAME(void *ctx)		\
+	{						\
+		return 0;				\
+	}
+
+SKIP_TEST(one_switch);
+SKIP_TEST(one_switch_non_zero_sec_off);
+SKIP_TEST(simple_test_other_sec);

tools/testing/selftests/bpf/progs/bpf_gotox.c line 433

Quoted 10 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +10 -0
+SKIP_TEST(two_switches);
+SKIP_TEST(big_jump_table);
+SKIP_TEST(one_jump_two_maps);
+SKIP_TEST(one_map_two_jumps);
+SKIP_TEST(use_static_global1);
+SKIP_TEST(use_static_global2);
+SKIP_TEST(use_static_global_other_sec);
+SKIP_TEST(use_nonstatic_global1);
+SKIP_TEST(use_nonstatic_global2);
+SKIP_TEST(use_nonstatic_global_other_sec);

Nice.
I double checked and tests are skipped when old clang is used and pass when new clang is used.

Quoted 3 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +3 -0
+#endif /* __BPF_FEATURE_GOTOX */
+
+char _license[] SEC("license") = "GPL";
#31 lore raw

Re: [PATCH v10 bpf-next 11/11] selftests/bpf: add C-level selftests for indirect jumps

Quoted 30 lines · Eduard Zingerman

On Sun, 2025-11-02 at 20:57 +0000, Anton Protopopov wrote:

[...]

Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +0 -0
new file mode 100644
index 000000000000..2a55fa91e1fa
@@ -0,0 +1,272 @@

[...]

Quoted 14 lines
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +14 -0
+static void check_simple_fentry(struct bpf_gotox *skel,
+				struct bpf_program *prog,
+				__u64 ctx_in,
+				__u64 expected)
+{
+	skel->bss->in_user = ctx_in;
+	skel->bss->ret_user = 0;
+
+	/* trigger */
+	usleep(1);
+
+	if (!ASSERT_EQ(skel->bss->ret_user, expected, "skel->bss->ret_user"))
+		return;
+}

[...]

Quoted 15 lines

tools/testing/selftests/bpf/prog_tests/bpf_gotox.c line 201

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/bpf_gotox.c new file +10 -0
+static void check_other_sec(struct bpf_gotox *skel)
+{
+	__u64 in[]   = {0, 1, 2, 3, 4,  5, 77};
+	__u64 out[]  = {2, 3, 4, 5, 7, 19, 19};
+	int i;
+
+	bpf_program__attach(skel->progs.simple_test_other_sec);
+	for (i = 0; i < ARRAY_SIZE(in); i++)
+		check_simple_fentry(skel, skel->progs.simple_test_other_sec, in[i], out[i]);
+}

The above means that fentry programs are accumulated for 'sys_nanosleep', right? In all 3 test cases that use check_simple_fentry() the identical 'out' values are used. Should the programs be detached here to avoid possible masking of a misbehaving program?

yes, thanks, will detach it

Quoted 15 lines

[...]

Quoted 5 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +0 -0
new file mode 100644
index 000000000000..8a84f4b225b2
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +5 -0
+SEC("fentry/" SYS_PREFIX "sys_nanosleep")
+int use_nonstatic_global_other_sec(void *ctx)
+{
+	return __nonstatic_global(in_user);
+}
Quoted 1 line

Should this check for target process pid?

And here will add a fix as well. Thanks

Quoted 29 lines

[...]

Quoted 9 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +9 -0
+#define SKIP_TEST(TEST_NAME)				\
+	SEC("syscall") int TEST_NAME(void *ctx)		\
+	{						\
+		return 0;				\
+	}
+
+SKIP_TEST(one_switch);
+SKIP_TEST(one_switch_non_zero_sec_off);
+SKIP_TEST(simple_test_other_sec);

tools/testing/selftests/bpf/progs/bpf_gotox.c line 433

Quoted 10 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +10 -0
+SKIP_TEST(two_switches);
+SKIP_TEST(big_jump_table);
+SKIP_TEST(one_jump_two_maps);
+SKIP_TEST(one_map_two_jumps);
+SKIP_TEST(use_static_global1);
+SKIP_TEST(use_static_global2);
+SKIP_TEST(use_static_global_other_sec);
+SKIP_TEST(use_nonstatic_global1);
+SKIP_TEST(use_nonstatic_global2);
+SKIP_TEST(use_nonstatic_global_other_sec);

Nice.
I double checked and tests are skipped when old clang is used and pass when new clang is used.

Quoted 3 lines
tools/testing/selftests/bpf/progs/bpf_gotox.c new file +3 -0
+#endif /* __BPF_FEATURE_GOTOX */
+
+char _license[] SEC("license") = "GPL";