BPF support for global data
Subsystems (7)
Daniel Borkmann
maintainerv3 cover BPF support for global data
This series is a major rework of previously submitted libbpf patches [0] in order to add global data support for BPF. The kernel has been extended to add proper infrastructure that allows for full .bss/.data/.rodata sections on BPF loader side based upon feedback from LPC discussions [1]. Latter support is then also added into libbpf in this series which allows for more natural C-like programming of BPF programs. For more information on loader, please refer to 'bpf, libbpf: support global data/bss/ rodata sections' patch in this series. Joint work with Joe Stringer.
Thanks a lot!
Note, here's the current state now updated with full BTF support in bpftool to be able to inspect global data maps. I'm planning to do one last round on the series (at latest end of week), so don't apply yet. ;)
rfc v3 -> v3:
- Implement BTF support in kernel, libbpf, bpftool, add tests
- Fix idx + off conversion (Andrii)
- Document lower / higher bits for direct value access (Andrii)
- Add tests with small value size (Andrii)
v2 -> rfc v3:
- Add index selection into ldimm64 (Andrii)
- Fix missing fdput() (Jann)
- Reject invalid flags in BPF_F_*_PROG (Jakub)
- Complete rework of libbpf support, includes:
- Add objname to map name (Stanislav)
- Make .rodata map full read-only after setup (Andrii)
- Merge relocation handling into single one (Andrii)
- Store global maps into obj->maps array (Andrii, Alexei)
- Debug message when skipping section (Andrii)
- Reject non-static global data till we have semantics for sharing them (Yonghong, Andrii, Alexei)
- More test cases and completely reworked prog test (Alexei)
- Fixes, cleanups, etc all over the set
- Not yet addressed:
- Make BTF mandatory for these maps (Alexei)
-> Waiting till BTF support for these lands first
v1 -> v2:
- Instead of 32-bit static data, implement full global data support (Alexei)
[0] https://patchwork.ozlabs.org/cover/1040290/
[1] http://vger.kernel.org/lpc-bpf2018.html#session-3
Daniel Borkmann (13):
bpf: implement lookup-free direct value access for maps
bpf: add program side {rd,wr}only support for maps
bpf: add syscall side map lock support
bpf: allow . char as part of the object name
bpf: add specification for BTF Var and DataSec kinds
bpf: kernel side support for BTF Var and DataSec
bpf: allow for key-less BTF in array map
bpf: sync {btf,bpf}.h uapi header from tools infrastructure bpf, libbpf: support global data/bss/rodata sections bpf, libbpf: add support for BTF Var and DataSec
bpf: bpftool support for dumping data/bss/rodata sections bpf, selftest: test {rd,wr}only flags and direct value access bpf, selftest: add test cases for BTF Var and DataSec
Joe Stringer (2):
bpf, libbpf: refactor relocation handling bpf, selftest: test global data/bss/rodata sections
Documentation/bpf/btf.rst | 57 +++ include/linux/bpf.h | 35 +- include/linux/bpf_verifier.h | 4 + include/linux/btf.h | 2 + include/uapi/linux/bpf.h | 24 +- include/uapi/linux/btf.h | 32 +- kernel/bpf/arraymap.c | 47 +- kernel/bpf/btf.c | 400 ++++++++++++++- kernel/bpf/core.c | 3 +- kernel/bpf/disasm.c | 5 +- kernel/bpf/hashtab.c | 6 +- kernel/bpf/local_storage.c | 8 +- kernel/bpf/lpm_trie.c | 6 +- kernel/bpf/queue_stack_maps.c | 6 +- kernel/bpf/syscall.c | 126 ++++- kernel/bpf/verifier.c | 155 ++++-- tools/bpf/bpftool/btf_dumper.c | 59 +++ tools/bpf/bpftool/map.c | 10 +- tools/bpf/bpftool/xlated_dumper.c | 6 + tools/include/linux/filter.h | 14 + tools/include/uapi/linux/bpf.h | 24 +- tools/include/uapi/linux/btf.h | 32 +- tools/lib/bpf/bpf.c | 10 + tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/btf.c | 97 +++- tools/lib/bpf/btf.h | 3 + tools/lib/bpf/libbpf.c | 537 ++++++++++++++++---- tools/lib/bpf/libbpf.h | 5 + tools/lib/bpf/libbpf.map | 7 + tools/testing/selftests/bpf/bpf_helpers.h | 8 +- .../testing/selftests/bpf/prog_tests/global_data.c | 157 ++++++ .../testing/selftests/bpf/progs/test_global_data.c | 106 ++++ tools/testing/selftests/bpf/test_btf.c | 552 ++++++++++++++++++++- tools/testing/selftests/bpf/test_verifier.c | 53 +- .../testing/selftests/bpf/verifier/array_access.c | 159 ++++++ .../selftests/bpf/verifier/direct_value_access.c | 262 ++++++++++ 36 files changed, 2812 insertions(+), 206 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/global_data.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_data.c create mode 100644 tools/testing/selftests/bpf/verifier/direct_value_access.c
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 1/15 bpf: implement lookup-free direct value access for maps
This generic extension to BPF maps allows for directly loading an address residing inside a BPF map value as a single BPF ldimm64 instruction.
The idea is similar to what BPF_PSEUDO_MAP_FD does today, which is a special src_reg flag for ldimm64 instruction that indicates that inside the first part of the double insns's imm field is a file descriptor which the verifier then replaces as a full 64bit address of the map into both imm parts. For the newly added BPF_PSEUDO_MAP_VALUE src_reg flag, the idea is the following:
the first part of the double insns's imm field is again a file descriptor corresponding to the map, and the second part of the imm field is an offset into the value. Both insns's off fields build the optional key resp. index to the map if it contains more than just one element. The verifier will then replace both imm parts with an address that points into the BPF map value for maps that support this operation. BPF_PSEUDO_MAP_VALUE is a distinct flag as otherwise with BPF_PSEUDO_MAP_FD we could not differ offset 0 between load of map pointer versus load of map's value at offset 0, and changing BPF_PSEUDO_MAP_FD's encoding into off by one to differ between regular map pointer and map value pointer would add unnecessary complexity and increases barrier for debuggability thus less suitable.
This extension allows for efficiently retrieving an address to a map value memory area without having to issue a helper call which needs to prepare registers according to calling convention, etc, without needing the extra NULL test, and without having to add the offset in an additional instruction to the value base pointer. The verifier then treats the destination register as PTR_TO_MAP_VALUE with constant reg->off from the user passed offset from the second imm field, and guarantees that this is within bounds of the map value. Any subsequent operations are normally treated as typical map value handling without anything else needed for verification.
The two map operations for direct value access have been added to array map for now. In future other types could be supported as well depending on the use case. The main use case for this commit is to allow for BPF loader support for global variables that reside in .data/.rodata/.bss sections such that we can directly load the address of them with minimal additional infrastructure required. Loader support has been added in subsequent commits for libbpf library.
--- include/linux/bpf.h | 6 +++ include/linux/bpf_verifier.h | 4 ++ include/uapi/linux/bpf.h | 13 ++++- kernel/bpf/arraymap.c | 29 ++++++++++ kernel/bpf/core.c | 3 +- kernel/bpf/disasm.c | 5 +- kernel/bpf/syscall.c | 31 ++++++++--- kernel/bpf/verifier.c | 109 ++++++++++++++++++++++++++++---------- tools/bpf/bpftool/xlated_dumper.c | 6 +++ 9 files changed, 168 insertions(+), 38 deletions(-)
include/linux/bpf.h +6 -0
@@ -57,6 +57,12 @@ struct bpf_map_ops {const struct btf *btf, const struct btf_type *key_type, const struct btf_type *value_type); + +/* Direct value access helpers. */ +int (*map_direct_value_addr)(const struct bpf_map *map, +u64 *imm,u32 idx,u32 off); +int (*map_direct_value_meta)(const struct bpf_map *map, +u64 imm,u32 *idx,u32 *off); };struct bpf_map {
include/linux/bpf_verifier.h +4 -0
@@ -223,6 +223,10 @@ struct bpf_insn_aux_data {unsigned long map_state;/* pointer/poison value for maps */ s32 call_imm;/* saved imm field of call insn */ u32 alu_limit;/* limit for add/sub register with pointer */ +struct { +u32 map_index;/* index into used_maps[] */ +u32 map_off;/* offset from value base address */ + }; };int ctx_field_size;/* the ctx field size for load insn, maybe 0 */ int sanitize_stack_off;/* stack slot to be cleared */
include/uapi/linux/bpf.h +12 -1
@@ -255,8 +255,19 @@ enum bpf_attach_type {*/ #define BPF_F_ANY_ALIGNMENT (1U << 1) -/* when bpf_ldimm64->src_reg == BPF_PSEUDO_MAP_FD, bpf_ldimm64->imm == fd */ +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have +* two extensions: +* +* insn[0].src_reg: BPF_PSEUDO_MAP_FD BPF_PSEUDO_MAP_VALUE +* insn[0].imm: map fd map fd +* insn[1].imm: 0 offset into value +* insn[0].off: 0 lower 16 bit of map index +* insn[1].off: 0 higher 16 bit of map index +* ldimm64 rewrite: address of map address of map[index]+offset +* verifier type: CONST_PTR_TO_MAP PTR_TO_MAP_VALUE +*/ #define BPF_PSEUDO_MAP_FD 1 +#define BPF_PSEUDO_MAP_VALUE 2/* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative * offset to another bpf function
kernel/bpf/arraymap.c +29 -0
@@ -160,6 +160,33 @@ static void *array_map_lookup_elem(struct bpf_map *map, void *key)return array->value + array->elem_size * (index & array->index_mask); } +static int array_map_direct_value_addr (const struct bpf_map *map,u64 *imm, +u32 index,u32 off) +{ + struct bpf_array *array = container_of(map, struct bpf_array, map); + +if (index >= map->max_entries || off >= map->value_size) +return -EINVAL ; + *imm = (unsigned long )(array->value + + array->elem_size * (index & array->index_mask)); +return 0 ; +} + +static int array_map_direct_value_meta (const struct bpf_map *map,u64 imm, +u32 *index,u32 *off) +{ + struct bpf_array *array = container_of(map, struct bpf_array, map); +u64 rem, base = (unsigned long )array->value, slot = array->elem_size; +u64 range = slot * map->max_entries; + +if (imm < base || imm >= base + range) +return -ENOENT ; + base = imm - base; + *index =div64_u64_rem (base, slot, &rem); + *off = rem; +return 0 ; +} +/* emit BPF instructions equivalent to C code of array_map_lookup_elem() */ static u32 array_map_gen_lookup (struct bpf_map *map,struct bpf_insn *insn_buf) {
@@ -419,6 +446,8 @@ const struct bpf_map_ops array_map_ops = { .map_update_elem = array_map_update_elem, .map_delete_elem = array_map_delete_elem, .map_gen_lookup = array_map_gen_lookup, + .map_direct_value_addr = array_map_direct_value_addr, + .map_direct_value_meta = array_map_direct_value_meta, .map_seq_show_elem = array_map_seq_show_elem, .map_check_btf = array_map_check_btf, };
kernel/bpf/core.c +2 -1
@@ -292,7 +292,8 @@ int bpf_prog_calc_tag(struct bpf_prog *fp) dst[i] = fp->insnsi[i];if (!was_ld_map && dst[i].code == (BPF_LD |BPF_IMM |BPF_DW ) && - dst[i].src_reg ==BPF_PSEUDO_MAP_FD ) { + (dst[i].src_reg ==BPF_PSEUDO_MAP_FD || + dst[i].src_reg ==BPF_PSEUDO_MAP_VALUE )) { was_ld_map = true; dst[i].imm =0 ; } else if (was_ld_map &&
kernel/bpf/disasm.c +3 -2
@@ -205,10 +205,11 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,* part of the ldimm64 insn is accessible. */ u64 imm = ((u64)(insn +1 )->imm <<32 ) | (u32 )insn->imm; -bool map_ptr = insn->src_reg ==BPF_PSEUDO_MAP_FD ; +bool is_ptr = insn->src_reg ==BPF_PSEUDO_MAP_FD || + insn->src_reg ==BPF_PSEUDO_MAP_VALUE ;char tmp[64 ]; -if (map_ptr && !allow_ptr_leaks) +if (is_ptr && !allow_ptr_leaks) imm =0 ; verbose(cbs->private_data, "(%02x) r%d = %s\n",
kernel/bpf/syscall.c +24 -7
@@ -2071,13 +2071,27 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) }static const struct bpf_map *bpf_map_from_imm (const struct bpf_prog *prog, -unsigned long addr) +unsigned long addr,u32 *idx, +u32 *off,u32 *type) { +const struct bpf_map *map;int i; -for (i =0 ; i < prog->aux->used_map_cnt; i++) -if (prog->aux->used_maps[i] == (void *)addr) -return prog->aux->used_maps[i]; + *off = *idx =0 ; +for (i =0 ; i < prog->aux->used_map_cnt; i++) { + map = prog->aux->used_maps[i]; +if (map == (void *)addr) { + *type =BPF_PSEUDO_MAP_FD ; +return map; + } +if (!map->ops->map_direct_value_meta) +continue ; +if (!map->ops->map_direct_value_meta (map, addr, idx, off)) { + *type =BPF_PSEUDO_MAP_VALUE ; +return map; + } + } +return NULL ; }
@@ -2085,6 +2099,7 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog) {const struct bpf_map *map;struct bpf_insn *insns; +u32 idx, off, type;u64 imm;int i;
@@ -2112,11 +2127,13 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog)continue ; imm = ((u64 )insns[i +1 ].imm <<32 ) | (u32 )insns[i].imm; - map =bpf_map_from_imm (prog, imm); + map =bpf_map_from_imm (prog, imm, &idx, &off, &type);if (map) { - insns[i].src_reg =BPF_PSEUDO_MAP_FD ; + insns[i].src_reg = type; insns[i].imm = map->id; - insns[i +1 ].imm =0 ; + insns[i].off = (u16 )idx; + insns[i +1 ].imm = off; + insns[i +1 ].off = (u16)(idx >>16 );continue ; } }
kernel/bpf/verifier.c +82 -27
172 lines not on this page · raw message
tools/bpf/bpftool/xlated_dumper.c +6 -0
@@ -195,6 +195,12 @@ static const char *print_imm(void *private_data,if (insn->src_reg ==BPF_PSEUDO_MAP_FD )snprintf (dd->scratch_buff,sizeof (dd->scratch_buff),"map[id:%u]" , insn->imm); +else if (insn->src_reg ==BPF_PSEUDO_MAP_VALUE ) +snprintf (dd->scratch_buff,sizeof (dd->scratch_buff), +"map[id:%u][%u]+%u" , insn->imm, + ((__u32 )(__u16 )insn[0 ].off) | + ((__u32 )(__u16 )insn[1 ].off) <<16 , + (insn +1 )->imm);else snprintf (dd->scratch_buff,sizeof (dd->scratch_buff),"0x%llx" , (unsigned long long )full_imm);
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 2/15 bpf: add program side {rd,wr}only support for maps
In reply to Daniel BorkmannThis work adds two new map creation flags BPF_F_RDONLY_PROG and BPF_F_WRONLY_PROG in order to allow for read-only or write-only BPF maps from a BPF program side.
Today we have BPF_F_RDONLY and BPF_F_WRONLY, but this only applies to system call side, meaning the BPF program has full read/write access to the map as usual while bpf(2) calls with map fd can either only read or write into the map depending on the flags. BPF_F_RDONLY_PROG and BPF_F_WRONLY_PROG allows for the exact opposite such that verifier is going to reject program loads if write into a read-only map or a read into a write-only map is detected. For read-only map case also some helpers are forbidden for programs that would alter the map state such as map deletion, update, etc.
We've enabled this generic map extension to various non-special maps holding normal user data: array, hash, lru, lpm, local storage, queue and stack. Further map types could be followed up in future depending on use-case. Main use case here is to forbid writes into .rodata map values from verifier side.
--- include/linux/bpf.h | 24 ++++++++++++++++++++++ include/uapi/linux/bpf.h | 10 +++++++++- kernel/bpf/arraymap.c | 3 ++- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 6 +++--- kernel/bpf/lpm_trie.c | 3 ++- kernel/bpf/queue_stack_maps.c | 6 +++--- kernel/bpf/syscall.c | 2 ++ kernel/bpf/verifier.c | 46 +++++++++++++++++++++++++++++++++++++++++-- 9 files changed, 92 insertions(+), 14 deletions(-)
include/linux/bpf.h +24 -0
@@ -427,6 +427,30 @@ struct bpf_array { }; }; +#define BPF_MAP_CAN_READ BIT(0) +#define BPF_MAP_CAN_WRITE BIT(1) + +static inline u32 bpf_map_flags_to_cap (struct bpf_map *map) +{ +u32 access_flags = map->map_flags & (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG ); + +/* Combination of BPF_F_RDONLY_PROG | BPF_F_WRONLY_PROG is +* not possible. +*/ +if (access_flags &BPF_F_RDONLY_PROG ) +return BPF_MAP_CAN_READ ; +else if (access_flags &BPF_F_WRONLY_PROG ) +return BPF_MAP_CAN_WRITE ; +else +return BPF_MAP_CAN_READ |BPF_MAP_CAN_WRITE ; +} + +static inline bool bpf_map_flags_access_ok (u32 access_flags) +{ +return (access_flags & (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG )) != + (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG ); +} +#define MAX_TAIL_CALL_CNT 32struct bpf_event_entry {
include/uapi/linux/bpf.h +9 -1
@@ -294,7 +294,7 @@ enum bpf_attach_type {#define BPF_OBJ_NAME_LEN 16U -/* Flags for accessing BPF object */ +/* Flags for accessing BPF object from syscall side. */ #define BPF_F_RDONLY (1U << 3)#define BPF_F_WRONLY (1U << 4)
@@ -304,6 +304,14 @@ enum bpf_attach_type {/* Zero-initialize hash function seed. This should only be used for testing. */ #define BPF_F_ZERO_SEED (1U << 6) +/* Flags for accessing BPF object from program side. */ +#define BPF_F_RDONLY_PROG (1U << 7) +#define BPF_F_WRONLY_PROG (1U << 8) +#define BPF_F_ACCESS_MASK (BPF_F_RDONLY | \ + BPF_F_RDONLY_PROG | \ + BPF_F_WRONLY | \ + BPF_F_WRONLY_PROG) +/* flags for BPF_PROG_QUERY */ #define BPF_F_QUERY_EFFECTIVE (1U << 0)
kernel/bpf/arraymap.c +2 -1
@@ -22,7 +22,7 @@#include "map_in_map.h" #define ARRAY_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK)static void bpf_array_free_percpu (struct bpf_array *array) {
@@ -63,6 +63,7 @@ int array_map_alloc_check(union bpf_attr *attr)if (attr->max_entries ==0 || attr->key_size !=4 || attr->value_size ==0 || attr->map_flags & ~ARRAY_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags) || (percpu && numa_node !=NUMA_NO_NODE ))return -EINVAL ;
kernel/bpf/hashtab.c +3 -3
@@ -23,7 +23,7 @@#define HTAB_CREATE_FLAG_MASK \ (BPF_F_NO_PREALLOC | BPF_F_NO_COMMON_LRU | BPF_F_NUMA_NODE | \ - BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_ZERO_SEED) + BPF_F_ACCESS_MASK | BPF_F_ZERO_SEED)struct bucket {struct hlist_nulls_head head;
@@ -262,8 +262,8 @@ static int htab_map_alloc_check(union bpf_attr *attr)/* Guard against local DoS, and discourage production use. */ return -EPERM ; -if (attr->map_flags & ~HTAB_CREATE_FLAG_MASK ) -/* reserved bits should not be used */ +if (attr->map_flags & ~HTAB_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags))return -EINVAL ;if (!lru && percpu_lru)
kernel/bpf/local_storage.c +3 -3
@@ -14,7 +14,7 @@ DEFINE_PER_CPU(struct bpf_cgroup_storage*, bpf_cgroup_storage[MAX_BPF_CGROUP_STO #ifdef CONFIG_CGROUP_BPF #define LOCAL_STORAGE_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK) struct bpf_cgroup_storage_map { struct bpf_map map;
@@ -282,8 +282,8 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr)if (attr->value_size >PAGE_SIZE )return ERR_PTR (-E2BIG ); -if (attr->map_flags & ~LOCAL_STORAGE_CREATE_FLAG_MASK ) -/* reserved bits should not be used */ +if (attr->map_flags & ~LOCAL_STORAGE_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags))return ERR_PTR (-EINVAL );if (attr->max_entries)
kernel/bpf/lpm_trie.c +2 -1
@@ -538,7 +538,7 @@ static int trie_delete_elem(struct bpf_map *map, void *_key)#define LPM_KEY_SIZE_MIN LPM_KEY_SIZE(LPM_DATA_SIZE_MIN)#define LPM_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_NUMA_NODE | \ - BPF_F_RDONLY | BPF_F_WRONLY) + BPF_F_ACCESS_MASK)static struct bpf_map *trie_alloc (union bpf_attr *attr) {
@@ -553,6 +553,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) if (attr->max_entries == 0 || !(attr->map_flags & BPF_F_NO_PREALLOC) || attr->map_flags & ~LPM_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok(attr->map_flags) || attr->key_size < LPM_KEY_SIZE_MIN || attr->key_size > LPM_KEY_SIZE_MAX || attr->value_size < LPM_VAL_SIZE_MIN ||
kernel/bpf/queue_stack_maps.c +3 -3
@@ -11,8 +11,7 @@#include "percpu_freelist.h" #define QUEUE_STACK_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) - + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK)struct bpf_queue_stack {struct bpf_map map;
@@ -52,7 +51,8 @@ static int queue_stack_map_alloc_check(union bpf_attr *attr)/* check sanity of attributes */ if (attr->max_entries ==0 || attr->key_size !=0 || attr->value_size ==0 || - attr->map_flags & ~QUEUE_STACK_CREATE_FLAG_MASK ) + attr->map_flags & ~QUEUE_STACK_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags))return -EINVAL ;if (attr->value_size >KMALLOC_MAX_SIZE )
kernel/bpf/syscall.c +2 -0
@@ -489,6 +489,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, map->spin_lock_off =btf_find_spin_lock (btf, value_type);if (map_value_has_spin_lock (map)) { +if (map->map_flags &BPF_F_RDONLY_PROG ) +return -EACCES ;if (map->map_type !=BPF_MAP_TYPE_HASH && map->map_type !=BPF_MAP_TYPE_ARRAY && map->map_type !=BPF_MAP_TYPE_CGROUP_STORAGE )
kernel/bpf/verifier.c +44 -2
80 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 3/15 bpf: add syscall side map lock support
In reply to Daniel BorkmannThis patch adds a new BPF_MAP_LOCK command which allows to lock the map globally as read-only/immutable from syscall side. Map permission handling has been refactored into map_get_sys_perms() and drops FMODE_CAN_WRITE in case of locked map. Main use case is to allow for setting up .rodata sections from the BPF ELF which are loaded into the kernel, meaning BPF loader first allocates map, sets up map value by copying .rodata section into it and once complete, it calls BPF_MAP_LOCK on the map fd to prevent further modifications. Given maps can be shared, we only grant the original creator of the map the ability to lock it as syscall-side read-only or only priviledged users otherwise.
Right now BPF_MAP_LOCK only takes map fd as argument while remaining bpf_attr members are required to be zero. I didn't add write-only locking here as counterpart since I don't have a concrete use-case for it on my side, and I think it makes probably more sense to wait once there is actually one. In that case bpf_attr can be extended as usual with a flag field and/or others where flag 0 means that we lock the map read-only hence this doesn't prevent to add further extensions to BPF_MAP_LOCK upon need.
--- include/linux/bpf.h | 5 +++- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 72 ++++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 65 insertions(+), 13 deletions(-)
include/linux/bpf.h +4 -1
@@ -87,7 +87,10 @@ struct bpf_map {struct btf *btf;u32 pages;bool unpriv_array; -/* 51 bytes hole */ +/* Next two members are write-once. */ +bool sys_immutable; +struct task_struct *creator; +/* 40 bytes hole */ /* The 3rd and 4th cacheline with misc members to avoid false sharing * particularly with refcounting.
include/uapi/linux/bpf.h +1 -0
@@ -105,6 +105,7 @@ enum bpf_cmd {BPF_BTF_GET_FD_BY_ID ,BPF_TASK_FD_QUERY ,BPF_MAP_LOOKUP_AND_DELETE_ELEM , + BPF_MAP_LOCK, }; enum bpf_map_type {
kernel/bpf/syscall.c +60 -12
144 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 4/15 bpf: allow . char as part of the object name
In reply to Daniel BorkmannTrivial addition to allow '.' aside from '_' as "special" characters in the object name. Used to allow for substrings in maps from loader side such as ".bss", ".data", ".rodata", but could also be useful for other purposes.
--- kernel/bpf/syscall.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/bpf/syscall.c +3 -3
@@ -464,10 +464,10 @@ static int bpf_obj_name_cpy(char *dst, const char *src)const char *end = src +BPF_OBJ_NAME_LEN ;memset (dst,0 ,BPF_OBJ_NAME_LEN ); - -/* Copy all isalnum() and '_' char */ +/* Copy all isalnum(), '_' and '.' chars. */ while (src < end && *src) { -if (!isalnum (*src) && *src !='_' ) +if (!isalnum (*src) && + *src !='_' && *src !='.' )return -EINVAL ; *dst++ = *src++; }
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 5/15 bpf: add specification for BTF Var and DataSec kinds
In reply to Daniel BorkmannThis adds the BTF specification and UAPI bits for supporting BTF Var and DataSec kinds. This is following LLVM upstream commit ac4082b77e07 ("[BPF] Add BTF Var and DataSec Support") which has been merged recently. Var itself is for describing a global variable and DataSec to describe ELF sections e.g. data/bss/rodata sections that hold one or multiple global variables.
--- Documentation/bpf/btf.rst | 57 +++++++++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/btf.h | 32 ++++++++++++++++++++++---- 2 files changed, 85 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +57 -0
@@ -82,6 +82,8 @@ sequentially and type id is assigned to each recognized type starting from id #define BTF_KIND_RESTRICT 11 /* Restrict */ #define BTF_KIND_FUNC 12 /* Function */ #define BTF_KIND_FUNC_PROTO 13 /* Function Proto */ + #define BTF_KIND_VAR 14 /* Variable */ + #define BTF_KIND_DATASEC 15 /* Section */ Note that the type section encodes debug info, not just pure types. ``BTF_KIND_FUNC`` is not a type, and it represents a defined subprogram.
@@ -393,6 +395,61 @@ refers to parameter type. If the function has variable arguments, the last parameter is encoded with ``name_off = 0`` and ``type = 0``. +2.2.14 BTF_KIND_VAR +~~~~~~~~~~~~~~~~~~~ + +``struct btf_type`` encoding requirement: + * ``name_off``: offset to a valid C identifier + * ``info.kind_flag``: 0 + * ``info.kind``: BTF_KIND_VAR + * ``info.vlen``: 0 + * ``type``: the type of the variable + +``btf_type`` is followed by a single ``struct btf_variable`` with the +following data:: + + struct btf_var { + __u32 linkage; + }; + +``struct btf_var`` encoding: + * ``linkage``: currently only static variable 0, or globally allocated + variable in ELF sections 1 + +Not all type of global variables are supported by LLVM at this point. +The following is currently available: + + * static variables with or without section attributes + * global variables with section attributes + +The latter is for future extraction of map key/value type id's from a +map definition. + +2.2.15 BTF_KIND_DATASEC +~~~~~~~~~~~~~~~~~~~~~~~ + +``struct btf_type`` encoding requirement: + * ``name_off``: offset to a valid name associated with a variable or + one of .data/.bss/.rodata + * ``info.kind_flag``: 0 + * ``info.kind``: BTF_KIND_DATASEC + * ``info.vlen``: # of variables + * ``size``: total section size in bytes (0 at compilation time, patched + to actual size by BPF loaders such as libbpf) + +``btf_type`` is followed by ``info.vlen`` number of ``struct btf_var_secinfo``.:: + + struct btf_var_secinfo { + __u32 type; + __u32 offset; + __u32 size; + }; + +``struct btf_var_secinfo`` encoding: + * ``type``: the type of the BTF_KIND_VAR variable + * ``offset``: the in-section offset of the variable + * ``size``: the size of the variable in bytes + 3. BTF Kernel API *****************
include/uapi/linux/btf.h +28 -4
@@ -39,11 +39,11 @@ struct btf_type {* struct, union and fwd */ __u32 info; -/* "size" is used by INT, ENUM, STRUCT and UNION. +/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC. * "size" tells the size of the type it is describing. * * "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT, -* FUNC and FUNC_PROTO. +* FUNC, FUNC_PROTO and VAR. * "type" is a type_id referring to another type. */ union {
@@ -70,8 +70,10 @@ struct btf_type {#define BTF_KIND_RESTRICT 11/* Restrict */ #define BTF_KIND_FUNC 12/* Function */ #define BTF_KIND_FUNC_PROTO 13/* Function Proto */ -#define BTF_KIND_MAX 13 -#define NR_BTF_KINDS 14 +#define BTF_KIND_VAR 14/* Variable */ +#define BTF_KIND_DATASEC 15/* Section */ +#define BTF_KIND_MAX BTF_KIND_DATASEC +#define NR_BTF_KINDS (BTF_KIND_MAX + 1)/* For some specific BTF_KIND, "struct btf_type" is immediately * followed by extra data.
@@ -138,4 +140,26 @@ struct btf_param { __u32 type; }; +enum { +BTF_VAR_STATIC =0 , +BTF_VAR_GLOBAL_ALLOCATED , +}; + +/* BTF_KIND_VAR is followed by a single "struct btf_var" to describe +* additional information related to the variable such as its linkage. +*/ +struct btf_var { +__u32 linkage; +}; + +/* BTF_KIND_DATASEC is followed by multiple "struct btf_var_secinfo" +* to describe all BTF_KIND_VAR types it contains along with it's +* in-section offset as well as size. +*/ +struct btf_var_secinfo { +__u32 type; +__u32 offset; +__u32 size; +}; +#endif /* _UAPI__LINUX_BTF_H__ */
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 6/15 bpf: kernel side support for BTF Var and DataSec
In reply to Daniel BorkmannThis work adds kernel-side verification, logging and seq_show dumping of BTF Var and DataSec kinds which are emitted with latest LLVM. The following constraints apply:
BTF Var must have:
- Its kind_flag is 0
- Its vlen is 0
- Must point to a valid type
- Type must not resolve to a forward type
- Must have a valid name
- Name may include dots (e.g. in case of static variables inside functions)
- Cannot be a member of a struct/union
- Linkage so far can either only be static or global/allocated
BTF DataSec must have:
- Its kind_flag is 0
- Its vlen cannot be 0
- Its size cannot be 0
- Must have a valid name
- Name may include dots (e.g. to represent .bss, .data, .rodata etc)
- Cannot be a member of a struct/union
- Inner btf_var_secinfo array with {type,offset,size} triple must be sorted by offset in ascending order
- Type must always point to BTF Var
- BTF resolved size of Var must be <= size provided by triple
- DataSec size must be >= sum of triple sizes (thus holes are allowed)
--- kernel/bpf/btf.c | 393 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 379 insertions(+), 14 deletions(-)
kernel/bpf/btf.c +379 -14
@@ -185,6 +185,16 @@ i <btf_type_vlen (struct_type); \ i++, member++) +#define for_each_vsi (i, struct_type, member) \ + for (i = 0, member = btf_type_var_secinfo(struct_type); \ + i < btf_type_vlen(struct_type); \ + i++, member++) + +#define for_each_vsi_from (i, from, struct_type, member) \ + for (i = from, member = btf_type_var_secinfo(struct_type) + from; \ + i < btf_type_vlen(struct_type); \ + i++, member++) +static DEFINE_IDR (btf_idr);static DEFINE_SPINLOCK (btf_idr_lock);
@@ -262,6 +272,8 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_RESTRICT] = "RESTRICT", [BTF_KIND_FUNC] = "FUNC", [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO", + [BTF_KIND_VAR] = "VAR", + [BTF_KIND_DATASEC] = "DATASEC", };struct btf_kind_operations {
@@ -308,6 +320,7 @@ static bool btf_type_is_modifier(const struct btf_type *t)case BTF_KIND_VOLATILE :case BTF_KIND_CONST :case BTF_KIND_RESTRICT : +case BTF_KIND_VAR :return true; }
@@ -375,13 +388,27 @@ static bool btf_type_is_int(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_INT ; } +static bool btf_type_is_var (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_VAR ; +} + +static bool btf_type_is_datasec (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ; +} +/* What types need to be resolved? * * btf_type_is_modifier() is an obvious one. * * btf_type_is_struct() because its member refers to * another type (through member->type). - +* +* btf_type_is_var() because the variable refers to +* another type. btf_type_is_datasec() holds multiple +* btf_type_is_var() types that need resolving. +* * btf_type_is_array() because its element (array->type) * refers to another type. Array can be thought of a * special case of struct while array just has the same
@@ -390,9 +417,11 @@ static bool btf_type_is_int(const struct btf_type *t)static bool btf_type_needs_resolve (const struct btf_type *t) {return btf_type_is_modifier (t) || -btf_type_is_ptr (t) || -btf_type_is_struct (t) || -btf_type_is_array (t); +btf_type_is_ptr (t) || +btf_type_is_struct (t) || +btf_type_is_array (t) || +btf_type_is_var (t) || +btf_type_is_datasec (t); }/* t->size can be used */
@@ -403,6 +432,7 @@ static bool btf_type_has_size(const struct btf_type *t)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC :return true; }
@@ -467,6 +497,16 @@ static const struct btf_enum *btf_type_enum(const struct btf_type *t)return (const struct btf_enum *)(t +1 ); } +static const struct btf_var *btf_type_var (const struct btf_type *t) +{ +return (const struct btf_var *)(t +1 ); +} + +static const struct btf_var_secinfo *btf_type_var_secinfo (const struct btf_type *t) +{ +return (const struct btf_var_secinfo *)(t +1 ); +} +static const struct btf_kind_operations *btf_type_ops (const struct btf_type *t) {return kind_ops[BTF_INFO_KIND (t->info)];
@@ -478,23 +518,31 @@ static bool btf_name_offset_valid(const struct btf *btf, u32 offset) offset < btf->hdr.str_len; } -/* Only C-style identifier is permitted. This can be relaxed if -* necessary. -*/ -static bool btf_name_valid_identifier (const struct btf *btf,u32 offset) +static bool __btf_name_char_ok (char c,bool first,bool dot_ok) +{ +if ((first ? !isalpha (c) : + !isalnum (c)) && + c !='_' && + ((c =='.' && !dot_ok) || + c !='.' )) +return false; +return true; +} + +static bool __btf_name_valid (const struct btf *btf,u32 offset,bool dot_ok) {/* offset must be valid */ const char *src = &btf->strings[offset];const char *src_limit; -if (!isalpha (*src) && *src !='_' ) +if (!__btf_name_char_ok (*src, true, dot_ok))return false;/* set a limit on identifier length */ src_limit = src +KSYM_NAME_LEN ; src++;while (*src && src < src_limit) { -if (!isalnum (*src) && *src !='_' ) +if (!__btf_name_char_ok (*src, false, dot_ok))return false; src++; }
@@ -502,6 +550,19 @@ static bool btf_name_valid_identifier(const struct btf *btf, u32 offset)return !*src; } +/* Only C-style identifier is permitted. This can be relaxed if +* necessary. +*/ +static bool btf_name_valid_identifier (const struct btf *btf,u32 offset) +{ +return __btf_name_valid (btf, offset, false); +} + +static bool btf_name_valid_section (const struct btf *btf,u32 offset) +{ +return __btf_name_valid (btf, offset, true); +} +static const char *__btf_name_by_offset (const struct btf *btf,u32 offset) {if (!offset)
@@ -697,6 +758,32 @@ static void btf_verifier_log_member(struct btf_verifier_env *env,__btf_verifier_log (log,"\n" ); } +__printf(4, 5) +static void btf_verifier_log_vsi (struct btf_verifier_env *env, +const struct btf_type *datasec_type, +const struct btf_var_secinfo *vsi, +const char *fmt, ...) +{ +struct bpf_verifier_log *log = &env->log; +va_list args; + +if (!bpf_verifier_log_needed (log)) +return ; +if (env->phase !=CHECK_META ) +btf_verifier_log_type (env, datasec_type,NULL ); + +__btf_verifier_log (log,"\t type_id=%u offset=%u size=%u" , + vsi->type, vsi->offset, vsi->size); +if (fmt && *fmt) { +__btf_verifier_log (log," " ); +va_start (args, fmt); +bpf_verifier_vlog (log, fmt, args); +va_end (args); + } + +__btf_verifier_log (log,"\n" ); +} +static void btf_verifier_log_hdr (struct btf_verifier_env *env,u32 btf_data_size) {
@@ -1542,6 +1629,53 @@ static int btf_modifier_resolve(struct btf_verifier_env *env,return 0 ; } +static int btf_var_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_type *next_type; +const struct btf_type *t = v->t; +u32 next_type_id = t->type; +struct btf *btf = env->btf; +u32 next_type_size; + + next_type =btf_type_by_id (btf, next_type_id); +if (!next_type) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +if (!env_type_is_resolve_sink (env, next_type) && + !env_type_is_resolved (env, next_type_id)) +return env_stack_push (env, next_type, next_type_id); + +if (btf_type_is_modifier (next_type)) { +const struct btf_type *resolved_type; +u32 resolved_type_id; + + resolved_type_id = next_type_id; + resolved_type =btf_type_id_resolve (btf, &resolved_type_id); + +if (btf_type_is_ptr (resolved_type) && + !env_type_is_resolve_sink (env, resolved_type) && + !env_type_is_resolved (env, resolved_type_id)) +return env_stack_push (env, resolved_type, + resolved_type_id); + } + +/* We must resolve to something concrete at this point, no +* forward types or similar that would resolve to size of +* zero is allowed. +*/ +if (!btf_type_id_size (btf, &next_type_id, &next_type_size)) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +env_stack_pop_resolved (env, next_type_id, next_type_size); + +return 0 ; +} +static int btf_ptr_resolve (struct btf_verifier_env *env,const struct resolve_vertex *v) {
@@ -1609,6 +1743,15 @@ static void btf_modifier_seq_show(const struct btf *btf,btf_type_ops (t)->seq_show (btf, t, type_id, data, bits_offset, m); } +static void btf_var_seq_show (const struct btf *btf,const struct btf_type *t, +u32 type_id,void *data,u8 bits_offset, +struct seq_file *m) +{ + t =btf_type_id_resolve (btf, &type_id); + +btf_type_ops (t)->seq_show (btf, t, type_id, data, bits_offset, m); +} +static void btf_ptr_seq_show (const struct btf *btf,const struct btf_type *t,u32 type_id,void *data,u8 bits_offset,struct seq_file *m)
@@ -2411,6 +2554,222 @@ static struct btf_kind_operations func_ops = { .seq_show = btf_df_seq_show, }; +static s32 btf_var_check_meta (struct btf_verifier_env *env, +const struct btf_type *t, +u32 meta_left) +{ +const struct btf_var *var; +u32 meta_needed =sizeof (*var); + +if (meta_left < meta_needed) { +btf_verifier_log_basic (env, t, +"meta_left:%u meta_needed:%u" , + meta_left, meta_needed); +return -EINVAL ; + } + +if (btf_type_vlen (t)) { +btf_verifier_log_type (env, t,"vlen != 0" ); +return -EINVAL ; + } + +if (btf_type_kflag (t)) { +btf_verifier_log_type (env, t,"Invalid btf_info kind_flag" ); +return -EINVAL ; + } + +if (!t->name_off || + !__btf_name_valid (env->btf, t->name_off, true)) { +btf_verifier_log_type (env, t,"Invalid name" ); +return -EINVAL ; + } + +/* A var cannot be in type void */ +if (!t->type || !BTF_TYPE_ID_VALID (t->type)) { +btf_verifier_log_type (env, t,"Invalid type_id" ); +return -EINVAL ; + } + + var =btf_type_var (t); +if (var->linkage !=BTF_VAR_STATIC && + var->linkage !=BTF_VAR_GLOBAL_ALLOCATED ) { +btf_verifier_log_type (env, t,"Linkage not supported" ); +return -EINVAL ; + } + +btf_verifier_log_type (env, t,NULL ); + +return meta_needed; +} + +static void btf_var_log (struct btf_verifier_env *env,const struct btf_type *t) +{ +const struct btf_var *var =btf_type_var (t); + +btf_verifier_log (env,"type_id=%u linkage=%u" , t->type, var->linkage); +} + +static const struct btf_kind_operations var_ops = { + .check_meta = btf_var_check_meta, + .resolve = btf_var_resolve, + .check_member = btf_df_check_member, + .check_kflag_member = btf_df_check_kflag_member, + .log_details = btf_var_log, + .seq_show = btf_var_seq_show, +}; + +static s32 btf_datasec_check_meta (struct btf_verifier_env *env, +const struct btf_type *t, +u32 meta_left) +{ +const struct btf_var_secinfo *vsi; +u64 last_vsi_end_off =0 , sum =0 ; +u32 i, meta_needed; + + meta_needed =btf_type_vlen (t) *sizeof (*vsi); +if (meta_left < meta_needed) { +btf_verifier_log_basic (env, t, +"meta_left:%u meta_needed:%u" , + meta_left, meta_needed); +return -EINVAL ; + } + +if (!btf_type_vlen (t)) { +btf_verifier_log_type (env, t,"vlen == 0" ); +return -EINVAL ; + } + +if (!t->size) { +btf_verifier_log_type (env, t,"size == 0" ); +return -EINVAL ; + } + +if (btf_type_kflag (t)) { +btf_verifier_log_type (env, t,"Invalid btf_info kind_flag" ); +return -EINVAL ; + } + +if (!t->name_off || + !btf_name_valid_section (env->btf, t->name_off)) { +btf_verifier_log_type (env, t,"Invalid name" ); +return -EINVAL ; + } + +btf_verifier_log_type (env, t,NULL ); + +for_each_vsi (i, t, vsi) { +/* A var cannot be in type void */ +if (!vsi->type || !BTF_TYPE_ID_VALID (vsi->type)) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid type_id" ); +return -EINVAL ; + } + +if (vsi->offset < last_vsi_end_off || vsi->offset >= t->size) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid offset" ); +return -EINVAL ; + } + +if (!vsi->size || vsi->size > t->size) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid size" ); +return -EINVAL ; + } + + last_vsi_end_off = vsi->offset + vsi->size; +if (last_vsi_end_off > t->size) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid offset+size" ); +return -EINVAL ; + } + +btf_verifier_log_vsi (env, t, vsi,NULL ); + sum += vsi->size; + } + +if (t->size < sum) { +btf_verifier_log_type (env, t,"Invalid btf_info size" ); +return -EINVAL ; + } + +return meta_needed; +} + +static int btf_datasec_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_var_secinfo *vsi; +struct btf *btf = env->btf; +u16 i; + +for_each_vsi_from (i, v->next_member, v->t, vsi) { +u32 var_type_id = vsi->type, type_id, type_size =0 ; +const struct btf_type *var_type =btf_type_by_id (env->btf, + var_type_id); +if (!var_type || !btf_type_is_var (var_type)) { +btf_verifier_log_vsi (env, v->t, vsi, +"Not a VAR kind member" ); +return -EINVAL ; + } + +if (!env_type_is_resolve_sink (env, var_type) && + !env_type_is_resolved (env, var_type_id)) { +env_stack_set_next_member (env, i +1 ); +return env_stack_push (env, var_type, var_type_id); + } + + type_id = var_type->type; +if (!btf_type_id_size (btf, &type_id, &type_size)) { +btf_verifier_log_vsi (env, v->t, vsi,"Invalid type" ); +return -EINVAL ; + } + +if (vsi->size < type_size) { +btf_verifier_log_vsi (env, v->t, vsi,"Invalid size" ); +return -EINVAL ; + } + } + +env_stack_pop_resolved (env,0 ,0 ); +return 0 ; +} + +static void btf_datasec_log (struct btf_verifier_env *env, +const struct btf_type *t) +{ +btf_verifier_log (env,"size=%u vlen=%u" , t->size,btf_type_vlen (t)); +} + +static void btf_datasec_seq_show (const struct btf *btf, +const struct btf_type *t,u32 type_id, +void *data,u8 bits_offset, +struct seq_file *m) +{ +const struct btf_var_secinfo *vsi; +const struct btf_type *var; +u32 i; + +seq_printf (m,"section (\"%s\") = {" ,__btf_name_by_offset (btf, t->name_off)); +for_each_vsi (i, t, vsi) { + var =btf_type_by_id (btf, vsi->type); +if (i) +seq_puts (m,"," ); +btf_type_ops (var)->seq_show (btf, var, vsi->type, + data + vsi->offset, bits_offset, m); + } +seq_puts (m,"}" ); +} + +static const struct btf_kind_operations datasec_ops = { + .check_meta = btf_datasec_check_meta, + .resolve = btf_datasec_resolve, + .check_member = btf_df_check_member, + .check_kflag_member = btf_df_check_kflag_member, + .log_details = btf_datasec_log, + .seq_show = btf_datasec_seq_show, +}; +static int btf_func_proto_check (struct btf_verifier_env *env,const struct btf_type *t) {
@@ -2542,6 +2901,8 @@ static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = { [BTF_KIND_RESTRICT] = &modifier_ops, [BTF_KIND_FUNC] = &func_ops, [BTF_KIND_FUNC_PROTO] = &func_proto_ops, + [BTF_KIND_VAR] = &var_ops, + [BTF_KIND_DATASEC] = &datasec_ops, }; static s32 btf_check_meta(struct btf_verifier_env *env,
@@ -2622,13 +2983,17 @@ static bool btf_resolve_valid(struct btf_verifier_env *env,if (!env_type_is_resolved (env, type_id))return false; -if (btf_type_is_struct (t)) +if (btf_type_is_struct (t) ||btf_type_is_datasec (t))return !btf->resolved_ids[type_id] && - !btf->resolved_sizes[type_id]; + !btf->resolved_sizes[type_id]; -if (btf_type_is_modifier (t) ||btf_type_is_ptr (t)) { +if (btf_type_is_modifier (t) ||btf_type_is_ptr (t) || +btf_type_is_var (t)) { t =btf_type_id_resolve (btf, &type_id); -return t && !btf_type_is_modifier (t); +return t && + !btf_type_is_modifier (t) && + !btf_type_is_var (t) && + !btf_type_is_datasec (t); }if (btf_type_is_array (t)) {
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v3 06/15] bpf: kernel side support for BTF Var and DataSec
Quoted 18 lines · Daniel Borkmann
This work adds kernel-side verification, logging and
seq_showdumping of BTF Var and DataSec kinds which are emitted with latest LLVM. The following constraints apply:BTF Var must have:
- Its
kind_flagis 0
- Its vlen is 0
- Must point to a valid type
- Type must not resolve to a forward type
- Must have a valid name
- Name may include dots (e.g. in case of static variables inside functions)
- Cannot be a member of a struct/union
- Linkage so far can either only be static or global/allocatedBTF DataSec must have:
Quoted 12 lines
- Its
kind_flagis 0
- Its vlen cannot be 0
- Its size cannot be 0
- Must have a valid name
- Name may include dots (e.g. to represent .bss, .data, .rodata etc)
- Cannot be a member of a struct/union
- Innerbtf_var_secinfoarray with {type,offset,size} triple must be sorted by offset in ascending order
- Type must always point to BTF Var
- BTF resolved size of Var must be <= size provided by triple
- DataSec size must be >= sum of triple sizes (thus holes are allowed)
Looks very good. A few questions.
[ ... ]
kernel/bpf/btf.c static bool btf_type_is_modifier(const struct btf_type *t) · line 323
Quoted 5 lines
kernel/bpf/btf.c +1
-0@@ -308,6 +320,7 @@ static bool btf_type_is_modifier(const struct btf_type *t)case BTF_KIND_VOLATILE :case BTF_KIND_CONST :case BTF_KIND_RESTRICT : +case BTF_KIND_VAR :
Why BTF_KIND_VAR is added here?
If possible, it may be better to keep is_modifier() as is since var intuitively does not belong to the is_modifier() test.
Quoted 33 lines
kernel/bpf/btf.c +15
-1return true; }@@ -375,13 +388,27 @@ static bool btf_type_is_int(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_INT ; } +static bool btf_type_is_var (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_VAR ; +} + +static bool btf_type_is_datasec (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ; +} +/* What types need to be resolved? * * btf_type_is_modifier() is an obvious one. * * btf_type_is_struct() because its member refers to * another type (through member->type). - +* +* btf_type_is_var() because the variable refers to +* another type. btf_type_is_datasec() holds multiple +* btf_type_is_var() types that need resolving. +* * btf_type_is_array() because its element (array->type) * refers to another type. Array can be thought of a * special case of struct while array just has the same
kernel/bpf/btf.c static bool btf_type_is_int(const struct btf_type *t) · line 423
Quoted 10 lines
kernel/bpf/btf.c +4
-3@@ -390,9 +417,11 @@ static bool btf_type_is_int(const struct btf_type *t)static bool btf_type_needs_resolve (const struct btf_type *t) {return btf_type_is_modifier (t) || -btf_type_is_ptr (t) || -btf_type_is_struct (t) || -btf_type_is_array (t); +btf_type_is_ptr (t) || +btf_type_is_struct (t) || +btf_type_is_array (t) || + btf_type_is_var(t) ||
is_var() is also tested here on top of is_modifier().
Quoted 12 lines
kernel/bpf/btf.c +2
-0+btf_type_is_datasec (t); }/* t->size can be used */ @@ -403,6 +432,7 @@ static bool btf_type_has_size(const struct btf_type *t)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC :return true; }
[ ... ]
kernel/bpf/btf.c static int btf_modifier_resolve(struct btf_verifier_env *env, · line 1641
Quoted 10 lines
kernel/bpf/btf.c +10
-0+static int btf_var_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_type *next_type; +const struct btf_type *t = v->t; +u32 next_type_id = t->type; +struct btf *btf = env->btf; +u32 next_type_size; + + next_type =btf_type_by_id (btf, next_type_id);
Does the BTF verifier complain if BTF_KIND_VAR -> BTF_KIND_VAR -> BTF_KIND_INT?
The same goes for BTF_KIND_PTR -> BTF_KIND_VAR -> BTF_KIND_INT.
kernel/bpf/btf.c static int btf_modifier_resolve(struct btf_verifier_env *env, · line 1651
Quoted 10 lines
kernel/bpf/btf.c +10
-0+if (!next_type) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +if (!env_type_is_resolve_sink (env, next_type) && + !env_type_is_resolved (env, next_type_id)) +return env_stack_push (env, next_type, next_type_id); + +if (btf_type_is_modifier (next_type)) {
May be a few comments on why this is needed. Together with the is_modifier() change above, it is not immediately clear to me.
I suspect this could be left to be done in the btf_ptr_resolve() as well if the ptr type happens to be behind the var type in the ".BTF" ELF.
Quoted 27 lines
kernel/bpf/btf.c +27
-0+const struct btf_type *resolved_type; +u32 resolved_type_id; + + resolved_type_id = next_type_id; + resolved_type =btf_type_id_resolve (btf, &resolved_type_id); + +if (btf_type_is_ptr (resolved_type) && + !env_type_is_resolve_sink (env, resolved_type) && + !env_type_is_resolved (env, resolved_type_id)) +return env_stack_push (env, resolved_type, + resolved_type_id); + } + +/* We must resolve to something concrete at this point, no +* forward types or similar that would resolve to size of +* zero is allowed. +*/ +if (!btf_type_id_size (btf, &next_type_id, &next_type_size)) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +env_stack_pop_resolved (env, next_type_id, next_type_size); + +return 0 ; +} +
[ ... ]
Quoted 25 lines
kernel/bpf/btf.c +8
-4@@ -2622,13 +2983,17 @@ static bool btf_resolve_valid(struct btf_verifier_env *env,if (!env_type_is_resolved (env, type_id))return false; -if (btf_type_is_struct (t)) +if (btf_type_is_struct (t) ||btf_type_is_datasec (t))return !btf->resolved_ids[type_id] && - !btf->resolved_sizes[type_id]; + !btf->resolved_sizes[type_id]; -if (btf_type_is_modifier (t) ||btf_type_is_ptr (t)) { +if (btf_type_is_modifier (t) ||btf_type_is_ptr (t) || +btf_type_is_var (t)) { t =btf_type_id_resolve (btf, &type_id); -return t && !btf_type_is_modifier (t); +return t && + !btf_type_is_modifier (t) && + !btf_type_is_var (t) && + !btf_type_is_datasec (t); }if (btf_type_is_array (t)) {Signature · 1 line
2.9.5
Re: [PATCH bpf-next v3 06/15] bpf: kernel side support for BTF Var and DataSec
Quoted 117 lines · Martin Lau
Quoted 18 lines · Daniel Borkmann
This work adds kernel-side verification, logging and
seq_showdumping of BTF Var and DataSec kinds which are emitted with latest LLVM. The following constraints apply:BTF Var must have:
- Its
kind_flagis 0
- Its vlen is 0
- Must point to a valid type
- Type must not resolve to a forward type
- Must have a valid name
- Name may include dots (e.g. in case of static variables inside functions)
- Cannot be a member of a struct/union
- Linkage so far can either only be static or global/allocatedBTF DataSec must have:
Quoted 12 lines
- Its
kind_flagis 0
- Its vlen cannot be 0
- Its size cannot be 0
- Must have a valid name
- Name may include dots (e.g. to represent .bss, .data, .rodata etc)
- Cannot be a member of a struct/union
- Innerbtf_var_secinfoarray with {type,offset,size} triple must be sorted by offset in ascending order
- Type must always point to BTF Var
- BTF resolved size of Var must be <= size provided by triple
- DataSec size must be >= sum of triple sizes (thus holes are allowed)Looks very good. A few questions.
[ ... ]
kernel/bpf/btf.c static bool btf_type_is_modifier(const struct btf_type *t) · line 323
Quoted 5 lines
kernel/bpf/btf.c +1
-0@@ -308,6 +320,7 @@ static bool btf_type_is_modifier(const struct btf_type *t)case BTF_KIND_VOLATILE :case BTF_KIND_CONST :case BTF_KIND_RESTRICT : +case BTF_KIND_VAR :Why
BTF_KIND_VARis added here?If possible, it may be better to keep
is_modifier()as is since var intuitively does not belong to theis_modifier()test.Quoted 33 lines
kernel/bpf/btf.c +15
-1return true; }@@ -375,13 +388,27 @@ static bool btf_type_is_int(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_INT ; } +static bool btf_type_is_var (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_VAR ; +} + +static bool btf_type_is_datasec (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ; +} +/* What types need to be resolved? * * btf_type_is_modifier() is an obvious one. * * btf_type_is_struct() because its member refers to * another type (through member->type). - +* +* btf_type_is_var() because the variable refers to +* another type. btf_type_is_datasec() holds multiple +* btf_type_is_var() types that need resolving. +* * btf_type_is_array() because its element (array->type) * refers to another type. Array can be thought of a * special case of struct while array just has the same kernel/bpf/btf.c static bool btf_type_is_int(const struct btf_type *t) · line 423
Quoted 10 lines
kernel/bpf/btf.c +4
-3@@ -390,9 +417,11 @@ static bool btf_type_is_int(const struct btf_type *t)static bool btf_type_needs_resolve (const struct btf_type *t) {return btf_type_is_modifier (t) || -btf_type_is_ptr (t) || -btf_type_is_struct (t) || -btf_type_is_array (t); +btf_type_is_ptr (t) || +btf_type_is_struct (t) || +btf_type_is_array (t) || + btf_type_is_var(t) ||
is_var()is also tested here on top ofis_modifier().Quoted 12 lines
kernel/bpf/btf.c +2
-0+btf_type_is_datasec (t); }/* t->size can be used */ @@ -403,6 +432,7 @@ static bool btf_type_has_size(const struct btf_type *t)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC :return true; }[ ... ]
kernel/bpf/btf.c static int btf_modifier_resolve(struct btf_verifier_env *env, · line 1641
Quoted 10 lines
kernel/bpf/btf.c +10
-0+static int btf_var_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_type *next_type; +const struct btf_type *t = v->t; +u32 next_type_id = t->type; +struct btf *btf = env->btf; +u32 next_type_size; + + next_type =btf_type_by_id (btf, next_type_id);Does the BTF verifier complain if
BTF_KIND_VAR->BTF_KIND_VAR->BTF_KIND_INT?
Quoted 1 line
The same goes for
BTF_KIND_PTR->BTF_KIND_VAR->BTF_KIND_INT.
I was about to try but it seems I cannot apply the set cleanly. Likely due to some recent changes.
After a quick skim through, the above cases seem possible.
If rejecting the above cases is desired,
I think it may be easier to check them at the individual type's _resolve() instead of depending on the final resort btf_resolve_valid(). The individual type's _resolve() should know better what are the acceptable next_type[s].
I was thinking btf_resolve_valid() is more like a "btf verifier internal error" to ensure the
earlier individual type's _resolve() is sane.
It seems at least the modifier_resolve() and ptr_resolve() are missing to reject BTF_KIND_FUNC. I will code up a patch to fix BTF_KIND_FUNC.
Quoted 73 lines
kernel/bpf/btf.c static int btf_modifier_resolve(struct btf_verifier_env *env, · line 1651
Quoted 11 lines
kernel/bpf/btf.c +10
-0+if (!next_type) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +if (!env_type_is_resolve_sink (env, next_type) && + !env_type_is_resolved (env, next_type_id)) +return env_stack_push (env, next_type, next_type_id); + +if (btf_type_is_modifier (next_type)) {May be a few comments on why this is needed. Together with the
is_modifier()change above, it is not immediately clear to me.I suspect this could be left to be done in the
btf_ptr_resolve()as well if the ptr type happens to be behind the var type in the ".BTF" ELF.Quoted 27 lines
kernel/bpf/btf.c +27
-0+const struct btf_type *resolved_type; +u32 resolved_type_id; + + resolved_type_id = next_type_id; + resolved_type =btf_type_id_resolve (btf, &resolved_type_id); + +if (btf_type_is_ptr (resolved_type) && + !env_type_is_resolve_sink (env, resolved_type) && + !env_type_is_resolved (env, resolved_type_id)) +return env_stack_push (env, resolved_type, + resolved_type_id); + } + +/* We must resolve to something concrete at this point, no +* forward types or similar that would resolve to size of +* zero is allowed. +*/ +if (!btf_type_id_size (btf, &next_type_id, &next_type_size)) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +env_stack_pop_resolved (env, next_type_id, next_type_size); + +return 0 ; +} +[ ... ]
Quoted 25 lines
kernel/bpf/btf.c +8
-4@@ -2622,13 +2983,17 @@ static bool btf_resolve_valid(struct btf_verifier_env *env,if (!env_type_is_resolved (env, type_id))return false; -if (btf_type_is_struct (t)) +if (btf_type_is_struct (t) ||btf_type_is_datasec (t))return !btf->resolved_ids[type_id] && - !btf->resolved_sizes[type_id]; + !btf->resolved_sizes[type_id]; -if (btf_type_is_modifier (t) ||btf_type_is_ptr (t)) { +if (btf_type_is_modifier (t) ||btf_type_is_ptr (t) || +btf_type_is_var (t)) { t =btf_type_id_resolve (btf, &type_id); -return t && !btf_type_is_modifier (t); +return t && + !btf_type_is_modifier (t) && + !btf_type_is_var (t) && + !btf_type_is_datasec (t); }if (btf_type_is_array (t)) {Signature · 1 line
2.9.5
Daniel Borkmann
maintainerRe: [PATCH bpf-next v3 06/15] bpf: kernel side support for BTF Var and DataSec
Hey Martin,
Thanks a lot for the feedback, much appreciated!
Quoted 43 lines · Martin Lau
Quoted 42 lines · Martin Lau
Quoted 18 lines · Daniel Borkmann
This work adds kernel-side verification, logging and
seq_showdumping of BTF Var and DataSec kinds which are emitted with latest LLVM. The following constraints apply:BTF Var must have:
- Its
kind_flagis 0
- Its vlen is 0
- Must point to a valid type
- Type must not resolve to a forward type
- Must have a valid name
- Name may include dots (e.g. in case of static variables inside functions)
- Cannot be a member of a struct/union
- Linkage so far can either only be static or global/allocatedBTF DataSec must have:
Quoted 12 lines
- Its
kind_flagis 0
- Its vlen cannot be 0
- Its size cannot be 0
- Must have a valid name
- Name may include dots (e.g. to represent .bss, .data, .rodata etc)
- Cannot be a member of a struct/union
- Innerbtf_var_secinfoarray with {type,offset,size} triple must be sorted by offset in ascending order
- Type must always point to BTF Var
- BTF resolved size of Var must be <= size provided by triple
- DataSec size must be >= sum of triple sizes (thus holes are allowed)Looks very good. A few questions.
[ ... ]
kernel/bpf/btf.c static bool btf_type_is_modifier(const struct btf_type *t) · line 323
Quoted 5 lines
kernel/bpf/btf.c +1
-0@@ -308,6 +320,7 @@ static bool btf_type_is_modifier(const struct btf_type *t)case BTF_KIND_VOLATILE :case BTF_KIND_CONST :case BTF_KIND_RESTRICT : +case BTF_KIND_VAR :Why
BTF_KIND_VARis added here?
Quoted 2 lines
Quoted 2 lines
If possible, it may be better to keep
is_modifier()as is since var intuitively does not belong to theis_modifier()test.
Hmm, agree, it's probably better to keep btf_type_is_modifier() as-is, I might have been mislead by the comment in that function. Need to double check whether that could trigger the WARN in btf_type_id_size().
Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
kernel/bpf/btf.c +15
-1return true; }@@ -375,13 +388,27 @@ static bool btf_type_is_int(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_INT ; } +static bool btf_type_is_var (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_VAR ; +} + +static bool btf_type_is_datasec (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ; +} +/* What types need to be resolved? * * btf_type_is_modifier() is an obvious one. * * btf_type_is_struct() because its member refers to * another type (through member->type). - +* +* btf_type_is_var() because the variable refers to +* another type. btf_type_is_datasec() holds multiple +* btf_type_is_var() types that need resolving. +* * btf_type_is_array() because its element (array->type) * refers to another type. Array can be thought of a * special case of struct while array just has the same @@ -390,9 +417,11 @@ static bool btf_type_is_int(const struct btf_type *t)static bool btf_type_needs_resolve (const struct btf_type *t) { return btf_type_is_modifier(t) ||
Quoted 8 lines
Quoted 8 lines
kernel/bpf/btf.c static bool btf_type_is_int(const struct btf_type *t) · line 423
Quoted 7 lines
kernel/bpf/btf.c +4
-3-btf_type_is_ptr (t) || -btf_type_is_struct (t) || -btf_type_is_array (t); +btf_type_is_ptr (t) || +btf_type_is_struct (t) || +btf_type_is_array (t) || + btf_type_is_var(t) ||
is_var()is also tested here on top ofis_modifier().
Yeah, correct.
Quoted 27 lines
Quoted 27 lines
Quoted 12 lines
kernel/bpf/btf.c +2
-0+btf_type_is_datasec (t); }/* t->size can be used */ @@ -403,6 +432,7 @@ static bool btf_type_has_size(const struct btf_type *t)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC :return true; }[ ... ]
kernel/bpf/btf.c static int btf_modifier_resolve(struct btf_verifier_env *env, · line 1641
Quoted 10 lines
kernel/bpf/btf.c +10
-0+static int btf_var_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_type *next_type; +const struct btf_type *t = v->t; +u32 next_type_id = t->type; +struct btf *btf = env->btf; +u32 next_type_size; + + next_type =btf_type_by_id (btf, next_type_id);Does the BTF verifier complain if
BTF_KIND_VAR->BTF_KIND_VAR->BTF_KIND_INT?
Quoted 5 lines
Quoted 1 line
The same goes for
BTF_KIND_PTR->BTF_KIND_VAR->BTF_KIND_INT.I was about to try but it seems I cannot apply the set cleanly. Likely due to some recent changes.
After a quick skim through, the above cases seem possible.
The tests currently only check whether we end up at BTF_KIND_VAR or BTF_KIND_DATASEC, but as far as I'm aware would accept some intermediate BTF_KIND_VAR type, which should be better rejected instead.
Quoted 5 lines
If rejecting the above cases is desired,
I think it may be easier to check them at the individual type's_resolve()instead of depending on the final resortbtf_resolve_valid(). The individual type's_resolve()should know better what are the acceptable next_type[s].
Makes sense, I'll change that today and add more test coverage for this particular case. Moving the check into individual type's _resolve() handler is indeed probably the best we can do at this point, though unfortunate duplication.
Quoted 7 lines
I was thinking
btf_resolve_valid()is more like a
"btf verifier internal error" to ensure the
earlier individual type's_resolve()is sane.It seems at least the
modifier_resolve()andptr_resolve()are missing to rejectBTF_KIND_FUNC. I will code up a patch to fixBTF_KIND_FUNC.
Okay.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
kernel/bpf/btf.c +4
-0+if (!next_type) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + }
Quoted 8 lines
Quoted 8 lines
kernel/bpf/btf.c static int btf_modifier_resolve(struct btf_verifier_env *env, · line 1651
Quoted 6 lines
kernel/bpf/btf.c +6
-0+ +if (!env_type_is_resolve_sink (env, next_type) && + !env_type_is_resolved (env, next_type_id)) +return env_stack_push (env, next_type, next_type_id); + +if (btf_type_is_modifier (next_type)) {May be a few comments on why this is needed. Together with the
is_modifier()change above, it is not immediately clear to me.
Yep, the change in is_modifier() needs to be dropped.
Quoted 59 lines
Quoted 59 lines
I suspect this could be left to be done in the
btf_ptr_resolve()as well if the ptr type happens to be behind the var type in the ".BTF" ELF.Quoted 27 lines
kernel/bpf/btf.c +27
-0+const struct btf_type *resolved_type; +u32 resolved_type_id; + + resolved_type_id = next_type_id; + resolved_type =btf_type_id_resolve (btf, &resolved_type_id); + +if (btf_type_is_ptr (resolved_type) && + !env_type_is_resolve_sink (env, resolved_type) && + !env_type_is_resolved (env, resolved_type_id)) +return env_stack_push (env, resolved_type, + resolved_type_id); + } + +/* We must resolve to something concrete at this point, no +* forward types or similar that would resolve to size of +* zero is allowed. +*/ +if (!btf_type_id_size (btf, &next_type_id, &next_type_size)) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +env_stack_pop_resolved (env, next_type_id, next_type_size); + +return 0 ; +} +[ ... ]
Quoted 25 lines
kernel/bpf/btf.c +8
-4@@ -2622,13 +2983,17 @@ static bool btf_resolve_valid(struct btf_verifier_env *env,if (!env_type_is_resolved (env, type_id))return false; -if (btf_type_is_struct (t)) +if (btf_type_is_struct (t) ||btf_type_is_datasec (t))return !btf->resolved_ids[type_id] && - !btf->resolved_sizes[type_id]; + !btf->resolved_sizes[type_id]; -if (btf_type_is_modifier (t) ||btf_type_is_ptr (t)) { +if (btf_type_is_modifier (t) ||btf_type_is_ptr (t) || +btf_type_is_var (t)) { t =btf_type_id_resolve (btf, &type_id); -return t && !btf_type_is_modifier (t); +return t && + !btf_type_is_modifier (t) && + !btf_type_is_var (t) && + !btf_type_is_datasec (t); }if (btf_type_is_array (t)) {Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 7/15 bpf: allow for key-less BTF in array map
In reply to Daniel BorkmannGiven we'll be reusing BPF array maps for global data/bss/rodata sections, we need a way to associate BTF DataSec type as its map value type. In usual cases we have this ugly BPF_ANNOTATE_KV_PAIR() macro hack e.g. via 38d5d3b3d5db ("bpf: Introduce BPF_ANNOTATE_KV_PAIR") to get initial map to type association going. While more use cases for it are discouraged, this also won't work for global data since the use of array map is a BPF loader detail and therefore unknown at compilation time. For array maps with just a single entry we make an exception in terms of BTF in that key type is declared optional if value type is of DataSec type. Latter LLVM is guaranteed to emit and it also aligns with how we regard global data maps as just a plain buffer area reusing existing map facilities for allowing things like introspection with existing tools.
--- include/linux/btf.h | 2 ++ kernel/bpf/arraymap.c | 15 ++++++++++++++- kernel/bpf/btf.c | 7 ++++++- kernel/bpf/local_storage.c | 2 ++ kernel/bpf/lpm_trie.c | 3 +++ kernel/bpf/syscall.c | 15 +++++++++++---- 6 files changed, 38 insertions(+), 6 deletions(-)
include/linux/btf.h +2 -0
@@ -51,6 +51,8 @@ bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,const struct btf_member *m, u32 expected_offset, u32 expected_size);int btf_find_spin_lock (const struct btf *btf,const struct btf_type *t); +const struct btf_type *btf_type_void_get (void ); +bool btf_type_is_void (const struct btf_type *t);#ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id);
kernel/bpf/arraymap.c +14 -1
@@ -388,7 +388,8 @@ static void array_map_seq_show_elem(struct bpf_map *map, void *key,return ; } -seq_printf (m,"%u: " , *(u32 *)key); +if (map->btf_key_type_id) +seq_printf (m,"%u: " , *(u32 *)key);btf_type_seq_show (map->btf, map->btf_value_type_id, value, m);seq_puts (m,"\n" );
@@ -425,6 +426,18 @@ static int array_map_check_btf(const struct bpf_map *map, {u32 int_data; +/* One exception for keyless BTF: .bss/.data/.rodata map */ +if (btf_type_is_void (key_type)) { +if (map->map_type !=BPF_MAP_TYPE_ARRAY || + map->max_entries !=1 ) +return -EINVAL ; + +if (BTF_INFO_KIND (value_type->info) !=BTF_KIND_DATASEC ) +return -EINVAL ; + +return 0 ; + } +if (BTF_INFO_KIND (key_type->info) !=BTF_KIND_INT )return -EINVAL ;
kernel/bpf/btf.c +6 -1
@@ -327,7 +327,12 @@ static bool btf_type_is_modifier(const struct btf_type *t)return false; } -static bool btf_type_is_void (const struct btf_type *t) +const struct btf_type *btf_type_void_get (void ) +{ +return &btf_void; +} + +bool btf_type_is_void (const struct btf_type *t) {return t == &btf_void; }
kernel/bpf/local_storage.c +2 -0
@@ -338,6 +338,8 @@ static int cgroup_storage_check_btf(const struct bpf_map *map,* __u32 attach_type; * }; */ +if (btf_type_is_void (key_type)) +return -EINVAL ;/* * Key_type must be a structure with two fields.
kernel/bpf/lpm_trie.c +3 -0
@@ -734,6 +734,9 @@ static int trie_check_btf(const struct bpf_map *map, const struct btf_type *key_type, const struct btf_type *value_type) { + if (btf_type_is_void(key_type)) + return -EINVAL; +/* Keys must have struct bpf_lpm_trie_key embedded. */ return BTF_INFO_KIND (key_type->info) !=BTF_KIND_STRUCT ? -EINVAL :0 ;
kernel/bpf/syscall.c +11 -4
@@ -494,9 +494,16 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,u32 key_size, value_size;int ret =0 ; - key_type =btf_type_id_size (btf, &btf_key_id, &key_size); -if (!key_type || key_size != map->key_size) -return -EINVAL ; +/* Some maps allow key to be unspecified. */ +if (btf_key_id) { + key_type =btf_type_id_size (btf, &btf_key_id, &key_size); +if (!key_type || key_size != map->key_size) +return -EINVAL ; + }else { + key_type =btf_type_void_get (); +if (!map->ops->map_check_btf) +return -EINVAL ; + } value_type =btf_type_id_size (btf, &btf_value_id, &value_size);if (!value_type || value_size != map->value_size)
@@ -564,7 +571,7 @@ static int map_create(union bpf_attr *attr)if (attr->btf_key_type_id || attr->btf_value_type_id) {struct btf *btf; -if (!attr->btf_key_type_id || !attr->btf_value_type_id) { +if (!attr->btf_value_type_id) { err = -EINVAL ; goto free_map_nouncharge; }
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 8/15 bpf: sync {btf,bpf}.h uapi header from tools infrastructure
In reply to Daniel BorkmannPull in latest changes from both headers, so we can make use of them in libbpf.
--- tools/include/uapi/linux/bpf.h | 24 ++++++++++++++++++++++-- tools/include/uapi/linux/btf.h | 32 ++++++++++++++++++++++++++++---- 2 files changed, 50 insertions(+), 6 deletions(-)
tools/include/uapi/linux/bpf.h +22 -2
@@ -105,6 +105,7 @@ enum bpf_cmd {BPF_BTF_GET_FD_BY_ID ,BPF_TASK_FD_QUERY ,BPF_MAP_LOOKUP_AND_DELETE_ELEM , + BPF_MAP_LOCK, }; enum bpf_map_type {
@@ -255,8 +256,19 @@ enum bpf_attach_type {*/ #define BPF_F_ANY_ALIGNMENT (1U << 1) -/* when bpf_ldimm64->src_reg == BPF_PSEUDO_MAP_FD, bpf_ldimm64->imm == fd */ +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have +* two extensions: +* +* insn[0].src_reg: BPF_PSEUDO_MAP_FD BPF_PSEUDO_MAP_VALUE +* insn[0].imm: map fd map fd +* insn[1].imm: 0 offset into value +* insn[0].off: 0 lower 16 bit of map index +* insn[1].off: 0 higher 16 bit of map index +* ldimm64 rewrite: address of map address of map[index]+offset +* verifier type: CONST_PTR_TO_MAP PTR_TO_MAP_VALUE +*/ #define BPF_PSEUDO_MAP_FD 1 +#define BPF_PSEUDO_MAP_VALUE 2/* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative * offset to another bpf function
@@ -283,7 +295,7 @@ enum bpf_attach_type {#define BPF_OBJ_NAME_LEN 16U -/* Flags for accessing BPF object */ +/* Flags for accessing BPF object from syscall side. */ #define BPF_F_RDONLY (1U << 3)#define BPF_F_WRONLY (1U << 4)
@@ -293,6 +305,14 @@ enum bpf_attach_type {/* Zero-initialize hash function seed. This should only be used for testing. */ #define BPF_F_ZERO_SEED (1U << 6) +/* Flags for accessing BPF object from program side. */ +#define BPF_F_RDONLY_PROG (1U << 7) +#define BPF_F_WRONLY_PROG (1U << 8) +#define BPF_F_ACCESS_MASK (BPF_F_RDONLY | \ + BPF_F_RDONLY_PROG | \ + BPF_F_WRONLY | \ + BPF_F_WRONLY_PROG) +/* flags for BPF_PROG_QUERY */ #define BPF_F_QUERY_EFFECTIVE (1U << 0)
tools/include/uapi/linux/btf.h +28 -4
@@ -39,11 +39,11 @@ struct btf_type {* struct, union and fwd */ __u32 info; -/* "size" is used by INT, ENUM, STRUCT and UNION. +/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC. * "size" tells the size of the type it is describing. * * "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT, -* FUNC and FUNC_PROTO. +* FUNC, FUNC_PROTO and VAR. * "type" is a type_id referring to another type. */ union {
@@ -70,8 +70,10 @@ struct btf_type {#define BTF_KIND_RESTRICT 11/* Restrict */ #define BTF_KIND_FUNC 12/* Function */ #define BTF_KIND_FUNC_PROTO 13/* Function Proto */ -#define BTF_KIND_MAX 13 -#define NR_BTF_KINDS 14 +#define BTF_KIND_VAR 14/* Variable */ +#define BTF_KIND_DATASEC 15/* Section */ +#define BTF_KIND_MAX BTF_KIND_DATASEC +#define NR_BTF_KINDS (BTF_KIND_MAX + 1)/* For some specific BTF_KIND, "struct btf_type" is immediately * followed by extra data.
@@ -138,4 +140,26 @@ struct btf_param { __u32 type; }; +enum { +BTF_VAR_STATIC =0 , +BTF_VAR_GLOBAL_ALLOCATED , +}; + +/* BTF_KIND_VAR is followed by a single "struct btf_var" to describe +* additional information related to the variable such as its linkage. +*/ +struct btf_var { +__u32 linkage; +}; + +/* BTF_KIND_DATASEC is followed by multiple "struct btf_var_secinfo" +* to describe all BTF_KIND_VAR types it contains along with it's +* in-section offset as well as size. +*/ +struct btf_var_secinfo { +__u32 type; +__u32 offset; +__u32 size; +}; +#endif /* _UAPI__LINUX_BTF_H__ */
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 9/15 bpf, libbpf: refactor relocation handling
In reply to Daniel BorkmannFrom: Joe Stringer <[email protected]>
Adjust the code for relocations slightly with no functional changes, so that upcoming patches that will introduce support for relocations into the .data, .rodata and .bss sections can be added independent of these changes.
--- tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++++++++------------------------ 1 file changed, 32 insertions(+), 30 deletions(-)
tools/lib/bpf/libbpf.c +32 -30
@@ -865,20 +865,20 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags) obj->efile.symbols = data; obj->efile.strtabidx = sh.sh_link; } - }else if ((sh.sh_type ==SHT_PROGBITS ) && - (sh.sh_flags &SHF_EXECINSTR ) && - (data->d_size >0 )) { -if (strcmp (name,".text" ) ==0 ) - obj->efile.text_shndx = idx; - err =bpf_object__add_program (obj, data->d_buf, - data->d_size, name, idx); -if (err) { -char errmsg[STRERR_BUFSIZE ]; -char *cp =libbpf_strerror_r (-err, errmsg, -sizeof (errmsg)); - -pr_warning ("failed to alloc program %s (%s): %s" , - name, obj->path, cp); + }else if (sh.sh_type ==SHT_PROGBITS && data->d_size >0 ) { +if (sh.sh_flags &SHF_EXECINSTR ) { +if (strcmp (name,".text" ) ==0 ) + obj->efile.text_shndx = idx; + err =bpf_object__add_program (obj, data->d_buf, + data->d_size, name, idx); +if (err) { +char errmsg[STRERR_BUFSIZE ]; +char *cp =libbpf_strerror_r (-err, errmsg, +sizeof (errmsg)); + +pr_warning ("failed to alloc program %s (%s): %s" , + name, obj->path, cp); + } } }else if (sh.sh_type ==SHT_REL ) {void *reloc = obj->efile.reloc;
@@ -1040,24 +1040,26 @@ bpf_program__collect_reloc(struct bpf_program *prog, GElf_Shdr *shdr,return -LIBBPF_ERRNO__RELOC ; } -/* TODO: 'maps' is sorted. We can use bsearch to make it faster. */ -for (map_idx =0 ; map_idx < nr_maps; map_idx++) { -if (maps[map_idx].offset == sym.st_value) { -pr_debug ("relocation: find map %zd (%s) for insn %u\n" , - map_idx, maps[map_idx].name, insn_idx); -break ; +if (sym.st_shndx == maps_shndx) { +/* TODO: 'maps' is sorted. We can use bsearch to make it faster. */ +for (map_idx =0 ; map_idx < nr_maps; map_idx++) { +if (maps[map_idx].offset == sym.st_value) { +pr_debug ("relocation: find map %zd (%s) for insn %u\n" , + map_idx, maps[map_idx].name, insn_idx); +break ; + } } - } -if (map_idx >= nr_maps) { -pr_warning ("bpf relocation: map_idx %d large than %d\n" , - (int )map_idx, (int )nr_maps -1 ); -return -LIBBPF_ERRNO__RELOC ; - } +if (map_idx >= nr_maps) { +pr_warning ("bpf relocation: map_idx %d large than %d\n" , + (int )map_idx, (int )nr_maps -1 ); +return -LIBBPF_ERRNO__RELOC ; + } - prog->reloc_desc[i].type =RELO_LD64 ; - prog->reloc_desc[i].insn_idx = insn_idx; - prog->reloc_desc[i].map_idx = map_idx; + prog->reloc_desc[i].type =RELO_LD64 ; + prog->reloc_desc[i].insn_idx = insn_idx; + prog->reloc_desc[i].map_idx = map_idx; + } }return 0 ; }
@@ -1419,7 +1421,7 @@ bpf_program__relocate(struct bpf_program *prog, struct bpf_object *obj) } insns[insn_idx].src_reg =BPF_PSEUDO_MAP_FD ; insns[insn_idx].imm = obj->maps[map_idx].fd; - }else { + }else if (prog->reloc_desc[i].type ==RELO_CALL ) { err =bpf_program__reloc_text (prog, obj, &prog->reloc_desc[i]);if (err)
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 10/15 bpf, libbpf: support global data/bss/rodata sections
In reply to Daniel BorkmannThis work adds BPF loader support for global data sections to libbpf. This allows to write BPF programs in more natural C-like way by being able to define global variables and const data.
Back at LPC 2018 [0] we presented a first prototype which implemented support for global data sections by extending BPF syscall where union bpf_attr would get additional memory/size pair for each section passed during prog load in order to later add this base address into the ldimm64 instruction along with the user provided offset when accessing a variable. Consensus from LPC was that for proper upstream support, it would be more desirable to use maps instead of bpf_attr extension as this would allow for introspection of these sections as well as potential live updates of their content. This work follows this path by taking the following steps from loader side:
1) In bpf_object__elf_collect() step we pick up ".data", ".rodata", and ".bss" section information.
2) If present, in bpf_object__init_internal_map() we add maps to the obj's map array that corresponds to each of the present sections. Given section size and access properties can differ, a single entry array map is created with value size that is corresponding to the ELF section size of .data, .bss or .rodata. These internal maps are integrated into the normal map handling of libbpf such that when user traverses all obj maps, they can be differentiated from user-created ones via bpf_map__is_internal(). In later steps when we actually create these maps in the kernel via bpf_object__create_maps(), then for .data and .rodata sections their content is copied into the map through bpf_map_update_elem(). For .bss this is not necessary since array map is already zero-initialized by default. Additionally, for .rodata the map is locked as read-only after setup, such that neither from program nor syscall side writes would be possible.
3) In bpf_program__collect_reloc() step, we record the corresponding map, insn index, and relocation type for the global data.
4) And last but not least in the actual relocation step in bpf_program__relocate(), we mark the ldimm64 instruction with src_reg = BPF_PSEUDO_MAP_VALUE where in the first imm field the map's file descriptor is stored as similarly done as in BPF_PSEUDO_MAP_FD, and in the second imm field (as ldimm64 is 2-insn wide) we store the access offset into the section. Given these maps have only single element ldimm64's off remains zero in both parts.
5) On kernel side, this special marked BPF_PSEUDO_MAP_VALUE load will then store the actual target address in order to have a 'map-lookup'-free access. That is, the actual map value base address + offset. The destination register in the verifier will then be marked as PTR_TO_MAP_VALUE, containing the fixed offset as reg->off and backing BPF map as reg->map_ptr. Meaning, it's treated as any other normal map value from verification side, only with efficient, direct value access instead of actual call to map lookup helper as in the typical case.
Currently, only support for static global variables has been added, and libbpf rejects non-static global variables from loading. This can be lifted until we have proper semantics for how BPF will treat these.
From BTF side, libbpf will set the value type id of the types corresponding to the ".bss", ".data" and ".rodata" names which LLVM will emit (w/o the object name prefix). The key type will be left as zero, thus making use of the key-less BTF option in array maps.
Simple example dump of program using globals vars in each section:
# bpftool prog
[...]
6784: sched_cls name load_static_dat tag a7e1291567277844 gpl
loaded_at 2019-03-11T15:39:34+0000 uid 0
xlated 1776B jited 993B memlock 4096B map_ids 2238,2237,2235,2236,2239,2240 # bpftool map show id 2237
2237: array name test_glo.bss flags 0x0
key 4B value 64B max_entries 1 memlock 4096B
# bpftool map show id 2235
2235: array name test_glo.data flags 0x0
key 4B value 64B max_entries 1 memlock 4096B
# bpftool map show id 2236
2236: array name test_glo.rodata flags 0x80
key 4B value 96B max_entries 1 memlock 4096B # bpftool prog dump xlated id 6784
int load_static_data(struct __sk_buff * skb):
; int load_static_data(struct __sk_buff *skb)
0: (b7) r6 = 0
; test_reloc(number, 0, &num0);
1: (63) *(u32 *)(r10 -4) = r6
2: (bf) r2 = r10
; int load_static_data(struct __sk_buff *skb)
3: (07) r2 += -4
; test_reloc(number, 0, &num0);
4: (18) r1 = map[id:2238]
6: (18) r3 = map[id:2237][0]+0 <-- direct addr in .bss area
8: (b7) r4 = 0
9: (85) call array_map_update_elem#100464
10: (b7) r1 = 1
; test_reloc(number, 1, &num1);
[...]
; test_reloc(string, 2, str2);
120: (18) r8 = map[id:2237][0]+16 <-- same here at offset +16
122: (18) r1 = map[id:2239]
124: (18) r3 = map[id:2237][0]+16
126: (b7) r4 = 0
127: (85) call array_map_update_elem#100464
128: (b7) r1 = 120
; str1[5] = 'x';
129: (73) *(u8 *)(r9 +5) = r1
; test_reloc(string, 3, str1);
130: (b7) r1 = 3
131: (63) *(u32 *)(r10 -4) = r1
132: (b7) r9 = 3
133: (bf) r2 = r10
; int load_static_data(struct __sk_buff *skb)
134: (07) r2 += -4
; test_reloc(string, 3, str1);
135: (18) r1 = map[id:2239]
137: (18) r3 = map[id:2235][0]+16 <-- direct addr in .data area
139: (b7) r4 = 0
140: (85) call array_map_update_elem#100464
141: (b7) r1 = 111
; __builtin_memcpy(&str2[2], "hello", sizeof("hello"));
142: (73) *(u8 *)(r8 +6) = r1 <-- further access based on .bss data
143: (b7) r1 = 108
144: (73) *(u8 *)(r8 +5) = r1
[...]Based upon recent fix in LLVM, commit c0db6b6bd444 ("[BPF] Don't fail for static variables").
[0] LPC 2018, BPF track, "ELF relocation for static data in BPF",
http://vger.kernel.org/lpc-bpf2018.html#session-3
--- tools/lib/bpf/bpf.c | 10 ++ tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 333 ++++++++++++++++++++++++++++++++++++++++------- tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/libbpf.map | 6 + 5 files changed, 304 insertions(+), 47 deletions(-)
tools/lib/bpf/bpf.c +10 -0
@@ -429,6 +429,16 @@ int bpf_map_get_next_key(int fd, const void *key, void *next_key)return sys_bpf (BPF_MAP_GET_NEXT_KEY , &attr,sizeof (attr)); } +int bpf_map_lock (int fd) +{ +union bpf_attr attr; + +memset (&attr,0 ,sizeof (attr)); + attr.map_fd = fd; + +return sys_bpf (BPF_MAP_LOCK , &attr,sizeof (attr)); +} +int bpf_obj_pin (int fd,const char *pathname) {union bpf_attr attr;
tools/lib/bpf/bpf.h +1 -0
@@ -117,6 +117,7 @@ LIBBPF_API int bpf_map_lookup_and_delete_elem(int fd, const void *key, void *value); LIBBPF_API int bpf_map_delete_elem(int fd, const void *key); LIBBPF_API int bpf_map_get_next_key(int fd, const void *key, void *next_key); +LIBBPF_API int bpf_map_lock(int fd); LIBBPF_API int bpf_obj_pin(int fd, const char *pathname); LIBBPF_API int bpf_obj_get(const char *pathname); LIBBPF_API int bpf_prog_attach(int prog_fd, int attachable_fd,
tools/lib/bpf/libbpf.c +286 -47
569 lines not on this page · raw message
tools/lib/bpf/libbpf.h +1 -0
@@ -301,6 +301,7 @@ LIBBPF_API void *bpf_map__priv(struct bpf_map *map); LIBBPF_API int bpf_map__reuse_fd(struct bpf_map *map, int fd); LIBBPF_API int bpf_map__resize(struct bpf_map *map, __u32 max_entries); LIBBPF_API bool bpf_map__is_offload_neutral(struct bpf_map *map); +LIBBPF_API bool bpf_map__is_internal(struct bpf_map *map); LIBBPF_API void bpf_map__set_ifindex(struct bpf_map *map, __u32 ifindex); LIBBPF_API int bpf_map__pin(struct bpf_map *map, const char *path); LIBBPF_API int bpf_map__unpin(struct bpf_map *map, const char *path);
tools/lib/bpf/libbpf.map +6 -0
@@ -157,3 +157,9 @@ LIBBPF_0.0.2 { bpf_program__bpil_addr_to_offs; bpf_program__bpil_offs_to_addr; } LIBBPF_0.0.1; + +LIBBPF_0.0.3 { + global: + bpf_map__is_internal; + bpf_map_lock; +} LIBBPF_0.0.2;
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 11/15 bpf, libbpf: add support for BTF Var and DataSec
In reply to Daniel BorkmannThis adds libbpf support for BTF Var and DataSec kinds. Main point here is that libbpf needs to do some preparatory work before the whole BTF object can be loaded into the kernel, that is, fixing up of DataSec size taken from the ELF section size and non-static variable offset which needs to be taken from the ELF's string section. Latest upstream LLVM didn't fix these up since at time of BTF emission this information wasn't available, hence loader does take care of it. Note, deduplication handling has not been in the scope of this work and needs to be addressed in a future commit.
--- tools/lib/bpf/btf.c | 97 +++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 3 + tools/lib/bpf/libbpf.c | 150 +++++++++++++++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.h | 4 ++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 235 insertions(+), 20 deletions(-)
tools/lib/bpf/btf.c +96 -1
@@ -24,6 +24,8 @@ ((k) == BTF_KIND_CONST) || \ ((k) == BTF_KIND_RESTRICT)) +#define IS_VAR (k) ((k) == BTF_KIND_VAR) +static struct btf_type btf_void;struct btf {
@@ -212,6 +214,10 @@ static int btf_type_size(struct btf_type *t)return base_size + vlen *sizeof (struct btf_member );case BTF_KIND_FUNC_PROTO :return base_size + vlen *sizeof (struct btf_param ); +case BTF_KIND_VAR : +return base_size +sizeof (struct btf_var ); +case BTF_KIND_DATASEC : +return base_size + vlen *sizeof (struct btf_var_secinfo );default :pr_debug ("Unsupported BTF_KIND:%u\n" ,BTF_INFO_KIND (t->info));return -EINVAL ;
@@ -283,6 +289,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC : size = t->size; goto done;case BTF_KIND_PTR :
@@ -292,6 +299,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)case BTF_KIND_VOLATILE :case BTF_KIND_CONST :case BTF_KIND_RESTRICT : +case BTF_KIND_VAR : type_id = t->type;break ;case BTF_KIND_ARRAY :
@@ -326,7 +334,8 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id) t =btf__type_by_id (btf, type_id);while (depth <MAX_RESOLVE_DEPTH && !btf_type_is_void_or_null (t) && -IS_MODIFIER (BTF_INFO_KIND (t->info))) { + (IS_MODIFIER (BTF_INFO_KIND (t->info)) || +IS_VAR (BTF_INFO_KIND (t->info)))) { type_id = t->type; t =btf__type_by_id (btf, type_id); depth++;
@@ -408,6 +417,92 @@ struct btf *btf__new(__u8 *data, __u32 size)return btf; } +static int compare_vsi_off (const void *_a,const void *_b) +{ +const struct btf_var_secinfo *a = _a; +const struct btf_var_secinfo *b = _b; + +return a->offset - b->offset; +} + +static int btf_fixup_datasec (struct bpf_object *obj,struct btf *btf, +struct btf_type *t) +{ +__u32 size =0 , off =0 , i, vars =BTF_INFO_VLEN (t->info); +const char *name =btf__name_by_offset (btf, t->name_off); +const struct btf_type *t_var; +struct btf_var_secinfo *vsi; +struct btf_var *var; +int ret; + +if (!name) { +pr_debug ("No name found in string section for DATASEC kind.\n" ); +return -ENOENT ; + } + + ret =bpf_object__section_size (obj, name, &size); +if (ret || !size || (t->size && t->size != size)) { +pr_debug ("Invalid size for section %s: %u bytes\n" , name, size); +return -ENOENT ; + } + + t->size = size; + +for (i =0 , vsi = (struct btf_var_secinfo *)(t +1 ); + i < vars; i++, vsi++) { + t_var =btf__type_by_id (btf, vsi->type); + var = (struct btf_var *)(t_var +1 ); + +if (BTF_INFO_KIND (t_var->info) !=BTF_KIND_VAR ) { +pr_debug ("Non-VAR type seen in section %s\n" , name); +return -EINVAL ; + } + +if (var->linkage ==BTF_VAR_STATIC ) +continue ; + + name =btf__name_by_offset (btf, t_var->name_off); +if (!name) { +pr_debug ("No name found in string section for VAR kind\n" ); +return -ENOENT ; + } + + ret =bpf_object__variable_offset (obj, name, &off); +if (ret) { +pr_debug ("No offset found in symbol table for VAR %s\n" , name); +return -ENOENT ; + } + + vsi->offset = off; + } + +qsort (t +1 , vars,sizeof (*vsi), compare_vsi_off); +return 0 ; +} + +int btf__finalize_data (struct bpf_object *obj,struct btf *btf) +{ +int err =0 ; +__u32 i; + +for (i =1 ; i <= btf->nr_types; i++) { +struct btf_type *t = btf->types[i]; + +/* Loader needs to fix up some of the things compiler +* couldn't get its hands on while emitting BTF. This +* is section size and global variable offset. We use +* the info from the ELF itself for this purpose. +*/ +if (BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ) { + err =btf_fixup_datasec (obj, btf, t); +if (err) +break ; + } + } + +return err; +} +int btf__load (struct btf *btf) {__u32 log_buf_size =BPF_LOG_BUF_SIZE ;
tools/lib/bpf/btf.h +3 -0
@@ -21,6 +21,8 @@ struct btf;struct btf_ext ;struct btf_type ; +struct bpf_object ; +/* * The .BTF.ext ELF section layout defined as * struct btf_ext_header
@@ -57,6 +59,7 @@ struct btf_ext_header { LIBBPF_API void btf__free(struct btf *btf); LIBBPF_API struct btf *btf__new(__u8 *data, __u32 size); +LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf); LIBBPF_API int btf__load(struct btf *btf); LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,const char *type_name);
tools/lib/bpf/libbpf.c +131 -19
181 lines not on this page · raw message
tools/lib/bpf/libbpf.h +4 -0
@@ -75,6 +75,10 @@ struct bpf_object *__bpf_object__open_xattr(struct bpf_object_open_attr *attr, LIBBPF_API struct bpf_object *bpf_object__open_buffer(void *obj_buf,size_t obj_buf_sz,const char *name); +int bpf_object__section_size (const struct bpf_object *obj,const char *name, +__u32 *size); +int bpf_object__variable_offset (const struct bpf_object *obj,const char *name, +__u32 *off); LIBBPF_API int bpf_object__pin_maps(struct bpf_object *obj, const char *path); LIBBPF_API int bpf_object__unpin_maps(struct bpf_object *obj,const char *path);
tools/lib/bpf/libbpf.map +1 -0
@@ -162,4 +162,5 @@ LIBBPF_0.0.3 { global: bpf_map__is_internal; bpf_map_lock; + btf__finalize_data; } LIBBPF_0.0.2;
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 12/15 bpf: bpftool support for dumping data/bss/rodata sections
In reply to Daniel BorkmannAdd the ability to bpftool to handle BTF Var and DataSec kinds in order to dump them out of btf_dumper_type(). The value has a single object with the section name, which itself holds an array of variables it dumps. A single variable is an object by itself printed along with its name. From there further type information is dumped along with corresponding value information.
Example output from .rodata:
# ./bpftool m d i 150
[{
"value": {
".rodata": [{
"load_static_data.bar": 18446744073709551615
},{
"num2": 24
},{
"num5": 43947
},{
"num6": 171
},{
"str0": [97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,0,0,0,0,0,0
]
},{
"struct0": {
"a": 42,
"b": 4278120431,
"c": 1229782938247303441
}
},{
"struct2": {
"a": 0,
"b": 0,
"c": 0
}
}
]
}
}
]--- tools/bpf/bpftool/btf_dumper.c | 59 ++++++++++++++++++++++++++++++++++++++++++ tools/bpf/bpftool/map.c | 10 ++++--- 2 files changed, 65 insertions(+), 4 deletions(-)
tools/bpf/bpftool/btf_dumper.c +59 -0
@@ -309,6 +309,48 @@ static int btf_dumper_struct(const struct btf_dumper *d, __u32 type_id,return ret; } +static int btf_dumper_var (const struct btf_dumper *d,__u32 type_id, +__u8 bit_offset,const void *data) +{ +const struct btf_type *t =btf__type_by_id (d->btf, type_id); +int ret; + +jsonw_start_object (d->jw); +jsonw_name (d->jw,btf__name_by_offset (d->btf, t->name_off)); + ret =btf_dumper_do_type (d, t->type, bit_offset, data); +jsonw_end_object (d->jw); + +return ret; +} + +static int btf_dumper_datasec (const struct btf_dumper *d,__u32 type_id, +const void *data) +{ +struct btf_var_secinfo *vsi; +const struct btf_type *t; +int ret =0 , i, vlen; + + t =btf__type_by_id (d->btf, type_id); +if (!t) +return -EINVAL ; + + vlen =BTF_INFO_VLEN (t->info); + vsi = (struct btf_var_secinfo *)(t +1 ); + +jsonw_start_object (d->jw); +jsonw_name (d->jw,btf__name_by_offset (d->btf, t->name_off)); +jsonw_start_array (d->jw); +for (i =0 ; i < vlen; i++) { + ret =btf_dumper_do_type (d, vsi[i].type,0 , data + vsi[i].offset); +if (ret) +break ; + } +jsonw_end_array (d->jw); +jsonw_end_object (d->jw); + +return ret; +} +static int btf_dumper_do_type (const struct btf_dumper *d,__u32 type_id,__u8 bit_offset,const void *data) {
@@ -341,6 +383,10 @@ static int btf_dumper_do_type(const struct btf_dumper *d, __u32 type_id,case BTF_KIND_CONST :case BTF_KIND_RESTRICT :return btf_dumper_modifier (d, type_id, bit_offset, data); +case BTF_KIND_VAR : +return btf_dumper_var (d, type_id, bit_offset, data); +case BTF_KIND_DATASEC : +return btf_dumper_datasec (d, type_id, data);default :jsonw_printf (d->jw,"(unsupported-kind" );return -EINVAL ;
@@ -377,6 +423,7 @@ static int __btf_dumper_type_only(const struct btf *btf, __u32 type_id, {const struct btf_type *proto_type;const struct btf_array *array; +const struct btf_var *var;const struct btf_type *t;if (!type_id) {
@@ -440,6 +487,18 @@ static int __btf_dumper_type_only(const struct btf *btf, __u32 type_id,if (pos ==-1 )return -1 ;break ; +case BTF_KIND_VAR : + var = (struct btf_var *)(t +1 ); +if (var->linkage ==BTF_VAR_STATIC ) +BTF_PRINT_ARG ("static " ); +BTF_PRINT_TYPE (t->type); +BTF_PRINT_ARG (" %s" , +btf__name_by_offset (btf, t->name_off)); +break ; +case BTF_KIND_DATASEC : +BTF_PRINT_ARG ("section (\"%s\") " , +btf__name_by_offset (btf, t->name_off)); +break ;case BTF_KIND_UNKN :default :return -1 ;
tools/bpf/bpftool/map.c +6 -4
@@ -153,11 +153,13 @@ static int do_dump_btf(const struct btf_dumper *d,/* start of key-value pair */ jsonw_start_object (d->jw); -jsonw_name (d->jw,"key" ); +if (map_info->btf_key_type_id) { +jsonw_name (d->jw,"key" ); - ret =btf_dumper_type (d, map_info->btf_key_type_id, key); -if (ret) - goto err_end_obj; + ret =btf_dumper_type (d, map_info->btf_key_type_id, key); +if (ret) + goto err_end_obj; + }if (!map_is_per_cpu (map_info->type)) {jsonw_name (d->jw,"value" );
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 13/15 bpf, selftest: test {rd,wr}only flags and direct value access
In reply to Daniel BorkmannExtend test_verifier with various test cases around the two kernel extensions, that is, {rd,wr}only map support as well as direct map value access. All passing, one skipped due to xskmap not present on test machine:
# ./test_verifier [...] #920/p XDP pkt read, pkt_data <= pkt_meta', bad access 1 OK #921/p XDP pkt read, pkt_data <= pkt_meta', bad access 2 OK Summary: 1366 PASSED, 1 SKIPPED, 0 FAILED
--- tools/include/linux/filter.h | 14 ++ tools/testing/selftests/bpf/test_verifier.c | 53 ++++- .../testing/selftests/bpf/verifier/array_access.c | 159 +++++++++++++ .../selftests/bpf/verifier/direct_value_access.c | 262 +++++++++++++++++++++ 4 files changed, 483 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/bpf/verifier/direct_value_access.c
tools/include/linux/filter.h +14 -0
@@ -283,6 +283,20 @@#define BPF_LD_MAP_FD (DST ,MAP_FD ) \ BPF_LD_IMM64_RAW(DST, BPF_PSEUDO_MAP_FD, MAP_FD) +#define BPF_LD_MAP_VALUE(DST, MAP_FD, VALUE_IDX, VALUE_OFF) \ + ((struct bpf_insn) { \ + .code = BPF_LD | BPF_DW | BPF_IMM, \ + .dst_reg = DST, \ + .src_reg = BPF_PSEUDO_MAP_VALUE, \ + .off = (__u16)(VALUE_IDX), \ + .imm = MAP_FD }), \ + ((struct bpf_insn) { \ + .code = 0, /* zero is reserved opcode */ \ + .dst_reg = 0, \ + .src_reg = 0, \ + .off = ((__u32)(VALUE_IDX)) >> 16, \ + .imm = VALUE_OFF }) +/* Relative call */ #define BPF_CALL_REL (TGT ) \
tools/testing/selftests/bpf/test_verifier.c +48 -5
99 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/array_access.c +159 -0
162 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/direct_value_access.c new file +262 -0
262 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 14/15 bpf, selftest: test global data/bss/rodata sections
In reply to Daniel BorkmannFrom: Joe Stringer <[email protected]>
Add tests for libbpf relocation of static variable references into the .data, .rodata and .bss sections of the ELF, also add read-only test for .rodata. All passing:
# ./test_progs [...] test_global_data:PASS:load program 0 nsec test_global_data:PASS:pass global data run 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .data reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .data reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_string:PASS:relocate .rodata reference 925 nsec test_global_data_string:PASS:relocate .data reference 925 nsec test_global_data_string:PASS:relocate .bss reference 925 nsec test_global_data_string:PASS:relocate .data reference 925 nsec test_global_data_string:PASS:relocate .bss reference 925 nsec test_global_data_struct:PASS:relocate .rodata reference 925 nsec test_global_data_struct:PASS:relocate .bss reference 925 nsec test_global_data_struct:PASS:relocate .rodata reference 925 nsec test_global_data_struct:PASS:relocate .data reference 925 nsec test_global_data_rdonly:PASS:test .rodata read-only map 925 nsec [...] Summary: 229 PASSED, 0 FAILED
Note map helper signatures have been changed to avoid warnings when passing in const data.
Joint work with Daniel Borkmann.
--- tools/testing/selftests/bpf/bpf_helpers.h | 8 +- .../testing/selftests/bpf/prog_tests/global_data.c | 157 +++++++++++++++++++++ .../testing/selftests/bpf/progs/test_global_data.c | 106 ++++++++++++++ 3 files changed, 267 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/global_data.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_data.c
tools/testing/selftests/bpf/bpf_helpers.h +4 -4
@@ -9,14 +9,14 @@#define SEC (NAME ) __attribute__((section(NAME), used))/* helper functions called from eBPF programs written in C */ -static void *(*bpf_map_lookup_elem)(void *map,void *key) = +static void *(*bpf_map_lookup_elem)(void *map,const void *key) = (void *) BPF_FUNC_map_lookup_elem; -static int (*bpf_map_update_elem)(void *map,void *key,void *value, +static int (*bpf_map_update_elem)(void *map,const void *key,const void *value,unsigned long long flags) = (void *) BPF_FUNC_map_update_elem; -static int (*bpf_map_delete_elem)(void *map,void *key) = +static int (*bpf_map_delete_elem)(void *map,const void *key) = (void *) BPF_FUNC_map_delete_elem; -static int (*bpf_map_push_elem)(void *map,void *value, +static int (*bpf_map_push_elem)(void *map,const void *value,unsigned long long flags) = (void *) BPF_FUNC_map_push_elem; static int (*bpf_map_pop_elem)(void *map, void *value) =
tools/testing/selftests/bpf/prog_tests/global_data.c new file +157 -0
157 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_global_data.c new file +106 -0
106 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v3 15/15 bpf, selftest: add test cases for BTF Var and DataSec
In reply to Daniel BorkmannExtend test_btf with various positive and negative tests around BTF verification of kind Var and DataSec. All passing as well:
# ./test_btf [...] BTF raw test[4] (global data test #1): OK BTF raw test[5] (global data test #2): OK BTF raw test[6] (global data test #3): OK BTF raw test[7] (global data test #4, unsupported linkage): OK BTF raw test[8] (global data test #5, invalid var type): OK BTF raw test[9] (global data test #6, invalid var type (fwd type)): OK BTF raw test[10] (global data test #7, not var kind): OK BTF raw test[11] (global data test #8, invalid var size): OK BTF raw test[12] (global data test #9, invalid var size): OK BTF raw test[13] (global data test #10, invalid var size): OK BTF raw test[14] (global data test #11, multiple section members): OK BTF raw test[15] (global data test #12, invalid offset): OK BTF raw test[16] (global data test #13, invalid offset): OK BTF raw test[17] (global data test #14, invalid offset): OK [...] PASS:159 SKIP:0 FAIL:0
--- tools/testing/selftests/bpf/test_btf.c | 552 ++++++++++++++++++++++++++++++++- 1 file changed, 550 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/test_btf.c +550 -2
@@ -85,6 +85,11 @@ static int __base_pr(enum libbpf_print_level level __attribute__((unused)),#define BTF_UNION_ENC (name, nr_elems, sz) \ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_UNION, 0, nr_elems), sz) +#define BTF_VAR_ENC (name, type, linkage) \ + BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_VAR, 0, 0), type), (linkage) +#define BTF_VAR_SECINFO_ENC (type, offset, size) \ + (type), (offset), (size) +#define BTF_MEMBER_ENC (name, type, bits_offset) \ (name), (type), (bits_offset)#define BTF_ENUM_ENC (name, val) (name), (val)
@@ -291,7 +296,6 @@ static struct btf_raw_test raw_tests[] = { .value_type_id = 3, .max_entries = 4, }, - { .descr = "struct test #3 Invalid member offset", .raw_types = {
@@ -319,7 +323,551 @@ static struct btf_raw_test raw_tests[] = { .btf_load_err = true, .err_str = "Invalid member bits_offset", }, - +/* +* struct A { +* unsigned long long m; +* int n; +* char o; +* [3 bytes hole] +* int p[8]; +* }; +*/ +{ + .descr = "global data test #1", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = "struct_test1_map", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 1, + .value_type_id = 5, + .max_entries = 4, +}, +/* + * struct A { + * unsigned long long m; + * int n; + * char o; + * [3 bytes hole] + * int p[8]; + * }; + * static struct A t; <- in .bss + */ +{ + .descr = "global data test #2", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 48), + BTF_VAR_SECINFO_ENC(6, 0, 48), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, +}, +{ + .descr = "global data test #3", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* static int t */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0t\0.bss", + .str_sec_size = sizeof("\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 3, + .max_entries = 1, +}, +{ + .descr = "global data test #4, unsupported linkage", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* static int t */ + BTF_VAR_ENC(NAME_TBD, 1, 2), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0t\0.bss", + .str_sec_size = sizeof("\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 3, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Linkage not supported", +}, +{ + .descr = "global data test #5, invalid var type", + .raw_types = { + /* static void t */ + BTF_VAR_ENC(NAME_TBD, 0, 0), /* [1] */ + /* .bss section */ /* [2] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(1, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0t\0.bss", + .str_sec_size = sizeof("\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 2, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #6, invalid var type (fwd type)", + .raw_types = { + /* union A */ + BTF_TYPE_ENC(NAME_TBD, + BTF_INFO_ENC(BTF_KIND_FWD, 1, 0), 0), /* [1] */ + /* static union A t */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0.bss", + .str_sec_size = sizeof("\0A\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 2, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type", +}, +{ + .descr = "global data test #7, invalid var type (fwd type)", + .raw_types = { + /* union A */ + BTF_TYPE_ENC(NAME_TBD, + BTF_INFO_ENC(BTF_KIND_FWD, 1, 0), 0), /* [1] */ + /* static union A t */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(1, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0.bss", + .str_sec_size = sizeof("\0A\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 2, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type", +}, +{ + .descr = "global data test #8, invalid var size", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 48), + BTF_VAR_SECINFO_ENC(6, 0, 47), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid size", +}, +{ + .descr = "global data test #9, invalid var size", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 46), + BTF_VAR_SECINFO_ENC(6, 0, 48), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid size", +}, +{ + .descr = "global data test #10, invalid var size", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 46), + BTF_VAR_SECINFO_ENC(6, 0, 46), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid size", +}, +{ + .descr = "global data test #11, multiple section members", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_VAR_SECINFO_ENC(7, 58, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, +}, +{ + .descr = "global data test #12, invalid offset", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_VAR_SECINFO_ENC(7, 60, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid offset+size", +}, +{ + .descr = "global data test #13, invalid offset", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_VAR_SECINFO_ENC(7, 12, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid offset", +}, +{ + .descr = "global data test #14, invalid offset", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(7, 58, 4), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid offset", +}, +{ + .descr = "global data test #15, not var kind", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(1, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0.bss", + .str_sec_size = sizeof("\0A\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 3, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Not a VAR kind member", +}, +{ + .descr = "global data test #16, invalid var referencing sec", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [2] */ + BTF_VAR_ENC(NAME_TBD, 2, 0), /* [3] */ + /* a section */ /* [4] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(3, 0, 4), + /* a section */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(6, 0, 4), + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [6] */ + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid resolve state", +}, +{ + .descr = "global data test #17, invalid var loop", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 2, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0aaa", + .str_sec_size = sizeof("\0A\0t\0aaa"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Loop detected", +}, /* Test member exceeds the size of struct. * * struct A {
Signature · 1 line
2.9.5