BPF support for global data
Subsystems (9)
Daniel Borkmann
maintainerv5 cover BPF support for global data
This series is a major rework of previously submitted libbpf patches [0] in order to add global data support for BPF. The kernel has been extended to add proper infrastructure that allows for full .bss/.data/.rodata sections on BPF loader side based upon feedback from LPC discussions [1]. Latter support is then also added into libbpf in this series which allows for more natural C-like programming of BPF programs. For more information on loader, please refer to 'bpf, libbpf: support global data/bss/ rodata sections' patch in this series.
Thanks a lot!
v4 -> v5:
- Removed index selection again for ldimm64 (Alexei)
- Adapted related test cases and added new ones to test
rejection of off != 0
v3 -> v4:
- Various fixes in BTF verification e.g. to disallow
Var and DataSec to be an intermediate type during resolve (Martin)
- More BTF test cases added
- Few cleanups in key-less BTF commit (Martin)
- Bump libbpf minor version from 2 to 3
- Renamed and simplified read-only locking
- Various minor improvements all over the place
v2 -> v3:
- Implement BTF support in kernel, libbpf, bpftool, add tests
- Fix idx + off conversion (Andrii)
- Document lower / higher bits for direct value access (Andrii)
- Add tests with small value size (Andrii)
- Add index selection into ldimm64 (Andrii)
- Fix missing fdput() (Jann)
- Reject invalid flags in BPF_F_*_PROG (Jakub)
- Complete rework of libbpf support, includes:
- Add objname to map name (Stanislav)
- Make .rodata map full read-only after setup (Andrii)
- Merge relocation handling into single one (Andrii)
- Store global maps into obj->maps array (Andrii, Alexei)
- Debug message when skipping section (Andrii)
- Reject non-static global data till we have
semantics for sharing them (Yonghong, Andrii, Alexei)
- More test cases and completely reworked prog test (Alexei)
- Fixes, cleanups, etc all over the set
- Not yet addressed:
- Make BTF mandatory for these maps (Alexei)
-> Waiting till BTF support for these lands first
v1 -> v2:
- Instead of 32-bit static data, implement full global data support (Alexei)
[0] https://patchwork.ozlabs.org/cover/1040290/
[1] http://vger.kernel.org/lpc-bpf2018.html#session-3
Daniel Borkmann (14):
bpf: implement lookup-free direct value access for maps
bpf: do not retain flags that are not tied to map lifetime
bpf: add program side {rd, wr}only support for maps
bpf: add syscall side map freeze support
bpf: allow . char as part of the object name
bpf: add specification for BTF Var and DataSec kinds
bpf: kernel side support for BTF Var and DataSec
bpf: allow for key-less BTF in array map
bpf: sync {btf, bpf}.h uapi header from tools infrastructure bpf, libbpf: support global data/bss/rodata sections bpf, libbpf: add support for BTF Var and DataSec
bpf: bpftool support for dumping data/bss/rodata sections bpf, selftest: test {rd, wr}only flags and direct value access bpf, selftest: add test cases for BTF Var and DataSec
Joe Stringer (2):
bpf, libbpf: refactor relocation handling bpf, selftest: test global data/bss/rodata sections
Documentation/bpf/btf.rst | 57 ++ include/linux/bpf.h | 38 +- include/linux/bpf_verifier.h | 4 + include/linux/btf.h | 1 + include/uapi/linux/bpf.h | 20 +- include/uapi/linux/btf.h | 32 +- kernel/bpf/arraymap.c | 53 +- kernel/bpf/btf.c | 419 ++++++++++++- kernel/bpf/core.c | 3 +- kernel/bpf/disasm.c | 5 +- kernel/bpf/hashtab.c | 6 +- kernel/bpf/local_storage.c | 6 +- kernel/bpf/lpm_trie.c | 3 +- kernel/bpf/queue_stack_maps.c | 6 +- kernel/bpf/syscall.c | 132 +++- kernel/bpf/verifier.c | 132 +++- tools/bpf/bpftool/btf_dumper.c | 59 ++ tools/bpf/bpftool/map.c | 10 +- tools/bpf/bpftool/xlated_dumper.c | 3 + tools/include/linux/filter.h | 21 +- tools/include/uapi/linux/bpf.h | 20 +- tools/include/uapi/linux/btf.h | 32 +- tools/lib/bpf/Makefile | 2 +- tools/lib/bpf/bpf.c | 10 + tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/btf.c | 97 ++- tools/lib/bpf/btf.h | 3 + tools/lib/bpf/libbpf.c | 546 ++++++++++++++--- tools/lib/bpf/libbpf.h | 5 + tools/lib/bpf/libbpf.map | 7 + tools/testing/selftests/bpf/bpf_helpers.h | 8 +- .../testing/selftests/bpf/prog_tests/global_data.c | 157 +++++ .../testing/selftests/bpf/progs/test_global_data.c | 106 ++++ tools/testing/selftests/bpf/test_btf.c | 665 ++++++++++++++++++++- tools/testing/selftests/bpf/test_verifier.c | 51 +- .../testing/selftests/bpf/verifier/array_access.c | 159 +++++ .../selftests/bpf/verifier/direct_value_access.c | 347 +++++++++++ 37 files changed, 3019 insertions(+), 207 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/global_data.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_data.c create mode 100644 tools/testing/selftests/bpf/verifier/direct_value_access.c
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 1/16 bpf: implement lookup-free direct value access for maps
This generic extension to BPF maps allows for directly loading an address residing inside a BPF map value as a single BPF ldimm64 instruction!
The idea is similar to what BPF_PSEUDO_MAP_FD does today, which is a special src_reg flag for ldimm64 instruction that indicates that inside the first part of the double insns's imm field is a file descriptor which the verifier then replaces as a full 64bit address of the map into both imm parts. For the newly added BPF_PSEUDO_MAP_VALUE src_reg flag, the idea is the following:
the first part of the double insns's imm field is again a file descriptor corresponding to the map, and the second part of the imm field is an offset into the value. The verifier will then replace both imm parts with an address that points into the BPF map value at the given value offset for maps that support this operation. Currently supported is array map with single entry. It is possible to support more than just single map element by reusing both 16bit off fields of the insns as a map index, so full array map lookup could be expressed that way. It hasn't been implemented here due to lack of concrete use case, but could easily be done so in future in a compatible way, since both off fields right now have to be 0 and would correctly denote a map index 0.
The BPF_PSEUDO_MAP_VALUE is a distinct flag as otherwise with BPF_PSEUDO_MAP_FD we could not differ offset 0 between load of map pointer versus load of map's value at offset 0, and changing BPF_PSEUDO_MAP_FD's encoding into off by one to differ between regular map pointer and map value pointer would add unnecessary complexity and increases barrier for debugability thus less suitable. Using the second part of the imm field as an offset into the value does /not/ come with limitations since maximum possible value size is in u32 universe anyway.
This optimization allows for efficiently retrieving an address to a map value memory area without having to issue a helper call which needs to prepare registers according to calling convention, etc, without needing the extra NULL test, and without having to add the offset in an additional instruction to the value base pointer. The verifier then treats the destination register as PTR_TO_MAP_VALUE with constant reg->off from the user passed offset from the second imm field, and guarantees that this is within bounds of the map value. Any subsequent operations are normally treated as typical map value handling without anything extra needed from verification side.
The two map operations for direct value access have been added to array map for now. In future other types could be supported as well depending on the use case. The main use case for this commit is to allow for BPF loader support for global variables that reside in .data/.rodata/.bss sections such that we can directly load the address of them with minimal additional infrastructure required. Loader support has been added in subsequent commits for libbpf library.
--- include/linux/bpf.h | 6 +++ include/linux/bpf_verifier.h | 4 ++ include/uapi/linux/bpf.h | 13 +++++- kernel/bpf/arraymap.c | 32 +++++++++++++++ kernel/bpf/core.c | 3 +- kernel/bpf/disasm.c | 5 ++- kernel/bpf/syscall.c | 28 +++++++++---- kernel/bpf/verifier.c | 86 ++++++++++++++++++++++++++++++--------- tools/bpf/bpftool/xlated_dumper.c | 3 ++ 9 files changed, 149 insertions(+), 31 deletions(-)
include/linux/bpf.h +6 -0
@@ -57,6 +57,12 @@ struct bpf_map_ops {const struct btf *btf, const struct btf_type *key_type, const struct btf_type *value_type); + +/* Direct value access helpers. */ +int (*map_direct_value_addr)(const struct bpf_map *map, +u64 *imm,u32 off); +int (*map_direct_value_meta)(const struct bpf_map *map, +u64 imm,u32 *off); };struct bpf_map {
include/linux/bpf_verifier.h +4 -0
@@ -224,6 +224,10 @@ struct bpf_insn_aux_data {unsigned long map_state;/* pointer/poison value for maps */ s32 call_imm;/* saved imm field of call insn */ u32 alu_limit;/* limit for add/sub register with pointer */ +struct { +u32 map_index;/* index into used_maps[] */ +u32 map_off;/* offset from value base address */ + }; };int ctx_field_size;/* the ctx field size for load insn, maybe 0 */ int sanitize_stack_off;/* stack slot to be cleared */
include/uapi/linux/bpf.h +12 -1
@@ -255,8 +255,19 @@ enum bpf_attach_type {*/ #define BPF_F_ANY_ALIGNMENT (1U << 1) -/* when bpf_ldimm64->src_reg == BPF_PSEUDO_MAP_FD, bpf_ldimm64->imm == fd */ +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have +* two extensions: +* +* insn[0].src_reg: BPF_PSEUDO_MAP_FD BPF_PSEUDO_MAP_VALUE +* insn[0].imm: map fd map fd +* insn[1].imm: 0 offset into value +* insn[0].off: 0 0 +* insn[1].off: 0 0 +* ldimm64 rewrite: address of map address of map[0]+offset +* verifier type: CONST_PTR_TO_MAP PTR_TO_MAP_VALUE +*/ #define BPF_PSEUDO_MAP_FD 1 +#define BPF_PSEUDO_MAP_VALUE 2/* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative * offset to another bpf function
kernel/bpf/arraymap.c +32 -0
@@ -160,6 +160,36 @@ static void *array_map_lookup_elem(struct bpf_map *map, void *key)return array->value + array->elem_size * (index & array->index_mask); } +static int array_map_direct_value_addr (const struct bpf_map *map,u64 *imm, +u32 off) +{ + struct bpf_array *array = container_of(map, struct bpf_array, map); + +if (map->max_entries !=1 ) +return -ENOTSUPP ; +if (off >= map->value_size) +return -EINVAL ; + + *imm = (unsigned long )array->value; +return 0 ; +} + +static int array_map_direct_value_meta (const struct bpf_map *map,u64 imm, +u32 *off) +{ + struct bpf_array *array = container_of(map, struct bpf_array, map); +u64 base = (unsigned long )array->value; +u64 range = array->elem_size; + +if (map->max_entries !=1 ) +return -ENOTSUPP ; +if (imm < base || imm >= base + range) +return -ENOENT ; + + *off = imm - base; +return 0 ; +} +/* emit BPF instructions equivalent to C code of array_map_lookup_elem() */ static u32 array_map_gen_lookup (struct bpf_map *map,struct bpf_insn *insn_buf) {
@@ -419,6 +449,8 @@ const struct bpf_map_ops array_map_ops = { .map_update_elem = array_map_update_elem, .map_delete_elem = array_map_delete_elem, .map_gen_lookup = array_map_gen_lookup, + .map_direct_value_addr = array_map_direct_value_addr, + .map_direct_value_meta = array_map_direct_value_meta, .map_seq_show_elem = array_map_seq_show_elem, .map_check_btf = array_map_check_btf, };
kernel/bpf/core.c +2 -1
@@ -292,7 +292,8 @@ int bpf_prog_calc_tag(struct bpf_prog *fp) dst[i] = fp->insnsi[i];if (!was_ld_map && dst[i].code == (BPF_LD |BPF_IMM |BPF_DW ) && - dst[i].src_reg ==BPF_PSEUDO_MAP_FD ) { + (dst[i].src_reg ==BPF_PSEUDO_MAP_FD || + dst[i].src_reg ==BPF_PSEUDO_MAP_VALUE )) { was_ld_map = true; dst[i].imm =0 ; } else if (was_ld_map &&
kernel/bpf/disasm.c +3 -2
@@ -205,10 +205,11 @@ void print_bpf_insn(const struct bpf_insn_cbs *cbs,* part of the ldimm64 insn is accessible. */ u64 imm = ((u64)(insn +1 )->imm <<32 ) | (u32 )insn->imm; -bool map_ptr = insn->src_reg ==BPF_PSEUDO_MAP_FD ; +bool is_ptr = insn->src_reg ==BPF_PSEUDO_MAP_FD || + insn->src_reg ==BPF_PSEUDO_MAP_VALUE ;char tmp[64 ]; -if (map_ptr && !allow_ptr_leaks) +if (is_ptr && !allow_ptr_leaks) imm =0 ; verbose(cbs->private_data, "(%02x) r%d = %s\n",
kernel/bpf/syscall.c +21 -7
@@ -2072,13 +2072,26 @@ static int bpf_map_get_fd_by_id(const union bpf_attr *attr) }static const struct bpf_map *bpf_map_from_imm (const struct bpf_prog *prog, -unsigned long addr) +unsigned long addr,u32 *off, +u32 *type) { +const struct bpf_map *map;int i; -for (i =0 ; i < prog->aux->used_map_cnt; i++) -if (prog->aux->used_maps[i] == (void *)addr) -return prog->aux->used_maps[i]; +for (i =0 , *off =0 ; i < prog->aux->used_map_cnt; i++) { + map = prog->aux->used_maps[i]; +if (map == (void *)addr) { + *type =BPF_PSEUDO_MAP_FD ; +return map; + } +if (!map->ops->map_direct_value_meta) +continue ; +if (!map->ops->map_direct_value_meta (map, addr, off)) { + *type =BPF_PSEUDO_MAP_VALUE ; +return map; + } + } +return NULL ; }
@@ -2086,6 +2099,7 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog) {const struct bpf_map *map;struct bpf_insn *insns; +u32 off, type;u64 imm;int i;
@@ -2113,11 +2127,11 @@ static struct bpf_insn *bpf_insn_prepare_dump(const struct bpf_prog *prog)continue ; imm = ((u64 )insns[i +1 ].imm <<32 ) | (u32 )insns[i].imm; - map =bpf_map_from_imm (prog, imm); + map =bpf_map_from_imm (prog, imm, &off, &type);if (map) { - insns[i].src_reg =BPF_PSEUDO_MAP_FD ; + insns[i].src_reg = type; insns[i].imm = map->id; - insns[i +1 ].imm =0 ; + insns[i +1 ].imm = off;continue ; } }
kernel/bpf/verifier.c +66 -20
138 lines not on this page · raw message
tools/bpf/bpftool/xlated_dumper.c +3 -0
@@ -195,6 +195,9 @@ static const char *print_imm(void *private_data,if (insn->src_reg ==BPF_PSEUDO_MAP_FD )snprintf (dd->scratch_buff,sizeof (dd->scratch_buff),"map[id:%u]" , insn->imm); +else if (insn->src_reg ==BPF_PSEUDO_MAP_VALUE ) +snprintf (dd->scratch_buff,sizeof (dd->scratch_buff), +"map[id:%u][0]+%u" , insn->imm, (insn +1 )->imm);else snprintf (dd->scratch_buff,sizeof (dd->scratch_buff),"0x%llx" , (unsigned long long )full_imm);
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 2/16 bpf: do not retain flags that are not tied to map lifetime
In reply to Daniel BorkmannBoth BPF_F_WRONLY / BPF_F_RDONLY flags are tied to the map file descriptor, but not to the map object itself! Meaning, at map creation time BPF_F_RDONLY can be set to make the map read-only from syscall side, but this holds only for the returned fd, so any other fd either retrieved via bpf file system or via map id for the very same underlying map object can have read-write access instead.
Given that, keeping the two flags around in the map_flags attribute and exposing them to user space upon map dump is misleading and may lead to false conclusions. Since these two flags are not tied to the map object lets also not store them as map property.
--- kernel/bpf/syscall.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
kernel/bpf/syscall.c +13 -1
@@ -166,13 +166,25 @@ void bpf_map_area_free(void *area)kvfree (area); } +static u32 bpf_map_flags_retain_permanent (u32 flags) +{ +/* Some map creation flags are not tied to the map object but +* rather to the map fd instead, so they have no meaning upon +* map object inspection since multiple file descriptors with +* different (access) properties can exist here. Thus, given +* this has zero meaning for the map itself, lets clear these +* from here. +*/ +return flags & ~(BPF_F_RDONLY |BPF_F_WRONLY ); +} +void bpf_map_init_from_attr (struct bpf_map *map,union bpf_attr *attr) { map->map_type = attr->map_type; map->key_size = attr->key_size; map->value_size = attr->value_size; map->max_entries = attr->max_entries; - map->map_flags = attr->map_flags; + map->map_flags =bpf_map_flags_retain_permanent (attr->map_flags); map->numa_node =bpf_map_attr_numa_node (attr); }
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 3/16 bpf: add program side {rd, wr}only support for maps
In reply to Daniel BorkmannThis work adds two new map creation flags BPF_F_RDONLY_PROG and BPF_F_WRONLY_PROG in order to allow for read-only or write-only BPF maps from a BPF program side.
Today we have BPF_F_RDONLY and BPF_F_WRONLY, but this only applies to system call side, meaning the BPF program has full read/write access to the map as usual while bpf(2) calls with map fd can either only read or write into the map depending on the flags. BPF_F_RDONLY_PROG and BPF_F_WRONLY_PROG allows for the exact opposite such that verifier is going to reject program loads if write into a read-only map or a read into a write-only map is detected. For read-only map case also some helpers are forbidden for programs that would alter the map state such as map deletion, update, etc. As opposed to the two BPF_F_RDONLY / BPF_F_WRONLY flags, BPF_F_RDONLY_PROG as well as BPF_F_WRONLY_PROG really do correspond to the map lifetime.
We've enabled this generic map extension to various non-special maps holding normal user data: array, hash, lru, lpm, local storage, queue and stack. Further generic map types could be followed up in future depending on use-case. Main use case here is to forbid writes into .rodata map values from verifier side.
--- include/linux/bpf.h | 29 +++++++++++++++++++++++++++ include/uapi/linux/bpf.h | 6 +++++- kernel/bpf/arraymap.c | 6 +++++- kernel/bpf/hashtab.c | 6 +++--- kernel/bpf/local_storage.c | 6 +++--- kernel/bpf/lpm_trie.c | 3 ++- kernel/bpf/queue_stack_maps.c | 6 +++--- kernel/bpf/syscall.c | 2 ++ kernel/bpf/verifier.c | 46 +++++++++++++++++++++++++++++++++++++++++-- 9 files changed, 96 insertions(+), 14 deletions(-)
include/linux/bpf.h +29 -0
@@ -430,6 +430,35 @@ struct bpf_array {#define BPF_COMPLEXITY_LIMIT_INSNS 1000000/* yes. 1M insns */ #define MAX_TAIL_CALL_CNT 32 +#define BPF_F_ACCESS_MASK (BPF_F_RDONLY | \ + BPF_F_RDONLY_PROG | \ + BPF_F_WRONLY | \ + BPF_F_WRONLY_PROG) + +#define BPF_MAP_CAN_READ BIT(0) +#define BPF_MAP_CAN_WRITE BIT(1) + +static inline u32 bpf_map_flags_to_cap (struct bpf_map *map) +{ +u32 access_flags = map->map_flags & (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG ); + +/* Combination of BPF_F_RDONLY_PROG | BPF_F_WRONLY_PROG is +* not possible. +*/ +if (access_flags &BPF_F_RDONLY_PROG ) +return BPF_MAP_CAN_READ ; +else if (access_flags &BPF_F_WRONLY_PROG ) +return BPF_MAP_CAN_WRITE ; +else +return BPF_MAP_CAN_READ |BPF_MAP_CAN_WRITE ; +} + +static inline bool bpf_map_flags_access_ok (u32 access_flags) +{ +return (access_flags & (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG )) != + (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG ); +} +struct bpf_event_entry {struct perf_event *event;struct file *perf_file;
include/uapi/linux/bpf.h +5 -1
@@ -294,7 +294,7 @@ enum bpf_attach_type {#define BPF_OBJ_NAME_LEN 16U -/* Flags for accessing BPF object */ +/* Flags for accessing BPF object from syscall side. */ #define BPF_F_RDONLY (1U << 3)#define BPF_F_WRONLY (1U << 4)
@@ -304,6 +304,10 @@ enum bpf_attach_type {/* Zero-initialize hash function seed. This should only be used for testing. */ #define BPF_F_ZERO_SEED (1U << 6) +/* Flags for accessing BPF object from program side. */ +#define BPF_F_RDONLY_PROG (1U << 7) +#define BPF_F_WRONLY_PROG (1U << 8) +/* flags for BPF_PROG_QUERY */ #define BPF_F_QUERY_EFFECTIVE (1U << 0)
kernel/bpf/arraymap.c +5 -1
@@ -22,7 +22,7 @@#include "map_in_map.h" #define ARRAY_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK)static void bpf_array_free_percpu (struct bpf_array *array) {
@@ -63,6 +63,7 @@ int array_map_alloc_check(union bpf_attr *attr)if (attr->max_entries ==0 || attr->key_size !=4 || attr->value_size ==0 || attr->map_flags & ~ARRAY_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags) || (percpu && numa_node !=NUMA_NO_NODE ))return -EINVAL ;
@@ -472,6 +473,9 @@ static int fd_array_map_alloc_check(union bpf_attr *attr)/* only file descriptors can be stored in this type of map */ if (attr->value_size !=sizeof (u32))return -EINVAL ; +/* Program read-only/write-only not supported for special maps yet. */ +if (attr->map_flags & (BPF_F_RDONLY_PROG |BPF_F_WRONLY_PROG )) +return -EINVAL ;return array_map_alloc_check (attr); }
kernel/bpf/hashtab.c +3 -3
@@ -23,7 +23,7 @@#define HTAB_CREATE_FLAG_MASK \ (BPF_F_NO_PREALLOC | BPF_F_NO_COMMON_LRU | BPF_F_NUMA_NODE | \ - BPF_F_RDONLY | BPF_F_WRONLY | BPF_F_ZERO_SEED) + BPF_F_ACCESS_MASK | BPF_F_ZERO_SEED)struct bucket {struct hlist_nulls_head head;
@@ -262,8 +262,8 @@ static int htab_map_alloc_check(union bpf_attr *attr)/* Guard against local DoS, and discourage production use. */ return -EPERM ; -if (attr->map_flags & ~HTAB_CREATE_FLAG_MASK ) -/* reserved bits should not be used */ +if (attr->map_flags & ~HTAB_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags))return -EINVAL ;if (!lru && percpu_lru)
kernel/bpf/local_storage.c +3 -3
@@ -14,7 +14,7 @@ DEFINE_PER_CPU(struct bpf_cgroup_storage*, bpf_cgroup_storage[MAX_BPF_CGROUP_STO #ifdef CONFIG_CGROUP_BPF #define LOCAL_STORAGE_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK) struct bpf_cgroup_storage_map { struct bpf_map map;
@@ -282,8 +282,8 @@ static struct bpf_map *cgroup_storage_map_alloc(union bpf_attr *attr)if (attr->value_size >PAGE_SIZE )return ERR_PTR (-E2BIG ); -if (attr->map_flags & ~LOCAL_STORAGE_CREATE_FLAG_MASK ) -/* reserved bits should not be used */ +if (attr->map_flags & ~LOCAL_STORAGE_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags))return ERR_PTR (-EINVAL );if (attr->max_entries)
kernel/bpf/lpm_trie.c +2 -1
@@ -538,7 +538,7 @@ static int trie_delete_elem(struct bpf_map *map, void *_key)#define LPM_KEY_SIZE_MIN LPM_KEY_SIZE(LPM_DATA_SIZE_MIN)#define LPM_CREATE_FLAG_MASK (BPF_F_NO_PREALLOC | BPF_F_NUMA_NODE | \ - BPF_F_RDONLY | BPF_F_WRONLY) + BPF_F_ACCESS_MASK)static struct bpf_map *trie_alloc (union bpf_attr *attr) {
@@ -553,6 +553,7 @@ static struct bpf_map *trie_alloc(union bpf_attr *attr) if (attr->max_entries == 0 || !(attr->map_flags & BPF_F_NO_PREALLOC) || attr->map_flags & ~LPM_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok(attr->map_flags) || attr->key_size < LPM_KEY_SIZE_MIN || attr->key_size > LPM_KEY_SIZE_MAX || attr->value_size < LPM_VAL_SIZE_MIN ||
kernel/bpf/queue_stack_maps.c +3 -3
@@ -11,8 +11,7 @@#include "percpu_freelist.h" #define QUEUE_STACK_CREATE_FLAG_MASK \ - (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY) - + (BPF_F_NUMA_NODE | BPF_F_ACCESS_MASK)struct bpf_queue_stack {struct bpf_map map;
@@ -52,7 +51,8 @@ static int queue_stack_map_alloc_check(union bpf_attr *attr)/* check sanity of attributes */ if (attr->max_entries ==0 || attr->key_size !=0 || attr->value_size ==0 || - attr->map_flags & ~QUEUE_STACK_CREATE_FLAG_MASK ) + attr->map_flags & ~QUEUE_STACK_CREATE_FLAG_MASK || + !bpf_map_flags_access_ok (attr->map_flags))return -EINVAL ;if (attr->value_size >KMALLOC_MAX_SIZE )
kernel/bpf/syscall.c +2 -0
@@ -501,6 +501,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, map->spin_lock_off =btf_find_spin_lock (btf, value_type);if (map_value_has_spin_lock (map)) { +if (map->map_flags &BPF_F_RDONLY_PROG ) +return -EACCES ;if (map->map_type !=BPF_MAP_TYPE_HASH && map->map_type !=BPF_MAP_TYPE_ARRAY && map->map_type !=BPF_MAP_TYPE_CGROUP_STORAGE )
kernel/bpf/verifier.c +44 -2
80 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 4/16 bpf: add syscall side map freeze support
In reply to Daniel BorkmannThis patch adds a new BPF_MAP_FREEZE command which allows to "freeze" the map globally as read-only / immutable from syscall side.
Map permission handling has been refactored into map_get_sys_perms() and drops FMODE_CAN_WRITE in case of locked map. Main use case is to allow for setting up .rodata sections from the BPF ELF which are loaded into the kernel, meaning BPF loader first allocates map, sets up map value by copying .rodata section into it and once complete, it calls BPF_MAP_FREEZE on the map fd to prevent further modifications.
Right now BPF_MAP_FREEZE only takes map fd as argument while remaining bpf_attr members are required to be zero. I didn't add write-only locking here as counterpart since I don't have a concrete use-case for it on my side, and I think it makes probably more sense to wait once there is actually one. In that case bpf_attr can be extended as usual with a flag field and/or others where flag 0 means that we lock the map read-only hence this doesn't prevent to add further extensions to BPF_MAP_FREEZE upon need.
A map creation flag like BPF_F_WRONCE was not considered for couple of reasons: i) in case of a generic implementation, a map can consist of more than just one element, thus there could be multiple map updates needed to set the map into a state where it can then be made immutable, ii) WRONCE indicates exact one-time write before it is then set immutable. A generic implementation would set a bit atomically on map update entry (if unset), indicating that every subsequent update from then onwards will need to bail out there. However, map updates can fail, so upon failure that flag would need to be unset again and the update attempt would need to be repeated for it to be eventually made immutable. While this can be made race-free, this approach feels less clean and in combination with reason i), it's not generic enough. A dedicated BPF_MAP_FREEZE command directly sets the flag, allows all pending operations to finish and caller has the guarantee that map is immutable from syscall side upon successful return, which is also more intuitive from an API point of view. A command name such as BPF_MAP_LOCK has been avoided as it's too close with BPF map spin locks (which already has BPF_F_LOCK flag). BPF_MAP_FREEZE is so far only enabled for privileged users.
--- include/linux/bpf.h | 3 ++- include/uapi/linux/bpf.h | 1 + kernel/bpf/syscall.c | 67 +++++++++++++++++++++++++++++++++++++++--------- 3 files changed, 58 insertions(+), 13 deletions(-)
include/linux/bpf.h +2 -1
@@ -87,7 +87,8 @@ struct bpf_map {struct btf *btf;u32 pages;bool unpriv_array; -/* 51 bytes hole */ +bool frozen;/* write-once */ +/* 48 bytes hole */ /* The 3rd and 4th cacheline with misc members to avoid false sharing * particularly with refcounting.
include/uapi/linux/bpf.h +1 -0
@@ -105,6 +105,7 @@ enum bpf_cmd {BPF_BTF_GET_FD_BY_ID ,BPF_TASK_FD_QUERY ,BPF_MAP_LOOKUP_AND_DELETE_ELEM , + BPF_MAP_FREEZE, }; enum bpf_map_type {
kernel/bpf/syscall.c +55 -12
127 lines not on this page · raw message
Signature · 1 line
2.9.5
Re: [PATCH bpf-next v5 04/16] bpf: add syscall side map freeze support
Quoted 41 lines · Daniel Borkmann
This patch adds a new
BPF_MAP_FREEZEcommand which allows to "freeze" the map globally as read-only / immutable from syscall side.Map permission handling has been refactored into
map_get_sys_perms()and dropsFMODE_CAN_WRITEin case of locked map. Main use case is to allow for setting up .rodata sections from the BPF ELF which are loaded into the kernel, meaning BPF loader first allocates map, sets up map value by copying .rodata section into it and once complete, it callsBPF_MAP_FREEZEon the map fd to prevent further modifications.Right now
BPF_MAP_FREEZEonly takes map fd as argument while remainingbpf_attrmembers are required to be zero. I didn't add write-only locking here as counterpart since I don't have a concrete use-case for it on my side, and I think it makes probably more sense to wait once there is actually one. In that casebpf_attrcan be extended as usual with a flag field and/or others where flag 0 means that we lock the map read-only hence this doesn't prevent to add further extensions toBPF_MAP_FREEZEupon need.A map creation flag like
BPF_F_WRONCEwas not considered for couple of reasons: i) in case of a generic implementation, a map can consist of more than just one element, thus there could be multiple map updates needed to set the map into a state where it can then be made immutable, ii) WRONCE indicates exact one-time write before it is then set immutable. A generic implementation would set a bit atomically on map update entry (if unset), indicating that every subsequent update from then onwards will need to bail out there. However, map updates can fail, so upon failure that flag would need to be unset again and the update attempt would need to be repeated for it to be eventually made immutable. While this can be made race-free, this approach feels less clean and in combination with reason i), it's not generic enough. A dedicatedBPF_MAP_FREEZEcommand directly sets the flag, allows all pending operations to finish and caller has the guarantee that map is immutable from syscall side upon successful return, which is also more intuitive from an API point of view. A command name such asBPF_MAP_LOCKhas been avoided as it's too close with BPF map spin locks (which already hasBPF_F_LOCKflag).BPF_MAP_FREEZEis so far only enabled for privileged users.
[...]
Quoted 10 lines
kernel/bpf/syscall.c +1
-2@@ -857,8 +870,7 @@ static int map_update_elem(union bpf_attr *attr) map =__bpf_map_get (f);if (IS_ERR (map))return PTR_ERR (map); - -if (!(f.file->f_mode &FMODE_CAN_WRITE )) { +if (!(map_get_sys_perms (map, f) &FMODE_CAN_WRITE )) { err = -EPERM ; goto err_put; }
[...]
Quoted 14 lines
kernel/bpf/syscall.c +14
-0+static int map_freeze (const union bpf_attr *attr) +{ +int err =0 , ufd = attr->map_fd; +struct bpf_map *map; +struct fd f; + +if (CHECK_ATTR (BPF_MAP_FREEZE )) +return -EINVAL ; + + f =fdget (ufd); + map =__bpf_map_get (f); +if (IS_ERR (map)) +return PTR_ERR (map); +if (READ_ONCE (map->frozen)) {
kernel/bpf/syscall.c static int map_lookup_and_delete_elem(union bpf_attr *attr) · line 1169
Quoted 10 lines
kernel/bpf/syscall.c +10
-0+ err = -EBUSY ; + goto err_put; + } +if (!capable (CAP_SYS_ADMIN )) { + err = -EPERM ; + goto err_put; + } + +WRITE_ONCE (map->frozen, true); +synchronize_rcu ();
What is this synchronize_rcu() doing? It seems like your intent might be to ensure that all pending writes from the syscall side have finished by the time this returns, but functions like map_update_elem() aren't in an RCU read-side critical section when they check for ->frozen, so that doesn't work, right?
Quoted 4 lines
kernel/bpf/syscall.c +4
-0+err_put: +fdput (f); +return err; +}
Daniel Borkmann
maintainerRe: [PATCH bpf-next v5 04/16] bpf: add syscall side map freeze support
Quoted 76 lines · Jann Horn
Quoted 41 lines · Daniel Borkmann
This patch adds a new
BPF_MAP_FREEZEcommand which allows to "freeze" the map globally as read-only / immutable from syscall side.Map permission handling has been refactored into
map_get_sys_perms()and dropsFMODE_CAN_WRITEin case of locked map. Main use case is to allow for setting up .rodata sections from the BPF ELF which are loaded into the kernel, meaning BPF loader first allocates map, sets up map value by copying .rodata section into it and once complete, it callsBPF_MAP_FREEZEon the map fd to prevent further modifications.Right now
BPF_MAP_FREEZEonly takes map fd as argument while remainingbpf_attrmembers are required to be zero. I didn't add write-only locking here as counterpart since I don't have a concrete use-case for it on my side, and I think it makes probably more sense to wait once there is actually one. In that casebpf_attrcan be extended as usual with a flag field and/or others where flag 0 means that we lock the map read-only hence this doesn't prevent to add further extensions toBPF_MAP_FREEZEupon need.A map creation flag like
BPF_F_WRONCEwas not considered for couple of reasons: i) in case of a generic implementation, a map can consist of more than just one element, thus there could be multiple map updates needed to set the map into a state where it can then be made immutable, ii) WRONCE indicates exact one-time write before it is then set immutable. A generic implementation would set a bit atomically on map update entry (if unset), indicating that every subsequent update from then onwards will need to bail out there. However, map updates can fail, so upon failure that flag would need to be unset again and the update attempt would need to be repeated for it to be eventually made immutable. While this can be made race-free, this approach feels less clean and in combination with reason i), it's not generic enough. A dedicatedBPF_MAP_FREEZEcommand directly sets the flag, allows all pending operations to finish and caller has the guarantee that map is immutable from syscall side upon successful return, which is also more intuitive from an API point of view. A command name such asBPF_MAP_LOCKhas been avoided as it's too close with BPF map spin locks (which already hasBPF_F_LOCKflag).BPF_MAP_FREEZEis so far only enabled for privileged users.[...]
Quoted 10 lines
kernel/bpf/syscall.c +1
-2@@ -857,8 +870,7 @@ static int map_update_elem(union bpf_attr *attr) map =__bpf_map_get (f);if (IS_ERR (map))return PTR_ERR (map); - -if (!(f.file->f_mode &FMODE_CAN_WRITE )) { +if (!(map_get_sys_perms (map, f) &FMODE_CAN_WRITE )) { err = -EPERM ; goto err_put; }[...]
Quoted 22 lines
kernel/bpf/syscall.c +22
-0+static int map_freeze (const union bpf_attr *attr) +{ +int err =0 , ufd = attr->map_fd; +struct bpf_map *map; +struct fd f; + +if (CHECK_ATTR (BPF_MAP_FREEZE )) +return -EINVAL ; + + f =fdget (ufd); + map =__bpf_map_get (f); +if (IS_ERR (map)) +return PTR_ERR (map); +if (READ_ONCE (map->frozen)) { + err = -EBUSY ; + goto err_put; + } +if (!capable (CAP_SYS_ADMIN )) { + err = -EPERM ; + goto err_put; + } +
Quoted 8 lines
kernel/bpf/syscall.c static int map_lookup_and_delete_elem(union bpf_attr *attr) · line 1169
Quoted 2 lines
kernel/bpf/syscall.c +2
-0+WRITE_ONCE (map->frozen, true); +synchronize_rcu ();What is this
synchronize_rcu()doing? It seems like your intent might be to ensure that all pending writes from the syscall side have finished by the time this returns, but functions likemap_update_elem()aren't in an RCU read-side critical section when they check for ->frozen, so that doesn't work, right?
Hmm, yes, true, good point this would only partially work. Considering that, it's best to just remove the synchronize_rcu() from there. Will do a v6 with that part removed.
Thanks,
Daniel
Daniel Borkmann
maintainerPATCH v5 5/16 bpf: allow . char as part of the object name
In reply to Daniel BorkmannTrivial addition to allow '.' aside from '_' as "special" characters in the object name. Used to allow for substrings in maps from loader side such as ".bss", ".data", ".rodata", but could also be useful for other purposes.
--- kernel/bpf/syscall.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/bpf/syscall.c +3 -3
@@ -474,10 +474,10 @@ static int bpf_obj_name_cpy(char *dst, const char *src)const char *end = src +BPF_OBJ_NAME_LEN ;memset (dst,0 ,BPF_OBJ_NAME_LEN ); - -/* Copy all isalnum() and '_' char */ +/* Copy all isalnum(), '_' and '.' chars. */ while (src < end && *src) { -if (!isalnum (*src) && *src !='_' ) +if (!isalnum (*src) && + *src !='_' && *src !='.' )return -EINVAL ; *dst++ = *src++; }
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 6/16 bpf: add specification for BTF Var and DataSec kinds
In reply to Daniel BorkmannThis adds the BTF specification and UAPI bits for supporting BTF Var and DataSec kinds. This is following LLVM upstream commit ac4082b77e07 ("[BPF] Add BTF Var and DataSec Support") which has been merged recently. Var itself is for describing a global variable and DataSec to describe ELF sections e.g. data/bss/rodata sections that hold one or multiple global variables.
--- Documentation/bpf/btf.rst | 57 +++++++++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/btf.h | 32 ++++++++++++++++++++++---- 2 files changed, 85 insertions(+), 4 deletions(-)
Documentation/bpf/btf.rst +57 -0
@@ -82,6 +82,8 @@ sequentially and type id is assigned to each recognized type starting from id #define BTF_KIND_RESTRICT 11 /* Restrict */ #define BTF_KIND_FUNC 12 /* Function */ #define BTF_KIND_FUNC_PROTO 13 /* Function Proto */ + #define BTF_KIND_VAR 14 /* Variable */ + #define BTF_KIND_DATASEC 15 /* Section */ Note that the type section encodes debug info, not just pure types. ``BTF_KIND_FUNC`` is not a type, and it represents a defined subprogram.
@@ -393,6 +395,61 @@ refers to parameter type. If the function has variable arguments, the last parameter is encoded with ``name_off = 0`` and ``type = 0``. +2.2.14 BTF_KIND_VAR +~~~~~~~~~~~~~~~~~~~ + +``struct btf_type`` encoding requirement: + * ``name_off``: offset to a valid C identifier + * ``info.kind_flag``: 0 + * ``info.kind``: BTF_KIND_VAR + * ``info.vlen``: 0 + * ``type``: the type of the variable + +``btf_type`` is followed by a single ``struct btf_variable`` with the +following data:: + + struct btf_var { + __u32 linkage; + }; + +``struct btf_var`` encoding: + * ``linkage``: currently only static variable 0, or globally allocated + variable in ELF sections 1 + +Not all type of global variables are supported by LLVM at this point. +The following is currently available: + + * static variables with or without section attributes + * global variables with section attributes + +The latter is for future extraction of map key/value type id's from a +map definition. + +2.2.15 BTF_KIND_DATASEC +~~~~~~~~~~~~~~~~~~~~~~~ + +``struct btf_type`` encoding requirement: + * ``name_off``: offset to a valid name associated with a variable or + one of .data/.bss/.rodata + * ``info.kind_flag``: 0 + * ``info.kind``: BTF_KIND_DATASEC + * ``info.vlen``: # of variables + * ``size``: total section size in bytes (0 at compilation time, patched + to actual size by BPF loaders such as libbpf) + +``btf_type`` is followed by ``info.vlen`` number of ``struct btf_var_secinfo``.:: + + struct btf_var_secinfo { + __u32 type; + __u32 offset; + __u32 size; + }; + +``struct btf_var_secinfo`` encoding: + * ``type``: the type of the BTF_KIND_VAR variable + * ``offset``: the in-section offset of the variable + * ``size``: the size of the variable in bytes + 3. BTF Kernel API *****************
include/uapi/linux/btf.h +28 -4
@@ -39,11 +39,11 @@ struct btf_type {* struct, union and fwd */ __u32 info; -/* "size" is used by INT, ENUM, STRUCT and UNION. +/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC. * "size" tells the size of the type it is describing. * * "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT, -* FUNC and FUNC_PROTO. +* FUNC, FUNC_PROTO and VAR. * "type" is a type_id referring to another type. */ union {
@@ -70,8 +70,10 @@ struct btf_type {#define BTF_KIND_RESTRICT 11/* Restrict */ #define BTF_KIND_FUNC 12/* Function */ #define BTF_KIND_FUNC_PROTO 13/* Function Proto */ -#define BTF_KIND_MAX 13 -#define NR_BTF_KINDS 14 +#define BTF_KIND_VAR 14/* Variable */ +#define BTF_KIND_DATASEC 15/* Section */ +#define BTF_KIND_MAX BTF_KIND_DATASEC +#define NR_BTF_KINDS (BTF_KIND_MAX + 1)/* For some specific BTF_KIND, "struct btf_type" is immediately * followed by extra data.
@@ -138,4 +140,26 @@ struct btf_param { __u32 type; }; +enum { +BTF_VAR_STATIC =0 , +BTF_VAR_GLOBAL_ALLOCATED , +}; + +/* BTF_KIND_VAR is followed by a single "struct btf_var" to describe +* additional information related to the variable such as its linkage. +*/ +struct btf_var { +__u32 linkage; +}; + +/* BTF_KIND_DATASEC is followed by multiple "struct btf_var_secinfo" +* to describe all BTF_KIND_VAR types it contains along with it's +* in-section offset as well as size. +*/ +struct btf_var_secinfo { +__u32 type; +__u32 offset; +__u32 size; +}; +#endif /* _UAPI__LINUX_BTF_H__ */
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 7/16 bpf: kernel side support for BTF Var and DataSec
In reply to Daniel BorkmannThis work adds kernel-side verification, logging and seq_show dumping of BTF Var and DataSec kinds which are emitted with latest LLVM. The following constraints apply:
BTF Var must have:
- Its kind_flag is 0
- Its vlen is 0
- Must point to a valid type
- Type must not resolve to a forward type
- Size of underlying type must be > 0
- Must have a valid name
- Can only be a source type, not sink or intermediate one
- Name may include dots (e.g. in case of static variables inside functions)
- Cannot be a member of a struct/union
- Linkage so far can either only be static or global/allocated
BTF DataSec must have:
- Its kind_flag is 0
- Its vlen cannot be 0
- Its size cannot be 0
- Must have a valid name
- Can only be a source type, not sink or intermediate one
- Name may include dots (e.g. to represent .bss, .data, .rodata etc)
- Cannot be a member of a struct/union
- Inner btf_var_secinfo array with {type,offset,size} triple must be sorted by offset in ascending order
- Type must always point to BTF Var
- BTF resolved size of Var must be <= size provided by triple
- DataSec size must be >= sum of triple sizes (thus holes are allowed)
btf_var_resolve(), btf_ptr_resolve() and btf_modifier_resolve() are on a high level quite similar but each come with slight, subtle differences. They could potentially be a bit refactored in future which hasn't been done here to ease review.
--- kernel/bpf/btf.c | 417 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 397 insertions(+), 20 deletions(-)
kernel/bpf/btf.c +397 -20
@@ -185,6 +185,16 @@ i <btf_type_vlen (struct_type); \ i++, member++) +#define for_each_vsi (i, struct_type, member) \ + for (i = 0, member = btf_type_var_secinfo(struct_type); \ + i < btf_type_vlen(struct_type); \ + i++, member++) + +#define for_each_vsi_from (i, from, struct_type, member) \ + for (i = from, member = btf_type_var_secinfo(struct_type) + from; \ + i < btf_type_vlen(struct_type); \ + i++, member++) +static DEFINE_IDR (btf_idr);static DEFINE_SPINLOCK (btf_idr_lock);
@@ -262,6 +272,8 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_RESTRICT] = "RESTRICT", [BTF_KIND_FUNC] = "FUNC", [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO", + [BTF_KIND_VAR] = "VAR", + [BTF_KIND_DATASEC] = "DATASEC", };struct btf_kind_operations {
@@ -375,13 +387,36 @@ static bool btf_type_is_int(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_INT ; } +static bool btf_type_is_var (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_VAR ; +} + +static bool btf_type_is_datasec (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ; +} + +/* Types that act only as a source, not sink or intermediate +* type when resolving. +*/ +static bool btf_type_is_resolve_source_only (const struct btf_type *t) +{ +return btf_type_is_var (t) || +btf_type_is_datasec (t); +} +/* What types need to be resolved? * * btf_type_is_modifier() is an obvious one. * * btf_type_is_struct() because its member refers to * another type (through member->type). - +* +* btf_type_is_var() because the variable refers to +* another type. btf_type_is_datasec() holds multiple +* btf_type_is_var() types that need resolving. +* * btf_type_is_array() because its element (array->type) * refers to another type. Array can be thought of a * special case of struct while array just has the same
@@ -390,9 +425,11 @@ static bool btf_type_is_int(const struct btf_type *t)static bool btf_type_needs_resolve (const struct btf_type *t) {return btf_type_is_modifier (t) || -btf_type_is_ptr (t) || -btf_type_is_struct (t) || -btf_type_is_array (t); +btf_type_is_ptr (t) || +btf_type_is_struct (t) || +btf_type_is_array (t) || +btf_type_is_var (t) || +btf_type_is_datasec (t); }/* t->size can be used */
@@ -403,6 +440,7 @@ static bool btf_type_has_size(const struct btf_type *t)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC :return true; }
@@ -467,6 +505,16 @@ static const struct btf_enum *btf_type_enum(const struct btf_type *t)return (const struct btf_enum *)(t +1 ); } +static const struct btf_var *btf_type_var (const struct btf_type *t) +{ +return (const struct btf_var *)(t +1 ); +} + +static const struct btf_var_secinfo *btf_type_var_secinfo (const struct btf_type *t) +{ +return (const struct btf_var_secinfo *)(t +1 ); +} +static const struct btf_kind_operations *btf_type_ops (const struct btf_type *t) {return kind_ops[BTF_INFO_KIND (t->info)];
@@ -478,23 +526,31 @@ static bool btf_name_offset_valid(const struct btf *btf, u32 offset) offset < btf->hdr.str_len; } -/* Only C-style identifier is permitted. This can be relaxed if -* necessary. -*/ -static bool btf_name_valid_identifier (const struct btf *btf,u32 offset) +static bool __btf_name_char_ok (char c,bool first,bool dot_ok) +{ +if ((first ? !isalpha (c) : + !isalnum (c)) && + c !='_' && + ((c =='.' && !dot_ok) || + c !='.' )) +return false; +return true; +} + +static bool __btf_name_valid (const struct btf *btf,u32 offset,bool dot_ok) {/* offset must be valid */ const char *src = &btf->strings[offset];const char *src_limit; -if (!isalpha (*src) && *src !='_' ) +if (!__btf_name_char_ok (*src, true, dot_ok))return false;/* set a limit on identifier length */ src_limit = src +KSYM_NAME_LEN ; src++;while (*src && src < src_limit) { -if (!isalnum (*src) && *src !='_' ) +if (!__btf_name_char_ok (*src, false, dot_ok))return false; src++; }
@@ -502,6 +558,19 @@ static bool btf_name_valid_identifier(const struct btf *btf, u32 offset)return !*src; } +/* Only C-style identifier is permitted. This can be relaxed if +* necessary. +*/ +static bool btf_name_valid_identifier (const struct btf *btf,u32 offset) +{ +return __btf_name_valid (btf, offset, false); +} + +static bool btf_name_valid_section (const struct btf *btf,u32 offset) +{ +return __btf_name_valid (btf, offset, true); +} +static const char *__btf_name_by_offset (const struct btf *btf,u32 offset) {if (!offset)
@@ -697,6 +766,32 @@ static void btf_verifier_log_member(struct btf_verifier_env *env,__btf_verifier_log (log,"\n" ); } +__printf(4, 5) +static void btf_verifier_log_vsi (struct btf_verifier_env *env, +const struct btf_type *datasec_type, +const struct btf_var_secinfo *vsi, +const char *fmt, ...) +{ +struct bpf_verifier_log *log = &env->log; +va_list args; + +if (!bpf_verifier_log_needed (log)) +return ; +if (env->phase !=CHECK_META ) +btf_verifier_log_type (env, datasec_type,NULL ); + +__btf_verifier_log (log,"\t type_id=%u offset=%u size=%u" , + vsi->type, vsi->offset, vsi->size); +if (fmt && *fmt) { +__btf_verifier_log (log," " ); +va_start (args, fmt); +bpf_verifier_vlog (log, fmt, args); +va_end (args); + } + +__btf_verifier_log (log,"\n" ); +} +static void btf_verifier_log_hdr (struct btf_verifier_env *env,u32 btf_data_size) {
@@ -974,7 +1069,8 @@ const struct btf_type *btf_type_id_size(const struct btf *btf, }else if (btf_type_is_ptr (size_type)) { size =sizeof (void *); }else { -if (WARN_ON_ONCE (!btf_type_is_modifier (size_type))) +if (WARN_ON_ONCE (!btf_type_is_modifier (size_type) && + !btf_type_is_var (size_type)))return NULL ; size = btf->resolved_sizes[size_type_id];
@@ -1509,7 +1605,7 @@ static int btf_modifier_resolve(struct btf_verifier_env *env,u32 next_type_size =0 ; next_type =btf_type_by_id (btf, next_type_id); -if (!next_type) { +if (!next_type ||btf_type_is_resolve_source_only (next_type)) {btf_verifier_log_type (env, v->t,"Invalid type_id" );return -EINVAL ; }
@@ -1542,6 +1638,53 @@ static int btf_modifier_resolve(struct btf_verifier_env *env,return 0 ; } +static int btf_var_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_type *next_type; +const struct btf_type *t = v->t; +u32 next_type_id = t->type; +struct btf *btf = env->btf; +u32 next_type_size; + + next_type =btf_type_by_id (btf, next_type_id); +if (!next_type ||btf_type_is_resolve_source_only (next_type)) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +if (!env_type_is_resolve_sink (env, next_type) && + !env_type_is_resolved (env, next_type_id)) +return env_stack_push (env, next_type, next_type_id); + +if (btf_type_is_modifier (next_type)) { +const struct btf_type *resolved_type; +u32 resolved_type_id; + + resolved_type_id = next_type_id; + resolved_type =btf_type_id_resolve (btf, &resolved_type_id); + +if (btf_type_is_ptr (resolved_type) && + !env_type_is_resolve_sink (env, resolved_type) && + !env_type_is_resolved (env, resolved_type_id)) +return env_stack_push (env, resolved_type, + resolved_type_id); + } + +/* We must resolve to something concrete at this point, no +* forward types or similar that would resolve to size of +* zero is allowed. +*/ +if (!btf_type_id_size (btf, &next_type_id, &next_type_size)) { +btf_verifier_log_type (env, v->t,"Invalid type_id" ); +return -EINVAL ; + } + +env_stack_pop_resolved (env, next_type_id, next_type_size); + +return 0 ; +} +static int btf_ptr_resolve (struct btf_verifier_env *env,const struct resolve_vertex *v) {
@@ -1551,7 +1694,7 @@ static int btf_ptr_resolve(struct btf_verifier_env *env,struct btf *btf = env->btf; next_type =btf_type_by_id (btf, next_type_id); -if (!next_type) { +if (!next_type ||btf_type_is_resolve_source_only (next_type)) {btf_verifier_log_type (env, v->t,"Invalid type_id" );return -EINVAL ; }
@@ -1609,6 +1752,15 @@ static void btf_modifier_seq_show(const struct btf *btf,btf_type_ops (t)->seq_show (btf, t, type_id, data, bits_offset, m); } +static void btf_var_seq_show (const struct btf *btf,const struct btf_type *t, +u32 type_id,void *data,u8 bits_offset, +struct seq_file *m) +{ + t =btf_type_id_resolve (btf, &type_id); + +btf_type_ops (t)->seq_show (btf, t, type_id, data, bits_offset, m); +} +static void btf_ptr_seq_show (const struct btf *btf,const struct btf_type *t,u32 type_id,void *data,u8 bits_offset,struct seq_file *m)
@@ -1776,7 +1928,8 @@ static int btf_array_resolve(struct btf_verifier_env *env,/* Check array->index_type */ index_type_id = array->index_type; index_type =btf_type_by_id (btf, index_type_id); -if (btf_type_nosize_or_null (index_type)) { +if (btf_type_is_resolve_source_only (index_type) || +btf_type_nosize_or_null (index_type)) {btf_verifier_log_type (env, v->t,"Invalid index" );return -EINVAL ; }
@@ -1795,7 +1948,8 @@ static int btf_array_resolve(struct btf_verifier_env *env,/* Check array->type */ elem_type_id = array->type; elem_type =btf_type_by_id (btf, elem_type_id); -if (btf_type_nosize_or_null (elem_type)) { +if (btf_type_is_resolve_source_only (elem_type) || +btf_type_nosize_or_null (elem_type)) {btf_verifier_log_type (env, v->t,"Invalid elem" );return -EINVAL ;
@@ -2016,7 +2170,8 @@ static int btf_struct_resolve(struct btf_verifier_env *env,const struct btf_type *member_type =btf_type_by_id (env->btf, member_type_id); -if (btf_type_nosize_or_null (member_type)) { +if (btf_type_is_resolve_source_only (member_type) || +btf_type_nosize_or_null (member_type)) {btf_verifier_log_member (env, v->t, member,"Invalid member" );return -EINVAL ;
@@ -2411,6 +2566,222 @@ static struct btf_kind_operations func_ops = { .seq_show = btf_df_seq_show, }; +static s32 btf_var_check_meta (struct btf_verifier_env *env, +const struct btf_type *t, +u32 meta_left) +{ +const struct btf_var *var; +u32 meta_needed =sizeof (*var); + +if (meta_left < meta_needed) { +btf_verifier_log_basic (env, t, +"meta_left:%u meta_needed:%u" , + meta_left, meta_needed); +return -EINVAL ; + } + +if (btf_type_vlen (t)) { +btf_verifier_log_type (env, t,"vlen != 0" ); +return -EINVAL ; + } + +if (btf_type_kflag (t)) { +btf_verifier_log_type (env, t,"Invalid btf_info kind_flag" ); +return -EINVAL ; + } + +if (!t->name_off || + !__btf_name_valid (env->btf, t->name_off, true)) { +btf_verifier_log_type (env, t,"Invalid name" ); +return -EINVAL ; + } + +/* A var cannot be in type void */ +if (!t->type || !BTF_TYPE_ID_VALID (t->type)) { +btf_verifier_log_type (env, t,"Invalid type_id" ); +return -EINVAL ; + } + + var =btf_type_var (t); +if (var->linkage !=BTF_VAR_STATIC && + var->linkage !=BTF_VAR_GLOBAL_ALLOCATED ) { +btf_verifier_log_type (env, t,"Linkage not supported" ); +return -EINVAL ; + } + +btf_verifier_log_type (env, t,NULL ); + +return meta_needed; +} + +static void btf_var_log (struct btf_verifier_env *env,const struct btf_type *t) +{ +const struct btf_var *var =btf_type_var (t); + +btf_verifier_log (env,"type_id=%u linkage=%u" , t->type, var->linkage); +} + +static const struct btf_kind_operations var_ops = { + .check_meta = btf_var_check_meta, + .resolve = btf_var_resolve, + .check_member = btf_df_check_member, + .check_kflag_member = btf_df_check_kflag_member, + .log_details = btf_var_log, + .seq_show = btf_var_seq_show, +}; + +static s32 btf_datasec_check_meta (struct btf_verifier_env *env, +const struct btf_type *t, +u32 meta_left) +{ +const struct btf_var_secinfo *vsi; +u64 last_vsi_end_off =0 , sum =0 ; +u32 i, meta_needed; + + meta_needed =btf_type_vlen (t) *sizeof (*vsi); +if (meta_left < meta_needed) { +btf_verifier_log_basic (env, t, +"meta_left:%u meta_needed:%u" , + meta_left, meta_needed); +return -EINVAL ; + } + +if (!btf_type_vlen (t)) { +btf_verifier_log_type (env, t,"vlen == 0" ); +return -EINVAL ; + } + +if (!t->size) { +btf_verifier_log_type (env, t,"size == 0" ); +return -EINVAL ; + } + +if (btf_type_kflag (t)) { +btf_verifier_log_type (env, t,"Invalid btf_info kind_flag" ); +return -EINVAL ; + } + +if (!t->name_off || + !btf_name_valid_section (env->btf, t->name_off)) { +btf_verifier_log_type (env, t,"Invalid name" ); +return -EINVAL ; + } + +btf_verifier_log_type (env, t,NULL ); + +for_each_vsi (i, t, vsi) { +/* A var cannot be in type void */ +if (!vsi->type || !BTF_TYPE_ID_VALID (vsi->type)) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid type_id" ); +return -EINVAL ; + } + +if (vsi->offset < last_vsi_end_off || vsi->offset >= t->size) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid offset" ); +return -EINVAL ; + } + +if (!vsi->size || vsi->size > t->size) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid size" ); +return -EINVAL ; + } + + last_vsi_end_off = vsi->offset + vsi->size; +if (last_vsi_end_off > t->size) { +btf_verifier_log_vsi (env, t, vsi, +"Invalid offset+size" ); +return -EINVAL ; + } + +btf_verifier_log_vsi (env, t, vsi,NULL ); + sum += vsi->size; + } + +if (t->size < sum) { +btf_verifier_log_type (env, t,"Invalid btf_info size" ); +return -EINVAL ; + } + +return meta_needed; +} + +static int btf_datasec_resolve (struct btf_verifier_env *env, +const struct resolve_vertex *v) +{ +const struct btf_var_secinfo *vsi; +struct btf *btf = env->btf; +u16 i; + +for_each_vsi_from (i, v->next_member, v->t, vsi) { +u32 var_type_id = vsi->type, type_id, type_size =0 ; +const struct btf_type *var_type =btf_type_by_id (env->btf, + var_type_id); +if (!var_type || !btf_type_is_var (var_type)) { +btf_verifier_log_vsi (env, v->t, vsi, +"Not a VAR kind member" ); +return -EINVAL ; + } + +if (!env_type_is_resolve_sink (env, var_type) && + !env_type_is_resolved (env, var_type_id)) { +env_stack_set_next_member (env, i +1 ); +return env_stack_push (env, var_type, var_type_id); + } + + type_id = var_type->type; +if (!btf_type_id_size (btf, &type_id, &type_size)) { +btf_verifier_log_vsi (env, v->t, vsi,"Invalid type" ); +return -EINVAL ; + } + +if (vsi->size < type_size) { +btf_verifier_log_vsi (env, v->t, vsi,"Invalid size" ); +return -EINVAL ; + } + } + +env_stack_pop_resolved (env,0 ,0 ); +return 0 ; +} + +static void btf_datasec_log (struct btf_verifier_env *env, +const struct btf_type *t) +{ +btf_verifier_log (env,"size=%u vlen=%u" , t->size,btf_type_vlen (t)); +} + +static void btf_datasec_seq_show (const struct btf *btf, +const struct btf_type *t,u32 type_id, +void *data,u8 bits_offset, +struct seq_file *m) +{ +const struct btf_var_secinfo *vsi; +const struct btf_type *var; +u32 i; + +seq_printf (m,"section (\"%s\") = {" ,__btf_name_by_offset (btf, t->name_off)); +for_each_vsi (i, t, vsi) { + var =btf_type_by_id (btf, vsi->type); +if (i) +seq_puts (m,"," ); +btf_type_ops (var)->seq_show (btf, var, vsi->type, + data + vsi->offset, bits_offset, m); + } +seq_puts (m,"}" ); +} + +static const struct btf_kind_operations datasec_ops = { + .check_meta = btf_datasec_check_meta, + .resolve = btf_datasec_resolve, + .check_member = btf_df_check_member, + .check_kflag_member = btf_df_check_kflag_member, + .log_details = btf_datasec_log, + .seq_show = btf_datasec_seq_show, +}; +static int btf_func_proto_check (struct btf_verifier_env *env,const struct btf_type *t) {
@@ -2542,6 +2913,8 @@ static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = { [BTF_KIND_RESTRICT] = &modifier_ops, [BTF_KIND_FUNC] = &func_ops, [BTF_KIND_FUNC_PROTO] = &func_proto_ops, + [BTF_KIND_VAR] = &var_ops, + [BTF_KIND_DATASEC] = &datasec_ops, }; static s32 btf_check_meta(struct btf_verifier_env *env,
@@ -2622,13 +2995,17 @@ static bool btf_resolve_valid(struct btf_verifier_env *env,if (!env_type_is_resolved (env, type_id))return false; -if (btf_type_is_struct (t)) +if (btf_type_is_struct (t) ||btf_type_is_datasec (t))return !btf->resolved_ids[type_id] && - !btf->resolved_sizes[type_id]; + !btf->resolved_sizes[type_id]; -if (btf_type_is_modifier (t) ||btf_type_is_ptr (t)) { +if (btf_type_is_modifier (t) ||btf_type_is_ptr (t) || +btf_type_is_var (t)) { t =btf_type_id_resolve (btf, &type_id); -return t && !btf_type_is_modifier (t); +return t && + !btf_type_is_modifier (t) && + !btf_type_is_var (t) && + !btf_type_is_datasec (t); }if (btf_type_is_array (t)) {
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 8/16 bpf: allow for key-less BTF in array map
In reply to Daniel BorkmannGiven we'll be reusing BPF array maps for global data/bss/rodata sections, we need a way to associate BTF DataSec type as its map value type. In usual cases we have this ugly BPF_ANNOTATE_KV_PAIR() macro hack e.g. via 38d5d3b3d5db ("bpf: Introduce BPF_ANNOTATE_KV_PAIR") to get initial map to type association going. While more use cases for it are discouraged, this also won't work for global data since the use of array map is a BPF loader detail and therefore unknown at compilation time. For array maps with just a single entry we make an exception in terms of BTF in that key type is declared optional if value type is of DataSec type. The latter LLVM is guaranteed to emit and it also aligns with how we regard global data maps as just a plain buffer area reusing existing map facilities for allowing things like introspection with existing tools.
--- include/linux/btf.h | 1 + kernel/bpf/arraymap.c | 15 ++++++++++++++- kernel/bpf/btf.c | 2 +- kernel/bpf/syscall.c | 15 +++++++++++---- 4 files changed, 27 insertions(+), 6 deletions(-)
include/linux/btf.h +1 -0
@@ -51,6 +51,7 @@ bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,const struct btf_member *m, u32 expected_offset, u32 expected_size);int btf_find_spin_lock (const struct btf *btf,const struct btf_type *t); +bool btf_type_is_void (const struct btf_type *t);#ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id);
kernel/bpf/arraymap.c +14 -1
@@ -391,7 +391,8 @@ static void array_map_seq_show_elem(struct bpf_map *map, void *key,return ; } -seq_printf (m,"%u: " , *(u32 *)key); +if (map->btf_key_type_id) +seq_printf (m,"%u: " , *(u32 *)key);btf_type_seq_show (map->btf, map->btf_value_type_id, value, m);seq_puts (m,"\n" );
@@ -428,6 +429,18 @@ static int array_map_check_btf(const struct bpf_map *map, {u32 int_data; +/* One exception for keyless BTF: .bss/.data/.rodata map */ +if (btf_type_is_void (key_type)) { +if (map->map_type !=BPF_MAP_TYPE_ARRAY || + map->max_entries !=1 ) +return -EINVAL ; + +if (BTF_INFO_KIND (value_type->info) !=BTF_KIND_DATASEC ) +return -EINVAL ; + +return 0 ; + } +if (BTF_INFO_KIND (key_type->info) !=BTF_KIND_INT )return -EINVAL ;
kernel/bpf/btf.c +1 -1
@@ -326,7 +326,7 @@ static bool btf_type_is_modifier(const struct btf_type *t)return false; } -static bool btf_type_is_void (const struct btf_type *t) +bool btf_type_is_void (const struct btf_type *t) {return t == &btf_void; }
kernel/bpf/syscall.c +11 -4
@@ -504,9 +504,16 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,u32 key_size, value_size;int ret =0 ; - key_type =btf_type_id_size (btf, &btf_key_id, &key_size); -if (!key_type || key_size != map->key_size) -return -EINVAL ; +/* Some maps allow key to be unspecified. */ +if (btf_key_id) { + key_type =btf_type_id_size (btf, &btf_key_id, &key_size); +if (!key_type || key_size != map->key_size) +return -EINVAL ; + }else { + key_type =btf_type_by_id (btf,0 ); +if (!map->ops->map_check_btf) +return -EINVAL ; + } value_type =btf_type_id_size (btf, &btf_value_id, &value_size);if (!value_type || value_size != map->value_size)
@@ -573,7 +580,7 @@ static int map_create(union bpf_attr *attr)if (attr->btf_key_type_id || attr->btf_value_type_id) {struct btf *btf; -if (!attr->btf_key_type_id || !attr->btf_value_type_id) { +if (!attr->btf_value_type_id) { err = -EINVAL ; goto free_map_nouncharge; }
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 9/16 bpf: sync {btf, bpf}.h uapi header from tools infrastructure
In reply to Daniel BorkmannPull in latest changes from both headers, so we can make use of them in libbpf.
--- tools/include/uapi/linux/bpf.h | 20 ++++++++++++++++++-- tools/include/uapi/linux/btf.h | 32 ++++++++++++++++++++++++++++---- 2 files changed, 46 insertions(+), 6 deletions(-)
tools/include/uapi/linux/bpf.h +18 -2
@@ -105,6 +105,7 @@ enum bpf_cmd {BPF_BTF_GET_FD_BY_ID ,BPF_TASK_FD_QUERY ,BPF_MAP_LOOKUP_AND_DELETE_ELEM , + BPF_MAP_FREEZE, }; enum bpf_map_type {
@@ -255,8 +256,19 @@ enum bpf_attach_type {*/ #define BPF_F_ANY_ALIGNMENT (1U << 1) -/* when bpf_ldimm64->src_reg == BPF_PSEUDO_MAP_FD, bpf_ldimm64->imm == fd */ +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have +* two extensions: +* +* insn[0].src_reg: BPF_PSEUDO_MAP_FD BPF_PSEUDO_MAP_VALUE +* insn[0].imm: map fd map fd +* insn[1].imm: 0 offset into value +* insn[0].off: 0 0 +* insn[1].off: 0 0 +* ldimm64 rewrite: address of map address of map[0]+offset +* verifier type: CONST_PTR_TO_MAP PTR_TO_MAP_VALUE +*/ #define BPF_PSEUDO_MAP_FD 1 +#define BPF_PSEUDO_MAP_VALUE 2/* when bpf_call->src_reg == BPF_PSEUDO_CALL, bpf_call->imm == pc-relative * offset to another bpf function
@@ -283,7 +295,7 @@ enum bpf_attach_type {#define BPF_OBJ_NAME_LEN 16U -/* Flags for accessing BPF object */ +/* Flags for accessing BPF object from syscall side. */ #define BPF_F_RDONLY (1U << 3)#define BPF_F_WRONLY (1U << 4)
@@ -293,6 +305,10 @@ enum bpf_attach_type {/* Zero-initialize hash function seed. This should only be used for testing. */ #define BPF_F_ZERO_SEED (1U << 6) +/* Flags for accessing BPF object from program side. */ +#define BPF_F_RDONLY_PROG (1U << 7) +#define BPF_F_WRONLY_PROG (1U << 8) +/* flags for BPF_PROG_QUERY */ #define BPF_F_QUERY_EFFECTIVE (1U << 0)
tools/include/uapi/linux/btf.h +28 -4
@@ -39,11 +39,11 @@ struct btf_type {* struct, union and fwd */ __u32 info; -/* "size" is used by INT, ENUM, STRUCT and UNION. +/* "size" is used by INT, ENUM, STRUCT, UNION and DATASEC. * "size" tells the size of the type it is describing. * * "type" is used by PTR, TYPEDEF, VOLATILE, CONST, RESTRICT, -* FUNC and FUNC_PROTO. +* FUNC, FUNC_PROTO and VAR. * "type" is a type_id referring to another type. */ union {
@@ -70,8 +70,10 @@ struct btf_type {#define BTF_KIND_RESTRICT 11/* Restrict */ #define BTF_KIND_FUNC 12/* Function */ #define BTF_KIND_FUNC_PROTO 13/* Function Proto */ -#define BTF_KIND_MAX 13 -#define NR_BTF_KINDS 14 +#define BTF_KIND_VAR 14/* Variable */ +#define BTF_KIND_DATASEC 15/* Section */ +#define BTF_KIND_MAX BTF_KIND_DATASEC +#define NR_BTF_KINDS (BTF_KIND_MAX + 1)/* For some specific BTF_KIND, "struct btf_type" is immediately * followed by extra data.
@@ -138,4 +140,26 @@ struct btf_param { __u32 type; }; +enum { +BTF_VAR_STATIC =0 , +BTF_VAR_GLOBAL_ALLOCATED , +}; + +/* BTF_KIND_VAR is followed by a single "struct btf_var" to describe +* additional information related to the variable such as its linkage. +*/ +struct btf_var { +__u32 linkage; +}; + +/* BTF_KIND_DATASEC is followed by multiple "struct btf_var_secinfo" +* to describe all BTF_KIND_VAR types it contains along with it's +* in-section offset as well as size. +*/ +struct btf_var_secinfo { +__u32 type; +__u32 offset; +__u32 size; +}; +#endif /* _UAPI__LINUX_BTF_H__ */
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 10/16 bpf, libbpf: refactor relocation handling
In reply to Daniel BorkmannFrom: Joe Stringer <[email protected]>
Adjust the code for relocations slightly with no functional changes, so that upcoming patches that will introduce support for relocations into the .data, .rodata and .bss sections can be added independent of these changes.
--- tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++++++++------------------------ 1 file changed, 32 insertions(+), 30 deletions(-)
tools/lib/bpf/libbpf.c +32 -30
@@ -871,20 +871,20 @@ static int bpf_object__elf_collect(struct bpf_object *obj, int flags) obj->efile.symbols = data; obj->efile.strtabidx = sh.sh_link; } - }else if ((sh.sh_type ==SHT_PROGBITS ) && - (sh.sh_flags &SHF_EXECINSTR ) && - (data->d_size >0 )) { -if (strcmp (name,".text" ) ==0 ) - obj->efile.text_shndx = idx; - err =bpf_object__add_program (obj, data->d_buf, - data->d_size, name, idx); -if (err) { -char errmsg[STRERR_BUFSIZE ]; -char *cp =libbpf_strerror_r (-err, errmsg, -sizeof (errmsg)); - -pr_warning ("failed to alloc program %s (%s): %s" , - name, obj->path, cp); + }else if (sh.sh_type ==SHT_PROGBITS && data->d_size >0 ) { +if (sh.sh_flags &SHF_EXECINSTR ) { +if (strcmp (name,".text" ) ==0 ) + obj->efile.text_shndx = idx; + err =bpf_object__add_program (obj, data->d_buf, + data->d_size, name, idx); +if (err) { +char errmsg[STRERR_BUFSIZE ]; +char *cp =libbpf_strerror_r (-err, errmsg, +sizeof (errmsg)); + +pr_warning ("failed to alloc program %s (%s): %s" , + name, obj->path, cp); + } } }else if (sh.sh_type ==SHT_REL ) {void *reloc = obj->efile.reloc;
@@ -1046,24 +1046,26 @@ bpf_program__collect_reloc(struct bpf_program *prog, GElf_Shdr *shdr,return -LIBBPF_ERRNO__RELOC ; } -/* TODO: 'maps' is sorted. We can use bsearch to make it faster. */ -for (map_idx =0 ; map_idx < nr_maps; map_idx++) { -if (maps[map_idx].offset == sym.st_value) { -pr_debug ("relocation: find map %zd (%s) for insn %u\n" , - map_idx, maps[map_idx].name, insn_idx); -break ; +if (sym.st_shndx == maps_shndx) { +/* TODO: 'maps' is sorted. We can use bsearch to make it faster. */ +for (map_idx =0 ; map_idx < nr_maps; map_idx++) { +if (maps[map_idx].offset == sym.st_value) { +pr_debug ("relocation: find map %zd (%s) for insn %u\n" , + map_idx, maps[map_idx].name, insn_idx); +break ; + } } - } -if (map_idx >= nr_maps) { -pr_warning ("bpf relocation: map_idx %d large than %d\n" , - (int )map_idx, (int )nr_maps -1 ); -return -LIBBPF_ERRNO__RELOC ; - } +if (map_idx >= nr_maps) { +pr_warning ("bpf relocation: map_idx %d large than %d\n" , + (int )map_idx, (int )nr_maps -1 ); +return -LIBBPF_ERRNO__RELOC ; + } - prog->reloc_desc[i].type =RELO_LD64 ; - prog->reloc_desc[i].insn_idx = insn_idx; - prog->reloc_desc[i].map_idx = map_idx; + prog->reloc_desc[i].type =RELO_LD64 ; + prog->reloc_desc[i].insn_idx = insn_idx; + prog->reloc_desc[i].map_idx = map_idx; + } }return 0 ; }
@@ -1425,7 +1427,7 @@ bpf_program__relocate(struct bpf_program *prog, struct bpf_object *obj) } insns[insn_idx].src_reg =BPF_PSEUDO_MAP_FD ; insns[insn_idx].imm = obj->maps[map_idx].fd; - }else { + }else if (prog->reloc_desc[i].type ==RELO_CALL ) { err =bpf_program__reloc_text (prog, obj, &prog->reloc_desc[i]);if (err)
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 11/16 bpf, libbpf: support global data/bss/rodata sections
In reply to Daniel BorkmannThis work adds BPF loader support for global data sections to libbpf. This allows to write BPF programs in more natural C-like way by being able to define global variables and const data.
Back at LPC 2018 [0] we presented a first prototype which implemented support for global data sections by extending BPF syscall where union bpf_attr would get additional memory/size pair for each section passed during prog load in order to later add this base address into the ldimm64 instruction along with the user provided offset when accessing a variable. Consensus from LPC was that for proper upstream support, it would be more desirable to use maps instead of bpf_attr extension as this would allow for introspection of these sections as well as potential live updates of their content. This work follows this path by taking the following steps from loader side:
1) In bpf_object__elf_collect() step we pick up ".data", ".rodata", and ".bss" section information.
2) If present, in bpf_object__init_internal_map() we add maps to the obj's map array that corresponds to each of the present sections. Given section size and access properties can differ, a single entry array map is created with value size that is corresponding to the ELF section size of .data, .bss or .rodata. These internal maps are integrated into the normal map handling of libbpf such that when user traverses all obj maps, they can be differentiated from user-created ones via bpf_map__is_internal(). In later steps when we actually create these maps in the kernel via bpf_object__create_maps(), then for .data and .rodata sections their content is copied into the map through bpf_map_update_elem(). For .bss this is not necessary since array map is already zero-initialized by default. Additionally, for .rodata the map is frozen as read-only after setup, such that neither from program nor syscall side writes would be possible.
3) In bpf_program__collect_reloc() step, we record the corresponding map, insn index, and relocation type for the global data.
4) And last but not least in the actual relocation step in bpf_program__relocate(), we mark the ldimm64 instruction with src_reg = BPF_PSEUDO_MAP_VALUE where in the first imm field the map's file descriptor is stored as similarly done as in BPF_PSEUDO_MAP_FD, and in the second imm field (as ldimm64 is 2-insn wide) we store the access offset into the section. Given these maps have only single element ldimm64's off remains zero in both parts.
5) On kernel side, this special marked BPF_PSEUDO_MAP_VALUE load will then store the actual target address in order to have a 'map-lookup'-free access. That is, the actual map value base address + offset. The destination register in the verifier will then be marked as PTR_TO_MAP_VALUE, containing the fixed offset as reg->off and backing BPF map as reg->map_ptr. Meaning, it's treated as any other normal map value from verification side, only with efficient, direct value access instead of actual call to map lookup helper as in the typical case.
Currently, only support for static global variables has been added, and libbpf rejects non-static global variables from loading. This can be lifted until we have proper semantics for how BPF will treat multi-object BPF loads. From BTF side, libbpf will set the value type id of the types corresponding to the ".bss", ".data" and ".rodata" names which LLVM will emit without the object name prefix. The key type will be left as zero, thus making use of the key-less BTF option in array maps.
Simple example dump of program using globals vars in each section:
# bpftool prog
[...]
6784: sched_cls name load_static_dat tag a7e1291567277844 gpl
loaded_at 2019-03-11T15:39:34+0000 uid 0
xlated 1776B jited 993B memlock 4096B map_ids 2238,2237,2235,2236,2239,2240 # bpftool map show id 2237
2237: array name test_glo.bss flags 0x0
key 4B value 64B max_entries 1 memlock 4096B
# bpftool map show id 2235
2235: array name test_glo.data flags 0x0
key 4B value 64B max_entries 1 memlock 4096B
# bpftool map show id 2236
2236: array name test_glo.rodata flags 0x80
key 4B value 96B max_entries 1 memlock 4096B # bpftool prog dump xlated id 6784
int load_static_data(struct __sk_buff * skb):
; int load_static_data(struct __sk_buff *skb)
0: (b7) r6 = 0
; test_reloc(number, 0, &num0);
1: (63) *(u32 *)(r10 -4) = r6
2: (bf) r2 = r10
; int load_static_data(struct __sk_buff *skb)
3: (07) r2 += -4
; test_reloc(number, 0, &num0);
4: (18) r1 = map[id:2238]
6: (18) r3 = map[id:2237][0]+0 <-- direct addr in .bss area
8: (b7) r4 = 0
9: (85) call array_map_update_elem#100464
10: (b7) r1 = 1
; test_reloc(number, 1, &num1);
[...]
; test_reloc(string, 2, str2);
120: (18) r8 = map[id:2237][0]+16 <-- same here at offset +16
122: (18) r1 = map[id:2239]
124: (18) r3 = map[id:2237][0]+16
126: (b7) r4 = 0
127: (85) call array_map_update_elem#100464
128: (b7) r1 = 120
; str1[5] = 'x';
129: (73) *(u8 *)(r9 +5) = r1
; test_reloc(string, 3, str1);
130: (b7) r1 = 3
131: (63) *(u32 *)(r10 -4) = r1
132: (b7) r9 = 3
133: (bf) r2 = r10
; int load_static_data(struct __sk_buff *skb)
134: (07) r2 += -4
; test_reloc(string, 3, str1);
135: (18) r1 = map[id:2239]
137: (18) r3 = map[id:2235][0]+16 <-- direct addr in .data area
139: (b7) r4 = 0
140: (85) call array_map_update_elem#100464
141: (b7) r1 = 111
; __builtin_memcpy(&str2[2], "hello", sizeof("hello"));
142: (73) *(u8 *)(r8 +6) = r1 <-- further access based on .bss data
143: (b7) r1 = 108
144: (73) *(u8 *)(r8 +5) = r1
[...]For Cilium use-case in particular, this enables migrating configuration constants from Cilium daemon's generated header defines into global data sections such that expensive runtime recompilations with LLVM can be avoided altogether. Instead, the ELF file becomes effectively a "template", meaning, it is compiled only once (!) and the Cilium daemon will then rewrite relevant configuration data from the ELF's .data or .rodata sections directly instead of recompiling the program. The updated ELF is then loaded into the kernel and atomically replaces the existing program in the networking datapath. More info in [0].
Based upon recent fix in LLVM, commit c0db6b6bd444 ("[BPF] Don't fail for static variables").
[0] LPC 2018, BPF track, "ELF relocation for static data in BPF",
http://vger.kernel.org/lpc-bpf2018.html#session-3
--- tools/lib/bpf/Makefile | 2 +- tools/lib/bpf/bpf.c | 10 ++ tools/lib/bpf/bpf.h | 1 + tools/lib/bpf/libbpf.c | 342 ++++++++++++++++++++++++++++++++++++++++------- tools/lib/bpf/libbpf.h | 1 + tools/lib/bpf/libbpf.map | 6 + 6 files changed, 314 insertions(+), 48 deletions(-)
tools/lib/bpf/Makefile +1 -1
@@ -3,7 +3,7 @@ BPF_VERSION = 0 BPF_PATCHLEVEL = 0 -BPF_EXTRAVERSION = 2 +BPF_EXTRAVERSION = 3 MAKEFLAGS += --no-print-directory
tools/lib/bpf/bpf.c +10 -0
@@ -429,6 +429,16 @@ int bpf_map_get_next_key(int fd, const void *key, void *next_key)return sys_bpf (BPF_MAP_GET_NEXT_KEY , &attr,sizeof (attr)); } +int bpf_map_freeze (int fd) +{ +union bpf_attr attr; + +memset (&attr,0 ,sizeof (attr)); + attr.map_fd = fd; + +return sys_bpf (BPF_MAP_FREEZE , &attr,sizeof (attr)); +} +int bpf_obj_pin (int fd,const char *pathname) {union bpf_attr attr;
tools/lib/bpf/bpf.h +1 -0
@@ -117,6 +117,7 @@ LIBBPF_API int bpf_map_lookup_and_delete_elem(int fd, const void *key, void *value); LIBBPF_API int bpf_map_delete_elem(int fd, const void *key); LIBBPF_API int bpf_map_get_next_key(int fd, const void *key, void *next_key); +LIBBPF_API int bpf_map_freeze(int fd); LIBBPF_API int bpf_obj_pin(int fd, const char *pathname); LIBBPF_API int bpf_obj_get(const char *pathname); LIBBPF_API int bpf_prog_attach(int prog_fd, int attachable_fd,
tools/lib/bpf/libbpf.c +295 -47
578 lines not on this page · raw message
tools/lib/bpf/libbpf.h +1 -0
@@ -301,6 +301,7 @@ LIBBPF_API void *bpf_map__priv(struct bpf_map *map); LIBBPF_API int bpf_map__reuse_fd(struct bpf_map *map, int fd); LIBBPF_API int bpf_map__resize(struct bpf_map *map, __u32 max_entries); LIBBPF_API bool bpf_map__is_offload_neutral(struct bpf_map *map); +LIBBPF_API bool bpf_map__is_internal(struct bpf_map *map); LIBBPF_API void bpf_map__set_ifindex(struct bpf_map *map, __u32 ifindex); LIBBPF_API int bpf_map__pin(struct bpf_map *map, const char *path); LIBBPF_API int bpf_map__unpin(struct bpf_map *map, const char *path);
tools/lib/bpf/libbpf.map +6 -0
@@ -157,3 +157,9 @@ LIBBPF_0.0.2 { bpf_program__bpil_addr_to_offs; bpf_program__bpil_offs_to_addr; } LIBBPF_0.0.1; + +LIBBPF_0.0.3 { + global: + bpf_map__is_internal; + bpf_map_freeze; +} LIBBPF_0.0.2;
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 12/16 bpf, libbpf: add support for BTF Var and DataSec
In reply to Daniel BorkmannThis adds libbpf support for BTF Var and DataSec kinds. Main point here is that libbpf needs to do some preparatory work before the whole BTF object can be loaded into the kernel, that is, fixing up of DataSec size taken from the ELF section size and non-static variable offset which needs to be taken from the ELF's string section.
Upstream LLVM doesn't fix these up since at time of BTF emission it is too early in the compilation process thus this information isn't available yet, hence loader needs to take care of it.
Note, deduplication handling has not been in the scope of this work and needs to be addressed in a future commit.
--- tools/lib/bpf/btf.c | 97 +++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 3 + tools/lib/bpf/libbpf.c | 150 +++++++++++++++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.h | 4 ++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 235 insertions(+), 20 deletions(-)
tools/lib/bpf/btf.c +96 -1
@@ -24,6 +24,8 @@ ((k) == BTF_KIND_CONST) || \ ((k) == BTF_KIND_RESTRICT)) +#define IS_VAR (k) ((k) == BTF_KIND_VAR) +static struct btf_type btf_void;struct btf {
@@ -212,6 +214,10 @@ static int btf_type_size(struct btf_type *t)return base_size + vlen *sizeof (struct btf_member );case BTF_KIND_FUNC_PROTO :return base_size + vlen *sizeof (struct btf_param ); +case BTF_KIND_VAR : +return base_size +sizeof (struct btf_var ); +case BTF_KIND_DATASEC : +return base_size + vlen *sizeof (struct btf_var_secinfo );default :pr_debug ("Unsupported BTF_KIND:%u\n" ,BTF_INFO_KIND (t->info));return -EINVAL ;
@@ -283,6 +289,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)case BTF_KIND_STRUCT :case BTF_KIND_UNION :case BTF_KIND_ENUM : +case BTF_KIND_DATASEC : size = t->size; goto done;case BTF_KIND_PTR :
@@ -292,6 +299,7 @@ __s64 btf__resolve_size(const struct btf *btf, __u32 type_id)case BTF_KIND_VOLATILE :case BTF_KIND_CONST :case BTF_KIND_RESTRICT : +case BTF_KIND_VAR : type_id = t->type;break ;case BTF_KIND_ARRAY :
@@ -326,7 +334,8 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id) t =btf__type_by_id (btf, type_id);while (depth <MAX_RESOLVE_DEPTH && !btf_type_is_void_or_null (t) && -IS_MODIFIER (BTF_INFO_KIND (t->info))) { + (IS_MODIFIER (BTF_INFO_KIND (t->info)) || +IS_VAR (BTF_INFO_KIND (t->info)))) { type_id = t->type; t =btf__type_by_id (btf, type_id); depth++;
@@ -408,6 +417,92 @@ struct btf *btf__new(__u8 *data, __u32 size)return btf; } +static int compare_vsi_off (const void *_a,const void *_b) +{ +const struct btf_var_secinfo *a = _a; +const struct btf_var_secinfo *b = _b; + +return a->offset - b->offset; +} + +static int btf_fixup_datasec (struct bpf_object *obj,struct btf *btf, +struct btf_type *t) +{ +__u32 size =0 , off =0 , i, vars =BTF_INFO_VLEN (t->info); +const char *name =btf__name_by_offset (btf, t->name_off); +const struct btf_type *t_var; +struct btf_var_secinfo *vsi; +struct btf_var *var; +int ret; + +if (!name) { +pr_debug ("No name found in string section for DATASEC kind.\n" ); +return -ENOENT ; + } + + ret =bpf_object__section_size (obj, name, &size); +if (ret || !size || (t->size && t->size != size)) { +pr_debug ("Invalid size for section %s: %u bytes\n" , name, size); +return -ENOENT ; + } + + t->size = size; + +for (i =0 , vsi = (struct btf_var_secinfo *)(t +1 ); + i < vars; i++, vsi++) { + t_var =btf__type_by_id (btf, vsi->type); + var = (struct btf_var *)(t_var +1 ); + +if (BTF_INFO_KIND (t_var->info) !=BTF_KIND_VAR ) { +pr_debug ("Non-VAR type seen in section %s\n" , name); +return -EINVAL ; + } + +if (var->linkage ==BTF_VAR_STATIC ) +continue ; + + name =btf__name_by_offset (btf, t_var->name_off); +if (!name) { +pr_debug ("No name found in string section for VAR kind\n" ); +return -ENOENT ; + } + + ret =bpf_object__variable_offset (obj, name, &off); +if (ret) { +pr_debug ("No offset found in symbol table for VAR %s\n" , name); +return -ENOENT ; + } + + vsi->offset = off; + } + +qsort (t +1 , vars,sizeof (*vsi), compare_vsi_off); +return 0 ; +} + +int btf__finalize_data (struct bpf_object *obj,struct btf *btf) +{ +int err =0 ; +__u32 i; + +for (i =1 ; i <= btf->nr_types; i++) { +struct btf_type *t = btf->types[i]; + +/* Loader needs to fix up some of the things compiler +* couldn't get its hands on while emitting BTF. This +* is section size and global variable offset. We use +* the info from the ELF itself for this purpose. +*/ +if (BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ) { + err =btf_fixup_datasec (obj, btf, t); +if (err) +break ; + } + } + +return err; +} +int btf__load (struct btf *btf) {__u32 log_buf_size =BPF_LOG_BUF_SIZE ;
tools/lib/bpf/btf.h +3 -0
@@ -21,6 +21,8 @@ struct btf;struct btf_ext ;struct btf_type ; +struct bpf_object ; +/* * The .BTF.ext ELF section layout defined as * struct btf_ext_header
@@ -57,6 +59,7 @@ struct btf_ext_header { LIBBPF_API void btf__free(struct btf *btf); LIBBPF_API struct btf *btf__new(__u8 *data, __u32 size); +LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf); LIBBPF_API int btf__load(struct btf *btf); LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,const char *type_name);
tools/lib/bpf/libbpf.c +131 -19
181 lines not on this page · raw message
tools/lib/bpf/libbpf.h +4 -0
@@ -75,6 +75,10 @@ struct bpf_object *__bpf_object__open_xattr(struct bpf_object_open_attr *attr, LIBBPF_API struct bpf_object *bpf_object__open_buffer(void *obj_buf,size_t obj_buf_sz,const char *name); +int bpf_object__section_size (const struct bpf_object *obj,const char *name, +__u32 *size); +int bpf_object__variable_offset (const struct bpf_object *obj,const char *name, +__u32 *off); LIBBPF_API int bpf_object__pin_maps(struct bpf_object *obj, const char *path); LIBBPF_API int bpf_object__unpin_maps(struct bpf_object *obj,const char *path);
tools/lib/bpf/libbpf.map +1 -0
@@ -162,4 +162,5 @@ LIBBPF_0.0.3 { global: bpf_map__is_internal; bpf_map_freeze; + btf__finalize_data; } LIBBPF_0.0.2;
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 13/16 bpf: bpftool support for dumping data/bss/rodata sections
In reply to Daniel BorkmannAdd the ability to bpftool to handle BTF Var and DataSec kinds in order to dump them out of btf_dumper_type(). The value has a single object with the section name, which itself holds an array of variables it dumps. A single variable is an object by itself printed along with its name. From there further type information is dumped along with corresponding value information.
Example output from .rodata:
# ./bpftool m d i 150
[{
"value": {
".rodata": [{
"load_static_data.bar": 18446744073709551615
},{
"num2": 24
},{
"num5": 43947
},{
"num6": 171
},{
"str0": [97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,0,0,0,0,0,0
]
},{
"struct0": {
"a": 42,
"b": 4278120431,
"c": 1229782938247303441
}
},{
"struct2": {
"a": 0,
"b": 0,
"c": 0
}
}
]
}
}
]--- tools/bpf/bpftool/btf_dumper.c | 59 ++++++++++++++++++++++++++++++++++++++++++ tools/bpf/bpftool/map.c | 10 ++++--- 2 files changed, 65 insertions(+), 4 deletions(-)
tools/bpf/bpftool/btf_dumper.c +59 -0
@@ -309,6 +309,48 @@ static int btf_dumper_struct(const struct btf_dumper *d, __u32 type_id,return ret; } +static int btf_dumper_var (const struct btf_dumper *d,__u32 type_id, +__u8 bit_offset,const void *data) +{ +const struct btf_type *t =btf__type_by_id (d->btf, type_id); +int ret; + +jsonw_start_object (d->jw); +jsonw_name (d->jw,btf__name_by_offset (d->btf, t->name_off)); + ret =btf_dumper_do_type (d, t->type, bit_offset, data); +jsonw_end_object (d->jw); + +return ret; +} + +static int btf_dumper_datasec (const struct btf_dumper *d,__u32 type_id, +const void *data) +{ +struct btf_var_secinfo *vsi; +const struct btf_type *t; +int ret =0 , i, vlen; + + t =btf__type_by_id (d->btf, type_id); +if (!t) +return -EINVAL ; + + vlen =BTF_INFO_VLEN (t->info); + vsi = (struct btf_var_secinfo *)(t +1 ); + +jsonw_start_object (d->jw); +jsonw_name (d->jw,btf__name_by_offset (d->btf, t->name_off)); +jsonw_start_array (d->jw); +for (i =0 ; i < vlen; i++) { + ret =btf_dumper_do_type (d, vsi[i].type,0 , data + vsi[i].offset); +if (ret) +break ; + } +jsonw_end_array (d->jw); +jsonw_end_object (d->jw); + +return ret; +} +static int btf_dumper_do_type (const struct btf_dumper *d,__u32 type_id,__u8 bit_offset,const void *data) {
@@ -341,6 +383,10 @@ static int btf_dumper_do_type(const struct btf_dumper *d, __u32 type_id,case BTF_KIND_CONST :case BTF_KIND_RESTRICT :return btf_dumper_modifier (d, type_id, bit_offset, data); +case BTF_KIND_VAR : +return btf_dumper_var (d, type_id, bit_offset, data); +case BTF_KIND_DATASEC : +return btf_dumper_datasec (d, type_id, data);default :jsonw_printf (d->jw,"(unsupported-kind" );return -EINVAL ;
@@ -377,6 +423,7 @@ static int __btf_dumper_type_only(const struct btf *btf, __u32 type_id, {const struct btf_type *proto_type;const struct btf_array *array; +const struct btf_var *var;const struct btf_type *t;if (!type_id) {
@@ -440,6 +487,18 @@ static int __btf_dumper_type_only(const struct btf *btf, __u32 type_id,if (pos ==-1 )return -1 ;break ; +case BTF_KIND_VAR : + var = (struct btf_var *)(t +1 ); +if (var->linkage ==BTF_VAR_STATIC ) +BTF_PRINT_ARG ("static " ); +BTF_PRINT_TYPE (t->type); +BTF_PRINT_ARG (" %s" , +btf__name_by_offset (btf, t->name_off)); +break ; +case BTF_KIND_DATASEC : +BTF_PRINT_ARG ("section (\"%s\") " , +btf__name_by_offset (btf, t->name_off)); +break ;case BTF_KIND_UNKN :default :return -1 ;
tools/bpf/bpftool/map.c +6 -4
@@ -153,11 +153,13 @@ static int do_dump_btf(const struct btf_dumper *d,/* start of key-value pair */ jsonw_start_object (d->jw); -jsonw_name (d->jw,"key" ); +if (map_info->btf_key_type_id) { +jsonw_name (d->jw,"key" ); - ret =btf_dumper_type (d, map_info->btf_key_type_id, key); -if (ret) - goto err_end_obj; + ret =btf_dumper_type (d, map_info->btf_key_type_id, key); +if (ret) + goto err_end_obj; + }if (!map_is_per_cpu (map_info->type)) {jsonw_name (d->jw,"value" );
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 14/16 bpf, selftest: test {rd, wr}only flags and direct value access
In reply to Daniel BorkmannExtend test_verifier with various test cases around the two kernel extensions, that is, {rd,wr}only map support as well as direct map value access. All passing, one skipped due to xskmap not present on test machine:
# ./test_verifier [...] #948/p XDP pkt read, pkt_meta' <= pkt_data, bad access 1 OK #949/p XDP pkt read, pkt_meta' <= pkt_data, bad access 2 OK #950/p XDP pkt read, pkt_data <= pkt_meta', good access OK #951/p XDP pkt read, pkt_data <= pkt_meta', bad access 1 OK #952/p XDP pkt read, pkt_data <= pkt_meta', bad access 2 OK Summary: 1410 PASSED, 1 SKIPPED, 0 FAILED
--- tools/include/linux/filter.h | 21 +- tools/testing/selftests/bpf/test_verifier.c | 51 ++- .../testing/selftests/bpf/verifier/array_access.c | 159 ++++++++++ .../selftests/bpf/verifier/direct_value_access.c | 347 +++++++++++++++++++++ 4 files changed, 573 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/bpf/verifier/direct_value_access.c
tools/include/linux/filter.h +20 -1
@@ -278,10 +278,29 @@ .off = 0, \ .imm = ((__u64) (IMM)) >> 32 }) +#define BPF_LD_IMM64_RAW_FULL(DST, SRC, OFF1, OFF2, IMM1, IMM2) \ + ((struct bpf_insn) { \ + .code = BPF_LD | BPF_DW | BPF_IMM, \ + .dst_reg = DST, \ + .src_reg = SRC, \ + .off = OFF1, \ + .imm = IMM1 }), \ + ((struct bpf_insn) { \ + .code = 0, /* zero is reserved opcode */ \ + .dst_reg = 0, \ + .src_reg = 0, \ + .off = OFF2, \ + .imm = IMM2 }) +/* pseudo BPF_LD_IMM64 insn used to refer to process-local map_fd */ #define BPF_LD_MAP_FD (DST ,MAP_FD ) \ - BPF_LD_IMM64_RAW(DST, BPF_PSEUDO_MAP_FD, MAP_FD) + BPF_LD_IMM64_RAW_FULL(DST, BPF_PSEUDO_MAP_FD, 0, 0, \ + MAP_FD, 0) + +#define BPF_LD_MAP_VALUE (DST ,MAP_FD ,VALUE_OFF ) \ + BPF_LD_IMM64_RAW_FULL(DST, BPF_PSEUDO_MAP_VALUE, 0, 0, \ + MAP_FD, VALUE_OFF)/* Relative call */
tools/testing/selftests/bpf/test_verifier.c +47 -4
91 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/array_access.c +159 -0
162 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/direct_value_access.c new file +347 -0
347 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 15/16 bpf, selftest: test global data/bss/rodata sections
In reply to Daniel BorkmannFrom: Joe Stringer <[email protected]>
Add tests for libbpf relocation of static variable references into the .data, .rodata and .bss sections of the ELF, also add read-only test for .rodata. All passing:
# ./test_progs [...] test_global_data:PASS:load program 0 nsec test_global_data:PASS:pass global data run 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .data reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .data reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .bss reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_number:PASS:relocate .rodata reference 925 nsec test_global_data_string:PASS:relocate .rodata reference 925 nsec test_global_data_string:PASS:relocate .data reference 925 nsec test_global_data_string:PASS:relocate .bss reference 925 nsec test_global_data_string:PASS:relocate .data reference 925 nsec test_global_data_string:PASS:relocate .bss reference 925 nsec test_global_data_struct:PASS:relocate .rodata reference 925 nsec test_global_data_struct:PASS:relocate .bss reference 925 nsec test_global_data_struct:PASS:relocate .rodata reference 925 nsec test_global_data_struct:PASS:relocate .data reference 925 nsec test_global_data_rdonly:PASS:test .rodata read-only map 925 nsec [...] Summary: 229 PASSED, 0 FAILED
Note map helper signatures have been changed to avoid warnings when passing in const data.
Joint work with Daniel Borkmann.
--- tools/testing/selftests/bpf/bpf_helpers.h | 8 +- .../testing/selftests/bpf/prog_tests/global_data.c | 157 +++++++++++++++++++++ .../testing/selftests/bpf/progs/test_global_data.c | 106 ++++++++++++++ 3 files changed, 267 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/global_data.c create mode 100644 tools/testing/selftests/bpf/progs/test_global_data.c
tools/testing/selftests/bpf/bpf_helpers.h +4 -4
@@ -9,14 +9,14 @@#define SEC (NAME ) __attribute__((section(NAME), used))/* helper functions called from eBPF programs written in C */ -static void *(*bpf_map_lookup_elem)(void *map,void *key) = +static void *(*bpf_map_lookup_elem)(void *map,const void *key) = (void *) BPF_FUNC_map_lookup_elem; -static int (*bpf_map_update_elem)(void *map,void *key,void *value, +static int (*bpf_map_update_elem)(void *map,const void *key,const void *value,unsigned long long flags) = (void *) BPF_FUNC_map_update_elem; -static int (*bpf_map_delete_elem)(void *map,void *key) = +static int (*bpf_map_delete_elem)(void *map,const void *key) = (void *) BPF_FUNC_map_delete_elem; -static int (*bpf_map_push_elem)(void *map,void *value, +static int (*bpf_map_push_elem)(void *map,const void *value,unsigned long long flags) = (void *) BPF_FUNC_map_push_elem; static int (*bpf_map_pop_elem)(void *map, void *value) =
tools/testing/selftests/bpf/prog_tests/global_data.c new file +157 -0
157 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_global_data.c new file +106 -0
106 lines not on this page · raw message
Signature · 1 line
2.9.5
Daniel Borkmann
maintainerPATCH v5 16/16 bpf, selftest: add test cases for BTF Var and DataSec
In reply to Daniel BorkmannExtend test_btf with various positive and negative tests around BTF verification of kind Var and DataSec. All passing as well:
# ./test_btf [...] BTF raw test[4] (global data test #1): OK BTF raw test[5] (global data test #2): OK BTF raw test[6] (global data test #3): OK BTF raw test[7] (global data test #4, unsupported linkage): OK BTF raw test[8] (global data test #5, invalid var type): OK BTF raw test[9] (global data test #6, invalid var type (fwd type)): OK BTF raw test[10] (global data test #7, invalid var type (fwd type)): OK BTF raw test[11] (global data test #8, invalid var size): OK BTF raw test[12] (global data test #9, invalid var size): OK BTF raw test[13] (global data test #10, invalid var size): OK BTF raw test[14] (global data test #11, multiple section members): OK BTF raw test[15] (global data test #12, invalid offset): OK BTF raw test[16] (global data test #13, invalid offset): OK BTF raw test[17] (global data test #14, invalid offset): OK BTF raw test[18] (global data test #15, not var kind): OK BTF raw test[19] (global data test #16, invalid var referencing sec): OK BTF raw test[20] (global data test #17, invalid var referencing var): OK BTF raw test[21] (global data test #18, invalid var loop): OK BTF raw test[22] (global data test #19, invalid var referencing var): OK BTF raw test[23] (global data test #20, invalid ptr referencing var): OK BTF raw test[24] (global data test #21, var included in struct): OK BTF raw test[25] (global data test #22, array of var): OK [...] PASS:167 SKIP:0 FAIL:0
--- tools/testing/selftests/bpf/test_btf.c | 665 ++++++++++++++++++++++++++++++++- 1 file changed, 663 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/test_btf.c +663 -2
@@ -85,6 +85,11 @@ static int __base_pr(enum libbpf_print_level level __attribute__((unused)),#define BTF_UNION_ENC (name, nr_elems, sz) \ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_UNION, 0, nr_elems), sz) +#define BTF_VAR_ENC (name, type, linkage) \ + BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_VAR, 0, 0), type), (linkage) +#define BTF_VAR_SECINFO_ENC (type, offset, size) \ + (type), (offset), (size) +#define BTF_MEMBER_ENC (name, type, bits_offset) \ (name), (type), (bits_offset)#define BTF_ENUM_ENC (name, val) (name), (val)
@@ -291,7 +296,6 @@ static struct btf_raw_test raw_tests[] = { .value_type_id = 3, .max_entries = 4, }, - { .descr = "struct test #3 Invalid member offset", .raw_types = {
@@ -319,7 +323,664 @@ static struct btf_raw_test raw_tests[] = { .btf_load_err = true, .err_str = "Invalid member bits_offset", }, - +/* +* struct A { +* unsigned long long m; +* int n; +* char o; +* [3 bytes hole] +* int p[8]; +* }; +*/ +{ + .descr = "global data test #1", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = "struct_test1_map", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 1, + .value_type_id = 5, + .max_entries = 4, +}, +/* + * struct A { + * unsigned long long m; + * int n; + * char o; + * [3 bytes hole] + * int p[8]; + * }; + * static struct A t; <- in .bss + */ +{ + .descr = "global data test #2", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 48), + BTF_VAR_SECINFO_ENC(6, 0, 48), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, +}, +{ + .descr = "global data test #3", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* static int t */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0t\0.bss", + .str_sec_size = sizeof("\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 3, + .max_entries = 1, +}, +{ + .descr = "global data test #4, unsupported linkage", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* static int t */ + BTF_VAR_ENC(NAME_TBD, 1, 2), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0t\0.bss", + .str_sec_size = sizeof("\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 3, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Linkage not supported", +}, +{ + .descr = "global data test #5, invalid var type", + .raw_types = { + /* static void t */ + BTF_VAR_ENC(NAME_TBD, 0, 0), /* [1] */ + /* .bss section */ /* [2] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(1, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0t\0.bss", + .str_sec_size = sizeof("\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 2, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #6, invalid var type (fwd type)", + .raw_types = { + /* union A */ + BTF_TYPE_ENC(NAME_TBD, + BTF_INFO_ENC(BTF_KIND_FWD, 1, 0), 0), /* [1] */ + /* static union A t */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0.bss", + .str_sec_size = sizeof("\0A\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 2, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type", +}, +{ + .descr = "global data test #7, invalid var type (fwd type)", + .raw_types = { + /* union A */ + BTF_TYPE_ENC(NAME_TBD, + BTF_INFO_ENC(BTF_KIND_FWD, 1, 0), 0), /* [1] */ + /* static union A t */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(1, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0.bss", + .str_sec_size = sizeof("\0A\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 2, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type", +}, +{ + .descr = "global data test #8, invalid var size", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 48), + BTF_VAR_SECINFO_ENC(6, 0, 47), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid size", +}, +{ + .descr = "global data test #9, invalid var size", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 46), + BTF_VAR_SECINFO_ENC(6, 0, 48), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid size", +}, +{ + .descr = "global data test #10, invalid var size", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* .bss section */ /* [7] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 46), + BTF_VAR_SECINFO_ENC(6, 0, 46), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 48, + .key_type_id = 0, + .value_type_id = 7, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid size", +}, +{ + .descr = "global data test #11, multiple section members", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_VAR_SECINFO_ENC(7, 58, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, +}, +{ + .descr = "global data test #12, invalid offset", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_VAR_SECINFO_ENC(7, 60, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid offset+size", +}, +{ + .descr = "global data test #13, invalid offset", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_VAR_SECINFO_ENC(7, 12, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid offset", +}, +{ + .descr = "global data test #14, invalid offset", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* unsigned long long */ + BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */ + /* char */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */ + /* int[8] */ + BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */ + /* struct A { */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 4), 48), + BTF_MEMBER_ENC(NAME_TBD, 2, 0), /* unsigned long long m;*/ + BTF_MEMBER_ENC(NAME_TBD, 1, 64),/* int n; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */ + BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */ + /* } */ + /* static struct A t */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [6] */ + /* static int u */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [7] */ + /* .bss section */ /* [8] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 2), 62), + BTF_VAR_SECINFO_ENC(7, 58, 4), + BTF_VAR_SECINFO_ENC(6, 10, 48), + BTF_END_RAW, + }, + .str_sec = "\0A\0m\0n\0o\0p\0t\0u\0.bss", + .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0t\0u\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 62, + .key_type_id = 0, + .value_type_id = 8, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid offset", +}, +{ + .descr = "global data test #15, not var kind", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(1, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0.bss", + .str_sec_size = sizeof("\0A\0t\0.bss"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 3, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Not a VAR kind member", +}, +{ + .descr = "global data test #16, invalid var referencing sec", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 5, 0), /* [2] */ + BTF_VAR_ENC(NAME_TBD, 2, 0), /* [3] */ + /* a section */ /* [4] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(3, 0, 4), + /* a section */ /* [5] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(6, 0, 4), + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [6] */ + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #17, invalid var referencing var", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [2] */ + BTF_VAR_ENC(NAME_TBD, 2, 0), /* [3] */ + /* a section */ /* [4] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(3, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #18, invalid var loop", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 2, 0), /* [2] */ + /* .bss section */ /* [3] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_DATASEC, 0, 1), 4), + BTF_VAR_SECINFO_ENC(2, 0, 4), + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0aaa", + .str_sec_size = sizeof("\0A\0t\0aaa"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #19, invalid var referencing var", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_VAR_ENC(NAME_TBD, 3, 0), /* [2] */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [3] */ + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #20, invalid ptr referencing var", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* PTR type_id=3 */ /* [2] */ + BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 3), + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [3] */ + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid type_id", +}, +{ + .descr = "global data test #21, var included in struct", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + /* struct A { */ /* [2] */ + BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), sizeof(int) * 2), + BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */ + BTF_MEMBER_ENC(NAME_TBD, 3, 32),/* VAR type_id=3; */ + /* } */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [3] */ + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid member", +}, +{ + .descr = "global data test #22, array of var", + .raw_types = { + /* int */ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ + BTF_TYPE_ARRAY_ENC(3, 1, 4), /* [2] */ + BTF_VAR_ENC(NAME_TBD, 1, 0), /* [3] */ + BTF_END_RAW, + }, + .str_sec = "\0A\0t\0s\0a\0a", + .str_sec_size = sizeof("\0A\0t\0s\0a\0a"), + .map_type = BPF_MAP_TYPE_ARRAY, + .map_name = ".bss", + .key_size = sizeof(int), + .value_size = 4, + .key_type_id = 0, + .value_type_id = 4, + .max_entries = 1, + .btf_load_err = true, + .err_str = "Invalid elem", +}, /* Test member exceeds the size of struct. * * struct A {
Signature · 1 line
2.9.5