bpf: kernel CO-RE relocation
cover bpf: kernel CO-RE relocation
From: Matteo Croce <[email protected]>
This series is a preliminary work to move the CO-RE relocations in the kernel. This is the first step to have signed BPF files, since userspace instruction patching makes any verification ineffective.
The first patch just copies the relocation code from tools/lib/bpf/, the second one refactors the whole code so to build it in kernel. It builds with a single warning, which can be suppressed by switching to dynamic allocation:
kernel/bpf/relo_core.c: In function 'bpf_core_apply_relo_insn':
kernel/bpf/relo_core.c:1457:1: warning: the frame size of 2776 bytes is larger than 1024 bytes [-Wframe-larger-than=]
To really use this code and do the the relocations done in kernel, we will need an API to pass the relocation informations along the BPF file, so don't consider this patches for inclusion, but just as a first step toward the full work.
Matteo Croce (2):
btf: copy relo_core from tools to kernel
btf: adapt relo_core for kernel compilation
include/linux/btf.h | 65 ++ kernel/bpf/Makefile | 1 + kernel/bpf/btf.c | 45 +- kernel/bpf/relo_core.c | 1457 ++++++++++++++++++++++++++++++++++++++++ kernel/bpf/relo_core.h | 100 +++ 5 files changed, 1630 insertions(+), 38 deletions(-) create mode 100644 kernel/bpf/relo_core.c create mode 100644 kernel/bpf/relo_core.h
Signature · 1 line
2.31.1
RFC 1/2 btf: copy relo_core from tools to kernel
From: Matteo Croce <[email protected]>
Just copy relo_core.{c,h} from tools/lib/bpf/ to kernel/bpf/ and adapt the Makefile.
--- kernel/bpf/Makefile | 1 + kernel/bpf/relo_core.c | 1295 ++++++++++++++++++++++++++++++++++++++++ kernel/bpf/relo_core.h | 100 ++++ 3 files changed, 1396 insertions(+) create mode 100644 kernel/bpf/relo_core.c create mode 100644 kernel/bpf/relo_core.h
kernel/bpf/Makefile +1 -0
@@ -36,3 +36,4 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o obj-${CONFIG_BPF_LSM} += bpf_lsm.o endif obj-$(CONFIG_BPF_PRELOAD) += preload/ +obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
kernel/bpf/relo_core.c new file +1295 -0
1,295 lines not on this page · raw message
kernel/bpf/relo_core.h new file +100 -0
100 lines not on this page · raw message
Signature · 1 line
2.31.1
RFC 2/2 btf: adapt relo_core for kernel compilation
In reply to Matteo CroceFrom: Matteo Croce <[email protected]>
Refactor kernel/bpf/relo_core.c so it builds in kernel contexxt.
- Replace lot of helpers used by the userspace code with the in-kernel equivalent (e.g. s/btf_is_composite/btf_type_is_struct/ and s/btf_vlen/btf_type_vlen)
- Move some static helpers from btf.c to btf.h (e.g. btf_type_is_array)
- Port utility functions (e.g. str_is_empty)
--- include/linux/btf.h | 65 ++++++++++ kernel/bpf/btf.c | 45 ++----- kernel/bpf/relo_core.c | 272 ++++++++++++++++++++++++++++++++--------- 3 files changed, 289 insertions(+), 93 deletions(-)
include/linux/btf.h +65 -0
@@ -123,6 +123,11 @@ const char *btf_type_str(const struct btf_type *t); i <btf_type_vlen (datasec_type); \ i++, member++) +static inline __u16 btf_kind (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info); +} +static inline bool btf_type_is_ptr (const struct btf_type *t) {return BTF_INFO_KIND (t->info) ==BTF_KIND_PTR ;
@@ -168,6 +173,34 @@ static inline bool btf_type_is_var(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_VAR ; } +static inline bool btf_type_is_array (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_ARRAY ; +} + +static inline bool btf_type_is_modifier (const struct btf_type *t) +{ +/* Some of them is not strictly a C modifier +* but they are grouped into the same bucket +* for BTF concern: +* A type (t) that refers to another +* type through t->type AND its size cannot +* be determined without following the t->type. +* +* ptr does not fall into this bucket +* because its size is always sizeof(void *). +*/ +switch (BTF_INFO_KIND (t->info)) { +case BTF_KIND_TYPEDEF : +case BTF_KIND_VOLATILE : +case BTF_KIND_CONST : +case BTF_KIND_RESTRICT : +return true; + } + +return false; +} +/* union is only a special case of struct: * all its offsetof(member) == 0 */
@@ -207,6 +240,21 @@ static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type, : 0; } +static inline __u8 btf_member_int_offset (const struct btf_type *t) +{ +return BTF_INT_OFFSET (*(__u32 *)(t +1 )); +} + +static inline __u8 btf_int_encoding (const struct btf_type *t) +{ +return BTF_INT_ENCODING (*(__u32 *)(t +1 )); +} + +static inline struct btf_param *btf_type_params (const struct btf_type *t) +{ +return (struct btf_param *)(t +1 ); +} +static inline const struct btf_member *btf_type_member (const struct btf_type *t) {return (const struct btf_member *)(t +1 );
@@ -218,6 +266,23 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(return (const struct btf_var_secinfo *)(t +1 ); } +static inline const struct btf_enum *btf_type_enum (const struct btf_type *t) +{ +return (const struct btf_enum *)(t +1 ); +} + +static inline const struct btf_array *btf_type_array (const struct btf_type *t) +{ +return (const struct btf_array *)(t +1 ); +} + +static inline bool is_ldimm64_insn (struct bpf_insn *insn) +{ +return insn->code == (BPF_LD |BPF_IMM |BPF_DW ); +} + +const struct btf_type *btf__type_by_id (const struct btf *btf,__u32 type_id); +#ifdef CONFIG_BPF_SYSCALL struct bpf_prog ;
kernel/bpf/btf.c +7 -38
69 lines not on this page · raw message
kernel/bpf/relo_core.c +217 -55
449 lines not on this page · raw message
Signature · 1 line
2.31.1
kernel test robot
build botRe: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation
reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross #…
Hi Matteo,
[FYI, it's a private test report for your RFC patch.] [auto build test WARNING on bpf/master]
url: https://github.com/0day-ci/linux/commits/Matteo-Croce/bpf-kernel-CO-RE-relocation/20210909-213900 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git master config: sh-allmodconfig (attached as .config) compiler: sh4-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c195b385facdf03a4ce9fc7a00d370b1753de6b6
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matteo-Croce/bpf-kernel-CO-RE-relocation/20210909-213900 git checkout c195b385facdf03a4ce9fc7a00d370b1753de6b6
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross ARCH=sh
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> kernel/bpf/relo_core.c:168:1: warning: no previous prototype for 'skip_mods_and_typedefs' [-Wmissing-prototypes]
168 | skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id)
| ^~~~~~~~~~~~~~~~~~~~~~
kernel/bpf/relo_core.c: In function 'bpf_core_apply_relo_insn':
>> kernel/bpf/relo_core.c:1457:1: warning: the frame size of 2188 bytes is larger than 1024 bytes [-Wframe-larger-than=]
1457 | }
| ^Kconfig warnings: (for reference only)
WARNING: unmet direct dependencies detected for SND_ATMEL_SOC_PDC
Depends on SOUND && !UML && SND && SND_SOC && SND_ATMEL_SOC && HAS_DMA Selected by
- SND_ATMEL_SOC_SSC && SOUND && !UML && SND && SND_SOC && SND_ATMEL_SOC
- SND_ATMEL_SOC_SSC_PDC && SOUND && !UML && SND && SND_SOC && SND_ATMEL_SOC && ATMEL_SSC
vim +/skip_mods_and_typedefs +168 kernel/bpf/relo_core.c
166
167 const struct btf_type *
> 168 skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id)
169 {
170 const struct btf_type *t = btf__type_by_id(btf, id);
171
172 if (res_id)
173 *res_id = id;
174
175 while (btf_type_is_modifier(t) || btf_type_is_typedef(t)) {
176 if (res_id)
177 *res_id = t->type;
178 t = btf__type_by_id(btf, t->type);
179 }
180
181 return t;
182 }
183
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation
In reply to Matteo CroceQuoted 9 lines · Matteo Croce
From: Matteo Croce <[email protected]>
Refactor
kernel/bpf/relo_core.cso it builds in kernel contexxt.- Replace lot of helpers used by the userspace code with the in-kernel equivalent (e.g. s/btf_is_composite/btf_type_is_struct/ and s/btf_vlen/btf_type_vlen)
- Move some static helpers frombtf.ctobtf.h(e.g.btf_type_is_array)
- Port utility functions (e.g.str_is_empty)
Cool. It's great to see that you're working on it.
Quoted 49 lines
(quoted diff) +49
-0+int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id, + const struct btf *targ_btf, __u32 targ_id) +{ + const struct btf_type *local_type, *targ_type; + int depth = 32; /* max recursion depth */ + + /* caller made sure that names match (ignoring flavor suffix) */ + local_type = btf__type_by_id(local_btf, local_id); + targ_type = btf__type_by_id(targ_btf, targ_id); + if (btf_kind(local_type) != btf_kind(targ_type)) + return 0; + +recur: + depth--; + if (depth < 0) + return -EINVAL; + + local_type = skip_mods_and_typedefs(local_btf, local_id, &local_id); + targ_type = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id); + if (!local_type || !targ_type) + return -EINVAL; + + if (btf_kind(local_type) != btf_kind(targ_type)) + return 0; + + switch (btf_kind(local_type)) { + case BTF_KIND_UNKN: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_ENUM: + case BTF_KIND_FWD: + return 1; + case BTF_KIND_INT: + /* just reject deprecated bitfield-like integers; all other + * integers are by default compatible between each other + */ + return btf_member_int_offset(local_type) == 0 && btf_member_int_offset(targ_type) == 0; + case BTF_KIND_PTR: + local_id = local_type->type; + targ_id = targ_type->type; + goto recur; + case BTF_KIND_ARRAY: + local_id = btf_type_array(local_type)->type; + targ_id = btf_type_array(targ_type)->type; + goto recur; + case BTF_KIND_FUNC_PROTO: { + struct btf_param *local_p = btf_type_params(local_type); + struct btf_param *targ_p = btf_type_params(targ_type); + __u16 local_vlen = btf_type_vlen(local_type);
Review
Quoted 10 lines
(quoted diff) +10
-0+ __u16 targ_vlen = btf_type_vlen(targ_type); + int i, err; + + if (local_vlen != targ_vlen) + return 0; + + for (i = 0; i < local_vlen; i++, local_p++, targ_p++) { + skip_mods_and_typedefs(local_btf, local_p->type, &local_id); + skip_mods_and_typedefs(targ_btf, targ_p->type, &targ_id); + err = bpf_core_types_are_compat(local_btf, local_id, targ_btf, targ_id);
The main todo for this function is to convert to non-recursive or limit the recursion to some small number (like 16).
Review
Quoted 3 lines
(quoted diff) +1
-1/* record enumerator name in a first accessor */ - acc->name = btf__name_by_offset(btf, btf_enum(t)[access_idx].name_off); + acc->name = btf_name_by_offset(btf, btf_type_enum(t)[access_idx].name_off);
Instead of doing this kind of change and diverge further between kernel and libbpf it would be better to agree on the same names for the helpers. They're really the same. There is no good reason for them to have different names in kernel's btf.h and libbpf's btf.h.
See attached patch how relo_core.c can be converted for kernel duty without copy-paste. We're doing double compile for disasm.c. We can do the same here.
The copy-paste will lead to code divergence.
The same bug/feature would have to be implemented twice in the future. The CO-RE algorithm can work in both kernel and user space unmodified. imo code sharing is almost always a win.
From 1e9b236914d3e7672eba2e3b99aa198ac2bdb7bd Mon Sep 17 00:00:00 2001
From: Alexei Starovoitov <[email protected]>
Date: Tue, 7 Sep 2021 15:45:44 -0700
Subject: [PATCH] bpf: Prepare relo_core.c for kernel duty.
Make relo_core.c to be compiled with kernel and with libbpf.
--- include/linux/btf.h | 84 +++++++++++++++++++++++++++++++++ kernel/bpf/Makefile | 4 ++ tools/lib/bpf/relo_core.c | 97 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 183 insertions(+), 2 deletions(-)
include/linux/btf.h +84 -0
@@ -143,6 +143,48 @@ static inline bool btf_type_is_enum(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_ENUM ; } +static inline u16 btf_kind (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info); +} + +static inline bool btf_is_enum (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_ENUM ; +} + +static inline bool btf_is_composite (const struct btf_type *t) +{ +u16 kind =btf_kind (t); + +return kind ==BTF_KIND_STRUCT || kind ==BTF_KIND_UNION ; +} + +static inline bool btf_is_array (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_ARRAY ; +} + +static inline bool btf_is_int (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_INT ; +} + +static inline bool btf_is_ptr (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_PTR ; +} + +static inline u8 btf_int_offset (const struct btf_type *t) +{ +return BTF_INT_OFFSET (*(u32 *)(t +1 )); +} + +static inline u8 btf_int_encoding (const struct btf_type *t) +{ +return BTF_INT_ENCODING (*(u32 *)(t +1 )); +} +static inline bool btf_type_is_scalar (const struct btf_type *t) {return btf_type_is_int (t) ||btf_type_is_enum (t);
@@ -183,6 +225,11 @@ static inline u16 btf_type_vlen(const struct btf_type *t)return BTF_INFO_VLEN (t->info); } +static inline u16 btf_vlen (const struct btf_type *t) +{ +return btf_type_vlen (t); +} +static inline u16 btf_func_linkage (const struct btf_type *t) {return BTF_INFO_VLEN (t->info);
@@ -193,6 +240,27 @@ static inline bool btf_type_kflag(const struct btf_type *t)return BTF_INFO_KFLAG (t->info); } +static inline struct btf_member *btf_members (const struct btf_type *t) +{ +return (struct btf_member *)(t +1 ); +} +#ifdef RELO_CORE +static inline u32 btf_member_bit_offset (const struct btf_type *t,u32 member_idx) +{ +const struct btf_member *m =btf_members (t) + member_idx; +bool kflag =btf_type_kflag (t); + +return kflag ?BTF_MEMBER_BIT_OFFSET (m->offset) : m->offset; +} + +static inline u32 btf_member_bitfield_size (const struct btf_type *t,u32 member_idx) +{ +const struct btf_member *m =btf_members (t) + member_idx; +bool kflag =btf_type_kflag (t); + +return kflag ?BTF_MEMBER_BITFIELD_SIZE (m->offset) :0 ; +} +#else static inline u32 btf_member_bit_offset (const struct btf_type *struct_type,const struct btf_member *member) {
@@ -206,12 +274,24 @@ static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type,return btf_type_kflag (struct_type) ?BTF_MEMBER_BITFIELD_SIZE (member->offset) : 0; } +#endif static inline const struct btf_member *btf_type_member (const struct btf_type *t) {return (const struct btf_member *)(t +1 ); } + +static inline struct btf_array *btf_array (const struct btf_type *t) +{ +return (struct btf_array *)(t +1 ); +} + +static inline struct btf_enum *btf_enum (const struct btf_type *t) +{ +return (struct btf_enum *)(t +1 ); +} +static inline const struct btf_var_secinfo *btf_type_var_secinfo (const struct btf_type *t) {
@@ -222,6 +302,10 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(struct bpf_prog ;const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id); +static inline const struct btf_type *btf__type_by_id (const struct btf *btf,u32 type_id) +{ +return btf_type_by_id (btf, type_id); +}const char *btf_name_by_offset (const struct btf *btf,u32 offset);struct btf *btf_parse_vmlinux (void );struct btf *bpf_prog_get_target_btf (const struct bpf_prog *prog);
kernel/bpf/Makefile +4 -0
@@ -36,3 +36,7 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o obj-${CONFIG_BPF_LSM} += bpf_lsm.o endif obj-$(CONFIG_BPF_PRELOAD) += preload/ + +obj-$(CONFIG_BPF_SYSCALL) += relo_core.o +$(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE + $(call if_changed_rule,cc_o_c)
tools/lib/bpf/relo_core.c +95 -2
116 lines not on this page · raw message
Signature · 1 line
2.30.2
Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation
Quoted 77 lines · Alexei Starovoitov
Quoted 9 lines · Matteo Croce
From: Matteo Croce <[email protected]>
Refactor
kernel/bpf/relo_core.cso it builds in kernel contexxt.- Replace lot of helpers used by the userspace code with the in-kernel equivalent (e.g. s/btf_is_composite/btf_type_is_struct/ and s/btf_vlen/btf_type_vlen)
- Move some static helpers frombtf.ctobtf.h(e.g.btf_type_is_array)
- Port utility functions (e.g.str_is_empty)Cool. It's great to see that you're working on it.
Quoted 49 lines
(quoted diff) +49
-0+int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id, + const struct btf *targ_btf, __u32 targ_id) +{ + const struct btf_type *local_type, *targ_type; + int depth = 32; /* max recursion depth */ + + /* caller made sure that names match (ignoring flavor suffix) */ + local_type = btf__type_by_id(local_btf, local_id); + targ_type = btf__type_by_id(targ_btf, targ_id); + if (btf_kind(local_type) != btf_kind(targ_type)) + return 0; + +recur: + depth--; + if (depth < 0) + return -EINVAL; + + local_type = skip_mods_and_typedefs(local_btf, local_id, &local_id); + targ_type = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id); + if (!local_type || !targ_type) + return -EINVAL; + + if (btf_kind(local_type) != btf_kind(targ_type)) + return 0; + + switch (btf_kind(local_type)) { + case BTF_KIND_UNKN: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_ENUM: + case BTF_KIND_FWD: + return 1; + case BTF_KIND_INT: + /* just reject deprecated bitfield-like integers; all other + * integers are by default compatible between each other + */ + return btf_member_int_offset(local_type) == 0 && btf_member_int_offset(targ_type) == 0; + case BTF_KIND_PTR: + local_id = local_type->type; + targ_id = targ_type->type; + goto recur; + case BTF_KIND_ARRAY: + local_id = btf_type_array(local_type)->type; + targ_id = btf_type_array(targ_type)->type; + goto recur; + case BTF_KIND_FUNC_PROTO: { + struct btf_param *local_p = btf_type_params(local_type); + struct btf_param *targ_p = btf_type_params(targ_type); + __u16 local_vlen = btf_type_vlen(local_type);Review
Quoted 10 lines
(quoted diff) +10
-0+ __u16 targ_vlen = btf_type_vlen(targ_type); + int i, err; + + if (local_vlen != targ_vlen) + return 0; + + for (i = 0; i < local_vlen; i++, local_p++, targ_p++) { + skip_mods_and_typedefs(local_btf, local_p->type, &local_id); + skip_mods_and_typedefs(targ_btf, targ_p->type, &targ_id); + err = bpf_core_types_are_compat(local_btf, local_id, targ_btf, targ_id);The main todo for this function is to convert to non-recursive or limit the recursion to some small number (like 16).
Quoted 9 lines
Review
Quoted 3 lines
/* record enumerator name in a first accessor */(quoted diff) +1
-1- acc->name = btf__name_by_offset(btf, btf_enum(t)[access_idx].name_off); + acc->name = btf_name_by_offset(btf, btf_type_enum(t)[access_idx].name_off);Instead of doing this kind of change and diverge further between kernel and libbpf it would be better to agree on the same names for the helpers. They're really the same. There is no good reason for them to have different names in kernel's
btf.hand libbpf'sbtf.h.
How can we share the helpers source too instead of duplicating it?
Quoted 2 lines
See attached patch how
relo_core.ccan be converted for kernel duty without copy-paste. We're doing double compile fordisasm.c. We can do the same here.
Agree.
Quoted 5 lines
The copy-paste will lead to code divergence.
The same bug/feature would have to be implemented twice in the future. The CO-RE algorithm can work in both kernel and user space unmodified. imo code sharing is almost always a win.
Indeed, I found a small difference between the userspace and kernel code.
In tools/lib/bpf/btf.h we have btf_is_mod() which returns true for { BTF_KIND_VOLATILE, BTF_KIND_CONST, BTF_KIND_RESTRICT }, while in kernel/bpf/btf.c we have btf_type_is_modifier() which returns true also for BTF_KIND_TYPEDEF.
Which is the right one?
Quoted 282 lines
From
1e9b236914d3e7672eba2e3b99aa198ac2bdb7bdMon Sep 17 00:00:00 2001
From: Alexei Starovoitov <[email protected]>
Date: Tue, 7 Sep 2021 15:45:44 -0700
Subject: [PATCH] bpf: Preparerelo_core.cfor kernel duty.Make
relo_core.cto be compiled with kernel and with libbpf.Signed-off-by Alexei Starovoitov <[email protected]>--- include/linux/btf.h | 84 +++++++++++++++++++++++++++++++++ kernel/bpf/Makefile | 4 ++ tools/lib/bpf/relo_core.c | 97 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 183 insertions(+), 2 deletions(-)include/linux/btf.h +84
-0@@ -143,6 +143,48 @@ static inline bool btf_type_is_enum(const struct btf_type *t)return BTF_INFO_KIND (t->info) ==BTF_KIND_ENUM ; } +static inline u16 btf_kind (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info); +} + +static inline bool btf_is_enum (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_ENUM ; +} + +static inline bool btf_is_composite (const struct btf_type *t) +{ +u16 kind =btf_kind (t); + +return kind ==BTF_KIND_STRUCT || kind ==BTF_KIND_UNION ; +} + +static inline bool btf_is_array (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_ARRAY ; +} + +static inline bool btf_is_int (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_INT ; +} + +static inline bool btf_is_ptr (const struct btf_type *t) +{ +return btf_kind (t) ==BTF_KIND_PTR ; +} + +static inline u8 btf_int_offset (const struct btf_type *t) +{ +return BTF_INT_OFFSET (*(u32 *)(t +1 )); +} + +static inline u8 btf_int_encoding (const struct btf_type *t) +{ +return BTF_INT_ENCODING (*(u32 *)(t +1 )); +} +static inline bool btf_type_is_scalar (const struct btf_type *t) {return btf_type_is_int (t) ||btf_type_is_enum (t);@@ -183,6 +225,11 @@ static inline u16 btf_type_vlen(const struct btf_type *t)return BTF_INFO_VLEN (t->info); } +static inline u16 btf_vlen (const struct btf_type *t) +{ +return btf_type_vlen (t); +} +static inline u16 btf_func_linkage (const struct btf_type *t) {return BTF_INFO_VLEN (t->info);@@ -193,6 +240,27 @@ static inline bool btf_type_kflag(const struct btf_type *t)return BTF_INFO_KFLAG (t->info); } +static inline struct btf_member *btf_members (const struct btf_type *t) +{ +return (struct btf_member *)(t +1 ); +} +#ifdef RELO_CORE +static inline u32 btf_member_bit_offset (const struct btf_type *t,u32 member_idx) +{ +const struct btf_member *m =btf_members (t) + member_idx; +bool kflag =btf_type_kflag (t); + +return kflag ?BTF_MEMBER_BIT_OFFSET (m->offset) : m->offset; +} + +static inline u32 btf_member_bitfield_size (const struct btf_type *t,u32 member_idx) +{ +const struct btf_member *m =btf_members (t) + member_idx; +bool kflag =btf_type_kflag (t); + +return kflag ?BTF_MEMBER_BITFIELD_SIZE (m->offset) :0 ; +} +#else static inline u32 btf_member_bit_offset (const struct btf_type *struct_type,const struct btf_member *member) {@@ -206,12 +274,24 @@ static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type,return btf_type_kflag (struct_type) ?BTF_MEMBER_BITFIELD_SIZE (member->offset) : 0; } +#endif static inline const struct btf_member *btf_type_member (const struct btf_type *t) {return (const struct btf_member *)(t +1 ); } + +static inline struct btf_array *btf_array (const struct btf_type *t) +{ +return (struct btf_array *)(t +1 ); +} + +static inline struct btf_enum *btf_enum (const struct btf_type *t) +{ +return (struct btf_enum *)(t +1 ); +} +static inline const struct btf_var_secinfo *btf_type_var_secinfo (const struct btf_type *t) {@@ -222,6 +302,10 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(struct bpf_prog ;const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id); +static inline const struct btf_type *btf__type_by_id (const struct btf *btf,u32 type_id) +{ +return btf_type_by_id (btf, type_id); +}const char *btf_name_by_offset (const struct btf *btf,u32 offset);struct btf *btf_parse_vmlinux (void );struct btf *bpf_prog_get_target_btf (const struct bpf_prog *prog);kernel/bpf/Makefile +4
-0@@ -36,3 +36,7 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o obj-${CONFIG_BPF_LSM} += bpf_lsm.o endif obj-$(CONFIG_BPF_PRELOAD) += preload/ + +obj-$(CONFIG_BPF_SYSCALL) += relo_core.o +$(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE + $(call if_changed_rule,cc_o_c)tools/lib/bpf/relo_core.c +95
-3@@ -1,6 +1,98 @@// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) /* Copyright (c) 2019 Facebook */ +#ifdef __KERNEL__ +#define RELO_CORE +#include <linux/bpf.h> +#include <linux/btf.h> +#include <linux/string.h> +#include "relo_core.h" + +static const char *btf_kind_str (const struct btf_type *t) +{ +return btf_type_str (t); +} + +static bool str_is_empty (const char *s) +{ +return !s || !s[0 ]; +} + +static bool is_ldimm64_insn (struct bpf_insn *insn) +{ +return insn->code == (BPF_LD |BPF_IMM |BPF_DW ); +} + +static const struct btf_type * +skip_mods_and_typedefs (const struct btf *btf,u32 id,u32 *res_id) +{ +return btf_type_skip_modifiers (btf, id, res_id); +} + +static const char *btf__name_by_offset (const struct btf *btf,u32 offset) +{ +return btf_name_by_offset (btf, offset); +} + +static s64 btf__resolve_size (const struct btf *btf,u32 type_id) +{ +const struct btf_type *t; +int size; + + t =btf_type_by_id (btf, type_id); + t =btf_resolve_size (btf, t, &size); +if (IS_ERR (t)) +return PTR_ERR (t); +return size; +} + +static bool bpf_core_is_flavor_sep (const char *s) +{ +/* check X___Y name pattern, where X and Y are not underscores */ +return s[0 ] !='_' &&/* X */ + s[1 ] =='_' && s[2 ] =='_' && s[3 ] =='_' &&/* ___ */ + s[4 ] !='_' ;/* Y */ +} + +size_t bpf_core_essential_name_len (const char *name) +{ +size_t n =strlen (name); +int i; + +for (i = n -5 ; i >=0 ; i--) { +if (bpf_core_is_flavor_sep (name + i)) +return i +1 ; + } +return n; +} + +enum libbpf_print_level { +LIBBPF_WARN , +LIBBPF_INFO , +LIBBPF_DEBUG , +}; +__attribute__((format (printf,2 ,3 ))) +void libbpf_print (enum libbpf_print_level level, +const char *format, ...) +{ +} +#define __pr (level, fmt, ...) \ +do { \ + libbpf_print(level, "libbpf: " fmt, ##__VA_ARGS__); \ +} while (0) + +#undef pr_warn +#undef pr_info +#undef pr_debug +#define pr_warn (fmt, ...) __pr(LIBBPF_WARN, fmt, ##__VA_ARGS__) +#define pr_info (fmt, ...) __pr(LIBBPF_INFO, fmt, ##__VA_ARGS__) +#define pr_debug (fmt, ...) __pr(LIBBPF_DEBUG, fmt, ##__VA_ARGS__) +int bpf_core_types_are_compat (const struct btf *local_btf,__u32 local_id, +const struct btf *targ_btf,__u32 targ_id) +{ +return 0 ; +} +#else #include <stdio.h> #include <string.h> #include <errno.h> @@ -12,8 +104,9 @@#include "btf.h" #include "str_error.h" #include "libbpf_internal.h" +#endif -#define BPF_CORE_SPEC_MAX_LEN 64 +#define BPF_CORE_SPEC_MAX_LEN 32/* represents BPF CO-RE field or array element accessor */ struct bpf_core_accessor {@@ -662,7 +755,7 @@ static int bpf_core_calc_field_relo(const char *prog_name, *validate = true;/* signedness is never ambiguous */ break ;case BPF_FIELD_LSHIFT_U64 : -#if __BYTE_ORDER == __LITTLE_ENDIAN +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ *val =64 - (bit_off + bit_sz - byte_off *8 );#else *val = (8 - byte_sz) *8 + (bit_off - byte_off *8 ); --2.30.2
Signature · 1 line
per aspera ad upstream
Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation
Quoted 2 lines · Matteo Croce
How can we share the helpers source too instead of duplicating it?
Ideally. Yes.
Andrii pointed out that libbpf's btf.h is installed,
so it's part of libbpf api.
Therefore it's safer to rename kernel helpers with equivalent meaning instead of risking libbpf renames.
Quoted 9 lines
Indeed, I found a small difference between the userspace and kernel code.
In
tools/lib/bpf/btf.hwe havebtf_is_mod()which returns true for {BTF_KIND_VOLATILE,BTF_KIND_CONST,BTF_KIND_RESTRICT}, while inkernel/bpf/btf.cwe havebtf_type_is_modifier()which returns true also forBTF_KIND_TYPEDEF.Which is the right one?
btf_is_mod() is part of libbpf btf.h, so we cannot change it. btf_type_is_modifier() is kernel internal helper.
It doesn't need to change and doesn't need to match. The equivalent helpers areskip_mods_and_typedefs() in the libbpf
andbtf_type_skip_modifiers() in the kernel.
In this case it's probably better to search-and-replace in libbpf. In most other cases kernel search-and-replace will be necessary. For example:btf_type_vlen->btf_vlenbtf_type_member->btf_members
Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation
Quoted 19 lines · Alexei Starovoitov
Quoted 2 lines · Matteo Croce
How can we share the helpers source too instead of duplicating it?
Ideally. Yes.
Andrii pointed out that libbpf'sbtf.his installed,
so it's part of libbpf api.
Therefore it's safer to rename kernel helpers with equivalent meaning instead of risking libbpf renames.Quoted 8 lines
Indeed, I found a small difference between the userspace and kernel code.
In
tools/lib/bpf/btf.hwe havebtf_is_mod()which returns true for {BTF_KIND_VOLATILE,BTF_KIND_CONST,BTF_KIND_RESTRICT}, while inkernel/bpf/btf.cwe havebtf_type_is_modifier()which returns true also forBTF_KIND_TYPEDEF.
Quoted 5 lines
Quoted 1 line
Which is the right one?
btf_is_mod()is part of libbpfbtf.h, so we cannot change it.btf_type_is_modifier()is kernel internal helper. It doesn't need to change and doesn't need to match.
libbpf's btf_is_mod() is not including typedef because there are cases where typedef shouldn't be skipped. I see one such case in btf_dump.c. So typedef is certainly not just a modifier from libbpf's point of view. Kernel doesn't care, though, which is why this difference exists.
Quoted 5 lines
The equivalent helpers are
skip_mods_and_typedefs()in the libbpf
andbtf_type_skip_modifiers()in the kernel.
In this case it's probably better to search-and-replace in libbpf.
27 uses in kernel vs 42 in libbpf, so, technically, libbpf should win. I don't mind mass-renaming of internal helpers, I'd just prefer the name of the function to reflect the actual logic (where as I pointed out above, libbpf knows and cares about the distinction between modifiers (const, volatile, restrict) and typedef). Should we rename both to btf_type_skip_mods_and_typedefs() or btf_type_skip_mods_typedefs()?
BTW, kernel might need to start caring about typedefs separately from modifiers in the context of btf_tag work by Yonghong. Typedefs might have btf_tags associated with them, I think.
Quoted 4 lines
In most other cases kernel search-and-replace will be necessary. For example:
btf_type_vlen->btf_vlenbtf_type_member->btf_members