Skip to contentSkip to conversation

bpf: kernel CO-RE relocation

RFC In review Matteo Croce 4 participants 8 messages Sep 9, 2021 – Sep 17, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover bpf: kernel CO-RE relocation

From: Matteo Croce <[email protected]>

This series is a preliminary work to move the CO-RE relocations in the kernel. This is the first step to have signed BPF files, since userspace instruction patching makes any verification ineffective.

The first patch just copies the relocation code from tools/lib/bpf/, the second one refactors the whole code so to build it in kernel. It builds with a single warning, which can be suppressed by switching to dynamic allocation:

kernel/bpf/relo_core.c: In function 'bpf_core_apply_relo_insn':
kernel/bpf/relo_core.c:1457:1: warning: the frame size of 2776 bytes is larger than 1024 bytes [-Wframe-larger-than=]

To really use this code and do the the relocations done in kernel, we will need an API to pass the relocation informations along the BPF file, so don't consider this patches for inclusion, but just as a first step toward the full work.

Matteo Croce (2):
btf: copy relo_core from tools to kernel
btf: adapt relo_core for kernel compilation

 include/linux/btf.h    |   65 ++
 kernel/bpf/Makefile    |    1 +
 kernel/bpf/btf.c       |   45 +-
 kernel/bpf/relo_core.c | 1457 ++++++++++++++++++++++++++++++++++++++++
 kernel/bpf/relo_core.h |  100 +++
 5 files changed, 1630 insertions(+), 38 deletions(-)
 create mode 100644 kernel/bpf/relo_core.c
 create mode 100644 kernel/bpf/relo_core.h
Signature · 1 line
2.31.1
#1 lore raw

RFC 1/2 btf: copy relo_core from tools to kernel

From: Matteo Croce <[email protected]>

Just copy relo_core.{c,h} from tools/lib/bpf/ to kernel/bpf/ and adapt the Makefile.

---
 kernel/bpf/Makefile    |    1 +
 kernel/bpf/relo_core.c | 1295 ++++++++++++++++++++++++++++++++++++++++
 kernel/bpf/relo_core.h |  100 ++++
 3 files changed, 1396 insertions(+)
 create mode 100644 kernel/bpf/relo_core.c
 create mode 100644 kernel/bpf/relo_core.h
kernel/bpf/Makefile +1 -0
index 7f33098ca63f..2ed7da9eba4a 100644
@@ -36,3 +36,4 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o
 obj-${CONFIG_BPF_LSM} += bpf_lsm.o
 endif
 obj-$(CONFIG_BPF_PRELOAD) += preload/
+obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
kernel/bpf/relo_core.c new file +1295 -0

1,295 lines not on this page · raw message

kernel/bpf/relo_core.h new file +100 -0

100 lines not on this page · raw message

Signature · 1 line
2.31.1
#2 lore raw

RFC 2/2 btf: adapt relo_core for kernel compilation

In reply to Matteo Croce

From: Matteo Croce <[email protected]>

Refactor kernel/bpf/relo_core.c so it builds in kernel contexxt.

- Replace lot of helpers used by the userspace code with the in-kernel equivalent (e.g. s/btf_is_composite/btf_type_is_struct/ and s/btf_vlen/btf_type_vlen)
- Move some static helpers from btf.c to btf.h (e.g. btf_type_is_array)
- Port utility functions (e.g. str_is_empty)

---
 include/linux/btf.h    |  65 ++++++++++
 kernel/bpf/btf.c       |  45 ++-----
 kernel/bpf/relo_core.c | 272 ++++++++++++++++++++++++++++++++---------
 3 files changed, 289 insertions(+), 93 deletions(-)
include/linux/btf.h +65 -0
index 214fde93214b..6c5bfbab9f23 100644
@@ -123,6 +123,11 @@ const char *btf_type_str(const struct btf_type *t);
 	     i < btf_type_vlen(datasec_type);			\
 	     i++, member++)
 
+static inline __u16 btf_kind(const struct btf_type *t)
+{
+	return BTF_INFO_KIND(t->info);
+}
+
 static inline bool btf_type_is_ptr(const struct btf_type *t)
 {
 	return BTF_INFO_KIND(t->info) == BTF_KIND_PTR;
@@ -168,6 +173,34 @@ static inline bool btf_type_is_var(const struct btf_type *t)
 	return BTF_INFO_KIND(t->info) == BTF_KIND_VAR;
 }
 
+static inline bool btf_type_is_array(const struct btf_type *t)
+{
+	return BTF_INFO_KIND(t->info) == BTF_KIND_ARRAY;
+}
+
+static inline bool btf_type_is_modifier(const struct btf_type *t)
+{
+	/* Some of them is not strictly a C modifier
+	 * but they are grouped into the same bucket
+	 * for BTF concern:
+	 *   A type (t) that refers to another
+	 *   type through t->type AND its size cannot
+	 *   be determined without following the t->type.
+	 *
+	 * ptr does not fall into this bucket
+	 * because its size is always sizeof(void *).
+	 */
+	switch (BTF_INFO_KIND(t->info)) {
+	case BTF_KIND_TYPEDEF:
+	case BTF_KIND_VOLATILE:
+	case BTF_KIND_CONST:
+	case BTF_KIND_RESTRICT:
+		return true;
+	}
+
+	return false;
+}
+
 /* union is only a special case of struct:
  * all its offsetof(member) == 0
  */
@@ -207,6 +240,21 @@ static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type,
 					   : 0;
 }
 
+static inline __u8 btf_member_int_offset(const struct btf_type *t)
+{
+	return BTF_INT_OFFSET(*(__u32 *)(t + 1));
+}
+
+static inline __u8 btf_int_encoding(const struct btf_type *t)
+{
+	return BTF_INT_ENCODING(*(__u32 *)(t + 1));
+}
+
+static inline struct btf_param *btf_type_params(const struct btf_type *t)
+{
+	return (struct btf_param *)(t + 1);
+}
+
 static inline const struct btf_member *btf_type_member(const struct btf_type *t)
 {
 	return (const struct btf_member *)(t + 1);
@@ -218,6 +266,23 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(
 	return (const struct btf_var_secinfo *)(t + 1);
 }
 
+static inline const struct btf_enum *btf_type_enum(const struct btf_type *t)
+{
+	return (const struct btf_enum *)(t + 1);
+}
+
+static inline const struct btf_array *btf_type_array(const struct btf_type *t)
+{
+	return (const struct btf_array *)(t + 1);
+}
+
+static inline bool is_ldimm64_insn(struct bpf_insn *insn)
+{
+	return insn->code == (BPF_LD | BPF_IMM | BPF_DW);
+}
+
+const struct btf_type *btf__type_by_id(const struct btf *btf, __u32 type_id);
+
 #ifdef CONFIG_BPF_SYSCALL
 struct bpf_prog;
 
kernel/bpf/btf.c +7 -38

69 lines not on this page · raw message

kernel/bpf/relo_core.c +217 -55

449 lines not on this page · raw message

Signature · 1 line
2.31.1
#3 lore raw

Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation

reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross #…

Hi Matteo,

[FYI, it's a private test report for your RFC patch.] [auto build test WARNING on bpf/master]

url:    https://github.com/0day-ci/linux/commits/Matteo-Croce/bpf-kernel-CO-RE-relocation/20210909-213900
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git master
config: sh-allmodconfig (attached as .config)
compiler: sh4-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c195b385facdf03a4ce9fc7a00d370b1753de6b6
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matteo-Croce/bpf-kernel-CO-RE-relocation/20210909-213900 git checkout c195b385facdf03a4ce9fc7a00d370b1753de6b6
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross ARCH=sh

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

>> kernel/bpf/relo_core.c:168:1: warning: no previous prototype for 'skip_mods_and_typedefs' [-Wmissing-prototypes]
     168 | skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id)
         | ^~~~~~~~~~~~~~~~~~~~~~
   kernel/bpf/relo_core.c: In function 'bpf_core_apply_relo_insn':
>> kernel/bpf/relo_core.c:1457:1: warning: the frame size of 2188 bytes is larger than 1024 bytes [-Wframe-larger-than=]
    1457 | }
         | ^

Kconfig warnings: (for reference only)
WARNING: unmet direct dependencies detected for SND_ATMEL_SOC_PDC
Depends on SOUND && !UML && SND && SND_SOC && SND_ATMEL_SOC && HAS_DMA Selected by
- SND_ATMEL_SOC_SSC && SOUND && !UML && SND && SND_SOC && SND_ATMEL_SOC
- SND_ATMEL_SOC_SSC_PDC && SOUND && !UML && SND && SND_SOC && SND_ATMEL_SOC && ATMEL_SSC

vim +/skip_mods_and_typedefs +168 kernel/bpf/relo_core.c

166
167 const struct btf_type *

 > 168	skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id)

169 {
170 const struct btf_type *t = btf__type_by_id(btf, id);
171
172 if (res_id)
173 *res_id = id;
174
175 while (btf_type_is_modifier(t) || btf_type_is_typedef(t)) {
176 if (res_id)
177 *res_id = t->type;
178 t = btf__type_by_id(btf, t->type);
179 }
180
181 return t;
182 }
183

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#4 lore raw

Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation

In reply to Matteo Croce
Quoted 9 lines · Matteo Croce

From: Matteo Croce <[email protected]>

Refactor kernel/bpf/relo_core.c so it builds in kernel contexxt.

- Replace lot of helpers used by the userspace code with the in-kernel equivalent (e.g. s/btf_is_composite/btf_type_is_struct/ and s/btf_vlen/btf_type_vlen)
- Move some static helpers from btf.c to btf.h (e.g. btf_type_is_array)
- Port utility functions (e.g. str_is_empty)

Cool. It's great to see that you're working on it.

Quoted 49 lines
(quoted diff) +49 -0
+int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
+			      const struct btf *targ_btf, __u32 targ_id)
+{
+	const struct btf_type *local_type, *targ_type;
+	int depth = 32; /* max recursion depth */
+
+	/* caller made sure that names match (ignoring flavor suffix) */
+	local_type = btf__type_by_id(local_btf, local_id);
+	targ_type = btf__type_by_id(targ_btf, targ_id);
+	if (btf_kind(local_type) != btf_kind(targ_type))
+		return 0;
+
+recur:
+	depth--;
+	if (depth < 0)
+		return -EINVAL;
+
+	local_type = skip_mods_and_typedefs(local_btf, local_id, &local_id);
+	targ_type = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id);
+	if (!local_type || !targ_type)
+		return -EINVAL;
+
+	if (btf_kind(local_type) != btf_kind(targ_type))
+		return 0;
+
+	switch (btf_kind(local_type)) {
+	case BTF_KIND_UNKN:
+	case BTF_KIND_STRUCT:
+	case BTF_KIND_UNION:
+	case BTF_KIND_ENUM:
+	case BTF_KIND_FWD:
+		return 1;
+	case BTF_KIND_INT:
+		/* just reject deprecated bitfield-like integers; all other
+		 * integers are by default compatible between each other
+		 */
+		return btf_member_int_offset(local_type) == 0 && btf_member_int_offset(targ_type) == 0;
+	case BTF_KIND_PTR:
+		local_id = local_type->type;
+		targ_id = targ_type->type;
+		goto recur;
+	case BTF_KIND_ARRAY:
+		local_id = btf_type_array(local_type)->type;
+		targ_id = btf_type_array(targ_type)->type;
+		goto recur;
+	case BTF_KIND_FUNC_PROTO: {
+		struct btf_param *local_p = btf_type_params(local_type);
+		struct btf_param *targ_p = btf_type_params(targ_type);
+		__u16 local_vlen = btf_type_vlen(local_type);

Review

Quoted 10 lines
(quoted diff) +10 -0
+		__u16 targ_vlen = btf_type_vlen(targ_type);
+		int i, err;
+
+		if (local_vlen != targ_vlen)
+			return 0;
+
+		for (i = 0; i < local_vlen; i++, local_p++, targ_p++) {
+			skip_mods_and_typedefs(local_btf, local_p->type, &local_id);
+			skip_mods_and_typedefs(targ_btf, targ_p->type, &targ_id);
+			err = bpf_core_types_are_compat(local_btf, local_id, targ_btf, targ_id);

The main todo for this function is to convert to non-recursive or limit the recursion to some small number (like 16).

Review

Quoted 3 lines
(quoted diff) +1 -1
 		/* record enumerator name in a first accessor */
-		acc->name = btf__name_by_offset(btf, btf_enum(t)[access_idx].name_off);
+		acc->name = btf_name_by_offset(btf, btf_type_enum(t)[access_idx].name_off);

Instead of doing this kind of change and diverge further between kernel and libbpf it would be better to agree on the same names for the helpers. They're really the same. There is no good reason for them to have different names in kernel's btf.h and libbpf's btf.h.

See attached patch how relo_core.c can be converted for kernel duty without copy-paste. We're doing double compile for disasm.c. We can do the same here.
The copy-paste will lead to code divergence.
The same bug/feature would have to be implemented twice in the future. The CO-RE algorithm can work in both kernel and user space unmodified. imo code sharing is almost always a win.

From 1e9b236914d3e7672eba2e3b99aa198ac2bdb7bd Mon Sep 17 00:00:00 2001
From: Alexei Starovoitov <[email protected]>
Date: Tue, 7 Sep 2021 15:45:44 -0700
Subject: [PATCH] bpf: Prepare relo_core.c for kernel duty.

Make relo_core.c to be compiled with kernel and with libbpf.

---
 include/linux/btf.h       | 84 +++++++++++++++++++++++++++++++++
 kernel/bpf/Makefile       |  4 ++
 tools/lib/bpf/relo_core.c | 97 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 183 insertions(+), 2 deletions(-)
include/linux/btf.h +84 -0
index 214fde93214b..152aff09ee2d 100644
@@ -143,6 +143,48 @@ static inline bool btf_type_is_enum(const struct btf_type *t)
 	return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM;
 }
 
+static inline u16 btf_kind(const struct btf_type *t)
+{
+	return BTF_INFO_KIND(t->info);
+}
+
+static inline bool btf_is_enum(const struct btf_type *t)
+{
+	return btf_kind(t) == BTF_KIND_ENUM;
+}
+
+static inline bool btf_is_composite(const struct btf_type *t)
+{
+	u16 kind = btf_kind(t);
+
+	return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION;
+}
+
+static inline bool btf_is_array(const struct btf_type *t)
+{
+	return btf_kind(t) == BTF_KIND_ARRAY;
+}
+
+static inline bool btf_is_int(const struct btf_type *t)
+{
+	return btf_kind(t) == BTF_KIND_INT;
+}
+
+static inline bool btf_is_ptr(const struct btf_type *t)
+{
+	return btf_kind(t) == BTF_KIND_PTR;
+}
+
+static inline u8 btf_int_offset(const struct btf_type *t)
+{
+	return BTF_INT_OFFSET(*(u32 *)(t + 1));
+}
+
+static inline u8 btf_int_encoding(const struct btf_type *t)
+{
+	return BTF_INT_ENCODING(*(u32 *)(t + 1));
+}
+
 static inline bool btf_type_is_scalar(const struct btf_type *t)
 {
 	return btf_type_is_int(t) || btf_type_is_enum(t);
@@ -183,6 +225,11 @@ static inline u16 btf_type_vlen(const struct btf_type *t)
 	return BTF_INFO_VLEN(t->info);
 }
 
+static inline u16 btf_vlen(const struct btf_type *t)
+{
+	return btf_type_vlen(t);
+}
+
 static inline u16 btf_func_linkage(const struct btf_type *t)
 {
 	return BTF_INFO_VLEN(t->info);
@@ -193,6 +240,27 @@ static inline bool btf_type_kflag(const struct btf_type *t)
 	return BTF_INFO_KFLAG(t->info);
 }
 
+static inline struct btf_member *btf_members(const struct btf_type *t)
+{
+	return (struct btf_member *)(t + 1);
+}
+#ifdef RELO_CORE
+static inline u32 btf_member_bit_offset(const struct btf_type *t, u32 member_idx)
+{
+	const struct btf_member *m = btf_members(t) + member_idx;
+	bool kflag = btf_type_kflag(t);
+
+	return kflag ? BTF_MEMBER_BIT_OFFSET(m->offset) : m->offset;
+}
+
+static inline u32 btf_member_bitfield_size(const struct btf_type *t, u32 member_idx)
+{
+	const struct btf_member *m = btf_members(t) + member_idx;
+	bool kflag = btf_type_kflag(t);
+
+	return kflag ? BTF_MEMBER_BITFIELD_SIZE(m->offset) : 0;
+}
+#else
 static inline u32 btf_member_bit_offset(const struct btf_type *struct_type,
 					const struct btf_member *member)
 {
@@ -206,12 +274,24 @@ static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type,
 	return btf_type_kflag(struct_type) ? BTF_MEMBER_BITFIELD_SIZE(member->offset)
 					   : 0;
 }
+#endif
 
 static inline const struct btf_member *btf_type_member(const struct btf_type *t)
 {
 	return (const struct btf_member *)(t + 1);
 }
 
+
+static inline struct btf_array *btf_array(const struct btf_type *t)
+{
+	return (struct btf_array *)(t + 1);
+}
+
+static inline struct btf_enum *btf_enum(const struct btf_type *t)
+{
+	return (struct btf_enum *)(t + 1);
+}
+
 static inline const struct btf_var_secinfo *btf_type_var_secinfo(
 		const struct btf_type *t)
 {
@@ -222,6 +302,10 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(
 struct bpf_prog;
 
 const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id);
+static inline const struct btf_type *btf__type_by_id(const struct btf *btf, u32 type_id)
+{
+	return btf_type_by_id(btf, type_id);
+}
 const char *btf_name_by_offset(const struct btf *btf, u32 offset);
 struct btf *btf_parse_vmlinux(void);
 struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog);
kernel/bpf/Makefile +4 -0
index 7f33098ca63f..3d5370c876b5 100644
@@ -36,3 +36,7 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o
 obj-${CONFIG_BPF_LSM} += bpf_lsm.o
 endif
 obj-$(CONFIG_BPF_PRELOAD) += preload/
+
+obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
+$(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE
+	$(call if_changed_rule,cc_o_c)
tools/lib/bpf/relo_core.c +95 -2

116 lines not on this page · raw message

Signature · 1 line
2.30.2
#5 lore raw

Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation

Quoted 77 lines · Alexei Starovoitov
Quoted 9 lines · Matteo Croce

From: Matteo Croce <[email protected]>

Refactor kernel/bpf/relo_core.c so it builds in kernel contexxt.

- Replace lot of helpers used by the userspace code with the in-kernel equivalent (e.g. s/btf_is_composite/btf_type_is_struct/ and s/btf_vlen/btf_type_vlen)
- Move some static helpers from btf.c to btf.h (e.g. btf_type_is_array)
- Port utility functions (e.g. str_is_empty)

Cool. It's great to see that you're working on it.

Quoted 49 lines
(quoted diff) +49 -0
+int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
+                           const struct btf *targ_btf, __u32 targ_id)
+{
+     const struct btf_type *local_type, *targ_type;
+     int depth = 32; /* max recursion depth */
+
+     /* caller made sure that names match (ignoring flavor suffix) */
+     local_type = btf__type_by_id(local_btf, local_id);
+     targ_type = btf__type_by_id(targ_btf, targ_id);
+     if (btf_kind(local_type) != btf_kind(targ_type))
+             return 0;
+
+recur:
+     depth--;
+     if (depth < 0)
+             return -EINVAL;
+
+     local_type = skip_mods_and_typedefs(local_btf, local_id, &local_id);
+     targ_type = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id);
+     if (!local_type || !targ_type)
+             return -EINVAL;
+
+     if (btf_kind(local_type) != btf_kind(targ_type))
+             return 0;
+
+     switch (btf_kind(local_type)) {
+     case BTF_KIND_UNKN:
+     case BTF_KIND_STRUCT:
+     case BTF_KIND_UNION:
+     case BTF_KIND_ENUM:
+     case BTF_KIND_FWD:
+             return 1;
+     case BTF_KIND_INT:
+             /* just reject deprecated bitfield-like integers; all other
+              * integers are by default compatible between each other
+              */
+             return btf_member_int_offset(local_type) == 0 && btf_member_int_offset(targ_type) == 0;
+     case BTF_KIND_PTR:
+             local_id = local_type->type;
+             targ_id = targ_type->type;
+             goto recur;
+     case BTF_KIND_ARRAY:
+             local_id = btf_type_array(local_type)->type;
+             targ_id = btf_type_array(targ_type)->type;
+             goto recur;
+     case BTF_KIND_FUNC_PROTO: {
+             struct btf_param *local_p = btf_type_params(local_type);
+             struct btf_param *targ_p = btf_type_params(targ_type);
+             __u16 local_vlen = btf_type_vlen(local_type);

Review

Quoted 10 lines
(quoted diff) +10 -0
+             __u16 targ_vlen = btf_type_vlen(targ_type);
+             int i, err;
+
+             if (local_vlen != targ_vlen)
+                     return 0;
+
+             for (i = 0; i < local_vlen; i++, local_p++, targ_p++) {
+                     skip_mods_and_typedefs(local_btf, local_p->type, &local_id);
+                     skip_mods_and_typedefs(targ_btf, targ_p->type, &targ_id);
+                     err = bpf_core_types_are_compat(local_btf, local_id, targ_btf, targ_id);

The main todo for this function is to convert to non-recursive or limit the recursion to some small number (like 16).

Quoted 9 lines

Review

Quoted 3 lines
              /* record enumerator name in a first accessor */
(quoted diff) +1 -1
-             acc->name = btf__name_by_offset(btf, btf_enum(t)[access_idx].name_off);
+             acc->name = btf_name_by_offset(btf, btf_type_enum(t)[access_idx].name_off);

Instead of doing this kind of change and diverge further between kernel and libbpf it would be better to agree on the same names for the helpers. They're really the same. There is no good reason for them to have different names in kernel's btf.h and libbpf's btf.h.

How can we share the helpers source too instead of duplicating it?

Quoted 2 lines

See attached patch how relo_core.c can be converted for kernel duty without copy-paste. We're doing double compile for disasm.c. We can do the same here.

Agree.

Quoted 5 lines

The copy-paste will lead to code divergence.
The same bug/feature would have to be implemented twice in the future. The CO-RE algorithm can work in both kernel and user space unmodified. imo code sharing is almost always a win.

Indeed, I found a small difference between the userspace and kernel code.

In tools/lib/bpf/btf.h we have btf_is_mod() which returns true for { BTF_KIND_VOLATILE, BTF_KIND_CONST, BTF_KIND_RESTRICT }, while in kernel/bpf/btf.c we have btf_type_is_modifier() which returns true also for BTF_KIND_TYPEDEF.

Which is the right one?

Quoted 282 lines

From 1e9b236914d3e7672eba2e3b99aa198ac2bdb7bd Mon Sep 17 00:00:00 2001
From: Alexei Starovoitov <[email protected]>
Date: Tue, 7 Sep 2021 15:45:44 -0700
Subject: [PATCH] bpf: Prepare relo_core.c for kernel duty.

Make relo_core.c to be compiled with kernel and with libbpf.

---
 include/linux/btf.h       | 84 +++++++++++++++++++++++++++++++++
 kernel/bpf/Makefile       |  4 ++
 tools/lib/bpf/relo_core.c | 97 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 183 insertions(+), 2 deletions(-)
include/linux/btf.h +84 -0
index 214fde93214b..152aff09ee2d 100644
@@ -143,6 +143,48 @@ static inline bool btf_type_is_enum(const struct btf_type *t)
        return BTF_INFO_KIND(t->info) == BTF_KIND_ENUM;
 }
 
+static inline u16 btf_kind(const struct btf_type *t)
+{
+       return BTF_INFO_KIND(t->info);
+}
+
+static inline bool btf_is_enum(const struct btf_type *t)
+{
+       return btf_kind(t) == BTF_KIND_ENUM;
+}
+
+static inline bool btf_is_composite(const struct btf_type *t)
+{
+       u16 kind = btf_kind(t);
+
+       return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION;
+}
+
+static inline bool btf_is_array(const struct btf_type *t)
+{
+       return btf_kind(t) == BTF_KIND_ARRAY;
+}
+
+static inline bool btf_is_int(const struct btf_type *t)
+{
+       return btf_kind(t) == BTF_KIND_INT;
+}
+
+static inline bool btf_is_ptr(const struct btf_type *t)
+{
+       return btf_kind(t) == BTF_KIND_PTR;
+}
+
+static inline u8 btf_int_offset(const struct btf_type *t)
+{
+       return BTF_INT_OFFSET(*(u32 *)(t + 1));
+}
+
+static inline u8 btf_int_encoding(const struct btf_type *t)
+{
+       return BTF_INT_ENCODING(*(u32 *)(t + 1));
+}
+
 static inline bool btf_type_is_scalar(const struct btf_type *t)
 {
        return btf_type_is_int(t) || btf_type_is_enum(t);
@@ -183,6 +225,11 @@ static inline u16 btf_type_vlen(const struct btf_type *t)
        return BTF_INFO_VLEN(t->info);
 }
 
+static inline u16 btf_vlen(const struct btf_type *t)
+{
+       return btf_type_vlen(t);
+}
+
 static inline u16 btf_func_linkage(const struct btf_type *t)
 {
        return BTF_INFO_VLEN(t->info);
@@ -193,6 +240,27 @@ static inline bool btf_type_kflag(const struct btf_type *t)
        return BTF_INFO_KFLAG(t->info);
 }
 
+static inline struct btf_member *btf_members(const struct btf_type *t)
+{
+       return (struct btf_member *)(t + 1);
+}
+#ifdef RELO_CORE
+static inline u32 btf_member_bit_offset(const struct btf_type *t, u32 member_idx)
+{
+       const struct btf_member *m = btf_members(t) + member_idx;
+       bool kflag = btf_type_kflag(t);
+
+       return kflag ? BTF_MEMBER_BIT_OFFSET(m->offset) : m->offset;
+}
+
+static inline u32 btf_member_bitfield_size(const struct btf_type *t, u32 member_idx)
+{
+       const struct btf_member *m = btf_members(t) + member_idx;
+       bool kflag = btf_type_kflag(t);
+
+       return kflag ? BTF_MEMBER_BITFIELD_SIZE(m->offset) : 0;
+}
+#else
 static inline u32 btf_member_bit_offset(const struct btf_type *struct_type,
                                        const struct btf_member *member)
 {
@@ -206,12 +274,24 @@ static inline u32 btf_member_bitfield_size(const struct btf_type *struct_type,
        return btf_type_kflag(struct_type) ? BTF_MEMBER_BITFIELD_SIZE(member->offset)
                                           : 0;
 }
+#endif
 
 static inline const struct btf_member *btf_type_member(const struct btf_type *t)
 {
        return (const struct btf_member *)(t + 1);
 }
 
+
+static inline struct btf_array *btf_array(const struct btf_type *t)
+{
+       return (struct btf_array *)(t + 1);
+}
+
+static inline struct btf_enum *btf_enum(const struct btf_type *t)
+{
+       return (struct btf_enum *)(t + 1);
+}
+
 static inline const struct btf_var_secinfo *btf_type_var_secinfo(
                const struct btf_type *t)
 {
@@ -222,6 +302,10 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(
 struct bpf_prog;
 
 const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id);
+static inline const struct btf_type *btf__type_by_id(const struct btf *btf, u32 type_id)
+{
+       return btf_type_by_id(btf, type_id);
+}
 const char *btf_name_by_offset(const struct btf *btf, u32 offset);
 struct btf *btf_parse_vmlinux(void);
 struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog);
kernel/bpf/Makefile +4 -0
index 7f33098ca63f..3d5370c876b5 100644
@@ -36,3 +36,7 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o
 obj-${CONFIG_BPF_LSM} += bpf_lsm.o
 endif
 obj-$(CONFIG_BPF_PRELOAD) += preload/
+
+obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
+$(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE
+       $(call if_changed_rule,cc_o_c)
tools/lib/bpf/relo_core.c +95 -3
index 4016ed492d0c..9d1f309f05fe 100644
@@ -1,6 +1,98 @@
 // SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 /* Copyright (c) 2019 Facebook */
 
+#ifdef __KERNEL__
+#define RELO_CORE
+#include <linux/bpf.h>
+#include <linux/btf.h>
+#include <linux/string.h>
+#include "relo_core.h"
+
+static const char *btf_kind_str(const struct btf_type *t)
+{
+       return btf_type_str(t);
+}
+
+static bool str_is_empty(const char *s)
+{
+       return !s || !s[0];
+}
+
+static bool is_ldimm64_insn(struct bpf_insn *insn)
+{
+       return insn->code == (BPF_LD | BPF_IMM | BPF_DW);
+}
+
+static const struct btf_type *
+skip_mods_and_typedefs(const struct btf *btf, u32 id, u32 *res_id)
+{
+       return btf_type_skip_modifiers(btf, id, res_id);
+}
+
+static const char *btf__name_by_offset(const struct btf *btf, u32 offset)
+{
+       return btf_name_by_offset(btf, offset);
+}
+
+static s64 btf__resolve_size(const struct btf *btf, u32 type_id)
+{
+       const struct btf_type *t;
+       int size;
+
+       t = btf_type_by_id(btf, type_id);
+       t = btf_resolve_size(btf, t, &size);
+       if (IS_ERR(t))
+               return PTR_ERR(t);
+       return size;
+}
+
+static bool bpf_core_is_flavor_sep(const char *s)
+{
+       /* check X___Y name pattern, where X and Y are not underscores */
+       return s[0] != '_' &&                                 /* X */
+              s[1] == '_' && s[2] == '_' && s[3] == '_' &&   /* ___ */
+              s[4] != '_';                                   /* Y */
+}
+
+size_t bpf_core_essential_name_len(const char *name)
+{
+       size_t n = strlen(name);
+       int i;
+
+       for (i = n - 5; i >= 0; i--) {
+               if (bpf_core_is_flavor_sep(name + i))
+                       return i + 1;
+       }
+       return n;
+}
+
+enum libbpf_print_level {
+        LIBBPF_WARN,
+        LIBBPF_INFO,
+        LIBBPF_DEBUG,
+};
+__attribute__((format(printf, 2, 3)))
+void libbpf_print(enum libbpf_print_level level,
+                 const char *format, ...)
+{
+}
+#define __pr(level, fmt, ...)  \
+do {                           \
+       libbpf_print(level, "libbpf: " fmt, ##__VA_ARGS__);     \
+} while (0)
+
+#undef pr_warn
+#undef pr_info
+#undef pr_debug
+#define pr_warn(fmt, ...)      __pr(LIBBPF_WARN, fmt, ##__VA_ARGS__)
+#define pr_info(fmt, ...)      __pr(LIBBPF_INFO, fmt, ##__VA_ARGS__)
+#define pr_debug(fmt, ...)     __pr(LIBBPF_DEBUG, fmt, ##__VA_ARGS__)
+int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
+                             const struct btf *targ_btf, __u32 targ_id)
+{
+       return 0;
+}
+#else
 #include <stdio.h>
 #include <string.h>
 #include <errno.h>
@@ -12,8 +104,9 @@
 #include "btf.h"
 #include "str_error.h"
 #include "libbpf_internal.h"
+#endif
 
-#define BPF_CORE_SPEC_MAX_LEN 64
+#define BPF_CORE_SPEC_MAX_LEN 32
 
 /* represents BPF CO-RE field or array element accessor */
 struct bpf_core_accessor {
@@ -662,7 +755,7 @@ static int bpf_core_calc_field_relo(const char *prog_name,
                        *validate = true; /* signedness is never ambiguous */
                break;
        case BPF_FIELD_LSHIFT_U64:
-#if __BYTE_ORDER == __LITTLE_ENDIAN
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
                *val = 64 - (bit_off + bit_sz - byte_off  * 8);
 #else
                *val = (8 - byte_sz) * 8 + (bit_off - byte_off * 8);
--

2.30.2

Signature · 1 line
per aspera ad upstream
#6 lore raw

Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation

Quoted 2 lines · Matteo Croce

How can we share the helpers source too instead of duplicating it?

Ideally. Yes.
Andrii pointed out that libbpf's btf.h is installed,
so it's part of libbpf api.
Therefore it's safer to rename kernel helpers with equivalent meaning instead of risking libbpf renames.

Quoted 9 lines

Indeed, I found a small difference between the userspace and kernel code.

In tools/lib/bpf/btf.h we have btf_is_mod() which returns true for { BTF_KIND_VOLATILE, BTF_KIND_CONST, BTF_KIND_RESTRICT }, while in kernel/bpf/btf.c we have btf_type_is_modifier() which returns true also for BTF_KIND_TYPEDEF.

Which is the right one?

btf_is_mod() is part of libbpf btf.h, so we cannot change it. btf_type_is_modifier() is kernel internal helper.
It doesn't need to change and doesn't need to match. The equivalent helpers are
skip_mods_and_typedefs() in the libbpf
and
btf_type_skip_modifiers() in the kernel.
In this case it's probably better to search-and-replace in libbpf. In most other cases kernel search-and-replace will be necessary. For example:
btf_type_vlen->btf_vlen
btf_type_member->btf_members

#7 lore raw

Re: [RFC bpf 2/2] btf: adapt relo_core for kernel compilation

Quoted 19 lines · Alexei Starovoitov
Quoted 2 lines · Matteo Croce

How can we share the helpers source too instead of duplicating it?

Ideally. Yes.
Andrii pointed out that libbpf's btf.h is installed,
so it's part of libbpf api.
Therefore it's safer to rename kernel helpers with equivalent meaning instead of risking libbpf renames.

Quoted 8 lines

Indeed, I found a small difference between the userspace and kernel code.

In tools/lib/bpf/btf.h we have btf_is_mod() which returns true for { BTF_KIND_VOLATILE, BTF_KIND_CONST, BTF_KIND_RESTRICT }, while in kernel/bpf/btf.c we have btf_type_is_modifier() which returns true also for BTF_KIND_TYPEDEF.

Quoted 5 lines
Quoted 1 line

Which is the right one?

btf_is_mod() is part of libbpf btf.h, so we cannot change it. btf_type_is_modifier() is kernel internal helper. It doesn't need to change and doesn't need to match.

libbpf's btf_is_mod() is not including typedef because there are cases where typedef shouldn't be skipped. I see one such case in btf_dump.c. So typedef is certainly not just a modifier from libbpf's point of view. Kernel doesn't care, though, which is why this difference exists.

Quoted 5 lines

The equivalent helpers are
skip_mods_and_typedefs() in the libbpf
and
btf_type_skip_modifiers() in the kernel.
In this case it's probably better to search-and-replace in libbpf.

27 uses in kernel vs 42 in libbpf, so, technically, libbpf should win. I don't mind mass-renaming of internal helpers, I'd just prefer the name of the function to reflect the actual logic (where as I pointed out above, libbpf knows and cares about the distinction between modifiers (const, volatile, restrict) and typedef). Should we rename both to btf_type_skip_mods_and_typedefs() or btf_type_skip_mods_typedefs()?

BTW, kernel might need to start caring about typedefs separately from modifiers in the context of btf_tag work by Yonghong. Typedefs might have btf_tags associated with them, I think.

Quoted 4 lines

In most other cases kernel search-and-replace will be necessary. For example:
btf_type_vlen->btf_vlen
btf_type_member->btf_members