Skip to contentSkip to conversation

Support BTF-powered BPF tracing programs for kernel modules

PATCH In review Andrii Nakryiko 3 participants 11 messages Dec 1, 2020

Subsystems (8)
lore
Share & tools
lore

Andrii Nakryiko

author maintainer
#0 lore raw

v2 cover Support BTF-powered BPF tracing programs for kernel modules

Building on top of two previous patch sets ([0] and not yet landed [1]), this patch sets extends kernel and libbpf with support for attaching BTF-powered raw tracepoint (tp_btf) and tracing (fentry/fexit/fmod_ret/lsm) BPF programs to BPF hooks defined in kernel modules.

Kernel UAPI for BPF_PROG_LOAD is extended with extra parameter (attach_btf_obj_id) which allows to specify kernel module BTF in which the BTF type is identifed by attach_btf_id.

From end user perspective there are no extra actions that need to happen. Libbpf will continue searching across all kernel module BTFs, if desired attach BTF type is not found in vmlinux. That way it doesn't matter if BPF hook that user is trying to attach to is built-in into vmlinux image or is loaded in kernel module.

Currently pahole doesn't generate BTF_KIND_FUNC info for ftrace-able static functions in kernel modules, so expose traced function in bpf_sidecar.ko. Once pahole is enhanced, we can go back to static function.

[0] https://patchwork.kernel.org/project/netdevbpf/list/?series=380759&state=*
[1] https://patchwork.kernel.org/project/netdevbpf/list/?series=393677&state=*

v1->v2:
- avoid increasing bpf_reg_state by reordering fields (Alexei);
- preserve btf_data_size in struct module;
- rebase on top of v3 of patch [1].

Andrii Nakryiko (8):
bpf: keep module's btf_data_size intact after load
bpf: remove hard-coded btf_vmlinux assumption from BPF verifier
bpf: allow to specify kernel module BTFs when attaching BPF programs
libbpf: factor out low-level BPF program loading helper
libbpf: support attachment of BPF tracing programs to kernel modules selftests/bpf: add tp_btf CO-RE reloc test for modules selftests/bpf: make bpf_testmod's traceable function global selftests/bpf: add fentry/fexit/fmod_ret selftest for kernel module

 include/linux/bpf.h                           |  13 +-
 include/linux/bpf_verifier.h                  |  28 +++-
 include/linux/btf.h                           |   7 +-
 include/uapi/linux/bpf.h                      |   1 +
 kernel/bpf/btf.c                              |  90 +++++++----
 kernel/bpf/syscall.c                          |  44 +++++-
 kernel/bpf/verifier.c                         |  77 ++++++----
 kernel/module.c                               |   1 -
 net/ipv4/bpf_tcp_ca.c                         |   3 +-
 tools/include/uapi/linux/bpf.h                |   1 +
 tools/lib/bpf/bpf.c                           | 101 ++++++++----
 tools/lib/bpf/libbpf.c                        | 145 +++++++++++++-----
 tools/lib/bpf/libbpf_internal.h               |  30 ++++
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |   3 +-
 .../selftests/bpf/prog_tests/core_reloc.c     |   3 +-
 .../selftests/bpf/prog_tests/module_attach.c  |  53 +++++++
 .../bpf/progs/test_core_reloc_module.c        |  32 +++-
 .../selftests/bpf/progs/test_module_attach.c  |  66 ++++++++
 18 files changed, 546 insertions(+), 152 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/module_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_module_attach.c
Signature · 1 line
2.24.1
author maintainer
#1 lore raw

PATCH v2 1/8 bpf: keep module's btf_data_size intact after load

Having real btf_data_size stored in struct module is benefitial to quickly determine which kernel modules have associated BTF object and which don't. There is no harm in keeping this info, as opposed to keeping invalid pointer.

Fixes 607c543f939d ("bpf: Sanitize BTF data pointer after module is loaded")
---
 kernel/module.c | 1 -
 1 file changed, 1 deletion(-)
kernel/module.c +0 -1
index 18f259d61d14..c3a9e972d3b2 100644
@@ -3712,7 +3712,6 @@ static noinline int do_init_module(struct module *mod)
 #ifdef CONFIG_DEBUG_INFO_BTF_MODULES
 	/* .BTF is not SHF_ALLOC and will get removed, so sanitize pointer */
 	mod->btf_data = NULL;
-	mod->btf_data_size = 0;
 #endif
 	/*
 	 * We want to free module_init, but be aware that kallsyms may be
Signature · 1 line
2.24.1
author maintainer
#2 lore raw

PATCH v2 2/8 bpf: remove hard-coded btf_vmlinux assumption from BPF verifier

In reply to Andrii Nakryiko

Remove a permeating assumption thoughout BPF verifier of vmlinux BTF. Instead, wherever BTF type IDs are involved, also track the instance of struct btf that goes along with the type ID. This allows to gradually add support for kernel module BTFs and using/tracking module types across BPF helper calls and registers.

This patch also renames btf_id() function to btf_obj_id() to minimize naming clash with using btf_id to denote BTF *type* ID, rather than BTF *object*'s ID.

Also, altough btf_vmlinux can't get destructed and thus doesn't need refcounting, module BTFs need that, so apply BTF refcounting universally when BPF program is using BTF-powered attachment (tp_btf, fentry/fexit, etc). This makes for simpler clean up code.

Now that BTF type ID is not enough to uniquely identify a BTF type, extend BPF trampoline key to include BTF object ID. To differentiate that from target program BPF ID, set 31st bit of type ID. BTF type IDs (at least currently) are not allowed to take full 32 bits, so there is no danger of confusing that bit with a valid BTF type ID.

---
 include/linux/bpf.h          | 13 ++++--
 include/linux/bpf_verifier.h | 28 +++++++++----
 include/linux/btf.h          |  5 ++-
 kernel/bpf/btf.c             | 65 ++++++++++++++++++++----------
 kernel/bpf/syscall.c         | 24 +++++++++--
 kernel/bpf/verifier.c        | 77 ++++++++++++++++++++++--------------
 net/ipv4/bpf_tcp_ca.c        |  3 +-
 7 files changed, 148 insertions(+), 67 deletions(-)
include/linux/bpf.h +9 -4
index e1bcb6d7345c..2f3b22887ef3 100644
@@ -421,7 +421,10 @@ struct bpf_insn_access_aux {
 	enum bpf_reg_type reg_type;
 	union {
 		int ctx_field_size;
-		u32 btf_id;
+		struct {
+			struct btf *btf;
+			u32 btf_id;
+		};
 	};
 	struct bpf_verifier_log *log; /* for verbose logs */
 };
@@ -458,6 +461,7 @@ struct bpf_verifier_ops {
 				  struct bpf_insn *dst,
 				  struct bpf_prog *prog, u32 *target_size);
 	int (*btf_struct_access)(struct bpf_verifier_log *log,
+				 const struct btf *btf,
 				 const struct btf_type *t, int off, int size,
 				 enum bpf_access_type atype,
 				 u32 *next_btf_id);
@@ -771,6 +775,7 @@ struct bpf_prog_aux {
 	u32 ctx_arg_info_size;
 	u32 max_rdonly_access;
 	u32 max_rdwr_access;
+	struct btf *attach_btf;
 	const struct bpf_ctx_arg_aux *ctx_arg_info;
 	struct mutex dst_mutex; /* protects dst_* pointers below, *after* prog becomes visible */
 	struct bpf_prog *dst_prog;
@@ -1005,7 +1010,6 @@ struct bpf_event_entry {
 
 bool bpf_prog_array_compatible(struct bpf_array *array, const struct bpf_prog *fp);
 int bpf_prog_calc_tag(struct bpf_prog *fp);
-const char *kernel_type_name(u32 btf_type_id);
 
 const struct bpf_func_proto *bpf_get_trace_printk_proto(void);
 
@@ -1430,12 +1434,13 @@ int bpf_prog_test_run_raw_tp(struct bpf_prog *prog,
 bool btf_ctx_access(int off, int size, enum bpf_access_type type,
 		    const struct bpf_prog *prog,
 		    struct bpf_insn_access_aux *info);
-int btf_struct_access(struct bpf_verifier_log *log,
+int btf_struct_access(struct bpf_verifier_log *log, const struct btf *btf,
 		      const struct btf_type *t, int off, int size,
 		      enum bpf_access_type atype,
 		      u32 *next_btf_id);
 bool btf_struct_ids_match(struct bpf_verifier_log *log,
-			  int off, u32 id, u32 need_type_id);
+			  const struct btf *btf, u32 id, int off,
+			  const struct btf *need_btf, u32 need_type_id);
 
 int btf_distill_func_proto(struct bpf_verifier_log *log,
 			   struct btf *btf,
include/linux/bpf_verifier.h +21 -7
index 306869d4743b..e941fe1484e5 100644
@@ -5,6 +5,7 @@
 #define _LINUX_BPF_VERIFIER_H 1
 
 #include <linux/bpf.h> /* for enum bpf_reg_type */
+#include <linux/btf.h> /* for struct btf and btf_id() */
 #include <linux/filter.h> /* for MAX_BPF_STACK */
 #include <linux/tnum.h>
 
@@ -43,6 +44,8 @@ enum bpf_reg_liveness {
 struct bpf_reg_state {
 	/* Ordering of fields matters.  See states_equal() */
 	enum bpf_reg_type type;
+	/* Fixed part of pointer offset, pointer types only */
+	s32 off;
 	union {
 		/* valid when type == PTR_TO_PACKET */
 		int range;
@@ -52,15 +55,20 @@ struct bpf_reg_state {
 		 */
 		struct bpf_map *map_ptr;
 
-		u32 btf_id; /* for PTR_TO_BTF_ID */
+		/* for PTR_TO_BTF_ID */
+		struct {
+			struct btf *btf;
+			u32 btf_id;
+		};
 
 		u32 mem_size; /* for PTR_TO_MEM | PTR_TO_MEM_OR_NULL */
 
 		/* Max size from any of the above. */
-		unsigned long raw;
+		struct {
+			unsigned long raw1;
+			unsigned long raw2;
+		} raw;
 	};
-	/* Fixed part of pointer offset, pointer types only */
-	s32 off;
 	/* For PTR_TO_PACKET, used to find other pointers with the same variable
 	 * offset, so they can share range knowledge.
 	 * For PTR_TO_MAP_VALUE_OR_NULL this is used to share which map value we
@@ -311,7 +319,10 @@ struct bpf_insn_aux_data {
 		struct {
 			enum bpf_reg_type reg_type;	/* type of pseudo_btf_id */
 			union {
-				u32 btf_id;	/* btf_id for struct typed var */
+				struct {
+					struct btf *btf;
+					u32 btf_id;	/* btf_id for struct typed var */
+				};
 				u32 mem_size;	/* mem_size for non-struct typed var */
 			};
 		} btf_var;
@@ -459,9 +470,12 @@ int check_ctx_reg(struct bpf_verifier_env *env,
 
 /* this lives here instead of in bpf.h because it needs to dereference tgt_prog */
 static inline u64 bpf_trampoline_compute_key(const struct bpf_prog *tgt_prog,
-					     u32 btf_id)
+					     struct btf *btf, u32 btf_id)
 {
-        return tgt_prog ? (((u64)tgt_prog->aux->id) << 32 | btf_id) : btf_id;
+	if (tgt_prog)
+		return ((u64)tgt_prog->aux->id << 32) | btf_id;
+	else
+		return ((u64)btf_obj_id(btf) << 32) | 0x80000000 | btf_id;
 }
 
 int bpf_check_attach_target(struct bpf_verifier_log *log,
include/linux/btf.h +4 -1
index 2bf641829664..fb608e4de076 100644
@@ -18,6 +18,7 @@ struct btf_show;
 
 extern const struct file_operations btf_fops;
 
+void btf_get(struct btf *btf);
 void btf_put(struct btf *btf);
 int btf_new_fd(const union bpf_attr *attr);
 struct btf *btf_get_by_fd(int fd);
@@ -88,7 +89,7 @@ int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj,
 			   char *buf, int len, u64 flags);
 
 int btf_get_fd_by_id(u32 id);
-u32 btf_id(const struct btf *btf);
+u32 btf_obj_id(const struct btf *btf);
 bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,
 			   const struct btf_member *m,
 			   u32 expected_offset, u32 expected_size);
@@ -206,6 +207,8 @@ static inline const struct btf_var_secinfo *btf_type_var_secinfo(
 }
 
 #ifdef CONFIG_BPF_SYSCALL
+struct bpf_prog;
+
 const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id);
 const char *btf_name_by_offset(const struct btf *btf, u32 offset);
 struct btf *btf_parse_vmlinux(void);
kernel/bpf/btf.c +44 -21

180 lines not on this page · raw message

kernel/bpf/syscall.c +21 -3
index f3fe9f53f93c..5ee00611af53 100644
@@ -1726,6 +1726,8 @@ static void __bpf_prog_put_rcu(struct rcu_head *rcu)
 	bpf_prog_uncharge_memlock(aux->prog);
 	security_bpf_prog_free(aux);
 	bpf_prog_free(aux->prog);
+	if (aux->attach_btf)
+		btf_put(aux->attach_btf);
 }
 
 static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)
@@ -2155,6 +2157,20 @@ static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
 
 	prog->expected_attach_type = attr->expected_attach_type;
 	prog->aux->attach_btf_id = attr->attach_btf_id;
+
+	if (attr->attach_btf_id && !attr->attach_prog_fd) {
+		struct btf *btf;
+
+		btf = bpf_get_btf_vmlinux();
+		if (IS_ERR(btf))
+			return PTR_ERR(btf);
+		if (!btf)
+			return -EINVAL;
+
+		btf_get(btf);
+		prog->aux->attach_btf = btf;
+	}
+
 	if (attr->attach_prog_fd) {
 		struct bpf_prog *dst_prog;
 
@@ -2255,6 +2271,8 @@ static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
 free_prog_sec:
 	security_bpf_prog_free(prog->aux);
 free_prog_nouncharge:
+	if (prog->aux->attach_btf)
+		btf_put(prog->aux->attach_btf);
 	bpf_prog_free(prog);
 	return err;
 }
@@ -2612,7 +2630,7 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog,
 			goto out_put_prog;
 		}
 
-		key = bpf_trampoline_compute_key(tgt_prog, btf_id);
+		key = bpf_trampoline_compute_key(tgt_prog, NULL, btf_id);
 	}
 
 	link = kzalloc(sizeof(*link), GFP_USER);
@@ -3589,7 +3607,7 @@ static int bpf_prog_get_info_by_fd(struct file *file,
 	}
 
 	if (prog->aux->btf)
-		info.btf_id = btf_id(prog->aux->btf);
+		info.btf_id = btf_obj_id(prog->aux->btf);
 
 	ulen = info.nr_func_info;
 	info.nr_func_info = prog->aux->func_info_cnt;
@@ -3692,7 +3710,7 @@ static int bpf_map_get_info_by_fd(struct file *file,
 	memcpy(info.name, map->name, sizeof(map->name));
 
 	if (map->btf) {
-		info.btf_id = btf_id(map->btf);
+		info.btf_id = btf_obj_id(map->btf);
 		info.btf_key_type_id = map->btf_key_type_id;
 		info.btf_value_type_id = map->btf_value_type_id;
 	}
kernel/bpf/verifier.c +47 -30

256 lines not on this page · raw message

net/ipv4/bpf_tcp_ca.c +2 -1
index 618954f82764..d520e61649c8 100644
@@ -95,6 +95,7 @@ static bool bpf_tcp_ca_is_valid_access(int off, int size,
 }
 
 static int bpf_tcp_ca_btf_struct_access(struct bpf_verifier_log *log,
+					const struct btf *btf,
 					const struct btf_type *t, int off,
 					int size, enum bpf_access_type atype,
 					u32 *next_btf_id)
@@ -102,7 +103,7 @@ static int bpf_tcp_ca_btf_struct_access(struct bpf_verifier_log *log,
 	size_t end;
 
 	if (atype == BPF_READ)
-		return btf_struct_access(log, t, off, size, atype, next_btf_id);
+		return btf_struct_access(log, btf, t, off, size, atype, next_btf_id);
 
 	if (t != tcp_sock_type) {
 		bpf_log(log, "only read is supported\n");
Signature · 1 line
2.24.1
author maintainer
#3 lore raw

PATCH v2 3/8 bpf: allow to specify kernel module BTFs when attaching BPF programs

In reply to Andrii Nakryiko

Add ability for user-space programs to specify non-vmlinux BTF when attaching BTF-powered BPF programs: raw_tp, fentry/fexit/fmod_ret, LSM, etc. For this, add attach_btf_obj_id field which contains BTF object ID for either vmlinux or module. For backwards compatibility (and simplicity) reasons, 0 denotes vmlinux BTF. Only kernel BTF (vmlinux or module) can be specified.

---
 include/linux/btf.h            |  2 ++
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/btf.c               | 25 +++++++++++++++++++------
 kernel/bpf/syscall.c           | 22 +++++++++++++++++++---
 tools/include/uapi/linux/bpf.h |  1 +
 5 files changed, 42 insertions(+), 9 deletions(-)
include/linux/btf.h +2 -0
index fb608e4de076..53ce2bc6dc54 100644
@@ -89,7 +89,9 @@ int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj,
 			   char *buf, int len, u64 flags);
 
 int btf_get_fd_by_id(u32 id);
+struct btf *btf_get_by_id(int id);
 u32 btf_obj_id(const struct btf *btf);
+bool btf_is_kernel(const struct btf *btf);
 bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,
 			   const struct btf_member *m,
 			   u32 expected_offset, u32 expected_size);
include/uapi/linux/bpf.h +1 -0
index c3458ec1f30a..60b95b51ccb8 100644
@@ -558,6 +558,7 @@ union bpf_attr {
 		__u32		line_info_cnt;	/* number of bpf_line_info records */
 		__u32		attach_btf_id;	/* in-kernel BTF type id to attach to */
 		__u32		attach_prog_fd; /* 0 to attach to vmlinux */
+		__u32		attach_btf_obj_id; /* vmlinux/module BTF object ID for BTF type */
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
kernel/bpf/btf.c +19 -6
index 7a19bf5bfe97..12876b272c6b 100644
@@ -5652,6 +5652,19 @@ struct btf *btf_get_by_fd(int fd)
 	return btf;
 }
 
+struct btf *btf_get_by_id(int id)
+{
+	struct btf *btf;
+
+	rcu_read_lock();
+	btf = idr_find(&btf_idr, id);
+	if (!btf || !refcount_inc_not_zero(&btf->refcnt))
+		btf = ERR_PTR(-ENOENT);
+	rcu_read_unlock();
+
+	return btf;
+}
+
 int btf_get_info_by_fd(const struct btf *btf,
 		       const union bpf_attr *attr,
 		       union bpf_attr __user *uattr)
@@ -5717,12 +5730,7 @@ int btf_get_fd_by_id(u32 id)
 	struct btf *btf;
 	int fd;
 
-	rcu_read_lock();
-	btf = idr_find(&btf_idr, id);
-	if (!btf || !refcount_inc_not_zero(&btf->refcnt))
-		btf = ERR_PTR(-ENOENT);
-	rcu_read_unlock();
-
+	btf = btf_get_by_id(id);
 	if (IS_ERR(btf))
 		return PTR_ERR(btf);
 
@@ -5738,6 +5746,11 @@ u32 btf_obj_id(const struct btf *btf)
 	return btf->id;
 }
 
+bool btf_is_kernel(const struct btf *btf)
+{
+	return btf->kernel_btf;
+}
+
 static int btf_id_cmp_func(const void *a, const void *b)
 {
 	const int *pa = a, *pb = b;
kernel/bpf/syscall.c +19 -3
index 5ee00611af53..3af073642664 100644
@@ -1968,7 +1968,7 @@ static void bpf_prog_load_fixup_attach_type(union bpf_attr *attr)
 static int
 bpf_prog_load_check_attach(enum bpf_prog_type prog_type,
 			   enum bpf_attach_type expected_attach_type,
-			   u32 btf_id, u32 prog_fd)
+			   u32 btf_obj_id, u32 btf_id, u32 prog_fd)
 {
 	if (btf_id) {
 		if (btf_id > BTF_MAX_TYPE)
@@ -1985,6 +1985,9 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type,
 		}
 	}
 
+	if (btf_obj_id && (!btf_id || prog_fd))
+		return -EINVAL;
+
 	if (prog_fd && prog_type != BPF_PROG_TYPE_TRACING &&
 	    prog_type != BPF_PROG_TYPE_EXT)
 		return -EINVAL;
@@ -2097,7 +2100,7 @@ static bool is_perfmon_prog_type(enum bpf_prog_type prog_type)
 }
 
 /* last field in 'union bpf_attr' used by this command */
-#define	BPF_PROG_LOAD_LAST_FIELD attach_prog_fd
+#define	BPF_PROG_LOAD_LAST_FIELD attach_btf_obj_id
 
 static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
 {
@@ -2146,6 +2149,7 @@ static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
 
 	bpf_prog_load_fixup_attach_type(attr);
 	if (bpf_prog_load_check_attach(type, attr->expected_attach_type,
+				       attr->attach_btf_obj_id,
 				       attr->attach_btf_id,
 				       attr->attach_prog_fd))
 		return -EINVAL;
@@ -2158,7 +2162,19 @@ static int bpf_prog_load(union bpf_attr *attr, union bpf_attr __user *uattr)
 	prog->expected_attach_type = attr->expected_attach_type;
 	prog->aux->attach_btf_id = attr->attach_btf_id;
 
-	if (attr->attach_btf_id && !attr->attach_prog_fd) {
+	if (attr->attach_btf_obj_id) {
+		struct btf *btf;
+
+		btf = btf_get_by_id(attr->attach_btf_obj_id);
+		if (IS_ERR(btf))
+			return PTR_ERR(btf);
+		if (!btf_is_kernel(btf)) {
+			btf_put(btf);
+			return -EINVAL;
+		}
+
+		prog->aux->attach_btf = btf;
+	} else if (attr->attach_btf_id && !attr->attach_prog_fd) {
 		struct btf *btf;
 
 		btf = bpf_get_btf_vmlinux();
tools/include/uapi/linux/bpf.h +1 -0
index c3458ec1f30a..60b95b51ccb8 100644
@@ -558,6 +558,7 @@ union bpf_attr {
 		__u32		line_info_cnt;	/* number of bpf_line_info records */
 		__u32		attach_btf_id;	/* in-kernel BTF type id to attach to */
 		__u32		attach_prog_fd; /* 0 to attach to vmlinux */
+		__u32		attach_btf_obj_id; /* vmlinux/module BTF object ID for BTF type */
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
Signature · 1 line
2.24.1
author maintainer
#4 lore raw

PATCH v2 4/8 libbpf: factor out low-level BPF program loading helper

In reply to Andrii Nakryiko

Refactor low-level API for BPF program loading to not rely on public API types. This allows painless extension without constant efforts to cleverly not break backwards compatibility.

---
 tools/lib/bpf/bpf.c             | 100 ++++++++++++++++++++++----------
 tools/lib/bpf/libbpf.c          |  34 +++++------
 tools/lib/bpf/libbpf_internal.h |  29 +++++++++
 3 files changed, 113 insertions(+), 50 deletions(-)
tools/lib/bpf/bpf.c +68 -32
index d27e34133973..e371ef91f5e8 100644
@@ -214,59 +214,52 @@ alloc_zero_tailing_info(const void *orecord, __u32 cnt,
 	return info;
 }
 
-int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
-			   char *log_buf, size_t log_buf_sz)
+int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
 {
 	void *finfo = NULL, *linfo = NULL;
 	union bpf_attr attr;
-	__u32 log_level;
 	int fd;
 
-	if (!load_attr || !log_buf != !log_buf_sz)
+	if (!load_attr->log_buf != !load_attr->log_buf_sz)
 		return -EINVAL;
 
-	log_level = load_attr->log_level;
-	if (log_level > (4 | 2 | 1) || (log_level && !log_buf))
+	if (load_attr->log_level > (4 | 2 | 1) || (load_attr->log_level && !load_attr->log_buf))
 		return -EINVAL;
 
 	memset(&attr, 0, sizeof(attr));
 	attr.prog_type = load_attr->prog_type;
 	attr.expected_attach_type = load_attr->expected_attach_type;
-	if (attr.prog_type == BPF_PROG_TYPE_STRUCT_OPS ||
-	    attr.prog_type == BPF_PROG_TYPE_LSM) {
-		attr.attach_btf_id = load_attr->attach_btf_id;
-	} else if (attr.prog_type == BPF_PROG_TYPE_TRACING ||
-		   attr.prog_type == BPF_PROG_TYPE_EXT) {
-		attr.attach_btf_id = load_attr->attach_btf_id;
-		attr.attach_prog_fd = load_attr->attach_prog_fd;
-	} else {
-		attr.prog_ifindex = load_attr->prog_ifindex;
-		attr.kern_version = load_attr->kern_version;
-	}
-	attr.insn_cnt = (__u32)load_attr->insns_cnt;
+
+	attr.attach_btf_id = load_attr->attach_btf_id;
+	attr.attach_prog_fd = load_attr->attach_prog_fd;
+
+	attr.prog_ifindex = load_attr->prog_ifindex;
+	attr.kern_version = load_attr->kern_version;
+
+	attr.insn_cnt = (__u32)load_attr->insn_cnt;
 	attr.insns = ptr_to_u64(load_attr->insns);
 	attr.license = ptr_to_u64(load_attr->license);
 
-	attr.log_level = log_level;
-	if (log_level) {
-		attr.log_buf = ptr_to_u64(log_buf);
-		attr.log_size = log_buf_sz;
-	} else {
-		attr.log_buf = ptr_to_u64(NULL);
-		attr.log_size = 0;
+	attr.log_level = load_attr->log_level;
+	if (attr.log_level) {
+		attr.log_buf = ptr_to_u64(load_attr->log_buf);
+		attr.log_size = load_attr->log_buf_sz;
 	}
 
 	attr.prog_btf_fd = load_attr->prog_btf_fd;
+	attr.prog_flags = load_attr->prog_flags;
+
 	attr.func_info_rec_size = load_attr->func_info_rec_size;
 	attr.func_info_cnt = load_attr->func_info_cnt;
 	attr.func_info = ptr_to_u64(load_attr->func_info);
+
 	attr.line_info_rec_size = load_attr->line_info_rec_size;
 	attr.line_info_cnt = load_attr->line_info_cnt;
 	attr.line_info = ptr_to_u64(load_attr->line_info);
+
 	if (load_attr->name)
 		memcpy(attr.prog_name, load_attr->name,
-		       min(strlen(load_attr->name), BPF_OBJ_NAME_LEN - 1));
-	attr.prog_flags = load_attr->prog_flags;
+		       min(strlen(load_attr->name), (size_t)BPF_OBJ_NAME_LEN - 1));
 
 	fd = sys_bpf_prog_load(&attr, sizeof(attr));
 	if (fd >= 0)
@@ -306,19 +299,19 @@ int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
 		}
 
 		fd = sys_bpf_prog_load(&attr, sizeof(attr));
-
 		if (fd >= 0)
 			goto done;
 	}
 
-	if (log_level || !log_buf)
+	if (load_attr->log_level || !load_attr->log_buf)
 		goto done;
 
 	/* Try again with log */
-	attr.log_buf = ptr_to_u64(log_buf);
-	attr.log_size = log_buf_sz;
+	attr.log_buf = ptr_to_u64(load_attr->log_buf);
+	attr.log_size = load_attr->log_buf_sz;
 	attr.log_level = 1;
-	log_buf[0] = 0;
+	load_attr->log_buf[0] = 0;
+
 	fd = sys_bpf_prog_load(&attr, sizeof(attr));
 done:
 	free(finfo);
@@ -326,6 +319,49 @@ int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
 	return fd;
 }
 
+int bpf_load_program_xattr(const struct bpf_load_program_attr *load_attr,
+			   char *log_buf, size_t log_buf_sz)
+{
+	struct bpf_prog_load_params p = {};
+
+	if (!load_attr || !log_buf != !log_buf_sz)
+		return -EINVAL;
+
+	p.prog_type = load_attr->prog_type;
+	p.expected_attach_type = load_attr->expected_attach_type;
+	switch (p.prog_type) {
+	case BPF_PROG_TYPE_STRUCT_OPS:
+	case BPF_PROG_TYPE_LSM:
+		p.attach_btf_id = load_attr->attach_btf_id;
+		break;
+	case BPF_PROG_TYPE_TRACING:
+	case BPF_PROG_TYPE_EXT:
+		p.attach_btf_id = load_attr->attach_btf_id;
+		p.attach_prog_fd = load_attr->attach_prog_fd;
+		break;
+	default:
+		p.prog_ifindex = load_attr->prog_ifindex;
+		p.kern_version = load_attr->kern_version;
+	}
+	p.insn_cnt = load_attr->insns_cnt;
+	p.insns = load_attr->insns;
+	p.license = load_attr->license;
+	p.log_level = load_attr->log_level;
+	p.log_buf = log_buf;
+	p.log_buf_sz = log_buf_sz;
+	p.prog_btf_fd = load_attr->prog_btf_fd;
+	p.func_info_rec_size = load_attr->func_info_rec_size;
+	p.func_info_cnt = load_attr->func_info_cnt;
+	p.func_info = load_attr->func_info;
+	p.line_info_rec_size = load_attr->line_info_rec_size;
+	p.line_info_cnt = load_attr->line_info_cnt;
+	p.line_info = load_attr->line_info;
+	p.name = load_attr->name;
+	p.prog_flags = load_attr->prog_flags;
+
+	return libbpf__bpf_prog_load(&p);
+}
+
 int bpf_load_program(enum bpf_prog_type type, const struct bpf_insn *insns,
 		     size_t insns_cnt, const char *license,
 		     __u32 kern_version, char *log_buf,
tools/lib/bpf/libbpf.c +16 -18
index 375698999375..0280df77cf5f 100644
@@ -6808,7 +6808,7 @@ static int
 load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
 	     char *license, __u32 kern_version, int *pfd)
 {
-	struct bpf_load_program_attr load_attr;
+	struct bpf_prog_load_params load_attr = {};
 	char *cp, errmsg[STRERR_BUFSIZE];
 	size_t log_buf_size = 0;
 	char *log_buf = NULL;
@@ -6817,7 +6817,6 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
 	if (!insns || !insns_cnt)
 		return -EINVAL;
 
-	memset(&load_attr, 0, sizeof(struct bpf_load_program_attr));
 	load_attr.prog_type = prog->type;
 	/* old kernels might not support specifying expected_attach_type */
 	if (!kernel_supports(FEAT_EXP_ATTACH_TYPE) && prog->sec_def &&
@@ -6828,19 +6827,14 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
 	if (kernel_supports(FEAT_PROG_NAME))
 		load_attr.name = prog->name;
 	load_attr.insns = insns;
-	load_attr.insns_cnt = insns_cnt;
+	load_attr.insn_cnt = insns_cnt;
 	load_attr.license = license;
-	if (prog->type == BPF_PROG_TYPE_STRUCT_OPS ||
-	    prog->type == BPF_PROG_TYPE_LSM) {
-		load_attr.attach_btf_id = prog->attach_btf_id;
-	} else if (prog->type == BPF_PROG_TYPE_TRACING ||
-		   prog->type == BPF_PROG_TYPE_EXT) {
-		load_attr.attach_prog_fd = prog->attach_prog_fd;
-		load_attr.attach_btf_id = prog->attach_btf_id;
-	} else {
-		load_attr.kern_version = kern_version;
-		load_attr.prog_ifindex = prog->prog_ifindex;
-	}
+	load_attr.attach_btf_id = prog->attach_btf_id;
+	load_attr.attach_prog_fd = prog->attach_prog_fd;
+	load_attr.attach_btf_id = prog->attach_btf_id;
+	load_attr.kern_version = kern_version;
+	load_attr.prog_ifindex = prog->prog_ifindex;
+
 	/* specify func_info/line_info only if kernel supports them */
 	btf_fd = bpf_object__btf_fd(prog->obj);
 	if (btf_fd >= 0 && kernel_supports(FEAT_BTF_FUNC)) {
@@ -6864,7 +6858,9 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
 		*log_buf = 0;
 	}
 
-	ret = bpf_load_program_xattr(&load_attr, log_buf, log_buf_size);
+	load_attr.log_buf = log_buf;
+	load_attr.log_buf_sz = log_buf_size;
+	ret = libbpf__bpf_prog_load(&load_attr);
 
 	if (ret >= 0) {
 		if (log_buf && load_attr.log_level)
@@ -6905,9 +6901,9 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
 		pr_warn("-- BEGIN DUMP LOG ---\n");
 		pr_warn("\n%s\n", log_buf);
 		pr_warn("-- END LOG --\n");
-	} else if (load_attr.insns_cnt >= BPF_MAXINSNS) {
+	} else if (load_attr.insn_cnt >= BPF_MAXINSNS) {
 		pr_warn("Program too large (%zu insns), at most %d insns\n",
-			load_attr.insns_cnt, BPF_MAXINSNS);
+			load_attr.insn_cnt, BPF_MAXINSNS);
 		ret = -LIBBPF_ERRNO__PROG2BIG;
 	} else if (load_attr.prog_type != BPF_PROG_TYPE_KPROBE) {
 		/* Wrong program type? */
@@ -6915,7 +6911,9 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt,
 
 		load_attr.prog_type = BPF_PROG_TYPE_KPROBE;
 		load_attr.expected_attach_type = 0;
-		fd = bpf_load_program_xattr(&load_attr, NULL, 0);
+		load_attr.log_buf = NULL;
+		load_attr.log_buf_sz = 0;
+		fd = libbpf__bpf_prog_load(&load_attr);
 		if (fd >= 0) {
 			close(fd);
 			ret = -LIBBPF_ERRNO__PROGTYPE;
tools/lib/bpf/libbpf_internal.h +29 -0
index e569ae63808e..681073a67ae3 100644
@@ -151,6 +151,35 @@ int parse_cpu_mask_file(const char *fcpu, bool **mask, int *mask_sz);
 int libbpf__load_raw_btf(const char *raw_types, size_t types_len,
 			 const char *str_sec, size_t str_len);
 
+struct bpf_prog_load_params {
+	enum bpf_prog_type prog_type;
+	enum bpf_attach_type expected_attach_type;
+	const char *name;
+	const struct bpf_insn *insns;
+	size_t insn_cnt;
+	const char *license;
+	__u32 kern_version;
+	__u32 attach_prog_fd;
+	__u32 attach_btf_id;
+	__u32 prog_ifindex;
+	__u32 prog_btf_fd;
+	__u32 prog_flags;
+
+	__u32 func_info_rec_size;
+	const void *func_info;
+	__u32 func_info_cnt;
+
+	__u32 line_info_rec_size;
+	const void *line_info;
+	__u32 line_info_cnt;
+
+	__u32 log_level;
+	char *log_buf;
+	size_t log_buf_sz;
+};
+
+int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr);
+
 int bpf_object__section_size(const struct bpf_object *obj, const char *name,
 			     __u32 *size);
 int bpf_object__variable_offset(const struct bpf_object *obj, const char *name,
Signature · 1 line
2.24.1
author maintainer
#5 lore raw

PATCH v2 5/8 libbpf: support attachment of BPF tracing programs to kernel modules

In reply to Andrii Nakryiko

Teach libbpf to search for BTF types in kernel modules for tracing BPF programs. This allows attachment of raw_tp/fentry/fexit/fmod_ret/etc BPF program types to tracepoints and functions in kernel modules.

---
 tools/lib/bpf/bpf.c             |   1 +
 tools/lib/bpf/libbpf.c          | 111 +++++++++++++++++++++++++-------
 tools/lib/bpf/libbpf_internal.h |   1 +
 3 files changed, 90 insertions(+), 23 deletions(-)
tools/lib/bpf/bpf.c +1 -0
index e371ef91f5e8..dfdad932ef72 100644
@@ -230,6 +230,7 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr)
 	attr.prog_type = load_attr->prog_type;
 	attr.expected_attach_type = load_attr->expected_attach_type;
 
+	attr.attach_btf_obj_id = load_attr->attach_btf_obj_id;
 	attr.attach_btf_id = load_attr->attach_btf_id;
 	attr.attach_prog_fd = load_attr->attach_prog_fd;
 
tools/lib/bpf/libbpf.c +88 -23

179 lines not on this page · raw message

tools/lib/bpf/libbpf_internal.h +1 -0
index 681073a67ae3..36236b4430ff 100644
@@ -160,6 +160,7 @@ struct bpf_prog_load_params {
 	const char *license;
 	__u32 kern_version;
 	__u32 attach_prog_fd;
+	__u32 attach_btf_obj_id;
 	__u32 attach_btf_id;
 	__u32 prog_ifindex;
 	__u32 prog_btf_fd;
Signature · 1 line
2.24.1
author maintainer
#6 lore raw

PATCH v2 6/8 selftests/bpf: add tp_btf CO-RE reloc test for modules

In reply to Andrii Nakryiko

Add another CO-RE relocation test for kernel module relocations. This time for tp_btf with direct memory reads.

---
 .../selftests/bpf/prog_tests/core_reloc.c     |  3 +-
 .../bpf/progs/test_core_reloc_module.c        | 32 ++++++++++++++++++-
 2 files changed, 33 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/prog_tests/core_reloc.c +2 -1
index bb980848cd77..06eb956ff7bb 100644
@@ -514,7 +514,8 @@ static struct core_reloc_test_case test_cases[] = {
 	},
 
 	/* validate we can find kernel module BTF types for relocs/attach */
-	MODULES_CASE("module", "raw_tp/bpf_testmod_test_read", "bpf_testmod_test_read"),
+	MODULES_CASE("module_probed", "raw_tp/bpf_testmod_test_read", "bpf_testmod_test_read"),
+	MODULES_CASE("module_direct", "tp_btf/bpf_testmod_test_read", NULL),
 
 	/* validate BPF program can use multiple flavors to match against
 	 * single target BTF type
tools/testing/selftests/bpf/progs/test_core_reloc_module.c +31 -1
index d1840c1a9d36..56363959f7b0 100644
@@ -36,7 +36,7 @@ struct core_reloc_module_output {
 };
 
 SEC("raw_tp/bpf_testmod_test_read")
-int BPF_PROG(test_core_module,
+int BPF_PROG(test_core_module_probed,
 	     struct task_struct *task,
 	     struct bpf_testmod_test_read_ctx *read_ctx)
 {
@@ -64,3 +64,33 @@ int BPF_PROG(test_core_module,
 
 	return 0;
 }
+
+SEC("tp_btf/bpf_testmod_test_read")
+int BPF_PROG(test_core_module_direct,
+	     struct task_struct *task,
+	     struct bpf_testmod_test_read_ctx *read_ctx)
+{
+	struct core_reloc_module_output *out = (void *)&data.out;
+	__u64 pid_tgid = bpf_get_current_pid_tgid();
+	__u32 real_tgid = (__u32)(pid_tgid >> 32);
+	__u32 real_pid = (__u32)pid_tgid;
+
+	if (data.my_pid_tgid != pid_tgid)
+		return 0;
+
+	if (task->pid != real_pid || task->tgid != real_tgid)
+		return 0;
+
+	out->len = read_ctx->len;
+	out->off = read_ctx->off;
+
+	out->read_ctx_sz = bpf_core_type_size(struct bpf_testmod_test_read_ctx);
+	out->read_ctx_exists = bpf_core_type_exists(struct bpf_testmod_test_read_ctx);
+	out->buf_exists = bpf_core_field_exists(read_ctx->buf);
+	out->off_exists = bpf_core_field_exists(read_ctx->off);
+	out->len_exists = bpf_core_field_exists(read_ctx->len);
+
+	out->comm_len = BPF_CORE_READ_STR_INTO(&out->comm, task, comm);
+
+	return 0;
+}
Signature · 1 line
2.24.1
author maintainer
#7 lore raw

PATCH v2 7/8 selftests/bpf: make bpf_testmod's traceable function global

In reply to Andrii Nakryiko

Pahole currently isn't able to generate BTF for static functions in kernel modules, so make sure traced sidecar function is global.

---
 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +2 -1
index 4ff0cf23607d..2df19d73ca49 100644
@@ -10,7 +10,7 @@
 #define CREATE_TRACE_POINTS
 #include "bpf_testmod-events.h"
 
-static noinline ssize_t
+noinline ssize_t
 bpf_testmod_test_read(struct file *file, struct kobject *kobj,
 		      struct bin_attribute *bin_attr,
 		      char *buf, loff_t off, size_t len)
@@ -25,6 +25,7 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj,
 
 	return -EIO; /* always fail */
 }
+EXPORT_SYMBOL(bpf_testmod_test_read);
 ALLOW_ERROR_INJECTION(bpf_testmod_test_read, ERRNO);
 
 static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = {
Signature · 1 line
2.24.1
author maintainer
#8 lore raw

PATCH v2 8/8 selftests/bpf: add fentry/fexit/fmod_ret selftest for kernel module

In reply to Andrii Nakryiko

Add new selftest checking attachment of fentry/fexit/fmod_ret (and raw tracepoint ones for completeness) BPF programs to kernel module function.

---
 .../selftests/bpf/prog_tests/module_attach.c  | 53 +++++++++++++++
 .../selftests/bpf/progs/test_module_attach.c  | 66 +++++++++++++++++++
 2 files changed, 119 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/module_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_module_attach.c
tools/testing/selftests/bpf/prog_tests/module_attach.c new file +53 -0
new file mode 100644
index 000000000000..4b65e9918764
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2020 Facebook */
+
+#include <test_progs.h>
+#include "test_module_attach.skel.h"
+
+static int duration;
+
+static int trigger_module_test_read(int read_sz)
+{
+	int fd, err;
+
+	fd = open("/sys/kernel/bpf_testmod", O_RDONLY);
+	err = -errno;
+	if (CHECK(fd < 0, "testmod_file_open", "failed: %d\n", err))
+		return err;
+
+	read(fd, NULL, read_sz);
+	close(fd);
+
+	return 0;
+}
+
+void test_module_attach(void)
+{
+	const int READ_SZ = 456;
+	struct test_module_attach* skel;
+	struct test_module_attach__bss *bss;
+	int err;
+
+	skel = test_module_attach__open_and_load();
+	if (CHECK(!skel, "skel_open", "failed to open skeleton\n"))
+		return;
+
+	bss = skel->bss;
+
+	err = test_module_attach__attach(skel);
+	if (CHECK(err, "skel_attach", "skeleton attach failed: %d\n", err))
+		goto cleanup;
+
+	/* trigger tracepoint */
+	ASSERT_OK(trigger_module_test_read(READ_SZ), "trigger_read");
+
+	ASSERT_EQ(bss->raw_tp_read_sz, READ_SZ, "raw_tp");
+	ASSERT_EQ(bss->tp_btf_read_sz, READ_SZ, "tp_btf");
+	ASSERT_EQ(bss->fentry_read_sz, READ_SZ, "fentry");
+	ASSERT_EQ(bss->fexit_read_sz, READ_SZ, "fexit");
+	ASSERT_EQ(bss->fexit_ret, -EIO, "fexit_tet");
+	ASSERT_EQ(bss->fmod_ret_read_sz, READ_SZ, "fmod_ret");
+
+cleanup:
+	test_module_attach__destroy(skel);
+}
tools/testing/selftests/bpf/progs/test_module_attach.c new file +66 -0

66 lines not on this page · raw message

Signature · 1 line
2.24.1
#9 lore raw
In reply to Andrii Nakryiko
Quoted 9 lines · Andrii Nakryiko

Building on top of two previous patch sets ([0] and not yet landed [1]), this patch sets extends kernel and libbpf with support for attaching BTF-powered raw tracepoint (tp_btf) and tracing (fentry/fexit/fmod_ret/lsm) BPF programs to BPF hooks defined in kernel modules.

Kernel UAPI for BPF_PROG_LOAD is extended with extra parameter (attach_btf_obj_id) which allows to specify kernel module BTF in which the BTF type is identifed by attach_btf_id.

Quoted 8 lines

From end user perspective there are no extra actions that need to happen.
Libbpf will continue searching across all kernel module BTFs, if desired attach BTF type is not found in vmlinux. That way it doesn't matter if BPF hook that user is trying to attach to is built-in into vmlinux image or is loaded in kernel module.

Currently pahole doesn't generate BTF_KIND_FUNC info for ftrace-able static functions in kernel modules, so expose traced function in bpf_sidecar.ko. Once

bpf_sidecar.ko => bpf_testmod.ko

Quoted 42 lines

pahole is enhanced, we can go back to static function.

   [0] https://patchwork.kernel.org/project/netdevbpf/list/?series=380759&state=*
   [1] https://patchwork.kernel.org/project/netdevbpf/list/?series=393677&state=*

v1->v2:
- avoid increasing bpf_reg_state by reordering fields (Alexei);
- preserve btf_data_size in struct module;
- rebase on top of v3 of patch [1].

Andrii Nakryiko (8):
bpf: keep module's btf_data_size intact after load
bpf: remove hard-coded btf_vmlinux assumption from BPF verifier
bpf: allow to specify kernel module BTFs when attaching BPF programs
libbpf: factor out low-level BPF program loading helper
libbpf: support attachment of BPF tracing programs to kernel modules selftests/bpf: add tp_btf CO-RE reloc test for modules selftests/bpf: make bpf_testmod's traceable function global selftests/bpf: add fentry/fexit/fmod_ret selftest for kernel module

  include/linux/bpf.h                           |  13 +-
  include/linux/bpf_verifier.h                  |  28 +++-
  include/linux/btf.h                           |   7 +-
  include/uapi/linux/bpf.h                      |   1 +
  kernel/bpf/btf.c                              |  90 +++++++----
  kernel/bpf/syscall.c                          |  44 +++++-
  kernel/bpf/verifier.c                         |  77 ++++++----
  kernel/module.c                               |   1 -
  net/ipv4/bpf_tcp_ca.c                         |   3 +-
  tools/include/uapi/linux/bpf.h                |   1 +
  tools/lib/bpf/bpf.c                           | 101 ++++++++----
  tools/lib/bpf/libbpf.c                        | 145 +++++++++++++-----
  tools/lib/bpf/libbpf_internal.h               |  30 ++++
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |   3 +-
  .../selftests/bpf/prog_tests/core_reloc.c     |   3 +-
  .../selftests/bpf/prog_tests/module_attach.c  |  53 +++++++
  .../bpf/progs/test_core_reloc_module.c        |  32 +++-
  .../selftests/bpf/progs/test_module_attach.c  |  66 ++++++++
  18 files changed, 546 insertions(+), 152 deletions(-)

create mode 100644 tools/testing/selftests/bpf/prog_tests/module_attach.c create mode 100644 tools/testing/selftests/bpf/progs/test_module_attach.c

#10 lore raw
Quoted 19 lines · Yonghong Song
Quoted 15 lines · Andrii Nakryiko

Building on top of two previous patch sets ([0] and not yet landed [1]), this patch sets extends kernel and libbpf with support for attaching BTF-powered raw tracepoint (tp_btf) and tracing (fentry/fexit/fmod_ret/lsm) BPF programs to BPF hooks defined in kernel modules.

Kernel UAPI for BPF_PROG_LOAD is extended with extra parameter (attach_btf_obj_id) which allows to specify kernel module BTF in which the BTF type is identifed by attach_btf_id.

From end user perspective there are no extra actions that need to happen.
Libbpf will continue searching across all kernel module BTFs, if desired attach BTF type is not found in vmlinux. That way it doesn't matter if BPF hook that user is trying to attach to is built-in into vmlinux image or is loaded in kernel module.

Quoted 5 lines
Quoted 2 lines

Currently pahole doesn't generate BTF_KIND_FUNC info for ftrace-able static functions in kernel modules, so expose traced function in bpf_sidecar.ko. Once

bpf_sidecar.ko => bpf_testmod.ko

will fix, thanks!

Quoted 42 lines
Quoted 42 lines

pahole is enhanced, we can go back to static function.

   [0] https://patchwork.kernel.org/project/netdevbpf/list/?series=380759&state=*
   [1] https://patchwork.kernel.org/project/netdevbpf/list/?series=393677&state=*

v1->v2:
- avoid increasing bpf_reg_state by reordering fields (Alexei);
- preserve btf_data_size in struct module;
- rebase on top of v3 of patch [1].

Andrii Nakryiko (8):
bpf: keep module's btf_data_size intact after load
bpf: remove hard-coded btf_vmlinux assumption from BPF verifier
bpf: allow to specify kernel module BTFs when attaching BPF programs
libbpf: factor out low-level BPF program loading helper
libbpf: support attachment of BPF tracing programs to kernel modules selftests/bpf: add tp_btf CO-RE reloc test for modules selftests/bpf: make bpf_testmod's traceable function global selftests/bpf: add fentry/fexit/fmod_ret selftest for kernel module

  include/linux/bpf.h                           |  13 +-
  include/linux/bpf_verifier.h                  |  28 +++-
  include/linux/btf.h                           |   7 +-
  include/uapi/linux/bpf.h                      |   1 +
  kernel/bpf/btf.c                              |  90 +++++++----
  kernel/bpf/syscall.c                          |  44 +++++-
  kernel/bpf/verifier.c                         |  77 ++++++----
  kernel/module.c                               |   1 -
  net/ipv4/bpf_tcp_ca.c                         |   3 +-
  tools/include/uapi/linux/bpf.h                |   1 +
  tools/lib/bpf/bpf.c                           | 101 ++++++++----
  tools/lib/bpf/libbpf.c                        | 145 +++++++++++++-----
  tools/lib/bpf/libbpf_internal.h               |  30 ++++
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |   3 +-
  .../selftests/bpf/prog_tests/core_reloc.c     |   3 +-
  .../selftests/bpf/prog_tests/module_attach.c  |  53 +++++++
  .../bpf/progs/test_core_reloc_module.c        |  32 +++-
  .../selftests/bpf/progs/test_module_attach.c  |  66 ++++++++
  18 files changed, 546 insertions(+), 152 deletions(-)

create mode 100644 tools/testing/selftests/bpf/prog_tests/module_attach.c create mode 100644 tools/testing/selftests/bpf/progs/test_module_attach.c