Registrating struct_ops types from modules

PATCH Superseded [email protected] 2 participants 16 messages Dec 7, 2023 – Dec 9, 2023

Subsystems (8)

Networking Network drivers Kselftest BPF +4 more subsystems

lore

Share & tools

lore

[email protected]

author

2023-12-07 01:40 UTC #0 lore raw

v12 cover Registrating struct_ops types from modules

From: Kui-Feng Lee <[email protected]>

Given the current constraints of the current implementation, struct_ops cannot be registered dynamically. This presents a significant limitation for modules like coming fuse-bpf, which seeks to implement a new struct_ops type. To address this issue, a new API is introduced that allows the registration of new struct_ops types from modules.

Previously, struct_ops types were defined in bpf_struct_ops_types.h and collected as a static array. The new API lets callers add new struct_ops types dynamically. The static array has been removed and replaced by the per-btf struct_ops_tab.

The struct_ops subsystem relies on BTF to determine the layout of values in a struct_ops map and identify the subsystem that the struct_ops map registers to. However, the kernel BTF does not include the type information of struct_ops types defined by a module. The struct_ops subsystem requires knowledge of the corresponding module for a given struct_ops map and the utilization of BTF information from that module. We empower libbpf to determine the correct module for accessing the BTF information and pass an identity (FD) of the module btf to the kernel. The kernel looks up type information and registered struct_ops types directly from the given btf.

If a module exits while one or more struct_ops maps still refer to a struct_ops type defined by the module, it can lead to unforeseen complications. Therefore, it is crucial to ensure that a module remains intact as long as any struct_ops map is still linked to a struct_ops type defined by the module. To achieve this, every struct_ops map holds a reference to the module while being registered.

Changes from v11:

- bpf_struct_ops_maps hold only the refcnt to the module, but not btf. (patch 1)

- Fix warning messages. (patch 1, 9 and 10)

- Remove unnecessary conditional compiling of CONFIG_BPF_JIT. (patch 4, 9 and 10)

- Fix the commit log of the patch 7 to explain how a btf is pass from the user space and how the kernel handle it.

- bpf_struct_ops_maps hold the module defining it's type, but not btf. A map will hold the module through its life-span from allocating to being free. (patch 8)

- Change selftests and tracing __bpf_struct_ops_map_free() to wait for the release of the bpf_testmod module.

- Include btf_obj_id in bpf_map_info. (patch 14)

Changes from v10:

- Guard btf.c from CONFIG_BPF_JIT=n. This patchset has introduced symbols from bpf_struct_ops.c which is only built when CONFIG_BPF_JIT=y.

- Fix the warning of unused errout_free label by moving code that is leaked to patch 8 to patch 7.

Changes from v9:

- Remove the call_rcu_tasks_trace() changes from kern_sync_rcu().

- Trace btf_put() in the test case to ensure the release of kmod's btf, or the consequent tests may fail for using kmod's unloaded old btf instead the new one created after loading again. The kmod's btf may live for awhile after unloading the kmod, for a map being freed asynchronized is still holding the btf.

- Split "add struct_ops_tab to btf" into tow patches by adding "make struct_ops_map support btfs other than btf_vmlinux".

- Flip the order of "pass attached BTF to the bpf_struct_ops subsystem" and "hold module for bpf_struct_ops_map" to make it more reasonable.

- Fix the compile errors of a missing header file.

Changes from v8:

- Rename bpf_struct_ops_init_one() to bpf_struct_ops_desc_init().

- Move code that using BTF_ID_LIST to the newly added patch 2.

- Move code that lookup struct_ops types from a given module to the newly added patch 5.

- Store the pointers of btf at st_maps.

- Add test cases for the cases of modules being unload.

- Call bpf_struct_ops_init() in btf_add_struct_ops() to fix an inconsistent issue.

Changes from v7:

- Fix check_struct_ops_btf_id() to use attach btf if there is instead of btf_vmlinux.

Changes from v6:

- Change returned error code to -EINVAL for the case of bpf_try_get_module().

- Return an error code from bpf_struct_ops_init().

- Fix the dependency issue of testing_helpers.c and rcu_tasks_trace_gp.skel.h.

Changes from v5:

- As the 2nd patch, we introduce "bpf_struct_ops_desc". This change involves moving certain members of "bpf_struct_ops" to "bpf_struct_ops_desc", which becomes a part of "btf_struct_ops_tab". This ensures that these members remain accessible even when the owner module of a "bpf_struct_ops" is unloaded.

- Correct the order of arguments when calling
in the 3rd patch.

- Remove the owner argument from bpf_struct_ops_init_one(). Instead, callers should fill in st_ops->owner.

- Make sure to hold the owner module when calling bpf_struct_ops_find() and bpf_struct_ops_find_value() in the 6th patch.

- Merge the functions register_bpf_struct_ops_btf() and register_bpf_struct_ops() into a single function and relocate it to btf.c for better organization and clarity.

- Undo the name modifications made to find_kernel_btf_id() and find_ksym_btf_id() in the 8th patch.

Changes from v4:

- Fix the dependency between testing_helpers.o and rcu_tasks_trace_gp.skel.h.

Changes from v3:

- Fix according to the feedback for v3.

- Change of the order of arguments to make btf as the first argument.

- Use btf_try_get_module() instead of try_get_module() since the module pointed by st_ops->owner can gone while some one is still holding its btf.

- Move variables defined by BPF_STRUCT_OPS_COMMON_VALUE to struct bpf_struct_ops_common_value to validation easier.

- Register the struct_ops type defined by bpf_testmod in its init function.

- Rename field name to 'value_type_btf_obj_fd' to make it explicit.

- Fix leaking of btf objects on error.

- st_maps hold their modules to keep modules alive and prevent they from unloading.

- bpf_map of libbpf keeps mod_btf_fd instead of a pointer to module_btf.

- Do call_rcu_tasks_trace() in kern_sync_rcu() to ensure the bpf_testmod is unloaded properly. It uses rcu_tasks_trace_gp to trigger call_rcu_tasks_trace() in the kernel.

- Merge and reorder patches in a reasonable order.

Changes from v2:

- Remove struct_ops array, and add a per-btf (module) struct_ops_tab to collect registered struct_ops types.

- Validate value_type by checking member names and types.

Kui-Feng Lee (14):
bpf: refactory struct_ops type initialization to a function.
bpf: get type information with BPF_ID_LIST
bpf, net: introduce bpf_struct_ops_desc.
bpf: add struct_ops_tab to btf.
bpf: make struct_ops_map support btfs other than btf_vmlinux.
bpf: lookup struct_ops types from a given module BTF.
bpf: pass attached BTF to the bpf_struct_ops subsystem
bpf: hold module for bpf_struct_ops_map.
bpf: validate value_type
bpf, net: switch to dynamic registration
libbpf: Find correct module BTFs for struct_ops maps and progs.
bpf: export btf_ctx_access to modules.
selftests/bpf: test case for register_bpf_struct_ops().
bpf: pass btf object id in bpf_map_info.

 include/linux/bpf.h                           |  55 ++-
 include/linux/bpf_verifier.h                  |   1 +
 include/linux/btf.h                           |   8 +
 include/uapi/linux/bpf.h                      |   7 +-
 kernel/bpf/bpf_struct_ops.c                   | 421 ++++++++++--------
 kernel/bpf/bpf_struct_ops_types.h             |  12 -
 kernel/bpf/btf.c                              | 124 +++++-
 kernel/bpf/syscall.c                          |   4 +-
 kernel/bpf/verifier.c                         |  25 +-
 net/bpf/bpf_dummy_struct_ops.c                |  22 +-
 net/ipv4/bpf_tcp_ca.c                         |  22 +-
 tools/include/uapi/linux/bpf.h                |   7 +-
 tools/lib/bpf/bpf.c                           |   4 +-
 tools/lib/bpf/bpf.h                           |   5 +-
 tools/lib/bpf/libbpf.c                        |  38 +-
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  52 +++
 .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   5 +
 .../bpf/prog_tests/test_struct_ops_module.c   |  92 ++++
 .../selftests/bpf/progs/struct_ops_module.c   |  30 ++
 .../selftests/bpf/progs/testmod_unload.c      |  25 ++
 20 files changed, 726 insertions(+), 233 deletions(-)
 delete mode 100644 kernel/bpf/bpf_struct_ops_types.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/testmod_unload.c

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #1 lore raw

PATCH v12 1/14 bpf: refactory struct_ops type initialization to a function.

From: Kui-Feng Lee <[email protected]>

Move the majority of the code to bpf_struct_ops_init_one(), which can then be utilized for the initialization of newly registered dynamically allocated struct_ops types in the following patches.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/btf.h         |   1 +
 kernel/bpf/bpf_struct_ops.c | 157 +++++++++++++++++++-----------------
 kernel/bpf/btf.c            |   5 ++
 3 files changed, 89 insertions(+), 74 deletions(-)

include/linux/btf.h +1 -0

index 59d404e22814..1d852dad7473 100644
@@ -137,6 +137,7 @@ struct btf_struct_metas {
 
 extern const struct file_operations btf_fops;
 
+const char *btf_get_name(const struct btf *btf);
 void btf_get(struct btf *btf);
 void btf_put(struct btf *btf);
 int btf_new_fd(const union bpf_attr *attr, bpfptr_t uattr, u32 uattr_sz);

kernel/bpf/bpf_struct_ops.c +83 ~~-74~~

185 lines not on this page · raw message

kernel/bpf/btf.c +5 -0

index 63cf4128fc05..6935a88ed190 100644
@@ -1707,6 +1707,11 @@ static void btf_free_rcu(struct rcu_head *rcu)
 	btf_free(btf);
 }
 
+const char *btf_get_name(const struct btf *btf)
+{
+	return btf->name;
+}
+
 void btf_get(struct btf *btf)
 {
 	refcount_inc(&btf->refcnt);

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #2 lore raw

PATCH v12 2/14 bpf: get type information with BPF_ID_LIST

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Get ready to remove bpf_struct_ops_init() in the future. By using BPF_ID_LIST, it is possible to gather type information while building instead of runtime.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/bpf_struct_ops.c | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

kernel/bpf/bpf_struct_ops.c +8 -9

index 813220c58cd3..409ab0327469 100644
@@ -108,7 +108,12 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = {
 #endif
 };
 
-static const struct btf_type *module_type;
+BTF_ID_LIST(st_ops_ids)
+BTF_ID(struct, module)
+
+enum {
+	IDX_MODULE_ID,
+};
 
 static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
 				    struct btf *btf,

@@ -197,7 +202,6 @@ static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
 void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
 {
 	struct bpf_struct_ops *st_ops;
-	s32 module_id;
 	u32 i;
 
 	/* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */

@@ -205,13 +209,6 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
 #include "bpf_struct_ops_types.h"
 #undef BPF_STRUCT_OPS_TYPE
 
-	module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT);
-	if (module_id < 0) {
-		pr_warn("Cannot find struct module in %s\n", btf_get_name(btf));
-		return;
-	}
-	module_type = btf_type_by_id(btf, module_id);
-
 	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {
 		st_ops = bpf_struct_ops[i];
 		bpf_struct_ops_init_one(st_ops, btf, log);

@@ -381,6 +378,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map;
 	const struct bpf_struct_ops *st_ops = st_map->st_ops;
 	struct bpf_struct_ops_value *uvalue, *kvalue;
+	const struct btf_type *module_type;
 	const struct btf_member *member;
 	const struct btf_type *t = st_ops->type;
 	struct bpf_tramp_links *tlinks;

@@ -428,6 +426,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	image = st_map->image;
 	image_end = st_map->image + PAGE_SIZE;
 
+	module_type = btf_type_by_id(btf_vmlinux, st_ops_ids[IDX_MODULE_ID]);
 	for_each_member(i, t, member) {
 		const struct btf_type *mtype, *ptype;
 		struct bpf_prog *prog;

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #3 lore raw

PATCH v12 3/14 bpf, net: introduce bpf_struct_ops_desc.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Move some of members of bpf_struct_ops to bpf_struct_ops_desc. When we introduce the new API to register new bpf_struct_ops types from modules, bpf_struct_ops may destroyed when the module is unloaded. Moving these members to bpf_struct_ops_desc make these data available even when the module is unloaded.

type_id is unavailabe in bpf_struct_ops anymore. Modules should get it from the btf received by kmod's init function.

Cc [email protected]

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h            | 13 ++++--
 kernel/bpf/bpf_struct_ops.c    | 80 +++++++++++++++++-----------------
 kernel/bpf/verifier.c          |  8 ++--
 net/bpf/bpf_dummy_struct_ops.c |  9 +++-
 net/ipv4/bpf_tcp_ca.c          |  8 +++-
 5 files changed, 70 insertions(+), 48 deletions(-)

include/linux/bpf.h +9 -4

index eb447b0a9423..73724e8caddf 100644
@@ -1634,17 +1634,22 @@ struct bpf_struct_ops {
 	void (*unreg)(void *kdata);
 	int (*update)(void *kdata, void *old_kdata);
 	int (*validate)(void *kdata);
-	const struct btf_type *type;
-	const struct btf_type *value_type;
 	const char *name;
 	struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS];
+};
+
+struct bpf_struct_ops_desc {
+	struct bpf_struct_ops *st_ops;
+
+	const struct btf_type *type;
+	const struct btf_type *value_type;
 	u32 type_id;
 	u32 value_id;
 };
 
 #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
 #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
-const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id);
+const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id);
 void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
 bool bpf_struct_ops_get(const void *kdata);
 void bpf_struct_ops_put(const void *kdata);

@@ -1687,7 +1692,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 			    union bpf_attr __user *uattr);
 #endif
 #else
-static inline const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id)
+static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id)
 {
 	return NULL;
 }

kernel/bpf/bpf_struct_ops.c +41 ~~-39~~

230 lines not on this page · raw message

kernel/bpf/verifier.c +5 -3

index cdb4f5f0ba79..5b9652a0d062 100644
@@ -20007,6 +20007,7 @@ static void print_verification_stats(struct bpf_verifier_env *env)
 static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 {
 	const struct btf_type *t, *func_proto;
+	const struct bpf_struct_ops_desc *st_ops_desc;
 	const struct bpf_struct_ops *st_ops;
 	const struct btf_member *member;
 	struct bpf_prog *prog = env->prog;

@@ -20019,14 +20020,15 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	btf_id = prog->aux->attach_btf_id;
-	st_ops = bpf_struct_ops_find(btf_id);
-	if (!st_ops) {
+	st_ops_desc = bpf_struct_ops_find(btf_id);
+	if (!st_ops_desc) {
 		verbose(env, "attach_btf_id %u is not a supported struct\n",
 			btf_id);
 		return -ENOTSUPP;
 	}
+	st_ops = st_ops_desc->st_ops;
 
-	t = st_ops->type;
+	t = st_ops_desc->type;
 	member_idx = prog->expected_attach_type;
 	if (member_idx >= btf_type_vlen(t)) {
 		verbose(env, "attach to invalid member idx %u of struct %s\n",

net/bpf/bpf_dummy_struct_ops.c +8 -1

index 5918d1b32e19..ffa224053a6c 100644
@@ -17,6 +17,8 @@ struct bpf_dummy_ops_test_args {
 	struct bpf_dummy_ops_state state;
 };
 
+static struct btf *bpf_dummy_ops_btf;
+
 static struct bpf_dummy_ops_test_args *
 dummy_ops_init_args(const union bpf_attr *kattr, unsigned int nr)
 {

@@ -85,9 +87,13 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 	void *image = NULL;
 	unsigned int op_idx;
 	int prog_ret;
+	u32 type_id;
 	int err;
 
-	if (prog->aux->attach_btf_id != st_ops->type_id)
+	type_id = btf_find_by_name_kind(bpf_dummy_ops_btf,
+					bpf_bpf_dummy_ops.name,
+					BTF_KIND_STRUCT);
+	if (prog->aux->attach_btf_id != type_id)
 		return -EOPNOTSUPP;
 
 	func_proto = prog->aux->attach_func_proto;

@@ -143,6 +149,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 
 static int bpf_dummy_init(struct btf *btf)
 {
+	bpf_dummy_ops_btf = btf;
 	return 0;
 }

net/ipv4/bpf_tcp_ca.c +7 -1

index 39dcccf0f174..3c8b76578a2a 100644
@@ -20,6 +20,7 @@ static u32 unsupported_ops[] = {
 
 static const struct btf_type *tcp_sock_type;
 static u32 tcp_sock_id, sock_id;
+static const struct btf_type *tcp_congestion_ops_type;
 
 static int bpf_tcp_ca_init(struct btf *btf)
 {

@@ -36,6 +37,11 @@ static int bpf_tcp_ca_init(struct btf *btf)
 	tcp_sock_id = type_id;
 	tcp_sock_type = btf_type_by_id(btf, tcp_sock_id);
 
+	type_id = btf_find_by_name_kind(btf, "tcp_congestion_ops", BTF_KIND_STRUCT);
+	if (type_id < 0)
+		return -EINVAL;
+	tcp_congestion_ops_type = btf_type_by_id(btf, type_id);
+
 	return 0;
 }

@@ -149,7 +155,7 @@ static u32 prog_ops_moff(const struct bpf_prog *prog)
 	u32 midx;
 
 	midx = prog->expected_attach_type;
-	t = bpf_tcp_congestion_ops.type;
+	t = tcp_congestion_ops_type;
 	m = &btf_type_member(t)[midx];
 
 	return __btf_member_bit_offset(t, m) / 8;

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #4 lore raw

PATCH v12 4/14 bpf: add struct_ops_tab to btf.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Maintain a registry of registered struct_ops types in the per-btf (module) struct_ops_tab. This registry allows for easy lookup of struct_ops types that are registered by a specific module.

It is a preparation work for supporting kernel module struct_ops in a latter patch. Each struct_ops will be registered under its own kernel module btf and will be stored in the newly added btf->struct_ops_tab. The bpf verifier and bpf syscall (e.g. prog and map cmd) can find the struct_ops and its btf type/size/id... information from btf->struct_ops_tab.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/btf.h |  5 ++++
 kernel/bpf/btf.c    | 73 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 78 insertions(+)

include/linux/btf.h +5 -0

index 1d852dad7473..e2f4b85cf82a 100644
@@ -584,4 +584,9 @@ static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type
 	return btf_type_is_struct(t);
 }
 
+struct bpf_struct_ops_desc;
+
+const struct bpf_struct_ops_desc *
+btf_get_struct_ops(struct btf *btf, u32 *ret_cnt);
+
 #endif

kernel/bpf/btf.c +73 -0

99 lines not on this page · raw message

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #5 lore raw

PATCH v12 5/14 bpf: make struct_ops_map support btfs other than btf_vmlinux.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Once new struct_ops can be registered from modules, btf_vmlinux is no longer the only btf that struct_ops_map would face. st_map should remember what btf it should use to get type information.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/bpf_struct_ops.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

kernel/bpf/bpf_struct_ops.c +15 -9

index 338b65f90692..9e5d77ea738a 100644
@@ -46,6 +46,8 @@ struct bpf_struct_ops_map {
 	 * "links[]".
 	 */
 	void *image;
+	/* The owner moduler's btf. */
+	struct btf *btf;
 	/* uvalue->data stores the kernel struct
 	 * (e.g. tcp_congestion_ops) that is more useful
 	 * to userspace than the kvalue.  For example,

@@ -314,7 +316,7 @@ static void bpf_struct_ops_map_put_progs(struct bpf_struct_ops_map *st_map)
 	}
 }
 
-static int check_zero_holes(const struct btf_type *t, void *data)
+static int check_zero_holes(const struct btf *btf, const struct btf_type *t, void *data)
 {
 	const struct btf_member *member;
 	u32 i, moff, msize, prev_mend = 0;

@@ -326,8 +328,8 @@ static int check_zero_holes(const struct btf_type *t, void *data)
 		    memchr_inv(data + prev_mend, 0, moff - prev_mend))
 			return -EINVAL;
 
-		mtype = btf_type_by_id(btf_vmlinux, member->type);
-		mtype = btf_resolve_size(btf_vmlinux, mtype, &msize);
+		mtype = btf_type_by_id(btf, member->type);
+		mtype = btf_resolve_size(btf, mtype, &msize);
 		if (IS_ERR(mtype))
 			return PTR_ERR(mtype);
 		prev_mend = moff + msize;

@@ -395,12 +397,12 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	if (*(u32 *)key != 0)
 		return -E2BIG;
 
-	err = check_zero_holes(st_ops_desc->value_type, value);
+	err = check_zero_holes(st_map->btf, st_ops_desc->value_type, value);
 	if (err)
 		return err;
 
 	uvalue = value;
-	err = check_zero_holes(t, uvalue->data);
+	err = check_zero_holes(st_map->btf, t, uvalue->data);
 	if (err)
 		return err;

@@ -436,7 +438,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 		u32 moff;
 
 		moff = __btf_member_bit_offset(t, member) / 8;
-		ptype = btf_type_resolve_ptr(btf_vmlinux, member->type, NULL);
+		ptype = btf_type_resolve_ptr(st_map->btf, member->type, NULL);
 		if (ptype == module_type) {
 			if (*(void **)(udata + moff))
 				goto reset_unlock;

@@ -461,8 +463,8 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 		if (!ptype || !btf_type_is_func_proto(ptype)) {
 			u32 msize;
 
-			mtype = btf_type_by_id(btf_vmlinux, member->type);
-			mtype = btf_resolve_size(btf_vmlinux, mtype, &msize);
+			mtype = btf_type_by_id(st_map->btf, member->type);
+			mtype = btf_resolve_size(st_map->btf, mtype, &msize);
 			if (IS_ERR(mtype)) {
 				err = PTR_ERR(mtype);
 				goto reset_unlock;

@@ -601,6 +603,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key)
 static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key,
 					     struct seq_file *m)
 {
+	struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map;
 	void *value;
 	int err;

@@ -610,7 +613,8 @@ static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key,
 
 	err = bpf_struct_ops_map_sys_lookup_elem(map, key, value);
 	if (!err) {
-		btf_type_seq_show(btf_vmlinux, map->btf_vmlinux_value_type_id,
+		btf_type_seq_show(st_map->btf,
+				  map->btf_vmlinux_value_type_id,
 				  value, m);
 		seq_puts(m, "\n");
 	}

@@ -720,6 +724,8 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 		return ERR_PTR(-ENOMEM);
 	}
 
+	st_map->btf = btf_vmlinux;
+
 	mutex_init(&st_map->lock);
 	set_vm_flush_reset_perms(st_map->image);
 	bpf_map_init_from_attr(map, attr);

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #6 lore raw

PATCH v12 6/14 bpf: lookup struct_ops types from a given module BTF.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

This is a preparation for searching for struct_ops types from a specified module. BTF is always btf_vmlinux now. This patch passes a pointer of BTF to bpf_struct_ops_find_value() and bpf_struct_ops_find(). Once the new registration API of struct_ops types is used, other BTFs besides btf_vmlinux can also be passed to them.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h         |  4 ++--
 kernel/bpf/bpf_struct_ops.c | 11 ++++++-----
 kernel/bpf/verifier.c       |  2 +-
 3 files changed, 9 insertions(+), 8 deletions(-)

include/linux/bpf.h +2 -2

index 73724e8caddf..59d26203f4d2 100644
@@ -1649,7 +1649,7 @@ struct bpf_struct_ops_desc {
 
 #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
 #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
-const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id);
+const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id);
 void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
 bool bpf_struct_ops_get(const void *kdata);
 void bpf_struct_ops_put(const void *kdata);

@@ -1692,7 +1692,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 			    union bpf_attr __user *uattr);
 #endif
 #else
-static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id)
+static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id)
 {
 	return NULL;
 }

kernel/bpf/bpf_struct_ops.c +6 -5

index 9e5d77ea738a..123ec76c48a9 100644
@@ -221,11 +221,11 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
 extern struct btf *btf_vmlinux;
 
 static const struct bpf_struct_ops_desc *
-bpf_struct_ops_find_value(u32 value_id)
+bpf_struct_ops_find_value(struct btf *btf, u32 value_id)
 {
 	unsigned int i;
 
-	if (!value_id || !btf_vmlinux)
+	if (!value_id || !btf)
 		return NULL;
 
 	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {

@@ -236,11 +236,12 @@ bpf_struct_ops_find_value(u32 value_id)
 	return NULL;
 }
 
-const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id)
+const struct bpf_struct_ops_desc *
+bpf_struct_ops_find(struct btf *btf, u32 type_id)
 {
 	unsigned int i;
 
-	if (!type_id || !btf_vmlinux)
+	if (!type_id || !btf)
 		return NULL;
 
 	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {

@@ -676,7 +677,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 	struct bpf_map *map;
 	int ret;
 
-	st_ops_desc = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id);
+	st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, attr->btf_vmlinux_value_type_id);
 	if (!st_ops_desc)
 		return ERR_PTR(-ENOTSUPP);

kernel/bpf/verifier.c +1 -1

index 5b9652a0d062..405aeee608aa 100644
@@ -20020,7 +20020,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	btf_id = prog->aux->attach_btf_id;
-	st_ops_desc = bpf_struct_ops_find(btf_id);
+	st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id);
 	if (!st_ops_desc) {
 		verbose(env, "attach_btf_id %u is not a supported struct\n",
 			btf_id);

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #7 lore raw

PATCH v12 7/14 bpf: pass attached BTF to the bpf_struct_ops subsystem

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Pass the fd of a btf from the userspace to the bpf() syscall, and then convert the fd into a btf. The btf is generated from the module that defines the target BPF struct_ops type.

In order to inform the kernel about the module that defines the target struct_ops type, the userspace program needs to provide a btf fd for the respective module's btf. This btf contains essential information on the types defined within the module, including the target struct_ops type.

A btf fd must be provided to the kernel for struct_ops maps struct_ops and for the bpf programs attached to those maps.

In the case of the bpf programs, the attach_btf_obj_fd parameter is passed as part of the bpf_attr and is converted into a btf. This btf is then stored in the prog->aux->attach_btf field. Here, it just let the verifier access attach_btf directly.

In the case of struct_ops maps, a btf fd is passed as value_type_btf_obj_fd of bpf_attr. The bpf_struct_ops_map_alloc() function converts the fd to a btf and stores it as st_map->btf.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/uapi/linux/bpf.h       |  5 +++
 kernel/bpf/bpf_struct_ops.c    | 56 +++++++++++++++++++++++-----------
 kernel/bpf/syscall.c           |  2 +-
 kernel/bpf/verifier.c          |  9 ++++--
 tools/include/uapi/linux/bpf.h |  5 +++
 5 files changed, 56 insertions(+), 21 deletions(-)

include/uapi/linux/bpf.h +5 -0

index e88746ba7d21..8e5a188e2883 100644
@@ -1401,6 +1401,11 @@ union bpf_attr {
 		 * to using 5 hash functions).
 		 */
 		__u64	map_extra;
+
+		__u32   value_type_btf_obj_fd;	/* fd pointing to a BTF
+						 * type data for
+						 * btf_vmlinux_value_type_id.
+						 */
 	};
 
 	struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

kernel/bpf/bpf_struct_ops.c +39 ~~-17~~

102 lines not on this page · raw message

kernel/bpf/syscall.c +1 -1

index 5e43ddd1b83f..6df7edc4b5e3 100644
@@ -1096,7 +1096,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,
 	return ret;
 }
 
-#define BPF_MAP_CREATE_LAST_FIELD map_extra
+#define BPF_MAP_CREATE_LAST_FIELD value_type_btf_obj_fd
 /* called via syscall */
 static int map_create(union bpf_attr *attr)
 {

kernel/bpf/verifier.c +6 -3

index 405aeee608aa..fccdab9db9ed 100644
@@ -20012,6 +20012,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	const struct btf_member *member;
 	struct bpf_prog *prog = env->prog;
 	u32 btf_id, member_idx;
+	struct btf *btf;
 	const char *mname;
 
 	if (!prog->gpl_compatible) {

@@ -20019,8 +20020,10 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 		return -EINVAL;
 	}
 
+	btf = prog->aux->attach_btf;
+
 	btf_id = prog->aux->attach_btf_id;
-	st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id);
+	st_ops_desc = bpf_struct_ops_find(btf, btf_id);
 	if (!st_ops_desc) {
 		verbose(env, "attach_btf_id %u is not a supported struct\n",
 			btf_id);

@@ -20037,8 +20040,8 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	member = &btf_type_member(t)[member_idx];
-	mname = btf_name_by_offset(btf_vmlinux, member->name_off);
-	func_proto = btf_type_resolve_func_ptr(btf_vmlinux, member->type,
+	mname = btf_name_by_offset(btf, member->name_off);
+	func_proto = btf_type_resolve_func_ptr(btf, member->type,
 					       NULL);
 	if (!func_proto) {
 		verbose(env, "attach to invalid member %s(@idx %u) of struct %s\n",

tools/include/uapi/linux/bpf.h +5 -0

index e88746ba7d21..8e5a188e2883 100644
@@ -1401,6 +1401,11 @@ union bpf_attr {
 		 * to using 5 hash functions).
 		 */
 		__u64	map_extra;
+
+		__u32   value_type_btf_obj_fd;	/* fd pointing to a BTF
+						 * type data for
+						 * btf_vmlinux_value_type_id.
+						 */
 	};
 
 	struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #8 lore raw

PATCH v12 8/14 bpf: hold module for bpf_struct_ops_map.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

To ensure that a module remains accessible whenever a struct_ops object of a struct_ops type provided by the module is still in use.

struct bpf_strct_ops_map doesn't hold a refcnt to btf anymore sicne a module will hold a refcnt to it's btf already. But, struct_ops programs are different. They hold their associated btf, not the module since they need only btf to assure their types (signatures).

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h          |  1 +
 include/linux/bpf_verifier.h |  1 +
 kernel/bpf/bpf_struct_ops.c  | 28 +++++++++++++++++++++++-----
 kernel/bpf/verifier.c        | 10 ++++++++++
 4 files changed, 35 insertions(+), 5 deletions(-)

include/linux/bpf.h +1 -0

index 59d26203f4d2..4cc2dfcd49e6 100644
@@ -1634,6 +1634,7 @@ struct bpf_struct_ops {
 	void (*unreg)(void *kdata);
 	int (*update)(void *kdata, void *old_kdata);
 	int (*validate)(void *kdata);
+	struct module *owner;
 	const char *name;
 	struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS];
 };

include/linux/bpf_verifier.h +1 -0

index 3378cc753061..353578e6cd24 100644
@@ -619,6 +619,7 @@ struct bpf_verifier_env {
 	u32 prev_insn_idx;
 	struct bpf_prog *prog;		/* eBPF program being verified */
 	const struct bpf_verifier_ops *ops;
+	struct module *attach_btf_mod;	/* The owner module of prog->aux->attach_btf */
 	struct bpf_verifier_stack_elem *head; /* stack of verifier states to be processed */
 	int stack_size;			/* number of states to be processed */
 	bool strict_alignment;		/* perform strict pointer alignment checks */

kernel/bpf/bpf_struct_ops.c +23 -5

index 3f647df50504..8ff86529b829 100644
@@ -635,12 +635,15 @@ static void __bpf_struct_ops_map_free(struct bpf_map *map)
 		bpf_jit_uncharge_modmem(PAGE_SIZE);
 	}
 	bpf_map_area_free(st_map->uvalue);
-	btf_put(st_map->btf);
 	bpf_map_area_free(st_map);
 }
 
 static void bpf_struct_ops_map_free(struct bpf_map *map)
 {
+	struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map;
+
+	module_put(st_map->st_ops_desc->st_ops->owner);
+
 	/* The struct_ops's function may switch to another struct_ops.
 	 *
 	 * For example, bpf_tcp_cc_x->init() may switch to

@@ -675,6 +678,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 	size_t st_map_size;
 	struct bpf_struct_ops_map *st_map;
 	const struct btf_type *t, *vt;
+	struct module *mod = NULL;
 	struct bpf_map *map;
 	struct btf *btf;
 	int ret;

@@ -684,10 +688,20 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 		btf = btf_get_by_fd(attr->value_type_btf_obj_fd);
 		if (IS_ERR(btf))
 			return ERR_PTR(PTR_ERR(btf));
-	} else {
+
+		if (btf != btf_vmlinux) {
+			mod = btf_try_get_module(btf);
+			if (!mod) {
+				btf_put(btf);
+				return ERR_PTR(-EINVAL);
+			}
+		}
+		/* mod (NULL for btf_vmlinux) holds a refcnt to btf. We
+		 * don't need an extra refcnt here.
+		 */
+		btf_put(btf);
+	} else
 		btf = btf_vmlinux;
-		btf_get(btf);
-	}
 
 	st_ops_desc = bpf_struct_ops_find_value(btf, attr->btf_vmlinux_value_type_id);
 	if (!st_ops_desc) {

@@ -751,7 +765,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 errout_free:
 	__bpf_struct_ops_map_free(map);
 errout:
-	btf_put(btf);
+	module_put(mod);
 
 	return ERR_PTR(ret);
 }

@@ -881,6 +895,10 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map
 	if (!bpf_struct_ops_valid_to_reg(new_map))
 		return -EINVAL;
 
+	/* The old map is holding the refcount for the owner module.  The
+	 * ownership of the owner module refcount is going to be
+	 * transferred from the old map to the new map.
+	 */
 	if (!st_map->st_ops_desc->st_ops->update)
 		return -EOPNOTSUPP;

kernel/bpf/verifier.c +10 -0

index fccdab9db9ed..cb282979d689 100644
@@ -20021,6 +20021,14 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	btf = prog->aux->attach_btf;
+	if (btf != btf_vmlinux) {
+		/* Make sure st_ops is valid through the lifetime of env */
+		env->attach_btf_mod = btf_try_get_module(btf);
+		if (!env->attach_btf_mod) {
+			verbose(env, "owner module of btf is not found\n");
+			return -ENOTSUPP;
+		}
+	}
 
 	btf_id = prog->aux->attach_btf_id;
 	st_ops_desc = bpf_struct_ops_find(btf, btf_id);

@@ -20735,6 +20743,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
 		env->prog->expected_attach_type = 0;
 
 	*prog = env->prog;
+
+	module_put(env->attach_btf_mod);
 err_unlock:
 	if (!is_priv)
 		mutex_unlock(&bpf_verifier_lock);

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #9 lore raw

PATCH v12 9/14 bpf: validate value_type

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

A value_type should consist of three components: refcnt, state, and data. refcnt and state has been move to struct bpf_struct_ops_common_value to make it easier to check the value type.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h         | 12 +++++
 kernel/bpf/bpf_struct_ops.c | 93 ++++++++++++++++++++++++-------------
 2 files changed, 72 insertions(+), 33 deletions(-)

include/linux/bpf.h +12 -0

index 4cc2dfcd49e6..bc4279cd61e9 100644
@@ -3239,4 +3239,16 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)
 	return prog->aux->func_idx != 0;
 }
 
+enum bpf_struct_ops_state {
+	BPF_STRUCT_OPS_STATE_INIT,
+	BPF_STRUCT_OPS_STATE_INUSE,
+	BPF_STRUCT_OPS_STATE_TOBEFREE,
+	BPF_STRUCT_OPS_STATE_READY,
+};
+
+struct bpf_struct_ops_common_value {
+	refcount_t refcnt;
+	enum bpf_struct_ops_state state;
+};
+
 #endif /* _LINUX_BPF_H */

kernel/bpf/bpf_struct_ops.c +60 ~~-33~~

189 lines not on this page · raw message

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #10 lore raw

PATCH v12 10/14 bpf, net: switch to dynamic registration

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Replace the static list of struct_ops types with per-btf struct_ops_tab to enable dynamic registration.

Both bpf_dummy_ops and bpf_tcp_ca now utilize the registration function instead of being listed in bpf_struct_ops_types.h.

Cc [email protected]

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h               | 28 ++++++++--
 include/linux/btf.h               |  2 +
 kernel/bpf/bpf_struct_ops.c       | 90 ++++++++++---------------------
 kernel/bpf/bpf_struct_ops_types.h | 12 -----
 kernel/bpf/btf.c                  | 49 +++++++++++++++--
 net/bpf/bpf_dummy_struct_ops.c    | 13 ++++-
 net/ipv4/bpf_tcp_ca.c             | 14 +++--
 7 files changed, 119 insertions(+), 89 deletions(-)
 delete mode 100644 kernel/bpf/bpf_struct_ops_types.h

include/linux/bpf.h +23 -5

index bc4279cd61e9..fdfceae25e08 100644
@@ -1651,7 +1651,6 @@ struct bpf_struct_ops_desc {
 #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
 #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
 const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id);
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
 bool bpf_struct_ops_get(const void *kdata);
 void bpf_struct_ops_put(const void *kdata);
 int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key,

@@ -1697,10 +1696,6 @@ static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *
 {
 	return NULL;
 }
-static inline void bpf_struct_ops_init(struct btf *btf,
-				       struct bpf_verifier_log *log)
-{
-}
 static inline bool bpf_try_module_get(const void *data, struct module *owner)
 {
 	return try_module_get(owner);

@@ -3239,6 +3234,14 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)
 	return prog->aux->func_idx != 0;
 }
 
+int register_bpf_struct_ops(struct bpf_struct_ops *st_ops);
+
+#define REGISTER_BPF_STRUCT_OPS(st_ops, type) \
+({					      \
+	BTF_STRUCT_OPS_TYPE_EMIT(type);	      \
+	register_bpf_struct_ops(st_ops);      \
+})
+
 enum bpf_struct_ops_state {
 	BPF_STRUCT_OPS_STATE_INIT,
 	BPF_STRUCT_OPS_STATE_INUSE,

@@ -3251,4 +3254,19 @@ struct bpf_struct_ops_common_value {
 	enum bpf_struct_ops_state state;
 };
 
+/* bpf_struct_ops_##_name (e.g. bpf_struct_ops_tcp_congestion_ops) is
+ * the map's value exposed to the userspace and its btf-type-id is
+ * stored at the map->btf_vmlinux_value_type_id.
+ *
+ */
+#define DEFINE_STRUCT_OPS_VALUE_TYPE(_name)			\
+struct bpf_struct_ops_##_name {					\
+	struct bpf_struct_ops_common_value common;		\
+	struct _name data ____cacheline_aligned_in_smp;		\
+}
+
+int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc,
+			     struct btf *btf,
+			     struct bpf_verifier_log *log);
+
 #endif /* _LINUX_BPF_H */

include/linux/btf.h +2 -0

index e2f4b85cf82a..cabab3db5216 100644
@@ -12,6 +12,8 @@
 #include <uapi/linux/bpf.h>
 
 #define BTF_TYPE_EMIT(type) ((void)(type *)0)
+#define BTF_STRUCT_OPS_TYPE_EMIT(type) \
+	((void)(struct bpf_struct_ops_##type *)0)
 #define BTF_TYPE_EMIT_ENUM(enum_val) ((void)enum_val)
 
 /* These need to be macros, as the expressions are used in assembler input */

kernel/bpf/bpf_struct_ops.c +27 ~~-63~~

176 lines not on this page · raw message

kernel/bpf/bpf_struct_ops_types.h deleted +0 ~~-12~~

deleted file mode 100644
index 5678a9ddf817..000000000000
@@ -1,12 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/* internal file - do not include directly */
-
-#ifdef CONFIG_BPF_JIT
-#ifdef CONFIG_NET
-BPF_STRUCT_OPS_TYPE(bpf_dummy_ops)
-#endif
-#ifdef CONFIG_INET
-#include <net/tcp.h>
-BPF_STRUCT_OPS_TYPE(tcp_congestion_ops)
-#endif
-#endif

kernel/bpf/btf.c +45 -4

79 lines not on this page · raw message

net/bpf/bpf_dummy_struct_ops.c +11 -2

index ffa224053a6c..65a89580d5d2 100644
@@ -7,7 +7,7 @@
 #include <linux/bpf.h>
 #include <linux/btf.h>
 
-extern struct bpf_struct_ops bpf_bpf_dummy_ops;
+static struct bpf_struct_ops bpf_bpf_dummy_ops;
 
 /* A common type for test_N with return value in bpf_dummy_ops */
 typedef int (*dummy_ops_test_ret_fn)(struct bpf_dummy_ops_state *state, ...);

@@ -223,11 +223,13 @@ static int bpf_dummy_reg(void *kdata)
 	return -EOPNOTSUPP;
 }
 
+DEFINE_STRUCT_OPS_VALUE_TYPE(bpf_dummy_ops);
+
 static void bpf_dummy_unreg(void *kdata)
 {
 }
 
-struct bpf_struct_ops bpf_bpf_dummy_ops = {
+static struct bpf_struct_ops bpf_bpf_dummy_ops = {
 	.verifier_ops = &bpf_dummy_verifier_ops,
 	.init = bpf_dummy_init,
 	.check_member = bpf_dummy_ops_check_member,

@@ -235,4 +237,11 @@ struct bpf_struct_ops bpf_bpf_dummy_ops = {
 	.reg = bpf_dummy_reg,
 	.unreg = bpf_dummy_unreg,
 	.name = "bpf_dummy_ops",
+	.owner = THIS_MODULE,
 };
+
+static int __init bpf_dummy_struct_ops_init(void)
+{
+	return REGISTER_BPF_STRUCT_OPS(&bpf_bpf_dummy_ops, bpf_dummy_ops);
+}
+late_initcall(bpf_dummy_struct_ops_init);

net/ipv4/bpf_tcp_ca.c +11 -3

index 3c8b76578a2a..69f0e8961628 100644
@@ -12,7 +12,7 @@
 #include <net/bpf_sk_storage.h>
 
 /* "extern" is to avoid sparse warning.  It is only used in bpf_struct_ops.c. */
-extern struct bpf_struct_ops bpf_tcp_congestion_ops;
+static struct bpf_struct_ops bpf_tcp_congestion_ops;
 
 static u32 unsupported_ops[] = {
 	offsetof(struct tcp_congestion_ops, get_info),

@@ -277,7 +277,9 @@ static int bpf_tcp_ca_validate(void *kdata)
 	return tcp_validate_congestion_control(kdata);
 }
 
-struct bpf_struct_ops bpf_tcp_congestion_ops = {
+DEFINE_STRUCT_OPS_VALUE_TYPE(tcp_congestion_ops);
+
+static struct bpf_struct_ops bpf_tcp_congestion_ops = {
 	.verifier_ops = &bpf_tcp_ca_verifier_ops,
 	.reg = bpf_tcp_ca_reg,
 	.unreg = bpf_tcp_ca_unreg,

@@ -287,10 +289,16 @@ struct bpf_struct_ops bpf_tcp_congestion_ops = {
 	.init = bpf_tcp_ca_init,
 	.validate = bpf_tcp_ca_validate,
 	.name = "tcp_congestion_ops",
+	.owner = THIS_MODULE,
 };
 
 static int __init bpf_tcp_ca_kfunc_init(void)
 {
-	return register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_tcp_ca_kfunc_set);
+	int ret;
+
+	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_tcp_ca_kfunc_set);
+	ret = ret ?: REGISTER_BPF_STRUCT_OPS(&bpf_tcp_congestion_ops, tcp_congestion_ops);
+
+	return ret;
 }
 late_initcall(bpf_tcp_ca_kfunc_init);

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #11 lore raw

PATCH v12 11/14 libbpf: Find correct module BTFs for struct_ops maps and progs.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Locate the module BTFs for struct_ops maps and progs and pass them to the kernel. This ensures that the kernel correctly resolves type IDs from the appropriate module BTFs.

For the map of a struct_ops object, the FD of the module BTF is set to bpf_map to keep a reference to the module BTF. The FD is passed to the kernel as value_type_btf_obj_fd when the struct_ops object is loaded.

For a bpf_struct_ops prog, attach_btf_obj_fd of bpf_prog is the FD of a module BTF in the kernel.

Signed-off-by Kui-Feng Lee <[email protected]>

Acked-by Andrii Nakryiko <[email protected]>

---
 tools/lib/bpf/bpf.c    |  4 +++-
 tools/lib/bpf/bpf.h    |  5 ++++-
 tools/lib/bpf/libbpf.c | 38 ++++++++++++++++++++++++++++----------
 3 files changed, 35 insertions(+), 12 deletions(-)

tools/lib/bpf/bpf.c +3 -1

index 9dc9625651dc..b133acfe08fb 100644
@@ -169,7 +169,8 @@ int bpf_map_create(enum bpf_map_type map_type,
 		   __u32 max_entries,
 		   const struct bpf_map_create_opts *opts)
 {
-	const size_t attr_sz = offsetofend(union bpf_attr, map_extra);
+	const size_t attr_sz = offsetofend(union bpf_attr,
+					   value_type_btf_obj_fd);
 	union bpf_attr attr;
 	int fd;

@@ -191,6 +192,7 @@ int bpf_map_create(enum bpf_map_type map_type,
 	attr.btf_key_type_id = OPTS_GET(opts, btf_key_type_id, 0);
 	attr.btf_value_type_id = OPTS_GET(opts, btf_value_type_id, 0);
 	attr.btf_vmlinux_value_type_id = OPTS_GET(opts, btf_vmlinux_value_type_id, 0);
+	attr.value_type_btf_obj_fd = OPTS_GET(opts, value_type_btf_obj_fd, 0);
 
 	attr.inner_map_fd = OPTS_GET(opts, inner_map_fd, 0);
 	attr.map_flags = OPTS_GET(opts, map_flags, 0);

tools/lib/bpf/bpf.h +4 -1

index d0f53772bdc0..ffdd81c0196a 100644
@@ -51,8 +51,11 @@ struct bpf_map_create_opts {
 
 	__u32 numa_node;
 	__u32 map_ifindex;
+
+	__u32 value_type_btf_obj_fd;
+	size_t:0;
 };
-#define bpf_map_create_opts__last_field map_ifindex
+#define bpf_map_create_opts__last_field value_type_btf_obj_fd
 
 LIBBPF_API int bpf_map_create(enum bpf_map_type map_type,
 			      const char *map_name,

tools/lib/bpf/libbpf.c +28 ~~-10~~

index ea9b8158c20d..deeb8fac7990 100644
@@ -527,6 +527,7 @@ struct bpf_map {
 	struct bpf_map_def def;
 	__u32 numa_node;
 	__u32 btf_var_idx;
+	int mod_btf_fd;
 	__u32 btf_key_type_id;
 	__u32 btf_value_type_id;
 	__u32 btf_vmlinux_value_type_id;

@@ -930,22 +931,29 @@ find_member_by_name(const struct btf *btf, const struct btf_type *t,
 	return NULL;
 }
 
+static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name,
+			    __u16 kind, struct btf **res_btf,
+			    struct module_btf **res_mod_btf);
+
 #define STRUCT_OPS_VALUE_PREFIX "bpf_struct_ops_"
 static int find_btf_by_prefix_kind(const struct btf *btf, const char *prefix,
 				   const char *name, __u32 kind);
 
 static int
-find_struct_ops_kern_types(const struct btf *btf, const char *tname,
+find_struct_ops_kern_types(struct bpf_object *obj, const char *tname,
+			   struct module_btf **mod_btf,
 			   const struct btf_type **type, __u32 *type_id,
 			   const struct btf_type **vtype, __u32 *vtype_id,
 			   const struct btf_member **data_member)
 {
 	const struct btf_type *kern_type, *kern_vtype;
 	const struct btf_member *kern_data_member;
+	struct btf *btf;
 	__s32 kern_vtype_id, kern_type_id;
 	__u32 i;
 
-	kern_type_id = btf__find_by_name_kind(btf, tname, BTF_KIND_STRUCT);
+	kern_type_id = find_ksym_btf_id(obj, tname, BTF_KIND_STRUCT,
+					&btf, mod_btf);
 	if (kern_type_id < 0) {
 		pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n",
 			tname);

@@ -999,14 +1007,16 @@ static bool bpf_map__is_struct_ops(const struct bpf_map *map)
 }
 
 /* Init the map's fields that depend on kern_btf */
-static int bpf_map__init_kern_struct_ops(struct bpf_map *map,
-					 const struct btf *btf,
-					 const struct btf *kern_btf)
+static int bpf_map__init_kern_struct_ops(struct bpf_map *map)
 {
 	const struct btf_member *member, *kern_member, *kern_data_member;
 	const struct btf_type *type, *kern_type, *kern_vtype;
 	__u32 i, kern_type_id, kern_vtype_id, kern_data_off;
+	struct bpf_object *obj = map->obj;
+	const struct btf *btf = obj->btf;
 	struct bpf_struct_ops *st_ops;
+	const struct btf *kern_btf;
+	struct module_btf *mod_btf;
 	void *data, *kern_data;
 	const char *tname;
 	int err;

@@ -1014,16 +1024,19 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map,
 	st_ops = map->st_ops;
 	type = st_ops->type;
 	tname = st_ops->tname;
-	err = find_struct_ops_kern_types(kern_btf, tname,
+	err = find_struct_ops_kern_types(obj, tname, &mod_btf,
 					 &kern_type, &kern_type_id,
 					 &kern_vtype, &kern_vtype_id,
 					 &kern_data_member);
 	if (err)
 		return err;
 
+	kern_btf = mod_btf ? mod_btf->btf : obj->btf_vmlinux;
+
 	pr_debug("struct_ops init_kern %s: type_id:%u kern_type_id:%u kern_vtype_id:%u\n",
 		 map->name, st_ops->type_id, kern_type_id, kern_vtype_id);
 
+	map->mod_btf_fd = mod_btf ? mod_btf->fd : 0;
 	map->def.value_size = kern_vtype->size;
 	map->btf_vmlinux_value_type_id = kern_vtype_id;

@@ -1099,6 +1112,8 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map,
 				return -ENOTSUP;
 			}
 
+			if (mod_btf)
+				prog->attach_btf_obj_fd = mod_btf->fd;
 			prog->attach_btf_id = kern_type_id;
 			prog->expected_attach_type = kern_member_idx;

@@ -1141,8 +1156,7 @@ static int bpf_object__init_kern_struct_ops_maps(struct bpf_object *obj)
 		if (!bpf_map__is_struct_ops(map))
 			continue;
 
-		err = bpf_map__init_kern_struct_ops(map, obj->btf,
-						    obj->btf_vmlinux);
+		err = bpf_map__init_kern_struct_ops(map);
 		if (err)
 			return err;
 	}

@@ -5201,8 +5215,10 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b
 	create_attr.numa_node = map->numa_node;
 	create_attr.map_extra = map->map_extra;
 
-	if (bpf_map__is_struct_ops(map))
+	if (bpf_map__is_struct_ops(map)) {
 		create_attr.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id;
+		create_attr.value_type_btf_obj_fd = map->mod_btf_fd;
+	}
 
 	if (obj->btf && btf__fd(obj->btf) >= 0) {
 		create_attr.btf_fd = btf__fd(obj->btf);

@@ -9546,7 +9562,9 @@ static int libbpf_find_attach_btf_id(struct bpf_program *prog, const char *attac
 		*btf_obj_fd = 0;
 		*btf_type_id = 1;
 	} else {
-		err = find_kernel_btf_id(prog->obj, attach_name, attach_type, btf_obj_fd, btf_type_id);
+		err = find_kernel_btf_id(prog->obj, attach_name,
+					 attach_type, btf_obj_fd,
+					 btf_type_id);
 	}
 	if (err) {
 		pr_warn("prog '%s': failed to find kernel BTF type ID of '%s': %d\n",

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #12 lore raw

PATCH v12 12/14 bpf: export btf_ctx_access to modules.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

The module requires the use of btf_ctx_access() to invoke bpf_tracing_btf_ctx_access() from a module. This function is valuable for implementing validation functions that ensure proper access to ctx.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/btf.c | 1 +
 1 file changed, 1 insertion(+)

kernel/bpf/btf.c +1 -0

index 5545dee3ff54..d9cdf41e8f34 100644
@@ -6142,6 +6142,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,
 		__btf_name_by_offset(btf, t->name_off));
 	return true;
 }
+EXPORT_SYMBOL_GPL(btf_ctx_access);
 
 enum bpf_struct_walk_result {
 	/* < 0 error */

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #13 lore raw

PATCH v12 13/14 selftests/bpf: test case for register_bpf_struct_ops().

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   | 52 +++++++++++
 .../selftests/bpf/bpf_testmod/bpf_testmod.h   |  5 +
 .../bpf/prog_tests/test_struct_ops_module.c   | 92 +++++++++++++++++++
 .../selftests/bpf/progs/struct_ops_module.c   | 30 ++++++
 .../selftests/bpf/progs/testmod_unload.c      | 25 +++++
 5 files changed, 204 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/testmod_unload.c

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +52 -0

index 91907b321f91..804a67bbb479 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Copyright (c) 2020 Facebook */
+#include <linux/bpf.h>
 #include <linux/btf.h>
 #include <linux/btf_ids.h>
 #include <linux/error-injection.h>

@@ -520,11 +521,61 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_static_unused_arg)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_offset)
 BTF_SET8_END(bpf_testmod_check_kfunc_ids)
 
+DEFINE_STRUCT_OPS_VALUE_TYPE(bpf_testmod_ops);
+
+static int bpf_testmod_ops_init(struct btf *btf)
+{
+	return 0;
+}
+
+static bool bpf_testmod_ops_is_valid_access(int off, int size,
+					    enum bpf_access_type type,
+					    const struct bpf_prog *prog,
+					    struct bpf_insn_access_aux *info)
+{
+	return bpf_tracing_btf_ctx_access(off, size, type, prog, info);
+}
+
+static int bpf_testmod_ops_init_member(const struct btf_type *t,
+				       const struct btf_member *member,
+				       void *kdata, const void *udata)
+{
+	return 0;
+}
+
 static const struct btf_kfunc_id_set bpf_testmod_kfunc_set = {
 	.owner = THIS_MODULE,
 	.set   = &bpf_testmod_check_kfunc_ids,
 };
 
+static const struct bpf_verifier_ops bpf_testmod_verifier_ops = {
+	.is_valid_access = bpf_testmod_ops_is_valid_access,
+};
+
+static int bpf_dummy_reg(void *kdata)
+{
+	struct bpf_testmod_ops *ops = kdata;
+	int r;
+
+	r = ops->test_2(4, 3);
+
+	return 0;
+}
+
+static void bpf_dummy_unreg(void *kdata)
+{
+}
+
+struct bpf_struct_ops bpf_bpf_testmod_ops = {
+	.verifier_ops = &bpf_testmod_verifier_ops,
+	.init = bpf_testmod_ops_init,
+	.init_member = bpf_testmod_ops_init_member,
+	.reg = bpf_dummy_reg,
+	.unreg = bpf_dummy_unreg,
+	.name = "bpf_testmod_ops",
+	.owner = THIS_MODULE,
+};
+
 extern int bpf_fentry_test1(int a);
 
 static int bpf_testmod_init(void)

@@ -535,6 +586,7 @@ static int bpf_testmod_init(void)
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_testmod_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_testmod_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_testmod_kfunc_set);
+	ret = ret ?: REGISTER_BPF_STRUCT_OPS(&bpf_bpf_testmod_ops, bpf_testmod_ops);
 	if (ret < 0)
 		return ret;
 	if (bpf_fentry_test1(0) < 0)

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h +5 -0

index f32793efe095..ca5435751c79 100644
@@ -28,4 +28,9 @@ struct bpf_iter_testmod_seq {
 	int cnt;
 };
 
+struct bpf_testmod_ops {
+	int (*test_1)(void);
+	int (*test_2)(int a, int b);
+};
+
 #endif /* _BPF_TESTMOD_H */

tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c new file +92 -0

92 lines not on this page · raw message

tools/testing/selftests/bpf/progs/struct_ops_module.c new file +30 -0

new file mode 100644
index 000000000000..cb305d04342f
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "../bpf_testmod/bpf_testmod.h"
+
+char _license[] SEC("license") = "GPL";
+
+int test_2_result = 0;
+
+SEC("struct_ops/test_1")
+int BPF_PROG(test_1)
+{
+	return 0xdeadbeef;
+}
+
+SEC("struct_ops/test_2")
+int BPF_PROG(test_2, int a, int b)
+{
+	test_2_result = a + b;
+	return a + b;
+}
+
+SEC(".struct_ops.link")
+struct bpf_testmod_ops testmod_1 = {
+	.test_1 = (void *)test_1,
+	.test_2 = (void *)test_2,
+};
+

tools/testing/selftests/bpf/progs/testmod_unload.c new file +25 -0

new file mode 100644
index 000000000000..bb17914ecca3
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_core_read.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+int bpf_testmod_put = 0;
+
+SEC("fentry/__bpf_struct_ops_map_free")
+int BPF_PROG(trace_map_free, struct bpf_map *map)
+{
+	static const char name[] = "testmod_1";
+	int i;
+
+	for (i = 0; i < sizeof(name); i++) {
+		if (map->name[i] != name[i])
+			return 0;
+	}
+
+	bpf_testmod_put = 1;
+
+	return 0;
+}

Signature · 1 line

2.34.1

[email protected]

author

2023-12-07 01:40 UTC #14 lore raw

PATCH v12 14/14 bpf: pass btf object id in bpf_map_info.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Include btf object id (btf_obj_id) in bpf_map_info so that tools (ex:
bpftools struct_ops dump) know the correct btf from the kernel to look up type information of struct_ops types.

Since struct_ops types can be defined and registered in a module. The type information of a struct_ops type are defined in the btf of the module defining it. The userspace tools need to know which btf is for the module defining a struct_ops type.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h            | 1 +
 include/uapi/linux/bpf.h       | 2 +-
 kernel/bpf/bpf_struct_ops.c    | 7 +++++++
 kernel/bpf/syscall.c           | 2 ++
 tools/include/uapi/linux/bpf.h | 2 +-
 5 files changed, 12 insertions(+), 2 deletions(-)

include/linux/bpf.h +1 -0

index fdfceae25e08..1cbaae59215c 100644
@@ -3268,5 +3268,6 @@ struct bpf_struct_ops_##_name {					\
 int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc,
 			     struct btf *btf,
 			     struct bpf_verifier_log *log);
+void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map);
 
 #endif /* _LINUX_BPF_H */

include/uapi/linux/bpf.h +1 -1

index 8e5a188e2883..0d3262bb4dc0 100644
@@ -6490,7 +6490,7 @@ struct bpf_map_info {
 	__u32 btf_id;
 	__u32 btf_key_type_id;
 	__u32 btf_value_type_id;
-	__u32 :32;	/* alignment pad */
+	__u32 btf_obj_id;
 	__u64 map_extra;
 } __attribute__((aligned(8)));

kernel/bpf/bpf_struct_ops.c +7 -0

index 9ca166c79d19..3aab2efa855a 100644
@@ -974,3 +974,10 @@ int bpf_struct_ops_link_create(union bpf_attr *attr)
 	kfree(link);
 	return err;
 }
+
+void bpf_map_struct_ops_info_fill(struct bpf_map_info *info, struct bpf_map *map)
+{
+	struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map;
+
+	info->btf_obj_id = btf_obj_id(st_map->btf);
+}

kernel/bpf/syscall.c +2 -0

index 6df7edc4b5e3..9e8847ca8962 100644
@@ -4621,6 +4621,8 @@ static int bpf_map_get_info_by_fd(struct file *file,
 		info.btf_value_type_id = map->btf_value_type_id;
 	}
 	info.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id;
+	if (map->map_type == BPF_MAP_TYPE_STRUCT_OPS)
+		bpf_map_struct_ops_info_fill(&info, map);
 
 	if (bpf_map_is_offloaded(map)) {
 		err = bpf_map_offload_info_fill(&info, map);

tools/include/uapi/linux/bpf.h +1 -1

index 8e5a188e2883..0d3262bb4dc0 100644
@@ -6490,7 +6490,7 @@ struct bpf_map_info {
 	__u32 btf_id;
 	__u32 btf_key_type_id;
 	__u32 btf_value_type_id;
-	__u32 :32;	/* alignment pad */
+	__u32 btf_obj_id;
 	__u64 map_extra;
 } __attribute__((aligned(8)));

Signature · 1 line

2.34.1

Kui-Feng Lee

2023-12-09 00:29 UTC #15 lore raw

In reply to [email protected]

I have sent v13 to solve the conflictions with the latest for-next.

Quoted 238 lines · On 12/6/23 17:39, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Given the current constraints of the current implementation, struct_ops cannot be registered dynamically. This presents a significant limitation for modules like coming fuse-bpf, which seeks to implement a new struct_ops type. To address this issue, a new API is introduced that allows the registration of new struct_ops types from modules.
Previously, struct_ops types were defined in bpf_struct_ops_types.h and collected as a static array. The new API lets callers add new struct_ops types dynamically. The static array has been removed and replaced by the per-btf struct_ops_tab.
The struct_ops subsystem relies on BTF to determine the layout of values in a struct_ops map and identify the subsystem that the struct_ops map registers to. However, the kernel BTF does not include the type information of struct_ops types defined by a module. The struct_ops subsystem requires knowledge of the corresponding module for a given struct_ops map and the utilization of BTF information from that module. We empower libbpf to determine the correct module for accessing the BTF information and pass an identity (FD) of the module btf to the kernel. The kernel looks up type information and registered struct_ops types directly from the given btf.
If a module exits while one or more struct_ops maps still refer to a struct_ops type defined by the module, it can lead to unforeseen complications. Therefore, it is crucial to ensure that a module remains intact as long as any struct_ops map is still linked to a struct_ops type defined by the module. To achieve this, every struct_ops map holds a reference to the module while being registered.
Changes from v11:
- bpf_struct_ops_maps hold only the refcnt to the module, but not
    btf. (patch 1)
- Fix warning messages. (patch 1, 9 and 10)
- Remove unnecessary conditional compiling of CONFIG_BPF_JIT.
    (patch 4, 9 and 10)
- Fix the commit log of the patch 7 to explain how a btf is pass from the user space and how the kernel handle it.
- bpf_struct_ops_maps hold the module defining it's type, but not btf. A map will hold the module through its life-span from allocating to being free. (patch 8)
- Change selftests and tracing __bpf_struct_ops_map_free() to wait for the release of the bpf_testmod module.
- Include btf_obj_id in bpf_map_info. (patch 14)
Changes from v10:
- Guard btf.c from CONFIG_BPF_JIT=n. This patchset has introduced symbols from bpf_struct_ops.c which is only built when CONFIG_BPF_JIT=y.
- Fix the warning of unused errout_free label by moving code that is leaked to patch 8 to patch 7.
Changes from v9:
- Remove the call_rcu_tasks_trace() changes from kern_sync_rcu().
- Trace btf_put() in the test case to ensure the release of kmod's btf, or the consequent tests may fail for using kmod's unloaded old btf instead the new one created after loading again. The kmod's btf may live for awhile after unloading the kmod, for a map being freed asynchronized is still holding the btf.
- Split "add struct_ops_tab to btf" into tow patches by adding "make struct_ops_map support btfs other than btf_vmlinux".
- Flip the order of "pass attached BTF to the bpf_struct_ops subsystem" and "hold module for bpf_struct_ops_map" to make it more reasonable.
- Fix the compile errors of a missing header file.
Changes from v8:
- Rename bpf_struct_ops_init_one() to bpf_struct_ops_desc_init().
- Move code that using BTF_ID_LIST to the newly added patch 2.
- Move code that lookup struct_ops types from a given module to the newly added patch 5.
- Store the pointers of btf at st_maps.
- Add test cases for the cases of modules being unload.
- Call bpf_struct_ops_init() in btf_add_struct_ops() to fix an inconsistent issue.
Changes from v7:
- Fix check_struct_ops_btf_id() to use attach btf if there is instead of btf_vmlinux.
Changes from v6:
- Change returned error code to -EINVAL for the case of
    bpf_try_get_module().
- Return an error code from bpf_struct_ops_init().
- Fix the dependency issue of testing_helpers.c and rcu_tasks_trace_gp.skel.h.
Changes from v5:
- As the 2nd patch, we introduce "bpf_struct_ops_desc". This change involves moving certain members of "bpf_struct_ops" to "bpf_struct_ops_desc", which becomes a part of "btf_struct_ops_tab". This ensures that these members remain accessible even when the owner module of a "bpf_struct_ops" is unloaded.
- Correct the order of arguments when calling
in the 3rd patch.
- Remove the owner argument from bpf_struct_ops_init_one(). Instead,
    callers should fill in st_ops->owner.
- Make sure to hold the owner module when calling bpf_struct_ops_find() and bpf_struct_ops_find_value() in the 6th patch.
- Merge the functions register_bpf_struct_ops_btf() and register_bpf_struct_ops() into a single function and relocate it to btf.c for better organization and clarity.
- Undo the name modifications made to find_kernel_btf_id() and find_ksym_btf_id() in the 8th patch.
Changes from v4:
- Fix the dependency between testing_helpers.o and rcu_tasks_trace_gp.skel.h.
Changes from v3:
- Fix according to the feedback for v3.
- Change of the order of arguments to make btf as the first argument.
- Use btf_try_get_module() instead of try_get_module() since the module pointed by st_ops->owner can gone while some one is still holding its btf.
- Move variables defined by BPF_STRUCT_OPS_COMMON_VALUE to struct bpf_struct_ops_common_value to validation easier.
- Register the struct_ops type defined by bpf_testmod in its init function.
- Rename field name to 'value_type_btf_obj_fd' to make it explicit.
- Fix leaking of btf objects on error.
- st_maps hold their modules to keep modules alive and prevent they from unloading.
- bpf_map of libbpf keeps mod_btf_fd instead of a pointer to module_btf.
- Do call_rcu_tasks_trace() in kern_sync_rcu() to ensure the bpf_testmod is unloaded properly. It uses rcu_tasks_trace_gp to trigger call_rcu_tasks_trace() in the kernel.
- Merge and reorder patches in a reasonable order.
Changes from v2:
- Remove struct_ops array, and add a per-btf (module) struct_ops_tab to collect registered struct_ops types.
- Validate value_type by checking member names and types.
---
v11: https://lore.kernel.org/all/[email protected]/
v10: https://lore.kernel.org/all/[email protected]/
v9: https://lore.kernel.org/all/[email protected]/
v8: https://lore.kernel.org/all/[email protected]/
v7: https://lore.kernel.org/all/[email protected]/
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
Kui-Feng Lee (14):
bpf: refactory struct_ops type initialization to a function.
bpf: get type information with BPF_ID_LIST
bpf, net: introduce bpf_struct_ops_desc.
bpf: add struct_ops_tab to btf.
bpf: make struct_ops_map support btfs other than btf_vmlinux.
bpf: lookup struct_ops types from a given module BTF.
bpf: pass attached BTF to the bpf_struct_ops subsystem
bpf: hold module for bpf_struct_ops_map.
bpf: validate value_type
bpf, net: switch to dynamic registration
libbpf: Find correct module BTFs for struct_ops maps and progs.
bpf: export btf_ctx_access to modules. selftests/bpf: test case for register_bpf_struct_ops().
bpf: pass btf object id in bpf_map_info.
  include/linux/bpf.h                           |  55 ++-
  include/linux/bpf_verifier.h                  |   1 +
  include/linux/btf.h                           |   8 +
  include/uapi/linux/bpf.h                      |   7 +-
  kernel/bpf/bpf_struct_ops.c                   | 421 ++++++++++--------
  kernel/bpf/bpf_struct_ops_types.h             |  12 -
  kernel/bpf/btf.c                              | 124 +++++-
  kernel/bpf/syscall.c                          |   4 +-
  kernel/bpf/verifier.c                         |  25 +-
  net/bpf/bpf_dummy_struct_ops.c                |  22 +-
  net/ipv4/bpf_tcp_ca.c                         |  22 +-
  tools/include/uapi/linux/bpf.h                |   7 +-
  tools/lib/bpf/bpf.c                           |   4 +-
  tools/lib/bpf/bpf.h                           |   5 +-
  tools/lib/bpf/libbpf.c                        |  38 +-
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  52 +++
  .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   5 +
  .../bpf/prog_tests/test_struct_ops_module.c   |  92 ++++
  .../selftests/bpf/progs/struct_ops_module.c   |  30 ++
  .../selftests/bpf/progs/testmod_unload.c      |  25 ++
  20 files changed, 726 insertions(+), 233 deletions(-)
delete mode 100644 kernel/bpf/bpf_struct_ops_types.h
create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/testmod_unload.c