Registrating struct_ops types from modules

PATCH Superseded [email protected] 4 participants 32 messages Nov 6, 2023 – Nov 27, 2023

Subsystems (8)

Networking Network drivers Kselftest BPF +4 more subsystems

lore

Share & tools

lore

[email protected]

author

2023-11-06 20:13 UTC #0 lore raw

v11 cover Registrating struct_ops types from modules

From: Kui-Feng Lee <[email protected]>

Given the current constraints of the current implementation, struct_ops cannot be registered dynamically. This presents a significant limitation for modules like coming fuse-bpf, which seeks to implement a new struct_ops type. To address this issue, a new API is introduced that allows the registration of new struct_ops types from modules.

Previously, struct_ops types were defined in bpf_struct_ops_types.h and collected as a static array. The new API lets callers add new struct_ops types dynamically. The static array has been removed and replaced by the per-btf struct_ops_tab.

The struct_ops subsystem relies on BTF to determine the layout of values in a struct_ops map and identify the subsystem that the struct_ops map registers to. However, the kernel BTF does not include the type information of struct_ops types defined by a module. The struct_ops subsystem requires knowledge of the corresponding module for a given struct_ops map and the utilization of BTF information from that module. We empower libbpf to determine the correct module for accessing the BTF information and pass an identity (FD) of the module btf to the kernel. The kernel looks up type information and registered struct_ops types directly from the given btf.

If a module exits while one or more struct_ops maps still refer to a struct_ops type defined by the module, it can lead to unforeseen complications. Therefore, it is crucial to ensure that a module remains intact as long as any struct_ops map is still linked to a struct_ops type defined by the module. To achieve this, every struct_ops map holds a reference to the module while being registered.

Changes from v10:

- Guard btf.c from CONFIG_BPF_JIT=n. This patchset has introduced symbols from bpf_struct_ops.c which is only built when CONFIG_BPF_JIT=y.

- Fix the warning of unused errout_free label by moving code that is leaked to patch 8 to patch 7.

Changes from v9:

- Remove the call_rcu_tasks_trace() changes from kern_sync_rcu().

- Trace btf_put() in the test case to ensure the release of kmod's btf, or the consequent tests may fail for using kmod's unloaded old btf instead the new one created after loading again. The kmod's btf may live for awhile after unloading the kmod, for a map being freed asynchronized is still holding the btf.

- Split "add struct_ops_tab to btf" into tow patches by adding "make struct_ops_map support btfs other than btf_vmlinux".

- Flip the order of "pass attached BTF to the bpf_struct_ops subsystem" and "hold module for bpf_struct_ops_map" to make it more reasonable.

- Fix the compile errors of a missing header file.

Changes from v8:

- Rename bpf_struct_ops_init_one() to bpf_struct_ops_desc_init().

- Move code that using BTF_ID_LIST to the newly added patch 2.

- Move code that lookup struct_ops types from a given module to the newly added patch 5.

- Store the pointers of btf at st_maps.

- Add test cases for the cases of modules being unload.

- Call bpf_struct_ops_init() in btf_add_struct_ops() to fix an inconsistent issue.

Changes from v7:

- Fix check_struct_ops_btf_id() to use attach btf if there is instead of btf_vmlinux.

Changes from v6:

- Change returned error code to -EINVAL for the case of bpf_try_get_module().

- Return an error code from bpf_struct_ops_init().

- Fix the dependency issue of testing_helpers.c and rcu_tasks_trace_gp.skel.h.

Changes from v5:

- As the 2nd patch, we introduce "bpf_struct_ops_desc". This change involves moving certain members of "bpf_struct_ops" to "bpf_struct_ops_desc", which becomes a part of "btf_struct_ops_tab". This ensures that these members remain accessible even when the owner module of a "bpf_struct_ops" is unloaded.

- Correct the order of arguments when calling
in the 3rd patch.

- Remove the owner argument from bpf_struct_ops_init_one(). Instead, callers should fill in st_ops->owner.

- Make sure to hold the owner module when calling bpf_struct_ops_find() and bpf_struct_ops_find_value() in the 6th patch.

- Merge the functions register_bpf_struct_ops_btf() and register_bpf_struct_ops() into a single function and relocate it to btf.c for better organization and clarity.

- Undo the name modifications made to find_kernel_btf_id() and find_ksym_btf_id() in the 8th patch.

Changes from v4:

- Fix the dependency between testing_helpers.o and rcu_tasks_trace_gp.skel.h.

Changes from v3:

- Fix according to the feedback for v3.

- Change of the order of arguments to make btf as the first argument.

- Use btf_try_get_module() instead of try_get_module() since the module pointed by st_ops->owner can gone while some one is still holding its btf.

- Move variables defined by BPF_STRUCT_OPS_COMMON_VALUE to struct bpf_struct_ops_common_value to validation easier.

- Register the struct_ops type defined by bpf_testmod in its init function.

- Rename field name to 'value_type_btf_obj_fd' to make it explicit.

- Fix leaking of btf objects on error.

- st_maps hold their modules to keep modules alive and prevent they from unloading.

- bpf_map of libbpf keeps mod_btf_fd instead of a pointer to module_btf.

- Do call_rcu_tasks_trace() in kern_sync_rcu() to ensure the bpf_testmod is unloaded properly. It uses rcu_tasks_trace_gp to trigger call_rcu_tasks_trace() in the kernel.

- Merge and reorder patches in a reasonable order.

Changes from v2:

- Remove struct_ops array, and add a per-btf (module) struct_ops_tab to collect registered struct_ops types.

- Validate value_type by checking member names and types.

---
v10: https://lore.kernel.org/all/[email protected]/
v9: https://lore.kernel.org/all/[email protected]/
v8: https://lore.kernel.org/all/[email protected]/
v7: https://lore.kernel.org/all/[email protected]/
v6: https://lore.kernel.org/all/[email protected]/
v5: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/

Kui-Feng Lee (13):
bpf: refactory struct_ops type initialization to a function.
bpf: get type information with BPF_ID_LIST
bpf, net: introduce bpf_struct_ops_desc.
bpf: add struct_ops_tab to btf.
bpf: make struct_ops_map support btfs other than btf_vmlinux.
bpf: lookup struct_ops types from a given module BTF.
bpf: pass attached BTF to the bpf_struct_ops subsystem
bpf: hold module for bpf_struct_ops_map.
bpf: validate value_type
bpf, net: switch to dynamic registration
libbpf: Find correct module BTFs for struct_ops maps and progs.
bpf: export btf_ctx_access to modules.
selftests/bpf: test case for register_bpf_struct_ops().

 include/linux/bpf.h                           |  52 ++-
 include/linux/bpf_verifier.h                  |   1 +
 include/linux/btf.h                           |  10 +
 include/uapi/linux/bpf.h                      |   5 +
 kernel/bpf/bpf_struct_ops.c                   | 437 ++++++++++--------
 kernel/bpf/bpf_struct_ops_types.h             |  12 -
 kernel/bpf/btf.c                              | 120 ++++-
 kernel/bpf/syscall.c                          |   2 +-
 kernel/bpf/verifier.c                         |  25 +-
 net/bpf/bpf_dummy_struct_ops.c                |  23 +-
 net/ipv4/bpf_tcp_ca.c                         |  24 +-
 tools/include/uapi/linux/bpf.h                |   5 +
 tools/lib/bpf/bpf.c                           |   4 +-
 tools/lib/bpf/bpf.h                           |   5 +-
 tools/lib/bpf/libbpf.c                        |  38 +-
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  59 +++
 .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   5 +
 .../bpf/prog_tests/test_struct_ops_module.c   | 144 ++++++
 .../selftests/bpf/progs/struct_ops_module.c   |  30 ++
 .../testing/selftests/bpf/progs/testmod_btf.c |  26 ++
 20 files changed, 796 insertions(+), 231 deletions(-)
 delete mode 100644 kernel/bpf/bpf_struct_ops_types.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/testmod_btf.c

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #1 lore raw

PATCH v11 1/13 bpf: refactory struct_ops type initialization to a function.

From: Kui-Feng Lee <[email protected]>

Move the majority of the code to bpf_struct_ops_init_one(), which can then be utilized for the initialization of newly registered dynamically allocated struct_ops types in the following patches.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/bpf_struct_ops.c | 157 +++++++++++++++++++-----------------
 1 file changed, 83 insertions(+), 74 deletions(-)

kernel/bpf/bpf_struct_ops.c +83 ~~-74~~

index db6176fb64dc..627cf1ea840a 100644
@@ -110,102 +110,111 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = {
 
 static const struct btf_type *module_type;
 
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
+static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
+				    struct btf *btf,
+				    struct bpf_verifier_log *log)
 {
-	s32 type_id, value_id, module_id;
 	const struct btf_member *member;
-	struct bpf_struct_ops *st_ops;
 	const struct btf_type *t;
+	s32 type_id, value_id;
 	char value_name[128];
 	const char *mname;
-	u32 i, j;
+	int i;
 
-	/* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */
-#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct bpf_struct_ops_##_name);
-#include "bpf_struct_ops_types.h"
-#undef BPF_STRUCT_OPS_TYPE
+	if (strlen(st_ops->name) + VALUE_PREFIX_LEN >=
+	    sizeof(value_name)) {
+		pr_warn("struct_ops name %s is too long\n",
+			st_ops->name);
+		return;
+	}
+	sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name);
 
-	module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT);
-	if (module_id < 0) {
-		pr_warn("Cannot find struct module in btf_vmlinux\n");
+	value_id = btf_find_by_name_kind(btf, value_name,
+					 BTF_KIND_STRUCT);
+	if (value_id < 0) {
+		pr_warn("Cannot find struct %s in btf_vmlinux\n",
+			value_name);
 		return;
 	}
-	module_type = btf_type_by_id(btf, module_id);
 
-	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {
-		st_ops = bpf_struct_ops[i];
+	type_id = btf_find_by_name_kind(btf, st_ops->name,
+					BTF_KIND_STRUCT);
+	if (type_id < 0) {
+		pr_warn("Cannot find struct %s in btf_vmlinux\n",
+			st_ops->name);
+		return;
+	}
+	t = btf_type_by_id(btf, type_id);
+	if (btf_type_vlen(t) > BPF_STRUCT_OPS_MAX_NR_MEMBERS) {
+		pr_warn("Cannot support #%u members in struct %s\n",
+			btf_type_vlen(t), st_ops->name);
+		return;
+	}
 
-		if (strlen(st_ops->name) + VALUE_PREFIX_LEN >=
-		    sizeof(value_name)) {
-			pr_warn("struct_ops name %s is too long\n",
+	for_each_member(i, t, member) {
+		const struct btf_type *func_proto;
+
+		mname = btf_name_by_offset(btf, member->name_off);
+		if (!*mname) {
+			pr_warn("anon member in struct %s is not supported\n",
 				st_ops->name);
-			continue;
+			break;
 		}
-		sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name);
 
-		value_id = btf_find_by_name_kind(btf, value_name,
-						 BTF_KIND_STRUCT);
-		if (value_id < 0) {
-			pr_warn("Cannot find struct %s in btf_vmlinux\n",
-				value_name);
-			continue;
+		if (__btf_member_bitfield_size(t, member)) {
+			pr_warn("bit field member %s in struct %s is not supported\n",
+				mname, st_ops->name);
+			break;
 		}
 
-		type_id = btf_find_by_name_kind(btf, st_ops->name,
-						BTF_KIND_STRUCT);
-		if (type_id < 0) {
-			pr_warn("Cannot find struct %s in btf_vmlinux\n",
-				st_ops->name);
-			continue;
-		}
-		t = btf_type_by_id(btf, type_id);
-		if (btf_type_vlen(t) > BPF_STRUCT_OPS_MAX_NR_MEMBERS) {
-			pr_warn("Cannot support #%u members in struct %s\n",
-				btf_type_vlen(t), st_ops->name);
-			continue;
+		func_proto = btf_type_resolve_func_ptr(btf,
+						       member->type,
+						       NULL);
+		if (func_proto &&
+		    btf_distill_func_proto(log, btf,
+					   func_proto, mname,
+					   &st_ops->func_models[i])) {
+			pr_warn("Error in parsing func ptr %s in struct %s\n",
+				mname, st_ops->name);
+			break;
 		}
+	}
 
-		for_each_member(j, t, member) {
-			const struct btf_type *func_proto;
+	if (i == btf_type_vlen(t)) {
+		if (st_ops->init(btf)) {
+			pr_warn("Error in init bpf_struct_ops %s\n",
+				st_ops->name);
+		} else {
+			st_ops->type_id = type_id;
+			st_ops->type = t;
+			st_ops->value_id = value_id;
+			st_ops->value_type = btf_type_by_id(btf,
+							    value_id);
+		}
+	}
+}
 
-			mname = btf_name_by_offset(btf, member->name_off);
-			if (!*mname) {
-				pr_warn("anon member in struct %s is not supported\n",
-					st_ops->name);
-				break;
-			}
+void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
+{
+	struct bpf_struct_ops *st_ops;
+	s32 module_id;
+	u32 i;
 
-			if (__btf_member_bitfield_size(t, member)) {
-				pr_warn("bit field member %s in struct %s is not supported\n",
-					mname, st_ops->name);
-				break;
-			}
+	/* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */
+#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct bpf_struct_ops_##_name);
+#include "bpf_struct_ops_types.h"
+#undef BPF_STRUCT_OPS_TYPE
 
-			func_proto = btf_type_resolve_func_ptr(btf,
-							       member->type,
-							       NULL);
-			if (func_proto &&
-			    btf_distill_func_proto(log, btf,
-						   func_proto, mname,
-						   &st_ops->func_models[j])) {
-				pr_warn("Error in parsing func ptr %s in struct %s\n",
-					mname, st_ops->name);
-				break;
-			}
-		}
+	module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT);
+	if (module_id < 0) {
+		pr_warn("Cannot find struct module in btf_vmlinux\n");
+		return;
+	}
+	module_type = btf_type_by_id(btf, module_id);
 
-		if (j == btf_type_vlen(t)) {
-			if (st_ops->init(btf)) {
-				pr_warn("Error in init bpf_struct_ops %s\n",
-					st_ops->name);
-			} else {
-				st_ops->type_id = type_id;
-				st_ops->type = t;
-				st_ops->value_id = value_id;
-				st_ops->value_type = btf_type_by_id(btf,
-								    value_id);
-			}
-		}
+	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {
+		st_ops = bpf_struct_ops[i];
+		bpf_struct_ops_init_one(st_ops, btf, log);
 	}
 }

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 01:11 UTC #2 lore raw

Re: [PATCH bpf-next v11 01/13] bpf: refactory struct_ops type initialization to a function.

Quoted 33 lines · On 11/6/23 12:12 PM, [email protected] wrote:

kernel/bpf/bpf_struct_ops.c +11 -8

index db6176fb64dc..627cf1ea840a 100644
@@ -110,102 +110,111 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = {
 
  static const struct btf_type *module_type;
 
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
+static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
+				    struct btf *btf,
+				    struct bpf_verifier_log *log)
  {
-	s32 type_id, value_id, module_id;
  	const struct btf_member *member;
-	struct bpf_struct_ops *st_ops;
  	const struct btf_type *t;
+	s32 type_id, value_id;
  	char value_name[128];
  	const char *mname;
-	u32 i, j;
+	int i;
 
-	/* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */
-#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct bpf_struct_ops_##_name);
-#include "bpf_struct_ops_types.h"
-#undef BPF_STRUCT_OPS_TYPE
+	if (strlen(st_ops->name) + VALUE_PREFIX_LEN >=
+	    sizeof(value_name)) {
+		pr_warn("struct_ops name %s is too long\n",
+			st_ops->name);
+		return;
+	}

kernel/bpf/bpf_struct_ops.c const struct bpf_prog_ops bpf_struct_ops_prog_ops = { · line 136

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +6 -3

+	sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name);
 
-	module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT);
-	if (module_id < 0) {
-		pr_warn("Cannot find struct module in btf_vmlinux\n");
+	value_id = btf_find_by_name_kind(btf, value_name,
+					 BTF_KIND_STRUCT);
+	if (value_id < 0) {
+		pr_warn("Cannot find struct %s in btf_vmlinux\n",
+			value_name);

"btf_vmlinux" needs to change in the pr_warn(). It should be btf->name but that may need a helper function to return btf->name.

kernel/bpf/bpf_struct_ops.c const struct bpf_prog_ops bpf_struct_ops_prog_ops = { · line 143

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +4 -3

  		return;
  	}
-	module_type = btf_type_by_id(btf, module_id);
 
-	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {
-		st_ops = bpf_struct_ops[i];
+	type_id = btf_find_by_name_kind(btf, st_ops->name,
+					BTF_KIND_STRUCT);
+	if (type_id < 0) {
+		pr_warn("Cannot find struct %s in btf_vmlinux\n",

Same here.

Kui-Feng Lee

2023-11-21 23:53 UTC #3 lore raw

Re: [PATCH bpf-next v11 01/13] bpf: refactory struct_ops type initialization to a function.

Quoted 36 lines · Martin KaFai Lau

Quoted 35 lines · On 11/6/23 12:12 PM, [email protected] wrote:

kernel/bpf/bpf_struct_ops.c +12 ~~-10~~

index db6176fb64dc..627cf1ea840a 100644
@@ -110,102 +110,111 @@ const struct bpf_prog_ops 
 bpf_struct_ops_prog_ops = {
   static const struct btf_type *module_type;
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
+static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
+                    struct btf *btf,
+                    struct bpf_verifier_log *log)
   {
-    s32 type_id, value_id, module_id;
       const struct btf_member *member;
-    struct bpf_struct_ops *st_ops;
       const struct btf_type *t;
+    s32 type_id, value_id;
       char value_name[128];
       const char *mname;
-    u32 i, j;
+    int i;
-    /* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */
-#define BPF_STRUCT_OPS_TYPE(_name) BTF_TYPE_EMIT(struct 
 bpf_struct_ops_##_name);
-#include "bpf_struct_ops_types.h"
-#undef BPF_STRUCT_OPS_TYPE
+    if (strlen(st_ops->name) + VALUE_PREFIX_LEN >=
+        sizeof(value_name)) {
+        pr_warn("struct_ops name %s is too long\n",
+            st_ops->name);
+        return;
+    }
+    sprintf(value_name, "%s%s", VALUE_PREFIX, st_ops->name);
-    module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT);
-    if (module_id < 0) {

Quoted 8 lines

kernel/bpf/bpf_struct_ops.c const struct bpf_prog_ops bpf_struct_ops_prog_ops = { · line 136
Quoted 6 lines
kernel/bpf/bpf_struct_ops.c +5 -1
-        pr_warn("Cannot find struct module in btf_vmlinux\n");
+    value_id = btf_find_by_name_kind(btf, value_name,
+                     BTF_KIND_STRUCT);
+    if (value_id < 0) {
+        pr_warn("Cannot find struct %s in btf_vmlinux\n",
+            value_name);
"btf_vmlinux" needs to change in the pr_warn(). It should be btf->name but that may need a helper function to return btf->name.

Right! I will add a helper function to return the name.

Quoted 12 lines

kernel/bpf/bpf_struct_ops.c const struct bpf_prog_ops bpf_struct_ops_prog_ops = { · line 143

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +4 -3

           return;
       }
-    module_type = btf_type_by_id(btf, module_id);
-    for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {
-        st_ops = bpf_struct_ops[i];
+    type_id = btf_find_by_name_kind(btf, st_ops->name,
+                    BTF_KIND_STRUCT);
+    if (type_id < 0) {
+        pr_warn("Cannot find struct %s in btf_vmlinux\n",

Same here.

[email protected]

author

2023-11-06 20:13 UTC #4 lore raw

PATCH v11 2/13 bpf: get type information with BPF_ID_LIST

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Get ready to remove bpf_struct_ops_init() in the future. By using BPF_ID_LIST, it is possible to gather type information while building instead of runtime.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/bpf_struct_ops.c | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

kernel/bpf/bpf_struct_ops.c +8 -9

index 627cf1ea840a..4ca4ca4998e0 100644
@@ -108,7 +108,12 @@ const struct bpf_prog_ops bpf_struct_ops_prog_ops = {
 #endif
 };
 
-static const struct btf_type *module_type;
+BTF_ID_LIST(st_ops_ids)
+BTF_ID(struct, module)
+
+enum {
+	IDX_MODULE_ID,
+};
 
 static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
 				    struct btf *btf,

@@ -197,7 +202,6 @@ static void bpf_struct_ops_init_one(struct bpf_struct_ops *st_ops,
 void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
 {
 	struct bpf_struct_ops *st_ops;
-	s32 module_id;
 	u32 i;
 
 	/* Ensure BTF type is emitted for "struct bpf_struct_ops_##_name" */

@@ -205,13 +209,6 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
 #include "bpf_struct_ops_types.h"
 #undef BPF_STRUCT_OPS_TYPE
 
-	module_id = btf_find_by_name_kind(btf, "module", BTF_KIND_STRUCT);
-	if (module_id < 0) {
-		pr_warn("Cannot find struct module in btf_vmlinux\n");
-		return;
-	}
-	module_type = btf_type_by_id(btf, module_id);
-
 	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {
 		st_ops = bpf_struct_ops[i];
 		bpf_struct_ops_init_one(st_ops, btf, log);

@@ -381,6 +378,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map;
 	const struct bpf_struct_ops *st_ops = st_map->st_ops;
 	struct bpf_struct_ops_value *uvalue, *kvalue;
+	const struct btf_type *module_type;
 	const struct btf_member *member;
 	const struct btf_type *t = st_ops->type;
 	struct bpf_tramp_links *tlinks;

@@ -428,6 +426,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	image = st_map->image;
 	image_end = st_map->image + PAGE_SIZE;
 
+	module_type = btf_type_by_id(btf_vmlinux, st_ops_ids[IDX_MODULE_ID]);
 	for_each_member(i, t, member) {
 		const struct btf_type *mtype, *ptype;
 		struct bpf_prog *prog;

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #5 lore raw

PATCH v11 3/13 bpf, net: introduce bpf_struct_ops_desc.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Move some of members of bpf_struct_ops to bpf_struct_ops_desc. When we introduce the new API to register new bpf_struct_ops types from modules, bpf_struct_ops may destroyed when the module is unloaded. Moving these members to bpf_struct_ops_desc make these data available even when the module is unloaded.

type_id is unavailabe in bpf_struct_ops anymore. Modules should get it from the btf received by kmod's init function.

Cc [email protected]

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h            | 13 ++++--
 kernel/bpf/bpf_struct_ops.c    | 80 +++++++++++++++++-----------------
 kernel/bpf/verifier.c          |  8 ++--
 net/bpf/bpf_dummy_struct_ops.c |  9 +++-
 net/ipv4/bpf_tcp_ca.c          |  8 +++-
 5 files changed, 70 insertions(+), 48 deletions(-)

include/linux/bpf.h +9 -4

index b4825d3cdb29..b55e27162df0 100644
@@ -1626,17 +1626,22 @@ struct bpf_struct_ops {
 	void (*unreg)(void *kdata);
 	int (*update)(void *kdata, void *old_kdata);
 	int (*validate)(void *kdata);
-	const struct btf_type *type;
-	const struct btf_type *value_type;
 	const char *name;
 	struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS];
+};
+
+struct bpf_struct_ops_desc {
+	struct bpf_struct_ops *st_ops;
+
+	const struct btf_type *type;
+	const struct btf_type *value_type;
 	u32 type_id;
 	u32 value_id;
 };
 
 #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
 #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
-const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id);
+const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id);
 void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
 bool bpf_struct_ops_get(const void *kdata);
 void bpf_struct_ops_put(const void *kdata);

@@ -1679,7 +1684,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 			    union bpf_attr __user *uattr);
 #endif
 #else
-static inline const struct bpf_struct_ops *bpf_struct_ops_find(u32 type_id)
+static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id)
 {
 	return NULL;
 }

kernel/bpf/bpf_struct_ops.c +41 ~~-39~~

230 lines not on this page · raw message

kernel/bpf/verifier.c +5 -3

index 857d76694517..290e3a7ee72f 100644
@@ -20081,6 +20081,7 @@ static void print_verification_stats(struct bpf_verifier_env *env)
 static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 {
 	const struct btf_type *t, *func_proto;
+	const struct bpf_struct_ops_desc *st_ops_desc;
 	const struct bpf_struct_ops *st_ops;
 	const struct btf_member *member;
 	struct bpf_prog *prog = env->prog;

@@ -20093,14 +20094,15 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	btf_id = prog->aux->attach_btf_id;
-	st_ops = bpf_struct_ops_find(btf_id);
-	if (!st_ops) {
+	st_ops_desc = bpf_struct_ops_find(btf_id);
+	if (!st_ops_desc) {
 		verbose(env, "attach_btf_id %u is not a supported struct\n",
 			btf_id);
 		return -ENOTSUPP;
 	}
+	st_ops = st_ops_desc->st_ops;
 
-	t = st_ops->type;
+	t = st_ops_desc->type;
 	member_idx = prog->expected_attach_type;
 	if (member_idx >= btf_type_vlen(t)) {
 		verbose(env, "attach to invalid member idx %u of struct %s\n",

net/bpf/bpf_dummy_struct_ops.c +8 -1

index 5918d1b32e19..ffa224053a6c 100644
@@ -17,6 +17,8 @@ struct bpf_dummy_ops_test_args {
 	struct bpf_dummy_ops_state state;
 };
 
+static struct btf *bpf_dummy_ops_btf;
+
 static struct bpf_dummy_ops_test_args *
 dummy_ops_init_args(const union bpf_attr *kattr, unsigned int nr)
 {

@@ -85,9 +87,13 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 	void *image = NULL;
 	unsigned int op_idx;
 	int prog_ret;
+	u32 type_id;
 	int err;
 
-	if (prog->aux->attach_btf_id != st_ops->type_id)
+	type_id = btf_find_by_name_kind(bpf_dummy_ops_btf,
+					bpf_bpf_dummy_ops.name,
+					BTF_KIND_STRUCT);
+	if (prog->aux->attach_btf_id != type_id)
 		return -EOPNOTSUPP;
 
 	func_proto = prog->aux->attach_func_proto;

@@ -143,6 +149,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 
 static int bpf_dummy_init(struct btf *btf)
 {
+	bpf_dummy_ops_btf = btf;
 	return 0;
 }

net/ipv4/bpf_tcp_ca.c +7 -1

index 39dcccf0f174..3c8b76578a2a 100644
@@ -20,6 +20,7 @@ static u32 unsupported_ops[] = {
 
 static const struct btf_type *tcp_sock_type;
 static u32 tcp_sock_id, sock_id;
+static const struct btf_type *tcp_congestion_ops_type;
 
 static int bpf_tcp_ca_init(struct btf *btf)
 {

@@ -36,6 +37,11 @@ static int bpf_tcp_ca_init(struct btf *btf)
 	tcp_sock_id = type_id;
 	tcp_sock_type = btf_type_by_id(btf, tcp_sock_id);
 
+	type_id = btf_find_by_name_kind(btf, "tcp_congestion_ops", BTF_KIND_STRUCT);
+	if (type_id < 0)
+		return -EINVAL;
+	tcp_congestion_ops_type = btf_type_by_id(btf, type_id);
+
 	return 0;
 }

@@ -149,7 +155,7 @@ static u32 prog_ops_moff(const struct bpf_prog *prog)
 	u32 midx;
 
 	midx = prog->expected_attach_type;
-	t = bpf_tcp_congestion_ops.type;
+	t = tcp_congestion_ops_type;
 	m = &btf_type_member(t)[midx];
 
 	return __btf_member_bit_offset(t, m) / 8;

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #6 lore raw

PATCH v11 4/13 bpf: add struct_ops_tab to btf.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Maintain a registry of registered struct_ops types in the per-btf (module) struct_ops_tab. This registry allows for easy lookup of struct_ops types that are registered by a specific module.

It is a preparation work for supporting kernel module struct_ops in a latter patch. Each struct_ops will be registered under its own kernel module btf and will be stored in the newly added btf->struct_ops_tab. The bpf verifier and bpf syscall (e.g. prog and map cmd) can find the struct_ops and its btf type/size/id... information from btf->struct_ops_tab.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/btf.h |  8 +++++
 kernel/bpf/btf.c    | 83 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)

include/linux/btf.h +8 -0

index c2231c64d60b..07ee6740e06a 100644
@@ -572,4 +572,12 @@ static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type
 	return btf_type_is_struct(t);
 }
 
+#ifdef CONFIG_BPF_JIT
+struct bpf_struct_ops_desc;
+
+const struct bpf_struct_ops_desc *
+btf_get_struct_ops(struct btf *btf, u32 *ret_cnt);
+
+#endif /* CONFIG_BPF_JIT */
+
 #endif

kernel/bpf/btf.c +83 -0

109 lines not on this page · raw message

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 01:35 UTC #7 lore raw

Re: [PATCH bpf-next v11 04/13] bpf: add struct_ops_tab to btf.

Quoted 24 lines · On 11/6/23 12:12 PM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Maintain a registry of registered struct_ops types in the per-btf (module) struct_ops_tab. This registry allows for easy lookup of struct_ops types that are registered by a specific module.
It is a preparation work for supporting kernel module struct_ops in a latter patch. Each struct_ops will be registered under its own kernel module btf and will be stored in the newly added btf->struct_ops_tab. The bpf verifier and bpf syscall (e.g. prog and map cmd) can find the struct_ops and its btf type/size/id... information from btf->struct_ops_tab.
Signed-off-by Kui-Feng Lee <[email protected]>
---
  include/linux/btf.h |  8 +++++
  kernel/bpf/btf.c    | 83 +++++++++++++++++++++++++++++++++++++++++++++
  2 files changed, 91 insertions(+)
include/linux/btf.h +0 -0
index c2231c64d60b..07ee6740e06a 100644

include/linux/btf.h static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type · line 575

Quoted 4 lines

include/linux/btf.h +1 -0

@@ -572,4 +572,12 @@ static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type
  	return btf_type_is_struct(t);
  }
 
+#ifdef CONFIG_BPF_JIT

There are many new ifdef CONFIG_BPF_JIT in btf.{h,c}. Could it be avoided? For example, having an empty bpf_struct_ops_desc_init() for the not CONFIG_BPF_JIT case, is it enough?

Quoted 8 lines

include/linux/btf.h +7 -0

+struct bpf_struct_ops_desc;
+
+const struct bpf_struct_ops_desc *
+btf_get_struct_ops(struct btf *btf, u32 *ret_cnt);
+
+#endif /* CONFIG_BPF_JIT */
+
  #endif

Kui-Feng Lee

2023-11-22 02:27 UTC #8 lore raw

Re: [PATCH bpf-next v11 04/13] bpf: add struct_ops_tab to btf.

Quoted 20 lines · Martin KaFai Lau

Quoted 19 lines · On 11/6/23 12:12 PM, [email protected] wrote:
From: Kui-Feng Lee <[email protected]>
Maintain a registry of registered struct_ops types in the per-btf (module) struct_ops_tab. This registry allows for easy lookup of struct_ops types that are registered by a specific module.
It is a preparation work for supporting kernel module struct_ops in a latter patch. Each struct_ops will be registered under its own kernel module btf and will be stored in the newly added btf->struct_ops_tab. The bpf verifier and bpf syscall (e.g. prog and map cmd) can find the struct_ops and its btf type/size/id... information from btf->struct_ops_tab.
Signed-off-by Kui-Feng Lee <[email protected]>
---
include/linux/btf.h | 8 +++++
kernel/bpf/btf.c | 83 +++++++++++++++++++++++++++++++++++++++++++++
  2 files changed, 91 insertions(+)

Quoted 11 lines

Quoted 9 lines
include/linux/btf.h +0 -0
index c2231c64d60b..07ee6740e06a 100644
@@ -572,4 +572,12 @@ static inline bool btf_type_is_struct_ptr(struct 
btf *btf, const struct btf_type
return btf_type_is_struct(t);
}
+#ifdef CONFIG_BPF_JIT
There are many new ifdef CONFIG_BPF_JIT in btf.{h,c}. Could it be avoided? For example, having an empty bpf_struct_ops_desc_init() for the not CONFIG_BPF_JIT case, is it enough?

It is enough. However, it also leaves dead code. Anyway, I just removed these conditions as you said.

Quoted 11 lines

Quoted 10 lines

include/linux/btf.h +7 -0

+struct bpf_struct_ops_desc;
+
+const struct bpf_struct_ops_desc *
+btf_get_struct_ops(struct btf *btf, u32 *ret_cnt);
+
+#endif /* CONFIG_BPF_JIT */
+

#endif

[email protected]

author

2023-11-06 20:13 UTC #9 lore raw

PATCH v11 5/13 bpf: make struct_ops_map support btfs other than btf_vmlinux.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Once new struct_ops can be registered from modules, btf_vmlinux is not longer the only btf tht struct_ops_map would face. st_map should remember what btf it should use to get type information.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/bpf_struct_ops.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

kernel/bpf/bpf_struct_ops.c +15 -9

index d804801c7864..a0291877a792 100644
@@ -46,6 +46,8 @@ struct bpf_struct_ops_map {
 	 * "links[]".
 	 */
 	void *image;
+	/* The owner moduler's btf. */
+	struct btf *btf;
 	/* uvalue->data stores the kernel struct
 	 * (e.g. tcp_congestion_ops) that is more useful
 	 * to userspace than the kvalue.  For example,

@@ -314,7 +316,7 @@ static void bpf_struct_ops_map_put_progs(struct bpf_struct_ops_map *st_map)
 	}
 }
 
-static int check_zero_holes(const struct btf_type *t, void *data)
+static int check_zero_holes(const struct btf *btf, const struct btf_type *t, void *data)
 {
 	const struct btf_member *member;
 	u32 i, moff, msize, prev_mend = 0;

@@ -326,8 +328,8 @@ static int check_zero_holes(const struct btf_type *t, void *data)
 		    memchr_inv(data + prev_mend, 0, moff - prev_mend))
 			return -EINVAL;
 
-		mtype = btf_type_by_id(btf_vmlinux, member->type);
-		mtype = btf_resolve_size(btf_vmlinux, mtype, &msize);
+		mtype = btf_type_by_id(btf, member->type);
+		mtype = btf_resolve_size(btf, mtype, &msize);
 		if (IS_ERR(mtype))
 			return PTR_ERR(mtype);
 		prev_mend = moff + msize;

@@ -395,12 +397,12 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	if (*(u32 *)key != 0)
 		return -E2BIG;
 
-	err = check_zero_holes(st_ops_desc->value_type, value);
+	err = check_zero_holes(st_map->btf, st_ops_desc->value_type, value);
 	if (err)
 		return err;
 
 	uvalue = value;
-	err = check_zero_holes(t, uvalue->data);
+	err = check_zero_holes(st_map->btf, t, uvalue->data);
 	if (err)
 		return err;

@@ -436,7 +438,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 		u32 moff;
 
 		moff = __btf_member_bit_offset(t, member) / 8;
-		ptype = btf_type_resolve_ptr(btf_vmlinux, member->type, NULL);
+		ptype = btf_type_resolve_ptr(st_map->btf, member->type, NULL);
 		if (ptype == module_type) {
 			if (*(void **)(udata + moff))
 				goto reset_unlock;

@@ -461,8 +463,8 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 		if (!ptype || !btf_type_is_func_proto(ptype)) {
 			u32 msize;
 
-			mtype = btf_type_by_id(btf_vmlinux, member->type);
-			mtype = btf_resolve_size(btf_vmlinux, mtype, &msize);
+			mtype = btf_type_by_id(st_map->btf, member->type);
+			mtype = btf_resolve_size(st_map->btf, mtype, &msize);
 			if (IS_ERR(mtype)) {
 				err = PTR_ERR(mtype);
 				goto reset_unlock;

@@ -601,6 +603,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key)
 static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key,
 					     struct seq_file *m)
 {
+	struct bpf_struct_ops_map *st_map = (struct bpf_struct_ops_map *)map;
 	void *value;
 	int err;

@@ -610,7 +613,8 @@ static void bpf_struct_ops_map_seq_show_elem(struct bpf_map *map, void *key,
 
 	err = bpf_struct_ops_map_sys_lookup_elem(map, key, value);
 	if (!err) {
-		btf_type_seq_show(btf_vmlinux, map->btf_vmlinux_value_type_id,
+		btf_type_seq_show(st_map->btf,
+				  map->btf_vmlinux_value_type_id,
 				  value, m);
 		seq_puts(m, "\n");
 	}

@@ -720,6 +724,8 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 		return ERR_PTR(-ENOMEM);
 	}
 
+	st_map->btf = btf_vmlinux;
+
 	mutex_init(&st_map->lock);
 	set_vm_flush_reset_perms(st_map->image);
 	bpf_map_init_from_attr(map, attr);

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 01:40 UTC #10 lore raw

Re: [PATCH bpf-next v11 05/13] bpf: make struct_ops_map support btfs other than btf_vmlinux.

Quoted 3 lines · On 11/6/23 12:12 PM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Once new struct_ops can be registered from modules, btf_vmlinux is not

s/not/no/ longer the only ...

Quoted 1 line

longer the only btf tht struct_ops_map would face. st_map should remember

s/tht/that/ (?)

Quoted 1 line

what btf it should use to get type information.

Kui-Feng Lee

2023-11-22 02:28 UTC #11 lore raw

Re: [PATCH bpf-next v11 05/13] bpf: make struct_ops_map support btfs other than btf_vmlinux.

Quoted 14 lines · Martin KaFai Lau

Quoted 3 lines · On 11/6/23 12:12 PM, [email protected] wrote:
From: Kui-Feng Lee <[email protected]>
Once new struct_ops can be registered from modules, btf_vmlinux is not
s/not/no/ longer the only ...
Quoted 1 line
longer the only btf tht struct_ops_map would face. st_map should remember
s/tht/that/ (?)
Quoted 1 line
what btf it should use to get type information.

Fixed! Thanks!

[email protected]

author

2023-11-06 20:13 UTC #12 lore raw

PATCH v11 6/13 bpf: lookup struct_ops types from a given module BTF.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

This is a preparation for searching for struct_ops types from a specified module. BTF is always btf_vmlinux now. This patch passes a pointer of BTF to bpf_struct_ops_find_value() and bpf_struct_ops_find(). Once the new registration API of struct_ops types is used, other BTFs besides btf_vmlinux can also be passed to them.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h         |  4 ++--
 kernel/bpf/bpf_struct_ops.c | 11 ++++++-----
 kernel/bpf/verifier.c       |  2 +-
 3 files changed, 9 insertions(+), 8 deletions(-)

include/linux/bpf.h +2 -2

index b55e27162df0..f0ed874d5ac3 100644
@@ -1641,7 +1641,7 @@ struct bpf_struct_ops_desc {
 
 #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
 #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
-const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id);
+const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id);
 void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
 bool bpf_struct_ops_get(const void *kdata);
 void bpf_struct_ops_put(const void *kdata);

@@ -1684,7 +1684,7 @@ int bpf_struct_ops_test_run(struct bpf_prog *prog, const union bpf_attr *kattr,
 			    union bpf_attr __user *uattr);
 #endif
 #else
-static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id)
+static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id)
 {
 	return NULL;
 }

kernel/bpf/bpf_struct_ops.c +6 -5

index a0291877a792..4ba6181ed1c4 100644
@@ -221,11 +221,11 @@ void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log)
 extern struct btf *btf_vmlinux;
 
 static const struct bpf_struct_ops_desc *
-bpf_struct_ops_find_value(u32 value_id)
+bpf_struct_ops_find_value(struct btf *btf, u32 value_id)
 {
 	unsigned int i;
 
-	if (!value_id || !btf_vmlinux)
+	if (!value_id || !btf)
 		return NULL;
 
 	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {

@@ -236,11 +236,12 @@ bpf_struct_ops_find_value(u32 value_id)
 	return NULL;
 }
 
-const struct bpf_struct_ops_desc *bpf_struct_ops_find(u32 type_id)
+const struct bpf_struct_ops_desc *
+bpf_struct_ops_find(struct btf *btf, u32 type_id)
 {
 	unsigned int i;
 
-	if (!type_id || !btf_vmlinux)
+	if (!type_id || !btf)
 		return NULL;
 
 	for (i = 0; i < ARRAY_SIZE(bpf_struct_ops); i++) {

@@ -676,7 +677,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 	struct bpf_map *map;
 	int ret;
 
-	st_ops_desc = bpf_struct_ops_find_value(attr->btf_vmlinux_value_type_id);
+	st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, attr->btf_vmlinux_value_type_id);
 	if (!st_ops_desc)
 		return ERR_PTR(-ENOTSUPP);

kernel/bpf/verifier.c +1 -1

index 290e3a7ee72f..bdd166cab977 100644
@@ -20094,7 +20094,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	btf_id = prog->aux->attach_btf_id;
-	st_ops_desc = bpf_struct_ops_find(btf_id);
+	st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id);
 	if (!st_ops_desc) {
 		verbose(env, "attach_btf_id %u is not a supported struct\n",
 			btf_id);

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #13 lore raw

PATCH v11 7/13 bpf: pass attached BTF to the bpf_struct_ops subsystem

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Every kernel module has its BTF, comprising information on types defined in the module. The BTF fd (attr->value_type_btf_obj_fd) passed from userspace helps the bpf_struct_ops to lookup type information and description of the struct_ops type, which is necessary for parsing the layout of map element values and registering maps. The descriptions are looked up by matching a type id (attr->btf_vmlinux_value_type_id) against bpf_struct_ops_desc(s) defined in a BTF. If the struct_ops type is defined in a module, the bpf_struct_ops needs to know the module BTF to lookup the bpf_struct_ops_desc.

The bpf_prog includes attach_btf in aux which is passed along with the bpf_attr when loading the program. The purpose of attach_btf is to determine the btf type of attach_btf_id. The attach_btf_id is then used to identify the traced function for a trace program. In the case of struct_ops programs, it is used to identify the struct_ops type of the struct_ops object that a program is attached to.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/uapi/linux/bpf.h       |  5 +++
 kernel/bpf/bpf_struct_ops.c    | 57 ++++++++++++++++++++++++----------
 kernel/bpf/syscall.c           |  2 +-
 kernel/bpf/verifier.c          |  9 ++++--
 tools/include/uapi/linux/bpf.h |  5 +++
 5 files changed, 57 insertions(+), 21 deletions(-)

include/uapi/linux/bpf.h +5 -0

index 0f6cdf52b1da..fd20c52606b2 100644
@@ -1398,6 +1398,11 @@ union bpf_attr {
 		 * to using 5 hash functions).
 		 */
 		__u64	map_extra;
+
+		__u32   value_type_btf_obj_fd;	/* fd pointing to a BTF
+						 * type data for
+						 * btf_vmlinux_value_type_id.
+						 */
 	};
 
 	struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

kernel/bpf/bpf_struct_ops.c +40 ~~-17~~

103 lines not on this page · raw message

kernel/bpf/syscall.c +1 -1

index 0ed286b8a0f0..974651fe2bee 100644
@@ -1096,7 +1096,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,
 	return ret;
 }
 
-#define BPF_MAP_CREATE_LAST_FIELD map_extra
+#define BPF_MAP_CREATE_LAST_FIELD value_type_btf_obj_fd
 /* called via syscall */
 static int map_create(union bpf_attr *attr)
 {

kernel/bpf/verifier.c +6 -3

index bdd166cab977..3f446f76d4bf 100644
@@ -20086,6 +20086,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	const struct btf_member *member;
 	struct bpf_prog *prog = env->prog;
 	u32 btf_id, member_idx;
+	struct btf *btf;
 	const char *mname;
 
 	if (!prog->gpl_compatible) {

@@ -20093,8 +20094,10 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 		return -EINVAL;
 	}
 
+	btf = prog->aux->attach_btf;
+
 	btf_id = prog->aux->attach_btf_id;
-	st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id);
+	st_ops_desc = bpf_struct_ops_find(btf, btf_id);
 	if (!st_ops_desc) {
 		verbose(env, "attach_btf_id %u is not a supported struct\n",
 			btf_id);

@@ -20111,8 +20114,8 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	member = &btf_type_member(t)[member_idx];
-	mname = btf_name_by_offset(btf_vmlinux, member->name_off);
-	func_proto = btf_type_resolve_func_ptr(btf_vmlinux, member->type,
+	mname = btf_name_by_offset(btf, member->name_off);
+	func_proto = btf_type_resolve_func_ptr(btf, member->type,
 					       NULL);
 	if (!func_proto) {
 		verbose(env, "attach to invalid member %s(@idx %u) of struct %s\n",

tools/include/uapi/linux/bpf.h +5 -0

index 0f6cdf52b1da..fd20c52606b2 100644
@@ -1398,6 +1398,11 @@ union bpf_attr {
 		 * to using 5 hash functions).
 		 */
 		__u64	map_extra;
+
+		__u32   value_type_btf_obj_fd;	/* fd pointing to a BTF
+						 * type data for
+						 * btf_vmlinux_value_type_id.
+						 */
 	};
 
 	struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 02:04 UTC #14 lore raw

Re: [PATCH bpf-next v11 07/13] bpf: pass attached BTF to the bpf_struct_ops subsystem

Quoted 4 lines · On 11/6/23 12:12 PM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Every kernel module has its BTF, comprising information on types defined in the module. The BTF fd (attr->value_type_btf_obj_fd) passed from userspace

I would highlight this patch (adds) value_type_btf_obj_fd.

Quoted 10 lines

helps the bpf_struct_ops to lookup type information and description of the struct_ops type, which is necessary for parsing the layout of map element values and registering maps. The descriptions are looked up by matching a type id (attr->btf_vmlinux_value_type_id) against bpf_struct_ops_desc(s) defined in a BTF. If the struct_ops type is defined in a module, the bpf_struct_ops needs to know the module BTF to lookup the bpf_struct_ops_desc.
The bpf_prog includes attach_btf in aux which is passed along with the bpf_attr when loading the program. The purpose of attach_btf is to

I read it as "attach_btf" is passed in the bpf_attr. This has been in my head for a while. I sort of know what is the actual uapi, so didn't get to it yet.

We have already discussed a bit of this offline. I think it meant attr->attach_btf_obj_fd here.

This patch is mainly about how the userspace passing kmod's btf to the kernel during map creation and prog load and also what uapi does it use. The commit message should mention this patch is reusing the existing attr->attach_btf_obj_fd for the userspace to pass the kmod's btf when loading the struct_ops prog. I need to go back to the syscall.c code to figure out and also leap forward to the later libbpf patch to confirm it.

I depend on the commit message to help the review. It is much appreciated if the commit message is clear and accurate on things like: what it wants to do, how it does it (addition/deletion/changes), and what are the major changes.

Quoted 80 lines

determine the btf type of attach_btf_id. The attach_btf_id is then used to identify the traced function for a trace program. In the case of struct_ops programs, it is used to identify the struct_ops type of the struct_ops object that a program is attached to.

Signed-off-by Kui-Feng Lee <[email protected]>

---
  include/uapi/linux/bpf.h       |  5 +++
  kernel/bpf/bpf_struct_ops.c    | 57 ++++++++++++++++++++++++----------
  kernel/bpf/syscall.c           |  2 +-
  kernel/bpf/verifier.c          |  9 ++++--
  tools/include/uapi/linux/bpf.h |  5 +++
  5 files changed, 57 insertions(+), 21 deletions(-)

include/uapi/linux/bpf.h +5 -0

index 0f6cdf52b1da..fd20c52606b2 100644
@@ -1398,6 +1398,11 @@ union bpf_attr {
  		 * to using 5 hash functions).
  		 */
  		__u64	map_extra;
+
+		__u32   value_type_btf_obj_fd;	/* fd pointing to a BTF
+						 * type data for
+						 * btf_vmlinux_value_type_id.
+						 */
  	};
 
  	struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

kernel/bpf/bpf_struct_ops.c +21 -5

index 4ba6181ed1c4..2fb1b21f989a 100644
@@ -635,6 +635,7 @@ static void __bpf_struct_ops_map_free(struct bpf_map *map)
  		bpf_jit_uncharge_modmem(PAGE_SIZE);
  	}
  	bpf_map_area_free(st_map->uvalue);
+	btf_put(st_map->btf);
  	bpf_map_area_free(st_map);
  }

@@ -675,15 +676,30 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
  	struct bpf_struct_ops_map *st_map;
  	const struct btf_type *t, *vt;
  	struct bpf_map *map;
+	struct btf *btf;
  	int ret;
 
-	st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, attr->btf_vmlinux_value_type_id);
-	if (!st_ops_desc)
-		return ERR_PTR(-ENOTSUPP);
+	if (attr->value_type_btf_obj_fd) {
+		/* The map holds btf for its whole life time. */
+		btf = btf_get_by_fd(attr->value_type_btf_obj_fd);
+		if (IS_ERR(btf))
+			return ERR_PTR(PTR_ERR(btf));
+	} else {
+		btf = btf_vmlinux;
+		btf_get(btf);
+	}
+
+	st_ops_desc = bpf_struct_ops_find_value(btf, attr->btf_vmlinux_value_type_id);
+	if (!st_ops_desc) {
+		ret = -ENOTSUPP;
+		goto errout;
+	}
 
  	vt = st_ops_desc->value_type;
-	if (attr->value_size != vt->size)
-		return ERR_PTR(-EINVAL);
+	if (attr->value_size != vt->size) {
+		ret = -EINVAL;
+		goto errout;
+	}
 
  	t = st_ops_desc->type;

@@ -694,17 +710,18 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
  		(vt->size - sizeof(struct bpf_struct_ops_value));

kernel/bpf/bpf_struct_ops.c static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) · line 718

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +5 -2

 
  	st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE);
-	if (!st_map)
-		return ERR_PTR(-ENOMEM);
+	if (!st_map) {
+		ret = -ENOMEM;
+		goto errout;
+	}
 
+	st_map->btf = btf;

How about do the "st_map->btf = btf;" assignment the same as where the current code is doing (a few lines below). Would it avoid the new "btf = NULL;" dance during the error case?

nit, if moving a line, I would move the following "st_map->st_ops_desc = st_ops_desc;" to the later and close to where "st_map->btf = btf;" is.

Quoted 24 lines

kernel/bpf/bpf_struct_ops.c +4 -6

  	st_map->st_ops_desc = st_ops_desc;
  	map = &st_map->map;
 
  	ret = bpf_jit_charge_modmem(PAGE_SIZE);
-	if (ret) {
-		__bpf_struct_ops_map_free(map);
-		return ERR_PTR(ret);
-	}
+	if (ret)
+		goto errout_free;
 
  	st_map->image = bpf_jit_alloc_exec(PAGE_SIZE);
  	if (!st_map->image) {

@@ -713,25 +730,31 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
  		 * here.
  		 */
  		bpf_jit_uncharge_modmem(PAGE_SIZE);
-		__bpf_struct_ops_map_free(map);
-		return ERR_PTR(-ENOMEM);
+		ret = -ENOMEM;
+		goto errout_free;
  	}
  	st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE);
  	st_map->links =

kernel/bpf/bpf_struct_ops.c static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr) · line 728

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +2 -3

  		bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_links *),
  				   NUMA_NO_NODE);
  	if (!st_map->uvalue || !st_map->links) {
-		__bpf_struct_ops_map_free(map);
-		return ERR_PTR(-ENOMEM);
+		ret = -ENOMEM;
+		goto errout_free;
  	}
 
-	st_map->btf = btf_vmlinux;

The old code initializes "st_map->btf" here.

Quoted 81 lines

kernel/bpf/bpf_struct_ops.c +8 -1

-
  	mutex_init(&st_map->lock);
  	set_vm_flush_reset_perms(st_map->image);
  	bpf_map_init_from_attr(map, attr);
 
  	return map;
+
+errout_free:
+	__bpf_struct_ops_map_free(map);
+	btf = NULL;		/* has been released */
+errout:
+	btf_put(btf);
+
+	return ERR_PTR(ret);
  }
 
  static u64 bpf_struct_ops_map_mem_usage(const struct bpf_map *map)

kernel/bpf/syscall.c +1 -1

index 0ed286b8a0f0..974651fe2bee 100644
@@ -1096,7 +1096,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,
  	return ret;
  }
 
-#define BPF_MAP_CREATE_LAST_FIELD map_extra
+#define BPF_MAP_CREATE_LAST_FIELD value_type_btf_obj_fd
  /* called via syscall */
  static int map_create(union bpf_attr *attr)
  {

kernel/bpf/verifier.c +6 -3

index bdd166cab977..3f446f76d4bf 100644
@@ -20086,6 +20086,7 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
  	const struct btf_member *member;
  	struct bpf_prog *prog = env->prog;
  	u32 btf_id, member_idx;
+	struct btf *btf;
  	const char *mname;
 
  	if (!prog->gpl_compatible) {

@@ -20093,8 +20094,10 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
  		return -EINVAL;
  	}
 
+	btf = prog->aux->attach_btf;
+
  	btf_id = prog->aux->attach_btf_id;
-	st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id);
+	st_ops_desc = bpf_struct_ops_find(btf, btf_id);
  	if (!st_ops_desc) {
  		verbose(env, "attach_btf_id %u is not a supported struct\n",
  			btf_id);

@@ -20111,8 +20114,8 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
  	}
 
  	member = &btf_type_member(t)[member_idx];
-	mname = btf_name_by_offset(btf_vmlinux, member->name_off);
-	func_proto = btf_type_resolve_func_ptr(btf_vmlinux, member->type,
+	mname = btf_name_by_offset(btf, member->name_off);
+	func_proto = btf_type_resolve_func_ptr(btf, member->type,
  					       NULL);
  	if (!func_proto) {
  		verbose(env, "attach to invalid member %s(@idx %u) of struct %s\n",

tools/include/uapi/linux/bpf.h +5 -0

index 0f6cdf52b1da..fd20c52606b2 100644
@@ -1398,6 +1398,11 @@ union bpf_attr {
  		 * to using 5 hash functions).
  		 */
  		__u64	map_extra;
+
+		__u32   value_type_btf_obj_fd;	/* fd pointing to a BTF
+						 * type data for
+						 * btf_vmlinux_value_type_id.
+						 */
  	};
 
  	struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

Kui-Feng Lee

2023-11-22 22:34 UTC #15 lore raw

Re: [PATCH bpf-next v11 07/13] bpf: pass attached BTF to the bpf_struct_ops subsystem

Quoted 19 lines · Martin KaFai Lau

Quoted 4 lines · On 11/6/23 12:12 PM, [email protected] wrote:
From: Kui-Feng Lee <[email protected]>
Every kernel module has its BTF, comprising information on types defined in the module. The BTF fd (attr->value_type_btf_obj_fd) passed from userspace
I would highlight this patch (adds) value_type_btf_obj_fd.
Quoted 10 lines
helps the bpf_struct_ops to lookup type information and description of the struct_ops type, which is necessary for parsing the layout of map element values and registering maps. The descriptions are looked up by matching a type id (attr->btf_vmlinux_value_type_id) against bpf_struct_ops_desc(s) defined in a BTF. If the struct_ops type is defined in a module, the bpf_struct_ops needs to know the module BTF to lookup the bpf_struct_ops_desc.
The bpf_prog includes attach_btf in aux which is passed along with the bpf_attr when loading the program. The purpose of attach_btf is to

Quoted 7 lines

I read it as "attach_btf" is passed in the bpf_attr. This has been in my head for a while. I sort of know what is the actual uapi, so didn't get to it yet.
We have already discussed a bit of this offline. I think it meant attr->attach_btf_obj_fd here.
This patch is mainly about how the userspace passing kmod's btf to the kernel during map creation and prog load and also what uapi does it use. The commit message should mention this patch is reusing the existing attr->attach_btf_obj_fd for the userspace to pass the kmod's btf when loading the struct_ops prog. I need to go back to the syscall.c code to figure out and also leap forward to the later libbpf patch to confirm it.
I depend on the commit message to help the review. It is much appreciated if the commit message is clear and accurate on things like: what it wants to do, how it does it (addition/deletion/changes), and what are the major changes.

Got it! I will rewrite the commit log to make it easier to read the patch.

Quoted 91 lines

Quoted 78 lines

Signed-off-by Kui-Feng Lee <[email protected]>

  5 files changed, 57 insertions(+), 21 deletions(-)

include/uapi/linux/bpf.h +5 -0

index 0f6cdf52b1da..fd20c52606b2 100644
@@ -1398,6 +1398,11 @@ union bpf_attr {
            * to using 5 hash functions).
            */
           __u64    map_extra;
+
+        __u32   value_type_btf_obj_fd;    /* fd pointing to a BTF
+                         * type data for
+                         * btf_vmlinux_value_type_id.
+                         */
       };

struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

kernel/bpf/bpf_struct_ops.c +21 -5

index 4ba6181ed1c4..2fb1b21f989a 100644
@@ -635,6 +635,7 @@ static void __bpf_struct_ops_map_free(struct 
 bpf_map *map)
           bpf_jit_uncharge_modmem(PAGE_SIZE);
       }
       bpf_map_area_free(st_map->uvalue);
+    btf_put(st_map->btf);
       bpf_map_area_free(st_map);
   }

@@ -675,15 +676,30 @@ static struct bpf_map 
 *bpf_struct_ops_map_alloc(union bpf_attr *attr)
       struct bpf_struct_ops_map *st_map;
       const struct btf_type *t, *vt;
       struct bpf_map *map;
+    struct btf *btf;
       int ret;
-    st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, 
 attr->btf_vmlinux_value_type_id);
-    if (!st_ops_desc)
-        return ERR_PTR(-ENOTSUPP);
+    if (attr->value_type_btf_obj_fd) {
+        /* The map holds btf for its whole life time. */
+        btf = btf_get_by_fd(attr->value_type_btf_obj_fd);
+        if (IS_ERR(btf))
+            return ERR_PTR(PTR_ERR(btf));
+    } else {
+        btf = btf_vmlinux;
+        btf_get(btf);
+    }
+
+    st_ops_desc = bpf_struct_ops_find_value(btf, 
 attr->btf_vmlinux_value_type_id);
+    if (!st_ops_desc) {
+        ret = -ENOTSUPP;
+        goto errout;
+    }
       vt = st_ops_desc->value_type;
-    if (attr->value_size != vt->size)
-        return ERR_PTR(-EINVAL);
+    if (attr->value_size != vt->size) {
+        ret = -EINVAL;
+        goto errout;
+    }
       t = st_ops_desc->type;

kernel/bpf/bpf_struct_ops.c static struct bpf_map · line 717

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +5 -2

@@ -694,17 +710,18 @@ static struct bpf_map 
 *bpf_struct_ops_map_alloc(union bpf_attr *attr)
           (vt->size - sizeof(struct bpf_struct_ops_value));
       st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE);
-    if (!st_map)
-        return ERR_PTR(-ENOMEM);
+    if (!st_map) {
+        ret = -ENOMEM;
+        goto errout;
+    }
+    st_map->btf = btf;

How about do the "st_map->btf = btf;" assignment the same as where the current code is doing (a few lines below). Would it avoid the new "btf = NULL;" dance during the error case?

Quoted 1 line

nit, if moving a line, I would move the following "st_map->st_ops_desc = st_ops_desc;" to the later and close to where "st_map->btf = btf;" is.

It would work. But, I also need to init st_map->btf as NULL. Or, it may fail at errout_free to free an invalid pointer if I read it correctly.

Quoted 115 lines

Quoted 25 lines

kernel/bpf/bpf_struct_ops.c +4 -6

       st_map->st_ops_desc = st_ops_desc;
       map = &st_map->map;
       ret = bpf_jit_charge_modmem(PAGE_SIZE);
-    if (ret) {
-        __bpf_struct_ops_map_free(map);
-        return ERR_PTR(ret);
-    }
+    if (ret)
+        goto errout_free;
       st_map->image = bpf_jit_alloc_exec(PAGE_SIZE);
       if (!st_map->image) {

@@ -713,25 +730,31 @@ static struct bpf_map 
 *bpf_struct_ops_map_alloc(union bpf_attr *attr)
            * here.
            */
           bpf_jit_uncharge_modmem(PAGE_SIZE);
-        __bpf_struct_ops_map_free(map);
-        return ERR_PTR(-ENOMEM);
+        ret = -ENOMEM;
+        goto errout_free;
       }
       st_map->uvalue = bpf_map_area_alloc(vt->size, NUMA_NO_NODE);

st_map->links =
bpf_map_area_alloc(btf_type_vlen(t) * sizeof(struct bpf_links *),

Quoted 8 lines

                     NUMA_NO_NODE);
      if (!st_map->uvalue || !st_map->links) {
-        __bpf_struct_ops_map_free(map);
-        return ERR_PTR(-ENOMEM);
+        ret = -ENOMEM;
+        goto errout_free;
      }
-    st_map->btf = btf_vmlinux;

The old code initializes "st_map->btf" here.

Quoted 78 lines

-
      mutex_init(&st_map->lock);
      set_vm_flush_reset_perms(st_map->image);
      bpf_map_init_from_attr(map, attr);
      return map;

kernel/bpf/bpf_struct_ops.c +8 -0

+
+errout_free:
+    __bpf_struct_ops_map_free(map);
+    btf = NULL;        /* has been released */
+errout:
+    btf_put(btf);
+
+    return ERR_PTR(ret);

}
static u64 bpf_struct_ops_map_mem_usage(const struct bpf_map *map)

kernel/bpf/syscall.c +1 -1

index 0ed286b8a0f0..974651fe2bee 100644
@@ -1096,7 +1096,7 @@ static int map_check_btf(struct bpf_map *map, 
 const struct btf *btf,
       return ret;
   }
-#define BPF_MAP_CREATE_LAST_FIELD map_extra
+#define BPF_MAP_CREATE_LAST_FIELD value_type_btf_obj_fd
   /* called via syscall */
   static int map_create(union bpf_attr *attr)
   {

kernel/bpf/verifier.c +6 -3

index bdd166cab977..3f446f76d4bf 100644
@@ -20086,6 +20086,7 @@ static int check_struct_ops_btf_id(struct 
 bpf_verifier_env *env)
       const struct btf_member *member;
       struct bpf_prog *prog = env->prog;
       u32 btf_id, member_idx;
+    struct btf *btf;
       const char *mname;
       if (!prog->gpl_compatible) {

@@ -20093,8 +20094,10 @@ static int check_struct_ops_btf_id(struct 
 bpf_verifier_env *env)
           return -EINVAL;
       }
+    btf = prog->aux->attach_btf;
+
       btf_id = prog->aux->attach_btf_id;
-    st_ops_desc = bpf_struct_ops_find(btf_vmlinux, btf_id);
+    st_ops_desc = bpf_struct_ops_find(btf, btf_id);
       if (!st_ops_desc) {
           verbose(env, "attach_btf_id %u is not a supported struct\n",
               btf_id);

@@ -20111,8 +20114,8 @@ static int check_struct_ops_btf_id(struct 
 bpf_verifier_env *env)
       }
       member = &btf_type_member(t)[member_idx];
-    mname = btf_name_by_offset(btf_vmlinux, member->name_off);
-    func_proto = btf_type_resolve_func_ptr(btf_vmlinux, member->type,
+    mname = btf_name_by_offset(btf, member->name_off);
+    func_proto = btf_type_resolve_func_ptr(btf, member->type,
                              NULL);
       if (!func_proto) {
           verbose(env, "attach to invalid member %s(@idx %u) of struct %s\n",

tools/include/uapi/linux/bpf.h +5 -0

index 0f6cdf52b1da..fd20c52606b2 100644
@@ -1398,6 +1398,11 @@ union bpf_attr {
            * to using 5 hash functions).
            */
           __u64    map_extra;
+
+        __u32   value_type_btf_obj_fd;    /* fd pointing to a BTF
+                         * type data for
+                         * btf_vmlinux_value_type_id.
+                         */
       };

struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */

Martin KaFai Lau

reviewer

2023-11-27 22:08 UTC #16 lore raw

Re: [PATCH bpf-next v11 07/13] bpf: pass attached BTF to the bpf_struct_ops subsystem

Quoted 60 lines · Kui-Feng Lee

Quoted 60 lines

Quoted 47 lines

kernel/bpf/bpf_struct_ops.c +21 -5

index 4ba6181ed1c4..2fb1b21f989a 100644
@@ -635,6 +635,7 @@ static void __bpf_struct_ops_map_free(struct bpf_map *map)
           bpf_jit_uncharge_modmem(PAGE_SIZE);
       }
       bpf_map_area_free(st_map->uvalue);
+    btf_put(st_map->btf);
       bpf_map_area_free(st_map);
   }

@@ -675,15 +676,30 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union 
 bpf_attr *attr)
       struct bpf_struct_ops_map *st_map;
       const struct btf_type *t, *vt;
       struct bpf_map *map;
+    struct btf *btf;
       int ret;
-    st_ops_desc = bpf_struct_ops_find_value(btf_vmlinux, 
 attr->btf_vmlinux_value_type_id);
-    if (!st_ops_desc)
-        return ERR_PTR(-ENOTSUPP);
+    if (attr->value_type_btf_obj_fd) {
+        /* The map holds btf for its whole life time. */
+        btf = btf_get_by_fd(attr->value_type_btf_obj_fd);
+        if (IS_ERR(btf))
+            return ERR_PTR(PTR_ERR(btf));
+    } else {
+        btf = btf_vmlinux;
+        btf_get(btf);
+    }
+
+    st_ops_desc = bpf_struct_ops_find_value(btf, 
 attr->btf_vmlinux_value_type_id);
+    if (!st_ops_desc) {
+        ret = -ENOTSUPP;
+        goto errout;
+    }
       vt = st_ops_desc->value_type;
-    if (attr->value_size != vt->size)
-        return ERR_PTR(-EINVAL);
+    if (attr->value_size != vt->size) {
+        ret = -EINVAL;
+        goto errout;
+    }
       t = st_ops_desc->type;

kernel/bpf/bpf_struct_ops.c static struct bpf_map *bpf_struct_ops_map_alloc(union · line 717

Quoted 10 lines

kernel/bpf/bpf_struct_ops.c +5 -2

@@ -694,17 +710,18 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union 
 bpf_attr *attr)
           (vt->size - sizeof(struct bpf_struct_ops_value));
       st_map = bpf_map_area_alloc(st_map_size, NUMA_NO_NODE);
-    if (!st_map)
-        return ERR_PTR(-ENOMEM);
+    if (!st_map) {
+        ret = -ENOMEM;
+        goto errout;
+    }
+    st_map->btf = btf;

How about do the "st_map->btf = btf;" assignment the same as where the current code is doing (a few lines below). Would it avoid the new "btf = NULL;" dance during the error case?

Quoted 4 lines

Quoted 1 line
nit, if moving a line, I would move the following "st_map->st_ops_desc = st_ops_desc;" to the later and close to where "st_map->btf = btf;" is.
It would work. But, I also need to init st_map->btf as NULL. Or, it may fail at errout_free to free an invalid pointer if I read it correctly.

st_map->btf should have been initialized to NULL. Please check bpf_map_area_alloc().

[email protected]

author

2023-11-06 20:13 UTC #17 lore raw

PATCH v11 8/13 bpf: hold module for bpf_struct_ops_map.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

To ensure that a module remains accessible whenever a struct_ops object of a struct_ops type provided by the module is still in use.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h          |  1 +
 include/linux/bpf_verifier.h |  1 +
 kernel/bpf/bpf_struct_ops.c  | 40 ++++++++++++++++++++++++++++++++++++
 kernel/bpf/verifier.c        | 10 +++++++++
 4 files changed, 52 insertions(+)

include/linux/bpf.h +1 -0

index f0ed874d5ac3..c287f42b2e48 100644
@@ -1626,6 +1626,7 @@ struct bpf_struct_ops {
 	void (*unreg)(void *kdata);
 	int (*update)(void *kdata, void *old_kdata);
 	int (*validate)(void *kdata);
+	struct module *owner;
 	const char *name;
 	struct btf_func_model func_models[BPF_STRUCT_OPS_MAX_NR_MEMBERS];
 };

include/linux/bpf_verifier.h +1 -0

index 24213a99cc79..c1461342f19e 100644
@@ -598,6 +598,7 @@ struct bpf_verifier_env {
 	u32 prev_insn_idx;
 	struct bpf_prog *prog;		/* eBPF program being verified */
 	const struct bpf_verifier_ops *ops;
+	struct module *attach_btf_mod;	/* The owner module of prog->aux->attach_btf */
 	struct bpf_verifier_stack_elem *head; /* stack of verifier states to be processed */
 	int stack_size;			/* number of states to be processed */
 	bool strict_alignment;		/* perform strict pointer alignment checks */

kernel/bpf/bpf_struct_ops.c +40 -0

index 2fb1b21f989a..d1af0ebaf02f 100644
@@ -387,6 +387,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	const struct btf_member *member;
 	const struct btf_type *t = st_ops_desc->type;
 	struct bpf_tramp_links *tlinks;
+	struct module *mod = NULL;
 	void *udata, *kdata;
 	int prog_fd, err;
 	void *image, *image_end;

@@ -424,6 +425,14 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 		goto unlock;
 	}
 
+	if (st_map->btf != btf_vmlinux) {
+		mod = btf_try_get_module(st_map->btf);
+		if (!mod) {
+			err = -EINVAL;
+			goto unlock;
+		}
+	}
+
 	memcpy(uvalue, value, map->value_size);
 
 	udata = &uvalue->data;

@@ -552,6 +561,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 		 * can be seen once BPF_STRUCT_OPS_STATE_INUSE is set.
 		 */
 		smp_store_release(&kvalue->state, BPF_STRUCT_OPS_STATE_INUSE);
+		/* Hold the owner module until the struct_ops is
+		 * unregistered
+		 */
+		mod = NULL;
 		goto unlock;
 	}

@@ -568,6 +581,7 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	memset(uvalue, 0, map->value_size);
 	memset(kvalue, 0, map->value_size);
 unlock:
+	module_put(mod);
 	kfree(tlinks);
 	mutex_unlock(&st_map->lock);
 	return err;

@@ -588,6 +602,7 @@ static long bpf_struct_ops_map_delete_elem(struct bpf_map *map, void *key)
 	switch (prev_state) {
 	case BPF_STRUCT_OPS_STATE_INUSE:
 		st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data);
+		module_put(st_map->st_ops_desc->st_ops->owner);
 		bpf_map_put(map);
 		return 0;
 	case BPF_STRUCT_OPS_STATE_TOBEFREE:

@@ -675,6 +690,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 	size_t st_map_size;
 	struct bpf_struct_ops_map *st_map;
 	const struct btf_type *t, *vt;
+	struct module *mod = NULL;
 	struct bpf_map *map;
 	struct btf *btf;
 	int ret;

@@ -684,6 +700,14 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 		btf = btf_get_by_fd(attr->value_type_btf_obj_fd);
 		if (IS_ERR(btf))
 			return ERR_PTR(PTR_ERR(btf));
+
+		if (btf != btf_vmlinux) {
+			mod = btf_try_get_module(btf);
+			if (!mod) {
+				ret = -EINVAL;
+				goto errout;
+			}
+		}
 	} else {
 		btf = btf_vmlinux;
 		btf_get(btf);

@@ -746,6 +770,8 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 	set_vm_flush_reset_perms(st_map->image);
 	bpf_map_init_from_attr(map, attr);
 
+	module_put(mod);
+
 	return map;
 
 errout_free:

@@ -753,6 +779,7 @@ static struct bpf_map *bpf_struct_ops_map_alloc(union bpf_attr *attr)
 	btf = NULL;		/* has been released */
 errout:
 	btf_put(btf);
+	module_put(mod);
 
 	return ERR_PTR(ret);
 }

@@ -836,6 +863,7 @@ static void bpf_struct_ops_map_link_dealloc(struct bpf_link *link)
 		 * bpf_struct_ops_link_create() fails to register.
 		 */
 		st_map->st_ops_desc->st_ops->unreg(&st_map->kvalue.data);
+		module_put(st_map->st_ops_desc->st_ops->owner);
 		bpf_map_put(&st_map->map);
 	}
 	kfree(st_link);

@@ -882,6 +910,10 @@ static int bpf_struct_ops_map_link_update(struct bpf_link *link, struct bpf_map
 	if (!bpf_struct_ops_valid_to_reg(new_map))
 		return -EINVAL;
 
+	/* The old map is holding the refcount for the owner module.  The
+	 * ownership of the owner module refcount is going to be
+	 * transferred from the old map to the new map.
+	 */
 	if (!st_map->st_ops_desc->st_ops->update)
 		return -EOPNOTSUPP;

@@ -927,6 +959,7 @@ int bpf_struct_ops_link_create(union bpf_attr *attr)
 	struct bpf_link_primer link_primer;
 	struct bpf_struct_ops_map *st_map;
 	struct bpf_map *map;
+	struct btf *btf;
 	int err;
 
 	map = bpf_map_get(attr->link_create.map_fd);

@@ -951,8 +984,15 @@ int bpf_struct_ops_link_create(union bpf_attr *attr)
 	if (err)
 		goto err_out;
 
+	/* Hold the owner module until the struct_ops is unregistered. */
+	btf = st_map->btf;
+	if (btf != btf_vmlinux && !btf_try_get_module(btf)) {
+		err = -EINVAL;
+		goto err_out;
+	}
 	err = st_map->st_ops_desc->st_ops->reg(st_map->kvalue.data);
 	if (err) {
+		module_put(st_map->st_ops_desc->st_ops->owner);
 		bpf_link_cleanup(&link_primer);
 		link = NULL;
 		goto err_out;

kernel/bpf/verifier.c +10 -0

index 3f446f76d4bf..20d6d9665983 100644
@@ -20095,6 +20095,14 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env)
 	}
 
 	btf = prog->aux->attach_btf;
+	if (btf != btf_vmlinux) {
+		/* Make sure st_ops is valid through the lifetime of env */
+		env->attach_btf_mod = btf_try_get_module(btf);
+		if (!env->attach_btf_mod) {
+			verbose(env, "owner module of btf is not found\n");
+			return -ENOTSUPP;
+		}
+	}
 
 	btf_id = prog->aux->attach_btf_id;
 	st_ops_desc = bpf_struct_ops_find(btf, btf_id);

@@ -20808,6 +20816,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, __u3
 		env->prog->expected_attach_type = 0;
 
 	*prog = env->prog;
+
+	module_put(env->attach_btf_mod);
 err_unlock:
 	if (!is_priv)
 		mutex_unlock(&bpf_verifier_lock);

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #18 lore raw

PATCH v11 9/13 bpf: validate value_type

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

A value_type should consist of three components: refcnt, state, and data. refcnt and state has been move to struct bpf_struct_ops_common_value to make it easier to check the value type.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h         | 14 ++++++
 kernel/bpf/bpf_struct_ops.c | 93 ++++++++++++++++++++++++-------------
 2 files changed, 74 insertions(+), 33 deletions(-)

include/linux/bpf.h +14 -0

index c287f42b2e48..48e97a255945 100644
@@ -3231,4 +3231,18 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)
 	return prog->aux->func_idx != 0;
 }
 
+#ifdef CONFIG_BPF_JIT
+enum bpf_struct_ops_state {
+	BPF_STRUCT_OPS_STATE_INIT,
+	BPF_STRUCT_OPS_STATE_INUSE,
+	BPF_STRUCT_OPS_STATE_TOBEFREE,
+	BPF_STRUCT_OPS_STATE_READY,
+};
+
+struct bpf_struct_ops_common_value {
+	refcount_t refcnt;
+	enum bpf_struct_ops_state state;
+};
+#endif /* CONFIG_BPF_JIT */
+
 #endif /* _LINUX_BPF_H */

kernel/bpf/bpf_struct_ops.c +60 ~~-33~~

189 lines not on this page · raw message

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 02:12 UTC #19 lore raw

Re: [PATCH bpf-next v11 09/13] bpf: validate value_type

Quoted 17 lines · On 11/6/23 12:12 PM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
A value_type should consist of three components: refcnt, state, and data. refcnt and state has been move to struct bpf_struct_ops_common_value to make it easier to check the value type.
Signed-off-by Kui-Feng Lee <[email protected]>
---
  include/linux/bpf.h         | 14 ++++++
  kernel/bpf/bpf_struct_ops.c | 93 ++++++++++++++++++++++++-------------
  2 files changed, 74 insertions(+), 33 deletions(-)
include/linux/bpf.h +0 -0
index c287f42b2e48..48e97a255945 100644

include/linux/bpf.h static inline bool bpf_is_subprog(const struct bpf_prog *prog) · line 3234

Quoted 4 lines

include/linux/bpf.h +1 -0

@@ -3231,4 +3231,18 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)
  	return prog->aux->func_idx != 0;
  }
 
+#ifdef CONFIG_BPF_JIT

There is an existing "#if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)" above and a few bpf_struct_ops_*() has already been there. Does it need another separate one which is only CONFIG_BPF_JIT here?

include/linux/bpf.h static inline bool bpf_is_subprog(const struct bpf_prog *prog) · line 3245

Quoted 11 lines

include/linux/bpf.h +11 -0

+enum bpf_struct_ops_state {
+	BPF_STRUCT_OPS_STATE_INIT,
+	BPF_STRUCT_OPS_STATE_INUSE,
+	BPF_STRUCT_OPS_STATE_TOBEFREE,
+	BPF_STRUCT_OPS_STATE_READY,
+};
+
+struct bpf_struct_ops_common_value {
+	refcount_t refcnt;
+	enum bpf_struct_ops_state state;
+};

Do the struct and enum really need to be in ifdef?

Quoted 2 lines

include/linux/bpf.h +2 -0
+#endif /* CONFIG_BPF_JIT */
+

Kui-Feng Lee

2023-11-22 23:47 UTC #20 lore raw

Re: [PATCH bpf-next v11 09/13] bpf: validate value_type

Quoted 39 lines · Martin KaFai Lau

Quoted 17 lines · On 11/6/23 12:12 PM, [email protected] wrote:
From: Kui-Feng Lee <[email protected]>
A value_type should consist of three components: refcnt, state, and data. refcnt and state has been move to struct bpf_struct_ops_common_value to make it easier to check the value type.
Signed-off-by Kui-Feng Lee <[email protected]>
---
include/linux/bpf.h | 14 ++++++ kernel/bpf/bpf_struct_ops.c | 93 ++++++++++++++++++++++++-------------
  2 files changed, 74 insertions(+), 33 deletions(-)
include/linux/bpf.h +0 -0
index c287f42b2e48..48e97a255945 100644
include/linux/bpf.h static inline bool bpf_is_subprog(const struct · line 3234
Quoted 4 lines
include/linux/bpf.h +1 -0
@@ -3231,4 +3231,18 @@ static inline bool bpf_is_subprog(const struct 
 bpf_prog *prog)
       return prog->aux->func_idx != 0;
   }
+#ifdef CONFIG_BPF_JIT
There is an existing "#if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)" above and a few bpf_struct_ops_*() has already been there. Does it need another separate one which is only CONFIG_BPF_JIT here?
include/linux/bpf.h static inline bool bpf_is_subprog(const struct bpf_prog *prog) · line 3245
Quoted 11 lines
include/linux/bpf.h +11 -0
+enum bpf_struct_ops_state {
+    BPF_STRUCT_OPS_STATE_INIT,
+    BPF_STRUCT_OPS_STATE_INUSE,
+    BPF_STRUCT_OPS_STATE_TOBEFREE,
+    BPF_STRUCT_OPS_STATE_READY,
+};
+
+struct bpf_struct_ops_common_value {
+    refcount_t refcnt;
+    enum bpf_struct_ops_state state;
+};
Do the struct and enum really need to be in ifdef?

Quoted 3 lines

include/linux/bpf.h static inline bool bpf_is_subprog(const struct bpf_prog *prog) · line 3247
Quoted 2 lines
include/linux/bpf.h +2 -0
+#endif /* CONFIG_BPF_JIT */
+

I just removed this pair of #if-else.
You are right! They are not necessary.

[email protected]

author

2023-11-06 20:13 UTC #21 lore raw

PATCH v11 10/13 bpf, net: switch to dynamic registration

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Replace the static list of struct_ops types with per-btf struct_ops_tab to enable dynamic registration.

Both bpf_dummy_ops and bpf_tcp_ca now utilize the registration function instead of being listed in bpf_struct_ops_types.h.

Cc [email protected]

Signed-off-by Kui-Feng Lee <[email protected]>

---
 include/linux/bpf.h               | 24 +++++++--
 include/linux/btf.h               |  2 +
 kernel/bpf/bpf_struct_ops.c       | 90 ++++++++++---------------------
 kernel/bpf/bpf_struct_ops_types.h | 12 -----
 kernel/bpf/btf.c                  | 40 ++++++++++++--
 net/bpf/bpf_dummy_struct_ops.c    | 14 ++++-
 net/ipv4/bpf_tcp_ca.c             | 16 ++++--
 7 files changed, 109 insertions(+), 89 deletions(-)
 delete mode 100644 kernel/bpf/bpf_struct_ops_types.h

include/linux/bpf.h +19 -5

index 48e97a255945..432c088d4001 100644
@@ -1643,7 +1643,6 @@ struct bpf_struct_ops_desc {
 #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
 #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
 const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id);
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
 bool bpf_struct_ops_get(const void *kdata);
 void bpf_struct_ops_put(const void *kdata);
 int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key,

@@ -1689,10 +1688,6 @@ static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *
 {
 	return NULL;
 }
-static inline void bpf_struct_ops_init(struct btf *btf,
-				       struct bpf_verifier_log *log)
-{
-}
 static inline bool bpf_try_module_get(const void *data, struct module *owner)
 {
 	return try_module_get(owner);

@@ -3232,6 +3227,8 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)
 }
 
 #ifdef CONFIG_BPF_JIT
+int register_bpf_struct_ops(struct bpf_struct_ops *st_ops);
+
 enum bpf_struct_ops_state {
 	BPF_STRUCT_OPS_STATE_INIT,
 	BPF_STRUCT_OPS_STATE_INUSE,

@@ -3243,6 +3240,23 @@ struct bpf_struct_ops_common_value {
 	refcount_t refcnt;
 	enum bpf_struct_ops_state state;
 };
+
+/* bpf_struct_ops_##_name (e.g. bpf_struct_ops_tcp_congestion_ops) is
+ * the map's value exposed to the userspace and its btf-type-id is
+ * stored at the map->btf_vmlinux_value_type_id.
+ *
+ */
+#define DEFINE_STRUCT_OPS_VALUE_TYPE(_name)			\
+extern struct bpf_struct_ops bpf_##_name;			\
+								\
+struct bpf_struct_ops_##_name {					\
+	struct bpf_struct_ops_common_value common;		\
+	struct _name data ____cacheline_aligned_in_smp;		\
+}
+
+extern int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc,
+				    struct btf *btf,
+				    struct bpf_verifier_log *log);
 #endif /* CONFIG_BPF_JIT */
 
 #endif /* _LINUX_BPF_H */

include/linux/btf.h +2 -0

index 07ee6740e06a..d6fd6c20b3e2 100644
@@ -12,6 +12,8 @@
 #include <uapi/linux/bpf.h>
 
 #define BTF_TYPE_EMIT(type) ((void)(type *)0)
+#define BTF_STRUCT_OPS_TYPE_EMIT(type) \
+	((void)(struct bpf_struct_ops_##type *)0)
 #define BTF_TYPE_EMIT_ENUM(enum_val) ((void)enum_val)
 
 /* These need to be macros, as the expressions are used in assembler input */

kernel/bpf/bpf_struct_ops.c +27 ~~-63~~

176 lines not on this page · raw message

kernel/bpf/bpf_struct_ops_types.h deleted +0 ~~-12~~

deleted file mode 100644
index 5678a9ddf817..000000000000
@@ -1,12 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/* internal file - do not include directly */
-
-#ifdef CONFIG_BPF_JIT
-#ifdef CONFIG_NET
-BPF_STRUCT_OPS_TYPE(bpf_dummy_ops)
-#endif
-#ifdef CONFIG_INET
-#include <net/tcp.h>
-BPF_STRUCT_OPS_TYPE(tcp_congestion_ops)
-#endif
-#endif

kernel/bpf/btf.c +36 -4

index 263715af10cb..7165a1beeed5 100644
@@ -19,6 +19,7 @@
 #include <linux/bpf_verifier.h>
 #include <linux/btf.h>
 #include <linux/btf_ids.h>
+#include <linux/bpf.h>
 #include <linux/bpf_lsm.h>
 #include <linux/skmsg.h>
 #include <linux/perf_event.h>

@@ -5796,8 +5797,6 @@ struct btf *btf_parse_vmlinux(void)
 	/* btf_parse_vmlinux() runs under bpf_verifier_lock */
 	bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]);
 
-	bpf_struct_ops_init(btf, log);
-
 	refcount_set(&btf->refcnt, 1);
 
 	err = btf_alloc_id(btf);

@@ -8628,10 +8627,11 @@ bool btf_type_ids_nocast_alias(struct bpf_verifier_log *log,
 #ifdef CONFIG_BPF_JIT
 
 static int
-btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops)
+btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops,
+		   struct bpf_verifier_log *log)
 {
 	struct btf_struct_ops_tab *tab, *new_tab;
-	int i;
+	int i, err;
 
 	if (!btf)
 		return -ENOENT;

@@ -8668,6 +8668,10 @@ btf_add_struct_ops(struct btf *btf, struct bpf_struct_ops *st_ops)
 
 	tab->ops[btf->struct_ops_tab->cnt].st_ops = st_ops;
 
+	err = bpf_struct_ops_desc_init(&tab->ops[btf->struct_ops_tab->cnt], btf, log);
+	if (err)
+		return err;
+
 	btf->struct_ops_tab->cnt++;
 
 	return 0;

@@ -8684,4 +8688,32 @@ const struct bpf_struct_ops_desc *btf_get_struct_ops(struct btf *btf, u32 *ret_c
 	return (const struct bpf_struct_ops_desc *)btf->struct_ops_tab->ops;
 }
 
+int register_bpf_struct_ops(struct bpf_struct_ops *st_ops)
+{
+	struct bpf_verifier_log *log;
+	struct btf *btf;
+	int err = 0;
+
+	btf = btf_get_module_btf(st_ops->owner);
+	if (!btf)
+		return -EINVAL;
+
+	log = kzalloc(sizeof(*log), GFP_KERNEL | __GFP_NOWARN);
+	if (!log) {
+		err = -ENOMEM;
+		goto errout;
+	}
+
+	log->level = BPF_LOG_KERNEL;
+
+	err = btf_add_struct_ops(btf, st_ops, log);
+
+errout:
+	kfree(log);
+	btf_put(btf);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(register_bpf_struct_ops);
+
 #endif /* CONFIG_BPF_JIT */

net/bpf/bpf_dummy_struct_ops.c +12 -2

index ffa224053a6c..148a5851c4fa 100644
@@ -7,7 +7,7 @@
 #include <linux/bpf.h>
 #include <linux/btf.h>
 
-extern struct bpf_struct_ops bpf_bpf_dummy_ops;
+static struct bpf_struct_ops bpf_bpf_dummy_ops;
 
 /* A common type for test_N with return value in bpf_dummy_ops */
 typedef int (*dummy_ops_test_ret_fn)(struct bpf_dummy_ops_state *state, ...);

@@ -223,11 +223,13 @@ static int bpf_dummy_reg(void *kdata)
 	return -EOPNOTSUPP;
 }
 
+DEFINE_STRUCT_OPS_VALUE_TYPE(bpf_dummy_ops);
+
 static void bpf_dummy_unreg(void *kdata)
 {
 }
 
-struct bpf_struct_ops bpf_bpf_dummy_ops = {
+static struct bpf_struct_ops bpf_bpf_dummy_ops = {
 	.verifier_ops = &bpf_dummy_verifier_ops,
 	.init = bpf_dummy_init,
 	.check_member = bpf_dummy_ops_check_member,

@@ -235,4 +237,12 @@ struct bpf_struct_ops bpf_bpf_dummy_ops = {
 	.reg = bpf_dummy_reg,
 	.unreg = bpf_dummy_unreg,
 	.name = "bpf_dummy_ops",
+	.owner = THIS_MODULE,
 };
+
+static int __init bpf_dummy_struct_ops_init(void)
+{
+	BTF_STRUCT_OPS_TYPE_EMIT(bpf_dummy_ops);
+	return register_bpf_struct_ops(&bpf_bpf_dummy_ops);
+}
+late_initcall(bpf_dummy_struct_ops_init);

net/ipv4/bpf_tcp_ca.c +13 -3

index 3c8b76578a2a..b36a19274e5b 100644
@@ -12,7 +12,7 @@
 #include <net/bpf_sk_storage.h>
 
 /* "extern" is to avoid sparse warning.  It is only used in bpf_struct_ops.c. */
-extern struct bpf_struct_ops bpf_tcp_congestion_ops;
+static struct bpf_struct_ops bpf_tcp_congestion_ops;
 
 static u32 unsupported_ops[] = {
 	offsetof(struct tcp_congestion_ops, get_info),

@@ -277,7 +277,9 @@ static int bpf_tcp_ca_validate(void *kdata)
 	return tcp_validate_congestion_control(kdata);
 }
 
-struct bpf_struct_ops bpf_tcp_congestion_ops = {
+DEFINE_STRUCT_OPS_VALUE_TYPE(tcp_congestion_ops);
+
+static struct bpf_struct_ops bpf_tcp_congestion_ops = {
 	.verifier_ops = &bpf_tcp_ca_verifier_ops,
 	.reg = bpf_tcp_ca_reg,
 	.unreg = bpf_tcp_ca_unreg,

@@ -287,10 +289,18 @@ struct bpf_struct_ops bpf_tcp_congestion_ops = {
 	.init = bpf_tcp_ca_init,
 	.validate = bpf_tcp_ca_validate,
 	.name = "tcp_congestion_ops",
+	.owner = THIS_MODULE,
 };
 
 static int __init bpf_tcp_ca_kfunc_init(void)
 {
-	return register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_tcp_ca_kfunc_set);
+	int ret;
+
+	BTF_STRUCT_OPS_TYPE_EMIT(tcp_congestion_ops);
+
+	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, &bpf_tcp_ca_kfunc_set);
+	ret = ret ?: register_bpf_struct_ops(&bpf_tcp_congestion_ops);
+
+	return ret;
 }
 late_initcall(bpf_tcp_ca_kfunc_init);

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 02:19 UTC #22 lore raw

Re: [PATCH bpf-next v11 10/13] bpf, net: switch to dynamic registration

Quoted 34 lines · On 11/6/23 12:12 PM, [email protected] wrote:

include/linux/bpf.h +2 -5

index 48e97a255945..432c088d4001 100644
@@ -1643,7 +1643,6 @@ struct bpf_struct_ops_desc {
  #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
  #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
  const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id);
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
  bool bpf_struct_ops_get(const void *kdata);
  void bpf_struct_ops_put(const void *kdata);
  int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key,

@@ -1689,10 +1688,6 @@ static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *
  {
  	return NULL;
  }
-static inline void bpf_struct_ops_init(struct btf *btf,
-				       struct bpf_verifier_log *log)
-{
-}
  static inline bool bpf_try_module_get(const void *data, struct module *owner)
  {
  	return try_module_get(owner);

@@ -3232,6 +3227,8 @@ static inline bool bpf_is_subprog(const struct bpf_prog *prog)
  }
 
  #ifdef CONFIG_BPF_JIT
+int register_bpf_struct_ops(struct bpf_struct_ops *st_ops);
+
  enum bpf_struct_ops_state {
  	BPF_STRUCT_OPS_STATE_INIT,
  	BPF_STRUCT_OPS_STATE_INUSE,

@@ -3243,6 +3240,23 @@ struct bpf_struct_ops_common_value {
  	refcount_t refcnt;

include/linux/bpf.h struct bpf_struct_ops_common_value { · line 3250

Quoted 10 lines

include/linux/bpf.h +8 -0

  	enum bpf_struct_ops_state state;
  };
+
+/* bpf_struct_ops_##_name (e.g. bpf_struct_ops_tcp_congestion_ops) is
+ * the map's value exposed to the userspace and its btf-type-id is
+ * stored at the map->btf_vmlinux_value_type_id.
+ *
+ */
+#define DEFINE_STRUCT_OPS_VALUE_TYPE(_name)			\
+extern struct bpf_struct_ops bpf_##_name;			\

Is it still needed?

include/linux/bpf.h struct bpf_struct_ops_common_value { · line 3259

Quoted 9 lines

include/linux/bpf.h +9 -0

+								\
+struct bpf_struct_ops_##_name {					\
+	struct bpf_struct_ops_common_value common;		\
+	struct _name data ____cacheline_aligned_in_smp;		\
+}
+
+extern int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc,
+				    struct btf *btf,
+				    struct bpf_verifier_log *log);

nit. Remove extern.

Quoted 3 lines

include/linux/bpf.h +0 -0

  #endif /* CONFIG_BPF_JIT */
 
  #endif /* _LINUX_BPF_H */

Kui-Feng Lee

2023-11-22 23:53 UTC #23 lore raw

Re: [PATCH bpf-next v11 10/13] bpf, net: switch to dynamic registration

Quoted 40 lines · Martin KaFai Lau

Quoted 39 lines · On 11/6/23 12:12 PM, [email protected] wrote:

include/linux/bpf.h +0 -5

index 48e97a255945..432c088d4001 100644
@@ -1643,7 +1643,6 @@ struct bpf_struct_ops_desc {
   #if defined(CONFIG_BPF_JIT) && defined(CONFIG_BPF_SYSCALL)
   #define BPF_MODULE_OWNER ((void *)((0xeB9FUL << 2) + POISON_POINTER_DELTA))
   const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf *btf, u32 type_id);
-void bpf_struct_ops_init(struct btf *btf, struct bpf_verifier_log *log);
   bool bpf_struct_ops_get(const void *kdata);
   void bpf_struct_ops_put(const void *kdata);
   int bpf_struct_ops_map_sys_lookup_elem(struct bpf_map *map, void *key,

@@ -1689,10 +1688,6 @@ static inline const struct bpf_struct_ops_desc 
 *bpf_struct_ops_find(struct btf *
   {
       return NULL;
   }
-static inline void bpf_struct_ops_init(struct btf *btf,
-                       struct bpf_verifier_log *log)
-{
-}
   static inline bool bpf_try_module_get(const void *data, struct module *owner)
   {

return try_module_get(owner);

include/linux/bpf.h +4 -0

@@ -3232,6 +3227,8 @@ static inline bool bpf_is_subprog(const struct 
 bpf_prog *prog)
   }
   #ifdef CONFIG_BPF_JIT
+int register_bpf_struct_ops(struct bpf_struct_ops *st_ops);
+
   enum bpf_struct_ops_state {
       BPF_STRUCT_OPS_STATE_INIT,
       BPF_STRUCT_OPS_STATE_INUSE,

@@ -3243,6 +3240,23 @@ struct bpf_struct_ops_common_value {
       refcount_t refcnt;
       enum bpf_struct_ops_state state;
   };
+
+/* bpf_struct_ops_##_name (e.g. bpf_struct_ops_tcp_congestion_ops) is

Quoted 8 lines

include/linux/bpf.h struct bpf_struct_ops_common_value { · line 3250
Quoted 6 lines
include/linux/bpf.h +6 -0
+ * the map's value exposed to the userspace and its btf-type-id is
+ * stored at the map->btf_vmlinux_value_type_id.
+ *
+ */
+#define DEFINE_STRUCT_OPS_VALUE_TYPE(_name)            \
+extern struct bpf_struct_ops bpf_##_name;            \
Is it still needed?

No, will remove it.

Quoted 13 lines

include/linux/bpf.h struct bpf_struct_ops_common_value { · line 3259

Quoted 11 lines

include/linux/bpf.h +9 -0

+                                \
+struct bpf_struct_ops_##_name {                    \
+    struct bpf_struct_ops_common_value common;        \
+    struct _name data ____cacheline_aligned_in_smp;        \
+}
+
+extern int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc 
 *st_ops_desc,
+                    struct btf *btf,
+                    struct bpf_verifier_log *log);

nit. Remove extern.

Sure!

Quoted 4 lines

Quoted 3 lines
include/linux/bpf.h +0 -0
   #endif /* CONFIG_BPF_JIT */
   #endif /* _LINUX_BPF_H */

[email protected]

author

2023-11-06 20:13 UTC #24 lore raw

PATCH v11 11/13 libbpf: Find correct module BTFs for struct_ops maps and progs.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Locate the module BTFs for struct_ops maps and progs and pass them to the kernel. This ensures that the kernel correctly resolves type IDs from the appropriate module BTFs.

For the map of a struct_ops object, the FD of the module BTF is set to bpf_map to keep a reference to the module BTF. The FD is passed to the kernel as value_type_btf_obj_fd when the struct_ops object is loaded.

For a bpf_struct_ops prog, attach_btf_obj_fd of bpf_prog is the FD of a module BTF in the kernel.

Signed-off-by Kui-Feng Lee <[email protected]>

Acked-by Andrii Nakryiko <[email protected]>

---
 tools/lib/bpf/bpf.c    |  4 +++-
 tools/lib/bpf/bpf.h    |  5 ++++-
 tools/lib/bpf/libbpf.c | 38 ++++++++++++++++++++++++++++----------
 3 files changed, 35 insertions(+), 12 deletions(-)

tools/lib/bpf/bpf.c +3 -1

index 9dc9625651dc..b133acfe08fb 100644
@@ -169,7 +169,8 @@ int bpf_map_create(enum bpf_map_type map_type,
 		   __u32 max_entries,
 		   const struct bpf_map_create_opts *opts)
 {
-	const size_t attr_sz = offsetofend(union bpf_attr, map_extra);
+	const size_t attr_sz = offsetofend(union bpf_attr,
+					   value_type_btf_obj_fd);
 	union bpf_attr attr;
 	int fd;

@@ -191,6 +192,7 @@ int bpf_map_create(enum bpf_map_type map_type,
 	attr.btf_key_type_id = OPTS_GET(opts, btf_key_type_id, 0);
 	attr.btf_value_type_id = OPTS_GET(opts, btf_value_type_id, 0);
 	attr.btf_vmlinux_value_type_id = OPTS_GET(opts, btf_vmlinux_value_type_id, 0);
+	attr.value_type_btf_obj_fd = OPTS_GET(opts, value_type_btf_obj_fd, 0);
 
 	attr.inner_map_fd = OPTS_GET(opts, inner_map_fd, 0);
 	attr.map_flags = OPTS_GET(opts, map_flags, 0);

tools/lib/bpf/bpf.h +4 -1

index d0f53772bdc0..ffdd81c0196a 100644
@@ -51,8 +51,11 @@ struct bpf_map_create_opts {
 
 	__u32 numa_node;
 	__u32 map_ifindex;
+
+	__u32 value_type_btf_obj_fd;
+	size_t:0;
 };
-#define bpf_map_create_opts__last_field map_ifindex
+#define bpf_map_create_opts__last_field value_type_btf_obj_fd
 
 LIBBPF_API int bpf_map_create(enum bpf_map_type map_type,
 			      const char *map_name,

tools/lib/bpf/libbpf.c +28 ~~-10~~

index e067be95da3c..5b53592b5217 100644
@@ -527,6 +527,7 @@ struct bpf_map {
 	struct bpf_map_def def;
 	__u32 numa_node;
 	__u32 btf_var_idx;
+	int mod_btf_fd;
 	__u32 btf_key_type_id;
 	__u32 btf_value_type_id;
 	__u32 btf_vmlinux_value_type_id;

@@ -930,22 +931,29 @@ find_member_by_name(const struct btf *btf, const struct btf_type *t,
 	return NULL;
 }
 
+static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name,
+			    __u16 kind, struct btf **res_btf,
+			    struct module_btf **res_mod_btf);
+
 #define STRUCT_OPS_VALUE_PREFIX "bpf_struct_ops_"
 static int find_btf_by_prefix_kind(const struct btf *btf, const char *prefix,
 				   const char *name, __u32 kind);
 
 static int
-find_struct_ops_kern_types(const struct btf *btf, const char *tname,
+find_struct_ops_kern_types(struct bpf_object *obj, const char *tname,
+			   struct module_btf **mod_btf,
 			   const struct btf_type **type, __u32 *type_id,
 			   const struct btf_type **vtype, __u32 *vtype_id,
 			   const struct btf_member **data_member)
 {
 	const struct btf_type *kern_type, *kern_vtype;
 	const struct btf_member *kern_data_member;
+	struct btf *btf;
 	__s32 kern_vtype_id, kern_type_id;
 	__u32 i;
 
-	kern_type_id = btf__find_by_name_kind(btf, tname, BTF_KIND_STRUCT);
+	kern_type_id = find_ksym_btf_id(obj, tname, BTF_KIND_STRUCT,
+					&btf, mod_btf);
 	if (kern_type_id < 0) {
 		pr_warn("struct_ops init_kern: struct %s is not found in kernel BTF\n",
 			tname);

@@ -999,14 +1007,16 @@ static bool bpf_map__is_struct_ops(const struct bpf_map *map)
 }
 
 /* Init the map's fields that depend on kern_btf */
-static int bpf_map__init_kern_struct_ops(struct bpf_map *map,
-					 const struct btf *btf,
-					 const struct btf *kern_btf)
+static int bpf_map__init_kern_struct_ops(struct bpf_map *map)
 {
 	const struct btf_member *member, *kern_member, *kern_data_member;
 	const struct btf_type *type, *kern_type, *kern_vtype;
 	__u32 i, kern_type_id, kern_vtype_id, kern_data_off;
+	struct bpf_object *obj = map->obj;
+	const struct btf *btf = obj->btf;
 	struct bpf_struct_ops *st_ops;
+	const struct btf *kern_btf;
+	struct module_btf *mod_btf;
 	void *data, *kern_data;
 	const char *tname;
 	int err;

@@ -1014,16 +1024,19 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map,
 	st_ops = map->st_ops;
 	type = st_ops->type;
 	tname = st_ops->tname;
-	err = find_struct_ops_kern_types(kern_btf, tname,
+	err = find_struct_ops_kern_types(obj, tname, &mod_btf,
 					 &kern_type, &kern_type_id,
 					 &kern_vtype, &kern_vtype_id,
 					 &kern_data_member);
 	if (err)
 		return err;
 
+	kern_btf = mod_btf ? mod_btf->btf : obj->btf_vmlinux;
+
 	pr_debug("struct_ops init_kern %s: type_id:%u kern_type_id:%u kern_vtype_id:%u\n",
 		 map->name, st_ops->type_id, kern_type_id, kern_vtype_id);
 
+	map->mod_btf_fd = mod_btf ? mod_btf->fd : 0;
 	map->def.value_size = kern_vtype->size;
 	map->btf_vmlinux_value_type_id = kern_vtype_id;

@@ -1099,6 +1112,8 @@ static int bpf_map__init_kern_struct_ops(struct bpf_map *map,
 				return -ENOTSUP;
 			}
 
+			if (mod_btf)
+				prog->attach_btf_obj_fd = mod_btf->fd;
 			prog->attach_btf_id = kern_type_id;
 			prog->expected_attach_type = kern_member_idx;

@@ -1141,8 +1156,7 @@ static int bpf_object__init_kern_struct_ops_maps(struct bpf_object *obj)
 		if (!bpf_map__is_struct_ops(map))
 			continue;
 
-		err = bpf_map__init_kern_struct_ops(map, obj->btf,
-						    obj->btf_vmlinux);
+		err = bpf_map__init_kern_struct_ops(map);
 		if (err)
 			return err;
 	}

@@ -5201,8 +5215,10 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b
 	create_attr.numa_node = map->numa_node;
 	create_attr.map_extra = map->map_extra;
 
-	if (bpf_map__is_struct_ops(map))
+	if (bpf_map__is_struct_ops(map)) {
 		create_attr.btf_vmlinux_value_type_id = map->btf_vmlinux_value_type_id;
+		create_attr.value_type_btf_obj_fd = map->mod_btf_fd;
+	}
 
 	if (obj->btf && btf__fd(obj->btf) >= 0) {
 		create_attr.btf_fd = btf__fd(obj->btf);

@@ -9546,7 +9562,9 @@ static int libbpf_find_attach_btf_id(struct bpf_program *prog, const char *attac
 		*btf_obj_fd = 0;
 		*btf_type_id = 1;
 	} else {
-		err = find_kernel_btf_id(prog->obj, attach_name, attach_type, btf_obj_fd, btf_type_id);
+		err = find_kernel_btf_id(prog->obj, attach_name,
+					 attach_type, btf_obj_fd,
+					 btf_type_id);
 	}
 	if (err) {
 		pr_warn("prog '%s': failed to find kernel BTF type ID of '%s': %d\n",

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #25 lore raw

PATCH v11 12/13 bpf: export btf_ctx_access to modules.

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

The module requires the use of btf_ctx_access() to invoke bpf_tracing_btf_ctx_access() from a module. This function is valuable for implementing validation functions that ensure proper access to ctx.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 kernel/bpf/btf.c | 1 +
 1 file changed, 1 insertion(+)

kernel/bpf/btf.c +1 -0

index 7165a1beeed5..e0812b00a6b2 100644
@@ -6146,6 +6146,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,
 		__btf_name_by_offset(btf, t->name_off));
 	return true;
 }
+EXPORT_SYMBOL_GPL(btf_ctx_access);
 
 enum bpf_struct_walk_result {
 	/* < 0 error */

Signature · 1 line

2.34.1

[email protected]

author

2023-11-06 20:13 UTC #26 lore raw

PATCH v11 13/13 selftests/bpf: test case for register_bpf_struct_ops().

In reply to [email protected]

From: Kui-Feng Lee <[email protected]>

Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module.

Signed-off-by Kui-Feng Lee <[email protected]>

---
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  59 +++++++
 .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   5 +
 .../bpf/prog_tests/test_struct_ops_module.c   | 144 ++++++++++++++++++
 .../selftests/bpf/progs/struct_ops_module.c   |  30 ++++
 .../testing/selftests/bpf/progs/testmod_btf.c |  26 ++++
 5 files changed, 264 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c
 create mode 100644 tools/testing/selftests/bpf/progs/testmod_btf.c

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +59 -0

index a5e246f7b202..418e10311c33 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Copyright (c) 2020 Facebook */
+#include <linux/bpf.h>
 #include <linux/btf.h>
 #include <linux/btf_ids.h>
 #include <linux/error-injection.h>

@@ -522,11 +523,66 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_static_unused_arg)
 BTF_ID_FLAGS(func, bpf_kfunc_call_test_offset)
 BTF_SET8_END(bpf_testmod_check_kfunc_ids)
 
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+
+DEFINE_STRUCT_OPS_VALUE_TYPE(bpf_testmod_ops);
+
+static int bpf_testmod_ops_init(struct btf *btf)
+{
+	return 0;
+}
+
+static bool bpf_testmod_ops_is_valid_access(int off, int size,
+					    enum bpf_access_type type,
+					    const struct bpf_prog *prog,
+					    struct bpf_insn_access_aux *info)
+{
+	return bpf_tracing_btf_ctx_access(off, size, type, prog, info);
+}
+
+static int bpf_testmod_ops_init_member(const struct btf_type *t,
+				       const struct btf_member *member,
+				       void *kdata, const void *udata)
+{
+	return 0;
+}
+
 static const struct btf_kfunc_id_set bpf_testmod_kfunc_set = {
 	.owner = THIS_MODULE,
 	.set   = &bpf_testmod_check_kfunc_ids,
 };
 
+static const struct bpf_verifier_ops bpf_testmod_verifier_ops = {
+	.is_valid_access = bpf_testmod_ops_is_valid_access,
+};
+
+static int bpf_dummy_reg(void *kdata)
+{
+	struct bpf_testmod_ops *ops = kdata;
+	int r;
+
+	BTF_STRUCT_OPS_TYPE_EMIT(bpf_testmod_ops);
+	r = ops->test_2(4, 3);
+
+	return 0;
+}
+
+static void bpf_dummy_unreg(void *kdata)
+{
+}
+
+struct bpf_struct_ops bpf_bpf_testmod_ops = {
+	.verifier_ops = &bpf_testmod_verifier_ops,
+	.init = bpf_testmod_ops_init,
+	.init_member = bpf_testmod_ops_init_member,
+	.reg = bpf_dummy_reg,
+	.unreg = bpf_dummy_unreg,
+	.name = "bpf_testmod_ops",
+	.owner = THIS_MODULE,
+};
+
+#endif /* CONFIG_DEBUG_INFO_BTF_MODULES */
+
 extern int bpf_fentry_test1(int a);
 
 static int bpf_testmod_init(void)

@@ -537,6 +593,9 @@ static int bpf_testmod_init(void)
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_testmod_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_testmod_kfunc_set);
 	ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_testmod_kfunc_set);
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+	ret = ret ?: register_bpf_struct_ops(&bpf_bpf_testmod_ops);
+#endif
 	if (ret < 0)
 		return ret;
 	if (bpf_fentry_test1(0) < 0)

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h +5 -0

index f32793efe095..ca5435751c79 100644
@@ -28,4 +28,9 @@ struct bpf_iter_testmod_seq {
 	int cnt;
 };
 
+struct bpf_testmod_ops {
+	int (*test_1)(void);
+	int (*test_2)(int a, int b);
+};
+
 #endif /* _BPF_TESTMOD_H */

tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c new file +144 -0

144 lines not on this page · raw message

tools/testing/selftests/bpf/progs/struct_ops_module.c new file +30 -0

new file mode 100644
index 000000000000..cb305d04342f
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "../bpf_testmod/bpf_testmod.h"
+
+char _license[] SEC("license") = "GPL";
+
+int test_2_result = 0;
+
+SEC("struct_ops/test_1")
+int BPF_PROG(test_1)
+{
+	return 0xdeadbeef;
+}
+
+SEC("struct_ops/test_2")
+int BPF_PROG(test_2, int a, int b)
+{
+	test_2_result = a + b;
+	return a + b;
+}
+
+SEC(".struct_ops.link")
+struct bpf_testmod_ops testmod_1 = {
+	.test_1 = (void *)test_1,
+	.test_2 = (void *)test_2,
+};
+

tools/testing/selftests/bpf/progs/testmod_btf.c new file +26 -0

new file mode 100644
index 000000000000..cfcd269f07fe
@@ -0,0 +1,26 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+#include <vmlinux.h>
+#include <bpf/bpf_core_read.h>
+#include <bpf/bpf_tracing.h>
+
+char _license[] SEC("license") = "GPL";
+
+int bpf_testmod_put = 0;
+
+SEC("kprobe/btf_put")
+int BPF_KPROBE(kprobe_btf_put, struct btf *btf)
+{
+	const char name[] = "bpf_testmod";
+	int i;
+
+	for (i = 0; i < sizeof(name); i++) {
+		if (BPF_CORE_READ(btf, name[i]) != name[i])
+			return 0;
+	}
+
+	if (BPF_CORE_READ(btf, refcnt.refs.counter) == 1)
+		bpf_testmod_put = 1;
+
+	return 0;
+}

Signature · 1 line

2.34.1

Martin KaFai Lau

reviewer

2023-11-10 02:23 UTC #27 lore raw

Re: [PATCH bpf-next v11 13/13] selftests/bpf: test case for register_bpf_struct_ops().

Quoted 30 lines · On 11/6/23 12:12 PM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module.
Signed-off-by Kui-Feng Lee <[email protected]>
---
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  59 +++++++
  .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   5 +
  .../bpf/prog_tests/test_struct_ops_module.c   | 144 ++++++++++++++++++
  .../selftests/bpf/progs/struct_ops_module.c   |  30 ++++
  .../testing/selftests/bpf/progs/testmod_btf.c |  26 ++++
  5 files changed, 264 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/testmod_btf.c
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +1 -0
index a5e246f7b202..418e10311c33 100644
@@ -1,5 +1,6 @@
  // SPDX-License-Identifier: GPL-2.0
  /* Copyright (c) 2020 Facebook */
+#include <linux/bpf.h>
  #include <linux/btf.h>
  #include <linux/btf_ids.h>
  #include <linux/error-injection.h>

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c BTF_ID_FLAGS(func, bpf_kfunc_call_test_static_unused_arg) · line 526

Quoted 4 lines

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +1 -0

@@ -522,11 +523,66 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_static_unused_arg)
  BTF_ID_FLAGS(func, bpf_kfunc_call_test_offset)
  BTF_SET8_END(bpf_testmod_check_kfunc_ids)
 
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES

I don't think it is needed. It should have been enabled (directly/indirectly) by the selftests/bpf/config already.

[ ... ]

Kui-Feng Lee

2023-11-22 23:59 UTC #28 lore raw

Re: [PATCH bpf-next v11 13/13] selftests/bpf: test case for register_bpf_struct_ops().

Quoted 29 lines · Martin KaFai Lau

Quoted 28 lines · On 11/6/23 12:12 PM, [email protected] wrote:
From: Kui-Feng Lee <[email protected]>
Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module.
Signed-off-by Kui-Feng Lee <[email protected]>
---
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  59 +++++++
  .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   5 +
  .../bpf/prog_tests/test_struct_ops_module.c   | 144 ++++++++++++++++++
  .../selftests/bpf/progs/struct_ops_module.c   |  30 ++++
  .../testing/selftests/bpf/progs/testmod_btf.c |  26 ++++
  5 files changed, 264 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_module.c create mode 100644 tools/testing/selftests/bpf/progs/testmod_btf.c
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +1 -0
index a5e246f7b202..418e10311c33 100644
@@ -1,5 +1,6 @@
   // SPDX-License-Identifier: GPL-2.0
   /* Copyright (c) 2020 Facebook */
+#include <linux/bpf.h>
   #include <linux/btf.h>
   #include <linux/btf_ids.h>

Quoted 8 lines

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c BTF_ID_FLAGS(func, · line 526
Quoted 6 lines
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +1 -0
   #include <linux/error-injection.h>
@@ -522,11 +523,66 @@ BTF_ID_FLAGS(func, 
 bpf_kfunc_call_test_static_unused_arg)
   BTF_ID_FLAGS(func, bpf_kfunc_call_test_offset)
   BTF_SET8_END(bpf_testmod_check_kfunc_ids)
+#ifdef CONFIG_DEBUG_INFO_BTF_MODULES
I don't think it is needed. It should have been enabled (directly/indirectly) by the selftests/bpf/config already.

Got it!

Quoted 3 lines

[ ... ]

Hou Tao

2023-11-17 10:45 UTC #29 lore raw

Re: [PATCH bpf-next v11 13/13] selftests/bpf: test case for register_bpf_struct_ops().

In reply to [email protected]

Hi,

Quoted 7 lines · On 11/7/2023 4:12 AM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module.
Signed-off-by Kui-Feng Lee <[email protected]>

SNIP

tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c line 6

Quoted 12 lines

tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c new file +6 -0

new file mode 100644
index 000000000000..49f4a4460642
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include <time.h>
+
+#include "rcu_tasks_trace_gp.skel.h"

It is no needed anymore. It seems it is a leftover from last revision.

Kui-Feng Lee

2023-11-23 00:01 UTC #30 lore raw

Re: [PATCH bpf-next v11 13/13] selftests/bpf: test case for register_bpf_struct_ops().

Quoted 11 lines · Hou Tao

Hi,
Quoted 7 lines · On 11/7/2023 4:12 AM, [email protected] wrote:
From: Kui-Feng Lee <[email protected]>
Create a new struct_ops type called bpf_testmod_ops within the bpf_testmod module. When a struct_ops object is registered, the bpf_testmod module will invoke test_2 from the module.
Signed-off-by Kui-Feng Lee <[email protected]>
SNIP

Quoted 15 lines

tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c line 6
Quoted 12 lines
tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c new file +6 -0
new file mode 100644
index 000000000000..49f4a4460642
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */
+#include <test_progs.h>
+#include <time.h>
+
+#include "rcu_tasks_trace_gp.skel.h"
It is no needed anymore. It seems it is a leftover from last revision.

You are right! I will clean it up.

Martin KaFai Lau

reviewer

2023-11-10 06:57 UTC #31 lore raw

In reply to [email protected]

Quoted 32 lines · On 11/6/23 12:12 PM, [email protected] wrote:

From: Kui-Feng Lee <[email protected]>
Given the current constraints of the current implementation, struct_ops cannot be registered dynamically. This presents a significant limitation for modules like coming fuse-bpf, which seeks to implement a new struct_ops type. To address this issue, a new API is introduced that allows the registration of new struct_ops types from modules.
Previously, struct_ops types were defined in bpf_struct_ops_types.h and collected as a static array. The new API lets callers add new struct_ops types dynamically. The static array has been removed and replaced by the per-btf struct_ops_tab.
The struct_ops subsystem relies on BTF to determine the layout of values in a struct_ops map and identify the subsystem that the struct_ops map registers to. However, the kernel BTF does not include the type information of struct_ops types defined by a module. The struct_ops subsystem requires knowledge of the corresponding module for a given struct_ops map and the utilization of BTF information from that module. We empower libbpf to determine the correct module for accessing the BTF information and pass an identity (FD) of the module btf to the kernel. The kernel looks up type information and registered struct_ops types directly from the given btf.
If a module exits while one or more struct_ops maps still refer to a struct_ops type defined by the module, it can lead to unforeseen complications. Therefore, it is crucial to ensure that a module remains intact as long as any struct_ops map is still linked to a struct_ops type defined by the module. To achieve this, every struct_ops map holds a reference to the module while being registered.

Quoted 8 lines

Changes from v10:
- Guard btf.c from CONFIG_BPF_JIT=n. This patchset has introduced symbols from bpf_struct_ops.c which is only built when CONFIG_BPF_JIT=y.
- Fix the warning of unused errout_free label by moving code that is leaked to patch 8 to patch 7.

Thanks for the patches and working on this feature.

One thing that still needs to check is the "bpftool struct_ops dump" support for kmod's btf. The bpftool changes can be a followup. However, please check if the current uapi has what it needs. A quick look is the userspace should be able to find the kmod btf from the map_info->btf_vmlinux_value_type_id.

We discussed a bit offline on patch 8 about putting the btf and module refcnt together in bpf_struct_ops_map_free (but before synchronize_rcu_mult) which should further simplify patch 8 also. hope that will work out.

Looking forward to v12.