Skip to contentSkip to conversation

Support cookie for link-based struct_ops attachment

RFC In review Amery Hung 4 participants 18 messages Jul 8, 2025 – Jul 14, 2025

Subsystems (5)
lore
Share & tools
lore

Amery Hung

author
#0 lore raw

cover Support cookie for link-based struct_ops attachment

* Motivation *

Currently, sched_ext only supports a single scheduler. scx kfuncs can reference the calling scheduler simply with a global pointer, scx_root, whether being called in struct_ops, tracing, syscall or async callback. However, when sched_ext moves to support multiple scx schedulers, scx kfunc without task_struct argument won't be able to easily refer to the calling scheduler without passing additional information.

The vision is to be able to view struct_ops, tracing, syscall programs and async callbacks in a file as part of a scheduler, and have the ability to refer to the scheduler in kfuncs called from any of the programs.

However, right now there is not a notion that connects bpf programs in a file in the kernel. Until it is clear what that should be, as a stopgap, we would like to reuse cookie to associate bpf programs belonging to the same scheduler.

* Design *

The interim solution is to use bpf cookie as the scheduler id, and add the cookie as an argument to scx kfuncs without task_struct. By maintaining the cookie to scheduler mapping in sched_ext, kfuncs will be able to refer to the calling scheduler internally using cookie.

For the user space loader, it should use the same cookie when attaching bpf programs. In bpf programs, they will call bpf_get_attach_cookie() to get the cookie and pass it to scx kfuncs.

The following patches are the first part of the kernel-side plumbing: to support cookie for link-based struct_ops attachment. struct_ops already uses bpf trampoline. The only problem is the trampoline is being populated during map_update when the cookie is not known yet. This patchset moves the trampoline preparation from map_update to link_create for link-based struct_ops attachment. Then, we extend the UAPI to take the cookie and propagate it all the way to the struct_ops link_create. Finally, bpf_get_attach_cookie() is allowed to struct_ops programs.

A problem that has not been addressed in this RFC is that by moving trampoline preparation to link_create, creating multiple links using the same struct_ops map becomes problematic. A simple solution may be just disable if there is no obvious use case for it.

* Alternative *

Martin has suggested a way that involves less kernel changes: Using struct_ops map id as the sched_ext scheduler id. Instead of further complicating struct_ops, sched_ext can create the struct_ops map id to scheduler mapping in reg(). The id will also be stored in a global variable in the file referenced by different types of programs. The rest is similar to the cookie approach: kfuncs without task_struct will need to add an argument, and they will lookup the scheduler by the id.

* Previous approach *

The previous solution was to set a "this pointer" pointing to the struct_ops struct during struct_ops map_update. Then, kfuncs can get a pointer to the scheduler with a __prog argument. However, this approach is broken as it would only work for struct_ops programs.

Some thought after Martin brought up the idea

I think we should use the map id instead of cookie. Especially if we are going to disable creating multiple links per map, the cookie effectively becomes the same as the map id.

Amery Hung (4):
bpf: Factor out bpf_struct_ops_prepare_attach()
bpf: Support cookie for linked-based struct_ops attachment
libbpf: Support link-based struct_ops attach with options selftests/bpf: Test bpf_get_attach_cookie() in struct_ops program

 include/uapi/linux/bpf.h                      |   3 +
 kernel/bpf/bpf_struct_ops.c                   | 199 ++++++++++++------
 kernel/bpf/helpers.c                          |  18 ++
 tools/include/uapi/linux/bpf.h                |   3 +
 tools/lib/bpf/bpf.c                           |   5 +
 tools/lib/bpf/bpf.h                           |   3 +
 tools/lib/bpf/libbpf.c                        |  14 +-
 tools/lib/bpf/libbpf.h                        |  10 +
 tools/lib/bpf/libbpf.map                      |   1 +
 .../bpf/prog_tests/test_struct_ops_cookie.c   |  42 ++++
 .../selftests/bpf/progs/struct_ops_cookie.c   |  48 +++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |   1 +
 12 files changed, 280 insertions(+), 67 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_cookie.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_cookie.c
Signature · 1 line
2.47.1
author
#1 lore raw

RFC 1/4 bpf: Factor out bpf_struct_ops_prepare_attach()

To support cookie for struct_ops attachment, the trampoline needs be created during link_create. To prepare for it, factor out the creation of trampoline and ksym from struct_ops map update. No functional change.

---
 kernel/bpf/bpf_struct_ops.c | 148 ++++++++++++++++++++++--------------
 1 file changed, 93 insertions(+), 55 deletions(-)
kernel/bpf/bpf_struct_ops.c +93 -55
index 96113633e391..4d150e99a86c 100644
@@ -673,6 +673,95 @@ static void bpf_struct_ops_map_free_ksyms(struct bpf_struct_ops_map *st_map)
 	}
 }
 
+static int bpf_struct_ops_prepare_attach(struct bpf_struct_ops_map *st_map)
+{
+	const struct bpf_struct_ops *st_ops = st_map->st_ops_desc->st_ops;
+	const struct btf_type *t = st_map->st_ops_desc->type;
+	struct bpf_link **plink = st_map->links;
+	struct bpf_ksym **pksym = st_map->ksyms;
+	const struct btf_member *member;
+	struct bpf_tramp_links *tlinks;
+	void *udata, *kdata;
+	int i, err = 0;
+	u32 trampoline_start, image_off = 0;
+	void *cur_image = NULL, *image = NULL;
+	const char *tname, *mname;
+
+	tlinks = kcalloc(BPF_TRAMP_MAX, sizeof(*tlinks), GFP_KERNEL);
+	if (!tlinks)
+		return -ENOMEM;
+
+	udata = &st_map->uvalue->data;
+	kdata = &st_map->kvalue.data;
+
+	tname = btf_name_by_offset(st_map->btf, t->name_off);
+
+	for_each_member(i, t, member) {
+		const struct btf_type *ptype;
+		struct bpf_tramp_link *link;
+		struct bpf_ksym *ksym;
+		u32 moff, id;
+
+		ptype = btf_type_resolve_ptr(st_map->btf, member->type, NULL);
+		if (!ptype || !btf_type_is_func_proto(ptype))
+			continue;
+
+		moff = __btf_member_bit_offset(t, member) / 8;
+
+		id = *(unsigned long *)(udata + moff);
+		if (!id)
+			continue;
+
+		mname = btf_name_by_offset(st_map->btf, member->name_off);
+		link = container_of(*plink++, struct bpf_tramp_link, link);
+
+		ksym = kzalloc(sizeof(*ksym), GFP_USER);
+		if (!ksym) {
+			err = -ENOMEM;
+			goto out;
+		}
+		*pksym++ = ksym;
+
+		trampoline_start = image_off;
+		err = bpf_struct_ops_prepare_trampoline(tlinks, link,
+						&st_ops->func_models[i],
+						*(void **)(st_ops->cfi_stubs + moff),
+						&image, &image_off,
+						st_map->image_pages_cnt < MAX_TRAMP_IMAGE_PAGES);
+		if (err)
+			goto out;
+
+		if (cur_image != image) {
+			st_map->image_pages[st_map->image_pages_cnt++] = image;
+			cur_image = image;
+			trampoline_start = 0;
+		}
+
+		*(void **)(kdata + moff) = image + trampoline_start + cfi_get_offset();
+
+		/* init ksym for this trampoline */
+		bpf_struct_ops_ksym_init(tname, mname,
+					 image + trampoline_start,
+					 image_off - trampoline_start,
+					 ksym);
+	}
+
+	if (st_ops->validate) {
+		err = st_ops->validate(kdata);
+		if (err)
+			goto out;
+	}
+	for (i = 0; i < st_map->image_pages_cnt; i++) {
+		err = arch_protect_bpf_trampoline(st_map->image_pages[i],
+						  PAGE_SIZE);
+		if (err)
+			goto out;
+	}
+out:
+	kfree(tlinks);
+	return err;
+}
+
 static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 					   void *value, u64 flags)
 {
@@ -683,14 +772,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	const struct btf_type *module_type;
 	const struct btf_member *member;
 	const struct btf_type *t = st_ops_desc->type;
-	struct bpf_tramp_links *tlinks;
 	void *udata, *kdata;
 	int prog_fd, err;
-	u32 i, trampoline_start, image_off = 0;
-	void *cur_image = NULL, *image = NULL;
+	u32 i;
 	struct bpf_link **plink;
-	struct bpf_ksym **pksym;
-	const char *tname, *mname;
 
 	if (flags)
 		return -EINVAL;
@@ -710,10 +795,6 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	if (uvalue->common.state || refcount_read(&uvalue->common.refcnt))
 		return -EINVAL;
 
-	tlinks = kcalloc(BPF_TRAMP_MAX, sizeof(*tlinks), GFP_KERNEL);
-	if (!tlinks)
-		return -ENOMEM;
-
 	uvalue = (struct bpf_struct_ops_value *)st_map->uvalue;
 	kvalue = (struct bpf_struct_ops_value *)&st_map->kvalue;
 
@@ -730,18 +811,14 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	kdata = &kvalue->data;
 
 	plink = st_map->links;
-	pksym = st_map->ksyms;
-	tname = btf_name_by_offset(st_map->btf, t->name_off);
 	module_type = btf_type_by_id(btf_vmlinux, st_ops_ids[IDX_MODULE_ID]);
 	for_each_member(i, t, member) {
 		const struct btf_type *mtype, *ptype;
 		struct bpf_prog *prog;
 		struct bpf_tramp_link *link;
-		struct bpf_ksym *ksym;
 		u32 moff;
 
 		moff = __btf_member_bit_offset(t, member) / 8;
-		mname = btf_name_by_offset(st_map->btf, member->name_off);
 		ptype = btf_type_resolve_ptr(st_map->btf, member->type, NULL);
 		if (ptype == module_type) {
 			if (*(void **)(udata + moff))
@@ -811,51 +888,13 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 			      &bpf_struct_ops_link_lops, prog);
 		*plink++ = &link->link;
 
-		ksym = kzalloc(sizeof(*ksym), GFP_USER);
-		if (!ksym) {
-			err = -ENOMEM;
-			goto reset_unlock;
-		}
-		*pksym++ = ksym;
-
-		trampoline_start = image_off;
-		err = bpf_struct_ops_prepare_trampoline(tlinks, link,
-						&st_ops->func_models[i],
-						*(void **)(st_ops->cfi_stubs + moff),
-						&image, &image_off,
-						st_map->image_pages_cnt < MAX_TRAMP_IMAGE_PAGES);
-		if (err)
-			goto reset_unlock;
-
-		if (cur_image != image) {
-			st_map->image_pages[st_map->image_pages_cnt++] = image;
-			cur_image = image;
-			trampoline_start = 0;
-		}
-
-		*(void **)(kdata + moff) = image + trampoline_start + cfi_get_offset();
-
 		/* put prog_id to udata */
 		*(unsigned long *)(udata + moff) = prog->aux->id;
-
-		/* init ksym for this trampoline */
-		bpf_struct_ops_ksym_init(tname, mname,
-					 image + trampoline_start,
-					 image_off - trampoline_start,
-					 ksym);
 	}
 
-	if (st_ops->validate) {
-		err = st_ops->validate(kdata);
-		if (err)
-			goto reset_unlock;
-	}
-	for (i = 0; i < st_map->image_pages_cnt; i++) {
-		err = arch_protect_bpf_trampoline(st_map->image_pages[i],
-						  PAGE_SIZE);
-		if (err)
-			goto reset_unlock;
-	}
+	err = bpf_struct_ops_prepare_attach(st_map);
+	if (err)
+		goto reset_unlock;
 
 	if (st_map->map.map_flags & BPF_F_LINK) {
 		err = 0;
@@ -897,7 +936,6 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
 	memset(uvalue, 0, map->value_size);
 	memset(kvalue, 0, map->value_size);
 unlock:
-	kfree(tlinks);
 	mutex_unlock(&st_map->lock);
 	if (!err)
 		bpf_struct_ops_map_add_ksyms(st_map);
Signature · 1 line
2.47.1
author
#2 lore raw

RFC 2/4 bpf: Support cookie for linked-based struct_ops attachment

In reply to Amery Hung

Support cookie when attaching a struct_ops map using link. The cookie is associated with the link and can be retrieved in bpf struct_ops program using bpf_get_attach_cookie().

Implementation wise, trampoline and ksyms preparation are deferred from map_update to link_create for struct_ops maps with BPF_F_LINK. Since bpf cookie is hardcoded to the trampoline in arch_prepare_bpf_trampoline(), it must be done when cookie is known (i.e., link_create). The trampoline and ksyms are freed once a link is detached as the struct_ops map is no longer associated with the link.

TODO:
A struct_ops map with BPF_F_LINK should be prevented from being used to create another link after this patch since a struct_ops map currently only support a set of trampoline. This may be done reusing the state from non-BPF_F_LINK struct_ops map: set the state from READY to INUSE in link_create, and check the state in link_create and link_update.

---
 include/uapi/linux/bpf.h       |  3 ++
 kernel/bpf/bpf_struct_ops.c    | 59 +++++++++++++++++++++++++---------
 kernel/bpf/helpers.c           | 18 +++++++++++
 tools/include/uapi/linux/bpf.h |  3 ++
 4 files changed, 68 insertions(+), 15 deletions(-)
include/uapi/linux/bpf.h +3 -0
index 0670e15a6100..4708d0783130 100644
@@ -1818,6 +1818,9 @@ union bpf_attr {
 				};
 				__u64		expected_revision;
 			} cgroup;
+			struct {
+				__u64		cookie;
+			} struct_ops;
 		};
 	} link_create;
 
kernel/bpf/bpf_struct_ops.c +44 -15

138 lines not on this page · raw message

kernel/bpf/helpers.c +18 -0
index 3d33181d5e67..4075fdd1533f 100644
@@ -1894,6 +1894,21 @@ static const struct bpf_func_proto bpf_dynptr_data_proto = {
 	.arg3_type	= ARG_CONST_ALLOC_SIZE_OR_ZERO,
 };
 
+BPF_CALL_1(bpf_get_attach_cookie_struct_ops, void *, ctx)
+{
+	struct bpf_tramp_run_ctx *run_ctx;
+
+	run_ctx = container_of(current->bpf_ctx, struct bpf_tramp_run_ctx, run_ctx);
+	return run_ctx->bpf_cookie;
+}
+
+static const struct bpf_func_proto bpf_get_attach_cookie_proto_struct_ops = {
+	.func		= bpf_get_attach_cookie_struct_ops,
+	.gpl_only	= false,
+	.ret_type	= RET_INTEGER,
+	.arg1_type	= ARG_PTR_TO_CTX,
+};
+
 const struct bpf_func_proto bpf_get_current_task_proto __weak;
 const struct bpf_func_proto bpf_get_current_task_btf_proto __weak;
 const struct bpf_func_proto bpf_probe_read_user_proto __weak;
@@ -1962,6 +1977,9 @@ bpf_base_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
 		return &bpf_get_ns_current_pid_tgid_proto;
 	case BPF_FUNC_get_current_uid_gid:
 		return &bpf_get_current_uid_gid_proto;
+	case BPF_FUNC_get_attach_cookie:
+		return prog->type == BPF_PROG_TYPE_STRUCT_OPS ?
+		       &bpf_get_attach_cookie_proto_struct_ops : NULL;
 	default:
 		break;
 	}
tools/include/uapi/linux/bpf.h +3 -0
index 0670e15a6100..4708d0783130 100644
@@ -1818,6 +1818,9 @@ union bpf_attr {
 				};
 				__u64		expected_revision;
 			} cgroup;
+			struct {
+				__u64		cookie;
+			} struct_ops;
 		};
 	} link_create;
 
Signature · 1 line
2.47.1
#3 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

kernel/bpf/bpf_struct_ops.c static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 5 lines · Amery Hung
kernel/bpf/bpf_struct_ops.c +1 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
  		goto unlock;
  	}
 
+	err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

If I understand correctly, the kfunc implementation needs to look up the scx_ops instance (i.e., the struct_ops map) from the map->id/cookie. There is a similar map->id lookup in bpf_map_get_fd_by_id(), which requires acquiring a spin_lock. If performance is a concern, we can investigate whether it can be rcu-ified. From a quick glance, bpf_map_free_id() is called before call_rcu(). Note that bpf_struct_ops_map_free() will wait for an RCU grace period.

#4 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 4 lines · Martin KaFai Lau

If I understand correctly, the kfunc implementation needs to look up the scx_ops instance (i.e., the struct_ops map) from the map->id/cookie. There is a similar map->id lookup in bpf_map_get_fd_by_id(), which requires acquiring a spin_lock. If performance is a concern, we can investigate whether it can be rcu-ified.
From a quick glance, bpf_map_free_id() is called before call_rcu(). Note that
bpf_struct_ops_map_free() will wait for an RCU grace period.

After looking at bpf_map_put more, not all BPF map frees wait for RCU. It may be cleaner to store the sched_ext_ops map in a separate (i.e., new) lookup data structure. That may also make it easier to ensure the looked up map is a struct_ops map (or in particular a sched_ext_ops).

author
#5 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

In reply to Martin KaFai Lau
Quoted 6 lines · Martin KaFai Lau
Quoted 4 lines · Amery Hung
kernel/bpf/bpf_struct_ops.c +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
              goto unlock;
      }
 
Quoted 9 lines

kernel/bpf/bpf_struct_ops.c static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
kernel/bpf/bpf_struct_ops.c +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

Quoted 7 lines

If I understand correctly, the kfunc implementation needs to look up the scx_ops instance (i.e., the struct_ops map) from the map->id/cookie. There is a similar map->id lookup in bpf_map_get_fd_by_id(), which requires acquiring a spin_lock. If performance is a concern, we can investigate whether it can be rcu-ified.
From a quick glance, bpf_map_free_id() is called before call_rcu(). Note that
bpf_struct_ops_map_free() will wait for an RCU grace period.

#6 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
kernel/bpf/bpf_struct_ops.c +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

kernel/bpf/bpf_struct_ops.c static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
kernel/bpf/bpf_struct_ops.c +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

For tracing program, the bpf cookie seems to be an existing mechanism that can have any value (?). Thus, user space is free to store the map->id in it also. It can also choose to store the map->id in a bpf global variable if it has other uses for the bpf cookie. I think both should work similarly.

author
#7 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
kernel/bpf/bpf_struct_ops.c +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

kernel/bpf/bpf_struct_ops.c static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
kernel/bpf/bpf_struct_ops.c +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the map->id stored in bpf_tramp_run_ctx::bpf_cookie. Maybe bpf_struct_ops_get_map_id()?

Another approach is to complete the plumbing of this patchset by moving trampoline and ksyms from map to link. Right now it is broken when creating multiple links from the same map as can be seen in the CI. I think this is better as we don't create another unique thing for struct_ops.

WDYT?

Quoted 4 lines

For tracing program, the bpf cookie seems to be an existing mechanism that can have any value (?). Thus, user space is free to store the map->id in it also. It can also choose to store the map->id in a bpf global variable if it has other uses for the bpf cookie. I think both should work similarly.

#8 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 42 lines · Amery Hung
Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
kernel/bpf/bpf_struct_ops.c +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

kernel/bpf/bpf_struct_ops.c static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
kernel/bpf/bpf_struct_ops.c +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

Quoted 6 lines

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the

yes, if absolutely necessary, sure, let's reuse the spot that is reserved for cookie inside the trampoline, but let's not expose this as real BPF cookie (i.e., let's not allow bpf_get_attach_cookie() helper for struct_ops), because BPF cookie is meant to be fully user controllable and used for whatever they deem necessary. Not necessarily to just identify the struct_ops map. So it will be a huge violation to just pre-define what BPF cookie value is for struct_ops.

Quoted 10 lines

map->id stored in bpf_tramp_run_ctx::bpf_cookie. Maybe bpf_struct_ops_get_map_id()?

Another approach is to complete the plumbing of this patchset by moving trampoline and ksyms from map to link. Right now it is broken when creating multiple links from the same map as can be seen in the CI. I think this is better as we don't create another unique thing for struct_ops.

WDYT?

I think that is a logical thing to do, because BPF link represents attachment, and trampoline should conceptually correspond to an attachment, not to the thing that is being attached (and might be attached to multiple places, potentially). We have this approach with the fentry/fexit program's trampoline, so it would be nice to move struct_ops to the same model.

Quoted 6 lines
Quoted 5 lines

For tracing program, the bpf cookie seems to be an existing mechanism that can have any value (?). Thus, user space is free to store the map->id in it also. It can also choose to store the map->id in a bpf global variable if it has other uses for the bpf cookie. I think both should work similarly.

author
#9 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 51 lines · Andrii Nakryiko
Quoted 42 lines · Amery Hung
Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
kernel/bpf/bpf_struct_ops.c +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

kernel/bpf/bpf_struct_ops.c static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
kernel/bpf/bpf_struct_ops.c +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

Quoted 6 lines

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the

Quoted 8 lines

yes, if absolutely necessary, sure, let's reuse the spot that is reserved for cookie inside the trampoline, but let's not expose this as real BPF cookie (i.e., let's not allow bpf_get_attach_cookie() helper for struct_ops), because BPF cookie is meant to be fully user controllable and used for whatever they deem necessary. Not necessarily to just identify the struct_ops map. So it will be a huge violation to just pre-define what BPF cookie value is for struct_ops.

We had some offline discussions and figured out this will not work well.

sched_ext users already call scx kfuncs in global subprograms. If we choose to add bpf_get_struct_ops_id() to get the id to be passed to scx kfuncs, it will force the user to create two sets of the same global subprog. The one called by struct_ops that calls bpf_get_struct_ops_id() and tracing programs that calls bpf_get_attach_cookie().

Quoted 24 lines
Quoted 10 lines

map->id stored in bpf_tramp_run_ctx::bpf_cookie. Maybe bpf_struct_ops_get_map_id()?

Another approach is to complete the plumbing of this patchset by moving trampoline and ksyms from map to link. Right now it is broken when creating multiple links from the same map as can be seen in the CI. I think this is better as we don't create another unique thing for struct_ops.

WDYT?

I think that is a logical thing to do, because BPF link represents attachment, and trampoline should conceptually correspond to an attachment, not to the thing that is being attached (and might be attached to multiple places, potentially). We have this approach with the fentry/fexit program's trampoline, so it would be nice to move struct_ops to the same model.

Quoted 6 lines
Quoted 5 lines

For tracing program, the bpf cookie seems to be an existing mechanism that can have any value (?). Thus, user space is free to store the map->id in it also. It can also choose to store the map->id in a bpf global variable if it has other uses for the bpf cookie. I think both should work similarly.

#10 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 63 lines · Amery Hung
Quoted 51 lines · Andrii Nakryiko
Quoted 42 lines · Amery Hung
Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
(quoted diff) +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

Review static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
(quoted diff) +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

Quoted 6 lines

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the

Quoted 8 lines

yes, if absolutely necessary, sure, let's reuse the spot that is reserved for cookie inside the trampoline, but let's not expose this as real BPF cookie (i.e., let's not allow bpf_get_attach_cookie() helper for struct_ops), because BPF cookie is meant to be fully user controllable and used for whatever they deem necessary. Not necessarily to just identify the struct_ops map. So it will be a huge violation to just pre-define what BPF cookie value is for struct_ops.

Quoted 8 lines

We had some offline discussions and figured out this will not work well.

sched_ext users already call scx kfuncs in global subprograms. If we choose to add bpf_get_struct_ops_id() to get the id to be passed to scx kfuncs, it will force the user to create two sets of the same global subprog. The one called by struct_ops that calls bpf_get_struct_ops_id() and tracing programs that calls bpf_get_attach_cookie().

Can we put cookie into map_extra during st_ops map creation time and later copy it into actual cookie place in a trampoline?

author
#11 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 74 lines · Alexei Starovoitov
Quoted 63 lines · Amery Hung
Quoted 51 lines · Andrii Nakryiko
Quoted 42 lines · Amery Hung
Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
(quoted diff) +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

Review static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
(quoted diff) +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

Quoted 6 lines

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the

Quoted 8 lines

yes, if absolutely necessary, sure, let's reuse the spot that is reserved for cookie inside the trampoline, but let's not expose this as real BPF cookie (i.e., let's not allow bpf_get_attach_cookie() helper for struct_ops), because BPF cookie is meant to be fully user controllable and used for whatever they deem necessary. Not necessarily to just identify the struct_ops map. So it will be a huge violation to just pre-define what BPF cookie value is for struct_ops.

Quoted 8 lines

We had some offline discussions and figured out this will not work well.

sched_ext users already call scx kfuncs in global subprograms. If we choose to add bpf_get_struct_ops_id() to get the id to be passed to scx kfuncs, it will force the user to create two sets of the same global subprog. The one called by struct_ops that calls bpf_get_struct_ops_id() and tracing programs that calls bpf_get_attach_cookie().

Quoted 2 lines

Can we put cookie into map_extra during st_ops map creation time and later copy it into actual cookie place in a trampoline?

It should work. Currently, it makes sense to have cookie in struct_ops map as all struct_ops implementers (hid, tcp cc, qdisc, sched_ext) do not allow multi-attachment of the same map. A struct_ops map is effectively an unique attachment for now.

Additionally, we will be able to support cookie for non-link struct_ops with this way.

This approach will not block future effort to support link-specific cookie if there is such a use case. We can revisit this patchset then.

#12 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 77 lines · Amery Hung
Quoted 74 lines · Alexei Starovoitov
Quoted 71 lines · Amery Hung
Quoted 51 lines · Andrii Nakryiko
Quoted 42 lines · Amery Hung
Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
(quoted diff) +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

Review static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
(quoted diff) +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

Quoted 6 lines

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the

Quoted 8 lines

yes, if absolutely necessary, sure, let's reuse the spot that is reserved for cookie inside the trampoline, but let's not expose this as real BPF cookie (i.e., let's not allow bpf_get_attach_cookie() helper for struct_ops), because BPF cookie is meant to be fully user controllable and used for whatever they deem necessary. Not necessarily to just identify the struct_ops map. So it will be a huge violation to just pre-define what BPF cookie value is for struct_ops.

We had some offline discussions and figured out this will not work well.

sched_ext users already call scx kfuncs in global subprograms. If we choose to add bpf_get_struct_ops_id() to get the id to be passed to scx kfuncs, it will force the user to create two sets of the same global subprog. The one called by struct_ops that calls bpf_get_struct_ops_id() and tracing programs that calls bpf_get_attach_cookie().

Quoted 7 lines
Quoted 2 lines

Can we put cookie into map_extra during st_ops map creation time and later copy it into actual cookie place in a trampoline?

It should work. Currently, it makes sense to have cookie in struct_ops map as all struct_ops implementers (hid, tcp cc, qdisc, sched_ext) do not allow multi-attachment of the same map. A struct_ops map is effectively an unique attachment for now.

Is there any conceptual reason why struct_ops map shouldn't be allowed to be attached to multiple places? If not, should we try to lift this restriction and not invent struct_ops-specific BPF cookie APIs?

From libbpf POV, struct_ops map is created implicitly from the struct_ops variable. There is no map_flags field there. We'll need to add new special APIs and/or conventions just to be able to set map_flags.

Quoted 6 lines

Additionally, we will be able to support cookie for non-link struct_ops with this way.

This approach will not block future effort to support link-specific cookie if there is such a use case. We can revisit this patchset then.

It will create two ways to specify BPF cookie for struct_ops: (legacy and special way) through map_flags and common one through the LINK_CREATE command (and I guess we'd need to reject LINK_CREATE if cookie was already set through map_flags, right?). Why confuse users like that?

From what I understand, the problem is that currently struct_ops map's BPF trampoline(s) are created a bit too early, before the attachment step. How hard would it be to move trampoline creation to an actual attachment time? Should we seriously consider this before we invent new struct_ops-specific exceptions for BPF cookie?

author
#13 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 109 lines · Andrii Nakryiko
Quoted 77 lines · Amery Hung
Quoted 74 lines · Alexei Starovoitov
Quoted 71 lines · Amery Hung
Quoted 51 lines · Andrii Nakryiko
Quoted 42 lines · Amery Hung
Quoted 29 lines · Martin KaFai Lau
Quoted 13 lines · Amery Hung
Quoted 5 lines · Amery Hung
Quoted 4 lines
(quoted diff) +0 -0
@@ -906,6 +904,10 @@ static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key,
               goto unlock;
       }
 
Quoted 8 lines

Review static long bpf_struct_ops_map_update_elem(struct bpf_map *map, void *key, · line 907

Quoted 1 line
(quoted diff) +1 -0
+     err = bpf_struct_ops_prepare_attach(st_map, 0);

A follow-up on the "using the map->id as the cookie" comment in the cover letter. I meant to use the map->id here instead of 0. If the cookie is intended to identify a particular struct_ops instance (i.e., the struct_ops map), then map->id should be a good fit, and it is automatically generated by the kernel during the map creation. As a result, I suspect that most of the changes in patch 1 and patch 2 will not be needed.

Quoted 3 lines

Do you mean keep using cookie as the mechanism to associate programs, but for struct_ops the cookie will be map->id (i.e., bpf_get_attah_cookie() in struct_ops will return map->id)?

I meant to use the map->id as the bpf_cookie stored in the bpf_tramp_run_ctx. Then there is no need for user space to generate a unique cookie during link_create. The kernel has already generated a unique ID in the map->id. The map->id is available during the bpf_struct_ops_map_update_elem(). Then there is also no need to distinguish between SEC(".struct_ops") vs SEC(".struct_ops.link"). Most of the patch 1 and patch 2 will not be needed.

A minor detail: note that the same struct ops program can be used in different trampolines. Thus, to be specific, the bpf cookie is stored in the trampoline.

Quoted 10 lines

If the question is about bpf global variable vs bpf cookie, yeah, I think using a bpf global variable should also work. The global variable can be initialized before libbpf's bpf_map__attach_struct_ops(). At that time, the map->id should be known already. I don't have a strong opinion on reusing the bpf cookie in the struct ops trampoline. No one is using it now, so it is available to be used. Exposing BPF_FUNC_get_attach_cookie for struct ops programs is pretty cheap also. Using bpf cookie to allow the struct ops program to tell which struct_ops map is calling it seems to fit well also after sleeping on it a bit. bpf global variable will also break if a bpf_prog.o has more than one SEC(".struct_ops").

Quoted 6 lines

While both of them work, using cookie instead of global variable is one less thing for the user to take care of (i.e., slightly better usability).

With the approach you suggested, to not mix the existing semantics of bpf cookie, I think a new struct_ops kfuncs is needed to retrieve the

Quoted 8 lines

yes, if absolutely necessary, sure, let's reuse the spot that is reserved for cookie inside the trampoline, but let's not expose this as real BPF cookie (i.e., let's not allow bpf_get_attach_cookie() helper for struct_ops), because BPF cookie is meant to be fully user controllable and used for whatever they deem necessary. Not necessarily to just identify the struct_ops map. So it will be a huge violation to just pre-define what BPF cookie value is for struct_ops.

We had some offline discussions and figured out this will not work well.

sched_ext users already call scx kfuncs in global subprograms. If we choose to add bpf_get_struct_ops_id() to get the id to be passed to scx kfuncs, it will force the user to create two sets of the same global subprog. The one called by struct_ops that calls bpf_get_struct_ops_id() and tracing programs that calls bpf_get_attach_cookie().

Quoted 7 lines
Quoted 2 lines

Can we put cookie into map_extra during st_ops map creation time and later copy it into actual cookie place in a trampoline?

It should work. Currently, it makes sense to have cookie in struct_ops map as all struct_ops implementers (hid, tcp cc, qdisc, sched_ext) do not allow multi-attachment of the same map. A struct_ops map is effectively an unique attachment for now.

Is there any conceptual reason why struct_ops map shouldn't be allowed to be attached to multiple places? If not, should we try to lift this restriction and not invent struct_ops-specific BPF cookie APIs?

From libbpf POV, struct_ops map is created implicitly from the struct_ops variable. There is no map_flags field there. We'll need to add new special APIs and/or conventions just to be able to set map_flags.

Quoted 6 lines

Additionally, we will be able to support cookie for non-link struct_ops with this way.

This approach will not block future effort to support link-specific cookie if there is such a use case. We can revisit this patchset then.

It will create two ways to specify BPF cookie for struct_ops: (legacy and special way) through map_flags and common one through the LINK_CREATE command (and I guess we'd need to reject LINK_CREATE if cookie was already set through map_flags, right?). Why confuse users like that?

Quoted 5 lines

From what I understand, the problem is that currently struct_ops map's BPF trampoline(s) are created a bit too early, before the attachment step. How hard would it be to move trampoline creation to an actual attachment time? Should we seriously consider this before we invent new struct_ops-specific exceptions for BPF cookie?

Yeah...

I realized that while map_extra works, the cookie needs to be passed to the kernel in map creation time and will create the confusion you mentioned in libbpf.

I am fixing the patchset by moving trampoline and ksyms to bpf_struct_ops_link. It shouldn't complicate struct_ops code too much (finger cross).

#14 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

Quoted 8 lines · Amery Hung
Quoted 8 lines
Quoted 4 lines

It should work. Currently, it makes sense to have cookie in struct_ops map as all struct_ops implementers (hid, tcp cc, qdisc, sched_ext) do not allow multi-attachment of the same map. A struct_ops map is effectively an unique attachment for now.

Is there any conceptual reason why struct_ops map shouldn't be allowed to be attached to multiple places? If not, should we try to lift this restriction and not invent struct_ops-specific BPF cookie APIs?

bpf_struct_ops map currently does allow attaching multiple times through the link interface. It is up to the subsystem tcp-cc/hid/qdisc/scx how to handle it.

Quoted 3 lines

I am fixing the patchset by moving trampoline and ksyms to bpf_struct_ops_link. It shouldn't complicate struct_ops code too much (finger cross).

seems reasonable. I think it will be useful to have comments in v2 on how their lifecycle will be handled.

#15 lore raw

Re: [RFC bpf-next v1 2/4] bpf: Support cookie for linked-based struct_ops attachment

In reply to Alexei Starovoitov
Quoted 2 lines · Alexei Starovoitov

Can we put cookie into map_extra during st_ops map creation time and later copy it into actual cookie place in a trampoline?

+1. good idea. This should do.

author
#16 lore raw

RFC 3/4 libbpf: Support link-based struct_ops attach with options

In reply to Amery Hung

Currently libbpf supports bpf_map__attach_struct_ops() with signature:
LIBBPF_API struct bpf_link * bpf_map__attach_struct_ops(const struct bpf_map *map);

To support cookie for struct_ops attachment, an additional argument is needed. The argument is a pointer to a structure containing all the option used for bpf_link_create.

Add the new API:
LIBBPF_API struct bpf_link * bpf_map__attach_struct_ops_opts(const struct bpf_map *map,

                                  const struct bpf_struct_ops_opts *opts);
---
 tools/lib/bpf/bpf.c      |  5 +++++
 tools/lib/bpf/bpf.h      |  3 +++
 tools/lib/bpf/libbpf.c   | 14 +++++++++++++-
 tools/lib/bpf/libbpf.h   | 10 ++++++++++
 tools/lib/bpf/libbpf.map |  1 +
 5 files changed, 32 insertions(+), 1 deletion(-)
tools/lib/bpf/bpf.c +5 -0
index ab40dbf9f020..c1e66e190982 100644
@@ -881,6 +881,11 @@ int bpf_link_create(int prog_fd, int target_fd,
 		if (!OPTS_ZEROED(opts, cgroup))
 			return libbpf_err(-EINVAL);
 		break;
+	case BPF_STRUCT_OPS:
+		attr.link_create.struct_ops.cookie = OPTS_GET(opts, struct_ops.cookie, 0);
+		if (!OPTS_ZEROED(opts, struct_ops))
+			return libbpf_err(-EINVAL);
+		break;
 	default:
 		if (!OPTS_ZEROED(opts, flags))
 			return libbpf_err(-EINVAL);
tools/lib/bpf/bpf.h +3 -0
index 7252150e7ad3..2386ad4a295c 100644
@@ -443,6 +443,9 @@ struct bpf_link_create_opts {
 			__u32 relative_id;
 			__u64 expected_revision;
 		} cgroup;
+		struct {
+			__u64 cookie;
+		} struct_ops;
 	};
 	size_t :0;
 };
tools/lib/bpf/libbpf.c +13 -1
index aee36402f0a3..f5b71eccd5cc 100644
@@ -13110,10 +13110,20 @@ static int bpf_link__detach_struct_ops(struct bpf_link *link)
 
 struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map)
 {
+	return bpf_map__attach_struct_ops_opts(map, NULL);
+}
+
+struct bpf_link *bpf_map__attach_struct_ops_opts(const struct bpf_map *map,
+						 const struct bpf_struct_ops_opts *opts)
+{
+	LIBBPF_OPTS(bpf_link_create_opts, link_create_opts);
 	struct bpf_link_struct_ops *link;
 	__u32 zero = 0;
 	int err, fd;
 
+	if (!OPTS_VALID(opts, bpf_struct_ops_opts))
+		return libbpf_err_ptr(-EINVAL);
+
 	if (!bpf_map__is_struct_ops(map)) {
 		pr_warn("map '%s': can't attach non-struct_ops map\n", map->name);
 		return libbpf_err_ptr(-EINVAL);
@@ -13149,7 +13159,9 @@ struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map)
 		return &link->link;
 	}
 
-	fd = bpf_link_create(map->fd, 0, BPF_STRUCT_OPS, NULL);
+	link_create_opts.struct_ops.cookie = OPTS_GET(opts, cookie, 0);
+
+	fd = bpf_link_create(map->fd, 0, BPF_STRUCT_OPS, &link_create_opts);
 	if (fd < 0) {
 		free(link);
 		return libbpf_err_ptr(fd);
tools/lib/bpf/libbpf.h +10 -0
index d1cf813a057b..a4a758a8a25b 100644
@@ -894,6 +894,16 @@ bpf_program__attach_cgroup_opts(const struct bpf_program *prog, int cgroup_fd,
 
 struct bpf_map;
 
+struct bpf_struct_ops_opts {
+	/* size of this struct, for forward/backward compatibility */
+	size_t sz;
+	__u64 cookie;
+	size_t :0;
+};
+#define bpf_struct_ops_opts__last_field cookie
+
+LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops_opts(const struct bpf_map *map,
+							    const struct bpf_struct_ops_opts* opts);
 LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);
 LIBBPF_API int bpf_link__update_map(struct bpf_link *link, const struct bpf_map *map);
 
tools/lib/bpf/libbpf.map +1 -0
index 1bbf77326420..6998300c766a 100644
@@ -439,6 +439,7 @@ LIBBPF_1.6.0 {
 		bpf_object__prepare;
 		bpf_prog_stream_read;
 		bpf_program__attach_cgroup_opts;
+		bpf_map__attach_struct_ops_opts;
 		bpf_program__func_info;
 		bpf_program__func_info_cnt;
 		bpf_program__line_info;
Signature · 1 line
2.47.1
author
#17 lore raw

RFC 4/4 selftests/bpf: Test bpf_get_attach_cookie() in struct_ops program

In reply to Amery Hung

Attach a struct_ops map with a cookie and test that struct_ops programs can read the cookie using bpf_get_attach_cookie().

---
 .../bpf/prog_tests/test_struct_ops_cookie.c   | 42 ++++++++++++++++
 .../selftests/bpf/progs/struct_ops_cookie.c   | 48 +++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  1 +
 3 files changed, 91 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_cookie.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_cookie.c
tools/testing/selftests/bpf/prog_tests/test_struct_ops_cookie.c new file +42 -0
new file mode 100644
index 000000000000..36179785b173
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+
+#include "struct_ops_cookie.skel.h"
+
+static void test_struct_ops_cookie_basic(void)
+{
+	LIBBPF_OPTS(bpf_struct_ops_opts, attach_opts);
+	LIBBPF_OPTS(bpf_test_run_opts, run_opts);
+	struct struct_ops_cookie *skel;
+	struct bpf_link *link;
+	int err, prog_fd;
+
+	skel = struct_ops_cookie__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "struct_ops_cookie__open_and_load"))
+		return;
+
+	attach_opts.cookie = 0x12345678;
+	link = bpf_map__attach_struct_ops_opts(skel->maps.testmod_cookie, &attach_opts);
+	if (!ASSERT_OK_PTR(link, "bpf_map__attach_struct_ops_opts"))
+		goto out;
+
+	prog_fd = bpf_program__fd(skel->progs.trigger_test_1);
+	err = bpf_prog_test_run_opts(prog_fd, &run_opts);
+	ASSERT_OK(err, "bpf_prog_test_run_opts");
+	ASSERT_EQ(skel->bss->cookie_test_1, 0x12345678, "cookie_1_value");
+
+	prog_fd = bpf_program__fd(skel->progs.trigger_test_2);
+	err = bpf_prog_test_run_opts(prog_fd, &run_opts);
+	ASSERT_OK(err, "bpf_prog_test_run_opts");
+	ASSERT_EQ(skel->bss->cookie_test_2, 0x12345678, "cookie_2_value");
+
+out:
+	bpf_link__destroy(link);
+	struct_ops_cookie__destroy(skel);
+}
+
+void serial_test_struct_ops_cookie(void)
+{
+	if (test__start_subtest("struct_ops_cookie_basic"))
+		test_struct_ops_cookie_basic();
+}
tools/testing/selftests/bpf/progs/struct_ops_cookie.c new file +48 -0

48 lines not on this page · raw message

tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +1 -0
index e9e918cdf31f..d273b327a1c0 100644
@@ -1139,6 +1139,7 @@ static const struct bpf_verifier_ops bpf_testmod_verifier_ops = {
 };
 
 static const struct bpf_verifier_ops bpf_testmod_verifier_ops3 = {
+	.get_func_proto	 = bpf_base_func_proto,
 	.is_valid_access = bpf_testmod_ops_is_valid_access,
 };
 
Signature · 1 line
2.47.1