Skip to contentSkip to conversation

Update KF_RCU_PROTECTED

PATCH Applied Kumar Kartikeya Dwivedi 4 participants 12 messages Sep 17, 2025 – Sep 18, 2025

Subsystems (6)
lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

v3 cover Update KF_RCU_PROTECTED

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

In addition to this, the aforementioned kfunc also needs to return an RCU protected pointer, which currently has no generic kfunc flag or annotation. Add such a flag as well while we are at it.

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

Changelog:
----------
v2 -> v3
v2: https://lore.kernel.org/bpf/[email protected]

* Add back lost hunk reworking documentation for KF_RCU_PROTECTED.

v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]

* Drop KF_RET_RCU and fold change into KF_RCU_PROTECTED. (Andrea, Alexei)
* Update tests for non-struct pointer return values with KF_RCU_PROTECTED.

Kumar Kartikeya Dwivedi (2):
bpf: Enforce RCU protection for KF_RCU_PROTECTED selftests/bpf: Add tests for KF_RCU_PROTECTED

 Documentation/bpf/kfuncs.rst                  | 19 +++++++-
 kernel/bpf/verifier.c                         | 10 ++++
 .../selftests/bpf/progs/cgroup_read_xattr.c   |  2 +-
 .../selftests/bpf/progs/iters_task_failure.c  |  4 +-
 .../selftests/bpf/progs/iters_testmod.c       | 46 +++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    | 12 +++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  2 +
 7 files changed, 91 insertions(+), 4 deletions(-)

base-commit: b13448dd64e27752fad252cec7da1a50ab9f0b6f

Signature · 1 line
2.51.0
author maintainer
#1 lore raw

PATCH v3 1/2 bpf: Enforce RCU protection for KF_RCU_PROTECTED

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

This would also mean that iterator APIs using KF_RCU_PROTECTED will error out earlier, instead of throwing an error for lack of RCU CS protection when next() or destroy() methods are invoked.

In addition to this, if the kfuncs tagged KF_RCU_PROTECTED return a pointer value, ensure that this pointer value is only usable in an RCU critical section. There might be edge cases where the return value is special and doesn't need to imply MEM_RCU semantics, but in general, the assumption should hold for the majority of kfuncs, and we can revisit things if necessary later.

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

---
 Documentation/bpf/kfuncs.rst                  | 19 ++++++++++++++++++-
 kernel/bpf/verifier.c                         | 10 ++++++++++
 .../selftests/bpf/progs/cgroup_read_xattr.c   |  2 +-
 .../selftests/bpf/progs/iters_task_failure.c  |  4 ++--
 4 files changed, 31 insertions(+), 4 deletions(-)
Documentation/bpf/kfuncs.rst +18 -1
index ae468b781d31..e38941370b90 100644
@@ -335,9 +335,26 @@ consider doing refcnt != 0 check, especially when returning a KF_ACQUIRE
 pointer. Note as well that a KF_ACQUIRE kfunc that is KF_RCU should very likely
 also be KF_RET_NULL.
 
+2.4.8 KF_RCU_PROTECTED flag
+---------------------------
+
+The KF_RCU_PROTECTED flag is used to indicate that the kfunc must be invoked in
+an RCU critical section. This is assumed by default in non-sleepable programs,
+and must be explicitly ensured by calling ``bpf_rcu_read_lock`` for sleepable
+ones.
+
+If the kfunc returns a pointer value, this flag also enforces that the returned
+pointer is RCU protected, and can only be used while the RCU critical section is
+active.
+
+The flag is distinct from the ``KF_RCU`` flag, which only ensures that its
+arguments are at least RCU protected pointers. This may transitively imply that
+RCU protection is ensured, but it does not work in cases of kfuncs which require
+RCU protection but do not take RCU protected arguments.
+
 .. _KF_deprecated_flag:
 
-2.4.8 KF_DEPRECATED flag
+2.4.9 KF_DEPRECATED flag
 ------------------------
 
 The KF_DEPRECATED flag is used for kfuncs which are scheduled to be
kernel/bpf/verifier.c +10 -0
index 1029380f84db..5f9ee5bbd0ff 100644
@@ -13916,6 +13916,11 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 		return -EACCES;
 	}
 
+	if (is_kfunc_rcu_protected(&meta) && !in_rcu_cs(env)) {
+		verbose(env, "kernel func %s requires RCU critical section protection\n", func_name);
+		return -EACCES;
+	}
+
 	/* In case of release function, we get register number of refcounted
 	 * PTR_TO_BTF_ID in bpf_kfunc_arg_meta, do the release now.
 	 */
@@ -14029,6 +14034,9 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			/* Ensures we don't access the memory after a release_reference() */
 			if (meta.ref_obj_id)
 				regs[BPF_REG_0].ref_obj_id = meta.ref_obj_id;
+
+			if (is_kfunc_rcu_protected(&meta))
+				regs[BPF_REG_0].type |= MEM_RCU;
 		} else {
 			mark_reg_known_zero(env, regs, BPF_REG_0);
 			regs[BPF_REG_0].btf = desc_btf;
@@ -14037,6 +14045,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
 			if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
 				regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+			else if (is_kfunc_rcu_protected(&meta))
+				regs[BPF_REG_0].type |= MEM_RCU;
 
 			if (is_iter_next_kfunc(&meta)) {
 				struct bpf_reg_state *cur_iter;
tools/testing/selftests/bpf/progs/cgroup_read_xattr.c +1 -1
index 092db1d0435e..88e13e17ec9e 100644
@@ -73,7 +73,7 @@ int BPF_PROG(use_css_iter_non_sleepable)
 }
 
 SEC("lsm.s/socket_connect")
-__failure __msg("expected an RCU CS")
+__failure __msg("kernel func bpf_iter_css_new requires RCU critical section protection")
 int BPF_PROG(use_css_iter_sleepable_missing_rcu_lock)
 {
 	u64 cgrp_id = bpf_get_current_cgroup_id();
tools/testing/selftests/bpf/progs/iters_task_failure.c +2 -2
index 6b1588d70652..fe3663dedbe1 100644
@@ -15,7 +15,7 @@ void bpf_rcu_read_lock(void) __ksym;
 void bpf_rcu_read_unlock(void) __ksym;
 
 SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
-__failure __msg("expected an RCU CS when using bpf_iter_task_next")
+__failure __msg("kernel func bpf_iter_task_new requires RCU critical section protection")
 int BPF_PROG(iter_tasks_without_lock)
 {
 	struct task_struct *pos;
@@ -27,7 +27,7 @@ int BPF_PROG(iter_tasks_without_lock)
 }
 
 SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
-__failure __msg("expected an RCU CS when using bpf_iter_css_next")
+__failure __msg("kernel func bpf_iter_css_new requires RCU critical section protection")
 int BPF_PROG(iter_css_without_lock)
 {
 	u64 cg_id = bpf_get_current_cgroup_id();
Signature · 1 line
2.51.0
maintainer Acked-by on v3 1/2
#2 lore raw

Re: [PATCH bpf-next v3 1/2] bpf: Enforce RCU protection for KF_RCU_PROTECTED

Quoted 22 lines · Kumar Kartikeya Dwivedi

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

This would also mean that iterator APIs using KF_RCU_PROTECTED will error out earlier, instead of throwing an error for lack of RCU CS protection when next() or destroy() methods are invoked.

In addition to this, if the kfuncs tagged KF_RCU_PROTECTED return a pointer value, ensure that this pointer value is only usable in an RCU critical section. There might be edge cases where the return value is special and doesn't need to imply MEM_RCU semantics, but in general, the assumption should hold for the majority of kfuncs, and we can revisit things if necessary later.

Quoted 6 lines

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

---

[...]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14052

Quoted 9 lines
kernel/bpf/verifier.c +2 -0
@@ -14037,6 +14045,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
 			if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
 				regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+			else if (is_kfunc_rcu_protected(&meta))
+				regs[BPF_REG_0].type |= MEM_RCU;
 
 			if (is_iter_next_kfunc(&meta)) {
 				struct bpf_reg_state *cur_iter;

The code below this hunk looks as follows:

			if (is_iter_next_kfunc(&meta)) {
				struct bpf_reg_state *cur_iter;

				cur_iter = get_iter_from_state(env->cur_state, &meta);

				if (cur_iter->type & MEM_RCU) /* KF_RCU_PROTECTED */
					regs[BPF_REG_0].type |= MEM_RCU;
				else
					regs[BPF_REG_0].type |= PTR_TRUSTED;
			}

Do we want to reduce it to:

			if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
				regs[BPF_REG_0].type |= PTR_UNTRUSTED;
			else if (is_kfunc_rcu_protected(&meta))
				regs[BPF_REG_0].type |= MEM_RCU;
			else if (is_iter_next_kfunc(&meta))
				regs[BPF_REG_0].type |= PTR_TRUSTED;

And mark relevant iterator next (and destroy?) functions as KF_RCU_PROTECTED? (bpf_iter_css_next, bpf_iter_task_next, bpf_iter_scx_dsq_next).

I ask, because setting |= MEM_RCU in two places of this if branch looks a bit iffy.

[...]

author maintainer
#3 lore raw

Re: [PATCH bpf-next v3 1/2] bpf: Enforce RCU protection for KF_RCU_PROTECTED

Quoted 58 lines · Eduard Zingerman
Quoted 22 lines · Kumar Kartikeya Dwivedi

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

This would also mean that iterator APIs using KF_RCU_PROTECTED will error out earlier, instead of throwing an error for lack of RCU CS protection when next() or destroy() methods are invoked.

In addition to this, if the kfuncs tagged KF_RCU_PROTECTED return a pointer value, ensure that this pointer value is only usable in an RCU critical section. There might be edge cases where the return value is special and doesn't need to imply MEM_RCU semantics, but in general, the assumption should hold for the majority of kfuncs, and we can revisit things if necessary later.

Quoted 6 lines

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

---

[...]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14052

Quoted 9 lines
kernel/bpf/verifier.c +2 -0
@@ -14037,6 +14045,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
                      if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
                              regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+                     else if (is_kfunc_rcu_protected(&meta))
+                             regs[BPF_REG_0].type |= MEM_RCU;
 
                      if (is_iter_next_kfunc(&meta)) {
                              struct bpf_reg_state *cur_iter;

The code below this hunk looks as follows:

                        if (is_iter_next_kfunc(&meta)) {
                                struct bpf_reg_state *cur_iter;

                                cur_iter = get_iter_from_state(env->cur_state, &meta);

                                if (cur_iter->type & MEM_RCU) /* KF_RCU_PROTECTED */
                                        regs[BPF_REG_0].type |= MEM_RCU;
                                else
                                        regs[BPF_REG_0].type |= PTR_TRUSTED;
                        }
Quoted 8 lines

Do we want to reduce it to:

                        if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
                                regs[BPF_REG_0].type |= PTR_UNTRUSTED;
                        else if (is_kfunc_rcu_protected(&meta))
                                regs[BPF_REG_0].type |= MEM_RCU;
                        else if (is_iter_next_kfunc(&meta))
                                regs[BPF_REG_0].type |= PTR_TRUSTED;

I thought so too but we cannot do this. Suppose that the RCU read lock is dropped and reacquired between new() and next(). Right now, we rely on MEM_RCU in iter->type stack object that gets invalidated properly. With such a change we'd lose the ability to track continued protection using RCU while the iterator is alive on the stack, and continue to mark returned pointers as MEM_RCU.

Quoted 8 lines

And mark relevant iterator next (and destroy?) functions as KF_RCU_PROTECTED? (bpf_iter_css_next, bpf_iter_task_next, bpf_iter_scx_dsq_next).

I ask, because setting |= MEM_RCU in two places of this if branch looks a bit iffy.

[...]

maintainer
#4 lore raw

Re: [PATCH bpf-next v3 1/2] bpf: Enforce RCU protection for KF_RCU_PROTECTED

Quoted 68 lines · Kumar Kartikeya Dwivedi
Quoted 58 lines · Eduard Zingerman
Quoted 22 lines · Kumar Kartikeya Dwivedi

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

This would also mean that iterator APIs using KF_RCU_PROTECTED will error out earlier, instead of throwing an error for lack of RCU CS protection when next() or destroy() methods are invoked.

In addition to this, if the kfuncs tagged KF_RCU_PROTECTED return a pointer value, ensure that this pointer value is only usable in an RCU critical section. There might be edge cases where the return value is special and doesn't need to imply MEM_RCU semantics, but in general, the assumption should hold for the majority of kfuncs, and we can revisit things if necessary later.

Quoted 6 lines

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

---

[...]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14052

Quoted 9 lines
kernel/bpf/verifier.c +2 -0
@@ -14037,6 +14045,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
                      if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
                              regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+                     else if (is_kfunc_rcu_protected(&meta))
+                             regs[BPF_REG_0].type |= MEM_RCU;
 
                      if (is_iter_next_kfunc(&meta)) {
                              struct bpf_reg_state *cur_iter;

The code below this hunk looks as follows:

                        if (is_iter_next_kfunc(&meta)) {
                                struct bpf_reg_state *cur_iter;

                                cur_iter = get_iter_from_state(env->cur_state, &meta);

                                if (cur_iter->type & MEM_RCU) /* KF_RCU_PROTECTED */
                                        regs[BPF_REG_0].type |= MEM_RCU;
                                else
                                        regs[BPF_REG_0].type |= PTR_TRUSTED;
                        }
Quoted 8 lines

Do we want to reduce it to:

                        if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
                                regs[BPF_REG_0].type |= PTR_UNTRUSTED;
                        else if (is_kfunc_rcu_protected(&meta))
                                regs[BPF_REG_0].type |= MEM_RCU;
                        else if (is_iter_next_kfunc(&meta))
                                regs[BPF_REG_0].type |= PTR_TRUSTED;
Quoted 6 lines

I thought so too but we cannot do this. Suppose that the RCU read lock is dropped and reacquired between new() and next(). Right now, we rely on MEM_RCU in iter->type stack object that gets invalidated properly. With such a change we'd lose the ability to track continued protection using RCU while the iterator is alive on the stack, and continue to mark returned pointers as MEM_RCU.

The change I suggest does not invalidate this mechanics. The iterator is still marked with MEM_RCU and this mark is converted to PTR_UNTRUSTED when RCU section exits.
The check for PTR_UNTRUSTED happens in process_iter_arg() called from check_kfunc_args().

Quoted 9 lines
Quoted 9 lines

And mark relevant iterator next (and destroy?) functions as KF_RCU_PROTECTED? (bpf_iter_css_next, bpf_iter_task_next, bpf_iter_scx_dsq_next).

I ask, because setting |= MEM_RCU in two places of this if branch looks a bit iffy.

[...]

author maintainer
#5 lore raw

Re: [PATCH bpf-next v3 1/2] bpf: Enforce RCU protection for KF_RCU_PROTECTED

Quoted 77 lines · Eduard Zingerman
Quoted 68 lines · Kumar Kartikeya Dwivedi
Quoted 58 lines · Eduard Zingerman
Quoted 22 lines · Kumar Kartikeya Dwivedi

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

This would also mean that iterator APIs using KF_RCU_PROTECTED will error out earlier, instead of throwing an error for lack of RCU CS protection when next() or destroy() methods are invoked.

In addition to this, if the kfuncs tagged KF_RCU_PROTECTED return a pointer value, ensure that this pointer value is only usable in an RCU critical section. There might be edge cases where the return value is special and doesn't need to imply MEM_RCU semantics, but in general, the assumption should hold for the majority of kfuncs, and we can revisit things if necessary later.

Quoted 6 lines

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

---

[...]

kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 14052

Quoted 9 lines
kernel/bpf/verifier.c +2 -0
@@ -14037,6 +14045,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
                      if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
                              regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+                     else if (is_kfunc_rcu_protected(&meta))
+                             regs[BPF_REG_0].type |= MEM_RCU;
 
                      if (is_iter_next_kfunc(&meta)) {
                              struct bpf_reg_state *cur_iter;

The code below this hunk looks as follows:

                        if (is_iter_next_kfunc(&meta)) {
                                struct bpf_reg_state *cur_iter;

                                cur_iter = get_iter_from_state(env->cur_state, &meta);

                                if (cur_iter->type & MEM_RCU) /* KF_RCU_PROTECTED */
                                        regs[BPF_REG_0].type |= MEM_RCU;
                                else
                                        regs[BPF_REG_0].type |= PTR_TRUSTED;
                        }
Quoted 8 lines

Do we want to reduce it to:

                        if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
                                regs[BPF_REG_0].type |= PTR_UNTRUSTED;
                        else if (is_kfunc_rcu_protected(&meta))
                                regs[BPF_REG_0].type |= MEM_RCU;
                        else if (is_iter_next_kfunc(&meta))
                                regs[BPF_REG_0].type |= PTR_TRUSTED;
Quoted 6 lines

I thought so too but we cannot do this. Suppose that the RCU read lock is dropped and reacquired between new() and next(). Right now, we rely on MEM_RCU in iter->type stack object that gets invalidated properly. With such a change we'd lose the ability to track continued protection using RCU while the iterator is alive on the stack, and continue to mark returned pointers as MEM_RCU.

Quoted 5 lines

The change I suggest does not invalidate this mechanics. The iterator is still marked with MEM_RCU and this mark is converted to PTR_UNTRUSTED when RCU section exits.
The check for PTR_UNTRUSTED happens in process_iter_arg() called from check_kfunc_args().

Oh, I see. You mean is_iter_valid_reg_init etc. will complain on seeing that PTR_UNTRUSTED?
Hmm, I guess that might work. I can respin or do it as a follow up, whatever works best.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

And mark relevant iterator next (and destroy?) functions as KF_RCU_PROTECTED? (bpf_iter_css_next, bpf_iter_task_next, bpf_iter_scx_dsq_next).

I ask, because setting |= MEM_RCU in two places of this if branch looks a bit iffy.

[...]

maintainer
#6 lore raw

Re: [PATCH bpf-next v3 1/2] bpf: Enforce RCU protection for KF_RCU_PROTECTED

On Thu, 2025-09-18 at 23:59 +0200, Kumar Kartikeya Dwivedi wrote:

[...]

Quoted 10 lines
Quoted 5 lines

The change I suggest does not invalidate this mechanics. The iterator is still marked with MEM_RCU and this mark is converted to PTR_UNTRUSTED when RCU section exits.
The check for PTR_UNTRUSTED happens in process_iter_arg() called from check_kfunc_args().

Oh, I see. You mean is_iter_valid_reg_init etc. will complain on seeing that PTR_UNTRUSTED?
Hmm, I guess that might work. I can respin or do it as a follow up, whatever works best.

Thank you, followup works for me.

[...]

author maintainer
#7 lore raw

PATCH v3 2/2 selftests/bpf: Add tests for KF_RCU_PROTECTED

In reply to Kumar Kartikeya Dwivedi

Add a couple of test cases to ensure RCU protection is kicked in automatically, and the return type is as expected.

---
 .../selftests/bpf/progs/iters_testmod.c       | 46 +++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    | 12 +++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  2 +
 3 files changed, 60 insertions(+)
tools/testing/selftests/bpf/progs/iters_testmod.c +46 -0
index 9e4b45201e69..5379e9960ffd 100644
@@ -123,3 +123,49 @@ int iter_next_ptr_mem_not_trusted(const void *ctx)
 	bpf_iter_num_destroy(&num_it);
 	return 0;
 }
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+__failure __msg("kernel func bpf_kfunc_ret_rcu_test requires RCU critical section protection")
+int iter_ret_rcu_test_protected(const void *ctx)
+{
+	struct task_struct *p;
+
+	p = bpf_kfunc_ret_rcu_test();
+	return p->pid;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+__failure __msg("R1 type=rcu_ptr_or_null_ expected=")
+int iter_ret_rcu_test_type(const void *ctx)
+{
+	struct task_struct *p;
+
+	bpf_rcu_read_lock();
+	p = bpf_kfunc_ret_rcu_test();
+	bpf_this_cpu_ptr(p);
+	bpf_rcu_read_unlock();
+	return 0;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+__failure __msg("kernel func bpf_kfunc_ret_rcu_test_nostruct requires RCU critical section protection")
+int iter_ret_rcu_test_protected_nostruct(const void *ctx)
+{
+	void *p;
+
+	p = bpf_kfunc_ret_rcu_test_nostruct(4);
+	return *(int *)p;
+}
+
+SEC("?fentry.s/" SYS_PREFIX "sys_getpgid")
+__failure __msg("R1 type=rdonly_rcu_mem_or_null expected=")
+int iter_ret_rcu_test_type_nostruct(const void *ctx)
+{
+	void *p;
+
+	bpf_rcu_read_lock();
+	p = bpf_kfunc_ret_rcu_test_nostruct(4);
+	bpf_this_cpu_ptr(p);
+	bpf_rcu_read_unlock();
+	return 0;
+}
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +12 -0
index 2beb9b2fcbd8..d6ce51df9ed4 100644
@@ -218,6 +218,16 @@ __bpf_kfunc void bpf_kfunc_rcu_task_test(struct task_struct *ptr)
 {
 }
 
+__bpf_kfunc struct task_struct *bpf_kfunc_ret_rcu_test(void)
+{
+	return NULL;
+}
+
+__bpf_kfunc int *bpf_kfunc_ret_rcu_test_nostruct(int rdonly_buf_size)
+{
+	return NULL;
+}
+
 __bpf_kfunc struct bpf_testmod_ctx *
 bpf_testmod_ctx_create(int *err)
 {
@@ -623,6 +633,8 @@ BTF_ID_FLAGS(func, bpf_kfunc_trusted_vma_test, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_kfunc_trusted_task_test, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_kfunc_trusted_num_test, KF_TRUSTED_ARGS)
 BTF_ID_FLAGS(func, bpf_kfunc_rcu_task_test, KF_RCU)
+BTF_ID_FLAGS(func, bpf_kfunc_ret_rcu_test, KF_RET_NULL | KF_RCU_PROTECTED)
+BTF_ID_FLAGS(func, bpf_kfunc_ret_rcu_test_nostruct, KF_RET_NULL | KF_RCU_PROTECTED)
 BTF_ID_FLAGS(func, bpf_testmod_ctx_create, KF_ACQUIRE | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_testmod_ctx_release, KF_RELEASE)
 BTF_ID_FLAGS(func, bpf_testmod_ops3_call_test_1)
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h +2 -0
index 286e7faa4754..4df6fa6a92cb 100644
@@ -158,6 +158,8 @@ void bpf_kfunc_trusted_vma_test(struct vm_area_struct *ptr) __ksym;
 void bpf_kfunc_trusted_task_test(struct task_struct *ptr) __ksym;
 void bpf_kfunc_trusted_num_test(int *ptr) __ksym;
 void bpf_kfunc_rcu_task_test(struct task_struct *ptr) __ksym;
+struct task_struct *bpf_kfunc_ret_rcu_test(void) __ksym;
+int *bpf_kfunc_ret_rcu_test_nostruct(int rdonly_buf_size) __ksym;
 
 int bpf_kfunc_multi_st_ops_test_1(struct st_ops_args *args, u32 id) __ksym;
 
Signature · 1 line
2.51.0
Reviewed-by on the v3 series
#9 lore raw
In reply to Kumar Kartikeya Dwivedi
Quoted 11 lines · Kumar Kartikeya Dwivedi

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

Quoted 6 lines

In addition to this, the aforementioned kfunc also needs to return an RCU protected pointer, which currently has no generic kfunc flag or annotation. Add such a flag as well while we are at it.

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

Everything looks good from a sched_ext perspective.

I've also tested this with the new scx_bpf_cpu_curr() kfunc, marked as KF_RCU_PROTECTED, and everything seems to work as expected.

Thanks,
-Andrea

Quoted 32 lines

Changelog:
----------
v2 -> v3
v2: https://lore.kernel.org/bpf/[email protected]

* Add back lost hunk reworking documentation for KF_RCU_PROTECTED.

v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]

* Drop KF_RET_RCU and fold change into KF_RCU_PROTECTED. (Andrea, Alexei)
* Update tests for non-struct pointer return values with KF_RCU_PROTECTED.

Kumar Kartikeya Dwivedi (2):
bpf: Enforce RCU protection for KF_RCU_PROTECTED selftests/bpf: Add tests for KF_RCU_PROTECTED

 Documentation/bpf/kfuncs.rst                  | 19 +++++++-
 kernel/bpf/verifier.c                         | 10 ++++
 .../selftests/bpf/progs/cgroup_read_xattr.c   |  2 +-
 .../selftests/bpf/progs/iters_task_failure.c  |  4 +-
 .../selftests/bpf/progs/iters_testmod.c       | 46 +++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    | 12 +++++
 .../bpf/test_kmods/bpf_testmod_kfunc.h        |  2 +
 7 files changed, 91 insertions(+), 4 deletions(-)

base-commit: b13448dd64e27752fad252cec7da1a50ab9f0b6f

Signature · 1 line
2.51.0
author maintainer
#10 lore raw
Quoted 20 lines · Andrea Righi
Quoted 11 lines · Kumar Kartikeya Dwivedi

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

In the interest of making this flag useful for other upcoming kfuncs, e.g. scx_bpf_cpu_curr() [0][1], add enforcement for invoking the kfunc in an RCU critical section in general.

Quoted 6 lines

In addition to this, the aforementioned kfunc also needs to return an RCU protected pointer, which currently has no generic kfunc flag or annotation. Add such a flag as well while we are at it.

[0]: https://lore.kernel.org/all/[email protected]
[1]: https://lore.kernel.org/all/[email protected]

Quoted 6 lines

Everything looks good from a sched_ext perspective.

I've also tested this with the new scx_bpf_cpu_curr() kfunc, marked as KF_RCU_PROTECTED, and everything seems to work as expected.

Thanks for quickly trying it out, Andrea!

Quoted 2 lines

[...]

#11 lore raw
In reply to Kumar Kartikeya Dwivedi
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 8 lines · On Wed, 17 Sep 2025 03:27:53 +0000 you wrote:

Currently, KF_RCU_PROTECTED only applies to iterator APIs and that too in a convoluted fashion: the presence of this flag on the kfunc is used to set MEM_RCU in iterator type, and the lack of RCU protection results in an error only later, once next() or destroy() methods are invoked on the iterator. While there is no bug, this is certainly a bit unintuitive, and makes the enforcement of the flag iterator specific.

[...]

Here is the summary with links:
- [bpf-next,v3,1/2] bpf: Enforce RCU protection for KF_RCU_PROTECTED

    https://git.kernel.org/bpf/bpf-next/c/1512231b6cc8
  - [bpf-next,v3,2/2] selftests/bpf: Add tests for KF_RCU_PROTECTED
    https://git.kernel.org/bpf/bpf-next/c/8b788d663861

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html