Remove KF_KPTR_GET kfunc flag
Subsystems (6)
cover Remove KF_KPTR_GET kfunc flag
We've managed to improve the UX for kptrs significantly over the last 9 months. All of the existing use cases which previously had KF_KPTR_GET kfuncs (struct bpf_cpumask *, struct task_struct *, and struct cgroup *) have all been updated to be synchronized using RCU. In other words, their KF_KPTR_GET kfuncs have been removed in favor of KF_RU | KF_ACQUIRE kfuncs, with the pointers themselves also being readable from maps in an RCU read region thanks to the types being RCU safe.
While KF_KPTR_GET was a logical starting point for kptrs, it's become clear that they're not the correct abstraction. KF_KPTR_GET is a flag that essentially does nothing other than enforcing that the argument to a function is a pointer to a referenced kptr map value. At first glance, that's a useful thing to guarantee to a kfunc. It gives kfuncs the ability to try and acquire a reference on that kptr without requiring the BPF prog to do something like this:
struct kptr_type *in_map, *new = NULL;
in_map = bpf_kptr_xchg(&map->value, NULL);
if (in_map) {
new = bpf_kptr_type_acquire(in_map);
in_map = bpf_kptr_xchg(&map->value, in_map);
if (in_map)
bpf_kptr_type_release(in_map);
}That's clearly a pretty ugly (and racy) UX, and if using KF_KPTR_GET is the only alternative, it's better than nothing. However, the problem with any KF_KPTR_GET kfunc lies in the fact that it always requires some kind of synchronization in order to safely do an opportunistic acquire of the kptr in the map. This is because a BPF program running on another CPU could do a bpf_kptr_xchg() on that map value, and free the kptr after it's been read by the KF_KPTR_GET kfunc. For example, the now-removed bpf_task_kptr_get() kfunc did the following:
struct task_struct *bpf_task_kptr_get(struct task_struct **pp)
{
struct task_struct *p;
rcu_read_lock();
p = READ_ONCE(*pp);
/* If p is non-NULL, it could still be freed by another CPU,
* so we have to do an opportunistic refcount_inc_not_zero()
* and return NULL if the task will be freed after the
* current RCU read region.
*/
|f (p && !refcount_inc_not_zero(&p->rcu_users))
p = NULL;
rcu_read_unlock();
return p;
}In other words, the kfunc uses RCU to ensure that the task remains valid after it's been peeked from the map. However, this is completely redundant with just defining a KF_RCU kfunc that itself does a refcount_inc_not_zero(), which is exactly what bpf_task_acquire() now does.
So, the question of whether KF_KPTR_GET is useful is actually, "Are there any synchronization mechanisms / safety flags that are required by certain kptrs, but which are not provided by the verifier to kfuncs?" The answer to that question today is "No", because every kptr we currently care about is RCU protected.
Even if the answer ever became "yes", the proper way to support that referenced kptr type would be to add support for whatever synchronization mechanism it requires in the verifier, rather than giving kfuncs a flag that says, "Here's a pointer to a referenced kptr in a map, do whatever you need to do."
With all that said -- so as to allow us to consolidate the kfunc API, and simplify the verifier, this patchset removes the KF_KPTR_GET kfunc flag.
David Vernet (3):
bpf: Remove bpf_kfunc_call_test_kptr_get() test kfunc
bpf: Remove KF_KPTR_GET kfunc flag
bpf,docs: Remove KF_KPTR_GET from documentation
Documentation/bpf/kfuncs.rst | 21 ++--- include/linux/btf.h | 15 ++-- kernel/bpf/verifier.c | 64 --------------- net/bpf/test_run.c | 12 --- tools/testing/selftests/bpf/progs/map_kptr.c | 40 ++-------- .../selftests/bpf/progs/map_kptr_fail.c | 78 ------------------- .../testing/selftests/bpf/verifier/map_kptr.c | 27 ------- 7 files changed, 18 insertions(+), 239 deletions(-)
Signature · 1 line
2.40.0
PATCH 1/3 bpf: Remove bpf_kfunc_call_test_kptr_get() test kfunc
We've managed to improve the UX for kptrs significantly over the last 9 months. All of the prior main use cases, struct bpf_cpumask *, struct task_struct *, and struct cgroup *, have all been updated to be synchronized using RCU. In other words, their KF_ACQUIRE kfunc calls are all KF_RCU, and the pointers themselves are MEM_RCU and can be accessed in an RCU read region in BPF.
In a follow-on change, we'll be removing the KF_KPTR_GET kfunc flag. This patch prepares for that by removing the bpf_kfunc_call_test_kptr_get() kfunc, and all associated selftests.
--- net/bpf/test_run.c | 12 --- tools/testing/selftests/bpf/progs/map_kptr.c | 40 ++-------- .../selftests/bpf/progs/map_kptr_fail.c | 78 ------------------- .../testing/selftests/bpf/verifier/map_kptr.c | 27 ------- 4 files changed, 5 insertions(+), 152 deletions(-)
net/bpf/test_run.c +0 -12
@@ -679,17 +679,6 @@ __bpf_kfunc void bpf_kfunc_call_int_mem_release(int *p) { } -__bpf_kfunc struct prog_test_ref_kfunc * -bpf_kfunc_call_test_kptr_get (struct prog_test_ref_kfunc **pp,int a,int b) -{ -struct prog_test_ref_kfunc *p =READ_ONCE (*pp); - -if (!p) -return NULL ; -refcount_inc (&p->cnt); -return p; -} -struct prog_test_pass1 {int x0;struct {
@@ -804,7 +793,6 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_test_get_rdwr_mem, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_kfunc_call_test_get_rdonly_mem, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_kfunc_call_test_acq_rdonly_mem, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_kfunc_call_int_mem_release, KF_RELEASE) -BTF_ID_FLAGS(func, bpf_kfunc_call_test_kptr_get, KF_ACQUIRE | KF_RET_NULL | KF_KPTR_GET) BTF_ID_FLAGS(func, bpf_kfunc_call_test_pass_ctx) BTF_ID_FLAGS(func, bpf_kfunc_call_test_pass1)BTF_ID_FLAGS (func, bpf_kfunc_call_test_pass2)
tools/testing/selftests/bpf/progs/map_kptr.c +5 -35
@@ -115,8 +115,6 @@ DEFINE_MAP_OF_MAP(BPF_MAP_TYPE_HASH_OF_MAPS, hash_malloc_map, hash_of_hash_malloDEFINE_MAP_OF_MAP (BPF_MAP_TYPE_HASH_OF_MAPS , lru_hash_map, hash_of_lru_hash_maps);extern struct prog_test_ref_kfunc *bpf_kfunc_call_test_acquire (unsigned long *sp) __ksym; -extern struct prog_test_ref_kfunc * -bpf_kfunc_call_test_kptr_get (struct prog_test_ref_kfunc **p,int a,int b) __ksym; extern void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p) __ksym; void bpf_kfunc_call_test_ref(struct prog_test_ref_kfunc *p) __ksym;
@@ -187,25 +185,10 @@ static void test_kptr_ref(struct map_value *v)bpf_kfunc_call_test_release (p); } -static void test_kptr_get (struct map_value *v) -{ -struct prog_test_ref_kfunc *p; - - p =bpf_kfunc_call_test_kptr_get (&v->ref_ptr,0 ,0 ); -if (!p) -return ; -if (p->a + p->b >100 ) { -bpf_kfunc_call_test_release (p); -return ; - } -bpf_kfunc_call_test_release (p); -} -static void test_kptr (struct map_value *v) {test_kptr_unref (v);test_kptr_ref (v); -test_kptr_get (v); }SEC ("tc" )
@@ -338,38 +321,25 @@ int test_map_kptr_ref_pre(struct map_value *v)if (p_st->cnt.refs.counter != ref)return 4 ; - p =bpf_kfunc_call_test_kptr_get (&v->ref_ptr,0 ,0 ); -if (!p) -return 5 ; - ref++; -if (p_st->cnt.refs.counter != ref) { - ret =6 ; - goto end; - } -bpf_kfunc_call_test_release (p); - ref--; -if (p_st->cnt.refs.counter != ref) -return 7 ; - p =bpf_kptr_xchg (&v->ref_ptr,NULL );if (!p) -return 8 ; +return 5 ;bpf_kfunc_call_test_release (p); ref--;if (p_st->cnt.refs.counter != ref) -return 9 ; +return 6 ; p =bpf_kfunc_call_test_acquire (&arg);if (!p) -return 10 ; +return 7 ; ref++; p =bpf_kptr_xchg (&v->ref_ptr, p);if (p) { - ret =11 ; + ret =8 ; goto end; }if (p_st->cnt.refs.counter != ref) -return 12 ; +return 9 ;/* Leave in map */ return 0 ;
tools/testing/selftests/bpf/progs/map_kptr_fail.c +0 -78
96 lines not on this page · raw message
tools/testing/selftests/bpf/verifier/map_kptr.c +0 -27
@@ -288,33 +288,6 @@ .result = REJECT, .errstr = "off=0 kptr isn't referenced kptr", }, -{ - "map_kptr: unref: bpf_kfunc_call_test_kptr_get rejected", - .insns = { -BPF_MOV64_REG (BPF_REG_2 ,BPF_REG_10 ), -BPF_LD_MAP_FD (BPF_REG_6 ,0 ), -BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_6 ), -BPF_ALU64_IMM (BPF_ADD ,BPF_REG_2 ,-4 ), -BPF_MOV64_IMM (BPF_REG_0 ,0 ), -BPF_ST_MEM (BPF_W ,BPF_REG_2 ,0 ,0 ), -BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,0 ,0 , BPF_FUNC_map_lookup_elem), -BPF_JMP_IMM (BPF_JNE ,BPF_REG_0 ,0 ,1 ), -BPF_EXIT_INSN (), -BPF_MOV64_REG (BPF_REG_1 ,BPF_REG_0 ), -BPF_MOV64_IMM (BPF_REG_2 ,0 ), -BPF_MOV64_IMM (BPF_REG_3 ,0 ), -BPF_RAW_INSN (BPF_JMP |BPF_CALL ,0 ,BPF_PSEUDO_KFUNC_CALL ,0 ,0 ), -BPF_MOV64_IMM (BPF_REG_0 ,0 ), - BPF_EXIT_INSN(), - }, - .prog_type =BPF_PROG_TYPE_SCHED_CLS , - .fixup_map_kptr = { 1 }, - .result = REJECT, - .errstr = "arg#0 no referenced kptr at map value offset=0", - .fixup_kfunc_btf_id = { - { "bpf_kfunc_call_test_kptr_get", 13 }, - } -},/* Tests for referenced PTR_TO_BTF_ID */ { "map_kptr: ref: loaded pointer marked as untrusted",
Signature · 1 line
2.40.0
PATCH 3/3 bpf,docs: Remove KF_KPTR_GET from documentation
In reply to David VernetA prior patch removed KF_KPTR_GET from the kernel. Now that it's no longer accessible to kfunc authors, this patch removes it from the BPF kfunc documentation.
--- Documentation/bpf/kfuncs.rst | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-)
Documentation/bpf/kfuncs.rst +6 -15
@@ -184,16 +184,7 @@ in. All copies of the pointer being released are invalidated as a result of invoking kfunc with this flag. KF_RELEASE kfuncs automatically receive the protection afforded by the KF_TRUSTED_ARGS flag described below. -2.4.4 KF_KPTR_GET flag ----------------------- - -The KF_KPTR_GET flag is used to indicate that the kfunc takes the first argument -as a pointer to kptr, safely increments the refcount of the object it points to, -and returns a reference to the user. The rest of the arguments may be normal -arguments of a kfunc. The KF_KPTR_GET flag should be used in conjunction with -KF_ACQUIRE and KF_RET_NULL flags. - -2.4.5 KF_TRUSTED_ARGS flag +2.4.4 KF_TRUSTED_ARGS flag -------------------------- The KF_TRUSTED_ARGS flag is used for kfuncs taking pointer arguments. It
@@ -205,7 +196,7 @@ exception described below). There are two types of pointers to kernel objects which are considered "valid": 1. Pointers which are passed as tracepoint or struct_ops callback arguments. -2. Pointers which were returned from a KF_ACQUIRE or KF_KPTR_GET kfunc. +2. Pointers which were returned from a KF_ACQUIRE kfunc. Pointers to non-BTF objects (e.g. scalar pointers) may also be passed to KF_TRUSTED_ARGS kfuncs, and may have a non-zero offset.
@@ -232,13 +223,13 @@ In other words, you must: 2. Specify the type and name of the trusted nested field. This field must match the field in the original type definition exactly. -2.4.6 KF_SLEEPABLE flag +2.4.5 KF_SLEEPABLE flag ----------------------- The KF_SLEEPABLE flag is used for kfuncs that may sleep. Such kfuncs can only be called by sleepable BPF programs (BPF_F_SLEEPABLE). -2.4.7 KF_DESTRUCTIVE flag +2.4.6 KF_DESTRUCTIVE flag -------------------------- The KF_DESTRUCTIVE flag is used to indicate functions calling which is
@@ -247,7 +238,7 @@ rebooting or panicking. Due to this additional restrictions apply to these calls. At the moment they only require CAP_SYS_BOOT capability, but more can be added later. -2.4.8 KF_RCU flag +2.4.7 KF_RCU flag ----------------- The KF_RCU flag is a weaker version of KF_TRUSTED_ARGS. The kfuncs marked with
@@ -260,7 +251,7 @@ also be KF_RET_NULL. .. _KF_deprecated_flag: -2.4.9 KF_DEPRECATED flag +2.4.8 KF_DEPRECATED flag ------------------------ The KF_DEPRECATED flag is used for kfuncs which are scheduled to be
Signature · 1 line
2.40.0
PATCH 2/3 bpf: Remove KF_KPTR_GET kfunc flag
In reply to David VernetWe've managed to improve the UX for kptrs significantly over the last 9 months. All of the existing use cases which previously had KF_KPTR_GET kfuncs (struct bpf_cpumask *, struct task_struct *, and struct cgroup *) have all been updated to be synchronized using RCU. In other words, their KF_KPTR_GET kfuncs have been removed in favor of KF_RU | KF_ACQUIRE kfuncs, with the pointers themselves also being readable from maps in an RCU read region thanks to the types being RCU safe.
While KF_KPTR_GET was a logical starting point for kptrs, it's become clear that they're not the correct abstraction. KF_KPTR_GET is a flag that essentially does nothing other than enforcing that the argument to a function is a pointer to a referenced kptr map value. At first glance, that's a useful thing to guarantee to a kfunc. It gives kfuncs the ability to try and acquire a reference on that kptr without requiring the BPF prog to do something like this:
struct kptr_type *in_map, *new = NULL;
in_map = bpf_kptr_xchg(&map->value, NULL);
if (in_map) {
new = bpf_kptr_type_acquire(in_map);
in_map = bpf_kptr_xchg(&map->value, in_map);
if (in_map)
bpf_kptr_type_release(in_map);
}That's clearly a pretty ugly (and racy) UX, and if using KF_KPTR_GET is the only alternative, it's better than nothing. However, the problem with any KF_KPTR_GET kfunc lies in the fact that it always requires some kind of synchronization in order to safely do an opportunistic acquire of the kptr in the map. This is because a BPF program running on another CPU could do a bpf_kptr_xchg() on that map value, and free the kptr after it's been read by the KF_KPTR_GET kfunc. For example, the now-removed bpf_task_kptr_get() kfunc did the following:
struct task_struct *bpf_task_kptr_get(struct task_struct **pp)
{
struct task_struct *p;
rcu_read_lock();
p = READ_ONCE(*pp);
/* If p is non-NULL, it could still be freed by another CPU,
* so we have to do an opportunistic refcount_inc_not_zero()
* and return NULL if the task will be freed after the
* current RCU read region.
*/
|f (p && !refcount_inc_not_zero(&p->rcu_users))
p = NULL;
rcu_read_unlock();
return p;
}In other words, the kfunc uses RCU to ensure that the task remains valid after it's been peeked from the map. However, this is completely redundant with just defining a KF_RCU kfunc that itself does a refcount_inc_not_zero(), which is exactly what bpf_task_acquire() now does.
So, the question of whether KF_KPTR_GET is useful is actually, "Are there any synchronization mechanisms / safety flags that are required by certain kptrs, but which are not provided by the verifier to kfuncs?" The answer to that question today is "No", because every kptr we currently care about is RCU protected.
Even if the answer ever became "yes", the proper way to support that referenced kptr type would be to add support for whatever synchronization mechanism it requires in the verifier, rather than giving kfuncs a flag that says, "Here's a pointer to a referenced kptr in a map, do whatever you need to do."
With all that said -- so as to allow us to consolidate the kfunc API, and simplify the verifier a bit, this patch removes KF_KPTR_GET, and all relevant logic from the verifier.
--- include/linux/btf.h | 15 +++++----- kernel/bpf/verifier.c | 64 ------------------------------------------- 2 files changed, 7 insertions(+), 72 deletions(-)
include/linux/btf.h +7 -8
@@ -18,7 +18,6 @@#define KF_ACQUIRE (1 << 0)/* kfunc is an acquire function */ #define KF_RELEASE (1 << 1)/* kfunc is a release function */ #define KF_RET_NULL (1 << 2)/* kfunc returns a pointer that may be NULL */ -#define KF_KPTR_GET (1 << 3)/* kfunc returns reference to a kptr */ /* Trusted arguments are those which are guaranteed to be valid when passed to * the kfunc. It is used to enforce that pointers obtained from either acquire * kfuncs, or from the main kernel on a tracepoint or struct_ops callback
@@ -67,14 +66,14 @@* return 0; * } */ -#define KF_TRUSTED_ARGS (1 << 4)/* kfunc only takes trusted pointer arguments */ -#define KF_SLEEPABLE (1 << 5)/* kfunc may sleep */ -#define KF_DESTRUCTIVE (1 << 6)/* kfunc performs destructive actions */ -#define KF_RCU (1 << 7)/* kfunc takes either rcu or trusted pointer arguments */ +#define KF_TRUSTED_ARGS (1 << 3)/* kfunc only takes trusted pointer arguments */ +#define KF_SLEEPABLE (1 << 4)/* kfunc may sleep */ +#define KF_DESTRUCTIVE (1 << 5)/* kfunc performs destructive actions */ +#define KF_RCU (1 << 6)/* kfunc takes either rcu or trusted pointer arguments */ /* only one of KF_ITER_{NEW,NEXT,DESTROY} could be specified per kfunc */ -#define KF_ITER_NEW (1 << 8)/* kfunc implements BPF iter constructor */ -#define KF_ITER_NEXT (1 << 9)/* kfunc implements BPF iter next method */ -#define KF_ITER_DESTROY (1 << 10)/* kfunc implements BPF iter destructor */ +#define KF_ITER_NEW (1 << 7)/* kfunc implements BPF iter constructor */ +#define KF_ITER_NEXT (1 << 8)/* kfunc implements BPF iter next method */ +#define KF_ITER_DESTROY (1 << 9)/* kfunc implements BPF iter destructor */ /* * Tag marking a kernel function as a kfunc. This is meant to minimize the
kernel/bpf/verifier.c +0 -64
94 lines not on this page · raw message
Signature · 1 line
2.40.0
Re: [PATCH bpf-next 2/3] bpf: Remove KF_KPTR_GET kfunc flag
Quoted 5 lines · David Vernet
We've managed to improve the UX for kptrs significantly over the last 9 months. All of the existing use cases which previously had
KF_KPTR_GETkfuncs (structbpf_cpumask*, structtask_struct*, and struct cgroup *) have all been updated to be synchronized using RCU. In other words, theirKF_KPTR_GETkfuncs have been removed in favor ofKF_RU|
typo: KF_RCU.
same typo is in cover letter.
Quoted 21 lines
include/linux/btf.h +1
-5@@ -18,7 +18,6 @@#define KF_ACQUIRE (1 << 0)/* kfunc is an acquire function */ #define KF_RELEASE (1 << 1)/* kfunc is a release function */ #define KF_RET_NULL (1 << 2)/* kfunc returns a pointer that may be NULL */ -#define KF_KPTR_GET (1 << 3)/* kfunc returns reference to a kptr */ /* Trusted arguments are those which are guaranteed to be valid when passed to * the kfunc. It is used to enforce that pointers obtained from either acquire * kfuncs, or from the main kernel on a tracepoint or struct_ops callback @@ -67,14 +66,14 @@* return 0; * } */ -#define KF_TRUSTED_ARGS (1 << 4)/* kfunc only takes trusted pointer arguments */ -#define KF_SLEEPABLE (1 << 5)/* kfunc may sleep */ -#define KF_DESTRUCTIVE (1 << 6)/* kfunc performs destructive actions */ -#define KF_RCU (1 << 7)/* kfunc takes either rcu or trusted pointer arguments */ +#define KF_TRUSTED_ARGS (1 << 3)/* kfunc only takes trusted pointer arguments */
include/linux/btf.h line 76
Quoted 10 lines
include/linux/btf.h +6
-3+#define KF_SLEEPABLE (1 << 4)/* kfunc may sleep */ +#define KF_DESTRUCTIVE (1 << 5)/* kfunc performs destructive actions */ +#define KF_RCU (1 << 6)/* kfunc takes either rcu or trusted pointer arguments */ /* only one of KF_ITER_{NEW,NEXT,DESTROY} could be specified per kfunc */ -#define KF_ITER_NEW (1 << 8)/* kfunc implements BPF iter constructor */ -#define KF_ITER_NEXT (1 << 9)/* kfunc implements BPF iter next method */ -#define KF_ITER_DESTROY (1 << 10)/* kfunc implements BPF iter destructor */ +#define KF_ITER_NEW (1 << 7)/* kfunc implements BPF iter constructor */ +#define KF_ITER_NEXT (1 << 8)/* kfunc implements BPF iter next method */ +#define KF_ITER_DESTROY (1 << 9)/* kfunc implements BPF iter destructor */
Great cleanup! but let's not renumber flags to reduce the churn. Just delete #define KF_KPTR_GET (1 << 3)
I bet soon enough we will add another KF_ flag in that place.