Enable cpumasks to be used as kptrs
Subsystems (5)
v2 cover Enable cpumasks to be used as kptrs
This is part 2 of https://lore.kernel.org/all/[email protected]/
Changelog:
----------
v1 -> v2:
- Put back 'static' keyword in bpf_find_btf_id()
(kernel test robot <[email protected]>)
- Surround cpumask kfuncs in __diag() blocks to avoid no-prototype build warnings (kernel test robot <[email protected]>)
- Enable ___init suffixes to a type definition to signal that a type is a nocast alias of another type. That is, that when passed to a kfunc that expects one of the two types, the verifier will reject the other even if they're equivalent according to the C standard (Kumar and Alexei)
- Reject NULL for all trusted args, not just PTR_TO_MEM (Kumar)
- Reject both NULL and PTR_MAYBE_NULL for all trusted args (Kumar and Alexei )
- Improve examples given in cpumask documentation (Alexei)
- Use __success macro for nested_trust test (Alexei)
- Fix comment typo in struct bpf_cpumask comment header.
- Fix another example in the bpf_cpumask doc examples.
- Add documentation for ___init suffix change mentioned above.
David Vernet (9):
bpf: Enable annotating trusted nested pointers
bpf: Allow trusted args to walk struct when checking BTF IDs
bpf: Disallow NULLable pointers for trusted kfuncs
bpf: Enable cpumasks to be queried and used as kptrs selftests/bpf: Add nested trust selftests suite selftests/bpf: Add selftest suite for cpumask kfuncs bpf/docs: Document cpumask kfuncs in a new file bpf/docs: Document how nested trusted fields may be defined bpf/docs: Document the nocast aliasing behavior of ___init
Documentation/bpf/cpumasks.rst | 396 +++++++++++++++ Documentation/bpf/index.rst | 1 + Documentation/bpf/kfuncs.rst | 76 ++- include/linux/bpf.h | 8 + kernel/bpf/Makefile | 1 + kernel/bpf/btf.c | 122 +++++ kernel/bpf/cpumask.c | 477 ++++++++++++++++++ kernel/bpf/verifier.c | 67 ++- tools/testing/selftests/bpf/DENYLIST.s390x | 2 + .../selftests/bpf/prog_tests/cgrp_kfunc.c | 4 +- .../selftests/bpf/prog_tests/cpumask.c | 74 +++ .../selftests/bpf/prog_tests/nested_trust.c | 12 + .../selftests/bpf/prog_tests/task_kfunc.c | 4 +- .../selftests/bpf/progs/cpumask_common.h | 114 +++++ .../selftests/bpf/progs/cpumask_failure.c | 125 +++++ .../selftests/bpf/progs/cpumask_success.c | 426 ++++++++++++++++ .../selftests/bpf/progs/nested_trust_common.h | 12 + .../bpf/progs/nested_trust_failure.c | 33 ++ .../bpf/progs/nested_trust_success.c | 31 ++ 19 files changed, 1976 insertions(+), 9 deletions(-) create mode 100644 Documentation/bpf/cpumasks.rst create mode 100644 kernel/bpf/cpumask.c create mode 100644 tools/testing/selftests/bpf/prog_tests/cpumask.c create mode 100644 tools/testing/selftests/bpf/prog_tests/nested_trust.c create mode 100644 tools/testing/selftests/bpf/progs/cpumask_common.h create mode 100644 tools/testing/selftests/bpf/progs/cpumask_failure.c create mode 100644 tools/testing/selftests/bpf/progs/cpumask_success.c create mode 100644 tools/testing/selftests/bpf/progs/nested_trust_common.h create mode 100644 tools/testing/selftests/bpf/progs/nested_trust_failure.c create mode 100644 tools/testing/selftests/bpf/progs/nested_trust_success.c
Signature · 1 line
2.39.0
PATCH v2 1/9 bpf: Enable annotating trusted nested pointers
In kfuncs, a "trusted" pointer is a pointer that the kfunc can assume is safe, and which the verifier will allow to be passed to a KF_TRUSTED_ARGS kfunc. Currently, a KF_TRUSTED_ARGS kfunc disallows any pointer to be passed at a nonzero offset, but sometimes this is in fact safe if the "nested" pointer's lifetime is inherited from its parent. For example, the const cpumask_t *cpus_ptr field in a struct task_struct will remain valid until the task itself is destroyed, and thus would also be safe to pass to a KF_TRUSTED_ARGS kfunc.
While it would be conceptually simple to enable this by using BTF tags, gcc unfortunately does not yet support this. In the interim, this patch enables support for this by using a type-naming convention. A new BTF_TYPE_SAFE_NESTED macro is defined in verifier.c which allows a developer to specify the nested fields of a type which are considered trusted if its parent is also trusted. The verifier is also updated to account for this. A patch with selftests will be added in a follow-on change, along with documentation for this feature.
--- include/linux/bpf.h | 4 +++ kernel/bpf/btf.c | 61 +++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 32 ++++++++++++++++++++--- 3 files changed, 94 insertions(+), 3 deletions(-)
include/linux/bpf.h +4 -0
@@ -2186,6 +2186,10 @@ struct bpf_core_ctx {const struct btf *btf; }; +bool btf_nested_type_is_trusted (struct bpf_verifier_log *log, +const struct bpf_reg_state *reg, +int off); +int bpf_core_apply (struct bpf_core_ctx *ctx,const struct bpf_core_relo *relo,int relo_idx,void *insn);
kernel/bpf/btf.c +61 -0
64 lines not on this page · raw message
kernel/bpf/verifier.c +29 -3
@@ -4755,6 +4755,25 @@ static int bpf_map_direct_read(struct bpf_map *map, int off, int size, u64 *val)return 0 ; } +#define BTF_TYPE_SAFE_NESTED (__type) __PASTE(__type, __safe_fields) + +BTF_TYPE_SAFE_NESTED(struct task_struct) { +const cpumask_t *cpus_ptr; +}; + +static bool nested_ptr_is_trusted (struct bpf_verifier_env *env, +struct bpf_reg_state *reg, +int off) +{ +/* If its parent is not trusted, it can't regain its trusted status. */ +if (!is_trusted_reg (reg)) +return false; + +BTF_TYPE_EMIT (BTF_TYPE_SAFE_NESTED(struct task_struct )); + +return btf_nested_type_is_trusted (&env->log, reg, off); +} + static int check_ptr_to_btf_access(struct bpf_verifier_env *env, struct bpf_reg_state *regs, int regno, int off, int size,
@@ -4843,10 +4862,17 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,if (type_flag (reg->type) &PTR_UNTRUSTED ) flag |=PTR_UNTRUSTED ; -/* By default any pointer obtained from walking a trusted pointer is -* no longer trusted except the rcu case below. +/* By default any pointer obtained from walking a trusted pointer is no +* longer trusted, unless the field being accessed has explicitly been +* marked as inheriting its parent's state of trust. +* +* An RCU-protected pointer can also be deemed trusted if we are in an +* RCU read region. This case is handled below. */ - flag &= ~PTR_TRUSTED ; +if (nested_ptr_is_trusted (env, reg, off)) + flag |=PTR_TRUSTED ; +else + flag &= ~PTR_TRUSTED ;if (flag &MEM_RCU ) {/* Mark value register as MEM_RCU only if it is protected by
Signature · 1 line
2.39.0
PATCH v2 2/9 bpf: Allow trusted args to walk struct when checking BTF IDs
In reply to David VernetWhen validating BTF types for KF_TRUSTED_ARGS kfuncs, the verifier currently enforces that the top-level type must match when calling the kfunc. In other words, the verifier does not allow the BPF program to pass a bitwise equivalent struct, despite it being allowed according to the C standard.
For example, if you have the following type:
struct nf_conn___init {
struct nf_conn ct;
};The C standard stipulates that it would be safe to pass a struct nf_conn___init to a kfunc expecting a struct nf_conn. The verifier currently disallows this, however, as semantically kfuncs may want to enforce that structs that have equivalent types according to the C standard, but have different BTF IDs, are not able to be passed to kfuncs expecting one or the other. For example, struct nf_conn___init may not be queried / looked up, as it is allocated but may not yet be fully initialized.
On the other hand, being able to pass types that are equivalent according to the C standard will be useful for other types of kfunc / kptrs enabled by BPF. For example, in a follow-on patch, a series of kfuncs will be added which allow programs to do bitwise queries on cpumasks that are either allocated by the program (in which case they'll be a 'struct bpf_cpumask' type that wraps a cpumask_t as its first element), or a cpumask that was allocated by the main kernel (in which case it will just be a straight cpumask_t, as in task->cpus_ptr).
Having the two types of cpumasks allows us to distinguish between the two for when a cpumask is read-only vs. mutatable. A struct bpf_cpumask can be mutated by e.g. bpf_cpumask_clear(), whereas a regular cpumask_t cannot be. On the other hand, a struct bpf_cpumask can of course be queried in the exact same manner as a cpumask_t, with e.g. bpf_cpumask_test_cpu().
If we were to enforce that top level types match, then a user that's passing a struct bpf_cpumask to a read-only cpumask_t argument would have to cast with something like bpf_cast_to_kern_ctx() (which itself would need to be updated to expect the alias, and currently it only accommodates a single alias per prog type). Additionally, not specifying KF_TRUSTED_ARGS is not an option, as some kfuncs take one argument as a struct bpf_cpumask *, and another as a struct cpumask * (i.e. cpumask_t).
In order to enable this, this patch relaxes the constraint that a KF_TRUSTED_ARGS kfunc must have strict type matching, and instead only enforces strict type matching if a type is observed to be a "no-cast alias" (i.e., that the type names are equivalent, but one is suffixed with ___init).
Additionally, in order to try and be conservative and match existing behavior / expectations, this patch also enforces strict type checking for acquire kfuncs. We were already enforcing it for release kfuncs, so this should also improve the consistency of the semantics for kfuncs.
--- include/linux/bpf.h | 4 +++ kernel/bpf/btf.c | 61 +++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 30 ++++++++++++++++++++- 3 files changed, 94 insertions(+), 1 deletion(-)
include/linux/bpf.h +4 -0
@@ -2190,6 +2190,10 @@ bool btf_nested_type_is_trusted(struct bpf_verifier_log *log,const struct bpf_reg_state *reg, int off); +bool btf_type_ids_nocast_alias (struct bpf_verifier_log *log, +const struct btf *reg_btf,u32 reg_id, +const struct btf *arg_btf,u32 arg_id); +int bpf_core_apply (struct bpf_core_ctx *ctx,const struct bpf_core_relo *relo,int relo_idx,void *insn);
kernel/bpf/btf.c +61 -0
70 lines not on this page · raw message
kernel/bpf/verifier.c +29 -1
@@ -8563,9 +8563,37 @@ static int process_kf_arg_ptr_to_btf_id(struct bpf_verifier_env *env, reg_ref_id = *reg2btf_ids[base_type (reg->type)]; } -if (is_kfunc_trusted_args (meta) || (is_kfunc_release (meta) && reg->ref_obj_id)) +/* Enforce strict type matching for calls to kfuncs that are acquiring +* or releasing a reference, or are no-cast aliases. We do _not_ +* enforce strict matching for plain KF_TRUSTED_ARGS kfuncs by default, +* as we want to enable BPF programs to pass types that are bitwise +* equivalent without forcing them to explicitly cast with something +* like bpf_cast_to_kern_ctx(). +* +* For example, say we had a type like the following: +* +* struct bpf_cpumask { +* cpumask_t cpumask; +* refcount_t usage; +* }; +* +* Note that as specified in <linux/cpumask.h>, cpumask_t is typedef'ed +* to a struct cpumask, so it would be safe to pass a struct +* bpf_cpumask * to a kfunc expecting a struct cpumask *. +* +* The philosophy here is similar to how we allow scalars of different +* types to be passed to kfuncs as long as the size is the same. The +* only difference here is that we're simply allowing +* btf_struct_ids_match() to walk the struct at the 0th offset, and +* resolve types. +*/ +if (is_kfunc_acquire (meta) || + (is_kfunc_release (meta) && reg->ref_obj_id) || +btf_type_ids_nocast_alias (&env->log, reg_btf, reg_ref_id, meta->btf, ref_id)) strict_type_match = true; +WARN_ON_ONCE (is_kfunc_trusted_args (meta) && reg->off); + reg_ref_t =btf_type_skip_modifiers (reg_btf, reg_ref_id, ®_ref_id); reg_ref_tname =btf_name_by_offset (reg_btf, reg_ref_t->name_off);if (!btf_struct_ids_match (&env->log, reg_btf, reg_ref_id, reg->off, meta->btf, ref_id, strict_type_match)) {
Signature · 1 line
2.39.0
PATCH v2 3/9 bpf: Disallow NULLable pointers for trusted kfuncs
In reply to David VernetKF_TRUSTED_ARGS kfuncs currently have a subtle and insidious bug in validating pointers to scalars. Say that you have a kfunc like the following, which takes an array as the first argument:
bool bpf_cpumask_empty(const struct cpumask *cpumask)
{
return cpumask_empty(cpumask);
}...BTF_ID_FLAGS(func, bpf_cpumask_empty, KF_TRUSTED_ARGS)
...
If a BPF program were to invoke the kfunc with a NULL argument, it would crash the kernel. The reason is that struct cpumask is defined as a bitmap, which is itself defined as an array, and is accessed as a memory address memory by bitmap operations. So when the verifier analyzes the register, it interprets it as a pointer to a scalar struct, which is an array of size 8. check_mem_reg() then sees that the register is NULL, and returns 0, and the kfunc crashes when it passes it down to the cpumask wrappers.
To fix this, this patch adds a check for KF_ARG_PTR_TO_MEM which verifies that the register doesn't contain a possibly-NULL pointer if the kfunc is KF_TRUSTED_ARGS.
--- kernel/bpf/verifier.c | 6 ++++++ tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c | 4 ++-- tools/testing/selftests/bpf/prog_tests/task_kfunc.c | 4 ++-- 3 files changed, 10 insertions(+), 4 deletions(-)
kernel/bpf/verifier.c +6 -0
@@ -8937,6 +8937,12 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_return -EINVAL ; } +if (is_kfunc_trusted_args (meta) && + (register_is_null (reg) ||type_may_be_null (reg->type))) { +verbose (env,"Possibly NULL pointer passed to trusted arg%d\n" , i); +return -EACCES ; + } +if (reg->ref_obj_id) {if (is_kfunc_release (meta) && meta->ref_obj_id) { verbose(env, "verifier internal error: more than one arg with ref_obj_id R%d %u %u\n",
tools/testing/selftests/bpf/prog_tests/cgrp_kfunc.c +2 -2
@@ -93,11 +93,11 @@ static struct {const char *prog_name;const char *expected_err_msg; } failure_tests[] = { - {"cgrp_kfunc_acquire_untrusted", "R1 must be referenced or trusted"}, + {"cgrp_kfunc_acquire_untrusted", "Possibly NULL pointer passed to trusted arg0"}, {"cgrp_kfunc_acquire_fp", "arg#0 pointer type STRUCT cgroup must point"}, {"cgrp_kfunc_acquire_unsafe_kretprobe", "reg type unsupported for arg#0 function"}, {"cgrp_kfunc_acquire_trusted_walked", "R1 must be referenced or trusted"}, - {"cgrp_kfunc_acquire_null", "arg#0 pointer type STRUCT cgroup must point"}, + {"cgrp_kfunc_acquire_null", "Possibly NULL pointer passed to trusted arg0"}, {"cgrp_kfunc_acquire_unreleased", "Unreleased reference"}, {"cgrp_kfunc_get_non_kptr_param", "arg#0 expected pointer to map value"}, {"cgrp_kfunc_get_non_kptr_acquired", "arg#0 expected pointer to map value"},
tools/testing/selftests/bpf/prog_tests/task_kfunc.c +2 -2
@@ -87,11 +87,11 @@ static struct {const char *prog_name;const char *expected_err_msg; } failure_tests[] = { - {"task_kfunc_acquire_untrusted", "R1 must be referenced or trusted"}, + {"task_kfunc_acquire_untrusted", "Possibly NULL pointer passed to trusted arg0"}, {"task_kfunc_acquire_fp", "arg#0 pointer type STRUCT task_struct must point"}, {"task_kfunc_acquire_unsafe_kretprobe", "reg type unsupported for arg#0 function"}, {"task_kfunc_acquire_trusted_walked", "R1 must be referenced or trusted"}, - {"task_kfunc_acquire_null", "arg#0 pointer type STRUCT task_struct must point"}, + {"task_kfunc_acquire_null", "Possibly NULL pointer passed to trusted arg0"}, {"task_kfunc_acquire_unreleased", "Unreleased reference"}, {"task_kfunc_get_non_kptr_param", "arg#0 expected pointer to map value"}, {"task_kfunc_get_non_kptr_acquired", "arg#0 expected pointer to map value"},
Signature · 1 line
2.39.0
PATCH v2 4/9 bpf: Enable cpumasks to be queried and used as kptrs
In reply to David VernetCertain programs may wish to be able to query cpumasks. For example, if a program that is tracing percpu operations may wish to track which tasks end up running on which CPUs, and it could be useful to associate that with the tasks' cpumasks. Similarly, a program tracking NUMA allocations, CPU scheduling domains, etc, would potentially benefit from being able to see which CPUs a task could be migrated to, etc.
This patch enables these such cases by introducing a series of bpf_cpumask_* kfuncs. Amongst these kfuncs, there are two separate "classes" of operations:
1. kfuncs which allow the caller to allocate and mutate their own cpumasks in the form of a struct bpf_cpumask * object. Such kfuncs include e.g. bpf_cpumask_create() to allocate the cpumask, and bpf_cpumask_or() to mutate it. "Regular" cpumasks such as p->cpus_ptr may not be passed to these kfuncs, and the verifier will ensure this is the case by comparing BTF IDs.
2. Read-only operations which operate on const struct cpumask * arguments. For example, bpf_cpumask_test_cpu(), which tests whether a CPU is set in the cpumask. Any trusted struct cpumask * or struct bpf_cpumask * may be passed to these kfuncs. The verifier allows struct bpf_cpumask * even though the kfunc is defined with struct cpumask * because the first element of a struct bpf_cpumask is a cpumask_t, so it is safe to cast.
A follow-on patch will add selftests which validate these kfuncs, and another will document them.
Note that some of the kfuncs that were added would benefit from additional verification logic. For example, any kfunc taking a CPU argument that exceeds the number of CPUs on the system, etc. For now, we silently check for and ignore these cases at runtime. When we have e.g. per-argument kfunc flags, it might be helpful to add another KF_CPU-type flag that specifies that the verifier should validate that it's a valid CPU.
--- kernel/bpf/Makefile | 1 + kernel/bpf/cpumask.c | 269 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 270 insertions(+) create mode 100644 kernel/bpf/cpumask.c
kernel/bpf/Makefile +1 -0
@@ -36,6 +36,7 @@ obj-$(CONFIG_DEBUG_INFO_BTF) += sysfs_btf.o endif ifeq ($(CONFIG_BPF_JIT),y) obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o +obj-$(CONFIG_BPF_SYSCALL) += cpumask.o obj-${CONFIG_BPF_LSM} += bpf_lsm.o endif obj-$(CONFIG_BPF_PRELOAD) += preload/
kernel/bpf/cpumask.c new file +269 -0
269 lines not on this page · raw message
Signature · 1 line
2.39.0
Re: [PATCH bpf-next v2 4/9] bpf: Enable cpumasks to be queried and used as kptrs
Quoted 36 lines · David Vernet
kernel/bpf/cpumask.c new file +36
-0+ +/** +* struct bpf_cpumask - refcounted BPF cpumask wrapper structure +* @cpumask: The actual cpumask embedded in the struct. +* @usage: Object reference counter. When the refcount goes to 0, the +* memory is released back to the BPF allocator, which provides +* RCU safety. +* +* Note that we explicitly embed a cpumask_t rather than a cpumask_var_t. This +* is done to avoid confusing the verifier due to the typedef of cpumask_var_t +* changing depending on whether CONFIG_CPUMASK_OFFSTACK is defined or not. See +* the details in <linux/cpumask.h>. The consequence is that this structure is +* likely a bit larger than it needs to be when CONFIG_CPUMASK_OFFSTACK is +* defined due to embedding the whole NR_CPUS-size bitmap, but the extra memory +* overhead is minimal. For the more typical case of CONFIG_CPUMASK_OFFSTACK +* not being defined, the structure is the same size regardless. +*/ +struct bpf_cpumask { +cpumask_t cpumask; +refcount_t usage; +}; + +static struct bpf_mem_alloc bpf_cpumask_ma; + +static bool cpu_valid (u32 cpu) +{ +return cpu < nr_cpu_ids; +} + +__diag_push (); +__diag_ignore_all ("-Wmissing-prototypes" , +"Global kfuncs as their definitions will be in BTF" ); + +struct bpf_cpumask *bpf_cpumask_create (void ) +{ +struct bpf_cpumask *cpumask;
kernel/bpf/cpumask.c line 54
Quoted 10 lines
kernel/bpf/cpumask.c new file +10
-0+ + cpumask =bpf_mem_alloc (&bpf_cpumask_ma,sizeof (*cpumask)); +if (!cpumask) +return NULL ; + +memset (cpumask,0 ,sizeof (*cpumask)); +refcount_set (&cpumask->usage,1 ); + +return cpumask; +}
Applied patches 1 and 2. Patch 3 doesn't apply anymore. Pls rebase.
I'm fine with existing bpf_cpumask proposal, but can we do better? This is so close to be a bitmap template.
Can we generalize it as
struct bpf_bitmap {
refcount_t refcnt;
int num_bits;
u64 bits[];
};struct bpf_bitmap *bpf_bitmap_create(int bits)
{
bitmap = bpf_mem_alloc(&bpf_cpumask_ma, sizeof(*bitmap) + BITS_TO_LONGS(bits) * sizeof(u64)); bitmap->num_bits = bits;
}
and special case few custom kfuncs in the verifier that allow type cast from bpf_bitmap with to 'struct cpumask *' ? Like struct cpumask *bpf_bitmap_cast_to_cpumask(struct bpf_bitmap *bitmap) {
if (bitmap->num_bits == nr_cpu_ids)
return bitmap->bits;
return NULL;
}BTF_ID_FLAGS(func, bpf_bitmap_cast_to_cpumask, KF_TRUSTED_ARGS | KF_RET_NULL)
The UX will be a bit worse, since bpf prog would need to do !=NULL check but with future bpf_assert() we may get rid of !=NULL check.
We can keep direct cpumask accessors as kfuncs:
u32 bpf_cpumask_first(const struct cpumask *cpumask); u32 bpf_cpumask_first_zero(const struct cpumask *cpumask);
and add bpf_find_first_bit() and the rest of bit manipulations.
Since all of the bpf_cpumask do run-time cpu_valid() check we're not sacrificing performance.
Feels more generic with wider applicability at the expense of little bit worse UX. I haven't thought about acq/rel consequences.
wdyt?
Re: [PATCH bpf-next v2 4/9] bpf: Enable cpumasks to be queried and used as kptrs
Quoted 41 lines · Alexei Starovoitov
Quoted 40 lines · David Vernet
kernel/bpf/cpumask.c new file +40
-0+ +/** +* struct bpf_cpumask - refcounted BPF cpumask wrapper structure +* @cpumask: The actual cpumask embedded in the struct. +* @usage: Object reference counter. When the refcount goes to 0, the +* memory is released back to the BPF allocator, which provides +* RCU safety. +* +* Note that we explicitly embed a cpumask_t rather than a cpumask_var_t. This +* is done to avoid confusing the verifier due to the typedef of cpumask_var_t +* changing depending on whether CONFIG_CPUMASK_OFFSTACK is defined or not. See +* the details in <linux/cpumask.h>. The consequence is that this structure is +* likely a bit larger than it needs to be when CONFIG_CPUMASK_OFFSTACK is +* defined due to embedding the whole NR_CPUS-size bitmap, but the extra memory +* overhead is minimal. For the more typical case of CONFIG_CPUMASK_OFFSTACK +* not being defined, the structure is the same size regardless. +*/ +struct bpf_cpumask { +cpumask_t cpumask; +refcount_t usage; +}; + +static struct bpf_mem_alloc bpf_cpumask_ma; + +static bool cpu_valid (u32 cpu) +{ +return cpu < nr_cpu_ids; +} + +__diag_push (); +__diag_ignore_all ("-Wmissing-prototypes" , +"Global kfuncs as their definitions will be in BTF" ); + +struct bpf_cpumask *bpf_cpumask_create (void ) +{ +struct bpf_cpumask *cpumask; + + cpumask =bpf_mem_alloc (&bpf_cpumask_ma,sizeof (*cpumask)); +if (!cpumask) +return NULL ;
Quoted 8 lines
kernel/bpf/cpumask.c line 54
Quoted 6 lines
kernel/bpf/cpumask.c new file +6
-0+ +memset (cpumask,0 ,sizeof (*cpumask)); +refcount_set (&cpumask->usage,1 ); + +return cpumask; +}Applied patches 1 and 2. Patch 3 doesn't apply anymore. Pls rebase.
Ack, will rebase for v3.
Quoted 2 lines
I'm fine with existing
bpf_cpumaskproposal, but can we do better? This is so close to be a bitmap template.
Agreed that they're close, but I'm not a fan of the UX taxes for what we get out of it. More below.
Quoted 7 lines
Can we generalize it as
struct bpf_bitmap { refcount_t refcnt; int num_bits; u64 bits[]; };
Quoted 5 lines
struct
bpf_bitmap*bpf_bitmap_create(int bits)
{
bitmap =bpf_mem_alloc(&bpf_cpumask_ma, sizeof(*bitmap) + BITS_TO_LONGS(bits) * sizeof(u64));bitmap->num_bits= bits;
}
+1 that having bitmap kfuncs would be nice to expose, and should be pretty easy to add. Happy to do so in a follow-on patch set.
Quoted 10 lines
and special case few custom kfuncs in the verifier that allow type cast from
bpf_bitmapwith to 'struct cpumask *' ? Like struct cpumask *bpf_bitmap_cast_to_cpumask(structbpf_bitmap*bitmap) {
if (bitmap->num_bits==nr_cpu_ids)return bitmap->bits;return NULL;
}BTF_ID_FLAGS(func, bpf_bitmap_cast_to_cpumask, KF_TRUSTED_ARGS | KF_RET_NULL)
This I'm not a huge fan of though. It seems like we're removing a useful abstraction and adding a UX tax just to avoid defining and exporting an additional small set of kfuncs for allocating, and acquire/releasing a struct bpf_cpumask. That logic is very minimal, just around 100 lines of code including doxygen comments.
It's kind of unfortunate that cpumask is so close to bitmap, but that's nothing new -- <linux/cpumask.h> in the kernel is little more than a thin wrapper around a bitmap that simply provides some more ergonomic APIs, along with some magic that makes it safe to access cpumask_var_t on the stack regardless of NR_CPUS. The latter doesn't apply to BPF, but the former does.
Quoted 9 lines
The UX will be a bit worse, since bpf prog would need to do !=NULL check but with future
bpf_assert()we may get rid of !=NULL check.We can keep direct cpumask accessors as kfuncs:
u32
bpf_cpumask_first(const struct cpumask *cpumask); u32bpf_cpumask_first_zero(const struct cpumask *cpumask);and add
bpf_find_first_bit()and the rest of bit manipulations.
Worth noting as well is that I think struct bpf_bitmap is going to be treated somewhat differently than struct bpf_cpumask and struct cpumask. There is no type-safety for bitmaps in the kernel. They're just represented as unsigned long *, so I don't we'll be able to allow programs to pass bitmaps allocated elsewhere in the kernel to read-only bitmap kfuncs like we do for struct cpumask *, as the verifier will just interpret them as pointers to statically sized scalars.
Quoted 5 lines
Since all of the
bpf_cpumaskdo run-timecpu_valid()check we're not sacrificing performance.Feels more generic with wider applicability at the expense of little bit worse UX. I haven't thought about acq/rel consequences.
The TL;DR from me is that I agree that having bitmap kfuncs is a great idea, but I don't see the need to tie the two at the hip at the cost of a worse UX. I'd prefer to push the extra complexity into the BPF backend in favor of a simpler programming front-end for users.
Thoughts?
Re: [PATCH bpf-next v2 4/9] bpf: Enable cpumasks to be queried and used as kptrs
Quoted 11 lines · David Vernet
Quoted 10 lines
The UX will be a bit worse, since bpf prog would need to do !=NULL check but with future
bpf_assert()we may get rid of !=NULL check.We can keep direct cpumask accessors as kfuncs:
u32
bpf_cpumask_first(const struct cpumask *cpumask); u32bpf_cpumask_first_zero(const struct cpumask *cpumask);and add
bpf_find_first_bit()and the rest of bit manipulations.
Quoted 7 lines
Worth noting as well is that I think struct
bpf_bitmapis going to be treated somewhat differently than structbpf_cpumaskand struct cpumask. There is no type-safety for bitmaps in the kernel. They're just represented as unsigned long *, so I don't we'll be able to allow programs to pass bitmaps allocated elsewhere in the kernel to read-only bitmap kfuncs like we do for struct cpumask *, as the verifier will just interpret them as pointers to statically sized scalars.
That's a good point. That's where run-time and verification-time safety hurts UX too much.
Quoted 6 lines
Quoted 5 lines
Since all of the
bpf_cpumaskdo run-timecpu_valid()check we're not sacrificing performance.Feels more generic with wider applicability at the expense of little bit worse UX. I haven't thought about acq/rel consequences.
Quoted 6 lines
The TL;DR from me is that I agree that having bitmap kfuncs is a great idea, but I don't see the need to tie the two at the hip at the cost of a worse UX. I'd prefer to push the extra complexity into the BPF backend in favor of a simpler programming front-end for users.
Thoughts?
Fair enough. Let's proceed with what you have.
PATCH v2 5/9 selftests/bpf: Add nested trust selftests suite
In reply to David VernetNow that defining trusted fields in a struct is supported, we should add selftests to verify the behavior. This patch adds a few such testcases.
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/nested_trust.c | 12 +++++++ .../selftests/bpf/progs/nested_trust_common.h | 12 +++++++ .../bpf/progs/nested_trust_failure.c | 33 +++++++++++++++++++ .../bpf/progs/nested_trust_success.c | 31 +++++++++++++++++ 5 files changed, 89 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/nested_trust.c create mode 100644 tools/testing/selftests/bpf/progs/nested_trust_common.h create mode 100644 tools/testing/selftests/bpf/progs/nested_trust_failure.c create mode 100644 tools/testing/selftests/bpf/progs/nested_trust_success.c
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -44,6 +44,7 @@ map_kptr # failed to open_and_load program: -524 modify_return # modify_return attach failed: -524 (trampoline) module_attach # skel_attach skeleton attach failed: -524 (trampoline) mptcp +nested_trust # JIT does not support calling kernel function netcnt # failed to load BPF skeleton 'netcnt_prog': -7 (?) probe_user # check_kprobe_res wrong kprobe res from probe read (?) rcu_read_lock # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3 (?)
tools/testing/selftests/bpf/prog_tests/nested_trust.c new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ + +#include <test_progs.h> +#include "nested_trust_failure.skel.h" +#include "nested_trust_success.skel.h" + +void test_nested_trust (void ) +{ +RUN_TESTS (nested_trust_success); +RUN_TESTS (nested_trust_failure); +}
tools/testing/selftests/bpf/progs/nested_trust_common.h new file +12 -0
@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ + +#ifndef _NESTED_TRUST_COMMON_H +#define _NESTED_TRUST_COMMON_H + +#include <stdbool.h> + +bool bpf_cpumask_test_cpu(unsigned int cpu, const struct cpumask *cpumask) __ksym; +bool bpf_cpumask_first_zero(const struct cpumask *cpumask) __ksym; + +#endif /* _NESTED_TRUST_COMMON_H */
tools/testing/selftests/bpf/progs/nested_trust_failure.c new file +33 -0
@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ + +#include <vmlinux.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +#include "nested_trust_common.h" + +char _license[] SEC("license") = "GPL"; + +/* Prototype for all of the program trace events below: +* +* TRACE_EVENT(task_newtask, +* TP_PROTO(struct task_struct *p, u64 clone_flags) +*/ + +SEC("tp_btf/task_newtask") +__failure __msg("R2 must be referenced or trusted") +int BPF_PROG(test_invalid_nested_user_cpus, struct task_struct *task, u64 clone_flags) +{ +bpf_cpumask_test_cpu (0 , task->user_cpus_ptr); +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("R1 must have zero offset when passed to release func or trusted arg to kfunc") +int BPF_PROG(test_invalid_nested_offset, struct task_struct *task, u64 clone_flags) +{ +bpf_cpumask_first_zero (&task->cpus_mask); +return 0 ; +}
tools/testing/selftests/bpf/progs/nested_trust_success.c new file +31 -0
@@ -0,0 +1,31 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ + +#include <vmlinux.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_helpers.h> +#include "bpf_misc.h" + +#include "nested_trust_common.h" + +char _license[] SEC("license") = "GPL"; + +int pid, err; + +static bool is_test_task (void ) +{ +int cur_pid =bpf_get_current_pid_tgid () >>32 ; + +return pid == cur_pid; +} + +SEC("tp_btf/task_newtask") +__success +int BPF_PROG (test_read_cpumask ,struct task_struct *task,u64 clone_flags) +{ +if (!is_test_task ()) +return 0 ; + +bpf_cpumask_test_cpu (0 , task->cpus_ptr); +return 0 ; +}
Signature · 1 line
2.39.0
PATCH v2 6/9 selftests/bpf: Add selftest suite for cpumask kfuncs
In reply to David VernetA recent patch added a new set of kfuncs for allocating, freeing, manipulating, and querying cpumasks. This patch adds a new 'cpumask' selftest suite which verifies their behavior.
--- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/cpumask.c | 74 +++ .../selftests/bpf/progs/cpumask_common.h | 114 +++++ .../selftests/bpf/progs/cpumask_failure.c | 125 +++++ .../selftests/bpf/progs/cpumask_success.c | 426 ++++++++++++++++++ 5 files changed, 740 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/cpumask.c create mode 100644 tools/testing/selftests/bpf/progs/cpumask_common.h create mode 100644 tools/testing/selftests/bpf/progs/cpumask_failure.c create mode 100644 tools/testing/selftests/bpf/progs/cpumask_success.c
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -13,6 +13,7 @@ cgroup_hierarchical_stats # JIT does not support calling kernel f cgrp_kfunc # JIT does not support calling kernel function cgrp_local_storage # prog_attach unexpected error: -524 (trampoline) core_read_macros # unknown func bpf_probe_read#4 (overlapping) +cpumask # JIT does not support calling kernel function d_path # failed to auto-attach program 'prog_stat': -524 (trampoline) decap_sanity # JIT does not support calling kernel function (kfunc) deny_namespace # failed to attach: ERROR: strerror_r(-524)=22 (trampoline)
tools/testing/selftests/bpf/prog_tests/cpumask.c new file +74 -0
74 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cpumask_common.h new file +114 -0
114 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cpumask_failure.c new file +125 -0
125 lines not on this page · raw message
tools/testing/selftests/bpf/progs/cpumask_success.c new file +426 -0
426 lines not on this page · raw message
Signature · 1 line
2.39.0
PATCH v2 7/9 bpf/docs: Document cpumask kfuncs in a new file
In reply to David VernetNow that we've added a series of new cpumask kfuncs, we should document them so users can easily use them. This patch adds a new cpumasks.rst file to document them.
--- Documentation/bpf/cpumasks.rst | 393 +++++++++++++++++++++++++++++++++ Documentation/bpf/index.rst | 1 + Documentation/bpf/kfuncs.rst | 11 + kernel/bpf/cpumask.c | 208 +++++++++++++++++ 4 files changed, 613 insertions(+) create mode 100644 Documentation/bpf/cpumasks.rst
Documentation/bpf/cpumasks.rst new file +393 -0
@@ -0,0 +1,393 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _cpumasks-header-label: + +================== +BPF cpumask kfuncs +================== + +1. Introduction +=============== + +``struct cpumask`` is a bitmap data structure in the kernel whose indices +reflect the CPUs on the system. Commonly, cpumasks are used to track which CPUs +a task is affinitized to, but they can also be used to e.g. track which cores +are associated with a scheduling domain, which cores on a machine are idle, +etc. + +BPF provides programs with a set of :ref:`kfuncs-header-label` that can be +used to allocate, mutate, query, and free cpumasks. + +2. BPF cpumask objects +====================== + +There are two different types of cpumasks that can be used by BPF programs. + +2.1 ``struct bpf_cpumask *`` +---------------------------- + +``struct bpf_cpumask *`` is a cpumask that is allocated by BPF, on behalf of a +BPF program, and whose lifecycle is entirely controlled by BPF. These cpumasks +are RCU-protected, can be mutated, can be used as kptrs, and can be safely cast +to a ``struct cpumask *``. + +2.1.1 ``struct bpf_cpumask *`` lifecycle +---------------------------------------- + +A ``struct bpf_cpumask *`` is allocated, acquired, and released, using the +following functions: + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_create + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_acquire + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_release + +For example: + +.. code-block:: c + + struct cpumask_map_value { + struct bpf_cpumask __kptr_ref * cpumask; + }; + + struct array_map { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct cpumask_map_value); + __uint(max_entries, 65536); + } cpumask_map SEC(".maps"); + + static int cpumask_map_insert(struct bpf_cpumask *mask, u32 pid) + { + struct cpumask_map_value local, *v; + long status; + struct bpf_cpumask *old; + u32 key = pid; + + local.cpumask = NULL; + status = bpf_map_update_elem(&cpumask_map, &key, &local, 0); + if (status) { + bpf_cpumask_release(mask); + return status; + } + + v = bpf_map_lookup_elem(&cpumask_map, &key); + if (!v) { + bpf_cpumask_release(mask); + return -ENOENT; + } + + old = bpf_kptr_xchg(&v->cpumask, mask); + if (old) + bpf_cpumask_release(old); + + return 0; + } + + /** + * A sample tracepoint showing how a task's cpumask can be queried and + * recorded as a kptr. + */ + SEC("tp_btf/task_newtask") + int BPF_PROG(record_task_cpumask, struct task_struct *task, u64 clone_flags) + { + struct bpf_cpumask *cpumask; + int ret; + + cpumask = bpf_cpumask_create(); + if (!cpumask) + return -ENOMEM; + + if (!bpf_cpumask_full(task->cpus_ptr)) + bpf_printk("task %s has CPU affinity", task->comm); + + bpf_cpumask_copy(cpumask, task->cpus_ptr); + return cpumask_map_insert(cpumask, task->pid); + } + +---- + +2.1.1 ``struct bpf_cpumask *`` as kptrs +--------------------------------------- + +As mentioned and illustrated above, these ``struct bpf_cpumask *`` objects can +also be stored in a map and used as kptrs. If a ``struct bpf_cpumask *`` is in +a map, the reference can be removed from the map with bpf_kptr_xchg(), or +opportunistically acquired with bpf_cpumask_kptr_get(): + + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_kptr_get + +Here is an example of a ``struct bpf_cpumask *`` being retrieved from a map: + +.. code-block:: c + + /* struct containing the struct bpf_cpumask kptr which is actually stored in the map. */ + struct cpumasks_kfunc_map_value { + struct bpf_cpumask __kptr_ref * bpf_cpumask; + }; + + /* The map containing struct cpumasks_kfunc_map_value entries. */ + struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct cpumasks_kfunc_map_value); + __uint(max_entries, 1); + } cpumasks_kfunc_map SEC(".maps"); + + /* ... */ + + /** + * A simple example tracepoint program showing how a + * struct bpf_cpumask * kptr that is stored in a map can + * be acquired using the bpf_cpumask_kptr_get() kfunc. + */ + SEC("tp_btf/cgroup_mkdir") + int BPF_PROG(cgrp_ancestor_example, struct cgroup *cgrp, const char *path) + { + struct bpf_cpumask *kptr; + struct cpumasks_kfunc_map_value *v; + u32 key = 0; + + /* Assume a bpf_cpumask * kptr was previously stored in the map. */ + v = bpf_map_lookup_elem(&cpumasks_kfunc_map, &key); + if (!v) + return -ENOENT; + + /* Acquire a reference to the bpf_cpumask * kptr that's already stored in the map. */ + kptr = bpf_cpumask_kptr_get(&v->cpumask); + if (!kptr) + /* If no bpf_cpumask was present in the map, it's because + * we're racing with another CPU that removed it with + * bpf_kptr_xchg() between the bpf_map_lookup_elem() + * above, and our call to bpf_cpumask_kptr_get(). + * bpf_cpumask_kptr_get() internally safely handles this + * race, and will return NULL if the cpumask is no longer + * present in the map by the time we invoke the kfunc. + */ + return -EBUSY; + + /* Free the reference we just took above. Note that the + * original struct bpf_cpumask * kptr is still in the map. It will + * be freed either at a later time if another context deletes + * it from the map, or automatically by the BPF subsystem if + * it's still present when the map is destroyed. + */ + bpf_cpumask_release(kptr); + + return 0; + } + +---- + +2.2 ``struct cpumask`` +---------------------- + +``struct cpumask`` is the object that actually contains the cpumask bitmap +being queried, mutated, etc. A ``struct bpf_cpumask`` wraps a ``struct +cpumask``, which is why it's safe to cast it as such (note however that it is +**not** safe to cast a ``struct cpumask *`` to a ``struct bpf_cpumask *``, and +the verifier will reject any program that tries to do so). + +As we'll see below, any kfunc that mutates its cpumask argument will take a +``struct bpf_cpumask *`` as that argument. Any argument that simply queries the +cpumask will instead take a ``struct cpumask *``. + +3. cpumask kfuncs +================= + +Above, we described the kfuncs that can be used to allocate, acquire, release, +etc a ``struct bpf_cpumask *``. This section of the document will describe the +kfuncs for mutating and querying cpumasks. + +3.1 Mutating cpumasks +--------------------- + +Some cpumask kfuncs are "read-only" in that they don't mutate any of their +arguments, whereas others mutate at least one argument (which means that the +argument must be a ``struct bpf_cpumask *``, as described above). + +This section will describe all of the cpumask kfuncs which mutate at least one +argument. :ref:`cpumasks-querying-label` below describes the read-only kfuncs. + +3.1.1 Setting and clearing CPUs +------------------------------- + +bpf_cpumask_set_cpu() and bpf_cpumask_clear_cpu() can be used to set and clear +a CPU in a ``struct bpf_cpumask`` respectively: + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_set_cpu bpf_cpumask_clear_cpu + +These kfuncs are pretty straightforward, and can be used, for example, as +follows: + +.. code-block:: c + + /** + * A sample tracepoint showing how a cpumask can be queried. + */ + SEC("tp_btf/task_newtask") + int BPF_PROG(test_set_clear_cpu, struct task_struct *task, u64 clone_flags) + { + struct bpf_cpumask *cpumask; + + cpumask = bpf_cpumask_create(); + if (!cpumask) + return -ENOMEM; + + bpf_cpumask_set_cpu(0, cpumask); + if (!bpf_cpumask_test_cpu(0, cast(cpumask))) + /* Should never happen. */ + goto release_exit; + + bpf_cpumask_clear_cpu(0, cpumask); + if (bpf_cpumask_test_cpu(0, cast(cpumask))) + /* Should never happen. */ + goto release_exit; + + /* struct cpumask * pointers such as task->cpus_ptr can also be queried. */ + if (bpf_cpumask_test_cpu(0, task->cpus_ptr)) + bpf_printk("task %s can use CPU %d", task->comm, 0); + + release_exit: + bpf_cpumask_release(cpumask); + return 0; + } + +---- + +bpf_cpumask_test_and_set_cpu() and bpf_cpumask_test_and_clear_cpu() are +analogous kfuncs that allow callers to atomically test and set (or clear) CPUs: + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_test_and_set_cpu bpf_cpumask_test_and_clear_cpu + +---- + +We can also set and clear entire ``struct bpf_cpumask *`` objects in one +operation using bpf_cpumask_setall() and bpf_cpumask_clear(): + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_setall bpf_cpumask_clear + +3.1.2 Operations between cpumasks +--------------------------------- + +In addition to setting and clearing individual CPUs in a single cpumask, +callers can also perform bitwise operations between multiple cpumasks using +bpf_cpumask_and(), bpf_cpumask_or(), and bpf_cpumask_xor(): + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_and bpf_cpumask_or bpf_cpumask_xor + +The following is an example of how they may be used. Note that some of the +kfuncs shown in this example will be covered in more detail below. + +.. code-block:: c + + /** + * A sample tracepoint showing how a cpumask can be mutated using + bitwise operators (and queried). + */ + SEC("tp_btf/task_newtask") + int BPF_PROG(test_and_or_xor, struct task_struct *task, u64 clone_flags) + { + struct bpf_cpumask *mask1, *mask2, *dst1, *dst2; + + mask1 = bpf_cpumask_create(); + if (!mask1) + return -ENOMEM; + + mask2 = bpf_cpumask_create(); + if (!mask2) { + bpf_cpumask_release(mask1); + return -ENOMEM; + } + + // ...Safely create the other two masks... */ + + bpf_cpumask_set_cpu(0, mask1); + bpf_cpumask_set_cpu(1, mask2); + bpf_cpumask_and(dst1, (const struct cpumask *)mask1, (const struct cpumask *)mask2); + if (!bpf_cpumask_empty((const struct cpumask *)dst1)) + /* Should never happen. */ + goto release_exit; + + bpf_cpumask_or(dst1, (const struct cpumask *)mask1, (const struct cpumask *)mask2); + if (!bpf_cpumask_test_cpu(0, (const struct cpumask *)dst1)) + /* Should never happen. */ + goto release_exit; + + if (!bpf_cpumask_test_cpu(1, (const struct cpumask *)dst1)) + /* Should never happen. */ + goto release_exit; + + bpf_cpumask_xor(dst2, (const struct cpumask *)mask1, (const struct cpumask *)mask2); + if (!bpf_cpumask_equal((const struct cpumask *)dst1, + (const struct cpumask *)dst2)) + /* Should never happen. */ + goto release_exit; + + release_exit: + bpf_cpumask_release(mask1); + bpf_cpumask_release(mask2); + bpf_cpumask_release(dst1); + bpf_cpumask_release(dst2); + return 0; + } + +---- + +The contents of an entire cpumask may be copied to another using +bpf_cpumask_copy(): + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_copy + +---- + +.. _cpumasks-querying-label: + +3.2 Querying cpumasks +--------------------- + +In addition to the above kfuncs, there is also a set of read-only kfuncs that +can be used to query the contents of cpumasks. + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_first bpf_cpumask_first_zero bpf_cpumask_test_cpu + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_equal bpf_cpumask_intersects bpf_cpumask_subset + bpf_cpumask_empty bpf_cpumask_full + +.. kernel-doc:: kernel/bpf/cpumask.c + :identifiers: bpf_cpumask_any bpf_cpumask_any_and + +---- + +Some example usages of these querying kfuncs were shown above. We will not +replicate those exmaples here. Note, however, that all of the aforementioned +kfuncs are tested in `tools/testing/selftests/bpf/progs/cpumask_success.c`_, so +please take a look there if you're looking for more examples of how they can be +used. + +.. _tools/testing/selftests/bpf/progs/cpumask_success.c: + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/tools/testing/selftests/bpf/progs/cpumask_success.c + + +4. Adding BPF cpumask kfuncs +============================ + +The set of supported BPF cpumask kfuncs are not (yet) a 1-1 match with the +cpumask operations in include/linux/cpumask.h. Any of those cpumask operations +could easily be encapsulated in a new kfunc if and when required. If you'd like +to support a new cpumask operation, please feel free to submit a patch. If you +do add a new cpumask kfunc, please document it here, and add any relevant +selftest testcases to the cpumask selftest suite.
Documentation/bpf/index.rst +1 -0
@@ -20,6 +20,7 @@ that goes into great technical depth about the BPF Architecture. syscall_api helpers kfuncs + cpumasks programs maps bpf_prog_run
Documentation/bpf/kfuncs.rst +11 -0
@@ -1,3 +1,7 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. _kfuncs-header-label: + ============================= BPF Kernel Functions (kfuncs) =============================
@@ -420,3 +424,10 @@ the verifier. bpf_cgroup_ancestor() can be used as follows: bpf_cgroup_release(parent); return 0; } + +3.3 struct cpumask * kfuncs +--------------------------- + +BPF provides a set of kfuncs that can be used to query, allocate, mutate, and +destroy struct cpumask * objects. Please refer to :ref:`cpumasks-header-label` +for more details.
kernel/bpf/cpumask.c +208 -0
341 lines not on this page · raw message
Signature · 1 line
2.39.0
PATCH v2 8/9 bpf/docs: Document how nested trusted fields may be defined
In reply to David VernetA prior change defined a new BTF_TYPE_SAFE_NESTED macro in the verifier which allows developers to specify when a pointee field in a struct type should inherit its parent pointer's trusted status. This patch updates the kfuncs documentation to specify this macro and how it can be used.
--- Documentation/bpf/kfuncs.rst | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)
Documentation/bpf/kfuncs.rst +21 -1
@@ -167,7 +167,8 @@ KF_ACQUIRE and KF_RET_NULL flags. The KF_TRUSTED_ARGS flag is used for kfuncs taking pointer arguments. It indicates that the all pointer arguments are valid, and that all pointers to BTF objects have been passed in their unmodified form (that is, at a zero -offset, and without having been obtained from walking another pointer). +offset, and without having been obtained from walking another pointer, with one +exception described below). There are two types of pointers to kernel objects which are considered "valid":
@@ -180,6 +181,25 @@ KF_TRUSTED_ARGS kfuncs, and may have a non-zero offset. The definition of "valid" pointers is subject to change at any time, and has absolutely no ABI stability guarantees. +As mentioned above, a nested pointer obtained from walking a trusted pointer is +no longer trusted, with one exception. If a struct type has a field that is +guaranteed to be valid as long as its parent pointer is trusted, the +``BTF_TYPE_SAFE_NESTED`` macro can be used to express that to the verifier as +follows: + +.. code-block:: c + + BTF_TYPE_SAFE_NESTED(struct task_struct) { + const cpumask_t *cpus_ptr; + }; + +In other words, you must: + +1. Wrap the trusted pointer type in the ``BTF_TYPE_SAFE_NESTED`` macro. + +2. Specify the type and name of the trusted nested field. This field must match + the field in the original type definition exactly. + 2.4.6 KF_SLEEPABLE flag -----------------------
Signature · 1 line
2.39.0
PATCH v2 9/9 bpf/docs: Document the nocast aliasing behavior of ___init
In reply to David VernetWhen comparing BTF IDs for pointers being passed to kfunc arguments, the verifier will allow pointer types that are equivalent according to the C standard. For example, for:
struct bpf_cpumask {
cpumask_t cpumask;
refcount_t usage;
};The verifier will allow a struct bpf_cpumask * to be passed to a kfunc that takes a const struct cpumask * (cpumask_t is a typedef of struct cpumask). The exception to this rule is if a type is suffixed with ___init, such as:
struct nf_conn___init {
struct nf_conn ct;
};The verifier will _not_ allow a struct nf_conn___init * to be passed to a kfunc that expects a struct nf_conn *. This patch documents this behavior in the kfuncs documentation page.
--- Documentation/bpf/kfuncs.rst | 43 ++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
Documentation/bpf/kfuncs.rst +43 -0
@@ -247,6 +247,49 @@ type. An example is shown below:: } late_initcall(init_subsystem); +2.6 Specifying no-cast aliases with ___init +-------------------------------------------- + +The verifier will always enforce that the BTF type of a pointer passed to a +kfunc by a BPF program, matches the type of pointer specified in the kfunc +definition. The verifier, does, however, allow types that are equivalent +according to the C standard to be passed to the same kfunc arg, even if their +BTF_IDs differ . + +For example, for the following type definition: + +.. code-block:: c + + struct bpf_cpumask { + cpumask_t cpumask; + refcount_t usage; + }; + +The verifier would allow a ``struct bpf_cpumask *`` to be passed to a kfunc +taking a ``cpumask_t *`` (which is a typedef of ``struct cpumask *``). For +instance, both ``struct cpumask *`` and ``struct bpf_cpmuask *`` can be passed +to bpf_cpumask_test_cpu(). + +In some cases, this type-aliasing behavior is not desired. ``struct +nf_conn___init`` is one such example: + +.. code-block:: c + + struct nf_conn___init { + struct nf_conn ct; + }; + +The C standard would consider these types to be equivalent, but it would not +always be safe to pass either type to a trusted kfunc. ``struct +nf_conn___init`` represents an allocated ``struct nf_conn`` object that has +*not yet been initialized*, so it would therefore be unsafe to pass a ``struct +nf_conn___init *`` to a kfunc that's expecting a fully initialized ``struct +nf_conn *`` (e.g. ``bpf_ct_change_timeout()``). + +In order to accommodate such requirements, the verifier will enforce strict +PTR_TO_BTF_ID type matching if two types have the exact same name, with one +being suffixed with ``___init``. + 3. Core kfuncs ==============
Signature · 1 line
2.39.0
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 23 lines · On Fri, 20 Jan 2023 13:25:14 -0600 you wrote:
This is part 2 of https://lore.kernel.org/all/[email protected]/
Changelog:
----------
v1 -> v2:
- Put back 'static' keyword inbpf_find_btf_id()
(kernel test robot <[email protected]>)
- Surround cpumask kfuncs in__diag()blocks to avoid no-prototype build warnings (kernel test robot <[email protected]>)
- Enable___initsuffixes to a type definition to signal that a type is a nocast alias of another type. That is, that when passed to a kfunc that expects one of the two types, the verifier will reject the other even if they're equivalent according to the C standard (Kumar and Alexei)
- Reject NULL for all trusted args, not justPTR_TO_MEM(Kumar)
- Reject both NULL andPTR_MAYBE_NULLfor all trusted args (Kumar and Alexei )
- Improve examples given in cpumask documentation (Alexei)
- Use__successmacro fornested_trusttest (Alexei)
- Fix comment typo in structbpf_cpumaskcomment header.
- Fix another example in thebpf_cpumaskdoc examples.
- Add documentation for___initsuffix change mentioned above.
Quoted 1 line
[...]
Here is the summary with links:
- [bpf-next,v2,1/9] bpf: Enable annotating trusted nested pointers
https://git.kernel.org/bpf/bpf-next/c/57539b1c0ac2 - [bpf-next,v2,2/9] bpf: Allow trusted args to walk struct when checking BTF IDs https://git.kernel.org/bpf/bpf-next/c/b613d335a743
- [bpf-next,v2,3/9] bpf: Disallow NULLable pointers for trusted kfuncs (no matching commit)
- [bpf-next,v2,4/9] bpf: Enable cpumasks to be queried and used as kptrs (no matching commit)
- [bpf-next,v2,5/9] selftests/bpf: Add nested trust selftests suite (no matching commit)
- [bpf-next,v2,6/9] selftests/bpf: Add selftest suite for cpumask kfuncs (no matching commit)
- [bpf-next,v2,7/9] bpf/docs: Document cpumask kfuncs in a new file (no matching commit)
- [bpf-next,v2,8/9] bpf/docs: Document how nested trusted fields may be defined (no matching commit)
- [bpf-next,v2,9/9] bpf/docs: Document the nocast aliasing behavior of ___init (no matching commit)
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html