bpf: introduce helper for populating bpf_cpumask
Subsystems (3)
Emil Tsalapatis
reviewerv3 cover bpf: introduce helper for populating bpf_cpumask
Some BPF programs like scx schedulers have their own internal CPU mask types, mask types, which they must transform into struct bpf_cpumask instances before passing them to scheduling-related kfuncs. There is currently no way to efficiently populate the bitfield of a bpf_cpumask from BPF memory, and programs must use multiple bpf_cpumask_[set, clear] calls to do so. Introduce a kfunc helper to populate the bitfield of a bpf_cpumask from valid BPF memory with a single call.
Changelog :
-----------v2->v3
v2: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Alexei Starovoitov:
* Added back patch descriptions dropped from v1->v2
* Elide the alignment check for archs with efficient unaligned accesses
v1->v2
v1: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Hou Tao:
* Add check that the input buffer is aligned to sizeof(long)
* Adjust input buffer size check to use bitmap_size()
* Add selftest for checking the bit pattern of the bpf_cpumask
* Moved all selftests into existing files
Emil Tsalapatis (2):
bpf: add kfunc for populating cpumask bits
selftests: bpf: add bpf_cpumask_fill selftests
kernel/bpf/cpumask.c | 28 +++++ .../selftests/bpf/prog_tests/cpumask.c | 3 + .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 4 files changed, 183 insertions(+)
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v3 1/2 bpf: add kfunc for populating cpumask bits
Add a helper kfunc that sets the bitmap of a bpf_cpumask from BPF memory.
--- kernel/bpf/cpumask.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
kernel/bpf/cpumask.c +28 -0
@@ -420,6 +420,33 @@ __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask)return cpumask_weight (cpumask); } +/** +* bpf_cpumask_fill() - Populate the CPU mask from the contents of +* a BPF memory region. +* +* @cpumask: The cpumask being populated. +* @src: The BPF memory holding the bit pattern. +* @src__sz: Length of the BPF memory region in bytes. +* +*/ +__bpf_kfunc int bpf_cpumask_fill(struct cpumask *cpumask, void *src, size_t src__sz) +{ +unsigned long source = (unsigned long )src; + +/* The memory region must be large enough to populate the entire CPU mask. */ +if (src__sz <bitmap_size (nr_cpu_ids)) +return -EACCES ; + +/* If avoiding unaligned accesses, the input region must be aligned to the nearest long. */ +if (!IS_ENABLED (CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS ) && + !IS_ALIGNED (source,sizeof (long ))) +return -EINVAL ; + +bitmap_copy (cpumask_bits (cpumask), src, nr_cpu_ids); + +return 0 ; +} +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (cpumask_kfunc_btf_ids)
@@ -448,6 +475,7 @@ BTF_ID_FLAGS(func, bpf_cpumask_copy, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_any_distribute, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_any_and_distribute, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_weight, KF_RCU) +BTF_ID_FLAGS(func, bpf_cpumask_fill, KF_RCU)BTF_KFUNCS_END (cpumask_kfunc_btf_ids) static const struct btf_kfunc_id_set cpumask_kfunc_set = {
Signature · 1 line
2.47.1
Re: [PATCH v3 1/2] bpf: add kfunc for populating cpumask bits
Quoted 15 lines · Emil Tsalapatis
Add a helper kfunc that sets the bitmap of a
bpf_cpumaskfrom BPF memory.Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- kernel/bpf/cpumask.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)kernel/bpf/cpumask.c +0
-0@@ -420,6 +420,33 @@ __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask)return cpumask_weight (cpumask); }
kernel/bpf/cpumask.c __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask) · line 431
Quoted 10 lines
kernel/bpf/cpumask.c +9
-0+/** +* bpf_cpumask_fill() - Populate the CPU mask from the contents of +* a BPF memory region. +* +* @cpumask: The cpumask being populated. +* @src: The BPF memory holding the bit pattern. +* @src__sz: Length of the BPF memory region in bytes. +* +*/
Since you're adding kdoc, make it complete.
Otherwise there is a warn during the build:
Quoted 1 line
kernel/bpf/cpumask.c:433: warning: No description found for return value of 'bpf_cpumask_fill'
and while at it, could you fix it for other kfuncs?
kernel/bpf/cpumask.c:50: warning: No description found for return value of 'bpf_cpumask_create'kernel/bpf/cpumask.c:76: warning: No description found for return value of 'bpf_cpumask_acquire'
Thanks!
Re: [PATCH v3 1/2] bpf: add kfunc for populating cpumask bits
In reply to Emil TsalapatisHello,
On Wed, Mar 05, 2025 at 11:13:26AM -0500, Emil Tsalapatis wrote:
...
Quoted 3 lines
+/**
+ *bpf_cpumask_fill()- Populate the CPU mask from the contents of
+ * a BPF memory region.
bitmap_fill() fills the bitmap with all 1's, so this name can be pretty confusing. Maybe bpf_cpumask_populate_from() or something like that?
Thanks.
Signature · 1 line
tejun
Emil Tsalapatis
reviewerPATCH v3 2/2 selftests: bpf: add bpf_cpumask_fill selftests
In reply to Emil TsalapatisAdd selftests for the bpf_cpumask_fill helper that sets a bpf_cpumask to a bit pattern provided by a BPF program.
--- .../selftests/bpf/prog_tests/cpumask.c | 3 + .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 3 files changed, 155 insertions(+)
tools/testing/selftests/bpf/prog_tests/cpumask.c +3 -0
@@ -25,6 +25,9 @@ static const char * const cpumask_success_testcases[] = {"test_global_mask_nested_deep_rcu" ,"test_global_mask_nested_deep_array_rcu" ,"test_cpumask_weight" , +"test_fill_reject_unaligned" , +"test_fill_reject_small_mask" , + "test_fill", };static void verify_success (const char *prog_name)
tools/testing/selftests/bpf/progs/cpumask_failure.c +38 -0
@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_fill_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_fill ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("leads to invalid memory access") +int BPF_PROG(test_fill_invalid_source, struct task_struct *task, u64 clone_flags) +{ +void *garbage = (void *)0x123456 ; +struct bpf_cpumask *local; +int ret; + + local =create_cpumask (); +if (!local) { + err =1 ; +return 0 ; + } + + ret =bpf_cpumask_fill ((struct cpumask *)local, garbage,8 ); +if (!ret) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}
tools/testing/selftests/bpf/progs/cpumask_success.c +114 -0
123 lines not on this page · raw message
Signature · 1 line
2.47.1