bpf: introduce helper for populating bpf_cpumask
Subsystems (3)
Emil Tsalapatis
reviewerv4 cover bpf: introduce helper for populating bpf_cpumask
Some BPF programs like scx schedulers have their own internal CPU mask types, mask types, which they must transform into struct bpf_cpumask instances before passing them to scheduling-related kfuncs. There is currently no way to efficiently populate the bitfield of a bpf_cpumask from BPF memory, and programs must use multiple bpf_cpumask_[set, clear] calls to do so. Introduce a kfunc helper to populate the bitfield of a bpf_cpumask from valid BPF memory with a single call.
Changelog :
-----------v3->v4
v3: https://lore.kernel.org/bpf/[email protected]/
* Removed new tests from tools/selftests/bpf/prog_tests/cpumask.c because
they were being run twice.
Addressed feedback by Alexei Starovoitov:
* Added missing return value in function kdoc
* Added an additional patch fixing some missing kdoc fields in kernel/bpf/cpumask.c
Addressed feedback by Tejun Heo:
* Renamed the kfunc to bpf_cpumask_populate to avoid confusion w/ bitmap_fill()
v2->v3
v2: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Alexei Starovoitov:
* Added back patch descriptions dropped from v1->v2
* Elide the alignment check for archs with efficient unaligned accesses
v1->v2
v1: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Hou Tao:
* Add check that the input buffer is aligned to sizeof(long)
* Adjust input buffer size check to use bitmap_size()
* Add selftest for checking the bit pattern of the bpf_cpumask
* Moved all selftests into existing files
Emil Tsalapatis (3):
bpf: add kfunc for populating cpumask bits
selftests: bpf: add bpf_cpumask_fill selftests
bpf: fix missing kdoc string fields in cpumask.c
kernel/bpf/cpumask.c | 53 ++++++++ .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 3 files changed, 205 insertions(+)
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v4 1/3 bpf: add kfunc for populating cpumask bits
Add a helper kfunc that sets the bitmap of a bpf_cpumask from BPF memory.
--- kernel/bpf/cpumask.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
kernel/bpf/cpumask.c +33 -0
@@ -420,6 +420,38 @@ __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask)return cpumask_weight (cpumask); } +/** +* bpf_cpumask_populate() - Populate the CPU mask from the contents of +* a BPF memory region. +* +* @cpumask: The cpumask being populated. +* @src: The BPF memory holding the bit pattern. +* @src__sz: Length of the BPF memory region in bytes. +* +* Return: +* * 0 if the struct cpumask * instance was populated successfully. +* * -EACCES if the memory region is too small to populate the cpumask. +* * -EINVAL if the memory region is not aligned to the size of a long +* and the architecture does not support efficient unaligned accesses. +*/ +__bpf_kfunc int bpf_cpumask_populate(struct cpumask *cpumask, void *src, size_t src__sz) +{ +unsigned long source = (unsigned long )src; + +/* The memory region must be large enough to populate the entire CPU mask. */ +if (src__sz <bitmap_size (nr_cpu_ids)) +return -EACCES ; + +/* If avoiding unaligned accesses, the input region must be aligned to the nearest long. */ +if (!IS_ENABLED (CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS ) && + !IS_ALIGNED (source,sizeof (long ))) +return -EINVAL ; + +bitmap_copy (cpumask_bits (cpumask), src, nr_cpu_ids); + +return 0 ; +} +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (cpumask_kfunc_btf_ids)
@@ -448,6 +480,7 @@ BTF_ID_FLAGS(func, bpf_cpumask_copy, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_any_distribute, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_any_and_distribute, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_weight, KF_RCU) +BTF_ID_FLAGS(func, bpf_cpumask_populate, KF_RCU)BTF_KFUNCS_END (cpumask_kfunc_btf_ids) static const struct btf_kfunc_id_set cpumask_kfunc_set = {
Signature · 1 line
2.47.1
Re: [PATCH v4 1/3] bpf: add kfunc for populating cpumask bits
Quoted 3 lines · Emil Tsalapatis
Add a helper kfunc that sets the bitmap of a
bpf_cpumaskfrom BPF memory.Signed-off-by Emil Tsalapatis (Meta) <[email protected]>
Emil Tsalapatis
reviewerPATCH v4 2/3 selftests: bpf: add bpf_cpumask_fill selftests
In reply to Emil TsalapatisAdd selftests for the bpf_cpumask_fill helper that sets a bpf_cpumask to a bit pattern provided by a BPF program.
--- .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 2 files changed, 152 insertions(+)
tools/testing/selftests/bpf/progs/cpumask_failure.c +38 -0
@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_populate_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_populate ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("leads to invalid memory access") +int BPF_PROG(test_populate_invalid_source, struct task_struct *task, u64 clone_flags) +{ +void *garbage = (void *)0x123456 ; +struct bpf_cpumask *local; +int ret; + + local =create_cpumask (); +if (!local) { + err =1 ; +return 0 ; + } + + ret =bpf_cpumask_populate ((struct cpumask *)local, garbage,8 ); +if (!ret) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}
tools/testing/selftests/bpf/progs/cpumask_success.c +114 -0
123 lines not on this page · raw message
Signature · 1 line
2.47.1
Re: [PATCH v4 2/3] selftests: bpf: add bpf_cpumask_fill selftests
Hi,
Quoted 8 lines · Emil Tsalapatis
Add selftests for the
bpf_cpumask_fillhelper that sets abpf_cpumaskto a bit pattern provided by a BPF program.Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 2 files changed, 152 insertions(+)
My local build failed due to the missed declaration of "bpf_cpumask_populate" in cpumask_common.h. It reported the following error:
progs/cpumask_success.c:788:8: error: call to undeclared function 'bpf_cpumask_populate'; ISO C99 and later do not support implicit fun ction declarations [-Wimplicit-function-declaration]
788 | ret = bpf_cpumask_populate((struct cpumask *)local, &toofewbits, sizeof(toofewbits));
Don't know the reason why CI succeeded.
Quoted 51 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +38
-0@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_populate_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_populate ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("leads to invalid memory access") +int BPF_PROG(test_populate_invalid_source, struct task_struct *task, u64 clone_flags) +{ +void *garbage = (void *)0x123456 ; +struct bpf_cpumask *local; +int ret; + + local =create_cpumask (); +if (!local) { + err =1 ; +return 0 ; + } + + ret =bpf_cpumask_populate ((struct cpumask *)local, garbage,8 ); +if (!ret) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}tools/testing/selftests/bpf/progs/cpumask_success.c +0
-0
tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 762
Quoted 6 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +1
-0@@ -757,6 +757,7 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl mask1 =bpf_cpumask_create (); mask2 =bpf_cpumask_create (); +if (!mask1 || !mask2) goto free_masks_return;
An extra newline.
tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 776
Quoted 8 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +3
-0@@ -770,3 +771,116 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_flbpf_cpumask_release (mask2);return 0 ; } + +SEC ("tp_btf/task_newtask" ) +__success
For tp_btf, bpf_prog_test_run() doesn't run the prog and it just returns directly, therefore, the prog below is not exercised at all. How about add test_populate_reject_small_mask into cpumask_success_testcases firstly, then switch these test cases to use __success() in a following patch ?
Quoted 19 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +19
-0+int BPF_PROG (test_populate_reject_small_mask ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *local; +u8 toofewbits; +int ret; + + local =create_cpumask (); +if (!local) +return 0 ; + +/* The kfunc should prevent this operation */ + ret =bpf_cpumask_populate ((struct cpumask *)local, &toofewbits,sizeof (toofewbits)); +if (ret != -EACCES ) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}
tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 805
Quoted 10 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +10
-0+ +/* Mask is guaranteed to be large enough for bpf_cpumask_t. */ +#define CPUMASK_TEST_MASKLEN (sizeof(cpumask_t)) + +/* Add an extra word for the test_populate_reject_unaligned test. */ +u64 bits[CPUMASK_TEST_MASKLEN /8 +1 ]; +extern bool CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS __kconfig __weak; + +SEC ("tp_btf/task_newtask" ) +__success
Same for test_populate_reject_unaligned.
Quoted 81 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +81
-0+int BPF_PROG (test_populate_reject_unaligned ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *mask; +char *src; +int ret; + +/* Skip if unaligned accesses are fine for this arch. */ +if (CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS ) +return 0 ; + + mask =bpf_cpumask_create (); +if (!mask) { + err =1 ; +return 0 ; + } + +/* Misalign the source array by a byte. */ + src = &((char *)bits)[1 ]; + + ret =bpf_cpumask_populate ((struct cpumask *)mask, src,CPUMASK_TEST_MASKLEN ); +if (ret != -EINVAL ) + err =2 ; + +bpf_cpumask_release (mask); + +return 0 ; +} + + +SEC("tp_btf/task_newtask") +__success +int BPF_PROG (test_populate ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *mask; +bool bit; +int ret; +int i; + +/* Set only odd bits. */ +__builtin_memset (bits,0xaa ,CPUMASK_TEST_MASKLEN ); + + mask =bpf_cpumask_create (); +if (!mask) { + err =1 ; +return 0 ; + } + +/* Pass the entire bits array, the kfunc will only copy the valid bits. */ + ret =bpf_cpumask_populate ((struct cpumask *)mask, bits,CPUMASK_TEST_MASKLEN ); +if (ret) { + err =2 ; + goto out; + } + +/* +* Test is there to appease the verifier. We cannot directly +* access NR_CPUS, the upper bound for nr_cpus, so we infer +* it from the size of cpumask_t. +*/ +if (nr_cpus <0 || nr_cpus >=CPUMASK_TEST_MASKLEN *8 ) { + err =3 ; + goto out; + } + + bpf_for(i, 0, nr_cpus) { +/* Odd-numbered bits should be set, even ones unset. */ + bit =bpf_cpumask_test_cpu (i, (const struct cpumask *)mask); +if (bit == (i %2 !=0 )) +continue ; + + err =4 ; +break ; + } + +out: +bpf_cpumask_release (mask); + +return 0 ; +} + +#undef CPUMASK_TEST_MASKLEN
Re: [PATCH v4 2/3] selftests: bpf: add bpf_cpumask_fill selftests
Quoted 16 lines · Hou Tao
Hi,
Quoted 8 lines · Emil Tsalapatis
Add selftests for the
bpf_cpumask_fillhelper that sets abpf_cpumaskto a bit pattern provided by a BPF program.Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 2 files changed, 152 insertions(+)My local build failed due to the missed declaration of "
bpf_cpumask_populate" incpumask_common.h. It reported the following error:
Quoted 7 lines
progs/cpumask_success.c:788:8: error: call to undeclared function 'bpf_cpumask_populate'; ISO C99 and later do not support implicit fun ction declarations [-Wimplicit-function-declaration]
788 | ret = bpf_cpumask_populate((struct cpumask *)local,
&toofewbits, sizeof(toofewbits));Don't know the reason why CI succeeded.
You need to upgrade pahole to make sure it emits kfuncs into vmlinux.h
Quoted 61 lines
Quoted 52 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +38
-0@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_populate_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_populate ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("leads to invalid memory access") +int BPF_PROG(test_populate_invalid_source, struct task_struct *task, u64 clone_flags) +{ +void *garbage = (void *)0x123456 ; +struct bpf_cpumask *local; +int ret; + + local =create_cpumask (); +if (!local) { + err =1 ; +return 0 ; + } + + ret =bpf_cpumask_populate ((struct cpumask *)local, garbage,8 ); +if (!ret) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}tools/testing/selftests/bpf/progs/cpumask_success.c +0
-0tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 762
Quoted 6 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +1
-0@@ -757,6 +757,7 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl mask1 =bpf_cpumask_create (); mask2 =bpf_cpumask_create (); +if (!mask1 || !mask2) goto free_masks_return;An extra newline.
Quoted 1 line
Quoted 13 lines
tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 776
Quoted 7 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +3
-0@@ -770,3 +771,116 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_flbpf_cpumask_release (mask2);return 0 ; } + +SEC ("tp_btf/task_newtask" ) +__successFor
tp_btf,bpf_prog_test_run()doesn't run the prog and it just returns directly, therefore, the prog below is not exercised at all. How about addtest_populate_reject_small_maskintocpumask_success_testcasesfirstly, then switch these test cases to use__success()in a following patch ?
Good point. I'll revert and wait for respin.
Emil Tsalapatis
reviewerRe: [PATCH v4 2/3] selftests: bpf: add bpf_cpumask_fill selftests
In reply to Hou TaoHi,
thank you for the feedback. I will address it in a v5.
Quoted 16 lines · Hou Tao
Hi,
Quoted 8 lines · Emil Tsalapatis
Add selftests for the
bpf_cpumask_fillhelper that sets abpf_cpumaskto a bit pattern provided by a BPF program.Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 2 files changed, 152 insertions(+)My local build failed due to the missed declaration of "
bpf_cpumask_populate" incpumask_common.h. It reported the following error:
Quoted 8 lines
progs/cpumask_success.c:788:8: error: call to undeclared function 'bpf_cpumask_populate'; ISO C99 and later do not support implicit fun ction declarations [-Wimplicit-function-declaration]
788 | ret = bpf_cpumask_populate((struct cpumask *)local,
&toofewbits, sizeof(toofewbits));Don't know the reason why CI succeeded.
Based on Alexei's email systems with recent pahole versions handle this fine (at least the CI and my local setup),
I will still add the definition in cpumask_common.h for uniformity since all the other kfuncs have one.
Quoted 60 lines
Quoted 51 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +38
-0@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_populate_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_populate ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("leads to invalid memory access") +int BPF_PROG(test_populate_invalid_source, struct task_struct *task, u64 clone_flags) +{ +void *garbage = (void *)0x123456 ; +struct bpf_cpumask *local; +int ret; + + local =create_cpumask (); +if (!local) { + err =1 ; +return 0 ; + } + + ret =bpf_cpumask_populate ((struct cpumask *)local, garbage,8 ); +if (!ret) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}tools/testing/selftests/bpf/progs/cpumask_success.c +0
-0tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 762
Quoted 6 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +1
-0@@ -757,6 +757,7 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl mask1 =bpf_cpumask_create (); mask2 =bpf_cpumask_create (); +if (!mask1 || !mask2) goto free_masks_return;An extra newline.
Quoted 1 line
Quoted 13 lines
tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 776
Quoted 7 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +3
-0@@ -770,3 +771,116 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_flbpf_cpumask_release (mask2);return 0 ; } + +SEC ("tp_btf/task_newtask" ) +__successFor
tp_btf,bpf_prog_test_run()doesn't run the prog and it just returns directly, therefore, the prog below is not exercised at all. How about addtest_populate_reject_small_maskintocpumask_success_testcasesfirstly, then switch these test cases to use__success()in a following patch ?
Sorry about that, I had the selftests properly hooked into prog_tests/cpumask.c until v3 but saw duplicate entries in the selftest log
and thought it was being run twice. I will add them back in.
Is __success() a different annotation? AFAICT __success is enough as long as err is set to nonzero on an error path, and all error paths are set like that in the selftests. In that case, shouldn't adding the new tests cpumask_success_testcases be enough to properly run the tests?
Quoted 113 lines
Quoted 19 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +19
-0+int BPF_PROG (test_populate_reject_small_mask ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *local; +u8 toofewbits; +int ret; + + local =create_cpumask (); +if (!local) +return 0 ; + +/* The kfunc should prevent this operation */ + ret =bpf_cpumask_populate ((struct cpumask *)local, &toofewbits,sizeof (toofewbits)); +if (ret != -EACCES ) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 805
Quoted 10 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +10
-0+ +/* Mask is guaranteed to be large enough for bpf_cpumask_t. */ +#define CPUMASK_TEST_MASKLEN (sizeof(cpumask_t)) + +/* Add an extra word for the test_populate_reject_unaligned test. */ +u64 bits[CPUMASK_TEST_MASKLEN /8 +1 ]; +extern bool CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS __kconfig __weak; + +SEC ("tp_btf/task_newtask" ) +__successSame for
test_populate_reject_unaligned.Quoted 81 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +81
-0+int BPF_PROG (test_populate_reject_unaligned ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *mask; +char *src; +int ret; + +/* Skip if unaligned accesses are fine for this arch. */ +if (CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS ) +return 0 ; + + mask =bpf_cpumask_create (); +if (!mask) { + err =1 ; +return 0 ; + } + +/* Misalign the source array by a byte. */ + src = &((char *)bits)[1 ]; + + ret =bpf_cpumask_populate ((struct cpumask *)mask, src,CPUMASK_TEST_MASKLEN ); +if (ret != -EINVAL ) + err =2 ; + +bpf_cpumask_release (mask); + +return 0 ; +} + + +SEC("tp_btf/task_newtask") +__success +int BPF_PROG (test_populate ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *mask; +bool bit; +int ret; +int i; + +/* Set only odd bits. */ +__builtin_memset (bits,0xaa ,CPUMASK_TEST_MASKLEN ); + + mask =bpf_cpumask_create (); +if (!mask) { + err =1 ; +return 0 ; + } + +/* Pass the entire bits array, the kfunc will only copy the valid bits. */ + ret =bpf_cpumask_populate ((struct cpumask *)mask, bits,CPUMASK_TEST_MASKLEN ); +if (ret) { + err =2 ; + goto out; + } + +/* +* Test is there to appease the verifier. We cannot directly +* access NR_CPUS, the upper bound for nr_cpus, so we infer +* it from the size of cpumask_t. +*/ +if (nr_cpus <0 || nr_cpus >=CPUMASK_TEST_MASKLEN *8 ) { + err =3 ; + goto out; + } + + bpf_for(i, 0, nr_cpus) { +/* Odd-numbered bits should be set, even ones unset. */ + bit =bpf_cpumask_test_cpu (i, (const struct cpumask *)mask); +if (bit == (i %2 !=0 )) +continue ; + + err =4 ; +break ; + } + +out: +bpf_cpumask_release (mask); + +return 0 ; +} + +#undef CPUMASK_TEST_MASKLEN
Re: [PATCH v4 2/3] selftests: bpf: add bpf_cpumask_fill selftests
Hi,
Quoted 10 lines · Emil Tsalapatis
Hi,
thank you for the feedback. I will address it in a v5.
Quoted 5 lines · Hou Tao
Hi,
Quoted 2 lines · Emil Tsalapatis
Add selftests for the
bpf_cpumask_fillhelper that sets abpf_cpumaskto a bit pattern provided by a BPF program.
Just find out, the name of bpf_cpumask_fill() also needs update.
Quoted 15 lines
Quoted 15 lines
Quoted 6 lines
Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 114 ++++++++++++++++++ 2 files changed, 152 insertions(+)My local build failed due to the missed declaration of "
bpf_cpumask_populate" incpumask_common.h. It reported the following error:
progs/cpumask_success.c:788:8: error: call to undeclared function 'bpf_cpumask_populate'; ISO C99 and later do not support implicit fun ction declarations [-Wimplicit-function-declaration]
788 | ret = bpf_cpumask_populate((struct cpumask *)local,
&toofewbits, sizeof(toofewbits));
Quoted 6 lines
Quoted 2 lines
Don't know the reason why CI succeeded.
Based on Alexei's email systems with recent pahole versions handle this fine (at least the CI and my local setup),
I will still add the definition incpumask_common.hfor uniformity since all the other kfuncs have one.
I see. Thanks for that.
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +16
-0@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_populate_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_populate ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} +
SNIP
Quoted 18 lines
Quoted 13 lines
An extra newline.
tools/testing/selftests/bpf/progs/cpumask_failure.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 776
Quoted 7 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +3
-0@@ -770,3 +771,116 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_flbpf_cpumask_release (mask2);return 0 ; } + +SEC ("tp_btf/task_newtask" ) +__successFor
tp_btf,bpf_prog_test_run()doesn't run the prog and it just returns directly, therefore, the prog below is not exercised at all. How about addtest_populate_reject_small_maskintocpumask_success_testcasesfirstly, then switch these test cases to use__success()in a following patch ?Sorry about that, I had the selftests properly hooked into
prog_tests/cpumask.cuntil v3 but saw duplicate entries in the selftest log
and thought it was being run twice. I will add them back in.
Quoted 5 lines
Is
__success()a different annotation? AFAICT__successis enough as long as err is set to nonzero on an error path, and all error paths are set like that in the selftests. In that case, shouldn't adding the new testscpumask_success_testcasesbe enough to properly run the tests?
Yes. __success() annotation is a bit different. It uses bpf_prog_test_run() to run the bpf prog directly instead of trigger the running of prog through an external event. I think adding new tests in cpumask_success_testcases will be enough. However, there is one success test test_refcount_null_tracking in cpumask_success.c which uses __success annotation, and it is still buggy. I think it would be better to switch all test cases to use __success annotation because the annotation provides much clarity.
Quoted 113 lines
Quoted 113 lines
Quoted 21 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +19
-0+int BPF_PROG (test_populate_reject_small_mask ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *local; +u8 toofewbits; +int ret; + + local =create_cpumask (); +if (!local) +return 0 ; + +/* The kfunc should prevent this operation */ + ret =bpf_cpumask_populate ((struct cpumask *)local, &toofewbits,sizeof (toofewbits)); +if (ret != -EACCES ) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}tools/testing/selftests/bpf/progs/cpumask_failure.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 805
Quoted 10 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +10
-0+ +/* Mask is guaranteed to be large enough for bpf_cpumask_t. */ +#define CPUMASK_TEST_MASKLEN (sizeof(cpumask_t)) + +/* Add an extra word for the test_populate_reject_unaligned test. */ +u64 bits[CPUMASK_TEST_MASKLEN /8 +1 ]; +extern bool CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS __kconfig __weak; + +SEC ("tp_btf/task_newtask" ) +__successSame for
test_populate_reject_unaligned.Quoted 81 lines
tools/testing/selftests/bpf/progs/cpumask_failure.c +81
-0+int BPF_PROG (test_populate_reject_unaligned ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *mask; +char *src; +int ret; + +/* Skip if unaligned accesses are fine for this arch. */ +if (CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS ) +return 0 ; + + mask =bpf_cpumask_create (); +if (!mask) { + err =1 ; +return 0 ; + } + +/* Misalign the source array by a byte. */ + src = &((char *)bits)[1 ]; + + ret =bpf_cpumask_populate ((struct cpumask *)mask, src,CPUMASK_TEST_MASKLEN ); +if (ret != -EINVAL ) + err =2 ; + +bpf_cpumask_release (mask); + +return 0 ; +} + + +SEC("tp_btf/task_newtask") +__success +int BPF_PROG (test_populate ,struct task_struct *task,u64 clone_flags) +{ +struct bpf_cpumask *mask; +bool bit; +int ret; +int i; + +/* Set only odd bits. */ +__builtin_memset (bits,0xaa ,CPUMASK_TEST_MASKLEN ); + + mask =bpf_cpumask_create (); +if (!mask) { + err =1 ; +return 0 ; + } + +/* Pass the entire bits array, the kfunc will only copy the valid bits. */ + ret =bpf_cpumask_populate ((struct cpumask *)mask, bits,CPUMASK_TEST_MASKLEN ); +if (ret) { + err =2 ; + goto out; + } + +/* +* Test is there to appease the verifier. We cannot directly +* access NR_CPUS, the upper bound for nr_cpus, so we infer +* it from the size of cpumask_t. +*/ +if (nr_cpus <0 || nr_cpus >=CPUMASK_TEST_MASKLEN *8 ) { + err =3 ; + goto out; + } + + bpf_for(i, 0, nr_cpus) { +/* Odd-numbered bits should be set, even ones unset. */ + bit =bpf_cpumask_test_cpu (i, (const struct cpumask *)mask); +if (bit == (i %2 !=0 )) +continue ; + + err =4 ; +break ; + } + +out: +bpf_cpumask_release (mask); + +return 0 ; +} + +#undef CPUMASK_TEST_MASKLEN
Emil Tsalapatis
reviewerPATCH v4 3/3 bpf: fix missing kdoc string fields in cpumask.c
In reply to Emil TsalapatisSome bpf_cpumask-related kfuncs have kdoc strings that are missing return values. Add a the missing descriptions for the return values.
--- kernel/bpf/cpumask.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
kernel/bpf/cpumask.c +20 -0
@@ -45,6 +45,10 @@ __bpf_kfunc_start_defs();* * bpf_cpumask_create() allocates memory using the BPF memory allocator, and * will not block. It may return NULL if no memory is available. +* +* Return: +* * A pointer to a new struct bpf_cpumask instance on success. +* * NULL if the BPF memory allocator is out of memory. */ __bpf_kfunc struct bpf_cpumask *bpf_cpumask_create(void) {
@@ -71,6 +75,10 @@ __bpf_kfunc struct bpf_cpumask *bpf_cpumask_create(void)* Acquires a reference to a BPF cpumask. The cpumask returned by this function * must either be embedded in a map as a kptr, or freed with * bpf_cpumask_release(). +* +* Return: +* * The struct bpf_cpumask pointer passed to the function. +* */ __bpf_kfunc struct bpf_cpumask *bpf_cpumask_acquire(struct bpf_cpumask *cpumask) {
@@ -106,6 +114,9 @@ CFI_NOSEAL(bpf_cpumask_release_dtor);* * Find the index of the first nonzero bit of the cpumask. A struct bpf_cpumask * pointer may be safely passed to this function. +* +* Return: +* * The index of the first nonzero bit in the struct cpumask. */ __bpf_kfunc u32 bpf_cpumask_first(const struct cpumask *cpumask) {
@@ -119,6 +130,9 @@ __bpf_kfunc u32 bpf_cpumask_first(const struct cpumask *cpumask)* * Find the index of the first unset bit of the cpumask. A struct bpf_cpumask * pointer may be safely passed to this function. +* +* Return: +* * The index of the first zero bit in the struct cpumask. */ __bpf_kfunc u32 bpf_cpumask_first_zero(const struct cpumask *cpumask) {
@@ -133,6 +147,9 @@ __bpf_kfunc u32 bpf_cpumask_first_zero(const struct cpumask *cpumask)* * Find the index of the first nonzero bit of the AND of two cpumasks. * struct bpf_cpumask pointers may be safely passed to @src1 and @src2. +* +* Return: +* * The index of the first bit that is nonzero in both cpumask instances. */ __bpf_kfunc u32 bpf_cpumask_first_and(const struct cpumask *src1,const struct cpumask *src2)
@@ -414,6 +431,9 @@ __bpf_kfunc u32 bpf_cpumask_any_and_distribute(const struct cpumask *src1,* @cpumask: The cpumask being queried. * * Count the number of set bits in the given cpumask. +* +* Return: +* * The number of bits set in the mask. */ __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask) {
Signature · 1 line
2.47.1
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 9 lines · On Wed, 5 Mar 2025 16:12:32 -0500 you wrote:
Some BPF programs like scx schedulers have their own internal CPU mask types, mask types, which they must transform into struct
bpf_cpumaskinstances before passing them to scheduling-related kfuncs. There is currently no way to efficiently populate the bitfield of abpf_cpumaskfrom BPF memory, and programs must use multiple bpf_cpumask_[set, clear] calls to do so. Introduce a kfunc helper to populate the bitfield of abpf_cpumaskfrom valid BPF memory with a single call.[...]
Here is the summary with links:
- [v4,1/3] bpf: add kfunc for populating cpumask bits
https://git.kernel.org/bpf/bpf-next/c/a6db20f88a63 - [v4,2/3] selftests: bpf: add bpf_cpumask_fill selftests https://git.kernel.org/bpf/bpf-next/c/66be130f8bf9 - [v4,3/3] bpf: fix missing kdoc string fields in cpumask.c https://git.kernel.org/bpf/bpf-next/c/f3839e9749b1
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html