bpf: introduce helper for populating bpf_cpumask
Subsystems (3)
Emil Tsalapatis
reviewerv5 cover bpf: introduce helper for populating bpf_cpumask
Some BPF programs like scx schedulers have their own internal CPU mask types, mask types, which they must transform into struct bpf_cpumask instances before passing them to scheduling-related kfuncs. There is currently no way to efficiently populate the bitfield of a bpf_cpumask from BPF memory, and programs must use multiple bpf_cpumask_[set, clear] calls to do so. Introduce a kfunc helper to populate the bitfield of a bpf_cpumask from valid BPF memory with a single call.
Changelog :
-----------v4->v5
v4: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Hou Tao:
* Readded the tests in tools/selftests/bpf/prog_tests/cpumask.c, turns out the selftest entries were not duplicates.
* Removed stray whitespace in selftest.
* Add patch the missing selftest to prog_tests/cpumask.c
* Explicitly annotate all cpumask selftests with __success
The last patch could very well be its own cleanup patch, but I rolled it into this series because it came up in the discussion. If the last patch in the series has any issues I'd be fine with applying the first 3 patches and dealing with it separately.
v3->v4
v3: https://lore.kernel.org/bpf/[email protected]/
* Removed new tests from tools/selftests/bpf/prog_tests/cpumask.c because
they were being run twice.
Addressed feedback by Alexei Starovoitov:
* Added missing return value in function kdoc
* Added an additional patch fixing some missing kdoc fields in kernel/bpf/cpumask.c
Addressed feedback by Tejun Heo:
* Renamed the kfunc to bpf_cpumask_populate to avoid confusion w/ bitmap_fill()
v2->v3
v2: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Alexei Starovoitov:
* Added back patch descriptions dropped from v1->v2
* Elide the alignment check for archs with efficient unaligned accesses
v1->v2
v1: https://lore.kernel.org/bpf/[email protected]/
Addressed feedback by Hou Tao:
* Add check that the input buffer is aligned to sizeof(long)
* Adjust input buffer size check to use bitmap_size()
* Add selftest for checking the bit pattern of the bpf_cpumask
* Moved all selftests into existing files
Emil Tsalapatis (4):
bpf: add kfunc for populating cpumask bits
selftests: bpf: add bpf_cpumask_fill selftests
bpf: fix missing kdoc string fields in cpumask.c
selftests: bpf: add missing cpumask test to runner and annotate
existing tests
kernel/bpf/cpumask.c | 53 +++++++ .../selftests/bpf/prog_tests/cpumask.c | 4 + .../selftests/bpf/progs/cpumask_common.h | 1 + .../selftests/bpf/progs/cpumask_failure.c | 38 +++++ .../selftests/bpf/progs/cpumask_success.c | 131 ++++++++++++++++++ 5 files changed, 227 insertions(+)
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v5 1/4 bpf: add kfunc for populating cpumask bits
Add a helper kfunc that sets the bitmap of a bpf_cpumask from BPF memory.
--- kernel/bpf/cpumask.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
kernel/bpf/cpumask.c +33 -0
@@ -420,6 +420,38 @@ __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask)return cpumask_weight (cpumask); } +/** +* bpf_cpumask_populate() - Populate the CPU mask from the contents of +* a BPF memory region. +* +* @cpumask: The cpumask being populated. +* @src: The BPF memory holding the bit pattern. +* @src__sz: Length of the BPF memory region in bytes. +* +* Return: +* * 0 if the struct cpumask * instance was populated successfully. +* * -EACCES if the memory region is too small to populate the cpumask. +* * -EINVAL if the memory region is not aligned to the size of a long +* and the architecture does not support efficient unaligned accesses. +*/ +__bpf_kfunc int bpf_cpumask_populate(struct cpumask *cpumask, void *src, size_t src__sz) +{ +unsigned long source = (unsigned long )src; + +/* The memory region must be large enough to populate the entire CPU mask. */ +if (src__sz <bitmap_size (nr_cpu_ids)) +return -EACCES ; + +/* If avoiding unaligned accesses, the input region must be aligned to the nearest long. */ +if (!IS_ENABLED (CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS ) && + !IS_ALIGNED (source,sizeof (long ))) +return -EINVAL ; + +bitmap_copy (cpumask_bits (cpumask), src, nr_cpu_ids); + +return 0 ; +} +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (cpumask_kfunc_btf_ids)
@@ -448,6 +480,7 @@ BTF_ID_FLAGS(func, bpf_cpumask_copy, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_any_distribute, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_any_and_distribute, KF_RCU) BTF_ID_FLAGS(func, bpf_cpumask_weight, KF_RCU) +BTF_ID_FLAGS(func, bpf_cpumask_populate, KF_RCU)BTF_KFUNCS_END (cpumask_kfunc_btf_ids) static const struct btf_kfunc_id_set cpumask_kfunc_set = {
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v5 2/4 selftests: bpf: add bpf_cpumask_fill selftests
In reply to Emil TsalapatisAdd selftests for the bpf_cpumask_fill helper that sets a bpf_cpumask to a bit pattern provided by a BPF program.
--- .../selftests/bpf/prog_tests/cpumask.c | 3 + .../selftests/bpf/progs/cpumask_common.h | 1 + .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 113 ++++++++++++++++++ 4 files changed, 155 insertions(+)
tools/testing/selftests/bpf/prog_tests/cpumask.c +3 -0
@@ -25,6 +25,9 @@ static const char * const cpumask_success_testcases[] = {"test_global_mask_nested_deep_rcu" ,"test_global_mask_nested_deep_array_rcu" ,"test_cpumask_weight" , +"test_populate_reject_small_mask" , +"test_populate_reject_unaligned" , + "test_populate", };static void verify_success (const char *prog_name)
tools/testing/selftests/bpf/progs/cpumask_common.h +1 -0
@@ -61,6 +61,7 @@ u32 bpf_cpumask_any_distribute(const struct cpumask *src) __ksym __weak; u32 bpf_cpumask_any_and_distribute(const struct cpumask *src1, const struct cpumask *src2) __ksym __weak; u32 bpf_cpumask_weight(const struct cpumask *cpumask) __ksym __weak; +int bpf_cpumask_populate(struct cpumask *cpumask, void *src, size_t src__sz) __ksym __weak; void bpf_rcu_read_lock(void) __ksym __weak; void bpf_rcu_read_unlock(void) __ksym __weak;
tools/testing/selftests/bpf/progs/cpumask_failure.c +38 -0
@@ -222,3 +222,41 @@ int BPF_PROG(test_invalid_nested_array, struct task_struct *task, u64 clone_flagreturn 0 ; } + +SEC("tp_btf/task_newtask") +__failure __msg("type=scalar expected=fp") +int BPF_PROG(test_populate_invalid_destination, struct task_struct *task, u64 clone_flags) +{ +struct bpf_cpumask *invalid = (struct bpf_cpumask *)0x123456 ; +u64 bits; +int ret; + + ret =bpf_cpumask_populate ((struct cpumask *)invalid, &bits,sizeof (bits)); +if (!ret) + err =2 ; + +return 0 ; +} + +SEC("tp_btf/task_newtask") +__failure __msg("leads to invalid memory access") +int BPF_PROG(test_populate_invalid_source, struct task_struct *task, u64 clone_flags) +{ +void *garbage = (void *)0x123456 ; +struct bpf_cpumask *local; +int ret; + + local =create_cpumask (); +if (!local) { + err =1 ; +return 0 ; + } + + ret =bpf_cpumask_populate ((struct cpumask *)local, garbage,8 ); +if (!ret) + err =2 ; + +bpf_cpumask_release (local); + +return 0 ; +}
tools/testing/selftests/bpf/progs/cpumask_success.c +113 -0
116 lines not on this page · raw message
Signature · 1 line
2.47.1
Re: [PATCH v5 2/4] selftests: bpf: add bpf_cpumask_fill selftests
Hi,
Quoted 2 lines · Emil Tsalapatis
Add selftests for the
bpf_cpumask_fillhelper that sets abpf_cpumaskto a bit pattern provided by a BPF program.
bpf_cpumask_populate instead of bpf_cpumask_fill ?
Quoted 35 lines
Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- .../selftests/bpf/prog_tests/cpumask.c | 3 + .../selftests/bpf/progs/cpumask_common.h | 1 + .../selftests/bpf/progs/cpumask_failure.c | 38 ++++++ .../selftests/bpf/progs/cpumask_success.c | 113 ++++++++++++++++++ 4 files changed, 155 insertions(+)tools/testing/selftests/bpf/prog_tests/cpumask.c +3
-0@@ -25,6 +25,9 @@ static const char * const cpumask_success_testcases[] = {"test_global_mask_nested_deep_rcu" ,"test_global_mask_nested_deep_array_rcu" ,"test_cpumask_weight" , +"test_populate_reject_small_mask" , +"test_populate_reject_unaligned" , + "test_populate", };static void verify_success (const char *prog_name)tools/testing/selftests/bpf/progs/cpumask_common.h +1
-0@@ -61,6 +61,7 @@ u32 bpf_cpumask_any_distribute(const struct cpumask *src) __ksym __weak; u32 bpf_cpumask_any_and_distribute(const struct cpumask *src1, const struct cpumask *src2) __ksym __weak; u32 bpf_cpumask_weight(const struct cpumask *cpumask) __ksym __weak; +int bpf_cpumask_populate(struct cpumask *cpumask, void *src, size_t src__sz) __ksym __weak; void bpf_rcu_read_lock(void) __ksym __weak; void bpf_rcu_read_unlock(void) __ksym __weak;
SNIP
tools/testing/selftests/bpf/progs/cpumask_success.c int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_fl · line 775
Quoted 11 lines
tools/testing/selftests/bpf/progs/cpumask_success.c +3
-0@@ -770,3 +770,116 @@ int BPF_PROG(test_refcount_null_tracking, struct task_struct *task, u64 clone_flbpf_cpumask_release (mask2);return 0 ; } + +SEC ("tp_btf/task_newtask" ) +__success
Why add __success annotation here ? It is unnecessary and will lead to double run as you had noticed.
Emil Tsalapatis
reviewerPATCH v5 3/4 bpf: fix missing kdoc string fields in cpumask.c
In reply to Emil TsalapatisSome bpf_cpumask-related kfuncs have kdoc strings that are missing return values. Add a the missing descriptions for the return values.
--- kernel/bpf/cpumask.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
kernel/bpf/cpumask.c +20 -0
@@ -45,6 +45,10 @@ __bpf_kfunc_start_defs();* * bpf_cpumask_create() allocates memory using the BPF memory allocator, and * will not block. It may return NULL if no memory is available. +* +* Return: +* * A pointer to a new struct bpf_cpumask instance on success. +* * NULL if the BPF memory allocator is out of memory. */ __bpf_kfunc struct bpf_cpumask *bpf_cpumask_create(void) {
@@ -71,6 +75,10 @@ __bpf_kfunc struct bpf_cpumask *bpf_cpumask_create(void)* Acquires a reference to a BPF cpumask. The cpumask returned by this function * must either be embedded in a map as a kptr, or freed with * bpf_cpumask_release(). +* +* Return: +* * The struct bpf_cpumask pointer passed to the function. +* */ __bpf_kfunc struct bpf_cpumask *bpf_cpumask_acquire(struct bpf_cpumask *cpumask) {
@@ -106,6 +114,9 @@ CFI_NOSEAL(bpf_cpumask_release_dtor);* * Find the index of the first nonzero bit of the cpumask. A struct bpf_cpumask * pointer may be safely passed to this function. +* +* Return: +* * The index of the first nonzero bit in the struct cpumask. */ __bpf_kfunc u32 bpf_cpumask_first(const struct cpumask *cpumask) {
@@ -119,6 +130,9 @@ __bpf_kfunc u32 bpf_cpumask_first(const struct cpumask *cpumask)* * Find the index of the first unset bit of the cpumask. A struct bpf_cpumask * pointer may be safely passed to this function. +* +* Return: +* * The index of the first zero bit in the struct cpumask. */ __bpf_kfunc u32 bpf_cpumask_first_zero(const struct cpumask *cpumask) {
@@ -133,6 +147,9 @@ __bpf_kfunc u32 bpf_cpumask_first_zero(const struct cpumask *cpumask)* * Find the index of the first nonzero bit of the AND of two cpumasks. * struct bpf_cpumask pointers may be safely passed to @src1 and @src2. +* +* Return: +* * The index of the first bit that is nonzero in both cpumask instances. */ __bpf_kfunc u32 bpf_cpumask_first_and(const struct cpumask *src1,const struct cpumask *src2)
@@ -414,6 +431,9 @@ __bpf_kfunc u32 bpf_cpumask_any_and_distribute(const struct cpumask *src1,* @cpumask: The cpumask being queried. * * Count the number of set bits in the given cpumask. +* +* Return: +* * The number of bits set in the mask. */ __bpf_kfunc u32 bpf_cpumask_weight(const struct cpumask *cpumask) {
Signature · 1 line
2.47.1
Emil Tsalapatis
reviewerPATCH v5 4/4 selftests: bpf: add missing cpumask test to runner and annotate existing tests
In reply to Emil TsalapatisThe BPF cpumask selftests are supposed to be run twice, once to ensure that they load properly and once to actually test their behavior. The load test is triggered by annotating the tests with __success, while the run test needs adding to tools/testing/selftests/bpf/prog_tests/cpumask.c the name of the new test. However, most existing tests are missing the __success annotation, and test_refcount_null_tracking is missing from the main test file. Add the missing annotations and test name.
--- .../testing/selftests/bpf/prog_tests/cpumask.c | 1 + .../selftests/bpf/progs/cpumask_success.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+)
tools/testing/selftests/bpf/prog_tests/cpumask.c +1 -0
@@ -25,6 +25,7 @@ static const char * const cpumask_success_testcases[] = {"test_global_mask_nested_deep_rcu" ,"test_global_mask_nested_deep_array_rcu" ,"test_cpumask_weight" , +"test_refcount_null_tracking" ,"test_populate_reject_small_mask" ,"test_populate_reject_unaligned" , "test_populate",
tools/testing/selftests/bpf/progs/cpumask_success.c +18 -0
@@ -136,6 +136,7 @@ static bool create_cpumask_set(struct bpf_cpumask **out1, } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_alloc_free_cpumask ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -152,6 +153,7 @@ int BPF_PROG(test_alloc_free_cpumask, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_set_clear_cpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -181,6 +183,7 @@ int BPF_PROG(test_set_clear_cpu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_setall_clear_cpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -210,6 +213,7 @@ int BPF_PROG(test_setall_clear_cpu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_first_firstzero_cpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -249,6 +253,7 @@ int BPF_PROG(test_first_firstzero_cpu, struct task_struct *task, u64 clone_flags } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_firstand_nocpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2;
@@ -281,6 +286,7 @@ int BPF_PROG(test_firstand_nocpu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_test_and_set_clear ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -313,6 +319,7 @@ int BPF_PROG(test_test_and_set_clear, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_and_or_xor ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2, *dst1, *dst2;
@@ -360,6 +367,7 @@ int BPF_PROG(test_and_or_xor, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_intersects_subset ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2, *dst1, *dst2;
@@ -402,6 +410,7 @@ int BPF_PROG(test_intersects_subset, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_copy_any_anyand ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2, *dst1, *dst2;
@@ -456,6 +465,7 @@ int BPF_PROG(test_copy_any_anyand, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_insert_leave ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -471,6 +481,7 @@ int BPF_PROG(test_insert_leave, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_insert_remove_release ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;
@@ -501,6 +512,7 @@ int BPF_PROG(test_insert_remove_release, struct task_struct *task, u64 clone_fla } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_rcu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *local, *prev;
@@ -534,6 +546,7 @@ int BPF_PROG(test_global_mask_rcu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_array_one_rcu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *local, *prev;
@@ -632,12 +645,14 @@ static int _global_mask_array_rcu(struct bpf_cpumask **mask0, } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_array_rcu ,struct task_struct *task,u64 clone_flags) {return _global_mask_array_rcu (&global_mask_array[0 ], &global_mask_array[1 ]); } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_array_l2_rcu ,struct task_struct *task,u64 clone_flags) {return _global_mask_array_rcu (&global_mask_array_l2[0 ][0 ], &global_mask_array_l2[1 ][0 ]);
@@ -670,6 +685,7 @@ int BPF_PROG(test_global_mask_nested_rcu, struct task_struct *task, u64 clone_fl* incorrect offset. */ SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_nested_deep_rcu ,struct task_struct *task,u64 clone_flags) {int r, i;
@@ -689,6 +705,7 @@ int BPF_PROG(test_global_mask_nested_deep_rcu, struct task_struct *task, u64 clo } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_nested_deep_array_rcu ,struct task_struct *task,u64 clone_flags) {int i;
@@ -706,6 +723,7 @@ int BPF_PROG(test_global_mask_nested_deep_array_rcu, struct task_struct *task, u } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_cpumask_weight ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *local;
Signature · 1 line
2.47.1
Re: [PATCH v5 4/4] selftests: bpf: add missing cpumask test to runner and annotate existing tests
Hi,
Quoted 7 lines · Emil Tsalapatis
The BPF cpumask selftests are supposed to be run twice, once to ensure that they load properly and once to actually test their behavior. The load test is triggered by annotating the tests with
__success, while the run test needs adding totools/testing/selftests/bpf/prog_tests/cpumask.cthe name of the new test. However, most existing tests are missing the__successannotation, andtest_refcount_null_trackingis missing from the main test file. Add the missing annotations and test name.
It seems I have misled you. There is no need to run these programs twice. __success() is used to annotate the test cases which will be run through RUN_TESTS() macros. RUN_TESTS() will run these programs directly through bpf_prog_test_run_opts(). However only specific program types support being run through bpf_prog_test_run_opts(), and tp_btf doesn't support being run through bpf_prog_test_run_opts(). Considering that multiple successful test cases use err to check whether there is any error during the running of program, I suggest only add test_refcount_null_tracking in cpumask_success_testcases and remove the __success annotation.
Quoted 166 lines
Signed-off-by Emil Tsalapatis (Meta) <[email protected]>--- .../testing/selftests/bpf/prog_tests/cpumask.c | 1 + .../selftests/bpf/progs/cpumask_success.c | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+)tools/testing/selftests/bpf/prog_tests/cpumask.c +1
-0@@ -25,6 +25,7 @@ static const char * const cpumask_success_testcases[] = {"test_global_mask_nested_deep_rcu" ,"test_global_mask_nested_deep_array_rcu" ,"test_cpumask_weight" , +"test_refcount_null_tracking" ,"test_populate_reject_small_mask" ,"test_populate_reject_unaligned" , "test_populate",tools/testing/selftests/bpf/progs/cpumask_success.c +18
-0@@ -136,6 +136,7 @@ static bool create_cpumask_set(struct bpf_cpumask **out1, } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_alloc_free_cpumask ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -152,6 +153,7 @@ int BPF_PROG(test_alloc_free_cpumask, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_set_clear_cpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -181,6 +183,7 @@ int BPF_PROG(test_set_clear_cpu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_setall_clear_cpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -210,6 +213,7 @@ int BPF_PROG(test_setall_clear_cpu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_first_firstzero_cpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -249,6 +253,7 @@ int BPF_PROG(test_first_firstzero_cpu, struct task_struct *task, u64 clone_flags } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_firstand_nocpu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2;@@ -281,6 +286,7 @@ int BPF_PROG(test_firstand_nocpu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_test_and_set_clear ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -313,6 +319,7 @@ int BPF_PROG(test_test_and_set_clear, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_and_or_xor ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2, *dst1, *dst2;@@ -360,6 +367,7 @@ int BPF_PROG(test_and_or_xor, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_intersects_subset ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2, *dst1, *dst2;@@ -402,6 +410,7 @@ int BPF_PROG(test_intersects_subset, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_copy_any_anyand ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *mask1, *mask2, *dst1, *dst2;@@ -456,6 +465,7 @@ int BPF_PROG(test_copy_any_anyand, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_insert_leave ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -471,6 +481,7 @@ int BPF_PROG(test_insert_leave, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_insert_remove_release ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *cpumask;@@ -501,6 +512,7 @@ int BPF_PROG(test_insert_remove_release, struct task_struct *task, u64 clone_fla } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_rcu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *local, *prev;@@ -534,6 +546,7 @@ int BPF_PROG(test_global_mask_rcu, struct task_struct *task, u64 clone_flags) } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_array_one_rcu ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *local, *prev;@@ -632,12 +645,14 @@ static int _global_mask_array_rcu(struct bpf_cpumask **mask0, } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_array_rcu ,struct task_struct *task,u64 clone_flags) {return _global_mask_array_rcu (&global_mask_array[0 ], &global_mask_array[1 ]); } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_array_l2_rcu ,struct task_struct *task,u64 clone_flags) {return _global_mask_array_rcu (&global_mask_array_l2[0 ][0 ], &global_mask_array_l2[1 ][0 ]);@@ -670,6 +685,7 @@ int BPF_PROG(test_global_mask_nested_rcu, struct task_struct *task, u64 clone_fl* incorrect offset. */ SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_nested_deep_rcu ,struct task_struct *task,u64 clone_flags) {int r, i;@@ -689,6 +705,7 @@ int BPF_PROG(test_global_mask_nested_deep_rcu, struct task_struct *task, u64 clo } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_global_mask_nested_deep_array_rcu ,struct task_struct *task,u64 clone_flags) {int i;@@ -706,6 +723,7 @@ int BPF_PROG(test_global_mask_nested_deep_array_rcu, struct task_struct *task, u } SEC("tp_btf/task_newtask") +__successint BPF_PROG (test_cpumask_weight ,struct task_struct *task,u64 clone_flags) {struct bpf_cpumask *local;