Skip to contentSkip to conversation

bpf/selftests: test_access_variable_array breaks due to sched_domain::span removal

PATCH Venkat Rao Bagalkote 5 participants 5 messages Apr 7 – Apr 8

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi,

While running BPF selftests on current linux-next, I noticed that test_access_variable_array fails to build due to reliance on
struct sched_domain::span, which is no longer appers to be BTF-visible after recent scheduler refactoring.

The Build error I am seeing is:

progs/test_access_variable_array.c:14:13: error: no member named 'span' in 'struct sched_domain'  CLNG-BPF [test_progs] test_check_mtu.bpf.o

   14 |         span = sd->span[0];
      |                ~~  ^

Below is a proposed update to the test that switches from sched_domain::span to sched_group::cpumask. This preserves the original intent of validating variable-length array access via BTF while avoiding reliance on removed scheduler internals.

tools/testing/selftests/bpf/progs/test_access_variable_array.c +0 -0
index 326b7d1f496a..c9f345ccde3c 100644
@@ -4,14 +4,18 @@

 #include "vmlinux.h"
 #include <bpf/bpf_helpers.h>
 #include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>

-unsigned long span = 0;
+unsigned long cpumask0 = 0;

-SEC("fentry/sched_balance_rq")
-int BPF_PROG(fentry_fentry, int this_cpu, struct rq *this_rq, -               struct sched_domain *sd) +SEC("fentry/sched_balance_find_dst_group_cpu")
+int BPF_PROG(fentry_fentry, struct sched_group *sg, struct task_struct *p, +               int this_cpu)
 {
-       span = sd->span[0];
+       unsigned long *mask;
+       /* Read pointer to variable-length CPU mask */
+       mask = BPF_CORE_READ(sg, cpumask);
+       cpumask0 = mask[0];

        return 0;
 }

I have tested this change, and it seems to be working as expected.

# ./test_progs -t access_variable_array
#1       access_variable_array:OK
Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED

Please note that this comes from my early days of working on upstream kernel contributions, and I am still learning the BPF and scheduler internals. My understanding here may be incomplete, so I wanted to share this primarily to report the breakage I am seeing and propose a possible direction for fixing the test.

I would appreciate any feedback on whether this is the right approach, or if there is a more appropriate structure or hook to use for preserving the variable-length array coverage in this selftest.

Regards,

Venkat.

#1 lore raw
Quoted 41 lines · Venkat Rao Bagalkote

Hi,

While running BPF selftests on current linux-next, I noticed that test_access_variable_array fails to build due to reliance on struct sched_domain::span, which is no longer appers to be BTF-visible after recent
scheduler refactoring.

The Build error I am seeing is:

progs/test_access_variable_array.c:14:13: error: no member named 'span' in 'struct sched_domain' CLNG-BPF [test_progs] test_check_mtu.bpf.o

14 | span = sd->span[0];
| ~~ ^

Below is a proposed update to the test that switches from sched_domain::span to sched_group::cpumask. This preserves the original intent of validating variable-length array access via BTF while avoiding reliance on removed scheduler internals.

diff --git a/tools/testing/selftests/bpf/progs/test_access_variable_array.c b/tools/testing/selftests/bpf/progs/test_access_variable_array.c index 326b7d1f496a..c9f345ccde3c 100644

tools/testing/selftests/bpf/progs/test_access_variable_array.c +2 -3
@@ -4,14 +4,18 @@
  #include "vmlinux.h"
  #include <bpf/bpf_helpers.h>
  #include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
 
-unsigned long span = 0;
+unsigned long cpumask0 = 0;
 
-SEC("fentry/sched_balance_rq")
-int BPF_PROG(fentry_fentry, int this_cpu, struct rq *this_rq,

tools/testing/selftests/bpf/progs/test_access_variable_array.c line 18

Quoted 10 lines
tools/testing/selftests/bpf/progs/test_access_variable_array.c +7 -2
-               struct sched_domain *sd)
+SEC("fentry/sched_balance_find_dst_group_cpu")
+int BPF_PROG(fentry_fentry, struct sched_group *sg, struct task_struct *p,
+               int this_cpu)
  {
-       span = sd->span[0];
+       unsigned long *mask;
+       /* Read pointer to variable-length CPU mask */
+       mask = BPF_CORE_READ(sg, cpumask);
+       cpumask0 = mask[0];

No. That defeats the point of the test.
It tests [] array in the kernel struct.
Just make one in testmod.ko and access that.

#2 lore raw
Quoted 41 lines · Alexei Starovoitov
Quoted 39 lines · Venkat Rao Bagalkote

Hi,

While running BPF selftests on current linux-next, I noticed that test_access_variable_array fails to build due to reliance on struct sched_domain::span, which is no longer appers to be BTF-visible after recent
scheduler refactoring.

The Build error I am seeing is:

progs/test_access_variable_array.c:14:13: error: no member named 'span' in 'struct sched_domain' CLNG-BPF [test_progs] test_check_mtu.bpf.o

14 | span = sd->span[0];
| ~~ ^

Below is a proposed update to the test that switches from sched_domain::span to sched_group::cpumask. This preserves the original intent of validating variable-length array access via BTF while avoiding reliance on removed scheduler internals.

diff --git a/tools/testing/selftests/bpf/progs/test_access_variable_array.c b/tools/testing/selftests/bpf/progs/test_access_variable_array.c index 326b7d1f496a..c9f345ccde3c 100644

tools/testing/selftests/bpf/progs/test_access_variable_array.c +2 -1
@@ -4,14 +4,18 @@
  #include "vmlinux.h"
  #include <bpf/bpf_helpers.h>
  #include <bpf/bpf_tracing.h>
+#include <bpf/bpf_core_read.h>
 
-unsigned long span = 0;
+unsigned long cpumask0 = 0;
 
Quoted 16 lines

tools/testing/selftests/bpf/progs/test_access_variable_array.c line 18

Quoted 12 lines
tools/testing/selftests/bpf/progs/test_access_variable_array.c +7 -4
-SEC("fentry/sched_balance_rq")
-int BPF_PROG(fentry_fentry, int this_cpu, struct rq *this_rq,
-               struct sched_domain *sd)
+SEC("fentry/sched_balance_find_dst_group_cpu")
+int BPF_PROG(fentry_fentry, struct sched_group *sg, struct task_struct *p,
+               int this_cpu)
  {
-       span = sd->span[0];
+       unsigned long *mask;
+       /* Read pointer to variable-length CPU mask */
+       mask = BPF_CORE_READ(sg, cpumask);
+       cpumask0 = mask[0];

No. That defeats the point of the test.
It tests [] array in the kernel struct.
Just make one in testmod.ko and access that.

Hi Alexei,

We do have cpumask as [] array in sched_group:

struct sched_group {
	struct sched_group	*next;			/* Must be a circular list */
	atomic_t		ref;

	unsigned int		group_weight;
	unsigned int		cores;
	struct sched_group_capacity *sgc;
	int			asym_prefer_cpu;	/* CPU of highest priority in group */
	int			flags;

	/*
	 * The CPUs this group covers.
	 *
	 * NOTE: this field is variable length. (Allocated dynamically
	 * by attaching extra space to the end of the structure,
	 * depending on how many CPUs the kernel has booted up with)
	 */
	unsigned long		cpumask[];
};

Wouldn't be good if it is used as:

cpumask0 = sg->cpumask[0];

Also, there is already a selftest that makes [] in testmod.ko and access that.

Thanks,
Saket

#3 lore raw
Quoted 13 lines · Saket Kumar Bhaskar

We do have cpumask as [] array in sched_group:

struct sched_group {
	struct sched_group	*next;			/* Must be a circular list */
	atomic_t		ref;

	unsigned int		group_weight;
	unsigned int		cores;
	struct sched_group_capacity *sgc;
	int			asym_prefer_cpu;	/* CPU of highest priority in group */
	int			flags;

	/*
Quoted 8 lines
	 * The CPUs this group covers.
	 *
	 * NOTE: this field is variable length. (Allocated dynamically
	 * by attaching extra space to the end of the structure,
	 * depending on how many CPUs the kernel has booted up with)
	 */
	unsigned long		cpumask[];
};

For now. But given how utterly broken flex arrays are, this thing will be gone the second we run into that same thing again.

Quoted 3 lines

Wouldn't be good if it is used as:

cpumask0 = sg->cpumask[0];

I'm thinking Alexei's point is that this test should not rely on random kernel code.

#4 lore raw
Quoted 27 lines · Peter Zijlstra
Quoted 13 lines · Saket Kumar Bhaskar

We do have cpumask as [] array in sched_group:

struct sched_group {
	struct sched_group	*next;			/* Must be a circular list */
	atomic_t		ref;

	unsigned int		group_weight;
	unsigned int		cores;
	struct sched_group_capacity *sgc;
	int			asym_prefer_cpu;	/* CPU of highest priority in group */
	int			flags;

	/*
Quoted 8 lines
	 * The CPUs this group covers.
	 *
	 * NOTE: this field is variable length. (Allocated dynamically
	 * by attaching extra space to the end of the structure,
	 * depending on how many CPUs the kernel has booted up with)
	 */
	unsigned long		cpumask[];
};

For now. But given how utterly broken flex arrays are, this thing will be gone the second we run into that same thing again.

Quoted 7 lines
Quoted 3 lines

Wouldn't be good if it is used as:

cpumask0 = sg->cpumask[0];

I'm thinking Alexei's point is that this test should not rely on random kernel code.

looks like we already have

struct bpf_testmod_struct_arg_3 {
        int a;
        int b[];
};

and function

int bpf_testmod_test_struct_arg_3(int a, int b, struct bpf_testmod_struct_arg_2 c);

...in selftests/bpf/test_kmods/bpf_testmod.c so we could potentially use that instead.

Alan