Skip to contentSkip to conversation

sched_ext: idle: Skip cross-node search with !CONFIG_NUMA

PATCH Applied Andrea Righi 3 participants 4 messages Jun 3, 2025 – Jun 4, 2025 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched_ext: idle: Skip cross-node search with !CONFIG_NUMA

In the idle CPU selection logic, attempting cross-node searches adds unnecessary complexity when CONFIG_NUMA is disabled.

Since there's no meaningful concept of nodes in this case, simplify the logic by restricting the idle CPU search to the current node only.

Fixes 48849271e6611 ("sched_ext: idle: Per-node idle cpumasks")
---
 kernel/sched/ext_idle.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/sched/ext_idle.c +8 -0
index 66da03cc0b338..8660d9ae40169 100644
@@ -138,6 +138,7 @@ static s32 pick_idle_cpu_in_node(const struct cpumask *cpus_allowed, int node, u
 		goto retry;
 }
 
+#ifdef CONFIG_NUMA
 /*
  * Tracks nodes that have not yet been visited when searching for an idle
  * CPU across all available nodes.
@@ -186,6 +187,13 @@ static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i
 
 	return cpu;
 }
+#else
+static inline s32
+pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, int node, u64 flags)
+{
+	return -EBUSY;
+}
+#endif
 
 /*
  * Find an idle CPU in the system, starting from @node.
Signature · 1 line
2.49.0
Applied
#1 lore raw
Quoted 8 lines · Andrea Righi

In the idle CPU selection logic, attempting cross-node searches adds unnecessary complexity when CONFIG_NUMA is disabled.

Since there's no meaningful concept of nodes in this case, simplify the logic by restricting the idle CPU search to the current node only.

Fixes 48849271e6611 ("sched_ext: idle: Per-node idle cpumasks")

Applied to sched_ext/for-6.16-fixes.

Thanks.

Signature · 1 line
tejun
#2 lore raw
In reply to Andrea Righi

Hi Andrea!

Quoted 17 lines · Andrea Righi

In the idle CPU selection logic, attempting cross-node searches adds unnecessary complexity when CONFIG_NUMA is disabled.

Since there's no meaningful concept of nodes in this case, simplify the logic by restricting the idle CPU search to the current node only.

Fixes 48849271e6611 ("sched_ext: idle: Per-node idle cpumasks")
---
 kernel/sched/ext_idle.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/sched/ext_idle.c +0 -0
index 66da03cc0b338..8660d9ae40169 100644

kernel/sched/ext_idle.c static s32 pick_idle_cpu_in_node(const struct cpumask *cpus_allowed, int node, u · line 141

Quoted 4 lines
kernel/sched/ext_idle.c +1 -0
@@ -138,6 +138,7 @@ static s32 pick_idle_cpu_in_node(const struct cpumask *cpus_allowed, int node, u
 		goto retry;
 }
 
+#ifdef CONFIG_NUMA

It would be more natural if you move this inside the function body, and not duplicate the function declaration.

kernel/sched/ext_idle.c static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i · line 195

Quoted 13 lines
kernel/sched/ext_idle.c +6 -0
 /*
  * Tracks nodes that have not yet been visited when searching for an idle
  * CPU across all available nodes.
@@ -186,6 +187,13 @@ static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i
 
 	return cpu;
 }
+#else
+static inline s32
+pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, int node, u64 flags)
+{
+	return -EBUSY;
+}

This is misleading errno. The system is nut busy, it is disabled. If it was a syscall, I would say you should return ENOSYS. ENODATA is another candidate. Or you have a special policy for the subsystem/

The above pick_idle_cpu_in_node() doesn't have CONFIG_NUMA protection as well. Is it safe against CONFIG_NUMA?

Quoted 6 lines
kernel/sched/ext_idle.c +1 -0
+#endif
 
 /*
  * Find an idle CPU in the system, starting from @node.
Signature · 1 line
2.49.0
#3 lore raw

Hi Yuri,

Quoted 23 lines · Yury Norov

Hi Andrea!

Quoted 20 lines · Andrea Righi

In the idle CPU selection logic, attempting cross-node searches adds unnecessary complexity when CONFIG_NUMA is disabled.

Since there's no meaningful concept of nodes in this case, simplify the logic by restricting the idle CPU search to the current node only.

Fixes 48849271e6611 ("sched_ext: idle: Per-node idle cpumasks")
---
 kernel/sched/ext_idle.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/sched/ext_idle.c +0 -0
index 66da03cc0b338..8660d9ae40169 100644
@@ -138,6 +138,7 @@ static s32 pick_idle_cpu_in_node(const struct cpumask *cpus_allowed, int node, u
 		goto retry;
 }
 
Quoted 4 lines

kernel/sched/ext_idle.c static s32 pick_idle_cpu_in_node(const struct cpumask *cpus_allowed, int node, u · line 141

Quoted 1 line
kernel/sched/ext_idle.c +1 -0
+#ifdef CONFIG_NUMA

It would be more natural if you move this inside the function body, and not duplicate the function declaration.

I was trying to catch both the function and the per_cpu_unvisited with a single #ifdef, but I can definitely split that and add another #ifdef inside the function body.

Quoted 6 lines
Quoted 6 lines
kernel/sched/ext_idle.c +0 -0
 /*
  * Tracks nodes that have not yet been visited when searching for an idle
  * CPU across all available nodes.
@@ -186,6 +187,13 @@ static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i
 
Quoted 12 lines

kernel/sched/ext_idle.c static s32 pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, i · line 195

Quoted 8 lines
kernel/sched/ext_idle.c +6 -0
 	return cpu;
 }
+#else
+static inline s32
+pick_idle_cpu_from_online_nodes(const struct cpumask *cpus_allowed, int node, u64 flags)
+{
+	return -EBUSY;
+}

This is misleading errno. The system is nut busy, it is disabled. If it was a syscall, I would say you should return ENOSYS. ENODATA is another candidate. Or you have a special policy for the subsystem/

So, this function is called only from scx_pick_idle_cpu(), that can still call pick_idle_cpu_from_online_nodes() even on kernels with !CONFIG_NUMA, if the BPF scheduler enables the per-node idle cpumask (setting the flag SCX_OPS_BUILTIN_IDLE_PER_NODE).

We can return -ENOSYS, but then we still need to return -EBUSY from scx_pick_idle_cpu(), since its logic is host-wide, so the choice of -EBUSY was to be consistent with that.

However, I don't have a strong opinion, if you think it's clearer to return -ENOSYS/ENODATA from pick_idle_cpu_from_online_nodes() I can change that, but I'd still return -EBUSY from scx_pick_idle_cpu().

Quoted 3 lines

The above pick_idle_cpu_in_node() doesn't have CONFIG_NUMA protection as well. Is it safe against CONFIG_NUMA?

pick_idle_cpu_in_node() is always called with a validated node (when passed from BPF) or a node from the kernel and idle_cpumask() is handling the NUMA_NO_NODE case, so that should be fine in theory.

Thanks,
-Andrea

PS Tejun already applied this patch to his tree, so I'll send all the changes as a followup patch, at least the original bug is fixed. :)