sched/topology: Fix for_each_node_numadist() lockup with !CONFIG_NUMA
PATCH v2 1/1 sched/topology: Fix for_each_node_numadist() lockup with !CONFIG_NUMA
for_each_node_numadist() can lead to hard lockups on kernels built without CONFIG_NUMA. For instance, the following was triggered by sched_ext:
watchdog: CPU5: Watchdog detected hard LOCKUP on cpu 5
...
RIP: 0010:_find_first_and_bit+0x8/0x60 ... Call Trace: <TASK> cpumask_any_and_distribute+0x49/0x80 pick_idle_cpu_in_node+0xcf/0x140 scx_bpf_pick_idle_cpu_node+0xaa/0x110 bpf_prog_16ee5b1f077af006_pick_idle_cpu+0x57f/0x5de bpf_prog_df2ce5cfac58ce09_bpfland_select_cpu+0x37/0xf4 bpf__sched_ext_ops_select_cpu+0x4b/0xb3
This happens because nearest_node_nodemask() always returns NUMA_NO_NODE (-1) when CONFIG_NUMA is disabled, causing the loop to never terminate, as the condition node >= MAX_NUMNODES is never satisfied.
Prevent this by providing a stub implementation based on for_each_node_mask() when CONFIG_NUMA is disabled, which can safely processes the single available node while still honoring the unvisited nodemask.
--- include/linux/topology.h | 5 +++++ 1 file changed, 5 insertions(+)
Changes in v2:
- Provide a stub implementation for the !CONFIG_NUMA case
- Link to v1: https://lore.kernel.org/all/[email protected]/
include/linux/topology.h +5 -0
@@ -304,12 +304,17 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops)* * Requires rcu_lock to be held. */ +#ifdef CONFIG_NUMA #define for_each_node_numadist (node, unvisited) \ for (int __start = (node), \ (node) = nearest_node_nodemask((__start), &(unvisited)); \ (node) < MAX_NUMNODES; \ node_clear((node), (unvisited)), \ (node) = nearest_node_nodemask((__start), &(unvisited))) +#else +#define for_each_node_numadist (node, unvisited) \ + for_each_node_mask((node), (unvisited)) +#endif /** * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance
Signature · 1 line
2.49.0
Quoted 41 lines · Andrea Righi
for_each_node_numadist()can lead to hard lockups on kernels built withoutCONFIG_NUMA. For instance, the following was triggered bysched_ext:watchdog: CPU5: Watchdog detected hard LOCKUP on cpu 5
...RIP: 0010:_find_first_and_bit+0x8/0x60 ... Call Trace: <TASK> cpumask_any_and_distribute+0x49/0x80 pick_idle_cpu_in_node+0xcf/0x140 scx_bpf_pick_idle_cpu_node+0xaa/0x110 bpf_prog_16ee5b1f077af006_pick_idle_cpu+0x57f/0x5de bpf_prog_df2ce5cfac58ce09_bpfland_select_cpu+0x37/0xf4 bpf__sched_ext_ops_select_cpu+0x4b/0xb3This happens because
nearest_node_nodemask()always returnsNUMA_NO_NODE(-1) whenCONFIG_NUMAis disabled, causing the loop to never terminate, as the condition node >=MAX_NUMNODESis never satisfied.Prevent this by providing a stub implementation based on
for_each_node_mask()whenCONFIG_NUMAis disabled, which can safely processes the single available node while still honoring the unvisited nodemask.Fixes f09177ca5f242 ("sched/topology: Introduce for_each_node_numadist() iterator")Signed-off-by Andrea Righi <[email protected]>--- include/linux/topology.h | 5 +++++ 1 file changed, 5 insertions(+)Changes in v2:
- Provide a stub implementation for the !CONFIG_NUMA case
- Link to v1: https://lore.kernel.org/all/[email protected]/include/linux/topology.h +0
-0
include/linux/topology.h sched_numa_hop_mask(unsigned int node, unsigned int hops) · line 307
Quoted 4 lines
include/linux/topology.h +1
-0@@ -304,12 +304,17 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops)* * Requires rcu_lock to be held. */ +#ifdef CONFIG_NUMA
While there, can you expand this optimization to MAX_NUMNODES == 1 case?
#if defined(CONFIG_NUMA) && (MAX_NUMNODES > 1)
With that:
Thanks,
Yury
Quoted 15 lines
include/linux/topology.h +4
-0#define for_each_node_numadist (node, unvisited) \ for (int __start = (node), \ (node) = nearest_node_nodemask((__start), &(unvisited)); \ (node) < MAX_NUMNODES; \ node_clear((node), (unvisited)), \ (node) = nearest_node_nodemask((__start), &(unvisited))) +#else +#define for_each_node_numadist (node, unvisited) \ + for_each_node_mask((node), (unvisited)) +#endif /** * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance Signature · 1 line
2.49.0
Hi Yury,
Quoted 49 lines · Yury Norov
Quoted 41 lines · Andrea Righi
for_each_node_numadist()can lead to hard lockups on kernels built withoutCONFIG_NUMA. For instance, the following was triggered bysched_ext:watchdog: CPU5: Watchdog detected hard LOCKUP on cpu 5
...RIP: 0010:_find_first_and_bit+0x8/0x60 ... Call Trace: <TASK> cpumask_any_and_distribute+0x49/0x80 pick_idle_cpu_in_node+0xcf/0x140 scx_bpf_pick_idle_cpu_node+0xaa/0x110 bpf_prog_16ee5b1f077af006_pick_idle_cpu+0x57f/0x5de bpf_prog_df2ce5cfac58ce09_bpfland_select_cpu+0x37/0xf4 bpf__sched_ext_ops_select_cpu+0x4b/0xb3This happens because
nearest_node_nodemask()always returnsNUMA_NO_NODE(-1) whenCONFIG_NUMAis disabled, causing the loop to never terminate, as the condition node >=MAX_NUMNODESis never satisfied.Prevent this by providing a stub implementation based on
for_each_node_mask()whenCONFIG_NUMAis disabled, which can safely processes the single available node while still honoring the unvisited nodemask.Fixes f09177ca5f242 ("sched/topology: Introduce for_each_node_numadist() iterator")Signed-off-by Andrea Righi <[email protected]>--- include/linux/topology.h | 5 +++++ 1 file changed, 5 insertions(+)Changes in v2:
- Provide a stub implementation for the !CONFIG_NUMA case
- Link to v1: https://lore.kernel.org/all/[email protected]/include/linux/topology.h +0
-0include/linux/topology.h sched_numa_hop_mask(unsigned int node, unsigned int hops) · line 307
Quoted 4 lines
include/linux/topology.h +1
-0@@ -304,12 +304,17 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops)* * Requires rcu_lock to be held. */ +#ifdef CONFIG_NUMA While there, can you expand this optimization to
MAX_NUMNODES== 1 case?
Quoted 1 line
#if defined(CONFIG_NUMA) && (MAX_NUMNODES > 1)
Makes sense, will send a v3, thanks!
-Andrea
Quoted 23 lines
With that:
Thanks,
YuryQuoted 15 lines
include/linux/topology.h +4
-0#define for_each_node_numadist (node, unvisited) \ for (int __start = (node), \ (node) = nearest_node_nodemask((__start), &(unvisited)); \ (node) < MAX_NUMNODES; \ node_clear((node), (unvisited)), \ (node) = nearest_node_nodemask((__start), &(unvisited))) +#else +#define for_each_node_numadist (node, unvisited) \ + for_each_node_mask((node), (unvisited)) +#endif /** * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance Signature · 1 line
2.49.0