riscv: smp: set CPU 0 possible in setup_smp()
PATCH v2 1/1 riscv: smp: set CPU 0 possible in setup_smp()
setup_smp() calls set_cpu_possible() for CPUs 1..nr_cpu_ids-1 but never for CPU 0 (the boot CPU). x86 handles this via init_cpu_possible(cpumask_of(0)); RISC-V has no equivalent.
Without CPU 0 in cpu_possible_mask, rcu_init_one()'s for_each_possible_cpu() loop skips it, leaving rdp->mynode=NULL. rcutree_prepare_cpu() then dereferences NULL and hangs.
Exposed on Sophgo SG2042 (64-hart, 4-NUMA) with Linux 7.2-rc3.
--- arch/riscv/kernel/smpboot.c | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/riscv/kernel/smpboot.c +9 -0
@@ -171,6 +171,15 @@ void __init setup_smp(void)for (cpuid =1 ; cpuid < nr_cpu_ids; cpuid++)if (cpuid_to_hartid_map (cpuid) !=INVALID_HARTID )set_cpu_possible (cpuid, true); + +/* +* Boot CPU (cpuid=0) is never set possible by the loop above. +* x86 and arm64 call init_cpu_possible(cpumask_of(0)) explicitly; +* RISC-V was missing this. Without it, rcu_init_one()'s +* for_each_possible_cpu loop skips CPU 0, leaving rdp->mynode=NULL, +* causing rcutree_prepare_cpu() to dereference NULL and hang. +*/ +set_cpu_possible (0 , true); }static int start_secondary_cpu (int cpu,struct task_struct *tidle)
Signature · 7 lines
2.53.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
Paul Walmsley
maintainerQuoted 4 lines · Paul Sherman
setup_smp()callsset_cpu_possible()for CPUs 1..nr_cpu_ids-1 but never for CPU 0 (the boot CPU). x86 handles this via init_cpu_possible(cpumask_of(0)); RISC-V has no equivalent.
Quoted 8 lines
Without CPU 0 in
cpu_possible_mask, rcu_init_one()'sfor_each_possible_cpu()loop skips it, leaving rdp->mynode=NULL.rcutree_prepare_cpu()then dereferences NULL and hangs.Exposed on Sophgo SG2042 (64-hart, 4-NUMA) with Linux 7.2-rc3.
Fixes a4166aec1130 ("riscv: Deduplicate code in setup_smp()")Signed-off-by Paul Sherman <[email protected]>
Thanks, queued for v7.2-rc.
- Paul
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 10 lines · Paul Walmsley
Quoted 8 lines · Paul Sherman
setup_smp()callsset_cpu_possible()for CPUs 1..nr_cpu_ids-1 but never for CPU 0 (the boot CPU). x86 handles this via init_cpu_possible(cpumask_of(0)); RISC-V has no equivalent.Without CPU 0 in
cpu_possible_mask, rcu_init_one()'sfor_each_possible_cpu()loop skips it, leaving rdp->mynode=NULL.rcutree_prepare_cpu()then dereferences NULL and hangs.
Quoted 5 lines
Quoted 4 lines
Exposed on Sophgo SG2042 (64-hart, 4-NUMA) with Linux 7.2-rc3.
Fixes a4166aec1130 ("riscv: Deduplicate code in setup_smp()")Signed-off-by Paul Sherman <[email protected]>Thanks, queued for v7.2-rc.
I hope it's not too late to object, but I don't see how this could happen. This is what happens in start_kernel():
assembly boot code
-> start_kernel()
-> boot_cpu_init();
-> int cpu = smp_processor_id();
/* cpu = 0 because INIT_THREAD_INFO(tsk) leaves it out */
/* Mark the boot cpu "present", "online" etc for SMP and UP case */
set_cpu_online(cpu, true);
set_cpu_active(cpu, true);
set_cpu_present(cpu, true);
set_cpu_possible(cpu, true); /* <---- HERE */
-> ...
-> setup_arch(&command_line);
-> setup_smp()
-> for (cpuid = 1; cpuid < nr_cpu_ids; cpuid++)
if (cpuid_to_hartid_map(cpuid) != INVALID_HARTID)
set_cpu_possible(cpuid, true);So CPU 0 is already in cpu_possible_mask. This is also what I see by adding prints to setup_smp().
* Paul Sherman: How did you reproduce this? Is your boot CPU not cpu 0 (note: not the same as hart 0)?
Vivian "dramforever" Wang
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Vivian, Paul,
Thank you for the precise analysis. You are correct for mainline, boot_cpu_init() calls set_cpu_possible(0, true) before setup_smp().
The hang was observed on a development tree based on Linux 7.0-rc2 with substantial percpu/SLUB allocator changes layered on top for large NUMA geometry (4 nodes, 128GB). In that context I instrumented rcu_init_one() and rcutree_prepare_cpu() via SBI ecall probes (no JTAG/denugger available on this hardware) and observed:
Za ffffffff81035000 <- per_cpu_ptr(&rcu_data, 0) in rcu_init_one
Zn 0000000000000000 <- mynode was NULL at write time
rdp ffffffff81035000 <- same address at rcutree_prepare_cpu
mynode = NULL <- confirmed NULL dereference
The for_each_possible_cpu loop in rcu_init_one was skipping CPU 0 because cpu_possible_mask had not yet been populated at that point in my tree's initialization order. The additional allocator changes may have altered the timing.
I am rebasing onto 7.2-rc3. If the condition cannot be reproduced there, please feel free to drop the patch.
The probe methodology above uses M-mode SBI ecalls as a poor man's logic analyzer; happy to share the infrastructure if useful.
Paul Sherman
Quoted 40 lines · Vivian Wang
Quoted 10 lines · Paul Walmsley
Quoted 8 lines · Paul Sherman
setup_smp()callsset_cpu_possible()for CPUs 1..nr_cpu_ids-1 but never for CPU 0 (the boot CPU). x86 handles this via init_cpu_possible(cpumask_of(0)); RISC-V has no equivalent.Without CPU 0 in
cpu_possible_mask, rcu_init_one()'sfor_each_possible_cpu()loop skips it, leaving rdp->mynode=NULL.rcutree_prepare_cpu()then dereferences NULL and hangs.Quoted 5 lines
Quoted 4 lines
Exposed on Sophgo SG2042 (64-hart, 4-NUMA) with Linux 7.2-rc3.
Fixes a4166aec1130 ("riscv: Deduplicate code in setup_smp()")Signed-off-by Paul Sherman <[email protected]>Thanks, queued for v7.2-rc.
I hope it's not too late to object, but I don't see how this could happen. This is what happens in
start_kernel():assembly boot code -> start_kernel() -> boot_cpu_init(); -> int cpu = smp_processor_id(); /* cpu = 0 because INIT_THREAD_INFO(tsk) leaves it out */ /* Mark the boot cpu "present", "online" etc for SMP and UP case */ set_cpu_online(cpu, true); set_cpu_active(cpu, true); set_cpu_present(cpu, true); set_cpu_possible(cpu, true); /* <---- HERE */ -> ... -> setup_arch(&command_line); -> setup_smp() -> for (cpuid = 1; cpuid < nr_cpu_ids; cpuid++) if (cpuid_to_hartid_map(cpuid) != INVALID_HARTID) set_cpu_possible(cpuid, true);
Quoted 8 lines
So CPU 0 is already in
cpu_possible_mask. This is also what I see by adding prints tosetup_smp().* Paul Sherman: How did you reproduce this? Is your boot CPU not cpu 0 (note: not the same as hart 0)?
Vivian "dramforever" Wang
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
On 7/15/26 12:51, Paul Sherman wrote:
[ Your reply was here - I moved it to the bottom. Please do not top-post. See section "Use trimmed interleaved replies in email discussions" in Documentation/process/submitting-patches.rst for how email replies are expected to be formatted on LKML. ]
Quoted 45 lines · Vivian Wang
Quoted 44 lines · Paul Walmsley
Quoted 11 lines · Paul Sherman
Quoted 9 lines
setup_smp()callsset_cpu_possible()for CPUs 1..nr_cpu_ids-1 but never for CPU 0 (the boot CPU). x86 handles this via init_cpu_possible(cpumask_of(0)); RISC-V has no equivalent.Without CPU 0 in
cpu_possible_mask, rcu_init_one()'sfor_each_possible_cpu()loop skips it, leaving rdp->mynode=NULL.rcutree_prepare_cpu()then dereferences NULL and hangs.Quoted 5 lines
Quoted 4 lines
Exposed on Sophgo SG2042 (64-hart, 4-NUMA) with Linux 7.2-rc3.
Fixes a4166aec1130 ("riscv: Deduplicate code in setup_smp()")Signed-off-by Paul Sherman <[email protected]>Thanks, queued for v7.2-rc.
I hope it's not too late to object, but I don't see how this could happen. This is what happens in
start_kernel():assembly boot code -> start_kernel() -> boot_cpu_init(); -> int cpu = smp_processor_id(); /* cpu = 0 because INIT_THREAD_INFO(tsk) leaves it out */ /* Mark the boot cpu "present", "online" etc for SMP and UP case */ set_cpu_online(cpu, true); set_cpu_active(cpu, true); set_cpu_present(cpu, true); set_cpu_possible(cpu, true); /* <---- HERE */ -> ... -> setup_arch(&command_line); -> setup_smp() -> for (cpuid = 1; cpuid < nr_cpu_ids; cpuid++) if (cpuid_to_hartid_map(cpuid) != INVALID_HARTID) set_cpu_possible(cpuid, true);So CPU 0 is already in
cpu_possible_mask. This is also what I see by adding prints tosetup_smp().* Paul Sherman: How did you reproduce this? Is your boot CPU not cpu 0 (note: not the same as hart 0)?
Quoted 6 lines
Quoted 2 lines
Vivian "dramforever" Wang
Vivian, Paul,
Thank you for the precise analysis. You are correct for mainline,
boot_cpu_init()callsset_cpu_possible(0, true)beforesetup_smp().
Okay, if it's not a bug in mainline, it's not a bug in mainline.
Quoted 10 lines
The hang was observed on a development tree based on Linux 7.0-rc2 with substantial percpu/SLUB allocator changes layered on top for large NUMA geometry (4 nodes, 128GB).
[...]
The
for_each_possible_cpuloop inrcu_init_onewas skipping CPU 0 becausecpu_possible_maskhad not yet been populated at that point in my tree's initialization order. The additional allocator changes may have altered the timing.
I don't know what you're tree is doing, obviously, but I want to note that boot_cpu_init() comes *before* setup_smp() in start_kernel(), and indeed both come *extremely* early. So maybe all you need to do is, when moving stuff around, to also keep boot_cpu_init() before setup_smp(), which comes (way) before rcu_init()?
Quoted 2 lines
I am rebasing onto 7.2-rc3. If the condition cannot be reproduced there, please feel free to drop the patch.
Given that neither visual analysis (see above) nor runtime testing (we have SG2042 machines in ISCAS running 7.0.x and 7.1.x without running into this) can confirm that this bug exists, I'm concluding that, unless further convincing evidence otherwise comes up, this is a bug you introduced in your tree and does not fixing for mainline.
Quoted 2 lines
The probe methodology above uses M-mode SBI ecalls as a poor man's logic analyzer; happy to share the infrastructure if useful.
If you can reproduce this on 7.2-rc3 *without* your stuff on top, that would be useful. Otherwise, nobody else knows what the bug even is...
Vivian "dramforever" Wang
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Paul Walmsley
maintainerQuoted 5 lines · Vivian Wang
Given that neither visual analysis (see above) nor runtime testing (we have SG2042 machines in ISCAS running 7.0.x and 7.1.x without running into this) can confirm that this bug exists, I'm concluding that, unless further convincing evidence otherwise comes up, this is a bug you introduced in your tree and does not fixing for mainline.
Thanks for taking a closer look, Vivian; I'll drop this patch for now until either Paul or someone else can reproduce this on mainline.
- Paul
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv