riscv: A few SMP fixes
Subsystems (3)
cover riscv: A few SMP fixes
tl;dr - these patches are improvement to the riscv framework to better support the SBI HSM tests.
The first patch is a simpler alternative to [1] and [2] which doesn't require us to decide how best to make the number configurable. The second patch just adds sanity checking to make sure we can expect the SBI implementation to accept all hartids mapped from the present mask. The third patch was already posted once before[3] with a slightly different summary. It and the last patch improve smp_boot_secondary() since the SBI HSM tests were attempting to make workarounds for odd behaviors.
[1] https://lore.kernel.org/all/20240820170150.377580-2-andrew.jones at linux.dev/
[2] https://lore.kernel.org/all/20240903143946.834864-6-andrew.jones at linux.dev/
[3] https://lore.kernel.org/all/20240904120812.1798715-2-andrew.jones at linux.dev/
Andrew Jones (4):
riscv: Bump NR_CPUS to 256
riscv: Filter unmanaged harts from present mask
riscv: Fix secondary_entry
riscv: Rework smp_boot_secondary
lib/riscv/asm/processor.h | 1 + lib/riscv/asm/setup.h | 2 +- lib/riscv/setup.c | 11 ++++++--- lib/riscv/smp.c | 49 +++++++++++++++++++++++++++------------ riscv/cstart.S | 7 +++--- 5 files changed, 48 insertions(+), 22 deletions(-)
Signature · 1 line
2.47.0
PATCH 1/4 riscv: Bump NR_CPUS to 256
Besides a bit more memory used for the .bss section, where there are NR_CPUS sized arrays, and a tiny bit more stack used by functions with cpumasks on their stacks, then there's no harm in bumping NR_CPUS. Bump it to 256, which should cover us for quite a while.
--- lib/riscv/asm/setup.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/riscv/asm/setup.h +1 -1
@@ -4,7 +4,7 @@#include <libcflat.h> #include <asm/processor.h> -#define NR_CPUS 16 +#define NR_CPUS 256extern struct thread_info cpus[NR_CPUS ];extern int nr_cpus;extern uint64_t timebase_frequency;
Signature · 1 line
2.47.0
PATCH 2/4 riscv: Filter unmanaged harts from present mask
In reply to Andrew JonesWe use SBI to manage harts and SBI may have a different idea of which harts it should manage than our hardware description. Filter out all harts which fail an SBI HSM status call from the present mask to ensure we don't try to use them.
--- lib/riscv/setup.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
lib/riscv/setup.c +8 -3
@@ -19,6 +19,7 @@#include <asm/mmu.h> #include <asm/page.h> #include <asm/processor.h> +#include <asm/sbi.h> #include <asm/setup.h> #include <asm/timer.h>
@@ -51,7 +52,9 @@ static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused) cpus[cpu].cpu = cpu; cpus[cpu].hartid = regval; -set_cpu_present (cpu, true); + +if (!sbi_hart_get_status (cpus[cpu].hartid).error) +set_cpu_present (cpu, true); }static void cpu_init_acpi (void )
@@ -61,7 +64,7 @@ static void cpu_init_acpi(void)static void cpu_init (void ) { -int ret; +int ret, me; nr_cpus =0 ;if (dt_available ()) {
@@ -71,7 +74,9 @@ static void cpu_init(void)cpu_init_acpi (); } -set_cpu_online (hartid_to_cpu (csr_read (CSR_SSCRATCH )), true); + me =hartid_to_cpu (csr_read (CSR_SSCRATCH )); +assert (cpu_present (me)); +set_cpu_online (me, true); cpu0_calls_idle = true; }
Signature · 1 line
2.47.0
PATCH 3/4 riscv: Fix secondary_entry
In reply to Andrew JonesThe last few instructions of secondary_entry had the right concept, but were the totally wrong implementation. Without setting ra, then, when the boot function doesn't stay in an infinite loop, like do_idle() would, we'd go off into the weeds when trying to return from it. Make sure we set ra to come back to where we can then call do_idle() instead. The bug was found by inspection since nobody is calling smp_boot_secondary() with anything other than do_idle() at this time.
--- riscv/cstart.S | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
riscv/cstart.S +3 -3
@@ -154,9 +154,9 @@ secondary_entry: mv a0, sp call secondary_cinit addi sp, sp, SECONDARY_DATA_SIZE - jr a0 - la a0, do_idle - jr a0 + jalr ra, a0 + call do_idle + j . /* unreachable */ /* * Save context to address in a0.
Signature · 1 line
2.47.0
PATCH 4/4 riscv: Rework smp_boot_secondary
In reply to Andrew JonesUse HSM status to determine when a secondary should be started. Also save the stack pointer so a secondary may be stopped and started again without leaking old stacks.
--- lib/riscv/asm/processor.h | 1 + lib/riscv/smp.c | 49 +++++++++++++++++++++++++++------------ riscv/cstart.S | 1 + 3 files changed, 36 insertions(+), 15 deletions(-)
lib/riscv/asm/processor.h +1 -0
@@ -13,6 +13,7 @@ struct thread_info {int cpu;unsigned long hartid;unsigned long isa[1 ]; +unsigned long sp;exception_fn exception_handlers[EXCEPTION_CAUSE_MAX ];exception_fn interrupt_handlers[INTERRUPT_CAUSE_MAX ]; };
lib/riscv/smp.c +34 -15
79 lines not on this page · raw message
riscv/cstart.S +1 -0
@@ -149,6 +149,7 @@ secondary_entry: csrw CSR_SSCRATCH, a0 mv sp, a1 mv fp, zero + addi sp, sp, -SECONDARY_DATA_SIZE REG_L a0, SECONDARY_STVEC(sp) csrw CSR_STVEC, a0 mv a0, sp
Signature · 1 line
2.47.0
Andrew Jones
AppliedQuoted 22 lines · Andrew Jones
tl;dr - these patches are improvement to the riscv framework to better support the SBI HSM tests.
The first patch is a simpler alternative to [1] and [2] which doesn't require us to decide how best to make the number configurable. The second patch just adds sanity checking to make sure we can expect the SBI implementation to accept all hartids mapped from the present mask. The third patch was already posted once before[3] with a slightly different summary. It and the last patch improve
smp_boot_secondary()since the SBI HSM tests were attempting to make workarounds for odd behaviors.[1] https://lore.kernel.org/all/20240820170150.377580-2-andrew.jones at linux.dev/
[2] https://lore.kernel.org/all/20240903143946.834864-6-andrew.jones at linux.dev/
[3] https://lore.kernel.org/all/20240904120812.1798715-2-andrew.jones at linux.dev/Andrew Jones (4):
riscv: BumpNR_CPUSto 256
riscv: Filter unmanaged harts from present mask
riscv: Fixsecondary_entry
riscv: Reworksmp_boot_secondary
Quoted 9 lines
lib/riscv/asm/processor.h | 1 + lib/riscv/asm/setup.h | 2 +- lib/riscv/setup.c | 11 ++++++--- lib/riscv/smp.c | 49 +++++++++++++++++++++++++++------------ riscv/cstart.S | 7 +++--- 5 files changed, 48 insertions(+), 22 deletions(-)Signature · 1 line
2.47.0
Merged to master through riscv/queue.
Thanks,
drew