Skip to contentSkip to conversation

riscv: A few SMP fixes

PATCH Applied Andrew Jones 1 participant 6 messages Oct 23, 2024 – Nov 11, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover riscv: A few SMP fixes

tl;dr - these patches are improvement to the riscv framework to better support the SBI HSM tests.

The first patch is a simpler alternative to [1] and [2] which doesn't require us to decide how best to make the number configurable. The second patch just adds sanity checking to make sure we can expect the SBI implementation to accept all hartids mapped from the present mask. The third patch was already posted once before[3] with a slightly different summary. It and the last patch improve smp_boot_secondary() since the SBI HSM tests were attempting to make workarounds for odd behaviors.

[1] https://lore.kernel.org/all/20240820170150.377580-2-andrew.jones at linux.dev/
[2] https://lore.kernel.org/all/20240903143946.834864-6-andrew.jones at linux.dev/
[3] https://lore.kernel.org/all/20240904120812.1798715-2-andrew.jones at linux.dev/

Andrew Jones (4):
riscv: Bump NR_CPUS to 256
riscv: Filter unmanaged harts from present mask
riscv: Fix secondary_entry
riscv: Rework smp_boot_secondary

 lib/riscv/asm/processor.h |  1 +
 lib/riscv/asm/setup.h     |  2 +-
 lib/riscv/setup.c         | 11 ++++++---
 lib/riscv/smp.c           | 49 +++++++++++++++++++++++++++------------
 riscv/cstart.S            |  7 +++---
 5 files changed, 48 insertions(+), 22 deletions(-)
Signature · 1 line
2.47.0
#1 lore raw

PATCH 1/4 riscv: Bump NR_CPUS to 256

Besides a bit more memory used for the .bss section, where there are NR_CPUS sized arrays, and a tiny bit more stack used by functions with cpumasks on their stacks, then there's no harm in bumping NR_CPUS. Bump it to 256, which should cover us for quite a while.

---
 lib/riscv/asm/setup.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/riscv/asm/setup.h +1 -1
index a13159bfe395..a031ebe7d762 100644
@@ -4,7 +4,7 @@
 #include <libcflat.h>
 #include <asm/processor.h>
 
-#define NR_CPUS 16
+#define NR_CPUS 256
 extern struct thread_info cpus[NR_CPUS];
 extern int nr_cpus;
 extern uint64_t timebase_frequency;
Signature · 1 line
2.47.0
#2 lore raw

PATCH 2/4 riscv: Filter unmanaged harts from present mask

In reply to Andrew Jones

We use SBI to manage harts and SBI may have a different idea of which harts it should manage than our hardware description. Filter out all harts which fail an SBI HSM status call from the present mask to ensure we don't try to use them.

---
 lib/riscv/setup.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
lib/riscv/setup.c +8 -3
index f347ad6352d7..211945448b0f 100644
@@ -19,6 +19,7 @@
 #include <asm/mmu.h>
 #include <asm/page.h>
 #include <asm/processor.h>
+#include <asm/sbi.h>
 #include <asm/setup.h>
 #include <asm/timer.h>
 
@@ -51,7 +52,9 @@ static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
 
 	cpus[cpu].cpu = cpu;
 	cpus[cpu].hartid = regval;
-	set_cpu_present(cpu, true);
+
+	if (!sbi_hart_get_status(cpus[cpu].hartid).error)
+		set_cpu_present(cpu, true);
 }
 
 static void cpu_init_acpi(void)
@@ -61,7 +64,7 @@ static void cpu_init_acpi(void)
 
 static void cpu_init(void)
 {
-	int ret;
+	int ret, me;
 
 	nr_cpus = 0;
 	if (dt_available()) {
@@ -71,7 +74,9 @@ static void cpu_init(void)
 		cpu_init_acpi();
 	}
 
-	set_cpu_online(hartid_to_cpu(csr_read(CSR_SSCRATCH)), true);
+	me = hartid_to_cpu(csr_read(CSR_SSCRATCH));
+	assert(cpu_present(me));
+	set_cpu_online(me, true);
 	cpu0_calls_idle = true;
 }
 
Signature · 1 line
2.47.0
#3 lore raw

PATCH 3/4 riscv: Fix secondary_entry

In reply to Andrew Jones

The last few instructions of secondary_entry had the right concept, but were the totally wrong implementation. Without setting ra, then, when the boot function doesn't stay in an infinite loop, like do_idle() would, we'd go off into the weeds when trying to return from it. Make sure we set ra to come back to where we can then call do_idle() instead. The bug was found by inspection since nobody is calling smp_boot_secondary() with anything other than do_idle() at this time.

Fixes 9c92b28e6b7b ("riscv: Add SMP support")
---
 riscv/cstart.S | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
riscv/cstart.S +3 -3
index bae1d2f5b4d5..687173706d83 100644
@@ -154,9 +154,9 @@ secondary_entry:
 	mv	a0, sp
 	call	secondary_cinit
 	addi	sp, sp, SECONDARY_DATA_SIZE
-	jr	a0
-	la	a0, do_idle
-	jr	a0
+	jalr	ra, a0
+	call	do_idle
+	j	.	/* unreachable */
 
 /*
  * Save context to address in a0.
Signature · 1 line
2.47.0
#4 lore raw

PATCH 4/4 riscv: Rework smp_boot_secondary

In reply to Andrew Jones

Use HSM status to determine when a secondary should be started. Also save the stack pointer so a secondary may be stopped and started again without leaking old stacks.

---
 lib/riscv/asm/processor.h |  1 +
 lib/riscv/smp.c           | 49 +++++++++++++++++++++++++++------------
 riscv/cstart.S            |  1 +
 3 files changed, 36 insertions(+), 15 deletions(-)
lib/riscv/asm/processor.h +1 -0
index 4063255a3475..401042724cee 100644
@@ -13,6 +13,7 @@ struct thread_info {
 	int cpu;
 	unsigned long hartid;
 	unsigned long isa[1];
+	unsigned long sp;
 	exception_fn exception_handlers[EXCEPTION_CAUSE_MAX];
 	exception_fn interrupt_handlers[INTERRUPT_CAUSE_MAX];
 };
lib/riscv/smp.c +34 -15

79 lines not on this page · raw message

riscv/cstart.S +1 -0
index 687173706d83..b7ee9b9c96b3 100644
@@ -149,6 +149,7 @@ secondary_entry:
 	csrw	CSR_SSCRATCH, a0
 	mv	sp, a1
 	mv	fp, zero
+	addi	sp, sp, -SECONDARY_DATA_SIZE
 	REG_L	a0, SECONDARY_STVEC(sp)
 	csrw	CSR_STVEC, a0
 	mv	a0, sp
Signature · 1 line
2.47.0
author Applied
#5 lore raw
In reply to Andrew Jones
Quoted 22 lines · Andrew Jones

tl;dr - these patches are improvement to the riscv framework to better support the SBI HSM tests.

The first patch is a simpler alternative to [1] and [2] which doesn't require us to decide how best to make the number configurable. The second patch just adds sanity checking to make sure we can expect the SBI implementation to accept all hartids mapped from the present mask. The third patch was already posted once before[3] with a slightly different summary. It and the last patch improve smp_boot_secondary() since the SBI HSM tests were attempting to make workarounds for odd behaviors.

[1] https://lore.kernel.org/all/20240820170150.377580-2-andrew.jones at linux.dev/
[2] https://lore.kernel.org/all/20240903143946.834864-6-andrew.jones at linux.dev/
[3] https://lore.kernel.org/all/20240904120812.1798715-2-andrew.jones at linux.dev/

Andrew Jones (4):
riscv: Bump NR_CPUS to 256
riscv: Filter unmanaged harts from present mask
riscv: Fix secondary_entry
riscv: Rework smp_boot_secondary

Quoted 9 lines
 lib/riscv/asm/processor.h |  1 +
 lib/riscv/asm/setup.h     |  2 +-
 lib/riscv/setup.c         | 11 ++++++---
 lib/riscv/smp.c           | 49 +++++++++++++++++++++++++++------------
 riscv/cstart.S            |  7 +++---
 5 files changed, 48 insertions(+), 22 deletions(-)
Signature · 1 line
2.47.0

Merged to master through riscv/queue.

Thanks,
drew