Skip to contentSkip to conversation

RISC-V: Increase range and default value of NR_CPUS

PATCH Superseded Anup Patel 3 participants 4 messages Mar 17, 2022 – Mar 19, 2022 RISC-V

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Anup Patel

author
#0 lore raw

PATCH 1/1 RISC-V: Increase range and default value of NR_CPUS

Currently, the range and default value of NR_CPUS is too restrictive for high-end RISC-V systems with large number of HARTs. The latest QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is restrictive for QEMU as well. Other major architectures (such as ARM64, x86_64, MIPS, etc) have a much higher range and default value of NR_CPUS.

This patch increases NR_CPUS range to 2-256 and default value to XLEN (i.e. 32 for RV32 and 64 for RV64).

---
 arch/riscv/Kconfig | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
arch/riscv/Kconfig +4 -3
index 5adcbd9b5e88..54f125eb10f6 100644
@@ -274,10 +274,11 @@ config SMP
 	  If you don't know what to do here, say N.
 
 config NR_CPUS
-	int "Maximum number of CPUs (2-32)"
-	range 2 32
+	int "Maximum number of CPUs (2-256)"
+	range 2 256
 	depends on SMP
-	default "8"
+	default "32" if 32BIT
+	default "64" if 64BIT
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"
Signature · 7 lines
2.25.1


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#1 lore raw
Quoted 6 lines · Anup Patel

Currently, the range and default value of NR_CPUS is too restrictive for high-end RISC-V systems with large number of HARTs. The latest QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is restrictive for QEMU as well. Other major architectures (such as ARM64, x86_64, MIPS, etc) have a much higher range and default value of NR_CPUS.

What's the memory overhead for increasing this?

Signature · 9 lines
Ben Dooks				http://www.codethink.co.uk/
Senior Engineer				Codethink - Providing Genius

https://www.codethink.co.uk/privacy.html

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#2 lore raw
Quoted 6 lines · Ben Dooks
Quoted 4 lines · Anup Patel

Currently, the range and default value of NR_CPUS is too restrictive for high-end RISC-V systems with large number of HARTs. The latest QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is restrictive for QEMU as well.

If qemu allows 512, what is the reason for limiting the kernel to 256?

Quoted 5 lines
Quoted 3 lines

Other major architectures (such as
ARM64, x86_64, MIPS, etc) have a much higher range and default value of NR_CPUS.

What's the memory overhead for increasing this?

It's supposed to be very small, I would expect three main sources of overhead:

- cpumask_t variables, those grow once you go beyond the size of
an unsigned long (32 or 64 bits), so with the default just on the limit, this makes no difference.
Note that you can run out of stack space with NR_CPUS values if CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
a problem for 512 or below.

- percpu variables: these are dynamically allocated based on the number of CPUs at boot time, so they should not have any real impact.

- NR_CPUS sized arrays, these are sometimes used in place of percpu data. This is only a problem if the array members individually are more than a few bytes. There are not too many of these in the kernel, as using those is discouraged.

Arnd

author
#3 lore raw
Quoted 10 lines · Arnd Bergmann
Quoted 6 lines · Ben Dooks
Quoted 4 lines · Anup Patel

Currently, the range and default value of NR_CPUS is too restrictive for high-end RISC-V systems with large number of HARTs. The latest QEMU virt machine supports upto 512 CPUs so the current NR_CPUS is restrictive for QEMU as well.

If qemu allows 512, what is the reason for limiting the kernel to 256?

There is no particular reason. I will change this to 2-512 range to match the maximum number of CPUs supported by the QEMU virt machine.

Quoted 7 lines
Quoted 6 lines
Quoted 4 lines

Other major architectures (such as
ARM64, x86_64, MIPS, etc) have a much higher range and default value of NR_CPUS.

What's the memory overhead for increasing this?

Quoted 8 lines

It's supposed to be very small, I would expect three main sources of overhead:

- cpumask_t variables, those grow once you go beyond the size of
an unsigned long (32 or 64 bits), so with the default just on the limit, this makes no difference.
Note that you can run out of stack space with NR_CPUS values if CONFIG_CPUMASK_OFFSTACK is disabled. Should not be
a problem for 512 or below.

Yes, the rationale is to have default NR_CPUS such that size of cpumask_t does not change for RV32 and RV64.

Quoted 8 lines

- percpu variables: these are dynamically allocated based on the number of CPUs at boot time, so they should not have any real impact.

- NR_CPUS sized arrays, these are sometimes used in place of percpu data. This is only a problem if the array members individually are more than a few bytes. There are not too many of these in the kernel, as using those is discouraged.

Separately, we are also trying to replace NR_CPUS sized arrays.

Regards,
Anup