Skip to contentSkip to conversation

ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512

PATCH Superseded Christoph Lameter (Ampere) 5 participants 13 messages Dec 15, 2023 – Jan 23, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512

Ampere Computing develops high end ARM processor that support an ever increasing number of processors. The default 256 processors are not enough for our newer products. The default is used by distros and therefore our customers cannot use distro kernels because the number of processors is not supported.

One of the objections against earlier patches to increase the limit was that the memory use becomes too high. There is a feature called CPUMASK_OFFSTACK that configures the cpumasks in the kernel to be dynamically allocated. This was used in the X86 architecture in the past to enable support for larger CPU configurations up to 8k cpus.

With that is becomes possible to dynamically size the allocation of the cpu bitmaps depending on the quantity of processors detected on bootup.

This patch enables that logic if more than 256 processors are configured and increases the default to 512 processors.

Further increases may be needed if ARM processor vendors start supporting more processors. Given the current inflationary trends in core counts from multiple processor manufacturers this may occur.

---

This was discussed earlier and this updated patch was proposed at https://lore.kernel.org/linux-arm-kernel/[email protected]/T/

arch/arm64/Kconfig +15 -1
Index: linux/arch/arm64/Kconfig
===================================================================
@@ -1407,7 +1407,21 @@ config SCHED_SMT
  config NR_CPUS
  	int "Maximum number of CPUs (2-4096)"
  	range 2 4096
-	default "256"
+	default 512
+
+#
+# Determines the placement of cpumasks.
+#
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256
 
  config HOTPLUG_CPU
  	bool "Support for hot-pluggable CPUs"
#1 lore raw

arch/arm64/Kconfig config SCHED_SMT · line 1409

Quoted 8 lines · Christoph Lameter (Ampere)

Index: linux/arch/arm64/Kconfig
===================================================================

arch/arm64/Kconfig +0 -0
@@ -1407,7 +1407,21 @@ config SCHED_SMT
  config NR_CPUS
  	int "Maximum number of CPUs (2-4096)"
  	range 2 4096

I think your mailer got to your patch and messed up the white space. There are two spaces before each of these lines rather than the usual one.

Quoted 6 lines
arch/arm64/Kconfig +5 -1
-	default "256"
+	default 512
+
+#
+# Determines the placement of cpumasks.
+#

arch/arm64/Kconfig config SCHED_SMT · line 1424

Quoted 10 lines
arch/arm64/Kconfig +10 -0
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256

Should that be ">= 256" ?

arch/arm64/Kconfig config SCHED_SMT · line 1427

Quoted 3 lines
arch/arm64/Kconfig +0 -0
  config HOTPLUG_CPU
  	bool "Support for hot-pluggable CPUs"

Same here.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
#2 lore raw
Quoted 24 lines · Russell King (Oracle)

arch/arm64/Kconfig config SCHED_SMT · line 1409

Quoted 8 lines · Christoph Lameter (Ampere)

Index: linux/arch/arm64/Kconfig
===================================================================

arch/arm64/Kconfig +0 -0
@@ -1407,7 +1407,21 @@ config SCHED_SMT
   config NR_CPUS
   	int "Maximum number of CPUs (2-4096)"
   	range 2 4096

I think your mailer got to your patch and messed up the white space. There are two spaces before each of these lines rather than the usual one.

Quoted 10 lines
arch/arm64/Kconfig +9 -1
-	default "256"
+	default 512
+
+#
+# Determines the placement of cpumasks.
+#
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
Quoted 8 lines

arch/arm64/Kconfig config SCHED_SMT · line 1424

Quoted 6 lines
arch/arm64/Kconfig +6 -0
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256

Should that be ">= 256" ?

Maybe just select CPUMASK_OFFSTACK if NR_CPUS >= 256,

But could we just make CPUMASK_OFFSTACK configurable and let user/distro to enable it?

lib/Kconfig +3 -1
index 5ddda7c2ed9b..4254be5aa843 100644
@@ -535,7 +535,9 @@ config CHECK_SIGNATURE
        bool
 
 config CPUMASK_OFFSTACK
-       bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS
+       bool "Force CPU masks off stack"
+       depends on SMP
+       default n
        help
          Use dynamic allocation for cpumask_var_t, instead of putting
          them on the stack.  This is a bit more expensive, but avoids
Quoted 7 lines

arch/arm64/Kconfig config SCHED_SMT · line 1426

Quoted 4 lines
arch/arm64/Kconfig +0 -0
   config HOTPLUG_CPU
   	bool "Support for hot-pluggable CPUs"

Same here.

#3 lore raw
Quoted 45 lines · Kefeng Wang
Quoted 24 lines · Russell King (Oracle)

arch/arm64/Kconfig config SCHED_SMT · line 1409

Quoted 8 lines · Christoph Lameter (Ampere)

Index: linux/arch/arm64/Kconfig
===================================================================

arch/arm64/Kconfig +0 -0
@@ -1407,7 +1407,21 @@ config SCHED_SMT
   config NR_CPUS
   	int "Maximum number of CPUs (2-4096)"
   	range 2 4096

I think your mailer got to your patch and messed up the white space. There are two spaces before each of these lines rather than the usual one.

Quoted 10 lines
arch/arm64/Kconfig +9 -1
-	default "256"
+	default 512
+
+#
+# Determines the placement of cpumasks.
+#
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
Quoted 8 lines

arch/arm64/Kconfig config SCHED_SMT · line 1424

Quoted 6 lines
arch/arm64/Kconfig +6 -0
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256

Should that be ">= 256" ?

Maybe just select CPUMASK_OFFSTACK if NR_CPUS >= 256,

But could we just make CPUMASK_OFFSTACK configurable and let user/distro to enable it?

lib/Kconfig +0 -0
index 5ddda7c2ed9b..4254be5aa843 100644

lib/Kconfig config CHECK_SIGNATURE · line 540

Quoted 7 lines
lib/Kconfig +3 -1
@@ -535,7 +535,9 @@ config CHECK_SIGNATURE
        bool
 
 config CPUMASK_OFFSTACK
-       bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS
+       bool "Force CPU masks off stack"
+       depends on SMP
+       default n

Please. No.

There is no point in defining a default of n. The default default is n. Therefore, specifying a default of n is utterly redundant as the option will still default to n and just adds clutter to Kconfig files.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
reviewer
#4 lore raw
In reply to Russell King (Oracle)
Quoted 15 lines · Russell King (Oracle)

arch/arm64/Kconfig config SCHED_SMT · line 1424

Quoted 12 lines · Christoph Lameter (Ampere)
arch/arm64/Kconfig +12 -0
+# Determines the placement of cpumasks.
+#
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256

Should that be ">= 256" ?

I don't think that ">= 256" makes sense. Note that since the cpumasks are arrays of unsigned long, they're chunked into groups of 64 bits:

    2 to  64 cpus:  1 x unsigned long =>  8 bytes
   65 to 128 cpus:  2 x unsigned long => 16 bytes
  129 to 192 cpus:  3 x unsigned long => 24 bytes
  193 to 256 cpus:  4 x unsigned long => 32 bytes
  257 to 320 cpus:  5 x unsigned long => 40 bytes

... and so if a mask for 256 CPUs is too big to go in the stack, so is any mask for 193+ CPUs, and so ">= 256" should be clamped down to ">= 193" or "> 192". The boundary should be just after a multiple of 64.

How did we choose 256 specifically? I note that x86-64 allows 512 CPUs before requiring CPUMASK_OFFSTACK, and I see that powerpc selects CPUMASK_OFFSTACK when NR_CPUS >= 8192.

Mark.

#5 lore raw
Quoted 30 lines · Mark Rutland
Quoted 15 lines · Russell King (Oracle)

arch/arm64/Kconfig config SCHED_SMT · line 1424

Quoted 12 lines · Christoph Lameter (Ampere)
arch/arm64/Kconfig +12 -0
+# Determines the placement of cpumasks.
+#
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256

Should that be ">= 256" ?

I don't think that ">= 256" makes sense. Note that since the cpumasks are arrays of unsigned long, they're chunked into groups of 64 bits:

    2 to  64 cpus:  1 x unsigned long =>  8 bytes
   65 to 128 cpus:  2 x unsigned long => 16 bytes
  129 to 192 cpus:  3 x unsigned long => 24 bytes
  193 to 256 cpus:  4 x unsigned long => 32 bytes
  257 to 320 cpus:  5 x unsigned long => 40 bytes

... and so if a mask for 256 CPUs is too big to go in the stack, so is any mask for 193+ CPUs, and so ">= 256" should be clamped down to ">= 193" or "> 192". The boundary should be just after a multiple of 64.

Quoted 5 lines

How did we choose 256 specifically? I note that x86-64 allows 512 CPUs before requiring CPUMASK_OFFSTACK, and I see that powerpc selects CPUMASK_OFFSTACK when NR_CPUS >= 8192.

Mark.

The suggestion for >= 256 may have been a zero-index/one-index mixup.

It seems > 256 was chosen as the cutoff simply because it preserves existing behavior. The patch description seems to imply there was pushback from distro maintainers on just increasing the default NR_CPUS.

The existing default value of 256 is probably already a strain for smaller ARM systems, to which x86-64 isn't a reasonable comparison. I'm not sure what the reaction to increasing from 64 to 256 in 2019 was like, but picking a pivot point for CPUMASK_OFFSTACK beyond 256 may skew the balance even less in favor of smaller ARM systems.

#6 lore raw
Quoted 4 lines · Eric Mackay

It seems > 256 was chosen as the cutoff simply because it preserves existing behavior. The patch description seems to imply there was pushback from distro maintainers on just increasing the default NR_CPUS.

Yup that was it.

Tested-by on v1 1/1
#7 lore raw
In reply to Christoph Lameter (Ampere)

Whitespace issues aside, I have applied the patch on top of kernel 6.1.55 and tested on both a dual-socket Ampere Altra machine with < 256 CPUs, and a dual-socket AmpereOne machine with > 256 CPUs. Works as expected, with all CPUs visible and functional.

arch/arm64/Kconfig config SCHED_SMT · line 1410

Quoted 5 lines

config NR_CPUS
int "Maximum number of CPUs (2-4096)"
range 2 4096

arch/arm64/Kconfig +1 -1
- 	default "256"
+ 	default 512

Nit: the new default value should be in quotation marks, if we want to be pedantic

#8 lore raw
Quoted 9 lines · Eric Mackay

Whitespace issues aside, I have applied the patch on top of kernel 6.1.55 and tested on both a dual-socket Ampere Altra machine with < 256 CPUs, and a dual-socket AmpereOne machine with > 256 CPUs. Works as expected, with all CPUs visible and functional.

arch/arm64/Kconfig config SCHED_SMT · line 1410

Quoted 5 lines

config NR_CPUS
int "Maximum number of CPUs (2-4096)"
range 2 4096

arch/arm64/Kconfig +1 -1
- 	default "256"
+ 	default 512

Nit: the new default value should be in quotation marks, if we want to be pedantic

I can't find anything that requires the quotes - and as "range" doesn't for consistency it seems that default shouldn't either. There's nothing in the documentation that indicates quotes should be used, and looking at the code, it's just treated as a string. The only thing that quotes seem to do is to ensure that whitespace will be included.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
#9 lore raw
Quoted 9 lines · Russell King (Oracle)
Quoted 8 lines · Eric Mackay

Whitespace issues aside, I have applied the patch on top of kernel 6.1.55 and tested on both a dual-socket Ampere Altra machine with < 256 CPUs, and a dual-socket AmpereOne machine with > 256 CPUs. Works as expected, with all CPUs visible and functional.

Quoted 5 lines

config NR_CPUS
int "Maximum number of CPUs (2-4096)"
range 2 4096

arch/arm64/Kconfig +1 -1
- 	default "256"
+ 	default 512
Quoted 11 lines
Quoted 1 line

Nit: the new default value should be in quotation marks, if we want to be pedantic

I can't find anything that requires the quotes - and as "range" doesn't for consistency it seems that default shouldn't either. There's nothing in the documentation that indicates quotes should be used, and looking at the code, it's just treated as a string. The only thing that quotes seem to do is to ensure that whitespace will be included.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

Ack. I withdraw my nit.

#11 lore raw
Quoted 5 lines · Christoph Lameter (Ampere)
Quoted 1 line · Eric Mackay

Ack. I withdraw my nit.

Alright. Are we done and should I post V2?

Yes, please go ahead

#12 lore raw

PATCH v2 1/1 ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512

Ampere Computing develops high end ARM processor that support an ever increasing number of processors. The default 256 processors are not enough for our newer products. The default is used by distros and therefore our customers cannot use distro kernels because the number of processors is not supported.

One of the objections against earlier patches to increase the limit was that the memory use becomes too high. There is a feature called CPUMASK_OFFSTACK that configures the cpumasks in the kernel to be dynamically allocated. This was used in the X86 architecture in the past to enable support for larger CPU configurations up to 8k cpus.

With that is becomes possible to dynamically size the allocation of the cpu bitmaps depending on the quantity of processors detected on bootup.

This patch enables that logic if more than 256 processors are configured and increases the default to 512 processors.

Further increases may be needed if ARM processor vendors start supporting more processors. Given the current inflationary trends in core counts from multiple processor manufacturers this may occur.

---

V1->V2
- Keep quotation marks
- Remove whiltespace damage
- Add tested by

arch/arm64/Kconfig +15 -1
Index: linux/arch/arm64/Kconfig
===================================================================
@@ -1407,7 +1407,21 @@ config SCHED_SMT
 config NR_CPUS
 	int "Maximum number of CPUs (2-4096)"
 	range 2 4096
-	default "256"
+	default "512"
+
+#
+# Determines the placement of cpumasks.
+#
+# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated.
+# Useful for machines with lots of core because it avoids increasing
+# the size of many of the data structures in the kernel.
+#
+# If this is off then the cpumasks have a static sizes and are
+# embedded within data structures.
+#
+config CPUMASK_OFFSTACK
+	def_bool y
+	depends on NR_CPUS > 256
 
 config HOTPLUG_CPU
 	bool "Support for hot-pluggable CPUs"