x86_64: Kconfig changes for NUMA
RFC 1/1 x86_64: Kconfig changes for NUMA
On x86_64 arches, there is no way to choose ACPI_NUMA without having to choose K8_NUMA. CONFIG_K8_NUMA is not needed for Intel EM64T NUMA boxes. It also looks odd if you have to select ACPI_NUMA from the power management menu. This patch fixes those oddities. Patch does the following:
1. Makes NUMA a config option like other arches
2. Makes topology detection options like K8_NUMA dependent on NUMA
3. Choosing ACPI NUMA detection can be done from the standard "Processor type and features" menu
Comments?
Thanks,
Kiran
arch/x86_64/Kconfig +31 -15
@@ -226,22 +226,41 @@ source "kernel/Kconfig.preempt" -config K8_NUMA - bool "K8 NUMA support" - select NUMA +config NUMA + bool "Non Uniform Memory Access (NUMA) Support" depends on SMP + default y if !MPSC + select X86_64_ACPI_NUMA if MPSC + help + Enable NUMA (Non Uniform Memory Access) support. The kernel + will try to allocate memory used by a CPU on the local memory + controller of the CPU and add some more NUMA awareness to the kernel. + This code is recommended on all multiprocessor Opteron systems. + If the system is EM64T, you should say N unless your system is EM64T + NUMA. + +config K8_NUMA + bool "K8 NUMA detection" + depends on NUMA && !MPSC + default y help - Enable NUMA (Non Unified Memory Architecture) support for - AMD Opteron Multiprocessor systems. The kernel will try to allocate - memory used by a CPU on the local memory controller of the CPU - and add some more NUMA awareness to the kernel. - This code is recommended on all multiprocessor Opteron systems - and normally doesn't hurt on others. + Enable K8 NUMA node topology detection. You should say Y here if + you have a multi processor AMD K8 system. + +# Dummy CONFIG option to select ACPI_NUMA from drivers/acpi/Kconfig. + +config X86_64_ACPI_NUMA + bool "ACPI NUMA detection" + depends on NUMA + select ACPI + select ACPI_NUMA + default y + help + Enable ACPI SRAT based node topology detection. config NUMA_EMU - bool "NUMA emulation support" - select NUMA - depends on SMP + bool "NUMA emulation" + depends on NUMA help Enable NUMA emulation. A flat machine will be split into virtual nodes when booted with "numa=fake=N", where N is the
@@ -252,9 +271,6 @@ depends on NUMA default y -config NUMA - bool - default n config ARCH_DISCONTIGMEM_ENABLE def_bool y
Quoted 6 lines · Ravikiran G Thirumalai
1. Makes NUMA a config option like other arches
2. Makes topology detection options likeK8_NUMAdependent on NUMA
3. Choosing ACPI NUMA detection can be done from the standard "Processor type and features" menu
Comments?
It's in principle ok except that I don't like the dependencies and defaults. K8_NUMA shouldn't be dependent on !M_PSC. And the defaults should be just dropped.
-Andi
Quoted 9 lines · Andi Kleen
Quoted 6 lines · Ravikiran G Thirumalai
1. Makes NUMA a config option like other arches
2. Makes topology detection options likeK8_NUMAdependent on NUMA
3. Choosing ACPI NUMA detection can be done from the standard "Processor type and features" menu
Comments?
Quoted 3 lines
It's in principle ok except that I don't like the dependencies and defaults.
K8_NUMAshouldn't be dependent on !M_PSC. And the defaults should be just dropped.
Well K8_NUMA will never be needed for EM64T, so I put in that dependency...I kept defaults because they were there earlier ...I can take out that too...
How's this?
Thanks,
Kiran
--
On x86_64 arches, there is no way to choose ACPI_NUMA without having to choose K8_NUMA. CONFIG_K8_NUMA is not needed for Intel EM64T NUMA boxes. It also looks odd if you have to select ACPI_NUMA from the power management menu. This patch fixes those oddities. Patch does the following:
1. Makes NUMA a config option like other arches
2. Makes topology detection options like K8_NUMA dependent on NUMA
3. Choosing ACPI NUMA detection can be done from the standard "Processor type and features" menu
arch/x86_64/Kconfig +27 -15
@@ -226,22 +226,37 @@ source "kernel/Kconfig.preempt" -config K8_NUMA - bool "K8 NUMA support" - select NUMA +config NUMA + bool "Non Uniform Memory Access (NUMA) Support" depends on SMP help - Enable NUMA (Non Unified Memory Architecture) support for - AMD Opteron Multiprocessor systems. The kernel will try to allocate - memory used by a CPU on the local memory controller of the CPU - and add some more NUMA awareness to the kernel. - This code is recommended on all multiprocessor Opteron systems - and normally doesn't hurt on others. + Enable NUMA (Non Uniform Memory Access) support. The kernel + will try to allocate memory used by a CPU on the local memory + controller of the CPU and add some more NUMA awareness to the kernel. + This code is recommended on all multiprocessor Opteron systems. + If the system is EM64T, you should say N unless your system is EM64T + NUMA. + +config K8_NUMA + bool "K8 NUMA detection" + depends on NUMA + help + Enable K8 NUMA node topology detection. You should say Y here if + you have a multi processor AMD K8 system. + +# Dummy CONFIG option to select ACPI_NUMA from drivers/acpi/Kconfig. + +config X86_64_ACPI_NUMA + bool "ACPI NUMA detection" + depends on NUMA + select ACPI + select ACPI_NUMA + help + Enable ACPI SRAT based node topology detection. config NUMA_EMU - bool "NUMA emulation support" - select NUMA - depends on SMP + bool "NUMA emulation" + depends on NUMA help Enable NUMA emulation. A flat machine will be split into virtual nodes when booted with "numa=fake=N", where N is the
@@ -252,9 +267,6 @@ depends on NUMA default y -config NUMA - bool - default n config ARCH_DISCONTIGMEM_ENABLE def_bool y
arch/x86_64/defconfig +1 -0
@@ -94,6 +94,7 @@ # CONFIG_PREEMPT is not set CONFIG_PREEMPT_BKL=y CONFIG_K8_NUMA=y +CONFIG_X86_64_ACPI_NUMA=y # CONFIG_NUMA_EMU is not set CONFIG_ARCH_DISCONTIGMEM_ENABLE=y CONFIG_NUMA=y
Quoted 1 line · Ravikiran G Thirumalai
How's this?
Looks good.
There is a small inaccuracy in the help text for K8 - dual core single socket systems don't need NUMA - but I can fix that.
-Andi
Quoted 9 lines · Andi Kleen
Quoted 6 lines · Ravikiran G Thirumalai
1. Makes NUMA a config option like other arches
2. Makes topology detection options likeK8_NUMAdependent on NUMA
3. Choosing ACPI NUMA detection can be done from the standard "Processor type and features" menu
Comments?
Quoted 3 lines
It's in principle ok except that I don't like the dependencies and defaults.
K8_NUMAshouldn't be dependent on !M_PSC. And the defaults should be just dropped.
While at it, could you please consider to remove the SMP dependency from NUMA_EMU? 2.6.14-rc5-git5 builds and works with !SMP and NUMA_EMU.
Why?
1. No need to force SMP when not needed.
2. qemu-system-x86_64 does not currently work with SMP kernels.
Thanks,
/ magnus
Quoted 14 lines · Magnus Damm
Quoted 9 lines · Andi Kleen
Quoted 6 lines · Ravikiran G Thirumalai
1. Makes NUMA a config option like other arches
2. Makes topology detection options likeK8_NUMAdependent on NUMA
3. Choosing ACPI NUMA detection can be done from the standard "Processor type and features" menu
Comments?Quoted 3 lines
It's in principle ok except that I don't like the dependencies and defaults.
K8_NUMAshouldn't be dependent on !M_PSC. And the defaults should be just dropped.
Quoted 7 lines
While at it, could you please consider to remove the SMP dependency from
NUMA_EMU? 2.6.14-rc5-git5 builds and works with !SMP andNUMA_EMU.Why?
1. No need to force SMP when not needed.
2. qemu-system-x86_64 does not currently work with SMP kernels.
Update:
Both CONFIG_NUMA_EMU and CONFIG_K8_NUMA build and run just fine without CONFIG_SMP. Not sure about CONFIG_ACPI_NUMA though.
/ magnus
Quoted 2 lines · Magnus Damm
Quoted 2 lines
While at it, could you please consider to remove the SMP dependency from
NUMA_EMU? 2.6.14-rc5-git5 builds and works with !SMP and
No.
Quoted 5 lines
Quoted 5 lines
NUMA_EMU.Why?
1. No need to force SMP when not needed.
2. qemu-system-x86_64 does not currently work with SMP kernels.
qemu needs to be fixed then.
Quoted 4 lines
Update:
Both
CONFIG_NUMA_EMUandCONFIG_K8_NUMAbuild and run just fine withoutCONFIG_SMP. Not sure aboutCONFIG_ACPI_NUMAthough.
I don't want too many weird combinations which no normal person uses like this. Undoubtedly there will be compile breakage for such stuff in the future (even if it happens to work by chance now) and best is to not open this can of worms in the first place. The number of variants has to be kept under control.
-And
Quoted 6 lines · Andi Kleen
Quoted 2 lines · Magnus Damm
Quoted 2 lines
While at it, could you please consider to remove the SMP dependency from
NUMA_EMU? 2.6.14-rc5-git5 builds and works with !SMP andNo.
That was quick.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
NUMA_EMU.Why?
1. No need to force SMP when not needed.
2. qemu-system-x86_64 does not currently work with SMP kernels.qemu needs to be fixed then.
Yep. Any day soon now (tm).
Quoted 10 lines
Quoted 4 lines
Update:
Both
CONFIG_NUMA_EMUandCONFIG_K8_NUMAbuild and run just fine withoutCONFIG_SMP. Not sure aboutCONFIG_ACPI_NUMAthough.I don't want too many weird combinations which no normal person uses like this. Undoubtedly there will be compile breakage for such stuff in the future (even if it happens to work by chance now) and best is to not open this can of worms in the first place. The number of variants has to be kept under control.
Yes, I understand that keeping the combinations low is a good thing.
But if the same logic is applied to the NUMA code, why then is there both k8topology.c and srat.c? Does non-ACPI systems exist in x86_64 land?
Thanks,
/ magnus
Quoted 4 lines · Magnus Damm
But if the same logic is applied to the NUMA code, why then is there both
k8topology.candsrat.c? Does non-ACPI systems exist inx86_64land?
Historical reasons. The first AMD NUMA systems indeed didn't have SRAT and before ACPI 3.0 was released there were some concerns about the Microsoft license of SRAT. So k8topology was implemented as a stopgap. The long term plan is to get rid of it though once the ACPI code has been proven completely stable.
-Andi