Skip to contentSkip to conversation

x86/numa: Add Devicetree support

PATCH In review Saurabh Sengar 4 participants 8 messages Jun 27, 2023 – Aug 24, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/numa: Add Devicetree support

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

---
 arch/x86/Kconfig        | 1 +
 arch/x86/kernel/setup.c | 2 +-
 arch/x86/mm/numa.c      | 3 +++
 3 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/Kconfig +1 -0
index c531b16ee0bf..a2ada193b2d8 100644
@@ -1567,6 +1567,7 @@ config NUMA
 	depends on X86_64 || (X86_32 && HIGHMEM64G && X86_BIGSMP)
 	default y if X86_BIGSMP
 	select USE_PERCPU_NUMA_NODE_ID
+	select OF_NUMA if OF
 	help
 	  Enable NUMA (Non-Uniform Memory Access) support.
 
arch/x86/kernel/setup.c +1 -1
index fd975a4a5200..940c92a6a5e9 100644
@@ -1220,6 +1220,7 @@ void __init setup_arch(char **cmdline_p)
 
 	early_acpi_boot_init();
 
+	x86_dtb_init();
 	initmem_init();
 	dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT);
 
@@ -1261,7 +1262,6 @@ void __init setup_arch(char **cmdline_p)
 	 * Read APIC and some other early information from ACPI tables.
 	 */
 	acpi_boot_init();
-	x86_dtb_init();
 
 	/*
 	 * get boot-time SMP configuration:
arch/x86/mm/numa.c +3 -0
index 2aadb2019b4f..a6d3d331fda2 100644
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/sched.h>
 #include <linux/topology.h>
+#include <linux/of.h>
 
 #include <asm/e820/api.h>
 #include <asm/proto.h>
@@ -733,6 +734,8 @@ void __init x86_numa_init(void)
 		if (!numa_init(amd_numa_init))
 			return;
 #endif
+		if (acpi_disabled && !numa_init(of_numa_init))
+			return;
 	}
 
 	numa_init(dummy_numa_init);
Signature · 1 line
2.34.1
#1 lore raw
Quoted 84 lines

-----Original Message-----
From: Saurabh Sengar <[email protected]>
Sent: Tuesday, June 27, 2023 12:58 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Michael Kelley (LINUX) <[email protected]>; [email protected]
Cc: [email protected]
Subject: [PATCH] x86/numa: Add Devicetree support

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

---
 arch/x86/Kconfig        | 1 +
 arch/x86/kernel/setup.c | 2 +-
 arch/x86/mm/numa.c      | 3 +++
 3 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/Kconfig +1 -0
index c531b16ee0bf..a2ada193b2d8 100644
@@ -1567,6 +1567,7 @@ config NUMA
 	depends on X86_64 || (X86_32 && HIGHMEM64G && X86_BIGSMP)
 	default y if X86_BIGSMP
 	select USE_PERCPU_NUMA_NODE_ID
+	select OF_NUMA if OF
 	help
 	  Enable NUMA (Non-Uniform Memory Access) support.
 
arch/x86/kernel/setup.c +1 -1
index fd975a4a5200..940c92a6a5e9 100644
@@ -1220,6 +1220,7 @@ void __init setup_arch(char **cmdline_p)
 
 	early_acpi_boot_init();
 
+	x86_dtb_init();
 	initmem_init();
 	dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT);
 
@@ -1261,7 +1262,6 @@ void __init setup_arch(char **cmdline_p)
 	 * Read APIC and some other early information from ACPI tables.
 	 */
 	acpi_boot_init();
-	x86_dtb_init();
 
 	/*
 	 * get boot-time SMP configuration:
arch/x86/mm/numa.c +3 -1
index 2aadb2019b4f..a6d3d331fda2 100644
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/sched.h>
 #include <linux/topology.h>
+#include <linux/of.h>
 
 #include <asm/e820/api.h>
 #include <asm/proto.h>
@@ -733,6 +734,8 @@ void __init x86_numa_init(void)
 		if (!numa_init(amd_numa_init))
 			return;
 #endif
+		if (acpi_disabled && !numa_init(of_numa_init))
+			return;
 	}
 
 	numa_init(dummy_numa_init);
--
Quoted 1 line

2.34.1

Hi x86 Maintainers,

May I get your feedback on this patch.

Regards,
Saurabh

#2 lore raw
Quoted 100 lines

-----Original Message-----
From: Saurabh Singh Sengar <[email protected]>
Sent: Wednesday, July 12, 2023 1:42 PM
To: Saurabh Sengar <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Michael Kelley (LINUX) <[email protected]>; linux- [email protected]; Sharath George John <[email protected]>
Subject: [EXTERNAL] RE: [PATCH] x86/numa: Add Devicetree support

Quoted 82 lines

-----Original Message-----
From: Saurabh Sengar <[email protected]>
Sent: Tuesday, June 27, 2023 12:58 PM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Michael Kelley (LINUX) <[email protected]>; [email protected]
Cc: [email protected]
Subject: [PATCH] x86/numa: Add Devicetree support

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

---
 arch/x86/Kconfig        | 1 +
 arch/x86/kernel/setup.c | 2 +-
 arch/x86/mm/numa.c      | 3 +++
 3 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/Kconfig index renamed from arch/x86/Kconfig +0 -0

c531b16ee0bf..a2ada193b2d8 100644

arch/x86/Kconfig +1 -0
@@ -1567,6 +1567,7 @@ config NUMA
 	depends on X86_64 || (X86_32 && HIGHMEM64G && X86_BIGSMP)
 	default y if X86_BIGSMP
 	select USE_PERCPU_NUMA_NODE_ID
+	select OF_NUMA if OF
 	help
 	  Enable NUMA (Non-Uniform Memory Access) support.
 
arch/x86/kernel/setup.c index renamed from arch/x86/kernel/setup.c +0 -0

fd975a4a5200..940c92a6a5e9 100644

arch/x86/kernel/setup.c +1 -1
@@ -1220,6 +1220,7 @@ void __init setup_arch(char **cmdline_p)
 
 	early_acpi_boot_init();
 
+	x86_dtb_init();
 	initmem_init();
 	dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT);
 
@@ -1261,7 +1262,6 @@ void __init setup_arch(char **cmdline_p)
 	 * Read APIC and some other early information from ACPI tables.
 	 */
 	acpi_boot_init();
-	x86_dtb_init();
 
 	/*
 	 * get boot-time SMP configuration:
arch/x86/mm/numa.c index renamed from arch/x86/mm/numa.c +0 -0

2aadb2019b4f..a6d3d331fda2 100644

arch/x86/mm/numa.c +3 -1
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/sched.h>
 #include <linux/topology.h>
+#include <linux/of.h>
 
 #include <asm/e820/api.h>
 #include <asm/proto.h>
@@ -733,6 +734,8 @@ void __init x86_numa_init(void)
 		if (!numa_init(amd_numa_init))
 			return;
 #endif
+		if (acpi_disabled && !numa_init(of_numa_init))
+			return;
 	}
 
 	numa_init(dummy_numa_init);
--
Quoted 1 line

2.34.1

Hi x86 Maintainers,

Quoted 4 lines

May I get your feedback on this patch.

Regards,
Saurabh

Kind reminder to review this patch.

- Saurabh

Reviewed-by on v1 1/1
#3 lore raw
In reply to Saurabh Sengar

From: Saurabh Sengar <[email protected]> Sent: Tuesday, June 27, 2023 12:28 AM

Quoted 73 lines

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

---
 arch/x86/Kconfig        | 1 +
 arch/x86/kernel/setup.c | 2 +-
 arch/x86/mm/numa.c      | 3 +++
 3 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/Kconfig +1 -0
index c531b16ee0bf..a2ada193b2d8 100644
@@ -1567,6 +1567,7 @@ config NUMA
 	depends on X86_64 || (X86_32 && HIGHMEM64G && X86_BIGSMP)
 	default y if X86_BIGSMP
 	select USE_PERCPU_NUMA_NODE_ID
+	select OF_NUMA if OF
 	help
 	  Enable NUMA (Non-Uniform Memory Access) support.
 
arch/x86/kernel/setup.c +1 -1
index fd975a4a5200..940c92a6a5e9 100644
@@ -1220,6 +1220,7 @@ void __init setup_arch(char **cmdline_p)
 
 	early_acpi_boot_init();
 
+	x86_dtb_init();
 	initmem_init();
 	dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT);
 
@@ -1261,7 +1262,6 @@ void __init setup_arch(char **cmdline_p)
 	 * Read APIC and some other early information from ACPI tables.
 	 */
 	acpi_boot_init();
-	x86_dtb_init();
 
 	/*
 	 * get boot-time SMP configuration:
arch/x86/mm/numa.c +3 -1
index 2aadb2019b4f..a6d3d331fda2 100644
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/sched.h>
 #include <linux/topology.h>
+#include <linux/of.h>
 
 #include <asm/e820/api.h>
 #include <asm/proto.h>
@@ -733,6 +734,8 @@ void __init x86_numa_init(void)
 		if (!numa_init(amd_numa_init))
 			return;
 #endif
+		if (acpi_disabled && !numa_init(of_numa_init))
+			return;
 	}
 
 	numa_init(dummy_numa_init);
--
Quoted 1 line

2.34.1

This looks good to me. I've looked and don't see any obvious issues in moving x86_dtb_init() earlier in setup_arch(). But I also know the setup ordering can be very tricky to get working correctly with a wide range of platforms, and I don't have that expertise. But with those caveats,

#4 lore raw
Quoted 94 lines

-----Original Message-----
From: Michael Kelley (LINUX) <[email protected]>
Sent: Thursday, August 3, 2023 12:19 AM
To: Saurabh Sengar <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] RE: [PATCH] x86/numa: Add Devicetree support

From: Saurabh Sengar <[email protected]> Sent: Tuesday, June 27, 2023 12:28 AM

Quoted 73 lines

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

---
 arch/x86/Kconfig        | 1 +
 arch/x86/kernel/setup.c | 2 +-
 arch/x86/mm/numa.c      | 3 +++
 3 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/Kconfig index renamed from arch/x86/Kconfig +0 -0

c531b16ee0bf..a2ada193b2d8 100644

arch/x86/Kconfig +1 -0
@@ -1567,6 +1567,7 @@ config NUMA
 	depends on X86_64 || (X86_32 && HIGHMEM64G && X86_BIGSMP)
 	default y if X86_BIGSMP
 	select USE_PERCPU_NUMA_NODE_ID
+	select OF_NUMA if OF
 	help
 	  Enable NUMA (Non-Uniform Memory Access) support.
 
arch/x86/kernel/setup.c index renamed from arch/x86/kernel/setup.c +0 -0

fd975a4a5200..940c92a6a5e9 100644

arch/x86/kernel/setup.c +1 -1
@@ -1220,6 +1220,7 @@ void __init setup_arch(char **cmdline_p)
 
 	early_acpi_boot_init();
 
+	x86_dtb_init();
 	initmem_init();
 	dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT);
 
@@ -1261,7 +1262,6 @@ void __init setup_arch(char **cmdline_p)
 	 * Read APIC and some other early information from ACPI tables.
 	 */
 	acpi_boot_init();
-	x86_dtb_init();
 
 	/*
 	 * get boot-time SMP configuration:
arch/x86/mm/numa.c index renamed from arch/x86/mm/numa.c +0 -0

2aadb2019b4f..a6d3d331fda2 100644

arch/x86/mm/numa.c +3 -1
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/sched.h>
 #include <linux/topology.h>
+#include <linux/of.h>
 
 #include <asm/e820/api.h>
 #include <asm/proto.h>
@@ -733,6 +734,8 @@ void __init x86_numa_init(void)
 		if (!numa_init(amd_numa_init))
 			return;
 #endif
+		if (acpi_disabled && !numa_init(of_numa_init))
+			return;
 	}
 
 	numa_init(dummy_numa_init);
--
Quoted 1 line

2.34.1

This looks good to me. I've looked and don't see any obvious issues in moving x86_dtb_init() earlier in setup_arch(). But I also know the setup ordering can be very tricky to get working correctly with a wide range of platforms, and I don't have that expertise.
But with those caveats,

Quoted 1 line

Hi X86 Maintainers,

Is this good to be accepted ?

- Saurabh

#5 lore raw
Quoted 94 lines · Saurabh Singh Sengar
Quoted 94 lines

-----Original Message-----
From: Michael Kelley (LINUX) <[email protected]>
Sent: Thursday, August 3, 2023 12:19 AM
To: Saurabh Sengar <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [EXTERNAL] RE: [PATCH] x86/numa: Add Devicetree support

From: Saurabh Sengar <[email protected]> Sent: Tuesday, June 27, 2023 12:28 AM

Quoted 73 lines

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

---
 arch/x86/Kconfig        | 1 +
 arch/x86/kernel/setup.c | 2 +-
 arch/x86/mm/numa.c      | 3 +++
 3 files changed, 5 insertions(+), 1 deletion(-)
arch/x86/Kconfig index renamed from arch/x86/Kconfig +0 -0

c531b16ee0bf..a2ada193b2d8 100644

arch/x86/Kconfig +1 -0
@@ -1567,6 +1567,7 @@ config NUMA
 	depends on X86_64 || (X86_32 && HIGHMEM64G && X86_BIGSMP)
 	default y if X86_BIGSMP
 	select USE_PERCPU_NUMA_NODE_ID
+	select OF_NUMA if OF
 	help
 	  Enable NUMA (Non-Uniform Memory Access) support.
 
arch/x86/kernel/setup.c index renamed from arch/x86/kernel/setup.c +0 -0

fd975a4a5200..940c92a6a5e9 100644

arch/x86/kernel/setup.c +1 -1
@@ -1220,6 +1220,7 @@ void __init setup_arch(char **cmdline_p)
 
 	early_acpi_boot_init();
 
+	x86_dtb_init();
 	initmem_init();
 	dma_contiguous_reserve(max_pfn_mapped << PAGE_SHIFT);
 
@@ -1261,7 +1262,6 @@ void __init setup_arch(char **cmdline_p)
 	 * Read APIC and some other early information from ACPI tables.
 	 */
 	acpi_boot_init();
-	x86_dtb_init();
 
 	/*
 	 * get boot-time SMP configuration:
arch/x86/mm/numa.c index renamed from arch/x86/mm/numa.c +0 -0

2aadb2019b4f..a6d3d331fda2 100644

arch/x86/mm/numa.c +3 -1
@@ -11,6 +11,7 @@
 #include <linux/nodemask.h>
 #include <linux/sched.h>
 #include <linux/topology.h>
+#include <linux/of.h>
 
 #include <asm/e820/api.h>
 #include <asm/proto.h>
@@ -733,6 +734,8 @@ void __init x86_numa_init(void)
 		if (!numa_init(amd_numa_init))
 			return;
 #endif
+		if (acpi_disabled && !numa_init(of_numa_init))
+			return;
 	}
 
 	numa_init(dummy_numa_init);
--
Quoted 1 line

2.34.1

This looks good to me. I've looked and don't see any obvious issues in moving x86_dtb_init() earlier in setup_arch(). But I also know the setup ordering can be very tricky to get working correctly with a wide range of platforms, and I don't have that expertise.
But with those caveats,

Quoted 5 lines
Quoted 1 line

Hi X86 Maintainers,

Is this good to be accepted ?

Thomas/Borislav,

Kind reminder to review this.

- Saurabh

Quoted 2 lines

- Saurabh

#6 lore raw
In reply to Saurabh Sengar
Quoted 10 lines · Saurabh Sengar

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

Moving dtb_init() is not really a good idea. The APIC/IO-APIC enumeration is post initmem_init() on purpose and the ongoing rework of the topology evaluation relies on that.

What you really want is to split dtb_init() into two parts:

1) x86_flattree_get_config() which can be invoked before initmem init like ACPI has an early init part so SRAT parsing can be done in the numa initialization.

2) The APIC/IOAPIC registration part, which stays where it is.

This split wants to be a seperate change.

Thanks,

tglx

#7 lore raw
Quoted 18 lines · Thomas Gleixner
Quoted 10 lines · Saurabh Sengar

Hyper-V has usecases where it need to fetch NUMA information from Devicetree. Currently, it is not possible to extract the NUMA information from Devicetree for x86 arch.

Add support for Devicetree in the x86_numa_init function, allowing the retrieval of NUMA node information from the Devicetree.

Additionally, relocate the x86_dtb_init function before initmem_init to ensure the Devicetree initialization prior to its utilization in x86_numa_init.

Moving dtb_init() is not really a good idea. The APIC/IO-APIC enumeration is post initmem_init() on purpose and the ongoing rework of the topology evaluation relies on that.

What you really want is to split dtb_init() into two parts:

Quoted 7 lines

1) x86_flattree_get_config() which can be invoked before initmem init like ACPI has an early init part so SRAT parsing can be done in the numa initialization.

2) The APIC/IOAPIC registration part, which stays where it is.

This split wants to be a seperate change.

Thanks for the review, I will send the v2 with suggested modifications.

- Saurabh

Quoted 8 lines

Thanks,

tglx