Skip to contentSkip to conversation

- Memoryless nodes

PATCH In review Jack Steiner 3 participants 7 messages Apr 17, 2009 – Apr 20, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 - Memoryless nodes

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

---
 arch/x86/kernel/tlb_uv.c |   12 +++++++++++-
 arch/x86/mm/srat_64.c    |    5 ++++-
 2 files changed, 15 insertions(+), 2 deletions(-)
arch/x86/mm/srat_64.c +4 -1
Index: linux/arch/x86/mm/srat_64.c
===================================================================
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -174,6 +175,7 @@ acpi_numa_processor_affinity_init(struct
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +404,8 @@ int __init acpi_scan_nodes(unsigned long
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)
#1 lore raw

[tip:x86/urgent] x86/uv: fix init of memory-less nodes

Commit-ID:  28468050e2f5410c04fd737c922f2bea73ddd690
Gitweb:     http://git.kernel.org/tip/28468050e2f5410c04fd737c922f2bea73ddd690
Author:     Jack Steiner <[email protected]>
AuthorDate: Fri, 17 Apr 2009 09:22:42 -0500
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 17 Apr 2009 16:45:29 +0200

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

[ Impact: fix potential boot crash on memory-less UV nodes. ]

LKML-Reference: <[email protected]>

---
 arch/x86/mm/srat_64.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)
arch/x86/mm/srat_64.c +4 -1
index c7d272b..1619026 100644
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -174,6 +175,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +404,8 @@ int __init acpi_scan_nodes(unsigned long start, unsigned long end)
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)
#2 lore raw

Re: [tip:x86/urgent] x86/uv: fix init of memory-less nodes

Quoted 7 lines · tip-bot for Jack Steiner
Commit-ID:  28468050e2f5410c04fd737c922f2bea73ddd690
Gitweb:     http://git.kernel.org/tip/28468050e2f5410c04fd737c922f2bea73ddd690
Author:     Jack Steiner <[email protected]>
AuthorDate: Fri, 17 Apr 2009 09:22:42 -0500
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 17 Apr 2009 16:45:29 +0200
Quoted 8 lines

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

[ Impact: fix potential boot crash on memory-less UV nodes. ]

This is incomplete, x2apic maps cpus to pxms with a type 2 srat entry and that pxm may not have affinity to any type 1 memory structure.

#3 lore raw

Re: [tip:x86/urgent] x86/uv: fix init of memory-less nodes

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

[ Impact: fix potential boot crash on memory-less nodes. ]

---

Fixes case caught by David - missed support for the x2apic SRAT table.

 arch/x86/mm/srat_64.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/mm/srat_64.c +5 -1
Index: linux/arch/x86/mm/srat_64.c
===================================================================
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -141,6 +142,7 @@ acpi_numa_x2apic_affinity_init(struct ac
 
 	apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -174,6 +176,7 @@ acpi_numa_processor_affinity_init(struct
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +405,8 @@ int __init acpi_scan_nodes(unsigned long
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)
#4 lore raw

[tip:x86/urgent] x86/uv: fix init of memory-less nodes

In reply to Jack Steiner
Commit-ID:  dc098551918093901d8ac8936e9d1a1b891b56ed
Gitweb:     http://git.kernel.org/tip/dc098551918093901d8ac8936e9d1a1b891b56ed
Author:     Jack Steiner <[email protected]>
AuthorDate: Fri, 17 Apr 2009 09:22:42 -0500
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 17 Apr 2009 22:42:12 +0200

x86/uv: fix init of memory-less nodes

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

v2: Fixes case caught by David Rientjes - missed support
for the x2apic SRAT table.

[ Impact: fix potential boot crash on memory-less UV nodes. ]

LKML-Reference: <[email protected]>

---
 arch/x86/mm/srat_64.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
arch/x86/mm/srat_64.c +5 -1
index c7d272b..33c5fa5 100644
@@ -28,6 +28,7 @@ int acpi_numa __initdata;
 static struct acpi_table_slit *acpi_slit;
 
 static nodemask_t nodes_parsed __initdata;
+static nodemask_t cpu_nodes_parsed __initdata;
 static struct bootnode nodes[MAX_NUMNODES] __initdata;
 static struct bootnode nodes_add[MAX_NUMNODES];
 static int found_add_area __initdata;
@@ -141,6 +142,7 @@ acpi_numa_x2apic_affinity_init(struct acpi_srat_x2apic_cpu_affinity *pa)
 
 	apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -174,6 +176,7 @@ acpi_numa_processor_affinity_init(struct acpi_srat_cpu_affinity *pa)
 	else
 		apic_id = pa->apic_id;
 	apicid_to_node[apic_id] = node;
+	node_set(node, cpu_nodes_parsed);
 	acpi_numa = 1;
 	printk(KERN_INFO "SRAT: PXM %u -> APIC %u -> Node %u\n",
 	       pxm, apic_id, node);
@@ -402,7 +405,8 @@ int __init acpi_scan_nodes(unsigned long start, unsigned long end)
 		return -1;
 	}
 
-	node_possible_map = nodes_parsed;
+	/* Account for nodes with cpus and no memory */
+	nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
 
 	/* Finally register nodes */
 	for_each_node_mask(i, node_possible_map)
#5 lore raw
In reply to Jack Steiner
Quoted 3 lines · Jack Steiner

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

That was intentional because a node is defined as an area of memory and an area of memory without memory didn't seem to make sense. I still don't think it does. Instead the CPUs are assigned to nearby nodes. Back then the generic code also couldn't deal with them at all, so you might well run into more problems.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#6 lore raw
Quoted 11 lines · Andi Kleen
Quoted 3 lines · Jack Steiner

Add support for nodes that have cpus but no memory. The current code was failing to add these nodes to the nodes_present_map.

That was intentional because a node is defined as an area of memory and an area of memory without memory didn't seem to make sense. I still don't think it does. Instead the CPUs are assigned to nearby nodes. Back then the generic code also couldn't deal with them at all, so you might well run into more problems.

Nodes can also be viewed as a collection of cpus and memory that are located "close together" - ie., same PXM in the SRAT. Normally nodes will have both memory & cpus but in theory either cpus or memory could be missing.

I have run a number of tests of nodes with cpus only and so far have not seen any issues. I certainly won't rule out problems but have not seen them yet.

If there are any corner cases that you are aware of, let me know & I can do some more targeted testing.

(Note: this is not a bizarre end case that I'm trying to fix. UV actually has blades with cpus and no memory available to the OS.)

--- jack