Skip to contentSkip to conversation

x86 : set numa_nodes_parsed at acpi_numa_memory_affinity_init

PATCH Superseded Yasuaki Ishimatsu 4 participants 6 messages May 29, 2012 – Jun 1, 2012

Subsystems (2)
x86 x86 MM

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Yasuaki Ishimatsu

author
#0 lore raw

PATCH 1/1 x86 : set numa_nodes_parsed at acpi_numa_memory_affinity_init

When hot-adding a CPU into my system, the system showed following message since node_to_cpumask_map[2] was not allocated memory.

Booting Node 2 Processor 32 APIC 0xc0
node_to_cpumask_map[2] NULL
Pid: 0, comm: swapper/32 Tainted: G A 3.3.5-acd #21

Call Trace:
 [<ffffffff81048845>] debug_cpumask_set_cpu+0x155/0x160
 [<ffffffff8105e28a>] ? add_timer_on+0xaa/0x120
 [<ffffffff8150665f>] numa_add_cpu+0x1e/0x22
 [<ffffffff815020bb>] identify_cpu+0x1df/0x1e4
 [<ffffffff815020d6>] identify_econdary_cpu+0x16/0x1d
 [<ffffffff81504614>] smp_store_cpu_info+0x3c/0x3e
 [<ffffffff81505263>] smp_callin+0x139/0x1be
 [<ffffffff815052fb>] start_secondary+0x13/0xeb

In my investigation, the root cause is that the bit of node 2 was not set at numa_nodes_parsed since hot-added cpu was not written in ACPI SRAT Table. But hot-added memory which is same PXM as the cpu is written in here.

ACPI Spec Rev 5.0 says about ACPI SRAT talbe as follows:
This optional table provides information that allows OSPM to associate processors and memory ranges, including ranges of memory provided by hot-added memory devices, with system localities / proximity domains and clock domains.

So I think the ACPI SRAT Table is not wrong. Thus numa_nodes_parsed should be set by not only acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init but also acpi_numa_memory_affinity_init for the case.

Signed-off-by Yasuaki Ishimatsu <[email protected]>
---
 arch/x86/mm/srat.c |    2 ++
 1 file changed, 2 insertions(+)
arch/x86/mm/srat.c +2 -0
Index: linux-3.4/arch/x86/mm/srat.c
===================================================================
@@ -176,6 +176,8 @@  acpi_numa_memory_affinity_init(struct ac
 		return;
 	}
 
+	node_set(node, numa_nodes_parsed);
+
 	printk(KERN_INFO "SRAT: Node %u PXM %u %Lx-%Lx\n", node, pxm,
 	       start, end);
 }
#1 lore raw

Hi Ishimatsu-san,

Quoted 16 lines · Yasuaki Ishimatsu

When hot-adding a CPU into my system, the system showed following message since node_to_cpumask_map[2] was not allocated memory.

Booting Node 2 Processor 32 APIC 0xc0
node_to_cpumask_map[2] NULL
Pid: 0, comm: swapper/32 Tainted: G A 3.3.5-acd #21

Call Trace:
  [<ffffffff81048845>] debug_cpumask_set_cpu+0x155/0x160
  [<ffffffff8105e28a>] ? add_timer_on+0xaa/0x120
  [<ffffffff8150665f>] numa_add_cpu+0x1e/0x22
  [<ffffffff815020bb>] identify_cpu+0x1df/0x1e4
  [<ffffffff815020d6>] identify_econdary_cpu+0x16/0x1d
  [<ffffffff81504614>] smp_store_cpu_info+0x3c/0x3e
  [<ffffffff81505263>] smp_callin+0x139/0x1be
  [<ffffffff815052fb>] start_secondary+0x13/0xeb
Quoted 3 lines

In my investigation, the root cause is that the bit of node 2 was not set at numa_nodes_parsed since hot-added cpu was not written in ACPI SRAT Table. But hot-added memory which is same PXM as the cpu is written in here.

I don't understand this explanation. If cpu is not written in srat table, it is firmware bug. isn't it? srat table should describe all possible cpus. Why doesn't your system describe cpu info in acpi table?

Quoted 21 lines

ACPI Spec Rev 5.0 says about ACPI SRAT talbe as follows:
This optional table provides information that allows OSPM to associate processors and memory ranges, including ranges of memory provided by hot-added memory devices, with system localities / proximity domains and clock domains.

So I think the ACPI SRAT Table is not wrong. Thus numa_nodes_parsed should be set by not only acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init but also acpi_numa_memory_affinity_init for the case.

Signed-off-by Yasuaki Ishimatsu<[email protected]>
---
  arch/x86/mm/srat.c |    2 ++
  1 file changed, 2 insertions(+)

Index: linux-3.4/arch/x86/mm/srat.c
===================================================================

arch/x86/mm/srat.c acpi_numa_memory_affinity_init(struct ac · line 180

Quoted 5 lines
arch/x86/mm/srat.c +2 -0
@@ -176,6 +176,8 @@  acpi_numa_memory_affinity_init(struct ac
  		return;
  	}
 
+	node_set(node, numa_nodes_parsed);
+

But...
I think this patch itself is correct. If a system have cpuless memory node, acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init can't find any cpu description. but we need memory node allocation.

Quoted 9 lines
arch/x86/mm/srat.c +0 -1
  	printk(KERN_INFO "SRAT: Node %u PXM %u %Lx-%Lx\n", node, pxm,
  	       start, end);
  }
 
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
Quoted 52 lines · KOSAKI Motohiro

Hi Ishimatsu-san,

Quoted 16 lines · Yasuaki Ishimatsu

When hot-adding a CPU into my system, the system showed following message since node_to_cpumask_map[2] was not allocated memory.

Booting Node 2 Processor 32 APIC 0xc0
node_to_cpumask_map[2] NULL
Pid: 0, comm: swapper/32 Tainted: G A 3.3.5-acd #21

Call Trace:
  [<ffffffff81048845>] debug_cpumask_set_cpu+0x155/0x160
  [<ffffffff8105e28a>] ? add_timer_on+0xaa/0x120
  [<ffffffff8150665f>] numa_add_cpu+0x1e/0x22
  [<ffffffff815020bb>] identify_cpu+0x1df/0x1e4
  [<ffffffff815020d6>] identify_econdary_cpu+0x16/0x1d
  [<ffffffff81504614>] smp_store_cpu_info+0x3c/0x3e
  [<ffffffff81505263>] smp_callin+0x139/0x1be
  [<ffffffff815052fb>] start_secondary+0x13/0xeb
Quoted 3 lines

In my investigation, the root cause is that the bit of node 2 was not set at numa_nodes_parsed since hot-added cpu was not written in ACPI SRAT Table. But hot-added memory which is same PXM as the cpu is written in here.

I don't understand this explanation. If cpu is not written in srat table, it is firmware bug. isn't it? srat table should describe all possible cpus. Why doesn't your system describe cpu info in acpi table?

Quoted 24 lines

ACPI Spec Rev 5.0 says about ACPI SRAT talbe as follows:
This optional table provides information that allows OSPM to associate processors and memory ranges, including ranges of memory provided by hot-added memory devices, with system localities / proximity domains and clock domains.

So I think the ACPI SRAT Table is not wrong. Thus numa_nodes_parsed should be set by not only acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init but also acpi_numa_memory_affinity_init for the case.

Signed-off-by Yasuaki Ishimatsu<[email protected]>
---
  arch/x86/mm/srat.c |    2 ++
  1 file changed, 2 insertions(+)

Index: linux-3.4/arch/x86/mm/srat.c
===================================================================

arch/x86/mm/srat.c +0 -0
@@ -176,6 +176,8 @@  acpi_numa_memory_affinity_init(struct ac
  		return;
  	}
Quoted 7 lines

arch/x86/mm/srat.c acpi_numa_memory_affinity_init(struct ac · line 180

Quoted 2 lines
arch/x86/mm/srat.c +2 -0
+	node_set(node, numa_nodes_parsed);
+

But...
I think this patch itself is correct. If a system have cpuless memory node, acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init can't find any cpu description. but we need memory node allocation.

SRAT table only provides information for CPUs present at boot time. For hot-added CPUs, the _PXM and _SLI methods provide the same information for them as SRAT table.

Quoted 17 lines
Quoted 11 lines
arch/x86/mm/srat.c +0 -1
  	printk(KERN_INFO "SRAT: Node %u PXM %u %Lx-%Lx\n", node, pxm,
  	       start, end);
  }
 
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

author
#3 lore raw
In reply to KOSAKI Motohiro

Hi Kosaki-san,

Quoted 20 lines · KOSAKI Motohiro

Hi Ishimatsu-san,

Quoted 16 lines · Yasuaki Ishimatsu

When hot-adding a CPU into my system, the system showed following message since node_to_cpumask_map[2] was not allocated memory.

Booting Node 2 Processor 32 APIC 0xc0
node_to_cpumask_map[2] NULL
Pid: 0, comm: swapper/32 Tainted: G A 3.3.5-acd #21

Call Trace:
   [<ffffffff81048845>] debug_cpumask_set_cpu+0x155/0x160
   [<ffffffff8105e28a>] ? add_timer_on+0xaa/0x120
   [<ffffffff8150665f>] numa_add_cpu+0x1e/0x22
   [<ffffffff815020bb>] identify_cpu+0x1df/0x1e4
   [<ffffffff815020d6>] identify_econdary_cpu+0x16/0x1d
   [<ffffffff81504614>] smp_store_cpu_info+0x3c/0x3e
   [<ffffffff81505263>] smp_callin+0x139/0x1be
   [<ffffffff815052fb>] start_secondary+0x13/0xeb
Quoted 7 lines
Quoted 3 lines

In my investigation, the root cause is that the bit of node 2 was not set at numa_nodes_parsed since hot-added cpu was not written in ACPI SRAT Table. But hot-added memory which is same PXM as the cpu is written in here.

I don't understand this explanation. If cpu is not written in srat table, it is firmware bug. isn't it? srat table should describe all possible cpus. Why doesn't your system describe cpu info in acpi table?

As jiang answered, hot-added CPU need not be written in SRAT Table. So I don't think it is firmware bug.

Thanks,
Yasuaki Ishimatsu

Quoted 51 lines
Quoted 22 lines

ACPI Spec Rev 5.0 says about ACPI SRAT talbe as follows:
This optional table provides information that allows OSPM to associate processors and memory ranges, including ranges of memory provided by hot-added memory devices, with system localities / proximity domains and clock domains.

So I think the ACPI SRAT Table is not wrong. Thus numa_nodes_parsed should be set by not only acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init but also acpi_numa_memory_affinity_init for the case.

Signed-off-by Yasuaki Ishimatsu<[email protected]>
---
   arch/x86/mm/srat.c |    2 ++
   1 file changed, 2 insertions(+)

Index: linux-3.4/arch/x86/mm/srat.c
===================================================================

arch/x86/mm/srat.c acpi_numa_memory_affinity_init(struct ac · line 180

Quoted 5 lines
arch/x86/mm/srat.c +2 -0
@@ -176,6 +176,8 @@  acpi_numa_memory_affinity_init(struct ac
   		return;
   	}
 
+	node_set(node, numa_nodes_parsed);
+

But...
I think this patch itself is correct. If a system have cpuless memory node, acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init can't find any cpu description. but we need memory node allocation.

Quoted 9 lines
arch/x86/mm/srat.c +0 -1
   	printk(KERN_INFO "SRAT: Node %u PXM %u %Lx-%Lx\n", node, pxm,
   	       start, end);
   }
 
--

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Acked-by
#4 lore raw
Quoted 27 lines · Yasuaki Ishimatsu

Hi Kosaki-san,

Quoted 24 lines · KOSAKI Motohiro

Hi Ishimatsu-san,

Quoted 19 lines · Yasuaki Ishimatsu

When hot-adding a CPU into my system, the system showed following message since node_to_cpumask_map[2] was not allocated memory.

Booting Node 2 Processor 32 APIC 0xc0
node_to_cpumask_map[2] NULL
Pid: 0, comm: swapper/32 Tainted: G A 3.3.5-acd #21

Call Trace:
    [<ffffffff81048845>] debug_cpumask_set_cpu+0x155/0x160
    [<ffffffff8105e28a>] ? add_timer_on+0xaa/0x120
    [<ffffffff8150665f>] numa_add_cpu+0x1e/0x22
    [<ffffffff815020bb>] identify_cpu+0x1df/0x1e4
    [<ffffffff815020d6>] identify_econdary_cpu+0x16/0x1d
    [<ffffffff81504614>] smp_store_cpu_info+0x3c/0x3e
    [<ffffffff81505263>] smp_callin+0x139/0x1be
    [<ffffffff815052fb>] start_secondary+0x13/0xeb

In my investigation, the root cause is that the bit of node 2 was not set at numa_nodes_parsed since hot-added cpu was not written in ACPI SRAT Table. But hot-added memory which is same PXM as the cpu is written in here.

Quoted 6 lines
Quoted 3 lines

I don't understand this explanation. If cpu is not written in srat table, it is firmware bug. isn't it? srat table should describe all possible cpus. Why doesn't your system describe cpu info in acpi table?

As jiang answered, hot-added CPU need not be written in SRAT Table. So I don't think it is firmware bug.

If so, please write it to the description. Your changelog was unclear. Other than that,

Acked-by KOSAKI Motohiro <[email protected]>
Acked-by on v1 1/1
#5 lore raw
In reply to Yasuaki Ishimatsu
Quoted 26 lines · Yasuaki Ishimatsu

When hot-adding a CPU into my system, the system showed following message since node_to_cpumask_map[2] was not allocated memory.

Booting Node 2 Processor 32 APIC 0xc0
node_to_cpumask_map[2] NULL
Pid: 0, comm: swapper/32 Tainted: G A 3.3.5-acd #21

Call Trace:
 [<ffffffff81048845>] debug_cpumask_set_cpu+0x155/0x160
 [<ffffffff8105e28a>] ? add_timer_on+0xaa/0x120
 [<ffffffff8150665f>] numa_add_cpu+0x1e/0x22
 [<ffffffff815020bb>] identify_cpu+0x1df/0x1e4
 [<ffffffff815020d6>] identify_econdary_cpu+0x16/0x1d
 [<ffffffff81504614>] smp_store_cpu_info+0x3c/0x3e
 [<ffffffff81505263>] smp_callin+0x139/0x1be
 [<ffffffff815052fb>] start_secondary+0x13/0xeb

In my investigation, the root cause is that the bit of node 2 was not set at numa_nodes_parsed since hot-added cpu was not written in ACPI SRAT Table. But hot-added memory which is same PXM as the cpu is written in here.

ACPI Spec Rev 5.0 says about ACPI SRAT talbe as follows:
This optional table provides information that allows OSPM to associate processors and memory ranges, including ranges of memory provided by hot-added memory devices, with system localities / proximity domains and clock domains.

Quoted 5 lines

So I think the ACPI SRAT Table is not wrong. Thus numa_nodes_parsed should be set by not only acpi_numa_processor_affinity_init / acpi_numa_x2apic_affinity_init but also acpi_numa_memory_affinity_init for the case.

Signed-off-by Yasuaki Ishimatsu <[email protected]>
Acked-by David Rientjes <[email protected]>