Skip to contentSkip to conversation

ACPI: Only create NUMA nodes from entries in SRAT or SRAT emulation.

PATCH In review Jonathan Cameron 2 participants 10 messages Aug 18, 2020 – Sep 24, 2020

Subsystems (9)
Memory management ACPI PCI IOMMU +5 more subsystems
lore
Share & tools
lore
#0 lore raw

v3 cover ACPI: Only create NUMA nodes from entries in SRAT or SRAT emulation.

This is a trivial rebase and resend of V2 now the merge window has closed.

Here, I will use the term Proximity Domains for the ACPI description and NUMA Nodes for the in kernel representation.

ACPI 6.3 included a clarification that only Static Resource Allocation Structures in SRAT may define the existence of proximity domains (sec 5.2.16). This clarification closed a possible interpretation that other parts of ACPI (e.g. DSDT _PXM, NFIT etc) could define new proximity domains that were not also mentioned in SRAT structures.

In practice the kernel has never allowed this alternative interpretation as such nodes are only partially initialized. This is architecture specific but to take an example, on x86 alloc_node_data has not been called. Any use of them for node specific allocation, will result in a crash as the infrastructure to fallback to a node with memory is not setup.

We ran into a problem when enabling _PXM handling for PCI devices and found there were boards out there advertising devices in proximity domains that didn't exist [2].

The fix suggested in this series is to replace instances that should not 'create' new nodes with pxm_to_node. This function needs a some additional hardening against invalid inputs to make sure it is safe for use in these new callers.

Patch 1 Hardens pxm_to_node() against numa_off, and pxm entry being too large.

Patch 2-4 change the various callers not related to SRAT entries so that they set this parameter to false, so do not attempt to initialize a new NUMA node if the relevant one does not already exist.

Patch 5 is a function rename to reflect change in functionality of acpi_map_pxm_to_online_node() as it no longer creates a new map, but just does a lookup of existing maps.

Patch 6 covers the one place we do not allow the full flexibility defined in the ACPI spec. For SRAT GIC Interrupt Translation Service (ITS) Affinity Structures, on ARM64, the driver currently makes an additional pass of SRAT later in the boot than the one used to identify NUMA domains. Note, this currently means that an ITS placed in a proximity domain that is not defined by another SRAT structure will result in the a crash.

To avoid this crash with minimal changes we do not create new NUMA nodes based on this particular entry type. Any current platform trying to do this will not boot, so this is an improvement, if perhaps not a perfect solution.

[1] Note in ACPI Specification 6.3 5.2.16 System Resource Affinity Table (SRAT)
[2] https://patchwork.kernel.org/patch/10597777/

Thanks to Bjorn Helgaas for review of v1 and Barry Song for internal reviews that lead to a slightly different approach for this v2.

Changes since v2.
* Trivial rebase to v5.9-rc1
* Collect up tags.

Changes since v1.
* Use pxm_to_node for what was previously the path using acpi_map_pxm_to_node with create=false. (Barry)
* Broke patch up into an initial noop stage followed by patches (Bjorn) to update each type of case in which partial creation of NUMA nodes is prevented.
* Added patch 5 to rename function to reflect change of functionality.
* Updated descriptions (now mostly in individual patches) inline with Bjorn's comments.

Jonathan Cameron (6):
ACPI: Add out of bounds and numa_off protections to pxm_to_node
ACPI: Do not create new NUMA domains from ACPI static tables that are
not SRAT
ACPI: Remove side effect of partly creating a node in
acpi_map_pxm_to_online_node
ACPI: Rename acpi_map_pxm_to_online_node to pxm_to_online_node
ACPI: Remove side effect of partly creating a node in acpi_get_node irq-chip/gic-v3-its: Fix crash if ITS is in a proximity domain without
processor or memory

 drivers/acpi/arm64/iort.c        |  2 +-
 drivers/acpi/nfit/core.c         |  6 ++----
 drivers/acpi/numa/hmat.c         |  4 ++--
 drivers/acpi/numa/srat.c         |  4 ++--
 drivers/iommu/intel/dmar.c       |  2 +-
 drivers/irqchip/irq-gic-v3-its.c |  7 ++++++-
 include/linux/acpi.h             | 15 +++++++--------
 7 files changed, 21 insertions(+), 19 deletions(-)
Signature · 1 line
2.19.1
#1 lore raw

PATCH v3 1/6 ACPI: Add out of bounds and numa_off protections to pxm_to_node

The function should check the validity of the pxm value before using it to index the pxm_to_node_map array.

Whilst hardening this code may be good in general, the main intent here is to enable following patches that use this function to replace acpi_map_pxm_to_node for non SRAT usecases which should return NO_NUMA_NODE for PXM entries not matching with those in SRAT.

---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
index 15bbaab8500b..1fb486f46ee2 100644
@@ -31,7 +31,7 @@ int acpi_numa __initdata;
 
 int pxm_to_node(int pxm)
 {
-	if (pxm < 0)
+	if (pxm < 0 || pxm >= MAX_PXM_DOMAINS || numa_off)
 		return NUMA_NO_NODE;
 	return pxm_to_node_map[pxm];
 }
Signature · 1 line
2.19.1
#2 lore raw

PATCH v3 2/6 ACPI: Do not create new NUMA domains from ACPI static tables that are not SRAT

In reply to Jonathan Cameron

Several ACPI static tables contain references to proximity domains. ACPI 6.3 has clarified that only entries in SRAT may define a new domain (sec 5.2.16).

Those tables described in the ACPI spec have additional clarifying text.

NFIT: Table 5-132,

"Integer that represents the proximity domain to which the memory belongs.
This number must match with corresponding entry in the SRAT table."

HMAT: Table 5-145,

"... This number must match with the corresponding entry in the SRAT
table's processor affinity structure ... if the initiator is a processor, or the Generic Initiator Affinity Structure if the initiator is a generic initiator".

IORT and DMAR are defined by external specifications.

Intel Virtualization Technology for Directed I/O Rev 3.1 does not make any explicit statements, but the general SRAT statement above will still apply.
https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf

IO Remapping Table, Platform Design Document rev D, also makes not explicit statement, but refers to ACPI SRAT table for more information and again the generic SRAT statement above applies.
https://developer.arm.com/documentation/den0049/d/

In conclusion, any proximity domain specified in these tables, should be a reference to a proximity domain also found in SRAT, and they should not be able to instantiate a new domain. Hence we switch to pxm_to_node() which will only return existing nodes.

---
 drivers/acpi/arm64/iort.c  | 2 +-
 drivers/acpi/nfit/core.c   | 3 +--
 drivers/acpi/numa/hmat.c   | 2 +-
 drivers/iommu/intel/dmar.c | 2 +-
 4 files changed, 4 insertions(+), 5 deletions(-)
drivers/acpi/arm64/iort.c +1 -1
index ec782e4a0fe4..26005a15cb8b 100644
@@ -1335,7 +1335,7 @@ static int  __init arm_smmu_v3_set_proximity(struct device *dev,
 
 	smmu = (struct acpi_iort_smmu_v3 *)node->node_data;
 	if (smmu->flags & ACPI_IORT_SMMU_V3_PXM_VALID) {
-		int dev_node = acpi_map_pxm_to_node(smmu->pxm);
+		int dev_node = pxm_to_node(smmu->pxm);
 
 		if (dev_node != NUMA_NO_NODE && !node_online(dev_node))
 			return -EINVAL;
drivers/acpi/nfit/core.c +1 -2
index 26dd208a0d63..ea0557cb54f7 100644
@@ -3008,8 +3008,7 @@ static int acpi_nfit_register_region(struct acpi_nfit_desc *acpi_desc,
 	if (spa->flags & ACPI_NFIT_PROXIMITY_VALID) {
 		ndr_desc->numa_node = acpi_map_pxm_to_online_node(
 						spa->proximity_domain);
-		ndr_desc->target_node = acpi_map_pxm_to_node(
-				spa->proximity_domain);
+		ndr_desc->target_node = pxm_to_node(spa->proximity_domain);
 	} else {
 		ndr_desc->numa_node = NUMA_NO_NODE;
 		ndr_desc->target_node = NUMA_NO_NODE;
drivers/acpi/numa/hmat.c +1 -1
index 2c32cfb72370..cf6df2df26cd 100644
@@ -666,7 +666,7 @@ static void hmat_register_target_device(struct memory_target *target,
 
 	pdev->dev.numa_node = acpi_map_pxm_to_online_node(target->memory_pxm);
 	info = (struct memregion_info) {
-		.target_node = acpi_map_pxm_to_node(target->memory_pxm),
+		.target_node = pxm_to_node(target->memory_pxm),
 	};
 	rc = platform_device_add_data(pdev, &info, sizeof(info));
 	if (rc < 0) {
drivers/iommu/intel/dmar.c +1 -1
index 93e6345f3414..2f3badd41e1b 100644
@@ -473,7 +473,7 @@ static int dmar_parse_one_rhsa(struct acpi_dmar_header *header, void *arg)
 	rhsa = (struct acpi_dmar_rhsa *)header;
 	for_each_drhd_unit(drhd) {
 		if (drhd->reg_base_addr = rhsa->base_address) {
-			int node = acpi_map_pxm_to_node(rhsa->proximity_domain);
+			int node = pxm_to_node(rhsa->proximity_domain);
 
 			if (!node_online(node))
 				node = NUMA_NO_NODE;
Signature · 1 line
2.19.1
#3 lore raw

PATCH v3 3/6 ACPI: Remove side effect of partly creating a node in acpi_map_pxm_to_online_node

In reply to Jonathan Cameron

Whilst this function will only return an online node, it can have the side effect of partially creating a new node. The existing comments suggest this is intentional, but the usecases of this function are related to NFIT and HMAT parsing, neither of which should be able to define new nodes.

One route by which the existing behaviour would cause a crash is to have a _PXM entry in ACPI DSDT attempt to place a device within this partly created proximity domain. A subsequent call to devm_kzalloc or similar would result in an attempt to allocate memory on a node for which zone lists have not been set up and a null pointer dereference.

We prevent such cases by switching to pxm_to_node() within acpi_map_pxm_to_online_node which cannot cause a new node to be partly created. If one would previously have been created we now return NO_NUMA_NODE. Documentation updated to reflect this change. We may want to think about renaming acpi_map_pxm_to_online_node to pxm_to_online_node to reflect this change.

---
 include/linux/acpi.h | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
include/linux/acpi.h +3 -4
index 1e4cdc6c7ae2..a9fd122ae878 100644
@@ -430,13 +430,12 @@ int acpi_get_node(acpi_handle handle);
  * ACPI device drivers, which are called after the NUMA initialization has
  * completed in the kernel, can call this interface to obtain their device
  * NUMA topology from ACPI tables.  Such drivers do not have to deal with
- * offline nodes.  A node may be offline when a device proximity ID is
- * unique, SRAT memory entry does not exist, or NUMA is disabled, ex.
- * "numa=off" on x86.
+ * offline nodes.  A node may be offline when SRAT memory entry does not exist,
+ * or NUMA is disabled, ex. "numa=off" on x86.
  */
 static inline int acpi_map_pxm_to_online_node(int pxm)
 {
-	int node = acpi_map_pxm_to_node(pxm);
+	int node = pxm_to_node(pxm);
 
 	return numa_map_to_online_node(node);
 }
Signature · 1 line
2.19.1
#4 lore raw

PATCH v3 4/6 ACPI: Rename acpi_map_pxm_to_online_node to pxm_to_online_node

In reply to Jonathan Cameron

As this function is no longer allowed to create new mappings let us rename it to reflect this.

Note all nodes should already exist before any of the users of this function are called.

---
 drivers/acpi/nfit/core.c | 3 +--
 drivers/acpi/numa/hmat.c | 2 +-
 include/linux/acpi.h     | 8 ++++----
 3 files changed, 6 insertions(+), 7 deletions(-)
drivers/acpi/nfit/core.c +1 -2
index ea0557cb54f7..8df2f16d1fdf 100644
@@ -3006,8 +3006,7 @@ static int acpi_nfit_register_region(struct acpi_nfit_desc *acpi_desc,
 	ndr_desc->provider_data = nfit_spa;
 	ndr_desc->attr_groups = acpi_nfit_region_attribute_groups;
 	if (spa->flags & ACPI_NFIT_PROXIMITY_VALID) {
-		ndr_desc->numa_node = acpi_map_pxm_to_online_node(
-						spa->proximity_domain);
+		ndr_desc->numa_node = pxm_to_online_node(spa->proximity_domain);
 		ndr_desc->target_node = pxm_to_node(spa->proximity_domain);
 	} else {
 		ndr_desc->numa_node = NUMA_NO_NODE;
drivers/acpi/numa/hmat.c +1 -1
index cf6df2df26cd..e7add2609c03 100644
@@ -664,7 +664,7 @@ static void hmat_register_target_device(struct memory_target *target,
 		goto out_pdev;
 	}
 
-	pdev->dev.numa_node = acpi_map_pxm_to_online_node(target->memory_pxm);
+	pdev->dev.numa_node = pxm_to_online_node(target->memory_pxm);
 	info = (struct memregion_info) {
 		.target_node = pxm_to_node(target->memory_pxm),
 	};
include/linux/acpi.h +4 -4
index a9fd122ae878..e9f6cd67943e 100644
@@ -420,10 +420,10 @@ int acpi_map_pxm_to_node(int pxm);
 int acpi_get_node(acpi_handle handle);
 
 /**
- * acpi_map_pxm_to_online_node - Map proximity ID to online node
+ * pxm_to_online_node - Map proximity ID to online node
  * @pxm: ACPI proximity ID
  *
- * This is similar to acpi_map_pxm_to_node(), but always returns an online
+ * This is similar to pxm_to_node(), but always returns an online
  * node.  When the mapped node from a given proximity ID is offline, it
  * looks up the node distance table and returns the nearest online node.
  *
@@ -433,14 +433,14 @@ int acpi_get_node(acpi_handle handle);
  * offline nodes.  A node may be offline when SRAT memory entry does not exist,
  * or NUMA is disabled, ex. "numa=off" on x86.
  */
-static inline int acpi_map_pxm_to_online_node(int pxm)
+static inline int pxm_to_online_node(int pxm)
 {
 	int node = pxm_to_node(pxm);
 
 	return numa_map_to_online_node(node);
 }
 #else
-static inline int acpi_map_pxm_to_online_node(int pxm)
+static inline int pxm_to_online_node(int pxm)
 {
 	return 0;
 }
Signature · 1 line
2.19.1
#5 lore raw

PATCH v3 5/6 ACPI: Remove side effect of partly creating a node in acpi_get_node

In reply to Jonathan Cameron

acpi_get_node calls acpi_get_pxm to evaluate the _PXM AML method for entries found in DSDT/SSDT. ACPI 6.3 sec 6.2.14 states
"_PXM evaluates to an integer that identifies a device as belonging to
a Proximity Domain defined in the System Resource Affinity Table (SRAT)."

Hence a _PXM method should not result in creation of a new NUMA node. Before this patch, _PXM could result in partial instantiation of NUMA node, missing elements such as zone lists. A call to devm_kzalloc for example results in a null pointer dereference.

This patch therefore replaces the acpi_map_pxm_to_node with a call to pxm_to_node.

---
 drivers/acpi/numa/srat.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/acpi/numa/srat.c +1 -1
index 1fb486f46ee2..2c9a66c203ff 100644
@@ -436,6 +436,6 @@ int acpi_get_node(acpi_handle handle)
 
 	pxm = acpi_get_pxm(handle);
 
-	return acpi_map_pxm_to_node(pxm);
+	return pxm_to_node(pxm);
 }
 EXPORT_SYMBOL(acpi_get_node);
Signature · 1 line
2.19.1
#6 lore raw

PATCH v3 6/6 irq-chip/gic-v3-its: Fix crash if ITS is in a proximity domain without processor or memory

In reply to Jonathan Cameron

Note this crash is present before any of the patches in this series, but as explained below it is highly unlikely anyone is shipping a firmware that causes it. Tests were done using an overriden SRAT.

On ARM64, the gic-v3 driver directly parses SRAT to locate GIC Interrupt Translation Service (ITS) Affinity Structures. This is done much later in the boot than the parses of SRAT which identify proximity domains.

As a result, an ITS placed in a proximity domain that is not defined by another SRAT structure will result in a NUMA node that is not completely configured and a crash.

ITS [mem 0x202100000-0x20211ffff]
ITS@0x0000000202100000: Using ITS number 0

Unable to handle kernel paging request at virtual address 0000000000001a08

...

Call trace:
  __alloc_pages_nodemask+0xe8/0x338
  alloc_pages_node.constprop.0+0x34/0x40
  its_probe_one+0x2f8/0xb18
  gic_acpi_parse_madt_its+0x108/0x150
  acpi_table_parse_entries_array+0x17c/0x264
  acpi_table_parse_entries+0x48/0x6c
  acpi_table_parse_madt+0x30/0x3c
  its_init+0x1c4/0x644
  gic_init_bases+0x4b8/0x4ec
  gic_acpi_init+0x134/0x264
  acpi_match_madt+0x4c/0x84
  acpi_table_parse_entries_array+0x17c/0x264
  acpi_table_parse_entries+0x48/0x6c
  acpi_table_parse_madt+0x30/0x3c
  __acpi_probe_device_table+0x8c/0xe8
  irqchip_init+0x3c/0x48
  init_IRQ+0xcc/0x100
  start_kernel+0x33c/0x548

ACPI 6.3 allows any set of Affinity Structures in SRAT to define a proximity domain. However, as we do not see this crash, we can conclude that no firmware is currently placing an ITS in a node that is separate from those containing memory and / or processors.

We could modify the SRAT parsing behavior to identify the existence of Proximity Domains unique to the ITS structures, and handle them as a special case of a generic initiator (once support for those merges).

This patch avoids the complexity that would be needed to handle this corner case, by not allowing the ITS entry parsing code to instantiate new NUMA Nodes. If one is encountered that does not already exist, then NO_NUMA_NODE is assigned and a warning printed just as if the value had been greater than allowed NUMA Nodes.

"SRAT: Invalid NUMA node -1 in ITS affinity"

Whilst this does not provide the full flexibility allowed by ACPI, it does fix the problem. We can revisit a more sophisticated solution if needed by future platforms.

Change is simply to replace acpi_map_pxm_to_node with pxm_to_node reflecting the fact a new mapping is not created.

---
 drivers/irqchip/irq-gic-v3-its.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/irqchip/irq-gic-v3-its.c +6 -1
index 95f097448f97..b59b63ee3fa6 100644
@@ -5263,7 +5263,12 @@ static int __init gic_acpi_parse_srat_its(union acpi_subtable_headers *header,
 		return -EINVAL;
 	}
 
-	node = acpi_map_pxm_to_node(its_affinity->proximity_domain);
+	/*
+	 * Note that in theory a new proximity node could be created by this
+	 * entry as it is an SRAT resource allocation structure.
+	 * We do not currently support doing so.
+	 */
+	node = pxm_to_node(its_affinity->proximity_domain);
 
 	if (node = NUMA_NO_NODE || node >= MAX_NUMNODES) {
 		pr_err("SRAT: Invalid NUMA node %d in ITS affinity\n", node);
Signature · 1 line
2.19.1
#7 lore raw
In reply to Jonathan Cameron
Quoted 43 lines · On Tue, 18 Aug 2020 22:24:24 +0800 Jonathan Cameron…

This is a trivial rebase and resend of V2 now the merge window has closed.

Here, I will use the term Proximity Domains for the ACPI description and NUMA Nodes for the in kernel representation.

ACPI 6.3 included a clarification that only Static Resource Allocation Structures in SRAT may define the existence of proximity domains (sec 5.2.16). This clarification closed a possible interpretation that other parts of ACPI (e.g. DSDT _PXM, NFIT etc) could define new proximity domains that were not also mentioned in SRAT structures.

In practice the kernel has never allowed this alternative interpretation as such nodes are only partially initialized. This is architecture specific but to take an example, on x86 alloc_node_data has not been called. Any use of them for node specific allocation, will result in a crash as the infrastructure to fallback to a node with memory is not setup.

We ran into a problem when enabling _PXM handling for PCI devices and found there were boards out there advertising devices in proximity domains that didn't exist [2].

The fix suggested in this series is to replace instances that should not 'create' new nodes with pxm_to_node. This function needs a some additional hardening against invalid inputs to make sure it is safe for use in these new callers.

Patch 1 Hardens pxm_to_node() against numa_off, and pxm entry being too large.

Patch 2-4 change the various callers not related to SRAT entries so that they set this parameter to false, so do not attempt to initialize a new NUMA node if the relevant one does not already exist.

Patch 5 is a function rename to reflect change in functionality of acpi_map_pxm_to_online_node() as it no longer creates a new map, but just does a lookup of existing maps.

Patch 6 covers the one place we do not allow the full flexibility defined in the ACPI spec. For SRAT GIC Interrupt Translation Service (ITS) Affinity Structures, on ARM64, the driver currently makes an additional pass of SRAT later in the boot than the one used to identify NUMA domains. Note, this currently means that an ITS placed in a proximity domain that is not defined by another SRAT structure will result in the a crash.

Quoted 6 lines

To avoid this crash with minimal changes we do not create new NUMA nodes based on this particular entry type. Any current platform trying to do this will not boot, so this is an improvement, if perhaps not a perfect solution.

[1] Note in ACPI Specification 6.3 5.2.16 System Resource Affinity Table (SRAT)
[2] https://patchwork.kernel.org/patch/10597777/

Looking for input from ARM (Lorenzo?), X86(not sure) and ACPI(Rafael?) people on this set. As it also touches NFIT handling perhaps someone who focuses on that as well. Please feel free to CC additional people who might be interested.

I'm fairly confident that it should be uncontroversial (famous last words) and it closes down a problem that lead to issues with seemingly obvious changes in the past. (The whole PCI _PXM issue on some threadripper platforms).

Thanks to Bjorn, Hanjun and Barry for feedback on earlier revisions.

Thanks,

Jonathan

Quoted 36 lines

Thanks to Bjorn Helgaas for review of v1 and Barry Song for internal reviews that lead to a slightly different approach for this v2.

Changes since v2.
* Trivial rebase to v5.9-rc1
* Collect up tags.

Changes since v1.
* Use pxm_to_node for what was previously the path using acpi_map_pxm_to_node with create=false. (Barry)
* Broke patch up into an initial noop stage followed by patches (Bjorn) to update each type of case in which partial creation of NUMA nodes is prevented.
* Added patch 5 to rename function to reflect change of functionality.
* Updated descriptions (now mostly in individual patches) inline with Bjorn's comments.

Jonathan Cameron (6):
ACPI: Add out of bounds and numa_off protections to pxm_to_node
ACPI: Do not create new NUMA domains from ACPI static tables that are
not SRAT
ACPI: Remove side effect of partly creating a node in
acpi_map_pxm_to_online_node
ACPI: Rename acpi_map_pxm_to_online_node to pxm_to_online_node
ACPI: Remove side effect of partly creating a node in acpi_get_node irq-chip/gic-v3-its: Fix crash if ITS is in a proximity domain without
processor or memory

 drivers/acpi/arm64/iort.c        |  2 +-
 drivers/acpi/nfit/core.c         |  6 ++----
 drivers/acpi/numa/hmat.c         |  4 ++--
 drivers/acpi/numa/srat.c         |  4 ++--
 drivers/iommu/intel/dmar.c       |  2 +-
 drivers/irqchip/irq-gic-v3-its.c |  7 ++++++-
 include/linux/acpi.h             | 15 +++++++--------
 7 files changed, 21 insertions(+), 19 deletions(-)
#8 lore raw
Quoted 57 lines · On Thu, 3 Sep 2020 13:33:18 +0100 Jonathan Cameron…
Quoted 43 lines · On Tue, 18 Aug 2020 22:24:24 +0800 Jonathan Cameron…

This is a trivial rebase and resend of V2 now the merge window has closed.

Here, I will use the term Proximity Domains for the ACPI description and NUMA Nodes for the in kernel representation.

ACPI 6.3 included a clarification that only Static Resource Allocation Structures in SRAT may define the existence of proximity domains (sec 5.2.16). This clarification closed a possible interpretation that other parts of ACPI (e.g. DSDT _PXM, NFIT etc) could define new proximity domains that were not also mentioned in SRAT structures.

In practice the kernel has never allowed this alternative interpretation as such nodes are only partially initialized. This is architecture specific but to take an example, on x86 alloc_node_data has not been called. Any use of them for node specific allocation, will result in a crash as the infrastructure to fallback to a node with memory is not setup.

We ran into a problem when enabling _PXM handling for PCI devices and found there were boards out there advertising devices in proximity domains that didn't exist [2].

The fix suggested in this series is to replace instances that should not 'create' new nodes with pxm_to_node. This function needs a some additional hardening against invalid inputs to make sure it is safe for use in these new callers.

Patch 1 Hardens pxm_to_node() against numa_off, and pxm entry being too large.

Patch 2-4 change the various callers not related to SRAT entries so that they set this parameter to false, so do not attempt to initialize a new NUMA node if the relevant one does not already exist.

Patch 5 is a function rename to reflect change in functionality of acpi_map_pxm_to_online_node() as it no longer creates a new map, but just does a lookup of existing maps.

Patch 6 covers the one place we do not allow the full flexibility defined in the ACPI spec. For SRAT GIC Interrupt Translation Service (ITS) Affinity Structures, on ARM64, the driver currently makes an additional pass of SRAT later in the boot than the one used to identify NUMA domains. Note, this currently means that an ITS placed in a proximity domain that is not defined by another SRAT structure will result in the a crash.

Quoted 6 lines

To avoid this crash with minimal changes we do not create new NUMA nodes based on this particular entry type. Any current platform trying to do this will not boot, so this is an improvement, if perhaps not a perfect solution.

[1] Note in ACPI Specification 6.3 5.2.16 System Resource Affinity Table (SRAT)
[2] https://patchwork.kernel.org/patch/10597777/

Looking for input from ARM (Lorenzo?), X86(not sure) and ACPI(Rafael?) people on this set. As it also touches NFIT handling perhaps someone who focuses on that as well. Please feel free to CC additional people who might be interested.

Quoted 6 lines

I'm fairly confident that it should be uncontroversial (famous last words) and it closes down a problem that lead to issues with seemingly obvious changes in the past. (The whole PCI _PXM issue on some threadripper platforms).

Thanks to Bjorn, Hanjun and Barry for feedback on earlier revisions.

Hi All,

Still after reviews on this set. If anyone one would like me to resend please let me know. Also any corrections to my suggested set of people to review would be welcome.

+CC Rafael who I've managed to drop from this version which won't have helped.

Thanks,

Jonathan

Quoted 46 lines

Thanks,

Jonathan

Quoted 27 lines

Thanks to Bjorn Helgaas for review of v1 and Barry Song for internal reviews that lead to a slightly different approach for this v2.

Changes since v2.
* Trivial rebase to v5.9-rc1
* Collect up tags.

Changes since v1.
* Use pxm_to_node for what was previously the path using acpi_map_pxm_to_node with create=false. (Barry)
* Broke patch up into an initial noop stage followed by patches (Bjorn) to update each type of case in which partial creation of NUMA nodes is prevented.
* Added patch 5 to rename function to reflect change of functionality.
* Updated descriptions (now mostly in individual patches) inline with Bjorn's comments.

Jonathan Cameron (6):
ACPI: Add out of bounds and numa_off protections to pxm_to_node
ACPI: Do not create new NUMA domains from ACPI static tables that are
not SRAT
ACPI: Remove side effect of partly creating a node in
acpi_map_pxm_to_online_node
ACPI: Rename acpi_map_pxm_to_online_node to pxm_to_online_node
ACPI: Remove side effect of partly creating a node in acpi_get_node irq-chip/gic-v3-its: Fix crash if ITS is in a proximity domain without
processor or memory

Quoted 9 lines
 drivers/acpi/arm64/iort.c        |  2 +-
 drivers/acpi/nfit/core.c         |  6 ++----
 drivers/acpi/numa/hmat.c         |  4 ++--
 drivers/acpi/numa/srat.c         |  4 ++--
 drivers/iommu/intel/dmar.c       |  2 +-
 drivers/irqchip/irq-gic-v3-its.c |  7 ++++++-
 include/linux/acpi.h             | 15 +++++++--------
 7 files changed, 21 insertions(+), 19 deletions(-)

_______________________________________________
Linuxarm mailing list
[email protected]
http://hulk.huawei.com/mailman/listinfo/linuxarm

#9 lore raw
Quoted 68 lines · Jonathan Cameron
Quoted 57 lines · On Thu, 3 Sep 2020 13:33:18 +0100 Jonathan Cameron…
Quoted 43 lines · On Tue, 18 Aug 2020 22:24:24 +0800 Jonathan Cameron…

This is a trivial rebase and resend of V2 now the merge window has closed.

Here, I will use the term Proximity Domains for the ACPI description and NUMA Nodes for the in kernel representation.

ACPI 6.3 included a clarification that only Static Resource Allocation Structures in SRAT may define the existence of proximity domains (sec 5.2.16). This clarification closed a possible interpretation that other parts of ACPI (e.g. DSDT _PXM, NFIT etc) could define new proximity domains that were not also mentioned in SRAT structures.

In practice the kernel has never allowed this alternative interpretation as such nodes are only partially initialized. This is architecture specific but to take an example, on x86 alloc_node_data has not been called. Any use of them for node specific allocation, will result in a crash as the infrastructure to fallback to a node with memory is not setup.

We ran into a problem when enabling _PXM handling for PCI devices and found there were boards out there advertising devices in proximity domains that didn't exist [2].

The fix suggested in this series is to replace instances that should not 'create' new nodes with pxm_to_node. This function needs a some additional hardening against invalid inputs to make sure it is safe for use in these new callers.

Patch 1 Hardens pxm_to_node() against numa_off, and pxm entry being too large.

Patch 2-4 change the various callers not related to SRAT entries so that they set this parameter to false, so do not attempt to initialize a new NUMA node if the relevant one does not already exist.

Patch 5 is a function rename to reflect change in functionality of acpi_map_pxm_to_online_node() as it no longer creates a new map, but just does a lookup of existing maps.

Patch 6 covers the one place we do not allow the full flexibility defined in the ACPI spec. For SRAT GIC Interrupt Translation Service (ITS) Affinity Structures, on ARM64, the driver currently makes an additional pass of SRAT later in the boot than the one used to identify NUMA domains. Note, this currently means that an ITS placed in a proximity domain that is not defined by another SRAT structure will result in the a crash.

Quoted 6 lines

To avoid this crash with minimal changes we do not create new NUMA nodes based on this particular entry type. Any current platform trying to do this will not boot, so this is an improvement, if perhaps not a perfect solution.

[1] Note in ACPI Specification 6.3 5.2.16 System Resource Affinity Table (SRAT)
[2] https://patchwork.kernel.org/patch/10597777/

Looking for input from ARM (Lorenzo?), X86(not sure) and ACPI(Rafael?) people on this set. As it also touches NFIT handling perhaps someone who focuses on that as well. Please feel free to CC additional people who might be interested.

Quoted 6 lines

I'm fairly confident that it should be uncontroversial (famous last words) and it closes down a problem that lead to issues with seemingly obvious changes in the past. (The whole PCI _PXM issue on some threadripper platforms).

Thanks to Bjorn, Hanjun and Barry for feedback on earlier revisions.

Quoted 7 lines

Hi All,

Still after reviews on this set. If anyone one would like me to resend please let me know. Also any corrections to my suggested set of people to review would be welcome.

+CC Rafael who I've managed to drop from this version which won't have helped.

Thanks!

I've just applied the series for 5.10 (with some minor edits in patch changelogs and subjects), so if anyone has any reservations regarding it, please let me know ASAP.

In particular, I took the Hanjun's R-by on the last patch to be sufficient from the ARM64 perspective.

Cheers!

Quoted 43 lines
Quoted 41 lines
Quoted 26 lines

Thanks to Bjorn Helgaas for review of v1 and Barry Song for internal reviews that lead to a slightly different approach for this v2.

Changes since v2.
* Trivial rebase to v5.9-rc1
* Collect up tags.

Changes since v1.
* Use pxm_to_node for what was previously the path using acpi_map_pxm_to_node with create=false. (Barry)
* Broke patch up into an initial noop stage followed by patches (Bjorn) to update each type of case in which partial creation of NUMA nodes is prevented.
* Added patch 5 to rename function to reflect change of functionality.
* Updated descriptions (now mostly in individual patches) inline with Bjorn's comments.

Jonathan Cameron (6):
ACPI: Add out of bounds and numa_off protections to pxm_to_node
ACPI: Do not create new NUMA domains from ACPI static tables that are
not SRAT
ACPI: Remove side effect of partly creating a node in
acpi_map_pxm_to_online_node
ACPI: Rename acpi_map_pxm_to_online_node to pxm_to_online_node
ACPI: Remove side effect of partly creating a node in acpi_get_node irq-chip/gic-v3-its: Fix crash if ITS is in a proximity domain without
processor or memory

Quoted 9 lines
 drivers/acpi/arm64/iort.c        |  2 +-
 drivers/acpi/nfit/core.c         |  6 ++----
 drivers/acpi/numa/hmat.c         |  4 ++--
 drivers/acpi/numa/srat.c         |  4 ++--
 drivers/iommu/intel/dmar.c       |  2 +-
 drivers/irqchip/irq-gic-v3-its.c |  7 ++++++-
 include/linux/acpi.h             | 15 +++++++--------
 7 files changed, 21 insertions(+), 19 deletions(-)

_______________________________________________
Linuxarm mailing list
[email protected]
http://hulk.huawei.com/mailman/listinfo/linuxarm