Skip to contentSkip to conversation

powerpc/papr_scm: Update the NUMA distance table for the target node

PATCH Applied Aneesh Kumar K.V 2 participants 2 messages Apr 4, 2023 – Apr 6, 2023 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 powerpc/papr_scm: Update the NUMA distance table for the target node

platform device helper routines won't update the NUMA distance table while creating a platform device, even if the device is present on a NUMA node that doesn't have memory or CPU. This is especially true for pmem devices. If the target node of the pmem device is not online, we find the nearest online node to the device and associate the pmem device with that online node. To find the nearest online node, we should have the numa distance table updated correctly. Update the distance information during the device probe.

For a papr scm device on NUMA node 3 distance_lookup_table value for distance_ref_points_depth = 2 before and after fix is below:

Before fix:

node 3 distance depth 0  - 0
node 3 distance depth 1  - 0
node 4 distance depth 0  - 4
node 4 distance depth 1  - 2
node 5 distance depth 0  - 5
node 5 distance depth 1  - 1

After fix

node 3 distance depth 0  - 3
node 3 distance depth 1  - 1
node 4 distance depth 0  - 4
node 4 distance depth 1  - 2
node 5 distance depth 0  - 5
node 5 distance depth 1  - 1

Without the fix, the nearest numa node to the pmem device(NUMA node 3) will be picked as 4. After the fix, we get the correct numa node which is 5.

Fixes da1115fdbd6e ("powerpc/nvdimm: Pick nearby online node if the device node is not online")

---
Changes from v1:
* Update commit message
* Update code comment

 arch/powerpc/mm/numa.c                    | 1 +
 arch/powerpc/platforms/pseries/papr_scm.c | 7 +++++++
 2 files changed, 8 insertions(+)
arch/powerpc/mm/numa.c +1 -0
index b44ce71917d7..16cfe56be05b 100644
@@ -366,6 +366,7 @@ void update_numa_distance(struct device_node *node)
 	WARN(numa_distance_table[nid][nid] == -1,
 	     "NUMA distance details for node %d not provided\n", nid);
 }
+EXPORT_SYMBOL_GPL(update_numa_distance);
 
 /*
  * ibm,numa-lookup-index-table= {N, domainid1, domainid2, ..... domainidN}
arch/powerpc/platforms/pseries/papr_scm.c +7 -0
index 2f8385523a13..500b2bd5417c 100644
@@ -1428,6 +1428,13 @@ static int papr_scm_probe(struct platform_device *pdev)
 		return -ENODEV;
 	}
 
+	/*
+	 * open firmware  platform device create won't update the
+	 * numa distance table. For papr scm device we use numa_map_to_online_node
+	 * to find the nearest online numa node and that requires corrrect
+	 * distance table information.
+	 */
+	update_numa_distance(dn);
 
 	p = kzalloc(sizeof(*p), GFP_KERNEL);
 	if (!p)
Signature · 1 line
2.39.2
#1 lore raw
Quoted 10 lines · Aneesh Kumar K.V

platform device helper routines won't update the NUMA distance table while creating a platform device, even if the device is present on a NUMA node that doesn't have memory or CPU. This is especially true for pmem devices. If the target node of the pmem device is not online, we find the nearest online node to the device and associate the pmem device with that online node. To find the nearest online node, we should have the numa distance table updated correctly. Update the distance information during the device probe.

[...]

Applied to powerpc/fixes.

[1/1] powerpc/papr_scm: Update the NUMA distance table for the target node

      https://git.kernel.org/powerpc/c/b277fc793daf258877b4c0744b52f69d6e6ba22e

cheers