Skip to contentSkip to conversation

of_numa: fix return -EINVAL when numa-node-id is not found in last node

PATCH In review Sang-Heon Jeon 2 participants 5 messages Jun 21 – Jun 23 Device tree

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 of_numa: fix return -EINVAL when numa-node-id is not found in last node

When the numa-node-id property is not found in the last memory node, of_property_read_u32() returns -EINVAL, which then becomes the return value of of_numa_parse_memory_nodes(), even though earlier memory nodes were parsed successfully.

Commit 7e488677a54a ("of, numa: return -EINVAL when no numa-node-id is found") meant -EINVAL to be returned only when the numa-node-id property is not found at all, not when it is found in an earlier memory node but not in the last.

Check whether at least one memory node was parsed successfully, and return 0 in that case, -EINVAL otherwise, so the return value of of_property_read_u32() for the last memory node no longer corrupts the overall return value.

Also include other minor changes for readability improvement with no functional change.

Fixes 7e488677a54a ("of, numa: return -EINVAL when no numa-node-id is found")

---
QEMU-based test results

DTB memory nodes defined in DT
- memory@80000000 (0x80000000-0x9fffffff) numa-node-id = <0>
- memory@c0000000 (0xc0000000-0xdfffffff) numa-node-id = <0>
- memory@e0000000 (0xe0000000-0xffffffff) numa-node-id = <1>
- memory@a0000000 (0xa0000000-0xbfffffff) (no numa-node-id)

1) AS-IS (before fix)

[    0.000000] NUMA: Faking a node at [mem 0x0000000080000000-0x00000000ffffffff]
[    0.069152] futex hash table entries: 512 (32768 bytes on 1 NUMA nodes, total 32 KiB, linear).

...

2) TO-BE (after fix)

[    0.000000] NUMA: Node 0 [mem 0x80000000-0x9fffffff] + [mem 0xc0000000-0xdfffffff] -> [mem 0x80000000-0xdfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000dfffffff]
[    0.000000] Initmem setup node 1 [mem 0x00000000e0000000-0x00000000ffffffff]
[    0.076854] futex hash table entries: 256 (16384 bytes on 2 NUMA nodes, total 32 KiB, linear).

...

Tested the scenarios below to confirm no regression, and all produced the same result on AS-IS and TO-BE:
- all / no memory nodes tagged
- untagged memory node exist, but not last
- empty / out-of-range numa-node-id
- memory node without reg

---
 drivers/of/of_numa.c | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)
drivers/of/of_numa.c +18 -11
index cd2dc8e825c9..13f2f6d238c9 100644
@@ -42,7 +42,8 @@ static int __init of_numa_parse_memory_nodes(void)
 	struct device_node *np = NULL;
 	struct resource rsrc;
 	u32 nid;
-	int i, r = -EINVAL;
+	int i, r;
+	bool found = false;
 
 	for_each_node_by_type(np, "memory") {
 		r = of_property_read_u32(np, "numa-node-id", &nid);
@@ -53,26 +54,32 @@ static int __init of_numa_parse_memory_nodes(void)
 			 * "numa-node-id" property
 			 */
 			continue;
+		if (r)
+			goto err;
 
 		if (nid >= MAX_NUMNODES) {
 			pr_warn("Node id %u exceeds maximum value\n", nid);
-			r = -EINVAL;
+			goto err;
 		}
 
-		for (i = 0; !r && !of_address_to_resource(np, i, &rsrc); i++) {
+		for (i = 0; !of_address_to_resource(np, i, &rsrc); i++) {
 			r = numa_add_memblk(nid, rsrc.start, rsrc.end + 1);
-			if (!r)
-				node_set(nid, numa_nodes_parsed);
+			if (r)
+				goto err;
+			node_set(nid, numa_nodes_parsed);
 		}
+		if (!i)
+			goto err;
 
-		if (!i || r) {
-			of_node_put(np);
-			pr_err("bad property in memory node\n");
-			return r ? : -EINVAL;
-		}
+		found = true;
 	}
 
-	return r;
+	return found ? 0 : -EINVAL;
+
+err:
+	of_node_put(np);
+	pr_err("bad property in memory node\n");
+	return r ?: -EINVAL;
 }
 
 static int __init of_numa_parse_distance_map_v1(struct device_node *map)
Signature · 1 line
2.43.0
maintainer
#1 lore raw
Quoted 10 lines · Sang-Heon Jeon

When the numa-node-id property is not found in the last memory node, of_property_read_u32() returns -EINVAL, which then becomes the return value of of_numa_parse_memory_nodes(), even though earlier memory nodes were parsed successfully.

Commit 7e488677a54a ("of, numa: return -EINVAL when no numa-node-id is found") meant -EINVAL to be returned only when the numa-node-id property is not found at all, not when it is found in an earlier memory node but not in the last.

Quoted 4 lines

Check whether at least one memory node was parsed successfully, and return 0 in that case, -EINVAL otherwise, so the return value of of_property_read_u32() for the last memory node no longer corrupts the overall return value.

IDK, it's arguable that an incomplete DT isn't valid and something we need to support. Is missing numa-node-id valid or it's just better to have at least partially
configured NUMA nodes?

Rob

#2 lore raw
Quoted 17 lines · Rob Herring
Quoted 10 lines · Sang-Heon Jeon

When the numa-node-id property is not found in the last memory node, of_property_read_u32() returns -EINVAL, which then becomes the return value of of_numa_parse_memory_nodes(), even though earlier memory nodes were parsed successfully.

Commit 7e488677a54a ("of, numa: return -EINVAL when no numa-node-id is found") meant -EINVAL to be returned only when the numa-node-id property is not found at all, not when it is found in an earlier memory node but not in the last.

Quoted 4 lines

Check whether at least one memory node was parsed successfully, and return 0 in that case, -EINVAL otherwise, so the return value of of_property_read_u32() for the last memory node no longer corrupts the overall return value.

Quoted 4 lines

IDK, it's arguable that an incomplete DT isn't valid and something we need to support. Is missing numa-node-id valid or it's just better to have at least partially
configured NUMA nodes?

You're right.

Since the first implementation 298535c00a2c ("of, numa: Add NUMA of binding implementation.") skips memory nodes without numa-node-id, I only try to fix the error case introduced by 7e488677a54a ("of,
numa: return -EINVAL when no numa-node-id is found").

IMHO, if changing the logic is acceptable, rejecting DT with an incomplete NUMA configuration also seems reasonable.
We already call memblock_validate_numa_coverage() after of_numa_parse_memory_nodes(),
so such an incomplete configuration would most likely fail from that validation anyway.

So, would it be OK to reject a DT with an incomplete NUMA configuration? If so, I'll make a patch in that direction instead.

Quoted 1 line

Rob

Best Regards,
Sang-Heon Jeon

maintainer
#3 lore raw
Quoted 38 lines · Sang-Heon Jeon
Quoted 17 lines · Rob Herring
Quoted 10 lines · Sang-Heon Jeon

When the numa-node-id property is not found in the last memory node, of_property_read_u32() returns -EINVAL, which then becomes the return value of of_numa_parse_memory_nodes(), even though earlier memory nodes were parsed successfully.

Commit 7e488677a54a ("of, numa: return -EINVAL when no numa-node-id is found") meant -EINVAL to be returned only when the numa-node-id property is not found at all, not when it is found in an earlier memory node but not in the last.

Quoted 4 lines

Check whether at least one memory node was parsed successfully, and return 0 in that case, -EINVAL otherwise, so the return value of of_property_read_u32() for the last memory node no longer corrupts the overall return value.

Quoted 4 lines

IDK, it's arguable that an incomplete DT isn't valid and something we need to support. Is missing numa-node-id valid or it's just better to have at least partially
configured NUMA nodes?

You're right.

Since the first implementation 298535c00a2c ("of, numa: Add NUMA of binding implementation.") skips memory nodes without numa-node-id, I only try to fix the error case introduced by 7e488677a54a ("of,
numa: return -EINVAL when no numa-node-id is found").

IMHO, if changing the logic is acceptable, rejecting DT with an incomplete NUMA configuration also seems reasonable.
We already call memblock_validate_numa_coverage() after of_numa_parse_memory_nodes(),
so such an incomplete configuration would most likely fail from that validation anyway.

Quoted 2 lines

So, would it be OK to reject a DT with an incomplete NUMA configuration? If so, I'll make a patch in that direction instead.

Reject? As in panic()? No, because why would we make a system that boots not boot and a panic at that stage won't be seen because your console is not up yet. If you want to add a warning that the NUMA information is incomplete, that would be fine.

Also, are we sure it's not valid to have some memory associated with a numa node and some that is not?

Rob

#4 lore raw
Quoted 40 lines · Rob Herring
Quoted 38 lines · Sang-Heon Jeon
Quoted 17 lines · Rob Herring
Quoted 10 lines · Sang-Heon Jeon

When the numa-node-id property is not found in the last memory node, of_property_read_u32() returns -EINVAL, which then becomes the return value of of_numa_parse_memory_nodes(), even though earlier memory nodes were parsed successfully.

Commit 7e488677a54a ("of, numa: return -EINVAL when no numa-node-id is found") meant -EINVAL to be returned only when the numa-node-id property is not found at all, not when it is found in an earlier memory node but not in the last.

Quoted 4 lines

Check whether at least one memory node was parsed successfully, and return 0 in that case, -EINVAL otherwise, so the return value of of_property_read_u32() for the last memory node no longer corrupts the overall return value.

Quoted 4 lines

IDK, it's arguable that an incomplete DT isn't valid and something we need to support. Is missing numa-node-id valid or it's just better to have at least partially
configured NUMA nodes?

You're right.

Since the first implementation 298535c00a2c ("of, numa: Add NUMA of binding implementation.") skips memory nodes without numa-node-id, I only try to fix the error case introduced by 7e488677a54a ("of,
numa: return -EINVAL when no numa-node-id is found").

IMHO, if changing the logic is acceptable, rejecting DT with an incomplete NUMA configuration also seems reasonable.
We already call memblock_validate_numa_coverage() after of_numa_parse_memory_nodes(),
so such an incomplete configuration would most likely fail from that validation anyway.

Quoted 7 lines
Quoted 2 lines

So, would it be OK to reject a DT with an incomplete NUMA configuration? If so, I'll make a patch in that direction instead.

Reject? As in panic()? No, because why would we make a system that boots not boot and a panic at that stage won't be seen because your console is not up yet. If you want to add a warning that the NUMA information is incomplete, that would be fine.

Sorry for the confusion. Previous "reject" meant returning a non-zero value from of_numa_parse_memory_nodes().
When of_numa_parse_memory_nodes() returns non-zero, NUMA initialization from DT is considered as failed and dummy_numa_init() is called,
which leaves the system with a single node covering all memory.

In conclusion, the direction that i meant is returning -EINVAL instead of skipping no numa-node-id node,

Quoted 2 lines

Also, are we sure it's not valid to have some memory associated with a numa node and some that is not?

As I described above, we already call memblock_validate_numa_coverage() after of_numa_parse_memory_nodes() success.
The threshold bytes of no node assigned memory is 0(arm64 / riscv) or 1MB(x86, introduced for e820/SRAT mismatch, but shared with DTB path as well),
so in practice existing logic already doesn't allow no node assigned memory, which corresponds to a missing numa-node-id in DT.

So as far as I know, the only case that passes memblock_validate_numa_coverage() with a DT missing numa-node-id is when numa_cleanup_meminfo() join two blocks of the same node (same scenario as QEMU test case). An unassigned region exists between two blocks of the same node, and no other node's memory exists in it.

So, in my opinion, at least for now, partial existence of numa-node-id DT is not valid.
But maybe, somewhere, someone might want to skip numa-node-id for the memory in the middle of the same node. So I can't be 100% sure.

Since neither of us is sure whether such a DT is valid, I'd rather keep the current patch's direction because it doesn't change existing policy.

Quoted 1 line

Rob

And always, thanks for taking the time to review.

Best Regards,
Sang-Heon Jeon