Skip to contentSkip to conversation

mm/fake-numa: fix under-allocation detection in uniform split

PATCH In review Sang-Heon Jeon 1 participant 2 messages Apr 17

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/fake-numa: fix under-allocation detection in uniform split

When split NUMA node uniformly, split_nodes_size_interleave_uniform() returns the next absolute node ID, not the number of nodes created.

The existing under-allocation detection logic compares next absolute node ID (ret) and request count (n), which only works when nid starts at 0.

For example, on a system with 2 physical NUMA nodes (node 0: 2GB, node 1: 128MB) and numa=fake=8U, 8 fake nodes are successfully created from node 0 and split_nodes_size_interleave_uniform() returns 8. For node 1, fake node nid starts at 8, but only 4 fake nodes are created due to current FAKE_NODE_MIN_SIZE being 32MB, and split_nodes_size_interleave_uniform() returns 12. By existing under-allocation detection logic, "ret < n" (12 < 8) is false, so the under-allocation will not be detected.

Fix under-allocation detection logic to compare the number of actually created nodes (ret - nid) against the request count (n).

Also, fix the outdated comment to match the actual return value.

Fixes cc9aec03e58f ("x86/numa_emulation: Introduce uniform split capability") # 4.19

---
Changes from RFC v1 [1]
- Merge patchset into once.
- Change base from linux-next to mm-unstable

Changes from RFC v2 [2]
- Fix error message to use the number of created node instead of returned node ID.
- Define nr_created variable to explicitly show the number of created noeds.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
---
QEMU-based test results for the commit message scenario.

1) AS-IS (before fix)

[    0.001878] NUMA: Node 0 [mem 0x00001000-0x0009ffff] + [mem 0x00100000-0x7fffffff] ]
[    0.001881] Fake node size 255MB too small, increasing to 256MB
[    0.001882] Faking node 0 at [mem 0x0000000000001000-0x0000000010000fff] (256MB)
[    0.001883] Faking node 1 at [mem 0x0000000010001000-0x0000000020000fff] (256MB)
[    0.001883] Faking node 2 at [mem 0x0000000020001000-0x0000000030000fff] (256MB)
[    0.001884] Faking node 3 at [mem 0x0000000030001000-0x0000000040000fff] (256MB)
[    0.001884] Faking node 4 at [mem 0x0000000040001000-0x0000000050000fff] (256MB)
[    0.001884] Faking node 5 at [mem 0x0000000050001000-0x0000000060000fff] (256MB)
[    0.001885] Faking node 6 at [mem 0x0000000060001000-0x0000000070000fff] (256MB)
[    0.001885] Faking node 7 at [mem 0x0000000070001000-0x000000007fffffff] (255MB)
[    0.001885] Fake node size 15MB too small, increasing to 32MB
[    0.001886] Faking node 8 at [mem 0x0000000080000000-0x0000000081ffffff] (32MB)
[    0.001886] Faking node 9 at [mem 0x0000000082000000-0x0000000083ffffff] (32MB)
[    0.001887] Faking node 10 at [mem 0x0000000084000000-0x0000000087fdcfff] (63MB)
[    0.001924] NODE_DATA(0) allocated [mem 0x0fffd6c0-0x10000fff]
[    0.019852] NODE_DATA(1) allocated [mem 0x1fffd6c0-0x20000fff]
[    0.022458] NODE_DATA(2) allocated [mem 0x2dffc6c0-0x2dffffff]
[    0.023293] NODE_DATA(3) allocated [mem 0x3fffd6c0-0x40000fff]
[    0.028522] NODE_DATA(4) allocated [mem 0x4fffd6c0-0x50000fff]
[    0.032397] NODE_DATA(5) allocated [mem 0x5fffd6c0-0x60000fff]
[    0.036552] NODE_DATA(6) allocated [mem 0x6fffd6c0-0x70000fff]
[    0.038746] NODE_DATA(7) allocated [mem 0x7fffc6c0-0x7fffffff]
[    0.040286] NODE_DATA(8) allocated [mem 0x81ffc6c0-0x81ffffff]
[    0.041517] NODE_DATA(9) allocated [mem 0x83ffc6c0-0x83ffffff]
[    0.043678] NODE_DATA(10) allocated [mem 0x87fd86c0-0x87fdbfff]

2) TO-BE (after fix)

[    0.001858] NUMA: Node 0 [mem 0x00001000-0x0009ffff] + [mem 0x00100000-0x7fffffff] ]
[    0.001860] Fake node size 255MB too small, increasing to 256MB
[    0.001861] Faking node 0 at [mem 0x0000000000001000-0x0000000010000fff] (256MB)
[    0.001861] Faking node 1 at [mem 0x0000000010001000-0x0000000020000fff] (256MB)
[    0.001862] Faking node 2 at [mem 0x0000000020001000-0x0000000030000fff] (256MB)
[    0.001862] Faking node 3 at [mem 0x0000000030001000-0x0000000040000fff] (256MB)
[    0.001863] Faking node 4 at [mem 0x0000000040001000-0x0000000050000fff] (256MB)
[    0.001863] Faking node 5 at [mem 0x0000000050001000-0x0000000060000fff] (256MB)
[    0.001863] Faking node 6 at [mem 0x0000000060001000-0x0000000070000fff] (256MB)
[    0.001864] Faking node 7 at [mem 0x0000000070001000-0x000000007fffffff] (255MB)
[    0.001864] Fake node size 15MB too small, increasing to 32MB
[    0.001864] Faking node 8 at [mem 0x0000000080000000-0x0000000081ffffff] (32MB)
[    0.001865] Faking node 9 at [mem 0x0000000082000000-0x0000000083ffffff] (32MB)
[    0.001865] Faking node 10 at [mem 0x0000000084000000-0x0000000087fdcfff] (63MB)
[    0.001866] numa_emulation: phys: 1 only got 3 of 8 nodes, failing
[    0.001867] NODE_DATA(0) allocated [mem 0x7fffc6c0-0x7fffffff]
[    0.001940] NODE_DATA(1) allocated [mem 0x87fd96c0-0x87fdcfff]

And also there is other scenario tested by Donghyeon. [1]

[1] https://lore.kernel.org/all/CAFPTC5e1OLpHa3HqwhtSPjS_PTQz+iG=ovM2cZ=VnOZ_5z7oxg@mail.gmail.com/

---
 mm/numa_emulation.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
mm/numa_emulation.c +6 -4
index 703c8fa05048..bc2f163e7c45 100644
@@ -214,7 +214,7 @@ static u64 uniform_size(u64 max_addr, u64 base, u64 hole, int nr_nodes)
  * Sets up fake nodes of `size' interleaved over physical nodes ranging from
  * `addr' to `max_addr'.
  *
- * Returns zero on success or negative on error.
+ * Returns absolute node ID on success or negative on error.
  */
 static int __init split_nodes_size_interleave_uniform(struct numa_meminfo *ei,
 					      struct numa_meminfo *pi,
@@ -398,7 +398,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
 	 */
 	if (strchr(emu_cmdline, 'U')) {
 		unsigned long n;
-		int nid = 0;
+		int nid = 0, nr_created;
 
 		n = simple_strtoul(emu_cmdline, &emu_cmdline, 0);
 		ret = -1;
@@ -416,9 +416,11 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
 					n, &pi.blk[0], nid);
 			if (ret < 0)
 				break;
-			if (ret < n) {
+
+			nr_created = ret - nid;
+			if (nr_created < n) {
 				pr_info("%s: phys: %d only got %d of %ld nodes, failing\n",
-						__func__, i, ret, n);
+						__func__, i, nr_created, n);
 				ret = -1;
 				break;
 			}
Signature · 1 line
2.43.0
#1 lore raw
Quoted 118 lines · Sang-Heon Jeon

When split NUMA node uniformly, split_nodes_size_interleave_uniform() returns the next absolute node ID, not the number of nodes created.

The existing under-allocation detection logic compares next absolute node ID (ret) and request count (n), which only works when nid starts at 0.

For example, on a system with 2 physical NUMA nodes (node 0: 2GB, node 1: 128MB) and numa=fake=8U, 8 fake nodes are successfully created from node 0 and split_nodes_size_interleave_uniform() returns 8. For node 1, fake node nid starts at 8, but only 4 fake nodes are created due to current FAKE_NODE_MIN_SIZE being 32MB, and split_nodes_size_interleave_uniform() returns 12. By existing under-allocation detection logic, "ret < n" (12 < 8) is false, so the under-allocation will not be detected.

Fix under-allocation detection logic to compare the number of actually created nodes (ret - nid) against the request count (n).

Also, fix the outdated comment to match the actual return value.

Fixes cc9aec03e58f ("x86/numa_emulation: Introduce uniform split capability") # 4.19

---
Changes from RFC v1 [1]
- Merge patchset into once.
- Change base from linux-next to mm-unstable

Changes from RFC v2 [2]
- Fix error message to use the number of created node instead of returned node ID.
- Define nr_created variable to explicitly show the number of created noeds.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
---
QEMU-based test results for the commit message scenario.

1) AS-IS (before fix)

[    0.001878] NUMA: Node 0 [mem 0x00001000-0x0009ffff] + [mem 0x00100000-0x7fffffff] ]
[    0.001881] Fake node size 255MB too small, increasing to 256MB
[    0.001882] Faking node 0 at [mem 0x0000000000001000-0x0000000010000fff] (256MB)
[    0.001883] Faking node 1 at [mem 0x0000000010001000-0x0000000020000fff] (256MB)
[    0.001883] Faking node 2 at [mem 0x0000000020001000-0x0000000030000fff] (256MB)
[    0.001884] Faking node 3 at [mem 0x0000000030001000-0x0000000040000fff] (256MB)
[    0.001884] Faking node 4 at [mem 0x0000000040001000-0x0000000050000fff] (256MB)
[    0.001884] Faking node 5 at [mem 0x0000000050001000-0x0000000060000fff] (256MB)
[    0.001885] Faking node 6 at [mem 0x0000000060001000-0x0000000070000fff] (256MB)
[    0.001885] Faking node 7 at [mem 0x0000000070001000-0x000000007fffffff] (255MB)
[    0.001885] Fake node size 15MB too small, increasing to 32MB
[    0.001886] Faking node 8 at [mem 0x0000000080000000-0x0000000081ffffff] (32MB)
[    0.001886] Faking node 9 at [mem 0x0000000082000000-0x0000000083ffffff] (32MB)
[    0.001887] Faking node 10 at [mem 0x0000000084000000-0x0000000087fdcfff] (63MB)
[    0.001924] NODE_DATA(0) allocated [mem 0x0fffd6c0-0x10000fff]
[    0.019852] NODE_DATA(1) allocated [mem 0x1fffd6c0-0x20000fff]
[    0.022458] NODE_DATA(2) allocated [mem 0x2dffc6c0-0x2dffffff]
[    0.023293] NODE_DATA(3) allocated [mem 0x3fffd6c0-0x40000fff]
[    0.028522] NODE_DATA(4) allocated [mem 0x4fffd6c0-0x50000fff]
[    0.032397] NODE_DATA(5) allocated [mem 0x5fffd6c0-0x60000fff]
[    0.036552] NODE_DATA(6) allocated [mem 0x6fffd6c0-0x70000fff]
[    0.038746] NODE_DATA(7) allocated [mem 0x7fffc6c0-0x7fffffff]
[    0.040286] NODE_DATA(8) allocated [mem 0x81ffc6c0-0x81ffffff]
[    0.041517] NODE_DATA(9) allocated [mem 0x83ffc6c0-0x83ffffff]
[    0.043678] NODE_DATA(10) allocated [mem 0x87fd86c0-0x87fdbfff]

2) TO-BE (after fix)

[    0.001858] NUMA: Node 0 [mem 0x00001000-0x0009ffff] + [mem 0x00100000-0x7fffffff] ]
[    0.001860] Fake node size 255MB too small, increasing to 256MB
[    0.001861] Faking node 0 at [mem 0x0000000000001000-0x0000000010000fff] (256MB)
[    0.001861] Faking node 1 at [mem 0x0000000010001000-0x0000000020000fff] (256MB)
[    0.001862] Faking node 2 at [mem 0x0000000020001000-0x0000000030000fff] (256MB)
[    0.001862] Faking node 3 at [mem 0x0000000030001000-0x0000000040000fff] (256MB)
[    0.001863] Faking node 4 at [mem 0x0000000040001000-0x0000000050000fff] (256MB)
[    0.001863] Faking node 5 at [mem 0x0000000050001000-0x0000000060000fff] (256MB)
[    0.001863] Faking node 6 at [mem 0x0000000060001000-0x0000000070000fff] (256MB)
[    0.001864] Faking node 7 at [mem 0x0000000070001000-0x000000007fffffff] (255MB)
[    0.001864] Fake node size 15MB too small, increasing to 32MB
[    0.001864] Faking node 8 at [mem 0x0000000080000000-0x0000000081ffffff] (32MB)
[    0.001865] Faking node 9 at [mem 0x0000000082000000-0x0000000083ffffff] (32MB)
[    0.001865] Faking node 10 at [mem 0x0000000084000000-0x0000000087fdcfff] (63MB)
[    0.001866] numa_emulation: phys: 1 only got 3 of 8 nodes, failing
[    0.001867] NODE_DATA(0) allocated [mem 0x7fffc6c0-0x7fffffff]
[    0.001940] NODE_DATA(1) allocated [mem 0x87fd96c0-0x87fdcfff]

And also there is other scenario tested by Donghyeon. [1]

[1] https://lore.kernel.org/all/CAFPTC5e1OLpHa3HqwhtSPjS_PTQz+iG=ovM2cZ=VnOZ_5z7oxg@mail.gmail.com/

---
 mm/numa_emulation.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
mm/numa_emulation.c +2 -2
index 703c8fa05048..bc2f163e7c45 100644
@@ -214,7 +214,7 @@ static u64 uniform_size(u64 max_addr, u64 base, u64 hole, int nr_nodes)
  * Sets up fake nodes of `size' interleaved over physical nodes ranging from
  * `addr' to `max_addr'.
  *
- * Returns zero on success or negative on error.
+ * Returns absolute node ID on success or negative on error.
  */
 static int __init split_nodes_size_interleave_uniform(struct numa_meminfo *ei,
                                              struct numa_meminfo *pi,
@@ -398,7 +398,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
         */
        if (strchr(emu_cmdline, 'U')) {
                unsigned long n;
-               int nid = 0;
+               int nid = 0, nr_created;
 
                n = simple_strtoul(emu_cmdline, &emu_cmdline, 0);
                ret = -1;

mm/numa_emulation.c void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt) · line 421

Quoted 7 lines
mm/numa_emulation.c +3 -1
@@ -416,9 +416,11 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
                                        n, &pi.blk[0], nid);
                        if (ret < 0)
                                break;
-                       if (ret < n) {
+
+                       nr_created = ret - nid;
+                       if (nr_created < n) {

sashiko [1] found a false failure point with memoryless nodes, I'll fix it and send v2 patch soon.

[1] https://sashiko.dev/#/patchset/20260417114127.1664283-1-ekffu200098%40gmail.com

Quoted 9 lines
mm/numa_emulation.c +1 -2
                                pr_info("%s: phys: %d only got %d of %ld nodes, failing\n",
-                                               __func__, i, ret, n);
+                                               __func__, i, nr_created, n);
                                ret = -1;
                                break;
                        }
--

2.43.0

Best Regards,
Sang-Heon Jeon