Skip to contentSkip to conversation

mm/hugetlb_cma: round up per_node before logging it

PATCH Applied Sang-Heon Jeon 2 participants 2 messages Apr 22 – Apr 23

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/hugetlb_cma: round up per_node before logging it

When the user requests a total hugetlb CMA size without per-node specification, hugetlb_cma_reserve() computes per_node from hugetlb_cma_size and the number of nodes that have memory

per_node = DIV_ROUND_UP(hugetlb_cma_size,
nodes_weight(hugetlb_bootmem_nodes));

The reservation loop later computes

size = round_up(min(per_node, hugetlb_cma_size - reserved),
PAGE_SIZE << order);

So the actually reserved per_node size is multiple of (PAGE_SIZE << order), but the logged per_node is not rounded up, so it may be smaller than the actual reserved size.

For example, as the existing comment describes, if a 3 GB area is requested on a machine with 4 NUMA nodes that have memory, 1 GB is allocated on the first three nodes, but the printed log is

hugetlb_cma: reserve 3072 MiB, up to 768 MiB per node

Round per_node up to (PAGE_SIZE << order) before logging so that the printed log always matches the actual reserved size. No functional change to the actual reservation size, as the following case analysis shows

1. remaining (hugetlb_cma_size - reserved) >= rounded per_node
- AS-IS: min() picks unrounded per_node;
round_up() returns rounded per_node
- TO-BE: min() picks rounded per_node;
round_up() returns rounded per_node (no-op)
2. remaining < unrounded per_node
- AS-IS: min() picks remaining;
round_up() returns round_up(remaining)
- TO-BE: min() picks remaining;
round_up() returns round_up(remaining)
3. unrounded per_node <= remaining < rounded per_node
- AS-IS: min() picks unrounded per_node;
round_up() returns rounded per_node
- TO-BE: min() picks remaining;
round_up() returns round_up(remaining) equals rounded per_node

Fixes cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma") # 5.7

---
Changes from RFC v1 [1]

- Fix missing semicolon
- Add Fixes tag
- Remove RFC tag

[1] https://lore.kernel.org/all/[email protected]/
---
QEMU-based test results

- arm64, 4GB per 4 NUMA nodes
- cmdline: hugetlb_cma=3G

1) AS-IS (beforce fix)

[    0.000000] psci: SMC Calling Convention v1.0
[    0.000000] hugetlb_cma: reserve 3072 MiB, up to 768 MiB per node
[    0.000000] cma: Reserved 1024 MiB in 1 range
[    0.000000] hugetlb_cma: reserved 1024 MiB on node 0
[    0.000000] cma: Reserved 1024 MiB in 1 range
[    0.000000] hugetlb_cma: reserved 1024 MiB on node 1
[    0.000000] cma: Reserved 1024 MiB in 1 range
[    0.000000] hugetlb_cma: reserved 1024 MiB on node 2
[    0.000000] Zone ranges:

2) TO-BE (after fix)

[    0.000000] psci: SMC Calling Convention v1.0
[    0.000000] hugetlb_cma: reserve 3072 MiB, up to 1024 MiB per node
[    0.000000] cma: Reserved 1024 MiB in 1 range
[    0.000000] hugetlb_cma: reserved 1024 MiB on node 0
[    0.000000] cma: Reserved 1024 MiB in 1 range
[    0.000000] hugetlb_cma: reserved 1024 MiB on node 1
[    0.000000] cma: Reserved 1024 MiB in 1 range
[    0.000000] hugetlb_cma: reserved 1024 MiB on node 2
[    0.000000] Zone ranges:
---
 mm/hugetlb_cma.c | 1 +
 1 file changed, 1 insertion(+)
mm/hugetlb_cma.c +1 -0
index f83ae4998990..7693ccefd0c6 100644
@@ -204,6 +204,7 @@ void __init hugetlb_cma_reserve(void)
 		 */
 		per_node = DIV_ROUND_UP(hugetlb_cma_size,
 					nodes_weight(hugetlb_bootmem_nodes));
+		per_node = round_up(per_node, PAGE_SIZE << order);
 		pr_info("hugetlb_cma: reserve %lu MiB, up to %lu MiB per node\n",
 			hugetlb_cma_size / SZ_1M, per_node / SZ_1M);
 	}
Signature · 1 line
2.43.0
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 46 lines · Sang-Heon Jeon

When the user requests a total hugetlb CMA size without per-node specification, hugetlb_cma_reserve() computes per_node from hugetlb_cma_size and the number of nodes that have memory

per_node = DIV_ROUND_UP(hugetlb_cma_size,
nodes_weight(hugetlb_bootmem_nodes));

The reservation loop later computes

size = round_up(min(per_node, hugetlb_cma_size - reserved),
PAGE_SIZE << order);

So the actually reserved per_node size is multiple of (PAGE_SIZE << order), but the logged per_node is not rounded up, so it may be smaller than the actual reserved size.

For example, as the existing comment describes, if a 3 GB area is requested on a machine with 4 NUMA nodes that have memory, 1 GB is allocated on the first three nodes, but the printed log is

hugetlb_cma: reserve 3072 MiB, up to 768 MiB per node

Round per_node up to (PAGE_SIZE << order) before logging so that the printed log always matches the actual reserved size. No functional change to the actual reservation size, as the following case analysis shows

1. remaining (hugetlb_cma_size - reserved) >= rounded per_node
- AS-IS: min() picks unrounded per_node;
round_up() returns rounded per_node
- TO-BE: min() picks rounded per_node;
round_up() returns rounded per_node (no-op)
2. remaining < unrounded per_node
- AS-IS: min() picks remaining;
round_up() returns round_up(remaining)
- TO-BE: min() picks remaining;
round_up() returns round_up(remaining)
3. unrounded per_node <= remaining < rounded per_node
- AS-IS: min() picks unrounded per_node;
round_up() returns rounded per_node
- TO-BE: min() picks remaining;
round_up() returns round_up(remaining) equals rounded per_node

Quoted 1 line
Fixes cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma") # 5.7

^
we do not add a suffix to Fixes tag. We usually add it to stable tag.

Thanks.