Skip to contentSkip to conversation

mm: hugetlb: avoid fallback for specific node allocation of 1G pages

PATCH Applied Luiz Capitulino 4 participants 5 messages Feb 11, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm: hugetlb: avoid fallback for specific node allocation of 1G pages

When using the HugeTLB kernel command-line to allocate 1G pages from a specific node, such as:

default_hugepagesz=1G hugepages=1:1

If node 1 happens to not have enough memory for the requested number of 1G pages, the allocation falls back to other nodes. A quick way to reproduce this is by creating a KVM guest with a memory-less node and trying to allocate 1 1G page from it. Instead of failing, the allocation will fallback to other nodes.

This defeats the purpose of node specific allocation. Also, specific node allocation for 2M pages don't have this behavior: the allocation will just fail for the pages it can't satisfy.

This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() for 1G boot-time allocation as this function falls back to other nodes if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() instead, which ensures that the allocation will only be satisfied from the specified node.

Fixes b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
---
 mm/hugetlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb.c +1 -1
index 65068671e460..163190e89ea1 100644
@@ -3145,7 +3145,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid)
 
 	/* do node specific alloc */
 	if (nid != NUMA_NO_NODE) {
-		m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h),
+		m = memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h),
 				0, MEMBLOCK_ALLOC_ACCESSIBLE, nid);
 		if (!m)
 			return 0;
Signature · 1 line
2.48.1
Acked-by on v1 1/1
#1 lore raw
Quoted 33 lines · Luiz Capitulino

When using the HugeTLB kernel command-line to allocate 1G pages from a specific node, such as:

default_hugepagesz=1G hugepages=1:1

If node 1 happens to not have enough memory for the requested number of 1G pages, the allocation falls back to other nodes. A quick way to reproduce this is by creating a KVM guest with a memory-less node and trying to allocate 1 1G page from it. Instead of failing, the allocation will fallback to other nodes.

This defeats the purpose of node specific allocation. Also, specific node allocation for 2M pages don't have this behavior: the allocation will just fail for the pages it can't satisfy.

This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() for 1G boot-time allocation as this function falls back to other nodes if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() instead, which ensures that the allocation will only be satisfied from the specified node.

Fixes b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
---
  mm/hugetlb.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb.c +0 -0
index 65068671e460..163190e89ea1 100644

mm/hugetlb.c int __alloc_bootmem_huge_page(struct hstate *h, int nid) · line 3151

Quoted 8 lines
mm/hugetlb.c +1 -1
@@ -3145,7 +3145,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid)
 
  	/* do node specific alloc */
  	if (nid != NUMA_NO_NODE) {
-		m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h),
+		m = memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h),
  				0, MEMBLOCK_ALLOC_ACCESSIBLE, nid);
  		if (!m)
  			return 0;

Yeah, documentation says "The node format specifies the number of huge pages to allocate on specific nodes."

Likely the patch simply copied the memblock_alloc_try_nid_raw() call; memblock_alloc_exact_nid_raw() seems to be the right thing to do

Signature · 3 lines
Cheers,

David / dhildenb
maintainer Acked-by on v1 1/1
#2 lore raw
In reply to Luiz Capitulino
Quoted 21 lines · Luiz Capitulino

When using the HugeTLB kernel command-line to allocate 1G pages from a specific node, such as:

default_hugepagesz=1G hugepages=1:1

If node 1 happens to not have enough memory for the requested number of 1G pages, the allocation falls back to other nodes. A quick way to reproduce this is by creating a KVM guest with a memory-less node and trying to allocate 1 1G page from it. Instead of failing, the allocation will fallback to other nodes.

This defeats the purpose of node specific allocation. Also, specific node allocation for 2M pages don't have this behavior: the allocation will just fail for the pages it can't satisfy.

This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() for 1G boot-time allocation as this function falls back to other nodes if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() instead, which ensures that the allocation will only be satisfied from the specified node.

Quoted 3 lines
Fixes b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")

This was discussed yesterday in [1], ccing Frank for awareness.

[1] https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/

Signature · 2 lines
Oscar Salvador
SUSE Labs
#3 lore raw
Quoted 24 lines · Oscar Salvador
Quoted 23 lines · Luiz Capitulino

When using the HugeTLB kernel command-line to allocate 1G pages from a specific node, such as:

default_hugepagesz=1G hugepages=1:1

If node 1 happens to not have enough memory for the requested number of 1G pages, the allocation falls back to other nodes. A quick way to reproduce this is by creating a KVM guest with a memory-less node and trying to allocate 1 1G page from it. Instead of failing, the allocation will fallback to other nodes.

This defeats the purpose of node specific allocation. Also, specific node allocation for 2M pages don't have this behavior: the allocation will just fail for the pages it can't satisfy.

This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() for 1G boot-time allocation as this function falls back to other nodes if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() instead, which ensures that the allocation will only be satisfied from the specified node.

Fixes b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
Quoted 7 lines
Quoted 1 line

This was discussed yesterday in [1], ccing Frank for awareness.

[1] https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/

Interesting, thanks for the reference.

I stumbled over this issue back in December when debugging a HugeTLB issue at Red Hat (David knows it ;) ) and had this patch pending for more than a week now...

#4 lore raw
Quoted 32 lines · Luiz Capitulino
Quoted 30 lines · Oscar Salvador
Quoted 21 lines · Luiz Capitulino

When using the HugeTLB kernel command-line to allocate 1G pages from a specific node, such as:

default_hugepagesz=1G hugepages=1:1

If node 1 happens to not have enough memory for the requested number of 1G pages, the allocation falls back to other nodes. A quick way to reproduce this is by creating a KVM guest with a memory-less node and trying to allocate 1 1G page from it. Instead of failing, the allocation will fallback to other nodes.

This defeats the purpose of node specific allocation. Also, specific node allocation for 2M pages don't have this behavior: the allocation will just fail for the pages it can't satisfy.

This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() for 1G boot-time allocation as this function falls back to other nodes if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() instead, which ensures that the allocation will only be satisfied from the specified node.

Quoted 3 lines
Fixes b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")

This was discussed yesterday in [1], ccing Frank for awareness.

Quoted 8 lines
Quoted 1 line

[1] https://patchwork.kernel.org/project/linux-mm/patch/[email protected]/

Interesting, thanks for the reference.

I stumbled over this issue back in December when debugging a HugeTLB issue at Red Hat (David knows it ;) ) and had this patch pending for more than a week now...

Looks good, I'll drop the same change from my upcoming v4 series. This will create a contextual dependency, but that's ok, this one will go in first in any case.