Skip to contentSkip to conversation

mm: hugetlb: fix CMA-related issues

PATCH In review Qi Zheng 2 participants 5 messages Nov 17

Subsystems (7)
lore
Share & tools
lore

Qi Zheng

author
#0 lore raw

v5 cover mm: hugetlb: fix CMA-related issues

From: Qi Zheng <[email protected]>

Hi all,

This series fixes three CMA-related issues in HugeTLB reservation, long-term pinning and allocation:

- Avoid ineffective CMA reservations on 32-bit x86 kernels running on CPUs that advertise 1 GiB page support.
- Allow long-term pins of explicitly CMA-allocated HugeTLB folios, which already own the CMA memory.
- Preserve the caller's GFP mask when allocating from CMA.

Changes in v5:
- Move the HugeTLB CMA ownership check into cma_folio_is_longterm_pinnable(), keeping folio_is_longterm_pinnable() inline in mm.h without a HugeTLB header dependency. (suggested by David Hildenbrand)
- Clarify CMA ownership in the commit message for patch #2 (pointed by Muchun Song)
- Rebase onto next-20261116.

Changes in v4:
- add comment to cma_alloc_frozen_compound().
(suggested by Oscar Salvador)
- collect Reviewed-by tag.
- rebase onto next-20261110.

Changes in v3:
- Drop the cpuset fix. A more comprehensive fix across the HugeTLB allocation paths will follow in a separate patchset.
- Check for __GFP_COMP instead of adding it in cma_alloc_frozen_compound(), and leave __GFP_NOWARN to callers. (suggested by Oscar Salvador)
- Clarify the CMA ownership comment in folio_is_longterm_pinnable(). (suggested by Muchun Song)
- Collect Reviewed-by and Acked-by tags.

Changes in v2:
- Replace the PF_MEMALLOC_PIN allocation restriction with a GUP exception for HugeTLB folios marked HPG_cma. (suggested by Muchun Song)
- Pass the caller's GFP mask through cma_alloc_frozen_compound() instead of skipping CMA when direct reclaim is disallowed. (suggested by sashiko)
- Remove unnecessary revalidation of successfully allocated folios after cpuset changes. Retry only failed allocations.
(pointed by Jose A. Perez de Azpillaga)
- collect Reviewed-by and Acked-by.
- Rebase onto next-20261104.

Thanks,
Qi

Qi Zheng (3):
x86: hugetlb: disable default CMA reservations on 32-bit kernel
mm: gup: allow long-term pins of CMA-allocated HugeTLB folios
mm: hugetlb: fix GFP mask propagation to CMA

 arch/x86/mm/hugetlbpage.c |  2 +-
 include/linux/cma.h       |  5 ++++-
 include/linux/mm.h        |  9 ++++++---
 mm/cma.c                  | 21 +++++++++++++++++++--
 mm/hugetlb_cma.c          |  4 ++--
 5 files changed, 32 insertions(+), 9 deletions(-)

base-commit: 749a4de370409464cf8501ec5b2451df45c84eb4

Signature · 1 line
2.54.0
author
#1 lore raw

PATCH v5 1/3 x86: hugetlb: disable default CMA reservations on 32-bit kernel

From: Qi Zheng <[email protected]>

A 32-bit x86 kernel can run on a 64-bit-capable CPU advertising GBPAGES. arch_hugetlb_cma_order() checks only this feature and returns a nonzero order, although gigantic HugeTLB support on x86 requires CONFIG_X86_64. Specifying hugetlb_cma= can therefore reserve a CMA area that cannot serve the intended gigantic HugeTLB allocations.

On non-PAE kernels, the folded PUD_SHIFT yields a 4 MiB order, which is equal to MAX_PAGE_ORDER. With CONFIG_DEBUG_VM enabled, this triggers the VM_WARN_ON(order <= MAX_PAGE_ORDER) check in hugetlb_cma_reserve(), warning that hugetlb_cma is attempting to reserve a non-gigantic order.

Require CONFIG_X86_64 as well as GBPAGES when selecting the default CMA order to avoid these ineffective reservations.

Fixes cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma")
---
 arch/x86/mm/hugetlbpage.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/hugetlbpage.c +1 -1
index 3b26621c91286..70726a99ede7b 100644
@@ -45,7 +45,7 @@ arch_initcall(gigantic_pages_init);
 
 unsigned int __init arch_hugetlb_cma_order(void)
 {
-	if (boot_cpu_has(X86_FEATURE_GBPAGES))
+	if (IS_ENABLED(CONFIG_X86_64) && boot_cpu_has(X86_FEATURE_GBPAGES))
 		return PUD_SHIFT - PAGE_SHIFT;
 
 	return 0;
Signature · 1 line
2.54.0
author
#2 lore raw

PATCH v5 2/3 mm: gup: allow long-term pins of CMA-allocated HugeTLB folios

In reply to Qi Zheng

From: Qi Zheng <[email protected]>

Long-term pinning rejects HugeTLB folios explicitly allocated from CMA, causing unnecessary migration or pinning failures. These folios already own their CMA memory and do not need to be migrated out for future CMA allocations.

Add cma_folio_is_longterm_pinnable() to recognize these allocations through HPG_cma. Allow them to be pinned even in ZONE_MOVABLE, since memory blocks overlapping boot-time CMA areas cannot be offlined. Ordinary movable allocations that borrow CMA memory remain unpinnable.

---
 include/linux/cma.h |  3 +++
 include/linux/mm.h  |  9 ++++++---
 mm/cma.c            | 11 +++++++++++
 3 files changed, 20 insertions(+), 3 deletions(-)
include/linux/cma.h +3 -0
index 8555d38a97b1a..d829fd0355bae 100644
@@ -21,6 +21,9 @@
 #define CMA_MIN_ALIGNMENT_BYTES (PAGE_SIZE * CMA_MIN_ALIGNMENT_PAGES)
 
 struct cma;
+struct folio;
+
+bool cma_folio_is_longterm_pinnable(struct folio *folio);
 
 extern unsigned long totalcma_pages;
 extern phys_addr_t cma_get_base(const struct cma *cma);
include/linux/mm.h +6 -3
index daf5c5b567ebc..e71aae8001831 100644
@@ -38,6 +38,7 @@
 #include <linux/bitops.h>
 #include <linux/iommu-debug-pagealloc.h>
 #include <linux/kcsan-checks.h>
+#include <linux/cma.h>
 
 struct mempolicy;
 struct anon_vma;
@@ -3015,15 +3016,18 @@ static inline bool is_zero_folio(const struct folio *folio)
 	return is_zero_page(&folio->page);
 }
 
-/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
 #ifdef CONFIG_MIGRATION
+/*
+ * MIGRATE_CMA and ZONE_MOVABLE folios cannot normally be long-term pinned.
+ * CMA may allow pinning folios explicitly allocated from its areas.
+ */
 static inline bool folio_is_longterm_pinnable(struct folio *folio)
 {
 #ifdef CONFIG_CMA
 	int mt = folio_migratetype(folio);
 
 	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
-		return false;
+		return cma_folio_is_longterm_pinnable(folio);
 #endif
 	/* The zero page can be "pinned" but gets special handling. */
 	if (is_zero_folio(folio))
@@ -3042,7 +3046,6 @@ static inline bool folio_is_longterm_pinnable(struct folio *folio)
 
 	/* Otherwise, non-movable zone folios can be pinned. */
 	return !folio_is_zone_movable(folio);
-
 }
 #else
 static inline bool folio_is_longterm_pinnable(struct folio *folio)
mm/cma.c +11 -0
index 9ec1d1fd5dff5..fea5d1015319d 100644
@@ -29,6 +29,7 @@
 #include <linux/highmem.h>
 #include <linux/io.h>
 #include <linux/kmemleak.h>
+#include <linux/hugetlb.h>
 #include <trace/events/cma.h>
 
 #include "internal.h"
@@ -38,6 +39,16 @@
 struct cma cma_areas[MAX_CMA_AREAS];
 unsigned int cma_area_count;
 
+bool cma_folio_is_longterm_pinnable(struct folio *folio)
+{
+	/*
+	 * Explicitly allocated CMA folios own their CMA memory and can be
+	 * pinned even in ZONE_MOVABLE, as CMA memory cannot be offlined.
+	 * For now, only HugeTLB tracks these allocations with a folio flag.
+	 */
+	return folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio);
+}
+
 phys_addr_t cma_get_base(const struct cma *cma)
 {
 	WARN_ON_ONCE(cma->nranges != 1);
Signature · 1 line
2.54.0
maintainer Reviewed-by on v5 2/3
#3 lore raw

Re: [PATCH v5 2/3] mm: gup: allow long-term pins of CMA-allocated HugeTLB folios

Quoted 101 lines · Qi Zheng

From: Qi Zheng <[email protected]>

Long-term pinning rejects HugeTLB folios explicitly allocated from CMA, causing unnecessary migration or pinning failures. These folios already own their CMA memory and do not need to be migrated out for future CMA allocations.

Add cma_folio_is_longterm_pinnable() to recognize these allocations through HPG_cma. Allow them to be pinned even in ZONE_MOVABLE, since memory blocks overlapping boot-time CMA areas cannot be offlined. Ordinary movable allocations that borrow CMA memory remain unpinnable.

---
 include/linux/cma.h |  3 +++
 include/linux/mm.h  |  9 ++++++---
 mm/cma.c            | 11 +++++++++++
 3 files changed, 20 insertions(+), 3 deletions(-)
include/linux/cma.h +3 -0
index 8555d38a97b1a..d829fd0355bae 100644
@@ -21,6 +21,9 @@
 #define CMA_MIN_ALIGNMENT_BYTES (PAGE_SIZE * CMA_MIN_ALIGNMENT_PAGES)
 
 struct cma;
+struct folio;
+
+bool cma_folio_is_longterm_pinnable(struct folio *folio);
 
 extern unsigned long totalcma_pages;
 extern phys_addr_t cma_get_base(const struct cma *cma);
include/linux/mm.h +6 -3
index daf5c5b567ebc..e71aae8001831 100644
@@ -38,6 +38,7 @@
 #include <linux/bitops.h>
 #include <linux/iommu-debug-pagealloc.h>
 #include <linux/kcsan-checks.h>
+#include <linux/cma.h>
 
 struct mempolicy;
 struct anon_vma;
@@ -3015,15 +3016,18 @@ static inline bool is_zero_folio(const struct folio *folio)
 	return is_zero_page(&folio->page);
 }
 
-/* MIGRATE_CMA and ZONE_MOVABLE do not allow pin folios */
 #ifdef CONFIG_MIGRATION
+/*
+ * MIGRATE_CMA and ZONE_MOVABLE folios cannot normally be long-term pinned.
+ * CMA may allow pinning folios explicitly allocated from its areas.
+ */
 static inline bool folio_is_longterm_pinnable(struct folio *folio)
 {
 #ifdef CONFIG_CMA
 	int mt = folio_migratetype(folio);
 
 	if (mt == MIGRATE_CMA || mt == MIGRATE_ISOLATE)
-		return false;
+		return cma_folio_is_longterm_pinnable(folio);
 #endif
 	/* The zero page can be "pinned" but gets special handling. */
 	if (is_zero_folio(folio))
@@ -3042,7 +3046,6 @@ static inline bool folio_is_longterm_pinnable(struct folio *folio)
 
 	/* Otherwise, non-movable zone folios can be pinned. */
 	return !folio_is_zone_movable(folio);
-
 }
 #else
 static inline bool folio_is_longterm_pinnable(struct folio *folio)
mm/cma.c +4 -0
index 9ec1d1fd5dff5..fea5d1015319d 100644
@@ -29,6 +29,7 @@
 #include <linux/highmem.h>
 #include <linux/io.h>
 #include <linux/kmemleak.h>
+#include <linux/hugetlb.h>
 #include <trace/events/cma.h>
 
 #include "internal.h"
@@ -38,6 +39,16 @@
 struct cma cma_areas[MAX_CMA_AREAS];
 unsigned int cma_area_count;
 
+bool cma_folio_is_longterm_pinnable(struct folio *folio)
+{
+	/*

mm/cma.c line 54

Quoted 10 lines
mm/cma.c +7 -0
+	 * Explicitly allocated CMA folios own their CMA memory and can be
+	 * pinned even in ZONE_MOVABLE, as CMA memory cannot be offlined.
+	 * For now, only HugeTLB tracks these allocations with a folio flag.
+	 */
+	return folio_test_hugetlb(folio) && folio_test_hugetlb_cma(folio);
+}
+
 phys_addr_t cma_get_base(const struct cma *cma)
 {
 	WARN_ON_ONCE(cma->nranges != 1);

Thanks.

This can go through Andrew's tree with the other hugetlb stuff.

Signature · 3 lines
Cheers,

David
author
#4 lore raw

PATCH v5 3/3 mm: hugetlb: fix GFP mask propagation to CMA

In reply to Qi Zheng

From: Qi Zheng <[email protected]>

cma_alloc_frozen_compound() uses GFP_KERNEL, allowing direct reclaim during the first MPOL_PREFERRED_MANY allocation attempt even though the caller clears __GFP_DIRECT_RECLAIM.

Pass the caller's GFP mask through the helper so CMA respects its reclaim constraints when trying the preferred nodes.

Fixes 9bda131c6093 ("mm: cma: add cma_alloc_frozen{_compound}()")
---
 include/linux/cma.h |  2 +-
 mm/cma.c            | 10 ++++++++--
 mm/hugetlb_cma.c    |  4 ++--
 3 files changed, 11 insertions(+), 5 deletions(-)
include/linux/cma.h +1 -1
index d829fd0355bae..d6e97e831d970 100644
@@ -56,7 +56,7 @@ extern bool cma_release(struct cma *cma, const struct page *pages, unsigned long
 
 struct page *cma_alloc_frozen(struct cma *cma, unsigned long count,
 		unsigned int align, bool no_warn);
-struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order);
+struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, gfp_t gfp);
 bool cma_release_frozen(struct cma *cma, const struct page *pages,
 		unsigned long count);
 
mm/cma.c +8 -2
index fea5d1015319d..225e3bbaecf9a 100644
@@ -944,9 +944,15 @@ struct page *cma_alloc_frozen(struct cma *cma, unsigned long count,
 	return __cma_alloc_frozen(cma, count, align, gfp);
 }
 
-struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order)
+struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, gfp_t gfp)
 {
-	gfp_t gfp = GFP_KERNEL | __GFP_COMP | __GFP_NOWARN;
+	/*
+	 * The only caller of cma_alloc_frozen_compound() is hugetlb, and
+	 * hugetlb constructs its gfp mask via htlb_alloc_mask(), which already
+	 * sets __GFP_COMP. Check here for usecases bypassing htlb_alloc_mask().
+	 */
+	if (WARN_ON_ONCE(!(gfp & __GFP_COMP)))
+		return NULL;
 
 	return __cma_alloc_frozen(cma, 1 << order, order, gfp);
 }
mm/hugetlb_cma.c +2 -2
index cd6f64da31a2c..1eb9d8d8732ba 100644
@@ -71,14 +71,14 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order, gfp_t gfp_mask,
 	}
 
 	if (hugetlb_cma[nid] && node_isset(nid, *nmask))
-		page = cma_alloc_frozen_compound(hugetlb_cma[nid], order);
+		page = cma_alloc_frozen_compound(hugetlb_cma[nid], order, gfp_mask);
 
 	if (!page && !(gfp_mask & __GFP_THISNODE)) {
 		for_each_node_mask(node, *nmask) {
 			if (node == nid || !hugetlb_cma[node])
 				continue;
 
-			page = cma_alloc_frozen_compound(hugetlb_cma[node], order);
+			page = cma_alloc_frozen_compound(hugetlb_cma[node], order, gfp_mask);
 			if (page)
 				break;
 		}
Signature · 1 line
2.54.0