Open HugeTLB allocation routine for more generic use

Previously, gbl_chg was passed from alloc_hugetlb_folio() into dequeue_hugetlb_folio_vma(), leaking the concept of gbl_chg into dequeue_hugetlb_folio_vma().

This patch consolidates the interpretation of gbl_chg into alloc_hugetlb_folio(), also renaming dequeue_hugetlb_folio_vma() to dequeue_hugetlb_folio() so dequeue_hugetlb_folio() can just focus on dequeuing a folio.

No functional change intended.

Signed-off-by Ackerley Tng <[email protected]>

Reviewed-by James Houghton <[email protected]>

---
 mm/hugetlb.c | 24 +++++++++---------------
 1 file changed, 9 insertions(+), 15 deletions(-)

mm/hugetlb.c +9 -15

index a1832da0f6236..fd067bd394ee0 100644
@@ -1380,7 +1380,7 @@ static unsigned long available_huge_pages(struct hstate *h)
 
 static struct folio *dequeue_hugetlb_folio_vma(struct hstate *h,
 				struct vm_area_struct *vma,
-				unsigned long address, long gbl_chg)
+				unsigned long address)
 {
 	struct folio *folio = NULL;
 	struct mempolicy *mpol;

@@ -1388,13 +1388,6 @@ static struct folio *dequeue_hugetlb_folio_vma(struct hstate *h,
 	nodemask_t *nodemask;
 	int nid;
 
-	/*
-	 * gbl_chg==1 means the allocation requires a new page that was not
-	 * reserved before.  Making sure there's at least one free page.
-	 */
-	if (gbl_chg && !available_huge_pages(h))
-		goto err;
-
 	gfp_mask = htlb_alloc_mask(h);
 	nid = huge_node(vma, address, gfp_mask, &mpol, &nodemask);
 

@@ -1412,9 +1405,6 @@ static struct folio *dequeue_hugetlb_folio_vma(struct hstate *h,
 
 	mpol_cond_put(mpol);
 	return folio;
-
-err:
-	return NULL;
 }
 
 #ifdef CONFIG_ARCH_HAS_GIGANTIC_PAGE

@@ -2962,12 +2952,16 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma,
 		goto out_uncharge_cgroup_reservation;
 
 	spin_lock_irq(&hugetlb_lock);
+
 	/*
-	 * glb_chg is passed to indicate whether or not a page must be taken
-	 * from the global free pool (global change).  gbl_chg == 0 indicates
-	 * a reservation exists for the allocation.
+	 * gbl_chg == 0 indicates a reservation exists for the allocation - so
+	 * try dequeuing a page. If there are available_huge_pages(), try using
+	 * them!
 	 */
-	folio = dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg);
+	folio = NULL;
+	if (!gbl_chg || available_huge_pages(h))
+		folio = dequeue_hugetlb_folio_vma(h, vma, addr);
+
 	if (!folio) {
 		spin_unlock_irq(&hugetlb_lock);
 		folio = alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr);

Signature · 1 line

2.53.0.310.g728cabbaf7-goog

Move memory policy interpretation out of alloc_buddy_hugetlb_folio_with_mpol() and into alloc_hugetlb_folio() to separate reading and interpretation of memory policy from actual allocation.

This will later allow memory policy to be interpreted outside of the process of allocating a hugetlb folio entirely. This opens doors for other callers of the HugeTLB folio allocation function, such as guest_memfd, where memory may not always be mapped and hence may not have an associated vma.

No functional change intended.

Signed-off-by Ackerley Tng <[email protected]>

Move memory policy interpretation out of dequeue_hugetlb_folio_vma() and into alloc_hugetlb_folio() to separate reading and interpretation of memory policy from actual allocation.

Also rename dequeue_hugetlb_folio_vma() to dequeue_hugetlb_folio_with_mpol() to remove association with vma and to align with alloc_buddy_hugetlb_folio_with_mpol().

This will later allow memory policy to be interpreted outside of the process of allocating a hugetlb folio entirely. This opens doors for other callers of the HugeTLB folio allocation function, such as guest_memfd, where memory may not always be mapped and hence may not have an associated vma.

No functional change intended.

Signed-off-by Ackerley Tng <[email protected]>

I would like to get feedback on:

1. Opening up HugeTLB's allocation for more generic use

2. Reverting and re-adopting the try-commit-cancel protocol for memory charging

To see how hugetlb_alloc_folio() is used by guest_memfd, the most recent patch series that uses this more generic HugeTLB allocation routine is at [1], and a newer revision of that patch series is at [2].

Independently of guest_memfd, I believe this change is useful in simplifying alloc_hugetlb_folio(). alloc_hugetlb_folio() was so coupled to a VMA that even HugeTLBfs allocates HugeTLB folios using a pseudo-VMA.

[1] https://lore.kernel.org/all/[email protected]/T/
[2] https://github.com/googleprodkernel/linux-cc/tree/wip-gmem-conversions-hugetlb-restructuring-12-08-25

Ackerley Tng (7):
mm: hugetlb: Consolidate interpretation of gbl_chg within
alloc_hugetlb_folio()
mm: hugetlb: Move mpol interpretation out of
alloc_buddy_hugetlb_folio_with_mpol()
mm: hugetlb: Move mpol interpretation out of
dequeue_hugetlb_folio_vma()
Revert "memcg/hugetlb: remove memcg hugetlb try-commit-cancel
protocol"
mm: hugetlb: Adopt memcg try-commit-cancel protocol
mm: memcontrol: Remove now-unused function mem_cgroup_charge_hugetlb
mm: hugetlb: Refactor out hugetlb_alloc_folio()

 include/linux/hugetlb.h    |  11 ++
 include/linux/memcontrol.h |  21 +++-
 mm/hugetlb.c               | 228 +++++++++++++++++++++----------------
 mm/memcontrol.c            |  77 ++++++++-----
 4 files changed, 212 insertions(+), 125 deletions(-)

base-commit: db9571a66156bfbc0273e66e5c77923869bda547 --
2.53.0.310.g728cabbaf7-goog

On Wed, 11 Feb 2026 16:37:11 -0800 Ackerley Tng <[email protected]> wrote:

Hi Ackerly, I hope you're donig well!

[...snip...]

Quoted 3 lines

I would like to get feedback on:

1. Opening up HugeTLB's allocation for more generic use

I'm not entirely familiar with guest_memfd, so pleae excuse my ignorance if I'm missing anything obvious.

But I'm wondering what hugeTLB offers
that other hugepage solutions cannot offer for guest_memfd, if the goal of this series is to decouple it from hugeTLBfs.

Quoted 2 lines

2. Reverting and re-adopting the try-commit-cancel protocol for memory charging

On the second point, I am wondering if reintroducing the try-commit-cancel protocol is tied to factoring out hugetlb_alloc_folio. When I removed the protocol a while back, the justification was that for the most part, grabbing a hugetlb folio was a relatively cheap & fast operation, since hugetlb mostly operates out of a preallocated pool.

So the cost of being wrong, going above the limit, and having to return the hugetlb folio was also relatively low.

It seems like this patch series introduces some new paths for hugetlb pages to be consumed (specifically, without a reservation or vma). I imagine that these new paths make the slowpath for hugetlb more frequent, which makes the cost of assuming that the memcg limit is OK higher? I think explicitly spelling this out in the justification for reintroducing the charging protocol could be helpful.

Thank you for the series, again. I hope you have a great day! Joshua

Quoted 37 lines

To see how hugetlb_alloc_folio() is used by guest_memfd, the most recent patch series that uses this more generic HugeTLB allocation routine is at [1], and a newer revision of that patch series is at [2].

Independently of guest_memfd, I believe this change is useful in simplifying alloc_hugetlb_folio(). alloc_hugetlb_folio() was so coupled to a VMA that even HugeTLBfs allocates HugeTLB folios using a pseudo-VMA.

[1] https://lore.kernel.org/all/[email protected]/T/
[2] https://github.com/googleprodkernel/linux-cc/tree/wip-gmem-conversions-hugetlb-restructuring-12-08-25

Ackerley Tng (7):
mm: hugetlb: Consolidate interpretation of gbl_chg within
alloc_hugetlb_folio()
mm: hugetlb: Move mpol interpretation out of
alloc_buddy_hugetlb_folio_with_mpol()
mm: hugetlb: Move mpol interpretation out of
dequeue_hugetlb_folio_vma()
Revert "memcg/hugetlb: remove memcg hugetlb try-commit-cancel
protocol"
mm: hugetlb: Adopt memcg try-commit-cancel protocol
mm: memcontrol: Remove now-unused function mem_cgroup_charge_hugetlb
mm: hugetlb: Refactor out hugetlb_alloc_folio()

 include/linux/hugetlb.h    |  11 ++
 include/linux/memcontrol.h |  21 +++-
 mm/hugetlb.c               | 228 +++++++++++++++++++++----------------
 mm/memcontrol.c            |  77 ++++++++-----
 4 files changed, 212 insertions(+), 125 deletions(-)

base-commit: db9571a66156bfbc0273e66e5c77923869bda547 --
2.53.0.310.g728cabbaf7-goog

Quoted 6 lines · Joshua Hahn

On Wed, 11 Feb 2026 16:37:11 -0800 Ackerley Tng <[email protected]> wrote:

Hi Ackerly, I hope you're donig well!

[...snip...]

Quoted 6 lines

Quoted 3 lines

I would like to get feedback on:

1. Opening up HugeTLB's allocation for more generic use

I'm not entirely familiar with guest_memfd, so pleae excuse my ignorance if I'm missing anything obvious.

Happy to take questions! Thank you for your thoughts and reviews!

Quoted 4 lines

But I'm wondering what hugeTLB offers
that other hugepage solutions cannot offer for guest_memfd, if the goal of this series is to decouple it from hugeTLBfs.

The one other huge page source that we've explored is THP pages from the buddy allocator. Compared to HugeTLB, huge pages from the buddy allocator

+ Has a maximum size of 2M
+ Does not guarantee huge pages the way HugeTLB does - HugeTLB pages are allocated at boot, and guest_memfd can reserve pages at guest_memfd creation time.
+ Allocation of HugeTLB pages is also really fast, it's just dequeuing from a preallocated pool

The last reason to use HugeTLB is not because of any inherent advantage of using HugeTLB over other sources of huge pages, but for administrative/scheduling purposes:

Given that existing non-guest_memfd workloads are already using HugeTLB, for optimal scheduling, machine memory is already carved up in HugeTLB pages for these workloads. Workloads that require using guest_memfd (like Confidential VMs) must also use HugeTLB to participate in optimial workload scheduling across machines.

Quoted 12 lines

Quoted 2 lines

2. Reverting and re-adopting the try-commit-cancel protocol for memory charging

On the second point, I am wondering if reintroducing the try-commit-cancel protocol is tied to factoring out hugetlb_alloc_folio. When I removed the protocol a while back, the justification was that for the most part, grabbing a hugetlb folio was a relatively cheap & fast operation, since hugetlb mostly operates out of a preallocated pool.

So the cost of being wrong, going above the limit, and having to return the hugetlb folio was also relatively low.

Thanks for this! I saw your patch to just optimistically grab a HugeTLB page :) For that patch, the primary reason was to simplify the logic, and the simplification was justifiable because grabbing a folio is cheap, right? (And so grabbing a folio being cheap wasn't a reason in itself?)

Quoted 7 lines

It seems like this patch series introduces some new paths for hugetlb pages to be consumed (specifically, without a reservation or vma). I imagine that these new paths make the slowpath for hugetlb more frequent, which makes the cost of assuming that the memcg limit is OK higher? I think explicitly spelling this out in the justification for reintroducing the charging protocol could be helpful.

Yes, I should have done that. Will copy the following to the next revision.

The main reason is that reintroducing the charging protocol is the clearest way (for me) to cleanly refactor out hugetlb_alloc_folio() without worrying about the edge cases around HugeTLB reservations and charging.

If I didn't reintroduce the charging protocol, I would have to depend on freeing the new hugetlb folio on memcg charging failure, and the freeing in turn depends on the subpool correctly being set in the folio, and the presence of the subpool influences (in free_huge_folio()) whether the reservation was returned to the global hstate. Aaannnd... there's also a hugetlb_restore_reserve flag that controls whether to return the folio to the subpool (and the hstate). I find folio_clear_hugetlb_restore_reserve() on certain code paths kind of magical/unexplained too.

I would rather iron out those charging and reservation details separately from this series (with more testing support).

On the other hand, reintroducing the charging protocol has the benefit of avoiding allocations (not just dequeuing, if surplus HugeTLB pages are required) if the memcg limit is hit. Also, if the original reason for removing the protocol was to simplify the code, refactoring out hugetlb_alloc_folio() also simplifies the code, and I think it's actually nice that memcg charging is done the same way as the other two (h_cg and h_cg_rsvd charging). After hugetlb_alloc_folio() is refactored out, the gotos make all three charging systems consistent and symmetric, which I think is nice to have :)

I hope the consistent/symmetric charging among all 3 systems is welcome, what do you think?

Quoted 8 lines · On Wed, 25 Feb 2026 19:37:04 -0800 Ackerley Tng <[email protected]> wrote:

Quoted 6 lines · Joshua Hahn

On Wed, 11 Feb 2026 16:37:11 -0800 Ackerley Tng <[email protected]> wrote:

Hi Ackerly, I hope you're donig well!

[...snip...]

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

I would like to get feedback on:

1. Opening up HugeTLB's allocation for more generic use

I'm not entirely familiar with guest_memfd, so pleae excuse my ignorance if I'm missing anything obvious.

Happy to take questions! Thank you for your thoughts and reviews!

Of course, thank you for your work, Ackerley!

Quoted 9 lines

Quoted 4 lines

But I'm wondering what hugeTLB offers
that other hugepage solutions cannot offer for guest_memfd, if the goal of this series is to decouple it from hugeTLBfs.

The one other huge page source that we've explored is THP pages from the buddy allocator. Compared to HugeTLB, huge pages from the buddy allocator

Quoted 6 lines

+ Has a maximum size of 2M
+ Does not guarantee huge pages the way HugeTLB does - HugeTLB pages are allocated at boot, and guest_memfd can reserve pages at guest_memfd creation time.
+ Allocation of HugeTLB pages is also really fast, it's just dequeuing from a preallocated pool

All of these make sense. Just wanted to know if guest_memfd had any unique usecases for hugeTLB that normal hugetlbfs didn't have.

Quoted 23 lines

The last reason to use HugeTLB is not because of any inherent advantage of using HugeTLB over other sources of huge pages, but for administrative/scheduling purposes:

Given that existing non-guest_memfd workloads are already using HugeTLB, for optimal scheduling, machine memory is already carved up in HugeTLB pages for these workloads. Workloads that require using guest_memfd (like Confidential VMs) must also use HugeTLB to participate in optimial workload scheduling across machines.

[...snip...]

On the other hand, reintroducing the charging protocol has the benefit of avoiding allocations (not just dequeuing, if surplus HugeTLB pages are required) if the memcg limit is hit. Also, if the original reason for removing the protocol was to simplify the code, refactoring out hugetlb_alloc_folio() also simplifies the code, and I think it's actually nice that memcg charging is done the same way as the other two (h_cg and h_cg_rsvd charging). After hugetlb_alloc_folio() is refactored out, the gotos make all three charging systems consistent and symmetric, which I think is nice to have :)

Quoted 2 lines

I hope the consistent/symmetric charging among all 3 systems is welcome, what do you think?

For the hugetlbfs case, the path to allocate a hugeTLB page on demand makes sense, so I definitely see the argument for avoiding allocations. Does guest_memfd also have a path to allocate a hugeTLB page outside of the boottime reservations? In that case I think it would be nice to clarify that the allocation failure case optimization is also for guest_memfd, not only for hugetlbfs.

Symmetric charging is definitely welcome : -) All of your reasons make sense to me, I just wanted to ask and make sure.

Thanks for your thoughts! I hope you have a great day!! Joshua