Skip to contentSkip to conversation

Beyond 2MB: Why Terabyte-Scale Machines Need 1GB Transparent Huge Pages

LSF/MM/BPF TOPIC Usama Arif 6 participants 16 messages Feb 19 – Feb 26 Memory management

lore
Share & tools
lore

Usama Arif

author
#0 lore raw

When 2M THPs were introduced, the new server hardware coming out had memory in the scale of low hunderds of gigabytes. Today, modern server hardware ship with several terabytes of memory. This is widely available at all hyperscalers (AWS, Azure, GCP, Meta, Oracle, etc).
While 2MB THP have mitigated some scalability bottlenecks, they are no longer "huge" in the context of terabyte-scale memory. There are concrete scalability walls that large-memory machines hit today. This includes LRU lock contention, zone lock contention when missing PCP cache at allocation, extremely low TLB coverage, amount of page tables used...

1G THPs come with their own set of challenges, more difficult to allocate, higher compaction times…

Why 1G THP over hugetlbfs?
==========================

As mentioned in the RFC for 1G THPs[1] while hugetlbfs provides 1GB huge pages today, it has significant limitations that make it unsuitable for many workloads.

The classic hugetlb user is a dedicated machine running a dedicated HPC workload. This approach just doesnt work when you run a multitude of general-purpose workloads co-located on the same host. Enlightening every one of these workloads to use hugetlbfs is impractical -- it requires application-level changes, explicit mmap flags, filesystem mounts, and per-workload capacity planning. Sharing a host between hugetlbfs consumers and regular workloads is equally difficult because hugetlb's static reservation model locks memory away from the rest of the system. In a multi-tenant environment where workloads are constantly being scheduled, resized, and migrated, this rigidity is a serious operational burden.

Concretely, hugetlbfs has the following limitations:

1. Static Reservation: hugetlbfs requires pre-allocating huge pages at boot or runtime, taking memory away. This requires capacity planning, administrative overhead, and makes workload orchestration much much more complex, especially colocating with workloads that don't use hugetlbfs.

2. No Fallback: If a 1GB huge page cannot be allocated, hugetlbfs fails rather than falling back to smaller pages. This makes it fragile under memory pressure.

3. No Splitting: hugetlbfs pages cannot be split when only partial access is needed, leading to memory waste and preventing partial reclaim. It would also make recovery from HWPOISON much easier, by splitting the 1G THP which is not possible with hugetlb.

4. Memory Accounting: hugetlbfs memory is accounted separately and cannot be easily shared with regular memory pools.

PUD THP solves these limitations by integrating 1GB pages into the existing THP infrastructure.

The RFC [1] coverletter contains performance numbers for 1G THPs on x86 and 512M PMD THPs on arm which I wont repeat here.
The RFC raised many good questions for how we can approach this and what the way forward would be. Some of these include:

Page table deposit strategy:
============================

The RFC deposited pagetables for the PMD page table and 512 PTE page tables, which means ~2MB of memory was going to be reserved (and unused) during the lifetime of the 1G THP. David raised a valid point if this is even needed for 2M THP, and I believe the answer is no. As part of cleaning up the current 2M implementation, I am currently working on seeing how kernel would look like without page table deposit for 2M THPs [2] (for everything apart from PowerPC hash MMU).

For 1G THPs a similar approach to [2] can be taken. And probably no initial support for 1G THPs on PowerPC hash MMU which requires page table deposit?

There will also be a lot of code reuse between PUD and PMD, and similar to page table deposit cleanup, it would be good to know what else needs to be targeted!

Is CMA needed to make this work?
================================

The short answer is no. 1G THPs can be gotten without it. CMA can help a lot ofcourse, but we dont *need* it. For e.g. I can run the very simple case of trying to get 1G pages in the upstream kernel without CMA on my server via hugetlb and it works. The server has been up for more than 2 weeks (so pretty fragmented), is running a bunch of stuff in the background, uses 0 CMA memory, and I tried to get 100x1G pages on it and it worked.
It uses folio_alloc_gigantic, which is exactly what this RFC uses:

$ uptime -p
up 2 weeks, 18 hours, 35 minutes
$ cat /proc/meminfo | grep -i cma
CmaTotal:              0 kB
CmaFree:               0 kB
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           1.0Ti        97Gi       297Gi       586Mi       623Gi       913Gi
Swap:          129Gi       659Mi       129Gi
$ echo 100 |   sudo tee /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
100
$ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
100
$ ./map_1g_hugepages
Mapping 100 x 1GB huge pages (100 GB total)
Mapped at 0x7f2d80000000
Touched page 0 at 0x7f2d80000000
Touched page 1 at 0x7f2dc0000000
Touched page 2 at 0x7f2e00000000
Touched page 3 at 0x7f2e40000000
..
..
Touched page 98 at 0x7f4600000000
Touched page 99 at 0x7f4640000000
Unmapped successfully

I see 1G THPs being opportunistically used ideally at the start of the application or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory available and a greater chance of getting 1G THPs.

Splitting strategy
==================

When PUD THP must be break -- for COW after fork, partial munmap, mprotect on a subregion, or reclaim -- it splits directly from PUD to PTE level, converting 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to PMDs and only the necessary PMDs to PTEs. This is something that would hopefully be possible with Davids proposal [3].

khugepaged support
==================

I believe the best strategy for 1G THPs would be to follow the same path as mTHPs, i.e. not having khugepaged support at the start. I have seen khugepaged working in ARM with 512M pages and 64K PAGE_SIZE, so maybe there is a case for it? But I I believe the initial implementation shouldn't have it.
Maybe MADV_COLLPASE only support makes more sense?
I would love to hear more thoughts on this.

Migration support
=================

It is going to be difficult to find a 1GB contiguous memory to migrate to. Maybe it's better to not allow migration of PUDs at all?
As Zi rightly mentioned [4], without migration, PUD THP loses its flexibility and transparency. But with its 1GB size, what exactly would the purpose of PUD THP migration be? It does not create memory fragmentation, since it is the largest folio size we have and contiguous. NUMA balancing 1GB THP seems too much work.

There are a lot more topics that would need to be discussed. But these are some of the big ones that came out of the RFC.

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] http://lore.kernel.org/all/[email protected]/
[4] https://lore.kernel.org/all/[email protected]/

#1 lore raw
Quoted 6 lines

I see 1G THPs being opportunistically used ideally at the start of the application or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory available and a greater chance of getting 1G THPs.

Quoted 8 lines

Splitting strategy
==================

When PUD THP must be break -- for COW after fork, partial munmap, mprotect on a subregion, or reclaim -- it splits directly from PUD to PTE level, converting 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to PMDs and only the necessary PMDs to PTEs. This is something that would hopefully be possible with Davids proposal [3].

There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

However, we cannot easily do the same when remapping a 1 GiB THP to be mapped by PMDs etc. I think there are examples where that just doesn't work or is not desired.

But I considered that in general (avoid folio_split()) an interesting approach. The remapping part is a bit different though.

Signature · 3 lines
Cheers,

David
#2 lore raw
Quoted 15 lines · David Hildenbrand (Arm)
Quoted 6 lines

I see 1G THPs being opportunistically used ideally at the start of the application or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory available and a greater chance of getting 1G THPs.

Quoted 8 lines

Splitting strategy
==================

When PUD THP must be break -- for COW after fork, partial munmap, mprotect on a subregion, or reclaim -- it splits directly from PUD to PTE level, converting 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to PMDs and only the necessary PMDs to PTEs. This is something that would hopefully be possible with Davids proposal [3].

Quoted 4 lines

There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

With TLB coalescing, there is benefit in preserving contiguity. If you lop off the last 4k of a 2M-backed range, a split still gives you 511 contiguously mapped pfns that can be coalesced.

It would be unfortunate to lose that for pure virtual memory splits, while there is no demand or no shortage of huge pages. But it might be possible to do this lazily, e.g. when somebody has trouble getting a larger page, scan the deferred split lists for candidates to migrate.

#7 lore raw
In reply to David Hildenbrand (Arm)
Quoted 5 lines · David Hildenbrand (Arm)
Quoted 5 lines

I see 1G THPs being opportunistically used ideally at the start of the application or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory available and a greater chance of getting 1G THPs.

Quoted 8 lines
Quoted 8 lines

Splitting strategy
==================

When PUD THP must be break -- for COW after fork, partial munmap, mprotect on a subregion, or reclaim -- it splits directly from PUD to PTE level, converting 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to PMDs and only the necessary PMDs to PTEs. This is something that would hopefully be possible with Davids proposal [3].

With mapping of folios > PMD with PMDs, you can use non uniform split to keep after-split folios as large as possible.

Quoted 2 lines

There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

This sounds more reasonable than splitting 1GB itself.

Quoted 4 lines

However, we cannot easily do the same when remapping a 1 GiB THP to be mapped by PMDs etc. I think there are examples where that just doesn't work or is not desired.

But I considered that in general (avoid folio_split()) an interesting approach. The remapping part is a bit different though.

If HW can support multiple TLB entries translating to the same physical frame and allow translation priority of TLB entries, this remapping would be easy and we can still keep the 1GB PUD mapping. Basically, we can have 1GB TLB entry pointing to the 1GB folio and another 4KB TLB entry pointing to the remapped region and overriding the part in the original 1GB vaddr region.

Without that, SW will need to split the PUD into PMDs and PTEs.

Best Regards,
Yan, Zi

#3 lore raw
In reply to Johannes Weiner
Quoted 16 lines · Johannes Weiner
Quoted 15 lines · David Hildenbrand (Arm)
Quoted 14 lines

I see 1G THPs being opportunistically used ideally at the start of the application or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory available and a greater chance of getting 1G THPs.

Splitting strategy
==================

When PUD THP must be break -- for COW after fork, partial munmap, mprotect on a subregion, or reclaim -- it splits directly from PUD to PTE level, converting 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to PMDs and only the necessary PMDs to PTEs. This is something that would hopefully be possible with Davids proposal [3].

Quoted 8 lines
Quoted 4 lines

There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

With TLB coalescing, there is benefit in preserving contiguity. If you lop off the last 4k of a 2M-backed range, a split still gives you 511 contiguously mapped pfns that can be coalesced.

Which CPU are you referring to? AMD’s PTE coalescing works up to 32KB and ARM’s contig PTE supports larger sizes. BTW, do we have PMD level ARM contiguous bit support?

Quoted 5 lines

It would be unfortunate to lose that for pure virtual memory splits, while there is no demand or no shortage of huge pages. But it might be possible to do this lazily, e.g. when somebody has trouble getting a larger page, scan the deferred split lists for candidates to migrate.

Best Regards,
Yan, Zi

#4 lore raw
Quoted 23 lines · Zi Yan
Quoted 21 lines · Johannes Weiner
Quoted 19 lines · David Hildenbrand (Arm)
Quoted 6 lines

I see 1G THPs being opportunistically used ideally at the start of the application or by the allocator (jemalloc/tcmalloc) when there is plenty of free memory available and a greater chance of getting 1G THPs.

Quoted 8 lines

Splitting strategy
==================

When PUD THP must be break -- for COW after fork, partial munmap, mprotect on a subregion, or reclaim -- it splits directly from PUD to PTE level, converting 1 PUD entry into 262,144 PTE entries. The ideal solution would be to split to PMDs and only the necessary PMDs to PTEs. This is something that would hopefully be possible with Davids proposal [3].

There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

Quoted 7 lines
Quoted 3 lines

With TLB coalescing, there is benefit in preserving contiguity. If you lop off the last 4k of a 2M-backed range, a split still gives you 511 contiguously mapped pfns that can be coalesced.

Which CPU are you referring to? AMD’s PTE coalescing works up to 32KB and ARM’s contig PTE supports larger sizes. BTW, do we have PMD level ARM contiguous bit support?

I'm not aware of a CPU that will coalesce the 511 entries into a single one. But *any* coalescing effects will be lost when the range is scattered into discontiguous 4k pagelets.

#6 lore raw
In reply to Zi Yan
Quoted 10 lines · Zi Yan
Quoted 8 lines · Johannes Weiner
Quoted 6 lines · David Hildenbrand (Arm)

There once was this proposal where we would, instead of splitting a THP, migrate all memory away instead. That means, instead of splitting the 1 GiB THP, you would instead return it to the page allocator where somebody else could use it.

Quoted 7 lines
Quoted 3 lines

With TLB coalescing, there is benefit in preserving contiguity. If you lop off the last 4k of a 2M-backed range, a split still gives you 511 contiguously mapped pfns that can be coalesced.

Which CPU are you referring to? AMD’s PTE coalescing works up to 32KB and ARM’s contig PTE supports larger sizes. BTW, do we have PMD level ARM contiguous bit support?

Yes. It's used for hugetlb only so far, obviously (np THP > PMD).

Signature · 3 lines
Cheers,

David
#5 lore raw
In reply to Johannes Weiner
Quoted 8 lines · Johannes Weiner
Quoted 7 lines · Zi Yan
Quoted 4 lines · Johannes Weiner

With TLB coalescing, there is benefit in preserving contiguity. If you lop off the last 4k of a 2M-backed range, a split still gives you 511 contiguously mapped pfns that can be coalesced.

Quoted 7 lines
Quoted 3 lines

Which CPU are you referring to? AMD’s PTE coalescing works up to 32KB and ARM’s contig PTE supports larger sizes. BTW, do we have PMD level ARM contiguous bit support?

I'm not aware of a CPU that will coalesce the 511 entries into a single one. But *any* coalescing effects will be lost when the range is scattered into discontiguous 4k pagelets.

You could of course migrate to larger folios, not necessarily 4k.

Signature · 3 lines
Cheers,

David
#8 lore raw
In reply to Zi Yan
Quoted 5 lines · Zi Yan

If HW can support multiple TLB entries translating to the same physical frame and allow translation priority of TLB entries, this remapping would be easy and we can still keep the 1GB PUD mapping. Basically, we can have 1GB TLB entry pointing to the 1GB folio and another 4KB TLB entry pointing to the remapped region and overriding the part in the original 1GB vaddr region.

Uh, do you know any hardware that supports that? Every CPU I'm familiar with has notes suggesting that trying to do this will cause you to Have A Very Bad Day.

#9 lore raw
Quoted 10 lines · Matthew Wilcox
Quoted 5 lines · Zi Yan

If HW can support multiple TLB entries translating to the same physical frame and allow translation priority of TLB entries, this remapping would be easy and we can still keep the 1GB PUD mapping. Basically, we can have 1GB TLB entry pointing to the 1GB folio and another 4KB TLB entry pointing to the remapped region and overriding the part in the original 1GB vaddr region.

Uh, do you know any hardware that supports that? Every CPU I'm familiar with has notes suggesting that trying to do this will cause you to Have A Very Bad Day.

No. I was imagining it. :)

But thinking about it more, that means for every >PTE TLB hit, HW needs to know whether any sub-range has an additional translation. It is easy if all sub-range translations are present in the TLB. Otherwise, a per sub range bitmap or rewalks of each sub range is needed. Never mind, thank you for waking me up in my daydream.

Best Regards,
Yan, Zi

#10 lore raw
In reply to Usama Arif
Quoted 4 lines · Usama Arif

Is CMA needed to make this work?
================================

Quoted 12 lines

The short answer is no. 1G THPs can be gotten without it. CMA can help a lot
ofcourse, but we dont *need* it. For e.g. I can run the very simple case of
trying to get 1G pages in the upstream kernel without CMA on my server via
hugetlb and it works. The server has been up for more than 2 weeks (so pretty
fragmented), is running a bunch of stuff in the background, uses 0 CMA memory,
and I tried to get 100x1G pages on it and it worked.
It uses folio_alloc_gigantic, which is exactly what this RFC uses:

While I agree with the idea of starting simple, I think we should ask the question of what we want physical memory handling to look like if 1TB pages become more common, and applications start to rely on them to meet their performance goals.

We have CMA balancing code today. It seems to work, but it likely is not the long term direction we want to go, mostly due to the way CMA does allocations.

It seems clear that in order to prevent memory fragmentation, we need to split up system memory in some way between an area that is used only for movable allocations, and an area where any kind of allocation can go.

This would need something similar to CMA balancing to prevent false OOMs for non-movable allocations.

However, beyond that I really do not have any idea of what things should look like.

What do we want the kernel to do here?

Signature · 1 line
All Rights Reversed.
#11 lore raw
Quoted 33 lines · Rik van Riel
Quoted 4 lines · Usama Arif

Is CMA needed to make this work?
================================

Quoted 12 lines

The short answer is no. 1G THPs can be gotten without it. CMA can help a lot
ofcourse, but we dont *need* it. For e.g. I can run the very simple case of
trying to get 1G pages in the upstream kernel without CMA on my server via
hugetlb and it works. The server has been up for more than 2 weeks (so pretty
fragmented), is running a bunch of stuff in the background, uses 0 CMA memory,
and I tried to get 100x1G pages on it and it worked.
It uses folio_alloc_gigantic, which is exactly what this RFC uses:

While I agree with the idea of starting simple, I think we should ask the question of what we want physical memory handling to look like if 1TB pages become more common, and applications start to rely on them to meet their performance goals.

We have CMA balancing code today. It seems to work, but it likely is not the long term direction we want to go, mostly due to the way CMA does allocations.

It seems clear that in order to prevent memory fragmentation, we need to split up system memory in some way between an area that is used only for movable allocations, and an area where any kind of allocation can go.

Quoted 7 lines

This would need something similar to CMA balancing to prevent false OOMs for non-movable allocations.

However, beyond that I really do not have any idea of what things should look like.

What do we want the kernel to do here?

This subtopic is certainly worth a separate session as it's quite involved, but I assume the right (tm) thing to do will be

(a) Teaching the buddy to manage pages larger than the current maximum buddy order. There will certainly be some work required to get to that point (and Zi Yan already did some work). It might also be fair to say that order > current buddy order might behave different at least to some degree (thinking about relation to zone alignment, section sizes etc).

If we require vmemmap for these larger orders, maybe the buddy order could more easily exceed the section size; I don't remember all of the details why that limitation was in place (but one of them was memmap continuity within a high-order buddy page, which is only guaranteed within a memory section with CONFIG_SPARSEMEM).

(b) Teaching compaction etc. to *also* compact/group on a larger granularity (in addition to current sized pageblocks). When we discussed that in the past we used the term superblock, that Zi Yan just brought up again in another thread [1].

There was a proposal a while ago to internally separate zones into chunks of memory (I think the proposal used DRAM banks, such that you could more easily power down unused DRAM banks). I'm not saying we should do that, but maybe something like sub-zones could be something to explore. Maybe not.

Big, more complex topic :)

[1] https://lore.kernel.org/r/[email protected]

Signature · 3 lines
Cheers,

David
#12 lore raw
In reply to Usama Arif
Quoted 2 lines · Usama Arif

Why 1G THP over hugetlbfs?
==========================

I know this isn't your intention, but one interesting aspect of supporting PUD mapped folios natively is that it could open the door to simplifying hugetlb as well.

We currently have all kinds of huge_vma checks scattered over the page table code, and entirely parallel paths for unmapping etc. With native PUD mappings, this could allow pushing the special casing out of the virtual memory layer and into where we deal with the page objects.

You might be able to take it as far as the only thing left of hugetlb is the reservation pool. Such that a naive application does mmap() as per usual, and it comes down to a separate allocation policy how the backing pages are served (buddy, CMA, boot-time reservations, ...)

Approaching it this way could help separate out the discussion on code impact and tech debt of PUD mappings, from the allocation technique question, which in itself is a fairly large topic.

#13 lore raw
Quoted 13 lines · Johannes Weiner
Quoted 2 lines · Usama Arif

Why 1G THP over hugetlbfs?
==========================

I know this isn't your intention, but one interesting aspect of supporting PUD mapped folios natively is that it could open the door to simplifying hugetlb as well.

We currently have all kinds of huge_vma checks scattered over the page table code, and entirely parallel paths for unmapping etc. With native PUD mappings, this could allow pushing the special casing out of the virtual memory layer and into where we deal with the page objects.

Quoted 8 lines

You might be able to take it as far as the only thing left of hugetlb is the reservation pool. Such that a naive application does mmap() as per usual, and it comes down to a separate allocation policy how the backing pages are served (buddy, CMA, boot-time reservations, ...)

Approaching it this way could help separate out the discussion on code impact and tech debt of PUD mappings, from the allocation technique question, which in itself is a fairly large topic.

I agree with this 100%. Adding 1GB folio support first, we then can think about what other THP features, e.g., split, migration, PMD/PTE mapping, are really needed and add them one by one. It is also going to be a good way of retiring hugetlb special code.

Best Regards,
Yan, Zi

#14 lore raw
Quoted 19 lines · Zi Yan
Quoted 18 lines · Johannes Weiner
Quoted 2 lines · Usama Arif

Why 1G THP over hugetlbfs?
==========================

I know this isn't your intention, but one interesting aspect of supporting PUD mapped folios natively is that it could open the door to simplifying hugetlb as well.

We currently have all kinds of huge_vma checks scattered over the page table code, and entirely parallel paths for unmapping etc. With native PUD mappings, this could allow pushing the special casing out of the virtual memory layer and into where we deal with the page objects.

You might be able to take it as far as the only thing left of hugetlb is the reservation pool. Such that a naive application does mmap() as per usual, and it comes down to a separate allocation policy how the backing pages are served (buddy, CMA, boot-time reservations, ...)

Quoted 8 lines
Quoted 3 lines

Approaching it this way could help separate out the discussion on code impact and tech debt of PUD mappings, from the allocation technique question, which in itself is a fairly large topic.

I agree with this 100%. Adding 1GB folio support first, we then can think about what other THP features, e.g., split, migration, PMD/PTE mapping, are really needed and add them one by one. It is also going to be a good way of retiring hugetlb special code.

But this hasn't happened yet for PMD-sized hugetlb, and there's no need to wait for PUD-sized THP to start this process. I don't think that introducing PUD-sized THP will actually motivate anyone to do this work.

I think we have four main things that hugetlb still offers:

- Reserved pool (mentioned above) which we don't yet have a THP replacement for
- shared page tables. mshare() is the replacement here, and that project is moving along nicely.
- Being able to allocate gigantic folios. This is also progressing.
- Guaranteeing that you don't get a fallback; you either get memory in the size you asked for, or you fail.

Every time this comes up, I offer the pagewalk code as an egregious example of where we force every user to know "oh, hugetlb is special". Getting rid of mm_walk_ops->hugetlb_entry() would be a great improvement.

People always look at the fault handler first and say "Ah, this is an obvious hugetlb-is-special case I can get rid of", but honestly it's not that painful to keep around and doesn't affect anyone else. mm_walk_ops affects everybody who walks page tables.

#15 lore raw
Quoted 40 lines · Matthew Wilcox
Quoted 16 lines · Zi Yan
Quoted 15 lines · Johannes Weiner

I know this isn't your intention, but one interesting aspect of supporting PUD mapped folios natively is that it could open the door to simplifying hugetlb as well.

We currently have all kinds of huge_vma checks scattered over the page table code, and entirely parallel paths for unmapping etc. With native PUD mappings, this could allow pushing the special casing out of the virtual memory layer and into where we deal with the page objects.

You might be able to take it as far as the only thing left of hugetlb is the reservation pool. Such that a naive application does mmap() as per usual, and it comes down to a separate allocation policy how the backing pages are served (buddy, CMA, boot-time reservations, ...)

Quoted 8 lines
Quoted 3 lines

Approaching it this way could help separate out the discussion on code impact and tech debt of PUD mappings, from the allocation technique question, which in itself is a fairly large topic.

I agree with this 100%. Adding 1GB folio support first, we then can think about what other THP features, e.g., split, migration, PMD/PTE mapping, are really needed and add them one by one. It is also going to be a good way of retiring hugetlb special code.

But this hasn't happened yet for PMD-sized hugetlb, and there's no need to wait for PUD-sized THP to start this process. I don't think that introducing PUD-sized THP will actually motivate anyone to do this work.

I think we have four main things that hugetlb still offers:

- Reserved pool (mentioned above) which we don't yet have a THP replacement for
- shared page tables. mshare() is the replacement here, and that project is moving along nicely.
- Being able to allocate gigantic folios. This is also progressing.
- Guaranteeing that you don't get a fallback; you either get memory in the size you asked for, or you fail.

Quoted 3 lines

Every time this comes up, I offer the pagewalk code as an egregious example of where we force every user to know "oh, hugetlb is special". Getting rid of mm_walk_ops->hugetlb_entry() would be a great improvement.

I'm all for having a better way to walk folios, and Oscar did some work on that in the past that got stalled (also partially my fault for being overloaded to provide feedback).

But that would be rather replacing mm_walk_ops, not changing it.

The real challenge are not PMD-mapped or PUD-mapped hugetlb folios. These are the easy case.

It's all about the hugetlb corner cases. PTE-batching to handle cont-pte, we already can support. Cont-PMD-mapped hugetlb folios would require similar support (rather easy).

But the real struggle is "fun" hugetlb things like
* PMD table sharing. I don't want ordinary page table walking code to mess with that monstrosity. It can mostly stay in mm/hugetlb.c where it can die a slow death.
* Powerpc stuff like a hugetlb folio that spwans two full page tables. There is no way we are going to add batching support (reading+writing) to generic page table walking code just for that purpose.

Instead, I think we should
* Leave all the mm_walk_ops->hugetlb_entry() stuff alone.
* Enlighten shmem to provide 2M + 1G folios from an alternative allocator (pool) (like guest_memfd is planning)
* Allow hugetlb to server as such a pool. Preparations happening in [1].
* Maybe enlighten shmem to be able to reserve folios in hugetlb.

Essentially: rebuild something like hugetlb for shared memory using shmem, and reuse the existing hugetlb pool. Avoid all the nastiness (weird architecture tweaks, odd folio sizes we don't want to support, huge pmd sharing, etc).

[1] https://lore.kernel.org/r/[email protected]

Signature · 3 lines
Cheers,

David