Skip to contentSkip to conversation

CMA, memdescs and folios

DISCUSSION Matthew Wilcox 3 participants 3 messages Feb 27, 2024 – Mar 5, 2024 Memory management

lore
Share & tools
lore
#0 lore raw

It may be helpful to look at

https://kernelnewbies.org/MatthewWilcox/Memdescs

I don't yet have a plan for what CMA should look like in the memdesc future. Partly I just don't know CMA very well. Some help would be appreciated ...

First, I'm pretty sure that cma allocations are freed as a single unit; there's no intended support for "allocate 2000MB from CMA, free 500MB-1500MB, use the first 500MB for one thing and the last 500MB for something else". Right?

Second, CMA doesn't actually grub around inside struct page itself, so it has no dependencies on what struct page contains. Is that true?

Third, I don't see where CMA manipulates the page refcount today. Does it rely on somebody else setting the page refcount to 1 before giving the pages to CMA?

Fourth, do users of CMA rely on pages being individually refcounted? Is there a reason you've never implemented an equivalent to __GFP_COMP before?

---

My strawman proposal is that, in a memdesc world, the individual pages that are free within CMA get a type 0 subtype to make them readily identifiable in memory dumps. At allocation time, the caller will pass in a memdesc to manage the pages (and CMA will assign it to all the pages, just like the BuddyAllocator will).

As a step towards that, we can change CMA soon to return pages which have a zero refcount. That should catch any users which rely on individual refcounts.

#1 lore raw
Quoted 13 lines · Matthew Wilcox

It may be helpful to look at

https://kernelnewbies.org/MatthewWilcox/Memdescs

I don't yet have a plan for what CMA should look like in the memdesc future. Partly I just don't know CMA very well. Some help would be appreciated ...

First, I'm pretty sure that cma allocations are freed as a single unit; there's no intended support for "allocate 2000MB from CMA, free 500MB-1500MB, use the first 500MB for one thing and the last 500MB for something else". Right?

Quoted 7 lines

Second, CMA doesn't actually grub around inside struct page itself, so it has no dependencies on what struct page contains. Is that true?

Third, I don't see where CMA manipulates the page refcount today. Does it rely on somebody else setting the page refcount to 1 before giving the pages to CMA?

isolate_freepages_range -> split_map_pages -> post_alloc_hook -> set_page_refcounted

Quoted 3 lines

Fourth, do users of CMA rely on pages being individually refcounted? Is there a reason you've never implemented an equivalent to __GFP_COMP before?

for powerpc kvm hash page table usage (kvm_alloc_hpt_cma()/kvm_free_hpt_cma()) I guess we don't expect them to be individually refcounted.

Quoted 12 lines

---

My strawman proposal is that, in a memdesc world, the individual pages that are free within CMA get a type 0 subtype to make them readily identifiable in memory dumps. At allocation time, the caller will pass in a memdesc to manage the pages (and CMA will assign it to all the pages, just like the BuddyAllocator will).

As a step towards that, we can change CMA soon to return pages which have a zero refcount. That should catch any users which rely on individual refcounts.

#2 lore raw
In reply to Matthew Wilcox
Quoted 4 lines · Matthew Wilcox

It may be helpful to look at

https://kernelnewbies.org/MatthewWilcox/Memdescs

Quoted 8 lines

I don't yet have a plan for what CMA should look like in the memdesc future. Partly I just don't know CMA very well. Some help would be appreciated ...

First, I'm pretty sure that cma allocations are freed as a single unit; there's no intended support for "allocate 2000MB from CMA, free 500MB-1500MB, use the first 500MB for one thing and the last 500MB for something else". Right?

hugetlb uses CMA to allocate gigantic folios.

I think we can demote them (e.g., 1 GiB -> 512 x 2 MiB), and then free individual demoted (is that a word?) folios back via CMA.

At least, there is a comment in __update_and_free_hugetlb_folio():

"Non-gigantic pages demoted from CMA allocated gigantic pages need to be given back to CMA in free_gigantic_folio."

And in fact, hugetlb_cma_reserve() prepares exactly for that: it sets the CMA granularity to the smallest folio size we could get after demoting.

"Note that 'order per bit' is based on smallest size that may be returned to CMA allocator in the case of huge page demotion."

I suspect there are other such users that free in different granularity than they allocated.

Signature · 3 lines
Cheers,

David / dhildenb