revisiting alloc_pages_bulks semantics?
Subsystems (3)
Hi all,
I've been looking into using alloc_pages_bulks in a few places lately, and have run into issues with the API. Here is my suggestions for how to make this more useful, although only some of them are something I'd feel comfortable to do myself:
1) early fail semantics
alloc_pages_bulks can do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiring GFP_NORETRY or similar to relax it?
2) pre-zeroed page array
There is one single user (svc_fill_pages in sunrpc) that relies on it. For everyone else it creates extra burden and is very error prone (speaking from experience).
3) page instead of folio
We're allocating folios, so we should have a folio API.
4) > order 0 support
The bulk allocator is limited to order 0 which limits it's usefulness these days. It would be really helpful to do bulk allocations for the pagecache or bounce buffering.
Quoted 6 lines · Christoph Hellwig
Hi all,
I've been looking into using
alloc_pages_bulksin a few places lately, and have run into issues with the API. Here is my suggestions for how to make this more useful, although only some of them are something I'd feel comfortable to do myself:
I have some questions below to get more details on your needs.
Quoted 8 lines
1) early fail semantics
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?
IIUC, current alloc_pages_bulks() tries to get free pages without doing compaction or reclaim unless none can be allocated. Does your “usual try” mean possible invocation of compaction and/or reclaim for every page allocation? I guess it also relates to the order > 0 bulk allocation below? My gut feeling is that if one “usual try” fails, the following “usual try” might not work. So making alloc_pages_bulks() do heavy allocation might not buy you much.
But can you elaborate on why looping alloc_pages_bulks() does not work well? That is essentially triggering compaction/reclaim repeatedly like your proposed “usual try” idea.
Quoted 6 lines
2) pre-zeroed page array
There is one single user (
svc_fill_pagesin sunrpc) that relies on it. For everyone else it creates extra burden and is very error prone (speaking from experience).
No comment for this one.
Quoted 4 lines
3) page instead of folio
We're allocating folios, so we should have a folio API.
Sounds reasonable to me.
Quoted 6 lines
4) > order 0 support
The bulk allocator is limited to order 0 which limits it's usefulness these days. It would be really helpful to do bulk allocations for the pagecache or bounce buffering.
Sounds reasonable to me, but when under memory pressure, I wonder how many > order 0 folios you can get in the end. And that might cause a storm of compaction and/or reclaim if combined with Idea 1. For > order 0 bulk allocations, are you thinking about 1) a try and bail-out early model or 2) a keep-trying model? For the latter, I wonder how large the allocation latency can be and if that is tolerable or even makes sense, since for THP allocations, we have seen >30s allocation latency when under memory pressure. Is waiting minutes for bulk > order 0 allocation making sense in your use cases?
Thanks.
Best Regards,
Yan, Zi
Quoted 10 lines · Zi Yan
Quoted 7 lines
1) early fail semantics
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?IIUC, current
alloc_pages_bulks()tries to get free pages without doing compaction or reclaim unless none can be allocated.
Yes, which is really odd, as other page/folio allocators make that an opt-in through GFP flags.
Quoted 3 lines
Does your “usual try”
mean possible invocation of compaction and/or reclaim for every page allocation?
If you look at most callers in tree, and my recently merged or to be merged work isn't any different, they just bloody want the pages just as any other allocator. Failing under grave memory pressure is fine of course, but just failing because getting the memory requires effort is not.
Quoted 4 lines
I guess it also relates to the order > 0 bulk allocation below? My gut feeling is that if one “usual try” fails, the following “usual try” might not work. So making
alloc_pages_bulks()do heavy allocation might not buy you much.
Well, we need to centralize this. Right now there is lots of divering cargo culting in the callers.
Quoted 3 lines
But can you elaborate on why looping
alloc_pages_bulks()does not work well? That is essentially triggering compaction/reclaim repeatedly like your proposed “usual try” idea.
I'm not even sure if it works well. There are some callers that do that, some use individual fallbacks. I don't really want to think about that when all I need is a few folios.
Quoted 7 lines
Quoted 3 lines
The bulk allocator is limited to order 0 which limits it's usefulness these days. It would be really helpful to do bulk allocations for the pagecache or bounce buffering.
Sounds reasonable to me, but when under memory pressure, I wonder how many > order 0 folios you can get in the end. And that might cause a storm of compaction and/or reclaim if combined with Idea 1.
Well, I really want them. In some cases I might be fine falling down to smaller sizes, but I also really don't want the logic in every caller.
Quoted 2 lines
For > order 0 bulk allocations, are you thinking about 1) a try and bail-out early model or 2) a keep-trying model?
Both are useful and as with other allocators should depend on the passed in GFP flags.
Quoted 5 lines
For the latter, I wonder how large the allocation latency can be and if that is tolerable or even makes sense, since for THP allocations, we have seen >30s allocation latency when under memory pressure. Is waiting minutes for bulk > order 0 allocation making sense in your use cases?
The allocations I have in mind would only require try hard allocations for typical file system blocks sizes (64k at most), while eveything larger is fair game for falling back.
Quoted 9 lines · Christoph Hellwig
Quoted 8 lines · Zi Yan
Quoted 7 lines
1) early fail semantics
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?
Quoted 5 lines
Quoted 2 lines
IIUC, current
alloc_pages_bulks()tries to get free pages without doing compaction or reclaim unless none can be allocated.Yes, which is really odd, as other page/folio allocators make that an opt-in through GFP flags.
Based on my understanding of the code, the GFP flags are respected at the __alloc_pages_noprof() in alloc_pages_bulk(). The loop of rmqueue_pcplist() is just a quick try of getting free pages. And I suspect it might be quicker than calling __alloc_pages_noprof() in a loop, since other preparation work in __alloc_pages_noprof() is only done once.
Quoted 31 lines
Quoted 4 lines
Does your “usual try”
mean possible invocation of compaction and/or reclaim for every page allocation?If you look at most callers in tree, and my recently merged or to be merged work isn't any different, they just bloody want the pages just as any other allocator. Failing under grave memory pressure is fine of course, but just failing because getting the memory requires effort is not.
Quoted 4 lines
I guess it also relates to the order > 0 bulk allocation below? My gut feeling is that if one “usual try” fails, the following “usual try” might not work. So making
alloc_pages_bulks()do heavy allocation might not buy you much.Well, we need to centralize this. Right now there is lots of divering cargo culting in the callers.
Quoted 3 lines
But can you elaborate on why looping
alloc_pages_bulks()does not work well? That is essentially triggering compaction/reclaim repeatedly like your proposed “usual try” idea.I'm not even sure if it works well. There are some callers that do that, some use individual fallbacks. I don't really want to think about that when all I need is a few folios.
Quoted 4 lines
Quoted 3 lines
The bulk allocator is limited to order 0 which limits it's usefulness these days. It would be really helpful to do bulk allocations for the pagecache or bounce buffering.
Quoted 7 lines
Quoted 3 lines
Sounds reasonable to me, but when under memory pressure, I wonder how many > order 0 folios you can get in the end. And that might cause a storm of compaction and/or reclaim if combined with Idea 1.
Well, I really want them. In some cases I might be fine falling down to smaller sizes, but I also really don't want the logic in every caller.
Based on your answers above, it sounds like a wrapper of __alloc_pages_bulk() that doing allocation in a loop until all requested pages are filled might be good enough for your case.
But let me know if I miss something.
Quoted 6 lines
Quoted 3 lines
For > order 0 bulk allocations, are you thinking about 1) a try and bail-out early model or 2) a keep-trying model?
Both are useful and as with other allocators should depend on the passed in GFP flags.
Like I said above, __alloc_pages_noprof() in alloc_pages_bulk() respects the GFP flags.
Quoted 10 lines
Quoted 6 lines
For the latter, I wonder how large the allocation latency can be and if that is tolerable or even makes sense, since for THP allocations, we have seen >30s allocation latency when under memory pressure. Is waiting minutes for bulk > order 0 allocation making sense in your use cases?
The allocations I have in mind would only require try hard allocations for typical file system blocks sizes (64k at most), while eveything larger is fair game for falling back.
Sure. In MM, PAGE_ALLOC_COSTLY_ORDER is 3, so pages bigger than that would take more effort to get and the allocation latency can be longer. So it might take a long time to allocate the last 64KB page in a bulk allocation.
I do not have any data for such scenarios, but some trick I can think of is to ask compaction and reclaim to aim for more free pages instead of just the requested order (not higher order), so that after one round of compaction and/or reclaim, more pages at the requested order can be allocated afterwards.
Best Regards,
Yan, Zi
Quoted 5 lines · Zi Yan
Quoted 2 lines
Yes, which is really odd, as other page/folio allocators make that an opt-in through GFP flags.
Based on my understanding of the code, the GFP flags are respected at the
__alloc_pages_noprof()inalloc_pages_bulk().
As __alloc_pages_noprof is the core of the regular page/folio allocator I'd expect that as well.
Quoted 5 lines
The loop of
rmqueue_pcplist()is just a quick try of getting free pages. And I suspect it might be quicker than calling__alloc_pages_noprof()in a loop, since other preparation work in__alloc_pages_noprof()is only done once.
Possibly. But that means a whole bunch of callers have the wrong assumption.
Quoted 9 lines
Quoted 3 lines
Well, I really want them. In some cases I might be fine falling down to smaller sizes, but I also really don't want the logic in every caller.
Based on your answers above, it sounds like a wrapper of
__alloc_pages_bulk()that doing allocation in a loop until all requested pages are filled might be good enough for your case.But let me know if I miss something.
Or just allocate all pages using a loop when alloc_pages_bulk_noprof doesn't get enough pages from the PCP list?
Quoted 8 lines
Quoted 3 lines
The allocations I have in mind would only require try hard allocations for typical file system blocks sizes (64k at most), while eveything larger is fair game for falling back.
Sure. In MM,
PAGE_ALLOC_COSTLY_ORDERis 3, so pages bigger than that would take more effort to get and the allocation latency can be longer. So it might take a long time to allocate the last 64KB page in a bulk allocation.
Based on the LSF/MM session on lage folios and MM fragmentation session it seems like we should raise it to 4 for 4k page size platforms, as this seems to be a proble for 64k folio allocations.
Quoted 7 lines · Christoph Hellwig
Hi all,
I've been looking into using
alloc_pages_bulksin a few places lately, and have run into issues with the API. Here is my suggestions for how to make this more useful, although only some of them are something I'd feel comfortable to do myself:
Quoted 7 lines
1) early fail semantics
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?
If we do that, do we keep the possibility of partial success, i.e. return how many were allocated? Seems wasteful to suceed N-1 and then throw all away, if the caller can use a fallback only for the last one. Do some callers need all-or-nothing semantics? Should a flag indicate which one to use?
Quoted 5 lines
2) pre-zeroed page array
There is one single user (
svc_fill_pagesin sunrpc) that relies on it. For everyone else it creates extra burden and is very error prone (speaking from experience).
Sounds good to me. Will sunrpc be easy to convert, or should it be another flag to opt-in to the current behavior, that it would use?
Quoted 3 lines
3) page instead of folio
We're allocating folios, so we should have a folio API.
Hm, folios initially started as "base or compound page" but then the semantics shifted and now they are also rmappable. See how folio_alloc_noprof() does page_rmappable_folio(). The differences might grow further with memdesc conversion I think.
So do all the callers actually want folios? If not, we could have both alloc_pages_bulk() and folio_alloc_bulk()?
Quoted 5 lines
4) > order 0 support
The bulk allocator is limited to order 0 which limits it's usefulness these days. It would be really helpful to do bulk allocations for the pagecache or bounce buffering.
Fine, with implications for the comment for 1)
Quoted 11 lines · Vlastimil Babka (SUSE)
Quoted 5 lines
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?If we do that, do we keep the possibility of partial success, i.e. return how many were allocated? Seems wasteful to suceed N-1 and then throw all away, if the caller can use a fallback only for the last one. Do some callers need all-or-nothing semantics? Should a flag indicate which one to use?
A lot of callers (but not all) need all or nothing semantics. But freeing already allocated pages is the not a major problem - the caller just has to add a release_pages call if it didn't already have one for cleaning up later failures.
Quoted 6 lines
Quoted 3 lines
There is one single user (
svc_fill_pagesin sunrpc) that relies on it. For everyone else it creates extra burden and is very error prone (speaking from experience).Sounds good to me. Will sunrpc be easy to convert, or should it be another flag to opt-in to the current behavior, that it would use?
I've added Chuck to the Cc list, but from memory sunrpc actually does make use of this feature and he objected to previous attempts to change it. So a first step would be to have a lower-level helper that works as-is and a wrapper that zeroes the array, even if that doesn't feel as efficient as it could be.
Quoted 10 lines
Quoted 3 lines
3) page instead of folio
We're allocating folios, so we should have a folio API.
Hm, folios initially started as "base or compound page" but then the semantics shifted and now they are also rmappable. See how
folio_alloc_noprof()doespage_rmappable_folio(). The differences might grow further with memdesc conversion I think.
So do all the callers actually want folios? If not, we could have bothalloc_pages_bulk()andfolio_alloc_bulk()?
I can't speak for all of them, but many do.
Quoted 4 lines · Christoph Hellwig
Quoted 4 lines
Quoted 3 lines
There is one single user (
svc_fill_pagesin sunrpc) that relies on it. For everyone else it creates extra burden and is very error prone (speaking from experience).
Quoted 8 lines
Quoted 2 lines
Sounds good to me. Will sunrpc be easy to convert, or should it be another flag to opt-in to the current behavior, that it would use?
I've added Chuck to the Cc list, but from memory sunrpc actually does make use of this feature and he objected to previous attempts to change it. So a first step would be to have a lower-level helper that works as-is and a wrapper that zeroes the array, even if that doesn't feel as efficient as it could be.
I think the problem is that sunrpc uses the pages as a queue instead of a stack. If it consumed pages from the end instead of the beginning, it could just refill the entire tail of the array.
Chuck Lever
Reviewed-by Jeff LaytonQuoted 13 lines · Christoph Hellwig
Quoted 11 lines · Vlastimil Babka (SUSE)
Quoted 5 lines
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?If we do that, do we keep the possibility of partial success, i.e. return how many were allocated? Seems wasteful to suceed N-1 and then throw all away, if the caller can use a fallback only for the last one. Do some callers need all-or-nothing semantics? Should a flag indicate which one to use?
Quoted 4 lines
A lot of callers (but not all) need all or nothing semantics. But freeing already allocated pages is the not a major problem - the caller just has to add a
release_pagescall if it didn't already have one for cleaning up later failures.
What the svc/nfsd thread is trying to avoid is sleeping uninterruptibly waiting for memory resources. That stalls server shutdown, among other things.
Guaranteeing forward progress is the bottom line.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
There is one single user (
svc_fill_pagesin sunrpc) that relies on it. For everyone else it creates extra burden and is very error prone (speaking from experience).
Quoted 8 lines
Quoted 2 lines
Sounds good to me. Will sunrpc be easy to convert, or should it be another flag to opt-in to the current behavior, that it would use?
I've added Chuck to the Cc list, but from memory sunrpc actually does make use of this feature and he objected to previous attempts to change it. So a first step would be to have a lower-level helper that works as-is and a wrapper that zeroes the array, even if that doesn't feel as efficient as it could be.
If sunrpc is the only user, it might be sensible to hoist the "zero fill" capability into sunrpc.ko.
The impact of walking the whole array, for this check, is measurable, and I've got a patch in 7.1 to mitigate that:
commit d7f3efd9ff474867b04e1ea784690f02450a245b (refs/patches/nfsd-fixes/sunrpc-optimize-rq_respages-allocation-in-svc_alloc_arg)
Author: Chuck Lever <[email protected]> AuthorDate: Thu Feb 26 09:47:39 2026 -0500 Commit: Chuck Lever <[email protected]> CommitDate: Sun Mar 29 21:25:09 2026 -0400
SUNRPC: Optimize rq_respages allocation in svc_alloc_arg
svc_alloc_arg() invokes alloc_pages_bulk() with the full rq_maxpages count (~259 for 1MB messages) for the rq_respages array, causing a full-array scan despite most slots holding valid pages.
svc_rqst_release_pages() NULLs only the range
[rq_respages, rq_next_page)
after each RPC, so only that range contains NULL entries. Limit the rq_respages fill in svc_alloc_arg() to that range instead of scanning the full array.
svc_init_buffer() initializes rq_next_page to span the entire rq_respages array, so the first svc_alloc_arg() call fills all slots.
Signature · 1 line
Chuck Lever
Quoted 14 lines · Chuck Lever
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Vlastimil Babka (SUSE)
Quoted 5 lines
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?If we do that, do we keep the possibility of partial success, i.e. return how many were allocated? Seems wasteful to suceed N-1 and then throw all away, if the caller can use a fallback only for the last one. Do some callers need all-or-nothing semantics? Should a flag indicate which one to use?
Quoted 8 lines
Quoted 4 lines
A lot of callers (but not all) need all or nothing semantics. But freeing already allocated pages is the not a major problem - the caller just has to add a
release_pagescall if it didn't already have one for cleaning up later failures.What the svc/nfsd thread is trying to avoid is sleeping uninterruptibly waiting for memory resources. That stalls server shutdown, among other things.
I'm not fully understanding the sentence. I guess you mean that you want svc_thread_should_stop to intercept some memory allocation waits?
Quoted 7 lines
Quoted 5 lines
I've added Chuck to the Cc list, but from memory sunrpc actually does make use of this feature and he objected to previous attempts to change it. So a first step would be to have a lower-level helper that works as-is and a wrapper that zeroes the array, even if that doesn't feel as efficient as it could be.
If sunrpc is the only user, it might be sensible to hoist the "zero fill" capability into sunrpc.ko.
As far as I can tell it is the only one. But I don't really see how you could implement that functionality outside the core, except by falling back to single allocations, or looking for empty slots.
I'm curious what you think about willy's comment, or if there is indeed a way to always use the pages from the beginning or end in sunrpc.
Quoted 7 lines
svc_rqst_release_pages()NULLs only the range[
rq_respages,rq_next_page)after each RPC, so only that range contains NULL entries. Limit the
rq_respagesfill insvc_alloc_arg()to that range instead of scanning the full array.
Does it NULL the entire range, or part of it? Because if it is the entire above range, you don't really need the check for NULL behavior at all but just point the bulk allocation to this range.
Quoted 20 lines · Christoph Hellwig
Quoted 19 lines · Chuck Lever
Quoted 17 lines · Christoph Hellwig
Quoted 11 lines · Vlastimil Babka (SUSE)
Quoted 5 lines
alloc_pages_bulkscan do partial allocations for some reasons, and users usually have a fallback by either looping and calling it again or falling back to single page allocations. This sucks! Why can't we get our usual try as hard as you can semantics, requiringGFP_NORETRYor similar to relax it?If we do that, do we keep the possibility of partial success, i.e. return how many were allocated? Seems wasteful to suceed N-1 and then throw all away, if the caller can use a fallback only for the last one. Do some callers need all-or-nothing semantics? Should a flag indicate which one to use?
A lot of callers (but not all) need all or nothing semantics. But freeing already allocated pages is the not a major problem - the caller just has to add a
release_pagescall if it didn't already have one for cleaning up later failures.
Quoted 7 lines
Quoted 3 lines
What the svc/nfsd thread is trying to avoid is sleeping uninterruptibly waiting for memory resources. That stalls server shutdown, among other things.
I'm not fully understanding the sentence. I guess you mean that you want
svc_thread_should_stopto intercept some memory allocation waits?
That's the gist of it.
Quoted 3 lines
I'm curious what you think about willy's comment, or if there is indeed a way to always use the pages from the beginning or end in sunrpc.
It is a ready supply of fresh pages, but it's not used as a simple queue. Many places in sunrpc point to rq_pages and use it as an array.
struct xdr_buf uses this array as the middle payload section of an RPC message:
struct xdr_buf {
struct kvec head[1], /* RPC header + non-page data */
tail[1]; /* Appended after page data */
struct bio_vec *bvec;
struct page ** pages; /* Array of pages */
unsigned int page_base, /* Start of page data */
page_len, /* Length of page data */
flags; /* Flags for data disposition */
unsigned int buflen, /* Total length of storage
buffer */
len; /* Length of XDR encoded message
*/
};
There's no actual array in struct xdr_buf: the "pages" fields points to the rq_pages array that contains the RPC message being processed.
Today, pages are consumed from the beginning of rq_pages.
Quoted 11 lines
Quoted 7 lines
svc_rqst_release_pages()NULLs only the range[
rq_respages,rq_next_page)after each RPC, so only that range contains NULL entries. Limit the
rq_respagesfill insvc_alloc_arg()to that range instead of scanning the full array.Does it NULL the entire range, or part of it? Because if it is the entire above range, you don't really need the check for NULL behavior at all but just point the bulk allocation to this range.
It's two phase.
Phase A prepares the thread (svc_rqst) for the next RPC, and it's done before an RPC is ready to be processed. alloc_bulk_pages refills the NULL entries.
Phase B is done while the client is waiting for the server to send an RPC reply, so it has to be low latency. This is where the pages are "removed" from the array and the array pointers are set to NULL.
Note that as struct xdr_buf is transitioned from "struct page **" to a bvec array, we indeed have good low-risk opportunities to restructure this code and how it uses rq_pages.
Signature · 1 line
Chuck Lever