alloc_pages_bulk
Subsystems (4)
Below is a first cut at a bulk page allocator. This has no testing whatsoever, not even being compiled ... I just want to get some feedback on the approach, so if I get slapped, I'm less far down the path that I have to back out of. The __alloc_pages cleanup is also tacked on the end because I'm lazy at creating diff trees - sorry ;-)
Comments, opinions, abuse?
M.
include/linux/gfp.h +12 -1
@@ -45,12 +45,23 @@static inline struct page *alloc_pages (unsigned int gfp_mask,unsigned int order) { +struct page *page;/* * Gets optimized away by the compiler. */ if (order >=MAX_ORDER )return NULL ; -return _alloc_pages (gfp_mask, order); +return _alloc_pages (gfp_mask, order,1 , &page); +} + +static inline struct page *alloc_pages_bulk (unsigned int gfp_mask,unsigned int order,unsigned long count,struct page **pages) +{ +/* +* Gets optimized away by the compiler. +*/ +if (order >=MAX_ORDER ) +return NULL ; +return _alloc_pages (gfp_mask, order, count, pages); }#define alloc_page (gfp_mask) alloc_pages(gfp_mask, 0)
mm/numa.c +57 -40
138 lines not on this page · raw message
mm/page_alloc.c +58 -44
183 lines not on this page · raw message
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 7 lines · Martin J. Bligh
Below is a first cut at a bulk page allocator. This has no testing whatsoever, not even being compiled ... I just want to get some feedback on the approach, so if I get slapped, I'm less far down the path that I have to back out of. The
__alloc_pagescleanup is also tacked on the end because I'm lazy at creating diff trees - sorry ;-)Comments, opinions, abuse?
The inline for alloc_pages is wasteful: regparm on 386 allows us to pass 3 arguments to a function without going to the stack; by making _alloc_pages take an additional argument which is a pointer, the stack manipulations and dereference add several instructions of bloat per alloc_pages inline. Keep a seperate entry point around for alloc_pages to avoid this. Also, what effect does this have on single page allocations for single processor and dual processor systems?
That said, why use an array? You could just have alloc_pages return a linked list of pages. This would allow you to make the allocation operation faster by doing a single snip of the portion of the list that has the required number of pages in the fast case.
-ben
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 1 line
The inline for
alloc_pagesis wasteful:
if I understand what you're hitting on here, that was already an inline - I didn't do that - if the order is static, which it would be most times, that all just dissappears into a call to _alloc_pages. If we ripped that out, and turned _alloc_pages into alloc_pages, we'd lose that.
Quoted 4 lines
regparm on 386 allows us to pass
3 arguments to a function without going to the stack; by making_alloc_pagestake an additional argument which is a pointer, the stack manipulations and dereference add several instructions of bloat peralloc_pagesinline. Keep
OK, now that I can understand.
Quoted 3 lines
a seperate entry point around for
alloc_pagesto avoid this. Also, what effect does this have on single page allocations for single processor and dual processor systems?
Unmeasured, though it will obviously need to be.
Quoted 4 lines
That said, why use an array? You could just have
alloc_pagesreturn a linked list of pages. This would allow you to make the allocation operation faster by doing a single snip of the portion of the list that has the required number of pages in the fast case.
By doing the mem allocation for that stuff in the top level alloc pages, it's just a local that gets freed before exit. If I allocate it lower down, I have to find some way to track and free it that gets messy. The reason I pushed it so high up was to avoid code duplication (see the NUMA version of _alloc_pages), and I though the cost would be trivial. But as you've pointed out, I've crossed the line onto the stack, which is obviously bad ... time for me to think things over again.
Glad I mailed it out before getting too carried away - thanks very much for the feedback ;-) I think I reorganised numa.c:_alloc_pages enough during the process that the code duplication issue may be totally moot ...
M.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Review
Quoted 5 lines · Martin J. Bligh
min += z->pages_min; if (z->free_pages > min) {(quoted diff) +1
-1- page = rmqueue(z, order); + page = rmqueue(z, order, count, pages);
This won't compile because your rmqueue() no longer returns a page pointer. Minor point, but the weightier question is:
what to do when you fix it up? If the attempt to allocate N pages would cause a watermark to be crossed then should we
a) ignore it, and just return the N pages anyway?
Has a risk that a zillion CPUs could all hit the same code at the same time and would exhaust the page reserves.
That's not very worrying, really.
b) Stop allocating pages when we hit the watermark and return a partial result to the caller
c) Stop allocating pages at the watermark, run off and do some page reclaim and start allocating pages again.
This implies that we should be releasing more than one page into current->local_pages.
Probably we should be doing this anyway - just queue _all_ the liberated pages onto local_pages, then satisfy the request from that pool, then gang-free the remainder. That's a straight-line speedup and lock-banging optimisation against the existing code.
d) Look at pages_min versus count before allocating any pages. If the allocation of `count' pagess would cause ->free_pages to fall below min, then go run some reclaim _first_, and then go grab a number of pages. That number is min(count, nr_pages_we_just_reclaimed). So the caller may see a partial result.
I think d), yes?
-
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 7 lines
Review
Quoted 4 lines
min += z->pages_min; if (z->free_pages > min) {(quoted diff) +1
-1- page = rmqueue(z, order); + page = rmqueue(z, order, count, pages);This won't compile because your
rmqueue()no longer returns a page pointer.
Arse. Will fix that in a second.
Quoted 7 lines
d) Look at
pages_minversus count before allocating any pages. If the allocation of `count' pagess would cause ->free_pages to fall below min, then go run some reclaim _first_, and then go grab a number of pages. That number ismin(count, nr_pages_we_just_reclaimed). So the caller may see a partial result.I think d), yes?
I was under the impression I was already doing that by this bit at the start of the loop:
- min = 1UL << order;
+ min = count << order;
Is that sufficient already (obviously my returns from rmqueue are total crap, but other than that ... ;-))
M.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 10 lines · Martin J. Bligh
Quoted 8 lines
Review
Quoted 5 lines
min += z->pages_min; if (z->free_pages > min) {(quoted diff) +1
-1- page = rmqueue(z, order); + page = rmqueue(z, order, count, pages);This won't compile because your
rmqueue()no longer returns a page pointer.Arse. Will fix that in a second.
No rush. Need to write some code which uses the interface first...
Quoted 14 lines
Quoted 7 lines
d) Look at
pages_minversus count before allocating any pages. If the allocation of `count' pagess would cause ->free_pages to fall below min, then go run some reclaim _first_, and then go grab a number of pages. That number ismin(count, nr_pages_we_just_reclaimed). So the caller may see a partial result.I think d), yes?
I was under the impression I was already doing that by this bit at the start of the loop:
(quoted diff) +1
-1- min = 1UL << order; + min = count << order;
Quoted 2 lines
Is that sufficient already (obviously my returns from rmqueue are total crap, but other than that ... ;-))
Ah. That'll work.
-
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/