5.14-rc1 mm/page_alloc.c stray patches
Subsystems (2)
cover 5.14-rc1 mm/page_alloc.c stray patches
This series is some fixes that would have likely have been included in the 5.14-rc1 merge window if they were on time. Mail indicates that some may already be picked up for mmotm but the tree is not up to date yet so I'm including them just in case.
Three are fixes to the bulk memory allocator and one is a fallout from cleaning up warnings that trips BTF that expected a symbol to be global.
mm/page_alloc.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-)
Signature · 1 line
2.26.2
PATCH 1/4 mm/page_alloc: Avoid page allocator recursion with pagesets.lock held
Syzbot is reporting potential deadlocks due to pagesets.lock when PAGE_OWNER is enabled. One example from Desmond Cheong Zhi Xi is as follows
__alloc_pages_bulk()
local_lock_irqsave(&pagesets.lock, flags) <---- outer lock here
prep_new_page():
post_alloc_hook():
set_page_owner():
__set_page_owner():
save_stack():
stack_depot_save():
alloc_pages():
alloc_page_interleave():
__alloc_pages():
get_page_from_freelist():
rm_queue():
rm_queue_pcplist():
local_lock_irqsave(&pagesets.lock, flags);
*** DEADLOCK ***Zhang, Qiang also reported
BUG: sleeping function called from invalid context at mm/page_alloc.c:5179 in_atomic(): 0, irqs_disabled(): 1, non_block: 0, pid: 1, name: swapper/0 ..... __dump_stack lib/dump_stack.c:79 [inline] dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:96 ___might_sleep.cold+0x1f1/0x237 kernel/sched/core.c:9153 prepare_alloc_pages+0x3da/0x580 mm/page_alloc.c:5179 __alloc_pages+0x12f/0x500 mm/page_alloc.c:5375 alloc_page_interleave+0x1e/0x200 mm/mempolicy.c:2147 alloc_pages+0x238/0x2a0 mm/mempolicy.c:2270 stack_depot_save+0x39d/0x4e0 lib/stackdepot.c:303 save_stack+0x15e/0x1e0 mm/page_owner.c:120 __set_page_owner+0x50/0x290 mm/page_owner.c:181 prep_new_page mm/page_alloc.c:2445 [inline] __alloc_pages_bulk+0x8b9/0x1870 mm/page_alloc.c:5313 alloc_pages_bulk_array_node include/linux/gfp.h:557 [inline] vm_area_alloc_pages mm/vmalloc.c:2775 [inline] __vmalloc_area_node mm/vmalloc.c:2845 [inline] __vmalloc_node_range+0x39d/0x960 mm/vmalloc.c:2947 __vmalloc_node mm/vmalloc.c:2996 [inline] vzalloc+0x67/0x80 mm/vmalloc.c:3066
There are a number of ways it could be fixed. The page owner code could be audited to strip GFP flags that allow sleeping but it'll impair the functionality of PAGE_OWNER if allocations fail. The bulk allocator could add a special case to release/reacquire the lock for prep_new_page and lookup PCP after the lock is reacquired at the cost of performance. The pages requiring prep could be tracked using the least significant bit and looping through the array although it is more complicated for the list interface. The options are relatively complex and the second one still incurs a performance penalty when PAGE_OWNER is active so this patch takes the simple approach -- disable bulk allocation if PAGE_OWNER is active. The caller will be forced to allocate one page at a time incurring a performance penalty but PAGE_OWNER is already a performance penalty.
--- mm/page_alloc.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
mm/page_alloc.c +12 -0
@@ -5239,6 +5239,18 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid,if (nr_pages - nr_populated ==1 ) goto failed; +#ifdef CONFIG_PAGE_OWNER +/* +* PAGE_OWNER may recurse into the allocator to allocate space to +* save the stack with pagesets.lock held. Releasing/reacquiring +* removes much of the performance benefit of bulk allocation so +* force the caller to allocate one page at a time as it'll have +* similar performance to added complexity to the bulk allocator. +*/ +if (static_branch_unlikely (&page_owner_inited)) + goto failed; +#endif +/* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */ gfp &= gfp_allowed_mask; alloc_gfp = gfp;
Signature · 1 line
2.26.2
PATCH 2/4 mm/page_alloc: correct return value when failing at preparing
In reply to Mel GormanFrom: Yanfei Xu <[email protected]>
If the array passed in is already partially populated, we should return "nr_populated" even failing at preparing arguments stage.
--- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_alloc.c +1 -1
@@ -5255,7 +5255,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, gfp &= gfp_allowed_mask; alloc_gfp = gfp;if (!prepare_alloc_pages (gfp,0 , preferred_nid, nodemask, &ac, &alloc_gfp, &alloc_flags)) -return 0 ; +return nr_populated; gfp = alloc_gfp;/* Find an allowed local zone that meets the low watermark. */
Signature · 1 line
2.26.2
Re: [PATCH 2/4] mm/page_alloc: correct return value when failing at preparing
Quoted 21 lines · Mel Gorman
From: Yanfei Xu <[email protected]>
If the array passed in is already partially populated, we should return "
nr_populated" even failing at preparing arguments stage.Signed-off-by Yanfei Xu <[email protected]>Signed-off-by Mel Gorman <[email protected]>---
mm/page_alloc.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)mm/page_alloc.c +0
-0
mm/page_alloc.c unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, · line 5258
Quoted 5 lines
mm/page_alloc.c +1
-1@@ -5255,7 +5255,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, gfp &= gfp_allowed_mask; alloc_gfp = gfp;if (!prepare_alloc_pages (gfp,0 , preferred_nid, nodemask, &ac, &alloc_gfp, &alloc_flags)) -return 0 ; +return nr_populated;
Can you restore the hunk in patch 3/4 that changes this "return nr_populated;" to "goto out;" ?
Quoted 6 lines
mm/page_alloc.c +0
-0gfp = alloc_gfp;/* Find an allowed local zone that meets the low watermark. */ Signature · 1 line
2.26.2
--
Chuck Lever
Re: [PATCH 2/4] mm/page_alloc: correct return value when failing at preparing
Quoted 16 lines · Chuck Lever III
Quoted 16 lines · Mel Gorman
From: Yanfei Xu <[email protected]>
If the array passed in is already partially populated, we should return "
nr_populated" even failing at preparing arguments stage.Signed-off-by Yanfei Xu <[email protected]>Signed-off-by Mel Gorman <[email protected]>---
mm/page_alloc.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
mm/page_alloc.c unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, · line 5258
Quoted 10 lines
mm/page_alloc.c +1
-1@@ -5255,7 +5255,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, gfp &= gfp_allowed_mask; alloc_gfp = gfp;if (!prepare_alloc_pages (gfp,0 , preferred_nid, nodemask, &ac, &alloc_gfp, &alloc_flags)) -return 0 ; +return nr_populated;Can you restore the hunk in patch 3/4 that changes this "return
nr_populated;" to "goto out;" ?
I fixed that up in the series I sent out. I applied this patch first then reconciled the collision with your patch.
Signature · 2 lines
Mel Gorman SUSE Labs
Re: [PATCH 2/4] mm/page_alloc: correct return value when failing at preparing
Quoted 36 lines · Mel Gorman
Quoted 16 lines · Chuck Lever III
Quoted 16 lines · Mel Gorman
From: Yanfei Xu <[email protected]>
If the array passed in is already partially populated, we should return "
nr_populated" even failing at preparing arguments stage.Signed-off-by Yanfei Xu <[email protected]>Signed-off-by Mel Gorman <[email protected]>---
mm/page_alloc.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)Quoted 14 lines
mm/page_alloc.c unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, · line 5258
Quoted 10 lines
mm/page_alloc.c +1
-1@@ -5255,7 +5255,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, gfp &= gfp_allowed_mask; alloc_gfp = gfp;if (!prepare_alloc_pages (gfp,0 , preferred_nid, nodemask, &ac, &alloc_gfp, &alloc_flags)) -return 0 ; +return nr_populated;Can you restore the hunk in patch 3/4 that changes this "return
nr_populated;" to "goto out;" ?
Quoted 2 lines
I fixed that up in the series I sent out. I applied this patch first then reconciled the collision with your patch.
3/4 still needs to change this "return nr_populated;" to "goto out;" as part of the clean up.
--
Chuck Lever
Re: [PATCH 2/4] mm/page_alloc: correct return value when failing at preparing
Quoted 36 lines · Chuck Lever III
Quoted 36 lines · Mel Gorman
Quoted 30 lines · Chuck Lever III
Quoted 21 lines · Mel Gorman
From: Yanfei Xu <[email protected]>
If the array passed in is already partially populated, we should return "
nr_populated" even failing at preparing arguments stage.Signed-off-by Yanfei Xu <[email protected]>Signed-off-by Mel Gorman <[email protected]>---
mm/page_alloc.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)mm/page_alloc.c +0
-0mm/page_alloc.c unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, · line 5258
Quoted 5 lines
mm/page_alloc.c +1
-1@@ -5255,7 +5255,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, gfp &= gfp_allowed_mask; alloc_gfp = gfp;if (!prepare_alloc_pages (gfp,0 , preferred_nid, nodemask, &ac, &alloc_gfp, &alloc_flags)) -return 0 ; +return nr_populated;Can you restore the hunk in patch 3/4 that changes this "return
nr_populated;" to "goto out;" ?
Quoted 6 lines
Quoted 2 lines
I fixed that up in the series I sent out. I applied this patch first then reconciled the collision with your patch.
3/4 still needs to change this "return
nr_populated;" to "goto out;" as part of the clean up.
Sorry, I had that in my git tree but didn't refresh the broken-out series properly before sending. I've sent a v2.
Signature · 2 lines
Mel Gorman SUSE Labs
PATCH 3/4 mm/page_alloc: Further fix __alloc_pages_bulk() return value
In reply to Mel GormanFrom: Chuck Lever <[email protected]>
The author of commit b3b64ebd3822 ("mm/page_alloc: do bulk array bounds check after checking populated elements") was possibly confused by the mixture of return values throughout the function.
The API contract is clear that the function "Returns the number of pages on the list or array." It does not list zero as a unique return value with a special meaning. Therefore zero is a plausible return value only if @nr_pages is zero or less.
Clean up the return logic to make it clear that the returned value is always the total number of pages in the array/list, not the number of pages that were allocated during this call.
The only change in behavior with this patch is the value returned if prepare_alloc_pages() fails. To match the API contract, the number of pages currently in the array/list is returned in this case.
The call site in __page_pool_alloc_pages_slow() also seems to be confused on this matter. It should be attended to by someone who is familiar with that code.
--- mm/page_alloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
mm/page_alloc.c +7 -5
@@ -5221,9 +5221,6 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid,unsigned int alloc_flags =ALLOC_WMARK_LOW ;int nr_populated =0 , nr_account =0 ; -if (unlikely (nr_pages <=0 )) -return 0 ; -/* * Skip populated array elements to determine if any pages need * to be allocated before disabling IRQs.
@@ -5231,9 +5228,13 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid,while (page_array && nr_populated < nr_pages && page_array[nr_populated]) nr_populated++; +/* No pages requested? */ +if (unlikely (nr_pages <=0 )) + goto out; +/* Already populated array? */ if (unlikely (page_array && nr_pages - nr_populated ==0 )) -return nr_populated; + goto out;/* Use the single page allocator for one page. */ if (nr_pages - nr_populated ==1 )
@@ -5323,6 +5324,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid,__count_zid_vm_events (PGALLOC ,zone_idx (zone), nr_account);zone_statistics (ac.preferred_zoneref->zone, zone, nr_account); +out:return nr_populated; failed_irq:
@@ -5338,7 +5340,7 @@ unsigned long __alloc_pages_bulk(gfp_t gfp, int preferred_nid, nr_populated++; } -return nr_populated; + goto out; }EXPORT_SYMBOL_GPL (__alloc_pages_bulk);
Signature · 1 line
2.26.2
PATCH 4/4 Revert "mm/page_alloc: make should_fail_alloc_page() static"
In reply to Mel GormanFrom: Matteo Croce <[email protected]>
This reverts commit f7173090033c70886d925995e9dfdfb76dbb2441.
Fix an unresolved symbol error when CONFIG_DEBUG_INFO_BTF=y:
LD vmlinux
BTFIDS vmlinux
FAILED unresolved symbol should_fail_alloc_page
make: *** [Makefile:1199: vmlinux] Error 255
make: *** Deleting file 'vmlinux'
--- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_alloc.c +1 -1
@@ -3820,7 +3820,7 @@ static inline bool __should_fail_alloc_page(gfp_t gfp_mask, unsigned int order)#endif /* CONFIG_FAIL_PAGE_ALLOC */ -static noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order) +noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order) {return __should_fail_alloc_page (gfp_mask, order); }
Signature · 1 line
2.26.2
Re: [PATCH 4/4] Revert "mm/page_alloc: make should_fail_alloc_page() static"
Quoted 5 lines · Mel Gorman
From: Matteo Croce <[email protected]>
This reverts commit
f7173090033c70886d925995e9dfdfb76dbb2441.Fix an unresolved symbol error when
CONFIG_DEBUG_INFO_BTF=y:
I still fundamentally disagreed with this "fix". Whatever code requires a function to be non-static without a prototype and reference is completely fucked up beyond rescue and needs to be disabled util it can be fixed instead of worked around like this.
Re: [PATCH 4/4] Revert "mm/page_alloc: make should_fail_alloc_page() static"
Quoted 11 lines · Christoph Hellwig
Quoted 5 lines · Mel Gorman
From: Matteo Croce <[email protected]>
This reverts commit
f7173090033c70886d925995e9dfdfb76dbb2441.Fix an unresolved symbol error when
CONFIG_DEBUG_INFO_BTF=y:I still fundamentally disagreed with this "fix". Whatever code requires a function to be non-static without a prototype and reference is completely fucked up beyond rescue and needs to be disabled util it can be fixed instead of worked around like this.
I'm definitely not happy with the fix but the breakage was unintentional and given that it was done for a W=1 warning, the patch was low priority and I felt that users that do error injection to stress failure paths at least had some value. If I was fixing something important, I would feel differently and we've slammed patches before that fixed warnings while introducing worse problems. I'm still hoping that BTF gets fixed because it's the right thing to do.
Signature · 2 lines
Mel Gorman SUSE Labs