mm/percpu: avoid IO dependency in alloc_percpu
Subsystems (2)
PATCH 1/1 mm/percpu: avoid IO dependency in alloc_percpu
percpu_alloc users within noio context may block on other percpu_alloc users doing IO due to the shared pcpu_alloc_mutex. pcpu_balance_workfn already grabs under noio context - update remaining holders to also run under noio.
Lockdep reported this as a circular locking dependency, with one caller having entered fs_reclaim holding pcpu_alloc_mutex, and the second caller (elevator init) grabbing pcpu_alloc_mutex in noio context.
--- mm/percpu.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
mm/percpu.c +7 -2
@@ -1749,8 +1749,13 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,size_t bits, bit_align; gfp =current_gfp_context (gfp); -/* whitelisted flags that can be passed to the backing allocators */ - pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN); +/* +* whitelisted flags that can be passed to the backing allocators. +* +* Allocations must be GFP_NOIO, else users in noio/nofs contexts may +* form dependency through pcpu_alloc_mutex +*/ + pcpu_gfp = gfp & (GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN); is_atomic = !gfpflags_allow_blocking (gfp); do_warn = !(gfp & __GFP_NOWARN);
Signature · 1 line
2.54.0.1189.g8c84645362-goog
Quoted 24 lines · Khazhismel Kumykov
percpu_allocusers within noio context may block on otherpercpu_allocusers doing IO due to the sharedpcpu_alloc_mutex.pcpu_balance_workfnalready grabs under noio context - update remaining holders to also run under noio.Lockdep reported this as a circular locking dependency, with one caller having entered
fs_reclaimholdingpcpu_alloc_mutex, and the second caller (elevator init) grabbingpcpu_alloc_mutexin noio context.Fixes 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic")Signed-off-by Khazhismel Kumykov <[email protected]>--- mm/percpu.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)mm/percpu.c +0
-0@@ -1749,8 +1749,13 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,size_t bits, bit_align;
mm/percpu.c void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved, · line 1758
Quoted 10 lines
mm/percpu.c +7
-2gfp =current_gfp_context (gfp); -/* whitelisted flags that can be passed to the backing allocators */ - pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN); +/* +* whitelisted flags that can be passed to the backing allocators. +* +* Allocations must be GFP_NOIO, else users in noio/nofs contexts may +* form dependency through pcpu_alloc_mutex +*/ + pcpu_gfp = gfp & (GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN);
ah, this is incomplete since we have subsequent allocations that use "GFP_KERNEL", I'll send a version with memalloc_noio instead. Apologies...
Quoted 6 lines
mm/percpu.c +0
-1is_atomic = !gfpflags_allow_blocking (gfp); do_warn = !(gfp & __GFP_NOWARN); --2.54.0.1189.g8c84645362-goog
Quoted 25 lines · Khazhy Kumykov
Quoted 24 lines · Khazhismel Kumykov
percpu_allocusers within noio context may block on otherpercpu_allocusers doing IO due to the sharedpcpu_alloc_mutex.pcpu_balance_workfnalready grabs under noio context - update remaining holders to also run under noio.Lockdep reported this as a circular locking dependency, with one caller having entered
fs_reclaimholdingpcpu_alloc_mutex, and the second caller (elevator init) grabbingpcpu_alloc_mutexin noio context.Fixes 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic")Signed-off-by Khazhismel Kumykov <[email protected]>--- mm/percpu.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)mm/percpu.c +0
-0@@ -1749,8 +1749,13 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,size_t bits, bit_align;
Quoted 13 lines
mm/percpu.c void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved, · line 1758
Quoted 10 lines
mm/percpu.c +7
-2gfp =current_gfp_context (gfp); -/* whitelisted flags that can be passed to the backing allocators */ - pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN); +/* +* whitelisted flags that can be passed to the backing allocators. +* +* Allocations must be GFP_NOIO, else users in noio/nofs contexts may +* form dependency through pcpu_alloc_mutex +*/ + pcpu_gfp = gfp & (GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN);ah, this is incomplete since we have subsequent allocations that use "
GFP_KERNEL", I'll send a version withmemalloc_noioinstead. Apologies...
Please see https://lore.kernel.org/all/[email protected]/
Quoted 6 lines
Quoted 6 lines
mm/percpu.c +0
-1is_atomic = !gfpflags_allow_blocking (gfp); do_warn = !(gfp & __GFP_NOWARN); --2.54.0.1189.g8c84645362-goog