Skip to contentSkip to conversation

mm/percpu: avoid IO dependency in alloc_percpu

PATCH In review Khazhismel Kumykov 2 participants 3 messages Jun 16

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/percpu: avoid IO dependency in alloc_percpu

percpu_alloc users within noio context may block on other percpu_alloc users doing IO due to the shared pcpu_alloc_mutex. pcpu_balance_workfn already grabs under noio context - update remaining holders to also run under noio.

Lockdep reported this as a circular locking dependency, with one caller having entered fs_reclaim holding pcpu_alloc_mutex, and the second caller (elevator init) grabbing pcpu_alloc_mutex in noio context.

Fixes 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic")
---
 mm/percpu.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
mm/percpu.c +7 -2
index b0676b8054ed..50057403ae79 100644
@@ -1749,8 +1749,13 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,
 	size_t bits, bit_align;
 
 	gfp = current_gfp_context(gfp);
-	/* whitelisted flags that can be passed to the backing allocators */
-	pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
+	/*
+	 * whitelisted flags that can be passed to the backing allocators.
+	 *
+	 * Allocations must be GFP_NOIO, else users in noio/nofs contexts may
+	 * form dependency through pcpu_alloc_mutex
+	 */
+	pcpu_gfp = gfp & (GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN);
 	is_atomic = !gfpflags_allow_blocking(gfp);
 	do_warn = !(gfp & __GFP_NOWARN);
 
Signature · 1 line
2.54.0.1189.g8c84645362-goog
#1 lore raw
Quoted 24 lines · Khazhismel Kumykov

percpu_alloc users within noio context may block on other percpu_alloc users doing IO due to the shared pcpu_alloc_mutex. pcpu_balance_workfn already grabs under noio context - update remaining holders to also run under noio.

Lockdep reported this as a circular locking dependency, with one caller having entered fs_reclaim holding pcpu_alloc_mutex, and the second caller (elevator init) grabbing pcpu_alloc_mutex in noio context.

Fixes 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic")
---
 mm/percpu.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
mm/percpu.c +0 -0
index b0676b8054ed..50057403ae79 100644
@@ -1749,8 +1749,13 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,
        size_t bits, bit_align;
 

mm/percpu.c void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved, · line 1758

Quoted 10 lines
mm/percpu.c +7 -2
        gfp = current_gfp_context(gfp);
-       /* whitelisted flags that can be passed to the backing allocators */
-       pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
+       /*
+        * whitelisted flags that can be passed to the backing allocators.
+        *
+        * Allocations must be GFP_NOIO, else users in noio/nofs contexts may
+        * form dependency through pcpu_alloc_mutex
+        */
+       pcpu_gfp = gfp & (GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN);

ah, this is incomplete since we have subsequent allocations that use "GFP_KERNEL", I'll send a version with memalloc_noio instead. Apologies...

Quoted 6 lines
mm/percpu.c +0 -1
        is_atomic = !gfpflags_allow_blocking(gfp);
        do_warn = !(gfp & __GFP_NOWARN);
 
--

2.54.0.1189.g8c84645362-goog

#2 lore raw
Quoted 25 lines · Khazhy Kumykov
Quoted 24 lines · Khazhismel Kumykov

percpu_alloc users within noio context may block on other percpu_alloc users doing IO due to the shared pcpu_alloc_mutex. pcpu_balance_workfn already grabs under noio context - update remaining holders to also run under noio.

Lockdep reported this as a circular locking dependency, with one caller having entered fs_reclaim holding pcpu_alloc_mutex, and the second caller (elevator init) grabbing pcpu_alloc_mutex in noio context.

Fixes 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic")
---
 mm/percpu.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
mm/percpu.c +0 -0
index b0676b8054ed..50057403ae79 100644
@@ -1749,8 +1749,13 @@ void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved,
        size_t bits, bit_align;
 
Quoted 13 lines

mm/percpu.c void __percpu *pcpu_alloc_noprof(size_t size, size_t align, bool reserved, · line 1758

Quoted 10 lines
mm/percpu.c +7 -2
        gfp = current_gfp_context(gfp);
-       /* whitelisted flags that can be passed to the backing allocators */
-       pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
+       /*
+        * whitelisted flags that can be passed to the backing allocators.
+        *
+        * Allocations must be GFP_NOIO, else users in noio/nofs contexts may
+        * form dependency through pcpu_alloc_mutex
+        */
+       pcpu_gfp = gfp & (GFP_NOIO | __GFP_NORETRY | __GFP_NOWARN);

ah, this is incomplete since we have subsequent allocations that use "GFP_KERNEL", I'll send a version with memalloc_noio instead. Apologies...

Please see https://lore.kernel.org/all/[email protected]/

Quoted 6 lines
Quoted 6 lines
mm/percpu.c +0 -1
        is_atomic = !gfpflags_allow_blocking(gfp);
        do_warn = !(gfp & __GFP_NOWARN);
 
--

2.54.0.1189.g8c84645362-goog