Skip to contentSkip to conversation

svcrdma: Avoid direct reclaim when allocating Read sink buffers

PATCH In review Chuck Lever 2 participants 3 messages Jun 5

Subsystems (4)
lore
Share & tools
lore

Chuck Lever

author maintainer
#0 lore raw

PATCH 1/1 svcrdma: Avoid direct reclaim when allocating Read sink buffers

From: Chuck Lever <[email protected]>

svc_rdma_alloc_read_pages() passes __GFP_NORETRY, which limits the allocator to a single round of direct reclaim and asynchronous compaction per attempt. Under memory pressure or fragmentation that round can take a long time, and the fallback loop repeats it at each order, multiplying the stall while the RPC waits for its Read sink buffer.

The contiguous allocation is opportunistic: when it fails, Read sink buffers come from the pages already in rq_pages[]. Direct reclaim effort buys little here. Allocate with GFP_NOWAIT instead, which omits __GFP_DIRECT_RECLAIM so the allocator takes pages only from the free lists and returns NULL immediately when none are available. GFP_NOWAIT retains __GFP_KSWAPD_RECLAIM, so a failed attempt still wakes kswapd to replenish higher-order pages in the background, and it already includes __GFP_NOWARN. __GFP_NORETRY has no effect once direct reclaim is off. skb_page_frag_refill() takes the same approach for its opportunistic high-order allocation.

Fixes 18755b8c2f24 ("svcrdma: Use contiguous pages for RDMA Read sink buffers")
---
 net/sunrpc/xprtrdma/svc_rdma_rw.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

Given the perf symbol resolution inaccuracies I can't swear this will fix the issue, but here's a stab at it.

net/sunrpc/xprtrdma/svc_rdma_rw.c +9 -9
index 587e4cd29303..efde26cac961 100644
@@ -746,10 +746,9 @@ int svc_rdma_prepare_reply_chunk(struct svcxprt_rdma *rdma,
 }
 
 /*
- * Cap contiguous RDMA Read sink allocations at order-4.
- * Higher orders risk allocation failure under
- * __GFP_NORETRY, which would negate the benefit of the
- * contiguous fast path.
+ * Cap contiguous RDMA Read sink allocations at order-4. Higher orders risk
+ * allocation failure under GFP_NOWAIT, which would negate the benefit of
+ * the contiguous fast path.
  */
 #define SVC_RDMA_CONTIG_MAX_ORDER	4
 
@@ -758,9 +757,11 @@ int svc_rdma_prepare_reply_chunk(struct svcxprt_rdma *rdma,
  * @nr_pages: number of pages needed
  * @order: on success, set to the allocation order
  *
- * Attempts a higher-order allocation, falling back to smaller orders.
- * The returned pages are split immediately so each sub-page has its
- * own refcount and can be freed independently.
+ * Attempts a higher-order allocation, falling back to smaller orders. The
+ * allocation is opportunistic: it takes pages only from the free lists,
+ * without direct reclaim, so it fails fast under memory pressure. The
+ * returned pages are split immediately so each sub-page has its own
+ * refcount and can be freed independently.
  *
  * Returns a pointer to the first page on success, or NULL if even
  * order-1 allocation fails.
@@ -775,8 +776,7 @@ svc_rdma_alloc_read_pages(unsigned int nr_pages, unsigned int *order)
 		SVC_RDMA_CONTIG_MAX_ORDER);
 
 	while (o >= 1) {
-		page = alloc_pages(GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN,
-				   o);
+		page = alloc_pages(GFP_NOWAIT, o);
 		if (page) {
 			split_page(page, o);
 			*order = o;
Signature · 1 line
2.54.0
#1 lore raw

Thanks Chuck, give me a few to test it out!

Quoted 10 lines

-----Original Message-----
From: Chuck Lever <[email protected]>
Sent: Friday, June 5, 2026 4:31 PM
To: Mike Snitzer <[email protected]>
Cc: [email protected]; [email protected]; Chuck Lever <[email protected]>; Jonathan Flynn <[email protected]>
Subject: [PATCH] svcrdma: Avoid direct reclaim when allocating Read sink buffers

Quoted 4 lines

From: Chuck Lever <[email protected]>

svc_rdma_alloc_read_pages() passes __GFP_NORETRY, which limits the allocator to a single round of direct reclaim and asynchronous

compaction per

Quoted 1 line

attempt. Under memory pressure or fragmentation that round can take a

long

Quoted 1 line

time, and the fallback loop repeats it at each order, multiplying the

stall while

Quoted 3 lines

the RPC waits for its Read sink buffer.

The contiguous allocation is opportunistic: when it fails, Read sink

buffers

Quoted 1 line

come from the pages already in rq_pages[]. Direct reclaim effort buys

little

Quoted 2 lines

here. Allocate with GFP_NOWAIT instead, which omits __GFP_DIRECT_RECLAIM so the allocator takes pages only from the free

lists

Quoted 2 lines

and returns NULL immediately when none are available. GFP_NOWAIT retains __GFP_KSWAPD_RECLAIM, so a failed attempt still wakes kswapd to

replenish

Quoted 3 lines

higher-order pages in the background, and it already includes __GFP_NOWARN. __GFP_NORETRY has no effect once direct reclaim is off. skb_page_frag_refill() takes the same approach for its opportunistic

high-

Quoted 4 lines

order allocation.

Fixes 18755b8c2f24 ("svcrdma: Use contiguous pages for RDMA Read sink
Quoted 8 lines

buffers")

---
 net/sunrpc/xprtrdma/svc_rdma_rw.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

Given the perf symbol resolution inaccuracies I can't swear this will

fix the

Quoted 56 lines

issue, but here's a stab at it.

net/sunrpc/xprtrdma/svc_rdma_rw.c +13 -9
index 587e4cd29303..efde26cac961 100644
@@ -746,10 +746,9 @@ int svc_rdma_prepare_reply_chunk(struct
 svcxprt_rdma *rdma,  }
 
 /*
- * Cap contiguous RDMA Read sink allocations at order-4.
- * Higher orders risk allocation failure under
- * __GFP_NORETRY, which would negate the benefit of the
- * contiguous fast path.
+ * Cap contiguous RDMA Read sink allocations at order-4. Higher orders
+ risk
+ * allocation failure under GFP_NOWAIT, which would negate the benefit
+ of
+ * the contiguous fast path.
  */
 #define SVC_RDMA_CONTIG_MAX_ORDER	4
@@ -758,9 +757,11 @@ int svc_rdma_prepare_reply_chunk(struct
 svcxprt_rdma *rdma,
  * @nr_pages: number of pages needed
  * @order: on success, set to the allocation order
  *
- * Attempts a higher-order allocation, falling back to smaller orders.
- * The returned pages are split immediately so each sub-page has its
- * own refcount and can be freed independently.
+ * Attempts a higher-order allocation, falling back to smaller orders.
+ The
+ * allocation is opportunistic: it takes pages only from the free
+ lists,
+ * without direct reclaim, so it fails fast under memory pressure. The
+ * returned pages are split immediately so each sub-page has its own
+ * refcount and can be freed independently.
  *
  * Returns a pointer to the first page on success, or NULL if even
  * order-1 allocation fails.
@@ -775,8 +776,7 @@ svc_rdma_alloc_read_pages(unsigned int nr_pages,
 unsigned int *order)
 		SVC_RDMA_CONTIG_MAX_ORDER);
 
 	while (o >= 1) {
-		page = alloc_pages(GFP_KERNEL | __GFP_NORETRY |
 __GFP_NOWARN,
-				   o);
+		page = alloc_pages(GFP_NOWAIT, o);
 		if (page) {
net/sunrpc/xprtrdma/svc_rdma_rw.c +0 -1
 			split_page(page, o);
 			*order = o;
--

2.54.0

#2 lore raw
In reply to Chuck Lever
Quoted 10 lines

-----Original Message-----
From: Chuck Lever <[email protected]>
Sent: Friday, June 5, 2026 4:31 PM
To: Mike Snitzer <[email protected]>
Cc: [email protected]; [email protected]; Chuck Lever <[email protected]>; Jonathan Flynn <[email protected]>
Subject: [PATCH] svcrdma: Avoid direct reclaim when allocating Read sink buffers

Quoted 4 lines

From: Chuck Lever <[email protected]>

svc_rdma_alloc_read_pages() passes __GFP_NORETRY, which limits the allocator to a single round of direct reclaim and asynchronous

compaction per

Quoted 1 line

attempt. Under memory pressure or fragmentation that round can take a

long

Quoted 1 line

time, and the fallback loop repeats it at each order, multiplying the

stall while

Quoted 3 lines

the RPC waits for its Read sink buffer.

The contiguous allocation is opportunistic: when it fails, Read sink

buffers

Quoted 1 line

come from the pages already in rq_pages[]. Direct reclaim effort buys

little

Quoted 2 lines

here. Allocate with GFP_NOWAIT instead, which omits __GFP_DIRECT_RECLAIM so the allocator takes pages only from the free

lists

Quoted 2 lines

and returns NULL immediately when none are available. GFP_NOWAIT retains __GFP_KSWAPD_RECLAIM, so a failed attempt still wakes kswapd to

replenish

Quoted 3 lines

higher-order pages in the background, and it already includes __GFP_NOWARN. __GFP_NORETRY has no effect once direct reclaim is off. skb_page_frag_refill() takes the same approach for its opportunistic

high-

Quoted 4 lines

order allocation.

Fixes 18755b8c2f24 ("svcrdma: Use contiguous pages for RDMA Read sink
Quoted 8 lines

buffers")

---
 net/sunrpc/xprtrdma/svc_rdma_rw.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

Given the perf symbol resolution inaccuracies I can't swear this will

fix the

Quoted 55 lines

issue, but here's a stab at it.

net/sunrpc/xprtrdma/svc_rdma_rw.c +13 -9
index 587e4cd29303..efde26cac961 100644
@@ -746,10 +746,9 @@ int svc_rdma_prepare_reply_chunk(struct
 svcxprt_rdma *rdma,  }
 
 /*
- * Cap contiguous RDMA Read sink allocations at order-4.
- * Higher orders risk allocation failure under
- * __GFP_NORETRY, which would negate the benefit of the
- * contiguous fast path.
+ * Cap contiguous RDMA Read sink allocations at order-4. Higher orders
+ risk
+ * allocation failure under GFP_NOWAIT, which would negate the benefit
+ of
+ * the contiguous fast path.
  */
 #define SVC_RDMA_CONTIG_MAX_ORDER	4
@@ -758,9 +757,11 @@ int svc_rdma_prepare_reply_chunk(struct
 svcxprt_rdma *rdma,
  * @nr_pages: number of pages needed
  * @order: on success, set to the allocation order
  *
- * Attempts a higher-order allocation, falling back to smaller orders.
- * The returned pages are split immediately so each sub-page has its
- * own refcount and can be freed independently.
+ * Attempts a higher-order allocation, falling back to smaller orders.
+ The
+ * allocation is opportunistic: it takes pages only from the free
+ lists,
+ * without direct reclaim, so it fails fast under memory pressure. The
+ * returned pages are split immediately so each sub-page has its own
+ * refcount and can be freed independently.
  *
  * Returns a pointer to the first page on success, or NULL if even
  * order-1 allocation fails.
@@ -775,8 +776,7 @@ svc_rdma_alloc_read_pages(unsigned int nr_pages,
 unsigned int *order)
 		SVC_RDMA_CONTIG_MAX_ORDER);
 
 	while (o >= 1) {
-		page = alloc_pages(GFP_KERNEL | __GFP_NORETRY |
 __GFP_NOWARN,
-				   o);
+		page = alloc_pages(GFP_NOWAIT, o);
 		if (page) {
net/sunrpc/xprtrdma/svc_rdma_rw.c +0 -1
 			split_page(page, o);
 			*order = o;
--
Quoted 1 line

2.54.0

Unfortunately, the GFP_NOWAIT change did not materially affect either throughput or the perf profile. The allocator-heavy stack rooted at svc_rdma_build_read_segment_contig() remains dominant, with alloc_pages_noprof() and rmqueue_buddy() continuing to account for a significant portion of the samples, similar to the original regressed build.

I have added a gfp-nowait directory to the OneDrive link referenced in my previous email. It contains the fio results, perf reports, and a flamegraph for the GFP_NOWAIT test.

I have also added a flamegraph to:

rpcrdma-regression/regressed/phase2/server

for the original regressed configuration.

-Jon