Allocate payload arrays dynamically
Subsystems (3)
[email protected]
maintainerv2 cover Allocate payload arrays dynamically
From: Chuck Lever <[email protected]>
In order to make RPCSVC_MAXPAYLOAD larger (or variable in size), we need to do something clever with the payload arrays embedded in struct svc_rqst and elsewhere.
My preference is to keep these arrays allocated all the time because allocating them on demand increases the risk of a memory allocation failure during a large I/O. This is a quick-and-dirty approach that might be replaced once NFSD is converted to use large folios.
The downside of this design choice is that it pins a few pages per NFSD thread (and that's the current situation already). But note that because RPCSVC_MAXPAGES is 259, each array is just over a page in size, making the allocation waste quite a bit of memory beyond the end of the array due to power-of-2 allocator round up. This gets worse as the MAXPAGES value is doubled or quadrupled.
This series also addresses similar issues in the socket and RDMA transports.
Chuck Lever (9):
sunrpc: Remove backchannel check in svc_init_buffer()
sunrpc: Add a helper to derive maxpages from sv_max_mesg
sunrpc: Replace the rq_pages array with dynamically-allocated memory
sunrpc: Replace the rq_vec array with dynamically-allocated memory
sunrpc: Replace the rq_bvec array with dynamically-allocated memory
sunrpc: Adjust size of socket's receive page array dynamically
svcrdma: Adjust the number of RDMA contexts per transport
svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages
svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages
fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 19 +++++++-- include/linux/sunrpc/svc_rdma.h | 6 ++- include/linux/sunrpc/svcsock.h | 4 +- net/sunrpc/svc.c | 51 +++++++++++++++--------- net/sunrpc/svc_xprt.c | 10 +---- net/sunrpc/svcsock.c | 15 ++++--- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 +++- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 ++++++-- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 12 files changed, 88 insertions(+), 48 deletions(-)
Signature · 31 lines
2.49.0 Chuck Lever (10): sunrpc: Remove backchannel check in svc_init_buffer() sunrpc: Add a helper to derive maxpages from sv_max_mesg sunrpc: Replace the rq_pages array with dynamically-allocated memory sunrpc: Replace the rq_vec array with dynamically-allocated memory sunrpc: Replace the rq_bvec array with dynamically-allocated memory sunrpc: Adjust size of socket's receive page array dynamically svcrdma: Adjust the number of RDMA contexts per transport svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages sunrpc: Remove the RPCSVC_MAXPAGES macro fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 31 +++++++++----- include/linux/sunrpc/svc_rdma.h | 6 ++- include/linux/sunrpc/svcsock.h | 4 +- net/sunrpc/svc.c | 51 +++++++++++++++--------- net/sunrpc/svc_xprt.c | 10 +---- net/sunrpc/svcsock.c | 15 ++++--- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 +++- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 ++++++-- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 12 files changed, 93 insertions(+), 55 deletions(-) -- 2.49.0
[email protected]
maintainerPATCH v2 1/10 sunrpc: Remove backchannel check in svc_init_buffer()
From: Chuck Lever <[email protected]>
A backchannel service might use forechannel send and receive buffers, but svc_recv() still calls svc_alloc_arg() on backchannel svc_rqsts, so it will populate the svc_rqst's rq_pages[] array anyway. The "is backchannel" check in svc_init_buffer() saves us nothing.
In a moment, I plan to replace the rq_pages[] array with a dynamically-allocated piece of memory, and svc_init_buffer() is where that memory will get allocated. Backchannel requests actually do use that array, so it has to be available to handle those requests without a segfault.
XXX: Or, make svc_alloc_arg() ignore backchannel requests too?
Could set rqstp->rq_maxpages to zero.
--- net/sunrpc/svc.c | 4 ---- 1 file changed, 4 deletions(-)
net/sunrpc/svc.c +0 -4
@@ -640,10 +640,6 @@ svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node) {unsigned long pages, ret; -/* bc_xprt uses fore channel allocated buffers */ -if (svc_is_backchannel (rqstp)) -return true; - pages = size /PAGE_SIZE +1 ;/* extra page as we hold both request and reply. * We assume one is at most one page */
Signature · 1 line
2.49.0
Re: [PATCH v2 01/10] sunrpc: Remove backchannel check in svc_init_buffer()
Quoted 8 lines · On Sat, 2025-04-19 at 13:28 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
A backchannel service might use forechannel send and receive buffers, but
svc_recv()still callssvc_alloc_arg()on backchannelsvc_rqsts, so it will populate the svc_rqst's rq_pages[] array anyway. The "is backchannel" check insvc_init_buffer()saves us nothing.
Quoted 9 lines
In a moment, I plan to replace the rq_pages[] array with a dynamically-allocated piece of memory, and
svc_init_buffer()is where that memory will get allocated. Backchannel requests actually do use that array, so it has to be available to handle those requests without a segfault.XXX: Or, make
svc_alloc_arg()ignore backchannel requests too?Could set rqstp->rq_maxpages to zero.
Maybe I'm confused here, but the backchannel still needs some pages to do its thing? I guess the main change here is that the pages are allocated at svc_create time instead of waiting until later?
Quoted 10 lines
Signed-off-by Chuck Lever <[email protected]>--- net/sunrpc/svc.c | 4 ---- 1 file changed, 4 deletions(-)net/sunrpc/svc.c +0
-0
net/sunrpc/svc.c svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node) · line 645
Quoted 10 lines
net/sunrpc/svc.c +0
-4@@ -640,10 +640,6 @@ svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node) {unsigned long pages, ret; -/* bc_xprt uses fore channel allocated buffers */ -if (svc_is_backchannel (rqstp)) -return true; - pages = size /PAGE_SIZE +1 ;/* extra page as we hold both request and reply. * We assume one is at most one page */
Chuck Lever
maintainerRe: [PATCH v2 01/10] sunrpc: Remove backchannel check in svc_init_buffer()
Quoted 15 lines · Jeff Layton
Quoted 14 lines · On Sat, 2025-04-19 at 13:28 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
A backchannel service might use forechannel send and receive buffers, but
svc_recv()still callssvc_alloc_arg()on backchannelsvc_rqsts, so it will populate the svc_rqst's rq_pages[] array anyway. The "is backchannel" check insvc_init_buffer()saves us nothing.In a moment, I plan to replace the rq_pages[] array with a dynamically-allocated piece of memory, and
svc_init_buffer()is where that memory will get allocated. Backchannel requests actually do use that array, so it has to be available to handle those requests without a segfault.
Quoted 7 lines
Quoted 3 lines
XXX: Or, make
svc_alloc_arg()ignore backchannel requests too?Could set rqstp->rq_maxpages to zero.Maybe I'm confused here, but the backchannel still needs some pages to do its thing? I guess the main change here is that the pages are allocated at
svc_createtime instead of waiting until later?
The backchannel does not use the pages in svc_rqst::rq_pages, it's rq_arg::pages comes from the RPC client's page allocator. Currently, svc_init_buffer() skips allocating pages in rq_pages for that reason.
Except that, svc_rqst::rq_pages is filled anyway when a backchannel svc_rqst is passed to svc_recv() -> and then to svc_alloc_arg().
This isn't really a problem at the moment, except that these pages are allocated but then never used AFAICT.
The problem is that in 03/10, in addition to populating rq_pages[], svc_init_buffer() also now allocates the rq_pages array itself. If that is skipped, then svc_alloc_args() chases a NULL pointer.
As I mentioned in the patch description, there are two or three ways to handle this. I'm not sure the way proposed here is best, but it does avoid introducing extra conditional logic in svc_alloc_args(), which is a hot path.
Let me know if you have a better idea. I will try to bolster the patch description here as well.
Quoted 22 lines
Quoted 10 lines
Signed-off-by Chuck Lever <[email protected]>--- net/sunrpc/svc.c | 4 ---- 1 file changed, 4 deletions(-)net/sunrpc/svc.c +0
-0net/sunrpc/svc.c svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node) · line 645
Quoted 10 lines
net/sunrpc/svc.c +0
-4@@ -640,10 +640,6 @@ svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node) {unsigned long pages, ret; -/* bc_xprt uses fore channel allocated buffers */ -if (svc_is_backchannel (rqstp)) -return true; - pages = size /PAGE_SIZE +1 ;/* extra page as we hold both request and reply. * We assume one is at most one page */ Acked-by Jeff Layton <[email protected]>
Signature · 1 line
Chuck Lever
[email protected]
maintainerPATCH v2 2/10 sunrpc: Add a helper to derive maxpages from sv_max_mesg
In reply to [email protected]From: Chuck Lever <[email protected]>
This page count is to be used to allocate various arrays of pages, bio_vecs, and kvecs, replacing the fixed RPCSVC_MAXPAGES value.
--- include/linux/sunrpc/svc.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)
include/linux/sunrpc/svc.h +12 -0
@@ -159,6 +159,18 @@ extern u32 svc_max_payload(const struct svc_rqst *rqstp);#define RPCSVC_MAXPAGES ((RPCSVC_MAXPAYLOAD+PAGE_SIZE-1)/PAGE_SIZE \ + 2 + 1) +/** +* svc_serv_maxpages - maximum pages/kvecs needed for one RPC message +* @serv: RPC service context +* +* Returns a count of pages or vectors that can hold the maximum +* size RPC message for @serv. +*/ +static inline unsigned long svc_serv_maxpages (const struct svc_serv *serv) +{ +return ((serv->sv_max_mesg +PAGE_SIZE -1 ) >>PAGE_SHIFT ) +2 +1 ; +} +/* * The context of a single thread, including the request currently being * processed.
Signature · 1 line
2.49.0
NeilBrown
reviewerRe: [PATCH v2 02/10] sunrpc: Add a helper to derive maxpages from sv_max_mesg
Quoted 20 lines · On Sun, 20 Apr 2025, [email protected] wrote:
From: Chuck Lever <[email protected]>
This page count is to be used to allocate various arrays of pages,
bio_vecs, and kvecs, replacing the fixedRPCSVC_MAXPAGESvalue.Signed-off-by Chuck Lever <[email protected]>--- include/linux/sunrpc/svc.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)include/linux/sunrpc/svc.h +2
-0@@ -159,6 +159,18 @@ extern u32 svc_max_payload(const struct svc_rqst *rqstp);#define RPCSVC_MAXPAGES ((RPCSVC_MAXPAYLOAD+PAGE_SIZE-1)/PAGE_SIZE \ + 2 + 1) +/** +* svc_serv_maxpages - maximum pages/kvecs needed for one RPC message
include/linux/sunrpc/svc.h extern u32 svc_max_payload(const struct svc_rqst *rqstp); · line 173
Quoted 10 lines
include/linux/sunrpc/svc.h +10
-0+* @serv: RPC service context +* +* Returns a count of pages or vectors that can hold the maximum +* size RPC message for @serv. +*/ +static inline unsigned long svc_serv_maxpages (const struct svc_serv *serv) +{ +return ((serv->sv_max_mesg +PAGE_SIZE -1 ) >>PAGE_SHIFT ) +2 +1 ; +} +
This looks like it should be
DIV_ROUND_UP(serv->sv_max_mesg, PAGE_SIZE) + 2
Could we document what the "+ 2" is for??
Thanks,
NeilBrown
Quoted 7 lines
include/linux/sunrpc/svc.h +0
-0/* * The context of a single thread, including the request currently being * processed. Signature · 1 line
2.49.0
Chuck Lever
maintainerRe: [PATCH v2 02/10] sunrpc: Add a helper to derive maxpages from sv_max_mesg
Quoted 33 lines · NeilBrown
Quoted 20 lines · On Sun, 20 Apr 2025, [email protected] wrote:
From: Chuck Lever <[email protected]>
This page count is to be used to allocate various arrays of pages,
bio_vecs, and kvecs, replacing the fixedRPCSVC_MAXPAGESvalue.Signed-off-by Chuck Lever <[email protected]>--- include/linux/sunrpc/svc.h | 12 ++++++++++++ 1 file changed, 12 insertions(+)include/linux/sunrpc/svc.h +2
-0@@ -159,6 +159,18 @@ extern u32 svc_max_payload(const struct svc_rqst *rqstp);#define RPCSVC_MAXPAGES ((RPCSVC_MAXPAYLOAD+PAGE_SIZE-1)/PAGE_SIZE \ + 2 + 1) +/** +* svc_serv_maxpages - maximum pages/kvecs needed for one RPC message include/linux/sunrpc/svc.h extern u32 svc_max_payload(const struct svc_rqst *rqstp); · line 173
Quoted 10 lines
include/linux/sunrpc/svc.h +10
-0+* @serv: RPC service context +* +* Returns a count of pages or vectors that can hold the maximum +* size RPC message for @serv. +*/ +static inline unsigned long svc_serv_maxpages (const struct svc_serv *serv) +{ +return ((serv->sv_max_mesg +PAGE_SIZE -1 ) >>PAGE_SHIFT ) +2 +1 ; +} +This looks like it should be
Quoted 1 line
DIV_ROUND_UP(serv->sv_max_mesg, PAGE_SIZE) + 2
Fair enough! I had forgotten about that macro.
Quoted 1 line
Could we document what the "+ 2" is for??
10/10 in this series moves an existing documenting comment for that purpose. I've relocated that hunk to this patch, but I think the text needs to be revisited because it ignores NFSv4 COMPOUNDs that may carry multiple payloads.
I plan to post a v3 of this series soon.
Quoted 12 lines
Thanks,
NeilBrownQuoted 7 lines
include/linux/sunrpc/svc.h +0
-0/* * The context of a single thread, including the request currently being * processed. Signature · 1 line
2.49.0
Signature · 1 line
Chuck Lever
[email protected]
maintainerPATCH v2 3/10 sunrpc: Replace the rq_pages array with dynamically-allocated memory
In reply to [email protected]From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, replace the fixed-size rq_vec[] array in struct svc_rqst with a chunk of dynamically-allocated memory.
On a system with 8-byte pointers and 4KB pages, pahole reports that the rq_pages[] array is 2080 bytes. Replacing it with a single pointer reduces the size of struct svc_rqst to just over 9500 bytes.
--- include/linux/sunrpc/svc.h | 3 ++- net/sunrpc/svc.c | 34 ++++++++++++++++++------------- net/sunrpc/svc_xprt.c | 10 +-------- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- 4 files changed, 24 insertions(+), 25 deletions(-)
include/linux/sunrpc/svc.h +2 -1
@@ -200,7 +200,8 @@ struct svc_rqst {struct xdr_stream rq_res_stream;struct page *rq_scratch_page;struct xdr_buf rq_res; -struct page *rq_pages[RPCSVC_MAXPAGES +1 ]; +unsigned long rq_maxpages;/* num of entries in rq_pages */ +struct page * *rq_pages;struct page * *rq_respages;/* points into rq_pages */ struct page * *rq_next_page;/* next reply page to use */ struct page * *rq_page_end;/* one past the last page */
net/sunrpc/svc.c +20 -14
@@ -636,20 +636,25 @@ svc_destroy(struct svc_serv **servp)EXPORT_SYMBOL_GPL (svc_destroy);static bool -svc_init_buffer (struct svc_rqst *rqstp,unsigned int size,int node) +svc_init_buffer (struct svc_rqst *rqstp,const struct svc_serv *serv,int node) { -unsigned long pages, ret; +unsigned long ret; - pages = size /PAGE_SIZE +1 ;/* extra page as we hold both request and reply. -* We assume one is at most one page -*/ -WARN_ON_ONCE (pages >RPCSVC_MAXPAGES ); -if (pages >RPCSVC_MAXPAGES ) - pages =RPCSVC_MAXPAGES ; +/* Add an extra page, as rq_pages holds both request and reply. +* We assume one of those is at most one page. +*/ + rqstp->rq_maxpages =svc_serv_maxpages (serv) +1 ; - ret =alloc_pages_bulk_node (GFP_KERNEL , node, pages, +/* rq_pages' last entry is NULL for historical reasons. */ + rqstp->rq_pages =kcalloc_node (rqstp->rq_maxpages +1 , +sizeof (struct page *), +GFP_KERNEL , node); +if (!rqstp->rq_pages) +return false; + + ret =alloc_pages_bulk_node (GFP_KERNEL , node, rqstp->rq_maxpages, rqstp->rq_pages); -return ret == pages; +return ret == rqstp->rq_maxpages; }/*
@@ -658,11 +663,12 @@ svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node)static void svc_release_buffer (struct svc_rqst *rqstp) { -unsigned int i; +unsigned long i; -for (i =0 ; i <ARRAY_SIZE (rqstp->rq_pages); i++) +for (i =0 ; i < rqstp->rq_maxpages; i++)if (rqstp->rq_pages[i])put_page (rqstp->rq_pages[i]); +kfree (rqstp->rq_pages); } static void
@@ -704,7 +710,7 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node)if (!rqstp->rq_resp) goto out_enomem; -if (!svc_init_buffer (rqstp, serv->sv_max_mesg, node)) +if (!svc_init_buffer (rqstp, serv, node)) goto out_enomem; rqstp->rq_err = -EAGAIN ;/* No error yet */
@@ -896,7 +902,7 @@ EXPORT_SYMBOL_GPL(svc_set_num_threads);bool svc_rqst_replace_page (struct svc_rqst *rqstp,struct page *page) {struct page **begin = rqstp->rq_pages; -struct page **end = &rqstp->rq_pages[RPCSVC_MAXPAGES ]; +struct page **end = &rqstp->rq_pages[rqstp->rq_maxpages];if (unlikely (rqstp->rq_next_page < begin || rqstp->rq_next_page > end)) {trace_svc_replace_page_err (rqstp);
net/sunrpc/svc_xprt.c +1 -9
@@ -651,18 +651,10 @@ static void svc_check_conn_limits(struct svc_serv *serv)static bool svc_alloc_arg (struct svc_rqst *rqstp) { -struct svc_serv *serv = rqstp->rq_server;struct xdr_buf *arg = &rqstp->rq_arg;unsigned long pages, filled, ret; - pages = (serv->sv_max_mesg +2 *PAGE_SIZE ) >>PAGE_SHIFT ; -if (pages >RPCSVC_MAXPAGES ) { -pr_warn_once ("svc: warning: pages=%lu > RPCSVC_MAXPAGES=%lu\n" , - pages,RPCSVC_MAXPAGES ); -/* use as many pages as possible */ - pages =RPCSVC_MAXPAGES ; - } - + pages = rqstp->rq_maxpages;for (filled =0 ; filled < pages; filled = ret) { ret =alloc_pages_bulk (GFP_KERNEL , pages, rqstp->rq_pages);if (ret > filled)
net/sunrpc/xprtrdma/svc_rdma_rw.c +1 -1
@@ -765,7 +765,7 @@ static int svc_rdma_build_read_segment(struct svc_rqst *rqstp, } len -= seg_len; -if (len && ((head->rc_curpage +1 ) >ARRAY_SIZE (rqstp->rq_pages))) +if (len && ((head->rc_curpage +1 ) > rqstp->rq_maxpages)) goto out_overrun; }
Signature · 1 line
2.49.0
Re: [PATCH v2 03/10] sunrpc: Replace the rq_pages array with dynamically-allocated memory
Quoted 134 lines · On Sat, 2025-04-19 at 13:28 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, replace the fixed-size rq_vec[] array in struct
svc_rqstwith a chunk of dynamically-allocated memory.On a system with 8-byte pointers and 4KB pages, pahole reports that the rq_pages[] array is 2080 bytes. Replacing it with a single pointer reduces the size of struct
svc_rqstto just over 9500 bytes.Signed-off-by Chuck Lever <[email protected]>--- include/linux/sunrpc/svc.h | 3 ++- net/sunrpc/svc.c | 34 ++++++++++++++++++------------- net/sunrpc/svc_xprt.c | 10 +-------- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- 4 files changed, 24 insertions(+), 25 deletions(-)include/linux/sunrpc/svc.h +2
-1@@ -200,7 +200,8 @@ struct svc_rqst {struct xdr_stream rq_res_stream;struct page *rq_scratch_page;struct xdr_buf rq_res; -struct page *rq_pages[RPCSVC_MAXPAGES +1 ]; +unsigned long rq_maxpages;/* num of entries in rq_pages */ +struct page * *rq_pages;struct page * *rq_respages;/* points into rq_pages */ struct page * *rq_next_page;/* next reply page to use */ struct page * *rq_page_end;/* one past the last page */ net/sunrpc/svc.c +20
-14@@ -636,20 +636,25 @@ svc_destroy(struct svc_serv **servp)EXPORT_SYMBOL_GPL (svc_destroy);static bool -svc_init_buffer (struct svc_rqst *rqstp,unsigned int size,int node) +svc_init_buffer (struct svc_rqst *rqstp,const struct svc_serv *serv,int node) { -unsigned long pages, ret; +unsigned long ret; - pages = size /PAGE_SIZE +1 ;/* extra page as we hold both request and reply. -* We assume one is at most one page -*/ -WARN_ON_ONCE (pages >RPCSVC_MAXPAGES ); -if (pages >RPCSVC_MAXPAGES ) - pages =RPCSVC_MAXPAGES ; +/* Add an extra page, as rq_pages holds both request and reply. +* We assume one of those is at most one page. +*/ + rqstp->rq_maxpages =svc_serv_maxpages (serv) +1 ; - ret =alloc_pages_bulk_node (GFP_KERNEL , node, pages, +/* rq_pages' last entry is NULL for historical reasons. */ + rqstp->rq_pages =kcalloc_node (rqstp->rq_maxpages +1 , +sizeof (struct page *), +GFP_KERNEL , node); +if (!rqstp->rq_pages) +return false; + + ret =alloc_pages_bulk_node (GFP_KERNEL , node, rqstp->rq_maxpages, rqstp->rq_pages); -return ret == pages; +return ret == rqstp->rq_maxpages; }/* @@ -658,11 +663,12 @@ svc_init_buffer(struct svc_rqst *rqstp, unsigned int size, int node)static void svc_release_buffer (struct svc_rqst *rqstp) { -unsigned int i; +unsigned long i; -for (i =0 ; i <ARRAY_SIZE (rqstp->rq_pages); i++) +for (i =0 ; i < rqstp->rq_maxpages; i++)if (rqstp->rq_pages[i])put_page (rqstp->rq_pages[i]); +kfree (rqstp->rq_pages); } static void@@ -704,7 +710,7 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node)if (!rqstp->rq_resp) goto out_enomem; -if (!svc_init_buffer (rqstp, serv->sv_max_mesg, node)) +if (!svc_init_buffer (rqstp, serv, node)) goto out_enomem; rqstp->rq_err = -EAGAIN ;/* No error yet */ @@ -896,7 +902,7 @@ EXPORT_SYMBOL_GPL(svc_set_num_threads);bool svc_rqst_replace_page (struct svc_rqst *rqstp,struct page *page) {struct page **begin = rqstp->rq_pages; -struct page **end = &rqstp->rq_pages[RPCSVC_MAXPAGES ]; +struct page **end = &rqstp->rq_pages[rqstp->rq_maxpages];if (unlikely (rqstp->rq_next_page < begin || rqstp->rq_next_page > end)) {trace_svc_replace_page_err (rqstp);net/sunrpc/svc_xprt.c +1
-9@@ -651,18 +651,10 @@ static void svc_check_conn_limits(struct svc_serv *serv)static bool svc_alloc_arg (struct svc_rqst *rqstp) { -struct svc_serv *serv = rqstp->rq_server;struct xdr_buf *arg = &rqstp->rq_arg;unsigned long pages, filled, ret; - pages = (serv->sv_max_mesg +2 *PAGE_SIZE ) >>PAGE_SHIFT ; -if (pages >RPCSVC_MAXPAGES ) { -pr_warn_once ("svc: warning: pages=%lu > RPCSVC_MAXPAGES=%lu\n" , - pages,RPCSVC_MAXPAGES ); -/* use as many pages as possible */ - pages =RPCSVC_MAXPAGES ; - } - + pages = rqstp->rq_maxpages;for (filled =0 ; filled < pages; filled = ret) { ret =alloc_pages_bulk (GFP_KERNEL , pages, rqstp->rq_pages);if (ret > filled)net/sunrpc/xprtrdma/svc_rdma_rw.c +0
-0
net/sunrpc/xprtrdma/svc_rdma_rw.c static int svc_rdma_build_read_segment(struct svc_rqst *rqstp, · line 771
Quoted 8 lines
net/sunrpc/xprtrdma/svc_rdma_rw.c +1
-1@@ -765,7 +765,7 @@ static int svc_rdma_build_read_segment(struct svc_rqst *rqstp, } len -= seg_len; -if (len && ((head->rc_curpage +1 ) >ARRAY_SIZE (rqstp->rq_pages))) +if (len && ((head->rc_curpage +1 ) > rqstp->rq_maxpages)) goto out_overrun; }
nit: I'd probably squash 2 and 3 together, but they both look right:
[email protected]
maintainerPATCH v2 4/10 sunrpc: Replace the rq_vec array with dynamically-allocated memory
In reply to [email protected]From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, replace the fixed-size rq_vec[] array in struct svc_rqst with a chunk of dynamically-allocated memory.
The rq_vec array is sized assuming request processing will need at most one kvec per page in a maximum-sized RPC message.
On a system with 8-byte pointers and 4KB pages, pahole reports that the rq_vec[] array is 4144 bytes. Replacing it with a single pointer reduces the size of struct svc_rqst to about 5400 bytes.
--- fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 2 +- net/sunrpc/svc.c | 8 +++++++- 4 files changed, 9 insertions(+), 4 deletions(-)
fs/nfsd/nfs4proc.c +0 -1
@@ -1228,7 +1228,6 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, write->wr_how_written = write->wr_stable_how; nvecs =svc_fill_write_vector (rqstp, &write->wr_payload); -WARN_ON_ONCE (nvecs >ARRAY_SIZE (rqstp->rq_vec)); status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf, write->wr_offset, rqstp->rq_vec, nvecs, &cnt,
fs/nfsd/vfs.c +1 -1
@@ -1094,7 +1094,7 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp, ++v; base =0 ; } -WARN_ON_ONCE (v >ARRAY_SIZE (rqstp->rq_vec)); +WARN_ON_ONCE (v > rqstp->rq_maxpages);trace_nfsd_read_vector (rqstp, fhp, offset, *count);iov_iter_kvec (&iter,ITER_DEST , rqstp->rq_vec, v, *count);
include/linux/sunrpc/svc.h +1 -1
@@ -207,7 +207,7 @@ struct svc_rqst {struct page * *rq_page_end;/* one past the last page */ struct folio_batch rq_fbatch; -struct kvec rq_vec[RPCSVC_MAXPAGES ];/* generally useful.. */ +struct kvec *rq_vec;struct bio_vec rq_bvec[RPCSVC_MAXPAGES ];__be32 rq_xid;/* transmission id */
net/sunrpc/svc.c +7 -1
@@ -675,6 +675,7 @@ static void svc_rqst_free(struct svc_rqst *rqstp) {folio_batch_release (&rqstp->rq_fbatch); +kfree (rqstp->rq_vec);svc_release_buffer (rqstp);if (rqstp->rq_scratch_page)put_page (rqstp->rq_scratch_page);
@@ -713,6 +714,11 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node)if (!svc_init_buffer (rqstp, serv, node)) goto out_enomem; + rqstp->rq_vec =kcalloc_node (rqstp->rq_maxpages,sizeof (struct kvec ), +GFP_KERNEL , node); +if (!rqstp->rq_vec) + goto out_enomem; + rqstp->rq_err = -EAGAIN ;/* No error yet */ serv->sv_nrthreads +=1 ;
@@ -1750,7 +1756,7 @@ unsigned int svc_fill_write_vector(struct svc_rqst *rqstp, ++pages; } -WARN_ON_ONCE (i >ARRAY_SIZE (rqstp->rq_vec)); +WARN_ON_ONCE (i > rqstp->rq_maxpages);return i; }EXPORT_SYMBOL_GPL (svc_fill_write_vector);
Signature · 1 line
2.49.0
Jeff Layton
maintainerRe: [PATCH v2 04/10] sunrpc: Replace the rq_vec array with dynamically-allocated memory
Quoted 6 lines · On Sat, 2025-04-19 at 13:28 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, replace the fixed-size rq_vec[] array in struct
svc_rqstwith a chunk of dynamically-allocated memory.
Quoted 6 lines
The
rq_vecarray is sized assuming request processing will need at most one kvec per page in a maximum-sized RPC message.On a system with 8-byte pointers and 4KB pages, pahole reports that the rq_vec[] array is 4144 bytes. Replacing it with a single pointer reduces the size of struct
svc_rqstto about 5400 bytes.
nit: so I guess the current struct is ~9k or so? If you're going to post numbers here, they should probably refer to the same thing.
I'm lazy -- don't make me do math.
Quoted 80 lines
Signed-off-by Chuck Lever <[email protected]>--- fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 2 +- net/sunrpc/svc.c | 8 +++++++- 4 files changed, 9 insertions(+), 4 deletions(-)fs/nfsd/nfs4proc.c +0
-1@@ -1228,7 +1228,6 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, write->wr_how_written = write->wr_stable_how; nvecs =svc_fill_write_vector (rqstp, &write->wr_payload); -WARN_ON_ONCE (nvecs >ARRAY_SIZE (rqstp->rq_vec)); status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf, write->wr_offset, rqstp->rq_vec, nvecs, &cnt,fs/nfsd/vfs.c +1
-1@@ -1094,7 +1094,7 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp, ++v; base =0 ; } -WARN_ON_ONCE (v >ARRAY_SIZE (rqstp->rq_vec)); +WARN_ON_ONCE (v > rqstp->rq_maxpages);trace_nfsd_read_vector (rqstp, fhp, offset, *count);iov_iter_kvec (&iter,ITER_DEST , rqstp->rq_vec, v, *count);include/linux/sunrpc/svc.h +1
-1@@ -207,7 +207,7 @@ struct svc_rqst {struct page * *rq_page_end;/* one past the last page */ struct folio_batch rq_fbatch; -struct kvec rq_vec[RPCSVC_MAXPAGES ];/* generally useful.. */ +struct kvec *rq_vec;struct bio_vec rq_bvec[RPCSVC_MAXPAGES ];__be32 rq_xid;/* transmission id */ net/sunrpc/svc.c +7
-1@@ -675,6 +675,7 @@ static void svc_rqst_free(struct svc_rqst *rqstp) {folio_batch_release (&rqstp->rq_fbatch); +kfree (rqstp->rq_vec);svc_release_buffer (rqstp);if (rqstp->rq_scratch_page)put_page (rqstp->rq_scratch_page);@@ -713,6 +714,11 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node)if (!svc_init_buffer (rqstp, serv, node)) goto out_enomem; + rqstp->rq_vec =kcalloc_node (rqstp->rq_maxpages,sizeof (struct kvec ), +GFP_KERNEL , node); +if (!rqstp->rq_vec) + goto out_enomem; + rqstp->rq_err = -EAGAIN ;/* No error yet */ serv->sv_nrthreads +=1 ;@@ -1750,7 +1756,7 @@ unsigned int svc_fill_write_vector(struct svc_rqst *rqstp, ++pages; } -WARN_ON_ONCE (i >ARRAY_SIZE (rqstp->rq_vec)); +WARN_ON_ONCE (i > rqstp->rq_maxpages);return i; }EXPORT_SYMBOL_GPL (svc_fill_write_vector);
Signature · 1 line
Jeff Layton <[email protected]>
Chuck Lever
maintainerRe: [PATCH v2 04/10] sunrpc: Replace the rq_vec array with dynamically-allocated memory
Quoted 7 lines · Jeff Layton
Quoted 6 lines · On Sat, 2025-04-19 at 13:28 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, replace the fixed-size rq_vec[] array in struct
svc_rqstwith a chunk of dynamically-allocated memory.
Quoted 8 lines
Quoted 6 lines
The
rq_vecarray is sized assuming request processing will need at most one kvec per page in a maximum-sized RPC message.On a system with 8-byte pointers and 4KB pages, pahole reports that the rq_vec[] array is 4144 bytes. Replacing it with a single pointer reduces the size of struct
svc_rqstto about 5400 bytes.nit: so I guess the current struct is ~9k or so?
11K plus, on my test system.
Quoted 4 lines
If you're going to
post numbers here, they should probably refer to the same thing.I'm lazy -- don't make me do math.
Well it's a running total. Each patch in this part of the series reduces the size of struct svc_rqst.
But I can update the descriptions. What math were you attempting to do?
Quoted 80 lines
Quoted 79 lines
Signed-off-by Chuck Lever <[email protected]>--- fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 2 +- net/sunrpc/svc.c | 8 +++++++- 4 files changed, 9 insertions(+), 4 deletions(-)fs/nfsd/nfs4proc.c +0
-1@@ -1228,7 +1228,6 @@ nfsd4_write(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, write->wr_how_written = write->wr_stable_how; nvecs =svc_fill_write_vector (rqstp, &write->wr_payload); -WARN_ON_ONCE (nvecs >ARRAY_SIZE (rqstp->rq_vec)); status = nfsd_vfs_write(rqstp, &cstate->current_fh, nf, write->wr_offset, rqstp->rq_vec, nvecs, &cnt,fs/nfsd/vfs.c +1
-1@@ -1094,7 +1094,7 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp, ++v; base =0 ; } -WARN_ON_ONCE (v >ARRAY_SIZE (rqstp->rq_vec)); +WARN_ON_ONCE (v > rqstp->rq_maxpages);trace_nfsd_read_vector (rqstp, fhp, offset, *count);iov_iter_kvec (&iter,ITER_DEST , rqstp->rq_vec, v, *count);include/linux/sunrpc/svc.h +1
-1@@ -207,7 +207,7 @@ struct svc_rqst {struct page * *rq_page_end;/* one past the last page */ struct folio_batch rq_fbatch; -struct kvec rq_vec[RPCSVC_MAXPAGES ];/* generally useful.. */ +struct kvec *rq_vec;struct bio_vec rq_bvec[RPCSVC_MAXPAGES ];__be32 rq_xid;/* transmission id */ net/sunrpc/svc.c +7
-1@@ -675,6 +675,7 @@ static void svc_rqst_free(struct svc_rqst *rqstp) {folio_batch_release (&rqstp->rq_fbatch); +kfree (rqstp->rq_vec);svc_release_buffer (rqstp);if (rqstp->rq_scratch_page)put_page (rqstp->rq_scratch_page);@@ -713,6 +714,11 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node)if (!svc_init_buffer (rqstp, serv, node)) goto out_enomem; + rqstp->rq_vec =kcalloc_node (rqstp->rq_maxpages,sizeof (struct kvec ), +GFP_KERNEL , node); +if (!rqstp->rq_vec) + goto out_enomem; + rqstp->rq_err = -EAGAIN ;/* No error yet */ serv->sv_nrthreads +=1 ;@@ -1750,7 +1756,7 @@ unsigned int svc_fill_write_vector(struct svc_rqst *rqstp, ++pages; } -WARN_ON_ONCE (i >ARRAY_SIZE (rqstp->rq_vec)); +WARN_ON_ONCE (i > rqstp->rq_maxpages);return i; }EXPORT_SYMBOL_GPL (svc_fill_write_vector);
Signature · 1 line
Chuck Lever
[email protected]
maintainerPATCH v2 5/10 sunrpc: Replace the rq_bvec array with dynamically-allocated memory
In reply to [email protected]From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, replace the fixed-size rq_bvec[] array in struct svc_rqst with a chunk of dynamically-allocated memory.
The rq_bvec[] array contains enough bio_vecs to handle each page in a maximum size RPC message.
On a system with 8-byte pointers and 4KB pages, pahole reports that the rq_bvec[] array is 4144 bytes. Replacing it with a single pointer reduces the size of struct svc_rqst to about 1240 bytes, or just over 19 cache lines.
--- include/linux/sunrpc/svc.h | 2 +- net/sunrpc/svc.c | 7 +++++++ net/sunrpc/svcsock.c | 7 +++---- 3 files changed, 11 insertions(+), 5 deletions(-)
include/linux/sunrpc/svc.h +1 -1
@@ -208,7 +208,7 @@ struct svc_rqst {struct folio_batch rq_fbatch;struct kvec *rq_vec; -struct bio_vec rq_bvec[RPCSVC_MAXPAGES ]; +struct bio_vec *rq_bvec;__be32 rq_xid;/* transmission id */ u32 rq_prog;/* program number */
net/sunrpc/svc.c +7 -0
@@ -675,6 +675,7 @@ static void svc_rqst_free(struct svc_rqst *rqstp) {folio_batch_release (&rqstp->rq_fbatch); +kfree (rqstp->rq_bvec);kfree (rqstp->rq_vec);svc_release_buffer (rqstp);if (rqstp->rq_scratch_page)
@@ -719,6 +720,12 @@ svc_prepare_thread(struct svc_serv *serv, struct svc_pool *pool, int node)if (!rqstp->rq_vec) goto out_enomem; + rqstp->rq_bvec =kcalloc_node (rqstp->rq_maxpages, +sizeof (struct bio_vec ), +GFP_KERNEL , node); +if (!rqstp->rq_bvec) + goto out_enomem; + rqstp->rq_err = -EAGAIN ;/* No error yet */ serv->sv_nrthreads +=1 ;
net/sunrpc/svcsock.c +3 -4
@@ -713,8 +713,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp)if (svc_xprt_is_dead (xprt)) goto out_notconn; - count =xdr_buf_to_bvec (rqstp->rq_bvec, -ARRAY_SIZE (rqstp->rq_bvec), xdr); + count =xdr_buf_to_bvec (rqstp->rq_bvec, rqstp->rq_maxpages, xdr);iov_iter_bvec (&msg.msg_iter,ITER_SOURCE , rqstp->rq_bvec, count, rqstp->rq_res.len);
@@ -1219,8 +1218,8 @@ static int svc_tcp_sendmsg(struct svc_sock *svsk, struct svc_rqst *rqstp,memcpy (buf, &marker,sizeof (marker));bvec_set_virt (rqstp->rq_bvec, buf,sizeof (marker)); - count =xdr_buf_to_bvec (rqstp->rq_bvec +1 , -ARRAY_SIZE (rqstp->rq_bvec) -1 , &rqstp->rq_res); + count =xdr_buf_to_bvec (rqstp->rq_bvec +1 , rqstp->rq_maxpages, + &rqstp->rq_res);iov_iter_bvec (&msg.msg_iter,ITER_SOURCE , rqstp->rq_bvec,1 + count,sizeof (marker) + rqstp->rq_res.len);
Signature · 1 line
2.49.0
[email protected]
maintainerPATCH v2 6/10 sunrpc: Adjust size of socket's receive page array dynamically
In reply to [email protected]From: Chuck Lever <[email protected]>
As a step towards making NFSD's maximum rsize and wsize variable at run-time, make sk_pages a flexible array.
--- include/linux/sunrpc/svcsock.h | 4 +++- net/sunrpc/svcsock.c | 8 ++++++-- 2 files changed, 9 insertions(+), 3 deletions(-)
include/linux/sunrpc/svcsock.h +3 -1
@@ -40,7 +40,9 @@ struct svc_sock {struct completion sk_handshake_done; -struct page * sk_pages[RPCSVC_MAXPAGES ];/* received data */ +/* received data */ +unsigned long sk_maxpages; + struct page * sk_pages[] __counted_by(sk_maxpages); };static inline u32 svc_sock_reclen (struct svc_sock *svsk)
net/sunrpc/svcsock.c +6 -2
@@ -1339,7 +1339,8 @@ static void svc_tcp_init(struct svc_sock *svsk, struct svc_serv *serv) svsk->sk_marker = xdr_zero; svsk->sk_tcplen =0 ; svsk->sk_datalen =0 ; -memset (&svsk->sk_pages[0 ],0 ,sizeof (svsk->sk_pages)); +memset (&svsk->sk_pages[0 ],0 , + svsk->sk_maxpages *sizeof (struct page *));tcp_sock_set_nodelay (sk);
@@ -1378,10 +1379,13 @@ static struct svc_sock *svc_setup_socket(struct svc_serv *serv,struct svc_sock *svsk;struct sock *inet;int pmap_register = !(flags &SVC_SOCK_ANONYMOUS ); +unsigned long pages; - svsk =kzalloc (sizeof (*svsk),GFP_KERNEL ); + pages =svc_serv_maxpages (serv); + svsk =kzalloc (struct_size (svsk, sk_pages, pages),GFP_KERNEL );if (!svsk)return ERR_PTR (-ENOMEM ); + svsk->sk_maxpages = pages; inet = sock->sk;
Signature · 1 line
2.49.0
[email protected]
maintainerPATCH v2 7/10 svcrdma: Adjust the number of RDMA contexts per transport
In reply to [email protected]From: Chuck Lever <[email protected]>
The RDMA accept code requests enough RDMA contexts to read and write one page per maximum size RPC message, plus one context that is getting recycled for the next RPC.
--- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/sunrpc/xprtrdma/svc_rdma_transport.c +1 -1
@@ -467,7 +467,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)* progress even if the client is using one rkey per page * in each Read chunk. */ - ctxts =3 *RPCSVC_MAXPAGES ; + ctxts =3 *svc_serv_maxpages (xprt->xpt_server); newxprt->sc_sq_depth = rq_depth + ctxts;if (newxprt->sc_sq_depth > dev->attrs.max_qp_wr) newxprt->sc_sq_depth = dev->attrs.max_qp_wr;
Signature · 1 line
2.49.0
[email protected]
maintainerPATCH v2 8/10 svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages
In reply to [email protected]From: Chuck Lever <[email protected]>
Allow allocation of more entries in the rc_pages[] array when the maximum size of an RPC message is increased.
--- include/linux/sunrpc/svc_rdma.h | 3 ++- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 ++++++-- 2 files changed, 8 insertions(+), 3 deletions(-)
include/linux/sunrpc/svc_rdma.h +2 -1
@@ -202,7 +202,8 @@ struct svc_rdma_recv_ctxt {struct svc_rdma_pcl rc_reply_pcl;unsigned int rc_page_count; -struct page *rc_pages[RPCSVC_MAXPAGES ]; +unsigned long rc_maxpages; + struct page *rc_pages[] __counted_by(rc_maxpages); };/*
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +6 -2
@@ -120,12 +120,16 @@ svc_rdma_recv_ctxt_alloc(struct svcxprt_rdma *rdma) {int node =ibdev_to_node (rdma->sc_cm_id->device);struct svc_rdma_recv_ctxt *ctxt; +unsigned long pages;dma_addr_t addr;void *buffer; - ctxt =kzalloc_node (sizeof (*ctxt),GFP_KERNEL , node); + pages =svc_serv_maxpages (rdma->sc_xprt.xpt_server); + ctxt =kzalloc_node (struct_size (ctxt, rc_pages, pages), +GFP_KERNEL , node);if (!ctxt) goto fail0; + ctxt->rc_maxpages = pages; buffer =kmalloc_node (rdma->sc_max_req_size,GFP_KERNEL , node);if (!buffer) goto fail1;
@@ -497,7 +501,7 @@ static bool xdr_check_write_chunk(struct svc_rdma_recv_ctxt *rctxt)* a computation, perform a simple range check. This is an * arbitrary but sensible limit (ie, not architectural). */ -if (unlikely (segcount >RPCSVC_MAXPAGES )) +if (unlikely (segcount > rctxt->rc_maxpages))return false; p = xdr_inline_decode(&rctxt->rc_stream,
Signature · 1 line
2.49.0
[email protected]
maintainerPATCH v2 9/10 svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages
In reply to [email protected]From: Chuck Lever <[email protected]>
Allow allocation of more entries in the sc_pages[] array when the maximum size of an RPC message is increased.
--- include/linux/sunrpc/svc_rdma.h | 3 ++- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 +++++++++++++--- 2 files changed, 15 insertions(+), 4 deletions(-)
include/linux/sunrpc/svc_rdma.h +2 -1
@@ -245,7 +245,8 @@ struct svc_rdma_send_ctxt {void *sc_xprt_buf;int sc_page_count;int sc_cur_sge_no; -struct page *sc_pages[RPCSVC_MAXPAGES ]; +unsigned long sc_maxpages; +struct page **sc_pages;struct ib_sge sc_sges[]; };
net/sunrpc/xprtrdma/svc_rdma_sendto.c +13 -3
@@ -118,6 +118,7 @@ svc_rdma_send_ctxt_alloc(struct svcxprt_rdma *rdma) {int node =ibdev_to_node (rdma->sc_cm_id->device);struct svc_rdma_send_ctxt *ctxt; +unsigned long pages;dma_addr_t addr;void *buffer;int i;
@@ -126,13 +127,19 @@ svc_rdma_send_ctxt_alloc(struct svcxprt_rdma *rdma) GFP_KERNEL, node);if (!ctxt) goto fail0; + pages =svc_serv_maxpages (rdma->sc_xprt.xpt_server); + ctxt->sc_pages =kcalloc_node (pages,sizeof (struct page *), +GFP_KERNEL , node); +if (!ctxt->sc_pages) + goto fail1; + ctxt->sc_maxpages = pages; buffer =kmalloc_node (rdma->sc_max_req_size,GFP_KERNEL , node);if (!buffer) - goto fail1; + goto fail2; addr =ib_dma_map_single (rdma->sc_pd->device, buffer, rdma->sc_max_req_size,DMA_TO_DEVICE );if (ib_dma_mapping_error (rdma->sc_pd->device, addr)) - goto fail2; + goto fail3;svc_rdma_send_cid_init (rdma, &ctxt->sc_cid);
@@ -151,8 +158,10 @@ svc_rdma_send_ctxt_alloc(struct svcxprt_rdma *rdma) ctxt->sc_sges[i].lkey = rdma->sc_pd->local_dma_lkey;return ctxt; -fail2: +fail3:kfree (buffer); +fail2: +kfree (ctxt->sc_pages); fail1:kfree (ctxt); fail0:
@@ -176,6 +185,7 @@ void svc_rdma_send_ctxts_destroy(struct svcxprt_rdma *rdma) rdma->sc_max_req_size, DMA_TO_DEVICE);kfree (ctxt->sc_xprt_buf); +kfree (ctxt->sc_pages);kfree (ctxt); } }
Signature · 1 line
2.49.0
[email protected]
maintainerPATCH v2 10/10 sunrpc: Remove the RPCSVC_MAXPAGES macro
In reply to [email protected]From: Chuck Lever <[email protected]>
It is no longer used.
--- include/linux/sunrpc/svc.h | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
include/linux/sunrpc/svc.h +5 -7
@@ -150,14 +150,7 @@ extern u32 svc_max_payload(const struct svc_rqst *rqstp);* list. xdr_buf.tail points to the end of the first page. * This assumes that the non-page part of an rpc reply will fit * in a page - NFSd ensures this. lockd also has no trouble. -* -* Each request/reply pair can have at most one "payload", plus two pages, -* one for the request, and one for the reply. -* We using ->sendfile to return read data, we might need one extra page -* if the request is not page-aligned. So add another '1'. */ -#define RPCSVC_MAXPAGES ((RPCSVC_MAXPAYLOAD+PAGE_SIZE-1)/PAGE_SIZE \ - + 2 + 1)/** * svc_serv_maxpages - maximum pages/kvecs needed for one RPC message
@@ -165,6 +158,11 @@ extern u32 svc_max_payload(const struct svc_rqst *rqstp);* * Returns a count of pages or vectors that can hold the maximum * size RPC message for @serv. +* +* Each request/reply pair can have at most one "payload", plus two +* pages, one for the request, and one for the reply. +* nfsd_splice_actor() might need an extra page when a READ payload +* is not page-aligned. */ static inline unsigned long svc_serv_maxpages (const struct svc_serv *serv) {
Signature · 1 line
2.49.0
Quoted 18 lines · On 4/19/25 1:28 PM, [email protected] wrote:
From: Chuck Lever <[email protected]>
In order to make
RPCSVC_MAXPAYLOADlarger (or variable in size), we need to do something clever with the payload arrays embedded in structsvc_rqstand elsewhere.My preference is to keep these arrays allocated all the time because allocating them on demand increases the risk of a memory allocation failure during a large I/O. This is a quick-and-dirty approach that might be replaced once NFSD is converted to use large folios.
The downside of this design choice is that it pins a few pages per NFSD thread (and that's the current situation already). But note that because
RPCSVC_MAXPAGESis 259, each array is just over a page in size, making the allocation waste quite a bit of memory beyond the end of the array due to power-of-2 allocator round up. This gets worse as the MAXPAGES value is doubled or quadrupled.
Quoted 2 lines
This series also addresses similar issues in the socket and RDMA transports.
I forgot to note that this series has other benefits besides making these arrays flexible in size. The reduction in size of struct svc_rqst to under half a page is kind of a big deal.
Quoted 25 lines
Chuck Lever (9):
sunrpc: Remove backchannel check insvc_init_buffer()
sunrpc: Add a helper to derive maxpages fromsv_max_mesg
sunrpc: Replace therq_pagesarray with dynamically-allocated memory
sunrpc: Replace therq_vecarray with dynamically-allocated memory
sunrpc: Replace therq_bvecarray with dynamically-allocated memory
sunrpc: Adjust size of socket's receive page array dynamically
svcrdma: Adjust the number of RDMA contexts per transport
svcrdma: Adjust the number of entries insvc_rdma_recv_ctxt::rc_pages
svcrdma: Adjust the number of entries insvc_rdma_send_ctxt::sc_pagesfs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 19 +++++++-- include/linux/sunrpc/svc_rdma.h | 6 ++- include/linux/sunrpc/svcsock.h | 4 +- net/sunrpc/svc.c | 51 +++++++++++++++--------- net/sunrpc/svc_xprt.c | 10 +---- net/sunrpc/svcsock.c | 15 ++++--- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 +++- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 ++++++-- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 12 files changed, 88 insertions(+), 48 deletions(-)
Signature · 1 line
Chuck Lever
Quoted 32 lines · On Sat, 2025-04-19 at 13:28 -0400, [email protected] wrote:
From: Chuck Lever <[email protected]>
In order to make
RPCSVC_MAXPAYLOADlarger (or variable in size), we need to do something clever with the payload arrays embedded in structsvc_rqstand elsewhere.My preference is to keep these arrays allocated all the time because allocating them on demand increases the risk of a memory allocation failure during a large I/O. This is a quick-and-dirty approach that might be replaced once NFSD is converted to use large folios.
The downside of this design choice is that it pins a few pages per NFSD thread (and that's the current situation already). But note that because
RPCSVC_MAXPAGESis 259, each array is just over a page in size, making the allocation waste quite a bit of memory beyond the end of the array due to power-of-2 allocator round up. This gets worse as the MAXPAGES value is doubled or quadrupled.This series also addresses similar issues in the socket and RDMA transports.
Chuck Lever (9):
sunrpc: Remove backchannel check insvc_init_buffer()
sunrpc: Add a helper to derive maxpages fromsv_max_mesg
sunrpc: Replace therq_pagesarray with dynamically-allocated memory
sunrpc: Replace therq_vecarray with dynamically-allocated memory
sunrpc: Replace therq_bvecarray with dynamically-allocated memory
sunrpc: Adjust size of socket's receive page array dynamically
svcrdma: Adjust the number of RDMA contexts per transport
svcrdma: Adjust the number of entries insvc_rdma_recv_ctxt::rc_pages
svcrdma: Adjust the number of entries insvc_rdma_send_ctxt::sc_pages
Quoted 44 lines
fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 19 +++++++-- include/linux/sunrpc/svc_rdma.h | 6 ++- include/linux/sunrpc/svcsock.h | 4 +- net/sunrpc/svc.c | 51 +++++++++++++++--------- net/sunrpc/svc_xprt.c | 10 +---- net/sunrpc/svcsock.c | 15 ++++--- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 +++- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 ++++++-- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 12 files changed, 88 insertions(+), 48 deletions(-)Signature · 28 lines
2.49.0 Chuck Lever (10): sunrpc: Remove backchannel check in svc_init_buffer() sunrpc: Add a helper to derive maxpages from sv_max_mesg sunrpc: Replace the rq_pages array with dynamically-allocated memory sunrpc: Replace the rq_vec array with dynamically-allocated memory sunrpc: Replace the rq_bvec array with dynamically-allocated memory sunrpc: Adjust size of socket's receive page array dynamically svcrdma: Adjust the number of RDMA contexts per transport svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages sunrpc: Remove the RPCSVC_MAXPAGES macro fs/nfsd/nfs4proc.c | 1 - fs/nfsd/vfs.c | 2 +- include/linux/sunrpc/svc.h | 31 +++++++++----- include/linux/sunrpc/svc_rdma.h | 6 ++- include/linux/sunrpc/svcsock.h | 4 +- net/sunrpc/svc.c | 51 +++++++++++++++--------- net/sunrpc/svc_xprt.c | 10 +---- net/sunrpc/svcsock.c | 15 ++++--- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 +++- net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 16 ++++++-- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 12 files changed, 93 insertions(+), 55 deletions(-)
I think the pile looks fine, modulo a few nits (which you can clean up if you like).