Skip to content

Allocate payload arrays dynamically

[email protected] linux-nfs linux-rdma 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest19 patches 8 repliesIn review
  2. v414 patches 37 repliesSuperseded
  3. v311 patches 2 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v210 patches9 repliesSuperseded

Changes in v5 from the cover letter

* Replace the use of rq_vec instead of allocating it dynamically

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/19svcrdma: Reduce the number of rdma_rw contexts per-QPnet/sunrpc/xprtrdma/svc_rdma_transport.c+8 -61 reply
2/19sunrpc: Add a helper to derive maxpages from sv_max_mesginclude/linux/sunrpc/svc.h+17 -00 replies
3/19sunrpc: Remove backchannel check in svc_init_buffer()net/sunrpc/svc.c+0 -40 replies
4/19sunrpc: Replace the rq_pages array with dynamically-allocated memoryinclude/linux/sunrpc/svc.h net/sunrpc/svc.c net/sunrpc/svc_xprt.c +1 more+21 -250 replies
5/19sunrpc: Replace the rq_bvec array with dynamically-allocated memoryinclude/linux/sunrpc/svc.h net/sunrpc/svc.c net/sunrpc/svcsock.c+11 -50 replies
6/19NFSD: Use rqstp->rq_bvec in nfsd_iter_read()fs/nfsd/vfs.c+7 -70 replies
7/19NFSD: De-duplicate the svc_fill_write_vector() call sitesfs/nfsd/nfs3proc.c fs/nfsd/nfs4proc.c fs/nfsd/nfsproc.c +4 more+54 -361 reply
8/19SUNRPC: Export xdr_buf_to_bvec()net/sunrpc/xdr.c+1 -00 replies
9/19NFSD: Use rqstp->rq_bvec in nfsd_iter_write()fs/nfsd/vfs.c+3 -20 replies
10/19SUNRPC: Remove svc_fill_write_vector()include/linux/sunrpc/svc.h net/sunrpc/svc.c+0 -420 replies
11/19SUNRPC: Remove svc_rqst :: rq_vecinclude/linux/sunrpc/svc.h+0 -10 replies
12/19sunrpc: Adjust size of socket's receive page array dynamicallyinclude/linux/sunrpc/svcsock.h net/sunrpc/svcsock.c+9 -30 replies
13/19svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pagesinclude/linux/sunrpc/svc_rdma.h net/sunrpc/xprtrdma/svc_rdma_recvfrom.c+8 -30 replies
14/19svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pagesinclude/linux/sunrpc/svc_rdma.h net/sunrpc/xprtrdma/svc_rdma_sendto.c+15 -40 replies
15/19sunrpc: Remove the RPCSVC_MAXPAGES macroinclude/linux/sunrpc/svc.h+0 -70 replies
16/19NFSD: Remove NFSD_BUFSIZEfs/nfsd/nfs4proc.c fs/nfsd/nfs4state.c fs/nfsd/nfs4xdr.c +1 more+3 -160 replies
17/19NFSD: Remove NFSSVC_MAXBLKSIZE_V2 macrofs/nfsd/nfsd.h fs/nfsd/nfsproc.c fs/nfsd/nfsxdr.c+4 -60 replies
18/19NFSD: Add a "default" block sizefs/nfsd/nfsd.h fs/nfsd/nfssvc.c+8 -30 replies
19/19SUNRPC: Bump the maximum payload size for the serverinclude/linux/sunrpc/svc.h+7 -75 replies

Reviews trailers per patch

16 of 19 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchNeilBrownChristoph HellwigJeff Layton
SeriesR-b
1/19R-b R-b R-b
2/19R-b R-b
3/19R-b R-b A-b
4/19R-b R-b
5/19R-b R-b
6/19R-b
7/19R-b
8/19
9/19
10/19
11/19R-b
12/19R-b R-b
13/19R-b R-b
14/19R-b R-b
15/19R-b R-b
16/19R-b R-b
17/19R-b R-b
18/19R-b
19/19R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v5 thread

Cover letter [PATCH v5 00/19] Allocate payload arrays dynamically

From: Chuck Lever <[email protected]>

In order to make RPCSVC_MAXPAYLOAD larger (or variable in size), we need to do something clever with the payload arrays embedded in struct svc_rqst and elsewhere.

My preference is to keep these arrays allocated all the time because allocating them on demand increases the risk of a memory allocation failure during a large I/O. This is a quick-and-dirty approach that might be replaced once NFSD is converted to use large folios.

The downside of this design choice is that it pins a few pages per NFSD thread (and that's the current situation already). But note that because RPCSVC_MAXPAGES is 259, each array is just over a page in size, making the allocation waste quite a bit of memory beyond the end of the array due to power-of-2 allocator round up. This gets worse as the MAXPAGES value is doubled or quadrupled.

This series also addresses similar issues in the socket and RDMA transports.

Changes since v4:
* Replace the use of rq_vec instead of allocating it dynamically

Changes since v3:
* Improved the rdma_rw context count estimate
* Dropped "NFSD: Remove NFSSVC_MAXBLKSIZE from .pc_xdrressize"
* Cleaned up the max size macros a bit
* Completed the implementation of adjustable max_block_size

Changes since v2:
* Address Jeff's review comments
* Address Neil's review comments
* Start removing a few uses of NFSSVC_MAXBLKSIZE

Chuck Lever (19):
svcrdma: Reduce the number of rdma_rw contexts per-QP
sunrpc: Add a helper to derive maxpages from sv_max_mesg
sunrpc: Remove backchannel check in svc_init_buffer()
sunrpc: Replace the rq_pages array with dynamically-allocated memory
sunrpc: Replace the rq_bvec array with dynamically-allocated memory
NFSD: Use rqstp->rq_bvec in nfsd_iter_read()
NFSD: De-duplicate the svc_fill_write_vector() call sites
SUNRPC: Export xdr_buf_to_bvec()
NFSD: Use rqstp->rq_bvec in nfsd_iter_write()
SUNRPC: Remove svc_fill_write_vector()
SUNRPC: Remove svc_rqst :: rq_vec
sunrpc: Adjust size of socket's receive page array dynamically
svcrdma: Adjust the number of entries in svc_rdma_recv_ctxt::rc_pages
svcrdma: Adjust the number of entries in svc_rdma_send_ctxt::sc_pages
sunrpc: Remove the RPCSVC_MAXPAGES macro
NFSD: Remove NFSD_BUFSIZE
NFSD: Remove NFSSVC_MAXBLKSIZE_V2 macro
NFSD: Add a "default" block size
SUNRPC: Bump the maximum payload size for the server

 fs/nfsd/nfs3proc.c                       |  5 +-
 fs/nfsd/nfs4proc.c                       | 10 +--
 fs/nfsd/nfs4state.c                      |  2 +-
 fs/nfsd/nfs4xdr.c                        |  2 +-
 fs/nfsd/nfsd.h                           | 24 +++----
 fs/nfsd/nfsproc.c                        | 13 ++--
 fs/nfsd/nfssvc.c                         |  2 +-
 fs/nfsd/nfsxdr.c                         |  4 +-
 fs/nfsd/vfs.c                            | 67 ++++++++++++++------
 fs/nfsd/vfs.h                            | 10 +--
 include/linux/sunrpc/svc.h               | 46 ++++++++------
 include/linux/sunrpc/svc_rdma.h          |  6 +-
 include/linux/sunrpc/svcsock.h           |  4 +-
 net/sunrpc/svc.c                         | 80 +++++++-----------------
 net/sunrpc/svc_xprt.c                    | 10 +--
 net/sunrpc/svcsock.c                     | 15 +++--
 net/sunrpc/xdr.c                         |  1 +
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c  |  8 ++-
 net/sunrpc/xprtrdma/svc_rdma_rw.c        |  2 +-
 net/sunrpc/xprtrdma/svc_rdma_sendto.c    | 16 ++++-
 net/sunrpc/xprtrdma/svc_rdma_transport.c | 14 +++--
 21 files changed, 170 insertions(+), 171 deletions(-)
Signature · 1 line
2.49.0