nfs: Enable PCI Peer-to-Peer DMA (P2PDMA) support
Subsystems (4)
cover nfs: Enable PCI Peer-to-Peer DMA (P2PDMA) support
As high-performance storage environments increasingly rely on direct data movement between PCIe endpoints (e.g., moving data directly between an NVMe Controller Memory Buffer and a Network Interface), support for Peer-to-Peer DMA (P2PDMA) in the network filesystem layer becomes essential. This series introduces P2PDMA support for the NFS Direct I/O.
Currently, NFS O_DIRECT operations fail with -EREMOTEIO if the user buffer resides in PCIe BAR memory. This is primarily due to the use of the legacy iov_iter_get_pages_alloc2() API, which cannot pass the required FOLL_PCI_P2PDMA flag, and a request lifecycle that is unaware
of the pinning requirements for P2P memory.
Design
=======
The proposed design centers around making the NFS request lifecycle "pin-aware" and upgrading the infrastructure to support modern memory extraction APIs.
1. 64-bit Capability Infrastructure
The existing nfs_server->caps bitmask is limited to 32 bits and is currently exhausted. This series expands the bitmask to 64 bits to accommodate NFS_CAP_P2PDMA. Crucially, it also refactors the NFS_CAP_* constants to use ULL definitions. This prevents a subtle 32-bit truncation bug where bitwise negations (e.g., caps &= ~NFS_CAP_ACLS) would accidentally clear the high bits of the 64-bit capability field.
2. Transport-Level Detection
P2PDMA support is a property of the local transport hardware. A new supports_p2pdma operation is added to the SunRPC transport ops. For RDMA, this is implemented by querying the underlying device via ib_dma_pci_p2p_dma_supported(). The NFS client queries this during mount and sets the NFS_CAP_P2PDMA bit accordingly.
3. Pin-Aware Request Lifecycle
Standard NFS requests use get_page() and put_page() for memory management. However, memory extracted via iov_iter_extract_pages() requires explicit pinning and unpinning (unpin_user_page()).
This series introduces a PG_PINNED flag in struct nfs_page. When set, the request lifecycle skips standard page referencing and ensures that unpin_user_page() is called only when the I/O is complete. This ensures that physical memory remains pinned for the duration of the DMA transfer
4. API Migration
The Direct I/O path is migrated to the modern iov_iter_extract_pages() API. The ITER_ALLOW_P2PDMA flag is passed to the iterator only when the local mount has signaled P2P support via the capability bit. This ensures that "normal" users on standard TCP/UDP transports see no change in behavior or overhead.
Call for review
===============
Any insights on the proposed changes to the nfs_page lifecycle and the 64-bit capability expansion are appreciated. If this approach is deemed incorrect or if there is a more idiomatic way for this, please direct me in the right direction.
Thanks,
Praan
Pranjal Shrivastava (4):
sunrpc: add supports_p2pdma to rpc_xprt_ops
nfs: add NFS_CAP_P2PDMA and detect transport support
nfs: make nfs_page pin-aware
nfs: allow P2PDMA in direct I/O path
fs/nfs/client.c | 8 ++++ fs/nfs/direct.c | 51 ++++++++++++++++++------- fs/nfs/nfs4_fs.h | 2 +- fs/nfs/pagelist.c | 18 ++++++--- fs/nfs/super.c | 2 +- include/linux/nfs_fs_sb.h | 67 +++++++++++++++++---------------- include/linux/nfs_page.h | 2 + include/linux/sunrpc/xprt.h | 1 + net/sunrpc/xprtrdma/transport.c | 9 +++++ 9 files changed, 106 insertions(+), 54 deletions(-)
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
RFC PATCH 1/4 sunrpc: add supports_p2pdma to rpc_xprt_ops
Add a new transport op, supports_p2pdma, to allow upper layers (such as NFS) to query whether the underlying RPC transport supports PCI Peer-to-Peer DMA (P2PDMA).
Since the capability is hardware-dependent. For the RDMA transport, implement this by querying the underlying InfiniBand device via ib_dma_pci_p2p_dma_supported() to ensures that P2PDMA is only attempted when both the transport and the HCA drivers support it.
--- include/linux/sunrpc/xprt.h | 1 + net/sunrpc/xprtrdma/transport.c | 9 +++++++++ 2 files changed, 10 insertions(+)
include/linux/sunrpc/xprt.h +1 -0
@@ -187,6 +187,7 @@ struct rpc_xprt_ops {void (*bc_free_rqst)(struct rpc_rqst *rqst);void (*bc_destroy)(struct rpc_xprt *xprt,unsigned int max_reqs); +bool (*supports_p2pdma)(struct rpc_xprt *xprt); };/*
net/sunrpc/xprtrdma/transport.c +9 -0
@@ -717,6 +717,14 @@ xprt_rdma_disable_swap(struct rpc_xprt *xprt) { } +static bool +xprt_rdma_supports_p2pdma (struct rpc_xprt *xprt) +{ +struct rpcrdma_xprt *r_xprt =rpcx_to_rdmax (xprt); + +return ib_dma_pci_p2p_dma_supported (r_xprt->rx_ep->re_id->device); +} +/* * Plumbing for rpc transport switch and kernel module */
@@ -742,6 +750,7 @@ static const struct rpc_xprt_ops xprt_rdma_procs = { .enable_swap = xprt_rdma_enable_swap, .disable_swap = xprt_rdma_disable_swap, .inject_disconnect = xprt_rdma_inject_disconnect, + .supports_p2pdma = xprt_rdma_supports_p2pdma,#if defined(CONFIG_SUNRPC_BACKCHANNEL ) .bc_setup = xprt_rdma_bc_setup, .bc_maxpayload = xprt_rdma_bc_maxpayload,
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
RFC PATCH 2/4 nfs: add NFS_CAP_P2PDMA and detect transport support
In reply to Pranjal ShrivastavaThe NFS server capabilities bitmask (server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.
Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.
Update nfs_server_set_init_caps() to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.
--- fs/nfs/client.c | 8 +++++ fs/nfs/nfs4_fs.h | 2 +- fs/nfs/super.c | 2 +- include/linux/nfs_fs_sb.h | 67 ++++++++++++++++++++------------------- 4 files changed, 44 insertions(+), 35 deletions(-)
fs/nfs/client.c +8 -0
@@ -712,6 +712,8 @@ static void nfs4_server_set_init_caps(struct nfs_server *server)void nfs_server_set_init_caps (struct nfs_server *server) { +struct rpc_xprt *xprt; +switch (server->nfs_client->rpc_ops->version) {case 2 : server->caps =NFS_CAP_HARDLINKS |NFS_CAP_SYMLINKS ;
@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)nfs4_server_set_init_caps (server);break ; } + +rcu_read_lock (); + xprt =rcu_dereference (server->client->cl_xprt); +if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma (xprt)) + server->caps |=NFS_CAP_P2PDMA ; +rcu_read_unlock (); }EXPORT_SYMBOL_GPL (nfs_server_set_init_caps);
fs/nfs/nfs4_fs.h +1 -1
@@ -60,7 +60,7 @@ enum nfs4_client_state {struct nfs_seqid_counter ;struct nfs4_minor_version_ops {u32 minor_version; -unsigned init_caps; +u64 init_caps;int (*init_client)(struct nfs_client *);void (*shutdown_client)(struct nfs_client *);
fs/nfs/super.c +1 -1
@@ -672,7 +672,7 @@ int nfs_show_stats(struct seq_file *m, struct dentry *root)show_implementation_id (m, nfss);seq_puts (m,"\n\tcaps:\t" ); -seq_printf (m,"caps=0x%x" , nfss->caps); +seq_printf (m,"caps=0x%llx" , nfss->caps);seq_printf (m,",wtmult=%u" , nfss->wtmult);seq_printf (m,",dtsize=%u" , nfss->dtsize);seq_printf (m,",bsize=%u" , nfss->bsize);
include/linux/nfs_fs_sb.h +34 -33
77 lines not on this page · raw message
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
Chuck Lever
maintainerRe: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support
Quoted 21 lines · Pranjal Shrivastava
The NFS server capabilities bitmask (
server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.Introduce a new capability bit,
NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.Update
nfs_server_set_init_caps()to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set theNFS_CAP_P2PDMAbit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.fs/nfs/client.c +0
-0@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)nfs4_server_set_init_caps (server);
fs/nfs/client.c void nfs_server_set_init_caps(struct nfs_server *server) · line 737
Quoted 10 lines
fs/nfs/client.c +6
-0break ; } + +rcu_read_lock (); + xprt =rcu_dereference (server->client->cl_xprt); +if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma (xprt)) + server->caps |=NFS_CAP_P2PDMA ; +rcu_read_unlock (); }EXPORT_SYMBOL_GPL (nfs_server_set_init_caps);
Is the transport even connected when the NFS client does this test? If it isn't, xprtrdma and the RDMA core have not chosen an underlying device yet.
Note that, even if this logic /is/ correct, if the transport connection is lost the transport will reconnect automatically, doing the RDMA CM dance again and possibly resolving to a different device. The NFS client layer will be none-the-wiser and the NFS_CAP_P2PDMA flag setting will be stale at that point, and quite possibly incorrect if the new connection's device is not P2P-enabled.
(Basically this is what happens when an RDMA device is removed).
So this detection has to be done as part of xprtrdma's connection flow, and it needs to set a flag somewhere in the rpc_xprt. The NFS direct I/O code path then has to look for that flag before choosing the mechanism/flags it uses for each iov iter.
Signature · 1 line
Chuck Lever
Re: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support
Quoted 46 lines · Chuck Lever
Quoted 21 lines · Pranjal Shrivastava
The NFS server capabilities bitmask (
server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.Introduce a new capability bit,
NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.Update
nfs_server_set_init_caps()to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set theNFS_CAP_P2PDMAbit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.fs/nfs/client.c +0
-0@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)nfs4_server_set_init_caps (server);fs/nfs/client.c void nfs_server_set_init_caps(struct nfs_server *server) · line 737
Quoted 10 lines
fs/nfs/client.c +6
-0break ; } + +rcu_read_lock (); + xprt =rcu_dereference (server->client->cl_xprt); +if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma (xprt)) + server->caps |=NFS_CAP_P2PDMA ; +rcu_read_unlock (); }EXPORT_SYMBOL_GPL (nfs_server_set_init_caps);Is the transport even connected when the NFS client does this test? If it isn't, xprtrdma and the RDMA core have not chosen an underlying device yet.
Note that, even if this logic /is/ correct, if the transport connection is lost the transport will reconnect automatically, doing the RDMA CM dance again and possibly resolving to a different device. The NFS client layer will be none-the-wiser and the
NFS_CAP_P2PDMAflag setting will be stale at that point, and quite possibly incorrect if the new connection's device is not P2P-enabled.
Quoted 7 lines
(Basically this is what happens when an RDMA device is removed).
So this detection has to be done as part of xprtrdma's connection flow, and it needs to set a flag somewhere in the
rpc_xprt. The NFS direct I/O code path then has to look for that flag before choosing the mechanism/flags it uses for each iov iter.
Ack. I agree, so should we start with an inital cap and then update it in the event of a transport change / disconnect? Or shall we populate the cap only when a transport is connected?
Thanks,
Praan
Chuck Lever
maintainerRe: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support
Quoted 55 lines · Pranjal Shrivastava
Quoted 46 lines · Chuck Lever
Quoted 21 lines · Pranjal Shrivastava
The NFS server capabilities bitmask (
server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.Introduce a new capability bit,
NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.Update
nfs_server_set_init_caps()to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set theNFS_CAP_P2PDMAbit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.fs/nfs/client.c +0
-0@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)nfs4_server_set_init_caps (server);fs/nfs/client.c void nfs_server_set_init_caps(struct nfs_server *server) · line 737
Quoted 10 lines
fs/nfs/client.c +6
-0break ; } + +rcu_read_lock (); + xprt =rcu_dereference (server->client->cl_xprt); +if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma (xprt)) + server->caps |=NFS_CAP_P2PDMA ; +rcu_read_unlock (); }EXPORT_SYMBOL_GPL (nfs_server_set_init_caps);Is the transport even connected when the NFS client does this test? If it isn't, xprtrdma and the RDMA core have not chosen an underlying device yet.
Note that, even if this logic /is/ correct, if the transport connection is lost the transport will reconnect automatically, doing the RDMA CM dance again and possibly resolving to a different device. The NFS client layer will be none-the-wiser and the
NFS_CAP_P2PDMAflag setting will be stale at that point, and quite possibly incorrect if the new connection's device is not P2P-enabled.Quoted 7 lines
(Basically this is what happens when an RDMA device is removed).
So this detection has to be done as part of xprtrdma's connection flow, and it needs to set a flag somewhere in the
rpc_xprt. The NFS direct I/O code path then has to look for that flag before choosing the mechanism/flags it uses for each iov iter.
Quoted 3 lines
Ack. I agree, so should we start with an inital cap and then update it in the event of a transport change / disconnect? Or shall we populate the cap only when a transport is connected?
IMO this flag does not belong in the NFS server CAPS, as it is a capability associated with each RPC transport. How should NFS_CAP_P2PDMA be set if there are two RPC transports, one with P2PDMA enabled and with it disabled? (Perhaps it should be a flag in the transport switch instance rather than the transport instance).
Which mechanism to use has to be re-decided every time a dreq is scheduled because the xprt can change between an original send and a retransmission (if, say, the COMMIT verifier changes due to a server reboot).
Trond and Anna will have the final say about how this works.
Signature · 1 line
Chuck Lever
RFC PATCH 3/4 nfs: make nfs_page pin-aware
In reply to Pranjal ShrivastavaThe migration to iov_iter_extract_pages() for Direct I/O introduces page pinning (GUP) instead of standard page referencing. To handle this correctly, nfs_page must track whether it holds a pin or a standard reference.
Add a new flag, PG_PINNED, to struct nfs_page. Update the creation path (nfs_page_create_from_page) to accept a 'pinned' boolean and set the flag accordingly. If the page is pinned, we skip the standard get_page() as the pin itself acts as a reference.
Update nfs_clear_request() to use unpin_user_page() instead of put_page() when the PG_PINNED flag is set. This ensures that memory remains safely locked for DMA and that kernel page accounting stays consistent. Ensure subrequests inherit the pin status from their parent request.
--- fs/nfs/direct.c | 4 ++-- fs/nfs/pagelist.c | 18 +++++++++++++----- include/linux/nfs_page.h | 2 ++ 3 files changed, 17 insertions(+), 7 deletions(-)
fs/nfs/direct.c +2 -2
@@ -370,7 +370,7 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,struct nfs_page *req;unsigned int req_len =min_t (size_t, bytes,PAGE_SIZE - pgbase);/* XXX do we need to do the eof zeroing found in async_filler? */ - req =nfs_page_create_from_page (dreq->ctx, pagevec[i], + req =nfs_page_create_from_page (dreq->ctx, pagevec[i], false, pgbase, pos, req_len);if (IS_ERR (req)) { result =PTR_ERR (req);
@@ -898,7 +898,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,struct nfs_page *req;unsigned int req_len =min_t (size_t, bytes,PAGE_SIZE - pgbase); - req =nfs_page_create_from_page (dreq->ctx, pagevec[i], + req =nfs_page_create_from_page (dreq->ctx, pagevec[i], false, pgbase, pos, req_len);if (IS_ERR (req)) { result =PTR_ERR (req);
fs/nfs/pagelist.c +13 -5
@@ -413,11 +413,14 @@ static void nfs_page_assign_folio(struct nfs_page *req, struct folio *folio) } } -static void nfs_page_assign_page (struct nfs_page *req,struct page *page) +static void nfs_page_assign_page (struct nfs_page *req,struct page *page,bool pinned) {if (page !=NULL ) { req->wb_page = page; -get_page (page); +if (pinned) +set_bit (PG_PINNED , &req->wb_flags); +else +get_page (page); } }
@@ -435,6 +438,7 @@ static void nfs_page_assign_page(struct nfs_page *req, struct page *page)*/ struct nfs_page *nfs_page_create_from_page (struct nfs_open_context *ctx,struct page *page, +bool pinned,unsigned int pgbase,loff_t offset,unsigned int count) {
@@ -446,7 +450,7 @@ struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx, ret =nfs_page_create (l_ctx, pgbase, offset >>PAGE_SHIFT ,offset_in_page (offset), count);if (!IS_ERR (ret)) { -nfs_page_assign_page (ret, page); +nfs_page_assign_page (ret, page, pinned);nfs_page_group_init (ret,NULL ); }nfs_put_lock_context (l_ctx);
@@ -500,7 +504,8 @@ nfs_create_subreq(struct nfs_page *req,if (folio)nfs_page_assign_folio (ret, folio);else -nfs_page_assign_page (ret, page); +nfs_page_assign_page (ret, page, +test_bit (PG_PINNED , &req->wb_flags));/* find the last request */ for (last = req->wb_head; last->wb_this_page != req->wb_head;
@@ -556,7 +561,10 @@ static void nfs_clear_request(struct nfs_page *req) req->wb_folio =NULL ;clear_bit (PG_FOLIO , &req->wb_flags); }else if (page !=NULL ) { -put_page (page); +if (test_and_clear_bit (PG_PINNED , &req->wb_flags)) +unpin_user_page (page); +else +put_page (page); req->wb_page =NULL ; }if (l_ctx !=NULL ) {
include/linux/nfs_page.h +2 -0
@@ -37,6 +37,7 @@ enum {PG_REMOVE ,/* page group sync bit in write path */ PG_CONTENDED1 ,/* Is someone waiting for a lock? */ PG_CONTENDED2 ,/* Is someone waiting for a lock? */ + PG_PINNED, /* page is pinned by GUP */ };struct nfs_inode ;
@@ -124,6 +125,7 @@ struct nfs_pageio_descriptor {extern struct nfs_page *nfs_page_create_from_page (struct nfs_open_context *ctx,struct page *page, +bool pinned,unsigned int pgbase,loff_t offset,unsigned int count);
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
Re: [RFC PATCH 3/4] nfs: make nfs_page pin-aware
This conversion really should go first as it is badly needed independent of any P2P support. And I wonder if it should go further - currently the NFS I/O code is using folios for buffered I/O, but pages for direct I/O, which makes larger I/O very inefficient.
The iov_iter_extract_bvecs wrapper allows to extract bvecs instead, which might be a good choice here either by passing down the bvecs or converting to an nfs_page inline. Or just open coding a variant of iov_iter_extract_bvecs that converts to nfs_page structures instead of bvecs. This would pair with a helper similar to __bio_release_pages on the unlock side.
Review
Quoted 3 lines
(quoted diff) +1
-0+ req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false, pgbase, pos, req_len);
A lot of this code reads pretty odd as it's overflowing the lines.
Re: [RFC PATCH 3/4] nfs: make nfs_page pin-aware
Quoted 5 lines · Christoph Hellwig
This conversion really should go first as it is badly needed independent of any P2P support. And I wonder if it should go further - currently the NFS I/O code is using folios for buffered I/O, but pages for direct I/O, which makes larger I/O very inefficient.
Ack. I'll send this out as a different series with migration to folios for direct I/O as well.
Quoted 6 lines
The
iov_iter_extract_bvecswrapper allows to extract bvecs instead, which might be a good choice here either by passing down the bvecs or converting to annfs_pageinline. Or just open coding a variant ofiov_iter_extract_bvecsthat converts tonfs_pagestructures instead of bvecs. This would pair with a helper similar to__bio_release_pageson the unlock side.
Ack. I'll attempt an open coded variant.
Quoted 7 lines
Review
Quoted 4 lines
(quoted diff) +1
-0+ req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false, pgbase, pos, req_len);A lot of this code reads pretty odd as it's overflowing the lines.
Ahh, my bad. For some reason even checkpatch didn't catch this, I'll fix this here and everywhere else.
Thanks,
Praan
Re: [RFC PATCH 3/4] nfs: make nfs_page pin-aware
Quoted 9 lines · Pranjal Shrivastava
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +1
-0+ req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false, pgbase, pos, req_len);A lot of this code reads pretty odd as it's overflowing the lines.
Ahh, my bad. For some reason even checkpatch didn't catch this, I'll fix this here and everywhere else.
checkpatch is unfortunately completely broken :( It misses lots of important bits, but at the same time has complete incoherent and crazy warnings.
RFC PATCH 4/4 nfs: allow P2PDMA in direct I/O path
In reply to Pranjal ShrivastavaMigrate the NFS Direct I/O path from the legacy iov_iter_get_pages_alloc2() API to the modern iov_iter_extract_pages() API. This migration enables support for PCI Peer-to-Peer DMA (P2PDMA) by allowing the setting the ITER_ALLOW_P2PDMA flag.
Pass ITER_ALLOW_P2PDMA to iov_iter_extract_pages() only if the local mount indicates support via the NFS_CAP_P2PDMA capability bit (detected at mount time for RDMA transports).
Fix the memory safety bug in the Direct I/O loop where pages were being unpinned immediately after request creation. Instead, we now leverage pin-aware nfs_page structures to hold the pins until the I/O is complete The manual release in the loop is updated to only clean up pages that failed to be handed over to an nfs_page request.
--- fs/nfs/direct.c | 51 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 37 insertions(+), 14 deletions(-)
fs/nfs/direct.c +37 -14
@@ -165,11 +165,17 @@ int nfs_swap_rw(struct kiocb *iocb, struct iov_iter *iter)return 0 ; } -static void nfs_direct_release_pages (struct page **pages,unsigned int npages) +static void nfs_direct_release_pages (struct page **pages,unsigned int npages, +bool pinned) {unsigned int i; -for (i =0 ; i < npages; i++) -put_page (pages[i]); + +if (pinned) { +unpin_user_pages (pages, npages); + }else { +for (i =0 ; i < npages; i++) +put_page (pages[i]); + } } void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo,
@@ -354,23 +360,30 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,inode_dio_begin (inode);while (iov_iter_count (iter)) { -struct page **pagevec; +/* Tell extract pages to allocate the page array */ +struct page **pagevec =NULL ;size_t bytes;size_t pgbase;unsigned npages, i; +bool pinned =iov_iter_extract_will_pin (iter); +iov_iter_extraction_t extraction_flags =0 ; + +if (NFS_SERVER (inode)->caps &NFS_CAP_P2PDMA ) + extraction_flags |=ITER_ALLOW_P2PDMA ; - result =iov_iter_get_pages_alloc2 (iter, &pagevec, - rsize, &pgbase); + result =iov_iter_extract_pages (iter, &pagevec, + rsize, ~0U , + extraction_flags, &pgbase);if (result <0 )break ; - + bytes = result; npages = (result + pgbase +PAGE_SIZE -1 ) /PAGE_SIZE ;for (i =0 ; i < npages; i++) {struct nfs_page *req;unsigned int req_len =min_t (size_t, bytes,PAGE_SIZE - pgbase);/* XXX do we need to do the eof zeroing found in async_filler? */ - req =nfs_page_create_from_page (dreq->ctx, pagevec[i], false, + req =nfs_page_create_from_page (dreq->ctx, pagevec[i], pinned, pgbase, pos, req_len);if (IS_ERR (req)) { result =PTR_ERR (req);
@@ -386,7 +399,8 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq, requested_bytes += req_len; pos += req_len; } -nfs_direct_release_pages (pagevec, npages); +if (i < npages) +nfs_direct_release_pages (pagevec + i, npages - i, pinned);kvfree (pagevec);if (result <0 )break ;
@@ -882,13 +896,21 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,NFS_I (inode)->write_io +=iov_iter_count (iter);while (iov_iter_count (iter)) { -struct page **pagevec; + +/* Tell extract pages to allocate the page array */ +struct page **pagevec =NULL ;size_t bytes;size_t pgbase;unsigned npages, i; +bool pinned =iov_iter_extract_will_pin (iter); +iov_iter_extraction_t extraction_flags =0 ; + +if (NFS_SERVER (inode)->caps &NFS_CAP_P2PDMA ) + extraction_flags |=ITER_ALLOW_P2PDMA ; - result =iov_iter_get_pages_alloc2 (iter, &pagevec, - wsize, &pgbase); + result =iov_iter_extract_pages (iter, &pagevec, + wsize, ~0U , + extraction_flags, &pgbase);if (result <0 )break ;
@@ -898,7 +920,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,struct nfs_page *req;unsigned int req_len =min_t (size_t, bytes,PAGE_SIZE - pgbase); - req =nfs_page_create_from_page (dreq->ctx, pagevec[i], false, + req =nfs_page_create_from_page (dreq->ctx, pagevec[i], pinned, pgbase, pos, req_len);if (IS_ERR (req)) { result =PTR_ERR (req);
@@ -942,7 +964,8 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq, desc.pg_error =0 ; defer = true; } -nfs_direct_release_pages (pagevec, npages); +if (i < npages) +nfs_direct_release_pages (pagevec + i, npages - i, pinned);kvfree (pagevec);if (result <0 )break ;
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
Re: [RFC PATCH 4/4] nfs: allow P2PDMA in direct I/O path
Quoted 8 lines · Pranjal Shrivastava
Migrate the NFS Direct I/O path from the legacy
iov_iter_get_pages_alloc2()API to the moderniov_iter_extract_pages()API. This migration enables support for PCI Peer-to-Peer DMA (P2PDMA) by allowing the setting theITER_ALLOW_P2PDMAflag.Pass
ITER_ALLOW_P2PDMAtoiov_iter_extract_pages()only if the local mount indicates support via theNFS_CAP_P2PDMAcapability bit (detected at mount time for RDMA transports).
Please split theconversion to iov_iter_extract_pages into a separate preparation patch, and even series. That is a long overdue change that fixes potential data corruption in XFS.
Re: [RFC PATCH 4/4] nfs: allow P2PDMA in direct I/O path
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Pranjal Shrivastava
Migrate the NFS Direct I/O path from the legacy
iov_iter_get_pages_alloc2()API to the moderniov_iter_extract_pages()API. This migration enables support for PCI Peer-to-Peer DMA (P2PDMA) by allowing the setting theITER_ALLOW_P2PDMAflag.
Quoted 8 lines
Quoted 3 lines
Pass
ITER_ALLOW_P2PDMAtoiov_iter_extract_pages()only if the local mount indicates support via theNFS_CAP_P2PDMAcapability bit (detected at mount time for RDMA transports).Please split theconversion to
iov_iter_extract_pagesinto a separate preparation patch, and even series. That is a long overdue change that fixes potential data corruption in XFS.
Sure, I'll send out a series with the migration to iov_iter_extract_pages, should I club this with the pin-aware + folios for direct I/O or send it as a separate series?
Thanks,
Praan
Re: [RFC PATCH 4/4] nfs: allow P2PDMA in direct I/O path
Quoted 8 lines · Pranjal Shrivastava
Quoted 4 lines
Please split theconversion to
iov_iter_extract_pagesinto a separate preparation patch, and even series. That is a long overdue change that fixes potential data corruption in XFS.Sure, I'll send out a series with the migration to
iov_iter_extract_pages, should I club this with the pin-aware + folios for direct I/O or send it as a separate series?
I think combining all this sounds find. I'd just do the P2P separately as it is bound to get quite a bit more complicated.
Besides the point of splitting out the iov_iter_extract* conversion this seems to ignore pNFS. You also need to check the layout driver for the current file range and propagate P2P support or lack of through that.
Note that the block-style layouts can also trivially support P2P and not just RPC-based ones.