Skip to contentSkip to conversation

nfs: Enable PCI Peer-to-Peer DMA (P2PDMA) support

RFC PATCH In review Pranjal Shrivastava 3 participants 15 messages Apr 1 – Apr 16

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover nfs: Enable PCI Peer-to-Peer DMA (P2PDMA) support

As high-performance storage environments increasingly rely on direct data movement between PCIe endpoints (e.g., moving data directly between an NVMe Controller Memory Buffer and a Network Interface), support for Peer-to-Peer DMA (P2PDMA) in the network filesystem layer becomes essential. This series introduces P2PDMA support for the NFS Direct I/O.

Currently, NFS O_DIRECT operations fail with -EREMOTEIO if the user buffer resides in PCIe BAR memory. This is primarily due to the use of the legacy iov_iter_get_pages_alloc2() API, which cannot pass the required FOLL_PCI_P2PDMA flag, and a request lifecycle that is unaware
of the pinning requirements for P2P memory.

Design
=======
The proposed design centers around making the NFS request lifecycle "pin-aware" and upgrading the infrastructure to support modern memory extraction APIs.

1. 64-bit Capability Infrastructure
The existing nfs_server->caps bitmask is limited to 32 bits and is currently exhausted. This series expands the bitmask to 64 bits to accommodate NFS_CAP_P2PDMA. Crucially, it also refactors the NFS_CAP_* constants to use ULL definitions. This prevents a subtle 32-bit truncation bug where bitwise negations (e.g., caps &= ~NFS_CAP_ACLS) would accidentally clear the high bits of the 64-bit capability field.

2. Transport-Level Detection
P2PDMA support is a property of the local transport hardware. A new supports_p2pdma operation is added to the SunRPC transport ops. For RDMA, this is implemented by querying the underlying device via ib_dma_pci_p2p_dma_supported(). The NFS client queries this during mount and sets the NFS_CAP_P2PDMA bit accordingly.

3. Pin-Aware Request Lifecycle
Standard NFS requests use get_page() and put_page() for memory management. However, memory extracted via iov_iter_extract_pages() requires explicit pinning and unpinning (unpin_user_page()).

This series introduces a PG_PINNED flag in struct nfs_page. When set, the request lifecycle skips standard page referencing and ensures that unpin_user_page() is called only when the I/O is complete. This ensures that physical memory remains pinned for the duration of the DMA transfer

4. API Migration
The Direct I/O path is migrated to the modern iov_iter_extract_pages() API. The ITER_ALLOW_P2PDMA flag is passed to the iterator only when the local mount has signaled P2P support via the capability bit. This ensures that "normal" users on standard TCP/UDP transports see no change in behavior or overhead.

Call for review
===============
Any insights on the proposed changes to the nfs_page lifecycle and the 64-bit capability expansion are appreciated. If this approach is deemed incorrect or if there is a more idiomatic way for this, please direct me in the right direction.

Thanks,
Praan

Pranjal Shrivastava (4):
sunrpc: add supports_p2pdma to rpc_xprt_ops
nfs: add NFS_CAP_P2PDMA and detect transport support
nfs: make nfs_page pin-aware
nfs: allow P2PDMA in direct I/O path

 fs/nfs/client.c                 |  8 ++++
 fs/nfs/direct.c                 | 51 ++++++++++++++++++-------
 fs/nfs/nfs4_fs.h                |  2 +-
 fs/nfs/pagelist.c               | 18 ++++++---
 fs/nfs/super.c                  |  2 +-
 include/linux/nfs_fs_sb.h       | 67 +++++++++++++++++----------------
 include/linux/nfs_page.h        |  2 +
 include/linux/sunrpc/xprt.h     |  1 +
 net/sunrpc/xprtrdma/transport.c |  9 +++++
 9 files changed, 106 insertions(+), 54 deletions(-)
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
#1 lore raw

RFC PATCH 1/4 sunrpc: add supports_p2pdma to rpc_xprt_ops

Add a new transport op, supports_p2pdma, to allow upper layers (such as NFS) to query whether the underlying RPC transport supports PCI Peer-to-Peer DMA (P2PDMA).

Since the capability is hardware-dependent. For the RDMA transport, implement this by querying the underlying InfiniBand device via ib_dma_pci_p2p_dma_supported() to ensures that P2PDMA is only attempted when both the transport and the HCA drivers support it.

---
 include/linux/sunrpc/xprt.h     | 1 +
 net/sunrpc/xprtrdma/transport.c | 9 +++++++++
 2 files changed, 10 insertions(+)
include/linux/sunrpc/xprt.h +1 -0
index f46d1fb8f71a..e451acd2e047 100644
@@ -187,6 +187,7 @@ struct rpc_xprt_ops {
 	void		(*bc_free_rqst)(struct rpc_rqst *rqst);
 	void		(*bc_destroy)(struct rpc_xprt *xprt,
 				      unsigned int max_reqs);
+	bool		(*supports_p2pdma)(struct rpc_xprt *xprt);
 };
 
 /*
net/sunrpc/xprtrdma/transport.c +9 -0
index 9a8ce5df83ca..1c1714189a29 100644
@@ -717,6 +717,14 @@ xprt_rdma_disable_swap(struct rpc_xprt *xprt)
 {
 }
 
+static bool
+xprt_rdma_supports_p2pdma(struct rpc_xprt *xprt)
+{
+	struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
+
+	return ib_dma_pci_p2p_dma_supported(r_xprt->rx_ep->re_id->device);
+}
+
 /*
  * Plumbing for rpc transport switch and kernel module
  */
@@ -742,6 +750,7 @@ static const struct rpc_xprt_ops xprt_rdma_procs = {
 	.enable_swap		= xprt_rdma_enable_swap,
 	.disable_swap		= xprt_rdma_disable_swap,
 	.inject_disconnect	= xprt_rdma_inject_disconnect,
+	.supports_p2pdma	= xprt_rdma_supports_p2pdma,
 #if defined(CONFIG_SUNRPC_BACKCHANNEL)
 	.bc_setup		= xprt_rdma_bc_setup,
 	.bc_maxpayload		= xprt_rdma_bc_maxpayload,
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
#2 lore raw

RFC PATCH 2/4 nfs: add NFS_CAP_P2PDMA and detect transport support

In reply to Pranjal Shrivastava

The NFS server capabilities bitmask (server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.

Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.

Update nfs_server_set_init_caps() to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.

---
 fs/nfs/client.c           |  8 +++++
 fs/nfs/nfs4_fs.h          |  2 +-
 fs/nfs/super.c            |  2 +-
 include/linux/nfs_fs_sb.h | 67 ++++++++++++++++++++-------------------
 4 files changed, 44 insertions(+), 35 deletions(-)
fs/nfs/client.c +8 -0
index be02bb227741..f177cf098d44 100644
@@ -712,6 +712,8 @@ static void nfs4_server_set_init_caps(struct nfs_server *server)
 
 void nfs_server_set_init_caps(struct nfs_server *server)
 {
+	struct rpc_xprt *xprt;
+
 	switch (server->nfs_client->rpc_ops->version) {
 	case 2:
 		server->caps = NFS_CAP_HARDLINKS | NFS_CAP_SYMLINKS;
@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)
 		nfs4_server_set_init_caps(server);
 		break;
 	}
+
+	rcu_read_lock();
+	xprt = rcu_dereference(server->client->cl_xprt);
+	if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma(xprt))
+		server->caps |= NFS_CAP_P2PDMA;
+	rcu_read_unlock();
 }
 EXPORT_SYMBOL_GPL(nfs_server_set_init_caps);
 
fs/nfs/nfs4_fs.h +1 -1
index b48e5b87cb2a..a309cc739fa3 100644
@@ -60,7 +60,7 @@ enum nfs4_client_state {
 struct nfs_seqid_counter;
 struct nfs4_minor_version_ops {
 	u32	minor_version;
-	unsigned init_caps;
+	u64	init_caps;
 
 	int	(*init_client)(struct nfs_client *);
 	void	(*shutdown_client)(struct nfs_client *);
fs/nfs/super.c +1 -1
index 7a318581f85b..b2de13a355df 100644
@@ -672,7 +672,7 @@ int nfs_show_stats(struct seq_file *m, struct dentry *root)
 	show_implementation_id(m, nfss);
 
 	seq_puts(m, "\n\tcaps:\t");
-	seq_printf(m, "caps=0x%x", nfss->caps);
+	seq_printf(m, "caps=0x%llx", nfss->caps);
 	seq_printf(m, ",wtmult=%u", nfss->wtmult);
 	seq_printf(m, ",dtsize=%u", nfss->dtsize);
 	seq_printf(m, ",bsize=%u", nfss->bsize);
include/linux/nfs_fs_sb.h +34 -33

77 lines not on this page · raw message

Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
maintainer
#3 lore raw

Re: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support

Quoted 21 lines · Pranjal Shrivastava

The NFS server capabilities bitmask (server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.

Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.

Update nfs_server_set_init_caps() to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.

fs/nfs/client.c +0 -0
index be02bb227741..f177cf098d44 100644
@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)
 		nfs4_server_set_init_caps(server);

fs/nfs/client.c void nfs_server_set_init_caps(struct nfs_server *server) · line 737

Quoted 10 lines
fs/nfs/client.c +6 -0
 		break;
 	}
+
+	rcu_read_lock();
+	xprt = rcu_dereference(server->client->cl_xprt);
+	if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma(xprt))
+		server->caps |= NFS_CAP_P2PDMA;
+	rcu_read_unlock();
 }
 EXPORT_SYMBOL_GPL(nfs_server_set_init_caps);

Is the transport even connected when the NFS client does this test? If it isn't, xprtrdma and the RDMA core have not chosen an underlying device yet.

Note that, even if this logic /is/ correct, if the transport connection is lost the transport will reconnect automatically, doing the RDMA CM dance again and possibly resolving to a different device. The NFS client layer will be none-the-wiser and the NFS_CAP_P2PDMA flag setting will be stale at that point, and quite possibly incorrect if the new connection's device is not P2P-enabled.

(Basically this is what happens when an RDMA device is removed).

So this detection has to be done as part of xprtrdma's connection flow, and it needs to set a flag somewhere in the rpc_xprt. The NFS direct I/O code path then has to look for that flag before choosing the mechanism/flags it uses for each iov iter.

Signature · 1 line
Chuck Lever
#4 lore raw

Re: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support

Quoted 46 lines · Chuck Lever
Quoted 21 lines · Pranjal Shrivastava

The NFS server capabilities bitmask (server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.

Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.

Update nfs_server_set_init_caps() to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.

fs/nfs/client.c +0 -0
index be02bb227741..f177cf098d44 100644
@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)
 		nfs4_server_set_init_caps(server);

fs/nfs/client.c void nfs_server_set_init_caps(struct nfs_server *server) · line 737

Quoted 10 lines
fs/nfs/client.c +6 -0
 		break;
 	}
+
+	rcu_read_lock();
+	xprt = rcu_dereference(server->client->cl_xprt);
+	if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma(xprt))
+		server->caps |= NFS_CAP_P2PDMA;
+	rcu_read_unlock();
 }
 EXPORT_SYMBOL_GPL(nfs_server_set_init_caps);

Is the transport even connected when the NFS client does this test? If it isn't, xprtrdma and the RDMA core have not chosen an underlying device yet.

Note that, even if this logic /is/ correct, if the transport connection is lost the transport will reconnect automatically, doing the RDMA CM dance again and possibly resolving to a different device. The NFS client layer will be none-the-wiser and the NFS_CAP_P2PDMA flag setting will be stale at that point, and quite possibly incorrect if the new connection's device is not P2P-enabled.

Quoted 7 lines

(Basically this is what happens when an RDMA device is removed).

So this detection has to be done as part of xprtrdma's connection flow, and it needs to set a flag somewhere in the rpc_xprt. The NFS direct I/O code path then has to look for that flag before choosing the mechanism/flags it uses for each iov iter.

Ack. I agree, so should we start with an inital cap and then update it in the event of a transport change / disconnect? Or shall we populate the cap only when a transport is connected?

Thanks,
Praan

maintainer
#5 lore raw

Re: [RFC PATCH 2/4] nfs: add NFS_CAP_P2PDMA and detect transport support

Quoted 55 lines · Pranjal Shrivastava
Quoted 46 lines · Chuck Lever
Quoted 21 lines · Pranjal Shrivastava

The NFS server capabilities bitmask (server->caps) is currently full, utilizing all 32 bits of the existing unsigned int. Expand the bitmask to 64 bits (u64) to allow for new feature flags.

Introduce a new capability bit, NFS_CAP_P2PDMA, to indicate that the local mount is backed by hardware and a transport capable of PCI Peer-to-Peer DMA.

Update nfs_server_set_init_caps() to query the underlying SunRPC transport for P2PDMA support during the mount process. If the transport (e.g., RDMA) signals support, set the NFS_CAP_P2PDMA bit in the mount's capabilities. This allows the high-performance Direct I/O path to efficiently determine if it should allow P2P memory buffers.

fs/nfs/client.c +0 -0
index be02bb227741..f177cf098d44 100644
@@ -725,6 +727,12 @@ void nfs_server_set_init_caps(struct nfs_server *server)
 		nfs4_server_set_init_caps(server);

fs/nfs/client.c void nfs_server_set_init_caps(struct nfs_server *server) · line 737

Quoted 10 lines
fs/nfs/client.c +6 -0
 		break;
 	}
+
+	rcu_read_lock();
+	xprt = rcu_dereference(server->client->cl_xprt);
+	if (xprt->ops->supports_p2pdma && xprt->ops->supports_p2pdma(xprt))
+		server->caps |= NFS_CAP_P2PDMA;
+	rcu_read_unlock();
 }
 EXPORT_SYMBOL_GPL(nfs_server_set_init_caps);

Is the transport even connected when the NFS client does this test? If it isn't, xprtrdma and the RDMA core have not chosen an underlying device yet.

Note that, even if this logic /is/ correct, if the transport connection is lost the transport will reconnect automatically, doing the RDMA CM dance again and possibly resolving to a different device. The NFS client layer will be none-the-wiser and the NFS_CAP_P2PDMA flag setting will be stale at that point, and quite possibly incorrect if the new connection's device is not P2P-enabled.

Quoted 7 lines

(Basically this is what happens when an RDMA device is removed).

So this detection has to be done as part of xprtrdma's connection flow, and it needs to set a flag somewhere in the rpc_xprt. The NFS direct I/O code path then has to look for that flag before choosing the mechanism/flags it uses for each iov iter.

Quoted 3 lines

Ack. I agree, so should we start with an inital cap and then update it in the event of a transport change / disconnect? Or shall we populate the cap only when a transport is connected?

IMO this flag does not belong in the NFS server CAPS, as it is a capability associated with each RPC transport. How should NFS_CAP_P2PDMA be set if there are two RPC transports, one with P2PDMA enabled and with it disabled? (Perhaps it should be a flag in the transport switch instance rather than the transport instance).

Which mechanism to use has to be re-decided every time a dreq is scheduled because the xprt can change between an original send and a retransmission (if, say, the COMMIT verifier changes due to a server reboot).

Trond and Anna will have the final say about how this works.

Signature · 1 line
Chuck Lever
#6 lore raw

RFC PATCH 3/4 nfs: make nfs_page pin-aware

In reply to Pranjal Shrivastava

The migration to iov_iter_extract_pages() for Direct I/O introduces page pinning (GUP) instead of standard page referencing. To handle this correctly, nfs_page must track whether it holds a pin or a standard reference.

Add a new flag, PG_PINNED, to struct nfs_page. Update the creation path (nfs_page_create_from_page) to accept a 'pinned' boolean and set the flag accordingly. If the page is pinned, we skip the standard get_page() as the pin itself acts as a reference.

Update nfs_clear_request() to use unpin_user_page() instead of put_page() when the PG_PINNED flag is set. This ensures that memory remains safely locked for DMA and that kernel page accounting stays consistent. Ensure subrequests inherit the pin status from their parent request.

---
 fs/nfs/direct.c          |  4 ++--
 fs/nfs/pagelist.c        | 18 +++++++++++++-----
 include/linux/nfs_page.h |  2 ++
 3 files changed, 17 insertions(+), 7 deletions(-)
fs/nfs/direct.c +2 -2
index 48d89716193a..c8429b430181 100644
@@ -370,7 +370,7 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 			struct nfs_page *req;
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 			/* XXX do we need to do the eof zeroing found in async_filler? */
-			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
 							pgbase, pos, req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
@@ -898,7 +898,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			struct nfs_page *req;
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 
-			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
 							pgbase, pos, req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
fs/nfs/pagelist.c +13 -5
index a9373de891c9..72d3da0fb654 100644
@@ -413,11 +413,14 @@ static void nfs_page_assign_folio(struct nfs_page *req, struct folio *folio)
 	}
 }
 
-static void nfs_page_assign_page(struct nfs_page *req, struct page *page)
+static void nfs_page_assign_page(struct nfs_page *req, struct page *page, bool pinned)
 {
 	if (page != NULL) {
 		req->wb_page = page;
-		get_page(page);
+		if (pinned)
+			set_bit(PG_PINNED, &req->wb_flags);
+		else
+			get_page(page);
 	}
 }
 
@@ -435,6 +438,7 @@ static void nfs_page_assign_page(struct nfs_page *req, struct page *page)
  */
 struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 					   struct page *page,
+					   bool pinned,
 					   unsigned int pgbase, loff_t offset,
 					   unsigned int count)
 {
@@ -446,7 +450,7 @@ struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 	ret = nfs_page_create(l_ctx, pgbase, offset >> PAGE_SHIFT,
 			      offset_in_page(offset), count);
 	if (!IS_ERR(ret)) {
-		nfs_page_assign_page(ret, page);
+		nfs_page_assign_page(ret, page, pinned);
 		nfs_page_group_init(ret, NULL);
 	}
 	nfs_put_lock_context(l_ctx);
@@ -500,7 +504,8 @@ nfs_create_subreq(struct nfs_page *req,
 		if (folio)
 			nfs_page_assign_folio(ret, folio);
 		else
-			nfs_page_assign_page(ret, page);
+			nfs_page_assign_page(ret, page,
+					     test_bit(PG_PINNED, &req->wb_flags));
 		/* find the last request */
 		for (last = req->wb_head;
 		     last->wb_this_page != req->wb_head;
@@ -556,7 +561,10 @@ static void nfs_clear_request(struct nfs_page *req)
 		req->wb_folio = NULL;
 		clear_bit(PG_FOLIO, &req->wb_flags);
 	} else if (page != NULL) {
-		put_page(page);
+		if (test_and_clear_bit(PG_PINNED, &req->wb_flags))
+			unpin_user_page(page);
+		else
+			put_page(page);
 		req->wb_page = NULL;
 	}
 	if (l_ctx != NULL) {
include/linux/nfs_page.h +2 -0
index afe1d8f09d89..cae67540a2ae 100644
@@ -37,6 +37,7 @@ enum {
 	PG_REMOVE,		/* page group sync bit in write path */
 	PG_CONTENDED1,		/* Is someone waiting for a lock? */
 	PG_CONTENDED2,		/* Is someone waiting for a lock? */
+	PG_PINNED,		/* page is pinned by GUP */
 };
 
 struct nfs_inode;
@@ -124,6 +125,7 @@ struct nfs_pageio_descriptor {
 
 extern struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 						  struct page *page,
+						  bool pinned,
 						  unsigned int pgbase,
 						  loff_t offset,
 						  unsigned int count);
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
#7 lore raw

Re: [RFC PATCH 3/4] nfs: make nfs_page pin-aware

This conversion really should go first as it is badly needed independent of any P2P support. And I wonder if it should go further - currently the NFS I/O code is using folios for buffered I/O, but pages for direct I/O, which makes larger I/O very inefficient.

The iov_iter_extract_bvecs wrapper allows to extract bvecs instead, which might be a good choice here either by passing down the bvecs or converting to an nfs_page inline. Or just open coding a variant of iov_iter_extract_bvecs that converts to nfs_page structures instead of bvecs. This would pair with a helper similar to __bio_release_pages on the unlock side.

Review

Quoted 3 lines
(quoted diff) +1 -0
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
 							pgbase, pos, req_len);

A lot of this code reads pretty odd as it's overflowing the lines.

#8 lore raw

Re: [RFC PATCH 3/4] nfs: make nfs_page pin-aware

Quoted 5 lines · Christoph Hellwig

This conversion really should go first as it is badly needed independent of any P2P support. And I wonder if it should go further - currently the NFS I/O code is using folios for buffered I/O, but pages for direct I/O, which makes larger I/O very inefficient.

Ack. I'll send this out as a different series with migration to folios for direct I/O as well.

Quoted 6 lines

The iov_iter_extract_bvecs wrapper allows to extract bvecs instead, which might be a good choice here either by passing down the bvecs or converting to an nfs_page inline. Or just open coding a variant of iov_iter_extract_bvecs that converts to nfs_page structures instead of bvecs. This would pair with a helper similar to __bio_release_pages on the unlock side.

Ack. I'll attempt an open coded variant.

Quoted 7 lines

Review

Quoted 4 lines
(quoted diff) +1 -0
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
 							pgbase, pos, req_len);

A lot of this code reads pretty odd as it's overflowing the lines.

Ahh, my bad. For some reason even checkpatch didn't catch this, I'll fix this here and everywhere else.

Thanks,
Praan

#9 lore raw

Re: [RFC PATCH 3/4] nfs: make nfs_page pin-aware

Quoted 9 lines · Pranjal Shrivastava
Quoted 6 lines

Review

Quoted 3 lines
(quoted diff) +1 -0
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
 							pgbase, pos, req_len);

A lot of this code reads pretty odd as it's overflowing the lines.

Ahh, my bad. For some reason even checkpatch didn't catch this, I'll fix this here and everywhere else.

checkpatch is unfortunately completely broken :( It misses lots of important bits, but at the same time has complete incoherent and crazy warnings.

#10 lore raw

RFC PATCH 4/4 nfs: allow P2PDMA in direct I/O path

In reply to Pranjal Shrivastava

Migrate the NFS Direct I/O path from the legacy iov_iter_get_pages_alloc2() API to the modern iov_iter_extract_pages() API. This migration enables support for PCI Peer-to-Peer DMA (P2PDMA) by allowing the setting the ITER_ALLOW_P2PDMA flag.

Pass ITER_ALLOW_P2PDMA to iov_iter_extract_pages() only if the local mount indicates support via the NFS_CAP_P2PDMA capability bit (detected at mount time for RDMA transports).

Fix the memory safety bug in the Direct I/O loop where pages were being unpinned immediately after request creation. Instead, we now leverage pin-aware nfs_page structures to hold the pins until the I/O is complete The manual release in the loop is updated to only clean up pages that failed to be handed over to an nfs_page request.

---
 fs/nfs/direct.c | 51 +++++++++++++++++++++++++++++++++++--------------
 1 file changed, 37 insertions(+), 14 deletions(-)
fs/nfs/direct.c +37 -14
index c8429b430181..6916541af9db 100644
@@ -165,11 +165,17 @@ int nfs_swap_rw(struct kiocb *iocb, struct iov_iter *iter)
 	return 0;
 }
 
-static void nfs_direct_release_pages(struct page **pages, unsigned int npages)
+static void nfs_direct_release_pages(struct page **pages, unsigned int npages,
+				     bool pinned)
 {
 	unsigned int i;
-	for (i = 0; i < npages; i++)
-		put_page(pages[i]);
+
+	if (pinned) {
+		unpin_user_pages(pages, npages);
+	} else {
+		for (i = 0; i < npages; i++)
+			put_page(pages[i]);
+	}
 }
 
 void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo,
@@ -354,23 +360,30 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 	inode_dio_begin(inode);
 
 	while (iov_iter_count(iter)) {
-		struct page **pagevec;
+		/* Tell extract pages to allocate the page array */
+		struct page **pagevec = NULL;
 		size_t bytes;
 		size_t pgbase;
 		unsigned npages, i;
+		bool pinned = iov_iter_extract_will_pin(iter);
+		iov_iter_extraction_t extraction_flags = 0;
+
+		if (NFS_SERVER(inode)->caps & NFS_CAP_P2PDMA)
+			extraction_flags |= ITER_ALLOW_P2PDMA;
 
-		result = iov_iter_get_pages_alloc2(iter, &pagevec,
-						  rsize, &pgbase);
+		result = iov_iter_extract_pages(iter, &pagevec,
+						rsize, ~0U,
+						extraction_flags, &pgbase);
 		if (result < 0)
 			break;
-	
+
 		bytes = result;
 		npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;
 		for (i = 0; i < npages; i++) {
 			struct nfs_page *req;
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 			/* XXX do we need to do the eof zeroing found in async_filler? */
-			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], pinned,
 							pgbase, pos, req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
@@ -386,7 +399,8 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 			requested_bytes += req_len;
 			pos += req_len;
 		}
-		nfs_direct_release_pages(pagevec, npages);
+		if (i < npages)
+			nfs_direct_release_pages(pagevec + i, npages - i, pinned);
 		kvfree(pagevec);
 		if (result < 0)
 			break;
@@ -882,13 +896,21 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 
 	NFS_I(inode)->write_io += iov_iter_count(iter);
 	while (iov_iter_count(iter)) {
-		struct page **pagevec;
+
+		/* Tell extract pages to allocate the page array */
+		struct page **pagevec = NULL;
 		size_t bytes;
 		size_t pgbase;
 		unsigned npages, i;
+		bool pinned = iov_iter_extract_will_pin(iter);
+		iov_iter_extraction_t extraction_flags = 0;
+
+		if (NFS_SERVER(inode)->caps & NFS_CAP_P2PDMA)
+			extraction_flags |= ITER_ALLOW_P2PDMA;
 
-		result = iov_iter_get_pages_alloc2(iter, &pagevec,
-						  wsize, &pgbase);
+		result = iov_iter_extract_pages(iter, &pagevec,
+						wsize, ~0U,
+						extraction_flags, &pgbase);
 		if (result < 0)
 			break;
 
@@ -898,7 +920,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			struct nfs_page *req;
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 
-			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], false,
+			req = nfs_page_create_from_page(dreq->ctx, pagevec[i], pinned,
 							pgbase, pos, req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
@@ -942,7 +964,8 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			desc.pg_error = 0;
 			defer = true;
 		}
-		nfs_direct_release_pages(pagevec, npages);
+		if (i < npages)
+			nfs_direct_release_pages(pagevec + i, npages - i, pinned);
 		kvfree(pagevec);
 		if (result < 0)
 			break;
Signature · 1 line
2.53.0.1185.g05d4b7b318-goog
#11 lore raw

Re: [RFC PATCH 4/4] nfs: allow P2PDMA in direct I/O path

Quoted 8 lines · Pranjal Shrivastava

Migrate the NFS Direct I/O path from the legacy iov_iter_get_pages_alloc2() API to the modern iov_iter_extract_pages() API. This migration enables support for PCI Peer-to-Peer DMA (P2PDMA) by allowing the setting the ITER_ALLOW_P2PDMA flag.

Pass ITER_ALLOW_P2PDMA to iov_iter_extract_pages() only if the local mount indicates support via the NFS_CAP_P2PDMA capability bit (detected at mount time for RDMA transports).

Please split theconversion to iov_iter_extract_pages into a separate preparation patch, and even series. That is a long overdue change that fixes potential data corruption in XFS.

#12 lore raw

Re: [RFC PATCH 4/4] nfs: allow P2PDMA in direct I/O path

Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Pranjal Shrivastava

Migrate the NFS Direct I/O path from the legacy iov_iter_get_pages_alloc2() API to the modern iov_iter_extract_pages() API. This migration enables support for PCI Peer-to-Peer DMA (P2PDMA) by allowing the setting the ITER_ALLOW_P2PDMA flag.

Quoted 8 lines
Quoted 3 lines

Pass ITER_ALLOW_P2PDMA to iov_iter_extract_pages() only if the local mount indicates support via the NFS_CAP_P2PDMA capability bit (detected at mount time for RDMA transports).

Please split theconversion to iov_iter_extract_pages into a separate preparation patch, and even series. That is a long overdue change that fixes potential data corruption in XFS.

Sure, I'll send out a series with the migration to iov_iter_extract_pages, should I club this with the pin-aware + folios for direct I/O or send it as a separate series?

Thanks,
Praan

#13 lore raw

Re: [RFC PATCH 4/4] nfs: allow P2PDMA in direct I/O path

Quoted 8 lines · Pranjal Shrivastava
Quoted 4 lines

Please split theconversion to iov_iter_extract_pages into a separate preparation patch, and even series. That is a long overdue change that fixes potential data corruption in XFS.

Sure, I'll send out a series with the migration to iov_iter_extract_pages, should I club this with the pin-aware + folios for direct I/O or send it as a separate series?

I think combining all this sounds find. I'd just do the P2P separately as it is bound to get quite a bit more complicated.

#14 lore raw
In reply to Pranjal Shrivastava

Besides the point of splitting out the iov_iter_extract* conversion this seems to ignore pNFS. You also need to check the layout driver for the current file range and propagate P2P support or lack of through that.

Note that the block-style layouts can also trivially support P2P and not just RPC-based ones.