Skip to contentSkip to conversation

nfs: Modernize Direct I/O path

PATCH Superseded Pranjal Shrivastava 4 participants 19 messages Jul 15 – Aug 5

Subsystems (4)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover nfs: Modernize Direct I/O path

Modernize the NFS Direct I/O path as a preparatory step to enable PCI Peer-to-Peer DMA (P2PDMA) support. Following feedback on the initial RFC [1], the modernization and architectural changes are split into different series. Additionally, based on the discussion in the v2 [2] of this series, the migration of NFS Direct I/O to folios would be handled in a separate follow-up series.

Currently, NFS O_DIRECT relies on the legacy iov_iter_get_pages_alloc2() API which does not support the pinning requirements for P2P memory. The implementation moves NFS to the modern iov_iter_extract_pages() API.

Design
======

1. Pin-Awareness
Standard NFS requests use get_page() and put_page() for memory management. However, memory extracted via iov_iter_extract_pages() requires explicit pinning.

Introduce a PG_PINNED flag and a wb_nr_pinned count to struct nfs_page. This allows the request lifecycle to track ownership of physical pins and ensure that unpinning is performed only when the I/O is complete.

2. API Migration
Migrate the Direct I/O path to the modern iov_iter_extract_pages() API. This aligns NFS with the modern extraction model and serves as the foundation for passing ITER_ALLOW_P2PDMA in a follow-up series.

Upcoming Work / Roadmap
================================

As decided in the RFC [1] & v2 [2], there will be separate series for P2PDMA Enablement and Migrating NFS Direct I/O to use folios.

This series lays the necessary groundwork for the upcoming work. Following this, two additional series are planned:

1. Migrating NFS Direct I/O to Folios
A series that introduces and exports helper from iov_iter.c to allow the nfs_direct_extract_pages() helper introduced here to aggregate multiple pages into a single large folio-based request, aiming to reduce the RPC overhead for hugepage I/O.

2. P2PDMA Enablement for NFS
Enabling ITER_ALLOW_P2PDMA for the Direct I/O path and introducing transport-level negotiation (discovery of P2P-capable RDMA/NVMe devices). This will build upon the PG_PINNED infrastructure introduced by this series.

Testing
=======
This series has been tested with xfstests [3] on RDMA & TCP transports by running the quick test suite for each transport vs. version combo:

./check -g quick -s rdma3 -s rdma40 -s rdma41 -s rdma42 -s tcp3 -s tcp40 -s tcp41 -s tcp42

The tests were run before & after applying the series. No regressions were observed.

The following summary was tabulated via a custom script [4] (on github) to depict that the tests failing in v1 [5] are now passing.

python3 display.py results/*/check.log

+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
| testcase     | rdma-sys-3   | rdma-sys-4.0 | rdma-sys-4.1 | rdma-sys-4.2 | tcp-sys-3    | tcp-sys-4.0  | tcp-sys-4.1  | tcp-sys-4.2  |
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+
| generic/091  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/130  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/139  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/143  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/154  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/155  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/183  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/188  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/190  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/196  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/198  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/203  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/214  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/240  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/263  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/287  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/290  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/292  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/330  | skipped      | skipped      | skipped      | pass         | skipped      | skipped      | skipped      | pass         |
| generic/444  | skipped      | skipped      | skipped      | skipped      | skipped      | skipped      | skipped      | skipped      |
| generic/450  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/451  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/586  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/647  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/708  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/729  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
| generic/760  | pass         | pass         | pass         | pass         | pass         | pass         | pass         | pass         |
+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+--------------+

Thanks,
Praan

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
[4] https://github.com/pran005/tools/blob/main/display.py
[5] https://lore.kernel.org/all/[email protected]/

[v3]
- Dropped patches that added folio support for NFS Direct I/O
- Folded requested_bytes accounting in patch 5 due to dropped folio support
- Rebased on fs-next

[v2]
- https://lore.kernel.org/all/[email protected]/
- Fix data corruption in nfs_direct_extract_pages() by correctly calculating intra-page offsets using offset_in_page().
- Fix requested_bytes accounting in direct read/write paths to only increment after successful RPC scheduling.
- Add missing kernel-doc descriptions for the @pinned parameter in nfs_page_create_from_page() and nfs_page_create_from_folio().
- Rebase on fs-next/

[v1] https://lore.kernel.org/all/[email protected]/

Pranjal Shrivastava (5):
nfs: make nfs_page pin-aware
nfs: Track number of pinned pages in nfs_page
nfs: Introduce nfs_release_request_list helper
nfs: migrate direct I/O to iov_iter_extract_pages
nfs: introduce nfs_direct_extract_pages helper

 fs/nfs/direct.c          | 143 ++++++++++++++++++++-------------------
 fs/nfs/pagelist.c        |  61 ++++++++++++++---
 fs/nfs/read.c            |   2 +-
 fs/nfs/write.c           |   2 +-
 include/linux/nfs_page.h |   8 ++-
 5 files changed, 134 insertions(+), 82 deletions(-)

base-commit: 98cd7881a2161fe187da5636e1c6cb53d741307a

Signature · 1 line
2.55.0.229.g6434b31f56-goog
#1 lore raw

PATCH v3 1/5 nfs: make nfs_page pin-aware

Modernizing the NFS Direct I/O path to use iov_iter_extract_pages() introduces page pinning (GUP) instead of standard page referencing. To handle this correctly, nfs_page must track whether it holds a pin or a standard reference.

Introduce a new flag, PG_PINNED, to struct nfs_page. Update the creation path (nfs_page_create_from_page and nfs_page_create_from_folio) to accept a pinned bool and set the flag accordingly. If the page is pinned, we skip the existing reference increment (get_page/folio_get) as the pin itself acts as a reference.

Update nfs_clear_request() & nfs_direct_release_pages() to use unpin_user_page() or unpin_user_folio() instead of only refcount decrement (put_page) when PG_PINNED flag is set. Finally, ensure subrequests inherit the pinning status from their parent request.

---
 fs/nfs/direct.c          | 22 +++++++++++++++-------
 fs/nfs/pagelist.c        | 38 ++++++++++++++++++++++++++++----------
 fs/nfs/read.c            |  2 +-
 fs/nfs/write.c           |  2 +-
 include/linux/nfs_page.h |  3 +++
 5 files changed, 48 insertions(+), 19 deletions(-)
fs/nfs/direct.c +15 -7
index e626c72495e6..19792a38c924 100644
@@ -165,11 +165,17 @@ int nfs_swap_rw(struct kiocb *iocb, struct iov_iter *iter)
 	return 0;
 }
 
-static void nfs_direct_release_pages(struct page **pages, unsigned int npages)
+static void nfs_direct_release_pages(struct page **pages, unsigned int npages,
+				     bool pinned)
 {
 	unsigned int i;
-	for (i = 0; i < npages; i++)
-		put_page(pages[i]);
+
+	if (pinned) {
+		unpin_user_pages(pages, npages);
+	} else {
+		for (i = 0; i < npages; i++)
+			put_page(pages[i]);
+	}
 }
 
 void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo,
@@ -371,7 +377,8 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 			/* XXX do we need to do the eof zeroing found in async_filler? */
 			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-							pgbase, pos, req_len);
+							false, pgbase, pos,
+							req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
 				break;
@@ -386,7 +393,7 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 			requested_bytes += req_len;
 			pos += req_len;
 		}
-		nfs_direct_release_pages(pagevec, npages);
+		nfs_direct_release_pages(pagevec, npages, false);
 		kvfree(pagevec);
 		if (result < 0)
 			break;
@@ -907,7 +914,8 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 
 			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-							pgbase, pos, req_len);
+							false, pgbase, pos,
+							req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
 				break;
@@ -950,7 +958,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			desc.pg_error = 0;
 			defer = true;
 		}
-		nfs_direct_release_pages(pagevec, npages);
+		nfs_direct_release_pages(pagevec, npages, false);
 		kvfree(pagevec);
 		if (result < 0)
 			break;
fs/nfs/pagelist.c +28 -10
index 7dd478ffc2fa..faa8bc1c6526 100644
@@ -404,20 +404,26 @@ static struct nfs_page *nfs_page_create(struct nfs_lock_context *l_ctx,
 	return req;
 }
 
-static void nfs_page_assign_folio(struct nfs_page *req, struct folio *folio)
+static void nfs_page_assign_folio(struct nfs_page *req, struct folio *folio, bool pinned)
 {
 	if (folio != NULL) {
 		req->wb_folio = folio;
-		folio_get(folio);
+		if (pinned)
+			set_bit(PG_PINNED, &req->wb_flags);
+		else
+			folio_get(folio);
 		set_bit(PG_FOLIO, &req->wb_flags);
 	}
 }
 
-static void nfs_page_assign_page(struct nfs_page *req, struct page *page)
+static void nfs_page_assign_page(struct nfs_page *req, struct page *page, bool pinned)
 {
 	if (page != NULL) {
 		req->wb_page = page;
-		get_page(page);
+		if (pinned)
+			set_bit(PG_PINNED, &req->wb_flags);
+		else
+			get_page(page);
 	}
 }
 
@@ -425,6 +431,7 @@ static void nfs_page_assign_page(struct nfs_page *req, struct page *page)
  * nfs_page_create_from_page - Create an NFS read/write request.
  * @ctx: open context to use
  * @page: page to write
+ * @pinned: true if page is pinned
  * @pgbase: starting offset within the page for the write
  * @offset: file offset for the write
  * @count: number of bytes to read/write
@@ -435,6 +442,7 @@ static void nfs_page_assign_page(struct nfs_page *req, struct page *page)
  */
 struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 					   struct page *page,
+					   bool pinned,
 					   unsigned int pgbase, loff_t offset,
 					   unsigned int count)
 {
@@ -446,7 +454,7 @@ struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 	ret = nfs_page_create(l_ctx, pgbase, offset >> PAGE_SHIFT,
 			      offset_in_page(offset), count);
 	if (!IS_ERR(ret)) {
-		nfs_page_assign_page(ret, page);
+		nfs_page_assign_page(ret, page, pinned);
 		nfs_page_group_init(ret, NULL);
 	}
 	nfs_put_lock_context(l_ctx);
@@ -457,6 +465,7 @@ struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
  * nfs_page_create_from_folio - Create an NFS read/write request.
  * @ctx: open context to use
  * @folio: folio to write
+ * @pinned: true if folio is pinned
  * @offset: starting offset within the folio for the write
  * @count: number of bytes to read/write
  *
@@ -466,6 +475,7 @@ struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
  */
 struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx,
 					    struct folio *folio,
+					    bool pinned,
 					    unsigned int offset,
 					    unsigned int count)
 {
@@ -476,7 +486,7 @@ struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx,
 		return ERR_CAST(l_ctx);
 	ret = nfs_page_create(l_ctx, offset, folio->index, offset, count);
 	if (!IS_ERR(ret)) {
-		nfs_page_assign_folio(ret, folio);
+		nfs_page_assign_folio(ret, folio, pinned);
 		nfs_page_group_init(ret, NULL);
 	}
 	nfs_put_lock_context(l_ctx);
@@ -498,9 +508,11 @@ nfs_create_subreq(struct nfs_page *req,
 			      offset, count);
 	if (!IS_ERR(ret)) {
 		if (folio)
-			nfs_page_assign_folio(ret, folio);
+			nfs_page_assign_folio(ret, folio,
+					      test_bit(PG_PINNED, &req->wb_flags));
 		else
-			nfs_page_assign_page(ret, page);
+			nfs_page_assign_page(ret, page,
+					     test_bit(PG_PINNED, &req->wb_flags));
 		/* find the last request */
 		for (last = req->wb_head;
 		     last->wb_this_page != req->wb_head;
@@ -552,11 +564,17 @@ static void nfs_clear_request(struct nfs_page *req)
 	struct nfs_open_context *ctx;
 
 	if (folio != NULL) {
-		folio_put(folio);
+		if (test_and_clear_bit(PG_PINNED, &req->wb_flags))
+			unpin_user_folio(folio, 1);
+		else
+			folio_put(folio);
 		req->wb_folio = NULL;
 		clear_bit(PG_FOLIO, &req->wb_flags);
 	} else if (page != NULL) {
-		put_page(page);
+		if (test_and_clear_bit(PG_PINNED, &req->wb_flags))
+			unpin_user_page(page);
+		else
+			put_page(page);
 		req->wb_page = NULL;
 	}
 	if (l_ctx != NULL) {
fs/nfs/read.c +1 -1
index 2b70bd2b934b..e7497b029d6c 100644
@@ -324,7 +324,7 @@ int nfs_read_add_folio(struct nfs_pageio_descriptor *pgio,
 
 	aligned_len = min_t(unsigned int, ALIGN(len, rsize), fsize);
 
-	new = nfs_page_create_from_folio(ctx, folio, 0, aligned_len);
+	new = nfs_page_create_from_folio(ctx, folio, false, 0, aligned_len);
 	if (IS_ERR(new)) {
 		error = PTR_ERR(new);
 		if (nfs_netfs_folio_unlock(folio))
fs/nfs/write.c +1 -1
index d2b03ceaeb4f..e565b811d8b7 100644
@@ -1088,7 +1088,7 @@ static struct nfs_page *nfs_setup_write_request(struct nfs_open_context *ctx,
 	req = nfs_try_to_update_request(folio, offset, bytes);
 	if (req != NULL)
 		goto out;
-	req = nfs_page_create_from_folio(ctx, folio, offset, bytes);
+	req = nfs_page_create_from_folio(ctx, folio, false, offset, bytes);
 	if (IS_ERR(req))
 		goto out;
 	nfs_inode_add_request(req);
include/linux/nfs_page.h +3 -0
index 4b9a35dbc062..fd7aafe7cb54 100644
@@ -38,6 +38,7 @@ enum {
 	PG_REMOVE,		/* page group sync bit in write path */
 	PG_CONTENDED1,		/* Is someone waiting for a lock? */
 	PG_CONTENDED2,		/* Is someone waiting for a lock? */
+	PG_PINNED,		/* page is pinned by GUP */
 };
 
 struct nfs_inode;
@@ -125,11 +126,13 @@ struct nfs_pageio_descriptor {
 
 extern struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 						  struct page *page,
+						  bool pinned,
 						  unsigned int pgbase,
 						  loff_t offset,
 						  unsigned int count);
 extern struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx,
 						   struct folio *folio,
+						   bool pinned,
 						   unsigned int offset,
 						   unsigned int count);
 extern	void nfs_release_request(struct nfs_page *);
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#2 lore raw

Re: [PATCH v3 1/5] nfs: make nfs_page pin-aware

It would be nice to avoid the overly long lines, but otherwise this looks good.

#3 lore raw

Re: [PATCH v3 1/5] nfs: make nfs_page pin-aware

Quoted 3 lines · Christoph Hellwig

It would be nice to avoid the overly long lines, but otherwise this looks good.

Ack, I'll trim the long lines and post a v4 quickly.

Thanks!
Praan

#4 lore raw

PATCH v3 2/5 nfs: Track number of pinned pages in nfs_page

In reply to Pranjal Shrivastava

Track the number of pinned pages in nfs_page to handle unpinning correctly, ensuring that only primary requests perform the final unpinning operation, preventing subrequests from incorrectly performing unpinning on behalf of their parent requests.

Add wb_nr_pinned to struct nfs_page to store the count of pinned pages owned by the request. Update request creation and cleanup helpers to initialize and use wb_nr_pinned for primary requests. Use the nfs_page_array_len() helper to calculate the number of pages spanned by a request's offset and length.

---
 fs/nfs/pagelist.c        | 9 +++++++--
 include/linux/nfs_page.h | 1 +
 2 files changed, 8 insertions(+), 2 deletions(-)
fs/nfs/pagelist.c +7 -2
index faa8bc1c6526..7d51e10fe97a 100644
@@ -455,6 +455,8 @@ struct nfs_page *nfs_page_create_from_page(struct nfs_open_context *ctx,
 			      offset_in_page(offset), count);
 	if (!IS_ERR(ret)) {
 		nfs_page_assign_page(ret, page, pinned);
+		if (pinned)
+			ret->wb_nr_pinned = 1;
 		nfs_page_group_init(ret, NULL);
 	}
 	nfs_put_lock_context(l_ctx);
@@ -487,6 +489,9 @@ struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx,
 	ret = nfs_page_create(l_ctx, offset, folio->index, offset, count);
 	if (!IS_ERR(ret)) {
 		nfs_page_assign_folio(ret, folio, pinned);
+		if (pinned)
+			ret->wb_nr_pinned = nfs_page_array_len(offset_in_page(offset),
+							      count);
 		nfs_page_group_init(ret, NULL);
 	}
 	nfs_put_lock_context(l_ctx);
@@ -565,14 +570,14 @@ static void nfs_clear_request(struct nfs_page *req)
 
 	if (folio != NULL) {
 		if (test_and_clear_bit(PG_PINNED, &req->wb_flags))
-			unpin_user_folio(folio, 1);
+			unpin_user_folio(folio, req->wb_nr_pinned);
 		else
 			folio_put(folio);
 		req->wb_folio = NULL;
 		clear_bit(PG_FOLIO, &req->wb_flags);
 	} else if (page != NULL) {
 		if (test_and_clear_bit(PG_PINNED, &req->wb_flags))
-			unpin_user_page(page);
+			unpin_user_pages(&page, req->wb_nr_pinned);
 		else
 			put_page(page);
 		req->wb_page = NULL;
include/linux/nfs_page.h +1 -0
index fd7aafe7cb54..080fa3e23580 100644
@@ -59,6 +59,7 @@ struct nfs_page {
 	struct nfs_page		*wb_this_page;  /* list of reqs for this page */
 	struct nfs_page		*wb_head;       /* head pointer for req list */
 	unsigned short		wb_nio;		/* Number of I/O attempts */
+	unsigned int		wb_nr_pinned;	/* Number of pinned pages */
 };
 
 struct nfs_pgio_mirror;
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#5 lore raw

Re: [PATCH v3 2/5] nfs: Track number of pinned pages in nfs_page

Nit: capitalization of the patch subjects is inconsistent. You probably one to stick to one version or another.

Otherwise this looks good to me.

#6 lore raw

Re: [PATCH v3 2/5] nfs: Track number of pinned pages in nfs_page

Quoted 2 lines · Christoph Hellwig

Nit: capitalization of the patch subjects is inconsistent. You probably one to stick to one version or another.

Ack. I'll stay with the lower-case convention as it seems to be common in the other commits. I'll fix that in the v4.

Quoted 3 lines

Otherwise this looks good to me.

Thanks!
Praan

#7 lore raw

PATCH v3 3/5 nfs: Introduce nfs_release_request_list helper

In reply to Pranjal Shrivastava

Introduce a centralized helper, nfs_release_request_list, to handle the bulk release of nfs_page requests from a list.

This serves as a preparatory step for two upcoming improvements:

1. Pin-Aware Cleanup: As we migrate to iov_iter_extract_* API, requests will hold pins (GUP) instead of standard references. The helper ensures that the correct unpinning logic gets applied consistently across all requests in a list.

2. Folio Support: In subsequent patches where nfs_page structures will cover multi-page folios, this helper provides a clean infrastructure to unlock these larger units of I/O in bulk during completion, similat to the pattern in bio_release_pages.

Additionally, refactor nfs_read_sync_pgio_error() to utilize this new helper.

---
 fs/nfs/direct.c          |  8 +-------
 fs/nfs/pagelist.c        | 18 ++++++++++++++++++
 include/linux/nfs_page.h |  4 ++--
 3 files changed, 21 insertions(+), 9 deletions(-)
fs/nfs/direct.c +1 -7
index 19792a38c924..96995736fac2 100644
@@ -314,13 +314,7 @@ static void nfs_direct_read_completion(struct nfs_pgio_header *hdr)
 
 static void nfs_read_sync_pgio_error(struct list_head *head, int error)
 {
-	struct nfs_page *req;
-
-	while (!list_empty(head)) {
-		req = nfs_list_entry(head->next);
-		nfs_list_remove_request(req);
-		nfs_release_request(req);
-	}
+	nfs_release_request_list(head);
 }
 
 static void nfs_direct_pgio_init(struct nfs_pgio_header *hdr)
fs/nfs/pagelist.c +18 -0
index 7d51e10fe97a..569bac4faff7 100644
@@ -622,6 +622,24 @@ void nfs_release_request(struct nfs_page *req)
 }
 EXPORT_SYMBOL_GPL(nfs_release_request);
 
+/*
+ * nfs_release_request_list - Release a list of NFS read/write requests
+ * @head: list of requests to release
+ *
+ * Removes each request from the list and drops it's refcount.
+ */
+void nfs_release_request_list(struct list_head *head)
+{
+	struct nfs_page *req;
+
+	while (!list_empty(head)) {
+		req = nfs_list_entry(head->next);
+		nfs_list_remove_request(req);
+		nfs_release_request(req);
+	}
+}
+EXPORT_SYMBOL_GPL(nfs_release_request_list);
+
 /*
  * nfs_generic_pg_test - determine if requests can be coalesced
  * @desc: pointer to descriptor
include/linux/nfs_page.h +2 -2
index 080fa3e23580..d23208ed3a33 100644
@@ -136,8 +136,8 @@ extern struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx,
 						   bool pinned,
 						   unsigned int offset,
 						   unsigned int count);
-extern	void nfs_release_request(struct nfs_page *);
-
+extern void nfs_release_request(struct nfs_page *req);
+extern void nfs_release_request_list(struct list_head *head);
 
 extern	void nfs_pageio_init(struct nfs_pageio_descriptor *desc,
 			     struct inode *inode,
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#8 lore raw

Re: [PATCH v3 3/5] nfs: Introduce nfs_release_request_list helper

Quoted 54 lines · Pranjal Shrivastava

Introduce a centralized helper, nfs_release_request_list, to handle the bulk release of nfs_page requests from a list.

This serves as a preparatory step for two upcoming improvements:

1. Pin-Aware Cleanup: As we migrate to iov_iter_extract_* API, requests will hold pins (GUP) instead of standard references. The helper ensures that the correct unpinning logic gets applied consistently across all requests in a list.

2. Folio Support: In subsequent patches where nfs_page structures will cover multi-page folios, this helper provides a clean infrastructure to unlock these larger units of I/O in bulk during completion, similat to the pattern in bio_release_pages.

Additionally, refactor nfs_read_sync_pgio_error() to utilize this new helper.

---
 fs/nfs/direct.c          |  8 +-------
 fs/nfs/pagelist.c        | 18 ++++++++++++++++++
 include/linux/nfs_page.h |  4 ++--
 3 files changed, 21 insertions(+), 9 deletions(-)
fs/nfs/direct.c +1 -7
index 19792a38c924..96995736fac2 100644
@@ -314,13 +314,7 @@ static void nfs_direct_read_completion(struct nfs_pgio_header *hdr)
 
 static void nfs_read_sync_pgio_error(struct list_head *head, int error)
 {
-	struct nfs_page *req;
-
-	while (!list_empty(head)) {
-		req = nfs_list_entry(head->next);
-		nfs_list_remove_request(req);
-		nfs_release_request(req);
-	}
+	nfs_release_request_list(head);
 }
 
 static void nfs_direct_pgio_init(struct nfs_pgio_header *hdr)
fs/nfs/pagelist.c +2 -0
index 7d51e10fe97a..569bac4faff7 100644
@@ -622,6 +622,24 @@ void nfs_release_request(struct nfs_page *req)
 }
 EXPORT_SYMBOL_GPL(nfs_release_request);
 
+/*
+ * nfs_release_request_list - Release a list of NFS read/write requests

fs/nfs/pagelist.c void nfs_release_request(struct nfs_page *req) · line 636

Quoted 10 lines
fs/nfs/pagelist.c +10 -0
+ * @head: list of requests to release
+ *
+ * Removes each request from the list and drops it's refcount.
+ */
+void nfs_release_request_list(struct list_head *head)
+{
+	struct nfs_page *req;
+
+	while (!list_empty(head)) {
+		req = nfs_list_entry(head->next);

req could/should be local here. Or you switch to list_first_or_null, which some folks prefer.

fs/nfs/pagelist.c void nfs_release_request(struct nfs_page *req) · line 638

Quoted 2 lines
fs/nfs/pagelist.c +2 -0
+extern void nfs_release_request(struct nfs_page *req);
+extern void nfs_release_request_list(struct list_head *head);

Please drop the superflous externs for new code.

Reviewed-by
#9 lore raw

Re: [PATCH v3 3/5] nfs: Introduce nfs_release_request_list helper

Quoted 12 lines · Christoph Hellwig
Quoted 10 lines · Pranjal Shrivastava

Introduce a centralized helper, nfs_release_request_list, to handle the bulk release of nfs_page requests from a list.

This serves as a preparatory step for two upcoming improvements:

1. Pin-Aware Cleanup: As we migrate to iov_iter_extract_* API, requests will hold pins (GUP) instead of standard references. The helper ensures that the correct unpinning logic gets applied consistently across all requests in a list.

Quoted 4 lines
Quoted 4 lines

2. Folio Support: In subsequent patches where nfs_page structures will cover multi-page folios, this helper provides a clean infrastructure to unlock these larger units of I/O in bulk during completion, similat to the pattern in bio_release_pages.

nit: similar -> similar

Quoted 54 lines
Quoted 40 lines

Additionally, refactor nfs_read_sync_pgio_error() to utilize this new helper.

---
 fs/nfs/direct.c          |  8 +-------
 fs/nfs/pagelist.c        | 18 ++++++++++++++++++
 include/linux/nfs_page.h |  4 ++--
 3 files changed, 21 insertions(+), 9 deletions(-)
fs/nfs/direct.c +1 -7
index 19792a38c924..96995736fac2 100644
@@ -314,13 +314,7 @@ static void nfs_direct_read_completion(struct nfs_pgio_header *hdr)
 
 static void nfs_read_sync_pgio_error(struct list_head *head, int error)
 {
-     struct nfs_page *req;
-
-     while (!list_empty(head)) {
-             req = nfs_list_entry(head->next);
-             nfs_list_remove_request(req);
-             nfs_release_request(req);
-     }
+     nfs_release_request_list(head);
 }
 
 static void nfs_direct_pgio_init(struct nfs_pgio_header *hdr)
fs/nfs/pagelist.c +2 -0
index 7d51e10fe97a..569bac4faff7 100644
@@ -622,6 +622,24 @@ void nfs_release_request(struct nfs_page *req)
 }
 EXPORT_SYMBOL_GPL(nfs_release_request);
 
+/*
+ * nfs_release_request_list - Release a list of NFS read/write requests

fs/nfs/pagelist.c void nfs_release_request(struct nfs_page *req) · line 636

Quoted 10 lines
fs/nfs/pagelist.c +10 -0
+ * @head: list of requests to release
+ *
+ * Removes each request from the list and drops it's refcount.
+ */
+void nfs_release_request_list(struct list_head *head)
+{
+     struct nfs_page *req;
+
+     while (!list_empty(head)) {
+             req = nfs_list_entry(head->next);

req could/should be local here. Or you switch to list_first_or_null, which some folks prefer.

Quoted 5 lines

fs/nfs/pagelist.c void nfs_release_request(struct nfs_page *req) · line 638

Quoted 2 lines
fs/nfs/pagelist.c +2 -0
+extern void nfs_release_request(struct nfs_page *req);
+extern void nfs_release_request_list(struct list_head *head);

Please drop the superflous externs for new code.

Small nit + fix christoph's comment,
else looks ok

#10 lore raw

Re: [PATCH v3 3/5] nfs: Introduce nfs_release_request_list helper

Quoted 13 lines · Shivaji Kant
Quoted 12 lines · Christoph Hellwig
Quoted 10 lines · Pranjal Shrivastava

Introduce a centralized helper, nfs_release_request_list, to handle the bulk release of nfs_page requests from a list.

This serves as a preparatory step for two upcoming improvements:

1. Pin-Aware Cleanup: As we migrate to iov_iter_extract_* API, requests will hold pins (GUP) instead of standard references. The helper ensures that the correct unpinning logic gets applied consistently across all requests in a list.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

2. Folio Support: In subsequent patches where nfs_page structures will cover multi-page folios, this helper provides a clean infrastructure to unlock these larger units of I/O in bulk during completion, similat to the pattern in bio_release_pages.

nit: similar -> similar

Ack. Fixing it,

Quoted 22 lines
Quoted 16 lines

fs/nfs/pagelist.c void nfs_release_request(struct nfs_page *req) · line 636

Quoted 12 lines
fs/nfs/pagelist.c +12 -0
+/*
+ * nfs_release_request_list - Release a list of NFS read/write requests
+ * @head: list of requests to release
+ *
+ * Removes each request from the list and drops it's refcount.
+ */
+void nfs_release_request_list(struct list_head *head)
+{
+     struct nfs_page *req;
+
+     while (!list_empty(head)) {
+             req = nfs_list_entry(head->next);

req could/should be local here. Or you switch to list_first_or_null, which some folks prefer.

Quoted 5 lines

include/linux/nfs_page.h extern struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx, · line 140

Quoted 2 lines
include/linux/nfs_page.h +2 -0
+extern void nfs_release_request(struct nfs_page *req);
+extern void nfs_release_request_list(struct list_head *head);

Please drop the superflous externs for new code.

Quoted 4 lines

Small nit + fix christoph's comment,
else looks ok

Thanks,
Praan

#11 lore raw

Re: [PATCH v3 3/5] nfs: Introduce nfs_release_request_list helper

In reply to Christoph Hellwig
Quoted 19 lines · Christoph Hellwig
Quoted 18 lines · Pranjal Shrivastava

Introduce a centralized helper, nfs_release_request_list, to handle the bulk release of nfs_page requests from a list.

This serves as a preparatory step for two upcoming improvements:

1. Pin-Aware Cleanup: As we migrate to iov_iter_extract_* API, requests will hold pins (GUP) instead of standard references. The helper ensures that the correct unpinning logic gets applied consistently across all requests in a list.

2. Folio Support: In subsequent patches where nfs_page structures will cover multi-page folios, this helper provides a clean infrastructure to unlock these larger units of I/O in bulk during completion, similat to the pattern in bio_release_pages.

Additionally, refactor nfs_read_sync_pgio_error() to utilize this new helper.

[...]

Quoted 7 lines
Quoted 7 lines
fs/nfs/pagelist.c +7 -0
+/*
+ * nfs_release_request_list - Release a list of NFS read/write requests
+ * @head: list of requests to release
+ *
+ * Removes each request from the list and drops it's refcount.
+ */
+void nfs_release_request_list(struct list_head *head)
Quoted 9 lines

fs/nfs/pagelist.c void nfs_release_request(struct nfs_page *req) · line 636

Quoted 5 lines
fs/nfs/pagelist.c +5 -0
+{
+	struct nfs_page *req;
+
+	while (!list_empty(head)) {
+		req = nfs_list_entry(head->next);

req could/should be local here. Or you switch to list_first_or_null, which some folks prefer.

Ack. Made it local

Quoted 5 lines

include/linux/nfs_page.h extern struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx, · line 140

Quoted 2 lines
include/linux/nfs_page.h +2 -0
+extern void nfs_release_request(struct nfs_page *req);
+extern void nfs_release_request_list(struct list_head *head);

Please drop the superflous externs for new code.

Ack. Dropping the externs.

Thanks,
Praan

#12 lore raw

PATCH v3 4/5 nfs: migrate direct I/O to iov_iter_extract_pages

In reply to Pranjal Shrivastava

Migrate the NFS Direct I/O path away from the legacy iov_iter_get_pages_alloc2() API to the modern iov_iter_extract_pages API. The transition aligns NFS with the modern VFS extraction model and serves as a preparatory step for supporting requirements such as page pinning via GUP for DMA.

The migration fixes a bug in the Direct I/O loop where pages were being unpinned immediately after request creation. With the new extraction model, pins are held until the I/O is complete. Manual release in the loop is correspondingly updated to only clean up failed pages.

---
 fs/nfs/direct.c | 26 +++++++++++++++-----------
 1 file changed, 15 insertions(+), 11 deletions(-)
fs/nfs/direct.c +15 -11
index 96995736fac2..b9ac0a67693c 100644
@@ -354,16 +354,17 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 	inode_dio_begin(inode);
 
 	while (iov_iter_count(iter)) {
-		struct page **pagevec;
+		struct page **pagevec = NULL;
 		size_t bytes;
 		size_t pgbase;
 		unsigned npages, i;
+		bool pinned = iov_iter_extract_will_pin(iter);
 
-		result = iov_iter_get_pages_alloc2(iter, &pagevec,
-						  rsize, &pgbase);
+		result = iov_iter_extract_pages(iter, &pagevec,
+						rsize, ~0U, 0, &pgbase);
 		if (result < 0)
 			break;
-	
+
 		bytes = result;
 		npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;
 		for (i = 0; i < npages; i++) {
@@ -371,7 +372,7 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 			/* XXX do we need to do the eof zeroing found in async_filler? */
 			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-							false, pgbase, pos,
+							pinned, pgbase, pos,
 							req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
@@ -387,7 +388,8 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 			requested_bytes += req_len;
 			pos += req_len;
 		}
-		nfs_direct_release_pages(pagevec, npages, false);
+		if (i < npages)
+			nfs_direct_release_pages(pagevec + i, npages - i, pinned);
 		kvfree(pagevec);
 		if (result < 0)
 			break;
@@ -891,13 +893,14 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 
 	NFS_I(inode)->write_io += iov_iter_count(iter);
 	while (iov_iter_count(iter)) {
-		struct page **pagevec;
+		struct page **pagevec = NULL;
 		size_t bytes;
 		size_t pgbase;
 		unsigned npages, i;
+		bool pinned = iov_iter_extract_will_pin(iter);
 
-		result = iov_iter_get_pages_alloc2(iter, &pagevec,
-						  wsize, &pgbase);
+		result = iov_iter_extract_pages(iter, &pagevec,
+						wsize, ~0U, 0, &pgbase);
 		if (result < 0)
 			break;
 
@@ -908,7 +911,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
 
 			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-							false, pgbase, pos,
+							pinned, pgbase, pos,
 							req_len);
 			if (IS_ERR(req)) {
 				result = PTR_ERR(req);
@@ -952,7 +955,8 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			desc.pg_error = 0;
 			defer = true;
 		}
-		nfs_direct_release_pages(pagevec, npages, false);
+		if (i < npages)
+			nfs_direct_release_pages(pagevec + i, npages - i, pinned);
 		kvfree(pagevec);
 		if (result < 0)
 			break;
Signature · 1 line
2.55.0.229.g6434b31f56-goog
#14 lore raw

PATCH v3 5/5 nfs: introduce nfs_direct_extract_pages helper

In reply to Pranjal Shrivastava

Introduce nfs_direct_extract_pages() in direct.c to centralize page extraction and request creation for the Direct I/O path. The helper manages extraction from the iters and builds a list of nfs_page requests

Refactor nfs_direct_read_schedule_iovec() and nfs_direct_write_schedule_iovec() to utilize the new helper, unifying the extraction logic on both paths.

---
 fs/nfs/direct.c | 127 ++++++++++++++++++++++++------------------------
 1 file changed, 64 insertions(+), 63 deletions(-)
fs/nfs/direct.c +64 -63
index b9ac0a67693c..d31e4720ffff 100644
@@ -178,6 +178,50 @@ static void nfs_direct_release_pages(struct page **pages, unsigned int npages,
 	}
 }
 
+static ssize_t nfs_direct_extract_pages(struct nfs_direct_req *dreq,
+					 struct iov_iter *iter,
+					 size_t size, loff_t *pos,
+					 struct list_head *list)
+{
+	bool pinned = iov_iter_extract_will_pin(iter);
+	struct page **pagevec = NULL;
+	ssize_t result, bytes = 0;
+	unsigned int npages, i;
+	size_t pgbase;
+
+	result = iov_iter_extract_pages(iter, &pagevec, size, ~0U, 0, &pgbase);
+	if (result <= 0)
+		return result;
+
+	npages = (result + pgbase + PAGE_SIZE - 1) >> PAGE_SHIFT;
+	for (i = 0; i < npages; i++) {
+		struct nfs_page *req;
+		unsigned int req_len = min_t(size_t, result - bytes, PAGE_SIZE - pgbase);
+
+		req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
+						pinned, pgbase, *pos,
+						req_len);
+		if (IS_ERR(req)) {
+			if (!bytes)
+				bytes = PTR_ERR(req);
+			break;
+		}
+
+		list_add_tail(&req->wb_list, list);
+		pgbase = 0;
+		bytes += req_len;
+		*pos += req_len;
+	}
+
+	if (i < npages) {
+		iov_iter_revert(iter, result - bytes);
+		nfs_direct_release_pages(pagevec + i, npages - i, pinned);
+	}
+
+	kvfree(pagevec);
+	return bytes;
+}
+
 void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo,
 			      struct nfs_direct_req *dreq)
 {
@@ -346,6 +390,7 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 	ssize_t result = -EINVAL;
 	size_t requested_bytes = 0;
 	size_t rsize = max_t(size_t, NFS_SERVER(inode)->rsize, PAGE_SIZE);
+	LIST_HEAD(nfs_page_list);
 
 	nfs_pageio_init_read(&desc, dreq->inode, false,
 			     &nfs_direct_read_completion_ops);
@@ -354,43 +399,23 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
 	inode_dio_begin(inode);
 
 	while (iov_iter_count(iter)) {
-		struct page **pagevec = NULL;
-		size_t bytes;
-		size_t pgbase;
-		unsigned npages, i;
-		bool pinned = iov_iter_extract_will_pin(iter);
-
-		result = iov_iter_extract_pages(iter, &pagevec,
-						rsize, ~0U, 0, &pgbase);
+		result = nfs_direct_extract_pages(dreq, iter, rsize, &pos, &nfs_page_list);
 		if (result < 0)
 			break;
 
-		bytes = result;
-		npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;
-		for (i = 0; i < npages; i++) {
-			struct nfs_page *req;
-			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
-			/* XXX do we need to do the eof zeroing found in async_filler? */
-			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-							pinned, pgbase, pos,
-							req_len);
-			if (IS_ERR(req)) {
-				result = PTR_ERR(req);
-				break;
-			}
+		while (!list_empty(&nfs_page_list)) {
+			struct nfs_page *req = nfs_list_entry(nfs_page_list.next);
+			size_t req_len = req->wb_bytes;
+
+			nfs_list_remove_request(req);
 			if (!nfs_pageio_add_request(&desc, req)) {
 				result = desc.pg_error;
 				nfs_release_request(req);
+				nfs_release_request_list(&nfs_page_list);
 				break;
 			}
-			pgbase = 0;
-			bytes -= req_len;
 			requested_bytes += req_len;
-			pos += req_len;
 		}
-		if (i < npages)
-			nfs_direct_release_pages(pagevec + i, npages - i, pinned);
-		kvfree(pagevec);
 		if (result < 0)
 			break;
 	}
@@ -881,6 +906,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 	ssize_t result = 0;
 	size_t requested_bytes = 0;
 	size_t wsize = max_t(size_t, NFS_SERVER(inode)->wsize, PAGE_SIZE);
+	LIST_HEAD(nfs_page_list);
 	bool defer = false;
 
 	trace_nfs_direct_write_schedule_iovec(dreq);
@@ -893,55 +919,32 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 
 	NFS_I(inode)->write_io += iov_iter_count(iter);
 	while (iov_iter_count(iter)) {
-		struct page **pagevec = NULL;
-		size_t bytes;
-		size_t pgbase;
-		unsigned npages, i;
-		bool pinned = iov_iter_extract_will_pin(iter);
-
-		result = iov_iter_extract_pages(iter, &pagevec,
-						wsize, ~0U, 0, &pgbase);
+		result = nfs_direct_extract_pages(dreq, iter, wsize, &pos, &nfs_page_list);
 		if (result < 0)
 			break;
 
-		bytes = result;
-		npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;
-		for (i = 0; i < npages; i++) {
-			struct nfs_page *req;
-			unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
-
-			req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-							pinned, pgbase, pos,
-							req_len);
-			if (IS_ERR(req)) {
-				result = PTR_ERR(req);
-				break;
-			}
-
-			if (desc.pg_error < 0) {
-				nfs_free_request(req);
-				result = desc.pg_error;
-				break;
-			}
-
-			pgbase = 0;
-			bytes -= req_len;
-			requested_bytes += req_len;
-			pos += req_len;
+		while (!list_empty(&nfs_page_list)) {
+			struct nfs_page *req = nfs_list_entry(nfs_page_list.next);
+			size_t req_len = req->wb_bytes;
 
+			nfs_list_remove_request(req);
 			if (defer) {
 				nfs_mark_request_commit(req, NULL, &cinfo, 0);
+				requested_bytes += req_len;
 				continue;
 			}
 
 			nfs_lock_request(req);
-			if (nfs_pageio_add_request(&desc, req))
+			if (nfs_pageio_add_request(&desc, req)) {
+				requested_bytes += req_len;
 				continue;
+			}
 
 			/* Exit on hard errors */
 			if (desc.pg_error < 0 && desc.pg_error != -EAGAIN) {
 				result = desc.pg_error;
 				nfs_unlock_and_release_request(req);
+				nfs_release_request_list(&nfs_page_list);
 				break;
 			}
 
@@ -952,12 +955,10 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 			spin_unlock(&dreq->lock);
 			nfs_unlock_request(req);
 			nfs_mark_request_commit(req, NULL, &cinfo, 0);
+			requested_bytes += req_len;
 			desc.pg_error = 0;
 			defer = true;
 		}
-		if (i < npages)
-			nfs_direct_release_pages(pagevec + i, npages - i, pinned);
-		kvfree(pagevec);
 		if (result < 0)
 			break;
 	}
Signature · 1 line
2.55.0.229.g6434b31f56-goog
Reviewed-by on v3 5/5
#16 lore raw

Re: [PATCH v3 5/5] nfs: introduce nfs_direct_extract_pages helper

In reply to Pranjal Shrivastava
Quoted 217 lines · Pranjal Shrivastava

Introduce nfs_direct_extract_pages() in direct.c to centralize page extraction and request creation for the Direct I/O path. The helper manages extraction from the iters and builds a list of nfs_page requests

Refactor nfs_direct_read_schedule_iovec() and nfs_direct_write_schedule_iovec() to utilize the new helper, unifying the extraction logic on both paths.

---
 fs/nfs/direct.c | 127 ++++++++++++++++++++++++------------------------
 1 file changed, 64 insertions(+), 63 deletions(-)
fs/nfs/direct.c +64 -64
index b9ac0a67693c..d31e4720ffff 100644
@@ -178,6 +178,50 @@ static void nfs_direct_release_pages(struct page **pages, unsigned int npages,
        }
 }
 
+static ssize_t nfs_direct_extract_pages(struct nfs_direct_req *dreq,
+                                        struct iov_iter *iter,
+                                        size_t size, loff_t *pos,
+                                        struct list_head *list)
+{
+       bool pinned = iov_iter_extract_will_pin(iter);
+       struct page **pagevec = NULL;
+       ssize_t result, bytes = 0;
+       unsigned int npages, i;
+       size_t pgbase;
+
+       result = iov_iter_extract_pages(iter, &pagevec, size, ~0U, 0, &pgbase);
+       if (result <= 0)
+               return result;
+
+       npages = (result + pgbase + PAGE_SIZE - 1) >> PAGE_SHIFT;
+       for (i = 0; i < npages; i++) {
+               struct nfs_page *req;
+               unsigned int req_len = min_t(size_t, result - bytes, PAGE_SIZE - pgbase);
+
+               req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
+                                               pinned, pgbase, *pos,
+                                               req_len);
+               if (IS_ERR(req)) {
+                       if (!bytes)
+                               bytes = PTR_ERR(req);
+                       break;
+               }
+
+               list_add_tail(&req->wb_list, list);
+               pgbase = 0;
+               bytes += req_len;
+               *pos += req_len;
+       }
+
+       if (i < npages) {
+               iov_iter_revert(iter, result - bytes);
+               nfs_direct_release_pages(pagevec + i, npages - i, pinned);
+       }
+
+       kvfree(pagevec);
+       return bytes;
+}
+
 void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo,
                              struct nfs_direct_req *dreq)
 {
@@ -346,6 +390,7 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
        ssize_t result = -EINVAL;
        size_t requested_bytes = 0;
        size_t rsize = max_t(size_t, NFS_SERVER(inode)->rsize, PAGE_SIZE);
+       LIST_HEAD(nfs_page_list);
 
        nfs_pageio_init_read(&desc, dreq->inode, false,
                             &nfs_direct_read_completion_ops);
@@ -354,43 +399,23 @@ static ssize_t nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
        inode_dio_begin(inode);
 
        while (iov_iter_count(iter)) {
-               struct page **pagevec = NULL;
-               size_t bytes;
-               size_t pgbase;
-               unsigned npages, i;
-               bool pinned = iov_iter_extract_will_pin(iter);
-
-               result = iov_iter_extract_pages(iter, &pagevec,
-                                               rsize, ~0U, 0, &pgbase);
+               result = nfs_direct_extract_pages(dreq, iter, rsize, &pos, &nfs_page_list);
                if (result < 0)
                        break;
 
-               bytes = result;
-               npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;
-               for (i = 0; i < npages; i++) {
-                       struct nfs_page *req;
-                       unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
-                       /* XXX do we need to do the eof zeroing found in async_filler? */
-                       req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-                                                       pinned, pgbase, pos,
-                                                       req_len);
-                       if (IS_ERR(req)) {
-                               result = PTR_ERR(req);
-                               break;
-                       }
+               while (!list_empty(&nfs_page_list)) {
+                       struct nfs_page *req = nfs_list_entry(nfs_page_list.next);
+                       size_t req_len = req->wb_bytes;
+
+                       nfs_list_remove_request(req);
                        if (!nfs_pageio_add_request(&desc, req)) {
                                result = desc.pg_error;
                                nfs_release_request(req);
+                               nfs_release_request_list(&nfs_page_list);
                                break;
                        }
-                       pgbase = 0;
-                       bytes -= req_len;
                        requested_bytes += req_len;
-                       pos += req_len;
                }
-               if (i < npages)
-                       nfs_direct_release_pages(pagevec + i, npages - i, pinned);
-               kvfree(pagevec);
                if (result < 0)
                        break;
        }
@@ -881,6 +906,7 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
        ssize_t result = 0;
        size_t requested_bytes = 0;
        size_t wsize = max_t(size_t, NFS_SERVER(inode)->wsize, PAGE_SIZE);
+       LIST_HEAD(nfs_page_list);
        bool defer = false;
 
        trace_nfs_direct_write_schedule_iovec(dreq);
@@ -893,55 +919,32 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 
        NFS_I(inode)->write_io += iov_iter_count(iter);
        while (iov_iter_count(iter)) {
-               struct page **pagevec = NULL;
-               size_t bytes;
-               size_t pgbase;
-               unsigned npages, i;
-               bool pinned = iov_iter_extract_will_pin(iter);
-
-               result = iov_iter_extract_pages(iter, &pagevec,
-                                               wsize, ~0U, 0, &pgbase);
+               result = nfs_direct_extract_pages(dreq, iter, wsize, &pos, &nfs_page_list);
                if (result < 0)
                        break;
 
-               bytes = result;
-               npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;
-               for (i = 0; i < npages; i++) {
-                       struct nfs_page *req;
-                       unsigned int req_len = min_t(size_t, bytes, PAGE_SIZE - pgbase);
-
-                       req = nfs_page_create_from_page(dreq->ctx, pagevec[i],
-                                                       pinned, pgbase, pos,
-                                                       req_len);
-                       if (IS_ERR(req)) {
-                               result = PTR_ERR(req);
-                               break;
-                       }
-
-                       if (desc.pg_error < 0) {
-                               nfs_free_request(req);
-                               result = desc.pg_error;
-                               break;
-                       }
-
-                       pgbase = 0;
-                       bytes -= req_len;
-                       requested_bytes += req_len;
-                       pos += req_len;
+               while (!list_empty(&nfs_page_list)) {
+                       struct nfs_page *req = nfs_list_entry(nfs_page_list.next);
+                       size_t req_len = req->wb_bytes;
 
+                       nfs_list_remove_request(req);
                        if (defer) {
                                nfs_mark_request_commit(req, NULL, &cinfo, 0);
+                               requested_bytes += req_len;
                                continue;
                        }
 
                        nfs_lock_request(req);
-                       if (nfs_pageio_add_request(&desc, req))
+                       if (nfs_pageio_add_request(&desc, req)) {
+                               requested_bytes += req_len;
                                continue;
+                       }
 
                        /* Exit on hard errors */
                        if (desc.pg_error < 0 && desc.pg_error != -EAGAIN) {
                                result = desc.pg_error;
                                nfs_unlock_and_release_request(req);
+                               nfs_release_request_list(&nfs_page_list);
                                break;
                        }
 
@@ -952,12 +955,10 @@ static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
                        spin_unlock(&dreq->lock);
                        nfs_unlock_request(req);
                        nfs_mark_request_commit(req, NULL, &cinfo, 0);
+                       requested_bytes += req_len;
                        desc.pg_error = 0;
                        defer = true;
                }
-               if (i < npages)
-                       nfs_direct_release_pages(pagevec + i, npages - i, pinned);
-               kvfree(pagevec);
                if (result < 0)
                        break;
        }
--
Quoted 2 lines

2.55.0.229.g6434b31f56-goog

Refactoring looks good. Consolidates extraction, error handling, and iov_iter_revert() into a single helper shared by both read and write paths.

maintainer
#17 lore raw

Re: [PATCH v3 5/5] nfs: introduce nfs_direct_extract_pages helper

In reply to Pranjal Shrivastava
Quoted 55 lines · Pranjal Shrivastava

Introduce nfs_direct_extract_pages() in direct.c to centralize page extraction and request creation for the Direct I/O path. The helper manages extraction from the iters and builds a list of nfs_page requests

Refactor nfs_direct_read_schedule_iovec() and nfs_direct_write_schedule_iovec() to utilize the new helper, unifying the extraction logic on both paths.

---
 fs/nfs/direct.c | 127 ++++++++++++++++++++++++---------------------- --

 1 file changed, 64 insertions(+), 63 deletions(-)
fs/nfs/direct.c +0 -0
index b9ac0a67693c..d31e4720ffff 100644
@@ -178,6 +178,50 @@ static void nfs_direct_release_pages(struct page
 **pages, unsigned int npages,
  	}
  }

fs/nfs/direct.c +12 -0
+static ssize_t nfs_direct_extract_pages(struct nfs_direct_req *dreq,
+					 struct iov_iter *iter,
+					 size_t size, loff_t *pos,
+					 struct list_head *list)
+{
+	bool pinned = iov_iter_extract_will_pin(iter);
+	struct page **pagevec = NULL;
+	ssize_t result, bytes = 0;
+	unsigned int npages, i;
+	size_t pgbase;
+
+	result = iov_iter_extract_pages(iter, &pagevec, size, ~0U,

0, &pgbase);

fs/nfs/direct.c +7 -0
+	if (result <= 0)
+		return result;
+
+	npages = (result + pgbase + PAGE_SIZE - 1) >> PAGE_SHIFT;
+	for (i = 0; i < npages; i++) {
+		struct nfs_page *req;
+		unsigned int req_len = min_t(size_t, result - bytes,

PAGE_SIZE - pgbase);

fs/nfs/direct.c +2 -0
+
+		req = nfs_page_create_from_page(dreq->ctx,

pagevec[i],

fs/nfs/direct.c +1 -0
+						pinned, pgbase,

*pos,

fs/nfs/direct.c +5 -0
+						req_len);
+		if (IS_ERR(req)) {
+			if (!bytes)
+				bytes = PTR_ERR(req);
+			break;

fs/nfs/direct.c static void nfs_direct_release_pages(struct page **pages, unsigned int npages, · line 217

Quoted 10 lines
fs/nfs/direct.c +10 -0
+		}
+
+		list_add_tail(&req->wb_list, list);
+		pgbase = 0;
+		bytes += req_len;
+		*pos += req_len;
+	}
+
+	if (i < npages) {
+		iov_iter_revert(iter, result - bytes);

Oopsable if bytes == PTR_ERR(req)

Quoted 2 lines
fs/nfs/direct.c +1 -0
+		nfs_direct_release_pages(pagevec + i, npages - i,

pinned);

See what happens above with swap over NFS (which sets pinned = false).

Quoted 185 lines
fs/nfs/direct.c +6 -0
+	}
+
+	kvfree(pagevec);
+	return bytes;
+}
+

 void nfs_init_cinfo_from_dreq(struct nfs_commit_info *cinfo, struct nfs_direct_req *dreq) {

fs/nfs/direct.c +1 -0
@@ -346,6 +390,7 @@ static ssize_t
 nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
  	ssize_t result = -EINVAL;
  	size_t requested_bytes = 0;
  	size_t rsize = max_t(size_t, NFS_SERVER(inode)->rsize,
 PAGE_SIZE);
+	LIST_HEAD(nfs_page_list);


  nfs_pageio_init_read(&desc, dreq->inode, false, &nfs_direct_read_completion_ops);

fs/nfs/direct.c +0 -0
@@ -354,43 +399,23 @@ static ssize_t
 nfs_direct_read_schedule_iovec(struct nfs_direct_req *dreq,
  	inode_dio_begin(inode);


  while (iov_iter_count(iter)) {

fs/nfs/direct.c +0 -8
-		struct page **pagevec = NULL;
-		size_t bytes;
-		size_t pgbase;
-		unsigned npages, i;
-		bool pinned = iov_iter_extract_will_pin(iter);
-
-		result = iov_iter_extract_pages(iter, &pagevec,
-						rsize, ~0U, 0,

&pgbase);

fs/nfs/direct.c +1 -0
+		result = nfs_direct_extract_pages(dreq, iter, rsize,

&pos, &nfs_page_list);
  if (result < 0)
  break;

fs/nfs/direct.c +0 -2
-		bytes = result;
-		npages = (result + pgbase + PAGE_SIZE - 1) /

PAGE_SIZE;

fs/nfs/direct.c +0 -3
-		for (i = 0; i < npages; i++) {
-			struct nfs_page *req;
-			unsigned int req_len = min_t(size_t, bytes,

PAGE_SIZE - pgbase);

fs/nfs/direct.c +0 -1
-			/* XXX do we need to do the eof zeroing

found in async_filler? */

fs/nfs/direct.c +0 -1
-			req = nfs_page_create_from_page(dreq->ctx,

pagevec[i],

fs/nfs/direct.c +0 -1
-							pinned,

pgbase, pos,

fs/nfs/direct.c +2 -5
-							req_len);
-			if (IS_ERR(req)) {
-				result = PTR_ERR(req);
-				break;
-			}
+		while (!list_empty(&nfs_page_list)) {
+			struct nfs_page *req =

nfs_list_entry(nfs_page_list.next);

fs/nfs/direct.c +3 -0
+			size_t req_len = req->wb_bytes;
+
+			nfs_list_remove_request(req);

  if (!nfs_pageio_add_request(&desc, req)) { result = desc.pg_error; nfs_release_request(req);

fs/nfs/direct.c +1 -0
+				nfs_release_request_list(&nfs_page_l

ist);
  break;
  }

fs/nfs/direct.c +0 -2
-			pgbase = 0;
-			bytes -= req_len;

  requested_bytes += req_len;

fs/nfs/direct.c +0 -1
-			pos += req_len;

  }

fs/nfs/direct.c +0 -4
-		if (i < npages)
-			nfs_direct_release_pages(pagevec + i, npages
- i, pinned);
-		kvfree(pagevec);

  if (result < 0)
  break;
  }

fs/nfs/direct.c +1 -0
@@ -881,6 +906,7 @@ static ssize_t
 nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
  	ssize_t result = 0;
  	size_t requested_bytes = 0;
  	size_t wsize = max_t(size_t, NFS_SERVER(inode)->wsize,
 PAGE_SIZE);
+	LIST_HEAD(nfs_page_list);
  	bool defer = false;

trace_nfs_direct_write_schedule_iovec(dreq);

fs/nfs/direct.c +0 -0
@@ -893,55 +919,32 @@ static ssize_t
 nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,


  NFS_I(inode)->write_io += iov_iter_count(iter); while (iov_iter_count(iter)) {

fs/nfs/direct.c +0 -8
-		struct page **pagevec = NULL;
-		size_t bytes;
-		size_t pgbase;
-		unsigned npages, i;
-		bool pinned = iov_iter_extract_will_pin(iter);
-
-		result = iov_iter_extract_pages(iter, &pagevec,
-						wsize, ~0U, 0,

&pgbase);

fs/nfs/direct.c +1 -0
+		result = nfs_direct_extract_pages(dreq, iter, wsize,

&pos, &nfs_page_list);
  if (result < 0)
  break;

fs/nfs/direct.c +0 -2
-		bytes = result;
-		npages = (result + pgbase + PAGE_SIZE - 1) /

PAGE_SIZE;

fs/nfs/direct.c +0 -3
-		for (i = 0; i < npages; i++) {
-			struct nfs_page *req;
-			unsigned int req_len = min_t(size_t, bytes,

PAGE_SIZE - pgbase);

fs/nfs/direct.c +0 -2
-
-			req = nfs_page_create_from_page(dreq->ctx,

pagevec[i],

fs/nfs/direct.c +0 -1
-							pinned,

pgbase, pos,

fs/nfs/direct.c +2 -16
-							req_len);
-			if (IS_ERR(req)) {
-				result = PTR_ERR(req);
-				break;
-			}
-
-			if (desc.pg_error < 0) {
-				nfs_free_request(req);
-				result = desc.pg_error;
-				break;
-			}
-
-			pgbase = 0;
-			bytes -= req_len;
-			requested_bytes += req_len;
-			pos += req_len;
+		while (!list_empty(&nfs_page_list)) {
+			struct nfs_page *req =

nfs_list_entry(nfs_page_list.next);

fs/nfs/direct.c +1 -0
+			size_t req_len = req->wb_bytes;

fs/nfs/direct.c +1 -0
+			nfs_list_remove_request(req);

  if (defer) { nfs_mark_request_commit(req, NULL, &cinfo, 0);

fs/nfs/direct.c +1 -0
+				requested_bytes += req_len;

  continue;
  }

  nfs_lock_request(req);

fs/nfs/direct.c +2 -1
-			if (nfs_pageio_add_request(&desc, req))
+			if (nfs_pageio_add_request(&desc, req)) {
+				requested_bytes += req_len;

  continue;

fs/nfs/direct.c +1 -0
+			}


  /* Exit on hard errors */
  if (desc.pg_error < 0 && desc.pg_error != - EAGAIN) {
  result = desc.pg_error; nfs_unlock_and_release_request(req);

fs/nfs/direct.c +1 -0
+				nfs_release_request_list(&nfs_page_l

ist);
  break;
  }

fs/nfs/direct.c +1 -4
@@ -952,12 +955,10 @@ static ssize_t
 nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
  			spin_unlock(&dreq->lock);
  			nfs_unlock_request(req);
  			nfs_mark_request_commit(req, NULL, &cinfo,
 0);
+			requested_bytes += req_len;
  			desc.pg_error = 0;
  			defer = true;
  		}
-		if (i < npages)
-			nfs_direct_release_pages(pagevec + i, npages
- i, pinned);
-		kvfree(pagevec);

  if (result < 0)
  break;
  }

Signature · 3 lines
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[email protected], [email protected]
#18 lore raw

Re: [PATCH v3 5/5] nfs: introduce nfs_direct_extract_pages helper

On Wed, Aug 05, 2026 at 01:12:28PM -0700, Trond Myklebust wrote:

Hi Trond,

Quoted 60 lines · Pranjal Shrivastava
Quoted 59 lines

Introduce nfs_direct_extract_pages() in direct.c to centralize page extraction and request creation for the Direct I/O path. The helper manages extraction from the iters and builds a list of nfs_page requests

Refactor nfs_direct_read_schedule_iovec() and nfs_direct_write_schedule_iovec() to utilize the new helper, unifying the extraction logic on both paths.

---
 fs/nfs/direct.c | 127 ++++++++++++++++++++++++---------------------- --

 1 file changed, 64 insertions(+), 63 deletions(-)
fs/nfs/direct.c +0 -0
index b9ac0a67693c..d31e4720ffff 100644
@@ -178,6 +178,50 @@ static void nfs_direct_release_pages(struct page
 **pages, unsigned int npages,
  	}
  }

fs/nfs/direct.c +12 -0
+static ssize_t nfs_direct_extract_pages(struct nfs_direct_req *dreq,
+					 struct iov_iter *iter,
+					 size_t size, loff_t *pos,
+					 struct list_head *list)
+{
+	bool pinned = iov_iter_extract_will_pin(iter);
+	struct page **pagevec = NULL;
+	ssize_t result, bytes = 0;
+	unsigned int npages, i;
+	size_t pgbase;
+
+	result = iov_iter_extract_pages(iter, &pagevec, size, ~0U,

0, &pgbase);

fs/nfs/direct.c +7 -0
+	if (result <= 0)
+		return result;
+
+	npages = (result + pgbase + PAGE_SIZE - 1) >> PAGE_SHIFT;
+	for (i = 0; i < npages; i++) {
+		struct nfs_page *req;
+		unsigned int req_len = min_t(size_t, result - bytes,

PAGE_SIZE - pgbase);

fs/nfs/direct.c +2 -0
+
+		req = nfs_page_create_from_page(dreq->ctx,

pagevec[i],

fs/nfs/direct.c +1 -0
+						pinned, pgbase,

*pos,

fs/nfs/direct.c +9 -0
+						req_len);
+		if (IS_ERR(req)) {
+			if (!bytes)
+				bytes = PTR_ERR(req);
+			break;
+		}
+
+		list_add_tail(&req->wb_list, list);
+		pgbase = 0;
Quoted 9 lines

fs/nfs/direct.c static void nfs_direct_release_pages(struct page **pages, unsigned int npages, · line 217

Quoted 6 lines
fs/nfs/direct.c +6 -0
+		bytes += req_len;
+		*pos += req_len;
+	}
+
+	if (i < npages) {
+		iov_iter_revert(iter, result - bytes);

Oopsable if bytes == PTR_ERR(req)

Ack. I'll catch errors in a separate variable something like:

...

    for (i = 0; i < npages; i++) {
 ...
        req = nfs_page_create_from_page(...);
        if (IS_ERR(req)) {
            err = PTR_ERR(req);
            break;
        }
 ...
        bytes += req_len;
    }

    if (i < npages) {
        iov_iter_revert(iter, result - bytes);
        nfs_direct_release_pages(pagevec + i, npages - i, pinned);
    }

    kvfree(pagevec);
    return bytes ? bytes : err;
Quoted 5 lines
Quoted 2 lines
fs/nfs/direct.c +1 -0
+		nfs_direct_release_pages(pagevec + i, npages - i,

pinned);

See what happens above with swap over NFS (which sets pinned = false).

I see what you mean, we never got a ref (after patch 4) but we'll call put_page() if pinned == false. I'll reduce the nfs_direct_release_pages to the following in patch 4 (when we start using the new extract API):

static void nfs_direct_release_pages(struct page **pages, unsigned int npages,
bool pinned)

{
	if (pinned) 
		unpin_user_pages(pages, npages);
}

Sounds good for v5?

Thanks,
Praan