Skip to contentSkip to conversation

NFS buffered write cleanup

PATCH In review Christoph Hellwig 2 participants 19 messages Jul 1, 2024 – Jul 5, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi all,

this series cleans up the nfs_page handling in the buffer write path.

The first patch was already sent independently but hasn't been picked up and this included here again.

The last patch fixes a bug where a request could get incorrectly reused. It would require the flexfiles layout and odd I/O timings, and without a flexfiles server I can't actually hit it. I'd appreciate a careful review of that one.

The series is against Trond's testing branch.

Diffstat:

 fs/nfs/file.c                  |    6 
 fs/nfs/filelayout/filelayout.c |    1 
 fs/nfs/fscache.c               |    2 
 fs/nfs/internal.h              |    8 -
 fs/nfs/pagelist.c              |  117 ---------------
 fs/nfs/pnfs.h                  |   22 --
 fs/nfs/pnfs_nfs.c              |   47 ------
 fs/nfs/read.c                  |    2 
 fs/nfs/write.c                 |  316 ++++++++++++++++++-----------------------
 include/linux/nfs_page.h       |    7 
 10 files changed, 157 insertions(+), 371 deletions(-)
#1 lore raw

PATCH 1/7 nfs: remove dead code for the old swap over NFS implementation

Remove the code testing folio_test_swapcache either explicitly or implicitly in pagemap.h headers, as is now handled using the direct I/O path and not the buffered I/O path that these helpers are located in.

---
 fs/nfs/file.c                  |   6 +-
 fs/nfs/filelayout/filelayout.c |   1 -
 fs/nfs/fscache.c               |   2 +-
 fs/nfs/internal.h              |   8 +--
 fs/nfs/pagelist.c              |   2 +-
 fs/nfs/pnfs.h                  |  22 ------
 fs/nfs/pnfs_nfs.c              |  47 ------------
 fs/nfs/read.c                  |   2 +-
 fs/nfs/write.c                 | 128 +++++++--------------------------
 include/linux/nfs_page.h       |   4 +-
 10 files changed, 36 insertions(+), 186 deletions(-)
fs/nfs/file.c +3 -3
index 834e612262e62b..0e2f87120cb840 100644
@@ -427,7 +427,7 @@ static int nfs_write_end(struct file *file, struct address_space *mapping,
 static void nfs_invalidate_folio(struct folio *folio, size_t offset,
 				size_t length)
 {
-	struct inode *inode = folio_file_mapping(folio)->host;
+	struct inode *inode = folio->mapping->host;
 	dfprintk(PAGECACHE, "NFS: invalidate_folio(%lu, %zu, %zu)\n",
 		 folio->index, offset, length);
 
@@ -454,7 +454,7 @@ static bool nfs_release_folio(struct folio *folio, gfp_t gfp)
 		if ((current_gfp_context(gfp) & GFP_KERNEL) != GFP_KERNEL ||
 		    current_is_kswapd())
 			return false;
-		if (nfs_wb_folio(folio_file_mapping(folio)->host, folio) < 0)
+		if (nfs_wb_folio(folio->mapping->host, folio) < 0)
 			return false;
 	}
 	return nfs_fscache_release_folio(folio, gfp);
@@ -606,7 +606,7 @@ static vm_fault_t nfs_vm_page_mkwrite(struct vm_fault *vmf)
 			   TASK_KILLABLE|TASK_FREEZABLE_UNSAFE);
 
 	folio_lock(folio);
-	mapping = folio_file_mapping(folio);
+	mapping = folio->mapping;
 	if (mapping != inode->i_mapping)
 		goto out_unlock;
 
fs/nfs/filelayout/filelayout.c +0 -1
index 29d84dc66ca391..b6e9aeaf4ce289 100644
@@ -1110,7 +1110,6 @@ static const struct pnfs_commit_ops filelayout_commit_ops = {
 	.clear_request_commit	= pnfs_generic_clear_request_commit,
 	.scan_commit_lists	= pnfs_generic_scan_commit_lists,
 	.recover_commit_reqs	= pnfs_generic_recover_commit_reqs,
-	.search_commit_reqs	= pnfs_generic_search_commit_reqs,
 	.commit_pagelist	= filelayout_commit_pagelist,
 };
 
fs/nfs/fscache.c +1 -1
index ddc1ee0319554c..7202ce84d0eb03 100644
@@ -341,7 +341,7 @@ void nfs_netfs_initiate_read(struct nfs_pgio_header *hdr)
 
 int nfs_netfs_folio_unlock(struct folio *folio)
 {
-	struct inode *inode = folio_file_mapping(folio)->host;
+	struct inode *inode = folio->mapping->host;
 
 	/*
 	 * If fscache is enabled, netfs will unlock pages.
fs/nfs/internal.h +4 -4
index 9f0f4534744ba4..87ebc4608c316a 100644
@@ -785,7 +785,7 @@ static inline void nfs_folio_mark_unstable(struct folio *folio,
 					   struct nfs_commit_info *cinfo)
 {
 	if (folio && !cinfo->dreq) {
-		struct inode *inode = folio_file_mapping(folio)->host;
+		struct inode *inode = folio->mapping->host;
 		long nr = folio_nr_pages(folio);
 
 		/* This page is really still in write-back - just that the
@@ -803,7 +803,7 @@ static inline void nfs_folio_mark_unstable(struct folio *folio,
 static inline
 unsigned int nfs_page_length(struct page *page)
 {
-	loff_t i_size = i_size_read(page_file_mapping(page)->host);
+	loff_t i_size = i_size_read(page->mapping->host);
 
 	if (i_size > 0) {
 		pgoff_t index = page_index(page);
@@ -821,10 +821,10 @@ unsigned int nfs_page_length(struct page *page)
  */
 static inline size_t nfs_folio_length(struct folio *folio)
 {
-	loff_t i_size = i_size_read(folio_file_mapping(folio)->host);
+	loff_t i_size = i_size_read(folio->mapping->host);
 
 	if (i_size > 0) {
-		pgoff_t index = folio_index(folio) >> folio_order(folio);
+		pgoff_t index = folio->index >> folio_order(folio);
 		pgoff_t end_index = (i_size - 1) >> folio_shift(folio);
 		if (index < end_index)
 			return folio_size(folio);
fs/nfs/pagelist.c +1 -1
index 040b6b79c75e59..3b006bcbcc87a2 100644
@@ -569,7 +569,7 @@ struct nfs_page *nfs_page_create_from_folio(struct nfs_open_context *ctx,
 
 	if (IS_ERR(l_ctx))
 		return ERR_CAST(l_ctx);
-	ret = nfs_page_create(l_ctx, offset, folio_index(folio), offset, count);
+	ret = nfs_page_create(l_ctx, offset, folio->index, offset, count);
 	if (!IS_ERR(ret)) {
 		nfs_page_assign_folio(ret, folio);
 		nfs_page_group_init(ret, NULL);
fs/nfs/pnfs.h +0 -22
index bb5142b4e67a59..1fc40afcbf1f50 100644
@@ -199,8 +199,6 @@ struct pnfs_commit_ops {
 				  int max);
 	void (*recover_commit_reqs) (struct list_head *list,
 				     struct nfs_commit_info *cinfo);
-	struct nfs_page * (*search_commit_reqs)(struct nfs_commit_info *cinfo,
-						struct folio *folio);
 };
 
 struct pnfs_layout_hdr {
@@ -409,8 +407,6 @@ void pnfs_generic_prepare_to_resend_writes(struct nfs_commit_data *data);
 void pnfs_generic_rw_release(void *data);
 void pnfs_generic_recover_commit_reqs(struct list_head *dst,
 				      struct nfs_commit_info *cinfo);
-struct nfs_page *pnfs_generic_search_commit_reqs(struct nfs_commit_info *cinfo,
-						 struct folio *folio);
 int pnfs_generic_commit_pagelist(struct inode *inode,
 				 struct list_head *mds_pages,
 				 int how,
@@ -570,17 +566,6 @@ pnfs_recover_commit_reqs(struct list_head *head, struct nfs_commit_info *cinfo)
 		fl_cinfo->ops->recover_commit_reqs(head, cinfo);
 }
 
-static inline struct nfs_page *
-pnfs_search_commit_reqs(struct inode *inode, struct nfs_commit_info *cinfo,
-			struct folio *folio)
-{
-	struct pnfs_ds_commit_info *fl_cinfo = cinfo->ds;
-
-	if (!fl_cinfo->ops || !fl_cinfo->ops->search_commit_reqs)
-		return NULL;
-	return fl_cinfo->ops->search_commit_reqs(cinfo, folio);
-}
-
 /* Should the pNFS client commit and return the layout upon a setattr */
 static inline bool
 pnfs_ld_layoutret_on_setattr(struct inode *inode)
@@ -882,13 +867,6 @@ pnfs_recover_commit_reqs(struct list_head *head, struct nfs_commit_info *cinfo)
 {
 }
 
-static inline struct nfs_page *
-pnfs_search_commit_reqs(struct inode *inode, struct nfs_commit_info *cinfo,
-			struct folio *folio)
-{
-	return NULL;
-}
-
 static inline int pnfs_layoutcommit_inode(struct inode *inode, bool sync)
 {
 	return 0;
fs/nfs/pnfs_nfs.c +0 -47

53 lines not on this page · raw message

fs/nfs/read.c +1 -1
index 1b0e06c11983cb..036ede4875cab6 100644
@@ -289,7 +289,7 @@ int nfs_read_add_folio(struct nfs_pageio_descriptor *pgio,
 		       struct nfs_open_context *ctx,
 		       struct folio *folio)
 {
-	struct inode *inode = folio_file_mapping(folio)->host;
+	struct inode *inode = folio->mapping->host;
 	struct nfs_server *server = NFS_SERVER(inode);
 	size_t fsize = folio_size(folio);
 	unsigned int rsize = server->rsize;
fs/nfs/write.c +24 -104

272 lines not on this page · raw message

include/linux/nfs_page.h +2 -2
index 1c315f854ea801..7bc31df457ea58 100644
@@ -208,8 +208,8 @@ static inline struct inode *nfs_page_to_inode(const struct nfs_page *req)
 	struct folio *folio = nfs_page_to_folio(req);
 
 	if (folio == NULL)
-		return page_file_mapping(req->wb_page)->host;
-	return folio_file_mapping(folio)->host;
+		return req->wb_page->mapping->host;
+	return folio->mapping->host;
 }
 
 /**
Signature · 1 line
2.43.0
#3 lore raw

PATCH 2/7 nfs: remove nfs_folio_private_request

In reply to Christoph Hellwig

nfs_folio_private_request is a trivial wrapper around, which itself has fallen out of favor and has been replaced with plain ->private dereferences in recent folio conversions. Do the same for nfs.

---
 fs/nfs/write.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)
fs/nfs/write.c +2 -7
index 641bdddeaad331..5410c18a006937 100644
@@ -169,11 +169,6 @@ nfs_cancel_remove_inode(struct nfs_page *req, struct inode *inode)
 	return 0;
 }
 
-static struct nfs_page *nfs_folio_private_request(struct folio *folio)
-{
-	return folio_get_private(folio);
-}
-
 /**
  * nfs_folio_find_head_request - find head request associated with a folio
  * @folio: pointer to folio
@@ -190,7 +185,7 @@ static struct nfs_page *nfs_folio_find_head_request(struct folio *folio)
 	if (!folio_test_private(folio))
 		return NULL;
 	spin_lock(&mapping->i_private_lock);
-	req = nfs_folio_private_request(folio);
+	req = folio->private;
 	if (req) {
 		WARN_ON_ONCE(req->wb_head != req);
 		kref_get(&req->wb_kref);
@@ -220,7 +215,7 @@ static struct nfs_page *nfs_folio_find_and_lock_request(struct folio *folio)
 			return ERR_PTR(ret);
 		}
 		/* Ensure that nobody removed the request before we locked it */
-		if (head == nfs_folio_private_request(folio))
+		if (head == folio->private)
 			break;
 		nfs_unlock_and_release_request(head);
 	}
Signature · 1 line
2.43.0
#5 lore raw

PATCH 3/7 nfs: simplify nfs_folio_find_and_lock_request

In reply to Christoph Hellwig

nfs_folio_find_and_lock_request and the nfs_page_group_lock_head helper called by it spend quite some effort to deal with head vs subrequests. But given that only the head request can be stashed in the folio private data, non of that is required.

Fold the locking logic from nfs_page_group_lock_head into nfs_folio_find_and_lock_request and simplify the result based on the invariant that we always find the head request in the folio private data.

---
 fs/nfs/pagelist.c        | 19 -------------------
 fs/nfs/write.c           | 38 +++++++++++++++++++++-----------------
 include/linux/nfs_page.h |  1 -
 3 files changed, 21 insertions(+), 37 deletions(-)
fs/nfs/pagelist.c +0 -19
index 3b006bcbcc87a2..e48cc69a2361aa 100644
@@ -187,25 +187,6 @@ nfs_async_iocounter_wait(struct rpc_task *task, struct nfs_lock_context *l_ctx)
 }
 EXPORT_SYMBOL_GPL(nfs_async_iocounter_wait);
 
-/*
- * nfs_page_lock_head_request - page lock the head of the page group
- * @req: any member of the page group
- */
-struct nfs_page *
-nfs_page_group_lock_head(struct nfs_page *req)
-{
-	struct nfs_page *head = req->wb_head;
-
-	while (!nfs_lock_request(head)) {
-		int ret = nfs_wait_on_request(head);
-		if (ret < 0)
-			return ERR_PTR(ret);
-	}
-	if (head != req)
-		kref_get(&head->wb_kref);
-	return head;
-}
-
 /*
  * nfs_unroll_locks -  unlock all newly locked reqs and wait on @req
  * @head: head request of page group, must be holding head lock
fs/nfs/write.c +21 -17
index 5410c18a006937..58e5b78ff436b9 100644
@@ -197,28 +197,32 @@ static struct nfs_page *nfs_folio_find_head_request(struct folio *folio)
 static struct nfs_page *nfs_folio_find_and_lock_request(struct folio *folio)
 {
 	struct inode *inode = folio->mapping->host;
-	struct nfs_page *req, *head;
+	struct nfs_page *head;
 	int ret;
 
-	for (;;) {
-		req = nfs_folio_find_head_request(folio);
-		if (!req)
-			return req;
-		head = nfs_page_group_lock_head(req);
-		if (head != req)
-			nfs_release_request(req);
-		if (IS_ERR(head))
-			return head;
-		ret = nfs_cancel_remove_inode(head, inode);
-		if (ret < 0) {
-			nfs_unlock_and_release_request(head);
+retry:
+	head = nfs_folio_find_head_request(folio);
+	if (!head)
+		return NULL;
+
+	while (!nfs_lock_request(head)) {
+		ret = nfs_wait_on_request(head);
+		if (ret < 0)
 			return ERR_PTR(ret);
-		}
-		/* Ensure that nobody removed the request before we locked it */
-		if (head == folio->private)
-			break;
+	}
+
+	/* Ensure that nobody removed the request before we locked it */
+	if (head != folio->private) {
 		nfs_unlock_and_release_request(head);
+		goto retry;
 	}
+
+	ret = nfs_cancel_remove_inode(head, inode);
+	if (ret < 0) {
+		nfs_unlock_and_release_request(head);
+		return ERR_PTR(ret);
+	}
+
 	return head;
 }
 
include/linux/nfs_page.h +0 -1
index 7bc31df457ea58..e799d93626f117 100644
@@ -155,7 +155,6 @@ extern size_t nfs_generic_pg_test(struct nfs_pageio_descriptor *desc,
 extern  int nfs_wait_on_request(struct nfs_page *);
 extern	void nfs_unlock_request(struct nfs_page *req);
 extern	void nfs_unlock_and_release_request(struct nfs_page *);
-extern	struct nfs_page *nfs_page_group_lock_head(struct nfs_page *req);
 extern	int nfs_page_group_lock_subrequests(struct nfs_page *head);
 extern void nfs_join_page_group(struct nfs_page *head,
 				struct nfs_commit_info *cinfo,
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH 3/7] nfs: simplify nfs_folio_find_and_lock_request

Quoted 62 lines · Christoph Hellwig

nfs_folio_find_and_lock_request and the nfs_page_group_lock_head helper called by it spend quite some effort to deal with head vs subrequests. But given that only the head request can be stashed in the folio private data, non of that is required.

Fold the locking logic from nfs_page_group_lock_head into nfs_folio_find_and_lock_request and simplify the result based on the invariant that we always find the head request in the folio private data.

---
  fs/nfs/pagelist.c        | 19 -------------------
  fs/nfs/write.c           | 38 +++++++++++++++++++++-----------------
  include/linux/nfs_page.h |  1 -
  3 files changed, 21 insertions(+), 37 deletions(-)
fs/nfs/pagelist.c +0 -19
index 3b006bcbcc87a2..e48cc69a2361aa 100644
@@ -187,25 +187,6 @@ nfs_async_iocounter_wait(struct rpc_task *task, struct nfs_lock_context *l_ctx)
  }
  EXPORT_SYMBOL_GPL(nfs_async_iocounter_wait);
 
-/*
- * nfs_page_lock_head_request - page lock the head of the page group
- * @req: any member of the page group
- */
-struct nfs_page *
-nfs_page_group_lock_head(struct nfs_page *req)
-{
-	struct nfs_page *head = req->wb_head;
-
-	while (!nfs_lock_request(head)) {
-		int ret = nfs_wait_on_request(head);
-		if (ret < 0)
-			return ERR_PTR(ret);
-	}
-	if (head != req)
-		kref_get(&head->wb_kref);
-	return head;
-}
-
  /*
   * nfs_unroll_locks -  unlock all newly locked reqs and wait on @req
   * @head: head request of page group, must be holding head lock
fs/nfs/write.c +1 -5
index 5410c18a006937..58e5b78ff436b9 100644
@@ -197,28 +197,32 @@ static struct nfs_page *nfs_folio_find_head_request(struct folio *folio)
  static struct nfs_page *nfs_folio_find_and_lock_request(struct folio *folio)
  {
  	struct inode *inode = folio->mapping->host;
-	struct nfs_page *req, *head;
+	struct nfs_page *head;
  	int ret;
 
-	for (;;) {
-		req = nfs_folio_find_head_request(folio);
-		if (!req)
-			return req;

fs/nfs/write.c static struct nfs_page *nfs_folio_find_head_request(struct folio *folio) · line 204

Quoted 10 lines
fs/nfs/write.c +2 -8
-		head = nfs_page_group_lock_head(req);
-		if (head != req)
-			nfs_release_request(req);
-		if (IS_ERR(head))
-			return head;
-		ret = nfs_cancel_remove_inode(head, inode);
-		if (ret < 0) {
-			nfs_unlock_and_release_request(head);
+retry:
+	head = nfs_folio_find_head_request(folio);

Is nfs_folio_find_head_reques() an appropriate name here? it doesn't search and find afaict (aside from internally the pnfs commits search). Anyways, Reviewed-by: Sagi Grimberg <[email protected]>

#7 lore raw

Re: [PATCH 3/7] nfs: simplify nfs_folio_find_and_lock_request

Quoted 2 lines · Sagi Grimberg

Is nfs_folio_find_head_reques() an appropriate name here? it doesn't search and find afaict (aside from internally the pnfs commits search). Anyways,

I could use a rename. If everyone agrees I can do that incrementally.

#8 lore raw

PATCH 4/7 nfs: fold nfs_folio_find_and_lock_request into nfs_lock_and_join_requests

In reply to Christoph Hellwig

Fold nfs_folio_find_and_lock_request into the only caller to prepare for changes to this code.

---
 fs/nfs/write.c | 68 ++++++++++++++++++++------------------------------
 1 file changed, 27 insertions(+), 41 deletions(-)
fs/nfs/write.c +27 -41
index 58e5b78ff436b9..2b139493168d87 100644
@@ -194,38 +194,6 @@ static struct nfs_page *nfs_folio_find_head_request(struct folio *folio)
 	return req;
 }
 
-static struct nfs_page *nfs_folio_find_and_lock_request(struct folio *folio)
-{
-	struct inode *inode = folio->mapping->host;
-	struct nfs_page *head;
-	int ret;
-
-retry:
-	head = nfs_folio_find_head_request(folio);
-	if (!head)
-		return NULL;
-
-	while (!nfs_lock_request(head)) {
-		ret = nfs_wait_on_request(head);
-		if (ret < 0)
-			return ERR_PTR(ret);
-	}
-
-	/* Ensure that nobody removed the request before we locked it */
-	if (head != folio->private) {
-		nfs_unlock_and_release_request(head);
-		goto retry;
-	}
-
-	ret = nfs_cancel_remove_inode(head, inode);
-	if (ret < 0) {
-		nfs_unlock_and_release_request(head);
-		return ERR_PTR(ret);
-	}
-
-	return head;
-}
-
 /* Adjust the file length if we're writing beyond the end */
 static void nfs_grow_file(struct folio *folio, unsigned int offset,
 			  unsigned int count)
@@ -530,26 +498,44 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
 	struct nfs_commit_info cinfo;
 	int ret;
 
-	nfs_init_cinfo_from_inode(&cinfo, inode);
 	/*
 	 * A reference is taken only on the head request which acts as a
 	 * reference to the whole page group - the group will not be destroyed
 	 * until the head reference is released.
 	 */
-	head = nfs_folio_find_and_lock_request(folio);
-	if (IS_ERR_OR_NULL(head))
-		return head;
+retry:
+	head = nfs_folio_find_head_request(folio);
+	if (!head)
+		return NULL;
 
-	/* lock each request in the page group */
-	ret = nfs_page_group_lock_subrequests(head);
-	if (ret < 0) {
+	while (!nfs_lock_request(head)) {
+		ret = nfs_wait_on_request(head);
+		if (ret < 0)
+			return ERR_PTR(ret);
+	}
+
+	/* Ensure that nobody removed the request before we locked it */
+	if (head != folio->private) {
 		nfs_unlock_and_release_request(head);
-		return ERR_PTR(ret);
+		goto retry;
 	}
 
-	nfs_join_page_group(head, &cinfo, inode);
+	ret = nfs_cancel_remove_inode(head, inode);
+	if (ret < 0)
+		goto out_unlock;
 
+	/* lock each request in the page group */
+	ret = nfs_page_group_lock_subrequests(head);
+	if (ret < 0)
+		goto out_unlock;
+
+	nfs_init_cinfo_from_inode(&cinfo, inode);
+	nfs_join_page_group(head, &cinfo, inode);
 	return head;
+
+out_unlock:
+	nfs_unlock_and_release_request(head);
+	return ERR_PTR(ret);
 }
 
 static void nfs_write_error(struct nfs_page *req, int error)
Signature · 1 line
2.43.0
Reviewed-by on v1 4/7
#9 lore raw

Re: [PATCH 4/7] nfs: fold nfs_folio_find_and_lock_request into nfs_lock_and_join_requests

Quoted 86 lines · Christoph Hellwig

Fold nfs_folio_find_and_lock_request into the only caller to prepare for changes to this code.

---
  fs/nfs/write.c | 68 ++++++++++++++++++++------------------------------
  1 file changed, 27 insertions(+), 41 deletions(-)
fs/nfs/write.c +13 -41
index 58e5b78ff436b9..2b139493168d87 100644
@@ -194,38 +194,6 @@ static struct nfs_page *nfs_folio_find_head_request(struct folio *folio)
  	return req;
  }
 
-static struct nfs_page *nfs_folio_find_and_lock_request(struct folio *folio)
-{
-	struct inode *inode = folio->mapping->host;
-	struct nfs_page *head;
-	int ret;
-
-retry:
-	head = nfs_folio_find_head_request(folio);
-	if (!head)
-		return NULL;
-
-	while (!nfs_lock_request(head)) {
-		ret = nfs_wait_on_request(head);
-		if (ret < 0)
-			return ERR_PTR(ret);
-	}
-
-	/* Ensure that nobody removed the request before we locked it */
-	if (head != folio->private) {
-		nfs_unlock_and_release_request(head);
-		goto retry;
-	}
-
-	ret = nfs_cancel_remove_inode(head, inode);
-	if (ret < 0) {
-		nfs_unlock_and_release_request(head);
-		return ERR_PTR(ret);
-	}
-
-	return head;
-}
-
  /* Adjust the file length if we're writing beyond the end */
  static void nfs_grow_file(struct folio *folio, unsigned int offset,
  			  unsigned int count)
@@ -530,26 +498,44 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
  	struct nfs_commit_info cinfo;
  	int ret;
 
-	nfs_init_cinfo_from_inode(&cinfo, inode);
  	/*
  	 * A reference is taken only on the head request which acts as a
  	 * reference to the whole page group - the group will not be destroyed
  	 * until the head reference is released.
  	 */
-	head = nfs_folio_find_and_lock_request(folio);
-	if (IS_ERR_OR_NULL(head))
-		return head;
+retry:
+	head = nfs_folio_find_head_request(folio);
+	if (!head)
+		return NULL;
 
-	/* lock each request in the page group */
-	ret = nfs_page_group_lock_subrequests(head);
-	if (ret < 0) {
+	while (!nfs_lock_request(head)) {
+		ret = nfs_wait_on_request(head);
+		if (ret < 0)
+			return ERR_PTR(ret);
+	}
+
+	/* Ensure that nobody removed the request before we locked it */
+	if (head != folio->private) {
  		nfs_unlock_and_release_request(head);
-		return ERR_PTR(ret);
+		goto retry;
  	}
 
-	nfs_join_page_group(head, &cinfo, inode);

fs/nfs/write.c static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio) · line 532

Quoted 10 lines
fs/nfs/write.c +9 -0
+	ret = nfs_cancel_remove_inode(head, inode);
+	if (ret < 0)
+		goto out_unlock;
 
+	/* lock each request in the page group */
+	ret = nfs_page_group_lock_subrequests(head);
+	if (ret < 0)
+		goto out_unlock;
+
+	nfs_init_cinfo_from_inode(&cinfo, inode);

Any reason why nfs_init_cinfo_from_inode() changed location?

Otherwise,

#10 lore raw

Re: [PATCH 4/7] nfs: fold nfs_folio_find_and_lock_request into nfs_lock_and_join_requests

Quoted 3 lines · Sagi Grimberg

fs/nfs/write.c static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio) · line 532

Quoted 1 line
fs/nfs/write.c +1 -0
+	nfs_init_cinfo_from_inode(&cinfo, inode);

Any reason why nfs_init_cinfo_from_inode() changed location?

I moved it early enough toward where it is being used to better reason about the code. But as a nice little advantage that also means it is only called when actually needed now.

#11 lore raw

PATCH 5/7 nfs: fold nfs_page_group_lock_subrequests into nfs_lock_and_join_requests

In reply to Christoph Hellwig

Fold nfs_page_group_lock_subrequests into nfs_lock_and_join_requests to prepare for future changes to this code, and move the helpers to write.c as well.

---
 fs/nfs/pagelist.c        | 77 ----------------------------------------
 fs/nfs/write.c           | 67 ++++++++++++++++++++++++++++++++--
 include/linux/nfs_page.h |  1 -
 3 files changed, 64 insertions(+), 81 deletions(-)
fs/nfs/pagelist.c +0 -77
index e48cc69a2361aa..fa7971072900b3 100644
@@ -187,83 +187,6 @@ nfs_async_iocounter_wait(struct rpc_task *task, struct nfs_lock_context *l_ctx)
 }
 EXPORT_SYMBOL_GPL(nfs_async_iocounter_wait);
 
-/*
- * nfs_unroll_locks -  unlock all newly locked reqs and wait on @req
- * @head: head request of page group, must be holding head lock
- * @req: request that couldn't lock and needs to wait on the req bit lock
- *
- * This is a helper function for nfs_lock_and_join_requests
- * returns 0 on success, < 0 on error.
- */
-static void
-nfs_unroll_locks(struct nfs_page *head, struct nfs_page *req)
-{
-	struct nfs_page *tmp;
-
-	/* relinquish all the locks successfully grabbed this run */
-	for (tmp = head->wb_this_page ; tmp != req; tmp = tmp->wb_this_page) {
-		if (!kref_read(&tmp->wb_kref))
-			continue;
-		nfs_unlock_and_release_request(tmp);
-	}
-}
-
-/*
- * nfs_page_group_lock_subreq -  try to lock a subrequest
- * @head: head request of page group
- * @subreq: request to lock
- *
- * This is a helper function for nfs_lock_and_join_requests which
- * must be called with the head request and page group both locked.
- * On error, it returns with the page group unlocked.
- */
-static int
-nfs_page_group_lock_subreq(struct nfs_page *head, struct nfs_page *subreq)
-{
-	int ret;
-
-	if (!kref_get_unless_zero(&subreq->wb_kref))
-		return 0;
-	while (!nfs_lock_request(subreq)) {
-		nfs_page_group_unlock(head);
-		ret = nfs_wait_on_request(subreq);
-		if (!ret)
-			ret = nfs_page_group_lock(head);
-		if (ret < 0) {
-			nfs_unroll_locks(head, subreq);
-			nfs_release_request(subreq);
-			return ret;
-		}
-	}
-	return 0;
-}
-
-/*
- * nfs_page_group_lock_subrequests -  try to lock the subrequests
- * @head: head request of page group
- *
- * This is a helper function for nfs_lock_and_join_requests which
- * must be called with the head request locked.
- */
-int nfs_page_group_lock_subrequests(struct nfs_page *head)
-{
-	struct nfs_page *subreq;
-	int ret;
-
-	ret = nfs_page_group_lock(head);
-	if (ret < 0)
-		return ret;
-	/* lock each request in the page group */
-	for (subreq = head->wb_this_page; subreq != head;
-			subreq = subreq->wb_this_page) {
-		ret = nfs_page_group_lock_subreq(head, subreq);
-		if (ret < 0)
-			return ret;
-	}
-	nfs_page_group_unlock(head);
-	return 0;
-}
-
 /*
  * nfs_page_set_headlock - set the request PG_HEADLOCK
  * @req: request that is to be locked
fs/nfs/write.c +64 -3

88 lines not on this page · raw message

include/linux/nfs_page.h +0 -1
index e799d93626f117..63eed97a18ade9 100644
@@ -155,7 +155,6 @@ extern size_t nfs_generic_pg_test(struct nfs_pageio_descriptor *desc,
 extern  int nfs_wait_on_request(struct nfs_page *);
 extern	void nfs_unlock_request(struct nfs_page *req);
 extern	void nfs_unlock_and_release_request(struct nfs_page *);
-extern	int nfs_page_group_lock_subrequests(struct nfs_page *head);
 extern void nfs_join_page_group(struct nfs_page *head,
 				struct nfs_commit_info *cinfo,
 				struct inode *inode);
Signature · 1 line
2.43.0
#13 lore raw

PATCH 6/7 nfs: move nfs_wait_on_request to write.c

In reply to Christoph Hellwig

nfs_wait_on_request is now only used in write.c. Move it there and mark it static.

---
 fs/nfs/pagelist.c        | 19 -------------------
 fs/nfs/write.c           | 17 +++++++++++++++++
 include/linux/nfs_page.h |  1 -
 3 files changed, 17 insertions(+), 20 deletions(-)
fs/nfs/pagelist.c +0 -19
index fa7971072900b3..04124f22666530 100644
@@ -598,25 +598,6 @@ void nfs_release_request(struct nfs_page *req)
 }
 EXPORT_SYMBOL_GPL(nfs_release_request);
 
-/**
- * nfs_wait_on_request - Wait for a request to complete.
- * @req: request to wait upon.
- *
- * Interruptible by fatal signals only.
- * The user is responsible for holding a count on the request.
- */
-int
-nfs_wait_on_request(struct nfs_page *req)
-{
-	if (!test_bit(PG_BUSY, &req->wb_flags))
-		return 0;
-	set_bit(PG_CONTENDED2, &req->wb_flags);
-	smp_mb__after_atomic();
-	return wait_on_bit_io(&req->wb_flags, PG_BUSY,
-			      TASK_UNINTERRUPTIBLE);
-}
-EXPORT_SYMBOL_GPL(nfs_wait_on_request);
-
 /*
  * nfs_generic_pg_test - determine if requests can be coalesced
  * @desc: pointer to descriptor
fs/nfs/write.c +17 -0
index 3ba298ebb0be14..2c089444303982 100644
@@ -476,6 +476,23 @@ void nfs_join_page_group(struct nfs_page *head, struct nfs_commit_info *cinfo,
 	nfs_destroy_unlinked_subrequests(destroy_list, head, inode);
 }
 
+/**
+ * nfs_wait_on_request - Wait for a request to complete.
+ * @req: request to wait upon.
+ *
+ * Interruptible by fatal signals only.
+ * The user is responsible for holding a count on the request.
+ */
+static int nfs_wait_on_request(struct nfs_page *req)
+{
+	if (!test_bit(PG_BUSY, &req->wb_flags))
+		return 0;
+	set_bit(PG_CONTENDED2, &req->wb_flags);
+	smp_mb__after_atomic();
+	return wait_on_bit_io(&req->wb_flags, PG_BUSY,
+			      TASK_UNINTERRUPTIBLE);
+}
+
 /*
  * nfs_unroll_locks -  unlock all newly locked reqs and wait on @req
  * @head: head request of page group, must be holding head lock
include/linux/nfs_page.h +0 -1
index 63eed97a18ade9..169b4ae30ff479 100644
@@ -152,7 +152,6 @@ extern	void nfs_pageio_cond_complete(struct nfs_pageio_descriptor *, pgoff_t);
 extern size_t nfs_generic_pg_test(struct nfs_pageio_descriptor *desc,
 				struct nfs_page *prev,
 				struct nfs_page *req);
-extern  int nfs_wait_on_request(struct nfs_page *);
 extern	void nfs_unlock_request(struct nfs_page *req);
 extern	void nfs_unlock_and_release_request(struct nfs_page *);
 extern void nfs_join_page_group(struct nfs_page *head,
Signature · 1 line
2.43.0
#15 lore raw

PATCH 7/7 nfs: don't reuse partially completed requests in nfs_lock_and_join_requests

In reply to Christoph Hellwig

When NFS requests are split into sub-requests, nfs_inode_remove_request calls nfs_page_group_sync_on_bit to set PG_REMOVE on this sub-request and only completes the head requests once PG_REMOVE is set on all requests. This means that when nfs_lock_and_join_requests sees a PG_REMOVE bit, I/O
on the request is in progress and has partially completed. If such a
request is returned to nfs_try_to_update_request, it could be extended with the newly dirtied region and I/O for the combined range will be re-scheduled, leading to extra I/O.

Change the logic to instead restart the search for a request when any PG_REMOVE bit is set, as the completion handler will remove the request as soon as it can take the page group lock. This not only avoid extending the I/O but also does the right thing for the callers that want to cancel or flush the request.

---
 fs/nfs/write.c | 49 ++++++++++++++++++++-----------------------------
 1 file changed, 20 insertions(+), 29 deletions(-)
fs/nfs/write.c +20 -29
index 2c089444303982..4dffdc5aadb2e2 100644
@@ -144,31 +144,6 @@ static void nfs_io_completion_put(struct nfs_io_completion *ioc)
 		kref_put(&ioc->refcount, nfs_io_completion_release);
 }
 
-static void
-nfs_page_set_inode_ref(struct nfs_page *req, struct inode *inode)
-{
-	if (!test_and_set_bit(PG_INODE_REF, &req->wb_flags)) {
-		kref_get(&req->wb_kref);
-		atomic_long_inc(&NFS_I(inode)->nrequests);
-	}
-}
-
-static int
-nfs_cancel_remove_inode(struct nfs_page *req, struct inode *inode)
-{
-	int ret;
-
-	if (!test_bit(PG_REMOVE, &req->wb_flags))
-		return 0;
-	ret = nfs_page_group_lock(req);
-	if (ret)
-		return ret;
-	if (test_and_clear_bit(PG_REMOVE, &req->wb_flags))
-		nfs_page_set_inode_ref(req, inode);
-	nfs_page_group_unlock(req);
-	return 0;
-}
-
 /**
  * nfs_folio_find_head_request - find head request associated with a folio
  * @folio: pointer to folio
@@ -564,6 +539,7 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
 	struct inode *inode = folio->mapping->host;
 	struct nfs_page *head, *subreq;
 	struct nfs_commit_info cinfo;
+	bool removed;
 	int ret;
 
 	/*
@@ -588,18 +564,18 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
 		goto retry;
 	}
 
-	ret = nfs_cancel_remove_inode(head, inode);
-	if (ret < 0)
-		goto out_unlock;
-
 	ret = nfs_page_group_lock(head);
 	if (ret < 0)
 		goto out_unlock;
 
+	removed = test_bit(PG_REMOVE, &head->wb_flags);
+
 	/* lock each request in the page group */
 	for (subreq = head->wb_this_page;
 	     subreq != head;
 	     subreq = subreq->wb_this_page) {
+		if (test_bit(PG_REMOVE, &subreq->wb_flags))
+			removed = true;
 		ret = nfs_page_group_lock_subreq(head, subreq);
 		if (ret < 0)
 			goto out_unlock;
@@ -607,6 +583,21 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
 
 	nfs_page_group_unlock(head);
 
+	/*
+	 * If PG_REMOVE is set on any request, I/O on that request has
+	 * completed, but some requests were still under I/O at the time
+	 * we locked the head request.
+	 *
+	 * In that case the above wait for all requests means that all I/O
+	 * has now finished, and we can restart from a clean slate.  Let the
+	 * old requests go away and start from scratch instead.
+	 */
+	if (removed) {
+		nfs_unroll_locks(head, head);
+		nfs_unlock_and_release_request(head);
+		goto retry;
+	}
+
 	nfs_init_cinfo_from_inode(&cinfo, inode);
 	nfs_join_page_group(head, &cinfo, inode);
 	return head;
Signature · 1 line
2.43.0
#16 lore raw

Re: [PATCH 7/7] nfs: don't reuse partially completed requests in nfs_lock_and_join_requests

Quoted 8 lines · Christoph Hellwig

When NFS requests are split into sub-requests, nfs_inode_remove_request calls nfs_page_group_sync_on_bit to set PG_REMOVE on this sub-request and only completes the head requests once PG_REMOVE is set on all requests. This means that when nfs_lock_and_join_requests sees a PG_REMOVE bit, I/O
on the request is in progress and has partially completed. If such a
request is returned to nfs_try_to_update_request, it could be extended with the newly dirtied region and I/O for the combined range will be re-scheduled, leading to extra I/O.

Probably worth noting in the change log that large folios makes this potentially much worse?

Quoted 87 lines

Change the logic to instead restart the search for a request when any PG_REMOVE bit is set, as the completion handler will remove the request as soon as it can take the page group lock. This not only avoid extending the I/O but also does the right thing for the callers that want to cancel or flush the request.

---
  fs/nfs/write.c | 49 ++++++++++++++++++++-----------------------------
  1 file changed, 20 insertions(+), 29 deletions(-)
fs/nfs/write.c +9 -29
index 2c089444303982..4dffdc5aadb2e2 100644
@@ -144,31 +144,6 @@ static void nfs_io_completion_put(struct nfs_io_completion *ioc)
  		kref_put(&ioc->refcount, nfs_io_completion_release);
  }
 
-static void
-nfs_page_set_inode_ref(struct nfs_page *req, struct inode *inode)
-{
-	if (!test_and_set_bit(PG_INODE_REF, &req->wb_flags)) {
-		kref_get(&req->wb_kref);
-		atomic_long_inc(&NFS_I(inode)->nrequests);
-	}
-}
-
-static int
-nfs_cancel_remove_inode(struct nfs_page *req, struct inode *inode)
-{
-	int ret;
-
-	if (!test_bit(PG_REMOVE, &req->wb_flags))
-		return 0;
-	ret = nfs_page_group_lock(req);
-	if (ret)
-		return ret;
-	if (test_and_clear_bit(PG_REMOVE, &req->wb_flags))
-		nfs_page_set_inode_ref(req, inode);
-	nfs_page_group_unlock(req);
-	return 0;
-}
-
  /**
   * nfs_folio_find_head_request - find head request associated with a folio
   * @folio: pointer to folio
@@ -564,6 +539,7 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
  	struct inode *inode = folio->mapping->host;
  	struct nfs_page *head, *subreq;
  	struct nfs_commit_info cinfo;
+	bool removed;
  	int ret;
 
  	/*
@@ -588,18 +564,18 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
  		goto retry;
  	}
 
-	ret = nfs_cancel_remove_inode(head, inode);
-	if (ret < 0)
-		goto out_unlock;
-
  	ret = nfs_page_group_lock(head);
  	if (ret < 0)
  		goto out_unlock;
 
+	removed = test_bit(PG_REMOVE, &head->wb_flags);
+
  	/* lock each request in the page group */
  	for (subreq = head->wb_this_page;
  	     subreq != head;
  	     subreq = subreq->wb_this_page) {
+		if (test_bit(PG_REMOVE, &subreq->wb_flags))
+			removed = true;
  		ret = nfs_page_group_lock_subreq(head, subreq);
  		if (ret < 0)
  			goto out_unlock;
@@ -607,6 +583,21 @@ static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio)
 
  	nfs_page_group_unlock(head);
 
+	/*
+	 * If PG_REMOVE is set on any request, I/O on that request has
+	 * completed, but some requests were still under I/O at the time
+	 * we locked the head request.

fs/nfs/write.c static struct nfs_page *nfs_lock_and_join_requests(struct folio *folio) · line 599

Quoted 10 lines
fs/nfs/write.c +10 -0
+	 *
+	 * In that case the above wait for all requests means that all I/O
+	 * has now finished, and we can restart from a clean slate.  Let the
+	 * old requests go away and start from scratch instead.
+	 */
+	if (removed) {
+		nfs_unroll_locks(head, head);
+		nfs_unlock_and_release_request(head);
+		goto retry;
+	}

Don't you need a waitqueue or something to avoid excessive restarts until the IO completes?

#17 lore raw

Re: [PATCH 7/7] nfs: don't reuse partially completed requests in nfs_lock_and_join_requests

Quoted 10 lines · Sagi Grimberg
Quoted 8 lines · Christoph Hellwig

When NFS requests are split into sub-requests, nfs_inode_remove_request calls nfs_page_group_sync_on_bit to set PG_REMOVE on this sub-request and only completes the head requests once PG_REMOVE is set on all requests. This means that when nfs_lock_and_join_requests sees a PG_REMOVE bit, I/O
on the request is in progress and has partially completed. If such a
request is returned to nfs_try_to_update_request, it could be extended with the newly dirtied region and I/O for the combined range will be re-scheduled, leading to extra I/O.

Quoted 3 lines

Probably worth noting in the change log that large folios makes this potentially much
worse?

That assumes large folios actually create more subrequest. One big reason to create subrequests is flexfiles mirroring, which of course doesn't change with large folio. The other is that if ->pg_test doesn't allow the nfs_page to cover everything, which is roughly bound by a page array allocation and for PNFS the layout segment size, and the chance for that to fail could very slightly increase.

#18 lore raw
In reply to Christoph Hellwig
Quoted 7 lines · Christoph Hellwig

Hi all,

this series cleans up the nfs_page handling in the buffer write path.

The first patch was already sent independently but hasn't been picked up and this included here again.

Quoted 6 lines

The last patch fixes a bug where a request could get incorrectly reused. It would require the flexfiles layout and odd I/O timings, and without a flexfiles server I can't actually hit it. I'd appreciate a careful review of that one.

The series is against Trond's testing branch.

Looks like Anna is doing this merge window despite the earlier testing branch from Trond. That already has the first patch applied, and the rest applies fine to it as well.