Skip to contentSkip to conversation

Initial NFS client support for RWF_DONTCACHE

PATCH Superseded Trond Myklebust 2 participants 11 messages Aug 18, 2025

Subsystems (5)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Trond Myklebust

author maintainer
#0 lore raw

v3 cover Initial NFS client support for RWF_DONTCACHE

From: Trond Myklebust <[email protected]>

The following patch set attempts to add support for the RWF_DONTCACHE flag in preadv2() and pwritev2() on NFS filesystems.

The main issue is allowing support on 2 stage writes (i.e. unstable WRITE followed by a COMMIT) since those don't follow the current assumption that the 'dropbehind' flag can be fulfilled as soon as the writeback lock is dropped.

v2:
- Make use of the new iocb parameter for nfs_write_begin()
v3:
- Set/clear PG_DROPBEHIND on the head of the nfs_page group
- Simplify helper folio_end_dropbehind

Trond Myklebust (2):
filemap: Add a helper for filesystems implementing dropbehind
NFS: Enable the RWF_DONTCACHE flag for the NFS client

 fs/nfs/file.c            |  6 ++----
 fs/nfs/nfs4file.c        |  1 +
 fs/nfs/write.c           | 20 +++++++++++++++++---
 include/linux/nfs_page.h |  1 +
 include/linux/pagemap.h  |  1 +
 mm/filemap.c             | 11 +++++++++++
 6 files changed, 33 insertions(+), 7 deletions(-)
Signature · 1 line
2.50.1
author maintainer
#1 lore raw

PATCH v3 1/2 filemap: Add a helper for filesystems implementing dropbehind

From: Trond Myklebust <[email protected]>

Add a helper to allow filesystems to attempt to free the 'dropbehind' folio.

---
 include/linux/pagemap.h |  1 +
 mm/filemap.c            | 11 +++++++++++
 2 files changed, 12 insertions(+)
include/linux/pagemap.h +1 -0
index 12a12dae727d..201b7c6f6441 100644
@@ -1221,6 +1221,7 @@ void folio_wait_writeback(struct folio *folio);
 int folio_wait_writeback_killable(struct folio *folio);
 void end_page_writeback(struct page *page);
 void folio_end_writeback(struct folio *folio);
+void folio_end_dropbehind(struct folio *folio);
 void folio_wait_stable(struct folio *folio);
 void __folio_mark_dirty(struct folio *folio, struct address_space *, int warn);
 void folio_account_cleaned(struct folio *folio, struct bdi_writeback *wb);
mm/filemap.c +11 -0
index 751838ef05e5..e35dd0f90850 100644
@@ -1666,6 +1666,17 @@ void folio_end_writeback(struct folio *folio)
 }
 EXPORT_SYMBOL(folio_end_writeback);
 
+/*
+ * Helper for filesystems that want to implement dropbehind, and that
+ * need to keep the folio around after folio_end_writeback, e.g. due to
+ * the need to first commit NFS stable writes.
+ */
+void folio_end_dropbehind(struct folio *folio)
+{
+	filemap_end_dropbehind_write(folio);
+}
+EXPORT_SYMBOL(folio_end_dropbehind);
+
 /**
  * __folio_lock - Get a lock on the folio, assuming we need to sleep to get it.
  * @folio: The folio to lock
Signature · 1 line
2.50.1
maintainer
#2 lore raw

Re: [PATCH v3 1/2] filemap: Add a helper for filesystems implementing dropbehind

mm/filemap.c void folio_end_writeback(struct folio *folio) · line 1678

Quoted 5 lines · Trond Myklebust
mm/filemap.c +5 -0
+void folio_end_dropbehind(struct folio *folio)
+{
+	filemap_end_dropbehind_write(folio);
+}
+EXPORT_SYMBOL(folio_end_dropbehind);

Why not just export filemap_end_dropbehind_write()?

author maintainer
#3 lore raw

Re: [PATCH v3 1/2] filemap: Add a helper for filesystems implementing dropbehind

Quoted 8 lines · Matthew Wilcox

mm/filemap.c void folio_end_writeback(struct folio *folio) · line 1678

Quoted 5 lines · Trond Myklebust
mm/filemap.c +5 -0
+void folio_end_dropbehind(struct folio *folio)
+{
+	filemap_end_dropbehind_write(folio);
+}
+EXPORT_SYMBOL(folio_end_dropbehind);

Why not just export filemap_end_dropbehind_write()?

It seemed more appropriate to use the 'folio' prefix when exporting, so that it is symmetric with folio_end_writeback().

I'm perfectly fine with changing that if people disagree.

Signature · 3 lines
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[email protected], [email protected]
maintainer
#4 lore raw

Re: [PATCH v3 1/2] filemap: Add a helper for filesystems implementing dropbehind

Quoted 10 lines · Trond Myklebust
Quoted 8 lines · Matthew Wilcox

mm/filemap.c void folio_end_writeback(struct folio *folio) · line 1678

Quoted 5 lines · Trond Myklebust
mm/filemap.c +5 -0
+void folio_end_dropbehind(struct folio *folio)
+{
+	filemap_end_dropbehind_write(folio);
+}
+EXPORT_SYMBOL(folio_end_dropbehind);

Why not just export filemap_end_dropbehind_write()?

Quoted 4 lines

It seemed more appropriate to use the 'folio' prefix when exporting, so that it is symmetric with folio_end_writeback().

I'm perfectly fine with changing that if people disagree.

I don't mind renaming filemap_end_dropbehind_write() to folio_end_dropbehind() as a solution!

author maintainer
#5 lore raw

PATCH v3 2/2 NFS: Enable the RWF_DONTCACHE flag for the NFS client

In reply to Trond Myklebust

From: Trond Myklebust <[email protected]>

While the NFS readahead code has no problems using the generic code to manage the dropbehind behaviour enabled by RWF_DONTCACHE, the write code needs to deal with the fact that NFS writeback uses a 2 step process (UNSTABLE write followed by COMMIT).
This commit replaces the use of the folio dropbehind flag with a local NFS request flag that triggers the dropbehind behaviour once the data has been written to stable storage.

---
 fs/nfs/file.c            |  6 ++----
 fs/nfs/nfs4file.c        |  1 +
 fs/nfs/write.c           | 20 +++++++++++++++++---
 include/linux/nfs_page.h |  1 +
 4 files changed, 21 insertions(+), 7 deletions(-)
fs/nfs/file.c +2 -4
index 86e36c630f09..f771bd92d61e 100644
@@ -348,7 +348,6 @@ static int nfs_write_begin(const struct kiocb *iocb,
 			   loff_t pos, unsigned len, struct folio **foliop,
 			   void **fsdata)
 {
-	fgf_t fgp = FGP_WRITEBEGIN;
 	struct folio *folio;
 	struct file *file = iocb->ki_filp;
 	int once_thru = 0;
@@ -357,10 +356,8 @@ static int nfs_write_begin(const struct kiocb *iocb,
 	dfprintk(PAGECACHE, "NFS: write_begin(%pD2(%lu), %u@%lld)\n",
 		file, mapping->host->i_ino, len, (long long) pos);
 
-	fgp |= fgf_set_order(len);
 start:
-	folio = __filemap_get_folio(mapping, pos >> PAGE_SHIFT, fgp,
-				    mapping_gfp_mask(mapping));
+	folio = write_begin_get_folio(iocb, mapping, pos >> PAGE_SHIFT, len);
 	if (IS_ERR(folio))
 		return PTR_ERR(folio);
 	*foliop = folio;
@@ -915,5 +912,6 @@ const struct file_operations nfs_file_operations = {
 	.splice_write	= iter_file_splice_write,
 	.check_flags	= nfs_check_flags,
 	.setlease	= simple_nosetlease,
+	.fop_flags	= FOP_DONTCACHE,
 };
 EXPORT_SYMBOL_GPL(nfs_file_operations);
fs/nfs/nfs4file.c +1 -0
index 1d6b5f4230c9..654fda6f362c 100644
@@ -454,4 +454,5 @@ const struct file_operations nfs4_file_operations = {
 #else
 	.llseek		= nfs_file_llseek,
 #endif
+	.fop_flags	= FOP_DONTCACHE,
 };
fs/nfs/write.c +17 -3
index 8b7c04737967..f0fd5162154f 100644
@@ -349,8 +349,12 @@ static void nfs_folio_end_writeback(struct folio *folio)
 static void nfs_page_end_writeback(struct nfs_page *req)
 {
 	if (nfs_page_group_sync_on_bit(req, PG_WB_END)) {
+		struct folio *folio = nfs_page_to_folio(req);
+
+		if (folio_test_clear_dropbehind(folio))
+			set_bit(PG_DROPBEHIND, &req->wb_head->wb_flags);
 		nfs_unlock_request(req);
-		nfs_folio_end_writeback(nfs_page_to_folio(req));
+		nfs_folio_end_writeback(folio);
 	} else
 		nfs_unlock_request(req);
 }
@@ -787,8 +791,15 @@ static void nfs_inode_remove_request(struct nfs_page *req)
 			clear_bit(PG_MAPPED, &req->wb_head->wb_flags);
 		}
 		spin_unlock(&mapping->i_private_lock);
-	}
-	nfs_page_group_unlock(req);
+		nfs_page_group_unlock(req);
+
+		if (test_and_clear_bit(PG_DROPBEHIND,
+				       &req->wb_head->wb_flags)) {
+			folio_set_dropbehind(folio);
+			folio_end_dropbehind(folio);
+		}
+	} else
+		nfs_page_group_unlock(req);
 
 	if (test_and_clear_bit(PG_INODE_REF, &req->wb_flags)) {
 		atomic_long_dec(&nfsi->nrequests);
@@ -2068,6 +2079,7 @@ int nfs_wb_folio(struct inode *inode, struct folio *folio)
 		.range_start = range_start,
 		.range_end = range_start + len - 1,
 	};
+	bool dropbehind = folio_test_clear_dropbehind(folio);
 	int ret;
 
 	trace_nfs_writeback_folio(inode, range_start, len);
@@ -2088,6 +2100,8 @@ int nfs_wb_folio(struct inode *inode, struct folio *folio)
 			goto out_error;
 	}
 out_error:
+	if (dropbehind)
+		folio_set_dropbehind(folio);
 	trace_nfs_writeback_folio_done(inode, range_start, len, ret);
 	return ret;
 }
include/linux/nfs_page.h +1 -0
index 9aed39abc94b..32ab7a590098 100644
@@ -37,6 +37,7 @@ enum {
 	PG_REMOVE,		/* page group sync bit in write path */
 	PG_CONTENDED1,		/* Is someone waiting for a lock? */
 	PG_CONTENDED2,		/* Is someone waiting for a lock? */
+	PG_DROPBEHIND,		/* Implement RWF_DONTCACHE */
 };
 
 struct nfs_inode;
Signature · 1 line
2.50.1
maintainer
#6 lore raw

Re: [PATCH v3 2/2] NFS: Enable the RWF_DONTCACHE flag for the NFS client

Quoted 12 lines · Trond Myklebust
fs/nfs/write.c +4 -0
@@ -349,8 +349,12 @@ static void nfs_folio_end_writeback(struct folio *folio)
 static void nfs_page_end_writeback(struct nfs_page *req)
 {
 	if (nfs_page_group_sync_on_bit(req, PG_WB_END)) {
+		struct folio *folio = nfs_page_to_folio(req);
+
+		if (folio_test_clear_dropbehind(folio))
+			set_bit(PG_DROPBEHIND, &req->wb_head->wb_flags);
 		nfs_unlock_request(req);
@@ -787,8 +791,15 @@ static void nfs_inode_remove_request(struct nfs_page *req)
 			clear_bit(PG_MAPPED, &req->wb_head->wb_flags);
 		}

fs/nfs/write.c static void nfs_inode_remove_request(struct nfs_page *req) · line 800

Quoted 10 lines
fs/nfs/write.c +7 -2
 		spin_unlock(&mapping->i_private_lock);
-	}
-	nfs_page_group_unlock(req);
+		nfs_page_group_unlock(req);
+
+		if (test_and_clear_bit(PG_DROPBEHIND,
+				       &req->wb_head->wb_flags)) {
+			folio_set_dropbehind(folio);
+			folio_end_dropbehind(folio);
+		}

I don't think this technique is "safe". By clearing the flag early, the page cache can't see that a folio that was created by dropbehind has now been reused and should have its dropbehind flag cleared. So we might see pages dropped from the cache that really should not be.

author maintainer
#7 lore raw

Re: [PATCH v3 2/2] NFS: Enable the RWF_DONTCACHE flag for the NFS client

Quoted 25 lines · Matthew Wilcox
Quoted 24 lines · Trond Myklebust
fs/nfs/write.c +4 -0
@@ -349,8 +349,12 @@ static void nfs_folio_end_writeback(struct
 folio *folio)
  static void nfs_page_end_writeback(struct nfs_page *req)
  {
  	if (nfs_page_group_sync_on_bit(req, PG_WB_END)) {
+		struct folio *folio = nfs_page_to_folio(req);
+
+		if (folio_test_clear_dropbehind(folio))
+			set_bit(PG_DROPBEHIND, &req->wb_head-
 >wb_flags);
  		nfs_unlock_request(req);
@@ -787,8 +791,15 @@ static void nfs_inode_remove_request(struct
 nfs_page *req)

  clear_bit(PG_MAPPED, &req->wb_head-

Quoted 1 line

wb_flags);

  } spin_unlock(&mapping->i_private_lock);

fs/nfs/write.c +5 -2
-	}
-	nfs_page_group_unlock(req);
+		nfs_page_group_unlock(req);
+
+		if (test_and_clear_bit(PG_DROPBEHIND,
+				       &req->wb_head->wb_flags)) {
+			folio_set_dropbehind(folio);
Quoted 8 lines

fs/nfs/write.c static void nfs_inode_remove_request(struct nfs_page *req) · line 800

Quoted 2 lines
fs/nfs/write.c +2 -0
+			folio_end_dropbehind(folio);
+		}

I don't think this technique is "safe".  By clearing the flag early, the page cache can't see that a folio that was created by dropbehind has now been reused and should have its dropbehind flag cleared.  So we
might see pages dropped from the cache that really should not be.

The only alternative would be to add back in a helper in mm/filemap.c that does the normal folio_end_writeback() routine, but ignores the dropbehind flag. (folio_end_writeback_ignore_dropbehind()?)

Signature · 3 lines
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[email protected], [email protected]
maintainer
#8 lore raw

Re: [PATCH v3 2/2] NFS: Enable the RWF_DONTCACHE flag for the NFS client

Quoted 10 lines · Trond Myklebust
Quoted 5 lines · Matthew Wilcox

I don't think this technique is "safe".  By clearing the flag early, the page cache can't see that a folio that was created by dropbehind has now been reused and should have its dropbehind flag cleared.  So we
might see pages dropped from the cache that really should not be.

The only alternative would be to add back in a helper in mm/filemap.c that does the normal folio_end_writeback() routine, but ignores the dropbehind flag. (folio_end_writeback_ignore_dropbehind()?)

Can you remind me why we clear the writeback flag as soon as the WRITE completes instead of leaving it set until the COMMIT completes?

author maintainer
#9 lore raw

Re: [PATCH v3 2/2] NFS: Enable the RWF_DONTCACHE flag for the NFS client

Quoted 11 lines · Matthew Wilcox
Quoted 10 lines · Trond Myklebust
Quoted 8 lines · Matthew Wilcox

I don't think this technique is "safe".  By clearing the flag early,
the page cache can't see that a folio that was created by dropbehind
has now been reused and should have its dropbehind flag cleared. So
we
might see pages dropped from the cache that really should not be.

Quoted 8 lines
Quoted 4 lines

The only alternative would be to add back in a helper in mm/filemap.c
that does the normal folio_end_writeback() routine, but ignores the dropbehind flag. (folio_end_writeback_ignore_dropbehind()?)

Can you remind me why we clear the writeback flag as soon as the WRITE
completes instead of leaving it set until the COMMIT completes?

It's about reducing latency.

An unstable WRITE is typically a quick operation because it only requires the server to cache the data.

COMMIT requires persistence, and so it is typically slower. Furthermore, the intention of COMMIT is to also allow the batching of writeback on the server, so that disk wakeup and seeks are minimised. While that is probably much less of a concern with modern SSDs vs older hard drives, the NFS client design has to cater to both.

So by clearing the writeback flag after the WRITE, we allow operations that want to further modify a specific folio to proceed without having to wait for persistence of the entire batch.

More importantly, it also speeds up stat() calls, since we can retrieve the updated mtime+ctime values from the server as soon as the WRITEs are complete, without having to wait for COMMIT.

Signature · 3 lines
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[email protected], [email protected]
maintainer
#10 lore raw

Re: [PATCH v3 2/2] NFS: Enable the RWF_DONTCACHE flag for the NFS client

Quoted 15 lines · Trond Myklebust
Quoted 3 lines

Can you remind me why we clear the writeback flag as soon as the WRITE
completes instead of leaving it set until the COMMIT completes?

It's about reducing latency.

An unstable WRITE is typically a quick operation because it only requires the server to cache the data.

COMMIT requires persistence, and so it is typically slower. Furthermore, the intention of COMMIT is to also allow the batching of writeback on the server, so that disk wakeup and seeks are minimised. While that is probably much less of a concern with modern SSDs vs older hard drives, the NFS client design has to cater to both.

Quoted 3 lines

So by clearing the writeback flag after the WRITE, we allow operations that want to further modify a specific folio to proceed without having to wait for persistence of the entire batch.

Ah; NFS sets SB_I_STABLE_WRITES. Unfortunately, 6c17260ca4ae is a little light on details about why.