Initial NFS client support for RWF_DONTCACHE
Subsystems (5)
Trond Myklebust
maintainerv4 cover Initial NFS client support for RWF_DONTCACHE
From: Trond Myklebust <[email protected]>
The following patch set attempts to add support for the RWF_DONTCACHE flag in preadv2() and pwritev2() on NFS filesystems.
The main issue is allowing support on 2 stage writes (i.e. unstable WRITE followed by a COMMIT) since those don't follow the current assumption that the 'dropbehind' flag can be fulfilled as soon as the writeback lock is dropped.
v2:
- Make use of the new iocb parameter for nfs_write_begin()
v3:
- Set/clear PG_DROPBEHIND on the head of the nfs_page group
- Simplify helper folio_end_dropbehind
v4:
- Replace filemap_end_dropbehind_write() with folio_end_dropbehind()
- Add a helper to replace folio_end_writeback with an equivalent that does not attempt to interpret the dropbehind flag
- Keep the folio dropbehind flag set until the NFS client is ready to call folio_end_dropbehind.
- Don't try to do a read-modify-write in nfs_write_begin() if the folio has the dropbehind flag set.
Trond Myklebust (3):
filemap: Add a helper for filesystems implementing dropbehind
filemap: Add a version of folio_end_writeback that ignores dropbehind
NFS: Enable use of the RWF_DONTCACHE flag on the NFS client
fs/nfs/file.c | 9 +++++---- fs/nfs/nfs4file.c | 1 + fs/nfs/write.c | 4 +++- include/linux/pagemap.h | 2 ++ mm/filemap.c | 34 ++++++++++++++++++++++++++-------- 5 files changed, 37 insertions(+), 13 deletions(-)
Signature · 1 line
2.50.1
Trond Myklebust
maintainerPATCH v4 1/3 filemap: Add a helper for filesystems implementing dropbehind
From: Trond Myklebust <[email protected]>
Add a helper to allow filesystems to attempt to free the 'dropbehind' folio.
--- include/linux/pagemap.h | 1 + mm/filemap.c | 5 +++-- 2 files changed, 4 insertions(+), 2 deletions(-)
include/linux/pagemap.h +1 -0
@@ -1221,6 +1221,7 @@ void folio_wait_writeback(struct folio *folio);int folio_wait_writeback_killable (struct folio *folio);void end_page_writeback (struct page *page);void folio_end_writeback (struct folio *folio); +void folio_end_dropbehind (struct folio *folio);void folio_wait_stable (struct folio *folio);void __folio_mark_dirty (struct folio *folio,struct address_space *,int warn);void folio_account_cleaned (struct folio *folio,struct bdi_writeback *wb);
mm/filemap.c +3 -2
@@ -1608,7 +1608,7 @@ static void filemap_end_dropbehind(struct folio *folio)* completes. Do that now. If we fail, it's likely because of a big folio - * just reset dropbehind for that case and latter completions should invalidate. */ -static void filemap_end_dropbehind_write (struct folio *folio) +void folio_end_dropbehind (struct folio *folio) {if (!folio_test_dropbehind (folio))return ;
@@ -1625,6 +1625,7 @@ static void filemap_end_dropbehind_write(struct folio *folio)folio_unlock (folio); } } +EXPORT_SYMBOL (folio_end_dropbehind);/** * folio_end_writeback - End writeback against a folio.
@@ -1660,7 +1661,7 @@ void folio_end_writeback(struct folio *folio)if (__folio_end_writeback (folio))folio_wake_bit (folio, PG_writeback); -filemap_end_dropbehind_write (folio); +folio_end_dropbehind (folio);acct_reclaim_writeback (folio);folio_put (folio); }
Signature · 1 line
2.50.1
Re: [PATCH v4 1/3] filemap: Add a helper for filesystems implementing dropbehind
Quoted 1 line · Trond Myklebust
+EXPORT_SYMBOL(folio_end_dropbehind);
This really should be a EXPORT_SYMBOL_GPL like for any other linux-only functionality. Same for the next patch.
Trond Myklebust
maintainerPATCH v4 2/3 filemap: Add a version of folio_end_writeback that ignores dropbehind
In reply to Trond MyklebustFrom: Trond Myklebust <[email protected]>
Filesystems such as NFS may need to defer dropbehind until after their 2-stage writes are done. This adds a helper folio_end_writeback_no_dropbehind() that allows them to release the writeback flag without immediately dropping the folio.
--- include/linux/pagemap.h | 1 + mm/filemap.c | 29 +++++++++++++++++++++++------ 2 files changed, 24 insertions(+), 6 deletions(-)
include/linux/pagemap.h +1 -0
@@ -1221,6 +1221,7 @@ void folio_wait_writeback(struct folio *folio);int folio_wait_writeback_killable (struct folio *folio);void end_page_writeback (struct page *page);void folio_end_writeback (struct folio *folio); +void folio_end_writeback_no_dropbehind (struct folio *folio);void folio_end_dropbehind (struct folio *folio);void folio_wait_stable (struct folio *folio);void __folio_mark_dirty (struct folio *folio,struct address_space *,int warn);
mm/filemap.c +23 -6
@@ -1628,14 +1628,15 @@ void folio_end_dropbehind(struct folio *folio)EXPORT_SYMBOL (folio_end_dropbehind);/** -* folio_end_writeback - End writeback against a folio. +* folio_end_writeback_no_dropbehind - End writeback against a folio. * @folio: The folio. * * The folio must actually be under writeback. +* This call is intended for filesystems that need to defer dropbehind. * * Context: May be called from process or interrupt context. */ -void folio_end_writeback (struct folio *folio) +void folio_end_writeback_no_dropbehind (struct folio *folio) {VM_BUG_ON_FOLIO (!folio_test_writeback (folio), folio);
@@ -1651,6 +1652,25 @@ void folio_end_writeback(struct folio *folio)folio_rotate_reclaimable (folio); } +if (__folio_end_writeback (folio)) +folio_wake_bit (folio, PG_writeback); + +acct_reclaim_writeback (folio); +} +EXPORT_SYMBOL (folio_end_writeback_no_dropbehind); + +/** +* folio_end_writeback - End writeback against a folio. +* @folio: The folio. +* +* The folio must actually be under writeback. +* +* Context: May be called from process or interrupt context. +*/ +void folio_end_writeback (struct folio *folio) +{ +VM_BUG_ON_FOLIO (!folio_test_writeback (folio), folio); +/* * Writeback does not hold a folio reference of its own, relying * on truncation to wait for the clearing of PG_writeback.
@@ -1658,11 +1678,8 @@ void folio_end_writeback(struct folio *folio)* reused before the folio_wake_bit(). */ folio_get (folio); -if (__folio_end_writeback (folio)) -folio_wake_bit (folio, PG_writeback); - +folio_end_writeback_no_dropbehind (folio);folio_end_dropbehind (folio); -acct_reclaim_writeback (folio);folio_put (folio); }EXPORT_SYMBOL (folio_end_writeback);
Signature · 1 line
2.50.1
Trond Myklebust
maintainerPATCH v4 3/3 NFS: Enable use of the RWF_DONTCACHE flag on the NFS client
In reply to Trond MyklebustFrom: Trond Myklebust <[email protected]>
The NFS client needs to defer dropbehind until after any writes to the folio have been persisted on the server. Since this may be a 2 step process, use folio_end_writeback_no_dropbehind() to allow release of the writeback flag, and then call folio_end_dropbehind() once the COMMIT is done.
--- fs/nfs/file.c | 9 +++++---- fs/nfs/nfs4file.c | 1 + fs/nfs/write.c | 4 +++- 3 files changed, 9 insertions(+), 5 deletions(-)
fs/nfs/file.c +5 -4
@@ -329,6 +329,8 @@ static bool nfs_want_read_modify_write(struct file *file, struct folio *folio,if (pnfs_ld_read_whole_page (file_inode (file)))return true; +if (folio_test_dropbehind (folio)) +return false;/* Open for reading too? */ if (file->f_mode &FMODE_READ )return true;
@@ -348,7 +350,6 @@ static int nfs_write_begin(const struct kiocb *iocb, loff_t pos, unsigned len, struct folio **foliop, void **fsdata) { - fgf_t fgp = FGP_WRITEBEGIN; struct folio *folio;struct file *file = iocb->ki_filp;int once_thru =0 ;
@@ -357,10 +358,8 @@ static int nfs_write_begin(const struct kiocb *iocb,dfprintk (PAGECACHE ,"NFS: write_begin(%pD2(%lu), %u@%lld)\n" , file, mapping->host->i_ino, len, (long long ) pos); - fgp |=fgf_set_order (len); start: - folio =__filemap_get_folio (mapping, pos >>PAGE_SHIFT , fgp, -mapping_gfp_mask (mapping)); + folio =write_begin_get_folio (iocb, mapping, pos >>PAGE_SHIFT , len);if (IS_ERR (folio))return PTR_ERR (folio); *foliop = folio;
@@ -372,6 +371,7 @@ static int nfs_write_begin(const struct kiocb *iocb, }else if (!once_thru &&nfs_want_read_modify_write (file, folio, pos, len)) { once_thru =1 ; +folio_clear_dropbehind (folio); ret =nfs_read_folio (file, folio);folio_put (folio);if (!ret)
@@ -915,5 +915,6 @@ const struct file_operations nfs_file_operations = { .splice_write = iter_file_splice_write, .check_flags = nfs_check_flags, .setlease = simple_nosetlease, + .fop_flags = FOP_DONTCACHE, };EXPORT_SYMBOL_GPL (nfs_file_operations);
fs/nfs/nfs4file.c +1 -0
@@ -454,4 +454,5 @@ const struct file_operations nfs4_file_operations = {#else .llseek = nfs_file_llseek, #endif + .fop_flags = FOP_DONTCACHE, };
fs/nfs/write.c +3 -1
@@ -338,7 +338,7 @@ static void nfs_folio_end_writeback(struct folio *folio) {struct nfs_server *nfss =NFS_SERVER (folio->mapping->host); -folio_end_writeback (folio); +folio_end_writeback_no_dropbehind (folio);if (atomic_long_dec_return (&nfss->writeback) <NFS_CONGESTION_OFF_THRESH ) { nfss->write_congested =0 ;
@@ -787,6 +787,8 @@ static void nfs_inode_remove_request(struct nfs_page *req)clear_bit (PG_MAPPED , &req->wb_head->wb_flags); }spin_unlock (&mapping->i_private_lock); + +folio_end_dropbehind (folio); }nfs_page_group_unlock (req);
Signature · 1 line
2.50.1
Trond Myklebust
maintainerv5 cover Initial NFS client support for RWF_DONTCACHE
In reply to Trond MyklebustFrom: Trond Myklebust <[email protected]>
The main issue is allowing support on 2 stage writes (i.e. unstable WRITE followed by a COMMIT) since those don't follow the current assumption that the 'dropbehind' flag can be fulfilled as soon as the writeback lock is dropped.
v2:
- Make use of the new iocb parameter for nfs_write_begin()
v3:
- Set/clear PG_DROPBEHIND on the head of the nfs_page group
- Simplify helper folio_end_dropbehind
v4:
- Replace filemap_end_dropbehind_write() with folio_end_dropbehind()
- Add a helper to replace folio_end_writeback with an equivalent that does not attempt to interpret the dropbehind flag
- Keep the folio dropbehind flag set until the NFS client is ready to call folio_end_dropbehind.
- Don't try to do a read-modify-write in nfs_write_begin() if the folio has the dropbehind flag set.
v5:
- Change helper function export types to EXPORT_SYMBOL_GPL
Trond Myklebust (3):
filemap: Add a helper for filesystems implementing dropbehind
filemap: Add a version of folio_end_writeback that ignores dropbehind
NFS: Enable use of the RWF_DONTCACHE flag on the NFS client
fs/nfs/file.c | 9 +++++---- fs/nfs/nfs4file.c | 1 + fs/nfs/write.c | 4 +++- include/linux/pagemap.h | 2 ++ mm/filemap.c | 34 ++++++++++++++++++++++++++-------- 5 files changed, 37 insertions(+), 13 deletions(-)
Signature · 1 line
2.51.0
Trond Myklebust
maintainerPATCH v5 1/3 filemap: Add a helper for filesystems implementing dropbehind
From: Trond Myklebust <[email protected]>
Add a helper to allow filesystems to attempt to free the 'dropbehind' folio.
--- include/linux/pagemap.h | 1 + mm/filemap.c | 5 +++-- 2 files changed, 4 insertions(+), 2 deletions(-)
include/linux/pagemap.h +1 -0
@@ -1221,6 +1221,7 @@ void folio_wait_writeback(struct folio *folio);int folio_wait_writeback_killable (struct folio *folio);void end_page_writeback (struct page *page);void folio_end_writeback (struct folio *folio); +void folio_end_dropbehind (struct folio *folio);void folio_wait_stable (struct folio *folio);void __folio_mark_dirty (struct folio *folio,struct address_space *,int warn);void folio_account_cleaned (struct folio *folio,struct bdi_writeback *wb);
mm/filemap.c +3 -2
@@ -1608,7 +1608,7 @@ static void filemap_end_dropbehind(struct folio *folio)* completes. Do that now. If we fail, it's likely because of a big folio - * just reset dropbehind for that case and latter completions should invalidate. */ -static void filemap_end_dropbehind_write (struct folio *folio) +void folio_end_dropbehind (struct folio *folio) {if (!folio_test_dropbehind (folio))return ;
@@ -1625,6 +1625,7 @@ static void filemap_end_dropbehind_write(struct folio *folio)folio_unlock (folio); } } +EXPORT_SYMBOL_GPL (folio_end_dropbehind);/** * folio_end_writeback - End writeback against a folio.
@@ -1660,7 +1661,7 @@ void folio_end_writeback(struct folio *folio)if (__folio_end_writeback (folio))folio_wake_bit (folio, PG_writeback); -filemap_end_dropbehind_write (folio); +folio_end_dropbehind (folio);acct_reclaim_writeback (folio);folio_put (folio); }
Signature · 1 line
2.51.0
Trond Myklebust
maintainerPATCH v5 2/3 filemap: Add a version of folio_end_writeback that ignores dropbehind
In reply to Trond MyklebustFrom: Trond Myklebust <[email protected]>
Filesystems such as NFS may need to defer dropbehind until after their 2-stage writes are done. This adds a helper folio_end_writeback_no_dropbehind() that allows them to release the writeback flag without immediately dropping the folio.
--- include/linux/pagemap.h | 1 + mm/filemap.c | 29 +++++++++++++++++++++++------ 2 files changed, 24 insertions(+), 6 deletions(-)
include/linux/pagemap.h +1 -0
@@ -1221,6 +1221,7 @@ void folio_wait_writeback(struct folio *folio);int folio_wait_writeback_killable (struct folio *folio);void end_page_writeback (struct page *page);void folio_end_writeback (struct folio *folio); +void folio_end_writeback_no_dropbehind (struct folio *folio);void folio_end_dropbehind (struct folio *folio);void folio_wait_stable (struct folio *folio);void __folio_mark_dirty (struct folio *folio,struct address_space *,int warn);
mm/filemap.c +23 -6
@@ -1628,14 +1628,15 @@ void folio_end_dropbehind(struct folio *folio)EXPORT_SYMBOL_GPL (folio_end_dropbehind);/** -* folio_end_writeback - End writeback against a folio. +* folio_end_writeback_no_dropbehind - End writeback against a folio. * @folio: The folio. * * The folio must actually be under writeback. +* This call is intended for filesystems that need to defer dropbehind. * * Context: May be called from process or interrupt context. */ -void folio_end_writeback (struct folio *folio) +void folio_end_writeback_no_dropbehind (struct folio *folio) {VM_BUG_ON_FOLIO (!folio_test_writeback (folio), folio);
@@ -1651,6 +1652,25 @@ void folio_end_writeback(struct folio *folio)folio_rotate_reclaimable (folio); } +if (__folio_end_writeback (folio)) +folio_wake_bit (folio, PG_writeback); + +acct_reclaim_writeback (folio); +} +EXPORT_SYMBOL_GPL (folio_end_writeback_no_dropbehind); + +/** +* folio_end_writeback - End writeback against a folio. +* @folio: The folio. +* +* The folio must actually be under writeback. +* +* Context: May be called from process or interrupt context. +*/ +void folio_end_writeback (struct folio *folio) +{ +VM_BUG_ON_FOLIO (!folio_test_writeback (folio), folio); +/* * Writeback does not hold a folio reference of its own, relying * on truncation to wait for the clearing of PG_writeback.
@@ -1658,11 +1678,8 @@ void folio_end_writeback(struct folio *folio)* reused before the folio_wake_bit(). */ folio_get (folio); -if (__folio_end_writeback (folio)) -folio_wake_bit (folio, PG_writeback); - +folio_end_writeback_no_dropbehind (folio);folio_end_dropbehind (folio); -acct_reclaim_writeback (folio);folio_put (folio); }EXPORT_SYMBOL (folio_end_writeback);
Signature · 1 line
2.51.0
Trond Myklebust
maintainerPATCH v5 3/3 NFS: Enable use of the RWF_DONTCACHE flag on the NFS client
In reply to Trond MyklebustFrom: Trond Myklebust <[email protected]>
The NFS client needs to defer dropbehind until after any writes to the folio have been persisted on the server. Since this may be a 2 step process, use folio_end_writeback_no_dropbehind() to allow release of the writeback flag, and then call folio_end_dropbehind() once the COMMIT is done.
--- fs/nfs/file.c | 9 +++++---- fs/nfs/nfs4file.c | 1 + fs/nfs/write.c | 4 +++- 3 files changed, 9 insertions(+), 5 deletions(-)
fs/nfs/file.c +5 -4
@@ -361,6 +361,8 @@ static bool nfs_want_read_modify_write(struct file *file, struct folio *folio,if (pnfs_ld_read_whole_page (file_inode (file)))return true; +if (folio_test_dropbehind (folio)) +return false;/* Open for reading too? */ if (file->f_mode &FMODE_READ )return true;
@@ -380,7 +382,6 @@ static int nfs_write_begin(const struct kiocb *iocb, loff_t pos, unsigned len, struct folio **foliop, void **fsdata) { - fgf_t fgp = FGP_WRITEBEGIN; struct folio *folio;struct file *file = iocb->ki_filp;int once_thru =0 ;
@@ -390,10 +391,8 @@ static int nfs_write_begin(const struct kiocb *iocb, file, mapping->host->i_ino, len, (long long) pos);nfs_truncate_last_folio (mapping,i_size_read (mapping->host), pos); - fgp |=fgf_set_order (len); start: - folio =__filemap_get_folio (mapping, pos >>PAGE_SHIFT , fgp, -mapping_gfp_mask (mapping)); + folio =write_begin_get_folio (iocb, mapping, pos >>PAGE_SHIFT , len);if (IS_ERR (folio))return PTR_ERR (folio); *foliop = folio;
@@ -405,6 +404,7 @@ static int nfs_write_begin(const struct kiocb *iocb, }else if (!once_thru &&nfs_want_read_modify_write (file, folio, pos, len)) { once_thru =1 ; +folio_clear_dropbehind (folio); ret =nfs_read_folio (file, folio);folio_put (folio);if (!ret)
@@ -949,5 +949,6 @@ const struct file_operations nfs_file_operations = { .splice_write = iter_file_splice_write, .check_flags = nfs_check_flags, .setlease = simple_nosetlease, + .fop_flags = FOP_DONTCACHE, };EXPORT_SYMBOL_GPL (nfs_file_operations);
fs/nfs/nfs4file.c +1 -0
@@ -456,4 +456,5 @@ const struct file_operations nfs4_file_operations = {#else .llseek = nfs_file_llseek, #endif + .fop_flags = FOP_DONTCACHE, };
fs/nfs/write.c +3 -1
@@ -296,7 +296,7 @@ static void nfs_folio_end_writeback(struct folio *folio) {struct nfs_server *nfss =NFS_SERVER (folio->mapping->host); -folio_end_writeback (folio); +folio_end_writeback_no_dropbehind (folio);if (atomic_long_dec_return (&nfss->writeback) <NFS_CONGESTION_OFF_THRESH ) { nfss->write_congested =0 ;
@@ -745,6 +745,8 @@ static void nfs_inode_remove_request(struct nfs_page *req)clear_bit (PG_MAPPED , &req->wb_head->wb_flags); }spin_unlock (&mapping->i_private_lock); + +folio_end_dropbehind (folio); }nfs_page_group_unlock (req);
Signature · 1 line
2.51.0
Trond Myklebust
maintainerHi Andrew,
Quoted 32 lines · Trond Myklebust
From: Trond Myklebust <[email protected]>
The main issue is allowing support on 2 stage writes (i.e. unstable WRITE followed by a COMMIT) since those don't follow the current assumption that the 'dropbehind' flag can be fulfilled as soon as the writeback lock is dropped.
v2:
- Make use of the new iocb parameter fornfs_write_begin()v3:
- Set/clearPG_DROPBEHINDon the head of thenfs_pagegroup
- Simplify helperfolio_end_dropbehind
v4:
- Replacefilemap_end_dropbehind_write()withfolio_end_dropbehind()
- Add a helper to replacefolio_end_writebackwith an equivalent that
does not attempt to interpret the dropbehind flag
- Keep the folio dropbehind flag set until the NFS client is ready to
callfolio_end_dropbehind.
- Don't try to do a read-modify-write innfs_write_begin()if the folio
has the dropbehind flag set.
v5:
- Change helper function export types toEXPORT_SYMBOL_GPLTrond Myklebust (3):
filemap: Add a helper for filesystems implementing dropbehind
filemap: Add a version offolio_end_writebackthat ignores dropbehind
NFS: Enable use of theRWF_DONTCACHEflag on the NFS client
Quoted 6 lines
fs/nfs/file.c | 9 +++++---- fs/nfs/nfs4file.c | 1 +
fs/nfs/write.c | 4 +++-include/linux/pagemap.h| 2 ++
mm/filemap.c | 34 ++++++++++++++++++++++++++--------5 files changed, 37 insertions(+), 13 deletions(-)
Since the above series has already done the rounds in the linux-nfs and linux-fsdevel mailing lists, could you please ask you to shepherd it in to the 6.18 merge window? As you can see above the larger set of changes are to mm/filemap.c rather than being NFS specific.
Cheers
Trond
Signature · 3 lines
Trond Myklebust Linux NFS client maintainer, Hammerspace [email protected], [email protected]