ceph: support RWF_DONTCACHE
Subsystems (3)
PATCH 1/1 ceph: support RWF_DONTCACHE
Similar to commit 902893e39076 ("NFS: Enable use of the RWF_DONTCACHE flag on the NFS client").
The Ceph client now supports RWF_DONTCACHE for buffered file IO, but it needs preserve the IOCB_DONTCACHE state through netfs_write_begin().
Unlike NFS, Ceph does not need to defer drop-behind until after a separate COMMIT step. Ceph's buffered writeback completes when the OSD write completes, so the existing folio_end_writeback() handling is sufficient.
This patch changes netfs because Ceph does not get its write_begin folio directly. ceph_write_begin() is a wrapper around netfs_write_begin(), and that helper was calling __filemap_get_folio(FGP_WRITEBEGIN, ...) without access to the kiocb. As a result, the per-IO IOCB_DONTCACHE flag was lost before folio allocation.
Fix this by passing the kiocb into netfs_write_begin() and switching it to write_begin_get_folio(iocb, ...), so IOCB_DONTCACHE is translated into FGP_DONTCACHE when appropriate. Then set FOP_DONTCACHE on ceph_file_fops so the VFS will accept RWF_DONTCACHE for files on Ceph.
--- fs/ceph/addr.c | 3 ++- fs/ceph/file.c | 1 + fs/netfs/buffered_read.c | 6 +++--- include/linux/netfs.h | 3 ++- 4 files changed, 8 insertions(+), 5 deletions(-)
fs/ceph/addr.c +2 -1
@@ -1865,7 +1865,8 @@ static int ceph_write_begin(const struct kiocb *iocb,struct ceph_inode_info *ci =ceph_inode (inode);int r; - r =netfs_write_begin (&ci->netfs, file, inode->i_mapping, pos, len, foliop,NULL ); + r =netfs_write_begin (iocb, &ci->netfs, file, inode->i_mapping, pos, + len, foliop,NULL );if (r <0 )return r;
fs/ceph/file.c +1 -0
@@ -3191,4 +3191,5 @@ const struct file_operations ceph_file_fops = { .compat_ioctl = compat_ptr_ioctl, .fallocate = ceph_fallocate, .copy_file_range = ceph_copy_file_range, + .fop_flags = FOP_DONTCACHE, };
fs/netfs/buffered_read.c +3 -3
@@ -589,6 +589,7 @@ static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len,/** * netfs_write_begin - Helper to prepare for writing [DEPRECATED] +* @iocb: The kiocb describing the write request * @ctx: The netfs context * @file: The file to read from * @mapping: The mapping to read from
@@ -620,7 +621,7 @@ static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len,* Note that this should be considered deprecated and netfs_perform_write() * used instead. */ -int netfs_write_begin (struct netfs_inode *ctx, +int netfs_write_begin (const struct kiocb *iocb,struct netfs_inode *ctx,struct file *file,struct address_space *mapping,loff_t pos,unsigned int len,struct folio **_folio,void **_fsdata)
@@ -631,8 +632,7 @@ int netfs_write_begin(struct netfs_inode *ctx,int ret; retry: - folio =__filemap_get_folio (mapping, index,FGP_WRITEBEGIN , -mapping_gfp_mask (mapping)); + folio =write_begin_get_folio (iocb, mapping, index, len);if (IS_ERR (folio))return PTR_ERR (folio);
include/linux/netfs.h +2 -1
@@ -414,7 +414,8 @@ int netfs_writeback_single(struct address_space *mapping,struct readahead_control ;void netfs_readahead (struct readahead_control *);int netfs_read_folio (struct file *,struct folio *); -int netfs_write_begin (struct netfs_inode *,struct file *, +int netfs_write_begin (const struct kiocb *iocb, +struct netfs_inode *ctx,struct file *file,struct address_space *,loff_t pos,unsigned int len,struct folio **,void **fsdata); int netfs_writepages(struct address_space *mapping,
Signature · 1 line
2.47.3
Quoted 62 lines · Max Kellermann
Similar to commit
902893e39076("NFS: Enable use of theRWF_DONTCACHEflag on the NFS client").The Ceph client now supports
RWF_DONTCACHEfor buffered file IO, but it needs preserve theIOCB_DONTCACHEstate throughnetfs_write_begin().Unlike NFS, Ceph does not need to defer drop-behind until after a separate COMMIT step. Ceph's buffered writeback completes when the OSD write completes, so the existing
folio_end_writeback()handling is sufficient.This patch changes netfs because Ceph does not get its
write_beginfolio directly.ceph_write_begin()is a wrapper aroundnetfs_write_begin(), and that helper was calling__filemap_get_folio(FGP_WRITEBEGIN, ...)without access to the kiocb. As a result, the per-IOIOCB_DONTCACHEflag was lost before folio allocation.Fix this by passing the kiocb into
netfs_write_begin()and switching it towrite_begin_get_folio(iocb, ...), soIOCB_DONTCACHEis translated intoFGP_DONTCACHEwhen appropriate. Then setFOP_DONTCACHEonceph_file_fopsso the VFS will acceptRWF_DONTCACHEfor files on Ceph.Signed-off-by Max Kellermann <[email protected]>--- fs/ceph/addr.c | 3 ++- fs/ceph/file.c | 1 + fs/netfs/buffered_read.c | 6 +++--- include/linux/netfs.h | 3 ++- 4 files changed, 8 insertions(+), 5 deletions(-)fs/ceph/addr.c +2
-1@@ -1865,7 +1865,8 @@ static int ceph_write_begin(const struct kiocb *iocb,struct ceph_inode_info *ci =ceph_inode (inode);int r; - r =netfs_write_begin (&ci->netfs, file, inode->i_mapping, pos, len, foliop,NULL ); + r =netfs_write_begin (iocb, &ci->netfs, file, inode->i_mapping, pos, + len, foliop,NULL );if (r <0 )return r;fs/ceph/file.c +1
-0@@ -3191,4 +3191,5 @@ const struct file_operations ceph_file_fops = { .compat_ioctl = compat_ptr_ioctl, .fallocate = ceph_fallocate, .copy_file_range = ceph_copy_file_range, + .fop_flags = FOP_DONTCACHE, };fs/netfs/buffered_read.c +0
-0
fs/netfs/buffered_read.c static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len, · line 591
Quoted 3 lines
fs/netfs/buffered_read.c +0
-0@@ -589,6 +589,7 @@ static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len,/** * netfs_write_begin - Helper to prepare for writing [DEPRECATED]
The netfs_write_begin() is deprecated function. Maybe, it's time to switch CephFS on netfs_perform_write() finally?
Thanks,
Slava.
Quoted 37 lines
fs/netfs/buffered_read.c +3
-3+* @iocb: The kiocb describing the write request * @ctx: The netfs context * @file: The file to read from * @mapping: The mapping to read from @@ -620,7 +621,7 @@ static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len,* Note that this should be considered deprecated and netfs_perform_write() * used instead. */ -int netfs_write_begin (struct netfs_inode *ctx, +int netfs_write_begin (const struct kiocb *iocb,struct netfs_inode *ctx,struct file *file,struct address_space *mapping,loff_t pos,unsigned int len,struct folio **_folio,void **_fsdata)@@ -631,8 +632,7 @@ int netfs_write_begin(struct netfs_inode *ctx,int ret; retry: - folio =__filemap_get_folio (mapping, index,FGP_WRITEBEGIN , -mapping_gfp_mask (mapping)); + folio =write_begin_get_folio (iocb, mapping, index, len);if (IS_ERR (folio))return PTR_ERR (folio);include/linux/netfs.h +2
-1@@ -414,7 +414,8 @@ int netfs_writeback_single(struct address_space *mapping,struct readahead_control ;void netfs_readahead (struct readahead_control *);int netfs_read_folio (struct file *,struct folio *); -int netfs_write_begin (struct netfs_inode *,struct file *, +int netfs_write_begin (const struct kiocb *iocb, +struct netfs_inode *ctx,struct file *file,struct address_space *,loff_t pos,unsigned int len,struct folio **,void **fsdata); int netfs_writepages(struct address_space *mapping,
David Howells
maintainerQuoted 2 lines · Viacheslav Dubeyko
The
netfs_write_begin()is deprecated function. Maybe, it's time to switch CephFS onnetfs_perform_write()finally?
I've got that on my to-do list for the next couple of weeks, wrapped around LSF/MM. The MM people would really like to get rid of some of the old things ceph uses, such as PG_private_2.
David
Hi Max Kellermann,
Thanks for the patch. This looks good to me.
One small commit message nit: "needs preserve" should be "needs to preserve".
Signature · 1 line
Alex Markuze