Add XFS support for RWF_DONTCACHE
Subsystems (3)
v2 cover Add XFS support for RWF_DONTCACHE
Hi,
Now that the main bits are in mainline, here are the XFS patches for adding RWF_DONTCACHE support. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flags FOP_DONTCACHE support in the file_operations struct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.
Patches are aginst 6.14-rc1.
Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick
Signature · 1 line
Jens Axboe
PATCH v2 1/2 iomap: make buffered writes work with RWF_DONTCACHE
Add iomap buffered write support for RWF_DONTCACHE. If RWF_DONTCACHE is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest.
--- Documentation/filesystems/iomap/design.rst | 5 +++++ Documentation/filesystems/iomap/operations.rst | 2 ++ fs/iomap/buffered-io.c | 4 ++++ include/linux/iomap.h | 1 + 4 files changed, 12 insertions(+)
Documentation/filesystems/iomap/design.rst +5 -0
@@ -352,6 +352,11 @@ operations: ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or ``RWF_NOWAIT``. + * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a + buffered file I/O and would like the kernel to drop the pagecache + after the I/O completes, if it isn't already being used by another + thread. + If it is necessary to read existing file contents from a `different <https://lore.kernel.org/all/[email protected]/>`_ device or address range on a device, the filesystem should return that
Documentation/filesystems/iomap/operations.rst +2 -0
@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap: * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``. + * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``. + Internal per-Folio State ------------------------
fs/iomap/buffered-io.c +4 -0
@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)if (iter->flags &IOMAP_NOWAIT ) fgp |=FGP_NOWAIT ; +if (iter->flags &IOMAP_DONTCACHE ) + fgp |=FGP_DONTCACHE ; fgp |=fgf_set_order (len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,if (iocb->ki_flags &IOCB_NOWAIT ) iter.flags |=IOMAP_NOWAIT ; +if (iocb->ki_flags &IOCB_DONTCACHE ) + iter.flags |=IOMAP_DONTCACHE ;while ((ret =iomap_iter (&iter, ops)) >0 ) iter.processed =iomap_write_iter (&iter, i);
include/linux/iomap.h +1 -0
@@ -183,6 +183,7 @@ struct iomap_folio_ops {#define IOMAP_DAX 0#endif /* CONFIG_FS_DAX */ #define IOMAP_ATOMIC (1 << 9) +#define IOMAP_DONTCACHE (1 << 10)struct iomap_ops {/*
Signature · 1 line
2.47.2
PATCH v2 2/2 xfs: flag as supporting FOP_DONTCACHE
In reply to Jens AxboeRead side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting FOP_DONTCACHE in the file_operations structure to enable full support for read and write uncached IO.
This provides similar benefits to using RWF_DONTCACHE with reads. Testing buffered writes on 32 files:
writing bs 65536, uncached 0
1s: 196035MB/sec
2s: 132308MB/sec
3s: 132438MB/sec
4s: 116528MB/sec
5s: 103898MB/sec
6s: 108893MB/sec
7s: 99678MB/sec
8s: 106545MB/sec
9s: 106826MB/sec
10s: 101544MB/sec
11s: 111044MB/sec
12s: 124257MB/sec
13s: 116031MB/sec
14s: 114540MB/sec
15s: 115011MB/sec
16s: 115260MB/sec
17s: 116068MB/sec
18s: 116096MB/sec
where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages.
Running the same test with uncached buffered writes:
writing bs 65536, uncached 1
1s: 198974MB/sec
2s: 189618MB/sec
3s: 193601MB/sec
4s: 188582MB/sec
5s: 193487MB/sec
6s: 188341MB/sec
7s: 194325MB/sec
8s: 188114MB/sec
9s: 192740MB/sec
10s: 189206MB/sec
11s: 193442MB/sec
12s: 189659MB/sec
13s: 191732MB/sec
14s: 190701MB/sec
15s: 191789MB/sec
16s: 191259MB/sec
17s: 190613MB/sec
18s: 191951MB/sec
and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write.
--- fs/xfs/xfs_file.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
fs/xfs/xfs_file.c +2 -1
@@ -1626,7 +1626,8 @@ const struct file_operations xfs_file_operations = { .fadvise = xfs_file_fadvise, .remap_file_range = xfs_file_remap_range, .fop_flags =FOP_MMAP_SYNC |FOP_BUFFER_RASYNC | - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE, +FOP_BUFFER_WASYNC |FOP_DIO_PARALLEL_WRITE | + FOP_DONTCACHE, };const struct file_operations xfs_dir_file_operations = {
Signature · 1 line
2.47.2
Re: [PATCH 2/2] xfs: flag as supporting FOP_DONTCACHE
Quoted 55 lines · Jens Axboe
Read side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting
FOP_DONTCACHEin thefile_operationsstructure to enable full support for read and write uncached IO.This provides similar benefits to using
RWF_DONTCACHEwith reads. Testing buffered writes on 32 files:writing bs 65536, uncached 0
1s: 196035MB/sec
2s: 132308MB/sec
3s: 132438MB/sec
4s: 116528MB/sec
5s: 103898MB/sec
6s: 108893MB/sec
7s: 99678MB/sec
8s: 106545MB/sec
9s: 106826MB/sec
10s: 101544MB/sec
11s: 111044MB/sec
12s: 124257MB/sec
13s: 116031MB/sec
14s: 114540MB/sec
15s: 115011MB/sec
16s: 115260MB/sec
17s: 116068MB/sec
18s: 116096MB/secwhere it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages.
Running the same test with uncached buffered writes:
writing bs 65536, uncached 1
1s: 198974MB/sec
2s: 189618MB/sec
3s: 193601MB/sec
4s: 188582MB/sec
5s: 193487MB/sec
6s: 188341MB/sec
7s: 194325MB/sec
8s: 188114MB/sec
9s: 192740MB/sec
10s: 189206MB/sec
11s: 193442MB/sec
12s: 189659MB/sec
13s: 191732MB/sec
14s: 190701MB/sec
15s: 191789MB/sec
16s: 191259MB/sec
17s: 190613MB/sec
18s: 191951MB/sec
Quoted 6 lines
and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write.
Signed-off-by Jens Axboe <[email protected]>
--D
Quoted 22 lines
--- fs/xfs/xfs_file.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)fs/xfs/xfs_file.c +2
-1@@ -1626,7 +1626,8 @@ const struct file_operations xfs_file_operations = { .fadvise = xfs_file_fadvise, .remap_file_range = xfs_file_remap_range, .fop_flags =FOP_MMAP_SYNC |FOP_BUFFER_RASYNC | - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE, +FOP_BUFFER_WASYNC |FOP_DIO_PARALLEL_WRITE | + FOP_DONTCACHE, };const struct file_operations xfs_dir_file_operations = {Signature · 1 line
2.47.2
Quoted 9 lines · Jens Axboe
Hi,
Now that the main bits are in mainline, here are the XFS patches for adding
RWF_DONTCACHEsupport. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flagsFOP_DONTCACHEsupport in thefile_operationsstruct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.Patches are aginst 6.14-rc1.
Gentle ping on this one.
Signature · 1 line
Jens Axboe
Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
In reply to Jens AxboeLooks good:
Re: [PATCH 2/2] xfs: flag as supporting FOP_DONTCACHE
In reply to Jens AxboeLooks good:
Quoted 8 lines · Jens Axboe
Hi,
Now that the main bits are in mainline, here are the XFS patches for adding
RWF_DONTCACHEsupport. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flagsFOP_DONTCACHEsupport in thefile_operationsstruct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.
Quoted 5 lines
Patches are aginst 6.14-rc1.
Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick
Is this slated for 6.15? I don't see it in the tree yet, but I also don't see a lot of other things in there. Would be a shame to miss the next release on this front.
Signature · 1 line
Jens Axboe
Carlos Maiolino
maintainerQuoted 9 lines · Jens Axboe
Quoted 8 lines · Jens Axboe
Hi,
Now that the main bits are in mainline, here are the XFS patches for adding
RWF_DONTCACHEsupport. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flagsFOP_DONTCACHEsupport in thefile_operationsstruct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.
Quoted 6 lines
Quoted 5 lines
Patches are aginst 6.14-rc1.
Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick
Hello Jens
Quoted 3 lines
Is this slated for 6.15? I don't see it in the tree yet, but I also don't see a lot of other things in there. Would be a shame to miss the next release on this front.
The changes looks fine, they are for 6.15, so I don't push things to for-next for the next release while working on the current one, I'm not sure I got your point. What else are these 'lot of other things' you don't see in there you mentioned?
Carlos
Quoted 3 lines
--
Jens Axboe
Carlos Maiolino
maintainerRe: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
In reply to Jens AxboeQuoted 6 lines · Jens Axboe
Add iomap buffered write support for
RWF_DONTCACHE. IfRWF_DONTCACHEis set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. Thewrite_iterhandler simply kicks off writeback for the pages, and writeback completion will take care of the rest.
[Adding Brauner to the loop as this usually goes through his tree.]
Christian, I'm pulling this into my tree for 6.15 if this is ok with you? Not sure if you're subscribed to linux-xfs, so, just in case, the link for the whole 2-patches series is below.
https://lore.kernel.org/linux-xfs/[email protected]/
Quoted 75 lines
Signed-off-by "Darrick J. Wong" <[email protected]>Signed-off-by Jens Axboe <[email protected]>--- Documentation/filesystems/iomap/design.rst | 5 +++++ Documentation/filesystems/iomap/operations.rst | 2 ++ fs/iomap/buffered-io.c | 4 ++++ include/linux/iomap.h | 1 + 4 files changed, 12 insertions(+)Documentation/filesystems/iomap/design.rst +5
-0@@ -352,6 +352,11 @@ operations: ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or ``RWF_NOWAIT``. + * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a + buffered file I/O and would like the kernel to drop the pagecache + after the I/O completes, if it isn't already being used by another + thread. + If it is necessary to read existing file contents from a `different <https://lore.kernel.org/all/[email protected]/>`_ device or address range on a device, the filesystem should return thatDocumentation/filesystems/iomap/operations.rst +2
-0@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap: * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``. + * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``. + Internal per-Folio State ------------------------fs/iomap/buffered-io.c +4
-0@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)if (iter->flags &IOMAP_NOWAIT ) fgp |=FGP_NOWAIT ; +if (iter->flags &IOMAP_DONTCACHE ) + fgp |=FGP_DONTCACHE ; fgp |=fgf_set_order (len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,if (iocb->ki_flags &IOCB_NOWAIT ) iter.flags |=IOMAP_NOWAIT ; +if (iocb->ki_flags &IOCB_DONTCACHE ) + iter.flags |=IOMAP_DONTCACHE ;while ((ret =iomap_iter (&iter, ops)) >0 ) iter.processed =iomap_write_iter (&iter, i);include/linux/iomap.h +1
-1@@ -183,6 +183,7 @@ struct iomap_folio_ops {#define IOMAP_DAX 0#endif /* CONFIG_FS_DAX */ #define IOMAP_ATOMIC (1 << 9) +#define IOMAP_DONTCACHE (1 << 10)struct iomap_ops {/* -- 2.47.2
Christian Brauner
maintainerRe: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
Quoted 8 lines · Carlos Maiolino
Quoted 6 lines · Jens Axboe
Add iomap buffered write support for
RWF_DONTCACHE. IfRWF_DONTCACHEis set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. Thewrite_iterhandler simply kicks off writeback for the pages, and writeback completion will take care of the rest.
Quoted 7 lines
[Adding Brauner to the loop as this usually goes through his tree.]
Christian, I'm pulling this into my tree for 6.15 if this is ok with you? Not sure if you're subscribed to linux-xfs, so, just in case, the link for the whole 2-patches series is below.
vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:
vfs-6.15.shared.iomap
Just pull it into your branch, please and I pull it into vfs-6.15.iomap.
I see that 2/2 adds a new FOP_UNCACHED flag to the VFS FOP_UNCACHED please make sure to Cc fsdevel for such additions and remind contributors as well.
Quoted 77 lines
Quoted 77 lines
Signed-off-by "Darrick J. Wong" <[email protected]>Signed-off-by Jens Axboe <[email protected]>--- Documentation/filesystems/iomap/design.rst | 5 +++++ Documentation/filesystems/iomap/operations.rst | 2 ++ fs/iomap/buffered-io.c | 4 ++++ include/linux/iomap.h | 1 + 4 files changed, 12 insertions(+)Documentation/filesystems/iomap/design.rst +5
-0@@ -352,6 +352,11 @@ operations: ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or ``RWF_NOWAIT``. + * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a + buffered file I/O and would like the kernel to drop the pagecache + after the I/O completes, if it isn't already being used by another + thread. + If it is necessary to read existing file contents from a `different <https://lore.kernel.org/all/[email protected]/>`_ device or address range on a device, the filesystem should return thatDocumentation/filesystems/iomap/operations.rst +2
-0@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap: * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``. + * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``. + Internal per-Folio State ------------------------fs/iomap/buffered-io.c +4
-0@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)if (iter->flags &IOMAP_NOWAIT ) fgp |=FGP_NOWAIT ; +if (iter->flags &IOMAP_DONTCACHE ) + fgp |=FGP_DONTCACHE ; fgp |=fgf_set_order (len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,if (iocb->ki_flags &IOCB_NOWAIT ) iter.flags |=IOMAP_NOWAIT ; +if (iocb->ki_flags &IOCB_DONTCACHE ) + iter.flags |=IOMAP_DONTCACHE ;while ((ret =iomap_iter (&iter, ops)) >0 ) iter.processed =iomap_write_iter (&iter, i);include/linux/iomap.h +1
-1@@ -183,6 +183,7 @@ struct iomap_folio_ops {#define IOMAP_DAX 0#endif /* CONFIG_FS_DAX */ #define IOMAP_ATOMIC (1 << 9) +#define IOMAP_DONTCACHE (1 << 10)struct iomap_ops {/* -- 2.47.2
Carlos Maiolino
maintainerRe: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
Quoted 15 lines · Christian Brauner
Quoted 14 lines · Carlos Maiolino
Quoted 6 lines · Jens Axboe
Add iomap buffered write support for
RWF_DONTCACHE. IfRWF_DONTCACHEis set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. Thewrite_iterhandler simply kicks off writeback for the pages, and writeback completion will take care of the rest.[Adding Brauner to the loop as this usually goes through his tree.]
Christian, I'm pulling this into my tree for 6.15 if this is ok with you? Not sure if you're subscribed to linux-xfs, so, just in case, the link for the whole 2-patches series is below.
Quoted 8 lines
Quoted 1 line
vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:
vfs-6.15.shared.iomap
Just pull it into your branch, please and I pull it into vfs-6.15.iomap.
Sounds good Chris. Thanks, I'll pull it once I start merging patches for 6.15 MW.
Carlos.
Quoted 82 lines
I see that 2/2 adds a new
FOP_UNCACHEDflag to the VFSFOP_UNCACHEDplease make sure to Cc fsdevel for such additions and remind contributors as well.Quoted 77 lines
Quoted 77 lines
Signed-off-by "Darrick J. Wong" <[email protected]>Signed-off-by Jens Axboe <[email protected]>--- Documentation/filesystems/iomap/design.rst | 5 +++++ Documentation/filesystems/iomap/operations.rst | 2 ++ fs/iomap/buffered-io.c | 4 ++++ include/linux/iomap.h | 1 + 4 files changed, 12 insertions(+)Documentation/filesystems/iomap/design.rst +5
-0@@ -352,6 +352,11 @@ operations: ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or ``RWF_NOWAIT``. + * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a + buffered file I/O and would like the kernel to drop the pagecache + after the I/O completes, if it isn't already being used by another + thread. + If it is necessary to read existing file contents from a `different <https://lore.kernel.org/all/[email protected]/>`_ device or address range on a device, the filesystem should return thatDocumentation/filesystems/iomap/operations.rst +2
-0@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap: * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``. + * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``. + Internal per-Folio State ------------------------fs/iomap/buffered-io.c +4
-0@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)if (iter->flags &IOMAP_NOWAIT ) fgp |=FGP_NOWAIT ; +if (iter->flags &IOMAP_DONTCACHE ) + fgp |=FGP_DONTCACHE ; fgp |=fgf_set_order (len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,if (iocb->ki_flags &IOCB_NOWAIT ) iter.flags |=IOMAP_NOWAIT ; +if (iocb->ki_flags &IOCB_DONTCACHE ) + iter.flags |=IOMAP_DONTCACHE ;while ((ret =iomap_iter (&iter, ops)) >0 ) iter.processed =iomap_write_iter (&iter, i);include/linux/iomap.h +1
-1@@ -183,6 +183,7 @@ struct iomap_folio_ops {#define IOMAP_DAX 0#endif /* CONFIG_FS_DAX */ #define IOMAP_ATOMIC (1 << 9) +#define IOMAP_DONTCACHE (1 << 10)struct iomap_ops {/* -- 2.47.2
Quoted 17 lines · Carlos Maiolino
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Jens Axboe
Hi,
Now that the main bits are in mainline, here are the XFS patches for adding
RWF_DONTCACHEsupport. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flagsFOP_DONTCACHEsupport in thefile_operationsstruct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.Quoted 6 lines
Quoted 5 lines
Patches are aginst 6.14-rc1.
Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick
Quoted 8 lines
Hello Jens
Quoted 3 lines
Is this slated for 6.15? I don't see it in the tree yet, but I also don't see a lot of other things in there. Would be a shame to miss the next release on this front.
The changes looks fine, they are for 6.15, so I don't push things to for-next for the next release while working on the current one, I'm
Hmm ok, that seems like an odd choice, surely you'd want things in for-next earlier rather than later? At least that's what everybody else is doing.
Quoted 2 lines
not sure I got your point. What else are these 'lot of other things' you don't see in there you mentioned?
Oh, it was just a comment on there not seeming to be any xfs changes for 6.15 in your for-next (or other branches) yet, which made me think that perhaps I was looking in the wrong spot. This where I looked, fwiw:
https://web.git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/
which does seem to have a few changes for 6.15, but really not much. Maybe this is all normal for xfs?
Signature · 1 line
Jens Axboe
Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
In reply to Christian BraunerQuoted 4 lines · Christian Brauner
Quoted 1 line
vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:
Note that we'll also need the earliest changes in vfs.iomap for the zoned xfs changes that are about to be ready. So we might as well pull in the entire vfs.iomap branch.
Christian Brauner
maintainerRe: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Christian Brauner
Quoted 1 line
vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:
Note that we'll also need the earliest changes in vfs.iomap for the zoned xfs changes that are about to be ready. So we might as well pull in the entire vfs.iomap branch.
Ok, then I'll just put the series into vfs-6.15.iomap and you can just pull that in.
Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
In reply to Jens Axboeinclude/linux/iomap.h struct iomap_folio_ops { · line 186
Quoted 7 lines · Jens Axboe
include/linux/iomap.h +1
-0@@ -183,6 +183,7 @@ struct iomap_folio_ops {#define IOMAP_DAX 0#endif /* CONFIG_FS_DAX */ #define IOMAP_ATOMIC (1 << 9) +#define IOMAP_DONTCACHE (1 << 10)
JFYI, IOMAP_FLAGS_STRINGS might need to be updated for this, but not adding it may be intentional. I did not see anyone mention this in the reviews.
Thanks,
John
Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE
In reply to Jens AxboeQuoted 9 lines · Jens Axboe
Add iomap buffered write support for
RWF_DONTCACHE. IfRWF_DONTCACHEis set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. Thewrite_iterhandler simply kicks off writeback for the pages, and writeback completion will take care of the rest.Signed-off-by "Darrick J. Wong"<[email protected]>Signed-off-by Jens Axboe<[email protected]>---
Looks good.
-ck
Re: [PATCH 2/2] xfs: flag as supporting FOP_DONTCACHE
In reply to Jens AxboeQuoted 55 lines · Jens Axboe
Read side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting
FOP_DONTCACHEin thefile_operationsstructure to enable full support for read and write uncached IO.This provides similar benefits to using
RWF_DONTCACHEwith reads. Testing buffered writes on 32 files:writing bs 65536, uncached 0
1s: 196035MB/sec
2s: 132308MB/sec
3s: 132438MB/sec
4s: 116528MB/sec
5s: 103898MB/sec
6s: 108893MB/sec
7s: 99678MB/sec
8s: 106545MB/sec
9s: 106826MB/sec
10s: 101544MB/sec
11s: 111044MB/sec
12s: 124257MB/sec
13s: 116031MB/sec
14s: 114540MB/sec
15s: 115011MB/sec
16s: 115260MB/sec
17s: 116068MB/sec
18s: 116096MB/secwhere it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages.
Running the same test with uncached buffered writes:
writing bs 65536, uncached 1
1s: 198974MB/sec
2s: 189618MB/sec
3s: 193601MB/sec
4s: 188582MB/sec
5s: 193487MB/sec
6s: 188341MB/sec
7s: 194325MB/sec
8s: 188114MB/sec
9s: 192740MB/sec
10s: 189206MB/sec
11s: 193442MB/sec
12s: 189659MB/sec
13s: 191732MB/sec
14s: 190701MB/sec
15s: 191789MB/sec
16s: 191259MB/sec
17s: 190613MB/sec
18s: 191951MB/sec
Quoted 6 lines
and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write.
Signed-off-by Jens Axboe<[email protected]>
Looks good.
-ck