Skip to contentSkip to conversation

Add XFS support for RWF_DONTCACHE

PATCH In review Jens Axboe 7 participants 18 messages Feb 4, 2025 – Mar 12, 2025

Subsystems (3)
lore
Share & tools
lore

Jens Axboe

author
#0 lore raw

v2 cover Add XFS support for RWF_DONTCACHE

Hi,

Now that the main bits are in mainline, here are the XFS patches for adding RWF_DONTCACHE support. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flags FOP_DONTCACHE support in the file_operations struct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.

Patches are aginst 6.14-rc1.

Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick

Signature · 1 line
Jens Axboe
author
#1 lore raw

PATCH v2 1/2 iomap: make buffered writes work with RWF_DONTCACHE

Add iomap buffered write support for RWF_DONTCACHE. If RWF_DONTCACHE is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest.

---
 Documentation/filesystems/iomap/design.rst     | 5 +++++
 Documentation/filesystems/iomap/operations.rst | 2 ++
 fs/iomap/buffered-io.c                         | 4 ++++
 include/linux/iomap.h                          | 1 +
 4 files changed, 12 insertions(+)
Documentation/filesystems/iomap/design.rst +5 -0
index b0d0188a095e..7b91546750f5 100644
@@ -352,6 +352,11 @@ operations:
    ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or
    ``RWF_NOWAIT``.
 
+ * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a
+   buffered file I/O and would like the kernel to drop the pagecache
+   after the I/O completes, if it isn't already being used by another
+   thread.
+
 If it is necessary to read existing file contents from a `different
 <https://lore.kernel.org/all/[email protected]/>`_
 device or address range on a device, the filesystem should return that
Documentation/filesystems/iomap/operations.rst +2 -0
index 2c7f5df9d8b0..584ff549f9a6 100644
@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap:
 
  * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``.
 
+ * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``.
+
 Internal per-Folio State
 ------------------------
 
fs/iomap/buffered-io.c +4 -0
index d303e6c8900c..ea863c3cf510 100644
@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
 
 	if (iter->flags & IOMAP_NOWAIT)
 		fgp |= FGP_NOWAIT;
+	if (iter->flags & IOMAP_DONTCACHE)
+		fgp |= FGP_DONTCACHE;
 	fgp |= fgf_set_order(len);
 
 	return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,
 
 	if (iocb->ki_flags & IOCB_NOWAIT)
 		iter.flags |= IOMAP_NOWAIT;
+	if (iocb->ki_flags & IOCB_DONTCACHE)
+		iter.flags |= IOMAP_DONTCACHE;
 
 	while ((ret = iomap_iter(&iter, ops)) > 0)
 		iter.processed = iomap_write_iter(&iter, i);
include/linux/iomap.h +1 -0
index 75bf54e76f3b..26b0dbe23e62 100644
@@ -183,6 +183,7 @@ struct iomap_folio_ops {
 #define IOMAP_DAX		0
 #endif /* CONFIG_FS_DAX */
 #define IOMAP_ATOMIC		(1 << 9)
+#define IOMAP_DONTCACHE		(1 << 10)
 
 struct iomap_ops {
 	/*
Signature · 1 line
2.47.2
maintainer
#3 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

In reply to Jens Axboe
Quoted 6 lines · Jens Axboe

Add iomap buffered write support for RWF_DONTCACHE. If RWF_DONTCACHE is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest.

[Adding Brauner to the loop as this usually goes through his tree.]

Christian, I'm pulling this into my tree for 6.15 if this is ok with you? Not sure if you're subscribed to linux-xfs, so, just in case, the link for the whole 2-patches series is below.

https://lore.kernel.org/linux-xfs/[email protected]/

Quoted 75 lines
---
 Documentation/filesystems/iomap/design.rst     | 5 +++++
 Documentation/filesystems/iomap/operations.rst | 2 ++
 fs/iomap/buffered-io.c                         | 4 ++++
 include/linux/iomap.h                          | 1 +
 4 files changed, 12 insertions(+)
Documentation/filesystems/iomap/design.rst +5 -0
index b0d0188a095e..7b91546750f5 100644
@@ -352,6 +352,11 @@ operations:
    ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or
    ``RWF_NOWAIT``.
 
+ * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a
+   buffered file I/O and would like the kernel to drop the pagecache
+   after the I/O completes, if it isn't already being used by another
+   thread.
+
 If it is necessary to read existing file contents from a `different
 <https://lore.kernel.org/all/[email protected]/>`_
 device or address range on a device, the filesystem should return that
Documentation/filesystems/iomap/operations.rst +2 -0
index 2c7f5df9d8b0..584ff549f9a6 100644
@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap:
 
  * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``.
 
+ * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``.
+
 Internal per-Folio State
 ------------------------
 
fs/iomap/buffered-io.c +4 -0
index d303e6c8900c..ea863c3cf510 100644
@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
 
 	if (iter->flags & IOMAP_NOWAIT)
 		fgp |= FGP_NOWAIT;
+	if (iter->flags & IOMAP_DONTCACHE)
+		fgp |= FGP_DONTCACHE;
 	fgp |= fgf_set_order(len);
 
 	return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,
 
 	if (iocb->ki_flags & IOCB_NOWAIT)
 		iter.flags |= IOMAP_NOWAIT;
+	if (iocb->ki_flags & IOCB_DONTCACHE)
+		iter.flags |= IOMAP_DONTCACHE;
 
 	while ((ret = iomap_iter(&iter, ops)) > 0)
 		iter.processed = iomap_write_iter(&iter, i);
include/linux/iomap.h +1 -1
index 75bf54e76f3b..26b0dbe23e62 100644
@@ -183,6 +183,7 @@ struct iomap_folio_ops {
 #define IOMAP_DAX		0
 #endif /* CONFIG_FS_DAX */
 #define IOMAP_ATOMIC		(1 << 9)
+#define IOMAP_DONTCACHE		(1 << 10)
 
 struct iomap_ops {
 	/*
--

2.47.2

#4 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

Quoted 8 lines · Carlos Maiolino
Quoted 6 lines · Jens Axboe

Add iomap buffered write support for RWF_DONTCACHE. If RWF_DONTCACHE is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest.

Quoted 7 lines

[Adding Brauner to the loop as this usually goes through his tree.]

Christian, I'm pulling this into my tree for 6.15 if this is ok with you? Not sure if you're subscribed to linux-xfs, so, just in case, the link for the whole 2-patches series is below.

https://lore.kernel.org/linux-xfs/[email protected]/

vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:

vfs-6.15.shared.iomap

Just pull it into your branch, please and I pull it into vfs-6.15.iomap.

I see that 2/2 adds a new FOP_UNCACHED flag to the VFS FOP_UNCACHED please make sure to Cc fsdevel for such additions and remind contributors as well.

Quoted 77 lines
Quoted 77 lines
---
 Documentation/filesystems/iomap/design.rst     | 5 +++++
 Documentation/filesystems/iomap/operations.rst | 2 ++
 fs/iomap/buffered-io.c                         | 4 ++++
 include/linux/iomap.h                          | 1 +
 4 files changed, 12 insertions(+)
Documentation/filesystems/iomap/design.rst +5 -0
index b0d0188a095e..7b91546750f5 100644
@@ -352,6 +352,11 @@ operations:
    ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or
    ``RWF_NOWAIT``.
 
+ * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a
+   buffered file I/O and would like the kernel to drop the pagecache
+   after the I/O completes, if it isn't already being used by another
+   thread.
+
 If it is necessary to read existing file contents from a `different
 <https://lore.kernel.org/all/[email protected]/>`_
 device or address range on a device, the filesystem should return that
Documentation/filesystems/iomap/operations.rst +2 -0
index 2c7f5df9d8b0..584ff549f9a6 100644
@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap:
 
  * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``.
 
+ * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``.
+
 Internal per-Folio State
 ------------------------
 
fs/iomap/buffered-io.c +4 -0
index d303e6c8900c..ea863c3cf510 100644
@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
 
 	if (iter->flags & IOMAP_NOWAIT)
 		fgp |= FGP_NOWAIT;
+	if (iter->flags & IOMAP_DONTCACHE)
+		fgp |= FGP_DONTCACHE;
 	fgp |= fgf_set_order(len);
 
 	return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,
 
 	if (iocb->ki_flags & IOCB_NOWAIT)
 		iter.flags |= IOMAP_NOWAIT;
+	if (iocb->ki_flags & IOCB_DONTCACHE)
+		iter.flags |= IOMAP_DONTCACHE;
 
 	while ((ret = iomap_iter(&iter, ops)) > 0)
 		iter.processed = iomap_write_iter(&iter, i);
include/linux/iomap.h +1 -1
index 75bf54e76f3b..26b0dbe23e62 100644
@@ -183,6 +183,7 @@ struct iomap_folio_ops {
 #define IOMAP_DAX		0
 #endif /* CONFIG_FS_DAX */
 #define IOMAP_ATOMIC		(1 << 9)
+#define IOMAP_DONTCACHE		(1 << 10)
 
 struct iomap_ops {
 	/*
--

2.47.2

maintainer
#5 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

Quoted 15 lines · Christian Brauner
Quoted 14 lines · Carlos Maiolino
Quoted 6 lines · Jens Axboe

Add iomap buffered write support for RWF_DONTCACHE. If RWF_DONTCACHE is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest.

[Adding Brauner to the loop as this usually goes through his tree.]

Christian, I'm pulling this into my tree for 6.15 if this is ok with you? Not sure if you're subscribed to linux-xfs, so, just in case, the link for the whole 2-patches series is below.

Quoted 8 lines
Quoted 1 line

https://lore.kernel.org/linux-xfs/[email protected]/

vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:

vfs-6.15.shared.iomap

Just pull it into your branch, please and I pull it into vfs-6.15.iomap.

Sounds good Chris. Thanks, I'll pull it once I start merging patches for 6.15 MW.

Carlos.

Quoted 82 lines

I see that 2/2 adds a new FOP_UNCACHED flag to the VFS FOP_UNCACHED please make sure to Cc fsdevel for such additions and remind contributors as well.

Quoted 77 lines
Quoted 77 lines
---
 Documentation/filesystems/iomap/design.rst     | 5 +++++
 Documentation/filesystems/iomap/operations.rst | 2 ++
 fs/iomap/buffered-io.c                         | 4 ++++
 include/linux/iomap.h                          | 1 +
 4 files changed, 12 insertions(+)
Documentation/filesystems/iomap/design.rst +5 -0
index b0d0188a095e..7b91546750f5 100644
@@ -352,6 +352,11 @@ operations:
    ``IOMAP_NOWAIT`` is often set on behalf of ``IOCB_NOWAIT`` or
    ``RWF_NOWAIT``.
 
+ * ``IOMAP_DONTCACHE`` is set when the caller wishes to perform a
+   buffered file I/O and would like the kernel to drop the pagecache
+   after the I/O completes, if it isn't already being used by another
+   thread.
+
 If it is necessary to read existing file contents from a `different
 <https://lore.kernel.org/all/[email protected]/>`_
 device or address range on a device, the filesystem should return that
Documentation/filesystems/iomap/operations.rst +2 -0
index 2c7f5df9d8b0..584ff549f9a6 100644
@@ -131,6 +131,8 @@ These ``struct kiocb`` flags are significant for buffered I/O with iomap:
 
  * ``IOCB_NOWAIT``: Turns on ``IOMAP_NOWAIT``.
 
+ * ``IOCB_DONTCACHE``: Turns on ``IOMAP_DONTCACHE``.
+
 Internal per-Folio State
 ------------------------
 
fs/iomap/buffered-io.c +4 -0
index d303e6c8900c..ea863c3cf510 100644
@@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len)
 
 	if (iter->flags & IOMAP_NOWAIT)
 		fgp |= FGP_NOWAIT;
+	if (iter->flags & IOMAP_DONTCACHE)
+		fgp |= FGP_DONTCACHE;
 	fgp |= fgf_set_order(len);
 
 	return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT,
@@ -1034,6 +1036,8 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i,
 
 	if (iocb->ki_flags & IOCB_NOWAIT)
 		iter.flags |= IOMAP_NOWAIT;
+	if (iocb->ki_flags & IOCB_DONTCACHE)
+		iter.flags |= IOMAP_DONTCACHE;
 
 	while ((ret = iomap_iter(&iter, ops)) > 0)
 		iter.processed = iomap_write_iter(&iter, i);
include/linux/iomap.h +1 -1
index 75bf54e76f3b..26b0dbe23e62 100644
@@ -183,6 +183,7 @@ struct iomap_folio_ops {
 #define IOMAP_DAX		0
 #endif /* CONFIG_FS_DAX */
 #define IOMAP_ATOMIC		(1 << 9)
+#define IOMAP_DONTCACHE		(1 << 10)
 
 struct iomap_ops {
 	/*
--

2.47.2

#6 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

In reply to Christian Brauner
Quoted 4 lines · Christian Brauner
Quoted 1 line

https://lore.kernel.org/linux-xfs/[email protected]/

vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:

Note that we'll also need the earliest changes in vfs.iomap for the zoned xfs changes that are about to be ready. So we might as well pull in the entire vfs.iomap branch.

#7 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

Quoted 9 lines · Christoph Hellwig
Quoted 4 lines · Christian Brauner
Quoted 1 line

https://lore.kernel.org/linux-xfs/[email protected]/

vfs-6.15.iomap has a lot of iomap changes already so we should use a shared tree we can both merge. I've put this on:

Note that we'll also need the earliest changes in vfs.iomap for the zoned xfs changes that are about to be ready. So we might as well pull in the entire vfs.iomap branch.

Ok, then I'll just put the series into vfs-6.15.iomap and you can just pull that in.

#8 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

In reply to Jens Axboe

include/linux/iomap.h struct iomap_folio_ops { · line 186

Quoted 7 lines · Jens Axboe
include/linux/iomap.h +1 -0
@@ -183,6 +183,7 @@ struct iomap_folio_ops {
  #define IOMAP_DAX		0
  #endif /* CONFIG_FS_DAX */
  #define IOMAP_ATOMIC		(1 << 9)
+#define IOMAP_DONTCACHE		(1 << 10)

JFYI, IOMAP_FLAGS_STRINGS might need to be updated for this, but not adding it may be intentional. I did not see anyone mention this in the reviews.

Thanks,
John

Reviewed-by on v2 1/2
#9 lore raw

Re: [PATCH 1/2] iomap: make buffered writes work with RWF_DONTCACHE

In reply to Jens Axboe
Quoted 9 lines · Jens Axboe

Add iomap buffered write support for RWF_DONTCACHE. If RWF_DONTCACHE is set for a write, mark the folios being written as uncached. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest.

---

Looks good.

-ck

author
#10 lore raw

PATCH v2 2/2 xfs: flag as supporting FOP_DONTCACHE

In reply to Jens Axboe

Read side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting FOP_DONTCACHE in the file_operations structure to enable full support for read and write uncached IO.

This provides similar benefits to using RWF_DONTCACHE with reads. Testing buffered writes on 32 files:

writing bs 65536, uncached 0
1s: 196035MB/sec
2s: 132308MB/sec
3s: 132438MB/sec
4s: 116528MB/sec
5s: 103898MB/sec
6s: 108893MB/sec
7s: 99678MB/sec
8s: 106545MB/sec
9s: 106826MB/sec
10s: 101544MB/sec
11s: 111044MB/sec
12s: 124257MB/sec
13s: 116031MB/sec
14s: 114540MB/sec
15s: 115011MB/sec
16s: 115260MB/sec
17s: 116068MB/sec
18s: 116096MB/sec

where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages.

Running the same test with uncached buffered writes:

writing bs 65536, uncached 1
1s: 198974MB/sec
2s: 189618MB/sec
3s: 193601MB/sec
4s: 188582MB/sec
5s: 193487MB/sec
6s: 188341MB/sec
7s: 194325MB/sec
8s: 188114MB/sec
9s: 192740MB/sec
10s: 189206MB/sec
11s: 193442MB/sec
12s: 189659MB/sec
13s: 191732MB/sec
14s: 190701MB/sec
15s: 191789MB/sec
16s: 191259MB/sec
17s: 190613MB/sec
18s: 191951MB/sec

and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write.

---
 fs/xfs/xfs_file.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/xfs/xfs_file.c +2 -1
index f7a7d89c345e..358987b6e2f8 100644
@@ -1626,7 +1626,8 @@ const struct file_operations xfs_file_operations = {
 	.fadvise	= xfs_file_fadvise,
 	.remap_file_range = xfs_file_remap_range,
 	.fop_flags	= FOP_MMAP_SYNC | FOP_BUFFER_RASYNC |
-			  FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE,
+			  FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE |
+			  FOP_DONTCACHE,
 };
 
 const struct file_operations xfs_dir_file_operations = {
Signature · 1 line
2.47.2
Reviewed-by on v2 2/2
#11 lore raw

Re: [PATCH 2/2] xfs: flag as supporting FOP_DONTCACHE

Quoted 55 lines · Jens Axboe

Read side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting FOP_DONTCACHE in the file_operations structure to enable full support for read and write uncached IO.

This provides similar benefits to using RWF_DONTCACHE with reads. Testing buffered writes on 32 files:

writing bs 65536, uncached 0
1s: 196035MB/sec
2s: 132308MB/sec
3s: 132438MB/sec
4s: 116528MB/sec
5s: 103898MB/sec
6s: 108893MB/sec
7s: 99678MB/sec
8s: 106545MB/sec
9s: 106826MB/sec
10s: 101544MB/sec
11s: 111044MB/sec
12s: 124257MB/sec
13s: 116031MB/sec
14s: 114540MB/sec
15s: 115011MB/sec
16s: 115260MB/sec
17s: 116068MB/sec
18s: 116096MB/sec

where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages.

Running the same test with uncached buffered writes:

writing bs 65536, uncached 1
1s: 198974MB/sec
2s: 189618MB/sec
3s: 193601MB/sec
4s: 188582MB/sec
5s: 193487MB/sec
6s: 188341MB/sec
7s: 194325MB/sec
8s: 188114MB/sec
9s: 192740MB/sec
10s: 189206MB/sec
11s: 193442MB/sec
12s: 189659MB/sec
13s: 191732MB/sec
14s: 190701MB/sec
15s: 191789MB/sec
16s: 191259MB/sec
17s: 190613MB/sec
18s: 191951MB/sec

Quoted 6 lines

and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write.

--D

Quoted 22 lines
---
 fs/xfs/xfs_file.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
fs/xfs/xfs_file.c +2 -1
index f7a7d89c345e..358987b6e2f8 100644
@@ -1626,7 +1626,8 @@ const struct file_operations xfs_file_operations = {
 	.fadvise	= xfs_file_fadvise,
 	.remap_file_range = xfs_file_remap_range,
 	.fop_flags	= FOP_MMAP_SYNC | FOP_BUFFER_RASYNC |
-			  FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE,
+			  FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE |
+			  FOP_DONTCACHE,
 };
 
 const struct file_operations xfs_dir_file_operations = {
Signature · 1 line
2.47.2
Reviewed-by on v2 2/2
#13 lore raw

Re: [PATCH 2/2] xfs: flag as supporting FOP_DONTCACHE

In reply to Jens Axboe
Quoted 55 lines · Jens Axboe

Read side was already fully supported, and with the write side appropriately punted to the worker queue, all that's needed now is setting FOP_DONTCACHE in the file_operations structure to enable full support for read and write uncached IO.

This provides similar benefits to using RWF_DONTCACHE with reads. Testing buffered writes on 32 files:

writing bs 65536, uncached 0
1s: 196035MB/sec
2s: 132308MB/sec
3s: 132438MB/sec
4s: 116528MB/sec
5s: 103898MB/sec
6s: 108893MB/sec
7s: 99678MB/sec
8s: 106545MB/sec
9s: 106826MB/sec
10s: 101544MB/sec
11s: 111044MB/sec
12s: 124257MB/sec
13s: 116031MB/sec
14s: 114540MB/sec
15s: 115011MB/sec
16s: 115260MB/sec
17s: 116068MB/sec
18s: 116096MB/sec

where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages.

Running the same test with uncached buffered writes:

writing bs 65536, uncached 1
1s: 198974MB/sec
2s: 189618MB/sec
3s: 193601MB/sec
4s: 188582MB/sec
5s: 193487MB/sec
6s: 188341MB/sec
7s: 194325MB/sec
8s: 188114MB/sec
9s: 192740MB/sec
10s: 189206MB/sec
11s: 193442MB/sec
12s: 189659MB/sec
13s: 191732MB/sec
14s: 190701MB/sec
15s: 191789MB/sec
16s: 191259MB/sec
17s: 190613MB/sec
18s: 191951MB/sec

Quoted 6 lines

and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write.

Looks good.

-ck

author
#14 lore raw
In reply to Jens Axboe
Quoted 9 lines · Jens Axboe

Hi,

Now that the main bits are in mainline, here are the XFS patches for adding RWF_DONTCACHE support. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flags FOP_DONTCACHE support in the file_operations struct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.

Patches are aginst 6.14-rc1.

Gentle ping on this one.

Signature · 1 line
Jens Axboe
author
#15 lore raw
In reply to Jens Axboe
Quoted 8 lines · Jens Axboe

Hi,

Now that the main bits are in mainline, here are the XFS patches for adding RWF_DONTCACHE support. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flags FOP_DONTCACHE support in the file_operations struct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.

Quoted 5 lines

Patches are aginst 6.14-rc1.

Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick

Is this slated for 6.15? I don't see it in the tree yet, but I also don't see a lot of other things in there. Would be a shame to miss the next release on this front.

Signature · 1 line
Jens Axboe
maintainer
#16 lore raw
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Jens Axboe

Hi,

Now that the main bits are in mainline, here are the XFS patches for adding RWF_DONTCACHE support. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flags FOP_DONTCACHE support in the file_operations struct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.

Quoted 6 lines
Quoted 5 lines

Patches are aginst 6.14-rc1.

Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick

Hello Jens

Quoted 3 lines

Is this slated for 6.15? I don't see it in the tree yet, but I also don't see a lot of other things in there. Would be a shame to miss the next release on this front.

The changes looks fine, they are for 6.15, so I don't push things to for-next for the next release while working on the current one, I'm not sure I got your point. What else are these 'lot of other things' you don't see in there you mentioned?

Carlos

Quoted 3 lines

--
Jens Axboe

author
#17 lore raw
Quoted 17 lines · Carlos Maiolino
Quoted 9 lines · Jens Axboe
Quoted 8 lines · Jens Axboe

Hi,

Now that the main bits are in mainline, here are the XFS patches for adding RWF_DONTCACHE support. It's pretty trivial - patch 1 adds the basic iomap flag and check, and patch 2 flags FOP_DONTCACHE support in the file_operations struct. Could be folded into a single patch at this point, I'll leave that up to you guys what you prefer.

Quoted 6 lines
Quoted 5 lines

Patches are aginst 6.14-rc1.

Since v1:
- Remove stale commit message paragraph
- Add comments for iomap flag from Darrick

Quoted 8 lines

Hello Jens

Quoted 3 lines

Is this slated for 6.15? I don't see it in the tree yet, but I also don't see a lot of other things in there. Would be a shame to miss the next release on this front.

The changes looks fine, they are for 6.15, so I don't push things to for-next for the next release while working on the current one, I'm

Hmm ok, that seems like an odd choice, surely you'd want things in for-next earlier rather than later? At least that's what everybody else is doing.

Quoted 2 lines

not sure I got your point. What else are these 'lot of other things' you don't see in there you mentioned?

Oh, it was just a comment on there not seeming to be any xfs changes for 6.15 in your for-next (or other branches) yet, which made me think that perhaps I was looking in the wrong spot. This where I looked, fwiw:

https://web.git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/

which does seem to have a few changes for 6.15, but really not much. Maybe this is all normal for xfs?

Signature · 1 line
Jens Axboe