RFC Add buffered write-through support to iomap & xfs

This is the next revision of writethrough series. I'll quote part of the original cover:

Hi all,

This patchset implements an early design prototype of buffered I/O write-through semantics in linux.

This idea mainly picked up traction to enable RWF_ATOMIC buffered IO, however write-through path can have many use cases beyond atomic writes,
- such as enabling truly async AIO buffered I/O when issued with O_DSYNC
- better scalability for buffered I/O

==============================================
** Changes since rfc v2 ** [1]
==============================================

1. Refined the error handling semantics:
----------------------------------------
Writethrough has 2 steps - memcpy to folio & IO to disk. Only the part of write completely submitted to disk is considered successful. With that in mind we have 2 types of errors. Suppose we have 16K write and we encounter an error after 4k:
a) If have only memcpy'd the 4k and we are able to submit it for IO, then we can return 4k as a short write, since everything is consistent.

b) If we have memcpy'd the whole 16k but faced and error (like EIO) after the first 4k write, then we return the error to user. Since the page cache is in inconsistent state, the user must treat this as a buffered IO fsync failure.

2. Introduce RWF_NOSERIAL to allow parallel writes
--------------------------------------------------
As per our discussions in LSFMM 2026, we noticed that writethrough (v2 design) suffered from a big regression (~65%) in workloads with multiple writers writing to a single file. This is because writethrough submits IO within the inode lock and since buffered IO has an exclusive lock in write, this hurt massively. In LSFMM, we discussed 3 approaches:
a) Defer IO submission outside inode lock
b) Avoid folio dirty - clear cycle
c) Shared lock for writes

We tried a) by only staging prepared folios in a list under inode lock and then submitting them outside. However, the initial implementation still showed around ~30% regression even though the code complexity was significantly higher. As the ROI was not worth it, we dropped this idea (if there is interest I can share a github link for these patches). So we finally decided to go with b) and c).

With b), we can avoid cycling folio through dirty and clear as we immediately submit the IO. We also don't use the heavy folio_start/end_writeback() and instead only use the writeback master bit on folio. This cuts back significantly on xa_lock contention.

With c), the NOSERIAL flag allows us to do writes under a shared lock if possible. This violates guarantees that XFS has historically provided however its expected to be an advanced feature that should be used by applications who know what they are doing. If used correctly, it can give a very good performance boost to parallel workloads. This is something that was also discussed at LSFMM 2026 [3] and since WRITETHROUGH is a new path it seems like a good time to introduce such a semantic change . For now we auto apply NOSERIAL for writethrough and disallow users to pass it till we finalize the semantics.

With b & c, we are able to see a good performance improvement in almost all of the cases that we regressing. More details and performance numbers specific to the NOSERIAL flag can be found in the respective patches.

3. Use REQ_SYNC | REQ_IDLE like dio
-----------------------------------
Writethrough doesn't go via the writeback mechanism and it's IO characteristics are similar to dio. Hence we pass REQ_SYNC | REQ_IDLE in the bio, just like dio, which allows us to bypass writeback throttling.

4. Move inode i_size update from completion path to write path
--------------------------------------------------------------
As per Dave's suggestion we originally wanted to update isize in completion like dio however this resulted in a big regression in extending IO because we end up holding the exclusive lock throughout the IO. To avoid this, we can just take the buffered IO approach of updating i_size in write path so that we can safely drop the inode lock and allow completion to finish outside the lock. This brings back the append IO performance in par with buffered IO.

5. There's a deadock in v2 that is fixed in the last patch. If needed this can be squashed in but I've kept it separate for now for easier review.

6. Added a patch to make stable writes more compatible to our usage model. Check patch 1 for details. Thanks to Darrick for suggesting this!.

7. Addressed reviews from Jan and Sashiko (thanks).

===================================================================
Performance Comparison Tables (with fio snippet)
===================================================================

Table 1: Extending writes using (libaio + O_DSYNC) - All writers on single file

+----------+-------------------+---------------------+
| numjobs  | Buffered IO       | Writethrough IO     |
+----------+-------------------+---------------------+
|    1     | 133 MiB/s         | 133 MiB/s (+0.0%)   |
|    2     | 179 MiB/s         | 243 MiB/s (+35.8%)  |
|    4     | 253 MiB/s         | 358 MiB/s (+41.5%)  |
|    8     | 366 MiB/s         | 376 MiB/s (+2.7%)   |
|   16     | 474 MiB/s         | 449 MiB/s (-5.3%)   |
+----------+-------------------+---------------------+
(fio --ioengine=libaio --writethrough=0/1 --bs=4k --rw=write \
     --iodepth=32 --sync=dsync--file_append=1)

Table 2: Random pure overwrites (libaio + O_DSYNC) - All writers on single file

+----------+-------------------+---------------------+
| numjobs  | Buffered IO       | Writethrough IO     |
+----------+-------------------+---------------------+
|    1     | 131 MiB/s         | 391 MiB/s (+198.5%) |
|    2     | 377 MiB/s         | 791 MiB/s (+109.8%) |
|    4     | 695 MiB/s         | 1591 MiB/s (+128.9%)|
|    8     | 1217 MiB/s        | 1846 MiB/s (+51.7%) |
|   16     | 1197 MiB/s        | 1844 MiB/s (+54.1%) |
+----------+-------------------+---------------------+
(fio --ioengine=libaio --bs=4k --size=2G --sync=dsync --writethrough=0/1 \
    --overwrite=1--rw=randwrite --iodepth=32)

Table 3: Random pure overwrites (libaio + O_DSYNC) - Each write writes own file

+----------+-------------------+---------------------+
| numjobs  | Buffered IO       | Writethrough IO     |
+----------+-------------------+---------------------+
|    1     | 187 MiB/s         | 389 MiB/s (+108.0%) |
|    2     | 373 MiB/s         | 781 MiB/s (+109.4%) |
|    4     | 706 MiB/s         | 1568 MiB/s (+122.1%)|
|    8     | 1222 MiB/s        | 1796 MiB/s (+47.0%) |
+----------+-------------------+---------------------+
(fio --ioengine=libaio --bs=4k --size=2G --filename_format=.test_file.\$jobnum \
     --sync=dsync --writethrough=0/1 --overwrite=1 --rw=randwrite --iodepth=32)

Table 4: Random 16KB overwrites with sync_file_range:16 - Single file (Roughly mimics postgresql IO pattern)

+----------+-------------------+---------------------+
| numjobs  | Buffered IO       | Writethrough IO     |
+----------+-------------------+---------------------+
|    1     | 1323 MiB/s        | 1275 MiB/s (-3.6%)  |
|    2     | 1779 MiB/s        | 2434 MiB/s (+36.8%) |
|    4     | 2092 MiB/s        | 2519 MiB/s (+20.4%) |
|    8     | 2272 MiB/s        | 2517 MiB/s (+10.8%) |
|   16     | 2328 MiB/s        | 2525 MiB/s (+8.5%)  |
+----------+-------------------+---------------------+
(fio --ioengine=libaio --bs=16k --size=5G --writethrough=0/1 --overwrite=1\
     --sync_file_range=wait_before,write:16 --rw=randwrite --iodepth=32)

* Environment details *

CPU : IBM Power 11 LPAR
Memory : 62Gi
Storage : Samsung PM173-series Enterprise NVMe SSD
Kernel : Linux 7.2-rc1
Filesystem : XFS (4k block size)

[1] https://lore.kernel.org/linux-xfs/[email protected]/
[2] https://github.com/OjaswinM/xfstests/tree/iomap-buf-writethrough2
[3] https://lwn.net/Articles/1072019

As usual, thoughts and suggestions are welcome :)

Regards,
ojaswin

Ojaswin Mujoo (11):
fs: Add counter to track inflight writes that need stable pages
mm: Refactor folio_clear_dirty_for_io()
iomap: Add helper to revert iomap iter
iomap: Add initial support for buffered RWF_WRITETHROUGH
xfs: Add RWF_WRITETHROUGH support to xfs
iomap: Add aio support to RWF_WRITETHROUGH
iomap: Add DSYNC support to RWF_WRITETHROUGH
fs: Introduce RWF_NOSERIAL flag to indicate parallel reads/writes
xfs: Implement RWF_NOSERIAL to parallelize RWF_WRITETHROUGH writes
iomap: Avoid folio dirtying in case of RWF_WRITETHROUGH
iomap: Handle deadlock due to repeating folios in RWF_WRITETHROUGH

 fs/iomap/buffered-io.c  | 570 +++++++++++++++++++++++++++++++++++++++-
 fs/iomap/iter.c         |   9 +
 fs/xfs/xfs_file.c       | 135 +++++++++-
 include/linux/fs.h      |  29 ++
 include/linux/iomap.h   |  50 ++++
 include/linux/pagemap.h |  16 +-
 include/uapi/linux/fs.h |   9 +-
 mm/filemap.c            |  22 ++
 mm/page-writeback.c     |  49 +++-
 9 files changed, 861 insertions(+), 28 deletions(-)

--
2.55.0