fuse: allow parallel direct writes for passthrough
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest2 patches 1 replyIn review
- v32 patches 2 repliesSuperseded
- v22 patches 2 repliesSuperseded
Changes in v4 from the cover letter
- Patch 1: keep only the FOPEN_IOMODE_CACHED() helper (drop the IS_DIRECT / IS_PASSTHROUGH macros, which described how I/O is performed rather than the ff->iomode state and were misleading); fold the O_DIRECT/fcntl note into the FOPEN_PARALLEL_DIRECT_WRITES comment; and simplify fuse_file_io_open() so a pure direct-IO open needs no early return. (Per Amir's review and suggested diff.)
- Patch 2: instead of duplicating fuse_dio_lock() into passthrough.c, generalise it: fuse_io_wr_exclusive_lock() now also handles passthrough, the uncached-io refcount is factored into fuse_parallel_dio_start/end() (no-ops for passthrough's permanent uncached iomode), and fuse_dio_lock()/fuse_dio_unlock() are renamed fuse_io_wr_lock()/ fuse_io_wr_unlock() and shared by both write paths. (Per Amir's review and suggested diff.)
Patches in v4 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/2 | fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opensfs/ | +12 | 0 replies | |
| 2/2 | fuse: allow parallel direct writes in passthrough write_iterfs/ fs/ fs/ | +47 | 0 replies |
Reviews trailers per patch
None of the 2 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Amir Goldstein |
|---|---|
| Series | R-b |
| 1/2 | |
| 2/2 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v4 0/2] fuse: allow parallel direct writes for passthrough
This is v4, about six weeks after v3 [1] -- apologies for the delay. Thanks to Amir Goldstein for the detailed review of v3 and for the suggested patches; this revision implements that feedback. The v3 thread [1] has the full context and Amir's comments.
The series is also available at:
https://github.com/russfellows/linux-fuse-passthrough-patches
FUSE passthrough writes are fully serialised on the inode lock, even when the server advertises FOPEN_PARALLEL_DIRECT_WRITES and the write is a direct, non-append, within-EOF overwrite. Native FUSE direct-IO writes already take a shared lock in that case; there is no reason passthrough cannot do the same. With a passthrough backing file on XFS, this is the difference between serialised and fully parallel 4K random-write O_DIRECT throughput.
Patch 1 stops fuse_file_io_open() from stripping FOPEN_PARALLEL_DIRECT_WRITES on passthrough opens (which bypass the page cache without FOPEN_DIRECT_IO). Patch 2 lets fuse_passthrough_write_iter() take the shared lock for eligible writes by sharing the existing direct-IO write-lock helper rather than duplicating it.
Rebased onto v6.18.39 (LTS).
Changes since v3:
- Patch 1: keep only the FOPEN_IOMODE_CACHED() helper (drop the IS_DIRECT / IS_PASSTHROUGH macros, which described how I/O is performed rather than the ff->iomode state and were misleading); fold the O_DIRECT/fcntl note into the FOPEN_PARALLEL_DIRECT_WRITES comment; and simplify fuse_file_io_open() so a pure direct-IO open needs no early return. (Per Amir's review and suggested diff.)
- Patch 2: instead of duplicating fuse_dio_lock() into passthrough.c, generalise it: fuse_io_wr_exclusive_lock() now also handles passthrough, the uncached-io refcount is factored into fuse_parallel_dio_start/end() (no-ops for passthrough's permanent uncached iomode), and fuse_dio_lock()/fuse_dio_unlock() are renamed fuse_io_wr_lock()/ fuse_io_wr_unlock() and shared by both write paths. (Per Amir's review and suggested diff.)
Testing (fstests / xfstests)
----------------------------
Run with the libfuse passthrough_hp server under xfstests (FSTYP=fuse), './check -fuse -g quick' (642 tests), on v6.18.39, backing XFS on a RAM-backed null_blk device. An unpatched v6.18.39 baseline was captured first, then the same tree with this series applied; results were diffed to isolate any patch-induced change. Two server configs were exercised, as suggested by Amir:
passthrough (default): unpatched 25 failed / patched 25 failed
passthrough --direct-io: unpatched 27 failed / patched 27 failed
In both configs the failing set is identical with and without the series (the --direct-io set is the default set plus generic/355 and generic/362). Every failure is pre-existing on the unpatched kernel and is a known passthrough_hp / fuse limitation (permission/ACL/atime/mmap-coherency tests), unrelated to this change:
=> zero regressions introduced by the series in either config.
No kernel warnings, lockdep reports, or splats were observed in any run.
Performance
-----------
Custom FUSE passthrough daemon advertising FOPEN_PASSTHROUGH | FOPEN_PARALLEL_DIRECT_WRITES, XFS on a RAM-backed null_blk device, kernel v6.18.39. fio 4K random write, O_DIRECT, libaio, iodepth=64, single shared file:
numjobs | unpatched (IOPS) | patched (IOPS)
--------|------------------|---------------
1 | 352,000 | 352,000
2 | 440,000 | 648,000
3 | 429,000 | 951,000
4 | 423,000 | 987,000Unpatched write throughput is flat regardless of job count (the exclusive inode lock serialises all writers to the file); with the series it scales with concurrency, ~2.3x at 4 jobs, up to the backing device ceiling. Single-job is unchanged (no lock contention), and 4K random *read* throughput is unchanged (~1.37M IOPS with and without the series), since reads never took the exclusive lock.
[1] v3: https://lore.kernel.org/all/[email protected]/
Russ Fellows (2):
fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens
fuse: allow parallel direct writes in passthrough write_iter
fs/fuse/file.c | 53 ++++++++++++++++++++++++++++++++++--------- fs/fuse/fuse_i.h | 2 ++ fs/fuse/iomode.c | 23 ++++++++++--------- fs/fuse/passthrough.c | 6 ++--- 4 files changed, 59 insertions(+), 25 deletions(-)
--
2.51.0