fuse: fix passthrough parallel direct writes
Subsystems (3)
v3 cover fuse: fix passthrough parallel direct writes
This series fixes FOPEN_PARALLEL_DIRECT_WRITES being silently ignored for FUSE passthrough opens.
Changes since v2:
- Patch 1: introduce FOPEN_IOMODE helper macros (FOPEN_IOMODE_IS_CACHED, FOPEN_IOMODE_IS_DIRECT, FOPEN_IOMODE_IS_PASSTHROUGH) to make the flag conditions self-documenting. Use FOPEN_IOMODE_IS_CACHED for the PARALLEL_DIRECT_WRITES guard. No behavioral change.
- Patch 2: replace open-coded lock/unlock with fuse_passthrough_lock() and fuse_passthrough_unlock() helpers local to passthrough.c. Add a post-lock re-check of the past-EOF condition to close the TOCTOU window between the initial check and acquiring the shared inode lock. Restore fuse_dio_lock()/fuse_dio_unlock() to file-private (static) and remove their declarations from fuse_i.h.
Patch 1 preserves FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens. fuse_file_io_open() stripped the flag for any open lacking FOPEN_DIRECT_IO. That rule is correct for the regular direct-IO path but wrong for passthrough:
passthrough already bypasses the page cache through the backing file without needing FOPEN_DIRECT_IO. The new FOPEN_IOMODE_IS_CACHED guard expresses the correct invariant: suppress the flag only for cached (page-cache) I/O mode.
Patch 2 makes fuse_passthrough_write_iter() respect FOPEN_PARALLEL_DIRECT_WRITES. Previously the function held the exclusive inode lock unconditionally, ignoring the flag entirely. The new fuse_passthrough_lock() allows shared inode locking only for direct within-EOF non-append overwrites, and serializes everything else with an exclusive lock. A past-EOF re-check after taking the shared lock closes the TOCTOU race where a concurrent EOF-extending write could slip in between the initial check and the lock acquisition.
Passthrough files are always in uncached iomode (committed at open time via fuse_file_uncached_io_open()), so the fuse_inode_uncached_io_start() guard from fuse_dio_lock() is not needed here.
Performance
-----------
Tested on kernel 6.17.13-p4min-v3, custom FUSE daemon with FOPEN_PASSTHROUGH | FOPEN_PARALLEL_DIRECT_WRITES, XFS on a RAM-backed null_blk device. fio randwrite 4K direct, iodepth=64:
numjobs | FUSE IOPS | Raw XFS IOPS
--------|------------|-------------
1 | 478,564 | 528,835
2 | 941,483 | ~1.0M *
4 | 1,457,284 | ~1.5M *
8 | 1,675,749 | 1,693,406* Raw XFS numjobs=2,4 not re-measured on this kernel; numjobs=1 and 8 directly measured. FUSE at numjobs=8 reaches 99% of raw XFS throughput, confirming the shared-lock path is fully active and passthrough overhead is negligible.
Correctness Testing
-------------------
All seven cases were run on kernel 6.17.13-p4min-v3+ with a custom FUSE daemon advertising FOPEN_PASSTHROUGH | FOPEN_PARALLEL_DIRECT_WRITES, backed by XFS on a RAM-backed null_blk device (8 GiB). Each case targets a distinct decision branch in fuse_passthrough_lock() and the surrounding write logic. The test harness verifies both completion status and exact final file size.
overwrite
Setup: Pre-seed a 1 GiB file via single-job sequential write.
Workload: 4 concurrent fio jobs, 4K direct randwrite over the full 1 GiB,
CRC32C verify on every block, runtime=30s.
Lock path: FOPEN_PARALLEL_DIRECT_WRITES set + IOCB_DIRECT + no IOCB_APPEND
+ pos+len <= i_size -> shared inode lock (TOCTOU re-check passes).
Validates: Core fix. Parallel passthrough writes produce correct data under
the shared lock; no corruption at any numjobs.
Result: PASS (zero CRC verify errors; final size 1 GiB)
read_write
Setup: Pre-seed a 1 GiB file.
Workload: 2 fio writer jobs on disjoint 512 MiB regions (CRC32C verify)
running simultaneously with 2 fio reader jobs roaming the whole file, all concurrent for 30s.
Lock path: Writers take shared inode lock; fuse_passthrough_read_iter()
takes no inode lock.
Validates: Shared-lock writers coexist correctly with concurrent unlocked
readers. A deadlock or lock regression would stall the workload; a corruption bug would trip the per-block CRC verify.
Result: PASS (zero verify errors; no stall; final size 1 GiB)
append
Setup: No pre-existing file.
Workload: 4 fio jobs, 4K direct write with --file_append=1 (pwrite() with
userspace-managed offsets into a pre-fallocated region); each job writes 4 MiB; no time_based.
Lock path: Writes land within the pre-allocated EOF -> shared inode lock
(same within-EOF path as overwrite).
Validates: Multi-job pwrite-based appends within a pre-allocated extent
complete without data loss or size mismatch.
Result: PASS (final size 16 MiB = 4 jobs x 4 MiB)
o_append
Setup: No pre-existing file.
Workload: 4 concurrent dd processes each open the file O_APPEND and issue
16 x 4 MiB writes. True O_APPEND: the kernel must atomically advance EOF before each write.
Lock path: O_APPEND sets IOCB_APPEND -> fuse_passthrough_write_needs_exclusive()
returns true -> exclusive inode lock unconditionally.
Validates: IOCB_APPEND branch of fuse_passthrough_lock() serializes correctly.
No two writers overlap; all 64 blocks land at unique offsets; O_APPEND atomicity is preserved through the passthrough path.
Result: PASS (final size 256 MiB = 4 x 16 x 4 MiB)
extend
Setup: truncate file to 64 MiB (sparse).
Workload: 4 fio jobs each write 64 MiB to a disjoint region starting beyond
the initial EOF (offsets 64, 128, 192, 256 MiB); 1M blocks; direct I/O; no time_based.
Lock path: Every write has pos+len > i_size -> the post-lock past-EOF re-check
in fuse_passthrough_lock() forces an upgrade to exclusive lock.
Validates: Exclusive lock is taken for all past-EOF writes; disjoint regions
are written correctly; i_size advances to the correct final value.
Result: PASS (final size 320 MiB = 64 MiB initial + 4 x 64 MiB)
extend_race
Setup: truncate file to 64 MiB (sparse).
Workload: 4 fio jobs all write the identical 64 MiB region [64 MiB, 128 MiB)
simultaneously; last writer wins per block.
Lock path: All writes have pos+len > i_size -> exclusive lock required for
all. The post-lock TOCTOU re-check is the key safety net: a job that passed the pre-lock past-EOF check and then lost a race with another writer must still upgrade to exclusive before proceeding.
Validates: TOCTOU re-check correctness; no deadlock among N simultaneous
past-EOF writers; i_size is consistent after all jobs complete.
Result: PASS (final size 128 MiB = 64 MiB + 64 MiB; no deadlock)
buffered
Setup: No pre-existing file.
Workload: 4 fio jobs, 4K buffered (non-O_DIRECT) randwrite over 1 GiB,
time_based 30s.
Lock path: fuse_file_write_iter() routes buffered writes to
fuse_cache_write_iter(), bypassing fuse_passthrough_write_iter() entirely. This patch is not in the call path.
Validates: Non-passthrough write path is completely unaffected by these changes.
Result: PASS (final size 1 GiB)
Russ Fellows (2):
fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens
fuse: allow parallel direct writes in passthrough write_iter
fs/fuse/file.c | 4 +-- fs/fuse/fuse_i.h | 2 -- fs/fuse/iomode.c | 30 ++++++++++++++++---- fs/fuse/passthrough.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 106 insertions(+), 12 deletions(-)
Signature · 1 line
2.51.0
PATCH v3 1/2 fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens
fuse_file_io_open() clears FOPEN_PARALLEL_DIRECT_WRITES for any open that lacks FOPEN_DIRECT_IO. That rule is too strict for passthrough opens, which already bypass the page cache through the backing file and do not need FOPEN_DIRECT_IO to guarantee direct-I/O semantics. Clearing the flag before the passthrough write path sees it prevents the kernel from ever taking the shared-lock path for passthrough writes.
Introduce FOPEN_IOMODE helper macros to classify the effective I/O mode of an open file:
FOPEN_IOMODE_IS_CACHED(oflags) - page-cache I/O (neither flag)
FOPEN_IOMODE_IS_DIRECT(oflags) - DIRECT_IO is set
FOPEN_IOMODE_IS_PASSTHROUGH(oflags) - PASSTHROUGH only (no DIRECT_IO)
Use FOPEN_IOMODE_IS_CACHED to express the PARALLEL_DIRECT_WRITES guard
cleanly: the flag is only suppressed for cached iomode, not for passthrough or direct-IO mode.
This is a prerequisite for passthrough write parallelism; without it the shared-lock path in the subsequent patch never activates.
--- fs/fuse/iomode.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-)
fs/fuse/iomode.c +27 -3
@@ -165,6 +165,23 @@ static void fuse_file_uncached_io_release(struct fuse_file *ff, (FOPEN_PASSTHROUGH |FOPEN_DIRECT_IO |FOPEN_PARALLEL_DIRECT_WRITES | \FOPEN_NOFLUSH ) +/* +* Helpers to classify the effective I/O mode of an open file. +* +* FOPEN_IOMODE() - extract the mode-relevant bits +* FOPEN_IOMODE_IS_CACHED - neither DIRECT_IO nor PASSTHROUGH: page-cache I/O +* FOPEN_IOMODE_IS_DIRECT - DIRECT_IO is set (may also have PASSTHROUGH) +* FOPEN_IOMODE_IS_PASSTHROUGH - PASSTHROUGH only, without DIRECT_IO + */ +#define FOPEN_IOMODE (oflags) \ + ((oflags) & (FOPEN_DIRECT_IO | FOPEN_PASSTHROUGH)) +#define FOPEN_IOMODE_IS_CACHED (oflags) \ + (FOPEN_IOMODE(oflags) == 0) +#define FOPEN_IOMODE_IS_DIRECT (oflags) \ + (FOPEN_IOMODE(oflags) & FOPEN_DIRECT_IO) +#define FOPEN_IOMODE_IS_PASSTHROUGH (oflags) \ + (FOPEN_IOMODE(oflags) == FOPEN_PASSTHROUGH) +static int fuse_file_passthrough_open (struct inode *inode,struct file *file) {struct fuse_file *ff = file->private_data;
@@ -216,9 +233,11 @@ int fuse_file_io_open(struct file *file, struct inode *inode) goto fail;/* -* FOPEN_PARALLEL_DIRECT_WRITES requires FOPEN_DIRECT_IO. +* FOPEN_PARALLEL_DIRECT_WRITES is not supported with cached iomode. +* Passthrough and direct-IO both bypass the page cache, so both are +* eligible to enable parallel direct writes. */ -if (!(ff->open_flags &FOPEN_DIRECT_IO )) +if (FOPEN_IOMODE_IS_CACHED (ff->open_flags)) ff->open_flags &= ~FOPEN_PARALLEL_DIRECT_WRITES ;/*
@@ -228,8 +247,13 @@ int fuse_file_io_open(struct file *file, struct inode *inode)* Note that if user opens a file open with O_DIRECT, but server did * not specify FOPEN_DIRECT_IO, a later fcntl() could remove O_DIRECT, * so we put the inode in caching mode to prevent parallel dio. +* +* Pure direct-IO (DIRECT_IO without PASSTHROUGH) needs no page-cache +* iomode machinery — return early. When DIRECT_IO and PASSTHROUGH are +* both set, write_iter uses the direct-IO path but the backing file +* still needs to be opened below. */ -if ((ff->open_flags &FOPEN_DIRECT_IO ) && +if (FOPEN_IOMODE_IS_DIRECT (ff->open_flags) && !(ff->open_flags &FOPEN_PASSTHROUGH ))return 0 ;
Signature · 1 line
2.51.0
Amir Goldstein
maintainerRe: [PATCH v3 1/2] fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens
Quoted 42 lines · Russ Fellows
fuse_file_io_open()clearsFOPEN_PARALLEL_DIRECT_WRITESfor any open that lacksFOPEN_DIRECT_IO. That rule is too strict for passthrough opens, which already bypass the page cache through the backing file and do not needFOPEN_DIRECT_IOto guarantee direct-I/O semantics. Clearing the flag before the passthrough write path sees it prevents the kernel from ever taking the shared-lock path for passthrough writes.Introduce
FOPEN_IOMODEhelper macros to classify the effective I/O mode of an open file:
FOPEN_IOMODE_IS_CACHED(oflags)- page-cache I/O (neither flag)
FOPEN_IOMODE_IS_DIRECT(oflags)-DIRECT_IOis set
FOPEN_IOMODE_IS_PASSTHROUGH(oflags)- PASSTHROUGH only (noDIRECT_IO)Use
FOPEN_IOMODE_IS_CACHEDto express thePARALLEL_DIRECT_WRITESguard
cleanly: the flag is only suppressed for cached iomode, not for passthrough or direct-IO mode.This is a prerequisite for passthrough write parallelism; without it the shared-lock path in the subsequent patch never activates.
Signed-off-by Russ Fellows <[email protected]>--- fs/fuse/iomode.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-)fs/fuse/iomode.c +7
-0@@ -165,6 +165,23 @@ static void fuse_file_uncached_io_release(struct fuse_file *ff, (FOPEN_PASSTHROUGH |FOPEN_DIRECT_IO |FOPEN_PARALLEL_DIRECT_WRITES | \FOPEN_NOFLUSH ) +/* +* Helpers to classify the effective I/O mode of an open file. +* +* FOPEN_IOMODE() - extract the mode-relevant bits +* FOPEN_IOMODE_IS_CACHED - neither DIRECT_IO nor PASSTHROUGH: page-cache I/O +* FOPEN_IOMODE_IS_DIRECT - DIRECT_IO is set (may also have PASSTHROUGH) +* FOPEN_IOMODE_IS_PASSTHROUGH - PASSTHROUGH only, without DIRECT_IO
fs/fuse/iomode.c static void fuse_file_uncached_io_release(struct fuse_file *ff, · line 184
Quoted 10 lines
fs/fuse/iomode.c +10
-0+*/ +#define FOPEN_IOMODE (oflags) \ + ((oflags) & (FOPEN_DIRECT_IO | FOPEN_PASSTHROUGH)) +#define FOPEN_IOMODE_IS_CACHED (oflags) \ + (FOPEN_IOMODE(oflags) == 0) +#define FOPEN_IOMODE_IS_DIRECT (oflags) \ + (FOPEN_IOMODE(oflags) & FOPEN_DIRECT_IO) +#define FOPEN_IOMODE_IS_PASSTHROUGH (oflags) \ + (FOPEN_IOMODE(oflags) == FOPEN_PASSTHROUGH) +
Sorry, mad bad I gave you wrong code samples.
The open iomode is not about how the io is performed direct/passthrough but it is about the state ff->iomode, so the FOPEN_IOMODE_IS_DIRECT() is misleading and it does not make the patch simpler which was the intention. Better loose all except FOPEN_IOMODE_CACHED() to avoid ambiguity.
Quoted 18 lines
fs/fuse/iomode.c +4
-2static int fuse_file_passthrough_open (struct inode *inode,struct file *file) {struct fuse_file *ff = file->private_data;@@ -216,9 +233,11 @@ int fuse_file_io_open(struct file *file, struct inode *inode) goto fail;/* -* FOPEN_PARALLEL_DIRECT_WRITES requires FOPEN_DIRECT_IO. +* FOPEN_PARALLEL_DIRECT_WRITES is not supported with cached iomode. +* Passthrough and direct-IO both bypass the page cache, so both are +* eligible to enable parallel direct writes. */ -if (!(ff->open_flags &FOPEN_DIRECT_IO )) +if (FOPEN_IOMODE_IS_CACHED (ff->open_flags)) ff->open_flags &= ~FOPEN_PARALLEL_DIRECT_WRITES ;/*
fs/fuse/iomode.c int fuse_file_io_open(struct file *file, struct inode *inode) · line 249
Quoted 3 lines
fs/fuse/iomode.c +0
-0@@ -228,8 +247,13 @@ int fuse_file_io_open(struct file *file, struct inode *inode)* Note that if user opens a file open with O_DIRECT, but server did * not specify FOPEN_DIRECT_IO, a later fcntl() could remove O_DIRECT, * so we put the inode in caching mode to prevent parallel dio.
Merge this "Note" into the comment above about FOPEN_PARALLEL_DIRECT_WRITES. It is out of context here.
fs/fuse/iomode.c int fuse_file_io_open(struct file *file, struct inode *inode) · line 254
Quoted 5 lines
fs/fuse/iomode.c +5
-0+* +* Pure direct-IO (DIRECT_IO without PASSTHROUGH) needs no page-cache +* iomode machinery — return early. When DIRECT_IO and PASSTHROUGH are +* both set, write_iter uses the direct-IO path but the backing file +* still needs to be opened below.
Loose this text, code is self explanatory
fs/fuse/iomode.c int fuse_file_io_open(struct file *file, struct inode *inode) · line 258
Quoted 5 lines
fs/fuse/iomode.c +1
-1*/ -if ((ff->open_flags &FOPEN_DIRECT_IO ) && +if (FOPEN_IOMODE_IS_DIRECT (ff->open_flags) && !(ff->open_flags &FOPEN_PASSTHROUGH ))return 0 ;
Sigh, this function is a mess, see my untested patch below to make it simpler
Thanks,
Amir.
fs/fuse/iomode.c +0 -0
@@ -192,12 +192,18 @@ static int fuse_file_passthrough_open(struct
inode *inode, struct file *file)
return err;
}
+/* Fuse uses page cache if no passthrough nor direct_io open specified */ +#define FOPEN_IOMODE(oflags) \
+ ((oflags) & (FOPEN_DIRECT_IO | FOPEN_PASSTHROUGH))
+#define FOPEN_IOMODE_CACHED(oflags) \
+ (FOPEN_IOMODE(oflags) == 0)
+
/* Request access to submit new io to inode via open file */ int fuse_file_io_open(struct file *file, struct inode *inode)
{
struct fuse_file *ff = file->private_data;
struct fuse_inode *fi = get_fuse_inode(inode);- int err;
+ int err = 0;
/*
* io modes are not relevant with DAX and with server that does not
@@ -216,25 +222,20 @@ int fuse_file_io_open(struct file *file, struct inode *inode)
/*
* FOPEN_PARALLEL_DIRECT_WRITES requires FOPEN_DIRECT_IO.
+ * Note that if user opens a file open with O_DIRECT, but server did
+ * not specify FOPEN_DIRECT_IO, a later fcntl() could remove O_DIRECT,
+ * so we put the inode in caching mode to prevent parallel dio.
*/- if (!(ff->open_flags & FOPEN_DIRECT_IO))
+ if (FOPEN_IOMODE_CACHED(ff->open_flags))
ff->open_flags &= ~FOPEN_PARALLEL_DIRECT_WRITES;
/*
* First passthrough file open denies caching inode io mode.
* First caching file open enters caching inode io mode.
- *
- * Note that if user opens a file open with O_DIRECT, but server did
- * not specify FOPEN_DIRECT_IO, a later fcntl() could remove O_DIRECT,
- * so we put the inode in caching mode to prevent parallel dio.
*/- if ((ff->open_flags & FOPEN_DIRECT_IO) &&
- !(ff->open_flags & FOPEN_PASSTHROUGH))
- return 0;
-
if (ff->open_flags & FOPEN_PASSTHROUGH)
err = fuse_file_passthrough_open(inode, file);
- else
+ else if (FOPEN_IOMODE_CACHED(ff->open_flags))
err = fuse_file_cached_io_open(inode, ff);
if (err)
goto fail;PATCH v3 2/2 fuse: allow parallel direct writes in passthrough write_iter
In reply to Russ Fellowsfuse_passthrough_write_iter() unconditionally called fuse_dio_lock() from file.c, which required those helpers to be exported. That coupling is unnecessary and the exported symbols are undesirable.
Replace the fuse_dio_lock()/fuse_dio_unlock() calls with a passthrough-specific pair, fuse_passthrough_lock() and fuse_passthrough_unlock(), that is self-contained in passthrough.c.
The new fuse_passthrough_lock() allows shared inode locking only when all of the following are true:
- the open carries FOPEN_PARALLEL_DIRECT_WRITES
- the write is direct I/O (IOCB_DIRECT)
- the write is not append (IOCB_APPEND absent)
- the write does not extend past EOF
The past-EOF check is made once before the lock is taken (fast path to choose lock type), and then re-checked after taking the shared lock. This re-check closes the TOCTOU window: a concurrent writer could extend EOF between the initial check and the lock acquisition; without the re-check a shared-lock writer could concurrently update i_size.
Passthrough files are always in uncached iomode (established at open time via fuse_file_uncached_io_open()), so the fuse_inode_uncached_io_start() guard from fuse_dio_lock() is not needed here.
Restore fuse_dio_lock() and fuse_dio_unlock() to file-private (static). Remove their declarations from fuse_i.h.
--- fs/fuse/file.c | 6 ++-- fs/fuse/fuse_i.h | 2 -- fs/fuse/passthrough.c | 74 +++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 75 insertions(+), 7 deletions(-)
fs/fuse/file.c +3 -3
@@ -1366,8 +1366,8 @@ static bool fuse_dio_wr_exclusive_lock(struct kiocb *iocb, struct iov_iter *fromreturn false; } -void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from, -bool *exclusive) +static void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from, +bool *exclusive) {struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode);
@@ -1393,7 +1393,7 @@ void fuse_dio_lock(struct kiocb *iocb, struct iov_iter *from, } } -void fuse_dio_unlock (struct kiocb *iocb,bool exclusive) +static void fuse_dio_unlock (struct kiocb *iocb,bool exclusive) {struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode);
fs/fuse/fuse_i.h +0 -2
@@ -1507,8 +1507,6 @@ int fuse_file_io_open(struct file *file, struct inode *inode);void fuse_file_io_release (struct fuse_file *ff,struct inode *inode);/* file.c */ -void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from,bool *exclusive); -void fuse_dio_unlock (struct kiocb *iocb,bool exclusive);struct fuse_file *fuse_file_open (struct fuse_mount *fm,u64 nodeid,unsigned int open_flags,bool isdir); void fuse_file_release(struct inode *inode, struct fuse_file *ff,
fs/fuse/passthrough.c +72 -2
88 lines not on this page · raw message
Signature · 1 line
2.51.0
Amir Goldstein
maintainerRe: [PATCH v3 2/2] fuse: allow parallel direct writes in passthrough write_iter
Quoted 142 lines · Russ Fellows
fuse_passthrough_write_iter()unconditionally calledfuse_dio_lock()fromfile.c, which required those helpers to be exported. That coupling is unnecessary and the exported symbols are undesirable.Replace the fuse_dio_lock()/fuse_dio_unlock() calls with a passthrough-specific pair,
fuse_passthrough_lock()andfuse_passthrough_unlock(), that is self-contained inpassthrough.c.The new
fuse_passthrough_lock()allows shared inode locking only when all of the following are true:- the open carries
FOPEN_PARALLEL_DIRECT_WRITES
- the write is direct I/O (IOCB_DIRECT)
- the write is not append (IOCB_APPENDabsent)
- the write does not extend past EOFThe past-EOF check is made once before the lock is taken (fast path to choose lock type), and then re-checked after taking the shared lock. This re-check closes the TOCTOU window: a concurrent writer could extend EOF between the initial check and the lock acquisition; without the re-check a shared-lock writer could concurrently update
i_size.Passthrough files are always in uncached iomode (established at open time via
fuse_file_uncached_io_open()), so thefuse_inode_uncached_io_start()guard fromfuse_dio_lock()is not needed here.Restore
fuse_dio_lock()andfuse_dio_unlock()to file-private (static). Remove their declarations fromfuse_i.h.Signed-off-by Russ Fellows <[email protected]>--- fs/fuse/file.c | 6 ++-- fs/fuse/fuse_i.h | 2 -- fs/fuse/passthrough.c | 74 +++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 75 insertions(+), 7 deletions(-)fs/fuse/file.c +3
-3@@ -1366,8 +1366,8 @@ static bool fuse_dio_wr_exclusive_lock(struct kiocb *iocb, struct iov_iter *fromreturn false; } -void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from, -bool *exclusive) +static void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from, +bool *exclusive) {struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode);@@ -1393,7 +1393,7 @@ void fuse_dio_lock(struct kiocb *iocb, struct iov_iter *from, } } -void fuse_dio_unlock (struct kiocb *iocb,bool exclusive) +static void fuse_dio_unlock (struct kiocb *iocb,bool exclusive) {struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode);fs/fuse/fuse_i.h +0
-2@@ -1507,8 +1507,6 @@ int fuse_file_io_open(struct file *file, struct inode *inode);void fuse_file_io_release (struct fuse_file *ff,struct inode *inode);/* file.c */ -void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from,bool *exclusive); -void fuse_dio_unlock (struct kiocb *iocb,bool exclusive);struct fuse_file *fuse_file_open (struct fuse_mount *fm,u64 nodeid,unsigned int open_flags,bool isdir); void fuse_file_release(struct inode *inode, struct fuse_file *ff,fs/fuse/passthrough.c +60
-0@@ -25,6 +25,76 @@ static void fuse_passthrough_end_write(struct kiocb *iocb, ssize_t ret)fuse_write_update_attr (inode, iocb->ki_pos, ret); } +static bool fuse_passthrough_io_past_eof (struct kiocb *iocb, +struct iov_iter *iter) +{ +struct inode *inode =file_inode (iocb->ki_filp); + +return iocb->ki_pos +iov_iter_count (iter) >i_size_read (inode); +} + +/* +* Decide whether an exclusive inode lock is required for a passthrough write +* before the lock is taken. Returns true (exclusive needed) unless all of: +* - server advertised FOPEN_PARALLEL_DIRECT_WRITES +* - write is direct I/O (not buffered) +* - write is not append +* - write does not appear to extend past EOF (re-checked after lock below) +*/ +static bool fuse_passthrough_write_needs_exclusive (struct kiocb *iocb, +struct iov_iter *iter) +{ +struct fuse_file *ff = iocb->ki_filp->private_data; + +if (!(ff->open_flags &FOPEN_PARALLEL_DIRECT_WRITES )) +return true; + +if (!(iocb->ki_flags &IOCB_DIRECT )) +return true; +if (iocb->ki_flags &IOCB_APPEND ) +return true; +if (fuse_passthrough_io_past_eof (iocb, iter)) +return true; + +return false; +} + +static void fuse_passthrough_lock (struct kiocb *iocb,struct iov_iter *iter, +bool *exclusive) +{ +struct inode *inode =file_inode (iocb->ki_filp); + + *exclusive =fuse_passthrough_write_needs_exclusive (iocb, iter); +if (*exclusive) { +inode_lock (inode); + }else { +inode_lock_shared (inode); +/* +* The past-EOF check in fuse_passthrough_write_needs_exclusive() +* was made without holding the inode lock and may have raced +* with a concurrent EOF-extending write. Re-check under the +* shared lock and upgrade to exclusive if the write now reaches +* past EOF, to ensure i_size is never updated without exclusive +* serialisation. +*/ +if (fuse_passthrough_io_past_eof (iocb, iter)) { +inode_unlock_shared (inode); +inode_lock (inode); + *exclusive = true; + } + } +} +
fs/fuse/passthrough.c static void fuse_passthrough_end_write(struct kiocb *iocb, ssize_t ret) · line 97
Quoted 10 lines
fs/fuse/passthrough.c +10
-0+static void fuse_passthrough_unlock (struct kiocb *iocb,bool exclusive) +{ +struct inode *inode =file_inode (iocb->ki_filp); + +if (exclusive) +inode_unlock (inode); +else +inode_unlock_shared (inode); +} +
I'll take the blame for phrasing my review comment "Either you duplicate fuse_dio_lock() -> fuse_passthrough_lock() or you find a way to have the two share common helper (prefered)"
I should have phrased it stronger.
Unless there is a very good reason not to share a common helper we need to use a common helper and not duplicate 99% identical code.
See my untested patch trying to do that below.
Thanks,
Amir.
fs/fuse/file.c +0 -0
@@ -1397,19 +1397,24 @@ static bool fuse_io_past_eof(struct kiocb
*iocb, struct iov_iter *iter)
}
/*
- * @return true if an exclusive lock for direct IO writes is needed
+ * @return true if an exclusive lock for direct IO or passthrough writes is needed
*/
-static bool fuse_dio_wr_exclusive_lock(struct kiocb *iocb, struct iov_iter *from)
+static bool fuse_io_wr_exclusive_lock(struct kiocb *iocb, struct iov_iter *from)
{
struct file *file = iocb->ki_filp;
struct fuse_file *ff = file->private_data;
struct inode *inode = file_inode(iocb->ki_filp);
struct fuse_inode *fi = get_fuse_inode(inode);+ bool fopen_direct_io = ff->open_flags & FOPEN_DIRECT_IO;
/* Server side has to advise that it supports parallel dio writes. */
if (!(ff->open_flags & FOPEN_PARALLEL_DIRECT_WRITES))
return true;+ /* Passthrough mode supports parallel IOCB_DIRECT writes */
+ if (!fopen_direct_io && !(iocb->ki_flags & IOCB_DIRECT))
+ return true;
+
/*
* Append will need to know the eventual EOF - always needs an
* exclusive lock.
@@ -1428,13 +1433,33 @@ static bool fuse_dio_wr_exclusive_lock(struct kiocb *iocb, struct iov_iter *from
return false;
}
static void fuse_dio_lock(struct kiocb *iocb, struct iov_iter *from,
- bool *exclusive)
+static int fuse_parallel_dio_start(struct fuse_file *ff, struct fuse_inode *fi) +{
+ /* Permanent uncached mode from passthrough open */
+ if (ff->iomode == IOM_UNCACHED)
+ return 0;
+
+ return fuse_inode_uncached_io_start(fi, NULL);
+}
+
+static void fuse_parallel_dio_end(struct fuse_file *ff, struct fuse_inode *fi) +{
+ /* Permanent uncached mode from passthrough open */
+ if (ff->iomode == IOM_UNCACHED)
+ return;
+
+ /* Allow opens in caching mode after last parallel dio end */
+ fuse_inode_uncached_io_end(fi);
+}
+
+/* Take shared/exclusive lock for direct IO or passthrough write */ +void fuse_io_wr_lock(struct kiocb *iocb, struct iov_iter *from, bool *exclusive)
{
+ struct fuse_file *ff = iocb->ki_filp->private_data;
struct inode *inode = file_inode(iocb->ki_filp);
struct fuse_inode *fi = get_fuse_inode(inode);- *exclusive = fuse_dio_wr_exclusive_lock(iocb, from);
+ *exclusive = fuse_io_wr_exclusive_lock(iocb, from);
if (*exclusive) {
inode_lock(inode);
} else {
@@ -1447,7 +1472,7 @@ static void fuse_dio_lock(struct kiocb *iocb,
struct iov_iter *from,
* have raced, so check it again.
*/
if (fuse_io_past_eof(iocb, from) ||
- fuse_inode_uncached_io_start(fi, NULL) != 0) {
+ fuse_parallel_dio_start(ff, fi) != 0) {
inode_unlock_shared(inode);
inode_lock(inode);
*exclusive = true;
@@ -1455,16 +1480,17 @@ static void fuse_dio_lock(struct kiocb *iocb,
struct iov_iter *from,
}}
-static void fuse_dio_unlock(struct kiocb *iocb, bool exclusive) +/* Release shared/exclusive lock taken for direct IO or passthrough write */ +void fuse_io_wr_unlock(struct kiocb *iocb, bool exclusive)
{
+ struct fuse_file *ff = iocb->ki_filp->private_data;
struct inode *inode = file_inode(iocb->ki_filp);
struct fuse_inode *fi = get_fuse_inode(inode);
if (exclusive) {
inode_unlock(inode);
} else {
- /* Allow opens in caching mode after last parallel dio end */
- fuse_inode_uncached_io_end(fi);
+ fuse_parallel_dio_end(ff, fi);
inode_unlock_shared(inode);
}
}
@@ -1793,7 +1819,7 @@ static ssize_t fuse_direct_write_iter(struct
kiocb *iocb, struct iov_iter *from)
ssize_t res;
bool exclusive;- fuse_dio_lock(iocb, from, &exclusive);
+ fuse_io_wr_lock(iocb, from, &exclusive);
res = generic_write_checks(iocb, from);
if (res > 0) {
task_io_account_write(res);
@@ -1807,7 +1833,7 @@ static ssize_t fuse_direct_write_iter(struct
kiocb *iocb, struct iov_iter *from)
fuse_write_update_attr(inode, iocb->ki_pos, res);
}
}- fuse_dio_unlock(iocb, exclusive);
+ fuse_io_wr_unlock(iocb, exclusive);
return res;
}
v4 cover fuse: allow parallel direct writes for passthrough
In reply to Russ FellowsThis is v4, about six weeks after v3 [1] -- apologies for the delay. Thanks to Amir Goldstein for the detailed review of v3 and for the suggested patches; this revision implements that feedback. The v3 thread [1] has the full context and Amir's comments.
The series is also available at:
https://github.com/russfellows/linux-fuse-passthrough-patches
FUSE passthrough writes are fully serialised on the inode lock, even when the server advertises FOPEN_PARALLEL_DIRECT_WRITES and the write is a direct, non-append, within-EOF overwrite. Native FUSE direct-IO writes already take a shared lock in that case; there is no reason passthrough cannot do the same. With a passthrough backing file on XFS, this is the difference between serialised and fully parallel 4K random-write O_DIRECT throughput.
Patch 1 stops fuse_file_io_open() from stripping FOPEN_PARALLEL_DIRECT_WRITES on passthrough opens (which bypass the page cache without FOPEN_DIRECT_IO). Patch 2 lets fuse_passthrough_write_iter() take the shared lock for eligible writes by sharing the existing direct-IO write-lock helper rather than duplicating it.
Rebased onto v6.18.39 (LTS).
Changes since v3:
- Patch 1: keep only the FOPEN_IOMODE_CACHED() helper (drop the IS_DIRECT / IS_PASSTHROUGH macros, which described how I/O is performed rather than the ff->iomode state and were misleading); fold the O_DIRECT/fcntl note into the FOPEN_PARALLEL_DIRECT_WRITES comment; and simplify fuse_file_io_open() so a pure direct-IO open needs no early return. (Per Amir's review and suggested diff.)
- Patch 2: instead of duplicating fuse_dio_lock() into passthrough.c, generalise it: fuse_io_wr_exclusive_lock() now also handles passthrough, the uncached-io refcount is factored into fuse_parallel_dio_start/end() (no-ops for passthrough's permanent uncached iomode), and fuse_dio_lock()/fuse_dio_unlock() are renamed fuse_io_wr_lock()/ fuse_io_wr_unlock() and shared by both write paths. (Per Amir's review and suggested diff.)
Testing (fstests / xfstests)
----------------------------
Run with the libfuse passthrough_hp server under xfstests (FSTYP=fuse), './check -fuse -g quick' (642 tests), on v6.18.39, backing XFS on a RAM-backed null_blk device. An unpatched v6.18.39 baseline was captured first, then the same tree with this series applied; results were diffed to isolate any patch-induced change. Two server configs were exercised, as suggested by Amir:
passthrough (default): unpatched 25 failed / patched 25 failed
passthrough --direct-io: unpatched 27 failed / patched 27 failed
In both configs the failing set is identical with and without the series (the --direct-io set is the default set plus generic/355 and generic/362). Every failure is pre-existing on the unpatched kernel and is a known passthrough_hp / fuse limitation (permission/ACL/atime/mmap-coherency tests), unrelated to this change:
=> zero regressions introduced by the series in either config.
No kernel warnings, lockdep reports, or splats were observed in any run.
Performance
-----------
Custom FUSE passthrough daemon advertising FOPEN_PASSTHROUGH | FOPEN_PARALLEL_DIRECT_WRITES, XFS on a RAM-backed null_blk device, kernel v6.18.39. fio 4K random write, O_DIRECT, libaio, iodepth=64, single shared file:
numjobs | unpatched (IOPS) | patched (IOPS)
--------|------------------|---------------
1 | 352,000 | 352,000
2 | 440,000 | 648,000
3 | 429,000 | 951,000
4 | 423,000 | 987,000Unpatched write throughput is flat regardless of job count (the exclusive inode lock serialises all writers to the file); with the series it scales with concurrency, ~2.3x at 4 jobs, up to the backing device ceiling. Single-job is unchanged (no lock contention), and 4K random *read* throughput is unchanged (~1.37M IOPS with and without the series), since reads never took the exclusive lock.
[1] v3: https://lore.kernel.org/all/[email protected]/
Russ Fellows (2):
fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens
fuse: allow parallel direct writes in passthrough write_iter
fs/fuse/file.c | 53 ++++++++++++++++++++++++++++++++++--------- fs/fuse/fuse_i.h | 2 ++ fs/fuse/iomode.c | 23 ++++++++++--------- fs/fuse/passthrough.c | 6 ++--- 4 files changed, 59 insertions(+), 25 deletions(-)
--
2.51.0
PATCH v4 1/2 fuse: preserve FOPEN_PARALLEL_DIRECT_WRITES for passthrough opens
fuse_file_io_open() clears FOPEN_PARALLEL_DIRECT_WRITES for any open that lacks FOPEN_DIRECT_IO. That rule is too strict for passthrough opens, which already bypass the page cache through the backing file and do not need FOPEN_DIRECT_IO to guarantee direct-I/O semantics. Clearing the flag before the passthrough write path sees it prevents the kernel from ever taking the shared-lock path for passthrough writes.
Introduce FOPEN_IOMODE_CACHED() to test for cached (page-cache) iomode -- neither FOPEN_DIRECT_IO nor FOPEN_PASSTHROUGH set -- and use it to guard the FOPEN_PARALLEL_DIRECT_WRITES stripping. Passthrough and direct-IO opens both bypass the page cache, so both are now eligible to enable parallel direct writes.
While here, simplify fuse_file_io_open(): drop the early return for the pure direct-IO case and dispatch the caching open through the same FOPEN_IOMODE_CACHED() test, so a pure direct-IO open simply performs no iomode setup instead of returning early.
This is a prerequisite for passthrough write parallelism; without it the shared-lock path in the subsequent patch never activates.
--- fs/fuse/iomode.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
fs/fuse/iomode.c +12 -11
@@ -192,12 +192,18 @@ static int fuse_file_passthrough_open(struct inode *inode, struct file *file)return err; } +/* Fuse uses page cache if no passthrough nor direct_io open specified */ +#define FOPEN_IOMODE (oflags) \ + ((oflags) & (FOPEN_DIRECT_IO | FOPEN_PASSTHROUGH)) +#define FOPEN_IOMODE_CACHED (oflags) \ + (FOPEN_IOMODE(oflags) == 0) +/* Request access to submit new io to inode via open file */ int fuse_file_io_open (struct file *file,struct inode *inode) {struct fuse_file *ff = file->private_data;struct fuse_inode *fi =get_fuse_inode (inode); -int err; +int err =0 ;/* * io modes are not relevant with DAX and with server that does not
@@ -216,25 +222,20 @@ int fuse_file_io_open(struct file *file, struct inode *inode)/* * FOPEN_PARALLEL_DIRECT_WRITES requires FOPEN_DIRECT_IO. +* Note that if user opens a file open with O_DIRECT, but server did +* not specify FOPEN_DIRECT_IO, a later fcntl() could remove O_DIRECT, +* so we put the inode in caching mode to prevent parallel dio. */ -if (!(ff->open_flags &FOPEN_DIRECT_IO )) +if (FOPEN_IOMODE_CACHED (ff->open_flags)) ff->open_flags &= ~FOPEN_PARALLEL_DIRECT_WRITES ;/* * First passthrough file open denies caching inode io mode. * First caching file open enters caching inode io mode. -* -* Note that if user opens a file open with O_DIRECT, but server did -* not specify FOPEN_DIRECT_IO, a later fcntl() could remove O_DIRECT, -* so we put the inode in caching mode to prevent parallel dio. */ -if ((ff->open_flags &FOPEN_DIRECT_IO ) && - !(ff->open_flags &FOPEN_PASSTHROUGH )) -return 0 ; -if (ff->open_flags &FOPEN_PASSTHROUGH ) err =fuse_file_passthrough_open (inode, file); -else +else if (FOPEN_IOMODE_CACHED (ff->open_flags)) err =fuse_file_cached_io_open (inode, ff);if (err) goto fail;
Signature · 1 line
2.51.0
PATCH v4 2/2 fuse: allow parallel direct writes in passthrough write_iter
In reply to Russ Fellowsfuse_passthrough_write_iter() unconditionally took the exclusive inode lock, so passthrough writes were fully serialised even when the server advertised FOPEN_PARALLEL_DIRECT_WRITES and the write was direct, non-append and within EOF. Native direct-IO writes already take a shared lock in that case via fuse_dio_lock(); passthrough should do the same.
Rather than duplicate the direct-IO locking logic, generalise it and share it between both write paths:
- rename fuse_dio_wr_exclusive_lock() to fuse_io_wr_exclusive_lock() and teach it that a passthrough open without FOPEN_DIRECT_IO only qualifies for the shared lock when the write itself is IOCB_DIRECT (buffered passthrough writes keep the exclusive lock);
- factor the uncached-io refcount dance into fuse_parallel_dio_start() and fuse_parallel_dio_end(), which are no-ops for the permanent uncached iomode that passthrough opens hold, so the shared-lock past-EOF re-check works unchanged for both paths;
- rename fuse_dio_lock()/fuse_dio_unlock() to fuse_io_wr_lock()/ fuse_io_wr_unlock(), export them via fuse_i.h, and call them from fuse_passthrough_write_iter().
No functional change for the native direct-IO write path.
--- fs/fuse/file.c | 53 ++++++++++++++++++++++++++++++++++--------- fs/fuse/fuse_i.h | 2 ++ fs/fuse/passthrough.c | 6 ++--- 3 files changed, 47 insertions(+), 14 deletions(-)
fs/fuse/file.c +42 -11
@@ -1349,19 +1349,29 @@ static bool fuse_io_past_eof(struct kiocb *iocb, struct iov_iter *iter) }/* -* @return true if an exclusive lock for direct IO writes is needed +* @return true if an exclusive lock for direct IO or passthrough writes is +* needed */ -static bool fuse_dio_wr_exclusive_lock (struct kiocb *iocb,struct iov_iter *from) +static bool fuse_io_wr_exclusive_lock (struct kiocb *iocb,struct iov_iter *from) {struct file *file = iocb->ki_filp;struct fuse_file *ff = file->private_data;struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode); +bool fopen_direct_io = ff->open_flags &FOPEN_DIRECT_IO ;/* Server side has to advise that it supports parallel dio writes. */ if (!(ff->open_flags &FOPEN_PARALLEL_DIRECT_WRITES ))return true; +/* +* Passthrough opens without FOPEN_DIRECT_IO only support parallel +* writes for I/O that is itself direct (IOCB_DIRECT); buffered +* passthrough writes still need the exclusive lock. +*/ +if (!fopen_direct_io && !(iocb->ki_flags &IOCB_DIRECT )) +return true; +/* * Append will need to know the eventual EOF - always needs an * exclusive lock.
@@ -1380,13 +1390,33 @@ static bool fuse_dio_wr_exclusive_lock(struct kiocb *iocb, struct iov_iter *fromreturn false; } -static void fuse_dio_lock (struct kiocb *iocb,struct iov_iter *from, -bool *exclusive) +static int fuse_parallel_dio_start (struct fuse_file *ff,struct fuse_inode *fi) +{ +/* Passthrough opens are in permanent uncached mode */ +if (ff->iomode ==IOM_UNCACHED ) +return 0 ; + +return fuse_inode_uncached_io_start (fi,NULL ); +} + +static void fuse_parallel_dio_end (struct fuse_file *ff,struct fuse_inode *fi) +{ +/* Passthrough opens are in permanent uncached mode */ +if (ff->iomode ==IOM_UNCACHED ) +return ; + +/* Allow opens in caching mode after last parallel dio end */ +fuse_inode_uncached_io_end (fi); +} + +/* Take shared/exclusive inode lock for a direct IO or passthrough write */ +void fuse_io_wr_lock (struct kiocb *iocb,struct iov_iter *from,bool *exclusive) { +struct fuse_file *ff = iocb->ki_filp->private_data;struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode); - *exclusive =fuse_dio_wr_exclusive_lock (iocb, from); + *exclusive =fuse_io_wr_exclusive_lock (iocb, from);if (*exclusive) {inode_lock (inode); }else {
@@ -1399,7 +1429,7 @@ static void fuse_dio_lock(struct kiocb *iocb, struct iov_iter *from,* have raced, so check it again. */ if (fuse_io_past_eof (iocb, from) || -fuse_inode_uncached_io_start (fi,NULL ) !=0 ) { +fuse_parallel_dio_start (ff, fi) !=0 ) {inode_unlock_shared (inode);inode_lock (inode); *exclusive = true;
@@ -1407,16 +1437,17 @@ static void fuse_dio_lock(struct kiocb *iocb, struct iov_iter *from, } } -static void fuse_dio_unlock (struct kiocb *iocb,bool exclusive) +/* Release the inode lock taken for a direct IO or passthrough write */ +void fuse_io_wr_unlock (struct kiocb *iocb,bool exclusive) { +struct fuse_file *ff = iocb->ki_filp->private_data;struct inode *inode =file_inode (iocb->ki_filp);struct fuse_inode *fi =get_fuse_inode (inode);if (exclusive) {inode_unlock (inode); }else { -/* Allow opens in caching mode after last parallel dio end */ -fuse_inode_uncached_io_end (fi); +fuse_parallel_dio_end (ff, fi);inode_unlock_shared (inode); } }
@@ -1759,7 +1790,7 @@ static ssize_t fuse_direct_write_iter(struct kiocb *iocb, struct iov_iter *from)ssize_t res;bool exclusive; -fuse_dio_lock (iocb, from, &exclusive); +fuse_io_wr_lock (iocb, from, &exclusive); res =generic_write_checks (iocb, from);if (res >0 ) {task_io_account_write (res);
@@ -1773,7 +1804,7 @@ static ssize_t fuse_direct_write_iter(struct kiocb *iocb, struct iov_iter *from)fuse_write_update_attr (inode, iocb->ki_pos, res); } } -fuse_dio_unlock (iocb, exclusive); +fuse_io_wr_unlock (iocb, exclusive);return res; }
fs/fuse/fuse_i.h +2 -0
@@ -1529,6 +1529,8 @@ struct fuse_file *fuse_file_open(struct fuse_mount *fm, u64 nodeid, unsigned int open_flags, bool isdir);void fuse_file_release (struct inode *inode,struct fuse_file *ff,unsigned int open_flags,fl_owner_t id,bool isdir); +void fuse_io_wr_lock (struct kiocb *iocb,struct iov_iter *from,bool *exclusive); +void fuse_io_wr_unlock (struct kiocb *iocb,bool exclusive);/* backing.c */ #ifdef CONFIG_FUSE_PASSTHROUGH
fs/fuse/passthrough.c +3 -3
@@ -54,11 +54,11 @@ ssize_t fuse_passthrough_write_iter(struct kiocb *iocb, struct iov_iter *iter) {struct file *file = iocb->ki_filp; -struct inode *inode =file_inode (file);struct fuse_file *ff = file->private_data;struct file *backing_file =fuse_file_passthrough (ff);size_t count =iov_iter_count (iter);ssize_t ret; +bool exclusive;struct backing_file_ctx ctx = { .cred = ff->cred, .end_write = fuse_passthrough_end_write,
@@ -70,10 +70,10 @@ ssize_t fuse_passthrough_write_iter(struct kiocb *iocb,if (!count)return 0 ; -inode_lock (inode); +fuse_io_wr_lock (iocb, iter, &exclusive); ret =backing_file_write_iter (backing_file, iter, iocb, iocb->ki_flags, &ctx); -inode_unlock (inode); +fuse_io_wr_unlock (iocb, exclusive);return ret; }
Signature · 1 line
2.51.0
Re: [PATCH v4 0/2] fuse: allow parallel direct writes for passthrough
In reply to Russ FellowsQuoted 17 lines · Russ Fellows
This is v4, about six weeks after v3 [1] -- apologies for the delay. Thanks to Amir Goldstein for the detailed review of v3 and for the suggested patches; this revision implements that feedback. The v3 thread [1] has the full context and Amir's comments.
The series is also available at:
https://github.com/russfellows/linux-fuse-passthrough-patchesFUSE passthrough writes are fully serialised on the inode lock, even when the server advertises
FOPEN_PARALLEL_DIRECT_WRITESand the write is a direct, non-append, within-EOF overwrite. Native FUSE direct-IO writes already take a shared lock in that case; there is no reason passthrough cannot do the same. With a passthrough backing file on XFS, this is the difference between serialised and fully parallel 4K random-writeO_DIRECTthroughput.
Quoted 7 lines
Patch 1 stops
fuse_file_io_open()from strippingFOPEN_PARALLEL_DIRECT_WRITESon passthrough opens (which bypass the page cache withoutFOPEN_DIRECT_IO). Patch 2 letsfuse_passthrough_write_iter()take the shared lock for eligible writes by sharing the existing direct-IO write-lock helper rather than duplicating it.Rebased onto v6.18.39 (LTS).
Hi Russ,
This version looks good and very well tested.
Please add
to both patches.
The only thing is that your patches need to apply to the current upstream master branch
not to an LTS kernel.
Kernel patches always need to be merged upstream and only then backported to LTS kernels.
The merge conflicts with master are minor, but please rebase and test on master. With that done, I hope that Miklos will be able to include v5 in 7.3 PR after he gets back from vacation.
Thanks,
Amir.
Quoted 73 lines
Changes since v3:
- Patch 1: keep only theFOPEN_IOMODE_CACHED()helper (drop theIS_DIRECT/IS_PASSTHROUGHmacros, which described how I/O is performed rather than theff->iomodestate and were misleading); fold the O_DIRECT/fcntl note into theFOPEN_PARALLEL_DIRECT_WRITEScomment; and simplifyfuse_file_io_open()so a pure direct-IO open needs no early return. (Per Amir's review and suggested diff.)
- Patch 2: instead of duplicatingfuse_dio_lock()intopassthrough.c, generalise it:fuse_io_wr_exclusive_lock()now also handles passthrough, the uncached-io refcount is factored into fuse_parallel_dio_start/end() (no-ops for passthrough's permanent uncached iomode), and fuse_dio_lock()/fuse_dio_unlock() are renamed fuse_io_wr_lock()/fuse_io_wr_unlock()and shared by both write paths. (Per Amir's review and suggested diff.)Testing (fstests / xfstests)
----------------------------
Run with the libfusepassthrough_hpserver under xfstests (FSTYP=fuse), './check -fuse -g quick' (642 tests), on v6.18.39, backing XFS on a RAM-backednull_blkdevice. An unpatched v6.18.39 baseline was captured first, then the same tree with this series applied; results were diffed to isolate any patch-induced change. Two server configs were exercised, as suggested by Amir:passthrough (default): unpatched 25 failed / patched 25 failed
passthrough --direct-io: unpatched 27 failed / patched 27 failedIn both configs the failing set is identical with and without the series (the --direct-io set is the default set plus generic/355 and generic/362). Every failure is pre-existing on the unpatched kernel and is a known
passthrough_hp/ fuse limitation (permission/ACL/atime/mmap-coherency tests), unrelated to this change:=> zero regressions introduced by the series in either config.
No kernel warnings, lockdep reports, or splats were observed in any run.
Performance
-----------
Custom FUSE passthrough daemon advertisingFOPEN_PASSTHROUGH|FOPEN_PARALLEL_DIRECT_WRITES, XFS on a RAM-backednull_blkdevice, kernel v6.18.39. fio 4K random write,O_DIRECT, libaio, iodepth=64, single shared file:numjobs | unpatched (IOPS) | patched (IOPS) --------|------------------|--------------- 1 | 352,000 | 352,000 2 | 440,000 | 648,000 3 | 429,000 | 951,000 4 | 423,000 | 987,000Unpatched write throughput is flat regardless of job count (the exclusive inode lock serialises all writers to the file); with the series it scales with concurrency, ~2.3x at 4 jobs, up to the backing device ceiling. Single-job is unchanged (no lock contention), and 4K random *read* throughput is unchanged (~1.37M IOPS with and without the series), since reads never took the exclusive lock.
[1] v3: https://lore.kernel.org/all/[email protected]/
Russ Fellows (2):
fuse: preserveFOPEN_PARALLEL_DIRECT_WRITESfor passthrough opens
fuse: allow parallel direct writes in passthroughwrite_iterfs/fuse/file.c | 53 ++++++++++++++++++++++++++++++++++--------- fs/fuse/fuse_i.h | 2 ++ fs/fuse/iomode.c | 23 ++++++++++--------- fs/fuse/passthrough.c | 6 ++--- 4 files changed, 59 insertions(+), 25 deletions(-)--
2.51.0