Skip to content

fuse: Add support for passthrough read/write

Amir Goldstein linux-fsdevel linux-unionfs 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v13 Latest10 patches 41 repliesIn review

Changes in v13 from the cover letter · 16 lines

- Rebase to v6.4-rc2
- Reword 'lower file' language to 'backing file'
- Add explicit FOPEN_PASSTHROUGH open flags
- Remove fuse_passthrough_out container
- Add FUSE_DEV_IOC_PASSTHROUGH_CLOSE ioctl
- Add experimental FUSE_DEV_IOC_PASSTHROUGH_SETUP ioctl
- Distinguished errors for failures to create passthrough id (EBADF, EOPNOTSUPP, ELOOP)
- idr and fuse_file point to refcounted passthrough object
- Use rcu_read_lock() to get passthrough object by id
- Handle errors to setup passthrough in atomic_open()
- Invalidate mtime/size after passthrough write
- Invalidate atime after passthrough read/mmap
- Bump FUSE protocol minor version
Alessio Balsini (2):
fs: Generic function to convert iocb to rw flags
fuse: Definitions and ioctl for passthrough

Patches in v13 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/10fs: Generic function to convert iocb to rw flagsfs/overlayfs/file.c include/linux/fs.h+10 -180 replies
2/10fuse: Definitions and ioctl for passthroughfs/fuse/Makefile fs/fuse/dev.c fs/fuse/dir.c +5 more+143 -164 replies
3/10fuse: Passthrough initialization and releasefs/fuse/fuse_i.h fs/fuse/inode.c fs/fuse/passthrough.c+87 -40 replies
4/10fuse: Introduce synchronous read and write for passthroughfs/fuse/file.c fs/fuse/fuse_i.h fs/fuse/passthrough.c+56 -20 replies
5/10fuse: Handle asynchronous read and write in passthroughfs/fuse/passthrough.c+75 -79 replies
6/10fuse: Use daemon creds in passthrough modefs/fuse/fuse_i.h fs/fuse/passthrough.c+19 -30 replies
7/10fuse: Introduce passthrough for mmapfs/fuse/file.c fs/fuse/fuse_i.h fs/fuse/passthrough.c+31 -00 replies
8/10fuse: update inode size/mtime after passthrough writefs/fuse/passthrough.c+42 -116 replies
9/10fuse: invalidate atime after passthrough read/mmapfs/fuse/passthrough.c+26 -00 replies
10/10fuse: setup a passthrough fd without a permanent backing idfs/fuse/dev.c fs/fuse/fuse_i.h fs/fuse/passthrough.c +1 more+74 -73 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v13 yet.

Read the v13 thread

Cover letter [PATCH v13 00/10] fuse: Add support for passthrough read/write

Miklos,

This patch set addresses your review feedback on Alesio's V12 patch set from 2021 [1] as well as other bugs that I have found since. This patch set uses refcounted backing files as we discussed recently [2].

I am posting this for several possible outcomes:

1. Either FUSE-BPF develpers can use this as a reference implementation for their 1st phase of "backing file passthrough"
2. Or they can tell me which API changes need to made to this patch set so the API is flexible enough to extend to "backing inode passthrough" and to "BPF filters" later on
3. We find there is little overlap in the APIs and merge this as is

These patches are available on github [3] along with libfuse patches [4]. I tested them by running xfstests (./check -fuse -g quick.rw) with latest libfuse xfstest support.

Without FOPEN_PASSTHROUGH, one test in this group fails (generic/451) which tests mixed buffered/aio writes.
With FOPEN_PASSTHROUGH, this test also passes.

This revision does not set any limitations on the number of backing files that can be mapped by the server. I considered several ways to address this and decided to try a different approach.

Patch 10 (with matching libfuse patch) is an RFC patch for an alternative API approach. Please see my comments on that patch.

Thanks,
Amir.

[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
[2] https://lore.kernel.org/linux-fsdevel/CAJfpegvbMKadnsBZmEvZpCxeWaMEGDRiDBqEZqaBSXcWyPZnpA@mail.gmail.com/
[3] https://github.com/amir73il/linux/commits/fuse-passthrough-fd
[4] https://github.com/amir73il/libfuse/commits/fuse-passthrough-fd

Changes since v12:
- Rebase to v6.4-rc2
- Reword 'lower file' language to 'backing file'
- Add explicit FOPEN_PASSTHROUGH open flags
- Remove fuse_passthrough_out container
- Add FUSE_DEV_IOC_PASSTHROUGH_CLOSE ioctl
- Add experimental FUSE_DEV_IOC_PASSTHROUGH_SETUP ioctl
- Distinguished errors for failures to create passthrough id (EBADF, EOPNOTSUPP, ELOOP)
- idr and fuse_file point to refcounted passthrough object
- Use rcu_read_lock() to get passthrough object by id
- Handle errors to setup passthrough in atomic_open()
- Invalidate mtime/size after passthrough write
- Invalidate atime after passthrough read/mmap
- Bump FUSE protocol minor version
Alessio Balsini (2):
fs: Generic function to convert iocb to rw flags
fuse: Definitions and ioctl for passthrough

Amir Goldstein (8):
fuse: Passthrough initialization and release
fuse: Introduce synchronous read and write for passthrough
fuse: Handle asynchronous read and write in passthrough
fuse: Use daemon creds in passthrough mode
fuse: Introduce passthrough for mmap
fuse: update inode size/mtime after passthrough write
fuse: invalidate atime after passthrough read/mmap
fuse: setup a passthrough fd without a permanent backing id

 fs/fuse/Makefile          |   1 +
 fs/fuse/dev.c             |  76 ++++++++-
 fs/fuse/dir.c             |   7 +-
 fs/fuse/file.c            |  28 +++-
 fs/fuse/fuse_i.h          |  48 +++++-
 fs/fuse/inode.c           |  22 ++-
 fs/fuse/passthrough.c     | 344 ++++++++++++++++++++++++++++++++++++++
 fs/overlayfs/file.c       |  23 +--
 include/linux/fs.h        |   5 +
 include/uapi/linux/fuse.h |  21 ++-
 10 files changed, 535 insertions(+), 40 deletions(-)
 create mode 100644 fs/fuse/passthrough.c
Signature · 1 line
2.34.1