Skip to contentSkip to conversation

fuse zero copy

fuse-devel Miklos Szeredi 2 participants 2 messages Aug 5, 2019 – Aug 7, 2019 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

Miklos Szeredi

author maintainer
#1 lore raw
In reply to a message not in the archive
Quoted 11 lines · Geert Custers

Hello,

I recently wrote a fuse driver for a custom on-disk file system. As far as I could tell, the documentation didn't have an explicit example for how to deal with filesystem that are on disk, most examples centered around a very simple in memory "filesystem". The way I do it now is by calling open() on the .img on init and fread()'ing and fwrite()'ing to it (with some caching to make it faster). I'm not sure this is the proper way to implement something like this, but I'll assume it is.

Quoted 11 lines

Implementing the file system I noticed that when performing fuse read()s and write()s that I do a lot of unneeded copying. Right now I fread() into the buffer passed to the read() function, but as far as I can tell this buffer is then copied from the fuse server to kernel space where it is copied back to the user program. A more natural way (the way I see it) would be a mechanism by which I could tell the kernel "read from fd 4 512 bytes starting at position 0x1000" for example. Then the whole operation involves only one copy operation. Reading around I have seen some ideas around this, but as far as I could tell this isn't actively being worked on... So my question is if there are any plans to implement a zero-copy system for fuse.

Actually it is being worked on. Attaching the current
proof-of-concept kernel patch for this.

I don't have a patch for libfuse yet, as I'm testing new ideas with a dummy filesystem that does raw /dev/fuse access. Also attached, needs to be run with "-m" to enable the file mapping mode.

To make this more useful, the kernel would need to cache the mapping, so it doesn't need to issue a MAP request on each read. That would also optimize the case of long extents, or files mirrored completely from an underlying filesystem (as done by the test program).

Thanks,
Miklos

From a90a38e4700fbf0e8f73ce19cb6dfe30db5902f2 Mon Sep 17 00:00:00 2001
From: Miklos Szeredi <[email protected]>
Date: Mon, 5 Aug 2019 13:44:59 +0200
Subject: [PATCH] fuse: add map request

---
 fs/fuse/Makefile          |  2 +-
 fs/fuse/dev.c             | 23 ++++++++++++++
 fs/fuse/file.c            | 64 +++++++++++++++++++++++++++++++++++++++
 fs/fuse/fuse_i.h          |  5 +++
 fs/fuse/map.c             | 58 +++++++++++++++++++++++++++++++++++
 include/uapi/linux/fuse.h | 10 ++++++
 6 files changed, 161 insertions(+), 1 deletion(-)
 create mode 100644 fs/fuse/map.c
fs/fuse/Makefile +1 -1
index 9485019c2a14..7e110c77d553 100644
@@ -6,4 +6,4 @@
 obj-$(CONFIG_FUSE_FS) += fuse.o
 obj-$(CONFIG_CUSE) += cuse.o
 
-fuse-objs := dev.o dir.o file.o inode.o control.o xattr.o acl.o readdir.o
+fuse-objs := dev.o dir.o file.o inode.o control.o xattr.o acl.o readdir.o map.o
fs/fuse/dev.c +23 -0
index ea8237513dfa..ed64ce383b11 100644
@@ -2306,6 +2306,26 @@ static int fuse_device_clone(struct fuse_conn *fc, struct file *new)
 	return 0;
 }
 
+static long fuse_dev_map_ioctl(unsigned int cmd, unsigned long arg)
+{
+	struct file *file;
+
+	switch (cmd) {
+	case FUSE_DEV_IOC_MAP_OPEN:
+		file = fget(arg);
+		if (!file)
+			return -EBADF;
+		return fuse_map_open(file);
+
+	case FUSE_DEV_IOC_MAP_CLOSE:
+		return fuse_map_close(arg);
+
+	default:
+		return -ENOTTY;
+	}
+
+}
+
 static long fuse_dev_ioctl(struct file *file, unsigned int cmd,
 			   unsigned long arg)
 {
@@ -2338,7 +2358,10 @@ static long fuse_dev_ioctl(struct file *file, unsigned int cmd,
 				fput(old);
 			}
 		}
+	} else {
+		err = fuse_dev_map_ioctl(cmd, arg);
 	}
+
 	return err;
 }
 
fs/fuse/file.c +64 -0

76 lines not on this page · raw message

fs/fuse/fuse_i.h +5 -0
index 24dbca777775..ea7b0548e034 100644
@@ -1093,4 +1093,9 @@ int fuse_set_acl(struct inode *inode, struct posix_acl *acl, int type);
 /* readdir.c */
 int fuse_readdir(struct file *file, struct dir_context *ctx);
 
+/* map.c */
+int fuse_map_open(struct file *file);
+int fuse_map_close(unsigned long mapfd);
+struct file *fuse_map_get(u64 mapfd);
+
 #endif /* _FS_FUSE_I_H */
fs/fuse/map.c new file +58 -0

58 lines not on this page · raw message

include/uapi/linux/fuse.h +10 -0
index 2971d29a42e4..65fca0128716 100644
@@ -244,6 +244,7 @@ struct fuse_file_lock {
 #define FOPEN_NONSEEKABLE	(1 << 2)
 #define FOPEN_CACHE_DIR		(1 << 3)
 #define FOPEN_STREAM		(1 << 4)
+#define FOPEN_MAP		(1 << 5)
 
 /**
  * INIT request/reply flags
@@ -422,6 +423,7 @@ enum fuse_opcode {
 	FUSE_RENAME2		= 45,
 	FUSE_LSEEK		= 46,
 	FUSE_COPY_FILE_RANGE	= 47,
+	FUSE_MAP		= 50,
 
 	/* CUSE specific operations */
 	CUSE_INIT		= 4096,
@@ -571,6 +573,12 @@ struct fuse_read_in {
 	uint32_t	padding;
 };
 
+struct fuse_map_out {
+	uint64_t	mapfd;
+	uint64_t	offset;
+	uint64_t	size;
+};
+
 #define FUSE_COMPAT_WRITE_IN_SIZE 24
 
 struct fuse_write_in {
@@ -823,6 +831,8 @@ struct fuse_notify_retrieve_in {
 
 /* Device ioctls: */
 #define FUSE_DEV_IOC_CLONE	_IOR(229, 0, uint32_t)
+#define FUSE_DEV_IOC_MAP_OPEN	_IO(229, 4)
+#define FUSE_DEV_IOC_MAP_CLOSE	_IO(229, 5)
 
 struct fuse_lseek_in {
 	uint64_t	fh;
Signature · 1 line
2.21.0
#2 lore raw
Quoted 6 lines

Actually it is being worked on. Attaching the current
proof-of-concept kernel patch for this.

I don't have a patch for libfuse yet, as I'm testing new ideas with a dummy filesystem that does raw /dev/fuse access. Also attached, needs to be run with "-m" to enable the file mapping mode.

I looked at the patch and proof of concept filesystem. I imagine the libfuse bindings would be relatively simple for this, something like a fuse_map_fd(), fuse_unmap_fd() and then a map() in the fuse_ops struct.

Quoted 4 lines

To make this more useful, the kernel would need to cache the mapping, so it doesn't need to issue a MAP request on each read. That would also optimize the case of long extents, or files mirrored completely from an underlying filesystem (as done by the test program).

I have some spare time so I could write some patches for libfuse and try add caching of the mapping on the kernel side. Maybe you can let me know your thoughts on what this would look like, so we are on the same page.

Thanks,
Geert