Skip to contentSkip to conversation

virtiofsd: Improve io bandwidth by replacing pwrite with pwritev

PATCH Superseded piaojun 2 participants 7 messages Aug 8, 2019 – Aug 9, 2019

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

piaojun

author
#0 lore raw

v3 cover virtiofsd: Improve io bandwidth by replacing pwrite with pwritev

Quoted 1 line

From my test, write bandwidth will be improved greatly by replacing

pwrite with pwritev, and the test result as below:

---
pwrite:

# fio -direct=1 -time_based -iodepth=64 -rw=randwrite -ioengine=libaio -bs=1M -size=1G -numjob=16 -runtime=30 -group_reporting -name=file -filename=/mnt/virtiofs/file
file: (g=0): rw=randwrite, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=64
...
fio-2.13
Starting 16 processes
Jobs: 16 (f=16): [w(16)] [100.0% done] [0KB/886.0MB/0KB /s] [0/886/0 iops] [eta 00m:00s]
file: (groupid=0, jobs=16): err= 0: pid=5799: Tue Aug 6 18:48:26 2019
write: io=26881MB, bw=916988KB/s, iops=895, runt= 30018msec

pwritev:

# fio -direct=1 -time_based -iodepth=64 -rw=randwrite -ioengine=libaio -bs=1M -size=1G -numjob=16 -runtime=30 -group_reporting -name=file -filename=/mnt/virtiofs/file
file: (g=0): rw=randwrite, bs=1M-1M/1M-1M/1M-1M, ioengine=libaio, iodepth=64
...
fio-2.13
Starting 16 processes
Jobs: 16 (f=16): [w(16)] [100.0% done] [0KB/1793MB/0KB /s] [0/1793/0 iops] [eta 00m:00s]
file: (groupid=0, jobs=16): err= 0: pid=6328: Tue Aug 6 18:22:17 2019
write: io=52775MB, bw=1758.7MB/s, iops=1758, runt= 30009msec
---

This patch introduces writev and pwritev for lo_write_buf(). I tried my best not doing harm to the origin code construct, and there will be some *useless* branches in fuse_buf_copy_one() which are hard to judge if they will be useful in the future. So I just leave them alone safely. If the cleanup work is necessary, please let me know.

v2
- Split into two patches
- Add the lost flags support, such as FUSE_BUF_PHYS_ADDR

v3
- use git send-email to make the patch set in one thread
- move fuse_buf_writev() into fuse_buf_copy()
- use writev for the src buffers when they're alread already mapped by the daemon process
- use calloc to replace malloc
- set res 0 if writev() returns 0

Jun Piao (2):
add definition of fuse_buf_writev().
use fuse_buf_writev to replace fuse_buf_write for better performance

---
 buffer.c      |   48 ++++++++++++++++++++++++++++++++++++++++++++++++
 fuse_common.h |   14 ++++++++++++++
 seccomp.c     |    2 ++
 3 files changed, 64 insertions(+)

--

author
#1 lore raw

PATCH v3 1/2 virtiofsd: add definition of fuse_buf_writev()

Define fuse_buf_writev() which use pwritev and writev to improve io bandwidth.

---
 contrib/virtiofsd/buffer.c      | 37 +++++++++++++++++++++++++++++++++++++
 contrib/virtiofsd/fuse_common.h | 14 ++++++++++++++
 contrib/virtiofsd/seccomp.c     |  2 ++
 3 files changed, 53 insertions(+)
contrib/virtiofsd/buffer.c +37 -0
index 655be137..3920cbf 100644
@@ -15,6 +15,7 @@
 #include <unistd.h>
 #include <errno.h>
 #include <assert.h>
+#include <stdlib.h>
 
 size_t fuse_buf_size(const struct fuse_bufvec *bufv)
 {
@@ -71,6 +72,42 @@ static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off,
 	return copied;
 }
 
+ssize_t fuse_buf_writev(fuse_req_t req,
+			     struct fuse_bufvec *out_buf,
+			     struct fuse_bufvec *in_buf,
+			     enum fuse_buf_copy_flags flags)
+{
+	ssize_t res, i, buf_index, iovcnt;
+	struct iovec * iov;
+	int fd = out_buf->buf[0].fd;
+	off_t pos = out_buf->buf[0].pos;
+
+	if (in_buf->count > 2)
+		iovcnt = in_buf->count - 1;
+	else
+		iovcnt = 1;
+
+	iov = calloc(iovcnt, sizeof(struct iovec));
+	if (!iov)
+		return -ENOMEM;
+
+	for (i = 0, buf_index = 1; i < iovcnt; i++, buf_index++) {
+		iov[i].iov_base = in_buf->buf[buf_index].mem;
+		iov[i].iov_len = in_buf->buf[buf_index].size;
+	}
+
+	if (flags & FUSE_BUF_FD_SEEK)
+		res = pwritev(fd, iov, iovcnt, pos);
+	else
+		res = writev(fd, iov, iovcnt);
+
+	if (res == -1)
+		res = -errno;
+
+	free(iov);
+	return res;
+}
+
 static ssize_t fuse_buf_read(const struct fuse_buf *dst, size_t dst_off,
 			     const struct fuse_buf *src, size_t src_off,
 			     size_t len)
contrib/virtiofsd/fuse_common.h +14 -0
index 001e6d6..7c07da7 100644
@@ -730,6 +730,20 @@ ssize_t fuse_buf_copy(fuse_req_t req,
 		      enum fuse_buf_copy_flags flags);
 
 /**
+ * Write data from one buffer vector to another
+ *
+ * @param req The request this copy is part of
+ * @param out_buf destination buffer vector
+ * @param in_buf source buffer vector
+ * @param flags flags controlling the copy
+ * @return actual number of bytes written or -errno on error
+ */
+ssize_t fuse_buf_writev(fuse_req_t req,
+		      struct fuse_bufvec *out_buf,
+		      struct fuse_bufvec *in_buf,
+		      enum fuse_buf_copy_flags flags);
+
+/**
  * Memory buffer iterator
  *
  */
contrib/virtiofsd/seccomp.c +2 -0
index 5f1c873..4bba30b 100644
@@ -60,6 +60,7 @@ static const int syscall_whitelist[] = {
 	SCMP_SYS(ppoll),
 	SCMP_SYS(prctl), /* TODO restrict to just PR_SET_NAME? */
 	SCMP_SYS(preadv),
+	SCMP_SYS(pwritev),
 	SCMP_SYS(pwrite64),
 	SCMP_SYS(read),
 	SCMP_SYS(readlinkat),
@@ -78,6 +79,7 @@ static const int syscall_whitelist[] = {
 	SCMP_SYS(unlinkat),
 	SCMP_SYS(utimensat),
 	SCMP_SYS(write),
+	SCMP_SYS(writev),
 };
 
 /* Syscalls used when --syslog is enabled */
author
#2 lore raw

PATCH v3 2/2 virtiofsd: use fuse_buf_writev to replace fuse_buf_write for better performance

fuse_buf_writev() only handles the normal write in which src is buffer and dest is fd. Specially if src buffer represents guest physical address that can't be mapped by the daemon process, IO must be bounced back to the VMM to do it by fuse_buf_copy().

---
 contrib/virtiofsd/buffer.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
contrib/virtiofsd/buffer.c +11 -0
index 3920cbf..f09d43c 100644
@@ -324,11 +324,22 @@ static int fuse_bufvec_advance(struct fuse_bufvec *bufv, size_t len)
 ssize_t fuse_buf_copy(fuse_req_t req, struct fuse_bufvec *dstv, struct fuse_bufvec *srcv,
 		      enum fuse_buf_copy_flags flags)
 {
+	size_t i;
 	size_t copied = 0;
 
 	if (dstv == srcv)
 		return fuse_buf_size(dstv);
 
+	/* use writev to improve bandwidth when all the
+	 * src buffers already mapped by the daemon
+	 * process */
+	for (i = 0; i < srcv->count; i++) {
+		if (srcv->buf[i].flags & FUSE_BUF_PHYS_ADDR)
+			break;
+	}
+	if (i == srcv->count)
+		return fuse_buf_writev(req, dstv, srcv, dstv->buf[0].flags);
+
 	for (;;) {
 		const struct fuse_buf *src = fuse_bufvec_current(srcv);
 		const struct fuse_buf *dst = fuse_bufvec_current(dstv);
#3 lore raw

Re: [Virtio-fs] [PATCH v3 1/2] virtiofsd: add definition of fuse_buf_writev()

In reply to piaojun

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 78

Quoted 8 lines · piaojun
contrib/virtiofsd/buffer.c +4 -0
@@ -71,6 +72,42 @@ static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off,
 	return copied;
 }
 
+ssize_t fuse_buf_writev(fuse_req_t req,
+			     struct fuse_bufvec *out_buf,
+			     struct fuse_bufvec *in_buf,
+			     enum fuse_buf_copy_flags flags)

FUSE_BUF_FD_SEEK is defined in enum fuse_buf_flags, not enum fuse_buf_copy_flags. This argument can be removed and the check below can be changed from:

if (flags & FUSE_BUF_FD_SEEK)

to

if (out_buf->buf[0].flags & FUSE_BUF_FD_SEEK)

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 83

Quoted 5 lines
contrib/virtiofsd/buffer.c +5 -0
+{
+	ssize_t res, i, buf_index, iovcnt;
+	struct iovec * iov;
+	int fd = out_buf->buf[0].fd;
+	off_t pos = out_buf->buf[0].pos;

A struct fuse_bufvec may have multiple elements but this function assumes it only has 1. Please use struct fuse_buf *out_buf instead. This way it's clear that only 1 fuse_buf will be written.

Quoted 4 lines
contrib/virtiofsd/buffer.c +4 -0
+
+	if (in_buf->count > 2)
+		iovcnt = in_buf->count - 1;
+	else

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 97

Quoted 10 lines
contrib/virtiofsd/buffer.c +10 -0
+		iovcnt = 1;
+
+	iov = calloc(iovcnt, sizeof(struct iovec));
+	if (!iov)
+		return -ENOMEM;
+
+	for (i = 0, buf_index = 1; i < iovcnt; i++, buf_index++) {
+		iov[i].iov_base = in_buf->buf[buf_index].mem;
+		iov[i].iov_len = in_buf->buf[buf_index].size;
+	}

Why is in_buf->buf[0] is skipped?

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 100

Quoted 3 lines
contrib/virtiofsd/buffer.c +3 -0
+
+	if (flags & FUSE_BUF_FD_SEEK)
+		res = pwritev(fd, iov, iovcnt, pos);

Please move off_t pos = out_buf->buf[0].pos into this if statement body to avoid a possible uninitialized memory access when !(flags & FUSE_BUF_FD_SEEK). This makes valgrind and other tools happy.

author
#4 lore raw

Re: [Virtio-fs] [PATCH v3 1/2] virtiofsd: add definition of fuse_buf_writev()

Hi Stefan,

Quoted 14 lines · Stefan Hajnoczi

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 78

Quoted 8 lines · piaojun
contrib/virtiofsd/buffer.c +4 -0
@@ -71,6 +72,42 @@ static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off,
 	return copied;
 }
 
+ssize_t fuse_buf_writev(fuse_req_t req,
+			     struct fuse_bufvec *out_buf,
+			     struct fuse_bufvec *in_buf,
+			     enum fuse_buf_copy_flags flags)

FUSE_BUF_FD_SEEK is defined in enum fuse_buf_flags, not enum fuse_buf_copy_flags. This argument can be removed and the check below can be changed from:

Quoted 5 lines

if (flags & FUSE_BUF_FD_SEEK)

to

if (out_buf->buf[0].flags & FUSE_BUF_FD_SEEK)

Accepted, and will be fixed in PATCH v4.

Quoted 10 lines

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 83

Quoted 6 lines
contrib/virtiofsd/buffer.c +5 -0
+{
+	ssize_t res, i, buf_index, iovcnt;
+	struct iovec * iov;
+	int fd = out_buf->buf[0].fd;
+	off_t pos = out_buf->buf[0].pos;

A struct fuse_bufvec may have multiple elements but this function assumes it only has 1. Please use struct fuse_buf *out_buf instead. This way it's clear that only 1 fuse_buf will be written.

Do you mean that assign *out_buf with &out_buf->buf[0] in fuse_buf_writev() and then operate out_buf in the following process?

Quoted 17 lines
Quoted 5 lines
contrib/virtiofsd/buffer.c +4 -0
+
+	if (in_buf->count > 2)
+		iovcnt = in_buf->count - 1;
+	else

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 97

Quoted 10 lines
contrib/virtiofsd/buffer.c +10 -0
+		iovcnt = 1;
+
+	iov = calloc(iovcnt, sizeof(struct iovec));
+	if (!iov)
+		return -ENOMEM;
+
+	for (i = 0, buf_index = 1; i < iovcnt; i++, buf_index++) {
+		iov[i].iov_base = in_buf->buf[buf_index].mem;
+		iov[i].iov_len = in_buf->buf[buf_index].size;
+	}

Why is in_buf->buf[0] is skipped?

This part of code seems a little complex and in_buf->buf[0].count just tricks me. Maybe I need handle two situations:
1. Iterate from in_buf->buf[0].mem when in_buf->buf[0].count is 1;
2. buf[0].size is set 0 in do_write_buf() when in_buf->buf[0].count > 1, and it's better to skip it. Perhaps it's also ok to iterate from buf[0], as writev skip the iov with zero len. This looks like a hack.

Quoted 8 lines

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 100

Quoted 4 lines
contrib/virtiofsd/buffer.c +3 -0
+
+	if (flags & FUSE_BUF_FD_SEEK)
+		res = pwritev(fd, iov, iovcnt, pos);

Please move off_t pos = out_buf->buf[0].pos into this if statement body to avoid a possible uninitialized memory access when !(flags & FUSE_BUF_FD_SEEK). This makes valgrind and other tools happy.

Accepted.

Thanks,
Jun

Quoted 1 line
#5 lore raw

Re: [Virtio-fs] [PATCH v3 1/2] virtiofsd: add definition of fuse_buf_writev()

Quoted 12 lines · piaojun
Quoted 10 lines · Stefan Hajnoczi

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 83

Quoted 5 lines · piaojun
contrib/virtiofsd/buffer.c +5 -0
+{
+	ssize_t res, i, buf_index, iovcnt;
+	struct iovec * iov;
+	int fd = out_buf->buf[0].fd;
+	off_t pos = out_buf->buf[0].pos;

A struct fuse_bufvec may have multiple elements but this function assumes it only has 1. Please use struct fuse_buf *out_buf instead. This way it's clear that only 1 fuse_buf will be written.

Quoted 2 lines

Do you mean that assign *out_buf with &out_buf->buf[0] in fuse_buf_writev() and then operate out_buf in the following process?

Since this function assumes that out_buf is only a single struct fuse_buf, please change the argument to struct fuse_buf instead of struct fuse_bufvec. This way it's clear what the function expects.

Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
contrib/virtiofsd/buffer.c +4 -0
+
+	if (in_buf->count > 2)
+		iovcnt = in_buf->count - 1;
+	else

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 97

Quoted 10 lines
contrib/virtiofsd/buffer.c +10 -0
+		iovcnt = 1;
+
+	iov = calloc(iovcnt, sizeof(struct iovec));
+	if (!iov)
+		return -ENOMEM;
+
+	for (i = 0, buf_index = 1; i < iovcnt; i++, buf_index++) {
+		iov[i].iov_base = in_buf->buf[buf_index].mem;
+		iov[i].iov_len = in_buf->buf[buf_index].size;
+	}

Why is in_buf->buf[0] is skipped?

Quoted 6 lines

This part of code seems a little complex and in_buf->buf[0].count just tricks me. Maybe I need handle two situations:
1. Iterate from in_buf->buf[0].mem when in_buf->buf[0].count is 1;
2. buf[0].size is set 0 in do_write_buf() when in_buf->buf[0].count > 1, and it's better to skip it. Perhaps it's also ok to iterate from buf[0], as writev skip the iov with zero len. This looks like a hack.

I would perform writev() with all of in_buf's buffers, even if some of them have 0 length. That way this function does not make any assumptions about in_buf's layout.

The only requirement is that in_buf buffers are all memory buffers (not file descriptors).

author
#6 lore raw

Re: [Virtio-fs] [PATCH v3 1/2] virtiofsd: add definition of fuse_buf_writev()

Hi Stefan,

Quoted 13 lines · Stefan Hajnoczi
Quoted 12 lines · piaojun
Quoted 10 lines · Stefan Hajnoczi

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 83

Quoted 5 lines · piaojun
contrib/virtiofsd/buffer.c +5 -0
+{
+	ssize_t res, i, buf_index, iovcnt;
+	struct iovec * iov;
+	int fd = out_buf->buf[0].fd;
+	off_t pos = out_buf->buf[0].pos;

A struct fuse_bufvec may have multiple elements but this function assumes it only has 1. Please use struct fuse_buf *out_buf instead. This way it's clear that only 1 fuse_buf will be written.

Quoted 6 lines
Quoted 2 lines

Do you mean that assign *out_buf with &out_buf->buf[0] in fuse_buf_writev() and then operate out_buf in the following process?

Since this function assumes that out_buf is only a single struct fuse_buf, please change the argument to struct fuse_buf instead of struct fuse_bufvec. This way it's clear what the function expects.

Agreed, already fixed in PATCH v4.

Quoted 26 lines
Quoted 19 lines
Quoted 18 lines
Quoted 6 lines
contrib/virtiofsd/buffer.c +4 -0
+
+	if (in_buf->count > 2)
+		iovcnt = in_buf->count - 1;
+	else

contrib/virtiofsd/buffer.c static ssize_t fuse_buf_write(const struct fuse_buf *dst, size_t dst_off, · line 97

Quoted 10 lines
contrib/virtiofsd/buffer.c +10 -0
+		iovcnt = 1;
+
+	iov = calloc(iovcnt, sizeof(struct iovec));
+	if (!iov)
+		return -ENOMEM;
+
+	for (i = 0, buf_index = 1; i < iovcnt; i++, buf_index++) {
+		iov[i].iov_base = in_buf->buf[buf_index].mem;
+		iov[i].iov_len = in_buf->buf[buf_index].size;
+	}

Why is in_buf->buf[0] is skipped?

Quoted 6 lines

This part of code seems a little complex and in_buf->buf[0].count just tricks me. Maybe I need handle two situations:
1. Iterate from in_buf->buf[0].mem when in_buf->buf[0].count is 1;
2. buf[0].size is set 0 in do_write_buf() when in_buf->buf[0].count > 1, and it's better to skip it. Perhaps it's also ok to iterate from buf[0], as writev skip the iov with zero len. This looks like a hack.

Quoted 3 lines

I would perform writev() with all of in_buf's buffers, even if some of them have 0 length. That way this function does not make any assumptions about in_buf's layout.

Agreed, already fixed in PATCH v4.

Jun

Quoted 4 lines

The only requirement is that in_buf buffers are all memory buffers (not file descriptors).