Skip to contentSkip to conversation

virtiofsd: print more verbose information when bailing out

PATCH In review Liu Bo 2 participants 6 messages Mar 20, 2020 – Mar 24, 2020 Arm/actions semi architecture

lore
Share & tools
lore

Liu Bo

author
#0 lore raw

PATCH v2 1/2 virtiofsd: print more verbose information when bailing out

It'd be helpful to know what is the exact value of arg's offset, size and flags.

---
 tools/virtiofsd/fuse_lowlevel.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/virtiofsd/fuse_lowlevel.c +2 -2
index a34a611..ca2056f 100644
@@ -1224,8 +1224,8 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid,
     }
 
     if (fuse_buf_size(pbufv) != arg->size) {
-        fuse_log(FUSE_LOG_ERR,
-                 "fuse: do_write_buf: buffer size doesn't match arg->size\n");
+        fuse_log(FUSE_LOG_ERR, "fuse: do_write_buf: buffer size %lu doesn't match arg->size %u offset %lu flags %u\n",
+                 fuse_buf_size(pbufv), arg->size, arg->offset, arg->write_flags);
         fuse_reply_err(req, EIO);
         return;
     }
Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH v2 2/2 virtiofsd: fix mmap write under nondax mode

When a file size is not aligned to PAGE_SIZE, a mmap write on it may encounter -EIO (can be observed from virtiofsd's log) due to the difference between the buf size and the size recorded in struct fuse_write_in. The difference comes from the fact that for mmap, writeback IO is used and guest kernel sets fuse_write_in's size to inode size if EOF, while the buf len still remains PAGE_SIZE aligned.

This handles the above special mmap case by truncating the last buf'size.

Fixes Commit 469f9d2f ("virtiofsd: Plumb fuse_bufvec through do_write_buf")
---
 tools/virtiofsd/fuse_lowlevel.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/virtiofsd/fuse_lowlevel.c +17 -0
index ca2056f..4f8bfb6 100644
@@ -1221,6 +1221,23 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid,
          * and the data in the rest, we need to skip that first element
          */
         ibufv->buf[0].size = 0;
+
+        /*
+         * In case of mmap, fuse_buf_size(pbufv) may need to truncate if
+         * arg->size has been cropped by inode size inside guest.  The
+         * diff can only be (0, PAGE_SIZE) because inode size must be
+         * overlapped with the last buf.
+         */
+        if (arg->write_flags & FUSE_WRITE_CACHE) {
+                size_t total = fuse_buf_size(pbufv);
+                int last = ibufv->count - 1;
+
+                if (total > arg->size) {
+                        size_t diff = total - arg->size;
+                        if (diff < ibufv->buf[last].size)
+                                ibufv->buf[last].size -= diff;
+                }
+        }
     }
 
     if (fuse_buf_size(pbufv) != arg->size) {
Signature · 1 line
1.8.3.1
#2 lore raw

Re: [Virtio-fs] [PATCH v2 2/2] virtiofsd: fix mmap write under nondax mode

Quoted 8 lines · * Liu Bo ([email protected]) wrote:

When a file size is not aligned to PAGE_SIZE, a mmap write on it may encounter -EIO (can be observed from virtiofsd's log) due to the difference between the buf size and the size recorded in struct fuse_write_in. The difference comes from the fact that for mmap, writeback IO is used and guest kernel sets fuse_write_in's size to inode size if EOF, while the buf len still remains PAGE_SIZE aligned.

This handles the above special mmap case by truncating the last buf'size.

Thanks,

Quoted 13 lines
Fixes Commit 469f9d2f ("virtiofsd: Plumb fuse_bufvec through do_write_buf")
---
 tools/virtiofsd/fuse_lowlevel.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/virtiofsd/fuse_lowlevel.c +0 -0
index ca2056f..4f8bfb6 100644
@@ -1221,6 +1221,23 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid,
          * and the data in the rest, we need to skip that first element

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1231

Quoted 10 lines
tools/virtiofsd/fuse_lowlevel.c +8 -0
          */
         ibufv->buf[0].size = 0;
+
+        /*
+         * In case of mmap, fuse_buf_size(pbufv) may need to truncate if
+         * arg->size has been cropped by inode size inside guest.  The
+         * diff can only be (0, PAGE_SIZE) because inode size must be
+         * overlapped with the last buf.
+         */
+        if (arg->write_flags & FUSE_WRITE_CACHE) {

Does this need to only do it in the WRITE_CACHE case - or should we just always truncate the write to arg->size?
Or is this just simpler?

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1238

Quoted 7 lines
tools/virtiofsd/fuse_lowlevel.c +7 -0
+                size_t total = fuse_buf_size(pbufv);
+                int last = ibufv->count - 1;
+
+                if (total > arg->size) {
+                        size_t diff = total - arg->size;
+                        if (diff < ibufv->buf[last].size)
+                                ibufv->buf[last].size -= diff;

I think that needs to modify pbufv->buf[last].size not ibufv because the two are only the same in some cases (although it's possible in this case the guest we try at the moment always falls in this side).

We should also do something in the else case - probably fail?

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1243

Quoted 5 lines
tools/virtiofsd/fuse_lowlevel.c +2 -0
+                }
+        }
     }
 
     if (fuse_buf_size(pbufv) != arg->size) {

If we now know that pbufv is now always shrung to size, then we only now need to check for the case where pbufv is too small.

Dave

Quoted 8 lines
Signature · 7 lines
1.8.3.1


_______________________________________________
Virtio-fs mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virtio-fs

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

author
#3 lore raw

Re: [Virtio-fs] [PATCH v2 2/2] virtiofsd: fix mmap write under nondax mode

Quoted 31 lines · Dr. David Alan Gilbert
Quoted 8 lines · * Liu Bo ([email protected]) wrote:

When a file size is not aligned to PAGE_SIZE, a mmap write on it may encounter -EIO (can be observed from virtiofsd's log) due to the difference between the buf size and the size recorded in struct fuse_write_in. The difference comes from the fact that for mmap, writeback IO is used and guest kernel sets fuse_write_in's size to inode size if EOF, while the buf len still remains PAGE_SIZE aligned.

This handles the above special mmap case by truncating the last buf'size.

Thanks,

Quoted 19 lines
Fixes Commit 469f9d2f ("virtiofsd: Plumb fuse_bufvec through do_write_buf")
---
 tools/virtiofsd/fuse_lowlevel.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/virtiofsd/fuse_lowlevel.c +4 -0
index ca2056f..4f8bfb6 100644
@@ -1221,6 +1221,23 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid,
          * and the data in the rest, we need to skip that first element
          */
         ibufv->buf[0].size = 0;
+
+        /*
+         * In case of mmap, fuse_buf_size(pbufv) may need to truncate if
+         * arg->size has been cropped by inode size inside guest.  The
Quoted 8 lines

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1231

Quoted 4 lines
tools/virtiofsd/fuse_lowlevel.c +4 -0
+         * diff can only be (0, PAGE_SIZE) because inode size must be
+         * overlapped with the last buf.
+         */
+        if (arg->write_flags & FUSE_WRITE_CACHE) {

Does this need to only do it in the WRITE_CACHE case - or should we just always truncate the write to arg->size?
Or is this just simpler?

For non-mmap IO, AFAICS, it's all synchronous IO (not using writepages) where the data part's length should be equal to arg->size here. So I think it's no harm to do it for both.

Quoted 9 lines

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1238

Quoted 8 lines
tools/virtiofsd/fuse_lowlevel.c +7 -0
+                size_t total = fuse_buf_size(pbufv);
+                int last = ibufv->count - 1;
+
+                if (total > arg->size) {
+                        size_t diff = total - arg->size;
+                        if (diff < ibufv->buf[last].size)
+                                ibufv->buf[last].size -= diff;
Quoted 4 lines

I think that needs to modify pbufv->buf[last].size not ibufv because the two are only the same in some cases (although it's possible in this case the guest we try at the moment always falls in this side).

OK.

Quoted 2 lines

We should also do something in the else case - probably fail?

If it fails, it then gets to the following check and report -EIO.

Quoted 11 lines

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1243

Quoted 5 lines
tools/virtiofsd/fuse_lowlevel.c +2 -0
+                }
+        }
     }
 
     if (fuse_buf_size(pbufv) != arg->size) {

If we now know that pbufv is now always shrung to size, then we only now need to check for the case where pbufv is too small.

From my understanding about both mmap IO and nonmmap IO, I think it's

arg->size that is always <= pbufv size.

thanks,
-liubo

#4 lore raw

Re: [Virtio-fs] [PATCH v2 2/2] virtiofsd: fix mmap write under nondax mode

Quoted 41 lines · * Liu Bo ([email protected]) wrote:
Quoted 31 lines · Dr. David Alan Gilbert
Quoted 8 lines · * Liu Bo ([email protected]) wrote:

When a file size is not aligned to PAGE_SIZE, a mmap write on it may encounter -EIO (can be observed from virtiofsd's log) due to the difference between the buf size and the size recorded in struct fuse_write_in. The difference comes from the fact that for mmap, writeback IO is used and guest kernel sets fuse_write_in's size to inode size if EOF, while the buf len still remains PAGE_SIZE aligned.

This handles the above special mmap case by truncating the last buf'size.

Thanks,

Quoted 19 lines
Fixes Commit 469f9d2f ("virtiofsd: Plumb fuse_bufvec through do_write_buf")
---
 tools/virtiofsd/fuse_lowlevel.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/virtiofsd/fuse_lowlevel.c +4 -0
index ca2056f..4f8bfb6 100644
@@ -1221,6 +1221,23 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid,
          * and the data in the rest, we need to skip that first element
          */
         ibufv->buf[0].size = 0;
+
+        /*
+         * In case of mmap, fuse_buf_size(pbufv) may need to truncate if
+         * arg->size has been cropped by inode size inside guest.  The
Quoted 8 lines

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1231

Quoted 4 lines
tools/virtiofsd/fuse_lowlevel.c +4 -0
+         * diff can only be (0, PAGE_SIZE) because inode size must be
+         * overlapped with the last buf.
+         */
+        if (arg->write_flags & FUSE_WRITE_CACHE) {

Does this need to only do it in the WRITE_CACHE case - or should we just always truncate the write to arg->size?
Or is this just simpler?

Quoted 3 lines

For non-mmap IO, AFAICS, it's all synchronous IO (not using writepages) where the data part's length should be equal to arg->size here. So I think it's no harm to do it for both.

OK, good that would simplify it.

Quoted 14 lines
Quoted 9 lines

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1238

Quoted 8 lines
tools/virtiofsd/fuse_lowlevel.c +7 -0
+                size_t total = fuse_buf_size(pbufv);
+                int last = ibufv->count - 1;
+
+                if (total > arg->size) {
+                        size_t diff = total - arg->size;
+                        if (diff < ibufv->buf[last].size)
+                                ibufv->buf[last].size -= diff;
Quoted 4 lines

I think that needs to modify pbufv->buf[last].size not ibufv because the two are only the same in some cases (although it's possible in this case the guest we try at the moment always falls in this side).

Quoted 6 lines

OK.

Quoted 2 lines

We should also do something in the else case - probably fail?

If it fails, it then gets to the following check and report -EIO.

Yes, but that error doesn't tell us why; so we'll see that error and not realise that it was because we failed to do this truncation.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
tools/virtiofsd/fuse_lowlevel.c +2 -0
+                }
+        }
     }
 
Quoted 8 lines
Quoted 5 lines

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1243

Quoted 1 line
tools/virtiofsd/fuse_lowlevel.c +0 -0
     if (fuse_buf_size(pbufv) != arg->size) {

If we now know that pbufv is now always shrung to size, then we only now need to check for the case where pbufv is too small.

From my understanding about both mmap IO and nonmmap IO, I think it's arg->size that is always <= pbufv size.

It should do, but this is a sanity check - remember we don't trust the guest.

Dave

Quoted 3 lines

thanks,
-liubo

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#5 lore raw
In reply to Liu Bo
Quoted 4 lines · * Liu Bo ([email protected]) wrote:

It'd be helpful to know what is the exact value of arg's offset, size and flags.

Thanks,

Quoted 9 lines
---
 tools/virtiofsd/fuse_lowlevel.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/virtiofsd/fuse_lowlevel.c +0 -0
index a34a611..ca2056f 100644

tools/virtiofsd/fuse_lowlevel.c static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid, · line 1228

Quoted 7 lines
tools/virtiofsd/fuse_lowlevel.c +2 -2
@@ -1224,8 +1224,8 @@ static void do_write_buf(fuse_req_t req, fuse_ino_t nodeid,
     }
 
     if (fuse_buf_size(pbufv) != arg->size) {
-        fuse_log(FUSE_LOG_ERR,
-                 "fuse: do_write_buf: buffer size doesn't match arg->size\n");
+        fuse_log(FUSE_LOG_ERR, "fuse: do_write_buf: buffer size %lu doesn't match arg->size %u offset %lu flags %u\n",
+                 fuse_buf_size(pbufv), arg->size, arg->offset, arg->write_flags);

Please use %zu for size_t's (i.e. fuse_buf_size) and %llu with a cast to (unsigned long long) for uint64_t's; also it's gone over the maximum line limit of 80 characters for qemu.

Dave

Quoted 11 lines
tools/virtiofsd/fuse_lowlevel.c +0 -0
         fuse_reply_err(req, EIO);
         return;
     }
Signature · 7 lines
1.8.3.1


_______________________________________________
Virtio-fs mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virtio-fs

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK