Skip to contentSkip to conversation

virtiofsd: Open fd O_RDONLY in setxattr/removexattr

PATCH In review Vivek Goyal 3 participants 6 messages Jan 8, 2020 – Jan 10, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 virtiofsd: Open fd O_RDONLY in setxattr/removexattr

Do not open fd O_RDWR as it will fail for directories with EISDIR. This code can be called both for regular files as well as directories.

I noticed this when I tried "setfattr -n user.foo -v test <dir>" inside the guest and got EISDIR.

To write xattr, we don't have to open fd with write permissions. Looks like kernel will do permission checks on inode.

---
 contrib/virtiofsd/passthrough_ll.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
contrib/virtiofsd/passthrough_ll.c +2 -2
Index: qemu/contrib/virtiofsd/passthrough_ll.c
===================================================================
@@ -2424,7 +2424,7 @@ static void lo_setxattr(fuse_req_t req,
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
@@ -2473,7 +2473,7 @@ static void lo_removexattr(fuse_req_t re
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
Reviewed-by on v1 1/1
#1 lore raw
Quoted 10 lines · Vivek Goyal

Do not open fd O_RDWR as it will fail for directories with EISDIR. This code can be called both for regular files as well as directories.

I noticed this when I tried "setfattr -n user.foo -v test <dir>" inside the guest and got EISDIR.

To write xattr, we don't have to open fd with write permissions. Looks like kernel will do permission checks on inode.

Looks good to me.

Quoted 31 lines
---
 contrib/virtiofsd/passthrough_ll.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: qemu/contrib/virtiofsd/passthrough_ll.c
===================================================================

contrib/virtiofsd/passthrough_ll.c +2 -2
@@ -2424,7 +2424,7 @@ static void lo_setxattr(fuse_req_t req,
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
@@ -2473,7 +2473,7 @@ static void lo_removexattr(fuse_req_t re
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;

_______________________________________________
Virtio-fs mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virtio-fs

#2 lore raw
Quoted 36 lines · Vivek Goyal
Quoted 10 lines

Do not open fd O_RDWR as it will fail for directories with EISDIR. This code can be called both for regular files as well as directories.

I noticed this when I tried "setfattr -n user.foo -v test <dir>" inside the guest and got EISDIR.

To write xattr, we don't have to open fd with write permissions. Looks like kernel will do permission checks on inode.

Looks good to me.

Quoted 20 lines
---
 contrib/virtiofsd/passthrough_ll.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: qemu/contrib/virtiofsd/passthrough_ll.c
===================================================================

contrib/virtiofsd/passthrough_ll.c +1 -1
@@ -2424,7 +2424,7 @@ static void lo_setxattr(fuse_req_t req,
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
@@ -2473,7 +2473,7 @@ static void lo_removexattr(fuse_req_t re
 	}
 
Quoted 7 lines

contrib/virtiofsd/passthrough_ll.c static void lo_removexattr(fuse_req_t re · line 2479

Quoted 7 lines
contrib/virtiofsd/passthrough_ll.c +1 -1
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;

Hello,

So I sent the same patch last October[1] and got comments
that it's would be better to fix the fundamental problem
(do not call openat() to non regular file/directory) [2].
[1] https://www.redhat.com/archives/virtio-fs/2019-October/msg00030.html
[2] https://www.redhat.com/archives/virtio-fs/2019-October/msg00046.html

I tried the suggested approach which uses fchdir(proc_self_fd) + ...xattr() + fchdir(root.fd) combination instead of current openat() + f...xattr().
However, it seems that always fchdir on xattr operation for regular files too incurs some performance overhead (~10% or more).

So I think hybrid approach is better:
For regular file/directory: use f...xattr
For other file types: use fchdir + ...xattr + fchdir

Attached patch adopts the above solution.
So could you check the patch's approach? If it is fine, I will send it as a formal patch (and I apologize that my response is delayed).

Thanks,
Misono

From 4be65d24ab400aa4322116c0f9cc79a35e5e500f Mon Sep 17 00:00:00 2001
From: Misono Tomohiro <[email protected]>
Date: Thu, 9 Jan 2020 14:10:53 +0900
Subject: [PATCH] virtiofs: Fix xattr operations

Current virtiofsd has problems about xattr operations and they does not work properly for directory/symlink/special file.

The fundamental cause is that virtiofsd uses openat() + f...xattr() systemcalls for xattr operation but we should not open symlink/special file in the daemon. Therefore the function is restricted.

Fix this problem by:
1. during setup of each thread, call unshare(CLONE_FS)
2. in xattr operations (i.e. lo_getxattr), if inode is not a regular file or directory, use fchdir(proc_loot_fd) + ...xattr() + fchdir(root.fd) instead of openat() + f...xattr()

(Note: for a regular file/directory openat() + f...xattr()
is still used for performance reason)

With this patch, xfstests generic/062 passes on virtiofs. This fix is suggested by Miklos Szeredi and Stefan Hajnoczi.

---
 tools/virtiofsd/fuse_virtio.c    |  13 +++
 tools/virtiofsd/passthrough_ll.c | 167 +++++++++++++++++++++++--------
 tools/virtiofsd/seccomp.c        |   6 ++
 3 files changed, 142 insertions(+), 44 deletions(-)
tools/virtiofsd/fuse_virtio.c +13 -0
index 254b0a71cd..99339e17b2 100644
@@ -460,6 +460,8 @@ err:
     return ret;
 }
 
+static __thread bool clone_fs_called;
+
 /* Process one FVRequest in a thread pool */
 static void fv_queue_worker(gpointer data, gpointer user_data)
 {
@@ -475,6 +477,17 @@ static void fv_queue_worker(gpointer data, gpointer user_data)
 
     assert(se->bufsize > sizeof(struct fuse_in_header));
 
+    if (!clone_fs_called) {
+        int ret;
+
+        /* unshare FS for xattr operation */
+        ret = unshare(CLONE_FS);
+        /* should not fail */
+        assert(ret == 0);
+
+        clone_fs_called = true;
+    }
+
     /*
      * An element contains one request and the space to send our response
      * They're spread over multiple descriptors in a scatter/gather set
tools/virtiofsd/passthrough_ll.c +123 -44

289 lines not on this page · raw message

tools/virtiofsd/seccomp.c +6 -0
index 33ec309fc8..87048c239e 100644
@@ -37,6 +37,7 @@ static const int syscall_whitelist[] = {
     SCMP_SYS(exit),
     SCMP_SYS(exit_group),
     SCMP_SYS(fallocate),
+    SCMP_SYS(fchdir),
     SCMP_SYS(fchmodat),
     SCMP_SYS(fchownat),
     SCMP_SYS(fcntl),
@@ -58,7 +59,9 @@ static const int syscall_whitelist[] = {
     SCMP_SYS(getpid),
     SCMP_SYS(gettid),
     SCMP_SYS(gettimeofday),
+    SCMP_SYS(getxattr),
     SCMP_SYS(linkat),
+    SCMP_SYS(listxattr),
     SCMP_SYS(lseek),
     SCMP_SYS(madvise),
     SCMP_SYS(mkdirat),
@@ -81,6 +84,7 @@ static const int syscall_whitelist[] = {
     SCMP_SYS(recvmsg),
     SCMP_SYS(renameat),
     SCMP_SYS(renameat2),
+    SCMP_SYS(removexattr),
     SCMP_SYS(rt_sigaction),
     SCMP_SYS(rt_sigprocmask),
     SCMP_SYS(rt_sigreturn),
@@ -88,10 +92,12 @@ static const int syscall_whitelist[] = {
     SCMP_SYS(setresgid),
     SCMP_SYS(setresuid),
     SCMP_SYS(set_robust_list),
+    SCMP_SYS(setxattr),
     SCMP_SYS(symlinkat),
     SCMP_SYS(time), /* Rarely needed, except on static builds */
     SCMP_SYS(tgkill),
     SCMP_SYS(unlinkat),
+    SCMP_SYS(unshare),
     SCMP_SYS(utimensat),
     SCMP_SYS(write),
     SCMP_SYS(writev),
Signature · 1 line
2.21.1
#3 lore raw
Quoted 46 lines · On Thu, Jan 09, 2020 at 09:27:15AM +0000, [email protected] wrote:
Quoted 36 lines · Vivek Goyal
Quoted 10 lines

Do not open fd O_RDWR as it will fail for directories with EISDIR. This code can be called both for regular files as well as directories.

I noticed this when I tried "setfattr -n user.foo -v test <dir>" inside the guest and got EISDIR.

To write xattr, we don't have to open fd with write permissions. Looks like kernel will do permission checks on inode.

Looks good to me.

Quoted 20 lines
---
 contrib/virtiofsd/passthrough_ll.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: qemu/contrib/virtiofsd/passthrough_ll.c
===================================================================

contrib/virtiofsd/passthrough_ll.c +1 -1
@@ -2424,7 +2424,7 @@ static void lo_setxattr(fuse_req_t req,
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
@@ -2473,7 +2473,7 @@ static void lo_removexattr(fuse_req_t re
 	}
 
Quoted 7 lines

contrib/virtiofsd/passthrough_ll.c static void lo_removexattr(fuse_req_t re · line 2479

Quoted 7 lines
contrib/virtiofsd/passthrough_ll.c +1 -1
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;

Hello,

Quoted 3 lines

So I sent the same patch last October[1] and got comments that it's would be better to fix the fundamental problem (do not call openat() to non regular file/directory) [2].

Right. So this patch is only solving the issue for directories (and not for symlinks other special files). We will need more improvements in this area.

I am trying to make overlayfs work on top of virtiofs so that it can be tested and see if it works well or not. Overlayfs does bunch of xattr operations on work/work directory and it fails. So this patch will atleast fix that and allow me to make progress.

Quoted 11 lines

[1] https://www.redhat.com/archives/virtio-fs/2019-October/msg00030.html
[2] https://www.redhat.com/archives/virtio-fs/2019-October/msg00046.html

I tried the suggested approach which uses fchdir(proc_self_fd) + ...xattr() + fchdir(root.fd) combination instead of current openat() + f...xattr().
However, it seems that always fchdir on xattr operation for regular files too incurs some performance overhead (~10% or more).

So I think hybrid approach is better:
For regular file/directory: use f...xattr
For other file types: use fchdir + ...xattr + fchdir

This sounds reasonable. I will give it a try. In fact Miklos's patch for allowing fsetxattr() on O_PATH is still being reviewed upstream. Once that is merged, our approach should probably be.

1. Try fsetxattr(O_PATH fd).
2. If 1 fails, try openat() followed by fsetxattr() for regular files and dirs.
3. For other file types use fchdir + ...xattr() + fchdir.

Thanks
Vivek

Quoted 7 lines


Attached patch adopts the above solution.
So could you check the patch's approach? If it is fine, I will send it as a formal patch (and I apologize that my response is delayed).

Thanks,
Misono

#4 lore raw
In reply to [email protected]
Quoted 46 lines · On Thu, Jan 09, 2020 at 09:27:15AM +0000, [email protected] wrote:
Quoted 36 lines · Vivek Goyal
Quoted 10 lines

Do not open fd O_RDWR as it will fail for directories with EISDIR. This code can be called both for regular files as well as directories.

I noticed this when I tried "setfattr -n user.foo -v test <dir>" inside the guest and got EISDIR.

To write xattr, we don't have to open fd with write permissions. Looks like kernel will do permission checks on inode.

Looks good to me.

Quoted 20 lines
---
 contrib/virtiofsd/passthrough_ll.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: qemu/contrib/virtiofsd/passthrough_ll.c
===================================================================

contrib/virtiofsd/passthrough_ll.c +1 -1
@@ -2424,7 +2424,7 @@ static void lo_setxattr(fuse_req_t req,
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
@@ -2473,7 +2473,7 @@ static void lo_removexattr(fuse_req_t re
 	}
 
Quoted 7 lines

contrib/virtiofsd/passthrough_ll.c static void lo_removexattr(fuse_req_t re · line 2479

Quoted 7 lines
contrib/virtiofsd/passthrough_ll.c +1 -1
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;

Hello,

Quoted 16 lines

So I sent the same patch last October[1] and got comments
that it's would be better to fix the fundamental problem
(do not call openat() to non regular file/directory) [2].
[1] https://www.redhat.com/archives/virtio-fs/2019-October/msg00030.html
[2] https://www.redhat.com/archives/virtio-fs/2019-October/msg00046.html

I tried the suggested approach which uses fchdir(proc_self_fd) + ...xattr() + fchdir(root.fd) combination instead of current openat() + f...xattr().
However, it seems that always fchdir on xattr operation for regular files too incurs some performance overhead (~10% or more).

So I think hybrid approach is better:
For regular file/directory: use f...xattr
For other file types: use fchdir + ...xattr + fchdir

Attached patch adopts the above solution.

Hi Misono,

Can you post this patch again separately so that it is easier to give comments and discuss this patch. I have a quick look and this patch does look reasonable.

Only minor concern I have is that now each thread has done unshare(CLONE_FS). Will we run into use cases where we want to enforce a particular change aross all threads.

Thanks
Vivek

#5 lore raw
Quoted 63 lines · On Thu, Jan 09, 2020 at 09:27:15AM +0000, [email protected] wrote:
Quoted 62 lines
Quoted 36 lines · Vivek Goyal
Quoted 10 lines

Do not open fd O_RDWR as it will fail for directories with EISDIR. This code can be called both for regular files as well as directories.

I noticed this when I tried "setfattr -n user.foo -v test <dir>" inside the guest and got EISDIR.

To write xattr, we don't have to open fd with write permissions. Looks like kernel will do permission checks on inode.

Looks good to me.

Quoted 20 lines
---
 contrib/virtiofsd/passthrough_ll.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: qemu/contrib/virtiofsd/passthrough_ll.c
===================================================================

contrib/virtiofsd/passthrough_ll.c +1 -1
@@ -2424,7 +2424,7 @@ static void lo_setxattr(fuse_req_t req,
 	}
 
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;
@@ -2473,7 +2473,7 @@ static void lo_removexattr(fuse_req_t re
 	}
 
Quoted 7 lines

contrib/virtiofsd/passthrough_ll.c static void lo_removexattr(fuse_req_t re · line 2479

Quoted 7 lines
contrib/virtiofsd/passthrough_ll.c +1 -1
 	sprintf(procname, "%i", inode->fd);
-	fd = openat(lo->proc_self_fd, procname, O_RDWR);
+	fd = openat(lo->proc_self_fd, procname, O_RDONLY);
 	if (fd < 0) {
 		saverr = errno;
 		goto out;

Hello,

So I sent the same patch last October[1] and got comments that it's would be better to fix the fundamental problem (do not call openat() to non regular file/directory) [2].
[1] https://www.redhat.com/archives/virtio-fs/2019-October/msg00030.html
[2]
https://www.redhat.com/archives/virtio-fs/2019-October/msg00046.html

I tried the suggested approach which uses fchdir(proc_self_fd) + ...xattr() + fchdir(root.fd) combination instead of current openat() + f...xattr(). However, it seems that always fchdir on xattr operation for regular files too incurs some performance overhead (~10% or more).

So I think hybrid approach is better:
For regular file/directory: use f...xattr For other file types: use fchdir + ...xattr + fchdir

Quoted 6 lines
Quoted 1 line

Attached patch adopts the above solution.

Hi Misono,

Can you post this patch again separately so that it is easier to give comments and discuss this patch. I have a quick look and this patch does look reasonable.

Thanks for the check, I will post it shortly.
Misono