Skip to contentSkip to conversation

virtiofsd: Add notion of unprivileged mode

RFC PATCH Superseded Vivek Goyal 3 participants 9 messages Jul 29, 2020 – Jul 30, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover virtiofsd: Add notion of unprivileged mode

Hi,

Daniel Berrange mentioned that having a unpriviliged mode in virtiofsd might be useful for certain use cases. Hence I decided to give it a try.

This is RFC patch series to allow running virtiofsd as unpriviliged user. This is still work in progress. I am posting it to get some early feedback.

These patches are dependent on Stefan's patch series for sandbox=chroot.

https://www.redhat.com/archives/virtio-fs/2020-July/msg00078.html

I can now run virtiofsd as user "test" and also export a directory into a VM running as user test.

This is ideally for the cases where user "test" inside VM will operate on this virtiofs mount point. Any filesystem operations which can't be done with the creds of "test" user on host, will fail.

Thanks
Vivek

Vivek Goyal (5):
virtiofsd: Add notion of unprivileged mode
virtiofsd: create lock/pid file in per user cache dir
virtiofsd: open /proc/self/fd/ in sandbox=NONE mode
virtiofsd: Open lo->source while setting up root in sandbox=NONE mode
virtiofsd: Skip setup_capabilities() in sandbox=NONE mode

 tools/virtiofsd/fuse_virtio.c    | 40 ++++++++++++++++++++++++++++----
 tools/virtiofsd/passthrough_ll.c | 29 ++++++++++++++++++++---
 2 files changed, 61 insertions(+), 8 deletions(-)
Signature · 1 line
2.25.4
#1 lore raw

PATCH 1/5 virtiofsd: Add notion of unprivileged mode

At startup if we are running as non-root user, then internall set unpriviliged mode set. Also add a notion of sandbox NONE and set that internally in unprivileged mode. setting up namespaces and chroot() fails when one does not have privileges.

---
 tools/virtiofsd/passthrough_ll.c | 8 ++++++++
 1 file changed, 8 insertions(+)
tools/virtiofsd/passthrough_ll.c +8 -0
index e2fbc614fd..cd91c4a831 100644
@@ -147,11 +147,13 @@ enum {
 enum {
     SANDBOX_NAMESPACE,
     SANDBOX_CHROOT,
+    SANDBOX_NONE,
 };
 
 struct lo_data {
     pthread_mutex_t mutex;
     int sandbox;
+    bool unprivileged;
     int debug;
     int writeback;
     int flock;
@@ -3288,6 +3290,12 @@ int main(int argc, char *argv[])
     lo_map_init(&lo.dirp_map);
     lo_map_init(&lo.fd_map);
 
+    if (geteuid() != 0) {
+       lo.unprivileged = true;
+       lo.sandbox = SANDBOX_NONE;
+       fuse_log(FUSE_LOG_DEBUG, "Running in unprivileged passthrough mode.\n");
+    }
+
     if (fuse_parse_cmdline(&args, &opts) != 0) {
         goto err_out1;
     }
Signature · 1 line
2.25.4
#2 lore raw

PATCH 2/5 virtiofsd: create lock/pid file in per user cache dir

In reply to Vivek Goyal

Right now we create lock/pid file in /usr/local/var/... and unprivliged user does not have access to create files there.

So create this file in per user cache dir as queried as specified by environment variable XDG_RUNTIME_DIR.

Note: "su $USER" does not update XDG_RUNTIME_DIR and it still points to root user's director. So for now I create a directory /tmp/$UID to save lock/pid file. Dan pointed out that it can be a problem if a malicious app already has /tmp/$UID created. So we probably need to get rid of this.

---
 tools/virtiofsd/fuse_virtio.c | 40 ++++++++++++++++++++++++++++++-----
 1 file changed, 35 insertions(+), 5 deletions(-)
tools/virtiofsd/fuse_virtio.c +35 -5
index 6b21a93841..f763a70ba5 100644
@@ -972,13 +972,43 @@ static bool fv_socket_lock(struct fuse_session *se)
     g_autofree gchar *pidfile = NULL;
     g_autofree gchar *dir = NULL;
     Error *local_err = NULL;
+    gboolean unprivileged = false;
 
-    dir = qemu_get_local_state_pathname("run/virtiofsd");
+    if (geteuid() != 0)
+        unprivileged = true;
 
-    if (g_mkdir_with_parents(dir, S_IRWXU) < 0) {
-        fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
-                 __func__, dir, strerror(errno));
-        return false;
+    /*
+     * Unpriviliged users don't have access to /usr/local/var. Hence
+     * store lock/pid file in per user directory. Use environment
+     * variable XDG_RUNTIME_DIR.
+     * If one logs into the system as root and then does "su" then
+     * XDG_RUNTIME_DIR still points to root user directory. In that
+     * case create a directory for user in /tmp/$UID
+     */
+    if (unprivileged) {
+        gchar *user_dir = NULL;
+        gboolean create_dir = false;
+        user_dir = g_strdup(g_get_user_runtime_dir());
+        if (!user_dir || g_str_has_suffix(user_dir, "/0")) {
+            user_dir = g_strdup_printf("/tmp/%d", geteuid());
+            create_dir = true;
+        }
+
+        if (create_dir && g_mkdir_with_parents(user_dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, user_dir, strerror(errno));
+            g_free(user_dir);
+            return false;
+        }
+        dir = g_strdup(user_dir);
+        g_free(user_dir);
+    } else {
+        dir = qemu_get_local_state_pathname("run/virtiofsd");
+        if (g_mkdir_with_parents(dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, dir, strerror(errno));
+            return false;
+        }
     }
 
     sk_name = g_strdup(se->vu_socket_path);
Signature · 1 line
2.25.4
#3 lore raw

Re: [PATCH 2/5] virtiofsd: create lock/pid file in per user cache dir

Quoted 10 lines · Vivek Goyal

Right now we create lock/pid file in /usr/local/var/... and unprivliged user does not have access to create files there.

So create this file in per user cache dir as queried as specified by environment variable XDG_RUNTIME_DIR.

Note: "su $USER" does not update XDG_RUNTIME_DIR and it still points to root user's director. So for now I create a directory /tmp/$UID to save lock/pid file. Dan pointed out that it can be a problem if a malicious app already has /tmp/$UID created. So we probably need to get rid of this.

IMHO use of "su $USER" is simply user error and we don't need to care about workarounds. They will see the startup fail due to EPERM on /run/user/0 directory, and then they'll have to fix their command to use "su - $USER" to setup a clean environment.

Quoted 6 lines
tools/virtiofsd/fuse_virtio.c +6 -0
+    /*
+     * Unpriviliged users don't have access to /usr/local/var. Hence
+     * store lock/pid file in per user directory. Use environment
+     * variable XDG_RUNTIME_DIR.
+     * If one logs into the system as root and then does "su" then
+     * XDG_RUNTIME_DIR still points to root user directory. In that

tools/virtiofsd/fuse_virtio.c static bool fv_socket_lock(struct fuse_session *se) · line 995

Quoted 10 lines
tools/virtiofsd/fuse_virtio.c +10 -0
+     * case create a directory for user in /tmp/$UID
+     */
+    if (unprivileged) {
+        gchar *user_dir = NULL;
+        gboolean create_dir = false;
+        user_dir = g_strdup(g_get_user_runtime_dir());
+        if (!user_dir || g_str_has_suffix(user_dir, "/0")) {
+            user_dir = g_strdup_printf("/tmp/%d", geteuid());
+            create_dir = true;
+        }

As above, I don't think we need to have this fallback code to deal with something that is just user error.

Also, g_get_user_runtime_dir() is guaranteed to return non-NULL.

tools/virtiofsd/fuse_virtio.c static bool fv_socket_lock(struct fuse_session *se) · line 1003

Quoted 8 lines
tools/virtiofsd/fuse_virtio.c +8 -0
+
+        if (create_dir && g_mkdir_with_parents(user_dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, user_dir, strerror(errno));
+            g_free(user_dir);
+            return false;
+        }
+        dir = g_strdup(user_dir);

Don't we also want to be appending "virtiofsd" to this directory path like we do in the privileged case ?

Quoted 14 lines
tools/virtiofsd/fuse_virtio.c +8 -0
+        g_free(user_dir);
+    } else {
+        dir = qemu_get_local_state_pathname("run/virtiofsd");
+        if (g_mkdir_with_parents(dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, dir, strerror(errno));
+            return false;
+        }
     }
 
     sk_name = g_strdup(se->vu_socket_path);
Signature · 1 line
2.25.4

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#4 lore raw

Re: [PATCH 2/5] virtiofsd: create lock/pid file in per user cache dir

Quoted 12 lines · Daniel P. Berrangé
Quoted 10 lines · Vivek Goyal

Right now we create lock/pid file in /usr/local/var/... and unprivliged user does not have access to create files there.

So create this file in per user cache dir as queried as specified by environment variable XDG_RUNTIME_DIR.

Note: "su $USER" does not update XDG_RUNTIME_DIR and it still points to root user's director. So for now I create a directory /tmp/$UID to save lock/pid file. Dan pointed out that it can be a problem if a malicious app already has /tmp/$UID created. So we probably need to get rid of this.

Quoted 4 lines

IMHO use of "su $USER" is simply user error and we don't need to care about workarounds. They will see the startup fail due to EPERM on /run/user/0 directory, and then they'll have to fix their command to use "su - $USER" to setup a clean environment.

I tried "su - $USER". That clears the old XDG_RUNTIME_DIR but does not set new one. So now we have an empty XDG_RUNTIME_DIR env variable. But good thing is that now g_get_user_runtime_dir() returns "/home/$USER/.cache" and we can store user specific temp files there.

So I agree that I will get rid of all the logic to create /tmp/$USER. "su $USER" will not be a supported path.

Quoted 19 lines
Quoted 8 lines
tools/virtiofsd/fuse_virtio.c +6 -0
+    /*
+     * Unpriviliged users don't have access to /usr/local/var. Hence
+     * store lock/pid file in per user directory. Use environment
+     * variable XDG_RUNTIME_DIR.
+     * If one logs into the system as root and then does "su" then
+     * XDG_RUNTIME_DIR still points to root user directory. In that

tools/virtiofsd/fuse_virtio.c static bool fv_socket_lock(struct fuse_session *se) · line 995

Quoted 10 lines
tools/virtiofsd/fuse_virtio.c +10 -0
+     * case create a directory for user in /tmp/$UID
+     */
+    if (unprivileged) {
+        gchar *user_dir = NULL;
+        gboolean create_dir = false;
+        user_dir = g_strdup(g_get_user_runtime_dir());
+        if (!user_dir || g_str_has_suffix(user_dir, "/0")) {
+            user_dir = g_strdup_printf("/tmp/%d", geteuid());
+            create_dir = true;
+        }
Quoted 4 lines

As above, I don't think we need to have this fallback code to deal with something that is just user error.

Also, g_get_user_runtime_dir() is guaranteed to return non-NULL.

Thanks. I will get rid of (!user_dir) case.

Quoted 10 lines

tools/virtiofsd/fuse_virtio.c static bool fv_socket_lock(struct fuse_session *se) · line 1003

Quoted 9 lines
tools/virtiofsd/fuse_virtio.c +8 -0
+
+        if (create_dir && g_mkdir_with_parents(user_dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, user_dir, strerror(errno));
+            g_free(user_dir);
+            return false;
+        }
+        dir = g_strdup(user_dir);
Quoted 2 lines

Don't we also want to be appending "virtiofsd" to this directory path like we do in the privileged case ?

Yes. I forgot to append "virtiofsd" dir. Will do.

Thanks
Vivek

#5 lore raw

Re: [Virtio-fs] [PATCH 2/5] virtiofsd: create lock/pid file in per user cache dir

In reply to Vivek Goyal
Quoted 2 lines · Vivek Goyal

Right now we create lock/pid file in /usr/local/var/... and unprivliged user does not have access to create files there.

Lock files should never be created in /usr, /usr should be considered readonly.  Lock files should

always be created in /run or /tmp if /run is not writable.  As this PR shows for rootless users XDG_RUNTIME_DIR

should be used.

Quoted 66 lines

So create this file in per user cache dir as queried as specified by environment variable XDG_RUNTIME_DIR.

Note: "su $USER" does not update XDG_RUNTIME_DIR and it still points to root user's director. So for now I create a directory /tmp/$UID to save lock/pid file. Dan pointed out that it can be a problem if a malicious app already has /tmp/$UID created. So we probably need to get rid of this.

---
 tools/virtiofsd/fuse_virtio.c | 40 ++++++++++++++++++++++++++++++-----
 1 file changed, 35 insertions(+), 5 deletions(-)
tools/virtiofsd/fuse_virtio.c +35 -5
index 6b21a93841..f763a70ba5 100644
@@ -972,13 +972,43 @@ static bool fv_socket_lock(struct fuse_session *se)
     g_autofree gchar *pidfile = NULL;
     g_autofree gchar *dir = NULL;
     Error *local_err = NULL;
+    gboolean unprivileged = false;
 
-    dir = qemu_get_local_state_pathname("run/virtiofsd");
+    if (geteuid() != 0)
+        unprivileged = true;
 
-    if (g_mkdir_with_parents(dir, S_IRWXU) < 0) {
-        fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
-                 __func__, dir, strerror(errno));
-        return false;
+    /*
+     * Unpriviliged users don't have access to /usr/local/var. Hence
+     * store lock/pid file in per user directory. Use environment
+     * variable XDG_RUNTIME_DIR.
+     * If one logs into the system as root and then does "su" then
+     * XDG_RUNTIME_DIR still points to root user directory. In that
+     * case create a directory for user in /tmp/$UID
+     */
+    if (unprivileged) {
+        gchar *user_dir = NULL;
+        gboolean create_dir = false;
+        user_dir = g_strdup(g_get_user_runtime_dir());
+        if (!user_dir || g_str_has_suffix(user_dir, "/0")) {
+            user_dir = g_strdup_printf("/tmp/%d", geteuid());
+            create_dir = true;
+        }
+
+        if (create_dir && g_mkdir_with_parents(user_dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, user_dir, strerror(errno));
+            g_free(user_dir);
+            return false;
+        }
+        dir = g_strdup(user_dir);
+        g_free(user_dir);
+    } else {
+        dir = qemu_get_local_state_pathname("run/virtiofsd");
+        if (g_mkdir_with_parents(dir, S_IRWXU) < 0) {
+            fuse_log(FUSE_LOG_ERR, "%s: Failed to create directory %s: %s",
+                     __func__, dir, strerror(errno));
+            return false;
+        }
     }
 
     sk_name = g_strdup(se->vu_socket_path);
#6 lore raw

PATCH 3/5 virtiofsd: open /proc/self/fd/ in sandbox=NONE mode

In reply to Vivek Goyal

We need /proc/self/fd descriptor even in sandbox=NONE mode.

---
 tools/virtiofsd/passthrough_ll.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
tools/virtiofsd/passthrough_ll.c +12 -1
index cd91c4a831..76ef891105 100644
@@ -2969,6 +2969,15 @@ static void setup_capabilities(char *modcaps_in)
     pthread_mutex_unlock(&cap.mutex);
 }
 
+static void setup_none(struct lo_data *lo)
+{
+    lo->proc_self_fd = open("/proc/self/fd", O_PATH);
+    if (lo->proc_self_fd == -1) {
+        fuse_log(FUSE_LOG_ERR, "open(\"/proc/self/fd\", O_PATH): %m\n");
+        exit(1);
+    }
+}
+
 /*
  * Use chroot as a weaker sandbox for environments where the process is
  * launched without CAP_SYS_ADMIN.
@@ -3014,8 +3023,10 @@ static void setup_sandbox(struct lo_data *lo, struct fuse_session *se,
     if (lo->sandbox == SANDBOX_NAMESPACE) {
         setup_namespaces(lo, se);
         setup_mounts(lo->source);
-    } else {
+    } else if (lo->sandbox == SANDBOX_CHROOT) {
         setup_chroot(lo);
+    } else {
+        setup_none(lo);
     }
 
     setup_seccomp(enable_syslog);
Signature · 1 line
2.25.4
#7 lore raw

PATCH 4/5 virtiofsd: Open lo->source while setting up root in sandbox=NONE mode

In reply to Vivek Goyal

In sandbox=NONE mode, lo->source points to the directory which is being exported. We have not done any chroot()/pivot_root(). So open lo->source.

---
 tools/virtiofsd/passthrough_ll.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/virtiofsd/passthrough_ll.c +4 -1
index 76ef891105..a6fa816b6c 100644
@@ -3209,7 +3209,10 @@ static void setup_root(struct lo_data *lo, struct lo_inode *root)
     int fd, res;
     struct stat stat;
 
-    fd = open("/", O_PATH);
+    if (lo->sandbox == SANDBOX_NONE)
+        fd = open(lo->source, O_PATH);
+    else
+        fd = open("/", O_PATH);
     if (fd == -1) {
         fuse_log(FUSE_LOG_ERR, "open(%s, O_PATH): %m\n", lo->source);
         exit(1);
Signature · 1 line
2.25.4
#8 lore raw

PATCH 5/5 virtiofsd: Skip setup_capabilities() in sandbox=NONE mode

In reply to Vivek Goyal

While running as unpriviliged user setup_capabilities() fails for me. So we are doing some operation which requires priviliges. For now simply skip it in sandbox=NONE mode.

---
 tools/virtiofsd/passthrough_ll.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tools/virtiofsd/passthrough_ll.c +2 -1
index a6fa816b6c..1a0b24cbf2 100644
@@ -3030,7 +3030,8 @@ static void setup_sandbox(struct lo_data *lo, struct fuse_session *se,
     }
 
     setup_seccomp(enable_syslog);
-    setup_capabilities(g_strdup(lo->modcaps));
+    if (lo->sandbox != SANDBOX_NONE)
+       setup_capabilities(g_strdup(lo->modcaps));
 }
 
 /* Set the maximum number of open file descriptors */
Signature · 1 line
2.25.4