Skip to contentSkip to conversation

docs: describe the security considerations with virtiofsd xattr mapping

PATCH In review Daniel P. Berrangé 4 participants 9 messages Jun 11, 2021 – Jun 30, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 docs: describe the security considerations with virtiofsd xattr mapping

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +50 -5
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.
+This is shown in example (1) below.
+
+If selectively mapping only a subset of xattr prefixes,
+then rules must be added to explicitly block direct
+access to the target of the remapping. This is shown
+in example (2) below.
+
+Mapping examples
+~~~~~~~~~~~~~~~~
 
 1) Prefix all attributes with 'user.virtiofs.'
 
@@ -270,7 +313,9 @@ stripping of 'user.virtiofs.'.
 The second rule hides unprefixed 'trusted.' attributes
 on the host.
 The third rule stops a guest from explicitly setting
-the 'user.virtiofs.' path directly.
+the 'user.virtiofs.' path directly to prevent access
+control bypass on the target of the earlier prefix
+remapping.
 Finally, the fourth rule lets all remaining attributes
 through.
 
Signature · 1 line
2.31.1
#1 lore raw
Quoted 75 lines · Daniel P. Berrangé

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +28 -4
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs

docs/tools/virtiofsd.rst Note: When the 'security.capability' xattr is remapped, the daemon has to do · line 270

Quoted 10 lines
docs/tools/virtiofsd.rst +10 -0
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.

Remapping all xattrs seem to make sense. It probably will lead to less confusion. Nested guests add another level of redirection.

BTW, remapping xattr has limitation that it does not work on symlinks. So "user.*" can't be set on symlink. And that means selinux relabeling of symlinks fails with remapped xattrs.

Not sure how to address this limitation. Host kernel imposes this limit. (man xattr).

docs/tools/virtiofsd.rst Note: When the 'security.capability' xattr is remapped, the daemon has to do · line 276

Quoted 6 lines
docs/tools/virtiofsd.rst +6 -0
+This is shown in example (1) below.
+
+If selectively mapping only a subset of xattr prefixes,
+then rules must be added to explicitly block direct
+access to the target of the remapping. This is shown
+in example (2) below.

Example (2) seems to block all the xattrs with prefix even if only one xattr has been remapped.

So if we remapped "trusted." to "user.virtiofs.trusted.", then client can't set any xattr starting with "user.virtiofs". I am wondering should it be limted to only blocking only "user.virtiofs.trusted.".

Thanks
Vivek

Quoted 24 lines
docs/tools/virtiofsd.rst +6 -1
+
+Mapping examples
+~~~~~~~~~~~~~~~~
 
 1) Prefix all attributes with 'user.virtiofs.'
 
@@ -270,7 +313,9 @@ stripping of 'user.virtiofs.'.
 The second rule hides unprefixed 'trusted.' attributes
 on the host.
 The third rule stops a guest from explicitly setting
-the 'user.virtiofs.' path directly.
+the 'user.virtiofs.' path directly to prevent access
+control bypass on the target of the earlier prefix
+remapping.
 Finally, the fourth rule lets all remaining attributes
 through.
 
Signature · 6 lines
2.31.1

_______________________________________________
Virtio-fs mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/virtio-fs
#2 lore raw
Quoted 90 lines · Vivek Goyal
Quoted 75 lines · Daniel P. Berrangé

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +28 -4
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs

docs/tools/virtiofsd.rst Note: When the 'security.capability' xattr is remapped, the daemon has to do · line 270

Quoted 10 lines
docs/tools/virtiofsd.rst +10 -0
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.

Remapping all xattrs seem to make sense. It probably will lead to less confusion. Nested guests add another level of redirection.

Quoted 6 lines

BTW, remapping xattr has limitation that it does not work on symlinks. So "user.*" can't be set on symlink. And that means selinux relabeling of symlinks fails with remapped xattrs.

Not sure how to address this limitation. Host kernel imposes this limit. (man xattr).

Oh fun, I had not noticed this limitation before :-(

Here are some ideas I had, none especially nice

- Use 'trusted.*' namespace for remapping instead of 'user.'

virtiofsd needs to have CAP_SYS_ADMIN though which is quite horrible if your goal is to confine its privileges in any meaningful way

- Store a symlinks' xattr on the target of the symlink

if the symlink has dev:inode 54:224, and points to a file 'foo', then on 'foo' create an xattr "user.virtiofs.link:54:224.<original xattrpath>"

This gets quite horrendous when you have symlinks pointing to symlinks pointing to symlinks. Does not work too well if the 'st_dev' value is not stable across reboots either.

- Don't use xattrs at all for remapping, instead use hidden files.

eg for a file 'foo', if an xattr is set then create a file '.foo.xattr' in the same directory and store the xattrs in that file in some format. Need to hide this lookaside files from the guest of course.

Quoted 10 lines

docs/tools/virtiofsd.rst Note: When the 'security.capability' xattr is remapped, the daemon has to do · line 276

Quoted 6 lines
docs/tools/virtiofsd.rst +6 -0
+This is shown in example (1) below.
+
+If selectively mapping only a subset of xattr prefixes,
+then rules must be added to explicitly block direct
+access to the target of the remapping. This is shown
+in example (2) below.

Example (2) seems to block all the xattrs with prefix even if only one xattr has been remapped.

Quoted 4 lines

So if we remapped "trusted." to "user.virtiofs.trusted.", then client can't set any xattr starting with "user.virtiofs". I am wondering should it be limted to only blocking only "user.virtiofs.trusted.".

Or maybe illustrate with two mappings, so we can show how blocking the parent xattr covers both

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#3 lore raw

On 6/15/21 10:46 AM, Daniel P. Berrangé wrote, in part:

Quoted 2 lines

- Don't use xattrs at all for remapping, instead use hidden files.

As not every host filesystem will have xattrs, or have xattrs enabled:
Isn't something like the above required of virtiofsd anyway?

Also, for security, you might consider creating a parallel hidden directory tree 'beside' the shared root (so not exposed to the guest) then put the xattrs in files there.  So if /var/shared/foo is visible to the guest, /var/shared/.foo.guest_xattrs/ parallels that.

#4 lore raw
In reply to Daniel P. Berrangé
Quoted 119 lines · Daniel P. Berrangé
Quoted 90 lines · Vivek Goyal
Quoted 75 lines · Daniel P. Berrangé

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +28 -4
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs

docs/tools/virtiofsd.rst Note: When the 'security.capability' xattr is remapped, the daemon has to do · line 270

Quoted 10 lines
docs/tools/virtiofsd.rst +10 -0
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.

Remapping all xattrs seem to make sense. It probably will lead to less confusion. Nested guests add another level of redirection.

Quoted 6 lines

BTW, remapping xattr has limitation that it does not work on symlinks. So "user.*" can't be set on symlink. And that means selinux relabeling of symlinks fails with remapped xattrs.

Not sure how to address this limitation. Host kernel imposes this limit. (man xattr).

Oh fun, I had not noticed this limitation before :-(

Here are some ideas I had, none especially nice

- Use 'trusted.*' namespace for remapping instead of 'user.'

virtiofsd needs to have CAP_SYS_ADMIN though which is quite horrible if your goal is to confine its privileges in any meaningful way

- Store a symlinks' xattr on the target of the symlink

if the symlink has dev:inode 54:224, and points to a file 'foo', then on 'foo' create an xattr "user.virtiofs.link:54:224.<original xattrpath>"

This gets quite horrendous when you have symlinks pointing to symlinks pointing to symlinks. Does not work too well if the 'st_dev' value is not stable across reboots either.

Quoted 7 lines

- Don't use xattrs at all for remapping, instead use hidden files.

eg for a file 'foo', if an xattr is set then create a file '.foo.xattr' in the same directory and store the xattrs in that file in some format. Need to hide this lookaside files from the guest of course.

I kind of like this idea of creating a regular file, say .user.virtiofs.foo.xattr. So any file prefixed by ".user.virtiofs" will be hidden from guest.

And probably same can be done for selinux labels for special files (device nodes, pipes, sockets etc).

Thanks
Vivek

#6 lore raw
In reply to Daniel P. Berrangé
Quoted 5 lines · Daniel P. Berrangé

- Store a symlinks' xattr on the target of the symlink

if the symlink has dev:inode 54:224, and points to a file 'foo', then on 'foo' create an xattr "user.virtiofs.link:54:224.<original xattrpath>"

Atomicity issues here.  If you do this, virtiofsd will need a 'xattr write intent journal' somewhere....

#5 lore raw
In reply to Vivek Goyal
Quoted 128 lines · * Vivek Goyal ([email protected]) wrote:
Quoted 119 lines · Daniel P. Berrangé
Quoted 90 lines · Vivek Goyal
Quoted 75 lines · Daniel P. Berrangé

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +28 -4
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs

docs/tools/virtiofsd.rst Note: When the 'security.capability' xattr is remapped, the daemon has to do · line 270

Quoted 10 lines
docs/tools/virtiofsd.rst +10 -0
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.

Remapping all xattrs seem to make sense. It probably will lead to less confusion. Nested guests add another level of redirection.

Quoted 6 lines

BTW, remapping xattr has limitation that it does not work on symlinks. So "user.*" can't be set on symlink. And that means selinux relabeling of symlinks fails with remapped xattrs.

Not sure how to address this limitation. Host kernel imposes this limit. (man xattr).

Oh fun, I had not noticed this limitation before :-(

Here are some ideas I had, none especially nice

- Use 'trusted.*' namespace for remapping instead of 'user.'

virtiofsd needs to have CAP_SYS_ADMIN though which is quite horrible if your goal is to confine its privileges in any meaningful way

- Store a symlinks' xattr on the target of the symlink

if the symlink has dev:inode 54:224, and points to a file 'foo', then on 'foo' create an xattr "user.virtiofs.link:54:224.<original xattrpath>"

This gets quite horrendous when you have symlinks pointing to symlinks pointing to symlinks. Does not work too well if the 'st_dev' value is not stable across reboots either.

Quoted 7 lines

- Don't use xattrs at all for remapping, instead use hidden files.

eg for a file 'foo', if an xattr is set then create a file '.foo.xattr' in the same directory and store the xattrs in that file in some format. Need to hide this lookaside files from the guest of course.

Quoted 6 lines

I kind of like this idea of creating a regular file, say .user.virtiofs.foo.xattr. So any file prefixed by ".user.virtiofs" will be hidden from guest.

And probably same can be done for selinux labels for special files (device nodes, pipes, sockets etc).

Except that splitting the attrs from the file makes it a nightmare to deal with renames and anything else that might be expected to happen atomically.

Dave

Quoted 2 lines

Thanks
Vivek

Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Reviewed-by on v1 1/1
#7 lore raw
In reply to Daniel P. Berrangé
Quoted 11 lines · * Daniel P. Berrangé ([email protected]) wrote:

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

Quoted 6 lines

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

Vivek's point about symlinks is something we should add but that's kind of separate to the clarification you've explained here.

Dave

Quoted 94 lines
---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +50 -5
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.
+This is shown in example (1) below.
+
+If selectively mapping only a subset of xattr prefixes,
+then rules must be added to explicitly block direct
+access to the target of the remapping. This is shown
+in example (2) below.
+
+Mapping examples
+~~~~~~~~~~~~~~~~
 
 1) Prefix all attributes with 'user.virtiofs.'
 
@@ -270,7 +313,9 @@ stripping of 'user.virtiofs.'.
 The second rule hides unprefixed 'trusted.' attributes
 on the host.
 The third rule stops a guest from explicitly setting
-the 'user.virtiofs.' path directly.
+the 'user.virtiofs.' path directly to prevent access
+control bypass on the target of the earlier prefix
+remapping.
 Finally, the fourth rule lets all remaining attributes
 through.
 
Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#8 lore raw
In reply to Daniel P. Berrangé
Quoted 11 lines · * Daniel P. Berrangé ([email protected]) wrote:

Different guest xattr prefixes have distinct access control rules applied by the guest. When remapping a guest xattr care must be taken that the remapping does not allow the a guest user to bypass guest kernel access control rules.

For example if 'trusted.*' which requires CAP_SYS_ADMIN is remapped to 'user.virtiofs.trusted.*', an unprivileged guest user which can write to 'user.*' can bypass the CAP_SYS_ADMIN control. Thus the target of any remapping must be explicitly blocked from read/writes by the guest, to prevent access control bypass.

Quoted 6 lines

The examples shown in the virtiofsd man page already do the right thing and ensure safety, but the security implications of getting this wrong were not made explicit. This could lead to host admins and apps unwittingly creating insecure configurations.

Queued

Quoted 95 lines
---
 docs/tools/virtiofsd.rst | 55 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 50 insertions(+), 5 deletions(-)
docs/tools/virtiofsd.rst +50 -5
index 00554c75bd..6370ab927c 100644
@@ -127,8 +127,8 @@ Options
   timeout.  ``always`` sets a long cache lifetime at the expense of coherency.
   The default is ``auto``.
 
-xattr-mapping
--------------
+Extended attribute (xattr) mapping
+----------------------------------
 
 By default the name of xattr's used by the client are passed through to the server
 file system.  This can be a problem where either those xattr names are used
@@ -136,6 +136,9 @@ by something on the server (e.g. selinux client/server confusion) or if the
 virtiofsd is running in a container with restricted privileges where it cannot
 access some attributes.
 
+Mapping syntax
+~~~~~~~~~~~~~~
+
 A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping``
 string consists of a series of rules.
 
@@ -232,8 +235,48 @@ Note: When the 'security.capability' xattr is remapped, the daemon has to do
 extra work to remove it during many operations, which the host kernel normally
 does itself.
 
-xattr-mapping Examples
-----------------------
+Security considerations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Operating systems typically partition the xattr namespace using
+well defined name prefixes. Each partition may have different
+access controls applied. For example, on Linux there are multiple
+partitions
+
+ * ``system.*`` - access varies depending on attribute & filesystem
+ * ``security.*`` - only processes with CAP_SYS_ADMIN
+ * ``trusted.*`` - only processes with CAP_SYS_ADMIN
+ * ``user.*`` - any process granted by file permissions / ownership
+
+While other OS such as FreeBSD have different name prefixes
+and access control rules.
+
+When remapping attributes on the host, it is important to
+ensure that the remapping does not allow a guest user to
+evade the guest access control rules.
+
+Consider if ``trusted.*`` from the guest was remapped to
+``user.virtiofs.trusted*`` in the host. An unprivileged
+user in a Linux guest has the ability to write to xattrs
+under ``user.*``. Thus the user can evade the access
+control restriction on ``trusted.*`` by instead writing
+to ``user.virtiofs.trusted.*``.
+
+As noted above, the partitions used and access controls
+applied, will vary across guest OS, so it is not wise to
+try to predict what the guest OS will use.
+
+The simplest way to avoid an insecure configuration is
+to remap all xattrs at once, to a given fixed prefix.
+This is shown in example (1) below.
+
+If selectively mapping only a subset of xattr prefixes,
+then rules must be added to explicitly block direct
+access to the target of the remapping. This is shown
+in example (2) below.
+
+Mapping examples
+~~~~~~~~~~~~~~~~
 
 1) Prefix all attributes with 'user.virtiofs.'
 
@@ -270,7 +313,9 @@ stripping of 'user.virtiofs.'.
 The second rule hides unprefixed 'trusted.' attributes
 on the host.
 The third rule stops a guest from explicitly setting
-the 'user.virtiofs.' path directly.
+the 'user.virtiofs.' path directly to prevent access
+control bypass on the target of the earlier prefix
+remapping.
 Finally, the fourth rule lets all remaining attributes
 through.
 
Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK