qemu VM entrypoints

Hi,

After a bit more discussion about integrating SELinux and KVM, it seems that there is little interest in adding enforcement hooks to KVM as it stands. Once KVM gets some type of inter-vm communication mechanism, MAC hooks will probably be added in that space.

Until then, there seems to be interest in adding MAC controls to control VM management operations, such as migrating VMs, or saving/resuming VMs.

One particular aspect of VM management which may be nice to control via SELinux is the loading of a virtual hard disk into a VM. Currently, administrators would have to rely on file permissions to control which files could be used as a virtual hard disks. The semantics of file permissions do not accomplish what is needed here. A domain needs to explicitly get permission from the policy to both use a file as a virtual disk and to use the contents of that virtual disk as an "entrypoint" to the new, virtual machine of a different integrity level.

Since there is no SELinux permission for this, I have created the vm { entrypoint } object class/permission pair to represent this type of access. Policy for allowing domain user_t to load a virtual disk of type qemu_virtdisk_t would look something like:

allow user_t qemu_virtdisk_t:file r_file_perms;
type_change user_t qemu_virtdisk_t:vm vm_user_t;
allow qemu_virtdisk_t user_vm_t:vm entrypoint;

Please note that this patch will only check the entrypoint permission, and does not actually facilitate transitioning on the type of the virtual disk. I want some comments before continuing with this approach.

When loading a virtual disk into a VM, qemu would consult the policy to see essentially three things: if the current process is allowed to read the virtual disk file, what the type of the VM should be after loading the disk, and if the virtual disk is in fact allowed to serve as an entrypoint to the target domain.

One problem with this approach is that loading a VM is not an exec-based operation. Dynamic transitions could be used, but could possibly be avoided by altering the patch to fork, then re-exec in the target domain.

This patch applies cleanly to kvm-userspace trunk.

Thoughts?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Quoted 7 lines · David Windsor

Hi,

After a bit more discussion about integrating SELinux and KVM, it seems that there is little interest in adding enforcement hooks to KVM as it stands. Once KVM gets some type of inter-vm communication mechanism, MAC hooks will probably be added in that space.

First, a patch like this has to go to qemu-devel.

Quoted 3 lines

Until then, there seems to be interest in adding MAC controls to control VM management operations, such as migrating VMs, or saving/resuming VMs.

Why is this interesting? Is it common to modify applications like this to have additional selinux hooks? Can you give examples?

What makes QEMU/KVM special such that it requires userspace selinux hooks?

Regards,

Anthony Liguori

Quoted 37 lines

One particular aspect of VM management which may be nice to control via SELinux is the loading of a virtual hard disk into a VM. Currently, administrators would have to rely on file permissions to control which files could be used as a virtual hard disks. The semantics of file permissions do not accomplish what is needed here. A domain needs to explicitly get permission from the policy to both use a file as a virtual disk and to use the contents of that virtual disk as an "entrypoint" to the new, virtual machine of a different integrity level.

Since there is no SELinux permission for this, I have created the vm { entrypoint } object class/permission pair to represent this type of access. Policy for allowing domain user_t to load a virtual disk of type qemu_virtdisk_t would look something like:

allow user_t qemu_virtdisk_t:file r_file_perms;
type_change user_t qemu_virtdisk_t:vm vm_user_t;
allow qemu_virtdisk_t user_vm_t:vm entrypoint;

Please note that this patch will only check the entrypoint permission, and does not actually facilitate transitioning on the type of the virtual disk. I want some comments before continuing with this approach.

When loading a virtual disk into a VM, qemu would consult the policy to see essentially three things: if the current process is allowed to read the virtual disk file, what the type of the VM should be after loading the disk, and if the virtual disk is in fact allowed to serve as an entrypoint to the target domain.

One problem with this approach is that loading a VM is not an exec-based operation. Dynamic transitions could be used, but could possibly be avoided by altering the patch to fork, then re-exec in the target domain.

This patch applies cleanly to kvm-userspace trunk.

Thoughts?

Quoted 10 lines

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Quoted 5 lines · Anthony Liguori

Quoted 3 lines · David Windsor

Hi,

After a bit more discussion about integrating SELinux and KVM, it

seems that

Quoted 9 lines

Quoted 1 line

there is little interest in adding enforcement hooks to KVM as it

stands.

Quoted 1 line

Once KVM gets some type of inter-vm communication mechanism, MAC

hooks will

Quoted 2 lines

probably be added in that space.

First, a patch like this has to go to qemu-devel.

Understood. This patch started out as a patch to KVM, but eventually found its way into userspace/qemu.

Quoted 8 lines

Quoted 1 line

Until then, there seems to be interest in adding MAC controls to

control VM

Quoted 2 lines

management operations, such as migrating VMs, or saving/resuming VMs.

Why is this interesting? Is it common to modify applications like this to have additional selinux hooks? Can you give examples?

This is interesting because currently there are only DAC controls on the loading of virtual hard disks into a qemu virtual machine. Further, this patch proposes additional SELinux functionality for transitioning the qemu process into the correct TE domain prior to launching the VM.

The use case this patch addresses is that of an administrator wanting to use KVM/qemu on a server to provide services for subnets of computers. It is important here to ensure that if the server is compromised, a qemu process on this compromised server can only use virtual disks permitted by policy, otherwise entire subnets of computers could be exposed to rogue VMs.

If the qemu process attempting to load the virutal disk does not possess the file:read permission on the virtual disk file, access will be denied right there. However, if the process can read the disk file, this patch proposes an additional control to see if the virtual disk can be used as an entrypoint to the new, target domain. If the vm:entrypoint permission is granted by policy, the qemu process would transition to the new domain.

As it stands, this patch does not perform any transitions upon loading a virtual disk. I think that a transition based on the label of the requesting process and the label of the virtual disk is appropriate here. Does anyone else have any thoughts about this?

Quoted 2 lines

What makes QEMU/KVM special such that it requires userspace selinux hooks?

qemu itself is special in that it is a userspace backend that is widely used with KVM, and the SELinux functionality proposed by this patch is significant. Hopefully this answers your question adequately.

Quoted 79 lines

Regards,

Anthony Liguori

Quoted 1 line

One particular aspect of VM management which may be nice to control

via

Quoted 2 lines

SELinux is the loading of a virtual hard disk into a VM. Currently, administrators would have to rely on file permissions to control

which files

Quoted 1 line

could be used as a virtual hard disks. The semantics of file

permissions do

Quoted 2 lines

not accomplish what is needed here. A domain needs to explicitly get permission from the policy to both use a file as a virtual disk and

to use

Quoted 1 line

the contents of that virtual disk as an "entrypoint" to the new,

virtual

Quoted 4 lines

machine of a different integrity level.

Since there is no SELinux permission for this, I have created the vm { entrypoint } object class/permission pair to represent this type of

access.

Quoted 8 lines

Policy for allowing domain user_t to load a virtual disk of type qemu_virtdisk_t would look something like:

allow user_t qemu_virtdisk_t:file r_file_perms;
type_change user_t qemu_virtdisk_t:vm vm_user_t;
allow qemu_virtdisk_t user_vm_t:vm entrypoint;

Please note that this patch will only check the entrypoint

permission, and

Quoted 1 line

does not actually facilitate transitioning on the type of the

virtual disk.

Quoted 3 lines

I want some comments before continuing with this approach.

When loading a virtual disk into a VM, qemu would consult the

policy to see

Quoted 1 line

essentially three things: if the current process is allowed to read

the

Quoted 1 line

virtual disk file, what the type of the VM should be after loading

the disk,

Quoted 1 line

and if the virtual disk is in fact allowed to serve as an

entrypoint to the

Quoted 3 lines

target domain.

One problem with this approach is that loading a VM is not an

exec-based

Quoted 1 line

operation. Dynamic transitions could be used, but could possibly

be avoided

Quoted 8 lines

by altering the patch to fork, then re-exec in the target domain.

This patch applies cleanly to kvm-userspace trunk.

Thoughts?

-------------------------------------------------------------------------

Quoted 9 lines

This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

-------------------------------------------------------------------------

This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Hi,

After a bit more discussion about integrating SELinux and KVM, it seems that there is little interest in adding enforcement hooks to KVM as it stands. Once KVM gets some type of inter-vm communication mechanism, MAC hooks will probably be added in that space.

Until then, there seems to be interest in adding MAC controls to control VM management operations, such as migrating VMs, or saving/resuming VMs.

One particular aspect of VM management which may be nice to control via SELinux is the loading of a virtual hard disk into a VM. Currently, administrators would have to rely on file permissions to control which files could be used as a virtual hard disks. The semantics of file permissions do not accomplish what is needed here. A domain needs to explicitly get permission from the policy to both use a file as a virtual disk and to use the contents of that virtual disk as an "entrypoint" to the new, virtual machine of a different integrity level.

Since there is no SELinux permission for this, I have created the vm { entrypoint } object class/permission pair to represent this type of access. Policy for allowing domain user_t to load a virtual disk of type qemu_virtdisk_t would look something like:

allow user_t qemu_virtdisk_t:file r_file_perms;
type_change user_t qemu_virtdisk_t:vm vm_user_t;
allow qemu_virtdisk_t user_vm_t:vm entrypoint;

Please note that this patch will only check the entrypoint permission, and does not actually facilitate transitioning on the type of the virtual disk. I want some comments before continuing with this approach.

When loading a virtual disk into a VM, qemu would consult the policy to see essentially three things: if the current process is allowed to read the virtual disk file, what the type of the VM should be after loading the disk, and if the virtual disk is in fact allowed to serve as an entrypoint to the target domain.

One problem with this approach is that loading a VM is not an exec-based operation. Dynamic transitions could be used, but could possibly be avoided by altering the patch to fork, then re-exec in the target domain.

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 2 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Then you can bypass MAC security by invoking qemu directly.

Quoted 3 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Then you can bypass MAC security by invoking qemu directly.

- James

Quoted 7 lines · James Morris

Quoted 3 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 2 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Quoted 10 lines · Anthony Liguori

Quoted 8 lines · James Morris

Quoted 4 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 6 lines

Quoted 3 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Yep, I was thinking only about qemu.

I guess you'd have OS policy preventing normal domains from accessing /dev/kvm (or /dev/lguest etc.), while a security-aware launcher would enforce access control policy over which domains could launch which disk images as VMs, and also setup the execution context & fork.

So, perhaps this would be better done at the libvirt layer (i.e. make libvirt the object manager).

- James

Quoted 5 lines

I guess you'd have OS policy preventing normal domains from accessing /dev/kvm (or /dev/lguest etc.), while a security-aware launcher would enforce access control policy over which domains could launch which disk images as VMs, and also setup the execution context & fork.

I really think you have to start with the assumption that a guest can access anything that QEMU can access and attempt to build security around that. If you want to restrict what the guest can see, restrict what QEMU can see.

At some point, we may do crazy stuff like syscall pass-through in which case, it would be all but impossible to have a "security-aware" launcher.

Quoted 8 lines · Anthony Liguori

Quoted 7 lines · James Morris

Quoted 3 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 5 lines

Quoted 2 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Yep, I was thinking only about qemu.

I guess you'd have OS policy preventing normal domains from accessing /dev/kvm (or /dev/lguest etc.), while a security-aware launcher would enforce access control policy over which domains could launch which disk images as VMs, and also setup the execution context & fork.

So, perhaps this would be better done at the libvirt layer (i.e. make libvirt the object manager).

- James

Quoted 8 lines · Anthony Liguori

Quoted 7 lines · James Morris

Quoted 3 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 5 lines

Quoted 2 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Yep, I was thinking only about qemu.

I guess you'd have OS policy preventing normal domains from accessing /dev/kvm (or /dev/lguest etc.), while a security-aware launcher would enforce access control policy over which domains could launch which disk images as VMs, and also setup the execution context & fork.

So, perhaps this would be better done at the libvirt layer (i.e. make libvirt the object manager).

- James

Quoted 8 lines · James Morris

Quoted 4 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 3 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Quoted 10 lines · Anthony Liguori

Quoted 8 lines · James Morris

Quoted 4 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 8 lines

Quoted 3 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

I guess modifying qemu makes sense if the modification gives you *more* permissions.

i.e. you start out just with the ability to access the disk image, and then you transition to a new domain that allows you to access some network.

Is that what is intended here?

Quoted 24 lines · Avi Kivity

Quoted 13 lines · Anthony Liguori

Quoted 10 lines · James Morris

Quoted 5 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 9 lines

Quoted 4 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Quoted 9 lines

I guess modifying qemu makes sense if the modification gives you *more* permissions.

i.e. you start out just with the ability to access the disk image, and then you transition to a new domain that allows you to access some network.

Is that what is intended here?

The intent here is to provide some mechanism for a policy-driven transition to occur when loading a virtual disk. So, an administrator would write policy for say, a topsecret_vm_t domain, in which the top secret VM can run. All access requests from this VM would have a source type of topsecret_vm_t on the host machine. The virtual disk, labeled perhaps as topsecret_disk_t, would need to be authorized in policy to be an entrypoint into the topsecret_vm_t domain. This patch proposes a mechanism that allows us to provide policy driven controls both over which files can be used as a disk image, and to which domain a qemu process will transition after loading the disk image.

Originally, I thought qemu was the proper place for this type of an enforcement hook, but libvirt may be more appropriate because qemu could be launched with a different security context if loading a virtual disk, rather than having qemu set up the security context of the VM. Thoughts?

Can you already write an selinux policy that changes the label of a process when it open()s a different file?

Quoted 2 lines · Anthony Liguori

Can you already write an selinux policy that changes the label of a process when it open()s a different file?

No, and you normally want to do this over an exec anyway, to ensure the new execution state is clean.

Quoted 10 lines · James Morris

Quoted 3 lines · Anthony Liguori

Can you already write an selinux policy that changes the label of a process when it open()s a different file?

No, and you normally want to do this over an exec anyway, to ensure the new execution state is clean.

This is correct and the object model being proposed isn't just about opening files.

Anthony - There are several userspace object managers for SELinux, basically they manage resources that are too abstract for the kernel to manage. These include DBUS connections, X windows/pixmaps/cursors/etc, postgresql tables and rows, etc. SELinux is designed to designed to have non-centralized enforcement of the policy (but centralized decision making). In this case the object model isn't just about opening a file, its about lauching and restricting a vm based on an abstract resource, in this case a virtual disk file, that the kernel can't differenciate from any other file.

The reason for this abstraction is simple, think about all the files that qemu has to open that aren't disk files. For example, if you have 3 vm's, unclass, secret, top secret, those qemu processes still have to share resources, library files, configuration files, devices, potentially log, pid etc. If someone could manage to get a top secret disk labeled lib_t they could boot any of those vm's with that disk image (using snapshot or some feature to prevent write attempts on it), and disclose top secret data where it shouldn't be disclosed.

With David's object model this kind of attack can be prevented because virtual disk files aren't 'just files' to qemu, they are an abstract resource that means more than just a regular file, just like a database file is more than just a regular file for SE-Postgresql.

Quoted 26 lines · Joshua Brindle

Quoted 10 lines · James Morris

Quoted 3 lines · Anthony Liguori

Can you already write an selinux policy that changes the label of a process when it open()s a different file?

No, and you normally want to do this over an exec anyway, to ensure the new execution state is clean.

This is correct and the object model being proposed isn't just about opening files.

Anthony - There are several userspace object managers for SELinux, basically they manage resources that are too abstract for the kernel to manage. These include DBUS connections, X windows/pixmaps/cursors/etc, postgresql tables and rows, etc. SELinux is designed to designed to have non-centralized enforcement of the policy (but centralized decision making). In this case the object model isn't just about opening a file, its about lauching and restricting a vm based on an abstract resource, in this case a virtual disk file, that the kernel can't differenciate from any other file.

Quoted 9 lines

The reason for this abstraction is simple, think about all the files that qemu has to open that aren't disk files. For example, if you have 3 vm's, unclass, secret, top secret, those qemu processes still have to share resources, library files, configuration files, devices, potentially log, pid etc. If someone could manage to get a top secret disk labeled lib_t they could boot any of those vm's with that disk image (using snapshot or some feature to prevent write attempts on it), and disclose top secret data where it shouldn't be disclosed.

The only way that the VM could possible see the top secret data is if the disk file was readable by the QEMU process running the secret guest. That's the security problem. You address this "attack" by simply not letting the user that launched the QEMU process have read
access to the top secret disk file. Why would you ever want to allow
the QEMU process that's running a "secret" domain to have any read access to top secret files?

Regards,

Anthony Liguori

Quoted 6 lines

With David's object model this kind of attack can be prevented because virtual disk files aren't 'just files' to qemu, they are an abstract resource that means more than just a regular file, just like a database file is more than just a regular file for SE-Postgresql.

Quoted 26 lines · Joshua Brindle

Quoted 10 lines · James Morris

Quoted 3 lines · Anthony Liguori

Can you already write an selinux policy that changes the label of a process when it open()s a different file?

No, and you normally want to do this over an exec anyway, to ensure the new execution state is clean.

This is correct and the object model being proposed isn't just about opening files.

Anthony - There are several userspace object managers for SELinux, basically they manage resources that are too abstract for the kernel to manage. These include DBUS connections, X windows/pixmaps/cursors/etc, postgresql tables and rows, etc. SELinux is designed to designed to have non-centralized enforcement of the policy (but centralized decision making). In this case the object model isn't just about opening a file, its about lauching and restricting a vm based on an abstract resource, in this case a virtual disk file, that the kernel can't differenciate from any other file.

Quoted 9 lines

The reason for this abstraction is simple, think about all the files that qemu has to open that aren't disk files. For example, if you have 3 vm's, unclass, secret, top secret, those qemu processes still have to share resources, library files, configuration files, devices, potentially log, pid etc. If someone could manage to get a top secret disk labeled lib_t they could boot any of those vm's with that disk image (using snapshot or some feature to prevent write attempts on it), and disclose top secret data where it shouldn't be disclosed.

The only way that the VM could possible see the top secret data is if the disk file was readable by the QEMU process running the secret guest. That's the security problem. You address this "attack" by simply not letting the user that launched the QEMU process have read
access to the top secret disk file. Why would you ever want to allow
the QEMU process that's running a "secret" domain to have any read access to top secret files?

Regards,

Anthony Liguori

Quoted 6 lines

With David's object model this kind of attack can be prevented because virtual disk files aren't 'just files' to qemu, they are an abstract resource that means more than just a regular file, just like a database file is more than just a regular file for SE-Postgresql.

Quoted 37 lines · David Windsor

Quoted 24 lines · Avi Kivity

Quoted 13 lines · Anthony Liguori

Quoted 10 lines · James Morris

Quoted 5 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Quoted 9 lines

Quoted 4 lines

Then you can bypass MAC security by invoking qemu directly.

You can bypass MAC security by writing your own binary that uses the KVM kernel interfaces.

Quoted 9 lines

I guess modifying qemu makes sense if the modification gives you *more* permissions.

i.e. you start out just with the ability to access the disk image, and then you transition to a new domain that allows you to access some network.

Is that what is intended here?

Quoted 11 lines

The intent here is to provide some mechanism for a policy-driven transition to occur when loading a virtual disk. So, an administrator would write policy for say, a topsecret_vm_t domain, in which the top secret VM can run. All access requests from this VM would have a source type of topsecret_vm_t on the host machine. The virtual disk, labeled perhaps as topsecret_disk_t, would need to be authorized in policy to be an entrypoint into the topsecret_vm_t domain. This patch proposes a mechanism that allows us to provide policy driven controls both over which files can be used as a disk image, and to which domain a qemu process will transition after loading the disk image.

Can you already write an selinux policy that changes the label of a process when it open()s a different file?

Regards,

Anthony Liguori

Quoted 8 lines

Originally, I thought qemu was the proper place for this type of an enforcement hook, but libvirt may be more appropriate because qemu could be launched with a different security context if loading a virtual disk, rather than having qemu set up the security context of the VM. Thoughts?

Quoted 2 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

Then you can bypass MAC security by invoking qemu directly.

- James
--
James Morris
<jmorris-gx6/[email protected]>

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Quoted 5 lines · James Morris

Quoted 2 lines · Daniel P. Berrange

It could be - if your put the policy at the control API layer instead of in QEMU itself.

I think that libvirt may be a bit too high in the virtualization stack for this control.
What benefits are there for placing such a hook in libvirt vs qemu? libvirt could still use the vm:entrypoint permission for other types of VMs it manages.

obviously the UNIX user has a corresponding SELinux domain. In the remote case, one could map x509 certificate IDs (the remote user's identify) to appropriate local SELinux domains.

One problem with this approach is that loading a VM is not an exec-based operation. Dynamic transitions could be used, but could possibly be avoided by altering the patch to fork, then re-exec in the target domain.

This patch applies cleanly to kvm-userspace trunk.

Thoughts?