VIRTIO adoption in other hypervisors
Hi,
I'm currently trying to get my head around virtio and was wondering how widespread adoption of virtio is amongst the various hypervisors and emulators out there.
Obviously I'm familiar with QEMU both via KVM and even when just doing plain emulation (although with some restrictions). As far as I'm aware the various Rust based VMMs have vary degrees of support for virtio devices over KVM as well. CrosVM specifically is embracing virtio for multi-process device emulation.
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
So some questions:
- Am I missing anything out in that summary?
- How about HyperV and the OSX equivalent?
- Do any other type-1 hypervisors support virtio?
Signature · 1 line
Alex Bennée
Quoted 22 lines · Alex Bennée
Hi,
I'm currently trying to get my head around virtio and was wondering how widespread adoption of virtio is amongst the various hypervisors and emulators out there.
Obviously I'm familiar with QEMU both via KVM and even when just doing plain emulation (although with some restrictions). As far as I'm aware the various Rust based VMMs have vary degrees of support for virtio devices over KVM as well. CrosVM specifically is embracing virtio for multi-process device emulation.
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
Quoted 8 lines
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
So some questions:
- Am I missing anything out in that summary?
- How about HyperV and the OSX equivalent?
- Do any other type-1 hypervisors support virtio?
From the top of my head, some other hypervisors with virtio support (irrespective of any classification):
https://wiki.freebsd.org/bhyve
https://projectacrn.org/
http://www.xhypervisor.org/
https://www.opensynergy.com/automotive-hypervisor/
But there are likely more.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Quoted 22 lines · Jan Kiszka
Quoted 21 lines · Alex Bennée
Hi,
I'm currently trying to get my head around virtio and was wondering how widespread adoption of virtio is amongst the various hypervisors and emulators out there.
Obviously I'm familiar with QEMU both via KVM and even when just doing plain emulation (although with some restrictions). As far as I'm aware the various Rust based VMMs have vary degrees of support for virtio devices over KVM as well. CrosVM specifically is embracing virtio for multi-process device emulation.
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
Quoted 5 lines
Quoted 5 lines
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
And to clarify the scope of this effort: virtio-over-ivshmem is not the fastest option to offer virtio to a guest (static "DMA" window), but it is the simplest one from the hypervisor PoV and, thus, also likely the easiest one to argue over when it comes to security and safety.
Jan
Quoted 19 lines
Quoted 6 lines
So some questions:
- Am I missing anything out in that summary?
- How about HyperV and the OSX equivalent?
- Do any other type-1 hypervisors support virtio?From the top of my head, some other hypervisors with virtio support
(irrespective of any classification):https://wiki.freebsd.org/bhyve
https://projectacrn.org/
http://www.xhypervisor.org/
https://www.opensynergy.com/automotive-hypervisor/But there are likely more.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Quoted 4 lines · Jan Kiszka
Quoted 3 lines · Jan Kiszka
Quoted 2 lines · Alex Bennée
Hi,
<snip>
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
And to clarify the scope of this effort: virtio-over-ivshmem is not the fastest option to offer virtio to a guest (static "DMA" window), but it is the simplest one from the hypervisor PoV and, thus, also likely the easiest one to argue over when it comes to security and safety.
So to drill down on this is this a particular problem with type-1 hypervisors?
It seems to me any KVM-like run loop trivially supports a range of virtio devices by virtue of trapping accesses to the signalling area of a virtqueue and allowing the VMM to handle the transaction which ever way it sees fit.
I've not quite understood the way Xen interfaces to QEMU aside from it's different to everything else. More over it seems the type-1 hypervisors are more interested in providing better isolation between segments of a system whereas VIRTIO currently assumes either the VMM or the hypervisor has full access the full guest address space. I've seen quite a lot of slides that want to isolate sections of device emulation to separate processes or even separate guest VMs.
Signature · 1 line
Alex Bennée
Quoted 36 lines · Alex Bennée
Quoted 4 lines · Jan Kiszka
Quoted 3 lines · Jan Kiszka
Quoted 2 lines · Alex Bennée
Hi,
<snip>
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
And to clarify the scope of this effort: virtio-over-ivshmem is not the fastest option to offer virtio to a guest (static "DMA" window), but it is the simplest one from the hypervisor PoV and, thus, also likely the easiest one to argue over when it comes to security and safety.
So to drill down on this is this a particular problem with type-1 hypervisors?
It seems to me any KVM-like run loop trivially supports a range of virtio devices by virtue of trapping accesses to the signalling area of a virtqueue and allowing the VMM to handle the transaction which ever way it sees fit.
Quoted 7 lines
I've not quite understood the way Xen interfaces to QEMU aside from it's different to everything else. More over it seems the type-1 hypervisors are more interested in providing better isolation between segments of a system whereas VIRTIO currently assumes either the VMM or the hypervisor has full access the full guest address space. I've seen quite a lot of slides that want to isolate sections of device emulation to separate processes or even separate guest VMs.
In Xen device emulation is done by other VMs. Normally the devices are emulated via dom0, but it is possible to have other driver domains, too (those need to get passed through the related PCI devices, of course).
PV device backends get access only to the guest pages the PV frontends allow. This is done via so called "grants", which are per guest. So a frontend can grant another Xen VM access to dedicated pages. The backend is using the grants to map those pages via the hypervisor in order to perform the I/O. After finishing the I/O the I/O-pages are unmapped by the backend again.
For legacy device emulation via qemu the guest running qemu needs to get access to all the guests memory, as the guest won't grant any pages to the emulating VM. It is possible to let qemu run in a small stub guest using PV devices in order to isolate the legacy guest from e.g. dom0.
Hope that makes it clearer,
Juergen
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 22 lines · Alex Bennée
Quoted 4 lines · Jan Kiszka
Quoted 3 lines · Jan Kiszka
Quoted 2 lines · Alex Bennée
Hi,
<snip>
Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
Quoted 8 lines
Quoted 5 lines
And to clarify the scope of this effort: virtio-over-ivshmem is not the fastest option to offer virtio to a guest (static "DMA" window), but it is the simplest one from the hypervisor PoV and, thus, also likely the easiest one to argue over when it comes to security and safety.
So to drill down on this is this a particular problem with type-1 hypervisors?
Well, this typing doesn't help here (like it rarely does). There are kvm-based setups that are stripped down and hardened in a way where other folks would rather think of "type 1". I just had a discussion around such a model for a cloud scenario that runs on kvm.
Quoted 6 lines
It seems to me any KVM-like run loop trivially supports a range of virtio devices by virtue of trapping accesses to the signalling area of a virtqueue and allowing the VMM to handle the transaction which ever way it sees fit.
Quoted 7 lines
I've not quite understood the way Xen interfaces to QEMU aside from it's different to everything else. More over it seems the type-1 hypervisors are more interested in providing better isolation between segments of a system whereas VIRTIO currently assumes either the VMM or the hypervisor has full access the full guest address space. I've seen quite a lot of slides that want to isolate sections of device emulation to separate processes or even separate guest VMs.
The point is in fact not only whether to trap IO accesses or to ask the guest to rather target something like ivshmem (in fact, that is where use cases I have in mind deviated from those of that cloud operator). It is specifically the question how the backend should be able to transfer data to/from the frontend. If you want to isolate the both from each other (driver VMs/domains/etc.), you either need a complex virtual IOMMU (or "grant tables") or a static DMA windows (like ivshmem). The former is more efficient with large transfers, the latter is much simpler and therefore more robust.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
Quoted 13 lines · Jan Kiszka
Quoted 5 lines
It seems to me any KVM-like run loop trivially supports a range of virtio devices by virtue of trapping accesses to the signalling area of a virtqueue and allowing the VMM to handle the transaction which ever way it sees fit.
Quoted 7 lines
I've not quite understood the way Xen interfaces to QEMU aside from it's different to everything else. More over it seems the type-1 hypervisors are more interested in providing better isolation between segments of a system whereas VIRTIO currently assumes either the VMM or the hypervisor has full access the full guest address space. I've seen quite a lot of slides that want to isolate sections of device emulation to separate processes or even separate guest VMs.
Quoted 8 lines
The point is in fact not only whether to trap IO accesses or to ask the guest to rather target something like ivshmem (in fact, that is where use cases I have in mind deviated from those of that cloud operator). It is specifically the question how the backend should be able to transfer data to/from the frontend. If you want to isolate the both from each other (driver VMs/domains/etc.), you either need a complex virtual IOMMU (or "grant tables") or a static DMA windows (like ivshmem). The former is more efficient with large transfers, the latter is much simpler and therefore more robust.
Jan explained it well +1
In addition to what Jan wrote, which is the most important aspect, there is also actually a problem with IO trapping with Xen x86 PV guests, but I think today is far less important than it used to be. We are talking about a type of guest designed to run without virtualization support in hardware. Trapping is not easy in that case. Today, on x86 we have PVH and HVM guests which use virtualization extensions. On ARM, all guests always had hardware virtualization support from the start. So, as of today, all guests except for old-style x86 PV guests can trap IO accesses without issues. IO trapping comes into play when you want to hook up something like the QEMU implementation of the PCI virtio backends. In fact, that works today with x86 HVM guests, but not with x86 PV guests. It doesn't work on ARM simply because Xen on ARM hasn't been using a QEMU emulator for anything yet, but there is nothing architectural that would prevent it from working. In fact, I have seen a demo of an emulator running together with Xen on ARM at a conference.
(FYI today you can run OpenAMP RPMesg, which is virtio-based, between two Xen guests by setting up pre-shared memory between them.)
Quoted 1 line · Alex Bennée
- How about HyperV and the OSX equivalent?
OS X Hypervisor.framework just uses QEMU, so it can use virtio devices too. VirtualBox also supports virtio devices.
Paolo
Quoted 5 lines · Paolo Bonzini
Quoted 1 line · Alex Bennée
- How about HyperV and the OSX equivalent?
OS X Hypervisor.framework just uses QEMU, so it can use virtio devices too. VirtualBox also supports virtio devices.
I guess these don't do any sort of vhost support so all virtio devices are handled directly in QEMU?
Signature · 5 lines
Alex Bennée _______________________________________________ Virtualization mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 4 lines · Alex Bennée
Quoted 2 lines
OS X Hypervisor.framework just uses QEMU, so it can use virtio devices too. VirtualBox also supports virtio devices.
I guess these don't do any sort of vhost support so all virtio devices are handled directly in QEMU?
OS X can use vhost-user.
Paolo
Quoted 18 lines · Alex Bennée
I'm currently trying to get my head around virtio and was wondering how widespread adoption of virtio is amongst the various hypervisors and emulators out there.
Obviously I'm familiar with QEMU both via KVM and even when just doing plain emulation (although with some restrictions). As far as I'm aware the various Rust based VMMs have vary degrees of support for virtio devices over KVM as well. CrosVM specifically is embracing virtio for multi-process device emulation.
I believe there has been some development work for supporting VIRTIO on Xen although it seems to have stalled according to:
https://wiki.xenproject.org/wiki/Virtio_On_Xen
Recently at KVM Forum there was Jan's talk about Inter-VM shared memory which proposed ivshmemv2 as a VIRTIO transport:
Quoted 8 lines
https://events19.linuxfoundation.org/events/kvm-forum-2019/program/schedule/
As I understood it this would allow Xen (and other hypervisors) a simple way to be able to carry virtio traffic between guest and end point.
So some questions:
- Am I missing anything out in that summary?
VirtualBox has virtio-net support:
https://www.virtualbox.org/manual/ch06.html
Quoted 1 line
- How about HyperV and the OSX equivalent?
macOS has *guest* drivers for VIRTIO devices:
https://www.kraxel.org/blog/2019/06/macos-qemu-guest/
Stefan