Implement a virtio GPU transport
Subsystems (3)
PATCH 1/1 Implement a virtio GPU transport
This patch implements a virtio-based transport for use by a virtualised OpenGL passthrough implementation.
The libGL and qemu-gl code to support this patch are available here:
http://gitorious.org/vm-gl-accel/qemu-gl
http://gitorious.org/vm-gl-accel/qemu-libgl
Comments please!
drivers/gpu/Makefile +1 -1
@@ -1 +1 @@ -obj-y += drm/ vga/ +obj-y += drm/ vga/ misc/
drivers/gpu/misc/Kconfig new file +8 -0
@@ -0,0 +1,8 @@ +config VIRTIOGL + tristate "Virtio userspace memory transport" + depends on VIRTIO_PCI + default n + help + A Driver to facilitate transferring data from userspace to a + hypervisor (eg. qemu) +
drivers/gpu/misc/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_VIRTIOGL) += virtio-gl.o
drivers/gpu/misc/virtio-gl.c new file +312 -0
312 lines not on this page · raw message
drivers/video/Kconfig +1 -0
@@ -16,6 +16,7 @@ source "drivers/char/agp/Kconfig" source "drivers/gpu/vga/Kconfig" source "drivers/gpu/drm/Kconfig" +source "drivers/gpu/misc/Kconfig" config VGASTATE tristate
include/linux/virtio_ids.h +1 -0
@@ -12,6 +12,7 @@#define VIRTIO_ID_CONSOLE 3/* virtio console */ #define VIRTIO_ID_RNG 4/* virtio ring */ #define VIRTIO_ID_BALLOON 5/* virtio balloon */ +#define VIRTIO_ID_GL 6/* virtio usermem */ #define VIRTIO_ID_9P 9/* 9p virtio console */ #endif /* _LINUX_VIRTIO_IDS_H */
Quoted 9 lines · Ian Molton
This patch implements a virtio-based transport for use by a virtualised OpenGL passthrough implementation.
The libGL and qemu-gl code to support this patch are available here:
http://gitorious.org/vm-gl-accel/qemu-gl
http://gitorious.org/vm-gl-accel/qemu-libglComments please!
1. copy qemu-devel an virtualization@, many virtio developers live there.
2. should start with a patch to the virtio-pci spec to document what you're doing
Quoted 10 lines
drivers/gpu/misc/virtio-gl.c new file +10
-0+static int put_data (struct virtio_gl_data *gldata) +{ +struct scatterlist *sg, *sg_list; +unsigned int count, ret, o_page, i_page, sg_entries; +struct virtio_gl_header *header = + (struct virtio_gl_header *)gldata->buffer; + + ret = header->buf_size; + + o_page = (header->buf_size +PAGE_SIZE -1 )>>PAGE_SHIFT ;
...
Quoted 8 lines
drivers/gpu/misc/virtio-gl.c new file +8
-0+sg_init_table (sg_list, sg_entries); + + sg =vmalloc_to_sg (sg_list, gldata->buffer, o_page); + sg =vmalloc_to_sg (sg, gldata->buffer, i_page); + +if (!sg) { + ret = -EIO ; + goto out_free;
drivers/gpu/misc/virtio-gl.c line 150
Quoted 10 lines
drivers/gpu/misc/virtio-gl.c new file +10
-0+ } + +/* Transfer data */ +if (virtqueue_add_buf (vq, sg_list, o_page, i_page, (void *)1 )>=0 ) { +virtqueue_kick (vq); +/* Chill out until it's done with the buffer. */ +while (!virtqueue_get_buf (vq,&count)) +cpu_relax (); + } +
This is pretty gross, and will burn lots of cpu if the hypervisor processes the queue asynchronously.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Ian Molton
This patch implements a virtio-based transport for use by a virtualised OpenGL passthrough implementation.
The libGL and qemu-gl code to support this patch are available here:
Quoted 7 lines
Quoted 5 lines
http://gitorious.org/vm-gl-accel/qemu-gl
http://gitorious.org/vm-gl-accel/qemu-libglComments please!
1. copy qemu-devel
Ok, will do.
Quoted 1 line
an virtualization@, many virtio developers live there.
you mean [email protected] ?
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Where can I find that spec?
Quoted 11 lines
Quoted 8 lines
+ /* Transfer data */
+ if (virtqueue_add_buf(vq, sg_list, o_page, i_page, (void *)1)>= 0) {
+virtqueue_kick(vq);
+ /* Chill out until it's done with the buffer. */
+ while (!virtqueue_get_buf(vq,&count))
+cpu_relax();
+ }
+This is pretty gross, and will burn lots of cpu if the hypervisor processes the queue asynchronously.
It doesnt, at present... It could be changed fairly easily ithout breaking anything if that happens though.
-Ian
Quoted 4 lines · Ian Molton
Quoted 2 lines
an virtualization@, many virtio developers live there.
you mean [email protected] ?
Yes.
Quoted 5 lines
Quoted 3 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Where can I find that spec?
http://ozlabs.org/~rusty/virtio-spec/
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
+ /* Transfer data */
+ if (virtqueue_add_buf(vq, sg_list, o_page, i_page, (void *)1)>= 0) {
+virtqueue_kick(vq);
+ /* Chill out until it's done with the buffer. */
+ while (!virtqueue_get_buf(vq,&count))
+cpu_relax();
+ }
+
Quoted 4 lines
Quoted 2 lines
This is pretty gross, and will burn lots of cpu if the hypervisor processes the queue asynchronously.
It doesnt, at present... It could be changed fairly easily ithout breaking anything if that happens though.
The hypervisor and the guest can be changed independently. The driver should be coded so that it doesn't depend on hypervisor implementation details.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 8 lines · Avi Kivity
Quoted 4 lines · Ian Molton
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Where can I find that spec?
Ok, but I'm not patching that until theres been some review.
There are links to the associated qemu and guest OS changes in my original email.
Quoted 6 lines
Quoted 2 lines
It doesnt, at present... It could be changed fairly easily ithout breaking anything if that happens though.
The hypervisor and the guest can be changed independently. The driver should be coded so that it doesn't depend on hypervisor implementation details.
Fixed - updated patch tested and attached.
-Ian
drivers/gpu/Makefile +1 -1
@@ -1 +1 @@ -obj-y += drm/ vga/ +obj-y += drm/ vga/ misc/
drivers/gpu/misc/Kconfig new file +8 -0
@@ -0,0 +1,8 @@ +config VIRTIOGL + tristate "Virtio userspace memory transport" + depends on VIRTIO_PCI + default n + help + A Driver to facilitate transferring data from userspace to a + hypervisor (eg. qemu) +
drivers/gpu/misc/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_VIRTIOGL) += virtio-gl.o
drivers/gpu/misc/virtio-gl.c new file +325 -0
325 lines not on this page · raw message
drivers/video/Kconfig +1 -0
@@ -16,6 +16,7 @@ source "drivers/char/agp/Kconfig" source "drivers/gpu/vga/Kconfig" source "drivers/gpu/drm/Kconfig" +source "drivers/gpu/misc/Kconfig" config VGASTATE tristate
include/linux/virtio_ids.h +1 -0
@@ -12,6 +12,7 @@#define VIRTIO_ID_CONSOLE 3/* virtio console */ #define VIRTIO_ID_RNG 4/* virtio ring */ #define VIRTIO_ID_BALLOON 5/* virtio balloon */ +#define VIRTIO_ID_GL 6/* virtio usermem */ #define VIRTIO_ID_9P 9/* 9p virtio console */ #endif /* _LINUX_VIRTIO_IDS_H */
Quoted 11 lines · Ian Molton
Quoted 8 lines · Avi Kivity
Quoted 4 lines · Ian Molton
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Where can I find that spec?
Ok, but I'm not patching that until theres been some review.
Well, I like to review an implementation against a spec.
Quoted 11 lines
There are links to the associated qemu and guest OS changes in my original email.
Quoted 6 lines
Quoted 2 lines
It doesnt, at present... It could be changed fairly easily ithout breaking anything if that happens though.
The hypervisor and the guest can be changed independently. The driver should be coded so that it doesn't depend on hypervisor implementation details.
Fixed - updated patch tested and attached.
drivers/gpu/misc/virtio-gl.c line 162
Quoted 8 lines
drivers/gpu/misc/virtio-gl.c new file +8
-0+ +/* Transfer data */ +if (virtqueue_add_buf (vq, sg_list, o_page, i_page, gldata)>=0 ) { +virtqueue_kick (vq); +/* Chill out until it's done with the buffer. */ +wait_for_completion (&gldata->done); + } +
Better, but still unsatisfying. If the server is busy, the caller would block. I guess it's expected since it's called from ->fsync(). I'm not sure whether that's the best interface, perhaps aio_writev is better.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 7 lines · Avi Kivity
Quoted 6 lines · Ian Molton
Quoted 5 lines · Avi Kivity
Quoted 3 lines · Ian Molton
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Where can I find that spec?
Ok, but I'm not patching that until theres been some review.
Well, I like to review an implementation against a spec.
True, but then all that would prove is that I can write a spec to match the code.
The code is proof of concept. the kernel bit is pretty simple, but I'd like to get some idea of whether the rest of the code will be accepted given that theres not much point in having any one (or two) of these components exist without the other.
Quoted 3 lines
Better, but still unsatisfying. If the server is busy, the caller would block. I guess it's expected since it's called from ->fsync(). I'm not sure whether that's the best interface, perhaps
aio_writevis better.
The caller is intended to block as the host must perform GL rendering before allowing the guests process to continue.
The only real bottleneck is that processes will block trying to submit data if another process is performing rendering, but that will only be solved when the renderer is made multithreaded. The same would happen on a real GPU if it had only one queue too.
If you look at the host code, you can see that the data is already buffered per-process, in a pretty sensible way. if the renderer itself were made a seperate thread, then this problem magically disappears (the queuing code on the host is pretty fast).
In testing, the overhead of this was pretty small anyway. Running a few dozen glxgears and a copy of ioquake3 simultaneously on an intel video card managed the same framerate with the same CPU utilisation, both with the old code and the version I just posted. Contention during rendering just isn't much of an issue.
-Ian
Quoted 4 lines · Ian Molton
Quoted 1 line
Well, I like to review an implementation against a spec.
True, but then all that would prove is that I can write a spec to match the code.
It would also allow us to check that the spec matches the requirements. Those two steps are easier than checking that the code matches the requirements.
Quoted 1 line
The code is proof of concept. the kernel bit is pretty simple, but I'd like to get some idea of whether the rest of the code will be accepted given that theres not much point in having any one (or two) of these components exist without the other.
I guess some graphics people need to be involved.
Quoted 6 lines
Quoted 4 lines
Better, but still unsatisfying. If the server is busy, the caller would block. I guess it's expected since it's called from ->fsync(). I'm not sure whether that's the best interface, perhaps
aio_writevis better.The caller is intended to block as the host must perform GL rendering before allowing the guests process to continue.
Why is that? Can't we pipeline the process?
Quoted 4 lines
The only real bottleneck is that processes will block trying to submit data if another process is performing rendering, but that will only be solved when the renderer is made multithreaded. The same would happen on a real GPU if it had only one queue too.
If you look at the host code, you can see that the data is already buffered per-process, in a pretty sensible way. if the renderer itself were made a seperate thread, then this problem magically disappears (the queuing code on the host is pretty fast).
Well, this is out of my area of expertise. I don't like it, but if it's acceptable to the gpu people, okay.
Quoted 2 lines
In testing, the overhead of this was pretty small anyway. Running a few dozen glxgears and a copy of ioquake3 simultaneously on an intel video card managed the same framerate with the same CPU utilisation, both with the old code and the version I just posted. Contention during rendering just isn't much of an issue.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 7 lines · Avi Kivity
Quoted 4 lines · Ian Molton
Quoted 1 line
Well, I like to review an implementation against a spec.
True, but then all that would prove is that I can write a spec to match the code.
It would also allow us to check that the spec matches the requirements. Those two steps are easier than checking that the code matches the requirements.
I'm extremely sceptical of any GL passthrough proposal. There have literally been half a dozen over the years and they never seem to leave proof-of-concept phase. My (limited) understanding is that it's a fundamentally hard problem that no one has adequately solved yet.
A specifically matters an awful lot less than an explanation of how the problem is being solved in a robust fashion such that it can be reviewed by people with a deeper understanding of the problem space.
Regards,
Anthony Liguori
Quoted 23 lines
Quoted 1 line
The code is proof of concept. the kernel bit is pretty simple, but I'd like to get some idea of whether the rest of the code will be accepted given that theres not much point in having any one (or two) of these components exist without the other.
I guess some graphics people need to be involved.
Quoted 6 lines
Quoted 4 lines
Better, but still unsatisfying. If the server is busy, the caller would block. I guess it's expected since it's called from ->fsync(). I'm not sure whether that's the best interface, perhaps
aio_writevis better.The caller is intended to block as the host must perform GL rendering before allowing the guests process to continue.
Why is that? Can't we pipeline the process?
Quoted 4 lines
The only real bottleneck is that processes will block trying to submit data if another process is performing rendering, but that will only be solved when the renderer is made multithreaded. The same would happen on a real GPU if it had only one queue too.
If you look at the host code, you can see that the data is already buffered per-process, in a pretty sensible way. if the renderer itself were made a seperate thread, then this problem magically disappears (the queuing code on the host is pretty fast).
Well, this is out of my area of expertise. I don't like it, but if it's acceptable to the gpu people, okay.
Quoted 2 lines
In testing, the overhead of this was pretty small anyway. Running a few dozen glxgears and a copy of ioquake3 simultaneously on an intel video card managed the same framerate with the same CPU utilisation, both with the old code and the version I just posted. Contention during rendering just isn't much of an issue.
Quoted 9 lines · Anthony Liguori
Quoted 8 lines · Avi Kivity
Quoted 2 lines · Ian Molton
True, but then all that would prove is that I can write a spec to match the code.
It would also allow us to check that the spec matches the requirements. Those two steps are easier than checking that the code matches the requirements.
There was no formal spec for this. The code was written to replace nasty undefined-instruction based data transport hacks in the (already existing) GL passthrough code.
Quoted 4 lines
I'm extremely sceptical of any GL passthrough proposal. There have literally been half a dozen over the years and they never seem to leave proof-of-concept phase. My (limited) understanding is that it's a fundamentally hard problem that no one has adequately solved yet.
The code in this case has been presented as a patch to qemu nearly 3 years ago. I've taken the patches and refactored them to use virtio rather than an undefined instruction (which fails under KVM, unlike my approach).
Its in use testing meego images and seems to be fairly reliable. it can handle compositing window managers, games, video, etc. We're currently supporting OpenGL 1.4 including shaders.
Quoted 3 lines
A specifically matters an awful lot less than an explanation of how the problem is being solved in a robust fashion such that it can be reviewed by people with a deeper understanding of the problem space.
I'm not sure there is a way to prevent nefarious tasks upsetting the hosts OpenGL with carefully crafted strings of commands, short of inspecting every single command, which is insane.
Really this needs to be done at a lower level by presenting a virtual GPU to the guest OS but I am not in a position to code that right now.
The code as it is is useful, and can always be superceeded by a virtual GPU implementation in future.
At least this breaks the chicken / egg cycle of people wanting GL support on virtual machines, but not writing stuff to take advantage of it because the support isn't there. its also a neatly encapsulated solution - if you dont want people to have access to the passthrough, simply tell qemu not to present the virtio-gl device to the guest, via qemus existing commandline options.
If this code was invasive to qemus core, I'd say 'no way' but its just not. and as the GL device is versioned, we can keep using it even if the passthrough is replaced by a virtual GPU.
Quoted 50 lines · Ian Molton
Quoted 9 lines · Anthony Liguori
Quoted 8 lines · Avi Kivity
Quoted 2 lines · Ian Molton
True, but then all that would prove is that I can write a spec to match the code.
It would also allow us to check that the spec matches the requirements. Those two steps are easier than checking that the code matches the requirements.
There was no formal spec for this. The code was written to replace nasty undefined-instruction based data transport hacks in the (already existing) GL passthrough code.
Quoted 4 lines
I'm extremely sceptical of any GL passthrough proposal. There have literally been half a dozen over the years and they never seem to leave proof-of-concept phase. My (limited) understanding is that it's a fundamentally hard problem that no one has adequately solved yet.
The code in this case has been presented as a patch to qemu nearly 3 years ago. I've taken the patches and refactored them to use virtio rather than an undefined instruction (which fails under KVM, unlike my approach).
Its in use testing meego images and seems to be fairly reliable. it can handle compositing window managers, games, video, etc. We're currently supporting OpenGL 1.4 including shaders.
Quoted 3 lines
A specifically matters an awful lot less than an explanation of how the problem is being solved in a robust fashion such that it can be reviewed by people with a deeper understanding of the problem space.
I'm not sure there is a way to prevent nefarious tasks upsetting the hosts OpenGL with carefully crafted strings of commands, short of inspecting every single command, which is insane.
Really this needs to be done at a lower level by presenting a virtual GPU to the guest OS but I am not in a position to code that right now.
The code as it is is useful, and can always be superceeded by a virtual GPU implementation in future.
At least this breaks the chicken / egg cycle of people wanting GL support on virtual machines, but not writing stuff to take advantage of it because the support isn't there. its also a neatly encapsulated solution - if you dont want people to have access to the passthrough, simply tell qemu not to present the virtio-gl device to the guest, via qemus existing commandline options.
Quoted 3 lines
If this code was invasive to qemus core, I'd say 'no way' but its just not. and as the GL device is versioned, we can keep using it even if the passthrough is replaced by a virtual GPU.
The virtio-gl implementation is basically duplicating virtio-serial. It looks like ti creates a totally separate window for the GL session. In the current form, is there really any advantage to having the code in QEMU? It could just as easily live outside of QEMU.
Regards,
Anthony Liguori
Quoted 8 lines · Anthony Liguori
Quoted 3 lines
If this code was invasive to qemus core, I'd say 'no way' but its just not. and as the GL device is versioned, we can keep using it even if the passthrough is replaced by a virtual GPU.
The virtio-gl implementation is basically duplicating virtio-serial. It looks like ti creates a totally separate window for the GL session. In the current form, is there really any advantage to having the code in QEMU? It could just as easily live outside of QEMU.
you could say much the same about any driver in qemu... you could serialise up the registers and ship the data off to be processed on another PC if you wanted to...
The code does not, however, create a seperate window for the GL session. the GL scene is rendered offscreen, and then piped back to the guest for display, so that it is fully composited into the guests graphical environment. From a user perspective, its as if the guest had hardware 3D. Performance is very reasonable, around 40fps in ioquake3 on modest (host) hardware.
In theory, the code *could* use a serial transport and render in a seperate process, but then that would make it much harder to evolve it into a GPU-like system in future.
-Ian
Quoted 4 lines · Avi Kivity
Quoted 2 lines
The caller is intended to block as the host must perform GL rendering before allowing the guests process to continue.
Why is that? Can't we pipeline the process?
No, not really. the guest may call for the scene to be rendered at any time and we have to wait for that to happen before we can return the data to it.
-Ian
Quoted 7 lines · Ian Molton
Quoted 4 lines · Avi Kivity
Quoted 2 lines
The caller is intended to block as the host must perform GL rendering before allowing the guests process to continue.
Why is that? Can't we pipeline the process?
No, not really. the guest may call for the scene to be rendered at any time and we have to wait for that to happen before we can return the data to it.
Waiting for a response is fine, but can't the guest issue a second batch while waiting for the first?
Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Ian Molton
Quoted 4 lines · Avi Kivity
Quoted 2 lines
The caller is intended to block as the host must perform GL rendering before allowing the guests process to continue.
Why is that? Can't we pipeline the process?
No, not really. the guest may call for the scene to be rendered at any time and we have to wait for that to happen before we can return the data to it.
Waiting for a response is fine, but can't the guest issue a second batch while waiting for the first?
In a threaded application I think you mean but all RPCs are dispatched holding a global lock so even within a threaded application, only a single GL call will be executed at a time.
The other scenario would be multiple applications trying to use GL but AFAICT, this is not supported in the current model.
Regards,
Anthony Liguori
Quoted 9 lines · Anthony Liguori
Quoted 5 lines · Avi Kivity
Quoted 1 line · Ian Molton
On 28/10/10 15:24, Avi Kivity wrote:
Waiting for a response is fine, but can't the guest issue a second batch while waiting for the first?
The other scenario would be multiple applications trying to use GL but AFAICT, this is not supported in the current model.
It very much is. It supports fully visually integrated rendering (no overlay windows) and even compositing GL window managers work fine, even if running 3D apps under them.
-Ian
Quoted 5 lines · Ian Molton
Quoted 3 lines · Anthony Liguori
Quoted 2 lines · Avi Kivity
Quoted 1 line · Ian Molton
On 28/10/10 15:24, Avi Kivity wrote:
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Waiting for a response is fine, but can't the guest issue a second batch while waiting for the first?
The other scenario would be multiple applications trying to use GL but AFAICT, this is not supported in the current model.
It very much is. It supports fully visually integrated rendering (no overlay windows) and even compositing GL window managers work fine, even if running 3D apps under them.
Does the kernel track userspace pid and pass that information to qemu?
Regards,
Anthony Liguori
Quoted 1 line
-Ian
Quoted 5 lines · Anthony Liguori
Quoted 3 lines
It very much is. It supports fully visually integrated rendering (no overlay windows) and even compositing GL window managers work fine, even if running 3D apps under them.
Does the kernel track userspace pid and pass that information to qemu?
Yes. And the qemu code tracks the PIDs and keeps multiple queues (one per pid).
-Ian
Quoted 6 lines · Avi Kivity
Quoted 3 lines
No, not really. the guest may call for the scene to be rendered at any time and we have to wait for that to happen before we can return the data to it.
Waiting for a response is fine, but can't the guest issue a second batch while waiting for the first?
No. Userspace doesnt do or expect that.
Multiple applications work fine, however.
-Ian
Quoted 11 lines · Ian Molton
Quoted 8 lines · Avi Kivity
Quoted 4 lines · Ian Molton
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Where can I find that spec?
Ok, but I'm not patching that until theres been some review.
Fair enough; it's a bit of a PITA to patch, so it makes sense to get the details nailed down first.
Quoted 11 lines
There are links to the associated qemu and guest OS changes in my original email.
Quoted 6 lines
Quoted 2 lines
It doesnt, at present... It could be changed fairly easily ithout breaking anything if that happens though.
The hypervisor and the guest can be changed independently. The driver should be coded so that it doesn't depend on hypervisor implementation details.
Fixed - updated patch tested and attached.
OK. FWIW, I think this is an awesome idea. I understand others are skeptical, but this seems simple and if it works and you're happy to maintain it I'm happy to let you do it :)
Cheers,
Rusty.
Quoted 26 lines · On Friday 29 October 2010 07:18:00 Rusty Russell wrote:
Quoted 11 lines · Ian Molton
Quoted 8 lines · Avi Kivity
Quoted 4 lines · Ian Molton
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Where can I find that spec?
Ok, but I'm not patching that until theres been some review.
Fair enough; it's a bit of a PITA to patch, so it makes sense to get the details nailed down first.
Quoted 10 lines
There are links to the associated qemu and guest OS changes in my original email.
Quoted 6 lines
Quoted 2 lines
It doesnt, at present... It could be changed fairly easily ithout breaking anything if that happens though.
The hypervisor and the guest can be changed independently. The driver should be coded so that it doesn't depend on hypervisor implementation details.
Quoted 5 lines
Quoted 1 line
Fixed - updated patch tested and attached.
OK. FWIW, I think this is an awesome idea. I understand others are skeptical, but this seems simple and if it works and you're happy to maintain it I'm happy to let you do it :)
From a kvm user's perspective this is a GREAT idea. Looks like Ian and friends have though about better solutions down the line and have made their code easy to superceed. If this is really the case I vote to get this in qemu/kvm asap - software opengl can be a real pain.
Thanks
Ed Tomlinson
Quoted 3 lines · Rusty Russell
Quoted 2 lines
Fixed - updated patch tested and attached.
OK. FWIW, I think this is an awesome idea.
Paravirtual OpenGL or the actual proposed implementation? Have you looked at the actual code?
If I understand correctly, the implementation is an RPC of opengl commands across virtio that are then rendered on the host to an offscreen buffer. The buffer is then sent back to the guest in rendered form.
But it's possible to mess around with other guests because the opengl commands are not scrubbed. There have been other proposals in the past that include a mini JIT that would translate opengl commands into a safe form.
Exposing just an opengl RPC transport doesn't seem that useful either. It ought to be part of a VGA card in order for it to be integrated with the rest of the graphics system without a round trip.
The alternative proposal is Spice which so far noone has mentioned. Right now, Spice seems to be taking the right approach to guest 3d support.
Regards,
Anthony Liguori
Quoted 8 lines
I understand others are skeptical,
but this seems simple and if it works and you're happy to maintain it I'm happy to let you do it :)Cheers,
Rusty.
Quoted 24 lines · ----- "Anthony Liguori
Quoted 3 lines · Rusty Russell
Quoted 2 lines
Fixed - updated patch tested and attached.
OK. FWIW, I think this is an awesome idea.
Paravirtual OpenGL or the actual proposed implementation? Have you looked at the actual code?
If I understand correctly, the implementation is an RPC of opengl commands across virtio that are then rendered on the host to an offscreen buffer. The buffer is then sent back to the guest in rendered
form.But it's possible to mess around with other guests because the opengl
commands are not scrubbed. There have been other proposals in the past
that include a mini JIT that would translate opengl commands into a safe
form.Exposing just an opengl RPC transport doesn't seem that useful either.
Quoted 8 lines
It ought to be part of a VGA card in order for it to be integrated with
the rest of the graphics system without a round trip.The alternative proposal is Spice which so far noone has mentioned. Right now, Spice seems to be taking the right approach to guest 3d support.
While we (speaking as part of the SPICE developers) want to have the same support in our virtual GPU for 3d as we have for 2d, we just don't at this point of time.
Quoted 13 lines
Regards,
Anthony Liguori
Quoted 2 lines
I understand others are skeptical,
but this seems simple and if it works and you're happy to maintainit I'm
Quoted 6 lines
happy to let you do it :)
Cheers,
Rusty.
Quoted 9 lines · Alon Levy
Quoted 5 lines
The alternative proposal is Spice which so far noone has mentioned. Right now, Spice seems to be taking the right approach to guest 3d support.
While we (speaking as part of the SPICE developers) want to have the same support in our virtual GPU for 3d as we have for 2d, we just don't at this point of time.
Yes, but I think the point is that are two general approaches to supporting 3d that are being proposed. One approach is to an RPC layer at the OpenGL level which essentially passes through the host OpenGL stack. That's what virtio-gl is. This has existed for quite a while and there are multiple transports for it. It supports serial ports, TCP sockets, a custom ISA extension for x86, and now a custom virtio transport.
Another approach would be to have a virtual GPU and to implement GPU-level commands for 3d. I have been repeated told that much of the complexity of Spice is absolutely needed for 3d and that that's a major part of the design.
GPU-level support for 3d operations has a number of advantages mainly that it's more reasonably portable to things like Windows since the 3d commands can be a superset of both OpenGL and Direct3d.
Also, Spice has an abstraction layer that doesn't simply passthrough graphics commands, but translates/sanitizes them first. That's another advantage over OpenGL passthrough. Without a layer to sanitize commands, a guest can do funky things with the host or other guests.
I think a Spice-like approach is the best thing long term. In the short term, I think doing the GL marshalling over virtio-serial makes a ton of sense since the kernel driver is already present upstream. It exists exactly for things like this.
In the very, very short term, I think an external backend to QEMU also makes a lot of sense because that's something that Just Works today. I think we can consider integrating it into QEMU (or at least simplifying the execution of the backend) but integrating into QEMU is going to require an awful lot of the existing code to be rewritten. Keeping it separate has the advantage of allowing something to Just Work as an interim solution as we wait for proper support in Spice.
Regards,
Anthony Liguori
Quoted 18 lines
Quoted 18 lines
Regards,
Anthony Liguori
Quoted 3 lines
I understand others are skeptical,
but this seems simple and if it works and you're happy to maintainit I'm
Quoted 7 lines
happy to let you do it :)
Cheers,
Rusty.
Quoted 5 lines · Anthony Liguori
Quoted 3 lines · Alon Levy
While we (speaking as part of the SPICE developers) want to have the same support in our virtual GPU for 3d as we have for 2d, we just don't at this point of time.
Would it be helpful to you to have /something/ that works in the interim? I'm happy to work with you guys so that we dont need to reinvent the wheel ;-)
Quoted 5 lines
Yes, but I think the point is that are two general approaches to supporting 3d that are being proposed. One approach is to an RPC layer at the OpenGL level which essentially passes through the host OpenGL stack. That's what virtio-gl is. This has existed for quite a while and there are multiple transports for it.
Well, sort of. this version is heavily modified and only the virtio transport is supported in it. Its quite a large code cleanup.
Quoted 1 line
It supports serial ports, TCP sockets, a custom ISA extension for x86,
The custom ISA idea is cut too since it fails to play nice with KVM. Virtio-gl offers better security too.
Quoted 4 lines
Another approach would be to have a virtual GPU and to implement GPU-level commands for 3d. I have been repeated told that much of the complexity of Spice is absolutely needed for 3d and that that's a major part of the design.
I've not seen any implementations of this type that are even close to useable.
Quoted 3 lines
GPU-level support for 3d operations has a number of advantages mainly that it's more reasonably portable to things like Windows since the 3d commands can be a superset of both OpenGL and Direct3d.
Agreed.
Quoted 3 lines
Also, Spice has an abstraction layer that doesn't simply passthrough graphics commands, but translates/sanitizes them first. That's another advantage over OpenGL passthrough.
I'm not sure that thats actually needed if you are careful about what commands you implement on the virtual GPU (and how)
Quoted 2 lines
Without a layer to sanitize commands,
a guest can do funky things with the host or other guests.
It shouldnt be possible for the guest to hurt other guests actual GL rendering (or the hosts) as each PID is handed a context (or contexts) of its own. It is possible for the guest to cause the host problems though. this is a design flaw in the whole idea of GL RPC. It *isnt* reasonable to make it secure, but it certainly is useful right now since theres no alternative available.
Quoted 4 lines
I think a Spice-like approach is the best thing long term. In the short term, I think doing the GL marshalling over virtio-serial makes a ton of sense since the kernel driver is already present upstream. It exists exactly for things like this.
The virtio driver enfoces the PID field and understands the packet format used. Its better than using serial. Its also just one driver - which doesnt have any special interdependencies and can be extended or got rid of in future if and when better things come along.
Quoted 2 lines
In the very, very short term, I think an external backend to QEMU also makes a lot of sense because that's something that Just Works today.
Whos written that? The 2007 patch I've been working on and updating simply fails to work altogether without huge alterations on current qemu.
My current patch touches a tiny part of the qemu sources. It works today.
Quoted 6 lines
I
think we can consider integrating it into QEMU (or at least simplifying the execution of the backend) but integrating into QEMU is going to require an awful lot of the existing code to be rewritten. Keeping it separate has the advantage of allowing something to Just Work as an interim solution as we wait for proper support in Spice.
I dont know why you think integrating it into qemu is hard? I've already done it. I added one virtio driver and a seperate offscreen renderer. it touches the qemu code in *one* place. There should be no need to rewrite anything.
Quoted 2 lines · Ian Molton
The virtio driver enfoces the PID field and understands the packet format used. Its better than using serial. Its also just one driver - which doesnt have any special interdependencies and can be extended or got rid of in future if and when better things come along.
Why is it better than using virtio-serial?
Quoted 7 lines
Quoted 3 lines
In the very, very short term, I think an external backend to QEMU also makes a lot of sense because that's something that Just Works today.
Whos written that? The 2007 patch I've been working on and updating simply fails to work altogether without huge alterations on current qemu.
My current patch touches a tiny part of the qemu sources. It works today.
But it's not at all mergable in the current form. If you want to do the work of getting it into a mergable state (cleaning up the coding style, moving it to hw/, etc.) than I'm willing to consider it. But I don't think a custom virtio transport is the right thing to do here.
However, if you want something that Just Works with the least amount of code possible, just split it into a separate process and we can stick it in a contrib/ directory or something.
Quoted 9 lines
Quoted 7 lines
I
think we can consider integrating it into QEMU (or at least simplifying the execution of the backend) but integrating into QEMU is going to require an awful lot of the existing code to be rewritten. Keeping it separate has the advantage of allowing something to Just Work as an interim solution as we wait for proper support in Spice.I dont know why you think integrating it into qemu is hard? I've already done it.
Adding a file that happens to compile as part of qemu even though it doesn't actually integrate with qemu in any meaningful way is not integrating. That's just build system manipulation.
Regards,
Anthony Liguori
Quoted 1 line
I added one virtio driver and a seperate offscreen renderer. it touches the qemu code in *one* place. There should be no need to rewrite anything.
Quoted 3 lines · Anthony Liguori
On 11/03/2010 01:03 PM, Ian Molton wrote:
Why is it better than using virtio-serial?
For one thing, it enforces the PID in kernel so the guests processes cant screw each other over by forging the PID passed to qemu.
Quoted 6 lines
Quoted 1 line
My current patch touches a tiny part of the qemu sources. It works today.
But it's not at all mergable in the current form. If you want to do the work of getting it into a mergable state (cleaning up the coding style, moving it to hw/, etc.) than I'm willing to consider it. But I don't think a custom virtio transport is the right thing to do here.
Hm, I thought I'd indented everything in qemus odd way... Is there a codingstyle document or a checkpatch-like tool for qemu?
I'm happy to make the code meet qemus coding style.
Quoted 3 lines
However, if you want something that Just Works with the least amount of code possible, just split it into a separate process and we can stick it in a contrib/ directory or something.
I dont see what splitting it into a seperate process buys us given we still need the virtio-gl driver in order to enforce the PID. The virtio driver is probably quite a bit more efficient at marshalling the data too, given that it was designed alongside the userspace code.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I
think we can consider integrating it into QEMU (or at least simplifying the execution of the backend) but integrating into QEMU is going to require an awful lot of the existing code to be rewritten.
Why? aside from codingstyle, whats massively wrong with it thats going to demand a total rewrite?
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Keeping it
separate has the advantage of allowing something to Just Work as an interim solution as we wait for proper support in Spice.
Why does keeping it seperate make life easier? qemu is in a git repo. when the time comes, if it reall is a total replacement, git-rm will nuke it all.
Quoted 6 lines
Quoted 2 lines
I dont know why you think integrating it into qemu is hard? I've already done it.
Adding a file that happens to compile as part of qemu even though it doesn't actually integrate with qemu in any meaningful way is not integrating. That's just build system manipulation.
Uh? Either it compiles and works as part of qemu (which it does) or it doesnt. How is that not integrated? I've added it to the configure script too.
-Ian
Ping ?
Quoted 59 lines · Ian Molton
Quoted 3 lines · Anthony Liguori
On 11/03/2010 01:03 PM, Ian Molton wrote:
Why is it better than using virtio-serial?
For one thing, it enforces the PID in kernel so the guests processes cant screw each other over by forging the PID passed to qemu.
Quoted 7 lines
Quoted 2 lines
My current patch touches a tiny part of the qemu sources. It works today.
But it's not at all mergable in the current form. If you want to do the work of getting it into a mergable state (cleaning up the coding style, moving it to hw/, etc.) than I'm willing to consider it. But I don't think a custom virtio transport is the right thing to do here.
Hm, I thought I'd indented everything in qemus odd way... Is there a codingstyle document or a checkpatch-like tool for qemu?
I'm happy to make the code meet qemus coding style.
Quoted 3 lines
However, if you want something that Just Works with the least amount of code possible, just split it into a separate process and we can stick it in a contrib/ directory or something.
I dont see what splitting it into a seperate process buys us given we still need the virtio-gl driver in order to enforce the PID. The virtio driver is probably quite a bit more efficient at marshalling the data too, given that it was designed alongside the userspace code.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I
think we can consider integrating it into QEMU (or at least simplifying the execution of the backend) but integrating into QEMU is going to require an awful lot of the existing code to be rewritten.Why? aside from codingstyle, whats massively wrong with it thats going to demand a total rewrite?
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Keeping it
separate has the advantage of allowing something to Just Work as an interim solution as we wait for proper support in Spice.Why does keeping it seperate make life easier? qemu is in a git repo. when the time comes, if it reall is a total replacement, git-rm will nuke it all.
Quoted 6 lines
Quoted 2 lines
I dont know why you think integrating it into qemu is hard? I've already done it.
Adding a file that happens to compile as part of qemu even though it doesn't actually integrate with qemu in any meaningful way is not integrating. That's just build system manipulation.
Uh? Either it compiles and works as part of qemu (which it does) or it doesnt. How is that not integrated? I've added it to the configure script too.
-Ian
Quoted 1 line · Ian Molton
Ping ?
I think the best way forward is to post patches.
To summarize what I was trying to express in the thread, I think this is not the right long term architecture but am not opposed to it as a short term solution. I think having a new virtio device is a bad design choice but am not totally opposed to it.
My advice is that using virtio-serial + an external tool is probably the least amount of work to get something working and usable with QEMU. If you want to go for the path of integration, you're going to have to fix all of the coding style issues and make the code fit into QEMU. Dropping a bunch of junk into target-i386/ is not making the code fit into QEMU.
If you post just what you have now in patch form, I can try to provide more concrete advice ignoring the coding style problems.
Regards,
Anthony Liguori
Quoted 4 lines · Anthony Liguori
Quoted 1 line · Ian Molton
Ping ?
I think the best way forward is to post patches.
I posted links to the git trees. I can post patches, but they are *large*. Do you really want me to post them?
Quoted 4 lines
To summarize what I was trying to express in the thread, I think this is not the right long term architecture but am not opposed to it as a short term solution. I think having a new virtio device is a bad design choice but am not totally opposed to it.
Ok! (I agree (that this should be a short term solution) :) )
Quoted 3 lines
you want to go for the path of integration, you're going to have to fix all of the coding style issues and make the code fit into QEMU. Dropping a bunch of junk into target-i386/ is not making the code fit into QEMU.
I agree. how about hw/gl for the renderer and hw/ for the virtio module?
Quoted 2 lines
If you post just what you have now in patch form, I can try to provide more concrete advice ignoring the coding style problems.
I can post patches, although I dont think LKML would appreciate the volume! I can post them to the qemu list if you do.
-Ian
Quoted 7 lines · Ian Molton
Quoted 4 lines · Anthony Liguori
Quoted 1 line · Ian Molton
Ping ?
I think the best way forward is to post patches.
I posted links to the git trees. I can post patches, but they are *large*. Do you really want me to post them?
Yes, and they have to be split up into something reviewable.
Quoted 6 lines
Quoted 5 lines
To summarize what I was trying to express in the thread, I think this is not the right long term architecture but am not opposed to it as a short term solution. I think having a new virtio device is a bad design choice but am not totally opposed to it.
Quoted 7 lines
Ok! (I agree (that this should be a short term solution) :) )
Quoted 3 lines
you want to go for the path of integration, you're going to have to fix all of the coding style issues and make the code fit into QEMU. Dropping a bunch of junk into target-i386/ is not making the code fit into QEMU.
I agree. how about hw/gl for the renderer and hw/ for the virtio module?
That would be fine.
Quoted 4 lines
Quoted 2 lines
If you post just what you have now in patch form, I can try to provide more concrete advice ignoring the coding style problems.
I can post patches, although I dont think LKML would appreciate the volume! I can post them to the qemu list if you do.
Yes, qemu is where I was suggesting you post them.
Regards,
Anthony Liguori
Quoted 1 line
-Ian
Quoted 11 lines · Ian Molton
Quoted 5 lines · Anthony Liguori
Quoted 3 lines · Alon Levy
While we (speaking as part of the SPICE developers) want to have the same support in our virtual GPU for 3d as we have for 2d, we just don't at this point of time.
Would it be helpful to you to have /something/ that works in the interim? I'm happy to work with you guys so that we dont need to reinvent the wheel ;-)
In case it wasn't clear, I think putting virtio-gl is a good idea, exactly because it works right now.
[Snip]
Quoted 8 lines · Alon Levy
Quoted 7 lines · Ian Molton
Quoted 5 lines · Anthony Liguori
Quoted 3 lines · Alon Levy
While we (speaking as part of the SPICE developers) want to have the same support in our virtual GPU for 3d as we have for 2d, we just don't at this point of time.
Quoted 5 lines
Quoted 3 lines
Would it be helpful to you to have /something/ that works in the interim? I'm happy to work with you guys so that we dont need to reinvent the wheel ;-)
In case it wasn't clear, I think putting virtio-gl is a good idea, exactly because it works right now.
Thanks :)
-Ian
Quoted 7 lines · Rusty Russell
Quoted 6 lines · Ian Molton
Quoted 5 lines · Avi Kivity
Quoted 3 lines · Ian Molton
Quoted 2 lines
2. should start with a patch to the virtio-pci spec to document what you're doing
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Where can I find that spec?
Ok, but I'm not patching that until theres been some review.
Fair enough; it's a bit of a PITA to patch, so it makes sense to get the details nailed down first.
I thought so :)
Quoted 5 lines
Quoted 1 line
Fixed - updated patch tested and attached.
OK. FWIW, I think this is an awesome idea. I understand others are skeptical, but this seems simple and if it works and you're happy to maintain it I'm happy to let you do it :)
Awesoe, thanks Rusty :)