Skip to contentSkip to conversation

VMware SVGA uses incorrect depth

PATCH In review Brian Kress 5 participants 5 messages Feb 16, 2009 – Mar 2, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 VMware SVGA uses incorrect depth

The VMware SVGA adapter is set to a 24 bit depth, where as the display surfaces (and just about everything else) are using a 32 bit depth. So when you use "-vga vmware" in either the sdl or vnc displays, you get some very odd video that has the wrong colors, is repeated and is only over 3/4 of the screen. Fix seems to be to change it to 32 bit. With this patch I can use VMware video in both sdl and vnc and have it display correctly.

vmware_vga.c +1 -1
Index: hw/vmware_vga.c
===================================================================
@@ -914,7 +914,7 @@
     s->width = -1;
     s->height = -1;
     s->svgaid = SVGA_ID;
-    s->depth = 24;
+    s->depth = 32;
     s->bypp = (s->depth + 7) >> 3;
     s->cursor.on = 0;
     s->redraw_fifo_first = 0;
Acked-by on v1 1/1
#1 lore raw
Quoted 7 lines · Brian Kress

The VMware SVGA adapter is set to a 24 bit depth, where as the display surfaces (and just about everything else) are using a 32 bit depth. So when you use "-vga vmware" in either the sdl or vnc displays, you get some very odd video that has the wrong colors, is repeated and is only over 3/4 of the screen. Fix seems to be to change it to 32 bit. With this patch I can use VMware video in both sdl and vnc and have it display correctly.

This certainly fixes the distorted output with vmware_vga.

#2 lore raw
In reply to Brian Kress
Quoted 14 lines · Brian Kress

The VMware SVGA adapter is set to a 24 bit depth, where as the display surfaces (and just about everything else) are using a 32 bit depth. So when you use "-vga vmware" in either the sdl or vnc displays, you get some very odd video that has the wrong colors, is repeated and is only over 3/4 of the screen. Fix seems to be to change it to 32 bit. With this patch I can use VMware video in both sdl and vnc and have it display correctly.

Index: hw/vmware_vga.c
===================================================================

vmware_vga.c line 919

Quoted 7 lines
vmware_vga.c +1 -1
@@ -914,7 +914,7 @@
     s->width = -1;
     s->height = -1;
     s->svgaid = SVGA_ID;
-    s->depth = 24;
+    s->depth = 32;
     s->bypp = (s->depth + 7) >> 3;
 

I think having:

s->depth = 24;
s->bypp = 4;

Is a better fix. Can you test that?

Regards,

Anthony Liguori

vmware_vga.c line 919

Quoted 2 lines
vmware_vga.c +0 -0
     s->cursor.on = 0;
    s->redraw_fifo_first = 0;
#3 lore raw

2009/2/27 Anthony Liguori <[email protected]>:

Quoted 31 lines
Quoted 20 lines · Brian Kress

The VMware SVGA adapter is set to a 24 bit depth, where as the display surfaces
(and just about everything else) are using a 32 bit depth.  So when you use "-vga vmware"
in either the sdl or vnc displays, you get some very odd video that has the wrong colors, is
repeated and is only over 3/4 of the screen.  Fix seems to be to change it to 32 bit.  With
this patch I can use VMware video in both sdl and vnc and have it display correctly.

Index: hw/vmware_vga.c
===================================================================

vmware_vga.c     (working copy) line 919

Quoted 7 lines
vmware_vga.c     (working copy) renamed from vmware_vga.c     (revision 6626) +1 -1
@@ -914,7 +914,7 @@
     s->width = -1;
     s->height = -1;
     s->svgaid = SVGA_ID;
-    s->depth = 24;
+    s->depth = 32;
     s->bypp = (s->depth + 7) >> 3;
 

I think having:

Quoted 2 lines

s->depth = 24;
s->bypp = 4;

I don't think hardcoding any depth is correct (not counting the obvious performance disadvantage). The s->depth value is what is returned by the SVGA_REG_HOST_BITS_PER_PIXEL register, so it should be the host's color depth if possible (may be impossible for VNC, but easy with SDL), iirc this broke only recently.

Cheers

#4 lore raw
Quoted 6 lines · andrzej zaborowski

I don't think hardcoding any depth is correct (not counting the obvious performance disadvantage). The s->depth value is what is returned by the SVGA_REG_HOST_BITS_PER_PIXEL register, so it should be the host's color depth if possible (may be impossible for VNC, but easy with SDL), iirc this broke only recently.

The problem is that the SDL frontend is initialized after the graphic device so it is not possible to get the host resolution from sdl.c at the time.

That said, I think that setting the vmware svga color depth to

s->depth = 24;
s->bypp = 4;

by default is reasonable since it covers 99% of the cases out there.