Skip to contentSkip to conversation

vmware_vga: Don't crash on too-big DEFINE_CURSOR command

PATCH New Roland Dreier 1 participant 1 message Dec 17, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vmware_vga: Don't crash on too-big DEFINE_CURSOR command

QEMU crashes with vmware_vga when running a Linux guest with the latest X.org vmware video driver if QEMU is using SDL for video output. In this case, vmware_vga advertises cursor acceleration to the guest, and the crash comes when the guest does a DEFINE_CURSOR command with a 64x64 32bpp cursor. This request overruns the image[] array in struct vmsvga_cursor_definition_s and QEMU ends up segfaulting because of memory corruption caused by writing past the end of the array.

Fix this by enlarging the image[] array to be able to hold 4096 32-bit pixels so we don't fail for the case of 64*64*32bpp, and also add error checking to avoid a crash if an even bigger request is sent by a guest.

---
 hw/vmware_vga.c |   11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)
hw/vmware_vga.c +10 -1
index f3e3749..d253a2e 100644
@@ -462,7 +462,7 @@ struct vmsvga_cursor_definition_s {
     int hot_x;
     int hot_y;
     uint32_t mask[1024];
-    uint32_t image[1024];
+    uint32_t image[4096];	/* allow for 64x64 32bpp cursor */
 };
 
 #define SVGA_BITMAP_SIZE(w, h)		((((w) + 31) >> 5) * (h))
@@ -557,6 +557,15 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s)
             cursor.height = y = vmsvga_fifo_read(s);
             vmsvga_fifo_read(s);
             cursor.bpp = vmsvga_fifo_read(s);
+
+	    if (SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask / sizeof (uint32_t) ||
+		SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image / sizeof (uint32_t)) {
+		    fprintf(stderr, "%s: DEFINE_CSURSOR too large x: %d, y: %d bpp: %d\n",
+			    __FUNCTION__, x, y, cursor.bpp);
+		    args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp);
+		    goto badcmd;
+	    }
+
             for (args = 0; args < SVGA_BITMAP_SIZE(x, y); args ++)
                 cursor.mask[args] = vmsvga_fifo_read_raw(s);
             for (args = 0; args < SVGA_PIXMAP_SIZE(x, y, cursor.bpp); args ++)