Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NLNUX-0006jt-EP for qemu-devel@nongnu.org; Thu, 17 Dec 2009 16:02:33 -0500 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1NLNUS-0006Wz-BJ for qemu-devel@nongnu.org; Thu, 17 Dec 2009 16:02:32 -0500 Received: from [199.232.76.173] (port=34669 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NLNUS-0006Wl-6c for qemu-devel@nongnu.org; Thu, 17 Dec 2009 16:02:28 -0500 Received: from sj-iport-4.cisco.com ([171.68.10.86]:4862) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.60) (envelope-from ) id 1NLNUR-0004fA-Is for qemu-devel@nongnu.org; Thu, 17 Dec 2009 16:02:27 -0500 Received: from xbh-sjc-231.amer.cisco.com (xbh-sjc-231.cisco.com [128.107.191.100]) by sj-core-3.cisco.com (8.13.8/8.14.3) with ESMTP id nBHL2O3b023594 for ; Thu, 17 Dec 2009 21:02:24 GMT From: Roland Dreier Date: Thu, 17 Dec 2009 13:02:23 -0800 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Qemu-devel] [PATCH] vmware_vga: Don't crash on too-big DEFINE_CURSOR command List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org QEMU crashes with vmware_vga when running a Linux guest with the latest X.org vmware video driver if QEMU is using SDL for video output. In this case, vmware_vga advertises cursor acceleration to the guest, and the crash comes when the guest does a DEFINE_CURSOR command with a 64x64 32bpp cursor. This request overruns the image[] array in struct vmsvga_cursor_definition_s and QEMU ends up segfaulting because of memory corruption caused by writing past the end of the array. Fix this by enlarging the image[] array to be able to hold 4096 32-bit pixels so we don't fail for the case of 64*64*32bpp, and also add error checking to avoid a crash if an even bigger request is sent by a guest. Signed-off-by: Roland Dreier --- hw/vmware_vga.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/hw/vmware_vga.c b/hw/vmware_vga.c index f3e3749..d253a2e 100644 --- a/hw/vmware_vga.c +++ b/hw/vmware_vga.c @@ -462,7 +462,7 @@ struct vmsvga_cursor_definition_s { int hot_x; int hot_y; uint32_t mask[1024]; - uint32_t image[1024]; + uint32_t image[4096]; /* allow for 64x64 32bpp cursor */ }; #define SVGA_BITMAP_SIZE(w, h) ((((w) + 31) >> 5) * (h)) @@ -557,6 +557,15 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s) cursor.height = y = vmsvga_fifo_read(s); vmsvga_fifo_read(s); cursor.bpp = vmsvga_fifo_read(s); + + if (SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask / sizeof (uint32_t) || + SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image / sizeof (uint32_t)) { + fprintf(stderr, "%s: DEFINE_CSURSOR too large x: %d, y: %d bpp: %d\n", + __FUNCTION__, x, y, cursor.bpp); + args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp); + goto badcmd; + } + for (args = 0; args < SVGA_BITMAP_SIZE(x, y); args ++) cursor.mask[args] = vmsvga_fifo_read_raw(s); for (args = 0; args < SVGA_PIXMAP_SIZE(x, y, cursor.bpp); args ++)