Crop VNC update requests to avoid segfaults

I was booting a guest that lowered the screen resolution after I logged in, so my VNC client was running at a larger resolution (1024x768) than the actual Qemu framebuffer's resolution (800x600). When the VNC client requested an update, Qemu tried to set the dirty bits and memset the data for an area of the screen that was non-existant, and it segfaulted.

I've written a patch that "crops" the coordinates (both x and y, even though only y is actually used) of the update region to the actual size of the display to avoid this problem. It is attached. I made it against Qemu CVS.

Comments, suggestions, and constructive criticism is appreciated.

Thank you,

Thomas Tuttle

------------------------------------------------------------------------

Index: vnc.c
===================================================================
RCS file: /sources/qemu/qemu/vnc.c,v
retrieving revision 1.13
diff -u -r1.13 vnc.c

vnc.c +0 -0

     int i;
     vs->need_update = 1;
     if (!incremental) {

Review

Quoted 6 lines · Thomas Tuttle

(quoted diff) +6 -0

+    if (x_position > vs->ds->width)  x_position = vs->ds->width;
+    if (y_position > vs->ds->height) y_position = vs->ds->height;
+    if (x_position + w >= vs->ds->width)  w = vs->ds->width  - x_position;
+    if (y_position + h >= vs->ds->height) h = vs->ds->height - y_position;
+    if (w < 0) w = 0;
+    if (h < 0) h = 0;

These last two lines aren't strictly needed since x_position cannot be > than vs->ds->width due to the first check but otherwise the patch looks good.