Skip to contentSkip to conversation

Fix vga segfaults or screen corruption with large memory guests

PATCH RESEND Applied Avi Kivity 2 participants 2 messages Apr 27, 2009

lore
Share & tools
lore

Avi Kivity

author
#0 lore raw

PATCH RESEND 1/1 Fix vga segfaults or screen corruption with large memory guests

page0 and friends are ram addresses; a smaller size will overflow and cause a segfault or random corruption.

Change them to ram_addr_t.

---
 hw/vga.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)
hw/vga.c +6 -5
index 2f122ee..517ce3d 100644
@@ -1584,8 +1584,9 @@ static void vga_sync_dirty_bitmap(VGAState *s)
  */
 static void vga_draw_graphic(VGAState *s, int full_update)
 {
-    int y1, y, update, page_min, page_max, linesize, y_start, double_scan, mask, depth;
-    int width, height, shift_control, line_offset, page0, page1, bwidth, bits;
+    int y1, y, update, linesize, y_start, double_scan, mask, depth;
+    int width, height, shift_control, line_offset, bwidth, bits;
+    ram_addr_t page0, page1, page_min, page_max;
     int disp_width, multi_scan, multi_run;
     uint8_t *d;
     uint32_t v, addr1, addr;
@@ -1723,8 +1724,8 @@ static void vga_draw_graphic(VGAState *s, int full_update)
     addr1 = (s->start_addr * 4);
     bwidth = (width * bits + 7) / 8;
     y_start = -1;
-    page_min = 0x7fffffff;
-    page_max = -1;
+    page_min = -1;
+    page_max = 0;
     d = ds_get_data(s->ds);
     linesize = ds_get_linesize(s->ds);
     y1 = 0;
@@ -1791,7 +1792,7 @@ static void vga_draw_graphic(VGAState *s, int full_update)
                    disp_width, y - y_start);
     }
     /* reset modified pages */
-    if (page_max != -1) {
+    if (page_max >= page_min) {
         cpu_physical_memory_reset_dirty(page_min, page_max + TARGET_PAGE_SIZE,
                                         VGA_DIRTY_FLAG);
     }
Signature · 1 line
1.6.1.1
Applied
#1 lore raw
Quoted 6 lines · Avi Kivity

page0 and friends are ram addresses; a smaller size will overflow and
cause a segfault or random corruption.

Change them to ram_addr_t.

Thanks, applied.

At least TCX needs a similar fix.