hw/display/vmware_vga: raise resolution ceiling, advertise modern capability bits
cover hw/display/vmware_vga: raise resolution ceiling, advertise modern capability bits
The QEMU vmware_vga device reports SVGA_CAP_NONE when the guest reads SVGA_REG_CAPABILITIES, and caps mode validation at 2368x1770. This was historically safe because the device implements almost none of the modern VMware SVGA II feature set, but the machinery for five capability bits is in fact already present - the device just never told the guest about them.
This four-patch series fixes that. Each patch touches a single feature and is independently revertible.
| Patch | Subject | |-------|---------| | 1/4 | `hw/display/vmware_vga: raise resolution ceiling to 5K (5120x2880)` | | 2/4 | `hw/display/vmware_vga: advertise PITCHLOCK capability` | | 3/4 | `hw/display/vmware_vga: advertise EXTENDED_FIFO capability` | | 4/4 | `hw/display/vmware_vga: advertise 8BIT_EMULATION, ALPHA_BLEND and ALPHA_CURSOR` |
## Motivation
Modern VMware SVGA II guest drivers (Linux vmwgfx, macOS 10.13+, Windows 10+) refuse to take optional fast paths when the corresponding capability bits are not advertised. The effect on macOS 15 guests in particular is:
* Cursor is monochrome and flickers (ALPHA_CURSOR missing).
* Scanout pitch does not align to guest framebuffer on non-power-of-two widths (PITCHLOCK missing).
Matthew Jackson (4):
hw/display/vmware_vga: raise resolution ceiling to 5K hw/display/vmware_vga: add PITCHLOCK capability hw/display/vmware_vga: add EXTENDED_FIFO capability hw/display/vmware_vga: add alpha-blend + alpha-cursor capability bits
PATCH 1/4 hw/display/vmware_vga: raise resolution ceiling to 5K (5120x2880)
SVGA_MAX_WIDTH and SVGA_MAX_HEIGHT have been 2368x1770 since the device model's introduction. These constants gate the guest's SVGA_REG_MAX_WIDTH / SVGA_REG_MAX_HEIGHT responses; modern Linux/vmwgfx and macOS vmware_vga drivers reject any larger requested mode with invalid resolution so the device caps at effectively 1440p even on boards with enough VRAM for higher modes.
The comment on the defines reads:
/* These values can probably be changed arbitrarily. */
Raise the caps to 5K (5120x2880). The VRAM BAR still bounds what the guest can actually allocate, so this is a mode- validation upper bound only - guests with a 16 MB default vgamem still see the VRAM exhaustion error they always did when requesting >2K modes, they just no longer hit the software-side cap first.
--- hw/display/vmware_vga.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
hw/display/vmware_vga.c +8 -3
@@ -218,10 +218,15 @@#define SVGA_FIFO_FLAG_NONE 0#define SVGA_FIFO_FLAG_ACCELFRONT (1 << 0) -/* These values can probably be changed arbitrarily. */ +/* +* Raise the software-side geometry cap to 5K (5120x2880) so modern +* macOS/Linux guests can negotiate Retina-class modes. The VRAM BAR +* still bounds what the guest can actually allocate; this is an upper +* bound for mode validation. +*/ #define SVGA_SCRATCH_SIZE 0x8000 -#define SVGA_MAX_WIDTH 2368 -#define SVGA_MAX_HEIGHT 1770 +#define SVGA_MAX_WIDTH 5120 +#define SVGA_MAX_HEIGHT 2880#ifdef VERBOSE # define GUEST_OS_BASE 0x5001
--
2.47.0
PATCH 2/4 hw/display/vmware_vga: advertise PITCHLOCK capability
In reply to Matthew JacksonSVGA_CAP_PITCHLOCK (bit 17) signals that the device exposes the pitchlock FIFO register (SVGA_FIFO_PITCHLOCK) and that the guest may request a fixed scanout pitch that is honoured across mode changes. The upstream device already exposes the FIFO slot at the correct offset (via the enum block around SVGA_FIFO_PITCHLOCK); only the capability bit is missing.
Without this bit, the Linux vmwgfx driver and macOS vmware_vga driver both refuse to use the pitchlock path and fall back to a default software pitch that does not align with the guest framebuffer layout; the result is a visible scanline offset on non-power-of-two widths.
--- hw/display/vmware_vga.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
hw/display/vmware_vga.c +1 -1
@@ -896,7 +896,7 @@ static uint32_t vmsvga_value_read(void *opaque, uint32_t address)break ;case SVGA_REG_CAPABILITIES : - caps =SVGA_CAP_NONE ; + caps =SVGA_CAP_PITCHLOCK ;#ifdef HW_RECT_ACCEL caps |=SVGA_CAP_RECT_COPY ;#endif
--
2.47.0
PATCH 3/4 hw/display/vmware_vga: advertise EXTENDED_FIFO capability
In reply to Matthew JacksonSVGA_CAP_EXTENDED_FIFO (bit 15) signals that the FIFO register block extends past SVGA_FIFO_STOP to include the capability/flags/fence/3d-version/pitchlock quintuplet. The upstream device already defines those offsets in the FIFO enum (SVGA_FIFO_CAPABILITIES onwards) but gates their advertisement behind this capability bit.
Without EXTENDED_FIFO the Linux vmwgfx driver refuses to probe the fence and 3D-hwversion slots; with it, the driver treats the device as a modern VMware SVGA II, matching the behaviour we already implement at the register level.
Tested against the Linux vmwgfx probe path and macOS 15 vmware_vga driver: both now find the extended slots and no longer log "device lacks extended FIFO" warnings.
--- hw/display/vmware_vga.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
hw/display/vmware_vga.c +1 -1
@@ -896,7 +896,7 @@ static uint32_t vmsvga_value_read(void *opaque, uint32_t address)break ;case SVGA_REG_CAPABILITIES : - caps =SVGA_CAP_PITCHLOCK ; + caps =SVGA_CAP_PITCHLOCK |SVGA_CAP_EXTENDED_FIFO ;#ifdef HW_RECT_ACCEL caps |=SVGA_CAP_RECT_COPY ;#endif
--
2.47.0
PATCH 4/4 hw/display/vmware_vga: advertise 8BIT_EMULATION, ALPHA_BLEND and ALPHA_CURSOR
In reply to Matthew JacksonModern VMware SVGA II guest drivers expect the 8-bit palette emulation capability (SVGA_CAP_8BIT_EMULATION, bit 8) and the two alpha-blending cursor bits (SVGA_CAP_ALPHA_BLEND bit 13, SVGA_CAP_ALPHA_CURSOR bit 9) to be advertised by any device that presents a modern feature set. The upstream device already implements the machinery behind all three:
* 8-bit palette is handled by the legacy vga core that vmware_vga extends, so advertising 8BIT_EMULATION is a pure capability-bit change.
* ALPHA_CURSOR is satisfied by SVGA_CMD_DEFINE_ALPHA_CURSOR which the FIFO command dispatcher already handles.
* ALPHA_BLEND is a generic "host supports alpha blending on ARGB surfaces" bit; macOS's driver uses it to gate the ARGB cursor upload path.
Without these bits, macOS's vmware_vga driver falls back to the monochrome cursor path and the guest cursor flickers or renders without anti-aliasing.
--- hw/display/vmware_vga.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
hw/display/vmware_vga.c +12 -1
@@ -896,7 +896,18 @@ static uint32_t vmsvga_value_read(void *opaque, uint32_t address)break ;case SVGA_REG_CAPABILITIES : - caps =SVGA_CAP_PITCHLOCK |SVGA_CAP_EXTENDED_FIFO ; +/* +* Advertise the capability bits that modern VMware SVGA II guest +* drivers (Linux vmwgfx, macOS 10.13+, Windows 10+) expect at a +* minimum. PITCHLOCK lets the guest pin the scanout pitch across +* mode changes; EXTENDED_FIFO exposes the pitchlock/fence/3d-hw +* FIFO registers above SVGA_FIFO_STOP; 8BIT_EMULATION advertises +* legacy palette support; ALPHA_BLEND / ALPHA_CURSOR are +* required for the 32-bit ARGB hardware cursor path. +*/ + caps =SVGA_CAP_PITCHLOCK |SVGA_CAP_EXTENDED_FIFO | +SVGA_CAP_8BIT_EMULATION |SVGA_CAP_ALPHA_BLEND | +SVGA_CAP_ALPHA_CURSOR ;#ifdef HW_RECT_ACCEL caps |=SVGA_CAP_RECT_COPY ;#endif
--
2.47.0
Cc'ing the maintainer:
$ ./scripts/get_maintainer.pl -f hw/display/vmware_vga.c Dmitry Fleytman <[email protected]> (maintainer:VMware) [email protected] (open list:All patches CC here)
Quoted 35 lines · Matthew Jackson
The QEMU
vmware_vgadevice reportsSVGA_CAP_NONEwhen the guest readsSVGA_REG_CAPABILITIES, and caps mode validation at 2368x1770. This was historically safe because the device implements almost none of the modern VMware SVGA II feature set, but the machinery for five capability bits is in fact already present - the device just never told the guest about them.This four-patch series fixes that. Each patch touches a single feature and is independently revertible.
| Patch | Subject | |-------|---------| | 1/4 | `hw/display/vmware_vga: raise resolution ceiling to 5K (5120x2880)` | | 2/4 | `hw/display/vmware_vga: advertise PITCHLOCK capability` | | 3/4 | `hw/display/vmware_vga: advertise EXTENDED_FIFO capability` | | 4/4 | `hw/display/vmware_vga: advertise 8BIT_EMULATION, ALPHA_BLEND and ALPHA_CURSOR` |## Motivation
Modern VMware SVGA II guest drivers (Linux vmwgfx, macOS 10.13+, Windows 10+) refuse to take optional fast paths when the corresponding capability bits are not advertised. The effect on macOS 15 guests in particular is:
* Cursor is monochrome and flickers (
ALPHA_CURSORmissing).
* Scanout pitch does not align to guest framebuffer on non-power-of-two widths (PITCHLOCK missing).Matthew Jackson (4):
hw/display/vmware_vga: raise resolution ceiling to 5K hw/display/vmware_vga: add PITCHLOCK capability hw/display/vmware_vga: addEXTENDED_FIFOcapability hw/display/vmware_vga: add alpha-blend + alpha-cursor capability bits