Skip to contentSkip to conversation

vVMX: VMX insn handling

PATCH Superseded Jan Beulich 2 participants 12 messages Jun 11, 2025

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover vVMX: VMX insn handling

Originally, while dealing with some other patch, I merely noticed the issue addressed by patch 2. That quickly grew then, though.

1: use reg_read()
2: adjust reg_read() for 32-bit guests
3: adjust reg_write() for 32-bit guests
4: prefer hvm_long_mode_active() in decode_vmx_inst() 5: operand size in decode_vmx_inst()
6: address size in decode_vmx_inst()

Jan

#1 lore raw

PATCH 1/6 vVMX: use reg_read()

Let's avoid such open-coding. There's also no need to use guest_cpu_user_regs(), when the function has a suitable parameter.

xen/arch/x86/hvm/vmx/vvmx.c +1 -1
@@ -2675,7 +2675,7 @@ int nvmx_n2_vmexit_handler(struct cpu_us
             {
             case VMX_CR_ACCESS_TYPE_MOV_TO_CR:
             {
-                val = *decode_gpr(guest_cpu_user_regs(), qual.gpr);
+                val = reg_read(regs, qual.gpr);
 
                 if ( qual.cr == 0 )
                 {
#3 lore raw

PATCH 2/6 vVMX: adjust reg_read() for 32-bit guests

In reply to Jan Beulich

Using the full 64-bit register values is wrong in this case; especially soon after a mode switch from long mode to 32-bit one upper halves of registers may continue to be non-zero.

Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")

---
Note that the affected VMX insns are invalid to use from compatibility mode, and hence the more expensive vmx_guest_x86_mode() doesn't need using here. (VMCALL and VMFUNC, which are permitted in compatibility mode, aren't taking this path. In fact both aren't dealt with at all [explicitly] in vvmx.c.)

xen/arch/x86/hvm/vmx/vvmx.c +6 -1
@@ -360,7 +360,12 @@ enum vmx_insn_errno set_vvmcs_real_safe(
 static unsigned long reg_read(struct cpu_user_regs *regs,
                               unsigned int index)
 {
-    return *decode_gpr(regs, index);
+    unsigned long val = *decode_gpr(regs, index);
+
+    if ( !hvm_long_mode_active(current) )
+        val = (uint32_t)val;
+
+    return val;
 }
 
 static void reg_write(struct cpu_user_regs *regs,
#4 lore raw

Re: [PATCH 2/6] vVMX: adjust reg_read() for 32-bit guests

Quoted 10 lines · Jan Beulich

Using the full 64-bit register values is wrong in this case; especially soon after a mode switch from long mode to 32-bit one upper halves of registers may continue to be non-zero.

Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")

---
Note that the affected VMX insns are invalid to use from compatibility mode, and hence the more expensive vmx_guest_x86_mode() doesn't need using here.

Fine, but you must have a comment to this effect in the code, and what prevents compatibility mode getting here?

Quoted 3 lines

(VMCALL and VMFUNC, which are permitted in compatibility mode, aren't taking this path. In fact both aren't dealt with at all [explicitly] in vvmx.c.)

VMCALL has no operands, implicit or otherwise.

VMFUNC does have implicit operands, but post-dates the last time anyone did any serious nested virt work, hence why it's unhandled.

Personally, I think vVMFUNC emulation can safely be left to whomever first wants it.  The only potential usecase I'm aware of has ceased to be for unrelated reasons.

~Andrew

#5 lore raw

Re: [PATCH 2/6] vVMX: adjust reg_read() for 32-bit guests

Quoted 12 lines · Andrew Cooper
Quoted 10 lines · Jan Beulich

Using the full 64-bit register values is wrong in this case; especially soon after a mode switch from long mode to 32-bit one upper halves of registers may continue to be non-zero.

Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")

---
Note that the affected VMX insns are invalid to use from compatibility mode, and hence the more expensive vmx_guest_x86_mode() doesn't need using here.

Quoted 2 lines

Fine, but you must have a comment to this effect in the code, and what prevents compatibility mode getting here?

Sure, I can add a comment there. As to compatibility mode - the insns will #UD, and hence no (instruction based) VMEXIT will occur.

Jan

#6 lore raw

PATCH 3/6 vVMX: adjust reg_write() for 32-bit guests

In reply to Jan Beulich

Using the full 64-bit register values is slightly wrong in this case; 32-bit writes of registers would normally zero-extend the value to 64 bits. The difference may be observable after switching (back) to 64-bit mode (even if as per the spec upper halves of registers are undefined after a mode switch, in reality they retain their values).

Fixes 33a7028fec44 ("Nexted VMX: Emulation of guest VMREAD")

---
Note that the sole affected VMX insn (VMREAD) is invalid to use from compatibility mode, and hence the more expensive vmx_guest_x86_mode() doesn't need using here.

xen/arch/x86/hvm/vmx/vvmx.c +2 -0
@@ -372,6 +372,8 @@ static void reg_write(struct cpu_user_re
                       unsigned int index,
                       unsigned long value)
 {
+    if ( !hvm_long_mode_active(current) )
+        value = (uint32_t)value;
     *decode_gpr(regs, index) = value;
 }
 
#7 lore raw

PATCH 4/6 vVMX: prefer hvm_long_mode_active() in decode_vmx_inst()

In reply to Jan Beulich

All affected VMX insns are invalid to use from compatibility mode, and hence the more expensive vmx_guest_x86_mode() doesn't need using here.

xen/arch/x86/hvm/vmx/vvmx.c +1 -1
@@ -418,7 +418,7 @@ static int decode_vmx_inst(struct cpu_us
     }
     else
     {
-        bool mode_64bit = vmx_guest_x86_mode(v) == X86_MODE_64BIT;
+        bool mode_64bit = hvm_long_mode_active(v);
 
         decode->type = VMX_INST_MEMREG_TYPE_MEMORY;
 
#8 lore raw

PATCH 5/6 vVMX: operand size in decode_vmx_inst()

In reply to Jan Beulich

Address size is entirely irrelevant to operand size determination; For VMREAD and VMWRITE outside of 64-bit mode operand size is 32 bits, while in 64-bit mode it's (naturally) 64 bits. For all other insns it's 64 bits (a physical address) or 128 bits (INVEPT, INVVPID). To limit the amount of change here, keep the latter at reading only 64 bits from guest space.

Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")

---
Beyond the wrong operand handling for INVEPT and INVVPID, the latter also doesn't even have the part read checked to have bits 16 and above all clear.

xen/arch/x86/hvm/vmx/vvmx.c +15 -12
@@ -399,13 +399,13 @@ static inline u32 __n2_secondary_exec_co
 
 static int decode_vmx_inst(struct cpu_user_regs *regs,
                            struct vmx_inst_decoded *decode,
-                           unsigned long *poperandS)
+                           unsigned long *poperandS, unsigned int size)
 {
     struct vcpu *v = current;
     union vmx_inst_info info;
     struct segment_register seg;
     unsigned long base, index, seg_base, disp, offset;
-    int scale, size;
+    unsigned int scale;
 
     __vmread(VMX_INSTRUCTION_INFO, &offset);
     info.word = offset;
@@ -437,7 +437,8 @@ static int decode_vmx_inst(struct cpu_us
 
         __vmread(EXIT_QUALIFICATION, &disp);
 
-        size = 1 << (info.fields.addr_size + 1);
+        if ( !size )
+            size = 4 << mode_64bit;
 
         offset = base + index * scale + disp;
         base = !mode_64bit || info.fields.segment >= x86_seg_fs ?
@@ -452,7 +453,9 @@ static int decode_vmx_inst(struct cpu_us
         if ( poperandS != NULL )
         {
             pagefault_info_t pfinfo;
-            int rc = hvm_copy_from_guest_linear(poperandS, base, size,
+            int rc = hvm_copy_from_guest_linear(poperandS, base,
+                                                min_t(unsigned int, size,
+                                                      sizeof(*poperandS)),
                                                 0, &pfinfo);
 
             if ( rc == HVMTRANS_bad_linear_to_gfn )
@@ -1549,7 +1552,7 @@ static int nvmx_handle_vmxon(struct cpu_
     uint32_t nvmcs_revid;
     int rc;
 
-    rc = decode_vmx_inst(regs, &decode, &gpa);
+    rc = decode_vmx_inst(regs, &decode, &gpa, sizeof(gpa));
     if ( rc != X86EMUL_OKAY )
         return rc;
 
@@ -1776,7 +1779,7 @@ static int nvmx_handle_vmptrld(struct cp
     unsigned long gpa = 0;
     int rc;
 
-    rc = decode_vmx_inst(regs, &decode, &gpa);
+    rc = decode_vmx_inst(regs, &decode, &gpa, sizeof(gpa));
     if ( rc != X86EMUL_OKAY )
         return rc;
 
@@ -1853,7 +1856,7 @@ static int nvmx_handle_vmptrst(struct cp
     unsigned long gpa = 0;
     int rc;
 
-    rc = decode_vmx_inst(regs, &decode, &gpa);
+    rc = decode_vmx_inst(regs, &decode, &gpa, sizeof(gpa));
     if ( rc != X86EMUL_OKAY )
         return rc;
 
@@ -1879,7 +1882,7 @@ static int nvmx_handle_vmclear(struct cp
     void *vvmcs;
     int rc;
 
-    rc = decode_vmx_inst(regs, &decode, &gpa);
+    rc = decode_vmx_inst(regs, &decode, &gpa, sizeof(gpa));
     if ( rc != X86EMUL_OKAY )
         return rc;
 
@@ -1941,7 +1944,7 @@ static int nvmx_handle_vmread(struct cpu
     u64 value = 0;
     int rc;
 
-    rc = decode_vmx_inst(regs, &decode, NULL);
+    rc = decode_vmx_inst(regs, &decode, NULL, 0);
     if ( rc != X86EMUL_OKAY )
         return rc;
 
@@ -1984,7 +1987,7 @@ static int nvmx_handle_vmwrite(struct cp
     enum vmx_insn_errno err;
     int rc;
 
-    rc = decode_vmx_inst(regs, &decode, &operand);
+    rc = decode_vmx_inst(regs, &decode, &operand, 0);
     if ( rc != X86EMUL_OKAY )
         return rc;
 
@@ -2026,7 +2029,7 @@ static int nvmx_handle_invept(struct cpu
     unsigned long eptp;
     int ret;
 
-    if ( (ret = decode_vmx_inst(regs, &decode, &eptp)) != X86EMUL_OKAY )
+    if ( (ret = decode_vmx_inst(regs, &decode, &eptp, 16)) != X86EMUL_OKAY )
         return ret;
 
     switch ( reg_read(regs, decode.reg2) )
@@ -2054,7 +2057,7 @@ static int nvmx_handle_invvpid(struct cp
     unsigned long vpid;
     int ret;
 
-    if ( (ret = decode_vmx_inst(regs, &decode, &vpid)) != X86EMUL_OKAY )
+    if ( (ret = decode_vmx_inst(regs, &decode, &vpid, 16)) != X86EMUL_OKAY )
         return ret;
 
     switch ( reg_read(regs, decode.reg2) )
#9 lore raw

Re: [PATCH 5/6] vVMX: operand size in decode_vmx_inst()

Quoted 7 lines · Jan Beulich

Address size is entirely irrelevant to operand size determination; For VMREAD and VMWRITE outside of 64-bit mode operand size is 32 bits, while in 64-bit mode it's (naturally) 64 bits. For all other insns it's 64 bits (a physical address) or 128 bits (INVEPT, INVVPID). To limit the amount of change here, keep the latter at reading only 64 bits from guest space.

Quoted 6 lines
Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")

---
Beyond the wrong operand handling for INVEPT and INVVPID, the latter also doesn't even have the part read checked to have bits 16 and above all clear.

There are more bugs than these.

https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/x86-next-euan

Judging by the choice of branch name, I think I'd collected those pending the re-opening of a dev window, and apparently forgotten.

At this point I don't think we want to take the branch as is (especially as I've forgotten why it wasn't taken before), but I think there is some salvageable work in there, beyond the changes in this patch.

~Andrew

#10 lore raw

Re: [PATCH 5/6] vVMX: operand size in decode_vmx_inst()

Quoted 17 lines · Andrew Cooper
Quoted 7 lines · Jan Beulich

Address size is entirely irrelevant to operand size determination; For VMREAD and VMWRITE outside of 64-bit mode operand size is 32 bits, while in 64-bit mode it's (naturally) 64 bits. For all other insns it's 64 bits (a physical address) or 128 bits (INVEPT, INVVPID). To limit the amount of change here, keep the latter at reading only 64 bits from guest space.

Quoted 6 lines
Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")

---
Beyond the wrong operand handling for INVEPT and INVVPID, the latter also doesn't even have the part read checked to have bits 16 and above all clear.

There are more bugs than these.

Quoted 8 lines

https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/x86-next-euan

Judging by the choice of branch name, I think I'd collected those pending the re-opening of a dev window, and apparently forgotten.

At this point I don't think we want to take the branch as is (especially as I've forgotten why it wasn't taken before), but I think there is some salvageable work in there, beyond the changes in this patch.

Interesting. Apparently it has been too long since this was posted, as I don't recall it at all. However, I'm not convinced everything's right there. E.g. in "x86/vvmx: Use correct sizes when reading operands" there is this hunk:

@@ -427,6 +427,7 @@ static int decode_vmx_inst(struct cpu_user_regs *regs,

     struct segment_register seg;
     unsigned long base, index, seg_base, disp, offset;
     int scale, size;

+ unsigned int bytes = vmx_guest_x86_mode(v);

     __vmread(VMX_INSTRUCTION_INFO, &offset);
     info.word = offset;

Which is hardly correct for 16-bit protected mode. INVEPT and INVVPID handling also isn't moving to 128-bit operands, despite what the title suggests. The commit message wrongly claims the upper halves don't need to be read.

Jan

#11 lore raw

PATCH 6/6 vVMX: address size in decode_vmx_inst()

In reply to Jan Beulich

While the original use of the address size file in the instruction info provided was wrong, it still wants using: The offset into the designated segment still may need truncating accordingly.

Fixes 09fce8016596 ("Nested VMX: Emulation of guest VMXON/OFF instruction")
xen/arch/x86/hvm/vmx/vvmx.c +7 -0
@@ -441,6 +441,13 @@ static int decode_vmx_inst(struct cpu_us
             size = 4 << mode_64bit;
 
         offset = base + index * scale + disp;
+        switch ( info.fields.addr_size )
+        {
+        case 0: offset = (uint16_t)offset; break;
+        case 1: offset = (uint32_t)offset; break;
+        default:                           break;
+        }
+
         base = !mode_64bit || info.fields.segment >= x86_seg_fs ?
                seg_base + offset : offset;
         if ( offset + size - 1 < offset ||