KVM: nVMX: emulating SMSW/LMSW/POPF/PUSHF for L2 injects a spurious #UD (vmx_check_intercept default)
Hi,
Reporting a nested-emulation fidelity bug (correctness / KVM-vs-hardware and VMX-vs-SVM differential; not memory-safety). It appears to be the same class the stalled series "KVM: nVMX: Synthesize nested VM-Exit for supported emulation intercepts" / "Consolidate missing X86EMUL_INTERCEPTED logic in L2 emulation" (v2, Jan 2025) set out to fix, which has not landed; v7.3-rc6 still carries the stub. Sending a concrete, reproduced list of the reachable instructions in case it helps revive that work, and a minimal stop-gap.
The defect
----------
The x86 instruction emulator has no notion of L1 vs L2; all nested-intercept fidelity is delegated to vmx_check_intercept() (arch/x86/kvm/vmx/vmx.c:8190), which is a partial stub ending in:
/* TODO: check more intercepts... */ default: return X86EMUL_UNHANDLEABLE; /* vmx.c:8272-8274 */
For an instruction VMX does NOT actually intercept, the architecturally correct answer is X86EMUL_CONTINUE (let it execute), not UNHANDLEABLE. SMSW and LMSW are decoded with an intercept hook (emulate.c, II() -> ctxt->intercept = x86_intercept_smsw/_lmsw) and the hook is consulted whenever check_intercepts is set, which x86.c sets precisely for L2:
r = x86_emulate_insn(ctxt, is_guest_mode(vcpu) && ...); /* x86.c */
So for L2, SMSW reaches vmx_check_intercept() -> default -> UNHANDLEABLE before em_smsw runs. UNHANDLEABLE becomes EMULATION_FAILED, handle_emulation_failure() queues #UD unconditionally, and the CPL0/userspace-exit escape hatch is gated on !is_guest_mode, so L2 is resumed with a spurious #UD pending. For L1 (is_guest_mode false) check_intercepts is false, em_smsw runs, and the instruction completes. SMSW merely reads CR0[15:0] and has no VMX intercept control, so #UD is architecturally impossible here.
Reachability: L0 only runs its emulator on an L2 instruction that faults to it, i.e. a memory operand resolving to an MMIO/unbacked GPA, which L1 controls via EPT12. No privilege beyond running a nested guest.
Reachable instructions (reproduced, fresh VM per arm)
-----------------------------------------------------
Confirmed at runtime on BOTH vendors (Intel 7.3-rc3, AMD 7.3-rc2):
SMSW #UD on VMX, clean on SVM (II SrcNone|DstMem|Mov)
LMSW #UD on VMX, clean on SVM (II SrcMem16|Mov|Priv; also has
a real
selective-CR0 intercept to honour)
POPF #UD on VMX, clean on SVM (II ImplicitOps|Stack)
PUSHF hang on VMX, clean on SVM (store re-faults the injected #UD until
the watchdog), clean on SVM
IRET fails on BOTH -> EXCLUDED (vendor-neutral emulator
limitation, not
this bug)
The VMX-vs-SVM split is the root cause made visible: svm_check_intercept() initialises ret = X86EMUL_CONTINUE and returns it for anything the nested VMCB does not actively intercept (svm.c:4852), whereas vmx_check_intercept()'s default returns UNHANDLEABLE.
Suggested fix
-------------
The general fix is the stalled series (synthesize a nested VM-exit for emulated-but-intercepted instructions; CONTINUE otherwise). As a minimal stop-gap, add cases to vmx_check_intercept(): SMSW -> X86EMUL_CONTINUE (no intercept exists); LMSW -> evaluate the nested CR0-load/selective-CR0 intercept and either reflect a nested VM-exit or CONTINUE; PUSHF/POPF -> CONTINUE (no VMX control). SVM already models all of these.
Reproducer: an EPT12-override harness points one L2 instruction's memory operand at an unbacked (MMIO) GPA and runs it; a MOV control through the same fault path emulates cleanly, isolating the instruction as the sole variable. Happy to share it or test patches.
Thanks,
Amirmohammad