RFC KVM: x86: Add a paranoid mode for CPUID verification
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest27 patches 27 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/27 | KVM: x86: Fix emulated CPUID features being applied to wrong sub-leafarch/ | +5 | 7 replies | |
| 2/27 | KVM: x86: Reorder the features for CPUID 7arch/ | +32 | 0 replies | |
| 3/27 | KVM: x86: Add definitions for CPUID overlaysarch/ | +23 | 0 replies | |
| 4/27 | KVM: x86: Extend F() and its variants for CPUID overlaysarch/ arch/ | +264 | 0 replies | |
| 5/27 | KVM: x86: Extend kvm_cpu_cap_{set/clear}() to configure overlaysarch/ arch/ arch/ +3 more | +73 | 0 replies | |
| 6/27 | KVM: x86: Populate TDX CPUID overlay with supported feature bitsarch/ arch/ arch/ | +184 | 0 replies | |
| 7/27 | KVM: x86: Support KVM_GET_{SUPPORTED,EMULATED}_CPUID as VM scope ioctlsarch/ arch/ arch/ | +33 | 0 replies | |
| 8/27 | KVM: x86: Thread @kvm to KVM CPU capability helpersarch/ arch/ arch/ +6 more | +98 | 1 reply | |
| 9/27 | KVM: x86: Use overlays of KVM CPU capabilitiesarch/ arch/ | +16 | 1 reply | |
| 10/27 | KVM: x86: Use vendor-specific overlay flags instead of F_CPUID_DEFAULTarch/ arch/ arch/ +1 more | +55 | 1 reply | |
| 11/27 | KVM: SVM: Drop unnecessary clears of unsupported common x86 featuresarch/ | +0 | 0 replies | |
| 12/27 | KVM: x86: Split KVM CPU cap leafs into two partsarch/ arch/ arch/ | +15 | 0 replies | |
| 13/27 | KVM: x86: Add a helper to initialize CPUID multi-bit fieldsarch/ | +11 | 0 replies | |
| 14/27 | KVM: x86: Add a helper to init multiple feature bits based on raw CPUIDarch/ arch/ | +33 | 0 replies | |
| 15/27 | KVM: x86: Add infrastructure to track CPUID entries ignored in paranoid modearch/ | +47 | 0 replies | |
| 16/27 | KVM: x86: Init allowed masks for basic CPUID range in paranoid modearch/ arch/ arch/ +2 more | +288 | 1 reply | |
| 17/27 | KVM: x86: Init allowed masks for extended CPUID range in paranoid modearch/ arch/ arch/ | +94 | 1 reply | |
| 18/27 | KVM: x86: Handle Centaur CPUID leafs in paranoid modearch/ | +5 | 0 replies | |
| 19/27 | KVM: x86: Track KVM PV CPUID features for paranoid modearch/ arch/ arch/ +1 more | +66 | 0 replies | |
| 20/27 | KVM: x86: Add per-VM flag to track CPUID paranoid modearch/ | +3 | 0 replies | |
| 21/27 | KVM: x86: Make kvm_vcpu_after_set_cpuid() return an error codearch/ arch/ | +7 | 1 reply | |
| 22/27 | KVM: x86: Verify userspace CPUID inputs in paranoid modearch/ | +113 | 1 reply | |
| 23/27 | KVM: x86: Account for runtime CPUID features in paranoid modearch/ | +24 | 1 reply | |
| 24/27 | KVM: x86: Skip paranoid CPUID check for KVM PV leafs when base is relocatedarch/ | +15 | 1 reply | |
| 25/27 | KVM: x86: Add new KVM_CAP_X86_CPUID_PARANOIDDocumentation/ arch/ arch/ +1 more | +40 | 0 replies | |
| 26/27 | KVM: x86: Add a helper to query the allowed CPUID maskarch/ arch/ | +17 | 0 replies | |
| 27/27 | KVM: TDX: Replace hardcoded CPUID filtering with the allowed maskarch/ | +13 | 1 reply |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [RFC PATCH 00/27] KVM: x86: Add a paranoid mode for CPUID verification
Hi,
This RFC series is to allow public capture of feedback from TDX developers before we have too much internal conversations on it and to initiate code review of Sashiko. It is not yet intended for review by KVM maintainers. Sean and Paolo, please feel free to ignore this version.
Originally, we had issues on TDX when a new hardware feature, which is a host state clobbering feature, is supported by new TDX modules/platforms. A host state clobbering feature requires KVM to save and restore the feature's related MSR(s) on host/guest transitions; otherwise, if the feature is used by TDs, the host state will be corrupted, leading to unexpected behavior on the host.
Currently KVM hardcodes a deny list for unsupported host clobbering features for TDX, i.e. HLE, RTM and WAITPKG. However, KVM can't keep a list of bits that it may not know about (e.g. the upcoming FRED support in TDX).
We had been working internally to propose a TDX specific solution to solve the host state clobbering feature issue. But during a PUCK meeting, Sean mentioned that KVM had a more permissive CPUID configuration interface than desired and there were problems due to it in the past for normal VMs as well. Sean suggested that KVM should introduce a more paranoid mode to check CPUID from userspace for VMs in general, as well as an opt-in interface for userspace. And TDX should use the infrastructure to enforce paranoid mode non-optionally.
This RFC patch series adds a paranoid CPUID verification mode for KVM on x86, where KVM must be explicitly aware of every CPUID feature exposed to the guest. When the CPUID paranoid mode is opted-in by userspace or enforced, KVM will reject any unknown or unsupported feature from userspace. And it starts to enforce paranoid CPUID verification for TDX.
This patch series touches a lot of lines and involves many subtle CPUID details. We may not expect reviews on these CPUID leaf specific details yet, but feedback is welcome on the framework to build the CPUID overlays and how paranoid CPUID verification is implemented.
The changes are only tested on Intel platforms. Compile-tested only for SVM.
The series is organized in following parts:
===========================================
- Patch 1 ~ 2: Cleanup patches.
- Patch 3 ~ 11: Construct CPUID overlays
This part extends kvm_cpu_caps[] into a 2D array indexed by an "overlay" dimension (CPUID_OL_DEFAULT, CPUID_OL_SVM, CPUID_OL_TDX), allowing each overlay to maintain its own set of supported CPUID features. Having separate overlays for VMX and TDX helps handle cases where KVM's support for certain features differs on Intel-compatible platforms, e.g., HLE, RTM and WAITPKG are not supported for TDX in KVM. There will be new host state clobbering features like this in the future.
Having separate overlays for VMX and SVM helps handle cases where a common feature has support on one vendor but not the other. Setting the support in common code requires additional handling in vendor specific code, e.g., SVM code needs to clear IBT, BUS_LOCK_DETECT and MSR_IMM.
More overlays could be added in the future if needed.
KVM_GET_SUPPORTED_CPUID and KVM_GET_EMULATED_CPUID are also promoted to VM-scoped IOCTLs so that userspace can query per-VM-type CPUID capabilities. CPUID overlays are a KVM internal concept; the overlay is decided by VM type and/or platform vendor.
- Patch 12 ~ 19: Build allowed CPUID values for different overlays This part builds a comprehensive table of allowed CPUID values covering the basic, extended, Centaur, and KVM paravirt CPUID ranges. For each CPUID output register, the validation follows one of three rules:
1. Ignored: the register is added to the ignored set and KVM skips validation of the userspace-provided value.
2. Mask/value check: a new KVM-only CPUID leaf enum is defined with a corresponding reverse_cpuid[] entry, and an allowed mask or fixed value is initialized per-overlay.
3. Zero check: for reserved registers or registers where no bits are supported, userspace input is checked against zero.
- Patch 20 ~ 25: Implement paranoid CPUID verification
This part adds CPUID paranoid verification to reject userspace CPUID configurations that set unsupported or unknown bits when paranoid mode is enabled for a VM.
Also, it adds the opt-in interface KVM_CAP_X86_CPUID_PARANOID for userspace and unconditionally enforces CPUID paranoid mode for TDs.
- Patch 26 ~ 27: Remove the hardcoded filter for TDX.
This part removes the hardcoded deny list for unsupported host clobbering features for TDX, and relies on the allowed mask for the TDX overlay to filter and check generically.
Opens:
======
- CPUID overlays VS. open-code checks for specific features in vendor specific callbacks.
Open-code checks for specific features in vendor callback will have less code changes, however, it tightly couples normal VM feature enablement with TDX. If a new host-state-clobbering feature is added for normal VMs, the developer has to remember to update the TDX filter list(s). Or when a common x86 feature is added for only VMX/SVM, the developer has to remember to clear the bit for the other vendor. Relying solely on mailing list reviews to catch these omissions may be more error-prone than using an overlay approach.
- This patch series uses a 2D array in common KVM code to accommodate KVM CPUID capabilities for different overlays. This avoids adding init ops and runtime ops to call into vendor modules for a few reasons:
1. kvm_ops_update() is called after ops->hardware_setup(), inside which the KVM CPU capabilities are built, runtime x86 ops can not be called. Need some workaround to allow it.
2. These inputs to build the KVM CPU capabilities for overlays are from the common KVM code or via the common KVM code helpers, which make the callbacks in vendor module just duplication of similar tedious code.
But conceptually, putting vendor-specific overlay data in the related vendor module is cleaner.
- This patch combines vCPU capability initialization and paranoid CPUID verification. It refactors the vCPU capability initialization to iterate over userspace CPUID entries rather than reverse_cpuid[], combining the paranoid check with capability setup. The purpose is to avoid iterating over CPUID entries twice for vCPU capability initialization and paranoid check separately. However, this can change the code for vCPU capability initialization a bit even when paranoid mode is disabled. It could be separated if we want to minimize the change for the non-paranoid mode.
- This patch series checks a CPUID register if part of the 32-bit range is reserved. I am not sure this is necessary for all cases. It could be simplified if we believe these reserved bits won’t cause problems according to the property of the CPUID register, so that they can be treated as ignored registers.
Binbin Wu (27):
KVM: x86: Fix emulated CPUID features being applied to wrong sub-leaf
KVM: x86: Reorder the features for CPUID 7
KVM: x86: Add definitions for CPUID overlays
KVM: x86: Extend F() and its variants for CPUID overlays
KVM: x86: Extend kvm_cpu_cap_{set/clear}() to configure overlays
KVM: x86: Populate TDX CPUID overlay with supported feature bits
KVM: x86: Support KVM_GET_{SUPPORTED,EMULATED}_CPUID as VM scope
ioctls
KVM: x86: Thread @kvm to KVM CPU capability helpers
KVM: x86: Use overlays of KVM CPU capabilities
KVM: x86: Use vendor-specific overlay flags instead of F_CPUID_DEFAULT
KVM: SVM: Drop unnecessary clears of unsupported common x86 features
KVM: x86: Split KVM CPU cap leafs into two parts
KVM: x86: Add a helper to initialize CPUID multi-bit fields
KVM: x86: Add a helper to init multiple feature bits based on raw
CPUID
KVM: x86: Add infrastructure to track CPUID entries ignored in
paranoid mode
KVM: x86: Init allowed masks for basic CPUID range in paranoid mode
KVM: x86: Init allowed masks for extended CPUID range in paranoid mode
KVM: x86: Handle Centaur CPUID leafs in paranoid mode
KVM: x86: Track KVM PV CPUID features for paranoid mode
KVM: x86: Add per-VM flag to track CPUID paranoid mode
KVM: x86: Make kvm_vcpu_after_set_cpuid() return an error code
KVM: x86: Verify userspace CPUID inputs in paranoid mode
KVM: x86: Account for runtime CPUID features in paranoid mode
KVM: x86: Skip paranoid CPUID check for KVM PV leafs when base is
relocated
KVM: x86: Add new KVM_CAP_X86_CPUID_PARANOID
KVM: x86: Add a helper to query the allowed CPUID mask
KVM: TDX: Replace hardcoded CPUID filtering with the allowed mask
Documentation/virt/kvm/api.rst | 18 + arch/x86/include/asm/kvm_host.h | 75 +- arch/x86/kvm/cpuid.c | 1224 +++++++++++++++++++++---------- arch/x86/kvm/cpuid.h | 118 ++- arch/x86/kvm/reverse_cpuid.h | 82 +++ arch/x86/kvm/svm/nested.c | 4 +- arch/x86/kvm/svm/sev.c | 6 +- arch/x86/kvm/svm/svm.c | 49 +- arch/x86/kvm/vmx/hyperv.c | 2 +- arch/x86/kvm/vmx/nested.c | 8 +- arch/x86/kvm/vmx/tdx.c | 60 +- arch/x86/kvm/vmx/vmx.c | 77 +- arch/x86/kvm/x86.c | 97 ++- arch/x86/kvm/x86.h | 2 +- include/uapi/linux/kvm.h | 1 + 15 files changed, 1298 insertions(+), 525 deletions(-)
base-commit: 6b802031877a995456c528095c41d1948546bf45
Signature · 1 line
2.46.0