KVM: X86: TDX support
Subsystems (8)
v2 cover KVM: X86: TDX support
From: Isaku Yamahata <[email protected]>
* What's TDX?
TDX stands for Trust Domain Extensions which isolates VMs from the virtual-machine manager (VMM)/hypervisor and any other software on the platform. [1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.
* The goal of this RFC patch
The purpose of this post is to get feedback early on high level design issue of KVM enhancement for TDX. The detailed coding (variable naming etc) is not cared of. This patch series is incomplete (not working). So it's RFC. Although multiple software components, not only KVM but also QEMU, guest Linux and virtual bios, need to be updated, this includes only KVM VMM part. For those who are curious to changes to other component, there are public repositories at github. [8], [9]
* Patch organization
The patch 66 is main change. The preceding patches(1-65) The preceding patches(01-61) are refactoring the code and introducing additional hooks.
- 01-12: They are preparations. introduce architecture constants, code refactoring, export symbols for following patches.
- 13-40: start to introduce the new type of VM and allow the coexistence of multiple type of VM. allow/disallow KVM ioctl where appropriate. Especially make per-system ioctl to per-VM ioctl.
- 41-65: refactoring KVM VMX/MMU and adding new hooks for Secure EPT.
- 66: main patch to add "basic" support for building/running TDX.
- 67: trace points for
- 68-69: Documentation
* TODOs
Those major features are missing from this patch series to keep this patch series small.
- load/initialize TDX module
split out from this patch series.
- unmapping private page
Will integrate Kirill's patch to show how kvm will utilize it.
- qemu gdb stub support
- Large page support
- guest PMU support
- TDP MMU support
- and more
Changes from v1:
- rebase to v5.13
- drop load/initialization of TDX module
- catch up the update of related specifications.
- rework on C-wrapper function to invoke seamcall
- various code clean up
[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF
Isaku Yamahata (11):
KVM: TDX: introduce config for KVM TDX support
KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
KVM: TDX: add a helper function for kvm to call seamcall
KVM: TDX: add trace point before/after TDX SEAMCALLs
KVM: TDX: Print the name of SEAMCALL status code
KVM: Add per-VM flag to mark read-only memory as unsupported
KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
KVM: TDX: add trace point for TDVMCALL and SEPT operation
KVM: TDX: add document on TDX MODULE Documentation/virtual/kvm: Add Trust Domain Extensions(TDX)
Kai Huang (2):
KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level
routes
cpu/hotplug: Document that TDX also depends on booting CPUs once
Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits
Sean Christopherson (53):
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add architectural definitions for structures and values
KVM: TDX: define and export helper functions for KVM TDX support
KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Split core of hypercall emulation to helper function
KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
KVM: Add infrastructure and macro to mark VM as bugged
KVM: Export kvm_make_all_cpus_request() for use in marking VMs as
bugged
KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the
VM
KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
KVM: Add max_vcpus field in common 'struct kvm'
KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
KVM: x86: Hoist kvm_dirty_regs check out of sync_regs()
KVM: x86: Introduce "protected guest" concept and block disallowed
ioctls
KVM: x86: Add per-VM flag to disable direct IRQ injection
KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
KVM: x86: Add flag to mark TSC as immutable (for TDX)
KVM: Add per-VM flag to disable dirty logging of memslots for TDs
KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: x86: Add guest_supported_xss placholder
KVM: Export kvm_is_reserved_pfn() for use by TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Refactor shadow walk in __direct_map() to reduce
indentation
KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
KVM: x86/mmu: Move 'pfn' variable to caller of direct_page_fault()
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Define EPT Violation architectural bits
KVM: VMX: Define VMCS encodings for shared EPT pointer
KVM: VMX: Add 'main.c' to wrap VMX and TDX
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: VMX: Move register caching logic to common code
KVM: TDX: Define TDCALL exit reason
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
KVM: VMX: MOVE GDT and IDT accessors to common code
KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX
code
KVM: TDX: Add "basic" support for building and running Trust Domains
Xiaoyao Li (2):
KVM: TDX: Introduce pr_seamcall_ex_ret_info() to print more info when
SEAMCALL fails
KVM: X86: Introduce initial_tsc_khz in struct kvm_arch
Documentation/virt/kvm/api.rst | 6 +- Documentation/virt/kvm/intel-tdx.rst | 441 ++++++ Documentation/virt/kvm/tdx-module.rst | 48 + arch/arm64/include/asm/kvm_host.h | 3 - arch/arm64/kvm/arm.c | 7 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/x86/Kbuild | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/kvm-x86-ops.h | 8 + arch/x86/include/asm/kvm_boot.h | 30 + arch/x86/include/asm/kvm_host.h | 55 +- arch/x86/include/asm/virtext.h | 25 + arch/x86/include/asm/vmx.h | 17 + arch/x86/include/uapi/asm/kvm.h | 60 + arch/x86/include/uapi/asm/vmx.h | 7 +- arch/x86/kernel/asm-offsets_64.c | 15 + arch/x86/kvm/Kconfig | 11 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/boot/Makefile | 6 + arch/x86/kvm/boot/seam/tdx_common.c | 242 +++ arch/x86/kvm/boot/seam/tdx_common.h | 13 + arch/x86/kvm/ioapic.c | 4 + arch/x86/kvm/irq_comm.c | 13 +- arch/x86/kvm/lapic.c | 7 +- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/mmu.h | 31 +- arch/x86/kvm/mmu/mmu.c | 526 +++++-- arch/x86/kvm/mmu/mmu_internal.h | 3 + arch/x86/kvm/mmu/paging_tmpl.h | 25 +- arch/x86/kvm/mmu/spte.c | 15 +- arch/x86/kvm/mmu/spte.h | 18 +- arch/x86/kvm/svm/svm.c | 18 +- arch/x86/kvm/trace.h | 138 ++ arch/x86/kvm/vmx/common.h | 178 +++ arch/x86/kvm/vmx/main.c | 1098 ++++++++++++++ arch/x86/kvm/vmx/posted_intr.c | 6 + arch/x86/kvm/vmx/seamcall.S | 64 + arch/x86/kvm/vmx/seamcall.h | 68 + arch/x86/kvm/vmx/tdx.c | 1958 +++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 267 ++++ arch/x86/kvm/vmx/tdx_arch.h | 370 +++++ arch/x86/kvm/vmx/tdx_errno.h | 202 +++ arch/x86/kvm/vmx/tdx_ops.h | 218 +++ arch/x86/kvm/vmx/tdx_stubs.c | 45 + arch/x86/kvm/vmx/vmcs.h | 11 - arch/x86/kvm/vmx/vmenter.S | 146 ++ arch/x86/kvm/vmx/vmx.c | 509 ++----- arch/x86/kvm/x86.c | 285 +++- include/linux/kvm_host.h | 51 +- include/uapi/linux/kvm.h | 2 + kernel/cpu.c | 4 + tools/arch/x86/include/uapi/asm/kvm.h | 55 + tools/include/uapi/linux/kvm.h | 2 + virt/kvm/kvm_main.c | 44 +- 54 files changed, 6717 insertions(+), 672 deletions(-) create mode 100644 Documentation/virt/kvm/intel-tdx.rst create mode 100644 Documentation/virt/kvm/tdx-module.rst create mode 100644 arch/x86/include/asm/kvm_boot.h create mode 100644 arch/x86/kvm/boot/Makefile create mode 100644 arch/x86/kvm/boot/seam/tdx_common.c create mode 100644 arch/x86/kvm/boot/seam/tdx_common.h create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/seamcall.S create mode 100644 arch/x86/kvm/vmx/seamcall.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/tdx_stubs.c
Signature · 1 line
2.25.1
RFC PATCH v2 1/69 KVM: TDX: introduce config for KVM TDX support
From: Isaku Yamahata <[email protected]>
Add new config KVM_INTEL_TDX for KVM TDX support. This patch introduce a config only. Later patch will implement feature depends on it.
--- arch/x86/kvm/Kconfig | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/x86/kvm/Kconfig +11 -0
@@ -96,6 +96,17 @@ config X86_SGX_KVM If unsure, say N. +config KVM_INTEL_TDX + bool "Trusted Domain eXtensions(TDX) Host support" + depends on KVM_INTEL && X86_64 + select FW_LOADER + + help + Extends KVM on Intel processors to support Trusted Domain Extensions. + Intel Trust Domain eXtensions(TDX) is an architecture based on CPU + instructions and TDX module to deploy hardware-isolated virtual + machines(VMs) which is called Trust Domains(TDs). + config KVM_AMD tristate "KVM for AMD processors support" depends on KVM
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 01/69] KVM: TDX: introduce config for KVM TDX support
Quoted 23 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add new config
KVM_INTEL_TDXfor KVM TDX support. This patch introduce a config only. Later patch will implement feature depends on it.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Kconfig | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/x86/kvm/Kconfig +4
-0@@ -96,6 +96,17 @@ config X86_SGX_KVM If unsure, say N. +config KVM_INTEL_TDX + bool "Trusted Domain eXtensions(TDX) Host support" + depends on KVM_INTEL && X86_64 + select FW_LOADER
arch/x86/kvm/Kconfig config X86_SGX_KVM · line 112
Quoted 10 lines
arch/x86/kvm/Kconfig +7
-0+ + help + Extends KVM on Intel processors to support Trusted Domain Extensions. + Intel Trust Domain eXtensions(TDX) is an architecture based on CPU + instructions and TDX module to deploy hardware-isolated virtual + machines(VMs) which is called Trust Domains(TDs). + config KVM_AMD tristate "KVM for AMD processors support" depends on KVM
Sean Christopherson
maintainerRe: [RFC PATCH v2 01/69] KVM: TDX: introduce config for KVM TDX support
In reply to [email protected]Quoted 15 lines · On Fri, Jul 02, 2021, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add new config
KVM_INTEL_TDXfor KVM TDX support. This patch introduce a config only. Later patch will implement feature depends on it.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Kconfig | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/x86/kvm/Kconfig +0
-0
arch/x86/kvm/Kconfig config X86_SGX_KVM · line 102
Quoted 7 lines
arch/x86/kvm/Kconfig +4
-0@@ -96,6 +96,17 @@ config X86_SGX_KVM If unsure, say N. +config KVM_INTEL_TDX + bool "Trusted Domain eXtensions(TDX) Host support" + depends on KVM_INTEL && X86_64 + select FW_LOADER
Similar to patch 2, it's impossible to review this as is since the FW_LOADER dependency is for the SEAMLDR side of things, which is not included in this series.
Quoted 13 lines
arch/x86/kvm/Kconfig +7
-0+ + help + Extends KVM on Intel processors to support Trusted Domain Extensions. + Intel Trust Domain eXtensions(TDX) is an architecture based on CPU + instructions and TDX module to deploy hardware-isolated virtual + machines(VMs) which is called Trust Domains(TDs). + config KVM_AMD tristate "KVM for AMD processors support" depends on KVMSignature · 1 line
2.25.1
RFC PATCH v2 2/69 KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This is preparatory clean up for TDX support.
Move out kvm_cpu_vmxon() from vmx.c to virtext.h with rename to vcpu_vmxon(). Which will be used outside of kvm.
SEAMLDER to load TDX module which occurs at kernel early boot phase. As bonus, this also increases the symetry with cpu_vmxoff().
No functional change is intended.
--- arch/x86/include/asm/virtext.h | 25 +++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 22 +--------------------- 2 files changed, 26 insertions(+), 21 deletions(-)
arch/x86/include/asm/virtext.h +25 -0
@@ -30,6 +30,31 @@ static inline int cpu_has_vmx(void) } +/** +* cpu_vmxon() - Enable VMX on the current CPU +* +* Set CR4.VMXE and enable VMX +*/ +static inline int cpu_vmxon (u64 vmxon_pointer) +{ +u64 msr; + +cr4_set_bits (X86_CR4_VMXE ); + + asm_volatile_goto("1: vmxon %[vmxon_pointer]\n\t" + _ASM_EXTABLE(1b, %l[fault]) + : : [vmxon_pointer] "m"(vmxon_pointer) + : : fault); +return 0 ; + +fault: +WARN_ONCE (1 ,"VMXON faulted, MSR_IA32_FEAT_CTL (0x3a) = 0x%llx\n" , +rdmsrl_safe (MSR_IA32_FEAT_CTL , &msr) ?0xdeadbeef : msr); +cr4_clear_bits (X86_CR4_VMXE ); + +return -EFAULT ; +} +/** * cpu_vmxoff() - Disable VMX on the current CPU *
arch/x86/kvm/vmx/vmx.c +1 -21
@@ -2376,26 +2376,6 @@ static __init int vmx_disabled_by_bios(void) !boot_cpu_has (X86_FEATURE_VMX ); } -static int kvm_cpu_vmxon (u64 vmxon_pointer) -{ -u64 msr; - -cr4_set_bits (X86_CR4_VMXE ); - - asm_volatile_goto("1: vmxon %[vmxon_pointer]\n\t" - _ASM_EXTABLE(1b, %l[fault]) - : : [vmxon_pointer] "m"(vmxon_pointer) - : : fault); -return 0 ; - -fault: -WARN_ONCE (1 ,"VMXON faulted, MSR_IA32_FEAT_CTL (0x3a) = 0x%llx\n" , -rdmsrl_safe (MSR_IA32_FEAT_CTL , &msr) ?0xdeadbeef : msr); -cr4_clear_bits (X86_CR4_VMXE ); - -return -EFAULT ; -} -static int hardware_enable (void ) {int cpu =raw_smp_processor_id ();
@@ -2415,7 +2395,7 @@ static int hardware_enable(void)intel_pt_handle_vmx (1 ); - r =kvm_cpu_vmxon (phys_addr); + r =cpu_vmxon (phys_addr);if (r) {intel_pt_handle_vmx (0 );return r;
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 02/69] KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
Quoted 85 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
This is preparatory clean up for TDX support.
Move outkvm_cpu_vmxon()fromvmx.ctovirtext.hwith rename tovcpu_vmxon(). Which will be used outside of kvm.
SEAMLDER to load TDX module which occurs at kernel early boot phase. As bonus, this also increases the symetry withcpu_vmxoff().No functional change is intended.
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/virtext.h | 25 +++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 22 +--------------------- 2 files changed, 26 insertions(+), 21 deletions(-)arch/x86/include/asm/virtext.h +25
-0@@ -30,6 +30,31 @@ static inline int cpu_has_vmx(void) } +/** +* cpu_vmxon() - Enable VMX on the current CPU +* +* Set CR4.VMXE and enable VMX +*/ +static inline int cpu_vmxon (u64 vmxon_pointer) +{ +u64 msr; + +cr4_set_bits (X86_CR4_VMXE ); + + asm_volatile_goto("1: vmxon %[vmxon_pointer]\n\t" + _ASM_EXTABLE(1b, %l[fault]) + : : [vmxon_pointer] "m"(vmxon_pointer) + : : fault); +return 0 ; + +fault: +WARN_ONCE (1 ,"VMXON faulted, MSR_IA32_FEAT_CTL (0x3a) = 0x%llx\n" , +rdmsrl_safe (MSR_IA32_FEAT_CTL , &msr) ?0xdeadbeef : msr); +cr4_clear_bits (X86_CR4_VMXE ); + +return -EFAULT ; +} +/** * cpu_vmxoff() - Disable VMX on the current CPU * arch/x86/kvm/vmx/vmx.c +0
-20@@ -2376,26 +2376,6 @@ static __init int vmx_disabled_by_bios(void) !boot_cpu_has (X86_FEATURE_VMX ); } -static int kvm_cpu_vmxon (u64 vmxon_pointer) -{ -u64 msr; - -cr4_set_bits (X86_CR4_VMXE ); - - asm_volatile_goto("1: vmxon %[vmxon_pointer]\n\t" - _ASM_EXTABLE(1b, %l[fault]) - : : [vmxon_pointer] "m"(vmxon_pointer) - : : fault); -return 0 ; - -fault: -WARN_ONCE (1 ,"VMXON faulted, MSR_IA32_FEAT_CTL (0x3a) = 0x%llx\n" , -rdmsrl_safe (MSR_IA32_FEAT_CTL , &msr) ?0xdeadbeef : msr); -cr4_clear_bits (X86_CR4_VMXE ); - -return -EFAULT ; -} -static int hardware_enable (void ) {int cpu =raw_smp_processor_id ();
arch/x86/kvm/vmx/vmx.c static int hardware_enable(void) · line 2401
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -2415,7 +2395,7 @@ static int hardware_enable(void)intel_pt_handle_vmx (1 ); - r =kvm_cpu_vmxon (phys_addr); + r =cpu_vmxon (phys_addr);if (r) {intel_pt_handle_vmx (0 );return r;
Sean Christopherson
maintainerRe: [RFC PATCH v2 02/69] KVM: X86: move kvm_cpu_vmxon() from vmx.c to virtext.h
In reply to [email protected]Quoted 6 lines · On Fri, Jul 02, 2021, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
This is preparatory clean up for TDX support.
Move outkvm_cpu_vmxon()fromvmx.ctovirtext.hwith rename tovcpu_vmxon(). Which will be used outside of kvm.
SEAMLDER to load TDX module which occurs at kernel early boot phase.
^^^^^^^^
SEAMLDR?
This patch can't be reviewed without seeing the TDX-Module load/init code, and that code has been moved to a different (not yet posted?) series. E.g. at the end of this series, KVM is still the only user.
Quoted 1 line
As bonus, this also increases the symetry with
cpu_vmxoff().
^^^^^^^
symmetryRFC PATCH v2 3/69 KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This is preparation for TDX support.
Because SEAMCALL instruction requires VMX enabled, it needs to initialize struct vmcs and load it before SEAMCALL instruction.[1] [2] Move out the definition of vmcs into a common x86 header, arch/x86/include/asm/vmx.h, so that seamloader code can share the same definition.
[1] Intel Trust Domain CPU Architectural Extensions
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-cpu-architectural-specification.pdf
[2] TDX Module spec
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
--- arch/x86/include/asm/vmx.h | 11 +++++++++++ arch/x86/kvm/vmx/vmcs.h | 11 ----------- 2 files changed, 11 insertions(+), 11 deletions(-)
arch/x86/include/asm/vmx.h +11 -0
@@ -17,6 +17,17 @@#include <uapi/asm/vmx.h> #include <asm/vmxfeatures.h> +struct vmcs_hdr { +u32 revision_id:31 ; +u32 shadow_vmcs:1 ; +}; + +struct vmcs { +struct vmcs_hdr hdr; +u32 abort; +char data[]; +}; +#define VMCS_CONTROL_BIT (x) BIT(VMX_FEATURE_##x & 0x1f)/*
arch/x86/kvm/vmx/vmcs.h +0 -11
@@ -11,17 +11,6 @@#include "capabilities.h" -struct vmcs_hdr { -u32 revision_id:31 ; -u32 shadow_vmcs:1 ; -}; - -struct vmcs { -struct vmcs_hdr hdr; -u32 abort; -char data[]; -}; - DECLARE_PER_CPU(struct vmcs *, current_vmcs);/*
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
Quoted 56 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
This is preparation for TDX support.
Because SEAMCALL instruction requires VMX enabled, it needs to initialize struct vmcs and load it before SEAMCALL instruction.[1] [2] Move out the definition of vmcs into a common x86 header,
arch/x86/include/asm/vmx.h, so that seamloader code can share the same definition.[1] Intel Trust Domain CPU Architectural Extensions
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-cpu-architectural-specification.pdf[2] TDX Module spec
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdfSigned-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/vmx.h | 11 +++++++++++ arch/x86/kvm/vmx/vmcs.h | 11 ----------- 2 files changed, 11 insertions(+), 11 deletions(-)arch/x86/include/asm/vmx.h +11
-0@@ -17,6 +17,17 @@#include <uapi/asm/vmx.h> #include <asm/vmxfeatures.h> +struct vmcs_hdr { +u32 revision_id:31 ; +u32 shadow_vmcs:1 ; +}; + +struct vmcs { +struct vmcs_hdr hdr; +u32 abort; +char data[]; +}; +#define VMCS_CONTROL_BIT (x) BIT(VMX_FEATURE_##x & 0x1f)/* arch/x86/kvm/vmx/vmcs.h +0
-4@@ -11,17 +11,6 @@#include "capabilities.h" -struct vmcs_hdr { -u32 revision_id:31 ; -u32 shadow_vmcs:1 ; -};
arch/x86/kvm/vmx/vmcs.h line 16
Quoted 10 lines
arch/x86/kvm/vmx/vmcs.h +0
-7- -struct vmcs { -struct vmcs_hdr hdr; -u32 abort; -char data[]; -}; - DECLARE_PER_CPU(struct vmcs *, current_vmcs);/*
Sean Christopherson
maintainerRe: [RFC PATCH v2 03/69] KVM: X86: move out the definition vmcs_hdr/vmcs from kvm to x86
In reply to [email protected]Quoted 8 lines · On Fri, Jul 02, 2021, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
This is preparation for TDX support.
Because SEAMCALL instruction requires VMX enabled, it needs to initialize struct vmcs and load it before SEAMCALL instruction.[1] [2] Move out the definition of vmcs into a common x86 header,
arch/x86/include/asm/vmx.h, so that seamloader code can share the same definition.
^^^^^^^^^^
SEAMLDR?I don't have a strong preference on what we call it, but we should be consistent in our usage.
Same comments as the first two patches, without seeing the actual SEAMLDR code it's impossible review this patch. I certainly have no objection to splitting up this behemoth, but the series should be self contained (within reason).
RFC PATCH v2 4/69 KVM: TDX: Add TDX "architectural" error codes
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add TDX completion status codes for SEAMCALL and TDG_VP_VMCALL.
TDX-SEAM uses bits 31:0 to return more information, so these error codes for SEAMCALL will only exactly match RAX[63:32]. [1] Section 17.1 Interface Function Completion Status Codes
Completion status codes for TDG.VP.VMCALL is defined in
[2] Chapter 3 TDG.VP.VMCALL Interface.
[1] Intel TDX Module Spec
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
[2] TDX Guest-Host Communication interface for Intel Trust Domain Extensions
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
--- arch/x86/kvm/vmx/tdx_errno.h | 106 +++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
arch/x86/kvm/vmx/tdx_errno.h new file +106 -0
@@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_SUCCESS 0x0000000000000000 +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000 +#define TDX_NON_RECOVERABLE_TD 0x4000000200000000 +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000 +#define TDX_INTERRUPTED_RESTARTABLE 0x8000000400000000 +#define TDX_NON_RECOVERABLE_TD_FATAL 0x4000000500000000 +#define TDX_INVALID_RESUMPTION 0xC000000600000000 +#define TDX_NON_RECOVERABLE_TD_NO_APIC 0xC000000700000000 +#define TDX_OPERAND_INVALID 0xC000010000000000 +#define TDX_OPERAND_ADDR_RANGE_ERROR 0xC000010100000000 +#define TDX_OPERAND_BUSY 0x8000020000000000 +#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000 +#define TDX_SYS_BUSY 0x8000020200000000 +#define TDX_PAGE_METADATA_INCORRECT 0xC000030000000000 +#define TDX_PAGE_ALREADY_FREE 0x0000030100000000 +#define TDX_PAGE_NOT_OWNED_BY_TD 0xC000030200000000 +#define TDX_PAGE_NOT_FREE 0xC000030300000000 +#define TDX_TD_ASSOCIATED_PAGES_EXIST 0xC000040000000000 +#define TDX_SYSINIT_NOT_PENDING 0xC000050000000000 +#define TDX_SYSINIT_NOT_DONE 0xC000050100000000 +#define TDX_SYSINITLP_NOT_DONE 0xC000050200000000 +#define TDX_SYSINITLP_DONE 0xC000050300000000 +#define TDX_SYS_NOT_READY 0xC000050500000000 +#define TDX_SYS_SHUTDOWN 0xC000050600000000 +#define TDX_SYSCONFIG_NOT_DONE 0xC000050700000000 +#define TDX_TD_NOT_INITIALIZED 0xC000060000000000 +#define TDX_TD_INITIALIZED 0xC000060100000000 +#define TDX_TD_NOT_FINALIZED 0xC000060200000000 +#define TDX_TD_FINALIZED 0xC000060300000000 +#define TDX_TD_FATAL 0xC000060400000000 +#define TDX_TD_NON_DEBUG 0xC000060500000000 +#define TDX_TDCX_NUM_INCORRECT 0xC000061000000000 +#define TDX_VCPU_STATE_INCORRECT 0xC000070000000000 +#define TDX_VCPU_ASSOCIATED 0x8000070100000000 +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000 +#define TDX_TDVPX_NUM_INCORRECT 0xC000070300000000 +#define TDX_NO_VALID_VE_INFO 0xC000070400000000 +#define TDX_MAX_VCPUS_EXCEEDED 0xC000070500000000 +#define TDX_TSC_ROLLBACK 0xC000070600000000 +#define TDX_FIELD_NOT_WRITABLE 0xC000072000000000 +#define TDX_FIELD_NOT_READABLE 0xC000072100000000 +#define TDX_TD_VMCS_FIELD_NOT_INITIALIZED 0xC000073000000000 +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000 +#define TDX_TD_KEYS_NOT_CONFIGURED 0x8000081000000000 +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000 +#define TDX_KEY_CONFIGURED 0x0000081500000000 +#define TDX_WBCACHE_NOT_COMPLETE 0x8000081700000000 +#define TDX_HKID_NOT_FREE 0xC000082000000000 +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000 +#define TDX_WBCACHE_RESUME_ERROR 0xC000082300000000 +#define TDX_FLUSHVP_NOT_DONE 0x8000082400000000 +#define TDX_NUM_ACTIVATED_HKIDS_NOT_SUPPORTED 0xC000082500000000 +#define TDX_INCORRECT_CPUID_VALUE 0xC000090000000000 +#define TDX_BOOT_NT4_SET 0xC000090100000000 +#define TDX_INCONSISTENT_CPUID_FIELD 0xC000090200000000 +#define TDX_CPUID_LEAF_1F_FORMAT_UNRECOGNIZED 0xC000090400000000 +#define TDX_INVALID_WBINVD_SCOPE 0xC000090500000000 +#define TDX_INVALID_PKG_ID 0xC000090600000000 +#define TDX_CPUID_LEAF_NOT_SUPPORTED 0xC000090800000000 +#define TDX_SMRR_NOT_LOCKED 0xC000091000000000 +#define TDX_INVALID_SMRR_CONFIGURATION 0xC000091100000000 +#define TDX_SMRR_OVERLAPS_CMR 0xC000091200000000 +#define TDX_SMRR_LOCK_NOT_SUPPORTED 0xC000091300000000 +#define TDX_SMRR_NOT_SUPPORTED 0xC000091400000000 +#define TDX_INCONSISTENT_MSR 0xC000092000000000 +#define TDX_INCORRECT_MSR_VALUE 0xC000092100000000 +#define TDX_SEAMREPORT_NOT_AVAILABLE 0xC000093000000000 +#define TDX_PERF_COUNTERS_ARE_PEBS_ENABLED 0x8000094000000000 +#define TDX_INVALID_TDMR 0xC0000A0000000000 +#define TDX_NON_ORDERED_TDMR 0xC0000A0100000000 +#define TDX_TDMR_OUTSIDE_CMRS 0xC0000A0200000000 +#define TDX_TDMR_ALREADY_INITIALIZED 0x00000A0300000000 +#define TDX_INVALID_PAMT 0xC0000A1000000000 +#define TDX_PAMT_OUTSIDE_CMRS 0xC0000A1100000000 +#define TDX_PAMT_OVERLAP 0xC0000A1200000000 +#define TDX_INVALID_RESERVED_IN_TDMR 0xC0000A2000000000 +#define TDX_NON_ORDERED_RESERVED_IN_TDMR 0xC0000A2100000000 +#define TDX_CMR_LIST_INVALID 0xC0000A2200000000 +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000 +#define TDX_EPT_ENTRY_FREE 0xC0000B0100000000 +#define TDX_EPT_ENTRY_NOT_FREE 0xC0000B0200000000 +#define TDX_EPT_ENTRY_NOT_PRESENT 0xC0000B0300000000 +#define TDX_EPT_ENTRY_NOT_LEAF 0xC0000B0400000000 +#define TDX_EPT_ENTRY_LEAF 0xC0000B0500000000 +#define TDX_GPA_RANGE_NOT_BLOCKED 0xC0000B0600000000 +#define TDX_GPA_RANGE_ALREADY_BLOCKED 0x00000B0700000000 +#define TDX_TLB_TRACKING_NOT_DONE 0xC0000B0800000000 +#define TDX_EPT_INVALID_PROMOTE_CONDITIONS 0xC0000B0900000000 +#define TDX_PAGE_ALREADY_ACCEPTED 0x00000B0A00000000 +#define TDX_PAGE_SIZE_MISMATCH 0xC0000B0B00000000 + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000 +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000 +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001 + +#endif /* __KVM_X86_TDX_ERRNO_H */
Signature · 1 line
2.25.1
RFC PATCH v2 5/69 KVM: TDX: Add architectural definitions for structures and values
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add structures and values that are architecturally defined in
[1] chapter 18 ABI Reference: Data Types and in [1] 20.2.1 SEAMCALL Instruction(Common) Table 20.4 SEAMCALL Instruction Leaf Numbers Definition.
[1] TDX Module Spec
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
--- arch/x86/kvm/vmx/tdx_arch.h | 307 ++++++++++++++++++++++++++++++++++++ 1 file changed, 307 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
arch/x86/kvm/vmx/tdx_arch.h new file +307 -0
@@ -0,0 +1,307 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define SEAMCALL_TDH_VP_ENTER 0 +#define SEAMCALL_TDH_MNG_ADDCX 1 +#define SEAMCALL_TDH_MEM_PAGE_ADD 2 +#define SEAMCALL_TDH_MEM_SEPT_ADD 3 +#define SEAMCALL_TDH_VP_ADDCX 4 +#define SEAMCALL_TDH_MEM_PAGE_AUG 6 +#define SEAMCALL_TDH_MEM_RANGE_BLOCK 7 +#define SEAMCALL_TDH_MNG_KEY_CONFIG 8 +#define SEAMCALL_TDH_MNG_CREATE 9 +#define SEAMCALL_TDH_VP_CREATE 10 +#define SEAMCALL_TDH_MNG_RD 11 +#define SEAMCALL_TDH_PHYMEM_PAGE_RD 12 +#define SEAMCALL_TDH_MNG_WR 13 +#define SEAMCALL_TDH_PHYMEM_PAGE_WR 14 +#define SEAMCALL_TDH_MEM_PAGE_DEMOTE 15 +#define SEAMCALL_TDH_MR_EXTEND 16 +#define SEAMCALL_TDH_MR_FINALIZE 17 +#define SEAMCALL_TDH_VP_FLUSH 18 +#define SEAMCALL_TDH_MNG_VPFLUSHDONE 19 +#define SEAMCALL_TDH_MNG_KEY_FREEID 20 +#define SEAMCALL_TDH_MNG_INIT 21 +#define SEAMCALL_TDH_VP_INIT 22 +#define SEAMCALL_TDH_MEM_PAGE_PROMOTE 23 +#define SEAMCALL_TDH_PHYMEM_PAGE_RDMD 24 +#define SEAMCALL_TDH_MEM_SEPT_RD 25 +#define SEAMCALL_TDH_VP_RD 26 +#define SEAMCALL_TDH_MNG_KEY_RECLAIMID 27 +#define SEAMCALL_TDH_PHYMEM_PAGE_RECLAIM 28 +#define SEAMCALL_TDH_MEM_PAGE_REMOVE 29 +#define SEAMCALL_TDH_MEM_SEPT_REMOVE 30 +#define SEAMCALL_TDH_SYS_KEY_CONFIG 31 +#define SEAMCALL_TDH_SYS_INFO 32 +#define SEAMCALL_TDH_SYS_INIT 33 +#define SEAMCALL_TDH_SYS_LP_INIT 35 +#define SEAMCALL_TDH_SYS_TDMR_INIT 36 +#define SEAMCALL_TDH_MEM_TRACK 38 +#define SEAMCALL_TDH_MEM_RANGE_UNBLOCK 39 +#define SEAMCALL_TDH_PHYMEM_CACHE_WB 40 +#define SEAMCALL_TDH_PHYMEM_PAGE_WBINVD 41 +#define SEAMCALL_TDH_MEM_SEPT_WR 42 +#define SEAMCALL_TDH_VP_WR 43 +#define SEAMCALL_TDH_SYS_LP_SHUTDOWN 44 +#define SEAMCALL_TDH_SYS_CONFIG 45 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define BUILD_TDX_FIELD (class, field) \ + (((u64)(class) << TDX_CLASS_SHIFT) | ((u64)(field) & TDX_FIELD_MASK)) + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS (field) BUILD_TDX_FIELD(0, (field)) + +/* +* @offset is the offset (in bytes) from the beginning of the architectural +* virtual APIC page. +*/ +#define TDVPS_APIC (offset) BUILD_TDX_FIELD(1, (offset)) + +/* @gpr is the index of a general purpose register, e.g. eax=0 */ +#define TDVPS_GPR (gpr) BUILD_TDX_FIELD(16, (gpr)) + +#define TDVPS_DR (dr) BUILD_TDX_FIELD(17, (0 + (dr))) + +enum tdx_guest_other_state { +TD_VCPU_XCR0 =32 , +TD_VCPU_IWK_ENCKEY0 =64 , +TD_VCPU_IWK_ENCKEY1 , +TD_VCPU_IWK_ENCKEY2 , +TD_VCPU_IWK_ENCKEY3 , +TD_VCPU_IWK_INTKEY0 =68 , +TD_VCPU_IWK_INTKEY1 , +TD_VCPU_IWK_FLAGS =70 , +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE (field) BUILD_TDX_FIELD(17, (field)) + +/* @msr is the MSR index */ +#define TDVPS_MSR (msr) BUILD_TDX_FIELD(19, (msr)) + +/* Management class fields */ +enum tdx_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(32, (field)) + +#define TDX1_NR_TDCX_PAGES 4 +#define TDX1_NR_TDVPX_PAGES 5 + +#define TDX1_MAX_NR_CPUID_CONFIGS 6 +#define TDX1_MAX_NR_CMRS 32 +#define TDX1_MAX_NR_TDMRS 64 +#define TDX1_MAX_NR_RSVD_AREAS 16 +#define TDX1_PAMT_ENTRY_SIZE 16 +#define TDX1_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_config { +u32 leaf; +u32 sub_leaf; +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +struct tdx_cpuid_value { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +#define TDX1_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX1_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX1_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX1_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +/* +* TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. +*/ +struct td_params { +u64 attributes; +u64 xfam; +u32 max_vcpus; +u32 reserved0; + +u64 eptp_controls; +u64 exec_controls; +u16 tsc_frequency; +u8 reserved1[38 ]; + +u64 mrconfigid[6 ]; +u64 mrowner[6 ]; +u64 mrownerconfig[6 ]; +u64 reserved2[4 ]; + +union { +struct tdx_cpuid_value cpuid_values[0 ]; +u8 reserved3[768 ]; + }; +} __packed__aligned(1024) ; + +/* Guest uses MAX_PA for GPAW when set. */ +#define TDX1_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* +* TDX1 requires the frequency to be defined in units of 25MHz, which is the +* frequency of the core crystal clock on TDX-capable platforms, i.e. TDX-SEAM +* can only program frequencies that are multiples of 25MHz. The frequency +* must be between 1ghz and 10ghz (inclusive). +*/ +#define TDX1_TSC_KHZ_TO_25MHZ (tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX1_TSC_25MHZ_TO_KHZ (tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX1_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX1_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +struct tdmr_reserved_area { +u64 offset; +u64 size; +} __packed; + +#define TDX_TDMR_ADDR_ALIGNMENT 512 +#define TDX_TDMR_INFO_ALIGNMENT 512 +struct tdmr_info { +u64 base; +u64 size; +u64 pamt_1g_base; +u64 pamt_1g_size; +u64 pamt_2m_base; +u64 pamt_2m_size; +u64 pamt_4k_base; +u64 pamt_4k_size; +struct tdmr_reserved_area reserved_areas[TDX1_MAX_NR_RSVD_AREAS ]; +} __packed__aligned(TDX_TDMR_INFO_ALIGNMENT) ; + +#define TDX_CMR_INFO_ARRAY_ALIGNMENT 512 +struct cmr_info { +u64 base; +u64 size; +} __packed; + +#define TDX_TDSYSINFO_STRUCT_ALIGNEMNT 1024 +struct tdsysinfo_struct { +/* TDX-SEAM Module Info */ +u32 attributes; +u32 vendor_id; +u32 build_date; +u16 build_num; +u16 minor_version; +u16 major_version; +u8 reserved0[14 ]; +/* Memory Info */ +u16 max_tdmrs; +u16 max_reserved_per_tdmr; +u16 pamt_entry_size; +u8 reserved1[10 ]; +/* Control Struct Info */ +u16 tdcs_base_size; +u8 reserved2[2 ]; +u16 tdvps_base_size; +u8 tdvps_xfam_dependent_size; +u8 reserved3[9 ]; +/* TD Capabilities */ +u64 attributes_fixed0; +u64 attributes_fixed1; +u64 xfam_fixed0; +u64 xfam_fixed1; +u8 reserved4[32 ]; +u32 num_cpuid_config; +union { +struct tdx_cpuid_config cpuid_configs[0 ]; +u8 reserved5[892 ]; + }; +} __packed__aligned(TDX_TDSYSINFO_STRUCT_ALIGNEMNT) ; + +struct tdx_ex_ret { +union { +/* Used to retrieve values from hardware. */ +struct { +u64 rcx; +u64 rdx; +u64 r8; +u64 r9; +u64 r10; +u64 r11; + }; +/* Functions that walk SEPT */ +struct { +u64 septe; +struct { +u64 level :3 ; +u64 sept_reserved_0 :5 ; +u64 state :8 ; +u64 sept_reserved_1 :48 ; + }; + }; +/* TD_MNG_{RD,WR} return the TDR, field code, and value. */ +struct { +u64 tdr; +u64 field; +u64 field_val; + }; +/* TD_MNG_{RD,WR}MEM return the address and its value. */ +struct { +u64 addr; +u64 val; + }; +/* TDH_PHYMEM_PAGE_RDMD and TDH_PHYMEM_PAGE_RECLAIM return page metadata. */ +struct { +u64 page_type; +u64 owner; +u64 page_size; + }; +/* +* TDH_SYS_INFO returns the buffer address and its size, and the +* CMR_INFO address and its number of entries. +*/ +struct { +u64 buffer; +u64 nr_bytes; +u64 cmr_info; +u64 nr_cmr_entries; + }; +/* +* TDH_MNG_INIT and TDH_SYS_INIT return CPUID info on error. Note, only +* the leaf and subleaf are valid on TDH_MNG_INIT error. +*/ +struct { +u32 leaf; +u32 subleaf; +u32 eax_mask; +u32 ebx_mask; +u32 ecx_mask; +u32 edx_mask; +u32 eax_val; +u32 ebx_val; +u32 ecx_val; +u32 edx_val; + }; +/* TDH_SYS_TDMR_INIT returns the input PA and next PA. */ +struct { +u64 prev; +u64 next; + }; + }; +}; + +#endif /* __KVM_X86_TDX_ARCH_H */
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 05/69] KVM: TDX: Add architectural definitions for structures and values
arch/x86/kvm/vmx/tdx_arch.h line 104
Quoted 7 lines · On Fri, Jul 2, 2021 at 3:05 PM <[email protected]> wrote:
arch/x86/kvm/vmx/tdx_arch.h new file +7
-0+/* Management class fields */ +enum tdx_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(32, (field))
I am a little confused with this. According to the spec, PEND_NMI has a field code of 0x200000000000000B
I can understand that 0x20 is the class code and the PEND_NMI field code is 0xB. On the other hand, for the LAST_EXIT_TSC the field code is 0xA00000000000000A. Based on your code and the table in the spec, I can see that there is an additional mask (1ULL<<63) for readonly fields.
Is this information correct and is this included in the spec? I tried to find it but somehow I do not see it clearly defined.
arch/x86/kvm/vmx/tdx_arch.h line 109
Quoted 4 lines
arch/x86/kvm/vmx/tdx_arch.h new file +4
-0+#define TDX1_NR_TDCX_PAGES 4 +#define TDX1_NR_TDVPX_PAGES 5 + +#define TDX1_MAX_NR_CPUID_CONFIGS 6
Why is this just 6? I am looking at the CPUID table in the spec and there are already more than 6 CPUID leaves there.
arch/x86/kvm/vmx/tdx_arch.h line 114
Quoted 5 lines
arch/x86/kvm/vmx/tdx_arch.h new file +5
-0+#define TDX1_MAX_NR_CMRS 32 +#define TDX1_MAX_NR_TDMRS 64 +#define TDX1_MAX_NR_RSVD_AREAS 16 +#define TDX1_PAMT_ENTRY_SIZE 16 +#define TDX1_EXTENDMR_CHUNKSIZE 256
I believe all of the defined variables above need to be enumerated with TDH.SYS.INFO.
Quoted 1 line
+#define
TDX_TDMR_ADDR_ALIGNMENT512
Is TDX_TDMR_ADDR_ALIGNMENT used anywhere or is it just for completeness?
Quoted 1 line
+#define
TDX_TDMR_INFO_ALIGNMENT512
Why do we have alignment of 512, I am assuming to make it cache line size aligned for efficiency?
Quoted 1 line
+#define
TDX_TDSYSINFO_STRUCT_ALIGNEMNT1024
typo: ALIGNEMNT -> ALIGNMENT
-Erdem
Re: [RFC PATCH v2 05/69] KVM: TDX: Add architectural definitions for structures and values
Quoted 14 lines · Erdem Aktas
arch/x86/kvm/vmx/tdx_arch.h line 104
Quoted 7 lines · On Fri, Jul 2, 2021 at 3:05 PM <[email protected]> wrote:
arch/x86/kvm/vmx/tdx_arch.h new file +7
-0+/* Management class fields */ +enum tdx_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(32, (field))I am a little confused with this. According to the spec,
PEND_NMIhas a field code of0x200000000000000B
I can understand that0x20is the class code and thePEND_NMIfield code is0xB. On the other hand, for theLAST_EXIT_TSCthe field code is0xA00000000000000A.
Quoted 2 lines
Based on your code and the table in the spec, I can see that there is an additional mask (1ULL<<63) for readonly fields
No. bit 63 is not for readonly fields, but for non_arch fields.
Please see 18.7.1 General definition
Quoted 9 lines
Is this information correct and is this included in the spec? I tried to find it but somehow I do not see it clearly defined.
arch/x86/kvm/vmx/tdx_arch.h line 109
Quoted 4 lines
arch/x86/kvm/vmx/tdx_arch.h new file +4
-0+#define TDX1_NR_TDCX_PAGES 4 +#define TDX1_NR_TDVPX_PAGES 5 + +#define TDX1_MAX_NR_CPUID_CONFIGS 6Why is this just 6? I am looking at the CPUID table in the spec and there are already more than 6 CPUID leaves there.
This is the number of CPUID config reported by TDH.SYS.INFO. Current KVM only reports 6 leaves.
Quoted 8 lines
arch/x86/kvm/vmx/tdx_arch.h line 114
Quoted 5 lines
arch/x86/kvm/vmx/tdx_arch.h new file +5
-0+#define TDX1_MAX_NR_CMRS 32 +#define TDX1_MAX_NR_TDMRS 64 +#define TDX1_MAX_NR_RSVD_AREAS 16 +#define TDX1_PAMT_ENTRY_SIZE 16 +#define TDX1_EXTENDMR_CHUNKSIZE 256I believe all of the defined variables above need to be enumerated with TDH.SYS.INFO.
No. Only TDX1_MAX_NR_TDMRS, TDX1_MAX_NR_RSVD_AREAS and TDX1_PAMT_ENTRY_SIZE can be enumerated from TDH.SYS.INFO.
- TDX1_MAX_NR_CMRS is described in 18.6.3 CMR_INFO, which tells
TDH.SYS.INFO leaf function returns a MAX_CMRS(32) entry array of CMR_INFO entries.
- TDX1_EXTENDMR_CHUNKSIZE is describe in 20.2.23 TDH.MR.EXTEND
Quoted 2 lines
Quoted 1 line
+#define
TDX_TDMR_ADDR_ALIGNMENT512Is
TDX_TDMR_ADDR_ALIGNMENTused anywhere or is it just for completeness?
It's the leftover during rebase. We will clean it up.
Quoted 3 lines
Quoted 1 line
+#define
TDX_TDMR_INFO_ALIGNMENT512Why do we have alignment of 512, I am assuming to make it cache line size aligned for efficiency?
It should be leftover too.
SEAMCALL TDH.SYS.INFO requires each cmr info in CMR_INFO_ARRAY to be 512B aligned
Quoted 7 lines
Quoted 2 lines
+#define
TDX_TDSYSINFO_STRUCT_ALIGNEMNT1024typo: ALIGNEMNT -> ALIGNMENT
-Erdem
Re: [RFC PATCH v2 05/69] KVM: TDX: Add architectural definitions for structures and values
Quoted 4 lines · Xiaoyao Li
No. bit 63 is not for readonly fields, but for
non_archfields.Please see 18.7.1 General definition
Thank you so much! make sense.
Quoted 10 lines
Quoted 9 lines
Is this information correct and is this included in the spec? I tried to find it but somehow I do not see it clearly defined.
arch/x86/kvm/vmx/tdx_arch.h line 109
Quoted 4 lines
arch/x86/kvm/vmx/tdx_arch.h new file +4
-0+#define TDX1_NR_TDCX_PAGES 4 +#define TDX1_NR_TDVPX_PAGES 5 + +#define TDX1_MAX_NR_CPUID_CONFIGS 6Why is this just 6? I am looking at the CPUID table in the spec and there are already more than 6 CPUID leaves there.
Quoted 2 lines
This is the number of CPUID config reported by TDH.SYS.INFO. Current KVM only reports 6 leaves.
I, personally, still think that it should be enumerated, rather than hardcoded. It is not clear to me why it is 6 and nothing in the spec says it will not change.
Quoted 12 lines
Quoted 8 lines
arch/x86/kvm/vmx/tdx_arch.h line 114
Quoted 5 lines
arch/x86/kvm/vmx/tdx_arch.h new file +5
-0+#define TDX1_MAX_NR_CMRS 32 +#define TDX1_MAX_NR_TDMRS 64 +#define TDX1_MAX_NR_RSVD_AREAS 16 +#define TDX1_PAMT_ENTRY_SIZE 16 +#define TDX1_EXTENDMR_CHUNKSIZE 256I believe all of the defined variables above need to be enumerated with TDH.SYS.INFO.
No. Only
TDX1_MAX_NR_TDMRS,TDX1_MAX_NR_RSVD_AREASandTDX1_PAMT_ENTRY_SIZEcan be enumerated from TDH.SYS.INFO.
Quoted 6 lines
-
TDX1_MAX_NR_CMRSis described in 18.6.3CMR_INFO, which tellsTDH.SYS.INFO leaf function returns a
MAX_CMRS(32)entry array ofCMR_INFOentries.-
TDX1_EXTENDMR_CHUNKSIZEis describe in 20.2.23 TDH.MR.EXTEND
Thanks for the pointers for MAX_CMRS and TDX1_EXTENDMR_CHUNKSIZE. Will the rest of it be enumerated or hardcoded?
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
+#define
TDX_TDMR_ADDR_ALIGNMENT512Is
TDX_TDMR_ADDR_ALIGNMENTused anywhere or is it just for completeness?It's the leftover during rebase. We will clean it up.
Thanks!
Quoted 2 lines
SEAMCALL TDH.SYS.INFO requires each cmr info in
CMR_INFO_ARRAYto be 512B aligned
Make sense, Thanks for the explanation.
Sean Christopherson
maintainerRe: [RFC PATCH v2 05/69] KVM: TDX: Add architectural definitions for structures and values
Quoted 11 lines · Erdem Aktas
Quoted 10 lines · Xiaoyao Li
Quoted 9 lines
Is this information correct and is this included in the spec? I tried to find it but somehow I do not see it clearly defined.
arch/x86/kvm/vmx/tdx_arch.h line 109
Quoted 4 lines
arch/x86/kvm/vmx/tdx_arch.h new file +4
-0+#define TDX1_NR_TDCX_PAGES 4 +#define TDX1_NR_TDVPX_PAGES 5 + +#define TDX1_MAX_NR_CPUID_CONFIGS 6Why is this just 6? I am looking at the CPUID table in the spec and there are already more than 6 CPUID leaves there.
Quoted 6 lines
Quoted 2 lines
This is the number of CPUID config reported by TDH.SYS.INFO. Current KVM only reports 6 leaves.
I, personally, still think that it should be enumerated, rather than hardcoded. It is not clear to me why it is 6 and nothing in the spec says it will not change.
It's both hardcoded and enumerated. KVM's hardcoded value is specifically the maximum value expected for TDX-Module modules supporting the so called "1.0" spec. It's certainly possible a spec change could bump the maximum, but KVM will refuse to use a module with higher maximums until Linux/KVM is updated to play nice with the new module.
Having hardcoded maximum allows for simpler and more efficient code, as loops and arrays can be statically defined instead of having to pass around the enumerated values.
And we'd want sanity checking anyways, e.g. if the TDX-module pulled a stupid and reported that it needs 4000 TDCX pages. This approach gives KVM documented values to sanity check, e.g. instead of arbitrary magic numbers.
The downside of this approach is that KVM will need to be updated to play nice with a new module if any of these maximums are raised. But, IMO that's acceptable because I can't imagine a scenario where anyone would want to load a TDX module without first testing the daylights out of the specific kernel+TDX combination, especially a TDX-module that by definition includes new features.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/vmx/tdx_arch.h line 114
Quoted 5 lines
arch/x86/kvm/vmx/tdx_arch.h new file +5
-0+#define TDX1_MAX_NR_CMRS 32 +#define TDX1_MAX_NR_TDMRS 64 +#define TDX1_MAX_NR_RSVD_AREAS 16 +#define TDX1_PAMT_ENTRY_SIZE 16 +#define TDX1_EXTENDMR_CHUNKSIZE 256I believe all of the defined variables above need to be enumerated with TDH.SYS.INFO.
And they are, though I believe the code for doing the actual SEAMCALL wasn't posted in this series. The output is sanity checked by tdx_hardware_enable():
+ tdx_caps.tdcs_nr_pages = tdsysinfo->tdcs_base_size / PAGE_SIZE;
+ if (tdx_caps.tdcs_nr_pages != TDX1_NR_TDCX_PAGES)
+ return -EIO;
+
+ tdx_caps.tdvpx_nr_pages = tdsysinfo->tdvps_base_size / PAGE_SIZE - 1;
+ if (tdx_caps.tdvpx_nr_pages != TDX1_NR_TDVPX_PAGES)
+ return -EIO;
+
+ tdx_caps.attrs_fixed0 = tdsysinfo->attributes_fixed0;
+ tdx_caps.attrs_fixed1 = tdsysinfo->attributes_fixed1;
+ tdx_caps.xfam_fixed0 = tdsysinfo->xfam_fixed0;
+ tdx_caps.xfam_fixed1 = tdsysinfo->xfam_fixed1;
+
+ tdx_caps.nr_cpuid_configs = tdsysinfo->num_cpuid_config;
+ if (tdx_caps.nr_cpuid_configs > TDX1_MAX_NR_CPUID_CONFIGS)
+ return -EIO;
+
RFC PATCH v2 6/69 KVM: TDX: add a helper function for kvm to call seamcall
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a helper function for kvm to call seamcall and a helper macro to check its return value. The later patches will use them.
--- arch/x86/kernel/asm-offsets_64.c | 15 ++++++++ arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/seamcall.S | 64 ++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/seamcall.h | 47 +++++++++++++++++++++++ 4 files changed, 127 insertions(+) create mode 100644 arch/x86/kvm/vmx/seamcall.S create mode 100644 arch/x86/kvm/vmx/seamcall.h
arch/x86/kernel/asm-offsets_64.c +15 -0
@@ -9,6 +9,11 @@#include <asm/kvm_para.h> #endif +#ifdef CONFIG_KVM_INTEL_TDX +#include <linux/kvm_types.h> +#include "../kvm/vmx/tdx_arch.h" +#endif +int main (void ) {#ifdef CONFIG_PARAVIRT
@@ -25,6 +30,16 @@ int main(void)BLANK ();#endif +#ifdef CONFIG_KVM_INTEL_TDX +OFFSET (TDX_SEAM_rcx, tdx_ex_ret, rcx); +OFFSET (TDX_SEAM_rdx, tdx_ex_ret, rdx); +OFFSET (TDX_SEAM_r8, tdx_ex_ret, r8); +OFFSET (TDX_SEAM_r9, tdx_ex_ret, r9); +OFFSET (TDX_SEAM_r10, tdx_ex_ret, r10); +OFFSET (TDX_SEAM_r11, tdx_ex_ret, r11); +BLANK (); +#endif +#define ENTRY (entry) OFFSET(pt_regs_ ## entry, pt_regs, entry)ENTRY (bx);ENTRY (cx);
arch/x86/kvm/Makefile +1 -0
@@ -24,6 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o +kvm-intel-$(CONFIG_KVM_INTEL_TDX) += vmx/seamcall.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/seamcall.S new file +64 -0
64 lines not on this page · raw message
arch/x86/kvm/vmx/seamcall.h new file +47 -0
47 lines not on this page · raw message
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 06/69] KVM: TDX: add a helper function for kvm to call seamcall
Quoted 15 lines · On 03/07/21 00:04, [email protected] wrote:
arch/x86/kvm/vmx/seamcall.S new file +15
-0+ +.Lseamcall: + seamcall + jmp .Lseamcall_ret +.Lspurious_fault: + call kvm_spurious_fault +.Lseamcall_ret: + + movq (FRAME_OFFSET + 8)(%rsp), %rdi + testq %rdi, %rdi + jz 1f + + /* If ex is non-NULL, store extra return values into it. */ + movq %rcx, TDX_SEAM_rcx(%rdi) + movq %rdx, TDX_SEAM_rdx(%rdi)
arch/x86/kvm/vmx/seamcall.S line 63
Quoted 10 lines
arch/x86/kvm/vmx/seamcall.S new file +10
-0+ movq %r8, TDX_SEAM_r8(%rdi) + movq %r9, TDX_SEAM_r9(%rdi) + movq %r10, TDX_SEAM_r10(%rdi) + movq %r11, TDX_SEAM_r11(%rdi) + +1: + FRAME_END + ret + + _ASM_EXTABLE(.Lseamcall, .Lspurious_fault)
Please use local labels and avoid unnecessary jmp, for example
1:
seamcall movq (FRAME_OFFSET + 8)(%rsp), %rdi testq %rdi, %rdi jz 2f /* If ex is non-NULL, store extra return values into it. */ movq %rcx, TDX_SEAM_rcx(%rdi) movq %rdx, TDX_SEAM_rdx(%rdi) movq %r8, TDX_SEAM_r8(%rdi) movq %r9, TDX_SEAM_r9(%rdi) movq %r10, TDX_SEAM_r10(%rdi) movq %r11, TDX_SEAM_r11(%rdi)
2:
FRAME_END
ret
3:
/* Probably it helps to write an error code in %rax? */ movq $0x4000000500000000, %rax cmpb $0, kvm_rebooting jne 2b ud2 _ASM_EXTABLE(1b, 3b)
Paolo
RFC PATCH v2 7/69 KVM: TDX: define and export helper functions for KVM TDX support
In reply to [email protected]From: Sean Christopherson <[email protected]>
NOTE: This is to make this patch series compile. other patch series that loads/initializes TDX module will replace this patch.
Define and export four helper functions commly used for for KVM TDX support and SEAMLDR. tdx_get_sysinfo(), tdx_seamcall_on_each_pkg(), tdx_keyid_alloc() and tdx_keyid_free(). The SEAMLDR logic will initializes at boot phase and KVM TDX will use those function to get system info, operation of package wide resource and, alloc/free tdx private key ID.
--- arch/x86/Kbuild | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/kvm_boot.h | 30 +++++ arch/x86/kvm/boot/Makefile | 6 + arch/x86/kvm/boot/seam/tdx_common.c | 167 ++++++++++++++++++++++++++++ arch/x86/kvm/boot/seam/tdx_common.h | 13 +++ 6 files changed, 219 insertions(+) create mode 100644 arch/x86/include/asm/kvm_boot.h create mode 100644 arch/x86/kvm/boot/Makefile create mode 100644 arch/x86/kvm/boot/seam/tdx_common.c create mode 100644 arch/x86/kvm/boot/seam/tdx_common.h
arch/x86/Kbuild +1 -0
@@ -4,6 +4,7 @@ obj-y += entry/ obj-$(CONFIG_PERF_EVENTS) += events/ obj-$(CONFIG_KVM) += kvm/ +obj-$(subst m,y,$(CONFIG_KVM)) += kvm/boot/ # Xen paravirtualization support obj-$(CONFIG_XEN) += xen/
arch/x86/include/asm/cpufeatures.h +2 -0
@@ -230,6 +230,8 @@#define X86_FEATURE_FLEXPRIORITY ( 8*32+ 2)/* Intel FlexPriority */ #define X86_FEATURE_EPT ( 8*32+ 3)/* Intel Extended Page Table */ #define X86_FEATURE_VPID ( 8*32+ 4)/* Intel Virtual Processor ID */ +#define X86_FEATURE_SEAM ( 8*32+ 5)/* "" Secure Arbitration Mode */ +#define X86_FEATURE_TDX ( 8*32+ 6)/* Intel Trusted Domain eXtensions */ #define X86_FEATURE_VMMCALL ( 8*32+15)/* Prefer VMMCALL to VMCALL */ #define X86_FEATURE_XENPV ( 8*32+16)/* "" Xen paravirtual guest */
arch/x86/include/asm/kvm_boot.h new file +30 -0
@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_X86_KVM_BOOT_H +#define _ASM_X86_KVM_BOOT_H + +#include <linux/cpumask.h> +#include <linux/mutex.h> +#include <linux/smp.h> +#include <linux/types.h> +#include <asm/processor.h> + +#ifdef CONFIG_KVM_INTEL_TDX + +/* +* Return pointer to TDX system info (TDSYSINFO_STRUCT) if TDX has been +* successfully initialized, or NULL. +*/ +struct tdsysinfo_struct ; +const struct tdsysinfo_struct *tdx_get_sysinfo (void ); + +extern u32 tdx_seam_keyid__ro_after_init ; + +int tdx_seamcall_on_each_pkg (int (*fn)(void *),void *param); + +/* TDX keyID allocation functions */ +extern int tdx_keyid_alloc (void ); +extern void tdx_keyid_free (int keyid); + +#endif + +#endif /* _ASM_X86_KVM_BOOT_H */
arch/x86/kvm/boot/Makefile new file +6 -0
@@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 + +asflags-y += -I$(srctree)/arch/x86/kvm +ccflags-y += -I$(srctree)/arch/x86/kvm + +obj-$(CONFIG_KVM_INTEL_TDX) += seam/tdx_common.o
arch/x86/kvm/boot/seam/tdx_common.c new file +167 -0
167 lines not on this page · raw message
arch/x86/kvm/boot/seam/tdx_common.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* common functions/symbols used by SEAMLDR and KVM */ + +#ifndef __BOOT_SEAM_TDX_COMMON_H +#define __BOOT_SEAM_TDX_COMMON_H + +extern struct tdsysinfo_struct tdx_tdsysinfo; +extern u32 tdx_keyids_start; +extern u32 tdx_nr_keyids; + +int __init init_package_masters (void ); + +#endif /* __BOOT_SEAM_TDX_COMMON_H */
Signature · 1 line
2.25.1
RE: [RFC PATCH v2 07/69] KVM: TDX: define and export helper functions for KVM TDX support
Quoted 6 lines · Isaku Yamahata
Subject: [RFC PATCH v2 07/69] KVM: TDX: define and export helper functions for KVM TDX support
arch/x86/kvm/boot/seam/tdx_common.c new file +4
-0+/* +* Setup one-cpu-per-pkg array to do package-scoped SEAMCALLs. The +array is +* only necessary if there are multiple packages.
arch/x86/kvm/boot/seam/tdx_common.c line 82
Quoted 10 lines
arch/x86/kvm/boot/seam/tdx_common.c new file +10
-0+*/ +int __init init_package_masters (void ) +{ +int cpu, pkg, nr_filled, nr_pkgs; + + nr_pkgs =topology_max_packages (); +if (nr_pkgs ==1 ) +return 0 ; + + tdx_package_masters =kcalloc (nr_pkgs,sizeof (int ),GFP_KERNEL );
Where is the corresponding kfree() invoked? (except the one invoked on error conditions below)
Quoted 10 lines
arch/x86/kvm/boot/seam/tdx_common.c new file +10
-0+if (!tdx_package_masters) +return -ENOMEM ; + +memset (tdx_package_masters,-1 , nr_pkgs *sizeof (int )); + + nr_filled =0 ; +for_each_online_cpu (cpu) { + pkg =topology_physical_package_id (cpu); +if (tdx_package_masters[pkg] >=0 ) +continue ;
arch/x86/kvm/boot/seam/tdx_common.c line 102
Quoted 10 lines
arch/x86/kvm/boot/seam/tdx_common.c new file +10
-0+ + tdx_package_masters[pkg] = cpu; +if (++nr_filled ==topology_max_packages ()) +break ; + } + +if (WARN_ON (nr_filled !=topology_max_packages ())) { +kfree (tdx_package_masters); +return -EIO ; + }
Best,
Wei
RFC PATCH v2 8/69 KVM: TDX: add trace point before/after TDX SEAMCALLs
In reply to [email protected]From: Isaku Yamahata <[email protected]>
--- arch/x86/kvm/trace.h | 80 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/seamcall.h | 22 ++++++++- arch/x86/kvm/vmx/tdx_arch.h | 47 ++++++++++++++++++ arch/x86/kvm/vmx/tdx_errno.h | 96 ++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 2 + 5 files changed, 246 insertions(+), 1 deletion(-)
arch/x86/kvm/trace.h +80 -0
@@ -8,6 +8,9 @@#include <asm/clocksource.h> #include <asm/pvclock-abi.h> +#include "vmx/tdx_arch.h" +#include "vmx/tdx_errno.h" +#undef TRACE_SYSTEM#define TRACE_SYSTEM kvm
@@ -659,6 +662,83 @@ TRACE_EVENT(kvm_nested_vmexit_inject, __entry->exit_int_info, __entry->exit_int_info_err) ); +/* +* Tracepoint for the start of TDX SEAMCALLs. +*/ +TRACE_EVENT (kvm_tdx_seamcall_enter, + TP_PROTO(int cpuid, __u64 op, __u64 rcx, __u64 rdx, __u64 r8, + __u64 r9, __u64 r10), +TP_ARGS (cpuid, op, rcx, rdx, r8, r9, r10), + +TP_STRUCT__entry ( + __field( int, cpuid ) + __field( __u64, op ) + __field( __u64, rcx ) + __field( __u64, rdx ) + __field( __u64, r8 ) + __field( __u64, r9 ) +__field ( __u64, r10 ) + ), + +TP_fast_assign ( + __entry->cpuid = cpuid; + __entry->op = op; + __entry->rcx = rcx; + __entry->rdx = rdx; + __entry->r8 = r8; + __entry->r9 = r9; + __entry->r10 = r10; + ), + +TP_printk ("cpu: %d op: %s rcx: 0x%llx rdx: 0x%llx r8: 0x%llx r9: 0x%llx r10: 0x%llx" , + __entry->cpuid, +__print_symbolic (__entry->op,TDX_SEAMCALL_OP_CODES ), + __entry->rcx, __entry->rdx, __entry->r8, + __entry->r9, __entry->r10) +); + +/* +* Tracepoint for the end of TDX SEAMCALLs. +*/ +TRACE_EVENT (kvm_tdx_seamcall_exit, + TP_PROTO(int cpuid, __u64 op, __u64 err, __u64 rcx, __u64 rdx, __u64 r8, + __u64 r9, __u64 r10, __u64 r11), +TP_ARGS (cpuid, op, err, rcx, rdx, r8, r9, r10, r11), + +TP_STRUCT__entry ( + __field( int, cpuid ) + __field( __u64, op ) + __field( __u64, err ) + __field( __u64, rcx ) + __field( __u64, rdx ) + __field( __u64, r8 ) + __field( __u64, r9 ) + __field( __u64, r10 ) +__field ( __u64, r11 ) + ), + +TP_fast_assign ( + __entry->cpuid = cpuid; + __entry->op = op; + __entry->err = err; + __entry->rcx = rcx; + __entry->rdx = rdx; + __entry->r8 = r8; + __entry->r9 = r9; + __entry->r10 = r10; + __entry->r11 = r11; + ), + +TP_printk ("cpu: %d op: %s err %s 0x%llx rcx: 0x%llx rdx: 0x%llx r8: 0x%llx r9: 0x%llx r10: 0x%llx r11: 0x%llx" , + __entry->cpuid, +__print_symbolic (__entry->op,TDX_SEAMCALL_OP_CODES ), +__print_symbolic (__entry->err &TDX_SEAMCALL_STATUS_MASK , +TDX_SEAMCALL_STATUS_CODES ), + __entry->err, + __entry->rcx, __entry->rdx, __entry->r8, + __entry->r9, __entry->r10, __entry->r11) +); +/* * Tracepoint for nested #vmexit because of interrupt pending */
arch/x86/kvm/vmx/seamcall.h +21 -1
@@ -9,12 +9,32 @@#else #ifndef seamcall +#include "trace.h" +struct tdx_ex_ret ; asmlinkage u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,struct tdx_ex_ret *ex); +static inline u64 _seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10, +struct tdx_ex_ret *ex) +{ +u64 err; + +trace_kvm_tdx_seamcall_enter (smp_processor_id (), op, + rcx, rdx, r8, r9, r10); + err =__seamcall (op, rcx, rdx, r8, r9, r10, ex); +if (ex) +trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, ex->rcx, + ex->rdx, ex->r8, ex->r9, ex->r10, + ex->r11); +else +trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, +0 ,0 ,0 ,0 ,0 ,0 ); +return err; +} +#define seamcall (op, rcx, rdx, r8, r9, r10, ex) \ - __seamcall(SEAMCALL_##op, (rcx), (rdx), (r8), (r9), (r10), (ex)) + _seamcall(SEAMCALL_##op, (rcx), (rdx), (r8), (r9), (r10), (ex))#endif static inline void __pr_seamcall_error(u64 op, const char *op_str,
arch/x86/kvm/vmx/tdx_arch.h +47 -0
53 lines not on this page · raw message
arch/x86/kvm/vmx/tdx_errno.h +96 -0
108 lines not on this page · raw message
arch/x86/kvm/x86.c +2 -0
@@ -11970,6 +11970,8 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_avic_unaccelerated_access);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_incomplete_ipi);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_ga_log);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_apicv_update_request); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_tdx_seamcall_enter); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_tdx_seamcall_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_enter);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_msr_protocol_enter);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 08/69] KVM: TDX: add trace point before/after TDX SEAMCALLs
arch/x86/kvm/vmx/seamcall.h line 32
Quoted 10 lines · On 03/07/21 00:04, [email protected] wrote:
arch/x86/kvm/vmx/seamcall.h +10
-0+trace_kvm_tdx_seamcall_enter (smp_processor_id (), op, + rcx, rdx, r8, r9, r10); + err =__seamcall (op, rcx, rdx, r8, r9, r10, ex); +if (ex) +trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, ex->rcx, + ex->rdx, ex->r8, ex->r9, ex->r10, + ex->r11); +else +trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, +0 ,0 ,0 ,0 ,0 ,0 );
Would it make sense to do the zeroing of ex directly in __seamcall in case there is an error?
Otherwise looks good.
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 08/69] KVM: TDX: add trace point before/after TDX SEAMCALLs
Quoted 14 lines · Paolo Bonzini
Review
Quoted 10 lines · On 03/07/21 00:04, [email protected] wrote:
(quoted diff) +10
-0+ trace_kvm_tdx_seamcall_enter(smp_processor_id(), op, + rcx, rdx, r8, r9, r10); + err = __seamcall(op, rcx, rdx, r8, r9, r10, ex); + if (ex) + trace_kvm_tdx_seamcall_exit(smp_processor_id(), op, err, ex->rcx, + ex->rdx, ex->r8, ex->r9, ex->r10, + ex->r11); + else + trace_kvm_tdx_seamcall_exit(smp_processor_id(), op, err, + 0, 0, 0, 0, 0, 0);Would it make sense to do the zeroing of ex directly in
__seamcallin case there is an error?
A better option would be to pass "ex" into the tracepoint. tdx_arch.h is already included by trace.h (though I'm not sure that's a good thing), and the cost of checking ex against NULL over and over is a non-issue because it's buried in the tracepoint, i.e. hidden behind a patch nop. The below reduces the footprint of _seamcall by 100+ bytes of code, presumably due to avoiding even more register shuffling (I didn't look too closely).
That said, I'm not sure adding generic tracepoints is a good idea. The flows that truly benefit from tracepoints will likely want to provide more/different information, e.g. the entry/exit flow already uses kvm_trace_entry/exit, and the SEPT flows have dedicated tracepoints. For flows like tdh_vp_flush(), which might benefit from a tracepoint, they'll only provide the host PA of the TDVPR, which is rather useless on its own. It's probably possible to cross-reference everything to understand what's going on, but it certainly won't be easy.
I can see the generic tracepoint being somewhat useful for debugging early development and/or a new TDX module, but otherwise I think it will be mostly overhead. E.g. if a TDX failure pops up in production, enabling the tracepoint might not even be viable. And even for the cases where the tracepoint is useful, I would be quite surprised if additional instrumentation wasn't needed to debug non-trivial issues.
arch/x86/kvm/trace.h +8 -9
@@ -701,9 +701,8 @@ TRACE_EVENT(kvm_tdx_seamcall_enter,* Tracepoint for the end of TDX SEAMCALLs. */ TRACE_EVENT(kvm_tdx_seamcall_exit, - TP_PROTO(int cpuid, __u64 op, __u64 err, __u64 rcx, __u64 rdx, __u64 r8, - __u64 r9, __u64 r10, __u64 r11), - TP_ARGS(cpuid, op, err, rcx, rdx, r8, r9, r10, r11), + TP_PROTO(int cpuid, __u64 op, __u64 err, struct tdx_ex_ret *ex), + TP_ARGS(cpuid, op, err, ex), TP_STRUCT__entry( __field( int, cpuid )
@@ -721,12 +720,12 @@ TRACE_EVENT(kvm_tdx_seamcall_exit, __entry->cpuid = cpuid; __entry->op = op; __entry->err = err; - __entry->rcx = rcx; - __entry->rdx = rdx; - __entry->r8 = r8; - __entry->r9 = r9; - __entry->r10 = r10; - __entry->r11 = r11; + __entry->rcx = ex ? ex->rcx :0 ; + __entry->rdx = ex ? ex->rdx :0 ; + __entry->r8 = ex ? ex->r8 :0 ; + __entry->r9 = ex ? ex->r9 :0 ; + __entry->r10 = ex ? ex->r10 :0 ; + __entry->r11 = ex ? ex->r11 :0 ; ), TP_printk("cpu: %d op: %s err %s 0x%llx rcx: 0x%llx rdx: 0x%llx r8: 0x%llx r9: 0x%llx r10: 0x%llx r11: 0x%llx",
arch/x86/kvm/vmx/seamcall.h +2 -7
@@ -23,13 +23,8 @@ static inline u64 _seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,trace_kvm_tdx_seamcall_enter (smp_processor_id (), op, rcx, rdx, r8, r9, r10); err =__seamcall (op, rcx, rdx, r8, r9, r10, ex); -if (ex) -trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, ex->rcx, - ex->rdx, ex->r8, ex->r9, ex->r10, - ex->r11); -else -trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, -0 ,0 ,0 ,0 ,0 ,0 ); +trace_kvm_tdx_seamcall_exit (smp_processor_id (), op, err, ex); +return err; }
Sean Christopherson
maintainerRe: [RFC PATCH v2 08/69] KVM: TDX: add trace point before/after TDX SEAMCALLs
In reply to [email protected]Quoted 26 lines · On Fri, Jul 02, 2021, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/trace.h | 80 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/seamcall.h | 22 ++++++++- arch/x86/kvm/vmx/tdx_arch.h | 47 ++++++++++++++++++ arch/x86/kvm/vmx/tdx_errno.h | 96 ++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 2 + 5 files changed, 246 insertions(+), 1 deletion(-)arch/x86/kvm/trace.h +3
-0@@ -8,6 +8,9 @@#include <asm/clocksource.h> #include <asm/pvclock-abi.h> +#include "vmx/tdx_arch.h" +#include "vmx/tdx_errno.h" +#undef TRACE_SYSTEM#define TRACE_SYSTEM kvm
arch/x86/kvm/trace.h TRACE_EVENT(kvm_nested_vmexit_inject, · line 668
Quoted 7 lines
arch/x86/kvm/trace.h +4
-0@@ -659,6 +662,83 @@ TRACE_EVENT(kvm_nested_vmexit_inject, __entry->exit_int_info, __entry->exit_int_info_err) ); +/* +* Tracepoint for the start of TDX SEAMCALLs. +*/ +TRACE_EVENT(kvm_tdx_seamcall_enter,
To avoid confusion, I think it makes sense to avoid "enter" and "exit". E.g. my first reaction was that the tracepoint was specific to TDENTER. And under the hood, SEAMCALL is technically an exit :-)
What about kvm_tdx_seamcall and kvm_tdx_seamret? If the seamret usage is too much of a stretch, kvm_tdx_seamcall_begin/end?
arch/x86/kvm/trace.h TRACE_EVENT(kvm_nested_vmexit_inject, · line 671
Quoted 3 lines
arch/x86/kvm/trace.h +3
-0+ TP_PROTO(int cpuid, __u64 op, __u64 rcx, __u64 rdx, __u64 r8, + __u64 r9, __u64 r10), + TP_ARGS(cpuid, op, rcx, rdx, r8, r9, r10),
"cpuid" is potentially confusing without looking at the caller. pcpu or pcpu_id would be preferable.
Quoted 12 lines
arch/x86/kvm/vmx/seamcall.h +2
-0@@ -9,12 +9,32 @@#else #ifndef seamcall +#include "trace.h" +struct tdx_ex_ret ; asmlinkage u64 __seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,
arch/x86/kvm/vmx/seamcall.h line 25
Quoted 10 lines
arch/x86/kvm/vmx/seamcall.h +8
-0struct tdx_ex_ret *ex); +static inline u64 _seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10, +struct tdx_ex_ret *ex) +{ +u64 err; + +trace_kvm_tdx_seamcall_enter (smp_processor_id (), op, + rcx, rdx, r8, r9, r10); + err =__seamcall (op, rcx, rdx, r8, r9, r10, ex);
What was the motivation behind switching from the macro magic[*] to a dedicated asm subroutine? The macros are gross, but IMO they yielded much more readable code for the upper level helpers, which is what people will look at the vast majority of time. E.g.
static inline u64 tdh_sys_lp_shutdown(void)
{
return seamcall(TDH_SYS_LP_SHUTDOWN, 0, 0, 0, 0, 0, NULL);
}
static inline u64 tdh_mem_track(hpa_t tdr)
{
return seamcall(TDH_MEM_TRACK, tdr, 0, 0, 0, 0, NULL);
}versus
static inline u64 tdsysshutdownlp(void)
{
seamcall_0(TDSYSSHUTDOWNLP);
}
static inline u64 tdtrack(hpa_t tdr)
{
seamcall_1(TDTRACK, tdr);
}The new approach also generates very suboptimal code due to the need to shuffle registers everywhere, e.g. gcc doesn't inline _seamcall because it's a whopping 200+ bytes.
arch/x86/kvm/vmx/seamcall.h line 27
Quoted 2 lines
arch/x86/kvm/vmx/seamcall.h +2
-0+if (ex) + trace_kvm_tdx_seamcall_exit(smp_processor_id(), op, err, ex->rcx,
smp_processor_id() is not stable since this code runs with IRQs and preemption enabled, e.g. if the task is preempted between the tracepoint and the actual SEAMCALL then the tracepoint may be wrong. There could also be weirdly "nested" tracepoints since migrating the task will generate TDH_VP_FLUSH.
Quoted 12 lines
arch/x86/kvm/vmx/seamcall.h +9
-1+ ex->rdx, ex->r8, ex->r9, ex->r10, + ex->r11); + else + trace_kvm_tdx_seamcall_exit(smp_processor_id(), op, err, + 0, 0, 0, 0, 0, 0); + return err; +} +#define seamcall (op, rcx, rdx, r8, r9, r10, ex) \ - __seamcall(SEAMCALL_##op, (rcx), (rdx), (r8), (r9), (r10), (ex)) + _seamcall(SEAMCALL_##op, (rcx), (rdx), (r8), (r9), (r10), (ex))#endif
RFC PATCH v2 9/69 KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX SEAMCALL interface is defined in [1] 20.2 Host-Side(SEAMCALL) interface Functions. Define C wrapper functions for SEAMCALLs which the later patches will use.
[1] TDX Module spec
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
--- arch/x86/kvm/vmx/tdx_ops.h | 205 +++++++++++++++++++++++++++++++++++++ 1 file changed, 205 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
arch/x86/kvm/vmx/tdx_ops.h new file +205 -0
@@ -0,0 +1,205 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_OPS_H +#define __KVM_X86_TDX_OPS_H + +#include <linux/compiler.h> + +#include <asm/asm.h> +#include <asm/kvm_host.h> + +#include "seamcall.h" + +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +return seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_add (hpa_t tdr,gpa_t gpa,hpa_t hpa,hpa_t source, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_ADD , gpa, tdr, hpa, source,0 , ex); +} + +static inline u64 tdh_mem_spet_add (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_ADD , gpa | level, tdr, page,0 ,0 , ex); +} + +static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) +{ +return seamcall (TDH_VP_ADDCX , addr, tdvpr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_aug (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_AUG , gpa, tdr, hpa,0 ,0 , ex); +} + +static inline u64 tdh_mem_range_block (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_RANGE_BLOCK , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mng_key_config (hpa_t tdr) +{ +return seamcall (TDH_MNG_KEY_CONFIG , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_create (hpa_t tdr,int hkid) +{ +return seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) +{ +return seamcall (TDH_VP_CREATE , tdvpr, tdr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_rd (hpa_t tdr,u64 field,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MNG_RD , tdr, field,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mng_wr (hpa_t tdr,u64 field,u64 val,u64 mask, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MNG_WR , tdr, field, val, mask,0 , ex); +} + +static inline u64 tdh_phymem_page_rd (hpa_t addr,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_PHYMEM_PAGE_RD , addr,0 ,0 ,0 ,0 , ex); +} + +static inline u64 tdh_phymem_page_wr (hpa_t addr,u64 val,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_PHYMEM_PAGE_WR , addr, val,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_page_demote (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_DEMOTE , gpa | level, tdr, page,0 ,0 , ex); +} + +static inline u64 tdh_mr_extend (hpa_t tdr,gpa_t gpa,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MR_EXTEND , gpa, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mr_finalize (hpa_t tdr) +{ +return seamcall (TDH_MR_FINALIZE , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_flush (hpa_t tdvpr) +{ +return seamcall (TDH_VP_FLUSH , tdvpr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_vpflushdone (hpa_t tdr) +{ +return seamcall (TDH_MNG_VPFLUSHDONE , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_key_freeid (hpa_t tdr) +{ +return seamcall (TDH_MNG_KEY_FREEID , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_init (hpa_t tdr,hpa_t td_params,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MNG_INIT , tdr, td_params,0 ,0 ,0 , ex); +} + +static inline u64 tdh_vp_init (hpa_t tdvpr,u64 rcx) +{ +return seamcall (TDH_VP_INIT , tdvpr, rcx,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_promote (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_PROMOTE , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_phymem_page_rdmd (hpa_t page,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_PHYMEM_PAGE_RDMD , page,0 ,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_sept_rd (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_RD , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_vp_rd (hpa_t tdvpr,u64 field,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_VP_RD , tdvpr, field,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mng_key_reclaimid (hpa_t tdr) +{ +return seamcall (TDH_MNG_KEY_RECLAIMID , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_phymem_page_reclaim (hpa_t page,struct tdx_ex_ret *ex) +{ +return seamcall (TDH_PHYMEM_PAGE_RECLAIM , page,0 ,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_page_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_PAGE_REMOVE , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_mem_sept_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_REMOVE , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_sys_lp_shutdown (void ) +{ +return seamcall (TDH_SYS_LP_SHUTDOWN ,0 ,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_track (hpa_t tdr) +{ +return seamcall (TDH_MEM_TRACK , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_range_unblock (hpa_t tdr,gpa_t gpa,int level, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_RANGE_UNBLOCK , gpa | level, tdr,0 ,0 ,0 , ex); +} + +static inline u64 tdh_phymem_cache_wb (bool resume) +{ +return seamcall (TDH_PHYMEM_CACHE_WB , resume ?1 :0 ,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_phymem_page_wbinvd (hpa_t page) +{ +return seamcall (TDH_PHYMEM_PAGE_WBINVD , page,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_sept_wr (hpa_t tdr,gpa_t gpa,int level,u64 val, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_MEM_SEPT_WR , gpa | level, tdr, val,0 ,0 , ex); +} + +static inline u64 tdh_vp_wr (hpa_t tdvpr,u64 field,u64 val,u64 mask, +struct tdx_ex_ret *ex) +{ +return seamcall (TDH_VP_WR , tdvpr, field, val, mask,0 , ex); +} + +#endif /* __KVM_X86_TDX_OPS_H */
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 09/69] KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
arch/x86/kvm/vmx/tdx_ops.h line 16
Quoted 5 lines · On 03/07/21 00:04, [email protected] wrote:
arch/x86/kvm/vmx/tdx_ops.h new file +5
-0+static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +return seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); +} +
Since you have wrappers anyway, I don't like having an extra macro level just to remove the SEAMCALL_ prefix. It messes up editors that look up the symbols.
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 09/69] KVM: TDX: Add C wrapper functions for TDX SEAMCALLs
Quoted 10 lines · Paolo Bonzini
arch/x86/kvm/vmx/tdx_ops.h line 16
Quoted 5 lines · On 03/07/21 00:04, [email protected] wrote:
arch/x86/kvm/vmx/tdx_ops.h new file +5
-0+static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +return seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); +} +Since you have wrappers anyway, I don't like having an extra macro level just to remove the
SEAMCALL_prefix. It messes up editors that look up the symbols.
True. On the other hand, prefixing SEAMCALL_ over and over is tedious and adds a lot of noise. What if we drop SEAMCALL_ from the #defines? The prefix made sense when there was no additional namespace, e.g. instead of having a bare TDCREATE, but that's mostly a non-issue now that all the SEAMCALLs are namespaced with TDH_.
RFC PATCH v2 10/69 KVM: TDX: Print the name of SEAMCALL status code
In reply to [email protected]From: Isaku Yamahata <[email protected]>
SEAMCALL error code is not intuitive to tell what's wrong in the SEAMCALL, print the error code name along with it.
--- arch/x86/kvm/boot/seam/tdx_common.c | 21 +++++++++++++++++++++ arch/x86/kvm/vmx/seamcall.h | 7 +++++-- 2 files changed, 26 insertions(+), 2 deletions(-)
arch/x86/kvm/boot/seam/tdx_common.c +21 -0
@@ -9,6 +9,7 @@#include <asm/kvm_boot.h> #include "vmx/tdx_arch.h" +#include "vmx/tdx_errno.h" /* * TDX system information returned by TDSYSINFO.
@@ -165,3 +166,23 @@ void tdx_keyid_free(int keyid)ida_free (&tdx_keyid_pool, keyid); }EXPORT_SYMBOL_GPL (tdx_keyid_free); + +static struct tdx_seamcall_status { +u64 err_code; +const char *err_name; +} tdx_seamcall_status_codes[] = {TDX_SEAMCALL_STATUS_CODES }; + +const char *tdx_seamcall_error_name (u64 error_code) +{ +struct tdx_seamcall_status status; +int i; + +for (i =0 ; i <ARRAY_SIZE (tdx_seamcall_status_codes); i++) { + status = tdx_seamcall_status_codes[i]; +if ((error_code &TDX_SEAMCALL_STATUS_MASK ) == status.err_code) +return status.err_name; + } + +return "Unknown SEAMCALL status code" ; +} +EXPORT_SYMBOL_GPL (tdx_seamcall_error_name);
arch/x86/kvm/vmx/seamcall.h +5 -2
@@ -37,11 +37,14 @@ static inline u64 _seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10, _seamcall(SEAMCALL_##op, (rcx), (rdx), (r8), (r9), (r10), (ex))#endif +const char *tdx_seamcall_error_name (u64 error_code); +static inline void __pr_seamcall_error (u64 op,const char *op_str,u64 err,struct tdx_ex_ret *ex) { -pr_err_ratelimited ("SEAMCALL[%s] failed on cpu %d: 0x%llx\n" , - op_str,smp_processor_id (), (err)); +pr_err_ratelimited ("SEAMCALL[%s] failed on cpu %d: %s (0x%llx)\n" , + op_str,smp_processor_id (), +tdx_seamcall_error_name (err), err);if (ex) pr_err_ratelimited( "RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n",
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 10/69] KVM: TDX: Print the name of SEAMCALL status code
Quoted 60 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
SEAMCALL error code is not intuitive to tell what's wrong in the SEAMCALL, print the error code name along with it.
Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/boot/seam/tdx_common.c | 21 +++++++++++++++++++++ arch/x86/kvm/vmx/seamcall.h | 7 +++++-- 2 files changed, 26 insertions(+), 2 deletions(-)arch/x86/kvm/boot/seam/tdx_common.c +21
-0@@ -9,6 +9,7 @@#include <asm/kvm_boot.h> #include "vmx/tdx_arch.h" +#include "vmx/tdx_errno.h" /* * TDX system information returned by TDSYSINFO. @@ -165,3 +166,23 @@ void tdx_keyid_free(int keyid)ida_free (&tdx_keyid_pool, keyid); }EXPORT_SYMBOL_GPL (tdx_keyid_free); + +static struct tdx_seamcall_status { +u64 err_code; +const char *err_name; +} tdx_seamcall_status_codes[] = {TDX_SEAMCALL_STATUS_CODES }; + +const char *tdx_seamcall_error_name (u64 error_code) +{ +struct tdx_seamcall_status status; +int i; + +for (i =0 ; i <ARRAY_SIZE (tdx_seamcall_status_codes); i++) { + status = tdx_seamcall_status_codes[i]; +if ((error_code &TDX_SEAMCALL_STATUS_MASK ) == status.err_code) +return status.err_name; + } + +return "Unknown SEAMCALL status code" ; +} +EXPORT_SYMBOL_GPL (tdx_seamcall_error_name);arch/x86/kvm/vmx/seamcall.h +2
-0@@ -37,11 +37,14 @@ static inline u64 _seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10, _seamcall(SEAMCALL_##op, (rcx), (rdx), (r8), (r9), (r10), (ex))#endif +const char *tdx_seamcall_error_name (u64 error_code); + static inline void __pr_seamcall_error(u64 op, const char *op_str,
arch/x86/kvm/vmx/seamcall.h static inline u64 _seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10, · line 50
Quoted 10 lines
arch/x86/kvm/vmx/seamcall.h +3
-2u64 err, struct tdx_ex_ret *ex) { - pr_err_ratelimited("SEAMCALL[%s] failed on cpu %d: 0x%llx\n", - op_str, smp_processor_id(), (err)); + pr_err_ratelimited("SEAMCALL[%s] failed on cpu %d: %s (0x%llx)\n", + op_str,smp_processor_id (), + tdx_seamcall_error_name(err), err); if (ex) pr_err_ratelimited( "RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n",
You can squash it in the earlier patch that introduced __pr_seamcall_error.
Paolo
RFC PATCH v2 11/69 KVM: TDX: Introduce pr_seamcall_ex_ret_info() to print more info when SEAMCALL fails
In reply to [email protected]From: Xiaoyao Li <[email protected]>
Various SEAMCALLs may have additional info in RCX, RDX, R8, R9, R10 when it completes. Introduce pr_seamcall_ex_ret_info() to parse those additional info based on returned SEAMCALL status code.
It only processes some cases for reference. More processes for other SEAMCALL status code can be added in the future on demand.
--- arch/x86/kvm/boot/seam/tdx_common.c | 54 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/seamcall.h | 6 ++-- 2 files changed, 56 insertions(+), 4 deletions(-)
arch/x86/kvm/boot/seam/tdx_common.c +54 -0
@@ -186,3 +186,57 @@ const char *tdx_seamcall_error_name(u64 error_code)return "Unknown SEAMCALL status code" ; }EXPORT_SYMBOL_GPL (tdx_seamcall_error_name); + +static const char *const TDX_SEPT_ENTRY_STATES [] = { +"SEPT_FREE" , +"SEPT_BLOCKED" , +"SEPT_PENDING" , +"SEPT_PENDING_BLOCKED" , +"SEPT_PRESENT" +}; + +void pr_seamcall_ex_ret_info (u64 op,u64 error_code,struct tdx_ex_ret *ex_ret) +{ +if (!ex_ret) +return ; + +switch (error_code &TDX_SEAMCALL_STATUS_MASK ) { +case TDX_INCORRECT_CPUID_VALUE : +pr_err ("Expected CPUID [leaf 0x%x subleaf 0x%x]: " +"eax 0x%x check_mask 0x%x, ebx 0x%x check_mask 0x%x, " +"ecx 0x%x check_mask 0x%x, edx 0x%x check_mask 0x%x\n" , + ex_ret->leaf, ex_ret->subleaf, + ex_ret->eax_val, ex_ret->eax_mask, + ex_ret->ebx_val, ex_ret->ebx_mask, + ex_ret->ecx_val, ex_ret->ecx_mask, + ex_ret->edx_val, ex_ret->edx_mask); +break ; +case TDX_INCONSISTENT_CPUID_FIELD : +pr_err ("Inconsistent CPUID [leaf 0x%x subleaf 0x%x]: " +"eax_mask 0x%x, ebx_mask 0x%x, ecx_mask %x, edx_mask 0x%x\n" , + ex_ret->leaf, ex_ret->subleaf, + ex_ret->eax_mask, ex_ret->ebx_mask, + ex_ret->ecx_mask, ex_ret->edx_mask); +break ; +case TDX_EPT_WALK_FAILED : { +const char *state; + +if (ex_ret->state >=ARRAY_SIZE (TDX_SEPT_ENTRY_STATES )) + state ="Invalid" ; +else + state =TDX_SEPT_ENTRY_STATES [ex_ret->state]; + +pr_err ("Secure EPT walk error: SEPTE 0x%llx, level %d, %s\n" , + ex_ret->septe, ex_ret->level, state); +break ; + } +default : +/* TODO: print only meaningful registers depending on op */ +pr_err ("RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, " +"R10 0x%llx, R11 0x%llx\n" , + ex_ret->rcx, ex_ret->rdx, ex_ret->r8, ex_ret->r9, + ex_ret->r10, ex_ret->r11); +break ; + } +} +EXPORT_SYMBOL_GPL (pr_seamcall_ex_ret_info);
arch/x86/kvm/vmx/seamcall.h +2 -4
@@ -38,6 +38,7 @@ static inline u64 _seamcall(u64 op, u64 rcx, u64 rdx, u64 r8, u64 r9, u64 r10,#endif const char *tdx_seamcall_error_name (u64 error_code); +void pr_seamcall_ex_ret_info (u64 op,u64 error_code,struct tdx_ex_ret *ex_ret);static inline void __pr_seamcall_error (u64 op,const char *op_str,u64 err,struct tdx_ex_ret *ex)
@@ -46,10 +47,7 @@ static inline void __pr_seamcall_error(u64 op, const char *op_str, op_str,smp_processor_id (), tdx_seamcall_error_name(err), err);if (ex) -pr_err_ratelimited ( -"RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n" , - (ex)->rcx, (ex)->rdx, (ex)->r8, (ex)->r9, (ex)->r10, - (ex)->r11); +pr_seamcall_ex_ret_info (op, err, ex); }#define pr_seamcall_error (op, err, ex) \
Signature · 1 line
2.25.1
RFC PATCH v2 12/69 KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
In reply to [email protected]From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
--- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+)
virt/kvm/kvm_main.c +1 -0
@@ -4606,6 +4606,7 @@ int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, r =__kvm_io_bus_read (vcpu, bus, &range, val);return r <0 ? r :0 ; } +EXPORT_SYMBOL_GPL (kvm_io_bus_read);/* Caller must hold slots_lock. */ int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 12/69] KVM: Export kvm_io_bus_read for use by TDX for PV MMIO
Quoted 16 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+)virt/kvm/kvm_main.c +0
-0
virt/kvm/kvm_main.c int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, · line 4612
Quoted 7 lines
virt/kvm/kvm_main.c +1
-0@@ -4606,6 +4606,7 @@ int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, r =__kvm_io_bus_read (vcpu, bus, &range, val);return r <0 ? r :0 ; } +EXPORT_SYMBOL_GPL (kvm_io_bus_read);/* Caller must hold slots_lock. */ int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
RFC PATCH v2 13/69 KVM: Enable hardware before doing arch VM initialization
In reply to [email protected]From: Sean Christopherson <[email protected]>
Swap the order of hardware_enable_all() and kvm_arch_init_vm() to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.
This also provides consistent ordering between kvm_create_vm() and kvm_destroy_vm() with respect to calling kvm_arch_destroy_vm() and hardware_disable_all().
--- virt/kvm/kvm_main.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
virt/kvm/kvm_main.c +8 -8
@@ -923,7 +923,7 @@ static struct kvm *kvm_create_vm(unsigned long type)struct kvm_memslots *slots =kvm_alloc_memslots ();if (!slots) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable;/* Generations must be different for each address space. */ slots->generation = i;rcu_assign_pointer (kvm->memslots[i], slots);
@@ -933,19 +933,19 @@ static struct kvm *kvm_create_vm(unsigned long type)rcu_assign_pointer (kvm->buses[i],kzalloc (sizeof (struct kvm_io_bus ),GFP_KERNEL_ACCOUNT ));if (!kvm->buses[i]) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable; } kvm->max_halt_poll_ns = halt_poll_ns; - r =kvm_arch_init_vm (kvm, type); -if (r) - goto out_err_no_arch_destroy_vm; - r =hardware_enable_all ();if (r) goto out_err_no_disable; + r =kvm_arch_init_vm (kvm, type); +if (r) + goto out_err_no_arch_destroy_vm; +#ifdef CONFIG_HAVE_KVM_IRQFD INIT_HLIST_HEAD (&kvm->irq_ack_notifier_list);#endif
@@ -972,10 +972,10 @@ static struct kvm *kvm_create_vm(unsigned long type)mmu_notifier_unregister (&kvm->mmu_notifier, current->mm);#endif out_err_no_mmu_notifier: -hardware_disable_all (); -out_err_no_disable:kvm_arch_destroy_vm (kvm); out_err_no_arch_destroy_vm: +hardware_disable_all (); +out_err_no_disable:WARN_ON_ONCE (!refcount_dec_and_test (&kvm->users_count));for (i =0 ; i <KVM_NR_BUSES ; i++)kfree (kvm_get_bus (kvm, i));
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 13/69] KVM: Enable hardware before doing arch VM initialization
virt/kvm/kvm_main.c static struct kvm *kvm_create_vm(unsigned long typ
Quoted 24 lines · On 03/07/21 00:04, [email protected] wrote:
This also provides consistent ordering between
kvm_create_vm()andkvm_destroy_vm()with respect to callingkvm_arch_destroy_vm()andhardware_disable_all().Signed-off-by Sean Christopherson<[email protected]>Signed-off-by Isaku Yamahata<[email protected]>--- virt/kvm/kvm_main.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)virt/kvm/kvm_main.c +1
-1@@ -923,7 +923,7 @@ static struct kvm *kvm_create_vm(unsigned long type)struct kvm_memslots *slots =kvm_alloc_memslots ();if (!slots) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable;/* Generations must be different for each address space. */ slots->generation = i;rcu_assign_pointer (kvm->memslots[i], slots);@@ -933,19 +933,19 @@ static struct kvm *kvm_create_vm(unsigned long typ
RFC PATCH v2 14/69 KVM: x86: Split core of hypercall emulation to helper function
In reply to [email protected]From: Sean Christopherson <[email protected]>
By necessity, TDX will use a different register ABI for hypercalls. Break out the core functionality so that it may be reused for TDX.
--- arch/x86/include/asm/kvm_host.h | 4 +++ arch/x86/kvm/x86.c | 55 ++++++++++++++++++++------------- 2 files changed, 38 insertions(+), 21 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -0
@@ -1667,6 +1667,10 @@ void kvm_vcpu_update_apicv(struct kvm_vcpu *vcpu);void kvm_request_apicv_update (struct kvm *kvm,bool activate,unsigned long bit); +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,
arch/x86/kvm/x86.c +34 -21
77 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 14/69] KVM: x86: Split core of hypercall emulation to helper function
Quoted 102 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
By necessity, TDX will use a different register ABI for hypercalls. Break out the core functionality so that it may be reused for TDX.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 4 +++ arch/x86/kvm/x86.c | 55 ++++++++++++++++++++------------- 2 files changed, 38 insertions(+), 21 deletions(-)arch/x86/include/asm/kvm_host.h +4
-0@@ -1667,6 +1667,10 @@ void kvm_vcpu_update_apicv(struct kvm_vcpu *vcpu);void kvm_request_apicv_update (struct kvm *kvm,bool activate,unsigned long bit); +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit);int kvm_emulate_hypercall (struct kvm_vcpu *vcpu); int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,arch/x86/kvm/x86.c +27
-21@@ -8406,26 +8406,15 @@ static void kvm_sched_yield(struct kvm_vcpu *vcpu, unsigned long dest_id)return ; } -int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) +unsigned long __kvm_emulate_hypercall (struct kvm_vcpu *vcpu,unsigned long nr, +unsigned long a0,unsigned long a1, +unsigned long a2,unsigned long a3, +int op_64_bit) { -unsigned long nr, a0, a1, a2, a3, ret; -int op_64_bit; - -if (kvm_xen_hypercall_enabled (vcpu->kvm)) -return kvm_xen_hypercall (vcpu); - -if (kvm_hv_hypercall_enabled (vcpu)) -return kvm_hv_hypercall (vcpu); - - nr =kvm_rax_read (vcpu); - a0 =kvm_rbx_read (vcpu); - a1 =kvm_rcx_read (vcpu); - a2 =kvm_rdx_read (vcpu); - a3 =kvm_rsi_read (vcpu); +unsigned long ret;trace_kvm_hypercall (nr, a0, a1, a2, a3); - op_64_bit =is_64_bit_mode (vcpu);if (!op_64_bit) { nr &=0xFFFFFFFF ; a0 &=0xFFFFFFFF ;@@ -8434,11 +8423,6 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) a3 &=0xFFFFFFFF ; } -if (static_call (kvm_x86_get_cpl)(vcpu) !=0 ) { - ret = -KVM_EPERM ; - goto out; - } - ret = -KVM_ENOSYS ;switch (nr) {@@ -8475,6 +8459,35 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) ret = -KVM_ENOSYS ;break ; } +return ret; +} +EXPORT_SYMBOL_GPL (__kvm_emulate_hypercall); + +int kvm_emulate_hypercall (struct kvm_vcpu *vcpu) +{ +unsigned long nr, a0, a1, a2, a3, ret; +int op_64_bit; + +if (kvm_xen_hypercall_enabled (vcpu->kvm)) +return kvm_xen_hypercall (vcpu); + +if (kvm_hv_hypercall_enabled (vcpu)) +return kvm_hv_hypercall (vcpu); + + nr =kvm_rax_read (vcpu); + a0 =kvm_rbx_read (vcpu); + a1 =kvm_rcx_read (vcpu); + a2 =kvm_rdx_read (vcpu); + a3 =kvm_rsi_read (vcpu); + + op_64_bit =is_64_bit_mode (vcpu);
arch/x86/kvm/x86.c int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) · line 8493
Quoted 10 lines
arch/x86/kvm/x86.c +7
-0+ +if (static_call (kvm_x86_get_cpl)(vcpu) !=0 ) { + ret = -KVM_EPERM ; + goto out; + } + + ret =__kvm_emulate_hypercall (vcpu, nr, a0, a1, a2, a3, op_64_bit); out:if (!op_64_bit) ret = (u32 )ret;
RFC PATCH v2 15/69 KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
In reply to [email protected]From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
--- arch/x86/kvm/x86.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/x86.c +1 -0
@@ -11962,6 +11962,7 @@ EXPORT_SYMBOL_GPL(kvm_sev_es_string_io);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_entry);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_inj_virq);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_page_fault);
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 15/69] KVM: x86: Export kvm_mmio tracepoint for use by TDX for PV MMIO
Quoted 16 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Later kvm_intel.ko will use it.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/x86.c | 1 + 1 file changed, 1 insertion(+)arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_sev_es_string_io); · line 11968
Quoted 7 lines
arch/x86/kvm/x86.c +1
-0@@ -11962,6 +11962,7 @@ EXPORT_SYMBOL_GPL(kvm_sev_es_string_io);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_entry);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_inj_virq);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_page_fault);
RFC PATCH v2 16/69 KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
In reply to [email protected]From: Sean Christopherson <[email protected]>
Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.
--- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +20 -1
@@ -85,6 +85,9 @@ __MODULE_PARM_TYPE(nx_huge_pages_recovery_ratio, "uint");static bool __read_mostly force_flush_and_sync_on_reuse;module_param_named (flush_on_reuse, force_flush_and_sync_on_reuse, bool,0644 ); +static bool __read_mostly memslot_update_zap_all; +module_param (memslot_update_zap_all, bool,0444 ); +/* * When setting this variable to true it enables Two-Dimensional-Paging * where the hardware walks 2 page tables:
@@ -5480,11 +5483,27 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)return unlikely (!list_empty_careful (&kvm->arch.zapped_obsolete_pages)); } +static void kvm_mmu_zap_memslot (struct kvm *kvm,struct kvm_memory_slot *slot) +{ +/* +* Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst +* case scenario we'll have unused shadow pages lying around until they +* are recycled due to age or when the VM is destroyed. +*/ +write_lock (&kvm->mmu_lock); +slot_handle_level (kvm, slot, kvm_zap_rmapp,PG_LEVEL_4K , +KVM_MAX_HUGEPAGE_LEVEL , true); +write_unlock (&kvm->mmu_lock); +} +static void kvm_mmu_invalidate_zap_pages_in_memslot (struct kvm *kvm,struct kvm_memory_slot *slot,struct kvm_page_track_notifier_node *node) { -kvm_mmu_zap_all_fast (kvm); +if (memslot_update_zap_all) +kvm_mmu_zap_all_fast (kvm); +else +kvm_mmu_zap_memslot (kvm, slot); }void kvm_mmu_init_vm (struct kvm *kvm)
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 16/69] KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
Quoted 46 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)arch/x86/kvm/mmu/mmu.c +16
-0@@ -85,6 +85,9 @@ __MODULE_PARM_TYPE(nx_huge_pages_recovery_ratio, "uint");static bool __read_mostly force_flush_and_sync_on_reuse;module_param_named (flush_on_reuse, force_flush_and_sync_on_reuse, bool,0644 ); +static bool __read_mostly memslot_update_zap_all; +module_param (memslot_update_zap_all, bool,0444 ); +/* * When setting this variable to true it enables Two-Dimensional-Paging * where the hardware walks 2 page tables: @@ -5480,11 +5483,27 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)return unlikely (!list_empty_careful (&kvm->arch.zapped_obsolete_pages)); } +static void kvm_mmu_zap_memslot (struct kvm *kvm,struct kvm_memory_slot *slot) +{ +/* +* Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst +* case scenario we'll have unused shadow pages lying around until they +* are recycled due to age or when the VM is destroyed. +*/ +write_lock (&kvm->mmu_lock); +slot_handle_level (kvm, slot, kvm_zap_rmapp,PG_LEVEL_4K , +KVM_MAX_HUGEPAGE_LEVEL , true); +write_unlock (&kvm->mmu_lock); +} + static void kvm_mmu_invalidate_zap_pages_in_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
arch/x86/kvm/mmu/mmu.c static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm) · line 5509
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +4
-1struct kvm_page_track_notifier_node *node) { -kvm_mmu_zap_all_fast (kvm); +if (memslot_update_zap_all) +kvm_mmu_zap_all_fast (kvm); +else +kvm_mmu_zap_memslot (kvm, slot); }void kvm_mmu_init_vm (struct kvm *kvm)
This is the old patch that broke VFIO for some unknown reason. The commit message should at least say why memslot_update_zap_all is not true by default. Also, IIUC the bug still there with NX hugepage splits disabled, but what if the TDP MMU is enabled?
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 16/69] KVM: x86/mmu: Zap only leaf SPTEs for deleted/moved memslot by default
Quoted 48 lines · Paolo Bonzini
Quoted 47 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Zap only leaf SPTEs when deleting/moving a memslot by default, and add a module param to allow reverting to the old behavior of zapping all SPTEs at all levels and memslots when any memslot is updated.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)arch/x86/kvm/mmu/mmu.c +17
-1@@ -85,6 +85,9 @@ __MODULE_PARM_TYPE(nx_huge_pages_recovery_ratio, "uint");static bool __read_mostly force_flush_and_sync_on_reuse;module_param_named (flush_on_reuse, force_flush_and_sync_on_reuse, bool,0644 ); +static bool __read_mostly memslot_update_zap_all; +module_param (memslot_update_zap_all, bool,0444 ); +/* * When setting this variable to true it enables Two-Dimensional-Paging * where the hardware walks 2 page tables: @@ -5480,11 +5483,27 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)return unlikely (!list_empty_careful (&kvm->arch.zapped_obsolete_pages)); } +static void kvm_mmu_zap_memslot (struct kvm *kvm,struct kvm_memory_slot *slot) +{ +/* +* Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst +* case scenario we'll have unused shadow pages lying around until they +* are recycled due to age or when the VM is destroyed. +*/ +write_lock (&kvm->mmu_lock); +slot_handle_level (kvm, slot, kvm_zap_rmapp,PG_LEVEL_4K , +KVM_MAX_HUGEPAGE_LEVEL , true); +write_unlock (&kvm->mmu_lock); +} +static void kvm_mmu_invalidate_zap_pages_in_memslot (struct kvm *kvm,struct kvm_memory_slot *slot,struct kvm_page_track_notifier_node *node) { -kvm_mmu_zap_all_fast (kvm); +if (memslot_update_zap_all)
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)
Quoted 6 lines
arch/x86/kvm/mmu/mmu.c +3
-0+kvm_mmu_zap_all_fast (kvm); + else +kvm_mmu_zap_memslot (kvm ,slot ); }void kvm_mmu_init_vm (struct kvm *kvm)This is the old patch that broke VFIO for some unknown reason.
Yes, my white whale :-/
Quoted 2 lines
The commit message should at least say why
memslot_update_zap_allis not true by default. Also, IIUC the bug still there with NX hugepage splits disabled,
I strongly suspect the bug is also there with hugepage splits enabled, it's just masked and/or harder to hit.
Quoted 1 line
but what if the TDP MMU is enabled?
This should not be a module param. IIRC, the original code I wrote had it as a per-VM flag that wasn't even exposed to the user, i.e. TDX guests always do the partial flush and non-TDX guests always do the full flush. I think that's the least awful approach if we can't figure out the underlying bug before TDX is ready for inclusion.
RFC PATCH v2 17/69 KVM: Add infrastructure and macro to mark VM as bugged
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- include/linux/kvm_host.h | 29 ++++++++++++++++++++++++++++- virt/kvm/kvm_main.c | 10 +++++----- 2 files changed, 33 insertions(+), 6 deletions(-)
include/linux/kvm_host.h +28 -1
@@ -149,6 +149,7 @@ static inline bool is_error_page(struct page *page)#define KVM_REQ_MMU_RELOAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_UNBLOCK 2#define KVM_REQ_UNHALT 3 +#define KVM_REQ_VM_BUGGED (4 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQUEST_ARCH_BASE 8#define KVM_ARCH_REQ_FLAGS (nr, flags) ({ \
@@ -585,6 +586,8 @@ struct kvm {pid_t userspace_pid;unsigned int max_halt_poll_ns;u32 dirty_ring_size; + +bool vm_bugged; };#define kvm_err (fmt, ...) \
@@ -613,6 +616,31 @@ struct kvm {#define vcpu_err (vcpu, fmt, ...) \ kvm_err("vcpu%i " fmt, (vcpu)->vcpu_id, ## __VA_ARGS__) +bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req); +static inline void kvm_vm_bugged (struct kvm *kvm) +{ + kvm->vm_bugged = true; +kvm_make_all_cpus_request (kvm,KVM_REQ_VM_BUGGED ); +} + +#define KVM_BUG(cond, kvm, fmt...) \ +({ \ + int __ret = (cond); \ + \ + if (WARN_ONCE(__ret && !(kvm)->vm_bugged, fmt)) \ + kvm_vm_bugged(kvm); \ + unlikely(__ret); \ +}) + +#define KVM_BUG_ON (cond, kvm) \ +({ \ + int __ret = (cond); \ + \ + if (WARN_ON_ONCE(__ret && !(kvm)->vm_bugged)) \ + kvm_vm_bugged(kvm); \ + unlikely(__ret); \ +}) +static inline bool kvm_dirty_log_manual_protect_and_init_set (struct kvm *kvm) {return !!(kvm->manual_dirty_log_protect &KVM_DIRTY_LOG_INITIALLY_SET );
@@ -930,7 +958,6 @@ void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc);bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except,unsigned long *vcpu_bitmap,cpumask_var_t tmp); -bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req);bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except); bool kvm_make_cpus_request_mask(struct kvm *kvm, unsigned int req,
virt/kvm/kvm_main.c +5 -5
@@ -3435,7 +3435,7 @@ static long kvm_vcpu_ioctl(struct file *filp,struct kvm_fpu *fpu =NULL ;struct kvm_sregs *kvm_sregs =NULL ; -if (vcpu->kvm->mm != current->mm) +if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)return -EIO ;if (unlikely (_IOC_TYPE (ioctl) !=KVMIO ))
@@ -3641,7 +3641,7 @@ static long kvm_vcpu_compat_ioctl(struct file *filp,void __user *argp =compat_ptr (arg);int r; -if (vcpu->kvm->mm != current->mm) +if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)return -EIO ;switch (ioctl) {
@@ -3707,7 +3707,7 @@ static long kvm_device_ioctl(struct file *filp, unsigned int ioctl, {struct kvm_device *dev = filp->private_data; -if (dev->kvm->mm != current->mm) +if (dev->kvm->mm != current->mm || dev->kvm->vm_bugged)return -EIO ;switch (ioctl) {
@@ -3991,7 +3991,7 @@ static long kvm_vm_ioctl(struct file *filp,void __user *argp = (void __user *)arg;int r; -if (kvm->mm != current->mm) +if (kvm->mm != current->mm || kvm->vm_bugged)return -EIO ;switch (ioctl) {case KVM_CREATE_VCPU :
@@ -4189,7 +4189,7 @@ static long kvm_vm_compat_ioctl(struct file *filp,struct kvm *kvm = filp->private_data;int r; -if (kvm->mm != current->mm) +if (kvm->mm != current->mm || kvm->vm_bugged)return -EIO ;switch (ioctl) {case KVM_GET_DIRTY_LOG : {
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 17/69] KVM: Add infrastructure and macro to mark VM as bugged
Quoted 112 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- include/linux/kvm_host.h | 29 ++++++++++++++++++++++++++++- virt/kvm/kvm_main.c | 10 +++++----- 2 files changed, 33 insertions(+), 6 deletions(-)include/linux/kvm_host.h +28
-1@@ -149,6 +149,7 @@ static inline bool is_error_page(struct page *page)#define KVM_REQ_MMU_RELOAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_UNBLOCK 2#define KVM_REQ_UNHALT 3 +#define KVM_REQ_VM_BUGGED (4 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQUEST_ARCH_BASE 8#define KVM_ARCH_REQ_FLAGS (nr, flags) ({ \@@ -585,6 +586,8 @@ struct kvm {pid_t userspace_pid;unsigned int max_halt_poll_ns;u32 dirty_ring_size; + +bool vm_bugged; };#define kvm_err (fmt, ...) \@@ -613,6 +616,31 @@ struct kvm {#define vcpu_err (vcpu, fmt, ...) \ kvm_err("vcpu%i " fmt, (vcpu)->vcpu_id, ## __VA_ARGS__) +bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req); +static inline void kvm_vm_bugged (struct kvm *kvm) +{ + kvm->vm_bugged = true; +kvm_make_all_cpus_request (kvm,KVM_REQ_VM_BUGGED ); +} + +#define KVM_BUG(cond, kvm, fmt...) \ +({ \ + int __ret = (cond); \ + \ + if (WARN_ONCE(__ret && !(kvm)->vm_bugged, fmt)) \ + kvm_vm_bugged(kvm); \ + unlikely(__ret); \ +}) + +#define KVM_BUG_ON (cond, kvm) \ +({ \ + int __ret = (cond); \ + \ + if (WARN_ON_ONCE(__ret && !(kvm)->vm_bugged)) \ + kvm_vm_bugged(kvm); \ + unlikely(__ret); \ +}) +static inline bool kvm_dirty_log_manual_protect_and_init_set (struct kvm *kvm) {return !!(kvm->manual_dirty_log_protect &KVM_DIRTY_LOG_INITIALLY_SET );@@ -930,7 +958,6 @@ void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc);bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except,unsigned long *vcpu_bitmap,cpumask_var_t tmp); -bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req);bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req,struct kvm_vcpu *except); bool kvm_make_cpus_request_mask(struct kvm *kvm, unsigned int req,virt/kvm/kvm_main.c +4
-4@@ -3435,7 +3435,7 @@ static long kvm_vcpu_ioctl(struct file *filp,struct kvm_fpu *fpu =NULL ;struct kvm_sregs *kvm_sregs =NULL ; -if (vcpu->kvm->mm != current->mm) +if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)return -EIO ;if (unlikely (_IOC_TYPE (ioctl) !=KVMIO ))@@ -3641,7 +3641,7 @@ static long kvm_vcpu_compat_ioctl(struct file *filp,void __user *argp =compat_ptr (arg);int r; -if (vcpu->kvm->mm != current->mm) +if (vcpu->kvm->mm != current->mm || vcpu->kvm->vm_bugged)return -EIO ;switch (ioctl) {@@ -3707,7 +3707,7 @@ static long kvm_device_ioctl(struct file *filp, unsigned int ioctl, {struct kvm_device *dev = filp->private_data; -if (dev->kvm->mm != current->mm) +if (dev->kvm->mm != current->mm || dev->kvm->vm_bugged)return -EIO ;switch (ioctl) {@@ -3991,7 +3991,7 @@ static long kvm_vm_ioctl(struct file *filp,void __user *argp = (void __user *)arg;int r; -if (kvm->mm != current->mm) +if (kvm->mm != current->mm || kvm->vm_bugged)return -EIO ;switch (ioctl) {case KVM_CREATE_VCPU :
virt/kvm/kvm_main.c static long kvm_vm_compat_ioctl(struct file *filp, · line 4195
Quoted 8 lines
virt/kvm/kvm_main.c +1
-1@@ -4189,7 +4189,7 @@ static long kvm_vm_compat_ioctl(struct file *filp,struct kvm *kvm = filp->private_data;int r; -if (kvm->mm != current->mm) +if (kvm->mm != current->mm || kvm->vm_bugged)return -EIO ;switch (ioctl) {case KVM_GET_DIRTY_LOG : {
RFC PATCH v2 18/69 KVM: Export kvm_make_all_cpus_request() for use in marking VMs as bugged
In reply to [email protected]From: Sean Christopherson <[email protected]>
Export kvm_make_all_cpus_request() and hoist the request helper declarations of request up to the KVM_REQ_* definitions in preparation for adding a "VM bugged" framework. The framework will add KVM_BUG() and KVM_BUG_ON() as alternatives to full BUG()/BUG_ON() for cases where KVM has definitely hit a bug (in itself or in silicon) and the VM is all but guaranteed to be hosed. Marking a VM bugged will trigger a request to all vCPUs to allow arch code to forcefully evict each vCPU from its run loop.
--- include/linux/kvm_host.h | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
include/linux/kvm_host.h +9 -9
@@ -158,6 +158,15 @@ static inline bool is_error_page(struct page *page) })#define KVM_ARCH_REQ (nr) KVM_ARCH_REQ_FLAGS(nr, 0) +bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req, +struct kvm_vcpu *except, +unsigned long *vcpu_bitmap,cpumask_var_t tmp); +bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req); +bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req, +struct kvm_vcpu *except); +bool kvm_make_cpus_request_mask (struct kvm *kvm,unsigned int req, +unsigned long *vcpu_bitmap); +#define KVM_USERSPACE_IRQ_SOURCE_ID 0#define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID 1
@@ -616,7 +625,6 @@ struct kvm {#define vcpu_err (vcpu, fmt, ...) \ kvm_err("vcpu%i " fmt, (vcpu)->vcpu_id, ## __VA_ARGS__) -bool kvm_make_all_cpus_request (struct kvm *kvm,unsigned int req);static inline void kvm_vm_bugged (struct kvm *kvm) { kvm->vm_bugged = true;
@@ -955,14 +963,6 @@ void kvm_mmu_free_memory_cache(struct kvm_mmu_memory_cache *mc);void *kvm_mmu_memory_cache_alloc (struct kvm_mmu_memory_cache *mc);#endif -bool kvm_make_vcpus_request_mask (struct kvm *kvm,unsigned int req, -struct kvm_vcpu *except, -unsigned long *vcpu_bitmap,cpumask_var_t tmp); -bool kvm_make_all_cpus_request_except (struct kvm *kvm,unsigned int req, -struct kvm_vcpu *except); -bool kvm_make_cpus_request_mask (struct kvm *kvm,unsigned int req, -unsigned long *vcpu_bitmap); -long kvm_arch_dev_ioctl (struct file *filp,unsigned int ioctl,unsigned long arg); long kvm_arch_vcpu_ioctl(struct file *filp,
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 18/69] KVM: Export kvm_make_all_cpus_request() for use in marking VMs as bugged
Quoted 11 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Export
kvm_make_all_cpus_request()and hoist the request helper declarations of request up to the KVM_REQ_* definitions in preparation for adding a "VM bugged" framework. The framework will addKVM_BUG()andKVM_BUG_ON()as alternatives to full BUG()/BUG_ON() for cases where KVM has definitely hit a bug (in itself or in silicon) and the VM is all but guaranteed to be hosed. Marking a VM bugged will trigger a request to all vCPUs to allow arch code to forcefully evict each vCPU from its run loop.
Quoted 2 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
This should be _before_ patch 17, not after.
Paolo
RFC PATCH v2 19/69 KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the VM
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 23 ++++++++++++++--------- arch/x86/kvm/x86.c | 4 ++++ 3 files changed, 19 insertions(+), 10 deletions(-)
arch/x86/kvm/svm/svm.c +1 -1
@@ -1526,7 +1526,7 @@ static void svm_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu));break ;default : -WARN_ON_ONCE (1 ); +KVM_BUG_ON (1 , vcpu->kvm); } }
arch/x86/kvm/vmx/vmx.c +14 -9
@@ -2360,7 +2360,7 @@ static void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) vcpu->arch.cr4 |=vmcs_readl (GUEST_CR4 ) & guest_owned_bits;break ;default : -WARN_ON_ONCE (1 ); +KVM_BUG_ON (1 , vcpu->kvm);break ; } }
@@ -5062,6 +5062,7 @@ static int handle_cr(struct kvm_vcpu *vcpu)return kvm_complete_insn_gp (vcpu, err);case 3 :WARN_ON_ONCE (enable_unrestricted_guest); + err =kvm_set_cr3 (vcpu, val);return kvm_complete_insn_gp (vcpu, err);case 4 :
@@ -5087,14 +5088,13 @@ static int handle_cr(struct kvm_vcpu *vcpu) }break ;case 2 :/* clts */ -WARN_ONCE (1 ,"Guest should always own CR0.TS" ); -vmx_set_cr0 (vcpu,kvm_read_cr0_bits (vcpu, ~X86_CR0_TS )); -trace_kvm_cr_write (0 ,kvm_read_cr0 (vcpu)); -return kvm_skip_emulated_instruction (vcpu); +KVM_BUG (1 , vcpu->kvm,"Guest always owns CR0.TS" ); +return -EIO ;case 1 :/*mov from cr*/ switch (cr) {case 3 :WARN_ON_ONCE (enable_unrestricted_guest); + val =kvm_read_cr3 (vcpu);kvm_register_write (vcpu, reg, val);trace_kvm_cr_read (cr, val);
@@ -5404,7 +5404,9 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu)static int handle_nmi_window (struct kvm_vcpu *vcpu) { -WARN_ON_ONCE (!enable_vnmi); +if (KVM_BUG_ON (!enable_vnmi, vcpu->kvm)) +return -EIO ; +exec_controls_clearbit (to_vmx (vcpu),CPU_BASED_NMI_WINDOW_EXITING ); ++vcpu->stat.nmi_window_exits;kvm_make_request (KVM_REQ_EVENT , vcpu);
@@ -5960,7 +5962,8 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)* below) should never happen as that means we incorrectly allowed a * nested VM-Enter with an invalid vmcs12. */ -WARN_ON_ONCE (vmx->nested.nested_run_pending); +if (KVM_BUG_ON (vmx->nested.nested_run_pending, vcpu->kvm)) +return -EIO ;/* If guest state is invalid, start emulating */ if (vmx->emulation_required)
@@ -6338,7 +6341,9 @@ static int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu)int max_irr;bool max_irr_updated; -WARN_ON (!vcpu->arch.apicv_active); +if (KVM_BUG_ON (!vcpu->arch.apicv_active, vcpu->kvm)) +return -EIO ; +if (pi_test_on (&vmx->pi_desc)) {pi_clear_on (&vmx->pi_desc);/*
@@ -6421,7 +6426,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu)unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ;gate_desc *desc = (gate_desc *)host_idt_base + vector; -if (WARN_ONCE (!is_external_intr (intr_info), +if (KVM_BUG (!is_external_intr (intr_info), vcpu->kvm,"KVM: unexpected VM-Exit interrupt info: 0x%x" , intr_info))return ;
arch/x86/kvm/x86.c +4 -0
@@ -9153,6 +9153,10 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) }if (kvm_request_pending (vcpu)) { +if (kvm_check_request (KVM_REQ_VM_BUGGED , vcpu)) { + r = -EIO ; + goto out; + }if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) {if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) { r =0 ;
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 19/69] KVM: x86: Use KVM_BUG/KVM_BUG_ON to handle bugs that are fatal to the VM
Quoted 109 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 23 ++++++++++++++--------- arch/x86/kvm/x86.c | 4 ++++ 3 files changed, 19 insertions(+), 10 deletions(-)arch/x86/kvm/svm/svm.c +1
-1@@ -1526,7 +1526,7 @@ static void svm_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg)load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu));break ;default : -WARN_ON_ONCE (1 ); +KVM_BUG_ON (1 , vcpu->kvm); } }arch/x86/kvm/vmx/vmx.c +14
-9@@ -2360,7 +2360,7 @@ static void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) vcpu->arch.cr4 |=vmcs_readl (GUEST_CR4 ) & guest_owned_bits;break ;default : -WARN_ON_ONCE (1 ); +KVM_BUG_ON (1 , vcpu->kvm);break ; } }@@ -5062,6 +5062,7 @@ static int handle_cr(struct kvm_vcpu *vcpu)return kvm_complete_insn_gp (vcpu, err);case 3 :WARN_ON_ONCE (enable_unrestricted_guest); + err =kvm_set_cr3 (vcpu, val);return kvm_complete_insn_gp (vcpu, err);case 4 :@@ -5087,14 +5088,13 @@ static int handle_cr(struct kvm_vcpu *vcpu) }break ;case 2 :/* clts */ -WARN_ONCE (1 ,"Guest should always own CR0.TS" ); -vmx_set_cr0 (vcpu,kvm_read_cr0_bits (vcpu, ~X86_CR0_TS )); -trace_kvm_cr_write (0 ,kvm_read_cr0 (vcpu)); -return kvm_skip_emulated_instruction (vcpu); +KVM_BUG (1 , vcpu->kvm,"Guest always owns CR0.TS" ); +return -EIO ;case 1 :/*mov from cr*/ switch (cr) {case 3 :WARN_ON_ONCE (enable_unrestricted_guest); + val =kvm_read_cr3 (vcpu);kvm_register_write (vcpu, reg, val);trace_kvm_cr_read (cr, val);@@ -5404,7 +5404,9 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu)static int handle_nmi_window (struct kvm_vcpu *vcpu) { -WARN_ON_ONCE (!enable_vnmi); +if (KVM_BUG_ON (!enable_vnmi, vcpu->kvm)) +return -EIO ; +exec_controls_clearbit (to_vmx (vcpu),CPU_BASED_NMI_WINDOW_EXITING ); ++vcpu->stat.nmi_window_exits;kvm_make_request (KVM_REQ_EVENT , vcpu);@@ -5960,7 +5962,8 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)* below) should never happen as that means we incorrectly allowed a * nested VM-Enter with an invalid vmcs12. */ -WARN_ON_ONCE (vmx->nested.nested_run_pending); +if (KVM_BUG_ON (vmx->nested.nested_run_pending, vcpu->kvm)) +return -EIO ;/* If guest state is invalid, start emulating */ if (vmx->emulation_required)@@ -6338,7 +6341,9 @@ static int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu)int max_irr;bool max_irr_updated; -WARN_ON (!vcpu->arch.apicv_active); +if (KVM_BUG_ON (!vcpu->arch.apicv_active, vcpu->kvm)) +return -EIO ; +if (pi_test_on (&vmx->pi_desc)) {pi_clear_on (&vmx->pi_desc);/* @@ -6421,7 +6426,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu)unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ;gate_desc *desc = (gate_desc *)host_idt_base + vector; -if (WARN_ONCE (!is_external_intr (intr_info), +if (KVM_BUG (!is_external_intr (intr_info), vcpu->kvm,"KVM: unexpected VM-Exit interrupt info: 0x%x" , intr_info))return ;arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · line 9162
Quoted 10 lines
arch/x86/kvm/x86.c +4
-0@@ -9153,6 +9153,10 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) }if (kvm_request_pending (vcpu)) { +if (kvm_check_request (KVM_REQ_VM_BUGGED , vcpu)) { + r = -EIO ; + goto out; + }if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) {if (unlikely (!kvm_x86_ops.nested_ops->get_nested_state_pages (vcpu))) { r =0 ;
RFC PATCH v2 20/69 KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -5091,7 +5091,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,if (r ==RET_PF_INVALID ) { r =kvm_mmu_do_page_fault (vcpu, cr2_or_gpa,lower_32_bits (error_code), false); -if (WARN_ON_ONCE (r ==RET_PF_INVALID )) +if (KVM_BUG_ON (r ==RET_PF_INVALID , vcpu->kvm))return -EIO ; }
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 20/69] KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
Quoted 14 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/mmu/mmu.c +0
-0
arch/x86/kvm/mmu/mmu.c int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, · line 5097
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-1@@ -5091,7 +5091,7 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,if (r ==RET_PF_INVALID ) { r =kvm_mmu_do_page_fault (vcpu, cr2_or_gpa,lower_32_bits (error_code), false); -if (WARN_ON_ONCE (r ==RET_PF_INVALID )) +if (KVM_BUG_ON (r ==RET_PF_INVALID , vcpu->kvm))return -EIO ; }
Sean Christopherson
maintainerRe: [RFC PATCH v2 20/69] KVM: x86/mmu: Mark VM as bugged if page fault returns RET_PF_INVALID
In reply to [email protected]Quoted 1 line · On Fri, Jul 02, 2021, [email protected] wrote:
From: Sean Christopherson <[email protected]>
For a changelog:
Mark a VM as bugged instead of simply warning if the core page fault handler unexpectedly returns RET_PF_INVALID. KVM's (undocumented) API for the page fault path does not allow returning RET_PF_INVALID, e.g. a fatal condition should be morphed to -errno.
Quoted 3 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
RFC PATCH v2 21/69 KVM: Add max_vcpus field in common 'struct kvm'
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/arm64/include/asm/kvm_host.h | 3 --- arch/arm64/kvm/arm.c | 7 ++----- arch/arm64/kvm/vgic/vgic-init.c | 6 +++--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 3 ++- 5 files changed, 8 insertions(+), 12 deletions(-)
arch/arm64/include/asm/kvm_host.h +0 -3
@@ -106,9 +106,6 @@ struct kvm_arch {/* VTCR_EL2 value for this VM */ u64 vtcr; -/* The maximum number of vCPUs depends on the used GIC model */ -int max_vcpus; -/* Interrupt controller */ struct vgic_dist vgic;
arch/arm64/kvm/arm.c +2 -5
@@ -145,7 +145,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)kvm_vgic_early_init (kvm);/* The maximum number of VCPUs is limited by the host's GIC model */ - kvm->arch.max_vcpus =kvm_arm_default_max_vcpus (); + kvm->max_vcpus =kvm_arm_default_max_vcpus ();set_default_spectre (kvm);
@@ -220,7 +220,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_MAX_VCPUS :case KVM_CAP_MAX_VCPU_ID :if (kvm) - r = kvm->arch.max_vcpus; + r = kvm->max_vcpus;else r =kvm_arm_default_max_vcpus ();break ;
@@ -299,9 +299,6 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)if (irqchip_in_kernel (kvm) &&vgic_initialized (kvm))return -EBUSY ; -if (id >= kvm->arch.max_vcpus) -return -EINVAL ; -return 0 ; }
arch/arm64/kvm/vgic/vgic-init.c +3 -3
@@ -97,11 +97,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) ret =0 ;if (type ==KVM_DEV_TYPE_ARM_VGIC_V2 ) - kvm->arch.max_vcpus =VGIC_V2_MAX_CPUS ; + kvm->max_vcpus =VGIC_V2_MAX_CPUS ;else - kvm->arch.max_vcpus =VGIC_V3_MAX_CPUS ; + kvm->max_vcpus =VGIC_V3_MAX_CPUS ; -if (atomic_read (&kvm->online_vcpus) > kvm->arch.max_vcpus) { +if (atomic_read (&kvm->online_vcpus) > kvm->max_vcpus) { ret = -E2BIG ; goto out_unlock; }
include/linux/kvm_host.h +1 -0
@@ -544,6 +544,7 @@ struct kvm {* and is accessed atomically. */ atomic_t online_vcpus; +int max_vcpus;int created_vcpus;int last_boosted_vcpu;struct list_head vm_list;
virt/kvm/kvm_main.c +2 -1
@@ -910,6 +910,7 @@ static struct kvm *kvm_create_vm(unsigned long type)mutex_init (&kvm->irq_lock);mutex_init (&kvm->slots_lock);INIT_LIST_HEAD (&kvm->devices); + kvm->max_vcpus =KVM_MAX_VCPUS ;BUILD_BUG_ON (KVM_MEM_SLOTS_NUM >SHRT_MAX );
@@ -3329,7 +3330,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)return -EINVAL ;mutex_lock (&kvm->lock); -if (kvm->created_vcpus ==KVM_MAX_VCPUS ) { +if (kvm->created_vcpus >= kvm->max_vcpus) {mutex_unlock (&kvm->lock);return -EINVAL ; }
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 21/69] KVM: Add max_vcpus field in common 'struct kvm'
Please replace "Add" with "Move" and add a couple lines to the commit message.
Paolo
Quoted 111 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 3 --- arch/arm64/kvm/arm.c | 7 ++----- arch/arm64/kvm/vgic/vgic-init.c | 6 +++--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 3 ++- 5 files changed, 8 insertions(+), 12 deletions(-)arch/arm64/include/asm/kvm_host.h +0
-3@@ -106,9 +106,6 @@ struct kvm_arch {/* VTCR_EL2 value for this VM */ u64 vtcr; -/* The maximum number of vCPUs depends on the used GIC model */ -int max_vcpus; -/* Interrupt controller */ struct vgic_dist vgic;arch/arm64/kvm/arm.c +2
-5@@ -145,7 +145,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)kvm_vgic_early_init (kvm);/* The maximum number of VCPUs is limited by the host's GIC model */ - kvm->arch.max_vcpus =kvm_arm_default_max_vcpus (); + kvm->max_vcpus =kvm_arm_default_max_vcpus ();set_default_spectre (kvm);@@ -220,7 +220,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_MAX_VCPUS :case KVM_CAP_MAX_VCPU_ID :if (kvm) - r = kvm->arch.max_vcpus; + r = kvm->max_vcpus;else r =kvm_arm_default_max_vcpus ();break ;@@ -299,9 +299,6 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)if (irqchip_in_kernel (kvm) &&vgic_initialized (kvm))return -EBUSY ; -if (id >= kvm->arch.max_vcpus) -return -EINVAL ; -return 0 ; }arch/arm64/kvm/vgic/vgic-init.c +3
-3@@ -97,11 +97,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) ret =0 ;if (type ==KVM_DEV_TYPE_ARM_VGIC_V2 ) - kvm->arch.max_vcpus =VGIC_V2_MAX_CPUS ; + kvm->max_vcpus =VGIC_V2_MAX_CPUS ;else - kvm->arch.max_vcpus =VGIC_V3_MAX_CPUS ; + kvm->max_vcpus =VGIC_V3_MAX_CPUS ; -if (atomic_read (&kvm->online_vcpus) > kvm->arch.max_vcpus) { +if (atomic_read (&kvm->online_vcpus) > kvm->max_vcpus) { ret = -E2BIG ; goto out_unlock; }include/linux/kvm_host.h +1
-0@@ -544,6 +544,7 @@ struct kvm {* and is accessed atomically. */ atomic_t online_vcpus; +int max_vcpus;int created_vcpus;int last_boosted_vcpu;struct list_head vm_list;virt/kvm/kvm_main.c +2
-1@@ -910,6 +910,7 @@ static struct kvm *kvm_create_vm(unsigned long type)mutex_init (&kvm->irq_lock);mutex_init (&kvm->slots_lock);INIT_LIST_HEAD (&kvm->devices); + kvm->max_vcpus =KVM_MAX_VCPUS ;BUILD_BUG_ON (KVM_MEM_SLOTS_NUM >SHRT_MAX );@@ -3329,7 +3330,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)return -EINVAL ;mutex_lock (&kvm->lock); -if (kvm->created_vcpus ==KVM_MAX_VCPUS ) { +if (kvm->created_vcpus >= kvm->max_vcpus) {mutex_unlock (&kvm->lock);return -EINVAL ; }
Sean Christopherson
maintainerRe: [RFC PATCH v2 21/69] KVM: Add max_vcpus field in common 'struct kvm'
Quoted 2 lines · Paolo Bonzini
Please replace "Add" with "Move" and add a couple lines to the commit message.
Move arm's per-VM max_vcpus field into the generic "struct kvm", and use it to check vcpus_created in the generic code instead of checking only the hardcoded absolute KVM-wide max. x86 TDX guests will reuse the generic check verbatim, as the max number of vCPUs for a TDX guest is user defined at VM creation and immutable thereafter.
RFC PATCH v2 22/69 KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a capability to effectively allow userspace to query what VM types are supported by KVM.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/uapi/asm/kvm.h | 4 ++++ arch/x86/kvm/svm/svm.c | 6 ++++++ arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/x86.c | 9 ++++++++- include/uapi/linux/kvm.h | 2 ++ tools/arch/x86/include/uapi/asm/kvm.h | 4 ++++ tools/include/uapi/linux/kvm.h | 2 ++ 9 files changed, 35 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -18,6 +18,7 @@ KVM_X86_OP_NULL(hardware_unsetup) KVM_X86_OP_NULL(cpu_has_accelerated_tpr) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -975,6 +975,7 @@ struct kvm_x86_msr_filter {#define APICV_INHIBIT_REASON_X2APIC 5struct kvm_arch { +unsigned long vm_type;unsigned long n_used_mmu_pages;unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;
@@ -1207,6 +1208,7 @@ struct kvm_x86_ops {bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); +bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);
arch/x86/include/uapi/asm/kvm.h +4 -0
@@ -490,4 +490,8 @@ struct kvm_pmu_event_filter {#define KVM_PMU_EVENT_ALLOW 0#define KVM_PMU_EVENT_DENY 1 +#define KVM_X86_LEGACY_VM 0 +#define KVM_X86_SEV_ES_VM 1 +#define KVM_X86_TDX_VM 2 +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/svm/svm.c +6 -0
@@ -4422,6 +4422,11 @@ static void svm_vm_destroy(struct kvm *kvm)sev_vm_destroy (kvm); } +static bool svm_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_LEGACY_VM ; +} +static int svm_vm_init (struct kvm *kvm) {if (!pause_filter_count || !pause_filter_thresh)
@@ -4448,6 +4453,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_free_vcpu, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy,
arch/x86/kvm/vmx/vmx.c +6 -0
@@ -6951,6 +6951,11 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu)return err; } +static bool vmx_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_LEGACY_VM ; +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
@@ -7605,6 +7610,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .cpu_has_accelerated_tpr = report_flexpriority, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init,
arch/x86/kvm/x86.c +8 -1
@@ -3995,6 +3995,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)else r =0 ;break ; +case KVM_CAP_VM_TYPES : + r =BIT (KVM_X86_LEGACY_VM ); +if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) + r |=BIT (KVM_X86_TDX_VM ); +break ;default :break ; }
@@ -10746,9 +10751,11 @@ void kvm_arch_free_vm(struct kvm *kvm)int kvm_arch_init_vm (struct kvm *kvm,unsigned long type) { -if (type) +if (!static_call (kvm_x86_is_vm_type_supported)(type))return -EINVAL ; + kvm->arch.vm_type = type; +INIT_HLIST_HEAD (&kvm->arch.mask_notifier_list);INIT_LIST_HEAD (&kvm->arch.active_mmu_pages);INIT_LIST_HEAD (&kvm->arch.zapped_obsolete_pages);
include/uapi/linux/kvm.h +2 -0
@@ -1084,6 +1084,8 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197#define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_VM_TYPES 1000 +#ifdef KVM_CAP_IRQ_ROUTING struct kvm_irq_routing_irqchip {
tools/arch/x86/include/uapi/asm/kvm.h +4 -0
@@ -490,4 +490,8 @@ struct kvm_pmu_event_filter {#define KVM_PMU_EVENT_ALLOW 0#define KVM_PMU_EVENT_DENY 1 +#define KVM_X86_LEGACY_VM 0 +#define KVM_X86_SEV_ES_VM 1 +#define KVM_X86_TDX_VM 2 +#endif /* _ASM_X86_KVM_H */
tools/include/uapi/linux/kvm.h +2 -0
@@ -1084,6 +1084,8 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197#define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_VM_TYPES 1000 +#ifdef KVM_CAP_IRQ_ROUTING struct kvm_irq_routing_irqchip {
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 22/69] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
arch/x86/include/uapi/asm/kvm.h struct kvm_pmu_event_filter { · line 496
Quoted 6 lines · On 03/07/21 00:04, [email protected] wrote:
#define KVM_PMU_EVENT_DENY 1arch/x86/include/uapi/asm/kvm.h +4
-0+#define KVM_X86_LEGACY_VM 0 +#define KVM_X86_SEV_ES_VM 1 +#define KVM_X86_TDX_VM 2 +
SEV-ES is not needed, and TDX_VM might be reused for SEV-SNP. Also "legacy VM" is not really the right name. Maybe NORMAL/TRUSTED?
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v2 22/69] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
In reply to [email protected]arch/x86/include/asm/kvm_host.h struct kvm_x86_msr_filter { · line 978
Quoted 3 lines · On 03/07/21 00:04, [email protected] wrote:
struct kvm_arch {arch/x86/include/asm/kvm_host.h +1
-0+unsigned long vm_type;
Also why not just int or u8?
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 22/69] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
Quoted 5 lines · Paolo Bonzini
arch/x86/include/asm/kvm_host.h struct kvm_x86_msr_filter { · line 978
Quoted 2 lines · On 03/07/21 00:04, [email protected] wrote:
struct kvm_arch {arch/x86/include/asm/kvm_host.h +1
-0+unsigned long vm_type;Also why not just int or u8?
Heh, because kvm_dev_ioctl_create_vm() takes an "unsigned long" for the type and it felt wrong to store it as something else. Storing it as a smaller field should be fine, I highly doubt we'll get to 256 types anytime soon :-)
I think kvm_x86_ops.is_vm_type_supported() should take the full size though.
Re: [RFC PATCH v2 22/69] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
Quoted 10 lines · Sean Christopherson
Quoted 5 lines · Paolo Bonzini
arch/x86/include/asm/kvm_host.h struct kvm_x86_msr_filter { · line 978
Quoted 2 lines · On 03/07/21 00:04, [email protected] wrote:
struct kvm_arch {arch/x86/include/asm/kvm_host.h +1
-0+unsigned long vm_type;Also why not just int or u8?
Heh, because
kvm_dev_ioctl_create_vm()takes an "unsigned long" for the type and it felt wrong to store it as something else. Storing it as a smaller field should be fine, I highly doubt we'll get to 256 types anytime soon :-)
It's the bit position. We can get only 8 types with u8 actually.
Quoted 3 lines
I think
kvm_x86_ops.is_vm_type_supported()should take the full size though.
Paolo Bonzini
maintainerRe: [RFC PATCH v2 22/69] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
Quoted 6 lines · Xiaoyao Li
Quoted 4 lines
Heh, because
kvm_dev_ioctl_create_vm()takes an "unsigned long" for the type and it felt wrong to store it as something else. Storing it as a smaller field should be fine, I highly doubt we'll get to 256 types anytime soon :-)It's the bit position. We can get only 8 types with u8 actually.
Every architecture defines the meaning, and for x86 we can say it's not a bit position.
Paolo
Re: [RFC PATCH v2 22/69] KVM: x86: Add vm_type to differentiate legacy VMs from protected VMs
Quoted 9 lines · Paolo Bonzini
Quoted 6 lines · Xiaoyao Li
Quoted 4 lines
Heh, because
kvm_dev_ioctl_create_vm()takes an "unsigned long" for the type and it felt wrong to store it as something else. Storing it as a smaller field should be fine, I highly doubt we'll get to 256 types anytime soon :-)It's the bit position. We can get only 8 types with u8 actually.
Every architecture defines the meaning, and for x86 we can say it's not a bit position.
Sorry, I find I was wrong. The types are not bit position but value.
KVM_CAP_VM_TYPES reports the supported vm types using bitmap that bit n represents type value n.
Quoted 2 lines
Paolo
RFC PATCH v2 23/69 KVM: x86: Hoist kvm_dirty_regs check out of sync_regs()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Move the kvm_dirty_regs vs. KVM_SYNC_X86_VALID_FIELDS check out of sync_regs() and into its sole caller, kvm_arch_vcpu_ioctl_run(). This allows a future patch to allow synchronizing select state for protected VMs.
--- arch/x86/kvm/x86.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +2 -4
@@ -9729,7 +9729,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) goto out; } -if (kvm_run->kvm_valid_regs & ~KVM_SYNC_X86_VALID_FIELDS ) { +if ((kvm_run->kvm_valid_regs & ~KVM_SYNC_X86_VALID_FIELDS ) || + (kvm_run->kvm_dirty_regs & ~KVM_SYNC_X86_VALID_FIELDS )) { r = -EINVAL ; goto out; }
@@ -10264,9 +10265,6 @@ static void store_regs(struct kvm_vcpu *vcpu)static int sync_regs (struct kvm_vcpu *vcpu) { -if (vcpu->run->kvm_dirty_regs & ~KVM_SYNC_X86_VALID_FIELDS ) -return -EINVAL ; -if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_REGS ) {__set_regs (vcpu, &vcpu->run->s.regs.regs); vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_REGS ;
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 23/69] KVM: x86: Hoist kvm_dirty_regs check out of sync_regs()
Quoted 29 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Move the
kvm_dirty_regsvs.KVM_SYNC_X86_VALID_FIELDScheck out ofsync_regs()and into its sole caller,kvm_arch_vcpu_ioctl_run(). This allows a future patch to allow synchronizing select state for protected VMs.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/x86.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)arch/x86/kvm/x86.c +2
-1@@ -9729,7 +9729,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) goto out; } -if (kvm_run->kvm_valid_regs & ~KVM_SYNC_X86_VALID_FIELDS ) { +if ((kvm_run->kvm_valid_regs & ~KVM_SYNC_X86_VALID_FIELDS ) || + (kvm_run->kvm_dirty_regs & ~KVM_SYNC_X86_VALID_FIELDS )) { r = -EINVAL ; goto out; }
arch/x86/kvm/x86.c static void store_regs(struct kvm_vcpu *vcpu) · line 10270
Quoted 9 lines
arch/x86/kvm/x86.c +0
-3@@ -10264,9 +10265,6 @@ static void store_regs(struct kvm_vcpu *vcpu)static int sync_regs (struct kvm_vcpu *vcpu) { -if (vcpu->run->kvm_dirty_regs & ~KVM_SYNC_X86_VALID_FIELDS ) -return -EINVAL ; -if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_REGS ) {__set_regs (vcpu, &vcpu->run->s.regs.regs); vcpu->run->kvm_dirty_regs &= ~KVM_SYNC_X86_REGS ;
RFC PATCH v2 24/69 KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add 'guest_state_protected' to mark a VM's state as being protected by hardware/firmware, e.g. SEV-ES or TDX-SEAM. Use the flag to disallow ioctls() and/or flows that attempt to access protected state.
Return an error if userspace attempts to get/set register state for a protected VM, e.g. a non-debug TDX guest. KVM can't provide sane data, it's userspace's responsibility to avoid attempting to read guest state when it's known to be inaccessible.
Retrieving vCPU events is the one exception, as the userspace VMM is allowed to inject NMIs.
--- arch/x86/kvm/x86.c | 104 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 86 insertions(+), 18 deletions(-)
arch/x86/kvm/x86.c +86 -18
@@ -4297,6 +4297,10 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_smi (struct kvm_vcpu *vcpu) { +/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +kvm_make_request (KVM_REQ_SMI , vcpu);return 0 ;
@@ -4343,6 +4347,10 @@ static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,unsigned bank_num = mcg_cap &0xff ;u64 *banks = vcpu->arch.mce_banks; +/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (mce->bank >= bank_num || !(mce->status &MCI_STATUS_VAL ))return -EINVAL ;/*
@@ -4438,7 +4446,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu, vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; events->interrupt.nr = vcpu->arch.interrupt.nr; events->interrupt.soft =0 ; - events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); +if (!vcpu->arch.guest_state_protected) + events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); events->nmi.injected = vcpu->arch.nmi_injected; events->nmi.pending = vcpu->arch.nmi_pending !=0 ;
@@ -4467,11 +4476,17 @@ static void kvm_smm_changed(struct kvm_vcpu *vcpu);static int kvm_vcpu_ioctl_x86_set_vcpu_events (struct kvm_vcpu *vcpu,struct kvm_vcpu_events *events) { -if (events->flags & ~(KVM_VCPUEVENT_VALID_NMI_PENDING - |KVM_VCPUEVENT_VALID_SIPI_VECTOR - |KVM_VCPUEVENT_VALID_SHADOW - |KVM_VCPUEVENT_VALID_SMM - |KVM_VCPUEVENT_VALID_PAYLOAD )) +u32 allowed_flags =KVM_VCPUEVENT_VALID_NMI_PENDING | +KVM_VCPUEVENT_VALID_SIPI_VECTOR | +KVM_VCPUEVENT_VALID_SHADOW | +KVM_VCPUEVENT_VALID_SMM | +KVM_VCPUEVENT_VALID_PAYLOAD ; + +/* TODO: introduce more precise flag */ +if (vcpu->arch.guest_state_protected) + allowed_flags =KVM_VCPUEVENT_VALID_NMI_PENDING ; + +if (events->flags & ~allowed_flags)return -EINVAL ;if (events->flags &KVM_VCPUEVENT_VALID_PAYLOAD ) {
@@ -4552,17 +4567,22 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,return 0 ; } -static void kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, -struct kvm_debugregs *dbgregs) +static int kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, +struct kvm_debugregs *dbgregs) {unsigned long val; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +memcpy (dbgregs->db, vcpu->arch.db,sizeof (vcpu->arch.db));kvm_get_dr (vcpu,6 , &val); dbgregs->dr6 = val; dbgregs->dr7 = vcpu->arch.dr7; dbgregs->flags =0 ;memset (&dbgregs->reserved,0 ,sizeof (dbgregs->reserved)); + +return 0 ; } static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,
@@ -4576,6 +4596,9 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,if (!kvm_dr7_valid (dbgregs->dr7))return -EINVAL ; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +memcpy (vcpu->arch.db, dbgregs->db,sizeof (vcpu->arch.db));kvm_update_dr0123 (vcpu); vcpu->arch.dr6 = dbgregs->dr6;
@@ -4671,11 +4694,14 @@ static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) } } -static void kvm_vcpu_ioctl_x86_get_xsave (struct kvm_vcpu *vcpu, -struct kvm_xsave *guest_xsave) +static int kvm_vcpu_ioctl_x86_get_xsave (struct kvm_vcpu *vcpu, +struct kvm_xsave *guest_xsave) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu) -return ; +return 0 ;if (boot_cpu_has (X86_FEATURE_XSAVE )) {memset (guest_xsave,0 ,sizeof (struct kvm_xsave ));
@@ -4687,6 +4713,8 @@ static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, *(u64 *)&guest_xsave->region[XSAVE_HDR_OFFSET /sizeof (u32)] =XFEATURE_MASK_FPSSE ; } + +return 0 ; }#define XSAVE_MXCSR_OFFSET 24
@@ -4697,6 +4725,9 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu,u64 xstate_bv;u32 mxcsr; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu)return 0 ;
@@ -4722,18 +4753,22 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu,return 0 ; } -static void kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu, -struct kvm_xcrs *guest_xcrs) +static int kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu, +struct kvm_xcrs *guest_xcrs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!boot_cpu_has (X86_FEATURE_XSAVE )) { guest_xcrs->nr_xcrs =0 ; -return ; +return 0 ; } guest_xcrs->nr_xcrs =1 ; guest_xcrs->flags =0 ; guest_xcrs->xcrs[0 ].xcr =XCR_XFEATURE_ENABLED_MASK ; guest_xcrs->xcrs[0 ].value = vcpu->arch.xcr0; +return 0 ; } static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu,
@@ -4741,6 +4776,9 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu, {int i, r =0 ; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!boot_cpu_has (X86_FEATURE_XSAVE ))return -EINVAL ;
@@ -5011,7 +5049,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_GET_DEBUGREGS : {struct kvm_debugregs dbgregs; -kvm_vcpu_ioctl_x86_get_debugregs (vcpu, &dbgregs); + r =kvm_vcpu_ioctl_x86_get_debugregs (vcpu, &dbgregs); +if (r) +break ; r = -EFAULT ; if (copy_to_user(argp, &dbgregs,
@@ -5037,7 +5077,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,if (!u.xsave)break ; -kvm_vcpu_ioctl_x86_get_xsave (vcpu, u.xsave); + r =kvm_vcpu_ioctl_x86_get_xsave (vcpu, u.xsave); +if (r) +break ; r = -EFAULT ;if (copy_to_user (argp, u.xsave,sizeof (struct kvm_xsave )))
@@ -5061,7 +5103,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,if (!u.xcrs)break ; -kvm_vcpu_ioctl_x86_get_xcrs (vcpu, u.xcrs); + r =kvm_vcpu_ioctl_x86_get_xcrs (vcpu, u.xcrs); +if (r) +break ; r = -EFAULT ; if (copy_to_user(argp, u.xcrs,
@@ -9735,6 +9779,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) goto out; } +if (vcpu->arch.guest_state_protected && + (kvm_run->kvm_valid_regs || kvm_run->kvm_dirty_regs)) { + r = -EINVAL ; + goto out; + } +if (kvm_run->kvm_dirty_regs) { r =sync_regs (vcpu);if (r !=0 )
@@ -9765,7 +9815,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) out:kvm_put_guest_fpu (vcpu); -if (kvm_run->kvm_valid_regs) +if (kvm_run->kvm_valid_regs && !vcpu->arch.guest_state_protected)store_regs (vcpu);post_kvm_run_save (vcpu);kvm_sigset_deactivate (vcpu);
@@ -9812,6 +9862,9 @@ static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int kvm_arch_vcpu_ioctl_get_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__get_regs (vcpu, regs);vcpu_put (vcpu);
@@ -9852,6 +9905,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int kvm_arch_vcpu_ioctl_set_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__set_regs (vcpu, regs);vcpu_put (vcpu);
@@ -9912,6 +9968,9 @@ static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs)int kvm_arch_vcpu_ioctl_get_sregs (struct kvm_vcpu *vcpu,struct kvm_sregs *sregs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__get_sregs (vcpu, sregs);vcpu_put (vcpu);
@@ -10112,6 +10171,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, {int ret; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu); ret =__set_sregs (vcpu, sregs);vcpu_put (vcpu);
@@ -10205,6 +10267,9 @@ int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) {struct fxregs_state *fxsave; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu)return 0 ;
@@ -10228,6 +10293,9 @@ int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) {struct fxregs_state *fxsave; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu)return 0 ;
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 24/69] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
Quoted 14 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add '
guest_state_protected' to mark a VM's state as being protected by hardware/firmware, e.g. SEV-ES or TDX-SEAM. Use the flag to disallowioctls()and/or flows that attempt to access protected state.Return an error if userspace attempts to get/set register state for a protected VM, e.g. a non-debug TDX guest. KVM can't provide sane data, it's userspace's responsibility to avoid attempting to read guest state when it's known to be inaccessible.
Retrieving vCPU events is the one exception, as the userspace VMM is allowed to inject NMIs.
Quoted 7 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/x86.c | 104 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 86 insertions(+), 18 deletions(-)
Looks good, but it should be checked whether it breaks QEMU for SEV-ES.
Tom, can you help?
Paolo
Quoted 283 lines
arch/x86/kvm/x86.c +86
-18@@ -4297,6 +4297,10 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_smi (struct kvm_vcpu *vcpu) { +/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +kvm_make_request (KVM_REQ_SMI , vcpu);return 0 ;@@ -4343,6 +4347,10 @@ static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,unsigned bank_num = mcg_cap &0xff ;u64 *banks = vcpu->arch.mce_banks; +/* TODO: use more precise flag */ +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (mce->bank >= bank_num || !(mce->status &MCI_STATUS_VAL ))return -EINVAL ;/* @@ -4438,7 +4446,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu, vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; events->interrupt.nr = vcpu->arch.interrupt.nr; events->interrupt.soft =0 ; - events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); +if (!vcpu->arch.guest_state_protected) + events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); events->nmi.injected = vcpu->arch.nmi_injected; events->nmi.pending = vcpu->arch.nmi_pending !=0 ;@@ -4467,11 +4476,17 @@ static void kvm_smm_changed(struct kvm_vcpu *vcpu);static int kvm_vcpu_ioctl_x86_set_vcpu_events (struct kvm_vcpu *vcpu,struct kvm_vcpu_events *events) { -if (events->flags & ~(KVM_VCPUEVENT_VALID_NMI_PENDING - |KVM_VCPUEVENT_VALID_SIPI_VECTOR - |KVM_VCPUEVENT_VALID_SHADOW - |KVM_VCPUEVENT_VALID_SMM - |KVM_VCPUEVENT_VALID_PAYLOAD )) +u32 allowed_flags =KVM_VCPUEVENT_VALID_NMI_PENDING | +KVM_VCPUEVENT_VALID_SIPI_VECTOR | +KVM_VCPUEVENT_VALID_SHADOW | +KVM_VCPUEVENT_VALID_SMM | +KVM_VCPUEVENT_VALID_PAYLOAD ; + +/* TODO: introduce more precise flag */ +if (vcpu->arch.guest_state_protected) + allowed_flags =KVM_VCPUEVENT_VALID_NMI_PENDING ; + +if (events->flags & ~allowed_flags)return -EINVAL ;if (events->flags &KVM_VCPUEVENT_VALID_PAYLOAD ) {@@ -4552,17 +4567,22 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,return 0 ; } -static void kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, -struct kvm_debugregs *dbgregs) +static int kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, +struct kvm_debugregs *dbgregs) {unsigned long val; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +memcpy (dbgregs->db, vcpu->arch.db,sizeof (vcpu->arch.db));kvm_get_dr (vcpu,6 , &val); dbgregs->dr6 = val; dbgregs->dr7 = vcpu->arch.dr7; dbgregs->flags =0 ;memset (&dbgregs->reserved,0 ,sizeof (dbgregs->reserved)); + +return 0 ; } static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,@@ -4576,6 +4596,9 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,if (!kvm_dr7_valid (dbgregs->dr7))return -EINVAL ; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +memcpy (vcpu->arch.db, dbgregs->db,sizeof (vcpu->arch.db));kvm_update_dr0123 (vcpu); vcpu->arch.dr6 = dbgregs->dr6;@@ -4671,11 +4694,14 @@ static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) } } -static void kvm_vcpu_ioctl_x86_get_xsave (struct kvm_vcpu *vcpu, -struct kvm_xsave *guest_xsave) +static int kvm_vcpu_ioctl_x86_get_xsave (struct kvm_vcpu *vcpu, +struct kvm_xsave *guest_xsave) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu) -return ; +return 0 ;if (boot_cpu_has (X86_FEATURE_XSAVE )) {memset (guest_xsave,0 ,sizeof (struct kvm_xsave ));@@ -4687,6 +4713,8 @@ static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, *(u64 *)&guest_xsave->region[XSAVE_HDR_OFFSET /sizeof (u32)] =XFEATURE_MASK_FPSSE ; } + +return 0 ; }#define XSAVE_MXCSR_OFFSET 24@@ -4697,6 +4725,9 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu,u64 xstate_bv;u32 mxcsr; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu)return 0 ;@@ -4722,18 +4753,22 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct kvm_vcpu *vcpu,return 0 ; } -static void kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu, -struct kvm_xcrs *guest_xcrs) +static int kvm_vcpu_ioctl_x86_get_xcrs (struct kvm_vcpu *vcpu, +struct kvm_xcrs *guest_xcrs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!boot_cpu_has (X86_FEATURE_XSAVE )) { guest_xcrs->nr_xcrs =0 ; -return ; +return 0 ; } guest_xcrs->nr_xcrs =1 ; guest_xcrs->flags =0 ; guest_xcrs->xcrs[0 ].xcr =XCR_XFEATURE_ENABLED_MASK ; guest_xcrs->xcrs[0 ].value = vcpu->arch.xcr0; +return 0 ; } static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu,@@ -4741,6 +4776,9 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(struct kvm_vcpu *vcpu, {int i, r =0 ; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!boot_cpu_has (X86_FEATURE_XSAVE ))return -EINVAL ;@@ -5011,7 +5049,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_GET_DEBUGREGS : {struct kvm_debugregs dbgregs; -kvm_vcpu_ioctl_x86_get_debugregs (vcpu, &dbgregs); + r =kvm_vcpu_ioctl_x86_get_debugregs (vcpu, &dbgregs); +if (r) +break ; r = -EFAULT ; if (copy_to_user(argp, &dbgregs,@@ -5037,7 +5077,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,if (!u.xsave)break ; -kvm_vcpu_ioctl_x86_get_xsave (vcpu, u.xsave); + r =kvm_vcpu_ioctl_x86_get_xsave (vcpu, u.xsave); +if (r) +break ; r = -EFAULT ;if (copy_to_user (argp, u.xsave,sizeof (struct kvm_xsave )))@@ -5061,7 +5103,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,if (!u.xcrs)break ; -kvm_vcpu_ioctl_x86_get_xcrs (vcpu, u.xcrs); + r =kvm_vcpu_ioctl_x86_get_xcrs (vcpu, u.xcrs); +if (r) +break ; r = -EFAULT ; if (copy_to_user(argp, u.xcrs,@@ -9735,6 +9779,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) goto out; } +if (vcpu->arch.guest_state_protected && + (kvm_run->kvm_valid_regs || kvm_run->kvm_dirty_regs)) { + r = -EINVAL ; + goto out; + } +if (kvm_run->kvm_dirty_regs) { r =sync_regs (vcpu);if (r !=0 )@@ -9765,7 +9815,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) out:kvm_put_guest_fpu (vcpu); -if (kvm_run->kvm_valid_regs) +if (kvm_run->kvm_valid_regs && !vcpu->arch.guest_state_protected)store_regs (vcpu);post_kvm_run_save (vcpu);kvm_sigset_deactivate (vcpu);@@ -9812,6 +9862,9 @@ static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int kvm_arch_vcpu_ioctl_get_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__get_regs (vcpu, regs);vcpu_put (vcpu);@@ -9852,6 +9905,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int kvm_arch_vcpu_ioctl_set_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__set_regs (vcpu, regs);vcpu_put (vcpu);@@ -9912,6 +9968,9 @@ static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs)int kvm_arch_vcpu_ioctl_get_sregs (struct kvm_vcpu *vcpu,struct kvm_sregs *sregs) { +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu);__get_sregs (vcpu, sregs);vcpu_put (vcpu);@@ -10112,6 +10171,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, {int ret; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +vcpu_load (vcpu); ret =__set_sregs (vcpu, sregs);vcpu_put (vcpu);@@ -10205,6 +10267,9 @@ int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) {struct fxregs_state *fxsave; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu)return 0 ;@@ -10228,6 +10293,9 @@ int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) {struct fxregs_state *fxsave; +if (vcpu->arch.guest_state_protected) +return -EINVAL ; +if (!vcpu->arch.guest_fpu)return 0 ;
Re: [RFC PATCH v2 24/69] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
Quoted 23 lines · Paolo Bonzini
Quoted 14 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add '
guest_state_protected' to mark a VM's state as being protected by hardware/firmware, e.g. SEV-ES or TDX-SEAM. Use the flag to disallowioctls()and/or flows that attempt to access protected state.Return an error if userspace attempts to get/set register state for a protected VM, e.g. a non-debug TDX guest. KVM can't provide sane data, it's userspace's responsibility to avoid attempting to read guest state when it's known to be inaccessible.
Retrieving vCPU events is the one exception, as the userspace VMM is allowed to inject NMIs.
Quoted 7 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
arch/x86/kvm/x86.c| 104 +++++++++++++++++++++++++++++++++++++--------1 file changed, 86 insertions(+), 18 deletions(-)
Quoted 2 lines
Looks good, but it should be checked whether it breaks QEMU for SEV-ES. Tom, can you help?
Sorry to take so long to get back to you... been really slammed, let me look into this a bit more. But, some quick thoughts...
Offhand, the SMI isn't a problem since SEV-ES doesn't support SMM.
For kvm_vcpu_ioctl_x86_{get,set}_xsave(), can TDX use what was added for SEV-ES:
ed02b213098a ("KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest")
Same for kvm_arch_vcpu_ioctl_{get,set}_fpu().
The changes to kvm_arch_vcpu_ioctl_{get,set}_sregs() might cause issues, since there are specific things allowed in __{get,set}_sregs. But I'll need to dig a bit more on that.
Thanks,
Tom
Quoted 272 lines
Paolo
Quoted 268 lines
arch/x86/kvm/x86.c +4
-0@@ -4297,6 +4297,10 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu) static int kvm_vcpu_ioctl_smi(struct kvm_vcpu *vcpu) { + /* TODO: use more precise flag */ + if (vcpu->arch.guest_state_protected) + return -EINVAL; + kvm_make_request(KVM_REQ_SMI, vcpu); return 0;@@ -4343,6 +4347,10 @@ static int kvm_vcpu_ioctl_x86_set_mce(structkvm_vcpu *vcpu, unsigned bank_num = mcg_cap & 0xff; u64 *banks = vcpu->arch.mce_banks;+ /* TODO: use more precise flag */
+ if (vcpu->arch.guest_state_protected)
+ return-EINVAL;
+
if (mce->bank>=bank_num|| !(mce->status &MCI_STATUS_VAL)) return-EINVAL;
/*arch/x86/kvm/x86.c +2
-1@@ -4438,7 +4446,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu *vcpu, vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; events->interrupt.nr = vcpu->arch.interrupt.nr; events->interrupt.soft = 0; - events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu); + if (!vcpu->arch.guest_state_protected) + events->interrupt.shadow =static_call (kvm_x86_get_interrupt_shadow)(vcpu);
events->nmi.injected=vcpu->arch.nmi_injected;events->nmi.pending=vcpu->arch.nmi_pending!= 0;arch/x86/kvm/x86.c +19
-6@@ -4467,11 +4476,17 @@ static void kvm_smm_changed(struct kvm_vcpu *vcpu); static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu, struct kvm_vcpu_events *events) { - if (events->flags & ~(KVM_VCPUEVENT_VALID_NMI_PENDING - | KVM_VCPUEVENT_VALID_SIPI_VECTOR - | KVM_VCPUEVENT_VALID_SHADOW - | KVM_VCPUEVENT_VALID_SMM - | KVM_VCPUEVENT_VALID_PAYLOAD)) + u32 allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING | + KVM_VCPUEVENT_VALID_SIPI_VECTOR | + KVM_VCPUEVENT_VALID_SHADOW | + KVM_VCPUEVENT_VALID_SMM | + KVM_VCPUEVENT_VALID_PAYLOAD; + + /* TODO: introduce more precise flag */ + if (vcpu->arch.guest_state_protected) + allowed_flags = KVM_VCPUEVENT_VALID_NMI_PENDING; + + if (events->flags & ~allowed_flags) return -EINVAL; if (events->flags & KVM_VCPUEVENT_VALID_PAYLOAD) {@@ -4552,17 +4567,22 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu, return 0; } -static void kvm_vcpu_ioctl_x86_get_debugregs(struct kvm_vcpu *vcpu, - struct kvm_debugregs *dbgregs) +static int kvm_vcpu_ioctl_x86_get_debugregs (struct kvm_vcpu *vcpu, + struct kvm_debugregs *dbgregs) { unsigned long val; + if (vcpu->arch.guest_state_protected) + return -EINVAL; + memcpy(dbgregs->db, vcpu->arch.db, sizeof(vcpu->arch.db)); kvm_get_dr(vcpu, 6, &val); dbgregs->dr6 = val; dbgregs->dr7 = vcpu->arch.dr7; dbgregs->flags = 0; memset(&dbgregs->reserved, 0, sizeof(dbgregs->reserved)); + + return 0; } static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu,@@ -4576,6 +4596,9 @@ static int kvm_vcpu_ioctl_x86_set_debugregs(struct kvm_vcpu *vcpu, if (!kvm_dr7_valid(dbgregs->dr7)) return -EINVAL; + if (vcpu->arch.guest_state_protected) + return -EINVAL; + memcpy(vcpu->arch.db, dbgregs->db, sizeof(vcpu->arch.db)); kvm_update_dr0123(vcpu);
vcpu->arch.dr6=dbgregs->dr6;arch/x86/kvm/x86.c +10
-2@@ -4671,11 +4694,14 @@ static void load_xsave(struct kvm_vcpu *vcpu, u8 *src) } } -static void kvm_vcpu_ioctl_x86_get_xsave(struct kvm_vcpu *vcpu, - struct kvm_xsave *guest_xsave) +static int kvm_vcpu_ioctl_x86_get_xsave (struct kvm_vcpu *vcpu, + struct kvm_xsave *guest_xsave) { + if (vcpu->arch.guest_state_protected) + return -EINVAL; + if (!vcpu->arch.guest_fpu) - return; + return 0; if (boot_cpu_has(X86_FEATURE_XSAVE)) { memset(guest_xsave, 0, sizeof(struct kvm_xsave));@@ -4687,6 +4713,8 @@ static void kvm_vcpu_ioctl_x86_get_xsave(structkvm_vcpu *vcpu, *(u64 *)&guest_xsave->region[XSAVE_HDR_OFFSET / sizeof(u32)] = XFEATURE_MASK_FPSSE; } + + return 0; } #define XSAVE_MXCSR_OFFSET 24@@ -4697,6 +4725,9 @@ static int kvm_vcpu_ioctl_x86_set_xsave(structkvm_vcpu *vcpu, u64 xstate_bv; u32 mxcsr; + if (vcpu->arch.guest_state_protected) + return -EINVAL; + if (!vcpu->arch.guest_fpu) return 0;@@ -4722,18 +4753,22 @@ static int kvm_vcpu_ioctl_x86_set_xsave(struct
kvm_vcpu*vcpu,
return 0;
}
-static voidkvm_vcpu_ioctl_x86_get_xcrs(struct kvm_vcpu *vcpu, - struct kvm_xcrs *guest_xcrs)
+static intkvm_vcpu_ioctl_x86_get_xcrs(struct kvm_vcpu *vcpu, + struct kvm_xcrs *guest_xcrs)
{
+ if (vcpu->arch.guest_state_protected)
+ return-EINVAL;
+
if (!boot_cpu_has(X86_FEATURE_XSAVE)) {guest_xcrs->nr_xcrs= 0;
- return;
+ return 0;
}
guest_xcrs->nr_xcrs= 1;
guest_xcrs->flags= 0;
guest_xcrs->xcrs[0].xcr =XCR_XFEATURE_ENABLED_MASK; guest_xcrs->xcrs[0].value =vcpu->arch.xcr0;
+ return 0;
}
static int kvm_vcpu_ioctl_x86_set_xcrs(structkvm_vcpu*vcpu,arch/x86/kvm/x86.c +2
-0@@ -4741,6 +4776,9 @@ static int kvm_vcpu_ioctl_x86_set_xcrs(structkvm_vcpu *vcpu, { int i, r = 0; + if (vcpu->arch.guest_state_protected) + return -EINVAL; + if (!boot_cpu_has(X86_FEATURE_XSAVE)) return -EINVAL;@@ -5011,7 +5049,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp, case
KVM_GET_DEBUGREGS: {
structkvm_debugregsdbgregs;
-kvm_vcpu_ioctl_x86_get_debugregs(vcpu, &dbgregs); + r =kvm_vcpu_ioctl_x86_get_debugregs(vcpu, &dbgregs); + if (r)
+ break;
r =-EFAULT;
if (copy_to_user(argp, &dbgregs,arch/x86/kvm/x86.c +6
-0@@ -5037,7 +5077,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp, if (!u.xsave) break; - kvm_vcpu_ioctl_x86_get_xsave(vcpu, u.xsave); + r = kvm_vcpu_ioctl_x86_get_xsave(vcpu, u.xsave); + if (r) + break; r = -EFAULT; if (copy_to_user(argp, u.xsave, sizeof(struct kvm_xsave)))@@ -5061,7 +5103,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp, if (!u.xcrs) break; - kvm_vcpu_ioctl_x86_get_xcrs(vcpu, u.xcrs); + r = kvm_vcpu_ioctl_x86_get_xcrs(vcpu, u.xcrs); + if (r) + break; r = -EFAULT; if (copy_to_user(argp, u.xcrs,@@ -9735,6 +9779,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) goto out; }+ if (
vcpu->arch.guest_state_protected&&
+ (kvm_run->kvm_valid_regs||kvm_run->kvm_dirty_regs)) { + r =-EINVAL;
+ goto out;
+ }
+
if (kvm_run->kvm_dirty_regs) {
r =sync_regs(vcpu);
if (r != 0)arch/x86/kvm/x86.c +0
-0@@ -9765,7 +9815,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)out:
kvm_put_guest_fpu(vcpu);
- if (kvm_run->kvm_valid_regs)
+ if (kvm_run->kvm_valid_regs&& !vcpu->arch.guest_state_protected)store_regs(vcpu);
post_kvm_run_save(vcpu);
kvm_sigset_deactivate(vcpu);arch/x86/kvm/x86.c +0
-0@@ -9812,6 +9862,9 @@ static void __get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int
kvm_arch_vcpu_ioctl_get_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
{
+ if (vcpu->arch.guest_state_protected)
+ return-EINVAL;
+
vcpu_load(vcpu);
__get_regs(vcpu, regs);
vcpu_put(vcpu);arch/x86/kvm/x86.c +0
-0@@ -9852,6 +9905,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)int
kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs)
{
+ if (vcpu->arch.guest_state_protected)
+ return-EINVAL;
+
vcpu_load(vcpu);
__set_regs(vcpu, regs);
vcpu_put(vcpu);arch/x86/kvm/x86.c +3
-0@@ -9912,6 +9968,9 @@ static void __get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) { + if (vcpu->arch.guest_state_protected) + return -EINVAL; + vcpu_load(vcpu); __get_sregs(vcpu, sregs);
vcpu_put(vcpu);arch/x86/kvm/x86.c +2
-0@@ -10112,6 +10171,9 @@ int kvm_arch_vcpu_ioctl_set_sregs(structkvm_vcpu *vcpu, { int ret; + if (vcpu->arch.guest_state_protected) + return -EINVAL; + vcpu_load(vcpu); ret = __set_sregs(vcpu, sregs);
vcpu_put(vcpu);arch/x86/kvm/x86.c +2
-0@@ -10205,6 +10267,9 @@ int kvm_arch_vcpu_ioctl_get_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu) { struct fxregs_state *fxsave; + if (vcpu->arch.guest_state_protected) + return -EINVAL; + if (!vcpu->arch.guest_fpu) return 0;@@ -10228,6 +10293,9 @@ int
kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu *vcpu, struct kvm_fpu *fpu)
{
structfxregs_state*fxsave;
+ if (vcpu->arch.guest_state_protected)
+ return-EINVAL;
+
if (!vcpu->arch.guest_fpu)
return 0;
Re: [RFC PATCH v2 24/69] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
Quoted 30 lines · Tom Lendacky
Quoted 23 lines · Paolo Bonzini
Quoted 14 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add '
guest_state_protected' to mark a VM's state as being protected by hardware/firmware, e.g. SEV-ES or TDX-SEAM. Use the flag to disallowioctls()and/or flows that attempt to access protected state.Return an error if userspace attempts to get/set register state for a protected VM, e.g. a non-debug TDX guest. KVM can't provide sane data, it's userspace's responsibility to avoid attempting to read guest state when it's known to be inaccessible.
Retrieving vCPU events is the one exception, as the userspace VMM is allowed to inject NMIs.
Quoted 7 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/x86.c | 104 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 86 insertions(+), 18 deletions(-)Quoted 2 lines
Looks good, but it should be checked whether it breaks QEMU for SEV-ES.
Tom, can you help?Sorry to take so long to get back to you... been really slammed, let me look into this a bit more. But, some quick thoughts...
Quoted 7 lines
Offhand, the SMI isn't a problem since SEV-ES doesn't support SMM.
For kvm_vcpu_ioctl_x86_{get,set}_xsave(), can TDX use what was added for SEV-ES:
ed02b213098a("KVM: SVM: Guest FPU state save/restore not needed for SEV-ES guest")Same for kvm_arch_vcpu_ioctl_{get,set}_fpu().
Tom,
I think what you did in this commit is not so correct. It just silently ignores the ioctls insteaf of returning an error to userspace to tell this IOCTL is not invalid to this VM. E.g., for kvm_arch_vcpu_ioctl_get_fpu(), QEMU just gets it succesful with fpu being all zeros.
So Paolo, what's your point on this?
Quoted 7 lines
The changes to kvm_arch_vcpu_ioctl_{get,set}_sregs() might cause issues, since there are specific things allowed in __{get,set}_sregs. But I'll need to dig a bit more on that.
Thanks,
Tom
Paolo Bonzini
maintainerRe: [RFC PATCH v2 24/69] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
Quoted 4 lines · Xiaoyao Li
Tom,
I think what you did in this commit is not so correct. It just silently ignores the ioctls insteaf of returning an error to userspace to tell this IOCTL is not invalid to this VM. E.g., for
kvm_arch_vcpu_ioctl_get_fpu(), QEMU just gets it succesful with fpu being all zeros.
Yes, it's a "cop out" that removes the need for more complex changes in QEMU.
I think for the get/set registers ioctls KVM_GET/SET_{REGS,SREGS,FPU,XSAVE,XCRS} we need to consider SEV-ES backwards compatibility. This means, at least for now, only apply the restriction to TDX (using a bool-returning function, see the review for 28/69).
For SMM, MCE, vCPU events and for kvm_valid/dirty_regs, it can be done as in this patch.
Paolo
Re: [RFC PATCH v2 24/69] KVM: x86: Introduce "protected guest" concept and block disallowed ioctls
Quoted 4 lines · Paolo Bonzini
Quoted 3 lines · Xiaoyao Li
Tom,
Quoted 8 lines
Quoted 1 line
I think what you did in this commit is not so correct. It just silently ignores the ioctls insteaf of returning an error to userspace to tell this IOCTL is not invalid to this VM. E.g., for
kvm_arch_vcpu_ioctl_get_fpu(), QEMU just gets it succesful with fpu being all zeros.Yes, it's a "cop out" that removes the need for more complex changes in QEMU.
I think for the get/set registers ioctls KVM_GET/SET_{REGS,SREGS,FPU,XSAVE,XCRS} we need to consider SEV-ES backwards compatibility. This means, at least for now, only apply the restriction to TDX (using a bool-returning function, see the review for 28/69).
For SMM, MCE, vCPU events and for kvm_valid/dirty_regs, it can be done as in this patch.
thank you Paolo,
I will go with this direction.
RFC PATCH v2 25/69 KVM: x86: Add per-VM flag to disable direct IRQ injection
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag to disable IRQ injection, which is not supported by TDX.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1063,6 +1063,7 @@ struct kvm_arch {bool exception_payload_enabled;bool bus_lock_detection_enabled; +bool irq_injection_disallowed;/* Deflect RDMSR and WRMSR to user space when they trigger a #GP */ u32 user_space_msr_mask;
arch/x86/kvm/x86.c +3 -1
@@ -4264,7 +4264,8 @@ static int kvm_vcpu_ready_for_interrupt_injection(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_interrupt (struct kvm_vcpu *vcpu,struct kvm_interrupt *irq) { -if (irq->irq >=KVM_NR_INTERRUPTS ) +if (irq->irq >=KVM_NR_INTERRUPTS || + vcpu->kvm->arch.irq_injection_disallowed)return -EINVAL ;if (!irqchip_in_kernel (vcpu->kvm)) {
@@ -8562,6 +8563,7 @@ static int emulator_fix_hypercall(struct x86_emulate_ctxt *ctxt)static int dm_request_for_irq_injection (struct kvm_vcpu *vcpu) {return vcpu->run->request_interrupt_window && + !vcpu->kvm->arch.irq_injection_disallowed &&likely (!pic_in_kernel (vcpu->kvm)); }
Signature · 1 line
2.25.1
RFC PATCH v2 26/69 KVM: x86: Add flag to disallow #MC injection / KVM_X86_SETUP_MCE
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag to disallow MCE injection and reject KVM_X86_SETUP_MCE with -EINVAL when set. TDX doesn't support injecting exceptions, including (virtual) #MCs.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1064,6 +1064,7 @@ struct kvm_arch {bool bus_lock_detection_enabled;bool irq_injection_disallowed; +bool mce_injection_disallowed;/* Deflect RDMSR and WRMSR to user space when they trigger a #GP */ u32 user_space_msr_mask;
arch/x86/kvm/x86.c +7 -7
@@ -4319,15 +4319,16 @@ static int vcpu_ioctl_tpr_access_reporting(struct kvm_vcpu *vcpu,static int kvm_vcpu_ioctl_x86_setup_mce (struct kvm_vcpu *vcpu,u64 mcg_cap) { -int r;unsigned bank_num = mcg_cap &0xff , bank; - r = -EINVAL ; +if (vcpu->kvm->arch.mce_injection_disallowed) +return -EINVAL ; +if (!bank_num || bank_num >KVM_MAX_MCE_BANKS ) - goto out; +return -EINVAL ;if (mcg_cap & ~(kvm_mce_cap_supported |0xff |0xff0000 )) - goto out; - r =0 ; +return -EINVAL ; + vcpu->arch.mcg_cap = mcg_cap;/* Init IA32_MCG_CTL to all 1s */ if (mcg_cap &MCG_CTL_P )
@@ -4337,8 +4338,7 @@ static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu, vcpu->arch.mce_banks[bank*4 ] = ~(u64 )0 ;static_call (kvm_x86_setup_mce)(vcpu); -out: -return r; +return 0 ; } static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu,
Signature · 1 line
2.25.1
RFC PATCH v2 27/69 KVM: x86: Add flag to mark TSC as immutable (for TDX)
In reply to [email protected]From: Sean Christopherson <[email protected]>
The TSC for TDX1 guests is fixed at TD creation time. Add tsc_immutable to reflect that the TSC of the guest cannot be changed in any way, and use it to short circuit all paths that lead to one of the myriad TSC adjustment flows.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 35 +++++++++++++++++++++++++-------- 2 files changed, 28 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1044,6 +1044,7 @@ struct kvm_arch {int audit_point;#endif +bool tsc_immutable;bool backwards_tsc_observed;bool boot_vcpu_runs_old_kvmclock;u32 bsp_vcpu_id;
arch/x86/kvm/x86.c +27 -8
@@ -2184,7 +2184,9 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale)u64 ratio;/* Guest TSC same frequency as host TSC? */ -if (!scale) { +if (!scale || vcpu->kvm->arch.tsc_immutable) { +if (scale) +pr_warn_ratelimited ("Guest TSC immutable, scaling not supported\n" ); vcpu->arch.tsc_scaling_ratio = kvm_default_tsc_scaling_ratio;return 0 ; }
@@ -2360,6 +2362,9 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data)bool already_matched;bool synchronizing = false; +if (WARN_ON_ONCE (vcpu->kvm->arch.tsc_immutable)) +return ; +raw_spin_lock_irqsave (&kvm->arch.tsc_write_lock, flags); offset =kvm_compute_tsc_offset (vcpu, data); ns =get_kvmclock_base_ns ();
@@ -2791,6 +2796,10 @@ static int kvm_guest_time_update(struct kvm_vcpu *v)u8 pvclock_flags;bool use_master_clock; +/* Unable to update guest time if the TSC is immutable. */ +if (ka->tsc_immutable) +return 0 ; + kernel_ns =0 ; host_tsc =0 ;
@@ -4142,7 +4151,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (tsc_delta <0 )mark_tsc_unstable ("KVM discovered backwards TSC" ); -if (kvm_check_tsc_unstable ()) { +if (kvm_check_tsc_unstable () && + !vcpu->kvm->arch.tsc_immutable) {u64 offset =kvm_compute_tsc_offset (vcpu, vcpu->arch.last_guest_tsc);kvm_vcpu_write_tsc_offset (vcpu, offset);
@@ -4156,7 +4166,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)* On a host with synchronized TSC, there is no need to update * kvmclock on vcpu->cpu migration */ -if (!vcpu->kvm->arch.use_master_clock || vcpu->cpu ==-1 ) +if ((!vcpu->kvm->arch.use_master_clock || vcpu->cpu ==-1 ) && + !vcpu->kvm->arch.tsc_immutable)kvm_make_request (KVM_REQ_GLOBAL_CLOCK_UPDATE , vcpu);if (vcpu->cpu != cpu)kvm_make_request (KVM_REQ_MIGRATE_TIMER , vcpu);
@@ -5126,10 +5137,11 @@ long kvm_arch_vcpu_ioctl(struct file *filp,break ; }case KVM_SET_TSC_KHZ : { -u32 user_tsc_khz; +u32 user_tsc_khz = (u32 )arg; r = -EINVAL ; - user_tsc_khz = (u32 )arg; +if (vcpu->kvm->arch.tsc_immutable) + goto out;if (kvm_has_tsc_control && user_tsc_khz >= kvm_max_guest_tsc_khz)
@@ -10499,9 +10511,12 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)if (mutex_lock_killable (&vcpu->mutex))return ; -vcpu_load (vcpu); -kvm_synchronize_tsc (vcpu,0 ); -vcpu_put (vcpu); + +if (!kvm->arch.tsc_immutable) { +vcpu_load (vcpu); +kvm_synchronize_tsc (vcpu,0 ); +vcpu_put (vcpu); + }/* poll control enabled by default */ vcpu->arch.msr_kvm_poll_control =1 ;
@@ -10696,6 +10711,10 @@ int kvm_arch_hardware_enable(void)if (backwards_tsc) {u64 delta_cyc = max_tsc - local_tsc;list_for_each_entry (kvm, &vm_list, vm_list) { +if (vcpu->kvm->arch.tsc_immutable) { +pr_warn_ratelimited ("Backwards TSC observed and guest with immutable TSC active\n" ); +continue ; + } kvm->arch.backwards_tsc_observed = true;kvm_for_each_vcpu (i, vcpu, kvm) { vcpu->arch.tsc_offset_adjustment += delta_cyc;
Signature · 1 line
2.25.1
RFC PATCH v2 28/69 KVM: Add per-VM flag to mark read-only memory as unsupported
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a flag for TDX to flag RO memory as unsupported and propagate it to KVM_MEM_READONLY to allow reporting RO memory as unsupported on a per-VM basis. TDX1 doesn't expose permission bits to the VMM in the SEPT tables, i.e. doesn't support read-only private memory.
--- arch/x86/kvm/x86.c | 4 +++- include/linux/kvm_host.h | 4 ++++ virt/kvm/kvm_main.c | 8 +++++--- 3 files changed, 12 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +3 -1
@@ -3897,7 +3897,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_ASYNC_PF_INT :case KVM_CAP_GET_TSC_KHZ :case KVM_CAP_KVMCLOCK_CTRL : -case KVM_CAP_READONLY_MEM :case KVM_CAP_HYPERV_TIME :case KVM_CAP_IOAPIC_POLARITY_IGNORED :case KVM_CAP_TSC_DEADLINE_TIMER :
@@ -4009,6 +4008,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) r |=BIT (KVM_X86_TDX_VM );break ; +case KVM_CAP_READONLY_MEM : + r = kvm && kvm->readonly_mem_unsupported ?0 :1 ; +break ;default :break ; }
include/linux/kvm_host.h +4 -0
@@ -597,6 +597,10 @@ struct kvm {unsigned int max_halt_poll_ns;u32 dirty_ring_size; +#ifdef __KVM_HAVE_READONLY_MEM +bool readonly_mem_unsupported; +#endif +bool vm_bugged; };
virt/kvm/kvm_main.c +5 -3
@@ -1258,12 +1258,14 @@ static void update_memslots(struct kvm_memslots *slots, } } -static int check_memory_region_flags (const struct kvm_userspace_memory_region *mem) +static int check_memory_region_flags (struct kvm *kvm, +const struct kvm_userspace_memory_region *mem) {u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEM - valid_flags |=KVM_MEM_READONLY ; +if (!kvm->readonly_mem_unsupported) + valid_flags |=KVM_MEM_READONLY ;#endif if (mem->flags & ~valid_flags)
@@ -1436,7 +1438,7 @@ int __kvm_set_memory_region(struct kvm *kvm,int as_id, id;int r; - r =check_memory_region_flags (mem); + r =check_memory_region_flags (kvm, mem);if (r)return r;
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 28/69] KVM: Add per-VM flag to mark read-only memory as unsupported
Quoted 76 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a flag for TDX to flag RO memory as unsupported and propagate it to
KVM_MEM_READONLYto allow reporting RO memory as unsupported on a per-VM basis. TDX1 doesn't expose permission bits to the VMM in the SEPT tables, i.e. doesn't support read-only private memory.Signed-off-by Isaku Yamahata <[email protected]>Co-developed-by Sean Christopherson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/x86.c | 4 +++- include/linux/kvm_host.h | 4 ++++ virt/kvm/kvm_main.c | 8 +++++--- 3 files changed, 12 insertions(+), 4 deletions(-)arch/x86/kvm/x86.c +3
-1@@ -3897,7 +3897,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_ASYNC_PF_INT :case KVM_CAP_GET_TSC_KHZ :case KVM_CAP_KVMCLOCK_CTRL : -case KVM_CAP_READONLY_MEM :case KVM_CAP_HYPERV_TIME :case KVM_CAP_IOAPIC_POLARITY_IGNORED :case KVM_CAP_TSC_DEADLINE_TIMER :@@ -4009,6 +4008,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) r |=BIT (KVM_X86_TDX_VM );break ; +case KVM_CAP_READONLY_MEM : + r = kvm && kvm->readonly_mem_unsupported ?0 :1 ; +break ;default :break ; }include/linux/kvm_host.h +4
-0@@ -597,6 +597,10 @@ struct kvm {unsigned int max_halt_poll_ns;u32 dirty_ring_size; +#ifdef __KVM_HAVE_READONLY_MEM +bool readonly_mem_unsupported; +#endif +bool vm_bugged; };virt/kvm/kvm_main.c +4
-2@@ -1258,12 +1258,14 @@ static void update_memslots(struct kvm_memslots *slots, } } -static int check_memory_region_flags (const struct kvm_userspace_memory_region *mem) +static int check_memory_region_flags (struct kvm *kvm, +const struct kvm_userspace_memory_region *mem) {u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEM - valid_flags |=KVM_MEM_READONLY ; +if (!kvm->readonly_mem_unsupported) + valid_flags |=KVM_MEM_READONLY ;#endif if (mem->flags & ~valid_flags)
virt/kvm/kvm_main.c int __kvm_set_memory_region(struct kvm *kvm, · line 1444
Quoted 8 lines
virt/kvm/kvm_main.c +1
-1@@ -1436,7 +1438,7 @@ int __kvm_set_memory_region(struct kvm *kvm,int as_id, id;int r; - r =check_memory_region_flags (mem); + r =check_memory_region_flags (kvm, mem);if (r)return r;
For all these flags, which of these limitations will be common to SEV-ES and SEV-SNP (ExtINT injection, MCE injection, changing TSC, read-only memory, dirty logging)? Would it make sense to use vm_type instead of all of them? I guess this also guides the choice of whether to use a single vm-type for TDX and SEV-SNP or two. Probably two is better, and there can be static inline bool functions to derive the support flags from the vm-type.
Paolo
Re: [RFC PATCH v2 28/69] KVM: Add per-VM flag to mark read-only memory as unsupported
Quoted 76 lines · Paolo Bonzini
Quoted 75 lines · On 03/07/21 00:04, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a flag for TDX to flag RO memory as unsupported and propagate it to
KVM_MEM_READONLYto allow reporting RO memory as unsupported on a per-VM basis. TDX1 doesn't expose permission bits to the VMM in the SEPT tables, i.e. doesn't support read-only private memory.Signed-off-by Isaku Yamahata <[email protected]>Co-developed-by Sean Christopherson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>---
arch/x86/kvm/x86.c | 4 +++-
include/linux/kvm_host.h| 4 ++++
virt/kvm/kvm_main.c | 8 +++++---3 files changed, 12 insertions(+), 4 deletions(-)arch/x86/kvm/x86.c +0
-0@@ -3897,7 +3897,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case
KVM_CAP_ASYNC_PF_INT:
caseKVM_CAP_GET_TSC_KHZ:
caseKVM_CAP_KVMCLOCK_CTRL:
- caseKVM_CAP_READONLY_MEM:
caseKVM_CAP_HYPERV_TIME:
caseKVM_CAP_IOAPIC_POLARITY_IGNORED:
caseKVM_CAP_TSC_DEADLINE_TIMER:arch/x86/kvm/x86.c +3
-0@@ -4009,6 +4008,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) if (static_call(kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM)) r |= BIT(KVM_X86_TDX_VM); break; + case KVM_CAP_READONLY_MEM: + r = kvm && kvm->readonly_mem_unsupported ? 0 : 1; + break;default:
break;
}include/linux/kvm_host.h +4
-0@@ -597,6 +597,10 @@ struct kvm { unsigned int max_halt_poll_ns; u32 dirty_ring_size; +#ifdef __KVM_HAVE_READONLY_MEM + bool readonly_mem_unsupported; +#endif + bool vm_bugged; };virt/kvm/kvm_main.c +4
-2@@ -1258,12 +1258,14 @@ static void update_memslots(struct kvm_memslots *slots, } } -static int check_memory_region_flags (const struct kvm_userspace_memory_region *mem) +static int check_memory_region_flags (struct kvm *kvm, + const struct kvm_userspace_memory_region *mem) { u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES; #ifdef __KVM_HAVE_READONLY_MEM - valid_flags |= KVM_MEM_READONLY; + if (!kvm->readonly_mem_unsupported) + valid_flags |= KVM_MEM_READONLY; #endif if (mem->flags & ~valid_flags)@@ -1436,7 +1438,7 @@ int __kvm_set_memory_region(struct kvm *kvm, int as_id, id;
Quoted 9 lines
virt/kvm/kvm_main.c int __kvm_set_memory_region(struct kvm *kvm, · line 1444
Quoted 6 lines
virt/kvm/kvm_main.c +1
-1int r; - r = check_memory_region_flags(mem); + r = check_memory_region_flags(kvm, mem); if (r) return r;For all these flags, which of these limitations will be common to SEV-ES and SEV-SNP (ExtINT injection, MCE injection, changing TSC, read-only memory, dirty logging)? Would it make sense to use
vm_typeinstead of all of them? I guess this also guides the choice of whether to use a single vm-type for TDX and SEV-SNP or two. Probably two is better, and there can be static inline bool functions to derive the support flags from the vm-type.
The SEV-ES does not need any of these flags. However, with SEV-SNP, we may able use the ExtINT injection.
-Brijesh
RFC PATCH v2 29/69 KVM: Add per-VM flag to disable dirty logging of memslots for TDs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag for TDX to mark dirty logging as unsupported.
--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-)
include/linux/kvm_host.h +1 -0
@@ -597,6 +597,7 @@ struct kvm {unsigned int max_halt_poll_ns;u32 dirty_ring_size; +bool dirty_log_unsupported;#ifdef __KVM_HAVE_READONLY_MEMbool readonly_mem_unsupported;#endif
virt/kvm/kvm_main.c +4 -1
@@ -1261,7 +1261,10 @@ static void update_memslots(struct kvm_memslots *slots,static int check_memory_region_flags (struct kvm *kvm,const struct kvm_userspace_memory_region *mem) { -u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ; +u32 valid_flags =0 ; + +if (!kvm->dirty_log_unsupported) + valid_flags |=KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEMif (!kvm->readonly_mem_unsupported)
Signature · 1 line
2.25.1
RFC PATCH v2 30/69 KVM: x86: Add per-VM flag to disable in-kernel I/O APIC and level routes
In reply to [email protected]From: Kai Huang <[email protected]>
Add a flag to let TDX disallow the in-kernel I/O APIC, level triggered routes for a userspace I/O APIC, and anything else that relies on being able to intercept EOIs. TDX-SEAM does not allow intercepting EOI.
Note, technically KVM could partially emulate the I/O APIC by allowing only edge triggered interrupts, but that adds a lot of complexity for basically zero benefit. Ideally KVM wouldn't even allow I/O APIC route reservation, but disabling that is a train wreck for Qemu.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/ioapic.c | 4 ++++ arch/x86/kvm/irq_comm.c | 9 +++++++-- arch/x86/kvm/lapic.c | 3 ++- arch/x86/kvm/x86.c | 6 ++++++ 5 files changed, 20 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1054,6 +1054,7 @@ struct kvm_arch {enum kvm_irqchip_mode irqchip_mode;u8 nr_reserved_ioapic_pins; +bool eoi_intercept_unsupported;bool disabled_lapic_found;
arch/x86/kvm/ioapic.c +4 -0
@@ -311,6 +311,10 @@ void kvm_arch_post_irq_ack_notifier_list_update(struct kvm *kvm) {if (!ioapic_in_kernel (kvm))return ; + +if (WARN_ON_ONCE (kvm->arch.eoi_intercept_unsupported)) +return ; +kvm_make_scan_ioapic_request (kvm); }
arch/x86/kvm/irq_comm.c +7 -2
@@ -123,7 +123,12 @@ EXPORT_SYMBOL_GPL(kvm_set_msi_irq);static inline bool kvm_msi_route_invalid (struct kvm *kvm,struct kvm_kernel_irq_routing_entry *e) { -return kvm->arch.x2apic_format && (e->msi.address_hi &0xff ); +struct msi_msg msg = { .address_lo = e->msi.address_lo, + .address_hi = e->msi.address_hi, + .data = e->msi.data }; +return (kvm->arch.eoi_intercept_unsupported && + msg.arch_data.is_level) || + (kvm->arch.x2apic_format && (msg.address_hi &0xff )); } int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
@@ -385,7 +390,7 @@ int kvm_setup_empty_irq_routing(struct kvm *kvm)void kvm_arch_post_irq_routing_update (struct kvm *kvm) { -if (!irqchip_split (kvm)) +if (!irqchip_split (kvm) || kvm->arch.eoi_intercept_unsupported)return ;kvm_make_scan_ioapic_request (kvm); }
arch/x86/kvm/lapic.c +2 -1
@@ -278,7 +278,8 @@ void kvm_recalculate_apic_map(struct kvm *kvm)if (old)call_rcu (&old->rcu, kvm_apic_map_free); -kvm_make_scan_ioapic_request (kvm); +if (!kvm->arch.eoi_intercept_unsupported) +kvm_make_scan_ioapic_request (kvm); }static inline void apic_set_spiv (struct kvm_lapic *apic,u32 val)
arch/x86/kvm/x86.c +6 -0
@@ -5744,6 +5744,9 @@ long kvm_arch_vm_ioctl(struct file *filp, goto create_irqchip_unlock; r = -EINVAL ; +if (kvm->arch.eoi_intercept_unsupported) + goto create_irqchip_unlock; +if (kvm->created_vcpus) goto create_irqchip_unlock;
@@ -5774,6 +5777,9 @@ long kvm_arch_vm_ioctl(struct file *filp, u.pit_config.flags =KVM_PIT_SPEAKER_DUMMY ; goto create_pit;case KVM_CREATE_PIT2 : + r = -EINVAL ; +if (kvm->arch.eoi_intercept_unsupported) + goto out; r = -EFAULT ;if (copy_from_user (&u.pit_config, argp,sizeof (struct kvm_pit_config )))
Signature · 1 line
2.25.1
RFC PATCH v2 31/69 KVM: x86: add per-VM flags to disable SMI/INIT/SIPI
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a flag to let TDX disallow to inject interrupt with delivery mode of SMI/INIT/SIPI. add a check to reject SMI/INIT interrupt delivery mode.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/irq_comm.c | 4 ++++ arch/x86/kvm/x86.c | 3 +-- 3 files changed, 7 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1055,6 +1055,8 @@ struct kvm_arch {enum kvm_irqchip_mode irqchip_mode;u8 nr_reserved_ioapic_pins;bool eoi_intercept_unsupported; +bool smm_unsupported; +bool init_sipi_unsupported;bool disabled_lapic_found;
arch/x86/kvm/irq_comm.c +4 -0
@@ -128,6 +128,10 @@ static inline bool kvm_msi_route_invalid(struct kvm *kvm, .data = e->msi.data };return (kvm->arch.eoi_intercept_unsupported && msg.arch_data.is_level) || + (kvm->arch.smm_unsupported && + msg.arch_data.delivery_mode ==APIC_DELIVERY_MODE_SMI ) || + (kvm->arch.init_sipi_unsupported && + msg.arch_data.delivery_mode ==APIC_DELIVERY_MODE_INIT ) || (kvm->arch.x2apic_format && (msg.address_hi &0xff )); }
arch/x86/kvm/x86.c +1 -2
@@ -4311,8 +4311,7 @@ static int kvm_vcpu_ioctl_nmi(struct kvm_vcpu *vcpu)static int kvm_vcpu_ioctl_smi (struct kvm_vcpu *vcpu) { -/* TODO: use more precise flag */ -if (vcpu->arch.guest_state_protected) +if (vcpu->kvm->arch.smm_unsupported)return -EINVAL ;kvm_make_request (KVM_REQ_SMI , vcpu);
Signature · 1 line
2.25.1
RFC PATCH v2 32/69 KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
In reply to [email protected]From: Sean Christopherson <[email protected]>
Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between ioctls(), e.g. this forces userspace to configure CPUID before MSRs. And for TDX, vCPUs will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.
--- arch/x86/kvm/x86.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +5 -2
@@ -437,8 +437,11 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info) {enum lapic_mode old_mode =kvm_get_apic_mode (vcpu);enum lapic_mode new_mode =kvm_apic_mode (msr_info->data); -u64 reserved_bits =kvm_vcpu_reserved_gpa_bits_raw (vcpu) |0x2ff | - (guest_cpuid_has (vcpu,X86_FEATURE_X2APIC ) ?0 :X2APIC_ENABLE ); +u64 reserved_bits =kvm_vcpu_reserved_gpa_bits_raw (vcpu) |0x2ff ; + +if (!msr_info->host_initiated && + !guest_cpuid_has (vcpu,X86_FEATURE_X2APIC )) + reserved_bits |=X2APIC_ENABLE ;if ((msr_info->data & reserved_bits) !=0 || new_mode ==LAPIC_MODE_INVALID )return 1 ;
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 32/69] KVM: x86: Allow host-initiated WRMSR to set X2APIC regardless of CPUID
Quoted 4 lines · On 03/07/21 00:04, [email protected] wrote:
Let userspace, or in the case of TDX, KVM itself, enable X2APIC even if X2APIC is not reported as supported in the guest's CPU model. KVM generally does not force specific ordering between
ioctls(), e.g. this forces userspace to configure CPUID before MSRs.
You already have to do this, see for example MSR_IA32_PERF_CAPABILITIES:
struct kvm_msr_entry msr_ent = {.index = msr, .data = 0};
if (!msr_info->host_initiated)
return 1;
if (guest_cpuid_has(vcpu, X86_FEATURE_PDCM) && kvm_get_msr_feature(&msr_ent))
return 1;
if (data & ~msr_ent.data)
return 1;Is this patch necessary? If not, I think it can be dropped.
Paolo
Quoted 3 lines
And for TDX, vCPUs
will always run with X2APIC enabled, e.g. KVM will want/need to enable X2APIC from time zero.
RFC PATCH v2 33/69 KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add hooks to cache and flush GPRs and invoke them from KVM_GET_REGS and KVM_SET_REGS respecitively. TDX will use the hooks to read/write GPRs from TDX-SEAM on-demand (for debug TDs).
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 6 ++++++ 2 files changed, 8 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1248,6 +1248,8 @@ struct kvm_x86_ops {void (*set_gdt)(struct kvm_vcpu *vcpu,struct desc_ptr *dt);void (*sync_dirty_debug_regs)(struct kvm_vcpu *vcpu);void (*set_dr7)(struct kvm_vcpu *vcpu,unsigned long value); +void (*cache_gprs)(struct kvm_vcpu *vcpu); +void (*flush_gprs)(struct kvm_vcpu *vcpu);void (*cache_reg)(struct kvm_vcpu *vcpu,enum kvm_reg reg);unsigned long (*get_rflags)(struct kvm_vcpu *vcpu);void (*set_rflags)(struct kvm_vcpu *vcpu,unsigned long rflags);
arch/x86/kvm/x86.c +6 -0
@@ -9850,6 +9850,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)static void __get_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (kvm_x86_ops.cache_gprs) + kvm_x86_ops.cache_gprs (vcpu); +if (vcpu->arch.emulate_regs_need_sync_to_vcpu) {/* * We are here if userspace calls get_regs() in the middle of
@@ -9924,6 +9927,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) vcpu->arch.exception.pending = false; +if (kvm_x86_ops.flush_gprs) + kvm_x86_ops.flush_gprs (vcpu); +kvm_make_request (KVM_REQ_EVENT , vcpu); }
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 33/69] KVM: x86: Add kvm_x86_ops .cache_gprs() and .flush_gprs()
Quoted 43 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add hooks to cache and flush GPRs and invoke them from
KVM_GET_REGSandKVM_SET_REGSrespecitively. TDX will use the hooks to read/write GPRs from TDX-SEAM on-demand (for debug TDs).Cc Tom Lendacky <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 6 ++++++ 2 files changed, 8 insertions(+)arch/x86/include/asm/kvm_host.h +2
-0@@ -1248,6 +1248,8 @@ struct kvm_x86_ops {void (*set_gdt)(struct kvm_vcpu *vcpu,struct desc_ptr *dt);void (*sync_dirty_debug_regs)(struct kvm_vcpu *vcpu);void (*set_dr7)(struct kvm_vcpu *vcpu,unsigned long value); +void (*cache_gprs)(struct kvm_vcpu *vcpu); +void (*flush_gprs)(struct kvm_vcpu *vcpu);void (*cache_reg)(struct kvm_vcpu *vcpu,enum kvm_reg reg);unsigned long (*get_rflags)(struct kvm_vcpu *vcpu);void (*set_rflags)(struct kvm_vcpu *vcpu,unsigned long rflags);arch/x86/kvm/x86.c +3
-0@@ -9850,6 +9850,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)static void __get_regs (struct kvm_vcpu *vcpu,struct kvm_regs *regs) { +if (kvm_x86_ops.cache_gprs) + kvm_x86_ops.cache_gprs (vcpu); +if (vcpu->arch.emulate_regs_need_sync_to_vcpu) {/* * We are here if userspace calls get_regs() in the middle of
arch/x86/kvm/x86.c static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) · line 9935
Quoted 9 lines
arch/x86/kvm/x86.c +3
-0@@ -9924,6 +9927,9 @@ static void __set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) vcpu->arch.exception.pending = false; +if (kvm_x86_ops.flush_gprs) + kvm_x86_ops.flush_gprs (vcpu); +kvm_make_request (KVM_REQ_EVENT , vcpu); }
RFC PATCH v2 34/69 KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 12 ++++++++++++ 2 files changed, 14 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1377,7 +1377,9 @@ struct kvm_x86_ops {int (*pre_leave_smm)(struct kvm_vcpu *vcpu,const char *smstate);void (*enable_smi_window)(struct kvm_vcpu *vcpu); +int (*mem_enc_op_dev)(void __user *argp);int (*mem_enc_op)(struct kvm *kvm,void __user *argp); +int (*mem_enc_op_vcpu)(struct kvm_vcpu *vcpu,void __user *argp);int (*mem_enc_reg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*mem_enc_unreg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*vm_copy_enc_context_from)(struct kvm *kvm,unsigned int source_fd);
arch/x86/kvm/x86.c +12 -0
@@ -4109,6 +4109,12 @@ long kvm_arch_dev_ioctl(struct file *filp,case KVM_GET_SUPPORTED_HV_CPUID : r =kvm_ioctl_get_supported_hv_cpuid (NULL , argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_dev) + goto out; + r = kvm_x86_ops.mem_enc_op_dev (argp); +break ;default : r = -EINVAL ;break ;
@@ -5263,6 +5269,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp, break; }#endif +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_vcpu) + goto out; + r = kvm_x86_ops.mem_enc_op_vcpu (vcpu, argp); +break ;default : r = -EINVAL ; }
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 34/69] KVM: x86: Add support for vCPU and device-scoped KVM_MEMORY_ENCRYPT_OP
Quoted 44 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 12 ++++++++++++ 2 files changed, 14 insertions(+)arch/x86/include/asm/kvm_host.h +2
-0@@ -1377,7 +1377,9 @@ struct kvm_x86_ops {int (*pre_leave_smm)(struct kvm_vcpu *vcpu,const char *smstate);void (*enable_smi_window)(struct kvm_vcpu *vcpu); +int (*mem_enc_op_dev)(void __user *argp);int (*mem_enc_op)(struct kvm *kvm,void __user *argp); +int (*mem_enc_op_vcpu)(struct kvm_vcpu *vcpu,void __user *argp);int (*mem_enc_reg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*mem_enc_unreg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*vm_copy_enc_context_from)(struct kvm *kvm,unsigned int source_fd);arch/x86/kvm/x86.c +6
-0@@ -4109,6 +4109,12 @@ long kvm_arch_dev_ioctl(struct file *filp,case KVM_GET_SUPPORTED_HV_CPUID : r =kvm_ioctl_get_supported_hv_cpuid (NULL , argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_dev) + goto out; + r = kvm_x86_ops.mem_enc_op_dev (argp); +break ;default : r = -EINVAL ;break ;@@ -5263,6 +5269,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,break ; }
arch/x86/kvm/x86.c long kvm_arch_vcpu_ioctl(struct file *filp, · line 5280
Quoted 10 lines
arch/x86/kvm/x86.c +6
-0#endif +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_vcpu) + goto out; + r = kvm_x86_ops.mem_enc_op_vcpu (vcpu, argp); +break ;default : r = -EINVAL ; }
RFC PATCH v2 35/69 KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resource, and finally free per-VM resources.
Note, this knowingly creates a discrepancy in nomenclature for SVM as svm_vm_teardown() invokes avic_vm_destroy() and sev_vm_destroy(). Moving the now-misnamed functions or renaming them is left to a future patch so as not to introduce a functional change for SVM.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/svm.c | 8 +++++++- arch/x86/kvm/vmx/vmx.c | 12 ++++++++++++ arch/x86/kvm/x86.c | 3 ++- 5 files changed, 23 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -20,6 +20,7 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP(vm_teardown) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1217,6 +1217,7 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*vm_teardown)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);/* Create, but do not attach this VCPU */
arch/x86/kvm/svm/svm.c +7 -1
@@ -4416,12 +4416,17 @@ static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)sev_vcpu_deliver_sipi_vector (vcpu, vector); } -static void svm_vm_destroy (struct kvm *kvm) +static void svm_vm_teardown (struct kvm *kvm) {avic_vm_destroy (kvm);sev_vm_destroy (kvm); } +static void svm_vm_destroy (struct kvm *kvm) +{ + +} +static bool svm_is_vm_type_supported (unsigned long type) {return type ==KVM_X86_LEGACY_VM ;
@@ -4456,6 +4461,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, + .vm_teardown = svm_vm_teardown, .vm_destroy = svm_vm_destroy, .prepare_guest_switch = svm_prepare_guest_switch,
arch/x86/kvm/vmx/vmx.c +12 -0
@@ -6995,6 +6995,16 @@ static int vmx_vm_init(struct kvm *kvm)return 0 ; } +static void vmx_vm_teardown (struct kvm *kvm) +{ + +} + +static void vmx_vm_destroy (struct kvm *kvm) +{ + +} +static int __init vmx_check_processor_compat (void ) {struct vmcs_config vmcs_conf;
@@ -7613,6 +7623,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init, + .vm_teardown = vmx_vm_teardown, + .vm_destroy = vmx_vm_destroy, .vcpu_create = vmx_create_vcpu, .vcpu_free = vmx_free_vcpu,
arch/x86/kvm/x86.c +2 -1
@@ -11043,7 +11043,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)__x86_set_memory_region (kvm,TSS_PRIVATE_MEMSLOT ,0 ,0 );mutex_unlock (&kvm->slots_lock); } -static_call_cond (kvm_x86_vm_destroy)(kvm); +static_call (kvm_x86_vm_teardown)(kvm);kvm_free_msr_filter (srcu_dereference_check (kvm->arch.msr_filter, &kvm->srcu,1 ));kvm_pic_destroy (kvm);kvm_ioapic_destroy (kvm);
@@ -11054,6 +11054,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_destroy)(kvm); }void kvm_arch_free_memslot (struct kvm *kvm,struct kvm_memory_slot *slot)
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 35/69] KVM: x86: Introduce vm_teardown() hook in kvm_arch_vm_destroy()
arch/x86/kvm/svm/svm.c static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) · line 4423
Quoted 6 lines · On 03/07/21 00:04, [email protected] wrote:
arch/x86/kvm/svm/svm.c +1
-1-static void svm_vm_destroy (struct kvm *kvm) +static void svm_vm_teardown (struct kvm *kvm) {avic_vm_destroy (kvm);sev_vm_destroy (kvm); }
Please keep "destroy" as is and use "free" from the final step.
arch/x86/kvm/svm/svm.c static void svm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) · line 4427
Quoted 3 lines
arch/x86/kvm/svm/svm.c +3
-0+static void svm_vm_destroy (struct kvm *kvm) +{ +
Please remove the empty lines.
Paolo
Quoted 65 lines
arch/x86/kvm/svm/svm.c +3
-0+} +static bool svm_is_vm_type_supported (unsigned long type) {return type ==KVM_X86_LEGACY_VM ;@@ -4456,6 +4461,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, + .vm_teardown = svm_vm_teardown, .vm_destroy = svm_vm_destroy, .prepare_guest_switch = svm_prepare_guest_switch,arch/x86/kvm/vmx/vmx.c +12
-0@@ -6995,6 +6995,16 @@ static int vmx_vm_init(struct kvm *kvm)return 0 ; } +static void vmx_vm_teardown (struct kvm *kvm) +{ + +} + +static void vmx_vm_destroy (struct kvm *kvm) +{ + +} +static int __init vmx_check_processor_compat (void ) {struct vmcs_config vmcs_conf;@@ -7613,6 +7623,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init, + .vm_teardown = vmx_vm_teardown, + .vm_destroy = vmx_vm_destroy, .vcpu_create = vmx_create_vcpu, .vcpu_free = vmx_free_vcpu,arch/x86/kvm/x86.c +2
-1@@ -11043,7 +11043,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)__x86_set_memory_region (kvm,TSS_PRIVATE_MEMSLOT ,0 ,0 );mutex_unlock (&kvm->slots_lock); } -static_call_cond (kvm_x86_vm_destroy)(kvm); +static_call (kvm_x86_vm_teardown)(kvm);kvm_free_msr_filter (srcu_dereference_check (kvm->arch.msr_filter, &kvm->srcu,1 ));kvm_pic_destroy (kvm);kvm_ioapic_destroy (kvm);@@ -11054,6 +11054,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_destroy)(kvm); }void kvm_arch_free_memslot (struct kvm *kvm,struct kvm_memory_slot *slot)
RFC PATCH v2 36/69 KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behavior
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a flag, KVM_DEBUGREG_AUTO_SWITCHED_GUEST, to skip saving/restoring DRs irrespective of any other flags. TDX-SEAM unconditionally saves and restores guest DRs and reset to architectural INIT state on TD exit. So, KVM needs to save host DRs before TD enter without restoring guest DRs and restore host DRs after TD exit.
Opportunistically convert the KVM_DEBUGREG_* definitions to use BIT().
--- arch/x86/include/asm/kvm_host.h | 11 ++++++++--- arch/x86/kvm/x86.c | 3 ++- 2 files changed, 10 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +8 -3
@@ -488,9 +488,14 @@ struct kvm_pmu {struct kvm_pmu_ops ;enum { -KVM_DEBUGREG_BP_ENABLED =1 , -KVM_DEBUGREG_WONT_EXIT =2 , -KVM_DEBUGREG_RELOAD =4 , +KVM_DEBUGREG_BP_ENABLED =BIT (0 ), +KVM_DEBUGREG_WONT_EXIT =BIT (1 ), +KVM_DEBUGREG_RELOAD =BIT (2 ), +/* +* Guest debug registers are saved/restored by hardware on exit from +* or enter guest. KVM needn't switch them. +*/ +KVM_DEBUGREG_AUTO_SWITCH_GUEST =BIT (3 ), };struct kvm_mtrr_range {
arch/x86/kvm/x86.c +2 -1
@@ -9441,7 +9441,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return (); -if (unlikely (vcpu->arch.switch_db_regs)) { +if (unlikely (vcpu->arch.switch_db_regs & ~KVM_DEBUGREG_AUTO_SWITCH_GUEST )) {set_debugreg (0 ,7 );set_debugreg (vcpu->arch.eff_db[0 ],0 );set_debugreg (vcpu->arch.eff_db[1 ],1 );
@@ -9473,6 +9473,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)*/ if (unlikely (vcpu->arch.switch_db_regs &KVM_DEBUGREG_WONT_EXIT )) {WARN_ON (vcpu->guest_debug &KVM_GUESTDBG_USE_HW_BP ); +WARN_ON (vcpu->arch.switch_db_regs &KVM_DEBUGREG_AUTO_SWITCH_GUEST );static_call (kvm_x86_sync_dirty_debug_regs)(vcpu);kvm_update_dr0123 (vcpu);kvm_update_dr7 (vcpu);
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 36/69] KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behavior
Quoted 29 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add a flag,
KVM_DEBUGREG_AUTO_SWITCHED_GUEST, to skip saving/restoring DRs irrespective of any other flags. TDX-SEAM unconditionally saves and restores guest DRs and reset to architectural INIT state on TD exit. So, KVM needs to save host DRs before TD enter without restoring guest DRs and restore host DRs after TD exit.Opportunistically convert the KVM_DEBUGREG_* definitions to use
BIT().Reported-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Co-developed-by Chao Gao <[email protected]>Signed-off-by Chao Gao <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 11 ++++++++--- arch/x86/kvm/x86.c | 3 ++- 2 files changed, 10 insertions(+), 4 deletions(-)arch/x86/include/asm/kvm_host.h +0
-1@@ -488,9 +488,14 @@ struct kvm_pmu {struct kvm_pmu_ops ;enum { -KVM_DEBUGREG_BP_ENABLED =1 ,
arch/x86/include/asm/kvm_host.h struct kvm_pmu { · line 498
Quoted 10 lines
arch/x86/include/asm/kvm_host.h +8
-2-KVM_DEBUGREG_WONT_EXIT =2 , - KVM_DEBUGREG_RELOAD = 4, +KVM_DEBUGREG_BP_ENABLED =BIT (0 ), +KVM_DEBUGREG_WONT_EXIT =BIT (1 ), +KVM_DEBUGREG_RELOAD =BIT (2 ), +/* +* Guest debug registers are saved/restored by hardware on exit from +* or enter guest. KVM needn't switch them. +*/ + KVM_DEBUGREG_AUTO_SWITCH_GUEST = BIT(3),
Maybe remove "_GUEST"? Apart from that,
Paolo
Quoted 25 lines
arch/x86/include/asm/kvm_host.h +0
-0};struct kvm_mtrr_range {arch/x86/kvm/x86.c +2
-1@@ -9441,7 +9441,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (test_thread_flag (TIF_NEED_FPU_LOAD ))switch_fpu_return (); -if (unlikely (vcpu->arch.switch_db_regs)) { +if (unlikely (vcpu->arch.switch_db_regs & ~KVM_DEBUGREG_AUTO_SWITCH_GUEST )) {set_debugreg (0 ,7 );set_debugreg (vcpu->arch.eff_db[0 ],0 );set_debugreg (vcpu->arch.eff_db[1 ],1 );@@ -9473,6 +9473,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)*/ if (unlikely (vcpu->arch.switch_db_regs &KVM_DEBUGREG_WONT_EXIT )) {WARN_ON (vcpu->guest_debug &KVM_GUESTDBG_USE_HW_BP ); +WARN_ON (vcpu->arch.switch_db_regs &KVM_DEBUGREG_AUTO_SWITCH_GUEST );static_call (kvm_x86_sync_dirty_debug_regs)(vcpu);kvm_update_dr0123 (vcpu);kvm_update_dr7 (vcpu);
RFC PATCH v2 37/69 KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Return true for kvm_vcpu_has_events() if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).
--- arch/x86/kvm/x86.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +3 -1
@@ -11341,7 +11341,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)if (kvm_arch_interrupt_allowed (vcpu) && (kvm_cpu_has_interrupt (vcpu) || -kvm_guest_apic_has_interrupt (vcpu))) +kvm_guest_apic_has_interrupt (vcpu) || + (vcpu->arch.apicv_active && + kvm_x86_ops.dy_apicv_has_pending_interrupt (vcpu))))return true;if (kvm_hv_has_stimer_pending (vcpu))
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 37/69] KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
Quoted 21 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Return true for
kvm_vcpu_has_events()if the vCPU has a pending APICv interrupt to support TDX's usage of APICv. Unlike VMX, TDX doesn't have access to vmcs.GUEST_INTR_STATUS and so can't emulate posted interrupts, i.e. needs to generate a posted interrupt and more importantly can't manually move requested interrupts into the vIRR (which it also doesn't have access to).Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/x86.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)arch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) · line 11349
Quoted 10 lines
arch/x86/kvm/x86.c +3
-1@@ -11341,7 +11341,9 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu)if (kvm_arch_interrupt_allowed (vcpu) && (kvm_cpu_has_interrupt (vcpu) || -kvm_guest_apic_has_interrupt (vcpu))) +kvm_guest_apic_has_interrupt (vcpu) || + (vcpu->arch.apicv_active && + kvm_x86_ops.dy_apicv_has_pending_interrupt (vcpu))))return true;if (kvm_hv_has_stimer_pending (vcpu))
Please remove "dy_" from the name of the callback, and use the static call. Also, if it makes sense, please consider using the same test as for patch 38 to choose *between* either kvm_cpu_has_interrupt() + kvm_guest_apic_has_interrupt() or kvm_x86_ops.dy_apicv_has_pending_interrupt().
Paolo
RFC PATCH v2 38/69 KVM: x86: Add option to force LAPIC expiration wait
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add an option to skip the IRR check in kvm_wait_lapic_expire(). This will be used by TDX to wait if there is an outstanding notification for a TD, i.e. a virtual interrupt is being triggered via posted interrupt processing. KVM TDX doesn't emulate PI processing, i.e. there will never be a bit set in IRR/ISR, so the default behavior for APICv of querying the IRR doesn't work as intended.
--- arch/x86/kvm/lapic.c | 4 ++-- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)
arch/x86/kvm/lapic.c +2 -2
@@ -1622,12 +1622,12 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)__wait_lapic_expire (vcpu, tsc_deadline - guest_tsc); } -void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu) +void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu,bool force_wait) {if (lapic_in_kernel (vcpu) && vcpu->arch.apic->lapic_timer.expired_tscdeadline && vcpu->arch.apic->lapic_timer.timer_advance_ns && -lapic_timer_int_injected (vcpu)) + (force_wait ||lapic_timer_int_injected (vcpu)))__kvm_wait_lapic_expire (vcpu); }EXPORT_SYMBOL_GPL (kvm_wait_lapic_expire);
arch/x86/kvm/lapic.h +1 -1
@@ -233,7 +233,7 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)bool kvm_apic_pending_eoi (struct kvm_vcpu *vcpu,int vector); -void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu); +void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu,bool force_wait);void kvm_bitmap_or_dest_vcpus (struct kvm *kvm,struct kvm_lapic_irq *irq,unsigned long *vcpu_bitmap);
arch/x86/kvm/svm/svm.c +1 -1
@@ -3774,7 +3774,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)clgi ();kvm_load_guest_xsave_state (vcpu); -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu, false);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -6727,7 +6727,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)if (enable_preemption_timer)vmx_update_hv_timer (vcpu); -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu, false);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 38/69] KVM: x86: Add option to force LAPIC expiration wait
Quoted 11 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add an option to skip the IRR check in
kvm_wait_lapic_expire(). This will be used by TDX to wait if there is an outstanding notification for a TD, i.e. a virtual interrupt is being triggered via posted interrupt processing. KVM TDX doesn't emulate PI processing, i.e. there will never be a bit set in IRR/ISR, so the default behavior for APICv of querying the IRR doesn't work as intended.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
Is there a better (existing after the previous patches) flag to test, or possibly can it use vm_type following the suggestion I gave for patch 28?
Paolo
Quoted 66 lines
--- arch/x86/kvm/lapic.c | 4 ++-- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)arch/x86/kvm/lapic.c +2
-2@@ -1622,12 +1622,12 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)__wait_lapic_expire (vcpu, tsc_deadline - guest_tsc); } -void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu) +void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu,bool force_wait) {if (lapic_in_kernel (vcpu) && vcpu->arch.apic->lapic_timer.expired_tscdeadline && vcpu->arch.apic->lapic_timer.timer_advance_ns && -lapic_timer_int_injected (vcpu)) + (force_wait ||lapic_timer_int_injected (vcpu)))__kvm_wait_lapic_expire (vcpu); }EXPORT_SYMBOL_GPL (kvm_wait_lapic_expire);arch/x86/kvm/lapic.h +1
-1@@ -233,7 +233,7 @@ static inline int kvm_lapic_latched_init(struct kvm_vcpu *vcpu)bool kvm_apic_pending_eoi (struct kvm_vcpu *vcpu,int vector); -void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu); +void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu,bool force_wait);void kvm_bitmap_or_dest_vcpus (struct kvm *kvm,struct kvm_lapic_irq *irq,unsigned long *vcpu_bitmap);arch/x86/kvm/svm/svm.c +1
-1@@ -3774,7 +3774,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)clgi ();kvm_load_guest_xsave_state (vcpu); -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu, false);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if arch/x86/kvm/vmx/vmx.c +1
-1@@ -6727,7 +6727,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)if (enable_preemption_timer)vmx_update_hv_timer (vcpu); -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu, false);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
Sean Christopherson
maintainerRe: [RFC PATCH v2 38/69] KVM: x86: Add option to force LAPIC expiration wait
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add an option to skip the IRR check in
kvm_wait_lapic_expire(). This will be used by TDX to wait if there is an outstanding notification for a TD, i.e. a virtual interrupt is being triggered via posted interrupt processing. KVM TDX doesn't emulate PI processing, i.e. there will never be a bit set in IRR/ISR, so the default behavior for APICv of querying the IRR doesn't work as intended.
Quoted 5 lines
Quoted 2 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Is there a better (existing after the previous patches) flag to test, or possibly can it use
vm_typefollowing the suggestion I gave for patch 28?
Not sure if there's a "better" flag, but there's most definitely a flag somewhere that will suffice :-)
RFC PATCH v2 39/69 KVM: x86: Add guest_supported_xss placholder
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a per-vcpu placeholder for the support XSS of the guest so that the TDX configuration code doesn't need to hack in manual computation of the supported XSS. KVM XSS enabling is currently being upstreamed, i.e. guest_supported_xss will no longer be a placeholder by the time TDX is ready for upstreaming (hopefully).
--- arch/x86/include/asm/kvm_host.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -656,6 +656,7 @@ struct kvm_vcpu_arch {u64 xcr0;u64 guest_supported_xcr0; +u64 guest_supported_xss;struct kvm_pio_request pio;void *pio_data;
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 39/69] KVM: x86: Add guest_supported_xss placholder
Quoted 20 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add a per-vcpu placeholder for the support XSS of the guest so that the TDX configuration code doesn't need to hack in manual computation of the supported XSS. KVM XSS enabling is currently being upstreamed, i.e.
guest_supported_xsswill no longer be a placeholder by the time TDX is ready for upstreaming (hopefully).Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + 1 file changed, 1 insertion(+)arch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_vcpu_arch { · line 662
Quoted 7 lines
arch/x86/include/asm/kvm_host.h +1
-0@@ -656,6 +656,7 @@ struct kvm_vcpu_arch {u64 xcr0;u64 guest_supported_xcr0; +u64 guest_supported_xss;struct kvm_pio_request pio;void *pio_data;
Please remove guest_supported_xss from patch 66 for now, instead.
Paolo
RFC PATCH v2 40/69 KVM: Export kvm_is_reserved_pfn() for use by TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX will use kvm_is_reserved_pfn() to prevent installing a reserved PFN int SEPT. Or rather, to prevent such an attempt, as reserved PFNs are not covered by TDMRs.
--- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+)
virt/kvm/kvm_main.c +1 -0
@@ -188,6 +188,7 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)return true; } +EXPORT_SYMBOL_GPL (kvm_is_reserved_pfn);bool kvm_is_transparent_hugepage (kvm_pfn_t pfn) {
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 40/69] KVM: Export kvm_is_reserved_pfn() for use by TDX
Quoted 18 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TDX will use
kvm_is_reserved_pfn()to prevent installing a reserved PFN int SEPT. Or rather, to prevent such an attempt, as reserved PFNs are not covered by TDMRs.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- virt/kvm/kvm_main.c | 1 + 1 file changed, 1 insertion(+)virt/kvm/kvm_main.c +0
-0
virt/kvm/kvm_main.c bool kvm_is_reserved_pfn(kvm_pfn_t pfn) · line 194
Quoted 7 lines
virt/kvm/kvm_main.c +1
-0@@ -188,6 +188,7 @@ bool kvm_is_reserved_pfn(kvm_pfn_t pfn)return true; } +EXPORT_SYMBOL_GPL (kvm_is_reserved_pfn);bool kvm_is_transparent_hugepage (kvm_pfn_t pfn) {
As before, there's no problem in squashing this in the patch that introduces the use of kvm_is_reserved_pfn. You could also move kvm_is_reserved_pfn and kvm_is_zone_device_pfn to a .h file.
Paolo
RFC PATCH v2 41/69 KVM: x86: Add infrastructure for stolen GPA bits
In reply to [email protected]From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.
With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faults
Since there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN, kvm_handle_hva() based operations will be applied to both aliases.
In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
--- arch/x86/kvm/mmu.h | 26 ++++++++++ arch/x86/kvm/mmu/mmu.c | 86 ++++++++++++++++++++++----------- arch/x86/kvm/mmu/mmu_internal.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 25 ++++++---- 4 files changed, 101 insertions(+), 37 deletions(-)
arch/x86/kvm/mmu.h +26 -0
@@ -232,4 +232,30 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu);int kvm_mmu_post_init_vm (struct kvm *kvm);void kvm_mmu_pre_destroy_vm (struct kvm *kvm); +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +/* Currently there are no stolen bits in KVM */ +return 0 ; +} + +static inline gfn_t vcpu_gfn_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gfn_stolen_mask (vcpu->kvm); +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return kvm_gfn_stolen_mask (kvm) <<PAGE_SHIFT ; +} + +static inline gpa_t vcpu_gpa_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gpa_stolen_mask (vcpu->kvm); +} + +static inline gfn_t vcpu_gpa_to_gfn_unalias (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +return (gpa >>PAGE_SHIFT ) & ~vcpu_gfn_stolen_mask (vcpu); +} +#endif
arch/x86/kvm/mmu/mmu.c +59 -27
163 lines not on this page · raw message
arch/x86/kvm/mmu/mmu_internal.h +1 -0
@@ -46,6 +46,7 @@ struct kvm_mmu_page {*/ union kvm_mmu_page_role role;gfn_t gfn; +gfn_t gfn_stolen_bits;u64 *spt;/* hold the gfn of each spte inside spt */
arch/x86/kvm/mmu/paging_tmpl.h +15 -10
@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS
@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page
@@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,
@@ -366,7 +367,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;
@@ -432,7 +433,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())
@@ -537,12 +538,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &vcpu_gpa_stolen_mask (vcpu)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); pfn = pte_prefetch_gfn_to_pfn(vcpu, gfn,
@@ -652,6 +655,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, direct_access = gw->pte_access; +WARN_ON (addr &vcpu_gpa_stolen_mask (vcpu)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;
@@ -1067,7 +1072,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; } - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 46 lines · On 03/07/21 00:04, [email protected] wrote:
From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faultsSince there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN,
kvm_handle_hva()based operations will be applied to both aliases.In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.
Quoted 6 lines
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
Signed-off-by Rick Edgecombe <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
Looks good, but the commit message is obsolete.
Paolo
Quoted 337 lines
arch/x86/kvm/mmu.h | 26 ++++++++++ arch/x86/kvm/mmu/mmu.c | 86 ++++++++++++++++++++++----------- arch/x86/kvm/mmu/mmu_internal.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 25 ++++++---- 4 files changed, 101 insertions(+), 37 deletions(-)arch/x86/kvm/mmu.h +26
-0@@ -232,4 +232,30 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu);int kvm_mmu_post_init_vm (struct kvm *kvm);void kvm_mmu_pre_destroy_vm (struct kvm *kvm); +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +/* Currently there are no stolen bits in KVM */ +return 0 ; +} + +static inline gfn_t vcpu_gfn_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gfn_stolen_mask (vcpu->kvm); +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return kvm_gfn_stolen_mask (kvm) <<PAGE_SHIFT ; +} + +static inline gpa_t vcpu_gpa_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gpa_stolen_mask (vcpu->kvm); +} + +static inline gfn_t vcpu_gpa_to_gfn_unalias (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +return (gpa >>PAGE_SHIFT ) & ~vcpu_gfn_stolen_mask (vcpu); +} +#endif arch/x86/kvm/mmu/mmu.c +59
-27@@ -188,27 +188,37 @@ static inline bool kvm_available_flush_tlb_with_range(void)return kvm_x86_ops.tlb_remote_flush_with_range; } -static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm, -struct kvm_tlb_range *range) -{ -int ret = -ENOTSUPP ; - -if (range && kvm_x86_ops.tlb_remote_flush_with_range) - ret =static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, range); - -if (ret) -kvm_flush_remote_tlbs (kvm); -} -void kvm_flush_remote_tlbs_with_address (struct kvm *kvm,u64 start_gfn,u64 pages) {struct kvm_tlb_range range; +u64 gfn_stolen_mask; + +if (!kvm_available_flush_tlb_with_range ()) + goto generic_flush; + +/* +* Fall back to the big hammer flush if there is more than one +* GPA alias that needs to be flushed. +*/ + gfn_stolen_mask =kvm_gfn_stolen_mask (kvm); +if (hweight64 (gfn_stolen_mask) >1 ) + goto generic_flush; range.start_gfn = start_gfn; range.pages = pages; +if (static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, &range)) + goto generic_flush; + +if (!gfn_stolen_mask) +return ; -kvm_flush_remote_tlbs_with_range (kvm, &range); + range.start_gfn |= gfn_stolen_mask; +static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, &range); +return ; + +generic_flush: +kvm_flush_remote_tlbs (kvm); }bool is_nx_huge_page_enabled (void )@@ -1949,14 +1959,16 @@ static void clear_sp_write_flooding_count(u64 *spte)__clear_sp_write_flooding_count (sptep_to_sp (spte)); } -static struct kvm_mmu_page *kvm_mmu_get_page (struct kvm_vcpu *vcpu, -gfn_t gfn, -gva_t gaddr, -unsigned level, -int direct, -unsigned int access) +static struct kvm_mmu_page *__kvm_mmu_get_page (struct kvm_vcpu *vcpu, +gfn_t gfn, +gfn_t gfn_stolen_bits, +gva_t gaddr, +unsigned int level, +int direct, +unsigned int access) {bool direct_mmu = vcpu->arch.mmu->direct_map; +gpa_t gfn_and_stolen = gfn | gfn_stolen_bits;union kvm_mmu_page_role role;struct hlist_head *sp_list;unsigned quadrant;@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.quadrant = quadrant; } - sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn)]; + sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn_and_stolen)];for_each_valid_sp (vcpu->kvm, sp, sp_list) { -if (sp->gfn != gfn) { +if ((sp->gfn | sp->gfn_stolen_bits) != gfn_and_stolen) { collisions++;continue ; }@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp =kvm_mmu_alloc_page (vcpu, direct); sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);if (!direct) {@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,return sp; } +static struct kvm_mmu_page *kvm_mmu_get_page (struct kvm_vcpu *vcpu,gfn_t gfn, +gva_t gaddr,unsigned int level, +int direct,unsigned int access) +{ +return __kvm_mmu_get_page (vcpu, gfn,0 , gaddr, level, direct, access); +} +static void shadow_walk_init_using_root (struct kvm_shadow_walk_iterator *iterator,struct kvm_vcpu *vcpu,hpa_t root,u64 addr)@@ -2637,7 +2657,9 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, gfn =kvm_mmu_page_get_gfn (sp, start - sp->spt); slot =gfn_to_memslot_dirty_bitmap (vcpu, gfn, access &ACC_WRITE_MASK ); -if (!slot) + +/* Don't map private memslots for stolen bits */ +if (!slot || (sp->gfn_stolen_bits && slot->id >=KVM_USER_MEM_SLOTS ))return -1 ; ret =gfn_to_page_many_atomic (slot, gfn, pages, end - start);@@ -2827,7 +2849,9 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,struct kvm_shadow_walk_iterator it;struct kvm_mmu_page *sp;int level, req_level, ret; -gfn_t gfn = gpa >>PAGE_SHIFT ; +gpa_t gpa_stolen_mask =vcpu_gpa_stolen_mask (vcpu); +gfn_t gfn = (gpa & ~gpa_stolen_mask) >>PAGE_SHIFT ; +gfn_t gfn_stolen_bits = (gpa & gpa_stolen_mask) >>PAGE_SHIFT ;gfn_t base_gfn = gfn;if (WARN_ON (!VALID_PAGE (vcpu->arch.mmu->root_hpa)))@@ -2852,8 +2876,9 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,drop_large_spte (vcpu, it.sptep);if (!is_shadow_present_pte (*it.sptep)) { - sp =kvm_mmu_get_page (vcpu, base_gfn, it.addr, - it.level -1 , true,ACC_ALL ); + sp =__kvm_mmu_get_page (vcpu, base_gfn, + gfn_stolen_bits, it.addr, + it.level -1 , true,ACC_ALL );link_shadow_page (vcpu, it.sptep, sp); if (is_tdp && huge_page_disallowed &&@@ -3689,6 +3714,13 @@ static bool try_async_pf(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn,if (slot && (slot->flags &KVM_MEMSLOT_INVALID ))return true; +/* Don't expose aliases for no slot GFNs or private memslots */ +if ((cr2_or_gpa &vcpu_gpa_stolen_mask (vcpu)) && + !kvm_is_visible_memslot (slot)) { + *pfn =KVM_PFN_NOSLOT ; +return false; + } +/* Don't expose private memslots to L2. */ if (is_guest_mode (vcpu) && !kvm_is_visible_memslot (slot)) { *pfn =KVM_PFN_NOSLOT ;@@ -3723,7 +3755,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,bool write = error_code &PFERR_WRITE_MASK ;bool map_writable; -gfn_t gfn = gpa >>PAGE_SHIFT ; +gfn_t gfn =vcpu_gpa_to_gfn_unalias (vcpu, gpa);unsigned long mmu_seq;kvm_pfn_t pfn;hva_t hva;@@ -3833,7 +3865,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, max_level >PG_LEVEL_4K ; max_level--) {int page_num =KVM_PAGES_PER_HPAGE (max_level); -gfn_t base = (gpa >>PAGE_SHIFT ) & ~(page_num -1 ); +gfn_t base =vcpu_gpa_to_gfn_unalias (vcpu, gpa) & ~(page_num -1 );if (kvm_mtrr_check_gfn_range_consistency (vcpu, base, page_num))break ;arch/x86/kvm/mmu/mmu_internal.h +1
-0@@ -46,6 +46,7 @@ struct kvm_mmu_page {*/ union kvm_mmu_page_role role;gfn_t gfn; +gfn_t gfn_stolen_bits;u64 *spt;/* hold the gfn of each spte inside spt */ arch/x86/kvm/mmu/paging_tmpl.h +15
-10@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page @@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,@@ -366,7 +367,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;@@ -432,7 +433,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())@@ -537,12 +538,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &vcpu_gpa_stolen_mask (vcpu)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); pfn = pte_prefetch_gfn_to_pfn(vcpu, gfn,@@ -652,6 +655,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, direct_access = gw->pte_access; +WARN_ON (addr &vcpu_gpa_stolen_mask (vcpu)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;@@ -1067,7 +1072,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; } - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
Re: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
In reply to [email protected]Quoted 193 lines · On Fri, 2 Jul 2021 15:04:47 -0700 [email protected] wrote:
From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA alising will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
-Specify physical memory by userspace via memslots -Map GPAs to TDP PTEs via RMAP
-Specify dirty tracking and write protection -Look up MTRR types
-Inject async page faultsSince there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN,
kvm_handle_hva()based operations will be applied to both aliases.In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the stolen bits case.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
Signed-off-by Rick Edgecombe <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu.h | 26 ++++++++++ arch/x86/kvm/mmu/mmu.c | 86 ++++++++++++++++++++++----------- arch/x86/kvm/mmu/mmu_internal.h | 1 + arch/x86/kvm/mmu/paging_tmpl.h | 25 ++++++---- 4 files changed, 101 insertions(+), 37 deletions(-)arch/x86/kvm/mmu.h +26
-0@@ -232,4 +232,30 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu);int kvm_mmu_post_init_vm (struct kvm *kvm);void kvm_mmu_pre_destroy_vm (struct kvm *kvm); +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +/* Currently there are no stolen bits in KVM */ +return 0 ; +} + +static inline gfn_t vcpu_gfn_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gfn_stolen_mask (vcpu->kvm); +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return kvm_gfn_stolen_mask (kvm) <<PAGE_SHIFT ; +} + +static inline gpa_t vcpu_gpa_stolen_mask (struct kvm_vcpu *vcpu) +{ +return kvm_gpa_stolen_mask (vcpu->kvm); +} + +static inline gfn_t vcpu_gpa_to_gfn_unalias (struct kvm_vcpu *vcpu,gpa_t gpa) +{ +return (gpa >>PAGE_SHIFT ) & ~vcpu_gfn_stolen_mask (vcpu); +} +#endif arch/x86/kvm/mmu/mmu.c +34
-21@@ -188,27 +188,37 @@ static inline bool kvm_available_flush_tlb_with_range(void)return kvm_x86_ops.tlb_remote_flush_with_range; } -static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm, -struct kvm_tlb_range *range) -{ -int ret = -ENOTSUPP ; - -if (range && kvm_x86_ops.tlb_remote_flush_with_range) - ret =static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, range); - -if (ret) -kvm_flush_remote_tlbs (kvm); -} -void kvm_flush_remote_tlbs_with_address (struct kvm *kvm,u64 start_gfn,u64 pages) {struct kvm_tlb_range range; +u64 gfn_stolen_mask; + +if (!kvm_available_flush_tlb_with_range ()) + goto generic_flush; + +/* +* Fall back to the big hammer flush if there is more than one +* GPA alias that needs to be flushed. +*/ + gfn_stolen_mask =kvm_gfn_stolen_mask (kvm); +if (hweight64 (gfn_stolen_mask) >1 ) + goto generic_flush; range.start_gfn = start_gfn; range.pages = pages; +if (static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, &range)) + goto generic_flush; + +if (!gfn_stolen_mask) +return ; -kvm_flush_remote_tlbs_with_range (kvm, &range); + range.start_gfn |= gfn_stolen_mask; +static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, &range); +return ; + +generic_flush: +kvm_flush_remote_tlbs (kvm); }bool is_nx_huge_page_enabled (void )@@ -1949,14 +1959,16 @@ static void clear_sp_write_flooding_count(u64 *spte)__clear_sp_write_flooding_count (sptep_to_sp (spte)); } -static struct kvm_mmu_page *kvm_mmu_get_page (struct kvm_vcpu *vcpu, -gfn_t gfn, -gva_t gaddr, -unsigned level, -int direct, -unsigned int access) +static struct kvm_mmu_page *__kvm_mmu_get_page (struct kvm_vcpu *vcpu, +gfn_t gfn, +gfn_t gfn_stolen_bits, +gva_t gaddr, +unsigned int level, +int direct, +unsigned int access) {bool direct_mmu = vcpu->arch.mmu->direct_map; +gpa_t gfn_and_stolen = gfn | gfn_stolen_bits;union kvm_mmu_page_role role;struct hlist_head *sp_list;unsigned quadrant;@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.quadrant = quadrant; } - sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn)]; + sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn_and_stolen)];for_each_valid_sp (vcpu->kvm, sp, sp_list) { -if (sp->gfn != gfn) { +if ((sp->gfn | sp->gfn_stolen_bits) != gfn_and_stolen) { collisions++;continue ; }@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp =kvm_mmu_alloc_page (vcpu, direct); sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);if (!direct) {
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2058
Quoted 2 lines
arch/x86/kvm/mmu/mmu.c +0
-0@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,return sp; }
Sorry for replying old thread, but to me it looks weird to have gfn_stolen_bits in 'struct kvm_mmu_page'. If I understand correctly, above code basically means that GFN with different stolen bit will have different 'struct kvm_mmu_page', but in the context of this patch, mappings with different stolen bits still use the same root, which means gfn_stolen_bits doesn't make a lot of sense at least for root page table.
Instead, having gfn_stolen_bits in 'struct kvm_mmu_page' only makes sense in the context of TDX, since TDX requires two separate roots for private and shared mappings.
So given we cannot tell whether the same root, or different roots should be used for different stolen bits, I think we should not add 'gfn_stolen_bits' to 'struct kvm_mmu_page' and use it to determine whether to allocate a new table for the same GFN, but should use a new role (i.e role.private) to determine.
And removing 'gfn_stolen_bits' in 'struct kvm_mmu_page' could also save some memory.
Quoted 197 lines
arch/x86/kvm/mmu/mmu.c +25
-6+static struct kvm_mmu_page *kvm_mmu_get_page (struct kvm_vcpu *vcpu,gfn_t gfn, +gva_t gaddr,unsigned int level, +int direct,unsigned int access) +{ +return __kvm_mmu_get_page (vcpu, gfn,0 , gaddr, level, direct, access); +} +static void shadow_walk_init_using_root (struct kvm_shadow_walk_iterator *iterator,struct kvm_vcpu *vcpu,hpa_t root,u64 addr)@@ -2637,7 +2657,9 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, gfn =kvm_mmu_page_get_gfn (sp, start - sp->spt); slot =gfn_to_memslot_dirty_bitmap (vcpu, gfn, access &ACC_WRITE_MASK ); -if (!slot) + +/* Don't map private memslots for stolen bits */ +if (!slot || (sp->gfn_stolen_bits && slot->id >=KVM_USER_MEM_SLOTS ))return -1 ; ret =gfn_to_page_many_atomic (slot, gfn, pages, end - start);@@ -2827,7 +2849,9 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,struct kvm_shadow_walk_iterator it;struct kvm_mmu_page *sp;int level, req_level, ret; -gfn_t gfn = gpa >>PAGE_SHIFT ; +gpa_t gpa_stolen_mask =vcpu_gpa_stolen_mask (vcpu); +gfn_t gfn = (gpa & ~gpa_stolen_mask) >>PAGE_SHIFT ; +gfn_t gfn_stolen_bits = (gpa & gpa_stolen_mask) >>PAGE_SHIFT ;gfn_t base_gfn = gfn;if (WARN_ON (!VALID_PAGE (vcpu->arch.mmu->root_hpa)))@@ -2852,8 +2876,9 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,drop_large_spte (vcpu, it.sptep);if (!is_shadow_present_pte (*it.sptep)) { - sp =kvm_mmu_get_page (vcpu, base_gfn, it.addr, - it.level -1 , true,ACC_ALL ); + sp =__kvm_mmu_get_page (vcpu, base_gfn, + gfn_stolen_bits, it.addr, + it.level -1 , true,ACC_ALL );link_shadow_page (vcpu, it.sptep, sp); if (is_tdp && huge_page_disallowed &&@@ -3689,6 +3714,13 @@ static bool try_async_pf(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn,if (slot && (slot->flags &KVM_MEMSLOT_INVALID ))return true; +/* Don't expose aliases for no slot GFNs or private memslots */ +if ((cr2_or_gpa &vcpu_gpa_stolen_mask (vcpu)) && + !kvm_is_visible_memslot (slot)) { + *pfn =KVM_PFN_NOSLOT ; +return false; + } +/* Don't expose private memslots to L2. */ if (is_guest_mode (vcpu) && !kvm_is_visible_memslot (slot)) { *pfn =KVM_PFN_NOSLOT ;@@ -3723,7 +3755,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,bool write = error_code &PFERR_WRITE_MASK ;bool map_writable; -gfn_t gfn = gpa >>PAGE_SHIFT ; +gfn_t gfn =vcpu_gpa_to_gfn_unalias (vcpu, gpa);unsigned long mmu_seq;kvm_pfn_t pfn;hva_t hva;@@ -3833,7 +3865,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, max_level >PG_LEVEL_4K ; max_level--) {int page_num =KVM_PAGES_PER_HPAGE (max_level); -gfn_t base = (gpa >>PAGE_SHIFT ) & ~(page_num -1 ); +gfn_t base =vcpu_gpa_to_gfn_unalias (vcpu, gpa) & ~(page_num -1 );if (kvm_mtrr_check_gfn_range_consistency (vcpu, base, page_num))break ;arch/x86/kvm/mmu/mmu_internal.h +1
-0@@ -46,6 +46,7 @@ struct kvm_mmu_page {*/ union kvm_mmu_page_role role;gfn_t gfn; +gfn_t gfn_stolen_bits;u64 *spt;/* hold the gfn of each spte inside spt */ arch/x86/kvm/mmu/paging_tmpl.h +15
-10@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~vcpu_gpa_stolen_mask(vcpu) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page @@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,@@ -366,7 +367,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;@@ -432,7 +433,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())@@ -537,12 +538,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &vcpu_gpa_stolen_mask (vcpu)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); pfn = pte_prefetch_gfn_to_pfn(vcpu, gfn,@@ -652,6 +655,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, gpa_t addr, direct_access = gw->pte_access; +WARN_ON (addr &vcpu_gpa_stolen_mask (vcpu)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;@@ -1067,7 +1072,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; } - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);Signature · 1 line
2.25.1
Sean Christopherson
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 5 lines · Kai Huang
Quoted 4 lines · On Fri, 2 Jul 2021 15:04:47 -0700 [email protected] wrote:
From: Rick Edgecombe <[email protected]>
arch/x86/kvm/mmu/mmu.c +0
-0@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp =kvm_mmu_alloc_page (vcpu, direct);
Quoted 11 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2058
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-0sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);if (!direct) {@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,return sp; }Sorry for replying old thread,
Ha, one month isn't old, it's barely even mature.
Quoted 5 lines
but to me it looks weird to have
gfn_stolen_bits
in 'structkvm_mmu_page'. If I understand correctly, above code basically means that GFN with different stolen bit will have different 'structkvm_mmu_page', but in the context of this patch, mappings with different stolen bits still use the same root,
You're conflating "mapping" with "PTE". The GFN is a per-PTE value. Yes, there is a final GFN that is representative of the mapping, but more directly the final GFN is associated with the leaf PTE.
TDX effectively adds the restriction that all PTEs used for a mapping must have the same shared/private status, so mapping and PTE are somewhat interchangeable when talking about stolen bits (the shared bit), but in the context of this patch, the stolen bits are a property of the PTE.
Back to your statement, it's incorrect. PTEs (effectively mappings in TDX) with different stolen bits will _not_ use the same root. kvm_mmu_get_page() includes the stolen bits in both the hash lookup and in the comparison, i.e. restores the stolen bits when looking for an existing shadow page at the target GFN.
@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,
role.quadrant = quadrant;
}- sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn(gfn)];
+ sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn(gfn_and_stolen)];
for_each_valid_sp(vcpu->kvm, sp, sp_list) {
- if (sp->gfn != gfn) {
+ if ((sp->gfn | sp->gfn_stolen_bits) != gfn_and_stolen) {
collisions++;
continue;
}Quoted 2 lines
which means
gfn_stolen_bitsdoesn't make a lot of sense at least for root page table.
It does make sense, even without a follow-up patch. In Rick's original series, stealing a bit for execute-only guest memory, there was only a single root. And except for TDX, there can only ever be a single root because the shared EPTP isn't usable, i.e. there's only the regular/private EPTP.
Quoted 8 lines
Instead, having
gfn_stolen_bitsin 'structkvm_mmu_page' only makes sense in the context of TDX, since TDX requires two separate roots for private and shared mappings.So given we cannot tell whether the same root, or different roots should be used for different stolen bits, I think we should not add '
gfn_stolen_bits' to 'structkvm_mmu_page' and use it to determine whether to allocate a new table for the same GFN, but should use a new role (i.e role.private) to determine.
A new role would work, too, but it has the disadvantage of not automagically working for all uses of stolen bits, e.g. XO support would have to add another role bit.
Quoted 2 lines
And removing '
gfn_stolen_bits' in 'structkvm_mmu_page' could also save some memory.
But I do like saving memory... One potentially bad idea would be to unionize gfn and stolen bits by shifting the stolen bits after they're extracted from the gpa, e.g.
union {
gfn_t gfn_and_stolen;
struct {
gfn_t gfn:52;
gfn_t stolen:12;
}
};the downsides being that accessing just the gfn would require an additional masking operation, and the stolen bits wouldn't align with reality.
Edgecombe, Rick P
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 121 lines · Sean Christopherson
Quoted 6 lines · Kai Huang
Quoted 5 lines · On Fri, 2 Jul 2021 15:04:47 -0700 [email protected] wrote:
From: Rick Edgecombe <[email protected]>
arch/x86/kvm/mmu/mmu.c +0
-0@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp = kvm_mmu_alloc_page(vcpu, direct);Quoted 12 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page · line 2059
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +1
-0sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);arch/x86/kvm/mmu/mmu.c +0
-0if (!direct) {@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, return sp; }Sorry for replying old thread,
Ha, one month isn't old, it's barely even mature.
Quoted 8 lines
but to me it looks weird to have
gfn_stolen_bits
in 'structkvm_mmu_page'. If I understand correctly, above code basically
means that GFN with different stolen bit will have different 'structkvm_mmu_page', but in the context of this patch, mappings with different
stolen bits still use the same root,You're conflating "mapping" with "PTE". The GFN is a per-PTE value. Yes, there
is a final GFN that is representative of the mapping, but more directly the final
GFN is associated with the leaf PTE.TDX effectively adds the restriction that all PTEs used for a mapping must have
the same shared/private status, so mapping and PTE are somewhat interchangeable
when talking about stolen bits (the shared bit), but in the context of this patch,
the stolen bits are a property of the PTE.Back to your statement, it's incorrect. PTEs (effectively mappings in TDX) with
different stolen bits will _not_ use the same
root.kvm_mmu_get_page()includes
the stolen bits in both the hash lookup and in the comparison, i.e. restores the
stolen bits when looking for an existing shadow page at the target GFN.(quoted diff) +2
-2@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.quadrant = quadrant; } - sp_list = &vcpu->kvm- >arch.mmu_page_hash[kvm_page_table_hashfn(gfn)]; + sp_list = &vcpu->kvm- >arch.mmu_page_hash[kvm_page_table_hashfn(gfn_and_stolen)]; for_each_valid_sp(vcpu->kvm, sp, sp_list) { - if (sp->gfn != gfn) { + if ((sp->gfn | sp->gfn_stolen_bits) !=gfn_and_stolen) { collisions++; continue; }Quoted 3 lines
which means
gfn_stolen_bitsdoesn't make a lot of sense at least for root
page table.It does make sense, even without a follow-up patch. In Rick's original series,
stealing a bit for execute-only guest memory, there was only a single root. And
except for TDX, there can only ever be a single root because the shared EPTP isn't
usable, i.e. there's only the regular/private EPTP.Quoted 5 lines
Instead, having
gfn_stolen_bitsin 'structkvm_mmu_page' only makes sense in
the context of TDX, since TDX requires two separate roots for private and
shared mappings.Quoted 8 lines
So given we cannot tell whether the same root, or different roots should be
used for different stolen bits, I think we should not add 'gfn_stolen_bits' to
'structkvm_mmu_page' and use it to determine whether to allocate a new table
for the same GFN, but should use a new role (i.e role.private) to determine.A new role would work, too, but it has the disadvantage of not automagically
working for all uses of stolen bits, e.g. XO support would have to add another
role bit.Quoted 3 lines
And removing '
gfn_stolen_bits' in 'structkvm_mmu_page' could also save some
memory.But I do like saving memory... One potentially bad idea would be to unionize
gfn and stolen bits by shifting the stolen bits after they're extracted from the
gpa, e.g.union { gfn_t gfn_and_stolen; struct { gfn_t gfn:52; gfn_t stolen:12; } };
Quoted 3 lines
the downsides being that accessing just the gfn would require an additional masking
operation, and the stolen bits wouldn't align with reality.
It definitely seems like the sp could be packed more efficiently. One other idea is the stolen bits could just be recovered from the role bits with a helper, like how the page fault error code stolen bits encoding version of this works.
If the stolen bits are not fed into the hash calculation though it would change the behavior a bit. Not sure if for better or worse. Also the calculation of hash collisions would need to be aware.
FWIW, I kind of like something like Sean's proposal. It's a bit convoluted, but there are more unused bits in the gfn than the role. Also they are a little more related.
Sean Christopherson
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 17 lines · Rick P
Quoted 16 lines · Sean Christopherson
Quoted 2 lines · Kai Huang
And removing '
gfn_stolen_bits' in 'structkvm_mmu_page' could also save some memory.But I do like saving memory... One potentially bad idea would be to unionize gfn and stolen bits by shifting the stolen bits after they're extracted from the gpa, e.g.
union { gfn_t gfn_and_stolen; struct { gfn_t gfn:52; gfn_t stolen:12; } };
Quoted 4 lines
Quoted 2 lines
the downsides being that accessing just the gfn would require an additional masking operation, and the stolen bits wouldn't align with reality.
It definitely seems like the sp could be packed more efficiently.
Yeah, in general it could be optimized. But for TDP/direct MMUs, we don't care thaaat much because there are relatively few shadow pages, versus indirect MMUs with thousands or tens of thousands of shadow pages. Of course, indirect MMUs are also the most gluttonous due to the unsync_child_bitmap, gfns, write flooding count, etc...
If we really want to reduce the memory footprint for the common case (TDP MMU), the crud that's used only by indirect shadow pages could be shoved into a different struct by abusing the struct layout and and wrapping accesses to the indirect-only fields with casts/container_of and helpers, e.g.
struct kvm_mmu_indirect_page {
struct kvm_mmu_page this;
gfn_t *gfns;
unsigned int unsync_children;
DECLARE_BITMAP(unsync_child_bitmap, 512);#ifdef CONFIG_X86_32 /* * Used out of the mmu-lock to avoid reading spte values while an * update is in progress; see the comments in __get_spte_lockless(). */ int clear_spte_count;
#endif
/* Number of writes since the last time traversal visited this page. */ atomic_t write_flooding_count; }
Quoted 3 lines
One other idea is the stolen bits could just be recovered from the role bits with a helper, like how the page fault error code stolen bits encoding version of this works.
As in, a generic "stolen_gfn_bits" in the role instead of a per-feature role bit? That would avoid the problem of per-feature role bits leading to a pile of marshalling code, and wouldn't suffer the masking cost when accessing ->gfn, though I'm not sure that matters much.
Quoted 3 lines
If the stolen bits are not fed into the hash calculation though it would change the behavior a bit. Not sure if for better or worse. Also the calculation of hash collisions would need to be aware.
The role is already factored into the collision logic.
Quoted 2 lines
FWIW, I kind of like something like Sean's proposal. It's a bit convoluted, but there are more unused bits in the gfn than the role.
And tightly bound, i.e. there can't be more than gfn_t gfn+gfn_stolen bits.
Quoted 1 line
Also they are a little more related.
Edgecombe, Rick P
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 38 lines · Sean Christopherson
Quoted 20 lines · Rick P
Quoted 19 lines · Sean Christopherson
Quoted 3 lines · Kai Huang
And removing '
gfn_stolen_bits' in 'structkvm_mmu_page' could also save
some memory.But I do like saving memory... One potentially bad idea would be to
unionize gfn and stolen bits by shifting the stolen bits after they're
extracted from the gpa, e.g.union { gfn_t gfn_and_stolen; struct { gfn_t gfn:52; gfn_t stolen:12; } };Quoted 6 lines
Quoted 4 lines
the downsides being that accessing just the gfn would require an additional
masking operation, and the stolen bits wouldn't align with reality.It definitely seems like the sp could be packed more efficiently.
Yeah, in general it could be optimized. But for TDP/direct MMUs, we don't care
thaaat much because there are relatively few shadow pages, versus indirect MMUs
with thousands or tens of thousands of shadow pages. Of course, indirect MMUs
are also the most gluttonous due to theunsync_child_bitmap, gfns, write flooding
count, etc...
Quoted 8 lines
If we really want to reduce the memory footprint for the common case (TDP MMU),
the crud that's used only by indirect shadow pages could be shoved into a
different struct by abusing the struct layout and and wrapping accesses to the
indirect-only fields with casts/container_of and helpers, e.g.
Wow, didn't realize classic MMU was that relegated already. Mostly an onlooker here, but does TDX actually need classic MMU support? Nice to have?
Quoted 28 lines
struct kvm_mmu_indirect_page { struct kvm_mmu_page this; gfn_t *gfns; unsigned int unsync_children; DECLARE_BITMAP(unsync_child_bitmap, 512);#ifdef CONFIG_X86_32 /* * Used out of the mmu-lock to avoid reading spte values while an * update is in progress; see the comments in __get_spte_lockless(). */ int clear_spte_count;#endif
/* Number of writes since the last time traversal visited this page. */ atomic_t write_flooding_count; }Quoted 4 lines
One other idea is the stolen bits could just be recovered from the role
bits with a helper, like how the page fault error code stolen bits encoding version of this works.
Quoted 7 lines
As in, a generic "
stolen_gfn_bits" in the role instead of a per- feature role bit?
That would avoid the problem of per-feature role bits leading to a pile of
marshalling code, and wouldn't suffer the masking cost when accessing ->gfn,
though I'm not sure that matters much.
Well I was thinking multiple types of aliases, like the pf err code stuff works, like this:
gfn_t stolen_bits(struct kvm *kvm, struct kvm_mmu_page *sp)
{
gfn_t stolen = 0;
if (sp->role.shared)
stolen |= kvm->arch.gfn_shared_mask;
if (sp->role.other_alias)
stolen |= kvm->arch.gfn_other_mask;
return stolen;
}But yea, there really only needs to be one. Still bit shifting seems better.
Quoted 7 lines
Quoted 5 lines
If the stolen bits are not fed into the hash calculation though it would change the behavior a bit. Not sure if for better or worse. Also
the calculation of hash collisions would need to be aware.The role is already factored into the collision logic.
I mean how aliases of the same gfn don't necessarily collide and the collisions counter is only incremented if the gfn/stolen matches, but not if the role is different.
Quoted 11 lines
Quoted 4 lines
FWIW, I kind of like something like Sean's proposal. It's a bit convoluted, but there are more unused bits in the gfn than the role.
And tightly bound, i.e. there can't be more than
gfn_tgfn+gfn_stolen bits.Quoted 1 line
Also they are a little more related.
Sean Christopherson
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 10 lines · Rick P
Quoted 6 lines · Sean Christopherson
If we really want to reduce the memory footprint for the common case (TDP MMU), the crud that's used only by indirect shadow pages could be shoved into a different struct by abusing the struct layout and and wrapping accesses to the indirect-only fields with casts/container_of and helpers, e.g.
Wow, didn't realize classic MMU was that relegated already. Mostly an onlooker here, but does TDX actually need classic MMU support? Nice to have?
Gah, bad verbiage on my part. I didn't me _the_ TDP MMU, I meant "MMU that uses TDP". The "TDP MMU" is being enabled by default in upstream, whether or not TDX needs to support the classic/legacy MMU is an unanswered question. From a maintenance perspective, I'd love to say no, but from a "what does Google actually want to use for TDX" perspective, I don't have a defininitive answer yet :-/
Quoted 5 lines
Quoted 1 line
The role is already factored into the collision logic.
I mean how aliases of the same gfn don't necessarily collide and the collisions counter is only incremented if the gfn/stolen matches, but not if the role is different.
Ah. I still think it would Just Work. The collisions tracking is purely for stats, presumably used in the past to see if the hash size was effective. If a hash lookup collides then it should be accounted, _why_ it collided doesn't factor in to the stats.
Your comments about the hash behavior being different still stand, though for TDX they wouldn't matter in practice since KVM is hosed if it has both shared and private versions of a shadow page.
Re: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
In reply to Sean ChristophersonQuoted 26 lines · On Thu, 5 Aug 2021 16:06:41 +0000 Sean Christopherson wrote:
Quoted 5 lines · Kai Huang
Quoted 4 lines · On Fri, 2 Jul 2021 15:04:47 -0700 [email protected] wrote:
From: Rick Edgecombe <[email protected]>
arch/x86/kvm/mmu/mmu.c +0
-0@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp =kvm_mmu_alloc_page (vcpu, direct);Quoted 11 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2058
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-0sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);if (!direct) {@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,return sp; }Sorry for replying old thread,
Ha, one month isn't old, it's barely even mature.
Quoted 5 lines
but to me it looks weird to have
gfn_stolen_bits
in 'structkvm_mmu_page'. If I understand correctly, above code basically means that GFN with different stolen bit will have different 'structkvm_mmu_page', but in the context of this patch, mappings with different stolen bits still use the same root,
Quoted 8 lines
You're conflating "mapping" with "PTE". The GFN is a per-PTE value. Yes, there is a final GFN that is representative of the mapping, but more directly the final GFN is associated with the leaf PTE.
TDX effectively adds the restriction that all PTEs used for a mapping must have the same shared/private status, so mapping and PTE are somewhat interchangeable when talking about stolen bits (the shared bit), but in the context of this patch, the stolen bits are a property of the PTE.
Yes it is a property of PTE, this is the reason that I think it's weird to have stolen bits in 'struct kvm_mmu_page'. Shouldn't stolen bits in 'struct kvm_mmu_page' imply that all PTEs (whether leaf or not) share the same stolen bit?
Quoted 9 lines
Back to your statement, it's incorrect. PTEs (effectively mappings in TDX) with different stolen bits will _not_ use the same root.
kvm_mmu_get_page()includes the stolen bits in both the hash lookup and in the comparison, i.e. restores the stolen bits when looking for an existing shadow page at the target GFN.arch/x86/kvm/mmu/mmu.c +0
-0@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.quadrant = quadrant;
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 1998
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +2
-2} - sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn)]; + sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn_and_stolen)];for_each_valid_sp (vcpu->kvm, sp, sp_list) { -if (sp->gfn != gfn) { +if ((sp->gfn | sp->gfn_stolen_bits) != gfn_and_stolen) { collisions++;continue ; }
This only works for non-root table, but there's only one single vcpu->arch.mmu->root_hpa, we don't have an array to have one root for each stolen bit, i.e. do a loop in mmu_alloc_direct_roots(), so effectively all stolen bits share one single root.
Quoted 12 lines
Quoted 2 lines
which means
gfn_stolen_bitsdoesn't make a lot of sense at least for root page table.It does make sense, even without a follow-up patch. In Rick's original series, stealing a bit for execute-only guest memory, there was only a single root. And except for TDX, there can only ever be a single root because the shared EPTP isn't usable, i.e. there's only the regular/private EPTP.
Quoted 3 lines
Instead, having
gfn_stolen_bitsin 'structkvm_mmu_page' only makes sense in the context of TDX, since TDX requires two separate roots for private and shared mappings.
Quoted 8 lines
Quoted 4 lines
So given we cannot tell whether the same root, or different roots should be used for different stolen bits, I think we should not add '
gfn_stolen_bits' to 'structkvm_mmu_page' and use it to determine whether to allocate a new table for the same GFN, but should use a new role (i.e role.private) to determine.A new role would work, too, but it has the disadvantage of not automagically working for all uses of stolen bits, e.g. XO support would have to add another role bit.
For each purpose of particular stolen bit, a new role can be defined. For instance, in __direct_map(), if you see stolen bit is TDX shared bit, you don't set role.private (otherwise set role.private). For XO, if you see the stolen bit is XO, you set role.xo.
We already have info of 'gfn_stolen_mask' in vcpu, so we just need to make sure all code paths can find the actual stolen bit based on sp->role and vcpu (I haven't gone through all though, assuming the annoying part is rmap).
Quoted 18 lines
Quoted 3 lines
And removing '
gfn_stolen_bits' in 'structkvm_mmu_page' could also save some memory.But I do like saving memory... One potentially bad idea would be to unionize gfn and stolen bits by shifting the stolen bits after they're extracted from the gpa, e.g.
union { gfn_t gfn_and_stolen; struct { gfn_t gfn:52; gfn_t stolen:12; } };the downsides being that accessing just the gfn would require an additional masking operation, and the stolen bits wouldn't align with reality.
Sean Christopherson
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 36 lines · Kai Huang
Quoted 26 lines · On Thu, 5 Aug 2021 16:06:41 +0000 Sean Christopherson wrote:
Quoted 5 lines · Kai Huang
Quoted 4 lines · On Fri, 2 Jul 2021 15:04:47 -0700 [email protected] wrote:
From: Rick Edgecombe <[email protected]>
arch/x86/kvm/mmu/mmu.c +0
-0@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp =kvm_mmu_alloc_page (vcpu, direct);Quoted 11 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2058
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-0sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);if (!direct) {@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,return sp; }Sorry for replying old thread,
Ha, one month isn't old, it's barely even mature.
Quoted 5 lines
but to me it looks weird to have
gfn_stolen_bits
in 'structkvm_mmu_page'. If I understand correctly, above code basically means that GFN with different stolen bit will have different 'structkvm_mmu_page', but in the context of this patch, mappings with different stolen bits still use the same root,Quoted 8 lines
You're conflating "mapping" with "PTE". The GFN is a per-PTE value. Yes, there is a final GFN that is representative of the mapping, but more directly the final GFN is associated with the leaf PTE.
TDX effectively adds the restriction that all PTEs used for a mapping must have the same shared/private status, so mapping and PTE are somewhat interchangeable when talking about stolen bits (the shared bit), but in the context of this patch, the stolen bits are a property of the PTE.
Quoted 4 lines
Yes it is a property of PTE, this is the reason that I think it's weird to have stolen bits in 'struct
kvm_mmu_page'. Shouldn't stolen bits in 'structkvm_mmu_page' imply that all PTEs (whether leaf or not) share the same stolen bit?
No, the stolen bits are the property of the shadow page. I'm using "PTE" above to mean "PTE for this shadow page", not PTEs within the shadow page, if that makes sense.
Quoted 9 lines
Quoted 9 lines
Back to your statement, it's incorrect. PTEs (effectively mappings in TDX) with different stolen bits will _not_ use the same root.
kvm_mmu_get_page()includes the stolen bits in both the hash lookup and in the comparison, i.e. restores the stolen bits when looking for an existing shadow page at the target GFN.arch/x86/kvm/mmu/mmu.c +0
-0@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.quadrant = quadrant; }
Quoted 14 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 1998
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +2
-2- sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn)]; + sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn_and_stolen)];for_each_valid_sp (vcpu->kvm, sp, sp_list) { -if (sp->gfn != gfn) { +if ((sp->gfn | sp->gfn_stolen_bits) != gfn_and_stolen) { collisions++;continue ; }This only works for non-root table, but there's only one single
vcpu->arch.mmu->root_hpa, we don't have an array to have one root for each stolen bit, i.e. do a loop inmmu_alloc_direct_roots(), so effectively all stolen bits share one single root.
Yes, and that's absolutely the required behavior for everything except for TDX with its two EPTPs. E.g. any other implement _must_ reject CR3s that set stolen gfn bits.
Quoted 21 lines
Quoted 12 lines
Quoted 2 lines
which means
gfn_stolen_bitsdoesn't make a lot of sense at least for root page table.It does make sense, even without a follow-up patch. In Rick's original series, stealing a bit for execute-only guest memory, there was only a single root. And except for TDX, there can only ever be a single root because the shared EPTP isn't usable, i.e. there's only the regular/private EPTP.
Quoted 3 lines
Instead, having
gfn_stolen_bitsin 'structkvm_mmu_page' only makes sense in the context of TDX, since TDX requires two separate roots for private and shared mappings.Quoted 8 lines
Quoted 4 lines
So given we cannot tell whether the same root, or different roots should be used for different stolen bits, I think we should not add '
gfn_stolen_bits' to 'structkvm_mmu_page' and use it to determine whether to allocate a new table for the same GFN, but should use a new role (i.e role.private) to determine.A new role would work, too, but it has the disadvantage of not automagically working for all uses of stolen bits, e.g. XO support would have to add another role bit.
Quoted 8 lines
For each purpose of particular stolen bit, a new role can be defined. For instance, in
__direct_map(), if you see stolen bit is TDX shared bit, you don't set role.private (otherwise set role.private). For XO, if you see the stolen bit is XO, you set role.xo.We already have info of '
gfn_stolen_mask' in vcpu, so we just need to make sure all code paths can find the actual stolen bit based onsp->roleand vcpu (I haven't gone through all though, assuming the annoying part is rmap).
Yes, and I'm not totally against the idea, but I'm also not 100% sold on it either, yet... The idea of a 'private' flag is growing on me.
If we're treating the shared bit as an attribute bit, which we are, then it's effectively an extension of role.access. Ditto for XO.
And looking at the code, I think it would be an improvement for TDX, as all of the is_private_gfn() calls that operate on a shadow page would be simplified and optimized as they wouldn't have to lookup both gfn_stolen_bits and the vcpu->kvm mask of the shared bit.
Actually, the more I think about it, the more I like it. For TDX, there's no risk of increased hash collisions, as we've already done messed up if there's a shared vs. private collision.
And for XO, if it ever makes it way upstream, I think we should flat out disallow referencing XO addresses in non-leaf PTEs, i.e. make the XO permission bit reserved in non-leaf PTEs. That would avoid any theoretical problems with the guest doing something stupid by polluting all its upper level PxEs with XO. Collisions would be purely limited to the case where the guest is intentionally creating an alternate mapping, which should be a rare event (or the guest is comprosied, which is also hopefully a rare event).
Re: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 37 lines · On Fri, 6 Aug 2021 19:02:39 +0000 Sean Christopherson wrote:
Quoted 36 lines · Kai Huang
Quoted 34 lines · On Thu, 5 Aug 2021 16:06:41 +0000 Sean Christopherson wrote:
Quoted 5 lines · Kai Huang
Quoted 4 lines · On Fri, 2 Jul 2021 15:04:47 -0700 [email protected] wrote:
From: Rick Edgecombe <[email protected]>
arch/x86/kvm/mmu/mmu.c +0
-0@@ -2020,6 +2032,7 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, sp =kvm_mmu_alloc_page (vcpu, direct);Quoted 11 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 2058
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-0sp->gfn = gfn; + sp->gfn_stolen_bits = gfn_stolen_bits; sp->role = role;hlist_add_head (&sp->hash_link, sp_list);if (!direct) {@@ -2044,6 +2057,13 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu,return sp; }Sorry for replying old thread,
Ha, one month isn't old, it's barely even mature.
Quoted 5 lines
but to me it looks weird to have
gfn_stolen_bits
in 'structkvm_mmu_page'. If I understand correctly, above code basically means that GFN with different stolen bit will have different 'structkvm_mmu_page', but in the context of this patch, mappings with different stolen bits still use the same root,You're conflating "mapping" with "PTE". The GFN is a per-PTE value. Yes, there is a final GFN that is representative of the mapping, but more directly the final GFN is associated with the leaf PTE.
TDX effectively adds the restriction that all PTEs used for a mapping must have the same shared/private status, so mapping and PTE are somewhat interchangeable when talking about stolen bits (the shared bit), but in the context of this patch, the stolen bits are a property of the PTE.
Quoted 8 lines
Quoted 4 lines
Yes it is a property of PTE, this is the reason that I think it's weird to have stolen bits in 'struct
kvm_mmu_page'. Shouldn't stolen bits in 'structkvm_mmu_page' imply that all PTEs (whether leaf or not) share the same stolen bit?No, the stolen bits are the property of the shadow page. I'm using "PTE" above to mean "PTE for this shadow page", not PTEs within the shadow page, if that makes sense.
I see.
Quoted 25 lines
Quoted 10 lines
Quoted 10 lines
Back to your statement, it's incorrect. PTEs (effectively mappings in TDX) with different stolen bits will _not_ use the same root.
kvm_mmu_get_page()includes the stolen bits in both the hash lookup and in the comparison, i.e. restores the stolen bits when looking for an existing shadow page at the target GFN.arch/x86/kvm/mmu/mmu.c +0
-0@@ -1978,9 +1990,9 @@ static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, role.quadrant = quadrant; }Quoted 14 lines
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *kvm_mmu_get_page(struct kvm_vcpu *vcpu, · line 1998
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +2
-2- sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn)]; + sp_list = &vcpu->kvm->arch.mmu_page_hash[kvm_page_table_hashfn (gfn_and_stolen)];for_each_valid_sp (vcpu->kvm, sp, sp_list) { -if (sp->gfn != gfn) { +if ((sp->gfn | sp->gfn_stolen_bits) != gfn_and_stolen) { collisions++;continue ; }This only works for non-root table, but there's only one single
vcpu->arch.mmu->root_hpa, we don't have an array to have one root for each stolen bit, i.e. do a loop inmmu_alloc_direct_roots(), so effectively all stolen bits share one single root.
Quoted 3 lines
Yes, and that's absolutely the required behavior for everything except for TDX with its two EPTPs. E.g. any other implement _must_ reject CR3s that set stolen gfn bits.
OK. I was thinking gfn_stolen_bits for 'struct kvm_mmu_page' for the table pointed by CR3 should still make sense.
Quoted 46 lines
Quoted 22 lines
Quoted 13 lines
Quoted 3 lines
which means
gfn_stolen_bitsdoesn't make a lot of sense at least for root page table.It does make sense, even without a follow-up patch. In Rick's original series, stealing a bit for execute-only guest memory, there was only a single root. And except for TDX, there can only ever be a single root because the shared EPTP isn't usable, i.e. there's only the regular/private EPTP.
Quoted 3 lines
Instead, having
gfn_stolen_bitsin 'structkvm_mmu_page' only makes sense in the context of TDX, since TDX requires two separate roots for private and shared mappings.Quoted 8 lines
Quoted 4 lines
So given we cannot tell whether the same root, or different roots should be used for different stolen bits, I think we should not add '
gfn_stolen_bits' to 'structkvm_mmu_page' and use it to determine whether to allocate a new table for the same GFN, but should use a new role (i.e role.private) to determine.A new role would work, too, but it has the disadvantage of not automagically working for all uses of stolen bits, e.g. XO support would have to add another role bit.
Quoted 8 lines
For each purpose of particular stolen bit, a new role can be defined. For instance, in
__direct_map(), if you see stolen bit is TDX shared bit, you don't set role.private (otherwise set role.private). For XO, if you see the stolen bit is XO, you set role.xo.We already have info of '
gfn_stolen_mask' in vcpu, so we just need to make sure all code paths can find the actual stolen bit based onsp->roleand vcpu (I haven't gone through all though, assuming the annoying part is rmap).Yes, and I'm not totally against the idea, but I'm also not 100% sold on it either, yet... The idea of a 'private' flag is growing on me.
If we're treating the shared bit as an attribute bit, which we are, then it's effectively an extension of role.access. Ditto for XO.
And looking at the code, I think it would be an improvement for TDX, as all of the
is_private_gfn()calls that operate on a shadow page would be simplified and optimized as they wouldn't have to lookup bothgfn_stolen_bitsand thevcpu->kvmmask of the shared bit.Actually, the more I think about it, the more I like it. For TDX, there's no risk of increased hash collisions, as we've already done messed up if there's a shared vs. private collision.
Quoted 9 lines
And for XO, if it ever makes it way upstream, I think we should flat out disallow referencing XO addresses in non-leaf PTEs, i.e. make the XO permission bit reserved in non-leaf PTEs. That would avoid any theoretical problems with the guest doing something stupid by polluting all its upper level PxEs with XO. Collisions would be purely limited to the case where the guest is intentionally creating an alternate mapping, which should be a rare event (or the guest is comprosied, which is also hopefully a rare event).
My main motivation is 'gfn_stolen_bits' doesn't quite make sense for 'struct kvm_mmu_page' for root, plus it seems it's a little bit redundant at first glance.
So could we have your final suggestion? :)
Thanks,
-Kai
Sean Christopherson
maintainerRe: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 1 line · Kai Huang
So could we have your final suggestion? :)
Try the kvm_mmu_page_role.private approach. So long as it doesn't end up splattering code everywhere, I think that's more aligned with how KVM generally wants to treat the shared bit.
In the changelog for this patch, make it very clear that ensuring different aliases get different shadow page (if necessary) is the responsiblity of each individual feature that leverages stolen gfn bits.
Re: [RFC PATCH v2 41/69] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 6 lines · On Fri, 6 Aug 2021 22:09:35 +0000 Sean Christopherson wrote:
Quoted 1 line · Kai Huang
So could we have your final suggestion? :)
Try the kvm_mmu_page_role.private approach. So long as it doesn't end up splattering code everywhere, I think that's more aligned with how KVM generally wants to treat the shared bit.
OK.
Quoted 4 lines
In the changelog for this patch, make it very clear that ensuring different aliases get different shadow page (if necessary) is the responsiblity of each individual feature that leverages stolen gfn bits.
Make sense. Thanks.
RFC PATCH v2 42/69 KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
In reply to [email protected]From: Sean Christopherson <[email protected]>
Explicity check for an MMIO spte in the fast page fault flow. TDX will use a not-present entry for MMIO sptes, which can be mistaken for an access-tracked spte since both have SPTE_SPECIAL_MASK set.
--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -3060,7 +3060,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,break ; sp =sptep_to_sp (iterator.sptep); -if (!is_last_spte (spte, sp->role.level)) +if (!is_last_spte (spte, sp->role.level) ||is_mmio_spte (spte))break ;/*
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 42/69] KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
Quoted 18 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Explicity check for an MMIO spte in the fast page fault flow. TDX will use a not-present entry for MMIO sptes, which can be mistaken for an access-tracked spte since both have
SPTE_SPECIAL_MASKset.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/mmu/mmu.c +0
-0
arch/x86/kvm/mmu/mmu.c static int fast_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, · line 3066
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +1
-1@@ -3060,7 +3060,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,break ; sp =sptep_to_sp (iterator.sptep); -if (!is_last_spte (spte, sp->role.level)) +if (!is_last_spte (spte, sp->role.level) ||is_mmio_spte (spte))break ;/*
RFC PATCH v2 43/69 KVM: x86/mmu: Allow non-zero init value for shadow PTE
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX will run with EPT violation #VEs enabled, which means KVM needs to set the "suppress #VE" bit in unused PTEs to avoid unintentionally reflecting not-present EPT violations into the guest.
--- arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 50 +++++++++++++++++++++++++++++++++++------ arch/x86/kvm/mmu/spte.c | 10 +++++++++ arch/x86/kvm/mmu/spte.h | 2 ++ 4 files changed, 56 insertions(+), 7 deletions(-)
arch/x86/kvm/mmu.h +1 -0
@@ -61,6 +61,7 @@ static __always_inline u64 rsvd_bits(int s, int e)void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask);void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only); +void kvm_mmu_set_spte_init_value (u64 init_value);void reset_shadow_zero_bits_mask (struct kvm_vcpu *vcpu,struct kvm_mmu *context);
arch/x86/kvm/mmu/mmu.c +43 -7
87 lines not on this page · raw message
arch/x86/kvm/mmu/spte.c +10 -0
@@ -34,6 +34,7 @@ u64 __read_mostly shadow_mmio_access_mask;u64 __read_mostly shadow_present_mask;u64 __read_mostly shadow_me_mask;u64 __read_mostly shadow_acc_track_mask; +u64 __read_mostly shadow_init_value;u64 __read_mostly shadow_nonpresent_or_rsvd_mask;u64 __read_mostly shadow_nonpresent_or_rsvd_lower_gfn_mask;
@@ -211,6 +212,14 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn)return new_spte; } +void kvm_mmu_set_spte_init_value (u64 init_value) +{ +if (WARN_ON (!IS_ENABLED (CONFIG_X86_64 ) && init_value)) + init_value =0 ; + shadow_init_value = init_value; +} +EXPORT_SYMBOL_GPL (kvm_mmu_set_spte_init_value); +static u8 kvm_get_shadow_phys_bits (void ) {/*
@@ -355,6 +364,7 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_present_mask =PT_PRESENT_MASK ; shadow_acc_track_mask =0 ; shadow_me_mask = sme_me_mask; + shadow_init_value =0 ; shadow_host_writable_mask =DEFAULT_SPTE_HOST_WRITEABLE ; shadow_mmu_writable_mask =DEFAULT_SPTE_MMU_WRITEABLE ;
arch/x86/kvm/mmu/spte.h +2 -0
@@ -152,6 +152,8 @@ extern u64 __read_mostly shadow_mmio_access_mask;extern u64 __read_mostly shadow_present_mask;extern u64 __read_mostly shadow_me_mask; +extern u64 __read_mostly shadow_init_value; +/* * SPTEs in MMUs without A/D bits are marked with SPTE_TDP_AD_DISABLED_MASK; * shadow_acc_track_mask is the set of bits to be cleared in non-accessed
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 43/69] KVM: x86/mmu: Allow non-zero init value for shadow PTE
Quoted 6 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TDX will run with EPT violation #VEs enabled, which means KVM needs to set the "suppress #VE" bit in unused PTEs to avoid unintentionally reflecting not-present EPT violations into the guest.
Quoted 8 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 50 +++++++++++++++++++++++++++++++++++------ arch/x86/kvm/mmu/spte.c | 10 +++++++++ arch/x86/kvm/mmu/spte.h | 2 ++ 4 files changed, 56 insertions(+), 7 deletions(-)
Please ensure that this also works for tdp_mmu.c (if anything, consider supporting TDX only for TDP MMU; it's quite likely that mmu.c support for EPT/NPT will go away).
Paolo
Quoted 158 lines
arch/x86/kvm/mmu.h +1
-0@@ -61,6 +61,7 @@ static __always_inline u64 rsvd_bits(int s, int e)void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask);void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only); +void kvm_mmu_set_spte_init_value (u64 init_value);void reset_shadow_zero_bits_mask (struct kvm_vcpu *vcpu,struct kvm_mmu *context);arch/x86/kvm/mmu/mmu.c +43
-7@@ -550,9 +550,9 @@ static int mmu_spte_clear_track_bits(u64 *sptep)u64 old_spte = *sptep;if (!spte_has_volatile_bits (old_spte)) -__update_clear_spte_fast (sptep,0ull ); +__update_clear_spte_fast (sptep, shadow_init_value);else - old_spte =__update_clear_spte_slow (sptep,0ull ); + old_spte =__update_clear_spte_slow (sptep, shadow_init_value);if (!is_shadow_present_pte (old_spte))return 0 ;@@ -582,7 +582,7 @@ static int mmu_spte_clear_track_bits(u64 *sptep)*/ static void mmu_spte_clear_no_track (u64 *sptep) { -__update_clear_spte_fast (sptep,0ull ); +__update_clear_spte_fast (sptep, shadow_init_value); }static u64 mmu_spte_get_lockless (u64 *sptep)@@ -660,6 +660,42 @@ static void walk_shadow_page_lockless_end(struct kvm_vcpu *vcpu)local_irq_enable (); } +static inline void kvm_init_shadow_page (void *page) +{ +#ifdef CONFIG_X86_64 +int ign; + + asmvolatile ( +"rep stosq\n\t" + :"=c" (ign),"=D" (page) + :"a" (shadow_init_value),"c" (4096 /8 ),"D" (page) + :"memory" + ); +#else +BUG (); +#endif +} + +static int mmu_topup_shadow_page_cache (struct kvm_vcpu *vcpu) +{ +struct kvm_mmu_memory_cache *mc = &vcpu->arch.mmu_shadow_page_cache; +int start, end, i, r; + +if (shadow_init_value) + start =kvm_mmu_memory_cache_nr_free_objects (mc); + + r =kvm_mmu_topup_memory_cache (mc,PT64_ROOT_MAX_LEVEL ); +if (r) +return r; + +if (shadow_init_value) { + end =kvm_mmu_memory_cache_nr_free_objects (mc); +for (i = start; i < end; i++) +kvm_init_shadow_page (mc->objects[i]); + } +return 0 ; +} +static int mmu_topup_memory_caches (struct kvm_vcpu *vcpu,bool maybe_indirect) {int r;@@ -669,8 +705,7 @@ static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu, bool maybe_indirect) 1 + PT64_ROOT_MAX_LEVEL + PTE_PREFETCH_NUM);if (r)return r; - r =kvm_mmu_topup_memory_cache (&vcpu->arch.mmu_shadow_page_cache, -PT64_ROOT_MAX_LEVEL ); + r =mmu_topup_shadow_page_cache (vcpu);if (r)return r;if (maybe_indirect) {@@ -3041,7 +3076,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,struct kvm_shadow_walk_iterator iterator;struct kvm_mmu_page *sp;int ret =RET_PF_INVALID ; -u64 spte =0ull ; +u64 spte = shadow_init_value;uint retry_count =0 ;if (!page_fault_can_be_fast (error_code))@@ -5383,7 +5418,8 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu) vcpu->arch.mmu_page_header_cache.kmem_cache = mmu_page_header_cache; vcpu->arch.mmu_page_header_cache.gfp_zero = __GFP_ZERO; - vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO; +if (!shadow_init_value) + vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO; vcpu->arch.mmu = &vcpu->arch.root_mmu; vcpu->arch.walk_mmu = &vcpu->arch.root_mmu;arch/x86/kvm/mmu/spte.c +10
-0@@ -34,6 +34,7 @@ u64 __read_mostly shadow_mmio_access_mask;u64 __read_mostly shadow_present_mask;u64 __read_mostly shadow_me_mask;u64 __read_mostly shadow_acc_track_mask; +u64 __read_mostly shadow_init_value;u64 __read_mostly shadow_nonpresent_or_rsvd_mask;u64 __read_mostly shadow_nonpresent_or_rsvd_lower_gfn_mask;@@ -211,6 +212,14 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn)return new_spte; } +void kvm_mmu_set_spte_init_value (u64 init_value) +{ +if (WARN_ON (!IS_ENABLED (CONFIG_X86_64 ) && init_value)) + init_value =0 ; + shadow_init_value = init_value; +} +EXPORT_SYMBOL_GPL (kvm_mmu_set_spte_init_value); +static u8 kvm_get_shadow_phys_bits (void ) {/* @@ -355,6 +364,7 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_present_mask =PT_PRESENT_MASK ; shadow_acc_track_mask =0 ; shadow_me_mask = sme_me_mask; + shadow_init_value =0 ; shadow_host_writable_mask =DEFAULT_SPTE_HOST_WRITEABLE ; shadow_mmu_writable_mask =DEFAULT_SPTE_MMU_WRITEABLE ;arch/x86/kvm/mmu/spte.h +2
-0@@ -152,6 +152,8 @@ extern u64 __read_mostly shadow_mmio_access_mask;extern u64 __read_mostly shadow_present_mask;extern u64 __read_mostly shadow_me_mask; +extern u64 __read_mostly shadow_init_value; +/* * SPTEs in MMUs without A/D bits are marked with SPTE_TDP_AD_DISABLED_MASK; * shadow_acc_track_mask is the set of bits to be cleared in non-accessed
Re: [RFC PATCH v2 43/69] KVM: x86/mmu: Allow non-zero init value for shadow PTE
Quoted 16 lines · Paolo Bonzini
Quoted 6 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TDX will run with EPT violation #VEs enabled, which means KVM needs to set the "suppress #VE" bit in unused PTEs to avoid unintentionally reflecting not-present EPT violations into the guest.
Quoted 8 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 50 +++++++++++++++++++++++++++++++++++------ arch/x86/kvm/mmu/spte.c | 10 +++++++++ arch/x86/kvm/mmu/spte.h | 2 ++ 4 files changed, 56 insertions(+), 7 deletions(-)
Quoted 3 lines
Please ensure that this also works for
tdp_mmu.c(if anything, consider supporting TDX only for TDP MMU; it's quite likely thatmmu.csupport for EPT/NPT will go away).
It's on my TODO list. Will address it.
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH v2 44/69 KVM: x86/mmu: Refactor shadow walk in __direct_map() to reduce indentation
In reply to [email protected]From: Sean Christopherson <[email protected]>
Employ a 'continue' to reduce the indentation for linking a new shadow page during __direct_map() in preparation for linking private pages.
--- arch/x86/kvm/mmu/mmu.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-)
arch/x86/kvm/mmu/mmu.c +9 -10
@@ -2910,16 +2910,15 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,break ;drop_large_spte (vcpu, it.sptep); -if (!is_shadow_present_pte (*it.sptep)) { - sp =__kvm_mmu_get_page (vcpu, base_gfn, - gfn_stolen_bits, it.addr, - it.level -1 , true,ACC_ALL ); - -link_shadow_page (vcpu, it.sptep, sp); -if (is_tdp && huge_page_disallowed && - req_level >= it.level) -account_huge_nx_page (vcpu->kvm, sp); - } +if (is_shadow_present_pte (*it.sptep)) +continue ; + + sp =__kvm_mmu_get_page (vcpu, base_gfn, gfn_stolen_bits, + it.addr, it.level -1 , true,ACC_ALL ); + +link_shadow_page (vcpu, it.sptep, sp); +if (is_tdp && huge_page_disallowed && req_level >= it.level) +account_huge_nx_page (vcpu->kvm, sp); } ret = mmu_set_spte(vcpu, it.sptep, ACC_ALL,
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 44/69] KVM: x86/mmu: Refactor shadow walk in __direct_map() to reduce indentation
Quoted 32 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Employ a 'continue' to reduce the indentation for linking a new shadow page during
__direct_map()in preparation for linking private pages.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-)arch/x86/kvm/mmu/mmu.c +2
-10@@ -2910,16 +2910,15 @@ static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,break ;drop_large_spte (vcpu, it.sptep); -if (!is_shadow_present_pte (*it.sptep)) { - sp =__kvm_mmu_get_page (vcpu, base_gfn, - gfn_stolen_bits, it.addr, - it.level -1 , true,ACC_ALL ); - -link_shadow_page (vcpu, it.sptep, sp); -if (is_tdp && huge_page_disallowed && - req_level >= it.level) -account_huge_nx_page (vcpu->kvm, sp); - } +if (is_shadow_present_pte (*it.sptep)) +continue ;
arch/x86/kvm/mmu/mmu.c static int __direct_map(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, · line 2924
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +7
-0+ + sp =__kvm_mmu_get_page (vcpu, base_gfn, gfn_stolen_bits, + it.addr, it.level -1 , true,ACC_ALL ); + +link_shadow_page (vcpu, it.sptep, sp); +if (is_tdp && huge_page_disallowed && req_level >= it.level) +account_huge_nx_page (vcpu->kvm, sp); } ret = mmu_set_spte(vcpu, it.sptep, ACC_ALL,
RFC PATCH v2 45/69 KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
In reply to [email protected]From: Sean Christopherson <[email protected]>
Return the old SPTE when clearing a SPTE and push the "old SPTE present" check to the caller. Private shadow page support will use the old SPTE in rmap_remove() to determine whether or not there is a linked private shadow page.
--- arch/x86/kvm/mmu/mmu.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
arch/x86/kvm/mmu/mmu.c +7 -5
@@ -542,9 +542,9 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte)* Rules for using mmu_spte_clear_track_bits: * It sets the sptep from present to nonpresent, and track the * state bits, it is used to clear the last level sptep. -* Returns non-zero if the PTE was previously valid. +* Returns the old PTE. */ -static int mmu_spte_clear_track_bits (u64 *sptep) +static u64 mmu_spte_clear_track_bits (u64 *sptep) {kvm_pfn_t pfn;u64 old_spte = *sptep;
@@ -555,7 +555,7 @@ static int mmu_spte_clear_track_bits(u64 *sptep) old_spte =__update_clear_spte_slow (sptep, shadow_init_value);if (!is_shadow_present_pte (old_spte)) -return 0 ; +return old_spte; pfn =spte_to_pfn (old_spte);
@@ -572,7 +572,7 @@ static int mmu_spte_clear_track_bits(u64 *sptep)if (is_dirty_spte (old_spte))kvm_set_pfn_dirty (pfn); -return 1 ; +return old_spte; }/*
@@ -1104,7 +1104,9 @@ static u64 *rmap_get_next(struct rmap_iterator *iter)static void drop_spte (struct kvm *kvm,u64 *sptep) { -if (mmu_spte_clear_track_bits (sptep)) +u64 old_spte =mmu_spte_clear_track_bits (sptep); + +if (is_shadow_present_pte (old_spte))rmap_remove (kvm, sptep); }
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 45/69] KVM: x86/mmu: Return old SPTE from mmu_spte_clear_track_bits()
Quoted 49 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Return the old SPTE when clearing a SPTE and push the "old SPTE present" check to the caller. Private shadow page support will use the old SPTE in
rmap_remove()to determine whether or not there is a linked private shadow page.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)arch/x86/kvm/mmu/mmu.c +4
-4@@ -542,9 +542,9 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte)* Rules for using mmu_spte_clear_track_bits: * It sets the sptep from present to nonpresent, and track the * state bits, it is used to clear the last level sptep. -* Returns non-zero if the PTE was previously valid. +* Returns the old PTE. */ -static int mmu_spte_clear_track_bits (u64 *sptep) +static u64 mmu_spte_clear_track_bits (u64 *sptep) {kvm_pfn_t pfn;u64 old_spte = *sptep;@@ -555,7 +555,7 @@ static int mmu_spte_clear_track_bits(u64 *sptep) old_spte =__update_clear_spte_slow (sptep, shadow_init_value);if (!is_shadow_present_pte (old_spte)) -return 0 ; +return old_spte; pfn =spte_to_pfn (old_spte);@@ -572,7 +572,7 @@ static int mmu_spte_clear_track_bits(u64 *sptep)if (is_dirty_spte (old_spte))kvm_set_pfn_dirty (pfn); -return 1 ; +return old_spte; }/*
arch/x86/kvm/mmu/mmu.c static u64 *rmap_get_next(struct rmap_iterator *iter) · line 1112
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +3
-1@@ -1104,7 +1104,9 @@ static u64 *rmap_get_next(struct rmap_iterator *iter)static void drop_spte (struct kvm *kvm,u64 *sptep) { -if (mmu_spte_clear_track_bits (sptep)) +u64 old_spte =mmu_spte_clear_track_bits (sptep); + +if (is_shadow_present_pte (old_spte))rmap_remove (kvm, sptep); }
RFC PATCH v2 46/69 KVM: x86/mmu: Frame in support for private/inaccessible shadow pages
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add kvm_x86_ops hooks to set/clear private SPTEs, i.e. SEPT entries, and to link/free private shadow pages, i.e. non-leaf SEPT pages.
Because SEAMCALLs are bloody expensive, and because KVM's MMU is already complex enough, TDX's SEPT will mirror KVM's shadow pages instead of replacing them outright. This costs extra memory, but is simpler and far more performant.
Add a separate list for tracking active private shadow pages. Zapping and freeing SEPT entries is subject to very different rules than normal pages/memory, and need to be preserved (along with their shadow page counterparts) when KVM gets trigger happy, e.g. zaps everything during a memslot update.
Zap any aliases of a GPA when mapping in a guest that supports guest private GPAs. This is necessary to avoid integrity failures with TDX due to pointing shared and private GPAs at the same HPA.
Do not prefetch private pages (this should probably be a property of the VM).
TDX needs one more bits in spte for SPTE_PRIVATE_ZAPPED. Steal 1 bit from MMIO generation as 62 bit and re-purpose it for non-present SPTE, SPTE_PRIVATE_ZAPPED.
--- arch/x86/include/asm/kvm-x86-ops.h | 6 + arch/x86/include/asm/kvm_host.h | 24 ++- arch/x86/kvm/mmu.h | 3 +- arch/x86/kvm/mmu/mmu.c | 292 ++++++++++++++++++++++++----- arch/x86/kvm/mmu/mmu_internal.h | 2 + arch/x86/kvm/mmu/spte.h | 16 +- arch/x86/kvm/x86.c | 4 +- 7 files changed, 293 insertions(+), 54 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +6 -0
@@ -88,6 +88,12 @@ KVM_X86_OP(set_tss_addr) KVM_X86_OP(set_identity_map_addr) KVM_X86_OP(get_mt_mask) KVM_X86_OP(load_mmu_pgd) +KVM_X86_OP(set_private_spte) +KVM_X86_OP(drop_private_spte) +KVM_X86_OP(zap_private_spte) +KVM_X86_OP(unzap_private_spte) +KVM_X86_OP(link_private_sp) +KVM_X86_OP(free_private_sp) KVM_X86_OP_NULL(has_wbinvd_exit) KVM_X86_OP(write_l1_tsc_offset) KVM_X86_OP(get_exit_info)
arch/x86/include/asm/kvm_host.h +22 -2
@@ -385,6 +385,7 @@ struct kvm_mmu { struct kvm_mmu_page *sp);void (*invlpg)(struct kvm_vcpu *vcpu,gva_t gva,hpa_t root_hpa);hpa_t root_hpa; +hpa_t private_root_hpa;gpa_t root_pgd;union kvm_mmu_role mmu_role;u8 root_level;
@@ -424,6 +425,7 @@ struct kvm_mmu {u8 last_nonleaf_level;bool nx; +bool no_prefetch;u64 pdptrs[4 ];/* pae */ };
@@ -639,6 +641,7 @@ struct kvm_vcpu_arch {struct kvm_mmu_memory_cache mmu_shadow_page_cache;struct kvm_mmu_memory_cache mmu_gfn_array_cache;struct kvm_mmu_memory_cache mmu_page_header_cache; +struct kvm_mmu_memory_cache mmu_private_sp_cache;/* * QEMU userspace and the guest each have their own FPU state.
@@ -989,6 +992,7 @@ struct kvm_arch {u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages; +struct list_head private_mmu_pages;struct list_head zapped_obsolete_pages;struct list_head lpage_disallowed_mmu_pages;struct kvm_page_track_notifier_node mmu_sp_tracker;
@@ -1137,6 +1141,8 @@ struct kvm_arch {*/ spinlock_t tdp_mmu_pages_lock;#endif /* CONFIG_X86_64 */ + +gfn_t gfn_shared_mask; };struct kvm_vm_stat {
@@ -1319,6 +1325,17 @@ struct kvm_x86_ops {void (*load_mmu_pgd)(struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +void (*set_private_spte)(struct kvm_vcpu *vcpu,gfn_t gfn,int level, +kvm_pfn_t pfn); +void (*drop_private_spte)(struct kvm *kvm,gfn_t gfn,int level, +kvm_pfn_t pfn); +void (*zap_private_spte)(struct kvm *kvm,gfn_t gfn,int level); +void (*unzap_private_spte)(struct kvm *kvm,gfn_t gfn,int level); +int (*link_private_sp)(struct kvm_vcpu *vcpu,gfn_t gfn,int level, +void *private_sp); +int (*free_private_sp)(struct kvm *kvm,gfn_t gfn,int level, +void *private_sp); +bool (*has_wbinvd_exit)(void );/* Returns actual tsc_offset set in active VMCS */
@@ -1490,7 +1507,8 @@ void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm, const struct kvm_memory_slot *memslot);void kvm_mmu_slot_leaf_clear_dirty (struct kvm *kvm,struct kvm_memory_slot *memslot); -void kvm_mmu_zap_all (struct kvm *kvm); +void kvm_mmu_zap_all_active (struct kvm *kvm); +void kvm_mmu_zap_all_private (struct kvm *kvm);void kvm_mmu_invalidate_mmio_sptes (struct kvm *kvm,u64 gen);unsigned long kvm_mmu_calculate_default_mmu_pages (struct kvm *kvm);void kvm_mmu_change_mmu_pages (struct kvm *kvm,unsigned long kvm_nr_mmu_pages);
@@ -1656,7 +1674,9 @@ static inline int __kvm_irq_line_state(unsigned long *irq_state,#define KVM_MMU_ROOT_CURRENT BIT(0)#define KVM_MMU_ROOT_PREVIOUS (i) BIT(1+i) -#define KVM_MMU_ROOTS_ALL (~0UL) +#define KVM_MMU_ROOT_PRIVATE BIT(1+KVM_MMU_NUM_PREV_ROOTS) +#define KVM_MMU_ROOTS_ALL ((u32)(~KVM_MMU_ROOT_PRIVATE)) +#define KVM_MMU_ROOTS_ALL_INC_PRIVATE (KVM_MMU_ROOTS_ALL | KVM_MMU_ROOT_PRIVATE)int kvm_pic_set_irq (struct kvm_pic *pic,int irq,int irq_source_id,int level);void kvm_pic_clear_all (struct kvm_pic *pic,int irq_source_id);
arch/x86/kvm/mmu.h +1 -2
@@ -235,8 +235,7 @@ void kvm_mmu_pre_destroy_vm(struct kvm *kvm);static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) { -/* Currently there are no stolen bits in KVM */ -return 0 ; +return kvm->arch.gfn_shared_mask; }static inline gfn_t vcpu_gfn_stolen_mask (struct kvm_vcpu *vcpu)
arch/x86/kvm/mmu/mmu.c +247 -45
603 lines not on this page · raw message
arch/x86/kvm/mmu/mmu_internal.h +2 -0
@@ -51,6 +51,8 @@ struct kvm_mmu_page {u64 *spt;/* hold the gfn of each spte inside spt */ gfn_t *gfns; +/* associated private shadow page, e.g. SEPT page */ +void *private_sp;/* Currently serving as active root */ union {int root_count;
arch/x86/kvm/mmu/spte.h +12 -4
@@ -14,6 +14,9 @@*/ #define SPTE_MMU_PRESENT_MASK BIT_ULL(11) +/* Masks that used to track metadata for not-present SPTEs. */ +#define SPTE_PRIVATE_ZAPPED BIT_ULL(62) +/* * TDP SPTES (more specifically, EPT SPTEs) may not have A/D bits, and may also * be restricted to using write-protection (for L2 when CPU dirty logging, i.e.
@@ -101,11 +104,11 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRACK_SAVED_MASK));#undef SHADOW_ACC_TRACK_SAVED_MASK/* -* Due to limited space in PTEs, the MMIO generation is a 19 bit subset of +* Due to limited space in PTEs, the MMIO generation is a 18 bit subset of * the memslots generation and is derived as follows: * * Bits 0-7 of the MMIO generation are propagated to spte bits 3-10 -* Bits 8-18 of the MMIO generation are propagated to spte bits 52-62 +* Bits 8-17 of the MMIO generation are propagated to spte bits 52-61 * * The KVM_MEMSLOT_GEN_UPDATE_IN_PROGRESS flag is intentionally not included in * the MMIO generation number, as doing so would require stealing a bit from
@@ -119,7 +122,7 @@ static_assert(!(EPT_SPTE_MMU_WRITABLE & SHADOW_ACC_TRACK_SAVED_MASK));#define MMIO_SPTE_GEN_LOW_END 10#define MMIO_SPTE_GEN_HIGH_START 52 -#define MMIO_SPTE_GEN_HIGH_END 62 +#define MMIO_SPTE_GEN_HIGH_END 61#define MMIO_SPTE_GEN_LOW_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_END, \ MMIO_SPTE_GEN_LOW_START)
@@ -132,7 +135,7 @@ static_assert(!(SPTE_MMU_PRESENT_MASK &#define MMIO_SPTE_GEN_HIGH_BITS (MMIO_SPTE_GEN_HIGH_END - MMIO_SPTE_GEN_HIGH_START + 1)/* remember to adjust the comment above as well if you change these */ -static_assert (MMIO_SPTE_GEN_LOW_BITS ==8 &&MMIO_SPTE_GEN_HIGH_BITS ==11 ); +static_assert (MMIO_SPTE_GEN_LOW_BITS ==8 &&MMIO_SPTE_GEN_HIGH_BITS ==10 );#define MMIO_SPTE_GEN_LOW_SHIFT (MMIO_SPTE_GEN_LOW_START - 0)#define MMIO_SPTE_GEN_HIGH_SHIFT (MMIO_SPTE_GEN_HIGH_START - MMIO_SPTE_GEN_LOW_BITS)
@@ -260,6 +263,11 @@ static inline bool is_access_track_spte(u64 spte)return !spte_ad_enabled (spte) && (spte & shadow_acc_track_mask) ==0 ; } +static inline bool is_zapped_private_pte (u64 pte) +{ +return !!(pte &SPTE_PRIVATE_ZAPPED ); +} +static inline bool is_large_pte (u64 pte) {return pte &PT_PAGE_SIZE_MASK ;
arch/x86/kvm/x86.c +3 -1
@@ -10874,6 +10874,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)INIT_HLIST_HEAD (&kvm->arch.mask_notifier_list);INIT_LIST_HEAD (&kvm->arch.active_mmu_pages); +INIT_LIST_HEAD (&kvm->arch.private_mmu_pages);INIT_LIST_HEAD (&kvm->arch.zapped_obsolete_pages);INIT_LIST_HEAD (&kvm->arch.lpage_disallowed_mmu_pages);INIT_LIST_HEAD (&kvm->arch.assigned_dev_head);
@@ -11299,7 +11300,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,void kvm_arch_flush_shadow_all (struct kvm *kvm) { -kvm_mmu_zap_all (kvm); +/* Zapping private pages must be deferred until VM destruction. */ +kvm_mmu_zap_all_active (kvm); } void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
Signature · 1 line
2.25.1
RFC PATCH v2 47/69 KVM: x86/mmu: Move 'pfn' variable to caller of direct_page_fault()
In reply to [email protected]From: Sean Christopherson <[email protected]>
When adding pages prior to boot, TDX will need the resulting host pfn so that it can be passed to TDADDPAGE (TDX-SEAM always works with physical addresses as it has its own page tables). Start plumbing pfn back up the page fault stack.
--- arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-)
arch/x86/kvm/mmu/mmu.c +15 -11
@@ -3969,14 +3969,14 @@ static bool try_async_pf(struct kvm_vcpu *vcpu, bool prefault, gfn_t gfn, }static int direct_page_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u32 error_code, -bool prefault,int max_level,bool is_tdp) +bool prefault,int max_level,bool is_tdp, +kvm_pfn_t *pfn) {bool write = error_code &PFERR_WRITE_MASK ;bool map_writable;gfn_t gfn =vcpu_gpa_to_gfn_unalias (vcpu, gpa);unsigned long mmu_seq; -kvm_pfn_t pfn;hva_t hva;int r;
@@ -3996,11 +3996,11 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, mmu_seq = vcpu->kvm->mmu_notifier_seq;smp_rmb (); -if (try_async_pf (vcpu, prefault, gfn, gpa, &pfn, &hva, +if (try_async_pf (vcpu, prefault, gfn, gpa, pfn, &hva, write, &map_writable))return RET_PF_RETRY ; -if (handle_abnormal_pfn (vcpu, is_tdp ?0 : gpa, gfn, pfn,ACC_ALL , &r)) +if (handle_abnormal_pfn (vcpu, is_tdp ?0 : gpa, gfn, *pfn,ACC_ALL , &r))return r; r =RET_PF_RETRY ;
@@ -4010,7 +4010,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,else write_lock (&vcpu->kvm->mmu_lock); -if (!is_noslot_pfn (pfn) &&mmu_notifier_retry_hva (vcpu->kvm, mmu_seq, hva)) +if (!is_noslot_pfn (*pfn) && +mmu_notifier_retry_hva (vcpu->kvm, mmu_seq, hva)) goto out_unlock; r =make_mmu_pages_available (vcpu);if (r)
@@ -4018,28 +4019,30 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,if (is_tdp_mmu_root (vcpu->kvm, vcpu->arch.mmu->root_hpa)) r =kvm_tdp_mmu_map (vcpu, gpa, error_code, map_writable, max_level, - pfn, prefault); + *pfn, prefault);else - r =__direct_map (vcpu, gpa, error_code, map_writable, max_level, pfn, - prefault, is_tdp); + r =__direct_map (vcpu, gpa, error_code, map_writable, max_level, + *pfn, prefault, is_tdp); out_unlock:if (is_tdp_mmu_root (vcpu->kvm, vcpu->arch.mmu->root_hpa))read_unlock (&vcpu->kvm->mmu_lock);else write_unlock (&vcpu->kvm->mmu_lock); -kvm_release_pfn_clean (pfn); +kvm_release_pfn_clean (*pfn);return r; }static int nonpaging_page_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u32 error_code,bool prefault) { +kvm_pfn_t pfn; +pgprintk ("%s: gva %lx error %x\n" , __func__, gpa, error_code);/* This path builds a PAE pagetable, we can map 2mb pages at maximum. */ return direct_page_fault (vcpu, gpa &PAGE_MASK , error_code, prefault, -PG_LEVEL_2M , false); +PG_LEVEL_2M , false, &pfn); } int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code,
@@ -4078,6 +4081,7 @@ EXPORT_SYMBOL_GPL(kvm_handle_page_fault);int kvm_tdp_page_fault (struct kvm_vcpu *vcpu,gpa_t gpa,u32 error_code,bool prefault) { +kvm_pfn_t pfn;int max_level; for (max_level = KVM_MAX_HUGEPAGE_LEVEL;
@@ -4091,7 +4095,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, }return direct_page_fault (vcpu, gpa, error_code, prefault, - max_level, true); + max_level, true, &pfn); } static void nonpaging_init_context(struct kvm_vcpu *vcpu,
Signature · 1 line
2.25.1
RFC PATCH v2 48/69 KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows TDX to get the resulting pfn and also allows the RET_PF_* enums to stay in mmu.c where they belong.
--- arch/x86/kvm/mmu.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+)
arch/x86/kvm/mmu.h +3 -0
@@ -125,6 +125,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,return vcpu->arch.mmu->page_fault (vcpu, cr2_or_gpa, err, prefault); } +kvm_pfn_t kvm_mmu_map_tdp_page (struct kvm_vcpu *vcpu,gpa_t gpa, +u32 error_code,int max_level); +/* * Currently, we have two sorts of write-protection, a) the first one * write-protects guest page to sync the guest modification, b) another one is
arch/x86/kvm/mmu/mmu.c +25 -0
@@ -4098,6 +4098,31 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code, max_level, true, &pfn); } +kvm_pfn_t kvm_mmu_map_tdp_page (struct kvm_vcpu *vcpu,gpa_t gpa, +u32 error_code,int max_level) +{ +kvm_pfn_t pfn; +int r; + +if (mmu_topup_memory_caches (vcpu, false)) +return KVM_PFN_ERR_FAULT ; + +/* +* Loop on the page fault path to handle the case where an mmu_notifier +* invalidation triggers RET_PF_RETRY. In the normal page fault path, +* KVM needs to resume the guest in case the invalidation changed any +* of the page fault properties, i.e. the gpa or error code. For this +* path, the gpa and error code are fixed by the caller, and the caller +* expects failure if and only if the page fault can't be fixed. +*/ +do { + r =direct_page_fault (vcpu, gpa, error_code, false, max_level, + true, &pfn); + }while (r ==RET_PF_RETRY && !is_error_noslot_pfn (pfn)); +return pfn; +} +EXPORT_SYMBOL_GPL (kvm_mmu_map_tdp_page); +static void nonpaging_init_context (struct kvm_vcpu *vcpu,struct kvm_mmu *context) {
Signature · 1 line
2.25.1
RFC PATCH v2 49/69 KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
In reply to [email protected]From: Sean Christopherson <[email protected]>
Pass intr_info to the NMI and INTR handlers instead of pulling it from vcpu_vmx in preparation for sharing the bulk of the handlers with TDX.
--- arch/x86/kvm/vmx/vmx.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/vmx.c +7 -8
@@ -6404,25 +6404,24 @@ static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,kvm_after_interrupt (vcpu); } -static void handle_exception_nmi_irqoff (struct vcpu_vmx *vmx) +static void handle_exception_nmi_irqoff (struct kvm_vcpu *vcpu,u32 intr_info) {const unsigned long nmi_entry = (unsigned long )asm_exc_nmi_noist; -u32 intr_info =vmx_get_intr_info (&vmx->vcpu);/* if exit due to PF check for async PF */ if (is_page_fault (intr_info)) - vmx->vcpu.arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); + vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags ();/* Handle machine checks before interrupts are enabled */ else if (is_machine_check (intr_info))kvm_machine_check ();/* We need to handle NMIs before interrupts are enabled */ else if (is_nmi (intr_info)) -handle_interrupt_nmi_irqoff (&vmx->vcpu, nmi_entry); +handle_interrupt_nmi_irqoff (vcpu, nmi_entry); } -static void handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu) +static void handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) { -u32 intr_info =vmx_get_intr_info (vcpu);unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ;gate_desc *desc = (gate_desc *)host_idt_base + vector;
@@ -6438,9 +6437,9 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);if (vmx->exit_reason.basic ==EXIT_REASON_EXTERNAL_INTERRUPT ) -handle_external_interrupt_irqoff (vcpu); +handle_external_interrupt_irqoff (vcpu,vmx_get_intr_info (vcpu));else if (vmx->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI ) -handle_exception_nmi_irqoff (vmx); +handle_exception_nmi_irqoff (vcpu,vmx_get_intr_info (vcpu)); }/*
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 49/69] KVM: VMX: Modify NMI and INTR handlers to take intr_info as param
Quoted 49 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Pass
intr_infoto the NMI and INTR handlers instead of pulling it fromvcpu_vmxin preparation for sharing the bulk of the handlers with TDX.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)arch/x86/kvm/vmx/vmx.c +5
-6@@ -6404,25 +6404,24 @@ static void handle_interrupt_nmi_irqoff(struct kvm_vcpu *vcpu,kvm_after_interrupt (vcpu); } -static void handle_exception_nmi_irqoff (struct vcpu_vmx *vmx) +static void handle_exception_nmi_irqoff (struct kvm_vcpu *vcpu,u32 intr_info) {const unsigned long nmi_entry = (unsigned long )asm_exc_nmi_noist; -u32 intr_info =vmx_get_intr_info (&vmx->vcpu);/* if exit due to PF check for async PF */ if (is_page_fault (intr_info)) - vmx->vcpu.arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); + vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags ();/* Handle machine checks before interrupts are enabled */ else if (is_machine_check (intr_info))kvm_machine_check ();/* We need to handle NMIs before interrupts are enabled */ else if (is_nmi (intr_info)) -handle_interrupt_nmi_irqoff (&vmx->vcpu, nmi_entry); +handle_interrupt_nmi_irqoff (vcpu, nmi_entry); } -static void handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu) +static void handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) { -u32 intr_info =vmx_get_intr_info (vcpu);unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ;gate_desc *desc = (gate_desc *)host_idt_base + vector;@@ -6438,9 +6437,9 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);
arch/x86/kvm/vmx/vmx.c static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) · line 6445
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +2
-2if (vmx->exit_reason.basic ==EXIT_REASON_EXTERNAL_INTERRUPT ) -handle_external_interrupt_irqoff (vcpu); +handle_external_interrupt_irqoff (vcpu,vmx_get_intr_info (vcpu));else if (vmx->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI ) -handle_exception_nmi_irqoff (vmx); +handle_exception_nmi_irqoff (vcpu,vmx_get_intr_info (vcpu)); }/*
RFC PATCH v2 50/69 KVM: VMX: Move NMI/exception handler to common helper
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/common.h | 52 +++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 52 ++++++--------------------------------- 2 files changed, 60 insertions(+), 44 deletions(-) create mode 100644 arch/x86/kvm/vmx/common.h
arch/x86/kvm/vmx/common.h new file +52 -0
@@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __KVM_X86_VMX_COMMON_H +#define __KVM_X86_VMX_COMMON_H + +#include <linux/kvm_host.h> + +#include <asm/traps.h> + +#include "vmcs.h" +#include "x86.h" + +extern unsigned long vmx_host_idt_base; +void vmx_do_interrupt_nmi_irqoff (unsigned long entry); + +static inline void vmx_handle_interrupt_nmi_irqoff (struct kvm_vcpu *vcpu, +unsigned long entry) +{ +kvm_before_interrupt (vcpu); +vmx_do_interrupt_nmi_irqoff (entry); +kvm_after_interrupt (vcpu); +} + +static inline void vmx_handle_exception_nmi_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) +{ +const unsigned long nmi_entry = (unsigned long )asm_exc_nmi_noist; + +/* if exit due to PF check for async PF */ +if (is_page_fault (intr_info)) + vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); +/* Handle machine checks before interrupts are enabled */ +else if (is_machine_check (intr_info)) +kvm_machine_check (); +/* We need to handle NMIs before interrupts are enabled */ +else if (is_nmi (intr_info)) +vmx_handle_interrupt_nmi_irqoff (vcpu, nmi_entry); +} + +static inline void vmx_handle_external_interrupt_irqoff (struct kvm_vcpu *vcpu, +u32 intr_info) +{ +unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ; +gate_desc *desc = (gate_desc *)vmx_host_idt_base + vector; + +if (KVM_BUG (!is_external_intr (intr_info), vcpu->kvm, +"KVM: unexpected VM-Exit interrupt info: 0x%x" , intr_info)) +return ; + +vmx_handle_interrupt_nmi_irqoff (vcpu,gate_offset (desc)); +} + +#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +8 -44
94 lines not on this page · raw message
Signature · 1 line
2.25.1
RFC PATCH v2 51/69 KVM: x86/mmu: Allow per-VM override of the TDP max page level
In reply to [email protected]From: Sean Christopherson <[email protected]>
TODO: This is tentative patch. Support large page and delete this patch.
Allow TDX to effectively disable large pages, as SEPT will initially support only 4k pages.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -989,6 +989,7 @@ struct kvm_arch {unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;unsigned int indirect_shadow_pages; +int tdp_max_page_level;u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages;
arch/x86/kvm/mmu/mmu.c +3 -1
@@ -4084,7 +4084,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,kvm_pfn_t pfn;int max_level; -for (max_level =KVM_MAX_HUGEPAGE_LEVEL ; +for (max_level = vcpu->kvm->arch.tdp_max_page_level; max_level >PG_LEVEL_4K ; max_level--) {int page_num =KVM_PAGES_PER_HPAGE (max_level);
@@ -5802,6 +5802,8 @@ void kvm_mmu_init_vm(struct kvm *kvm) node->track_write = kvm_mmu_pte_write; node->track_flush_slot = kvm_mmu_invalidate_zap_pages_in_memslot;kvm_page_track_register_notifier (kvm, node); + + kvm->arch.tdp_max_page_level =KVM_MAX_HUGEPAGE_LEVEL ; }void kvm_mmu_uninit_vm (struct kvm *kvm)
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 51/69] KVM: x86/mmu: Allow per-VM override of the TDP max page level
Quoted 41 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TODO: This is tentative patch. Support large page and delete this patch.
Allow TDX to effectively disable large pages, as SEPT will initially support only 4k pages.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm_host.h +1
-0@@ -989,6 +989,7 @@ struct kvm_arch {unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;unsigned int indirect_shadow_pages; +int tdp_max_page_level;u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages;arch/x86/kvm/mmu/mmu.c +1
-1@@ -4084,7 +4084,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,kvm_pfn_t pfn;int max_level; -for (max_level =KVM_MAX_HUGEPAGE_LEVEL ; +for (max_level = vcpu->kvm->arch.tdp_max_page_level; max_level >PG_LEVEL_4K ; max_level--) {int page_num =KVM_PAGES_PER_HPAGE (max_level);
arch/x86/kvm/mmu/mmu.c void kvm_mmu_init_vm(struct kvm *kvm) · line 5809
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c +2
-0@@ -5802,6 +5802,8 @@ void kvm_mmu_init_vm(struct kvm *kvm) node->track_write = kvm_mmu_pte_write; node->track_flush_slot = kvm_mmu_invalidate_zap_pages_in_memslot;kvm_page_track_register_notifier (kvm, node); + + kvm->arch.tdp_max_page_level =KVM_MAX_HUGEPAGE_LEVEL ; }void kvm_mmu_uninit_vm (struct kvm *kvm)
Seems good enough for now.
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 51/69] KVM: x86/mmu: Allow per-VM override of the TDP max page level
Quoted 7 lines · Paolo Bonzini
Quoted 6 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TODO: This is tentative patch. Support large page and delete this patch.
Allow TDX to effectively disable large pages, as SEPT will initially support only 4k pages.
...
Quoted 1 line
Seems good enough for now.
Looks like SNP needs a dynamic check, i.e. a kvm_x86_ops hook, to handle an edge case in the RMP. That's probably the better route given that this is a short-term hack (hopefully :-D).
RFC PATCH v2 52/69 KVM: VMX: Split out guts of EPT violation to common/exposed function
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/common.h | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 33 +++++---------------------------- 2 files changed, 34 insertions(+), 28 deletions(-)
arch/x86/kvm/vmx/common.h +29 -0
@@ -5,8 +5,11 @@#include <linux/kvm_host.h> #include <asm/traps.h> +#include <asm/vmx.h> +#include "mmu.h" #include "vmcs.h" +#include "vmx.h" #include "x86.h" extern unsigned long vmx_host_idt_base;
@@ -49,4 +52,30 @@ static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, vmx_handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc)); } +static inline int __vmx_handle_ept_violation (struct kvm_vcpu *vcpu,gpa_t gpa, +unsigned long exit_qualification) +{ +u64 error_code; + +/* Is it a read fault? */ + error_code = (exit_qualification &EPT_VIOLATION_ACC_READ ) + ?PFERR_USER_MASK :0 ; +/* Is it a write fault? */ + error_code |= (exit_qualification &EPT_VIOLATION_ACC_WRITE ) + ?PFERR_WRITE_MASK :0 ; +/* Is it a fetch fault? */ + error_code |= (exit_qualification &EPT_VIOLATION_ACC_INSTR ) + ?PFERR_FETCH_MASK :0 ; +/* ept page table entry is present? */ + error_code |= (exit_qualification & + (EPT_VIOLATION_READABLE |EPT_VIOLATION_WRITABLE | +EPT_VIOLATION_EXECUTABLE )) + ?PFERR_PRESENT_MASK :0 ; + + error_code |= (exit_qualification &EPT_VIOLATION_GVA_TRANSLATED ) !=0 ? +PFERR_GUEST_FINAL_MASK :PFERR_GUEST_PAGE_MASK ; + +return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +} +#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +5 -28
@@ -5328,11 +5328,10 @@ static int handle_task_switch(struct kvm_vcpu *vcpu)static int handle_ept_violation (struct kvm_vcpu *vcpu) { -unsigned long exit_qualification; -gpa_t gpa; -u64 error_code; +unsigned long exit_qualification =vmx_get_exit_qual (vcpu); +gpa_t gpa =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); - exit_qualification =vmx_get_exit_qual (vcpu); +trace_kvm_page_fault (gpa, exit_qualification);/* * EPT violation happened while executing iret from NMI,
@@ -5341,31 +5340,9 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)* AAK134, BY25. */ if (!(to_vmx (vcpu)->idt_vectoring_info &VECTORING_INFO_VALID_MASK ) && - enable_vnmi && - (exit_qualification &INTR_INFO_UNBLOCK_NMI )) + enable_vnmi && (exit_qualification &INTR_INFO_UNBLOCK_NMI ))vmcs_set_bits (GUEST_INTERRUPTIBILITY_INFO ,GUEST_INTR_STATE_NMI ); - gpa =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); -trace_kvm_page_fault (gpa, exit_qualification); - -/* Is it a read fault? */ - error_code = (exit_qualification &EPT_VIOLATION_ACC_READ ) - ?PFERR_USER_MASK :0 ; -/* Is it a write fault? */ - error_code |= (exit_qualification &EPT_VIOLATION_ACC_WRITE ) - ?PFERR_WRITE_MASK :0 ; -/* Is it a fetch fault? */ - error_code |= (exit_qualification &EPT_VIOLATION_ACC_INSTR ) - ?PFERR_FETCH_MASK :0 ; -/* ept page table entry is present? */ - error_code |= (exit_qualification & - (EPT_VIOLATION_READABLE |EPT_VIOLATION_WRITABLE | -EPT_VIOLATION_EXECUTABLE )) - ?PFERR_PRESENT_MASK :0 ; - - error_code |= (exit_qualification &EPT_VIOLATION_GVA_TRANSLATED ) !=0 ? -PFERR_GUEST_FINAL_MASK :PFERR_GUEST_PAGE_MASK ; - vcpu->arch.exit_qualification = exit_qualification;/*
@@ -5379,7 +5356,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)if (unlikely (allow_smaller_maxphyaddr &&kvm_vcpu_is_illegal_gpa (vcpu, gpa)))return kvm_emulate_instruction (vcpu,0 ); -return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +return __vmx_handle_ept_violation (vcpu, gpa, exit_qualification); }static int handle_ept_misconfig (struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 52/69] KVM: VMX: Split out guts of EPT violation to common/exposed function
Quoted 110 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/common.h | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 33 +++++---------------------------- 2 files changed, 34 insertions(+), 28 deletions(-)arch/x86/kvm/vmx/common.h +29
-0@@ -5,8 +5,11 @@#include <linux/kvm_host.h> #include <asm/traps.h> +#include <asm/vmx.h> +#include "mmu.h" #include "vmcs.h" +#include "vmx.h" #include "x86.h" extern unsigned long vmx_host_idt_base;@@ -49,4 +52,30 @@ static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, vmx_handle_interrupt_nmi_irqoff(vcpu, gate_offset(desc)); } +static inline int __vmx_handle_ept_violation (struct kvm_vcpu *vcpu,gpa_t gpa, +unsigned long exit_qualification) +{ +u64 error_code; + +/* Is it a read fault? */ + error_code = (exit_qualification &EPT_VIOLATION_ACC_READ ) + ?PFERR_USER_MASK :0 ; +/* Is it a write fault? */ + error_code |= (exit_qualification &EPT_VIOLATION_ACC_WRITE ) + ?PFERR_WRITE_MASK :0 ; +/* Is it a fetch fault? */ + error_code |= (exit_qualification &EPT_VIOLATION_ACC_INSTR ) + ?PFERR_FETCH_MASK :0 ; +/* ept page table entry is present? */ + error_code |= (exit_qualification & + (EPT_VIOLATION_READABLE |EPT_VIOLATION_WRITABLE | +EPT_VIOLATION_EXECUTABLE )) + ?PFERR_PRESENT_MASK :0 ; + + error_code |= (exit_qualification &EPT_VIOLATION_GVA_TRANSLATED ) !=0 ? +PFERR_GUEST_FINAL_MASK :PFERR_GUEST_PAGE_MASK ; + +return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +} +#endif /* __KVM_X86_VMX_COMMON_H */ arch/x86/kvm/vmx/vmx.c +4
-27@@ -5328,11 +5328,10 @@ static int handle_task_switch(struct kvm_vcpu *vcpu)static int handle_ept_violation (struct kvm_vcpu *vcpu) { -unsigned long exit_qualification; -gpa_t gpa; -u64 error_code; +unsigned long exit_qualification =vmx_get_exit_qual (vcpu); +gpa_t gpa =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); - exit_qualification =vmx_get_exit_qual (vcpu); +trace_kvm_page_fault (gpa, exit_qualification);/* * EPT violation happened while executing iret from NMI, @@ -5341,31 +5340,9 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)* AAK134, BY25. */ if (!(to_vmx (vcpu)->idt_vectoring_info &VECTORING_INFO_VALID_MASK ) && - enable_vnmi && - (exit_qualification &INTR_INFO_UNBLOCK_NMI )) + enable_vnmi && (exit_qualification &INTR_INFO_UNBLOCK_NMI ))vmcs_set_bits (GUEST_INTERRUPTIBILITY_INFO ,GUEST_INTR_STATE_NMI ); - gpa =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); -trace_kvm_page_fault (gpa, exit_qualification); - -/* Is it a read fault? */ - error_code = (exit_qualification &EPT_VIOLATION_ACC_READ ) - ?PFERR_USER_MASK :0 ; -/* Is it a write fault? */ - error_code |= (exit_qualification &EPT_VIOLATION_ACC_WRITE ) - ?PFERR_WRITE_MASK :0 ; -/* Is it a fetch fault? */ - error_code |= (exit_qualification &EPT_VIOLATION_ACC_INSTR ) - ?PFERR_FETCH_MASK :0 ; -/* ept page table entry is present? */ - error_code |= (exit_qualification & - (EPT_VIOLATION_READABLE |EPT_VIOLATION_WRITABLE | -EPT_VIOLATION_EXECUTABLE )) - ?PFERR_PRESENT_MASK :0 ; - - error_code |= (exit_qualification &EPT_VIOLATION_GVA_TRANSLATED ) !=0 ? -PFERR_GUEST_FINAL_MASK :PFERR_GUEST_PAGE_MASK ; - vcpu->arch.exit_qualification = exit_qualification;/*
arch/x86/kvm/vmx/vmx.c static int handle_ept_violation(struct kvm_vcpu *vcpu) · line 5362
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -5379,7 +5356,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)if (unlikely (allow_smaller_maxphyaddr &&kvm_vcpu_is_illegal_gpa (vcpu, gpa)))return kvm_emulate_instruction (vcpu,0 ); -return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +return __vmx_handle_ept_violation (vcpu, gpa, exit_qualification); }static int handle_ept_misconfig (struct kvm_vcpu *vcpu)
This should be in main.c, not in a header (and named __vt_handle_ept_qualification).
Paolo
Sean Christopherson
maintainerRe: [RFC PATCH v2 52/69] KVM: VMX: Split out guts of EPT violation to common/exposed function
Quoted 4 lines · Paolo Bonzini
Quoted 3 lines · On 03/07/21 00:04, [email protected] wrote:
arch/x86/kvm/vmx/common.h +3
-0+static inline int __vmx_handle_ept_violation (struct kvm_vcpu *vcpu,gpa_t gpa, +unsigned long exit_qualification) +{
...
Quoted 2 lines
Quoted 2 lines
arch/x86/kvm/vmx/common.h +2
-0+} +
...
Quoted 11 lines
arch/x86/kvm/vmx/vmx.c static int handle_ept_violation(struct kvm_vcpu *vcpu) · line 5361
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -5379,7 +5356,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu)if (unlikely (allow_smaller_maxphyaddr &&kvm_vcpu_is_illegal_gpa (vcpu, gpa)))return kvm_emulate_instruction (vcpu,0 ); -return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); +return __vmx_handle_ept_violation (vcpu, gpa, exit_qualification); }static int handle_ept_misconfig (struct kvm_vcpu *vcpu)This should be in
main.c, not in a header (and named__vt_handle_ept_qualification).
Yar, though I'm guessing you meant __vt_handle_ept_violation?
RFC PATCH v2 53/69 KVM: VMX: Define EPT Violation architectural bits
In reply to [email protected]From: Sean Christopherson <[email protected]>
Define the EPT Violation #VE control bit, #VE info VMCS fields, and the suppress #VE bit for EPT entries.
--- arch/x86/include/asm/vmx.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/include/asm/vmx.h +4 -0
@@ -78,6 +78,7 @@ struct vmcs {#define SECONDARY_EXEC_ENCLS_EXITING VMCS_CONTROL_BIT(ENCLS_EXITING)#define SECONDARY_EXEC_RDSEED_EXITING VMCS_CONTROL_BIT(RDSEED_EXITING)#define SECONDARY_EXEC_ENABLE_PML VMCS_CONTROL_BIT(PAGE_MOD_LOGGING) +#define SECONDARY_EXEC_EPT_VIOLATION_VE VMCS_CONTROL_BIT(EPT_VIOLATION_VE)#define SECONDARY_EXEC_PT_CONCEAL_VMX VMCS_CONTROL_BIT(PT_CONCEAL_VMX)#define SECONDARY_EXEC_XSAVES VMCS_CONTROL_BIT(XSAVES)#define SECONDARY_EXEC_MODE_BASED_EPT_EXEC VMCS_CONTROL_BIT(MODE_BASED_EPT_EXEC)
@@ -226,6 +227,8 @@ enum vmcs_field {VMREAD_BITMAP_HIGH =0x00002027 ,VMWRITE_BITMAP =0x00002028 ,VMWRITE_BITMAP_HIGH =0x00002029 , +VE_INFO_ADDRESS =0x0000202A , +VE_INFO_ADDRESS_HIGH =0x0000202B ,XSS_EXIT_BITMAP =0x0000202C ,XSS_EXIT_BITMAP_HIGH =0x0000202D , ENCLS_EXITING_BITMAP = 0x0000202E,
@@ -509,6 +512,7 @@ enum vmcs_field {#define VMX_EPT_IPAT_BIT (1ull << 6)#define VMX_EPT_ACCESS_BIT (1ull << 8)#define VMX_EPT_DIRTY_BIT (1ull << 9) +#define VMX_EPT_SUPPRESS_VE_BIT (1ull << 63)#define VMX_EPT_RWX_MASK (VMX_EPT_READABLE_MASK | \ VMX_EPT_WRITABLE_MASK | \ VMX_EPT_EXECUTABLE_MASK)
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 53/69] KVM: VMX: Define EPT Violation architectural bits
Quoted 4 lines · On 03/07/21 00:04, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Define the EPT Violation #VE control bit, #VE info VMCS fields, and the suppress #VE bit for EPT entries.
Better: "KVM: VMX: Define EPT Violation #VE architectural bits".
Paolo
Quoted 36 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/vmx.h | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/include/asm/vmx.h +4
-0@@ -78,6 +78,7 @@ struct vmcs {#define SECONDARY_EXEC_ENCLS_EXITING VMCS_CONTROL_BIT(ENCLS_EXITING)#define SECONDARY_EXEC_RDSEED_EXITING VMCS_CONTROL_BIT(RDSEED_EXITING)#define SECONDARY_EXEC_ENABLE_PML VMCS_CONTROL_BIT(PAGE_MOD_LOGGING) +#define SECONDARY_EXEC_EPT_VIOLATION_VE VMCS_CONTROL_BIT(EPT_VIOLATION_VE)#define SECONDARY_EXEC_PT_CONCEAL_VMX VMCS_CONTROL_BIT(PT_CONCEAL_VMX)#define SECONDARY_EXEC_XSAVES VMCS_CONTROL_BIT(XSAVES)#define SECONDARY_EXEC_MODE_BASED_EPT_EXEC VMCS_CONTROL_BIT(MODE_BASED_EPT_EXEC)@@ -226,6 +227,8 @@ enum vmcs_field {VMREAD_BITMAP_HIGH =0x00002027 ,VMWRITE_BITMAP =0x00002028 ,VMWRITE_BITMAP_HIGH =0x00002029 , +VE_INFO_ADDRESS =0x0000202A , +VE_INFO_ADDRESS_HIGH =0x0000202B ,XSS_EXIT_BITMAP =0x0000202C ,XSS_EXIT_BITMAP_HIGH =0x0000202D , ENCLS_EXITING_BITMAP = 0x0000202E,@@ -509,6 +512,7 @@ enum vmcs_field {#define VMX_EPT_IPAT_BIT (1ull << 6)#define VMX_EPT_ACCESS_BIT (1ull << 8)#define VMX_EPT_DIRTY_BIT (1ull << 9) +#define VMX_EPT_SUPPRESS_VE_BIT (1ull << 63)#define VMX_EPT_RWX_MASK (VMX_EPT_READABLE_MASK | \ VMX_EPT_WRITABLE_MASK | \ VMX_EPT_EXECUTABLE_MASK)
RFC PATCH v2 54/69 KVM: VMX: Define VMCS encodings for shared EPT pointer
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add the VMCS field encoding for the shared EPTP, which will be used by TDX to have separate EPT walks for private GPAs (existing EPTP) versus shared GPAs (new shared EPTP).
--- arch/x86/include/asm/vmx.h | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/include/asm/vmx.h +2 -0
@@ -235,6 +235,8 @@ enum vmcs_field {ENCLS_EXITING_BITMAP_HIGH =0x0000202F ,TSC_MULTIPLIER =0x00002032 ,TSC_MULTIPLIER_HIGH =0x00002033 , +SHARED_EPT_POINTER =0x0000203C , +SHARED_EPT_POINTER_HIGH =0x0000203D ,GUEST_PHYSICAL_ADDRESS =0x00002400 ,GUEST_PHYSICAL_ADDRESS_HIGH =0x00002401 , VMCS_LINK_POINTER = 0x00002800,
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 54/69] KVM: VMX: Define VMCS encodings for shared EPT pointer
Quoted 18 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add the VMCS field encoding for the shared EPTP, which will be used by TDX to have separate EPT walks for private GPAs (existing EPTP) versus shared GPAs (new shared EPTP).
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/vmx.h | 2 ++ 1 file changed, 2 insertions(+)arch/x86/include/asm/vmx.h +0
-0
arch/x86/include/asm/vmx.h enum vmcs_field { · line 242
Quoted 8 lines
arch/x86/include/asm/vmx.h +2
-0@@ -235,6 +235,8 @@ enum vmcs_field {ENCLS_EXITING_BITMAP_HIGH =0x0000202F ,TSC_MULTIPLIER =0x00002032 ,TSC_MULTIPLIER_HIGH =0x00002033 , +SHARED_EPT_POINTER =0x0000203C , +SHARED_EPT_POINTER_HIGH =0x0000203D ,GUEST_PHYSICAL_ADDRESS =0x00002400 ,GUEST_PHYSICAL_ADDRESS_HIGH =0x00002401 , VMCS_LINK_POINTER = 0x00002800,
RFC PATCH v2 55/69 KVM: VMX: Add 'main.c' to wrap VMX and TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
Wrap the VMX kvm_x86_ops hooks in preparation of adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 710 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 283 ++++------------ 3 files changed, 768 insertions(+), 227 deletions(-) create mode 100644 arch/x86/kvm/vmx/main.c
arch/x86/kvm/Makefile +1 -1
@@ -21,7 +21,7 @@ kvm-y += x86.o emulate.o i8259.o irq.o lapic.o \ kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_XEN) += xen.o -kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ +kvm-intel-y += vmx/main.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_INTEL_TDX) += vmx/seamcall.o
arch/x86/kvm/vmx/main.c new file +710 -0
710 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +57 -226
385 lines not on this page · raw message
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 55/69] KVM: VMX: Add 'main.c' to wrap VMX and TDX
Quoted 1 line · On 03/07/21 00:05, [email protected] wrote:
+#include "
vmx.c"
What makes it particularly hard to have this as a separate .o file rather than an #include?
Paolo
Re: [RFC PATCH v2 55/69] KVM: VMX: Add 'main.c' to wrap VMX and TDX
Quoted 5 lines · Paolo Bonzini
Quoted 1 line · On 03/07/21 00:05, [email protected] wrote:
+#include "
vmx.c"What makes it particularly hard to have this as a separate .o file rather than an #include?
It's to let complier to optimize functionc call of "if (tdx) tdx_xxx() else vmx_xxx()", given x86_ops static call story.
Signature · 1 line
Isaku Yamahata <[email protected]>
Paolo Bonzini
maintainerRe: [RFC PATCH v2 55/69] KVM: VMX: Add 'main.c' to wrap VMX and TDX
Quoted 11 lines · Isaku Yamahata
Quoted 5 lines · Paolo Bonzini
Quoted 1 line · On 03/07/21 00:05, [email protected] wrote:
+#include "
vmx.c"What makes it particularly hard to have this as a separate .o file rather than an #include?
It's to let complier to optimize functionc call of "if (tdx)
tdx_xxx()elsevmx_xxx()", givenx86_opsstatic call story.
As long as it's not an indirect call, not inlining tdx_xxx and vmx_xxx is unlikely to give a lot of benefit.
What you could do, is use a static branch that bypasses the "is_tdx_vcpu/vm" check if no TDX VM is running. A similar technique is used to bypass the test for in-kernel APIC if all VM have it.
Paolo
RFC PATCH v2 56/69 KVM: VMX: Move setting of EPT MMU masks to common VT-x code
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/main.c | 4 ++++ arch/x86/kvm/vmx/vmx.c | 4 ---- 2 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/main.c +4 -0
@@ -34,6 +34,10 @@ static __init int vt_hardware_setup(void)if (ret)return ret; +if (enable_ept) +kvm_mmu_set_ept_masks (enable_ept_ad_bits, +cpu_has_vmx_ept_execute_only ()); +return 0 ; }
arch/x86/kvm/vmx/vmx.c +0 -4
@@ -7644,10 +7644,6 @@ static __init int hardware_setup(void)set_bit (0 , vmx_vpid_bitmap);/* 0 is reserved for host */ -if (enable_ept) -kvm_mmu_set_ept_masks (enable_ept_ad_bits, -cpu_has_vmx_ept_execute_only ()); -if (!enable_ept) ept_lpage_level =0 ;else if (cpu_has_vmx_ept_1g_page ())
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 56/69] KVM: VMX: Move setting of EPT MMU masks to common VT-x code
Quoted 30 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 4 ++++ arch/x86/kvm/vmx/vmx.c | 4 ---- 2 files changed, 4 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/main.c +4
-0@@ -34,6 +34,10 @@ static __init int vt_hardware_setup(void)if (ret)return ret; +if (enable_ept) +kvm_mmu_set_ept_masks (enable_ept_ad_bits, +cpu_has_vmx_ept_execute_only ()); +return 0 ; }arch/x86/kvm/vmx/vmx.c +0
-0
arch/x86/kvm/vmx/vmx.c static __init int hardware_setup(void) · line 7649
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +0
-4@@ -7644,10 +7644,6 @@ static __init int hardware_setup(void)set_bit (0 , vmx_vpid_bitmap);/* 0 is reserved for host */ -if (enable_ept) -kvm_mmu_set_ept_masks (enable_ept_ad_bits, -cpu_has_vmx_ept_execute_only ()); -if (!enable_ept) ept_lpage_level =0 ;else if (cpu_has_vmx_ept_1g_page ())
RFC PATCH v2 57/69 KVM: VMX: Move register caching logic to common code
In reply to [email protected]From: Sean Christopherson <[email protected]>
Move the guts of vmx_cache_reg() to vt_cache_reg() in preparation for reusing the bulk of the code for TDX, which can access guest state for debug TDs.
Use kvm_x86_ops.cache_reg() in ept_update_paging_mode_cr0() rather than trying to expose vt_cache_reg() to vmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest. Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.
--- arch/x86/kvm/vmx/main.c | 37 +++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.c | 42 +---------------------------------------- 2 files changed, 37 insertions(+), 42 deletions(-)
arch/x86/kvm/vmx/main.c +36 -1
@@ -341,7 +341,42 @@ static void vt_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)static void vt_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) { -vmx_cache_reg (vcpu, reg); +unsigned long guest_owned_bits; + +kvm_register_mark_available (vcpu, reg); + +switch (reg) { +case VCPU_REGS_RSP : + vcpu->arch.regs[VCPU_REGS_RSP ] =vmcs_readl (GUEST_RSP ); +break ; +case VCPU_REGS_RIP : + vcpu->arch.regs[VCPU_REGS_RIP ] =vmcs_readl (GUEST_RIP ); +break ; +case VCPU_EXREG_PDPTR : +if (enable_ept) +ept_save_pdptrs (vcpu); +break ; +case VCPU_EXREG_CR0 : + guest_owned_bits = vcpu->arch.cr0_guest_owned_bits; + + vcpu->arch.cr0 &= ~guest_owned_bits; + vcpu->arch.cr0 |=vmcs_readl (GUEST_CR0 ) & guest_owned_bits; +break ; +case VCPU_EXREG_CR3 : +if (is_unrestricted_guest (vcpu) || + (enable_ept &&is_paging (vcpu))) + vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); +break ; +case VCPU_EXREG_CR4 : + guest_owned_bits = vcpu->arch.cr4_guest_owned_bits; + + vcpu->arch.cr4 &= ~guest_owned_bits; + vcpu->arch.cr4 |=vmcs_readl (GUEST_CR4 ) & guest_owned_bits; +break ; +default : +KVM_BUG_ON (1 , vcpu->kvm); +break ; + } }static unsigned long vt_get_rflags (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.c +1 -41
54 lines not on this page · raw message
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 57/69] KVM: VMX: Move register caching logic to common code
Quoted 121 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Move the guts of
vmx_cache_reg()tovt_cache_reg()in preparation for reusing the bulk of the code for TDX, which can access guest state for debug TDs.Use
kvm_x86_ops.cache_reg()inept_update_paging_mode_cr0()rather than trying to exposevt_cache_reg()tovmx.c, even though it means taking a retpoline. The code runs if and only if EPT is enabled but unrestricted guest. Only one generation of CPU, Nehalem, supports EPT but not unrestricted guest, and disabling unrestricted guest without also disabling EPT is, to put it bluntly, dumb.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 37 +++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.c | 42 +---------------------------------------- 2 files changed, 37 insertions(+), 42 deletions(-)arch/x86/kvm/vmx/main.c +36
-1@@ -341,7 +341,42 @@ static void vt_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)static void vt_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) { -vmx_cache_reg (vcpu, reg); +unsigned long guest_owned_bits; + +kvm_register_mark_available (vcpu, reg); + +switch (reg) { +case VCPU_REGS_RSP : + vcpu->arch.regs[VCPU_REGS_RSP ] =vmcs_readl (GUEST_RSP ); +break ; +case VCPU_REGS_RIP : + vcpu->arch.regs[VCPU_REGS_RIP ] =vmcs_readl (GUEST_RIP ); +break ; +case VCPU_EXREG_PDPTR : +if (enable_ept) +ept_save_pdptrs (vcpu); +break ; +case VCPU_EXREG_CR0 : + guest_owned_bits = vcpu->arch.cr0_guest_owned_bits; + + vcpu->arch.cr0 &= ~guest_owned_bits; + vcpu->arch.cr0 |=vmcs_readl (GUEST_CR0 ) & guest_owned_bits; +break ; +case VCPU_EXREG_CR3 : +if (is_unrestricted_guest (vcpu) || + (enable_ept &&is_paging (vcpu))) + vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); +break ; +case VCPU_EXREG_CR4 : + guest_owned_bits = vcpu->arch.cr4_guest_owned_bits; + + vcpu->arch.cr4 &= ~guest_owned_bits; + vcpu->arch.cr4 |=vmcs_readl (GUEST_CR4 ) & guest_owned_bits; +break ; +default : +KVM_BUG_ON (1 , vcpu->kvm); +break ; + } }static unsigned long vt_get_rflags (struct kvm_vcpu *vcpu)arch/x86/kvm/vmx/vmx.c +0
-40@@ -2326,46 +2326,6 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return ret; } -static void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) -{ -unsigned long guest_owned_bits; - -kvm_register_mark_available (vcpu, reg); - -switch (reg) { -case VCPU_REGS_RSP : - vcpu->arch.regs[VCPU_REGS_RSP ] =vmcs_readl (GUEST_RSP ); -break ; -case VCPU_REGS_RIP : - vcpu->arch.regs[VCPU_REGS_RIP ] =vmcs_readl (GUEST_RIP ); -break ; -case VCPU_EXREG_PDPTR : -if (enable_ept) -ept_save_pdptrs (vcpu); -break ; -case VCPU_EXREG_CR0 : - guest_owned_bits = vcpu->arch.cr0_guest_owned_bits; - - vcpu->arch.cr0 &= ~guest_owned_bits; - vcpu->arch.cr0 |=vmcs_readl (GUEST_CR0 ) & guest_owned_bits; -break ; -case VCPU_EXREG_CR3 : -if (is_unrestricted_guest (vcpu) || - (enable_ept &&is_paging (vcpu))) - vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); -break ; -case VCPU_EXREG_CR4 : - guest_owned_bits = vcpu->arch.cr4_guest_owned_bits; - - vcpu->arch.cr4 &= ~guest_owned_bits; - vcpu->arch.cr4 |=vmcs_readl (GUEST_CR4 ) & guest_owned_bits; -break ; -default : -KVM_BUG_ON (1 , vcpu->kvm); -break ; - } -} -static __init int vmx_disabled_by_bios (void ) { return !boot_cpu_has(X86_FEATURE_MSR_IA32_FEAT_CTL) ||
arch/x86/kvm/vmx/vmx.c static void ept_update_paging_mode_cr0(unsigned long *hw_cr0, · line 3032
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -3066,7 +3026,7 @@ static void ept_update_paging_mode_cr0(unsigned long *hw_cr0,struct vcpu_vmx *vmx =to_vmx (vcpu);if (!kvm_register_is_available (vcpu,VCPU_EXREG_CR3 )) -vmx_cache_reg (vcpu,VCPU_EXREG_CR3 ); + kvm_x86_ops.cache_reg (vcpu,VCPU_EXREG_CR3 );if (!(cr0 &X86_CR0_PG )) {/* From paging/starting to nonpaging */ exec_controls_setbit(vmx, CPU_BASED_CR3_LOAD_EXITING |
This shows the problem with #including vmx.c. You should have a .h file for both vmx.h and main.h (e.g. kvm_intel.h), so that here you can just use vt_cache_reg.
Paolo
RFC PATCH v2 58/69 KVM: TDX: Define TDCALL exit reason
In reply to [email protected]From: Sean Christopherson <[email protected]>
Define the TDCALL exit reason, which is carved out from the VMX exit reason namespace as the TDCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit.
--- arch/x86/include/uapi/asm/vmx.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/include/uapi/asm/vmx.h +3 -1
@@ -91,6 +91,7 @@#define EXIT_REASON_UMWAIT 67#define EXIT_REASON_TPAUSE 68#define EXIT_REASON_BUS_LOCK 74 +#define EXIT_REASON_TDCALL 77#define VMX_EXIT_REASONS \ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \
@@ -153,7 +154,8 @@ { EXIT_REASON_XRSTORS, "XRSTORS" }, \ { EXIT_REASON_UMWAIT, "UMWAIT" }, \ { EXIT_REASON_TPAUSE, "TPAUSE" }, \ - { EXIT_REASON_BUS_LOCK, "BUS_LOCK" } + { EXIT_REASON_BUS_LOCK, "BUS_LOCK" }, \ + { EXIT_REASON_TDCALL, "TDCALL" } #define VMX_EXIT_REASON_FLAGS \ { VMX_EXIT_REASONS_FAILED_VMENTRY, "FAILED_VMENTRY" }
Signature · 1 line
2.25.1
RFC PATCH v2 59/69 KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
In reply to [email protected]From: Sean Christopherson <[email protected]>
Stub in kvm_tdx, vcpu_tdx, their various accessors, and VMCS helpers. The VMCS helpers, which rely on the stubs, will be used by preparatory patches to move VMX functions for accessing VMCS state to common code.
--- arch/x86/kvm/vmx/tdx.h | 170 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx.h
arch/x86/kvm/vmx/tdx.h new file +170 -0
@@ -0,0 +1,170 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_H +#define __KVM_X86_TDX_H + +#include <linux/list.h> +#include <linux/kvm_host.h> + +#include "tdx_arch.h" +#include "tdx_errno.h" +#include "tdx_ops.h" + +#ifdef CONFIG_KVM_INTEL_TDX + +struct tdx_td_page { +unsigned long va; +hpa_t pa; +bool added; +}; + +struct kvm_tdx { +struct kvm kvm; + +struct tdx_td_page tdr; +struct tdx_td_page tdcs[TDX1_NR_TDCX_PAGES ]; +}; + +struct vcpu_tdx { +struct kvm_vcpu vcpu; + +struct tdx_td_page tdvpr; +struct tdx_td_page tdvpx[TDX1_NR_TDVPX_PAGES ]; +}; + +static inline bool is_td (struct kvm *kvm) +{ +return kvm->arch.vm_type ==KVM_X86_TDX_VM ; +} + +static inline bool is_td_vcpu (struct kvm_vcpu *vcpu) +{ +return is_td (vcpu->kvm); +} + +static inline bool is_debug_td (struct kvm_vcpu *vcpu) +{ +return !vcpu->arch.guest_state_protected; +} + +static inline struct kvm_tdx *to_kvm_tdx (struct kvm *kvm) +{ + return container_of(kvm, struct kvm_tdx, kvm); +} + +static inline struct vcpu_tdx *to_tdx (struct kvm_vcpu *vcpu) +{ + return container_of(vcpu, struct vcpu_tdx, vcpu); +} + +static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) +{ +BUILD_BUG_ON_MSG (__builtin_constant_p (field) && (field) &0x1 , +"Read/Write to TD VMCS *_HIGH fields not supported" ); + +BUILD_BUG_ON (bits !=16 && bits !=32 && bits !=64 ); + +BUILD_BUG_ON_MSG (bits !=64 &&__builtin_constant_p (field) && + (((field) &0x6000 ) ==0x2000 || + ((field) &0x6000 ) ==0x6000 ), +"Invalid TD VMCS access for 64-bit field" ); +BUILD_BUG_ON_MSG (bits !=32 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x4000 , +"Invalid TD VMCS access for 32-bit field" ); +BUILD_BUG_ON_MSG (bits !=16 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x0000 , +"Invalid TD VMCS access for 16-bit field" ); +} + +static __always_inline void tdvps_gpr_check(u64 field, u8 bits) +{ +BUILD_BUG_ON_MSG (__builtin_constant_p (field) && (field) >=NR_VCPU_REGS , +"Invalid TD guest GPR index" ); +} + +static __always_inline void tdvps_apic_check(u64 field, u8 bits) {} +static __always_inline void tdvps_dr_check(u64 field, u8 bits) {} +static __always_inline void tdvps_state_check(u64 field, u8 bits) {} +static __always_inline void tdvps_msr_check(u64 field, u8 bits) {} +static __always_inline void tdvps_management_check(u64 field, u8 bits) {} + +#define TDX_BUILD_TDVPS_ACCESSORS (bits, uclass, lclass) \ +static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ + u32 field) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_rd(tdx->tdvpr.pa, TDVPS_##uclass(field), &ex_ret); \ + if (unlikely(err)) { \ + pr_err("TDH_VP_RD["#uclass".0x%x] failed: %s (0x%llx)\n", \ + field, tdx_seamcall_error_name(err), err); \ + return 0; \ + } \ + return (u##bits)ex_ret.r8; \ +} \ +static __always_inline void td_##lclass##_write##bits(struct vcpu_tdx *tdx, \ + u32 field, u##bits val) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), val, \ + GENMASK_ULL(bits - 1, 0), &ex_ret); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] = 0x%llx failed: %s (0x%llx)\n", \ + field, (u64)val, tdx_seamcall_error_name(err), err); \ +} \ +static __always_inline void td_##lclass##_setbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), bit, bit, \ + &ex_ret); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] |= 0x%llx failed: %s (0x%llx)\n", \ + field, bit, tdx_seamcall_error_name(err), err); \ +} \ +static __always_inline void td_##lclass##_clearbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_ex_ret ex_ret; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), 0, bit, \ + &ex_ret); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] &= ~0x%llx failed: %s (0x%llx)\n", \ + field, bit, tdx_seamcall_error_name(err), err); \ +} + +TDX_BUILD_TDVPS_ACCESSORS (16 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (32 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (64 ,VMCS , vmcs); + +TDX_BUILD_TDVPS_ACCESSORS (64 ,APIC , apic); +TDX_BUILD_TDVPS_ACCESSORS (64 ,GPR , gpr); +TDX_BUILD_TDVPS_ACCESSORS (64 ,DR , dr); +TDX_BUILD_TDVPS_ACCESSORS (64 ,STATE , state); +TDX_BUILD_TDVPS_ACCESSORS (64 ,MSR , msr); +TDX_BUILD_TDVPS_ACCESSORS (8 ,MANAGEMENT , management); + +#else + +struct kvm_tdx ; +struct vcpu_tdx ; + +static inline bool is_td (struct kvm *kvm) {return false; } +static inline bool is_td_vcpu (struct kvm_vcpu *vcpu) {return false; } +static inline bool is_debug_td (struct kvm_vcpu *vcpu) {return false; } +static inline struct kvm_tdx *to_kvm_tdx (struct kvm *kvm) {return NULL ; } +static inline struct vcpu_tdx *to_tdx (struct kvm_vcpu *vcpu) {return NULL ; } + +#endif /* CONFIG_KVM_INTEL_TDX */ + +#endif /* __KVM_X86_TDX_H */
Signature · 1 line
2.25.1
RFC PATCH v2 60/69 KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.
Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.
--- arch/x86/kvm/vmx/common.h | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
arch/x86/kvm/vmx/common.h +41 -0
@@ -11,6 +11,47 @@#include "vmcs.h" #include "vmx.h" #include "x86.h" +#include "tdx.h" + +#ifdef CONFIG_KVM_INTEL_TDX +#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return 0; \ + return td_vmcs_read##tdbits(to_tdx(vcpu), field); \ + } \ + return vmcs_read##bits(field); \ +} \ +static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu, \ + unsigned long field, type value) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return; \ + return td_vmcs_write##tdbits(to_tdx(vcpu), field, value); \ + } \ + vmcs_write##bits(field, value); \ +} +#else +#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + return vmcs_read##bits(field); \ +} \ +static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu, \ + unsigned long field, type value) \ +{ \ + vmcs_write##bits(field, value); \ +} +#endif /* CONFIG_KVM_INTEL_TDX */ +VT_BUILD_VMCS_HELPERS (u16,16 ,16 ); +VT_BUILD_VMCS_HELPERS (u32,32 ,32 ); +VT_BUILD_VMCS_HELPERS (u64,64 ,64 ); +VT_BUILD_VMCS_HELPERS(unsigned long, l, 64);extern unsigned long vmx_host_idt_base;void vmx_do_interrupt_nmi_irqoff (unsigned long entry);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v2 60/69] KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
Quoted 20 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.
Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/common.h | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)arch/x86/kvm/vmx/common.h +0
-0
arch/x86/kvm/vmx/common.h line 16
Quoted 6 lines
arch/x86/kvm/vmx/common.h +3
-0@@ -11,6 +11,47 @@#include "vmcs.h" #include "vmx.h" #include "x86.h" +#include "tdx.h" + +#ifdef CONFIG_KVM_INTEL_TDX
Is this #ifdef needed at all if tdx.h properly stubs is_td_vcpu (to return false) and possibly declares a dummy version of td_vmcs_read/td_vmcs_write?
Paolo
Quoted 42 lines
arch/x86/kvm/vmx/common.h +38
-0+#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return 0; \ + return td_vmcs_read##tdbits(to_tdx(vcpu), field); \ + } \ + return vmcs_read##bits(field); \ +} \ +static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu, \ + unsigned long field, type value) \ +{ \ + if (unlikely(is_td_vcpu(vcpu))) { \ + if (KVM_BUG_ON(!is_debug_td(vcpu), vcpu->kvm)) \ + return; \ + return td_vmcs_write##tdbits(to_tdx(vcpu), field, value); \ + } \ + vmcs_write##bits(field, value); \ +} +#else +#define VT_BUILD_VMCS_HELPERS (type, bits, tdbits) \ +static __always_inline type vmread##bits(struct kvm_vcpu *vcpu, \ + unsigned long field) \ +{ \ + return vmcs_read##bits(field); \ +} \ +static __always_inline void vmwrite##bits(struct kvm_vcpu *vcpu, \ + unsigned long field, type value) \ +{ \ + vmcs_write##bits(field, value); \ +} +#endif /* CONFIG_KVM_INTEL_TDX */ +VT_BUILD_VMCS_HELPERS (u16,16 ,16 ); +VT_BUILD_VMCS_HELPERS (u32,32 ,32 ); +VT_BUILD_VMCS_HELPERS (u64,64 ,64 ); +VT_BUILD_VMCS_HELPERS(unsigned long, l, 64);extern unsigned long vmx_host_idt_base;void vmx_do_interrupt_nmi_irqoff (unsigned long entry);
Sean Christopherson
maintainerRe: [RFC PATCH v2 60/69] KVM: VMX: Add macro framework to read/write VMCS for VMs and TDs
Quoted 16 lines · Paolo Bonzini
Quoted 15 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add a macro framework to hide VMX vs. TDX details of VMREAD and VMWRITE so the VMX and TDX can shared common flows, e.g. accessing DTs.
Note, the TDX paths are dead code at this time. There is no great way to deal with the chicken-and-egg scenario of having things in place for TDX without first having TDX.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/common.h | 41 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
Quoted 14 lines
arch/x86/kvm/vmx/common.h line 16
Quoted 11 lines
arch/x86/kvm/vmx/common.h +3
-0@@ -11,6 +11,47 @@#include "vmcs.h" #include "vmx.h" #include "x86.h" +#include "tdx.h" + +#ifdef CONFIG_KVM_INTEL_TDX Is this #ifdef needed at all if
tdx.hproperly stubsis_td_vcpu(to return false) and possibly declares a dummy version of td_vmcs_read/td_vmcs_write?
IIRC, it requires dummy versions of is_debug_td() and all the ##bits variants of td_vmcs_read/write(). I'm not sure if I ever actually tried that, e.g. to see if the compiler completely elided the TDX crud when CONFIG_KVM_INTEL_TDX=n.
RFC PATCH v2 61/69 KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
In reply to [email protected]From: Sean Christopherson <[email protected]>
Move the AR_BYTES helpers to common.h so that future patches can reuse them to decode/encode AR for TDX.
--- arch/x86/kvm/vmx/common.h | 41 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 47 ++++----------------------------------- 2 files changed, 45 insertions(+), 43 deletions(-)
arch/x86/kvm/vmx/common.h +41 -0
@@ -4,6 +4,7 @@#include <linux/kvm_host.h> +#include <asm/kvm.h> #include <asm/traps.h> #include <asm/vmx.h>
@@ -119,4 +120,44 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); } +static inline u32 vmx_encode_ar_bytes (struct kvm_segment *var) +{ +u32 ar; + +if (var->unusable || !var->present) + ar =1 <<16 ; +else { + ar = var->type &15 ; + ar |= (var->s &1 ) <<4 ; + ar |= (var->dpl &3 ) <<5 ; + ar |= (var->present &1 ) <<7 ; + ar |= (var->avl &1 ) <<12 ; + ar |= (var->l &1 ) <<13 ; + ar |= (var->db &1 ) <<14 ; + ar |= (var->g &1 ) <<15 ; + } + +return ar; +} + +static inline void vmx_decode_ar_bytes (u32 ar,struct kvm_segment *var) +{ + var->unusable = (ar >>16 ) &1 ; + var->type = ar &15 ; + var->s = (ar >>4 ) &1 ; + var->dpl = (ar >>5 ) &3 ; +/* +* Some userspaces do not preserve unusable property. Since usable +* segment has to be present according to VMX spec we can use present +* property to amend userspace bug by making unusable segment always +* nonpresent. vmx_encode_ar_bytes() already marks nonpresent +* segment as unusable. +*/ + var->present = !var->unusable; + var->avl = (ar >>12 ) &1 ; + var->l = (ar >>13 ) &1 ; + var->db = (ar >>14 ) &1 ; + var->g = (ar >>15 ) &1 ; +} +#endif /* __KVM_X86_VMX_COMMON_H */
arch/x86/kvm/vmx/vmx.c +4 -43
89 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 61/69] KVM: VMX: Move AR_BYTES encoder/decoder helpers to common.h
Quoted 162 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Move the
AR_BYTEShelpers tocommon.hso that future patches can reuse them to decode/encode AR for TDX.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/common.h | 41 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 47 ++++----------------------------------- 2 files changed, 45 insertions(+), 43 deletions(-)arch/x86/kvm/vmx/common.h +41
-0@@ -4,6 +4,7 @@#include <linux/kvm_host.h> +#include <asm/kvm.h> #include <asm/traps.h> #include <asm/vmx.h> @@ -119,4 +120,44 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); } +static inline u32 vmx_encode_ar_bytes (struct kvm_segment *var) +{ +u32 ar; + +if (var->unusable || !var->present) + ar =1 <<16 ; +else { + ar = var->type &15 ; + ar |= (var->s &1 ) <<4 ; + ar |= (var->dpl &3 ) <<5 ; + ar |= (var->present &1 ) <<7 ; + ar |= (var->avl &1 ) <<12 ; + ar |= (var->l &1 ) <<13 ; + ar |= (var->db &1 ) <<14 ; + ar |= (var->g &1 ) <<15 ; + } + +return ar; +} + +static inline void vmx_decode_ar_bytes (u32 ar,struct kvm_segment *var) +{ + var->unusable = (ar >>16 ) &1 ; + var->type = ar &15 ; + var->s = (ar >>4 ) &1 ; + var->dpl = (ar >>5 ) &3 ; +/* +* Some userspaces do not preserve unusable property. Since usable +* segment has to be present according to VMX spec we can use present +* property to amend userspace bug by making unusable segment always +* nonpresent. vmx_encode_ar_bytes() already marks nonpresent +* segment as unusable. +*/ + var->present = !var->unusable; + var->avl = (ar >>12 ) &1 ; + var->l = (ar >>13 ) &1 ; + var->db = (ar >>14 ) &1 ; + var->g = (ar >>15 ) &1 ; +} +#endif /* __KVM_X86_VMX_COMMON_H */ arch/x86/kvm/vmx/vmx.c +3
-42@@ -365,8 +365,6 @@ static const struct kernel_param_ops vmentry_l1d_flush_ops = { };module_param_cb (vmentry_l1d_flush, &vmentry_l1d_flush_ops,NULL ,0644 ); -static u32 vmx_segment_access_rights (struct kvm_segment *var); -void vmx_vmexit (void ); #define vmx_insn_failed(fmt...) \@@ -2826,7 +2824,7 @@ static void fix_rmode_seg(int seg, struct kvm_segment *save)vmcs_write16 (sf->selector, var.selector);vmcs_writel (sf->base, var.base);vmcs_write32 (sf->limit, var.limit); -vmcs_write32 (sf->ar_bytes,vmx_segment_access_rights (&var)); +vmcs_write32 (sf->ar_bytes,vmx_encode_ar_bytes (&var)); }static void enter_rmode (struct kvm_vcpu *vcpu)@@ -3217,7 +3215,6 @@ void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)void vmx_get_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg) {struct vcpu_vmx *vmx =to_vmx (vcpu); -u32 ar;if (vmx->rmode.vm86_active && seg !=VCPU_SREG_LDTR ) { *var = vmx->rmode.segs[seg];@@ -3231,23 +3228,7 @@ void vmx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) var->base =vmx_read_guest_seg_base (vmx, seg); var->limit =vmx_read_guest_seg_limit (vmx, seg); var->selector =vmx_read_guest_seg_selector (vmx, seg); - ar =vmx_read_guest_seg_ar (vmx, seg); - var->unusable = (ar >>16 ) &1 ; - var->type = ar &15 ; - var->s = (ar >>4 ) &1 ; - var->dpl = (ar >>5 ) &3 ; -/* -* Some userspaces do not preserve unusable property. Since usable -* segment has to be present according to VMX spec we can use present -* property to amend userspace bug by making unusable segment always -* nonpresent. vmx_segment_access_rights() already marks nonpresent -* segment as unusable. -*/ - var->present = !var->unusable; - var->avl = (ar >>12 ) &1 ; - var->l = (ar >>13 ) &1 ; - var->db = (ar >>14 ) &1 ; - var->g = (ar >>15 ) &1 ; +vmx_decode_ar_bytes (vmx_read_guest_seg_ar (vmx, seg), var); }static u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg)@@ -3273,26 +3254,6 @@ int vmx_get_cpl(struct kvm_vcpu *vcpu) } } -static u32 vmx_segment_access_rights (struct kvm_segment *var) -{ -u32 ar; - -if (var->unusable || !var->present) - ar =1 <<16 ; -else { - ar = var->type &15 ; - ar |= (var->s &1 ) <<4 ; - ar |= (var->dpl &3 ) <<5 ; - ar |= (var->present &1 ) <<7 ; - ar |= (var->avl &1 ) <<12 ; - ar |= (var->l &1 ) <<13 ; - ar |= (var->db &1 ) <<14 ; - ar |= (var->g &1 ) <<15 ; - } - -return ar; -} -void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -3327,7 +3288,7 @@ void vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg)if (is_unrestricted_guest (vcpu) && (seg !=VCPU_SREG_LDTR )) var->type |=0x1 ;/* Accessed */ -vmcs_write32 (sf->ar_bytes,vmx_segment_access_rights (var)); +vmcs_write32 (sf->ar_bytes,vmx_encode_ar_bytes (var)); out: vmx->emulation_required =emulation_required (vcpu);
arch/x86/kvm/vmx/vmx.c static bool rmode_segment_valid(struct kvm_vcpu *vcpu, int seg) · line 3341
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +1
-1@@ -3374,7 +3335,7 @@ static bool rmode_segment_valid(struct kvm_vcpu *vcpu, int seg) var.dpl =0x3 ;if (seg ==VCPU_SREG_CS ) var.type =0x3 ; - ar =vmx_segment_access_rights (&var); + ar =vmx_encode_ar_bytes (&var);if (var.base != (var.selector <<4 ))return false;
RFC PATCH v2 62/69 KVM: VMX: MOVE GDT and IDT accessors to common code
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/main.c | 6 ++++-- arch/x86/kvm/vmx/vmx.c | 12 ------------ 2 files changed, 4 insertions(+), 14 deletions(-)
arch/x86/kvm/vmx/main.c +4 -2
@@ -311,7 +311,8 @@ static int vt_set_efer(struct kvm_vcpu *vcpu, u64 efer)static void vt_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { -vmx_get_idt (vcpu, dt); + dt->size =vmread32 (vcpu,GUEST_IDTR_LIMIT ); + dt->address =vmreadl (vcpu,GUEST_IDTR_BASE ); }static void vt_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt)
@@ -321,7 +322,8 @@ static void vt_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)static void vt_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { -vmx_get_gdt (vcpu, dt); + dt->size =vmread32 (vcpu,GUEST_GDTR_LIMIT ); + dt->address =vmreadl (vcpu,GUEST_GDTR_BASE ); }static void vt_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt)
arch/x86/kvm/vmx/vmx.c +0 -12
@@ -3302,24 +3302,12 @@ static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *l = (ar >>13 ) &1 ; } -static void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) -{ - dt->size =vmcs_read32 (GUEST_IDTR_LIMIT ); - dt->address =vmcs_readl (GUEST_IDTR_BASE ); -} -static void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_IDTR_LIMIT , dt->size);vmcs_writel (GUEST_IDTR_BASE , dt->address); } -static void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) -{ - dt->size =vmcs_read32 (GUEST_GDTR_LIMIT ); - dt->address =vmcs_readl (GUEST_GDTR_BASE ); -} -static void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_GDTR_LIMIT , dt->size);
Signature · 1 line
2.25.1
Re: [RFC PATCH v2 62/69] KVM: VMX: MOVE GDT and IDT accessors to common code
Quoted 53 lines · On 03/07/21 00:05, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 6 ++++-- arch/x86/kvm/vmx/vmx.c | 12 ------------ 2 files changed, 4 insertions(+), 14 deletions(-)arch/x86/kvm/vmx/main.c +4
-2@@ -311,7 +311,8 @@ static int vt_set_efer(struct kvm_vcpu *vcpu, u64 efer)static void vt_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { -vmx_get_idt (vcpu, dt); + dt->size =vmread32 (vcpu,GUEST_IDTR_LIMIT ); + dt->address =vmreadl (vcpu,GUEST_IDTR_BASE ); }static void vt_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt)@@ -321,7 +322,8 @@ static void vt_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt)static void vt_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { -vmx_get_gdt (vcpu, dt); + dt->size =vmread32 (vcpu,GUEST_GDTR_LIMIT ); + dt->address =vmreadl (vcpu,GUEST_GDTR_BASE ); }static void vt_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt)arch/x86/kvm/vmx/vmx.c +0
-6@@ -3302,24 +3302,12 @@ static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *l = (ar >>13 ) &1 ; } -static void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) -{ - dt->size =vmcs_read32 (GUEST_IDTR_LIMIT ); - dt->address =vmcs_readl (GUEST_IDTR_BASE ); -} -static void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_IDTR_LIMIT , dt->size);vmcs_writel (GUEST_IDTR_BASE , dt->address); }
arch/x86/kvm/vmx/vmx.c static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) · line 3313
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +0
-6-static void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) -{ - dt->size =vmcs_read32 (GUEST_GDTR_LIMIT ); - dt->address =vmcs_readl (GUEST_GDTR_BASE ); -} -static void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_GDTR_LIMIT , dt->size);
RFC PATCH v2 63/69 KVM: VMX: Move .get_interrupt_shadow() implementation to common VMX code
In reply to [email protected]From: Sean Christopherson <[email protected]>
--- arch/x86/kvm/vmx/common.h | 14 ++++++++++++++ arch/x86/kvm/vmx/vmx.c | 10 +--------- 2 files changed, 15 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/common.h +14 -0
@@ -120,6 +120,20 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa,return kvm_mmu_page_fault (vcpu, gpa, error_code,NULL ,0 ); } +static inline u32 __vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) +{ +u32 interruptibility; +int ret =0 ; + + interruptibility =vmread32 (vcpu,GUEST_INTERRUPTIBILITY_INFO ); +if (interruptibility &GUEST_INTR_STATE_STI ) + ret |=KVM_X86_SHADOW_INT_STI ; +if (interruptibility &GUEST_INTR_STATE_MOV_SS ) + ret |=KVM_X86_SHADOW_INT_MOV_SS ; + +return ret; +} +static inline u32 vmx_encode_ar_bytes (struct kvm_segment *var) {u32 ar;
arch/x86/kvm/vmx/vmx.c +1 -9
@@ -1467,15 +1467,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)u32 vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu) { -u32 interruptibility =vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); -int ret =0 ; - -if (interruptibility &GUEST_INTR_STATE_STI ) - ret |=KVM_X86_SHADOW_INT_STI ; -if (interruptibility &GUEST_INTR_STATE_MOV_SS ) - ret |=KVM_X86_SHADOW_INT_MOV_SS ; - -return ret; +return __vmx_get_interrupt_shadow (vcpu); }void vmx_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask)
Signature · 1 line
2.25.1