KVM TDX basic feature support
Subsystems (8)
v5 cover KVM TDX basic feature support
From: Isaku Yamahata <[email protected]>
Hi. Now TDX host kernel patch series was posted, I've rebased this patch series to it and make it work.
https://lore.kernel.org/lkml/[email protected]/
Changes from v4:
- rebased to TDX host kernel patch series.
- include all the patches to make this patch series working.
- add [MARKER] patches to mark the patch layer clear.
Thanks,
* What's TDX?
TDX stands for Trust Domain Extensions, which extends Intel Virtual Machines Extensions (VMX) to introduce a kind of virtual machine guest called a Trust Domain (TD) for confidential computing.
A TD runs in a CPU mode that is designed to protect the confidentiality of its memory contents and its CPU state from any other software, including the hosting Virtual Machine Monitor (VMM), unless explicitly shared by the TD itself.
We have more detailed explanations below (***). We have the high-level design of TDX KVM below (****).
In this patch series, we use "TD" or "guest TD" to differentiate it from the current "VM" (Virtual Machine), which is supported by KVM today.
* The organization of this patch series
This patch series is on top of the patches series "TDX host kernel support":
https://lore.kernel.org/lkml/[email protected]/
this patch series is available at
https://github.com/intel/tdx/releases/tag/kvm-upstream
The corresponding patches to qemu are available at
https://github.com/intel/qemu-tdx/commits/tdx-upstream
The relations of the layers are depicted as follows.
The arrows below show the order of patch reviews we would like to have.
The below layers are chosen so that the device model, for example, qemu can exercise each layering step by step. Check if TDX is supported, create TD VM, create TD vcpu, allow vcpu running, populate TD guest private memory, and handle vcpu exits/hypercalls/interrupts to run TD fully.
TDX vcpu
interrupt/exits/hypercall<------------\
^ |
| |
TD finalization |
^ |
| |
TDX EPT violation<------------\ |
^ | |
| | |
TD vcpu enter/exit | |
^ | |
| | |
TD vcpu creation/destruction | \-------KVM TDP MMU MapGPA
^ | ^
| | |
TD VM creation/destruction \---------------KVM TDP MMU hooks ^ ^
| |
TDX architectural definitions KVM TDP refactoring for TDX
^ ^
| |
TDX, VMX <--------TDX host kernel KVM MMU GPA stolen bits
coexistence support
The followings are explanations of each layer. Each layer has a dummy commit that starts with [MARKER] in subject. It is intended to help to identify where each layer starts.
TDX host kernel support:
https://lore.kernel.org/lkml/[email protected]/
The guts of system-wide initialization of TDX module. There is an independent patch series for host x86. TDX KVM patches call functions this patch series provides to initialize the TDX module.
TDX, VMX coexistence:
Infrastructure to allow TDX to coexist with VMX and trigger the initialization of the TDX module.
This layer starts with
"KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX"
TDX architectural definitions:
Add TDX architectural definitions and helper functions
This layer starts with
"[MARKER] The start of TDX KVM patch series: TDX architectural definitions".
TD VM creation/destruction:
Guest TD creation/destroy allocation and releasing of TDX specific vm and vcpu structure. Create an initial guest memory image with TDX measurement.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD VM creation/destruction".
TD vcpu creation/destruction:
guest TD creation/destroy Allocation and releasing of TDX specific vm and vcpu structure. Create an initial guest memory image with TDX measurement.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD vcpu creation/destruction"
TDX EPT violation:
Create an initial guest memory image with TDX measurement. Handle secure EPT violations to populate guest pages with TDX SEAMCALLs. This layer starts with
"[MARKER] The start of TDX KVM patch series: TDX EPT violation"
TD vcpu enter/exit:
Allow TDX vcpu to enter into TD and exit from TD. Save CPU state before entering into TD. Restore CPU state after exiting from TD.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD vcpu enter/exit"
TD vcpu interrupts/exit/hypercall:
Handle various exits/hypercalls and allow interrupts to be injected so that TD vcpu can continue running.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD vcpu exits/interrupts/hypercalls"
KVM MMU GPA stolen bits:
Introduce framework to handle stolen repurposed bit of GPA TDX repurposed a bit of GPA to indicate shared or private. If it's shared, it's the same as the conventional VMX EPT case. VMM can access shared guest pages. If it's private, it's handled by Secure-EPT and the guest page is encrypted.
This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM MMU GPA stolen bits"
KVM TDP refactoring for TDX:
TDX Secure EPT requires different constants. e.g. initial value EPT entry value etc. Various refactoring for those differences. This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM TDP refactoring for TDX"
KVM TDP MMU hooks:
Introduce framework to TDP MMU to add hooks in addition to direct EPT access TDX added Secure EPT which is an enhancement to VMX EPT. Unlike conventional VMX EPT, CPU can't directly read/write Secure EPT. Instead, use TDX SEAMCALLs to operate on Secure EPT.
This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM TDP MMU hooks"
KVM TDP MMU MapGPA:
Introduce framework to handle switching guest pages from private/shared to shared/private. For a given GPA, a guest page can be assigned to a private GPA or a shared GPA exclusively. With TDX MapGPA hypercall, guest TD converts GPA assignments from private (or shared) to shared (or private).
This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM TDP MMU MapGPA "
KVM guest private memory: (not shown in the above diagram)
[PATCH v4 00/12] KVM: mm: fd-based approach for supporting KVM guest private
memory: https://lkml.org/lkml/2022/1/18/395
Guest private memory requires different memory management in KVM. The patch proposes a way for it. Integration with TDX KVM.
(***)
* TDX module
A CPU-attested software module called the "TDX module" is designed to implement the TDX architecture, and it is loaded by the UEFI firmware today. It can be loaded by the kernel or driver at runtime, but in this patch series we assume that the TDX module is already loaded and initialized.
The TDX module provides two main new logical modes of operation built upon the new SEAM (Secure Arbitration Mode) root and non-root CPU modes added to the VMX architecture. TDX root mode is mostly identical to the VMX root operation mode, and the TDX functions (described later) are triggered by the new SEAMCALL instruction with the desired interface function selected by an input operand (leaf number, in RAX). TDX non-root mode is used for TD guest operation. TDX non-root operation (i.e. "guest TD" mode) is similar to the VMX non-root operation (i.e. guest VM), with changes and restrictions to better assure that no other software or hardware has direct visibility of the TD memory and state.
TDX transitions between TDX root operation and TDX non-root operation include TD Entries, from TDX root to TDX non-root mode, and TD Exits from TDX non-root to TDX root mode. A TD Exit might be asynchronous, triggered by some external event (e.g., external interrupt or SMI) or an exception, or it might be synchronous, triggered by a TDCALL (TDG.VP.VMCALL) function.
TD VCPUs can be entered using SEAMCALL(TDH.VP.ENTER) by KVM. TDH.VP.ENTER is one of the TDX interface functions as mentioned above, and "TDH" stands for Trust Domain Host. Those host-side TDX interface functions are categorized into various areas just for better organization, such as SYS (TDX module management), MNG (TD management), VP (VCPU), PHYSMEM (physical memory), MEM (private memory), etc. For example, SEAMCALL(TDH.SYS.INFO) returns the TDX module information.
TDCS (Trust Domain Control Structure) is the main control structure of a guest TD, and encrypted (using the guest TD's ephemeral private key). At a high level, TDCS holds information for controlling TD operation as a whole, execution, EPTP, MSR bitmaps, etc that KVM needs to set it up. Note that MSR bitmaps are held as part of TDCS (unlike VMX) because they are meant to have the same value for all VCPUs of the same TD.
Trust Domain Virtual Processor State (TDVPS) is the root control structure of a TD VCPU. It helps the TDX module control the operation of the VCPU, and holds the VCPU state while the VCPU is not running. TDVPS is opaque to software and DMA access, accessible only by using the TDX module interface functions (such as TDH.VP.RD, TDH.VP.WR). TDVPS includes TD VMCS, and TD VMCS auxiliary structures, such as virtual APIC page, virtualization exception information, etc.
Several VMX control structures (such as Shared EPT and Posted interrupt descriptor) are directly managed and accessed by the host VMM. These control structures are pointed to by fields in the TD VMCS.
The above means that 1) KVM needs to allocate different data structures for TDs,
2) KVM can reuse the existing code for TDs for some operations, 3) it needs to define TD-specific handling for others. 3) Redirect operations to . 3) Redirect operations to the TDX specific callbacks, like "if (is_td_vcpu(vcpu)) tdx_callback() else vmx_callback();".
*TD Private Memory
TD private memory is designed to hold TD private content, encrypted by the CPU using the TD ephemeral key. An encryption engine holds a table of encryption keys, and an encryption key is selected for each memory transaction based on a Host Key Identifier (HKID). By design, the host VMM does not have access to the encryption keys.
In the first generation of MKTME, HKID is "stolen" from the physical address by allocating a configurable number of bits from the top of the physical address. The HKID space is partitioned into shared HKIDs for legacy MKTME accesses and private HKIDs for SEAM-mode-only accesses. We use 0 for the shared HKID on the host so that MKTME can be opaque or bypassed on the host.
During TDX non-root operation (i.e. guest TD), memory accesses can be qualified as either shared or private, based on the value of a new SHARED bit in the Guest Physical Address (GPA). The CPU translates shared GPAs using the usual VMX EPT (Extended Page Table) or "Shared EPT" (in this document), which resides in host VMM memory. The Shared EPT is directly managed by the host VMM - the same as with the current VMX. Since guest TDs usually require I/O, and the data exchange needs to be done via shared memory, thus KVM needs to use the current EPT functionality even for TDs.
* Secure EPT and Minoring using the TDP code
The CPU translates private GPAs using a separate Secure EPT. The Secure EPT pages are encrypted and integrity-protected with the TD's ephemeral private key. Secure EPT can be managed _indirectly_ by the host VMM, using the TDX interface functions, and thus conceptually Secure EPT is a subset of EPT (why "subset"). Since execution of such interface functions takes much longer time than accessing memory directly, in KVM we use the existing TDP code to minor the Secure EPT for the TD.
This way, we can effectively walk Secure EPT without using the TDX interface functions.
* VM life cycle and TDX specific operations
The userspace VMM, such as QEMU, needs to build and treat TDs differently. For example, a TD needs to boot in private memory, and the host software cannot copy the initial image to private memory.
* TSC Virtualization
The TDX module helps TDs maintain reliable TSC (Time Stamp Counter) values (e.g. consistent among the TD VCPUs) and the virtual TSC frequency is determined by TD configuration, i.e. when the TD is created, not per VCPU. The current KVM owns TSC virtualization for VMs, but the TDX module does for TDs.
* MCE support for TDs
The TDX module doesn't allow VMM to inject MCE. Instead PV way is needed for TD to communicate with VMM. For now, KVM silently ignores MCE request by VMM. MSRs related to MCE (e.g, MCE bank registers) can be naturally emulated by paravirtualizing MSR access.
[1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.
* Restrictions or future work
Some features are not included to reduce patch size. Those features are addressed as future independent patch series.
- large page (2M, 1G)
- qemu gdb stub
- guest PMU
- and more
* Prerequisites
It's required to load the TDX module and initialize it. It's out of the scope of this patch series. Another independent patch for the common x86 code is planned. It defines CONFIG_INTEL_TDX_HOST and this patch series uses CONFIG_INTEL_TDX_HOST. It's assumed that With CONFIG_INTEL_TDX_HOST=y, the TDX module is initialized and ready for KVM to use the TDX module APIs for TDX guest life cycle like tdh.mng.init are ready to use.
Concretely Global initialization, LP (Logical Processor) initialization, global configuration, the key configuration, and TDMR and PAMT initialization are done. The state of the TDX module is SYS_READY. Please refer to the TDX module specification, the chapter Intel TDX Module Lifecycle State Machine
** Detecting the TDX module readiness.
TDX host patch series implements the detection of the TDX module availability and its initialization so that KVM can use it. Also it manages Host KeyID (HKID) assigned to guest TD.
The assumed APIs the TDX host patch series provides are
- int seamrr_enabled()
Check if required cpu feature (SEAM mode) is available. This only check CPU feature availability. At this point, the TDX module may not be ready for KVM to use.
- int init_tdx(void);
Initialization of TDX module so that the TDX module is ready for KVM to use.
- const struct tdsysinfo_struct *tdx_get_sysinfo(void);
Return the system wide information about the TDX module. NULL if the TDX isn't initialized.
- u32 tdx_get_global_keyid(void);
Return global key id that is used for the TDX module itself.
- int tdx_keyid_alloc(void);
Allocate HKID for guest TD.
- void tdx_keyid_free(int keyid);
Free HKID for guest TD.
(****)
* TDX KVM high-level design
- Host key ID management
Host Key ID (HKID) needs to be assigned to each TDX guest for memory encryption. It is assumed The TDX host patch series implements necessary functions, u32 tdx_get_global_keyid(void), int tdx_keyid_alloc(void) and,
void tdx_keyid_free(int keyid).
- Data structures and VM type
Because TDX is different from VMX, define its own VM/VCPU structures, struct kvm_tdx and struct vcpu_tdx instead of struct kvm_vmx and struct vcpu_vmx. To identify the VM, introduce VM-type to specify which VM type, VMX (default) or TDX, is used.
- VM life cycle and TDX specific operations
Re-purpose the existing KVM_MEMORY_ENCRYPT_OP to add TDX specific operations. New commands are used to get the TDX system parameters, set TDX specific VM/VCPU parameters, set initial guest memory and measurement.
The creation of TDX VM requires five additional operations in addition to the conventional VM creation.
- Get KVM system capability to check if TDX VM type is supported
- VM creation (KVM_CREATE_VM)
- New: Get the TDX specific system parameters. KVM_TDX_GET_CAPABILITY.
- New: Set TDX specific VM parameters. KVM_TDX_INIT_VM.
- VCPU creation (KVM_CREATE_VCPU)
- New: Set TDX specific VCPU parameters. KVM_TDX_INIT_VCPU.
- New: Initialize guest memory as boot state and extend the measurement with the memory. KVM_TDX_INIT_MEM_REGION.
- New: Finalize VM. KVM_TDX_FINALIZE. Complete measurement of the initial TDX VM contents.
- VCPU RUN (KVM_VCPU_RUN)
- Protected guest state
Because the guest state (CPU state and guest memory) is protected, the KVM VMM can't operate on them. For example, accessing CPU registers, injecting exceptions, and accessing guest memory. Those operations are handled as silently ignored, returning zero or initial reset value when it's requested via KVM API ioctls.
VM/VCPU state and callbacks for TDX specific operations. Define tdx specific VM state and VCPU state instead of VMX ones. Redirect operations to TDX specific callbacks. "if (tdx) tdx_op() else vmx_op()".
Operations on the CPU state
silently ignore operations on the guest state. For example, the write to CPU registers is ignored and the read from CPU registers returns 0.
. ignore access to CPU registers except for allowed ones.
. TSC: add a check if tsc is immutable and return an error. Because the KVM
implementation updates the internal tsc state and it's difficult to back out those changes. Instead, skip the logic.
. dirty logging: add check if dirty logging is supported.
. exceptions/SMI/MCE/SIPI/INIT: silently ignore
Note: virtual external interrupt and NMI can be injected into TDX guests.
- KVM MMU integration
One bit of the guest physical address (bit 51 or 47) is repurposed to indicate if the guest physical address is private (the bit is cleared) or shared (the bit is set). The bits are called stolen bits.
- Stolen bits framework
systematically tracks which guest physical address, shared or private, is used.
- Shared EPT and secure EPT
There are two EPTs. Shared EPT (the conventional one) and Secure EPT(the new one). Shared EPT is handled the same for the stolen bit set. Secure EPT points to private guest pages. To resolve EPT violation, KVM walks one of two EPTs based on faulted GPA. Because it's costly to access secure EPT during walking EPTs with SEAMCALLs for the private guest physical address, another private EPT is used as a shadow of Secure-EPT with the existing logic at the cost of extra memory.
The following depicts the relationship.
KVM | TDX module
| | |
-------------+---------- | |
| | | |
V V | |
shared GPA private GPA | |
CPU shared EPT pointer KVM private EPT pointer | CPU secure EPT pointer
| | | |
| | | |
V V | V
shared EPT private EPT<-------mirror----->Secure EPT
| | | |
| \--------------------+------\ |
| | | |
V | V V
shared guest page | private guest page
|
|
non-encrypted memory | encrypted memory
| - Operating on Secure EPT
Use the TDX module APIs to operate on Secure EPT. To call the TDX API during resolving EPT violation, add hooks to additional operation and wiring it to TDX backend.
* References
[1] TDX specification
https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf
[3] Intel CPU Architectural Extensions Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 EAS
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
[5] Intel TDX Loader Interface Specification
https://software.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf
[7] Intel TDX Virtual Firmware Design Guide
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF
Chao Gao (1):
KVM: x86: Allow to update cached values in kvm_user_return_msrs w/o
wrmsr
Isaku Yamahata (73):
x86/virt/tdx: export platform_has_tdx
KVM: TDX: Detect CPU feature on kernel module initialization
KVM: x86: Refactor KVM VMX module init/exit functions
KVM: TDX: Add placeholders for TDX VM/vcpu structure x86/virt/tdx: Add a helper function to return system wide info about
TDX module
KVM: TDX: Add a function to initialize TDX module
KVM: TDX: Make TDX VM type supported
[MARKER] The start of TDX KVM patch series: TDX architectural
definitions
KVM: TDX: Define TDX architectural definitions
KVM: TDX: Add a function for KVM to invoke SEAMCALL
KVM: TDX: add a helper function for KVM to issue SEAMCALL
KVM: TDX: Add helper functions to print TDX SEAMCALL error [MARKER] The start of TDX KVM patch series: TD VM creation/destruction
KVM: TDX: allocate per-package mutex
x86/cpu: Add helper functions to allocate/free MKTME keyid
KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl
KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters [MARKER] The start of TDX KVM patch series: TD vcpu
creation/destruction
KVM: TDX: allocate/free TDX vcpu structure
[MARKER] The start of TDX KVM patch series: KVM MMU GPA stolen bits
KVM: x86/mmu: introduce config for PRIVATE KVM MMU [MARKER] The start of TDX KVM patch series: KVM TDP refactoring for
TDX
KVM: x86/mmu: Disallow fast page fault on private GPA [MARKER] The start of TDX KVM patch series: KVM TDP MMU hooks
KVM: x86/tdp_mmu: make REMOVED_SPTE include shadow_initial value
KVM: x86/tdp_mmu: refactor kvm_tdp_mmu_map()
KVM: x86/mmu: add a private pointer to struct kvm_mmu_page
KVM: x86/tdp_mmu: Support TDX private mapping for TDP MMU
KVM: x86/tdp_mmu: Ignore unsupported mmu operation on private GFNs [MARKER] The start of TDX KVM patch series: TDX EPT violation
KVM: TDX: TDP MMU TDX support
[MARKER] The start of TDX KVM patch series: KVM TDP MMU MapGPA
KVM: x86/mmu: steal software usable bit for EPT to represent shared
page
KVM: x86/tdp_mmu: Keep PRIVATE_PROHIBIT bit when zapping
KVM: x86/tdp_mmu: prevent private/shared map based on PRIVATE_PROHIBIT
KVM: x86/tdp_mmu: implement MapGPA hypercall for TDX
KVM: x86/mmu: Focibly use TDP MMU for TDX
[MARKER] The start of TDX KVM patch series: TD finalization
KVM: TDX: Create initial guest memory
KVM: TDX: Finalize VM initialization
[MARKER] The start of TDX KVM patch series: TD vcpu enter/exit
KVM: TDX: Add helper assembly function to TDX vcpu
KVM: TDX: Implement TDX vcpu enter/exit path
KVM: TDX: vcpu_run: save/restore host state(host kernel gs)
KVM: TDX: restore host xsave state when exit from the guest TD
KVM: TDX: restore user ret MSRs
[MARKER] The start of TDX KVM patch series: TD vcpu
exits/interrupts/hypercalls
KVM: TDX: complete interrupts after tdexit
KVM: TDX: restore debug store when TD exit
KVM: TDX: handle vcpu migration over logical processor
KVM: TDX: track LP tdx vcpu run and teardown vcpus on descroing the
guest TD
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: TDX: Implement interrupt injection
KVM: TDX: Implements vcpu request_immediate_exit
KVM: TDX: Implement methods to inject NMI
KVM: TDX: Add a place holder to handle TDX VM exit
KVM: TDX: handle EXIT_REASON_OTHER_SMI
KVM: TDX: handle ept violation/misconfig exit
KVM: TDX: handle EXCEPTION_NMI and EXTERNAL_INTERRUPT
KVM: TDX: Add TDG.VP.VMCALL accessors to access guest vcpu registers
KVM: TDX: handle KVM hypercall with TDG.VP.VMCALL
KVM: TDX: Handle TDX PV CPUID hypercall
KVM: TDX: Handle TDX PV HLT hypercall
KVM: TDX: Handle TDX PV port io hypercall
KVM: TDX: Implement callbacks for MSR operations for TDX
KVM: TDX: Handle TDX PV rdmsr hypercall
KVM: TDX: Handle TDX PV wrmsr hypercall
KVM: TDX: Handle TDX PV report fatal error hypercall
KVM: TDX: Handle TDX PV map_gpa hypercall
KVM: TDX: Silently discard SMI request
KVM: TDX: Silently ignore INIT/SIPI Documentation/virtual/kvm: Document on Trust Domain Extensions(TDX)
KVM: x86: design documentation on TDX support of x86 KVM TDP MMU
Kai Huang (1):
KVM: x86: Introduce hooks to free VM callback prezap and vm_free
Rick Edgecombe (1):
KVM: x86: Add infrastructure for stolen GPA bits
Sean Christopherson (26):
KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
KVM: Enable hardware before doing arch VM initialization
KVM: x86: Introduce vm_type to differentiate default VMs from
confidential VMs
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module
KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
KVM: Add max_vcpus field in common 'struct kvm'
KVM: TDX: create/destroy VM structure
KVM: TDX: Do TDX specific vcpu initialization
KVM: x86/mmu: Disallow dirty logging for x86 TDX
KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
KVM: x86/mmu: Allow non-zero init value for shadow PTE
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: x86/mmu: Track shadow MMIO value/mask on a per-VM basis
KVM: TDX: Add load_mmu_pgd method for TDX
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: x86: Check for pending APICv interrupt in kvm_vcpu_has_events()
KVM: x86: Add option to force LAPIC expiration wait
KVM: VMX: Modify NMI and INTR handlers to take intr_info as function
argument
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86: Split core of hypercall emulation to helper function
KVM: TDX: Add a placeholder for handler of TDX hypercalls
(TDG.VP.VMCALL)
KVM: TDX: Handle TDX PV MMIO hypercall
KVM: TDX: Add methods to ignore accesses to CPU state
Xiaoyao Li (1):
KVM: TDX: initialize VM with TDX specific parameters
Yuan Yao (1):
KVM: TDX: Use vcpu_to_pi_desc() uniformly in posted_intr.c
Documentation/virt/kvm/api.rst | 24 +- .../virt/kvm/intel-tdx-layer-status.rst | 33 + Documentation/virt/kvm/intel-tdx.rst | 360 +++ Documentation/virt/kvm/tdx-tdp-mmu.rst | 466 ++++ arch/arm64/include/asm/kvm_host.h | 3 - arch/arm64/kvm/arm.c | 6 +- arch/arm64/kvm/vgic/vgic-init.c | 6 +- arch/x86/events/intel/ds.c | 1 + arch/x86/include/asm/kvm-x86-ops.h | 5 + arch/x86/include/asm/kvm_host.h | 38 +- arch/x86/include/asm/tdx.h | 61 + arch/x86/include/asm/vmx.h | 2 + arch/x86/include/uapi/asm/kvm.h | 59 + arch/x86/include/uapi/asm/vmx.h | 5 +- arch/x86/kvm/Kconfig | 4 + arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/lapic.c | 25 +- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/mmu.h | 65 +- arch/x86/kvm/mmu/mmu.c | 232 +- arch/x86/kvm/mmu/mmu_internal.h | 84 + arch/x86/kvm/mmu/paging_tmpl.h | 25 +- arch/x86/kvm/mmu/spte.c | 48 +- arch/x86/kvm/mmu/spte.h | 40 +- arch/x86/kvm/mmu/tdp_iter.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 642 ++++- arch/x86/kvm/mmu/tdp_mmu.h | 16 +- arch/x86/kvm/svm/svm.c | 10 +- arch/x86/kvm/vmx/common.h | 155 ++ arch/x86/kvm/vmx/main.c | 1026 ++++++++ arch/x86/kvm/vmx/posted_intr.c | 8 +- arch/x86/kvm/vmx/seamcall.S | 55 + arch/x86/kvm/vmx/seamcall.h | 25 + arch/x86/kvm/vmx/tdx.c | 2337 +++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 253 ++ arch/x86/kvm/vmx/tdx_arch.h | 158 ++ arch/x86/kvm/vmx/tdx_errno.h | 29 + arch/x86/kvm/vmx/tdx_error.c | 22 + arch/x86/kvm/vmx/tdx_ops.h | 174 ++ arch/x86/kvm/vmx/vmenter.S | 146 + arch/x86/kvm/vmx/vmx.c | 619 ++--- arch/x86/kvm/vmx/x86_ops.h | 235 ++ arch/x86/kvm/x86.c | 123 +- arch/x86/kvm/x86.h | 8 + arch/x86/virt/tdxcall.S | 8 +- arch/x86/virt/vmx/tdx.c | 50 +- arch/x86/virt/vmx/tdx.h | 52 - include/linux/kvm_host.h | 2 + include/uapi/linux/kvm.h | 1 + tools/arch/x86/include/uapi/asm/kvm.h | 59 + tools/include/uapi/linux/kvm.h | 1 + virt/kvm/kvm_main.c | 35 +- 52 files changed, 7142 insertions(+), 706 deletions(-) create mode 100644 Documentation/virt/kvm/intel-tdx-layer-status.rst create mode 100644 Documentation/virt/kvm/intel-tdx.rst create mode 100644 Documentation/virt/kvm/tdx-tdp-mmu.rst create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/seamcall.S create mode 100644 arch/x86/kvm/vmx/seamcall.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_error.c create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/x86_ops.h
Signature · 1 line
2.25.1
RFC PATCH v5 1/104 KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
From: Sean Christopherson <[email protected]>
KVM accesses Virtual Machine Control Structure (VMCS) with VMX instructions to operate on VM. TDX defines its data structure and TDX SEAMCALL APIs for VMM to operate on Trust Domain (TD) instead.
Trust Domain Virtual Processor State (TDVPS) is the root control structure of a TD VCPU. It helps the TDX module control the operation of the VCPU, and holds the VCPU state while the VCPU is not running. TDVPS is opaque to software and DMA access, accessible only by using the TDX module interface functions (such as TDH.VP.RD, TDH.VP.WR ,..). TDVPS includes TD VMCS, and TD VMCS auxiliary structures, such as virtual APIC page, virtualization exception information, etc. TDVPS is composed of Trust Domain Virtual Processor Root (TDVPR) which is the root page of TDVPS and Trust Domain Virtual Processor eXtension (TDVPX) pages which extend TDVPR to help provide enough physical space for the logical TDVPS structure.
Also, we have a new structure, Trust Domain Control Structure (TDCS) is the main control structure of a guest TD, and encrypted (using the guest TD's ephemeral private key). At a high level, TDCS holds information for controlling TD operation as a whole, execution, EPTP, MSR bitmaps, etc. KVM needs to set it up. Note that MSR bitmaps are held as part of TDCS (unlike VMX) because they are meant to have the same value for all VCPUs of the same TD. TDCS is a multi-page logical structure composed of multiple Trust Domain Control Extension (TDCX) physical pages. Trust Domain Root (TDR) is the root control structure of a guest TD and is encrypted using the TDX global private key. It holds a minimal set of state variables that enable guest TD control even during times when the TD's private key is not known, or when the TD's key management state does not permit access to memory encrypted using the TD's private key.
The following shows the relationship between those structures.
TDR--> TDCS per-TD
| \--> TDCX
\
\--> TDVPS per-TD VCPU
\--> TDVPR and TDVPXThe existing global struct kvm_x86_ops already defines an interface which fits with TDX. But kvm_x86_ops is system-wide, not per-VM structure. To allow VMX to coexist with TDs, the kvm_x86_ops callbacks will have wrappers "if (tdx) tdx_op() else vmx_op()" to switch VMX or TDX at run time.
To split the runtime switch, the VMX implementation, and the TDX implementation, add main.c, and move out the vmx_x86_ops hooks in preparation for adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.
The current code looks as follows.
In vmx.c
static vmx_op() { ... }
static struct kvm_x86_ops vmx_x86_ops = {
.op = vmx_op,initialization code
The eventually converted code will look like In vmx.c, keep the VMX operations.
vmx_op() { ... }
VMX initialization
In tdx.c, define the TDX operations.
tdx_op() { ... }
TDX initialization
In x86_ops.h, declare the VMX and TDX operations.
vmx_op();
tdx_op();
In main.c, define common wrappers for VMX and VMX.
static vt_ops() { if (tdx) tdx_ops() else vmx_ops() }
static struct kvm_x86_ops vt_x86_ops = {
.op = vt_op,initialization to call VMX and TDX initialization
Opportunistically, fix the name inconsistency from vmx_create_vcpu() and vmx_free_vcpu() to vmx_vcpu_create() and vxm_vcpu_free().
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 154 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 360 +++++++++++-------------------------- arch/x86/kvm/vmx/x86_ops.h | 126 +++++++++++++ 4 files changed, 385 insertions(+), 257 deletions(-) create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/x86_ops.h
arch/x86/kvm/Makefile +1 -1
@@ -22,7 +22,7 @@ kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/evmcs.o vmx/nested.o vmx/posted_intr.o + vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/main.c new file +154 -0
154 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +104 -256
905 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h new file +126 -0
126 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 001/104] KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
Quoted 73 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
KVM accesses Virtual Machine Control Structure (VMCS) with VMX instructions to operate on VM. TDX defines its data structure and TDX SEAMCALL APIs for VMM to operate on Trust Domain (TD) instead.
Trust Domain Virtual Processor State (TDVPS) is the root control structure of a TD VCPU. It helps the TDX module control the operation of the VCPU, and holds the VCPU state while the VCPU is not running. TDVPS is opaque to software and DMA access, accessible only by using the TDX module interface functions (such as TDH.VP.RD, TDH.VP.WR ,..). TDVPS includes TD VMCS, and TD VMCS auxiliary structures, such as virtual APIC page, virtualization exception information, etc. TDVPS is composed of Trust Domain Virtual Processor Root (TDVPR) which is the root page of TDVPS and Trust Domain Virtual Processor eXtension (TDVPX) pages which extend TDVPR to help provide enough physical space for the logical TDVPS structure.
Also, we have a new structure, Trust Domain Control Structure (TDCS) is the main control structure of a guest TD, and encrypted (using the guest TD's ephemeral private key). At a high level, TDCS holds information for controlling TD operation as a whole, execution, EPTP, MSR bitmaps, etc. KVM needs to set it up. Note that MSR bitmaps are held as part of TDCS (unlike VMX) because they are meant to have the same value for all VCPUs of the same TD. TDCS is a multi-page logical structure composed of multiple Trust Domain Control Extension (TDCX) physical pages. Trust Domain Root (TDR) is the root control structure of a guest TD and is encrypted using the TDX global private key. It holds a minimal set of state variables that enable guest TD control even during times when the TD's private key is not known, or when the TD's key management state does not permit access to memory encrypted using the TD's private key.
The following shows the relationship between those structures.
TDR--> TDCS per-TD | \--> TDCX \ \--> TDVPS per-TD VCPU \--> TDVPR and TDVPXThe existing global struct
kvm_x86_opsalready defines an interface which fits with TDX. Butkvm_x86_opsis system-wide, not per-VM structure. To allow VMX to coexist with TDs, thekvm_x86_opscallbacks will have wrappers "if (tdx)tdx_op()elsevmx_op()" to switch VMX or TDX at run time.To split the runtime switch, the VMX implementation, and the TDX implementation, add
main.c, and move out thevmx_x86_opshooks in preparation for adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.The current code looks as follows.
Invmx.c
staticvmx_op(){ ... }static struct kvm_x86_ops vmx_x86_ops = { .op = vmx_op,initialization code
The eventually converted code will look like
Invmx.c, keep the VMX operations.
vmx_op(){ ... }
VMX initialization
Intdx.c, define the TDX operations.
tdx_op(){ ... }
TDX initialization
Inx86_ops.h, declare the VMX and TDX operations.vmx_op(); tdx_op(); In main.c, define common wrappers for VMX and VMX. static vt_ops() { if (tdx) tdx_ops() else vmx_ops() } static struct kvm_x86_ops vt_x86_ops = { .op = vt_op,initialization to call VMX and TDX initialization
Quoted 7 lines
Opportunistically, fix the name inconsistency from
vmx_create_vcpu()andvmx_free_vcpu()tovmx_vcpu_create()andvxm_vcpu_free().Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
Quoted 1,305 lines
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 154 ++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 360 +++++++++++-------------------------- arch/x86/kvm/vmx/x86_ops.h | 126 +++++++++++++ 4 files changed, 385 insertions(+), 257 deletions(-)create mode 100644
arch/x86/kvm/vmx/main.c
create mode 100644arch/x86/kvm/vmx/x86_ops.harch/x86/kvm/Makefile +1
-1@@ -22,7 +22,7 @@ kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/evmcs.o vmx/nested.o vmx/posted_intr.o + vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.oarch/x86/kvm/vmx/main.c new file +154
-0@@ -0,0 +1,154 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/moduleparam.h> + +#include "x86_ops.h" +#include "vmx.h" +#include "nested.h" +#include "pmu.h" + +struct kvm_x86_ops vt_x86_ops__initdata = { + .name ="kvm_intel" , + + .hardware_unsetup = vmx_hardware_unsetup, + + .hardware_enable = vmx_hardware_enable, + .hardware_disable = vmx_hardware_disable, + .cpu_has_accelerated_tpr = report_flexpriority, + .has_emulated_msr = vmx_has_emulated_msr, + + .vm_size =sizeof (struct kvm_vmx ), + .vm_init = vmx_vm_init, + + .vcpu_create = vmx_vcpu_create, + .vcpu_free = vmx_vcpu_free, + .vcpu_reset = vmx_vcpu_reset, + + .prepare_guest_switch = vmx_prepare_switch_to_guest, + .vcpu_load = vmx_vcpu_load, + .vcpu_put = vmx_vcpu_put, + + .update_exception_bitmap = vmx_update_exception_bitmap, + .get_msr_feature = vmx_get_msr_feature, + .get_msr = vmx_get_msr, + .set_msr = vmx_set_msr, + .get_segment_base = vmx_get_segment_base, + .get_segment = vmx_get_segment, + .set_segment = vmx_set_segment, + .get_cpl = vmx_get_cpl, + .get_cs_db_l_bits = vmx_get_cs_db_l_bits, + .set_cr0 = vmx_set_cr0, + .is_valid_cr4 = vmx_is_valid_cr4, + .set_cr4 = vmx_set_cr4, + .set_efer = vmx_set_efer, + .get_idt = vmx_get_idt, + .set_idt = vmx_set_idt, + .get_gdt = vmx_get_gdt, + .set_gdt = vmx_set_gdt, + .set_dr7 = vmx_set_dr7, + .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, + .cache_reg = vmx_cache_reg, + .get_rflags = vmx_get_rflags, + .set_rflags = vmx_set_rflags, + .get_if_flag = vmx_get_if_flag, + + .tlb_flush_all = vmx_flush_tlb_all, + .tlb_flush_current = vmx_flush_tlb_current, + .tlb_flush_gva = vmx_flush_tlb_gva, + .tlb_flush_guest = vmx_flush_tlb_guest, + + .vcpu_pre_run = vmx_vcpu_pre_run, + .run = vmx_vcpu_run, + .handle_exit = vmx_handle_exit, + .skip_emulated_instruction = vmx_skip_emulated_instruction, + .update_emulated_instruction = vmx_update_emulated_instruction, + .set_interrupt_shadow = vmx_set_interrupt_shadow, + .get_interrupt_shadow = vmx_get_interrupt_shadow, + .patch_hypercall = vmx_patch_hypercall, + .set_irq = vmx_inject_irq, + .set_nmi = vmx_inject_nmi, + .queue_exception = vmx_queue_exception, + .cancel_injection = vmx_cancel_injection, + .interrupt_allowed = vmx_interrupt_allowed, + .nmi_allowed = vmx_nmi_allowed, + .get_nmi_mask = vmx_get_nmi_mask, + .set_nmi_mask = vmx_set_nmi_mask, + .enable_nmi_window = vmx_enable_nmi_window, + .enable_irq_window = vmx_enable_irq_window, + .update_cr8_intercept = vmx_update_cr8_intercept, + .set_virtual_apic_mode = vmx_set_virtual_apic_mode, + .set_apic_access_page_addr = vmx_set_apic_access_page_addr, + .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, + .load_eoi_exitmap = vmx_load_eoi_exitmap, + .apicv_post_state_restore = vmx_apicv_post_state_restore, + .check_apicv_inhibit_reasons = vmx_check_apicv_inhibit_reasons, + .hwapic_irr_update = vmx_hwapic_irr_update, + .hwapic_isr_update = vmx_hwapic_isr_update, + .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, + .sync_pir_to_irr = vmx_sync_pir_to_irr, + .deliver_interrupt = vmx_deliver_interrupt, + .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, + + .set_tss_addr = vmx_set_tss_addr, + .set_identity_map_addr = vmx_set_identity_map_addr, + .get_mt_mask = vmx_get_mt_mask, + + .get_exit_info = vmx_get_exit_info, + + .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, + + .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, + + .get_l2_tsc_offset = vmx_get_l2_tsc_offset, + .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, + .write_tsc_offset = vmx_write_tsc_offset, + .write_tsc_multiplier = vmx_write_tsc_multiplier, + + .load_mmu_pgd = vmx_load_mmu_pgd, + + .check_intercept = vmx_check_intercept, + .handle_exit_irqoff = vmx_handle_exit_irqoff, + + .request_immediate_exit = vmx_request_immediate_exit, + + .sched_in = vmx_sched_in, + + .cpu_dirty_log_size =PML_ENTITY_NUM , + .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, + + .pmu_ops = &intel_pmu_ops, + .nested_ops = &vmx_nested_ops, + + .update_pi_irte = pi_update_irte, + .start_assignment = vmx_pi_start_assignment, + +#ifdef CONFIG_X86_64 + .set_hv_timer = vmx_set_hv_timer, + .cancel_hv_timer = vmx_cancel_hv_timer, +#endif + + .setup_mce = vmx_setup_mce, + + .smi_allowed = vmx_smi_allowed, + .enter_smm = vmx_enter_smm, + .leave_smm = vmx_leave_smm, + .enable_smi_window = vmx_enable_smi_window, + + .can_emulate_instruction = vmx_can_emulate_instruction, + .apic_init_signal_blocked = vmx_apic_init_signal_blocked, + .migrate_timers = vmx_migrate_timers, + + .msr_filter_changed = vmx_msr_filter_changed, + .complete_emulated_msr = kvm_complete_insn_gp, + + .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, +}; + +struct kvm_x86_init_ops vt_init_ops__initdata = { + .cpu_has_kvm_support = vmx_cpu_has_kvm_support, + .disabled_by_bios = vmx_disabled_by_bios, + .check_processor_compatibility = vmx_check_processor_compat, + .hardware_setup = vmx_hardware_setup, + .handle_intel_pt_intr =NULL , + + .runtime_ops = &vt_x86_ops, +};arch/x86/kvm/vmx/vmx.c +104
-256@@ -66,6 +66,7 @@#include "vmcs12.h" #include "vmx.h" #include "x86.h" +#include "x86_ops.h" MODULE_AUTHOR ("Qumranet" );MODULE_LICENSE ("GPL" );@@ -541,7 +542,7 @@ static inline bool cpu_need_virtualize_apic_accesses(struct kvm_vcpu *vcpu)return flexpriority_enabled &&lapic_in_kernel (vcpu); } -static inline bool report_flexpriority (void ) +bool report_flexpriority (void ) {return flexpriority_enabled; }@@ -1316,7 +1317,7 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu,* Switches to specified vcpu, until a matching vcpu_put(), but assumes * vcpu mutex is already taken. */ -static void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu) +void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -1327,7 +1328,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vmx->host_debugctlmsr =get_debugctlmsr (); } -static void vmx_vcpu_put (struct kvm_vcpu *vcpu) +void vmx_vcpu_put (struct kvm_vcpu *vcpu) {vmx_vcpu_pi_put (vcpu);@@ -1381,7 +1382,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) vmx->emulation_required =vmx_emulation_required (vcpu); } -static bool vmx_get_if_flag (struct kvm_vcpu *vcpu) +bool vmx_get_if_flag (struct kvm_vcpu *vcpu) {return vmx_get_rflags (vcpu) &X86_EFLAGS_IF ; }@@ -1487,8 +1488,8 @@ static int vmx_rtit_ctl_check(struct kvm_vcpu *vcpu, u64 data)return 0 ; } -static bool vmx_can_emulate_instruction (struct kvm_vcpu *vcpu,int emul_type, -void *insn,int insn_len) +bool vmx_can_emulate_instruction (struct kvm_vcpu *vcpu,int emul_type, +void *insn,int insn_len) {/* * Emulation of instructions in SGX enclaves is impossible as RIP does @@ -1572,7 +1573,7 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)* Recognizes a pending MTF VM-exit and records the nested state for later * delivery. */ -static void vmx_update_emulated_instruction (struct kvm_vcpu *vcpu) +void vmx_update_emulated_instruction (struct kvm_vcpu *vcpu) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -1595,7 +1596,7 @@ static void vmx_update_emulated_instruction(struct kvm_vcpu *vcpu) vmx->nested.mtf_pending = false; } -static int vmx_skip_emulated_instruction (struct kvm_vcpu *vcpu) +int vmx_skip_emulated_instruction (struct kvm_vcpu *vcpu) {vmx_update_emulated_instruction (vcpu);return skip_emulated_instruction (vcpu);@@ -1614,7 +1615,7 @@ static void vmx_clear_hlt(struct kvm_vcpu *vcpu)vmcs_write32 (GUEST_ACTIVITY_STATE ,GUEST_ACTIVITY_ACTIVE ); } -static void vmx_queue_exception (struct kvm_vcpu *vcpu) +void vmx_queue_exception (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);unsigned nr = vcpu->arch.exception.nr;@@ -1727,12 +1728,12 @@ u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu)return kvm_default_tsc_scaling_ratio; } -static void vmx_write_tsc_offset (struct kvm_vcpu *vcpu,u64 offset) +void vmx_write_tsc_offset (struct kvm_vcpu *vcpu,u64 offset) {vmcs_write64 (TSC_OFFSET , offset); } -static void vmx_write_tsc_multiplier (struct kvm_vcpu *vcpu,u64 multiplier) +void vmx_write_tsc_multiplier (struct kvm_vcpu *vcpu,u64 multiplier) {vmcs_write64 (TSC_MULTIPLIER , multiplier); }@@ -1756,7 +1757,7 @@ static inline bool vmx_feature_control_msr_valid(struct kvm_vcpu *vcpu,return !(val & ~valid_bits); } -static int vmx_get_msr_feature (struct kvm_msr_entry *msr) +int vmx_get_msr_feature (struct kvm_msr_entry *msr) {switch (msr->index) { case MSR_IA32_VMX_BASIC ... MSR_IA32_VMX_VMFUNC:@@ -1776,7 +1777,7 @@ static int vmx_get_msr_feature(struct kvm_msr_entry *msr)* Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -static int vmx_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) +int vmx_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) {struct vcpu_vmx *vmx =to_vmx (vcpu);struct vmx_uret_msr *msr;@@ -1954,7 +1955,7 @@ static u64 vcpu_supported_debugctl(struct kvm_vcpu *vcpu)* Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -static int vmx_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) +int vmx_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) {struct vcpu_vmx *vmx =to_vmx (vcpu);struct vmx_uret_msr *msr;@@ -2267,7 +2268,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return ret; } -static void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) +void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) {unsigned long guest_owned_bits;@@ -2310,12 +2311,12 @@ static void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) } } -static __init int cpu_has_kvm_support (void ) +__init int vmx_cpu_has_kvm_support (void ) {return cpu_has_vmx (); } -static __init int vmx_disabled_by_bios (void ) +__init int vmx_disabled_by_bios (void ) {return !boot_cpu_has (X86_FEATURE_MSR_IA32_FEAT_CTL ) || !boot_cpu_has (X86_FEATURE_VMX );@@ -2341,7 +2342,7 @@ static int kvm_cpu_vmxon(u64 vmxon_pointer)return -EFAULT ; } -static int hardware_enable (void ) +int vmx_hardware_enable (void ) {int cpu =raw_smp_processor_id ();u64 phys_addr =__pa (per_cpu (vmxarea, cpu));@@ -2382,7 +2383,7 @@ static void vmclear_local_loaded_vmcss(void)__loaded_vmcs_clear (v); } -static void hardware_disable (void ) +void vmx_hardware_disable (void ) {vmclear_local_loaded_vmcss ();@@ -2924,7 +2925,7 @@ static void exit_lmode(struct kvm_vcpu *vcpu)#endif -static void vmx_flush_tlb_all (struct kvm_vcpu *vcpu) +void vmx_flush_tlb_all (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -2954,7 +2955,7 @@ static inline int vmx_get_current_vpid(struct kvm_vcpu *vcpu)return to_vmx (vcpu)->vpid; } -static void vmx_flush_tlb_current (struct kvm_vcpu *vcpu) +void vmx_flush_tlb_current (struct kvm_vcpu *vcpu) {struct kvm_mmu *mmu = vcpu->arch.mmu;u64 root_hpa = mmu->root_hpa;@@ -2970,7 +2971,7 @@ static void vmx_flush_tlb_current(struct kvm_vcpu *vcpu)vpid_sync_context (vmx_get_current_vpid (vcpu)); } -static void vmx_flush_tlb_gva (struct kvm_vcpu *vcpu,gva_t addr) +void vmx_flush_tlb_gva (struct kvm_vcpu *vcpu,gva_t addr) {/* * vpid_sync_vcpu_addr() is a nop if vpid==0, see the comment in @@ -2979,7 +2980,7 @@ static void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr)vpid_sync_vcpu_addr (vmx_get_current_vpid (vcpu), addr); } -static void vmx_flush_tlb_guest (struct kvm_vcpu *vcpu) +void vmx_flush_tlb_guest (struct kvm_vcpu *vcpu) {/* * vpid_sync_context() is a nop if vpid==0, e.g. if enable_vpid==0 or a @@ -3134,8 +3135,7 @@ u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level)return eptp; } -static void vmx_load_mmu_pgd (struct kvm_vcpu *vcpu,hpa_t root_hpa, -int root_level) +void vmx_load_mmu_pgd (struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level) {struct kvm *kvm = vcpu->kvm;bool update_guest_cr3 = true;@@ -3163,8 +3163,7 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,vmcs_writel (GUEST_CR3 , guest_cr3); } - -static bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) +bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) {/* * We operate under the default treatment of SMM, so VMX cannot be @@ -3280,7 +3279,7 @@ void vmx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) var->g = (ar >>15 ) &1 ; } -static u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg) +u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg) {struct kvm_segment s;@@ -3360,14 +3359,14 @@ void __vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg)vmcs_write32 (sf->ar_bytes,vmx_segment_access_rights (var)); } -static void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg) +void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg) {__vmx_set_segment (vcpu, var, seg);to_vmx (vcpu)->emulation_required =vmx_emulation_required (vcpu); } -static void vmx_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l) +void vmx_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l) {u32 ar =vmx_read_guest_seg_ar (to_vmx (vcpu),VCPU_SREG_CS );@@ -3375,25 +3374,25 @@ static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *l = (ar >>13 ) &1 ; } -static void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { dt->size =vmcs_read32 (GUEST_IDTR_LIMIT ); dt->address =vmcs_readl (GUEST_IDTR_BASE ); } -static void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_IDTR_LIMIT , dt->size);vmcs_writel (GUEST_IDTR_BASE , dt->address); } -static void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { dt->size =vmcs_read32 (GUEST_GDTR_LIMIT ); dt->address =vmcs_readl (GUEST_GDTR_BASE ); } -static void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_GDTR_LIMIT , dt->size);vmcs_writel (GUEST_GDTR_BASE , dt->address);@@ -3889,7 +3888,7 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu) } } -static bool vmx_guest_apic_has_interrupt (struct kvm_vcpu *vcpu) +bool vmx_guest_apic_has_interrupt (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);void *vapic_page;@@ -3909,7 +3908,7 @@ static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu)return ((rvi &0xf0 ) > (vppr &0xf0 )); } -static void vmx_msr_filter_changed (struct kvm_vcpu *vcpu) +void vmx_msr_filter_changed (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);u32 i;@@ -4041,8 +4040,8 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)return 0 ; } -static void vmx_deliver_interrupt (struct kvm_lapic *apic,int delivery_mode, -int trig_mode,int vector) +void vmx_deliver_interrupt (struct kvm_lapic *apic,int delivery_mode, +int trig_mode,int vector) {struct kvm_vcpu *vcpu = apic->vcpu;@@ -4185,7 +4184,7 @@ static u32 vmx_vmexit_ctrl(void) ~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |VM_EXIT_LOAD_IA32_EFER ); } -static void vmx_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu) +void vmx_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -4508,7 +4507,7 @@ static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu) vmx->pi_desc.sn =1 ; } -static void vmx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +void vmx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -4565,12 +4564,12 @@ static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)vpid_sync_context (vmx->vpid); } -static void vmx_enable_irq_window (struct kvm_vcpu *vcpu) +void vmx_enable_irq_window (struct kvm_vcpu *vcpu) {exec_controls_setbit (to_vmx (vcpu),CPU_BASED_INTR_WINDOW_EXITING ); } -static void vmx_enable_nmi_window (struct kvm_vcpu *vcpu) +void vmx_enable_nmi_window (struct kvm_vcpu *vcpu) {if (!enable_vnmi ||vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) &GUEST_INTR_STATE_STI ) {@@ -4581,7 +4580,7 @@ static void vmx_enable_nmi_window(struct kvm_vcpu *vcpu)exec_controls_setbit (to_vmx (vcpu),CPU_BASED_NMI_WINDOW_EXITING ); } -static void vmx_inject_irq (struct kvm_vcpu *vcpu) +void vmx_inject_irq (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);uint32_t intr;@@ -4609,7 +4608,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu)vmx_clear_hlt (vcpu); } -static void vmx_inject_nmi (struct kvm_vcpu *vcpu) +void vmx_inject_nmi (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -4687,7 +4686,7 @@ bool vmx_nmi_blocked(struct kvm_vcpu *vcpu) GUEST_INTR_STATE_NMI)); } -static int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection) +int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection) {if (to_vmx (vcpu)->nested.nested_run_pending)return -EBUSY ;@@ -4709,7 +4708,7 @@ bool vmx_interrupt_blocked(struct kvm_vcpu *vcpu) (GUEST_INTR_STATE_STI | GUEST_INTR_STATE_MOV_SS)); } -static int vmx_interrupt_allowed (struct kvm_vcpu *vcpu,bool for_injection) +int vmx_interrupt_allowed (struct kvm_vcpu *vcpu,bool for_injection) {if (to_vmx (vcpu)->nested.nested_run_pending)return -EBUSY ;@@ -4724,7 +4723,7 @@ static int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection)return !vmx_interrupt_blocked (vcpu); } -static int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr) +int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr) {void __user *ret;@@ -4744,7 +4743,7 @@ static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr)return init_rmode_tss (kvm, ret); } -static int vmx_set_identity_map_addr (struct kvm *kvm,u64 ident_addr) +int vmx_set_identity_map_addr (struct kvm *kvm,u64 ident_addr) {to_kvm_vmx (kvm)->ept_identity_map_addr = ident_addr;return 0 ;@@ -5023,8 +5022,7 @@ static int handle_io(struct kvm_vcpu *vcpu)return kvm_fast_pio (vcpu, size, port, in); } -static void -vmx_patch_hypercall (struct kvm_vcpu *vcpu,unsigned char *hypercall) +void vmx_patch_hypercall (struct kvm_vcpu *vcpu,unsigned char *hypercall) {/* * Patch in the VMCALL instruction: @@ -5234,7 +5232,7 @@ static int handle_dr(struct kvm_vcpu *vcpu)return kvm_complete_insn_gp (vcpu, err); } -static void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu) +void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu) {get_debugreg (vcpu->arch.db[0 ],0 );get_debugreg (vcpu->arch.db[1 ],1 );@@ -5253,7 +5251,7 @@ static void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)set_debugreg (DR6_RESERVED ,6 ); } -static void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val) +void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val) {vmcs_writel (GUEST_DR7 , val); }@@ -5519,7 +5517,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)return 1 ; } -static int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu) +int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu) {if (vmx_emulation_required_with_pending_exception (vcpu)) {kvm_prepare_emulation_failure_exit (vcpu);@@ -5756,9 +5754,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = {static const int kvm_vmx_max_exit_handlers =ARRAY_SIZE (kvm_vmx_exit_handlers); -static void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, -u64 *info1,u64 *info2, -u32 *intr_info,u32 *error_code) +void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *intr_info,u32 *error_code) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6191,7 +6188,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)return 0 ; } -static int vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath) +int vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath) {int ret =__vmx_handle_exit (vcpu, exit_fastpath);@@ -6279,7 +6276,7 @@ static noinstr void vmx_l1d_flush(struct kvm_vcpu *vcpu) : "eax", "ebx", "ecx", "edx"); } -static void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr) +void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);int tpr_threshold;@@ -6349,7 +6346,7 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu)vmx_update_msr_bitmap_x2apic (vcpu); } -static void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu) +void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu) {struct page *page;@@ -6377,7 +6374,7 @@ static void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu)put_page (page); } -static void vmx_hwapic_isr_update (struct kvm_vcpu *vcpu,int max_isr) +void vmx_hwapic_isr_update (struct kvm_vcpu *vcpu,int max_isr) {u16 status;u8 old;@@ -6411,7 +6408,7 @@ static void vmx_set_rvi(int vector) } } -static void vmx_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr) +void vmx_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr) {/* * When running L2, updating RVI is only relevant when @@ -6425,7 +6422,7 @@ static void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr)vmx_set_rvi (max_irr); } -static int vmx_sync_pir_to_irr (struct kvm_vcpu *vcpu) +int vmx_sync_pir_to_irr (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);int max_irr;@@ -6471,7 +6468,7 @@ static int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu)return max_irr; } -static void vmx_load_eoi_exitmap (struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap) +void vmx_load_eoi_exitmap (struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap) {if (!kvm_vcpu_apicv_active (vcpu))return ;@@ -6482,7 +6479,7 @@ static void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)vmcs_write64 (EOI_EXIT_BITMAP3 , eoi_exit_bitmap[3 ]); } -static void vmx_apicv_post_state_restore (struct kvm_vcpu *vcpu) +void vmx_apicv_post_state_restore (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6554,7 +6551,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu)handle_interrupt_nmi_irqoff (vcpu,gate_offset (desc)); } -static void vmx_handle_exit_irqoff (struct kvm_vcpu *vcpu) +void vmx_handle_exit_irqoff (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6571,7 +6568,7 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)* The kvm parameter can be NULL (module initialization, or invocation before * VM creation). Be sure to check the kvm parameter before using it. */ -static bool vmx_has_emulated_msr (struct kvm *kvm,u32 index) +bool vmx_has_emulated_msr (struct kvm *kvm,u32 index) {switch (index) {case MSR_IA32_SMBASE :@@ -6692,7 +6689,7 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx) IDT_VECTORING_ERROR_CODE); } -static void vmx_cancel_injection (struct kvm_vcpu *vcpu) +void vmx_cancel_injection (struct kvm_vcpu *vcpu) { __vmx_complete_interrupts(vcpu, vmcs_read32(VM_ENTRY_INTR_INFO_FIELD),@@ -6788,7 +6785,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,guest_state_exit_irqoff (); } -static fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu) +fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);unsigned long cr4;@@ -6969,7 +6966,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)return vmx_exit_handlers_fastpath (vcpu); } -static void vmx_free_vcpu (struct kvm_vcpu *vcpu) +void vmx_vcpu_free (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6980,7 +6977,7 @@ static void vmx_free_vcpu(struct kvm_vcpu *vcpu)free_loaded_vmcs (vmx->loaded_vmcs); } -static int vmx_create_vcpu (struct kvm_vcpu *vcpu) +int vmx_vcpu_create (struct kvm_vcpu *vcpu) {struct vmx_uret_msr *tsx_ctrl;struct vcpu_vmx *vmx;@@ -7085,7 +7082,7 @@ static int vmx_create_vcpu(struct kvm_vcpu *vcpu)#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" -static int vmx_vm_init (struct kvm *kvm) +int vmx_vm_init (struct kvm *kvm) {if (!ple_gap) kvm->arch.pause_in_guest = true;@@ -7116,7 +7113,7 @@ static int vmx_vm_init(struct kvm *kvm)return 0 ; } -static int __init vmx_check_processor_compat (void ) +int __init vmx_check_processor_compat (void ) {struct vmcs_config vmcs_conf;struct vmx_capability vmx_cap;@@ -7139,7 +7136,7 @@ static int __init vmx_check_processor_compat(void)return 0 ; } -static u64 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio) +u64 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio) {u8 cache;@@ -7328,7 +7325,7 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i *4 )); } -static void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu) +void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -7433,7 +7430,7 @@ static __init void vmx_set_cpu_caps(void)kvm_cpu_cap_check_and_set (X86_FEATURE_WAITPKG ); } -static void vmx_request_immediate_exit (struct kvm_vcpu *vcpu) +void vmx_request_immediate_exit (struct kvm_vcpu *vcpu) {to_vmx (vcpu)->req_immediate_exit = true; }@@ -7472,10 +7469,10 @@ static int vmx_check_intercept_io(struct kvm_vcpu *vcpu,return intercept ?X86EMUL_UNHANDLEABLE :X86EMUL_CONTINUE ; } -static int vmx_check_intercept (struct kvm_vcpu *vcpu, -struct x86_instruction_info *info, -enum x86_intercept_stage stage, -struct x86_exception *exception) +int vmx_check_intercept (struct kvm_vcpu *vcpu, +struct x86_instruction_info *info, +enum x86_intercept_stage stage, +struct x86_exception *exception) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);@@ -7540,8 +7537,8 @@ static inline int u64_shl_div_u64(u64 a, unsigned int shift,return 0 ; } -static int vmx_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, -bool *expired) +int vmx_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired) {struct vcpu_vmx *vmx;u64 tscl, guest_tscl, delta_tsc, lapic_timer_advance_cycles;@@ -7580,13 +7577,13 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, return 0; } -static void vmx_cancel_hv_timer (struct kvm_vcpu *vcpu) +void vmx_cancel_hv_timer (struct kvm_vcpu *vcpu) {to_vmx (vcpu)->hv_deadline_tsc =-1 ; }#endif -static void vmx_sched_in (struct kvm_vcpu *vcpu,int cpu) +void vmx_sched_in (struct kvm_vcpu *vcpu,int cpu) {if (!kvm_pause_in_guest (vcpu->kvm))shrink_ple_window (vcpu);@@ -7612,7 +7609,7 @@ void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)secondary_exec_controls_clearbit (vmx,SECONDARY_EXEC_ENABLE_PML ); } -static void vmx_setup_mce (struct kvm_vcpu *vcpu) +void vmx_setup_mce (struct kvm_vcpu *vcpu) {if (vcpu->arch.mcg_cap &MCG_LMCE_P ) to_vmx(vcpu)->msr_ia32_feature_control_valid_bits |=@@ -7622,7 +7619,7 @@ static void vmx_setup_mce(struct kvm_vcpu *vcpu) ~FEAT_CTL_LMCE_ENABLED ; } -static int vmx_smi_allowed (struct kvm_vcpu *vcpu,bool for_injection) +int vmx_smi_allowed (struct kvm_vcpu *vcpu,bool for_injection) {/* we need a nested vmexit to enter SMM, postpone if run is pending */ if (to_vmx (vcpu)->nested.nested_run_pending)@@ -7630,7 +7627,7 @@ static int vmx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection)return !is_smm (vcpu); } -static int vmx_enter_smm (struct kvm_vcpu *vcpu,char *smstate) +int vmx_enter_smm (struct kvm_vcpu *vcpu,char *smstate) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -7644,7 +7641,7 @@ static int vmx_enter_smm(struct kvm_vcpu *vcpu, char *smstate)return 0 ; } -static int vmx_leave_smm (struct kvm_vcpu *vcpu,const char *smstate) +int vmx_leave_smm (struct kvm_vcpu *vcpu,const char *smstate) {struct vcpu_vmx *vmx =to_vmx (vcpu);int ret;@@ -7665,17 +7662,17 @@ static int vmx_leave_smm(struct kvm_vcpu *vcpu, const char *smstate)return 0 ; } -static void vmx_enable_smi_window (struct kvm_vcpu *vcpu) +void vmx_enable_smi_window (struct kvm_vcpu *vcpu) {/* RSM will cause a vmexit anyway. */ } -static bool vmx_apic_init_signal_blocked (struct kvm_vcpu *vcpu) +bool vmx_apic_init_signal_blocked (struct kvm_vcpu *vcpu) {return to_vmx (vcpu)->nested.vmxon && !is_guest_mode (vcpu); } -static void vmx_migrate_timers (struct kvm_vcpu *vcpu) +void vmx_migrate_timers (struct kvm_vcpu *vcpu) {if (is_guest_mode (vcpu)) {struct hrtimer *timer = &to_vmx (vcpu)->nested.preemption_timer;@@ -7685,7 +7682,7 @@ static void vmx_migrate_timers(struct kvm_vcpu *vcpu) } } -static void hardware_unsetup (void ) +void vmx_hardware_unsetup (void ) {kvm_set_posted_intr_wakeup_handler (NULL );@@ -7695,7 +7692,7 @@ static void hardware_unsetup(void)free_kvm_area (); } -static bool vmx_check_apicv_inhibit_reasons (ulong bit) +bool vmx_check_apicv_inhibit_reasons (ulong bit) {ulong supported =BIT (APICV_INHIBIT_REASON_DISABLE ) | BIT(APICV_INHIBIT_REASON_ABSENT) |@@ -7705,143 +7702,6 @@ static bool vmx_check_apicv_inhibit_reasons(ulong bit)return supported &BIT (bit); } -static struct kvm_x86_ops vmx_x86_ops__initdata = { - .name ="kvm_intel" , - - .hardware_unsetup = hardware_unsetup, - - .hardware_enable = hardware_enable, - .hardware_disable = hardware_disable, - .cpu_has_accelerated_tpr = report_flexpriority, - .has_emulated_msr = vmx_has_emulated_msr, - - .vm_size =sizeof (struct kvm_vmx ), - .vm_init = vmx_vm_init, - - .vcpu_create = vmx_create_vcpu, - .vcpu_free = vmx_free_vcpu, - .vcpu_reset = vmx_vcpu_reset, - - .prepare_guest_switch = vmx_prepare_switch_to_guest, - .vcpu_load = vmx_vcpu_load, - .vcpu_put = vmx_vcpu_put, - - .update_exception_bitmap = vmx_update_exception_bitmap, - .get_msr_feature = vmx_get_msr_feature, - .get_msr = vmx_get_msr, - .set_msr = vmx_set_msr, - .get_segment_base = vmx_get_segment_base, - .get_segment = vmx_get_segment, - .set_segment = vmx_set_segment, - .get_cpl = vmx_get_cpl, - .get_cs_db_l_bits = vmx_get_cs_db_l_bits, - .set_cr0 = vmx_set_cr0, - .is_valid_cr4 = vmx_is_valid_cr4, - .set_cr4 = vmx_set_cr4, - .set_efer = vmx_set_efer, - .get_idt = vmx_get_idt, - .set_idt = vmx_set_idt, - .get_gdt = vmx_get_gdt, - .set_gdt = vmx_set_gdt, - .set_dr7 = vmx_set_dr7, - .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, - .cache_reg = vmx_cache_reg, - .get_rflags = vmx_get_rflags, - .set_rflags = vmx_set_rflags, - .get_if_flag = vmx_get_if_flag, - - .tlb_flush_all = vmx_flush_tlb_all, - .tlb_flush_current = vmx_flush_tlb_current, - .tlb_flush_gva = vmx_flush_tlb_gva, - .tlb_flush_guest = vmx_flush_tlb_guest, - - .vcpu_pre_run = vmx_vcpu_pre_run, - .run = vmx_vcpu_run, - .handle_exit = vmx_handle_exit, - .skip_emulated_instruction = vmx_skip_emulated_instruction, - .update_emulated_instruction = vmx_update_emulated_instruction, - .set_interrupt_shadow = vmx_set_interrupt_shadow, - .get_interrupt_shadow = vmx_get_interrupt_shadow, - .patch_hypercall = vmx_patch_hypercall, - .set_irq = vmx_inject_irq, - .set_nmi = vmx_inject_nmi, - .queue_exception = vmx_queue_exception, - .cancel_injection = vmx_cancel_injection, - .interrupt_allowed = vmx_interrupt_allowed, - .nmi_allowed = vmx_nmi_allowed, - .get_nmi_mask = vmx_get_nmi_mask, - .set_nmi_mask = vmx_set_nmi_mask, - .enable_nmi_window = vmx_enable_nmi_window, - .enable_irq_window = vmx_enable_irq_window, - .update_cr8_intercept = vmx_update_cr8_intercept, - .set_virtual_apic_mode = vmx_set_virtual_apic_mode, - .set_apic_access_page_addr = vmx_set_apic_access_page_addr, - .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, - .load_eoi_exitmap = vmx_load_eoi_exitmap, - .apicv_post_state_restore = vmx_apicv_post_state_restore, - .check_apicv_inhibit_reasons = vmx_check_apicv_inhibit_reasons, - .hwapic_irr_update = vmx_hwapic_irr_update, - .hwapic_isr_update = vmx_hwapic_isr_update, - .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, - .sync_pir_to_irr = vmx_sync_pir_to_irr, - .deliver_interrupt = vmx_deliver_interrupt, - .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, - - .set_tss_addr = vmx_set_tss_addr, - .set_identity_map_addr = vmx_set_identity_map_addr, - .get_mt_mask = vmx_get_mt_mask, - - .get_exit_info = vmx_get_exit_info, - - .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, - - .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, - - .get_l2_tsc_offset = vmx_get_l2_tsc_offset, - .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, - .write_tsc_offset = vmx_write_tsc_offset, - .write_tsc_multiplier = vmx_write_tsc_multiplier, - - .load_mmu_pgd = vmx_load_mmu_pgd, - - .check_intercept = vmx_check_intercept, - .handle_exit_irqoff = vmx_handle_exit_irqoff, - - .request_immediate_exit = vmx_request_immediate_exit, - - .sched_in = vmx_sched_in, - - .cpu_dirty_log_size =PML_ENTITY_NUM , - .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, - - .pmu_ops = &intel_pmu_ops, - .nested_ops = &vmx_nested_ops, - - .update_pi_irte = pi_update_irte, - .start_assignment = vmx_pi_start_assignment, - -#ifdef CONFIG_X86_64 - .set_hv_timer = vmx_set_hv_timer, - .cancel_hv_timer = vmx_cancel_hv_timer, -#endif - - .setup_mce = vmx_setup_mce, - - .smi_allowed = vmx_smi_allowed, - .enter_smm = vmx_enter_smm, - .leave_smm = vmx_leave_smm, - .enable_smi_window = vmx_enable_smi_window, - - .can_emulate_instruction = vmx_can_emulate_instruction, - .apic_init_signal_blocked = vmx_apic_init_signal_blocked, - .migrate_timers = vmx_migrate_timers, - - .msr_filter_changed = vmx_msr_filter_changed, - .complete_emulated_msr = kvm_complete_insn_gp, - - .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, -}; -static unsigned int vmx_handle_intel_pt_intr (void ) {struct kvm_vcpu *vcpu =kvm_get_running_vcpu ();@@ -7882,9 +7742,7 @@ static __init void vmx_setup_user_return_msrs(void)kvm_add_user_return_msr (vmx_uret_msrs_list[i]); } -static struct kvm_x86_init_ops vmx_init_ops__initdata ; - -static __init int hardware_setup (void ) +__init int vmx_hardware_setup (void ) {unsigned long host_bndcfgs;struct desc_ptr dt;@@ -7944,16 +7802,16 @@ static __init int hardware_setup(void)* using the APIC_ACCESS_ADDR VMCS field. */ if (!flexpriority_enabled) - vmx_x86_ops.set_apic_access_page_addr =NULL ; + vt_x86_ops.set_apic_access_page_addr =NULL ;if (!cpu_has_vmx_tpr_shadow ()) - vmx_x86_ops.update_cr8_intercept =NULL ; + vt_x86_ops.update_cr8_intercept =NULL ;#if IS_ENABLED (CONFIG_HYPERV )if (ms_hyperv.nested_features &HV_X64_NESTED_GUEST_MAPPING_FLUSH && enable_ept) { - vmx_x86_ops.tlb_remote_flush = hv_remote_flush_tlb; - vmx_x86_ops.tlb_remote_flush_with_range = + vt_x86_ops.tlb_remote_flush = hv_remote_flush_tlb; + vt_x86_ops.tlb_remote_flush_with_range = hv_remote_flush_tlb_with_range; }#endif @@ -7969,7 +7827,7 @@ static __init int hardware_setup(void)if (!cpu_has_vmx_apicv ()) enable_apicv =0 ;if (!enable_apicv) - vmx_x86_ops.sync_pir_to_irr =NULL ; + vt_x86_ops.sync_pir_to_irr =NULL ;if (cpu_has_vmx_tsc_scaling ()) { kvm_has_tsc_control = true;@@ -7996,7 +7854,7 @@ static __init int hardware_setup(void) enable_pml =0 ;if (!enable_pml) - vmx_x86_ops.cpu_dirty_log_size =0 ; + vt_x86_ops.cpu_dirty_log_size =0 ;if (!cpu_has_vmx_preemption_timer ()) enable_preemption_timer = false;@@ -8023,9 +7881,9 @@ static __init int hardware_setup(void) }if (!enable_preemption_timer) { - vmx_x86_ops.set_hv_timer =NULL ; - vmx_x86_ops.cancel_hv_timer =NULL ; - vmx_x86_ops.request_immediate_exit = __kvm_request_immediate_exit; + vt_x86_ops.set_hv_timer =NULL ; + vt_x86_ops.cancel_hv_timer =NULL ; + vt_x86_ops.request_immediate_exit = __kvm_request_immediate_exit; } kvm_mce_cap_supported |=MCG_LMCE_P ;@@ -8035,9 +7893,9 @@ static __init int hardware_setup(void)if (!enable_ept || !cpu_has_vmx_intel_pt ()) pt_mode =PT_MODE_SYSTEM ;if (pt_mode ==PT_MODE_HOST_GUEST ) - vmx_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr; + vt_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr;else - vmx_init_ops.handle_intel_pt_intr =NULL ; + vt_init_ops.handle_intel_pt_intr =NULL ;setup_default_sgx_lepubkeyhash ();@@ -8061,16 +7919,6 @@ static __init int hardware_setup(void)return r; } -static struct kvm_x86_init_ops vmx_init_ops__initdata = { - .cpu_has_kvm_support = cpu_has_kvm_support, - .disabled_by_bios = vmx_disabled_by_bios, - .check_processor_compatibility = vmx_check_processor_compat, - .hardware_setup = hardware_setup, - .handle_intel_pt_intr =NULL , - - .runtime_ops = &vmx_x86_ops, -}; -static void vmx_cleanup_l1d_flush (void ) {if (vmx_l1d_flush_pages) {@@ -8149,7 +7997,7 @@ static int __init vmx_init(void) }if (ms_hyperv.nested_features &HV_X64_NESTED_DIRECT_FLUSH ) - vmx_x86_ops.enable_direct_tlbflush + vt_x86_ops.enable_direct_tlbflush = hv_enable_direct_tlbflush; }else {@@ -8157,8 +8005,8 @@ static int __init vmx_init(void) }#endif - r =kvm_init (&vmx_init_ops,sizeof (struct vcpu_vmx ), - __alignof__(struct vcpu_vmx ),THIS_MODULE ); + r =kvm_init (&vt_init_ops,sizeof (struct vcpu_vmx ), + __alignof__(struct vcpu_vmx ),THIS_MODULE );if (r)return r;arch/x86/kvm/vmx/x86_ops.h new file +126
-0@@ -0,0 +1,126 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_VMX_X86_OPS_H +#define __KVM_X86_VMX_X86_OPS_H + +#include <linux/kvm_host.h> + +#include <asm/virtext.h> + +#include "x86.h" + +extern struct kvm_x86_init_ops vt_init_ops__initdata ; + +__init int vmx_cpu_has_kvm_support (void ); +__init int vmx_disabled_by_bios (void ); +int __init vmx_check_processor_compat (void ); +__init int vmx_hardware_setup (void ); + +extern struct kvm_x86_ops vt_x86_ops__initdata ; +extern struct kvm_x86_init_ops vt_init_ops__initdata ; + +void vmx_hardware_unsetup (void ); +int vmx_hardware_enable (void ); +void vmx_hardware_disable (void ); +bool report_flexpriority (void ); +int vmx_vm_init (struct kvm *kvm); +int vmx_vcpu_create (struct kvm_vcpu *vcpu); +int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu); +fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu); +void vmx_vcpu_free (struct kvm_vcpu *vcpu); +void vmx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); +void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu); +void vmx_vcpu_put (struct kvm_vcpu *vcpu); +int vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath); +void vmx_handle_exit_irqoff (struct kvm_vcpu *vcpu); +int vmx_skip_emulated_instruction (struct kvm_vcpu *vcpu); +void vmx_update_emulated_instruction (struct kvm_vcpu *vcpu); +int vmx_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info); +int vmx_smi_allowed (struct kvm_vcpu *vcpu,bool for_injection); +int vmx_enter_smm (struct kvm_vcpu *vcpu,char *smstate); +int vmx_leave_smm (struct kvm_vcpu *vcpu,const char *smstate); +void vmx_enable_smi_window (struct kvm_vcpu *vcpu); +bool vmx_can_emulate_instruction (struct kvm_vcpu *vcpu,int emul_type, +void *insn,int insn_len); +int vmx_check_intercept (struct kvm_vcpu *vcpu, +struct x86_instruction_info *info, +enum x86_intercept_stage stage, +struct x86_exception *exception); +bool vmx_apic_init_signal_blocked (struct kvm_vcpu *vcpu); +void vmx_migrate_timers (struct kvm_vcpu *vcpu); +void vmx_set_virtual_apic_mode (struct kvm_vcpu *vcpu); +void vmx_apicv_post_state_restore (struct kvm_vcpu *vcpu); +bool vmx_check_apicv_inhibit_reasons (ulong bit); +void vmx_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr); +void vmx_hwapic_isr_update (struct kvm_vcpu *vcpu,int max_isr); +bool vmx_guest_apic_has_interrupt (struct kvm_vcpu *vcpu); +int vmx_sync_pir_to_irr (struct kvm_vcpu *vcpu); +void vmx_deliver_interrupt (struct kvm_lapic *apic,int delivery_mode, +int trig_mode,int vector); +void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu); +bool vmx_has_emulated_msr (struct kvm *kvm,u32 index); +void vmx_msr_filter_changed (struct kvm_vcpu *vcpu); +void vmx_prepare_switch_to_guest (struct kvm_vcpu *vcpu); +void vmx_update_exception_bitmap (struct kvm_vcpu *vcpu); +int vmx_get_msr_feature (struct kvm_msr_entry *msr); +int vmx_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info); +u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg); +void vmx_get_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg); +void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg); +int vmx_get_cpl (struct kvm_vcpu *vcpu); +void vmx_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l); +void vmx_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0); +void vmx_load_mmu_pgd (struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +void vmx_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); +bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); +int vmx_set_efer (struct kvm_vcpu *vcpu,u64 efer); +void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val); +void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu); +void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg); +unsigned long vmx_get_rflags (struct kvm_vcpu *vcpu); +void vmx_set_rflags (struct kvm_vcpu *vcpu,unsigned long rflags); +bool vmx_get_if_flag (struct kvm_vcpu *vcpu); +void vmx_flush_tlb_all (struct kvm_vcpu *vcpu); +void vmx_flush_tlb_current (struct kvm_vcpu *vcpu); +void vmx_flush_tlb_gva (struct kvm_vcpu *vcpu,gva_t addr); +void vmx_flush_tlb_guest (struct kvm_vcpu *vcpu); +void vmx_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask); +u32 vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu); +void vmx_patch_hypercall (struct kvm_vcpu *vcpu,unsigned char *hypercall); +void vmx_inject_irq (struct kvm_vcpu *vcpu); +void vmx_inject_nmi (struct kvm_vcpu *vcpu); +void vmx_queue_exception (struct kvm_vcpu *vcpu); +void vmx_cancel_injection (struct kvm_vcpu *vcpu); +int vmx_interrupt_allowed (struct kvm_vcpu *vcpu,bool for_injection); +int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection); +bool vmx_get_nmi_mask (struct kvm_vcpu *vcpu); +void vmx_set_nmi_mask (struct kvm_vcpu *vcpu,bool masked); +void vmx_enable_nmi_window (struct kvm_vcpu *vcpu); +void vmx_enable_irq_window (struct kvm_vcpu *vcpu); +void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr); +void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu); +void vmx_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu); +void vmx_load_eoi_exitmap (struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap); +int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr); +int vmx_set_identity_map_addr (struct kvm *kvm,u64 ident_addr); +u64 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio); +void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *intr_info,u32 *error_code); +u64 vmx_get_l2_tsc_offset (struct kvm_vcpu *vcpu); +u64 vmx_get_l2_tsc_multiplier (struct kvm_vcpu *vcpu); +void vmx_write_tsc_offset (struct kvm_vcpu *vcpu,u64 offset); +void vmx_write_tsc_multiplier (struct kvm_vcpu *vcpu,u64 multiplier); +void vmx_request_immediate_exit (struct kvm_vcpu *vcpu); +void vmx_sched_in (struct kvm_vcpu *vcpu,int cpu); +void vmx_update_cpu_dirty_logging (struct kvm_vcpu *vcpu); +#ifdef CONFIG_X86_64 +int vmx_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired); +void vmx_cancel_hv_timer (struct kvm_vcpu *vcpu); +#endif +void vmx_setup_mce (struct kvm_vcpu *vcpu); + +#endif /* __KVM_X86_VMX_X86_OPS_H */
RFC PATCH v5 2/104 x86/virt/tdx: export platform_has_tdx
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX KVM uses platform_has_tdx() via hardware_setup().
--- arch/x86/virt/vmx/tdx.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/virt/vmx/tdx.c +1 -0
@@ -1630,3 +1630,4 @@ bool platform_has_tdx(void) {return seamrr_enabled () &&tdx_keyid_sufficient (); } +EXPORT_SYMBOL_GPL (platform_has_tdx);
Signature · 1 line
2.25.1
RFC PATCH v5 3/104 KVM: TDX: Detect CPU feature on kernel module initialization
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX), detect TDX module availability, and initialize TDX module. This patch implements the first step to detect CPU feature. Because VMX isn't enabled yet by VMXON instruction on KVM kernel module initialization, defer further initialization step until VMX is enabled by hardware_enable callback.
Introduce a module parameter, enable_tdx, to explicitly enable TDX KVM support. It's off by default to keep same behavior for those who don't use TDX. Implement CPU feature detection at KVM kernel module initialization as hardware_setup callback to check if CPU feature is available and get some CPU parameters.
--- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/main.c | 15 ++++++++++- arch/x86/kvm/vmx/tdx.c | 53 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 6 +++++ 4 files changed, 74 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx.c
arch/x86/kvm/Makefile +1 -0
@@ -24,6 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/main.c +14 -1
@@ -6,6 +6,19 @@#include "nested.h" #include "pmu.h" +static __init int vt_hardware_setup (void ) +{ +int ret; + + ret =vmx_hardware_setup (); +if (ret) +return ret; + +tdx_hardware_setup (&vt_x86_ops); + +return 0 ; +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,
@@ -147,7 +160,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .cpu_has_kvm_support = vmx_cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios, .check_processor_compatibility = vmx_check_processor_compat, - .hardware_setup = vmx_hardware_setup, + .hardware_setup = vt_hardware_setup, .handle_intel_pt_intr = NULL, .runtime_ops = &vt_x86_ops,
arch/x86/kvm/vmx/tdx.c new file +53 -0
53 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +6 -0
@@ -123,4 +123,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);#endif void vmx_setup_mce (struct kvm_vcpu *vcpu); +#ifdef CONFIG_INTEL_TDX_HOST +void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +#else +static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {} +#endif +#endif /* __KVM_X86_VMX_X86_OPS_H */
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 003/104] KVM: TDX: Detect CPU feature on kernel module initialization
Quoted 95 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX), detect TDX module availability, and initialize TDX module. This patch implements the first step to detect CPU feature. Because VMX isn't enabled yet by VMXON instruction on KVM kernel module initialization, defer further initialization step until VMX is enabled by
hardware_enablecallback.Introduce a module parameter,
enable_tdx, to explicitly enable TDX KVM support. It's off by default to keep same behavior for those who don't use TDX. Implement CPU feature detection at KVM kernel module initialization ashardware_setupcallback to check if CPU feature is available and get some CPU parameters.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/main.c | 15 ++++++++++- arch/x86/kvm/vmx/tdx.c | 53 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 6 +++++ 4 files changed, 74 insertions(+), 1 deletion(-)create mode 100644
arch/x86/kvm/vmx/tdx.carch/x86/kvm/Makefile +1
-0@@ -24,6 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.oarch/x86/kvm/vmx/main.c +14
-1@@ -6,6 +6,19 @@#include "nested.h" #include "pmu.h" +static __init int vt_hardware_setup (void ) +{ +int ret; + + ret =vmx_hardware_setup (); +if (ret) +return ret; + +tdx_hardware_setup (&vt_x86_ops); + +return 0 ; +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,@@ -147,7 +160,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .cpu_has_kvm_support = vmx_cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios, .check_processor_compatibility = vmx_check_processor_compat, - .hardware_setup = vmx_hardware_setup, + .hardware_setup = vt_hardware_setup, .handle_intel_pt_intr = NULL, .runtime_ops = &vt_x86_ops,arch/x86/kvm/vmx/tdx.c new file +20
-0@@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/cpu.h> + +#include <asm/tdx.h> + +#include "capabilities.h" +#include "x86_ops.h" + +#undef pr_fmt +#define pr_fmt (fmt) "tdx: " fmt + +static bool __read_mostly enable_tdx = true; +module_param_named (tdx, enable_tdx, bool,0644 ); + +static u64 hkid_mask__ro_after_init ; +static u8 hkid_start_pos__ro_after_init ; + +static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) +{ +u32 max_pa;
arch/x86/kvm/vmx/tdx.c line 30
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c new file +10
-0+ +if (!enable_ept) { +pr_warn ("Cannot enable TDX with EPT disabled\n" ); +return -EINVAL ; + } + +if (!platform_has_tdx ()) { +pr_warn ("Cannot enable TDX with SEAMRR disabled\n" ); +return -ENODEV ; + }
This will cause a pr_warn in the logs on all machines that don't have TDX. Perhaps you can restrict the pr_warn() to machines that have __seamrr_enabled() == true?
Paolo
Quoted 37 lines
arch/x86/kvm/vmx/tdx.c new file +22
-0+if (WARN_ON_ONCE (x86_ops->tlb_remote_flush)) +return -EIO ; + + max_pa =cpuid_eax (0x80000008 ) &0xff ; + hkid_start_pos = boot_cpu_data.x86_phys_bits; + hkid_mask =GENMASK_ULL (max_pa -1 , hkid_start_pos); + +return 0 ; +} + +void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops) +{ +/* +* This function is called at the initialization. No need to protect +* enable_tdx. +*/ +if (!enable_tdx) +return ; + +if (__tdx_hardware_setup (&vt_x86_ops)) + enable_tdx = false; +}arch/x86/kvm/vmx/x86_ops.h +6
-0@@ -123,4 +123,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);#endif void vmx_setup_mce (struct kvm_vcpu *vcpu); +#ifdef CONFIG_INTEL_TDX_HOST +void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +#else +static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {} +#endif +#endif /* __KVM_X86_VMX_X86_OPS_H */
Re: [RFC PATCH v5 003/104] KVM: TDX: Detect CPU feature on kernel module initialization
Thanks for comment.
Quoted 11 lines · Paolo Bonzini
Quoted 10 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
(quoted diff) +9
-0+static int __init __tdx_hardware_setup(struct kvm_x86_ops *x86_ops) +{ + u32 max_pa; + + if (!enable_ept) { + pr_warn("Cannot enable TDX with EPT disabled\n"); + return -EINVAL; + } +
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ if (!platform_has_tdx()) { + pr_warn("Cannot enable TDX with SEAMRR disabled\n"); + return -ENODEV; + }This will cause a
pr_warnin the logs on all machines that don't have TDX. Perhaps you can restrict thepr_warn()to machines that have__seamrr_enabled()== true?
Makes sense. I'll include the following change.
arch/x86/include/asm/tdx.h +2 -0
@@ -134,6 +134,7 @@ struct tdsysinfo_struct { }; } __packed__aligned(TDSYSINFO_STRUCT_ALIGNMENT) ; +bool __seamrr_enabled (void );void tdx_detect_cpu (struct cpuinfo_x86 *c);int tdx_detect (void );int tdx_init (void );
@@ -143,6 +144,7 @@ u32 tdx_get_global_keyid(void);int tdx_keyid_alloc (void );void tdx_keyid_free (int keyid);#else +static inline bool __seamrr_enabled (void ) {return false; }static inline void tdx_detect_cpu (struct cpuinfo_x86 *c) { }static inline int tdx_detect (void ) {return -ENODEV ; }static inline int tdx_init (void ) {return -ENODEV ; }
arch/x86/kvm/vmx/tdx.c +2 -1
@@ -2625,7 +2625,8 @@ static int __init __tdx_hardware_setup(struct kvm_x86_ops *x86_ops) }if (!platform_has_tdx ()) { -pr_warn ("Cannot enable TDX with SEAMRR disabled\n" ); +if (__seamrr_enabled ()) +pr_warn ("Cannot enable TDX with SEAMRR disabled\n" );return -ENODEV ; }
arch/x86/virt/vmx/tdx.c +2 -1
@@ -186,10 +186,11 @@ static const struct kernel_param_ops tdx_trace_ops = {module_param_cb (tdx_trace_level, &tdx_trace_ops, &tdx_trace_level,0644 );MODULE_PARM_DESC (tdx_trace_level,"TDX module trace level" ); -static bool __seamrr_enabled (void ) +bool __seamrr_enabled (void ) {return (seamrr_mask &SEAMRR_ENABLED_BITS ) ==SEAMRR_ENABLED_BITS ; } +EXPORT_SYMBOL_GPL (__seamrr_enabled);static void detect_seam_bsp (struct cpuinfo_x86 *c) {
Signature · 1 line
Isaku Yamahata <[email protected]>
Sean Christopherson
maintainerRe: [RFC PATCH v5 003/104] KVM: TDX: Detect CPU feature on kernel module initialization
In reply to [email protected]Quoted 9 lines · On Fri, Mar 04, 2022, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c new file +3
-0@@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/cpu.h> +
arch/x86/kvm/vmx/tdx.c line 13
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c new file +10
-0+#include <asm/tdx.h> + +#include "capabilities.h" +#include "x86_ops.h" + +#undef pr_fmt +#define pr_fmt (fmt) "tdx: " fmt + +static bool __read_mostly enable_tdx = true; +module_param_named (tdx, enable_tdx, bool,0644 );
This is comically unsafe, userspace must not be allowed to toggle enable_tdx after KVM is loaded.
Quoted 28 lines
arch/x86/kvm/vmx/tdx.c new file +28
-0+static u64 hkid_mask__ro_after_init ; +static u8 hkid_start_pos__ro_after_init ; + +static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) +{ +u32 max_pa; + +if (!enable_ept) { +pr_warn ("Cannot enable TDX with EPT disabled\n" ); +return -EINVAL ; + } + +if (!platform_has_tdx ()) { +pr_warn ("Cannot enable TDX with SEAMRR disabled\n" ); +return -ENODEV ; + } + +if (WARN_ON_ONCE (x86_ops->tlb_remote_flush)) +return -EIO ; + + max_pa =cpuid_eax (0x80000008 ) &0xff ; + hkid_start_pos = boot_cpu_data.x86_phys_bits; + hkid_mask =GENMASK_ULL (max_pa -1 , hkid_start_pos); + +return 0 ; +} + +void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops)
arch/x86/kvm/vmx/tdx.c line 51
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c new file +10
-0+{ +/* +* This function is called at the initialization. No need to protect +* enable_tdx. +*/ +if (!enable_tdx) +return ; + +if (__tdx_hardware_setup (&vt_x86_ops)) + enable_tdx = false;
Clearing enable_tdx here unnecessarily risks introducing bugs in the caller, e.g. acting on enable_tdx before tdx_hardware_setup() is invoked. I'm guessing this was the result of trying to defer module load until VM creation. With that gone, the flag can be moved to vmx/main.c, as there should be zero reason for tdx.c to check/modify enable_tdx, i.e. functions in tdx.c should never be called if enabled_tdx = false.
An alteranative to
if (enable_tdx && tdx_hardware_setup(&vt_x86_ops)) enable_tdx = false;
would be
enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops);
I actually prefer the latter (no "if"), but I already generated and wiped the below diff before thinking of that.
arch/x86/kvm/vmx/main.c +5 -1
@@ -6,6 +6,9 @@#include "nested.h" #include "pmu.h" +static bool __read_mostly enable_tdx =IS_ENABLED (CONFIG_INTEL_TDX_HOST ); +module_param_named (tdx, enable_tdx, bool,0644 ); +static __init int vt_hardware_setup (void ) {int ret;
@@ -14,7 +17,8 @@ static __init int vt_hardware_setup(void)if (ret)return ret; -tdx_hardware_setup (&vt_x86_ops); +if (enable_tdx &&tdx_hardware_setup (&vt_x86_ops)) + enable_tdx = false;return 0 ; }
arch/x86/kvm/vmx/tdx.c +1 -17
@@ -9,13 +9,10 @@#undef pr_fmt#define pr_fmt (fmt) "tdx: " fmt -static bool __read_mostly enable_tdx = true; -module_param_named (tdx, enable_tdx, bool,0644 ); -static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ; -static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) +static int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {u32 max_pa;
@@ -38,16 +35,3 @@ static int __init __tdx_hardware_setup(struct kvm_x86_ops *x86_ops)return 0 ; } - -void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops) -{ -/* -* This function is called at the initialization. No need to protect -* enable_tdx. -*/ -if (!enable_tdx) -return ; - -if (__tdx_hardware_setup (&vt_x86_ops)) - enable_tdx = false; -}
arch/x86/kvm/vmx/x86_ops.h +2 -2
@@ -124,9 +124,9 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);void vmx_setup_mce (struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST -void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);#else -static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {} +static inline int void tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return 0; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
RFC PATCH v5 4/104 KVM: Enable hardware before doing arch VM initialization
In reply to [email protected]From: Sean Christopherson <[email protected]>
Swap the order of hardware_enable_all() and kvm_arch_init_vm() to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.
This also provides consistent ordering between kvm_create_vm() and kvm_destroy_vm() with respect to calling kvm_arch_destroy_vm() and hardware_disable_all().
--- virt/kvm/kvm_main.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
virt/kvm/kvm_main.c +7 -7
@@ -1105,19 +1105,19 @@ static struct kvm *kvm_create_vm(unsigned long type)rcu_assign_pointer (kvm->buses[i],kzalloc (sizeof (struct kvm_io_bus ),GFP_KERNEL_ACCOUNT ));if (!kvm->buses[i]) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable; } kvm->max_halt_poll_ns = halt_poll_ns; - r =kvm_arch_init_vm (kvm, type); -if (r) - goto out_err_no_arch_destroy_vm; - r =hardware_enable_all ();if (r) goto out_err_no_disable; + r =kvm_arch_init_vm (kvm, type); +if (r) + goto out_err_no_arch_destroy_vm; +#ifdef CONFIG_HAVE_KVM_IRQFD INIT_HLIST_HEAD (&kvm->irq_ack_notifier_list);#endif
@@ -1145,10 +1145,10 @@ static struct kvm *kvm_create_vm(unsigned long type)mmu_notifier_unregister (&kvm->mmu_notifier, current->mm);#endif out_err_no_mmu_notifier: -hardware_disable_all (); -out_err_no_disable:kvm_arch_destroy_vm (kvm); out_err_no_arch_destroy_vm: +hardware_disable_all (); +out_err_no_disable:WARN_ON_ONCE (!refcount_dec_and_test (&kvm->users_count));for (i =0 ; i <KVM_NR_BUSES ; i++)kfree (kvm_get_bus (kvm, i));
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 004/104] KVM: Enable hardware before doing arch VM initialization
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Swap the order of
hardware_enable_all()andkvm_arch_init_vm()to accommodate Intel's TDX, which needs VMX to be enabled during VM init in order to make SEAMCALLs.
Quoted 6 lines
This also provides consistent ordering between
kvm_create_vm()andkvm_destroy_vm()with respect to callingkvm_arch_destroy_vm()andhardware_disable_all().Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
and please submit this as a preparation patch that can be committed separately.
Paolo
Quoted 46 lines
--- virt/kvm/kvm_main.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)virt/kvm/kvm_main.c +7
-7@@ -1105,19 +1105,19 @@ static struct kvm *kvm_create_vm(unsigned long type)rcu_assign_pointer (kvm->buses[i],kzalloc (sizeof (struct kvm_io_bus ),GFP_KERNEL_ACCOUNT ));if (!kvm->buses[i]) - goto out_err_no_arch_destroy_vm; + goto out_err_no_disable; } kvm->max_halt_poll_ns = halt_poll_ns; - r =kvm_arch_init_vm (kvm, type); -if (r) - goto out_err_no_arch_destroy_vm; - r =hardware_enable_all ();if (r) goto out_err_no_disable; + r =kvm_arch_init_vm (kvm, type); +if (r) + goto out_err_no_arch_destroy_vm; +#ifdef CONFIG_HAVE_KVM_IRQFD INIT_HLIST_HEAD (&kvm->irq_ack_notifier_list);#endif @@ -1145,10 +1145,10 @@ static struct kvm *kvm_create_vm(unsigned long type)mmu_notifier_unregister (&kvm->mmu_notifier, current->mm);#endif out_err_no_mmu_notifier: -hardware_disable_all (); -out_err_no_disable:kvm_arch_destroy_vm (kvm); out_err_no_arch_destroy_vm: +hardware_disable_all (); +out_err_no_disable:WARN_ON_ONCE (!refcount_dec_and_test (&kvm->users_count));for (i =0 ; i <KVM_NR_BUSES ; i++)kfree (kvm_get_bus (kvm, i));
RFC PATCH v5 5/104 KVM: x86: Refactor KVM VMX module init/exit functions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.
The current module initialization flow is, 1.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 2.) report those sizes to the KVM common layer and KVM common initialization, and 3.) VMX specific system-wide initialization.
Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. We have a wrapper function,
"vt_init() {vmx_pre_kvm_init(); kvm_init(); vmx_init(); }" in main.c, and vmx_pre_kvm_init() and vmx_init() in vmx.c. vmx_pre_kvm_init() calculates the sizes of VMX kvm structure and KVM vcpu structure, kvm_init() does system-wide initialization of the KVM common layer, and vmx_init() does system-wide VMX initialization.
The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures, add tdx_pre_kvm_init() to report the sizes of them to the KVM common layer.
The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c.
--- arch/x86/kvm/vmx/main.c | 33 +++++++++++++ arch/x86/kvm/vmx/vmx.c | 97 +++++++++++++++++++------------------- arch/x86/kvm/vmx/x86_ops.h | 5 +- 3 files changed, 86 insertions(+), 49 deletions(-)
arch/x86/kvm/vmx/main.c +33 -0
@@ -165,3 +165,36 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .runtime_ops = &vt_x86_ops, }; + +static int __init vt_init (void ) +{ +unsigned int vcpu_size =0 , vcpu_align =0 ; +int r; + +vmx_pre_kvm_init (&vcpu_size, &vcpu_align); + + r =kvm_init (&vt_init_ops, vcpu_size, vcpu_align,THIS_MODULE ); +if (r) + goto err_vmx_post_exit; + + r =vmx_init (); +if (r) + goto err_kvm_exit; + +return 0 ; + +err_kvm_exit: +kvm_exit (); +err_vmx_post_exit: +vmx_post_kvm_exit (); +return r; +} +module_init (vt_init); + +static void vt_exit (void ) +{ +vmx_exit (); +kvm_exit (); +vmx_post_kvm_exit (); +} +module_exit (vt_exit);
arch/x86/kvm/vmx/vmx.c +49 -48
121 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +4 -1
@@ -8,7 +8,10 @@#include "x86.h" -extern struct kvm_x86_init_ops vt_init_ops__initdata ; +void __init vmx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align); +int __init vmx_init (void ); +void vmx_exit (void ); +void vmx_post_kvm_exit (void );__init int vmx_cpu_has_kvm_support (void );__init int vmx_disabled_by_bios (void );
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 005/104] KVM: x86: Refactor KVM VMX module init/exit functions
Quoted 125 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.
The current module initialization flow is, 1.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 2.) report those sizes to the KVM common layer and KVM common initialization, and 3.) VMX specific system-wide initialization.
Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in
vmx.cfrom a file,main.c, common among VMX and TDX. We have a wrapper function,
"vt_init(){vmx_pre_kvm_init();kvm_init();vmx_init(); }" inmain.c, andvmx_pre_kvm_init()andvmx_init()invmx.c.vmx_pre_kvm_init()calculates the sizes of VMX kvm structure and KVM vcpu structure,kvm_init()does system-wide initialization of the KVM common layer, andvmx_init()does system-wide VMX initialization.The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct
vmx_kvm{ struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures, addtdx_pre_kvm_init()to report the sizes of them to the KVM common layer.The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in
vmx.cfrommain.c.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 33 +++++++++++++ arch/x86/kvm/vmx/vmx.c | 97 +++++++++++++++++++------------------- arch/x86/kvm/vmx/x86_ops.h | 5 +- 3 files changed, 86 insertions(+), 49 deletions(-)arch/x86/kvm/vmx/main.c +33
-0@@ -165,3 +165,36 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .runtime_ops = &vt_x86_ops, }; + +static int __init vt_init (void ) +{ +unsigned int vcpu_size =0 , vcpu_align =0 ; +int r; + +vmx_pre_kvm_init (&vcpu_size, &vcpu_align); + + r =kvm_init (&vt_init_ops, vcpu_size, vcpu_align,THIS_MODULE ); +if (r) + goto err_vmx_post_exit; + + r =vmx_init (); +if (r) + goto err_kvm_exit; + +return 0 ; + +err_kvm_exit: +kvm_exit (); +err_vmx_post_exit: +vmx_post_kvm_exit (); +return r; +} +module_init (vt_init); + +static void vt_exit (void ) +{ +vmx_exit (); +kvm_exit (); +vmx_post_kvm_exit (); +} +module_exit (vt_exit);arch/x86/kvm/vmx/vmx.c +1
-34@@ -7929,47 +7929,12 @@ static void vmx_cleanup_l1d_flush(void) l1tf_vmx_mitigation =VMENTER_L1D_FLUSH_AUTO ; } -static void vmx_exit (void ) +void __init vmx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align) { -#ifdef CONFIG_KEXEC_CORE -RCU_INIT_POINTER (crash_vmclear_loaded_vmcss,NULL ); -synchronize_rcu (); -#endif - -kvm_exit (); - -#if IS_ENABLED (CONFIG_HYPERV ) -if (static_branch_unlikely (&enable_evmcs)) { -int cpu; -struct hv_vp_assist_page *vp_ap; -/* -* Reset everything to support using non-enlightened VMCS -* access later (e.g. when we reload the module with -* enlightened_vmcs=0) -*/ -for_each_online_cpu (cpu) { - vp_ap =hv_get_vp_assist_page (cpu); - -if (!vp_ap) -continue ; - - vp_ap->nested_control.features.directhypercall =0 ; - vp_ap->current_nested_vmcs =0 ; - vp_ap->enlighten_vmentry =0 ; - } - -static_branch_disable (&enable_evmcs); - } -#endif -vmx_cleanup_l1d_flush (); - - allow_smaller_maxphyaddr = false;
arch/x86/kvm/vmx/vmx.c static void vmx_cleanup_l1d_flush(void) · line 7937
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +4
-6-} -module_exit (vmx_exit); - -static int __init vmx_init (void ) -{ -int r, cpu; +if (sizeof (struct vcpu_vmx ) > *vcpu_size) + *vcpu_size =sizeof (struct vcpu_vmx ); +if (__alignof__(struct vcpu_vmx ) > *vcpu_align) + *vcpu_align = __alignof__(struct vcpu_vmx );
Please keep these four lines in vt_init, and rename the rest of vmx_pre_kvm_init to hv_vp_assist_page_init. Likewise, rename vmx_post_kvm_exit to hv_vp_assist_page_exit.
Adjusting the vcpu_size and vcpu_align for TDX (I guess) can be added later when TDX ops are introduced.
Paolo
Quoted 91 lines
arch/x86/kvm/vmx/vmx.c +44
-8#if IS_ENABLED (CONFIG_HYPERV )/* @@ -8004,11 +7969,38 @@ static int __init vmx_init(void) enlightened_vmcs = false; }#endif +} - r =kvm_init (&vt_init_ops,sizeof (struct vcpu_vmx ), - __alignof__(struct vcpu_vmx ),THIS_MODULE ); -if (r) -return r; +void vmx_post_kvm_exit (void ) +{ +#if IS_ENABLED (CONFIG_HYPERV ) +if (static_branch_unlikely (&enable_evmcs)) { +int cpu; +struct hv_vp_assist_page *vp_ap; +/* +* Reset everything to support using non-enlightened VMCS +* access later (e.g. when we reload the module with +* enlightened_vmcs=0) +*/ +for_each_online_cpu (cpu) { + vp_ap =hv_get_vp_assist_page (cpu); + +if (!vp_ap) +continue ; + + vp_ap->nested_control.features.directhypercall =0 ; + vp_ap->current_nested_vmcs =0 ; + vp_ap->enlighten_vmentry =0 ; + } + +static_branch_disable (&enable_evmcs); + } +#endif +} + +int __init vmx_init (void ) +{ +int r, cpu;/* * Must be called after kvm_init() so enable_ept is properly set @@ -8018,10 +8010,8 @@ static int __init vmx_init(void)* mitigation mode. */ r =vmx_setup_l1d_flush (vmentry_l1d_flush_param); -if (r) { -vmx_exit (); +if (r)return r; - }for_each_possible_cpu (cpu) {INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu));@@ -8045,4 +8035,15 @@ static int __init vmx_init(void)return 0 ; } -module_init (vmx_init); + +void vmx_exit (void ) +{ +#ifdef CONFIG_KEXEC_CORE +RCU_INIT_POINTER (crash_vmclear_loaded_vmcss,NULL ); +synchronize_rcu (); +#endif + +vmx_cleanup_l1d_flush (); + + allow_smaller_maxphyaddr = false; +}arch/x86/kvm/vmx/x86_ops.h +4
-1@@ -8,7 +8,10 @@#include "x86.h" -extern struct kvm_x86_init_ops vt_init_ops__initdata ; +void __init vmx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align); +int __init vmx_init (void ); +void vmx_exit (void ); +void vmx_post_kvm_exit (void );__init int vmx_cpu_has_kvm_support (void );__init int vmx_disabled_by_bios (void );
Re: [RFC PATCH v5 005/104] KVM: x86: Refactor KVM VMX module init/exit functions
Thanks for review.
Quoted 13 lines · Paolo Bonzini
arch/x86/kvm/vmx/vmx.c static void vmx_cleanup_l1d_flush(void) · line 7937
Quoted 7 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/vmx.c +4
-3-static int __init vmx_init (void ) -{ -int r, cpu; +if (sizeof (struct vcpu_vmx ) > *vcpu_size) + *vcpu_size =sizeof (struct vcpu_vmx ); +if (__alignof__(struct vcpu_vmx ) > *vcpu_align) + *vcpu_align = __alignof__(struct vcpu_vmx );Please keep these four lines in
vt_init, and rename the rest ofvmx_pre_kvm_inittohv_vp_assist_page_init. Likewise, renamevmx_post_kvm_exittohv_vp_assist_page_exit.
Quoted 2 lines
Adjusting the
vcpu_sizeandvcpu_alignfor TDX (I guess) can be added later when TDX ops are introduced.
sure. I'll make it like
vcpu_size = max(sizeof(struct vcpu_vmx), sizeof(vcpu_tdx));
...
Thanks,
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH v5 6/104 KVM: TDX: Add placeholders for TDX VM/vcpu structure
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add placeholders TDX VM/vcpu structure that overlays with VMX VM/vcpu structures. Initialize VM structure size and vcpu size/align so that x86 KVM common code knows those size irrespective of VMX or TDX. Those structures will be populated as guest creation logic develops.
Add helper functions to check if the VM is guest TD and add conversion functions between KVM VM/VCPU and TDX VM/VCPU.
--- arch/x86/kvm/vmx/main.c | 3 +++ arch/x86/kvm/vmx/tdx.c | 11 +++++++++ arch/x86/kvm/vmx/tdx.h | 47 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ 4 files changed, 65 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx.h
arch/x86/kvm/vmx/main.c +3 -0
@@ -171,6 +171,9 @@ static int __init vt_init(void)unsigned int vcpu_size =0 , vcpu_align =0 ;int r; +/* tdx_pre_kvm_init must be called before vmx_pre_kvm_init(). */ +tdx_pre_kvm_init (&vcpu_size, &vcpu_align, &vt_x86_ops.vm_size); +vmx_pre_kvm_init (&vcpu_size, &vcpu_align); r =kvm_init (&vt_init_ops, vcpu_size, vcpu_align,THIS_MODULE );
arch/x86/kvm/vmx/tdx.c +11 -0
@@ -5,6 +5,7 @@#include "capabilities.h" #include "x86_ops.h" +#include "tdx.h" #undef pr_fmt#define pr_fmt (fmt) "tdx: " fmt
@@ -51,3 +52,13 @@ void __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)if (__tdx_hardware_setup (&vt_x86_ops)) enable_tdx = false; } + +void __init tdx_pre_kvm_init (unsigned int *vcpu_size, +unsigned int *vcpu_align,unsigned int *vm_size) +{ + *vcpu_size =sizeof (struct vcpu_tdx ); + *vcpu_align = __alignof__(struct vcpu_tdx ); + +if (sizeof (struct kvm_tdx ) > *vm_size) + *vm_size =sizeof (struct kvm_tdx ); +}
arch/x86/kvm/vmx/tdx.h new file +47 -0
47 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +4 -0
@@ -127,8 +127,12 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);void vmx_setup_mce (struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST +void __init tdx_pre_kvm_init (unsigned int *vcpu_size, +unsigned int *vcpu_align,unsigned int *vm_size);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);#else +static inline void tdx_pre_kvm_init ( +unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {}#endif
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 006/104] KVM: TDX: Add placeholders for TDX VM/vcpu structure
arch/x86/kvm/vmx/tdx.c void __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) · line 64
Quoted 9 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +9
-0+void __init tdx_pre_kvm_init (unsigned int *vcpu_size, +unsigned int *vcpu_align,unsigned int *vm_size) +{ + *vcpu_size =sizeof (struct vcpu_tdx ); + *vcpu_align = __alignof__(struct vcpu_tdx ); + +if (sizeof (struct kvm_tdx ) > *vm_size) + *vm_size =sizeof (struct kvm_tdx ); +}
No need for this function, I would just do
vcpu_size = max(sizeof vcpu_vmx, sizeof vcpu_tdx); vcpu_align = max(...); vt_x86_ops.vm_size = max(...);
in vt_init.
Paolo
RFC PATCH v5 7/104 x86/virt/tdx: Add a helper function to return system wide info about TDX module
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX KVM needs system-wide information about the TDX module, struct tdsysinfo_struct. Add a helper function tdx_get_sysinfo() to return it instead of KVM getting it with various error checks. Move out the struct definition about it to common place tdx_host.h.
--- arch/x86/include/asm/tdx.h | 55 ++++++++++++++++++++++++++++++++++++++ arch/x86/virt/vmx/tdx.c | 16 +++++++++-- arch/x86/virt/vmx/tdx.h | 52 ----------------------------------- 3 files changed, 69 insertions(+), 54 deletions(-)
arch/x86/include/asm/tdx.h +55 -0
@@ -82,15 +82,70 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,#endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */ #ifdef CONFIG_INTEL_TDX_HOST +struct tdx_cpuid_config { +u32 leaf; +u32 sub_leaf; +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +#define TDSYSINFO_STRUCT_SIZE 1024 +#define TDSYSINFO_STRUCT_ALIGNMENT 1024 + +struct tdsysinfo_struct { +/* TDX-SEAM Module Info */ +u32 attributes; +u32 vendor_id; +u32 build_date; +u16 build_num; +u16 minor_version; +u16 major_version; +u8 reserved0[14 ]; +/* Memory Info */ +u16 max_tdmrs; +u16 max_reserved_per_tdmr; +u16 pamt_entry_size; +u8 reserved1[10 ]; +/* Control Struct Info */ +u16 tdcs_base_size; +u8 reserved2[2 ]; +u16 tdvps_base_size; +u8 tdvps_xfam_dependent_size; +u8 reserved3[9 ]; +/* TD Capabilities */ +u64 attributes_fixed0; +u64 attributes_fixed1; +u64 xfam_fixed0; +u64 xfam_fixed1; +u8 reserved4[32 ]; +u32 num_cpuid_config; +/* +* The actual number of CPUID_CONFIG depends on above +* 'num_cpuid_config'. The size of 'struct tdsysinfo_struct' +* is 1024B defined by TDX architecture. Use a union with +* specific padding to make 'sizeof(struct tdsysinfo_struct)' +* equal to 1024. +*/ +union { +struct tdx_cpuid_config cpuid_configs[0 ]; +u8 reserved5[892 ]; + }; +} __packed__aligned(TDSYSINFO_STRUCT_ALIGNMENT) ; +void tdx_detect_cpu (struct cpuinfo_x86 *c);int tdx_detect (void );int tdx_init (void );bool platform_has_tdx (void ); +const struct tdsysinfo_struct *tdx_get_sysinfo (void );#else static inline void tdx_detect_cpu (struct cpuinfo_x86 *c) { }static inline int tdx_detect (void ) {return -ENODEV ; }static inline int tdx_init (void ) {return -ENODEV ; }static inline bool platform_has_tdx (void ) {return false; } +struct tdsysinfo_struct ; +static inline const struct tdsysinfo_struct *tdx_get_sysinfo (void ) {return NULL ; }#endif /* CONFIG_INTEL_TDX_HOST */ #endif /* !__ASSEMBLY__ */
arch/x86/virt/vmx/tdx.c +14 -2
@@ -641,7 +641,7 @@ static int sanitize_cmrs(struct cmr_info *cmr_array, int cmr_num)return 0 ; } -static int tdx_get_sysinfo (void ) +static int __tdx_get_sysinfo (void ) {struct tdx_module_output out;u64 tdsysinfo_sz, cmr_num;
@@ -676,6 +676,18 @@ static int tdx_get_sysinfo(void)return sanitize_cmrs (tdx_cmr_array, cmr_num); } +const struct tdsysinfo_struct *tdx_get_sysinfo (void ) +{ +const struct tdsysinfo_struct *r =NULL ; + +mutex_lock (&tdx_module_lock); +if (tdx_module_status ==TDX_MODULE_INITIALIZED ) + r = &tdx_sysinfo; +mutex_unlock (&tdx_module_lock); +return r; +} +EXPORT_SYMBOL_GPL (tdx_get_sysinfo); +/* * Only E820_TYPE_RAM and E820_TYPE_PRAM are considered as candidate for * TDX usable memory. The latter is treated as RAM because it is created
@@ -1383,7 +1395,7 @@ static int init_tdx_module(void) goto out;/* Get TDX module information and CMRs */ - ret =tdx_get_sysinfo (); + ret =__tdx_get_sysinfo ();if (ret) goto out;
arch/x86/virt/vmx/tdx.h +0 -52
58 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 007/104] x86/virt/tdx: Add a helper function to return system wide info about TDX module
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Signed-off-by Isaku Yamahata<[email protected]>--- arch/x86/include/asm/tdx.h | 55 ++++++++++++++++++++++++++++++++++++++ arch/x86/virt/vmx/tdx.c | 16 +++++++++-- arch/x86/virt/vmx/tdx.h | 52 ----------------------------------- 3 files changed, 69 insertions(+), 54 deletions(-)
Patch looks good, but place these definitions in arch/x86/include/asm/tdx.h already in Kai's series if possible.
Apart from that,
Paolo
Re: [RFC PATCH v5 007/104] x86/virt/tdx: Add a helper function to return system wide info about TDX module
Quoted 8 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Signed-off-by Isaku Yamahata<[email protected]>--- arch/x86/include/asm/tdx.h | 55 ++++++++++++++++++++++++++++++++++++++ arch/x86/virt/vmx/tdx.c | 16 +++++++++-- arch/x86/virt/vmx/tdx.h | 52 ----------------------------------- 3 files changed, 69 insertions(+), 54 deletions(-)
Quoted 8 lines
Patch looks good, but place these definitions in
arch/x86/include/asm/tdx.halready in Kai's series if possible.Apart from that,
Reviewed-by Paolo Bonzini <[email protected]>Paolo
Does it make more sense for me to just include this patch (and couple of other patches such as exporting information of TDX KeyIDs) to host kernel support series?
Signature · 2 lines
Thanks, -Kai
RFC PATCH v5 8/104 KVM: TDX: Add a function to initialize TDX module
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Memory used for TDX is encrypted with an encryption key. An encryption key is assigned to guest TD, and TDX memory is encrypted. VMM calculates Trust Domain Memory Range (TDMR), a range of memory pages that can hold TDX memory encrypted with an encryption key. VMM allocates memory regions for Physical Address Metadata Table (PAMT) which the TDX module uses to track page states. Used for TDX memory, assigned to which guest TD, etc. VMM gives PAMT regions to the TDX module and initializes it which is also encrypted.
TDX requires more initialization steps in addition to VMX. As a preparation step, check if the CPU feature is available and enable VMX because the TDX module API requires VMX to be enabled to be functional. The next step is basic platform initialization. Check if TDX module API is available, call system-wide initialization API (TDH.SYS.INIT), and call LP initialization API (TDH.SYS.LP.INIT). Lastly, get system-wide parameters (TDH.SYS.INFO), allocate PAMT for TDX module to track page states (TDH.SYS.CONFIG), configure encryption key (TDH.SYS.KEY.CONFIG), and initialize PAMT (TDH.SYS.TDMR.INIT).
A TDX host patch series implements those details and it provides APIs, seamrr_enabled() to check if CPU feature is available, init_tdx() to initialize the TDX module, tdx_get_tdsysinfo() to get TDX system parameters.
Add a wrapper function to initialize the TDX module and get system-wide parameters via those APIs. Because TDX requires VMX enabled, It will be called on-demand when the first guest TD is created via x86 KVM init_vm callback.
--- arch/x86/kvm/vmx/tdx.c | 89 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 4 ++ 2 files changed, 93 insertions(+)
arch/x86/kvm/vmx/tdx.c +89 -0
@@ -13,9 +13,98 @@static bool __read_mostly enable_tdx = true;module_param_named (tdx, enable_tdx, bool,0644 ); +#define TDX_MAX_NR_CPUID_CONFIGS \ + ((sizeof(struct tdsysinfo_struct) - \ + offsetof(struct tdsysinfo_struct, cpuid_configs)) \ + / sizeof(struct tdx_cpuid_config)) + +struct tdx_capabilities { +u8 tdcs_nr_pages; +u8 tdvpx_nr_pages; + +u64 attrs_fixed0; +u64 attrs_fixed1; +u64 xfam_fixed0; +u64 xfam_fixed1; + +u32 nr_cpuid_configs; +struct tdx_cpuid_config cpuid_configs[TDX_MAX_NR_CPUID_CONFIGS ]; +}; + +/* Capabilities of KVM + the TDX module. */ +struct tdx_capabilities tdx_caps; +static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ; +static int __tdx_module_setup (void ) +{ +const struct tdsysinfo_struct *tdsysinfo; +int ret =0 ; + +BUILD_BUG_ON (sizeof (*tdsysinfo) !=1024 ); +BUILD_BUG_ON (TDX_MAX_NR_CPUID_CONFIGS !=37 ); + + ret =tdx_detect (); +if (ret) { +pr_info ("Failed to detect TDX module.\n" ); +return ret; + } + + ret =tdx_init (); +if (ret) { +pr_info ("Failed to initialize TDX module.\n" ); +return ret; + } + + tdsysinfo =tdx_get_sysinfo (); +if (tdx_caps.nr_cpuid_configs >TDX_MAX_NR_CPUID_CONFIGS ) +return -EIO ; + + tdx_caps = (struct tdx_capabilities ) { + .tdcs_nr_pages = tdsysinfo->tdcs_base_size /PAGE_SIZE , +/* +* TDVPS = TDVPR(4K page) + TDVPX(multiple 4K pages). +* -1 for TDVPR. +*/ + .tdvpx_nr_pages = tdsysinfo->tdvps_base_size /PAGE_SIZE -1 , + .attrs_fixed0 = tdsysinfo->attributes_fixed0, + .attrs_fixed1 = tdsysinfo->attributes_fixed1, + .xfam_fixed0 = tdsysinfo->xfam_fixed0, + .xfam_fixed1 = tdsysinfo->xfam_fixed1, + .nr_cpuid_configs = tdsysinfo->num_cpuid_config, + }; +if (!memcpy (tdx_caps.cpuid_configs, tdsysinfo->cpuid_configs, + tdsysinfo->num_cpuid_config * +sizeof (struct tdx_cpuid_config ))) +return -EIO ; + +return 0 ; +} + +int tdx_module_setup (void ) +{ +static DEFINE_MUTEX (tdx_init_lock); +static bool __read_mostly tdx_module_initialized; +int ret =0 ; + +mutex_lock (&tdx_init_lock); + +if (!tdx_module_initialized) { +if (enable_tdx) { + ret =__tdx_module_setup (); +if (ret) + enable_tdx = false; +else + tdx_module_initialized = true; + }else + ret = -EOPNOTSUPP ; + } + +mutex_unlock (&tdx_init_lock); +return ret; +} +static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {u32 max_pa;
arch/x86/kvm/vmx/tdx.h +4 -0
@@ -3,6 +3,8 @@#define __KVM_X86_TDX_H#ifdef CONFIG_INTEL_TDX_HOST +int tdx_module_setup (void ); +struct kvm_tdx {struct kvm kvm; };
@@ -35,6 +37,8 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_tdx, vcpu); }#else +static inline int tdx_module_setup (void ) {return -ENODEV ; }; +struct kvm_tdx ;struct vcpu_tdx ;
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
arch/x86/kvm/vmx/tdx.c line 97
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +6
-0+ +if (!tdx_module_initialized) { +if (enable_tdx) { + ret =__tdx_module_setup (); +if (ret) + enable_tdx = false;
"enable_tdx = false" isn't great to do only when a VM is created. Does it make sense to anticipate this to the point when the kvm_intel.ko module is loaded?
Paolo
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+else + tdx_module_initialized = true; + }else + ret = -EOPNOTSUPP ; + } +
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 11 lines · Paolo Bonzini
arch/x86/kvm/vmx/tdx.c line 97
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +6
-0+ +if (!tdx_module_initialized) { +if (enable_tdx) { + ret =__tdx_module_setup (); +if (ret) + enable_tdx = false;"
enable_tdx= false" isn't great to do only when a VM is created. Does it make sense to anticipate this to the point when the kvm_intel.ko module is loaded?
It's possible. I have the following two reasons to chose to defer TDX module initialization until creating first TD. Given those reasons, do you still want the initialization at loading kvm_intel.ko module? If yes, I'll change it.
- memory over head: The initialization of TDX module requires to allocate physically contiguous memory whose size is about 0.43% of the system memory. If user don't use TD, it will be wasted.
- VMXON on all pCPUs: The TDX module initialization requires to enable VMX (VMXON) on all present pCPUs. vmx_hardware_enable() which is called on creating guest does it. It naturally fits with the TDX module initialization at creating first TD. I wanted to avoid code to enable VMXON on loading the kvm_intel.ko.
Signature · 1 line
Isaku Yamahata <[email protected]>
Sean Christopherson
maintainerRe: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Paolo Bonzini
arch/x86/kvm/vmx/tdx.c line 97
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +6
-0+ +if (!tdx_module_initialized) { +if (enable_tdx) { + ret =__tdx_module_setup (); +if (ret) + enable_tdx = false;"
enable_tdx= false" isn't great to do only when a VM is created. Does it make sense to anticipate this to the point when the kvm_intel.ko module is loaded?
Quoted 3 lines
It's possible. I have the following two reasons to chose to defer TDX module initialization until creating first TD. Given those reasons, do you still want the initialization at loading kvm_intel.ko module? If yes, I'll change it.
Yes, TDX module setup needs to be done at load time. The loss of memory is unfortunate, e.g. if the host is part of a pool that _might_ run TDX guests, but the alternatives are worse. If TDX fails to initialize, e.g. due to low mem, then the host will be unable to run TDX guests despite saying "I support TDX". Or this gem :-)
/*
* TDH.SYS.KEY.CONFIG may fail with entropy error (which is
* a recoverable error). Assume this is exceedingly rare and
* just return error if encountered instead of retrying. */
The CPU overhead of initializing the TDX module is also non-trivial, and it doesn't affect just this CPU, e.g. all CPUs need to do certain SEAMCALLs and at least one WBINVD. The can cause noisy neighbor problems.
Quoted 8 lines
- memory over head: The initialization of TDX module requires to allocate physically contiguous memory whose size is about 0.43% of the system memory. If user don't use TD, it will be wasted.
- VMXON on all pCPUs: The TDX module initialization requires to enable VMX (VMXON) on all present pCPUs.
vmx_hardware_enable()which is called on creating guest does it. It naturally fits with the TDX module initialization at creating first TD. I wanted to avoid code to enable VMXON on loading the kvm_intel.ko.
That's a solvable problem, though making it work without exporting hardware_enable_all() could get messy.
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 9 lines
Quoted 6 lines
- VMXON on all pCPUs: The TDX module initialization requires to enable VMX (VMXON) on all present pCPUs.
vmx_hardware_enable()which is called on creating guest does it. It naturally fits with the TDX module initialization at creating first TD. I wanted to avoid code to enable VMXON on loading the kvm_intel.ko.That's a solvable problem, though making it work without exporting
hardware_enable_all()could get messy.
Could you elaborate a little bit on how to resolve?
Signature · 2 lines
Thanks, -Kai
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
In reply to Sean ChristophersonQuoted 20 lines · Sean Christopherson
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Paolo Bonzini
arch/x86/kvm/vmx/tdx.c line 97
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +6
-0+ +if (!tdx_module_initialized) { +if (enable_tdx) { + ret =__tdx_module_setup (); +if (ret) + enable_tdx = false;"
enable_tdx= false" isn't great to do only when a VM is created. Does it make sense to anticipate this to the point when the kvm_intel.ko module is loaded?Quoted 3 lines
It's possible. I have the following two reasons to chose to defer TDX module initialization until creating first TD. Given those reasons, do you still want the initialization at loading kvm_intel.ko module? If yes, I'll change it.
Quoted 5 lines
Yes, TDX module setup needs to be done at load time. The loss of memory is unfortunate, e.g. if the host is part of a pool that _might_ run TDX guests, but the alternatives are worse. If TDX fails to initialize, e.g. due to low mem, then the host will be unable to run TDX guests despite saying "I support TDX". Or this gem :-)
Ok.
Quoted 11 lines
Quoted 8 lines
- memory over head: The initialization of TDX module requires to allocate physically contiguous memory whose size is about 0.43% of the system memory. If user don't use TD, it will be wasted.
- VMXON on all pCPUs: The TDX module initialization requires to enable VMX (VMXON) on all present pCPUs.
vmx_hardware_enable()which is called on creating guest does it. It naturally fits with the TDX module initialization at creating first TD. I wanted to avoid code to enable VMXON on loading the kvm_intel.ko.That's a solvable problem, though making it work without exporting
hardware_enable_all()could get messy.
Could you please explain any reason why it's bad idea to export it?
Signature · 1 line
Isaku Yamahata <[email protected]>
Sean Christopherson
maintainerRe: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
+Chao Gao
Quoted 11 lines · Isaku Yamahata
Quoted 8 lines · Sean Christopherson
Quoted 4 lines · Isaku Yamahata
- VMXON on all pCPUs: The TDX module initialization requires to enable VMX (VMXON) on all present pCPUs.
vmx_hardware_enable()which is called on creating guest does it. It naturally fits with the TDX module initialization at creating first TD. I wanted to avoid code to enable VMXON on loading the kvm_intel.ko.That's a solvable problem, though making it work without exporting
hardware_enable_all()could get messy.Could you please explain any reason why it's bad idea to export it?
I'd really prefer to keep the hardware enable/disable logic internal to kvm_main.c so that all architectures share a common flow, and so that kvm_main.c is the sole owner. I'm worried that exposing the helper will lead to other arch/vendor usage, and that will end up with what is effectively duplicate flows. Deduplicating arch code into generic KVM is usually very difficult.
This might also be a good opportunity to make KVM slightly more robust. Ooh, and we can kill two birds with one stone. There's an in-flight series to add compatibility checks to hotplug[*]. But rather than special case hotplug, what if we instead do hardware enable/disable during module load, and move the compatibility check into the hardware_enable path? That fixes the hotplug issue, gives TDX a window for running post-VMXON code in kvm_init(), and makes the broadcast IPI less wasteful on architectures that don't have compatiblity checks.
I'm thinking something like this, maybe as a modificatyion to patch 6 in Chao's series, or more likely as a patch 7 so that the hotplug compat checks still get in even if the early hardware enable doesn't work on all architectures for some reason.
virt/kvm/kvm_main.c +12 -19
@@ -4838,8 +4838,13 @@ static void hardware_enable_nolock(void *junk)cpumask_set_cpu (cpu, cpus_hardware_enabled); + r =kvm_arch_check_processor_compat (); +if (r) + goto out; + r =kvm_arch_hardware_enable (); +out:if (r) {cpumask_clear_cpu (cpu, cpus_hardware_enabled);atomic_inc (&hardware_enable_failed);
@@ -5636,18 +5641,6 @@ void kvm_unregister_perf_callbacks(void) }#endif -struct kvm_cpu_compat_check { -void *opaque; -int *ret; -}; - -static void check_processor_compat (void *data) -{ -struct kvm_cpu_compat_check *c = data; - - *c->ret =kvm_arch_check_processor_compat (c->opaque); -} -int kvm_init (void *opaque,unsigned vcpu_size,unsigned vcpu_align,struct module *module) {
@@ -5679,13 +5672,13 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align,if (r <0 ) goto out_free_1; - c.ret = &r; - c.opaque = opaque; -for_each_online_cpu (cpu) { -smp_call_function_single (cpu, check_processor_compat, &c,1 ); -if (r <0 ) - goto out_free_2; - } + r =hardware_enable_all (); +if (r) + goto out_free_2; + +kvm_arch_post_hardware_enable_setup (); + +hardware_disable_all (); r =cpuhp_setup_state_nocalls (CPUHP_AP_KVM_STARTING ,"kvm/cpu:starting" , kvm_starting_cpu, kvm_dying_cpu);
A message that is not in the archive (sent off-list or not yet received): 20220401032741.GA2806@gao-cwp
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
The original reply was sent to Sean only by mistake. Add others back.
Quoted 48 lines · Chao Gao
Quoted 21 lines · Sean Christopherson
+Chao Gao
Quoted 11 lines · Isaku Yamahata
Quoted 8 lines · Sean Christopherson
Quoted 4 lines · Isaku Yamahata
- VMXON on all pCPUs: The TDX module initialization requires to enable VMX (VMXON) on all present pCPUs.
vmx_hardware_enable()which is called on creating guest does it. It naturally fits with the TDX module initialization at creating first TD. I wanted to avoid code to enable VMXON on loading the kvm_intel.ko.That's a solvable problem, though making it work without exporting
hardware_enable_all()could get messy.Could you please explain any reason why it's bad idea to export it?
I'd really prefer to keep the hardware enable/disable logic internal to
kvm_main.cso that all architectures share a common flow, and so thatkvm_main.cis the sole owner. I'm worried that exposing the helper will lead to otherarch/vendorusage, and that will end up with what is effectively duplicate flows. Deduplicating arch code into generic KVM is usually very difficult.Quoted 7 lines
This might also be a good opportunity to make KVM slightly more robust. Ooh, and we can kill two birds with one stone. There's an in-flight series to add compatibility checks to hotplug[*]. But rather than special case hotplug, what if we instead do hardware enable/disable during module load, and move the compatibility check into the
hardware_enablepath? That fixes the hotplug issue, gives TDX a window for running post-VMXON code inkvm_init(), and makes the broadcast IPI less wasteful on architectures that don't have compatiblity checks.Sounds good. But more time is wasted on compat checks on architectures that have them because they are done each time of enabling hardware. A solution for this is caching the result of
kvm_arch_check_processor_compat().Quoted 7 lines
I'm thinking something like this, maybe as a modificatyion to patch 6 in Chao's series, or more likely as a patch 7 so that the hotplug compat checks still get in even
if the early hardware enable doesn't work on all architectures for some reason.
By "early", do you mean hardware enable during module loading or during CPU hotplug?
And if below change is put into my series,
kvm_arch_post_hardware_enable_setup()will be an empty function for all architectures until TDX series gets merged. So, I prefer to dropkvm_arch_post_hardware_enable_setup()and let TDX series introduce it.
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
In reply to [email protected]Quoted 10 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Memory used for TDX is encrypted with an encryption key. An encryption key is assigned to guest TD, and TDX memory is encrypted. VMM calculates Trust Domain Memory Range (TDMR), a range of memory pages that can hold TDX memory encrypted with an encryption key. VMM allocates memory regions for Physical Address Metadata Table (PAMT) which the TDX module uses to track page states. Used for TDX memory, assigned to which guest TD, etc. VMM gives PAMT regions to the TDX module and initializes it which is also encrypted.
Not sure why above are related to this patch. Perhaps you can just say TDX module is detected and initialized on demand via tdx_detect() and tdx_init().
Quoted 4 lines
TDX requires more initialization steps in addition to VMX. As a preparation step, check if the CPU feature is available and enable VMX because the TDX module API requires VMX to be enabled to be functional.
Those are not reflected in this patch either.
Quoted 6 lines
The next step is basic platform initialization. Check if TDX module API is available, call system-wide initialization API (TDH.SYS.INIT), and call LP initialization API (TDH.SYS.LP.INIT). Lastly, get system-wide parameters (TDH.SYS.INFO), allocate PAMT for TDX module to track page states (TDH.SYS.CONFIG), configure encryption key (TDH.SYS.KEY.CONFIG), and initialize PAMT (TDH.SYS.TDMR.INIT).
Again, not sure why those are related.
Quoted 5 lines
A TDX host patch series implements those details and it provides APIs,
seamrr_enabled()to check if CPU feature is available,init_tdx()to initialize the TDX module,tdx_get_tdsysinfo()to get TDX system parameters.
init_tdx() -> tdx_init().
"A TDX host patch series" should not be in the formal commit message, I suppose.
Quoted 5 lines
Add a wrapper function to initialize the TDX module and get system-wide parameters via those APIs. Because TDX requires VMX enabled, It will be called on-demand when the first guest TD is created via x86 KVM
init_vmcallback.
Why not just merge this patch with the change where you implement the init_vm callback? Then you can just declare this patch as "detect and initialize TDX module when first VM is created", or something like that..
Quoted 28 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/tdx.c | 89 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 4 ++ 2 files changed, 93 insertions(+)arch/x86/kvm/vmx/tdx.c +13
-0@@ -13,9 +13,98 @@static bool __read_mostly enable_tdx = true;module_param_named (tdx, enable_tdx, bool,0644 ); +#define TDX_MAX_NR_CPUID_CONFIGS \ + ((sizeof(struct tdsysinfo_struct) - \ + offsetof(struct tdsysinfo_struct, cpuid_configs)) \ + / sizeof(struct tdx_cpuid_config)) + +struct tdx_capabilities { +u8 tdcs_nr_pages; +u8 tdvpx_nr_pages; + +u64 attrs_fixed0; +u64 attrs_fixed1; +u64 xfam_fixed0; +u64 xfam_fixed1;
arch/x86/kvm/vmx/tdx.c line 38
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +8
-0+ +u32 nr_cpuid_configs; +struct tdx_cpuid_config cpuid_configs[TDX_MAX_NR_CPUID_CONFIGS ]; +}; + +/* Capabilities of KVM + the TDX module. */ +struct tdx_capabilities tdx_caps; +static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ;
The two seems are not used in this patch.
Please make sure each patch can compile w/o warning.
Quoted 43 lines
arch/x86/kvm/vmx/tdx.c +42
-0+static int __tdx_module_setup (void ) +{ +const struct tdsysinfo_struct *tdsysinfo; +int ret =0 ; + +BUILD_BUG_ON (sizeof (*tdsysinfo) !=1024 ); +BUILD_BUG_ON (TDX_MAX_NR_CPUID_CONFIGS !=37 ); + + ret =tdx_detect (); +if (ret) { +pr_info ("Failed to detect TDX module.\n" ); +return ret; + } + + ret =tdx_init (); +if (ret) { +pr_info ("Failed to initialize TDX module.\n" ); +return ret; + } + + tdsysinfo =tdx_get_sysinfo (); +if (tdx_caps.nr_cpuid_configs >TDX_MAX_NR_CPUID_CONFIGS ) +return -EIO ; + + tdx_caps = (struct tdx_capabilities ) { + .tdcs_nr_pages = tdsysinfo->tdcs_base_size /PAGE_SIZE , +/* +* TDVPS = TDVPR(4K page) + TDVPX(multiple 4K pages). +* -1 for TDVPR. +*/ + .tdvpx_nr_pages = tdsysinfo->tdvps_base_size /PAGE_SIZE -1 , + .attrs_fixed0 = tdsysinfo->attributes_fixed0, + .attrs_fixed1 = tdsysinfo->attributes_fixed1, + .xfam_fixed0 = tdsysinfo->xfam_fixed0, + .xfam_fixed1 = tdsysinfo->xfam_fixed1, + .nr_cpuid_configs = tdsysinfo->num_cpuid_config, + }; +if (!memcpy (tdx_caps.cpuid_configs, tdsysinfo->cpuid_configs, + tdsysinfo->num_cpuid_config * +sizeof (struct tdx_cpuid_config ))) +return -EIO ; +
arch/x86/kvm/vmx/tdx.c line 91
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+return 0 ; +} + +int tdx_module_setup (void ) +{ +static DEFINE_MUTEX (tdx_init_lock); +static bool __read_mostly tdx_module_initialized; +int ret =0 ; + +mutex_lock (&tdx_init_lock);
It took me a while to figure out why this mutex is needed. Please see my above suggestion to merge this patch to the change that implements init_vm() callback.
arch/x86/kvm/vmx/tdx.c line 95
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4
-0+ +if (!tdx_module_initialized) { +if (enable_tdx) { + ret =__tdx_module_setup ();
I think you can move tdx_detect() and tdx_init() out of your mutex. They are internally protected by mutex.
Quoted 37 lines
arch/x86/kvm/vmx/tdx.c +12
-0+if (ret) + enable_tdx = false; +else + tdx_module_initialized = true; + }else + ret = -EOPNOTSUPP ; + } + +mutex_unlock (&tdx_init_lock); +return ret; +} +static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {u32 max_pa;arch/x86/kvm/vmx/tdx.h +4
-0@@ -3,6 +3,8 @@#define __KVM_X86_TDX_H#ifdef CONFIG_INTEL_TDX_HOST +int tdx_module_setup (void ); +struct kvm_tdx {struct kvm kvm; };@@ -35,6 +37,8 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_tdx, vcpu); }#else +static inline int tdx_module_setup (void ) {return -ENODEV ; }; +struct kvm_tdx ;struct vcpu_tdx ;
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 11 lines · Kai Huang
Quoted 6 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a wrapper function to initialize the TDX module and get system-wide parameters via those APIs. Because TDX requires VMX enabled, It will be called on-demand when the first guest TD is created via x86 KVM
init_vmcallback.Why not just merge this patch with the change where you implement the
init_vmcallback? Then you can just declare this patch as "detect and initialize TDX module when first VM is created", or something like that..
Ok. Anyway in the next respoin, tdx module initialization will be done when loading kvm_intel.ko. So the whole part will be changed and will be a part of module loading.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 11 lines · Isaku Yamahata
Quoted 8 lines · Kai Huang
Quoted 6 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a wrapper function to initialize the TDX module and get system-wide parameters via those APIs. Because TDX requires VMX enabled, It will be called on-demand when the first guest TD is created via x86 KVM
init_vmcallback.
Quoted 7 lines
Quoted 3 lines
Why not just merge this patch with the change where you implement the
init_vmcallback? Then you can just declare this patch as "detect and initialize TDX module when first VM is created", or something like that..Ok. Anyway in the next respoin, tdx module initialization will be done when loading kvm_intel.ko. So the whole part will be changed and will be a part of module loading.
Will we change the GET_TDX_CAPABILITIES ioctl back to KVM scope?
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 16 lines · Xiaoyao Li
Quoted 15 lines · Isaku Yamahata
Quoted 11 lines · Kai Huang
Quoted 6 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a wrapper function to initialize the TDX module and get system-wide parameters via those APIs. Because TDX requires VMX enabled, It will be called on-demand when the first guest TD is created via x86 KVM
init_vmcallback.Why not just merge this patch with the change where you implement the
init_vmcallback? Then you can just declare this patch as "detect and initialize TDX module when first VM is created", or something like that..
Quoted 5 lines
Quoted 3 lines
Ok. Anyway in the next respoin, tdx module initialization will be done when loading kvm_intel.ko. So the whole part will be changed and will be a part of module loading.
Will we change the
GET_TDX_CAPABILITIESioctl back to KVM scope?
No because it system scoped KVM_TDX_CAPABILITIES requires one more callback for it. We can reduce the change.
Or do you have any use case for system scoped KVM_TDX_CAPABILITIES?
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH v5 008/104] KVM: TDX: Add a function to initialize TDX module
Quoted 23 lines · Isaku Yamahata
Quoted 20 lines · Xiaoyao Li
Quoted 18 lines · Isaku Yamahata
Quoted 11 lines · Kai Huang
Quoted 6 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a wrapper function to initialize the TDX module and get system-wide parameters via those APIs. Because TDX requires VMX enabled, It will be called on-demand when the first guest TD is created via x86 KVM
init_vmcallback.Why not just merge this patch with the change where you implement the
init_vmcallback? Then you can just declare this patch as "detect and initialize TDX module when first VM is created", or something like that..Ok. Anyway in the next respoin, tdx module initialization will be done when loading kvm_intel.ko. So the whole part will be changed and will be a part of module loading.
Quoted 6 lines
Quoted 1 line
Will we change the
GET_TDX_CAPABILITIESioctl back to KVM scope?No because it system scoped
KVM_TDX_CAPABILITIESrequires one more callback for it. We can reduce the change.Or do you have any use case for system scoped
KVM_TDX_CAPABILITIES?
No. Just to confirm.
on the other hand, vm-scope IOCTL seems more flexible if different capabilities are reported per VM in the future.
RFC PATCH v5 9/104 KVM: x86: Introduce vm_type to differentiate default VMs from confidential VMs
In reply to [email protected]From: Sean Christopherson <[email protected]>
Unlike default VMs, confidential VMs (Intel TDX and AMD SEV-ES) don't allow some operations (e.g., memory read/write, register state access, etc).
Introduce vm_type to track the type of the VM to x86 KVM. Other arch KVMs already use vm_type, KVM_INIT_VM accepts vm_type, and x86 KVM callback vm_init accepts vm_type. So follow them. Further, a different policy can be made based on vm_type. Define KVM_X86_DEFAULT_VM for default VM as default and define KVM_X86_TDX_VM for Intel TDX VM. The wrapper function will be defined as "bool is_td(kvm) { return vm_type == VM_TYPE_TDX; }"
Add a capability KVM_CAP_VM_TYPES to effectively allow device model, e.g. qemu, to query what VM types are supported by KVM. This (introduce a new capability and add vm_type) is chosen to align with other arch KVMs that have VM types already. Other arch KVMs uses different name to query supported vm types and there is no common name for it, so new name was chosen.
--- Documentation/virt/kvm/api.rst | 15 +++++++++++++++ arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/uapi/asm/kvm.h | 3 +++ arch/x86/kvm/svm/svm.c | 6 ++++++ arch/x86/kvm/vmx/main.c | 1 + arch/x86/kvm/vmx/tdx.h | 6 +----- arch/x86/kvm/vmx/vmx.c | 5 +++++ arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 9 ++++++++- include/uapi/linux/kvm.h | 1 + tools/arch/x86/include/uapi/asm/kvm.h | 3 +++ tools/include/uapi/linux/kvm.h | 1 + 13 files changed, 48 insertions(+), 6 deletions(-)
Documentation/virt/kvm/api.rst +15 -0
@@ -147,15 +147,30 @@ described as 'basic' will be available. The new VM has no virtual cpus and no memory. You probably want to use 0 as machine type. +X86: +^^^^ + +Supported vm type can be queried from KVM_CAP_VM_TYPES, which returns the +bitmap of supported vm types. The 1-setting of bit @n means vm type with +value @n is supported. + +S390: +^^^^^ + In order to create user controlled virtual machines on S390, check KVM_CAP_S390_UCONTROL and use the flag KVM_VM_S390_UCONTROL as privileged user (CAP_SYS_ADMIN). +MIPS: +^^^^^ + To use hardware assisted virtualization on MIPS (VZ ASE) rather than the default trap & emulate implementation (which changes the virtual memory layout to fit in user mode), check KVM_CAP_MIPS_VZ and use the flag KVM_VM_MIPS_VZ. +ARM64: +^^^^^^ On arm64, the physical address size for a VM (IPA Size limit) is limited to 40bits by default. The limit can be configured if the host supports the
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -18,6 +18,7 @@ KVM_X86_OP_NULL(hardware_unsetup) KVM_X86_OP_NULL(cpu_has_accelerated_tpr) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1048,6 +1048,7 @@ struct kvm_x86_msr_filter {#define APICV_INHIBIT_REASON_ABSENT 7struct kvm_arch { +unsigned long vm_type;unsigned long n_used_mmu_pages;unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;
@@ -1322,6 +1323,7 @@ struct kvm_x86_ops {bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); +bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);
arch/x86/include/uapi/asm/kvm.h +3 -0
@@ -525,4 +525,7 @@ struct kvm_pmu_event_filter {#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ #define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_TDX_VM 1 +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/svm/svm.c +6 -0
@@ -4512,6 +4512,11 @@ static void svm_vm_destroy(struct kvm *kvm)sev_vm_destroy (kvm); } +static bool svm_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_DEFAULT_VM ; +} +static int svm_vm_init (struct kvm *kvm) {if (!pause_filter_count || !pause_filter_thresh)
@@ -4539,6 +4544,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_free_vcpu, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy,
arch/x86/kvm/vmx/main.c +1 -0
@@ -29,6 +29,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .cpu_has_accelerated_tpr = report_flexpriority, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init,
arch/x86/kvm/vmx/tdx.h +1 -5
@@ -15,11 +15,7 @@ struct vcpu_tdx {static inline bool is_td (struct kvm *kvm) { -/* -* TDX VM type isn't defined yet. -* return kvm->arch.vm_type == KVM_X86_TDX_VM; -*/ -return false; +return kvm->arch.vm_type ==KVM_X86_TDX_VM ; }static inline bool is_td_vcpu (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/vmx.c +5 -0
@@ -7079,6 +7079,11 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } +bool vmx_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_DEFAULT_VM ; +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
arch/x86/kvm/vmx/x86_ops.h +1 -0
@@ -25,6 +25,7 @@ void vmx_hardware_unsetup(void);int vmx_hardware_enable (void );void vmx_hardware_disable (void );bool report_flexpriority (void ); +bool vmx_is_vm_type_supported (unsigned long type);int vmx_vm_init (struct kvm *kvm);int vmx_vcpu_create (struct kvm_vcpu *vcpu);int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +8 -1
@@ -4344,6 +4344,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r =sizeof (struct kvm_xsave );break ; } +case KVM_CAP_VM_TYPES : + r =BIT (KVM_X86_DEFAULT_VM ); +if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) + r |=BIT (KVM_X86_TDX_VM ); +break ;default :break ; }
@@ -11583,9 +11588,11 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)int ret;unsigned long flags; -if (type) +if (!static_call (kvm_x86_is_vm_type_supported)(type))return -EINVAL ; + kvm->arch.vm_type = type; + ret =kvm_page_track_init (kvm);if (ret)return ret;
include/uapi/linux/kvm.h +1 -0
@@ -1135,6 +1135,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_XSAVE2 208#define KVM_CAP_SYS_ATTRIBUTES 209#define KVM_CAP_PPC_AIL_MODE_3 210 +#define KVM_CAP_VM_TYPES 21#ifdef KVM_CAP_IRQ_ROUTING
tools/arch/x86/include/uapi/asm/kvm.h +3 -0
@@ -525,4 +525,7 @@ struct kvm_pmu_event_filter {#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ #define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_TDX_VM 1 +#endif /* _ASM_X86_KVM_H */
tools/include/uapi/linux/kvm.h +1 -0
@@ -1135,6 +1135,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_XSAVE2 208#define KVM_CAP_SYS_ATTRIBUTES 209#define KVM_CAP_PPC_AIL_MODE_3 210 +#define KVM_CAP_VM_TYPES 211#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 009/104] KVM: x86: Introduce vm_type to differentiate default VMs from confidential VMs
Quoted 19 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Unlike default VMs, confidential VMs (Intel TDX and AMD SEV-ES) don't allow some operations (e.g., memory read/write, register state access, etc).
Introduce
vm_typeto track the type of the VM to x86 KVM. Other arch KVMs already usevm_type,KVM_INIT_VMacceptsvm_type, and x86 KVM callbackvm_initacceptsvm_type. So follow them. Further, a different policy can be made based onvm_type. DefineKVM_X86_DEFAULT_VMfor default VM as default and defineKVM_X86_TDX_VMfor Intel TDX VM. The wrapper function will be defined as "boolis_td(kvm){ returnvm_type==VM_TYPE_TDX; }"Add a capability
KVM_CAP_VM_TYPESto effectively allow device model, e.g. qemu, to query what VM types are supported by KVM. This (introduce a new capability and addvm_type) is chosen to align with other arch KVMs that have VM types already. Other arch KVMs uses different name to query supported vm types and there is no common name for it, so new name was chosen.
Quoted 4 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
Quoted 241 lines
--- Documentation/virt/kvm/api.rst | 15 +++++++++++++++ arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/include/uapi/asm/kvm.h | 3 +++ arch/x86/kvm/svm/svm.c | 6 ++++++ arch/x86/kvm/vmx/main.c | 1 + arch/x86/kvm/vmx/tdx.h | 6 +----- arch/x86/kvm/vmx/vmx.c | 5 +++++ arch/x86/kvm/vmx/x86_ops.h | 1 + arch/x86/kvm/x86.c | 9 ++++++++- include/uapi/linux/kvm.h | 1 + tools/arch/x86/include/uapi/asm/kvm.h | 3 +++ tools/include/uapi/linux/kvm.h | 1 + 13 files changed, 48 insertions(+), 6 deletions(-)Documentation/virt/kvm/api.rst +15
-0@@ -147,15 +147,30 @@ described as 'basic' will be available. The new VM has no virtual cpus and no memory. You probably want to use 0 as machine type. +X86: +^^^^ + +Supported vm type can be queried from KVM_CAP_VM_TYPES, which returns the +bitmap of supported vm types. The 1-setting of bit @n means vm type with +value @n is supported. + +S390: +^^^^^ + In order to create user controlled virtual machines on S390, check KVM_CAP_S390_UCONTROL and use the flag KVM_VM_S390_UCONTROL as privileged user (CAP_SYS_ADMIN). +MIPS: +^^^^^ + To use hardware assisted virtualization on MIPS (VZ ASE) rather than the default trap & emulate implementation (which changes the virtual memory layout to fit in user mode), check KVM_CAP_MIPS_VZ and use the flag KVM_VM_MIPS_VZ. +ARM64: +^^^^^^ On arm64, the physical address size for a VM (IPA Size limit) is limited to 40bits by default. The limit can be configured if the host supports thearch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -18,6 +18,7 @@ KVM_X86_OP_NULL(hardware_unsetup) KVM_X86_OP_NULL(cpu_has_accelerated_tpr) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_NULL(vm_destroy) KVM_X86_OP(vcpu_create)arch/x86/include/asm/kvm_host.h +2
-0@@ -1048,6 +1048,7 @@ struct kvm_x86_msr_filter {#define APICV_INHIBIT_REASON_ABSENT 7struct kvm_arch { +unsigned long vm_type;unsigned long n_used_mmu_pages;unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;@@ -1322,6 +1323,7 @@ struct kvm_x86_ops {bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); +bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);arch/x86/include/uapi/asm/kvm.h +3
-0@@ -525,4 +525,7 @@ struct kvm_pmu_event_filter {#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ #define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_TDX_VM 1 +#endif /* _ASM_X86_KVM_H */ arch/x86/kvm/svm/svm.c +6
-0@@ -4512,6 +4512,11 @@ static void svm_vm_destroy(struct kvm *kvm)sev_vm_destroy (kvm); } +static bool svm_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_DEFAULT_VM ; +} +static int svm_vm_init (struct kvm *kvm) {if (!pause_filter_count || !pause_filter_thresh)@@ -4539,6 +4544,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_free_vcpu, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy,arch/x86/kvm/vmx/main.c +1
-0@@ -29,6 +29,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .cpu_has_accelerated_tpr = report_flexpriority, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init,arch/x86/kvm/vmx/tdx.h +1
-5@@ -15,11 +15,7 @@ struct vcpu_tdx {static inline bool is_td (struct kvm *kvm) { -/* -* TDX VM type isn't defined yet. -* return kvm->arch.vm_type == KVM_X86_TDX_VM; -*/ -return false; +return kvm->arch.vm_type ==KVM_X86_TDX_VM ; }static inline bool is_td_vcpu (struct kvm_vcpu *vcpu)arch/x86/kvm/vmx/vmx.c +5
-0@@ -7079,6 +7079,11 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } +bool vmx_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_DEFAULT_VM ; +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"arch/x86/kvm/vmx/x86_ops.h +1
-0@@ -25,6 +25,7 @@ void vmx_hardware_unsetup(void);int vmx_hardware_enable (void );void vmx_hardware_disable (void );bool report_flexpriority (void ); +bool vmx_is_vm_type_supported (unsigned long type);int vmx_vm_init (struct kvm *kvm);int vmx_vcpu_create (struct kvm_vcpu *vcpu);int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +8
-1@@ -4344,6 +4344,11 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r =sizeof (struct kvm_xsave );break ; } +case KVM_CAP_VM_TYPES : + r =BIT (KVM_X86_DEFAULT_VM ); +if (static_call (kvm_x86_is_vm_type_supported)(KVM_X86_TDX_VM )) + r |=BIT (KVM_X86_TDX_VM ); +break ;default :break ; }@@ -11583,9 +11588,11 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)int ret;unsigned long flags; -if (type) +if (!static_call (kvm_x86_is_vm_type_supported)(type))return -EINVAL ; + kvm->arch.vm_type = type; + ret =kvm_page_track_init (kvm);if (ret)return ret;include/uapi/linux/kvm.h +1
-0@@ -1135,6 +1135,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_XSAVE2 208#define KVM_CAP_SYS_ATTRIBUTES 209#define KVM_CAP_PPC_AIL_MODE_3 210 +#define KVM_CAP_VM_TYPES 21#ifdef KVM_CAP_IRQ_ROUTING tools/arch/x86/include/uapi/asm/kvm.h +3
-0@@ -525,4 +525,7 @@ struct kvm_pmu_event_filter {#define KVM_VCPU_TSC_CTRL 0/* control group for the timestamp counter (TSC) */ #define KVM_VCPU_TSC_OFFSET 0/* attribute for the TSC offset */ +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_TDX_VM 1 +#endif /* _ASM_X86_KVM_H */ tools/include/uapi/linux/kvm.h +1
-0@@ -1135,6 +1135,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_XSAVE2 208#define KVM_CAP_SYS_ATTRIBUTES 209#define KVM_CAP_PPC_AIL_MODE_3 210 +#define KVM_CAP_VM_TYPES 211#ifdef KVM_CAP_IRQ_ROUTING
RFC PATCH v5 10/104 KVM: TDX: Make TDX VM type supported
In reply to [email protected]From: Isaku Yamahata <[email protected]>
As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is vm_init callback for KVM_CREATE_VM. Add a place holder function and call a function to initialize TDX module on demand because in that callback VMX is enabled by hardware_enable callback (vmx_hardware_enable).
--- arch/x86/kvm/vmx/main.c | 24 ++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.c | 5 +++++ arch/x86/kvm/vmx/vmx.c | 5 ----- arch/x86/kvm/vmx/x86_ops.h | 3 ++- 4 files changed, 29 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/main.c +22 -2
@@ -5,6 +5,12 @@#include "vmx.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" + +static bool vt_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_DEFAULT_VM ||tdx_is_vm_type_supported (type); +}static __init int vt_hardware_setup (void ) {
@@ -19,6 +25,20 @@ static __init int vt_hardware_setup(void)return 0 ; } +static int vt_vm_init (struct kvm *kvm) +{ +int ret; + +if (is_td (kvm)) { + ret =tdx_module_setup (); +if (ret) +return ret; +return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ + } + +return vmx_vm_init (kvm); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,
@@ -29,9 +49,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .cpu_has_accelerated_tpr = report_flexpriority, .has_emulated_msr = vmx_has_emulated_msr, - .is_vm_type_supported = vmx_is_vm_type_supported, + .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), - .vm_init = vmx_vm_init, + .vm_init = vt_vm_init, .vcpu_create = vmx_vcpu_create, .vcpu_free = vmx_vcpu_free,
arch/x86/kvm/vmx/tdx.c +5 -0
@@ -105,6 +105,11 @@ int tdx_module_setup(void)return ret; } +bool tdx_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_TDX_VM &&READ_ONCE (enable_tdx); +} +static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {u32 max_pa;
arch/x86/kvm/vmx/vmx.c +0 -5
@@ -7079,11 +7079,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } -bool vmx_is_vm_type_supported (unsigned long type) -{ -return type ==KVM_X86_DEFAULT_VM ; -} -#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
arch/x86/kvm/vmx/x86_ops.h +2 -1
@@ -25,7 +25,6 @@ void vmx_hardware_unsetup(void);int vmx_hardware_enable (void );void vmx_hardware_disable (void );bool report_flexpriority (void ); -bool vmx_is_vm_type_supported (unsigned long type);int vmx_vm_init (struct kvm *kvm);int vmx_vcpu_create (struct kvm_vcpu *vcpu);int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu);
@@ -130,10 +129,12 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST void __init tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size); +bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {} +static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {}#endif
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 010/104] KVM: TDX: Make TDX VM type supported
Quoted 7 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is
vm_initcallback forKVM_CREATE_VM. Add a place holder function and call a function to initialize TDX module on demand because in that callback VMX is enabled byhardware_enablecallback (vmx_hardware_enable).
Should we put this patch at the end of series until all changes required to run TD are introduced? This patch essentially tells userspace KVM is ready to support a TD but actually it's not ready. And this might also cause bisect issue I suppose?
Signature · 2 lines
Thanks, -Kai
Quoted 116 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 24 ++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.c | 5 +++++ arch/x86/kvm/vmx/vmx.c | 5 ----- arch/x86/kvm/vmx/x86_ops.h | 3 ++- 4 files changed, 29 insertions(+), 8 deletions(-)arch/x86/kvm/vmx/main.c +22
-2@@ -5,6 +5,12 @@#include "vmx.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" + +static bool vt_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_DEFAULT_VM ||tdx_is_vm_type_supported (type); +}static __init int vt_hardware_setup (void ) {@@ -19,6 +25,20 @@ static __init int vt_hardware_setup(void)return 0 ; } +static int vt_vm_init (struct kvm *kvm) +{ +int ret; + +if (is_td (kvm)) { + ret =tdx_module_setup (); +if (ret) +return ret; +return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ + } + +return vmx_vm_init (kvm); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,@@ -29,9 +49,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .cpu_has_accelerated_tpr = report_flexpriority, .has_emulated_msr = vmx_has_emulated_msr, - .is_vm_type_supported = vmx_is_vm_type_supported, + .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), - .vm_init = vmx_vm_init, + .vm_init = vt_vm_init, .vcpu_create = vmx_vcpu_create, .vcpu_free = vmx_vcpu_free,arch/x86/kvm/vmx/tdx.c +5
-0@@ -105,6 +105,11 @@ int tdx_module_setup(void)return ret; } +bool tdx_is_vm_type_supported (unsigned long type) +{ +return type ==KVM_X86_TDX_VM &&READ_ONCE (enable_tdx); +} +static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {u32 max_pa;arch/x86/kvm/vmx/vmx.c +0
-5@@ -7079,11 +7079,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } -bool vmx_is_vm_type_supported (unsigned long type) -{ -return type ==KVM_X86_DEFAULT_VM ; -} -#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"arch/x86/kvm/vmx/x86_ops.h +2
-1@@ -25,7 +25,6 @@ void vmx_hardware_unsetup(void);int vmx_hardware_enable (void );void vmx_hardware_disable (void );bool report_flexpriority (void ); -bool vmx_is_vm_type_supported (unsigned long type);int vmx_vm_init (struct kvm *kvm);int vmx_vcpu_create (struct kvm_vcpu *vcpu);int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu);@@ -130,10 +129,12 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST void __init tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size); +bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {} +static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {}#endif
Re: [RFC PATCH v5 010/104] KVM: TDX: Make TDX VM type supported
Quoted 9 lines · Kai Huang
Quoted 7 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is
vm_initcallback forKVM_CREATE_VM. Add a place holder function and call a function to initialize TDX module on demand because in that callback VMX is enabled byhardware_enablecallback (vmx_hardware_enable).
Quoted 4 lines
Should we put this patch at the end of series until all changes required to run TD are introduced? This patch essentially tells userspace KVM is ready to support a TD but actually it's not ready. And this might also cause bisect issue I suppose?
The intention is that developers can exercise the new code step-by-step even if the TDX KVM isn't complete.
How about introducing new config and remove it at the last of the patch series?
arch/x86/kvm/Kconfig +14 -0
@@ -98,6 +98,20 @@ config X86_SGX_KVM If unsure, say N. +config X86_TDX_KVM_EXPERIMENTAL + bool "EXPERIMENTAL Trust Domian Extensions (TDX) KVM support" + default n + depends on INTEL_TDX_HOST + depends on KVM_INTEL + help + Enable experimental TDX KVM support. TDX KVM needs many patches and + the patches will be merged step by step, not at once. Even if TDX KVM + support is incomplete, enable TDX KVM support so that developper can + exercise TDX KVM code. TODO: Remove this configuration once the + (first step of) TDX KVM support is complete. + + If unsure, say N. + config KVM_AMD tristate "KVM for AMD processors support" depends on KVM
arch/x86/kvm/vmx/tdx.c +4 -0
@@ -170,7 +170,11 @@ int tdx_module_setup(void)bool tdx_is_vm_type_supported (unsigned long type) { +#ifdef CONFIG_X86_TDX_KVM_EXPERIMENTAL return type ==KVM_X86_TDX_VM &&READ_ONCE (enable_tdx); +#else +return false; +#endif }static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops)
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH v5 010/104] KVM: TDX: Make TDX VM type supported
Quoted 12 lines · Isaku Yamahata
Quoted 9 lines · Kai Huang
Quoted 7 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is
vm_initcallback forKVM_CREATE_VM. Add a place holder function and call a function to initialize TDX module on demand because in that callback VMX is enabled byhardware_enablecallback (vmx_hardware_enable).
Quoted 7 lines
Quoted 4 lines
Should we put this patch at the end of series until all changes required to run TD are introduced? This patch essentially tells userspace KVM is ready to support a TD but actually it's not ready. And this might also cause bisect issue I suppose?
The intention is that developers can exercise the new code step-by-step even if the TDX KVM isn't complete.
What is the purpose/value to allow developers to exercise the new code step-by- step? Userspace cannot create TD successfully anyway until all patches are ready.
Signature · 2 lines
Thanks, -Kai
Paolo Bonzini
maintainerRe: [RFC PATCH v5 010/104] KVM: TDX: Make TDX VM type supported
Quoted 5 lines · Kai Huang
Quoted 2 lines
The intention is that developers can exercise the new code step-by-step even if the TDX KVM isn't complete.
What is the purpose/value to allow developers to exercise the new code step-by- step? Userspace cannot create TD successfully anyway until all patches are ready.
We can move this to the end when the patch is committed, but I think there is value in showing that the series works (for partial definitions of "work") at every step of the enablement process.
Paolo
RFC PATCH v5 11/104 The start of TDX KVM patch series: TDX architectural definitions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of TDX architectural definitions.
--- .../virt/kvm/intel-tdx-layer-status.rst | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 Documentation/virt/kvm/intel-tdx-layer-status.rst
Documentation/virt/kvm/intel-tdx-layer-status.rst new file +29 -0
@@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +Intel Trust Dodmain Extensions(TDX) +=================================== + +Layer status +============ +What qemu can do +---------------- +- TDX VM TYPE is exposed to Qemu. +- Qemu can try to create VM of TDX VM type and then fails. + +Patch Layer status +------------------ + Patch layer Status +* TDX, VMX coexistence: Applied +* TDX architectural definitions: Applying +* TD VM creation/destruction: Not yet +* TD vcpu creation/destruction: Not yet +* TDX EPT violation: Not yet +* TD finalization: Not yet +* TD vcpu enter/exit: Not yet +* TD vcpu interrupts/exit/hypercall: Not yet + +* KVM MMU GPA stolen bits: Not yet +* KVM TDP refactoring for TDX: Not yet +* KVM TDP MMU hooks: Not yet +* KVM TDP MMU MapGPA: Not yet
Signature · 1 line
2.25.1
RFC PATCH v5 12/104 KVM: TDX: Define TDX architectural definitions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Define architectural definitions for KVM to issue the TDX SEAMCALLs.
Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference.
--- arch/x86/kvm/vmx/tdx_arch.h | 158 ++++++++++++++++++++++++++++++++++++ 1 file changed, 158 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
arch/x86/kvm/vmx/tdx_arch.h new file +158 -0
@@ -0,0 +1,158 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40 +#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_NON_ARCH BIT_ULL(63) +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define __BUILD_TDX_FIELD (non_arch, class, field) \ + (((non_arch) ? TDX_NON_ARCH : 0) | \ + ((u64)(class) << TDX_CLASS_SHIFT) | \ + ((u64)(field) & TDX_FIELD_MASK)) + +#define BUILD_TDX_FIELD (class, field) \ + __BUILD_TDX_FIELD(false, (class), (field)) + +#define BUILD_TDX_FIELD_NON_ARCH (class, field) \ + __BUILD_TDX_FIELD(true, (class), (field)) + + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS (field) BUILD_TDX_FIELD(0, (field)) + +enum tdx_guest_other_state { +TD_VCPU_STATE_DETAILS_NON_ARCH =0x100 , +}; + +union tdx_vcpu_state_details { +struct { +u64 vmxip :1 ; +u64 reserved :63 ; + }; +u64 full; +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE (field) BUILD_TDX_FIELD(17, (field)) +#define TDVPS_STATE_NON_ARCH (field) BUILD_TDX_FIELD_NON_ARCH(17, (field)) + +/* Management class fields */ +enum tdx_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(32, (field)) + +enum tdx_tdcs_execution_control { +TD_TDCS_EXEC_TSC_OFFSET =10 , +}; + +/* @field is any of enum tdx_tdcs_execution_control */ +#define TDCS_EXEC (field) BUILD_TDX_FIELD(17, (field)) + +#define TDX_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_value { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +#define TDX_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +#define TDX_TD_XFAM_LBR BIT_ULL(15) +#define TDX_TD_XFAM_AMX (BIT_ULL(17) | BIT_ULL(18)) + +/* +* TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. +*/ +struct td_params { +u64 attributes; +u64 xfam; +u32 max_vcpus; +u32 reserved0; + +u64 eptp_controls; +u64 exec_controls; +u16 tsc_frequency; +u8 reserved1[38 ]; + +u64 mrconfigid[6 ]; +u64 mrowner[6 ]; +u64 mrownerconfig[6 ]; +u64 reserved2[4 ]; + +union { +struct tdx_cpuid_value cpuid_values[0 ]; +u8 reserved3[768 ]; + }; +} __packed__aligned(1024) ; + +/* +* Guest uses MAX_PA for GPAW when set. +* 0: GPA.SHARED bit is GPA[47] +* 1: GPA.SHARED bit is GPA[51] +*/ +#define TDX_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* +* TDX requires the frequency to be defined in units of 25MHz, which is the +* frequency of the core crystal clock on TDX-capable platforms, i.e. the TDX +* module can only program frequencies that are multiples of 25MHz. The +* frequency must be between 100mhz and 10ghz (inclusive). +*/ +#define TDX_TSC_KHZ_TO_25MHZ (tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX_TSC_25MHZ_TO_KHZ (tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +#endif /* __KVM_X86_TDX_ARCH_H */
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 012/104] KVM: TDX: Define TDX architectural definitions
Quoted 7 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Define architectural definitions for KVM to issue the TDX SEAMCALLs.
Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference.
Quoted 7 lines
Co-developed-by Sean Christopherson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/tdx_arch.h | 158 ++++++++++++++++++++++++++++++++++++ 1 file changed, 158 insertions(+)create mode 100644
arch/x86/kvm/vmx/tdx_arch.h
Quoted 164 lines
arch/x86/kvm/vmx/tdx_arch.h new file +158
-0@@ -0,0 +1,158 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40 +#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_NON_ARCH BIT_ULL(63) +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define __BUILD_TDX_FIELD (non_arch, class, field) \ + (((non_arch) ? TDX_NON_ARCH : 0) | \ + ((u64)(class) << TDX_CLASS_SHIFT) | \ + ((u64)(field) & TDX_FIELD_MASK)) + +#define BUILD_TDX_FIELD (class, field) \ + __BUILD_TDX_FIELD(false, (class), (field)) + +#define BUILD_TDX_FIELD_NON_ARCH (class, field) \ + __BUILD_TDX_FIELD(true, (class), (field)) + + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS (field) BUILD_TDX_FIELD(0, (field)) + +enum tdx_guest_other_state { +TD_VCPU_STATE_DETAILS_NON_ARCH =0x100 , +}; + +union tdx_vcpu_state_details { +struct { +u64 vmxip :1 ; +u64 reserved :63 ; + }; +u64 full; +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE (field) BUILD_TDX_FIELD(17, (field)) +#define TDVPS_STATE_NON_ARCH (field) BUILD_TDX_FIELD_NON_ARCH(17, (field)) + +/* Management class fields */ +enum tdx_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(32, (field)) + +enum tdx_tdcs_execution_control { +TD_TDCS_EXEC_TSC_OFFSET =10 , +}; + +/* @field is any of enum tdx_tdcs_execution_control */ +#define TDCS_EXEC (field) BUILD_TDX_FIELD(17, (field)) + +#define TDX_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_value { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +#define TDX_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +#define TDX_TD_XFAM_LBR BIT_ULL(15) +#define TDX_TD_XFAM_AMX (BIT_ULL(17) | BIT_ULL(18)) + +/* +* TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. +*/ +struct td_params { +u64 attributes; +u64 xfam; +u32 max_vcpus; +u32 reserved0; + +u64 eptp_controls; +u64 exec_controls; +u16 tsc_frequency; +u8 reserved1[38 ]; + +u64 mrconfigid[6 ]; +u64 mrowner[6 ]; +u64 mrownerconfig[6 ]; +u64 reserved2[4 ]; + +union { +struct tdx_cpuid_value cpuid_values[0 ]; +u8 reserved3[768 ]; + }; +} __packed__aligned(1024) ; + +/* +* Guest uses MAX_PA for GPAW when set. +* 0: GPA.SHARED bit is GPA[47] +* 1: GPA.SHARED bit is GPA[51] +*/ +#define TDX_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* +* TDX requires the frequency to be defined in units of 25MHz, which is the +* frequency of the core crystal clock on TDX-capable platforms, i.e. the TDX +* module can only program frequencies that are multiples of 25MHz. The +* frequency must be between 100mhz and 10ghz (inclusive). +*/ +#define TDX_TSC_KHZ_TO_25MHZ (tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX_TSC_25MHZ_TO_KHZ (tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +#endif /* __KVM_X86_TDX_ARCH_H */
RFC PATCH v5 13/104 KVM: TDX: Add TDX "architectural" error codes
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed.
TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.
--- arch/x86/kvm/vmx/tdx_errno.h | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
arch/x86/kvm/vmx/tdx_errno.h new file +29 -0
@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_SUCCESS 0x0000000000000000ULL +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_LIFECYCLE_STATE_INCORRECT 0xC000060700000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001ULL + +#endif /* __KVM_X86_TDX_ERRNO_H */
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 013/104] KVM: TDX: Add TDX "architectural" error codes
Quoted 8 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed.
Quoted 6 lines
TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
Quoted 40 lines
--- arch/x86/kvm/vmx/tdx_errno.h | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)create mode 100644
arch/x86/kvm/vmx/tdx_errno.harch/x86/kvm/vmx/tdx_errno.h new file +29
-0@@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_SUCCESS 0x0000000000000000ULL +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_LIFECYCLE_STATE_INCORRECT 0xC000060700000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001ULL + +#endif /* __KVM_X86_TDX_ERRNO_H */
RFC PATCH v5 14/104 KVM: TDX: Add a function for KVM to invoke SEAMCALL
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add an assembly function for KVM to call the TDX module because __seamcall defined in arch/x86/virt/vmx/seamcall.S doesn't fit for the KVM use case.
TDX module API returns extended error information in registers, rcx, rdx, r8, r9, r10, and r11 in addition to success case. KVM uses those extended error information in addition to the status code returned in RAX. Update the assembly code to optionally return those outputs even in the error case and define the specific version for KVM to call the TDX module.
SEAMCALL to the SEAM module (P-SEAMLDR or TDX module) can result in the error of VmFailInvalid indicated by CF=1 when VMX isn't enabled by VMXON instruction. Because KVM guarantees that VMX is enabled, VmFailInvalid error won't happen. Don't check the error for KVM.
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/seamcall.S | 55 +++++++++++++++++++++++++++++++++++++ arch/x86/virt/tdxcall.S | 8 ++++-- 3 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 arch/x86/kvm/vmx/seamcall.S
arch/x86/kvm/Makefile +1 -1
@@ -24,7 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/seamcall.S new file +55 -0
55 lines not on this page · raw message
arch/x86/virt/tdxcall.S +6 -2
@@ -13,7 +13,7 @@ #define tdcall .byte 0x66,0x0f,0x01,0xcc #define seamcall .byte 0x66,0x0f,0x01,0xcf -.macro TDX_MODULE_CALL host:req +.macro TDX_MODULE_CALL host:req error_check=1 /* * R12 will be used as temporary storage for struct tdx_module_output * pointer. Since R12-R15 registers are not used by TDCALL/SEAMCALL
@@ -51,9 +51,11 @@ * * Set %rax to TDX_SEAMCALL_VMFAILINVALID for VMfailInvalid. * This value will never be used as actual SEAMCALL error code. - */ + */ + .if \error_check jnc .Lno_vmfailinvalid mov $TDX_SEAMCALL_VMFAILINVALID, %rax + .endif .Lno_vmfailinvalid: .else tdcall
@@ -66,8 +68,10 @@ pop %r12 /* Check for success: 0 - Successful, otherwise failed */ + .if \error_check test %rax, %rax jnz .Lno_output_struct + .endif /* * Since this function can be initiated without an output pointer,
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 014/104] KVM: TDX: Add a function for KVM to invoke SEAMCALL
Quoted 103 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add an assembly function for KVM to call the TDX module because
__seamcalldefined inarch/x86/virt/vmx/seamcall.Sdoesn't fit for the KVM use case.TDX module API returns extended error information in registers, rcx, rdx, r8, r9, r10, and r11 in addition to success case. KVM uses those extended error information in addition to the status code returned in RAX. Update the assembly code to optionally return those outputs even in the error case and define the specific version for KVM to call the TDX module.
SEAMCALL to the SEAM module (P-SEAMLDR or TDX module) can result in the error of VmFailInvalid indicated by CF=1 when VMX isn't enabled by VMXON instruction. Because KVM guarantees that VMX is enabled, VmFailInvalid error won't happen. Don't check the error for KVM.
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/seamcall.S | 55 +++++++++++++++++++++++++++++++++++++ arch/x86/virt/tdxcall.S | 8 ++++-- 3 files changed, 62 insertions(+), 3 deletions(-)create mode 100644
arch/x86/kvm/vmx/seamcall.Sarch/x86/kvm/Makefile +1
-1@@ -24,7 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.oarch/x86/kvm/vmx/seamcall.S new file +55
-0@@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include <linux/linkage.h> +#include <asm/export.h> +#include <asm/frame.h> + +#include "../../virt/tdxcall.S" + +/* + * kvm_seamcall() - Host-side interface functions to SEAM software (TDX module) + * + * Transform function call register arguments into the SEAMCALL register + * ABI. Return the completion status of the SEAMCALL. Additional output + * operands are saved in @out (if it is provided by the user). + * It doesn't check TDX_SEAMCALL_VMFAILINVALID unlike __semcall() because KVM + * guarantees that VMX is enabled so that TDX_SEAMCALL_VMFAILINVALID doesn't + * happen. In the case of error completion status code, extended error code may + * be stored in leaf specific output registers. + * + *------------------------------------------------------------------------- + * SEAMCALL ABI: + *------------------------------------------------------------------------- + * Input Registers: + * + * RAX - SEAMCALL Leaf number. + * RCX,RDX,R8-R9 - SEAMCALL Leaf specific input registers. + * + * Output Registers: + * + * RAX - SEAMCALL completion status code. + * RCX,RDX,R8-R11 - SEAMCALL Leaf specific output registers. + * + *------------------------------------------------------------------------- + * + * kvm_seamcall() function ABI: + * + * @fn (RDI) - SEAMCALL Leaf number, moved to RAX + * @rcx (RSI) - Input parameter 1, moved to RCX + * @rdx (RDX) - Input parameter 2, moved to RDX + * @r8 (RCX) - Input parameter 3, moved to R8 + * @r9 (R8) - Input parameter 4, moved to R9 + * + * @out (R9) - struct tdx_module_output pointer + * stored temporarily in R12 (not + * shared with the TDX module). It + * can be NULL. + * + * Return (via RAX) the completion status of the SEAMCALL + */ +SYM_FUNC_START(kvm_seamcall) + FRAME_BEGIN + TDX_MODULE_CALL host=1 error_check=0 + FRAME_END + ret +SYM_FUNC_END(kvm_seamcall) +EXPORT_SYMBOL_GPL(kvm_seamcall)arch/x86/virt/tdxcall.S +0
-0
arch/x86/virt/tdxcall.S line 16
Quoted 5 lines
arch/x86/virt/tdxcall.S +1
-1@@ -13,7 +13,7 @@ #define tdcall .byte 0x66,0x0f,0x01,0xcc #define seamcall .byte 0x66,0x0f,0x01,0xcf -.macro TDX_MODULE_CALL host:req +.macro TDX_MODULE_CALL host:req error_check=1
Perhaps name this argument ext_error_out and reverse it (that is, 0 is the current behavior while 1 is what KVM needs).
Paolo
Quoted 27 lines
arch/x86/virt/tdxcall.S +5
-1/* * R12 will be used as temporary storage for struct tdx_module_output * pointer. Since R12-R15 registers are not used by TDCALL/SEAMCALL@@ -51,9 +51,11 @@ * * Set %rax to TDX_SEAMCALL_VMFAILINVALID for VMfailInvalid. * This value will never be used as actual SEAMCALL error code. - */ + */ + .if \error_check jnc .Lno_vmfailinvalid mov $TDX_SEAMCALL_VMFAILINVALID, %rax + .endif .Lno_vmfailinvalid: .else tdcall@@ -66,8 +68,10 @@ pop %r12 /* Check for success: 0 - Successful, otherwise failed */ + .if \error_check test %rax, %rax jnz .Lno_output_struct + .endif /* * Since this function can be initiated without an output pointer,
Re: [RFC PATCH v5 014/104] KVM: TDX: Add a function for KVM to invoke SEAMCALL
In reply to [email protected]Quoted 10 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add an assembly function for KVM to call the TDX module because
__seamcalldefined inarch/x86/virt/vmx/seamcall.Sdoesn't fit for the KVM use case.TDX module API returns extended error information in registers, rcx, rdx, r8, r9, r10, and r11 in addition to success case. KVM uses those extended error information in addition to the status code returned in RAX. Update the assembly code to optionally return those outputs even in the error case and define the specific version for KVM to call the TDX module.
+Kirill.
Kirill's patches hasn't been merged yet. Can Kirill just change his patch so you don't have to change the assembly code again? Btw this change is reasonable for host kernel support series as well. Sorry that I failed to notice.
Btw, SEAMCALL C function is already implemented in host kernel support series:
I think maybe you can just export __seamcall() and use it?
Quoted 5 lines
SEAMCALL to the SEAM module (P-SEAMLDR or TDX module) can result in the error of VmFailInvalid indicated by CF=1 when VMX isn't enabled by VMXON instruction. Because KVM guarantees that VMX is enabled, VmFailInvalid error won't happen. Don't check the error for KVM.
Perhaps you can introduce kvm_seamcall() which calls __seamcall(), but WARN() if it returns VMfailInvalid?
Quoted 111 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/seamcall.S | 55 +++++++++++++++++++++++++++++++++++++ arch/x86/virt/tdxcall.S | 8 ++++-- 3 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 arch/x86/kvm/vmx/seamcall.Sarch/x86/kvm/Makefile +1
-1@@ -24,7 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.oarch/x86/kvm/vmx/seamcall.S new file +55
-0@@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include <linux/linkage.h> +#include <asm/export.h> +#include <asm/frame.h> + +#include "../../virt/tdxcall.S" + +/* + * kvm_seamcall() - Host-side interface functions to SEAM software (TDX module) + * + * Transform function call register arguments into the SEAMCALL register + * ABI. Return the completion status of the SEAMCALL. Additional output + * operands are saved in @out (if it is provided by the user). + * It doesn't check TDX_SEAMCALL_VMFAILINVALID unlike __semcall() because KVM + * guarantees that VMX is enabled so that TDX_SEAMCALL_VMFAILINVALID doesn't + * happen. In the case of error completion status code, extended error code may + * be stored in leaf specific output registers. + * + *------------------------------------------------------------------------- + * SEAMCALL ABI: + *------------------------------------------------------------------------- + * Input Registers: + * + * RAX - SEAMCALL Leaf number. + * RCX,RDX,R8-R9 - SEAMCALL Leaf specific input registers. + * + * Output Registers: + * + * RAX - SEAMCALL completion status code. + * RCX,RDX,R8-R11 - SEAMCALL Leaf specific output registers. + * + *------------------------------------------------------------------------- + * + * kvm_seamcall() function ABI: + * + * @fn (RDI) - SEAMCALL Leaf number, moved to RAX + * @rcx (RSI) - Input parameter 1, moved to RCX + * @rdx (RDX) - Input parameter 2, moved to RDX + * @r8 (RCX) - Input parameter 3, moved to R8 + * @r9 (R8) - Input parameter 4, moved to R9 + * + * @out (R9) - struct tdx_module_output pointer + * stored temporarily in R12 (not + * shared with the TDX module). It + * can be NULL. + * + * Return (via RAX) the completion status of the SEAMCALL + */ +SYM_FUNC_START(kvm_seamcall) + FRAME_BEGIN + TDX_MODULE_CALL host=1 error_check=0 + FRAME_END + ret +SYM_FUNC_END(kvm_seamcall) +EXPORT_SYMBOL_GPL(kvm_seamcall)arch/x86/virt/tdxcall.S +4
-2@@ -13,7 +13,7 @@ #define tdcall .byte 0x66,0x0f,0x01,0xcc #define seamcall .byte 0x66,0x0f,0x01,0xcf -.macro TDX_MODULE_CALL host:req +.macro TDX_MODULE_CALL host:req error_check=1 /* * R12 will be used as temporary storage for struct tdx_module_output * pointer. Since R12-R15 registers are not used by TDCALL/SEAMCALL@@ -51,9 +51,11 @@ * * Set %rax to TDX_SEAMCALL_VMFAILINVALID for VMfailInvalid. * This value will never be used as actual SEAMCALL error code. - */ + */ + .if \error_check jnc .Lno_vmfailinvalid mov $TDX_SEAMCALL_VMFAILINVALID, %rax + .endif .Lno_vmfailinvalid: .else tdcall
arch/x86/virt/tdxcall.S line 75
Quoted 8 lines
arch/x86/virt/tdxcall.S +2
-0@@ -66,8 +68,10 @@ pop %r12 /* Check for success: 0 - Successful, otherwise failed */ + .if \error_check test %rax, %rax jnz .Lno_output_struct + .endif
Checking VMfailInvalid, and checking whether %rax is 0 are totally different things. I don't think it's good to use one single 'error_check' to handle.
As I mentioned above, I think checking whether %rax is 0 can be just removed in Kirill's patch. But I don't have strong opinion on whether you should add another parameter to determine whether to check VMfailInvalid, or you should just call __seamcall() which is implemented in host support series.
RFC PATCH v5 15/104 KVM: TDX: add a helper function for KVM to issue SEAMCALL
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TODO: Consolidate seamcall helper function with TDX host/guest patch series. For now, this is kept to make this patch series compile/work.
A VMM interacts with the TDX module using a new instruction (SEAMCALL). A TDX VMM uses SEAMCALLs where a VMX VMM would have directly interacted with VMX instructions. For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs.
Add a helper function for KVM C code to execute SEAMCALL instruction to hide its SEAMCALL ABI details. Although the x86 TDX host patch series defines a similar wrapper, the KVM TDX patch series defines its own because KVM TDX case is performance-critical, unlike the x86 TDX one that does one-time initialization. The difference is that the KVM TDX one is defined as a static inline function without an error check that is known to not happen so that compiler can optimize it better. The wrapper fiction in the x86 TDX host patch is defined as a function written in assembly code with error check so that it can detect errors that can occur only during the initialization.
--- arch/x86/kvm/vmx/seamcall.h | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 arch/x86/kvm/vmx/seamcall.h
arch/x86/kvm/vmx/seamcall.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_VMX_SEAMCALL_H +#define __KVM_VMX_SEAMCALL_H + +#ifdef CONFIG_INTEL_TDX_HOST + +#ifdef __ASSEMBLY__ + +.macro seamcall + .byte 0x66, 0x0f, 0x01, 0xcf +.endm + +#else + +struct tdx_module_output ; +u64 kvm_seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10, +struct tdx_module_output *out); + +#endif /* !__ASSEMBLY__ */ + +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_VMX_SEAMCALL_H */
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 015/104] KVM: TDX: add a helper function for KVM to issue SEAMCALL
Quoted 11 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TODO: Consolidate seamcall helper function with TDX host/guest patch series. For now, this is kept to make this patch series compile/work.
A VMM interacts with the TDX module using a new instruction (SEAMCALL). A TDX VMM uses SEAMCALLs where a VMX VMM would have directly interacted with VMX instructions. For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs.
Quoted 10 lines
Add a helper function for KVM C code to execute SEAMCALL instruction to hide its SEAMCALL ABI details. Although the x86 TDX host patch series defines a similar wrapper, the KVM TDX patch series defines its own because KVM TDX case is performance-critical, unlike the x86 TDX one that does one-time initialization. The difference is that the KVM TDX one is defined as a static inline function without an error check that is known to not happen so that compiler can optimize it better. The wrapper fiction in the x86 TDX host patch is defined as a function written in assembly code with error check so that it can detect errors that can occur only during the initialization.
I assume whatever survives of this patch will be merged in the previous one.
Paolo
Quoted 37 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/seamcall.h | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)create mode 100644
arch/x86/kvm/vmx/seamcall.harch/x86/kvm/vmx/seamcall.h new file +23
-0@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_VMX_SEAMCALL_H +#define __KVM_VMX_SEAMCALL_H + +#ifdef CONFIG_INTEL_TDX_HOST + +#ifdef __ASSEMBLY__ + +.macro seamcall + .byte 0x66, 0x0f, 0x01, 0xcf +.endm + +#else + +struct tdx_module_output ; +u64 kvm_seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10, +struct tdx_module_output *out); + +#endif /* !__ASSEMBLY__ */ + +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_VMX_SEAMCALL_H */
RFC PATCH v5 16/104 KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX SEAMCALL interface is defined in the TDX module specification. Define C wrapper functions for SEAMCALLs for readability.
--- arch/x86/kvm/vmx/tdx_ops.h | 161 +++++++++++++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
arch/x86/kvm/vmx/tdx_ops.h new file +161 -0
@@ -0,0 +1,161 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_OPS_H +#define __KVM_X86_TDX_OPS_H + +#include <linux/compiler.h> + +#include <asm/asm.h> +#include <asm/kvm_host.h> + +#include "tdx_errno.h" +#include "tdx_arch.h" +#include "seamcall.h" + +#ifdef CONFIG_INTEL_TDX_HOST + +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +return kvm_seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_add (hpa_t tdr,gpa_t gpa,hpa_t hpa,hpa_t source, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MEM_PAGE_ADD , gpa, tdr, hpa, source,0 , out); +} + +static inline u64 tdh_mem_sept_add (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MEM_SEPT_ADD , gpa | level, tdr, page,0 ,0 , out); +} + +static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) +{ +return kvm_seamcall (TDH_VP_ADDCX , addr, tdvpr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_page_aug (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MEM_PAGE_AUG , gpa, tdr, hpa,0 ,0 , out); +} + +static inline u64 tdh_mem_range_block (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MEM_RANGE_BLOCK , gpa | level, tdr,0 ,0 ,0 , out); +} + +static inline u64 tdh_mng_key_config (hpa_t tdr) +{ +return kvm_seamcall (TDH_MNG_KEY_CONFIG , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_create (hpa_t tdr,int hkid) +{ +return kvm_seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) +{ +return kvm_seamcall (TDH_VP_CREATE , tdvpr, tdr,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_rd (hpa_t tdr,u64 field,struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MNG_RD , tdr, field,0 ,0 ,0 , out); +} + +static inline u64 tdh_mr_extend (hpa_t tdr,gpa_t gpa,struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MR_EXTEND , gpa, tdr,0 ,0 ,0 , out); +} + +static inline u64 tdh_mr_finalize (hpa_t tdr) +{ +return kvm_seamcall (TDH_MR_FINALIZE , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_flush (hpa_t tdvpr) +{ +return kvm_seamcall (TDH_VP_FLUSH , tdvpr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_vpflushdone (hpa_t tdr) +{ +return kvm_seamcall (TDH_MNG_VPFLUSHDONE , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_key_freeid (hpa_t tdr) +{ +return kvm_seamcall (TDH_MNG_KEY_FREEID , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mng_init (hpa_t tdr,hpa_t td_params,struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MNG_INIT , tdr, td_params,0 ,0 ,0 , out); +} + +static inline u64 tdh_vp_init (hpa_t tdvpr,u64 rcx) +{ +return kvm_seamcall (TDH_VP_INIT , tdvpr, rcx,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_rd (hpa_t tdvpr,u64 field,struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_VP_RD , tdvpr, field,0 ,0 ,0 , out); +} + +static inline u64 tdh_mng_key_reclaimid (hpa_t tdr) +{ +return kvm_seamcall (TDH_MNG_KEY_RECLAIMID , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_phymem_page_reclaim (hpa_t page,struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_PHYMEM_PAGE_RECLAIM , page,0 ,0 ,0 ,0 , out); +} + +static inline u64 tdh_mem_page_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MEM_PAGE_REMOVE , gpa | level, tdr,0 ,0 ,0 , out); +} + +static inline u64 tdh_sys_lp_shutdown (void ) +{ +return kvm_seamcall (TDH_SYS_LP_SHUTDOWN ,0 ,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_track (hpa_t tdr) +{ +return kvm_seamcall (TDH_MEM_TRACK , tdr,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_mem_range_unblock (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_MEM_RANGE_UNBLOCK , gpa | level, tdr,0 ,0 ,0 , out); +} + +static inline u64 tdh_phymem_cache_wb (bool resume) +{ +return kvm_seamcall (TDH_PHYMEM_CACHE_WB , resume ?1 :0 ,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_phymem_page_wbinvd (hpa_t page) +{ +return kvm_seamcall (TDH_PHYMEM_PAGE_WBINVD , page,0 ,0 ,0 ,0 ,NULL ); +} + +static inline u64 tdh_vp_wr (hpa_t tdvpr,u64 field,u64 val,u64 mask, +struct tdx_module_output *out) +{ +return kvm_seamcall (TDH_VP_WR , tdvpr, field, val, mask,0 , out); +} +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_X86_TDX_OPS_H */
Signature · 1 line
2.25.1
RFC PATCH v5 17/104 KVM: TDX: Add helper functions to print TDX SEAMCALL error
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add helper functions to print out errors from the TDX module in a uniform manner.
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/seamcall.h | 2 ++ arch/x86/kvm/vmx/tdx_error.c | 22 ++++++++++++++++++++++ 3 files changed, 25 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx_error.c
arch/x86/kvm/Makefile +1 -1
@@ -24,7 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/seamcall.h +2 -0
@@ -16,6 +16,8 @@ struct tdx_module_output;u64 kvm_seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10,struct tdx_module_output *out); +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_output *out); +#endif /* !__ASSEMBLY__ */ #endif /* CONFIG_INTEL_TDX_HOST */
arch/x86/kvm/vmx/tdx_error.c new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 +/* functions to record TDX SEAMCALL error */ + +#include <linux/kernel.h> +#include <linux/bug.h> + +#include "tdx_ops.h" + +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_output *out) +{ +if (!out) { +pr_err_ratelimited ("SEAMCALL[%lld] failed: 0x%llx\n" , + op, error_code); +return ; + } + +pr_err_ratelimited ( +"SEAMCALL[%lld] failed: 0x%llx " +"RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n" , + op, error_code, + out->rcx, out->rdx, out->r8, out->r9, out->r10, out->r11); +}
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 017/104] KVM: TDX: Add helper functions to print TDX SEAMCALL error
Quoted 57 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add helper functions to print out errors from the TDX module in a uniform manner.
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/seamcall.h | 2 ++ arch/x86/kvm/vmx/tdx_error.c | 22 ++++++++++++++++++++++ 3 files changed, 25 insertions(+), 1 deletion(-)create mode 100644
arch/x86/kvm/vmx/tdx_error.carch/x86/kvm/Makefile +1
-1@@ -24,7 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.oarch/x86/kvm/vmx/seamcall.h +2
-0@@ -16,6 +16,8 @@ struct tdx_module_output;u64 kvm_seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10,struct tdx_module_output *out); +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_output *out); +#endif /* !__ASSEMBLY__ */ #endif /* CONFIG_INTEL_TDX_HOST */ arch/x86/kvm/vmx/tdx_error.c new file +12
-0@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 +/* functions to record TDX SEAMCALL error */ + +#include <linux/kernel.h> +#include <linux/bug.h> + +#include "tdx_ops.h" + +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_output *out) +{ +if (!out) { + pr_err_ratelimited("SEAMCALL[%lld] failed: 0x%llx\n",
arch/x86/kvm/vmx/tdx_error.c line 22
Quoted 10 lines
arch/x86/kvm/vmx/tdx_error.c new file +10
-0+ op, error_code); +return ; + } + +pr_err_ratelimited ( +"SEAMCALL[%lld] failed: 0x%llx " +"RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n" , + op, error_code, + out->rcx, out->rdx, out->r8, out->r9, out->r10, out->r11); +}
Paolo Bonzini
maintainerRe: [RFC PATCH v5 017/104] KVM: TDX: Add helper functions to print TDX SEAMCALL error
In reply to [email protected]Quoted 5 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add helper functions to print out errors from the TDX module in a uniform manner.
Quoted 7 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/seamcall.h | 2 ++ arch/x86/kvm/vmx/tdx_error.c | 22 ++++++++++++++++++++++ 3 files changed, 25 insertions(+), 1 deletion(-)create mode 100644
arch/x86/kvm/vmx/tdx_error.c
When rebasing against tip/x86/tdx, the new .c file needs to include asm/tdx.h.
Paolo
Quoted 54 lines
arch/x86/kvm/Makefile +1
-1@@ -24,7 +24,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/evmcs.o vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/seamcall.o vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.oarch/x86/kvm/vmx/seamcall.h +2
-0@@ -16,6 +16,8 @@ struct tdx_module_output;u64 kvm_seamcall (u64 op,u64 rcx,u64 rdx,u64 r8,u64 r9,u64 r10,struct tdx_module_output *out); +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_output *out); +#endif /* !__ASSEMBLY__ */ #endif /* CONFIG_INTEL_TDX_HOST */ arch/x86/kvm/vmx/tdx_error.c new file +22
-0@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 +/* functions to record TDX SEAMCALL error */ + +#include <linux/kernel.h> +#include <linux/bug.h> + +#include "tdx_ops.h" + +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_output *out) +{ +if (!out) { +pr_err_ratelimited ("SEAMCALL[%lld] failed: 0x%llx\n" , + op, error_code); +return ; + } + +pr_err_ratelimited ( +"SEAMCALL[%lld] failed: 0x%llx " +"RCX 0x%llx, RDX 0x%llx, R8 0x%llx, R9 0x%llx, R10 0x%llx, R11 0x%llx\n" , + op, error_code, + out->rcx, out->rdx, out->r8, out->r9, out->r10, out->r11); +}
RFC PATCH v5 18/104 The start of TDX KVM patch series: TD VM creation/destruction
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of TD VM creation/destruction.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2
@@ -15,8 +15,8 @@ Patch Layer status ------------------ Patch layer Status * TDX, VMX coexistence: Applied -* TDX architectural definitions: Applying -* TD VM creation/destruction: Not yet +* TDX architectural definitions: Applied +* TD VM creation/destruction: Applying * TD vcpu creation/destruction: Not yet * TDX EPT violation: Not yet * TD finalization: Not yet
Signature · 1 line
2.25.1
RFC PATCH v5 19/104 KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
In reply to [email protected]From: Sean Christopherson <[email protected]>
Stub in kvm_tdx, vcpu_tdx, and their various accessors. TDX defines SEAMCALL APIs to access TDX control structures corresponding to the VMX VMCS. Introduce helper accessors to hide its SEAMCALL ABI details.
--- arch/x86/kvm/vmx/tdx.h | 101 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
arch/x86/kvm/vmx/tdx.h +101 -0
@@ -3,14 +3,29 @@#define __KVM_X86_TDX_H#ifdef CONFIG_INTEL_TDX_HOST + +#include "tdx_ops.h" +int tdx_module_setup (void ); +struct tdx_td_page { +unsigned long va; +hpa_t pa; +bool added; +}; +struct kvm_tdx {struct kvm kvm; + +struct tdx_td_page tdr; +struct tdx_td_page *tdcs; };struct vcpu_tdx {struct kvm_vcpu vcpu; + +struct tdx_td_page tdvpr; +struct tdx_td_page *tdvpx; };static inline bool is_td (struct kvm *kvm)
@@ -32,6 +47,92 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_tdx, vcpu); } + +static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) +{ +BUILD_BUG_ON_MSG (__builtin_constant_p (field) && (field) &0x1 , +"Read/Write to TD VMCS *_HIGH fields not supported" ); + +BUILD_BUG_ON (bits !=16 && bits !=32 && bits !=64 ); + +BUILD_BUG_ON_MSG (bits !=64 &&__builtin_constant_p (field) && + (((field) &0x6000 ) ==0x2000 || + ((field) &0x6000 ) ==0x6000 ), +"Invalid TD VMCS access for 64-bit field" ); +BUILD_BUG_ON_MSG (bits !=32 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x4000 , +"Invalid TD VMCS access for 32-bit field" ); +BUILD_BUG_ON_MSG (bits !=16 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x0000 , +"Invalid TD VMCS access for 16-bit field" ); +} + +static __always_inline void tdvps_state_non_arch_check(u64 field, u8 bits) {} +static __always_inline void tdvps_management_check(u64 field, u8 bits) {} + +#define TDX_BUILD_TDVPS_ACCESSORS (bits, uclass, lclass) \ +static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ + u32 field) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_rd(tdx->tdvpr.pa, TDVPS_##uclass(field), &out); \ + if (unlikely(err)) { \ + pr_err("TDH_VP_RD["#uclass".0x%x] failed: 0x%llx\n", \ + field, err); \ + return 0; \ + } \ + return (u##bits)out.r8; \ +} \ +static __always_inline void td_##lclass##_write##bits(struct vcpu_tdx *tdx, \ + u32 field, u##bits val) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), val, \ + GENMASK_ULL(bits - 1, 0), &out); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] = 0x%llx failed: 0x%llx\n", \ + field, (u64)val, err); \ +} \ +static __always_inline void td_##lclass##_setbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), bit, bit, \ + &out); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] |= 0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} \ +static __always_inline void td_##lclass##_clearbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), 0, bit, \ + &out); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] &= ~0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} + +TDX_BUILD_TDVPS_ACCESSORS (16 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (32 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (64 ,VMCS , vmcs); + +TDX_BUILD_TDVPS_ACCESSORS (64 ,STATE_NON_ARCH , state_non_arch); +TDX_BUILD_TDVPS_ACCESSORS (8 ,MANAGEMENT , management); +#else static inline int tdx_module_setup (void ) {return -ENODEV ; };
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 019/104] KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers
Quoted 11 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Stub in
kvm_tdx,vcpu_tdx, and their various accessors. TDX defines SEAMCALL APIs to access TDX control structures corresponding to the VMX VMCS. Introduce helper accessors to hide its SEAMCALL ABI details.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/tdx.h | 101 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
When rebasing against tip/x86/tdx, the new .h file needs to include asm/tdx.h.
Paolo
Quoted 127 lines
arch/x86/kvm/vmx/tdx.h +101
-0@@ -3,14 +3,29 @@#define __KVM_X86_TDX_H#ifdef CONFIG_INTEL_TDX_HOST + +#include "tdx_ops.h" +int tdx_module_setup (void ); +struct tdx_td_page { +unsigned long va; +hpa_t pa; +bool added; +}; +struct kvm_tdx {struct kvm kvm; + +struct tdx_td_page tdr; +struct tdx_td_page *tdcs; };struct vcpu_tdx {struct kvm_vcpu vcpu; + +struct tdx_td_page tdvpr; +struct tdx_td_page *tdvpx; };static inline bool is_td (struct kvm *kvm)@@ -32,6 +47,92 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_tdx, vcpu); } + +static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) +{ +BUILD_BUG_ON_MSG (__builtin_constant_p (field) && (field) &0x1 , +"Read/Write to TD VMCS *_HIGH fields not supported" ); + +BUILD_BUG_ON (bits !=16 && bits !=32 && bits !=64 ); + +BUILD_BUG_ON_MSG (bits !=64 &&__builtin_constant_p (field) && + (((field) &0x6000 ) ==0x2000 || + ((field) &0x6000 ) ==0x6000 ), +"Invalid TD VMCS access for 64-bit field" ); +BUILD_BUG_ON_MSG (bits !=32 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x4000 , +"Invalid TD VMCS access for 32-bit field" ); +BUILD_BUG_ON_MSG (bits !=16 &&__builtin_constant_p (field) && + ((field) &0x6000 ) ==0x0000 , +"Invalid TD VMCS access for 16-bit field" ); +} + +static __always_inline void tdvps_state_non_arch_check(u64 field, u8 bits) {} +static __always_inline void tdvps_management_check(u64 field, u8 bits) {} + +#define TDX_BUILD_TDVPS_ACCESSORS (bits, uclass, lclass) \ +static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ + u32 field) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_rd(tdx->tdvpr.pa, TDVPS_##uclass(field), &out); \ + if (unlikely(err)) { \ + pr_err("TDH_VP_RD["#uclass".0x%x] failed: 0x%llx\n", \ + field, err); \ + return 0; \ + } \ + return (u##bits)out.r8; \ +} \ +static __always_inline void td_##lclass##_write##bits(struct vcpu_tdx *tdx, \ + u32 field, u##bits val) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), val, \ + GENMASK_ULL(bits - 1, 0), &out); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] = 0x%llx failed: 0x%llx\n", \ + field, (u64)val, err); \ +} \ +static __always_inline void td_##lclass##_setbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), bit, bit, \ + &out); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] |= 0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} \ +static __always_inline void td_##lclass##_clearbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_module_output out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr.pa, TDVPS_##uclass(field), 0, bit, \ + &out); \ + if (unlikely(err)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] &= ~0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} + +TDX_BUILD_TDVPS_ACCESSORS (16 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (32 ,VMCS , vmcs); +TDX_BUILD_TDVPS_ACCESSORS (64 ,VMCS , vmcs); + +TDX_BUILD_TDVPS_ACCESSORS (64 ,STATE_NON_ARCH , state_non_arch); +TDX_BUILD_TDVPS_ACCESSORS (8 ,MANAGEMENT , management); +#else static inline int tdx_module_setup (void ) {return -ENODEV ; };
RFC PATCH v5 20/104 KVM: TDX: allocate per-package mutex
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Several TDX SEAMCALLs are per-package scope (concretely per memory controller) and they need to be serialized per-package. Allocate mutex for it.
--- arch/x86/kvm/vmx/main.c | 8 +++++++- arch/x86/kvm/vmx/tdx.c | 18 ++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ 3 files changed, 27 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/main.c +7 -1
@@ -25,6 +25,12 @@ static __init int vt_hardware_setup(void)return 0 ; } +static void vt_hardware_unsetup (void ) +{ +tdx_hardware_unsetup (); +vmx_hardware_unsetup (); +} +static int vt_vm_init (struct kvm *kvm) {int ret;
@@ -42,7 +48,7 @@ static int vt_vm_init(struct kvm *kvm)struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" , - .hardware_unsetup = vmx_hardware_unsetup, + .hardware_unsetup = vt_hardware_unsetup, .hardware_enable = vmx_hardware_enable, .hardware_disable = vmx_hardware_disable,
arch/x86/kvm/vmx/tdx.c +18 -0
@@ -34,6 +34,8 @@ struct tdx_capabilities {/* Capabilities of KVM + the TDX module. */ struct tdx_capabilities tdx_caps; +static struct mutex *tdx_mng_key_config_lock; +static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ;
@@ -112,7 +114,9 @@ bool tdx_is_vm_type_supported(unsigned long type)static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) { +int max_pkgs;u32 max_pa; +int i;if (!enable_ept) {pr_warn ("Cannot enable TDX with EPT disabled\n" );
@@ -127,6 +131,14 @@ static int __init __tdx_hardware_setup(struct kvm_x86_ops *x86_ops)if (WARN_ON_ONCE (x86_ops->tlb_remote_flush))return -EIO ; + max_pkgs =topology_max_packages (); + tdx_mng_key_config_lock =kcalloc (max_pkgs,sizeof (*tdx_mng_key_config_lock), +GFP_KERNEL ); +if (!tdx_mng_key_config_lock) +return -ENOMEM ; +for (i =0 ; i < max_pkgs; i++) +mutex_init (&tdx_mng_key_config_lock[i]); + max_pa =cpuid_eax (0x80000008 ) &0xff ; hkid_start_pos = boot_cpu_data.x86_phys_bits; hkid_mask =GENMASK_ULL (max_pa -1 , hkid_start_pos);
@@ -147,6 +159,12 @@ void __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) enable_tdx = false; } +void tdx_hardware_unsetup (void ) +{ +/* kfree accepts NULL. */ +kfree (tdx_mng_key_config_lock); +} +void __init tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {
arch/x86/kvm/vmx/x86_ops.h +2 -0
@@ -131,11 +131,13 @@ void __init tdx_pre_kvm_init(unsigned int *vcpu_size, unsigned int *vcpu_align, unsigned int *vm_size);bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +void tdx_hardware_unsetup (void );#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {} +static inline void tdx_hardware_unsetup (void ) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 020/104] KVM: TDX: allocate per-package mutex
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Several TDX SEAMCALLs are per-package scope (concretely per memory controller) and they need to be serialized per-package. Allocate mutex for it.
Quoted 6 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 8 +++++++- arch/x86/kvm/vmx/tdx.c | 18 ++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ 3 files changed, 27 insertions(+), 1 deletion(-)
Please define here the lock/unlock functions as well:
static inline int tdx_mng_key_lock(void)
{
int cpu = get_cpu();
cur_pkg = topology_physical_package_id(cpu);
mutex_lock(&tdx_mng_key_config_lock[cur_pkg]);
return cur_pkg;
}static inline void tdx_mng_key_unlock(int cur_pkg)
{
mutex_unlock(&tdx_mng_key_config_lock[cur_pkg]);
put_cpu();
}Thanks,
Paolo
Quoted 96 lines
arch/x86/kvm/vmx/main.c +7
-1@@ -25,6 +25,12 @@ static __init int vt_hardware_setup(void)return 0 ; } +static void vt_hardware_unsetup (void ) +{ +tdx_hardware_unsetup (); +vmx_hardware_unsetup (); +} +static int vt_vm_init (struct kvm *kvm) {int ret;@@ -42,7 +48,7 @@ static int vt_vm_init(struct kvm *kvm)struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" , - .hardware_unsetup = vmx_hardware_unsetup, + .hardware_unsetup = vt_hardware_unsetup, .hardware_enable = vmx_hardware_enable, .hardware_disable = vmx_hardware_disable,arch/x86/kvm/vmx/tdx.c +18
-0@@ -34,6 +34,8 @@ struct tdx_capabilities {/* Capabilities of KVM + the TDX module. */ struct tdx_capabilities tdx_caps; +static struct mutex *tdx_mng_key_config_lock; +static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ;@@ -112,7 +114,9 @@ bool tdx_is_vm_type_supported(unsigned long type)static int __init __tdx_hardware_setup (struct kvm_x86_ops *x86_ops) { +int max_pkgs;u32 max_pa; +int i;if (!enable_ept) {pr_warn ("Cannot enable TDX with EPT disabled\n" );@@ -127,6 +131,14 @@ static int __init __tdx_hardware_setup(struct kvm_x86_ops *x86_ops)if (WARN_ON_ONCE (x86_ops->tlb_remote_flush))return -EIO ; + max_pkgs =topology_max_packages (); + tdx_mng_key_config_lock =kcalloc (max_pkgs,sizeof (*tdx_mng_key_config_lock), +GFP_KERNEL ); +if (!tdx_mng_key_config_lock) +return -ENOMEM ; +for (i =0 ; i < max_pkgs; i++) +mutex_init (&tdx_mng_key_config_lock[i]); + max_pa =cpuid_eax (0x80000008 ) &0xff ; hkid_start_pos = boot_cpu_data.x86_phys_bits; hkid_mask =GENMASK_ULL (max_pa -1 , hkid_start_pos);@@ -147,6 +159,12 @@ void __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) enable_tdx = false; } +void tdx_hardware_unsetup (void ) +{ +/* kfree accepts NULL. */ +kfree (tdx_mng_key_config_lock); +} +void __init tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {arch/x86/kvm/vmx/x86_ops.h +2
-0@@ -131,11 +131,13 @@ void __init tdx_pre_kvm_init(unsigned int *vcpu_size, unsigned int *vcpu_align, unsigned int *vm_size);bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +void tdx_hardware_unsetup (void );#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {} +static inline void tdx_hardware_unsetup (void ) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
Re: [RFC PATCH v5 020/104] KVM: TDX: allocate per-package mutex
Quoted 25 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Several TDX SEAMCALLs are per-package scope (concretely per memory controller) and they need to be serialized per-package. Allocate mutex for it.
Quoted 6 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 8 +++++++- arch/x86/kvm/vmx/tdx.c | 18 ++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ 3 files changed, 27 insertions(+), 1 deletion(-)Please define here the lock/unlock functions as well:
static inline int
tdx_mng_key_lock(void){ int cpu = get_cpu(); cur_pkg = topology_physical_package_id(cpu); mutex_lock(&tdx_mng_key_config_lock[cur_pkg]); return cur_pkg; }
Quoted 5 lines
static inline void
tdx_mng_key_unlock(int cur_pkg){ mutex_unlock(&tdx_mng_key_config_lock[cur_pkg]); put_cpu(); }
Sure, will do in the next respoin.
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH v5 21/104 KVM: x86: Introduce hooks to free VM callback prezap and vm_free
In reply to [email protected]From: Kai Huang <[email protected]>
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. keyID must have been reclaimed, etc). Add prezap callback before tearing down private page tables for it.
TDX needs to free some resources after other resources (i.e. vcpu related resources). Add vm_free callback at the end of kvm_arch_destroy_vm().
--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 8 ++++++++ 3 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +2 -0
@@ -20,7 +20,9 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP_NULL(mmu_prezap) KVM_X86_OP_NULL(vm_destroy) +KVM_X86_OP_NULL(vm_free) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free) KVM_X86_OP(vcpu_reset)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1326,7 +1326,9 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*mmu_prezap)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm); +void (*vm_free)(struct kvm *kvm);/* Create, but do not attach this VCPU */ int (*vcpu_create)(struct kvm_vcpu *vcpu);
arch/x86/kvm/x86.c +8 -0
@@ -11779,6 +11779,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_free)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot)
@@ -12036,6 +12037,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,void kvm_arch_flush_shadow_all (struct kvm *kvm) { +/* +* kvm_mmu_zap_all() zaps both private and shared page tables. Before +* tearing down private page tables, TDX requires some TD resources to +* be destroyed (i.e. keyID must have been reclaimed, etc). Invoke +* kvm_x86_mmu_prezap() for this. +*/ +static_call_cond (kvm_x86_mmu_prezap)(kvm);kvm_mmu_zap_all (kvm); }
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 021/104] KVM: x86: Introduce hooks to free VM callback prezap and vm_free
Quoted 61 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Kai Huang <[email protected]>
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. keyID must have been reclaimed, etc). Add prezap callback before tearing down private page tables for it.
TDX needs to free some resources after other resources (i.e. vcpu related resources). Add
vm_freecallback at the end ofkvm_arch_destroy_vm().Signed-off-by Kai Huang <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 8 ++++++++ 3 files changed, 12 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +2
-0@@ -20,7 +20,9 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP_NULL(mmu_prezap) KVM_X86_OP_NULL(vm_destroy) +KVM_X86_OP_NULL(vm_free) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free) KVM_X86_OP(vcpu_reset)arch/x86/include/asm/kvm_host.h +2
-0@@ -1326,7 +1326,9 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*mmu_prezap)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm); +void (*vm_free)(struct kvm *kvm);/* Create, but do not attach this VCPU */ int (*vcpu_create)(struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +1
-0@@ -11779,6 +11779,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_free)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot)@@ -12036,6 +12037,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,void kvm_arch_flush_shadow_all (struct kvm *kvm) {
arch/x86/kvm/x86.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 12049
Quoted 10 lines
arch/x86/kvm/x86.c +7
-0+/* +* kvm_mmu_zap_all() zaps both private and shared page tables. Before +* tearing down private page tables, TDX requires some TD resources to +* be destroyed (i.e. keyID must have been reclaimed, etc). Invoke +* kvm_x86_mmu_prezap() for this. +*/ +static_call_cond (kvm_x86_mmu_prezap)(kvm);kvm_mmu_zap_all (kvm); }
The two callbacks are introduced here but they are actually implemented in 2 patches later (patch 24 KVM: TDX: create/destroy VM structure). Why not just squash this patch to patch 24? Or at least you can put this patch right before patch 24.
Please feel free to remove my SoB and From if this bothers you to squash.
Re: [RFC PATCH v5 021/104] KVM: x86: Introduce hooks to free VM callback prezap and vm_free
Quoted 69 lines · Kai Huang
Quoted 68 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Kai Huang <[email protected]>
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. keyID must have been reclaimed, etc). Add prezap callback before tearing down private page tables for it.
TDX needs to free some resources after other resources (i.e. vcpu related resources). Add
vm_freecallback at the end ofkvm_arch_destroy_vm().Signed-off-by Kai Huang <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 8 ++++++++ 3 files changed, 12 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +2
-0@@ -20,7 +20,9 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP_NULL(mmu_prezap) KVM_X86_OP_NULL(vm_destroy) +KVM_X86_OP_NULL(vm_free) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free) KVM_X86_OP(vcpu_reset)arch/x86/include/asm/kvm_host.h +2
-0@@ -1326,7 +1326,9 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*mmu_prezap)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm); +void (*vm_free)(struct kvm *kvm);/* Create, but do not attach this VCPU */ int (*vcpu_create)(struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +8
-0@@ -11779,6 +11779,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_free)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot)@@ -12036,6 +12037,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,void kvm_arch_flush_shadow_all (struct kvm *kvm) { +/* +* kvm_mmu_zap_all() zaps both private and shared page tables. Before +* tearing down private page tables, TDX requires some TD resources to +* be destroyed (i.e. keyID must have been reclaimed, etc). Invoke +* kvm_x86_mmu_prezap() for this. +*/ +static_call_cond (kvm_x86_mmu_prezap)(kvm);
Quoted 8 lines
arch/x86/kvm/x86.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 12049
Quoted 3 lines
arch/x86/kvm/x86.c +0
-0kvm_mmu_zap_all (kvm); }The two callbacks are introduced here but they are actually implemented in 2 patches later (patch 24 KVM: TDX: create/destroy VM structure). Why not just squash this patch to patch 24? Or at least you can put this patch right before patch 24.
Ok. I'll squash this patch into it.
Signature · 1 line
Isaku Yamahata <[email protected]>
Paolo Bonzini
maintainerRe: [RFC PATCH v5 021/104] KVM: x86: Introduce hooks to free VM callback prezap and vm_free
In reply to [email protected]Quoted 59 lines · On 3/4/22 20:48, [email protected] wrote:
From: Kai Huang <[email protected]>
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. keyID must have been reclaimed, etc). Add prezap callback before tearing down private page tables for it.
TDX needs to free some resources after other resources (i.e. vcpu related resources). Add
vm_freecallback at the end ofkvm_arch_destroy_vm().Signed-off-by Kai Huang <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 8 ++++++++ 3 files changed, 12 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +2
-0@@ -20,7 +20,9 @@ KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) +KVM_X86_OP_NULL(mmu_prezap) KVM_X86_OP_NULL(vm_destroy) +KVM_X86_OP_NULL(vm_free) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free) KVM_X86_OP(vcpu_reset)arch/x86/include/asm/kvm_host.h +2
-0@@ -1326,7 +1326,9 @@ struct kvm_x86_ops {bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm); +void (*mmu_prezap)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm); +void (*vm_free)(struct kvm *kvm);/* Create, but do not attach this VCPU */ int (*vcpu_create)(struct kvm_vcpu *vcpu);arch/x86/kvm/x86.c +1
-0@@ -11779,6 +11779,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_free)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot)@@ -12036,6 +12037,13 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
arch/x86/kvm/x86.c void kvm_arch_commit_memory_region(struct kvm *kvm, · line 12047
Quoted 10 lines
arch/x86/kvm/x86.c +7
-0void kvm_arch_flush_shadow_all (struct kvm *kvm) { +/* +* kvm_mmu_zap_all() zaps both private and shared page tables. Before +* tearing down private page tables, TDX requires some TD resources to +* be destroyed (i.e. keyID must have been reclaimed, etc). Invoke +* kvm_x86_mmu_prezap() for this. +*/ +static_call_cond (kvm_x86_mmu_prezap)(kvm);kvm_mmu_zap_all (kvm);
Please rename the hook to (*flush_shadow_all_private).
Otherwise ok,
Paolo
Quoted 2 lines
arch/x86/kvm/x86.c +0
-0}
RFC PATCH v5 22/104 KVM: Add max_vcpus field in common 'struct kvm'
In reply to [email protected]From: Sean Christopherson <[email protected]>
For TDX guests, the maximum number of vcpus needs to be specified when the TDX guest VM is initialized (creating the TDX data corresponding to TDX guest) before creating vcpu. It needs to record the maximum number of vcpus on VM creation (KVM_CREATE_VM) and return error if the number of vcpus exceeds it
Because there is already max_vcpu member in arm64 struct kvm_arch, move it to common struct kvm and initialize it to KVM_MAX_VCPUS before kvm_arch_init_vm() instead of adding it to x86 struct kvm_arch.
--- arch/arm64/include/asm/kvm_host.h | 3 --- arch/arm64/kvm/arm.c | 6 +++--- arch/arm64/kvm/vgic/vgic-init.c | 6 +++--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 3 ++- 5 files changed, 9 insertions(+), 10 deletions(-)
arch/arm64/include/asm/kvm_host.h +0 -3
@@ -107,9 +107,6 @@ struct kvm_arch {/* VTCR_EL2 value for this VM */ u64 vtcr; -/* The maximum number of vCPUs depends on the used GIC model */ -int max_vcpus; -/* Interrupt controller */ struct vgic_dist vgic;
arch/arm64/kvm/arm.c +3 -3
@@ -153,7 +153,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)kvm_vgic_early_init (kvm);/* The maximum number of VCPUs is limited by the host's GIC model */ - kvm->arch.max_vcpus =kvm_arm_default_max_vcpus (); + kvm->max_vcpus =kvm_arm_default_max_vcpus ();set_default_spectre (kvm);
@@ -229,7 +229,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_MAX_VCPUS :case KVM_CAP_MAX_VCPU_ID :if (kvm) - r = kvm->arch.max_vcpus; + r = kvm->max_vcpus;else r =kvm_arm_default_max_vcpus ();break ;
@@ -305,7 +305,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)if (irqchip_in_kernel (kvm) &&vgic_initialized (kvm))return -EBUSY ; -if (id >= kvm->arch.max_vcpus) +if (id >= kvm->max_vcpus)return -EINVAL ;return 0 ;
arch/arm64/kvm/vgic/vgic-init.c +3 -3
@@ -98,11 +98,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) ret =0 ;if (type ==KVM_DEV_TYPE_ARM_VGIC_V2 ) - kvm->arch.max_vcpus =VGIC_V2_MAX_CPUS ; + kvm->max_vcpus =VGIC_V2_MAX_CPUS ;else - kvm->arch.max_vcpus =VGIC_V3_MAX_CPUS ; + kvm->max_vcpus =VGIC_V3_MAX_CPUS ; -if (atomic_read (&kvm->online_vcpus) > kvm->arch.max_vcpus) { +if (atomic_read (&kvm->online_vcpus) > kvm->max_vcpus) { ret = -E2BIG ; goto out_unlock; }
include/linux/kvm_host.h +1 -0
@@ -715,6 +715,7 @@ struct kvm {* and is accessed atomically. */ atomic_t online_vcpus; +int max_vcpus;int created_vcpus;int last_boosted_vcpu;struct list_head vm_list;
virt/kvm/kvm_main.c +2 -1
@@ -1075,6 +1075,7 @@ static struct kvm *kvm_create_vm(unsigned long type)spin_lock_init (&kvm->gpc_lock);INIT_LIST_HEAD (&kvm->devices); + kvm->max_vcpus =KVM_MAX_VCPUS ;BUILD_BUG_ON (KVM_MEM_SLOTS_NUM >SHRT_MAX );
@@ -3718,7 +3719,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)return -EINVAL ;mutex_lock (&kvm->lock); -if (kvm->created_vcpus ==KVM_MAX_VCPUS ) { +if (kvm->created_vcpus >= kvm->max_vcpus) {mutex_unlock (&kvm->lock);return -EINVAL ; }
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 022/104] KVM: Add max_vcpus field in common 'struct kvm'
Quoted 111 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
For TDX guests, the maximum number of vcpus needs to be specified when the TDX guest VM is initialized (creating the TDX data corresponding to TDX guest) before creating vcpu. It needs to record the maximum number of vcpus on VM creation (
KVM_CREATE_VM) and return error if the number of vcpus exceeds itBecause there is already
max_vcpumember in arm64 structkvm_arch, move it to common struct kvm and initialize it toKVM_MAX_VCPUSbeforekvm_arch_init_vm()instead of adding it to x86 structkvm_arch.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/arm64/include/asm/kvm_host.h | 3 --- arch/arm64/kvm/arm.c | 6 +++--- arch/arm64/kvm/vgic/vgic-init.c | 6 +++--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 3 ++- 5 files changed, 9 insertions(+), 10 deletions(-)arch/arm64/include/asm/kvm_host.h +0
-3@@ -107,9 +107,6 @@ struct kvm_arch {/* VTCR_EL2 value for this VM */ u64 vtcr; -/* The maximum number of vCPUs depends on the used GIC model */ -int max_vcpus; -/* Interrupt controller */ struct vgic_dist vgic;arch/arm64/kvm/arm.c +3
-3@@ -153,7 +153,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)kvm_vgic_early_init (kvm);/* The maximum number of VCPUs is limited by the host's GIC model */ - kvm->arch.max_vcpus =kvm_arm_default_max_vcpus (); + kvm->max_vcpus =kvm_arm_default_max_vcpus ();set_default_spectre (kvm);@@ -229,7 +229,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_MAX_VCPUS :case KVM_CAP_MAX_VCPU_ID :if (kvm) - r = kvm->arch.max_vcpus; + r = kvm->max_vcpus;else r =kvm_arm_default_max_vcpus ();break ;@@ -305,7 +305,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)if (irqchip_in_kernel (kvm) &&vgic_initialized (kvm))return -EBUSY ; -if (id >= kvm->arch.max_vcpus) +if (id >= kvm->max_vcpus)return -EINVAL ;return 0 ;arch/arm64/kvm/vgic/vgic-init.c +3
-3@@ -98,11 +98,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) ret =0 ;if (type ==KVM_DEV_TYPE_ARM_VGIC_V2 ) - kvm->arch.max_vcpus =VGIC_V2_MAX_CPUS ; + kvm->max_vcpus =VGIC_V2_MAX_CPUS ;else - kvm->arch.max_vcpus =VGIC_V3_MAX_CPUS ; + kvm->max_vcpus =VGIC_V3_MAX_CPUS ; -if (atomic_read (&kvm->online_vcpus) > kvm->arch.max_vcpus) { +if (atomic_read (&kvm->online_vcpus) > kvm->max_vcpus) { ret = -E2BIG ; goto out_unlock; }include/linux/kvm_host.h +1
-0@@ -715,6 +715,7 @@ struct kvm {* and is accessed atomically. */ atomic_t online_vcpus; +int max_vcpus;int created_vcpus;int last_boosted_vcpu;struct list_head vm_list;virt/kvm/kvm_main.c +1
-0@@ -1075,6 +1075,7 @@ static struct kvm *kvm_create_vm(unsigned long type)spin_lock_init (&kvm->gpc_lock);INIT_LIST_HEAD (&kvm->devices); + kvm->max_vcpus =KVM_MAX_VCPUS ;BUILD_BUG_ON (KVM_MEM_SLOTS_NUM >SHRT_MAX );
virt/kvm/kvm_main.c static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id) · line 3725
Quoted 8 lines
virt/kvm/kvm_main.c +1
-1@@ -3718,7 +3719,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, u32 id)return -EINVAL ;mutex_lock (&kvm->lock); -if (kvm->created_vcpus ==KVM_MAX_VCPUS ) { +if (kvm->created_vcpus >= kvm->max_vcpus) {mutex_unlock (&kvm->lock);return -EINVAL ; }
Queued this one already, thanks.
Paolo
RFC PATCH v5 23/104 x86/cpu: Add helper functions to allocate/free MKTME keyid
In reply to [email protected]From: Isaku Yamahata <[email protected]>
MKTME keyid is assigned to guest TD. The memory controller encrypts guest TD memory with key id. Add helper functions to allocate/free MKTME keyid so that TDX KVM assign keyid.
Also export MKTME global keyid that is used to encrypt TDX module and its memory.
--- arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/virt/vmx/tdx.c | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-)
arch/x86/include/asm/tdx.h +6 -0
@@ -139,6 +139,9 @@ int tdx_detect(void);int tdx_init (void );bool platform_has_tdx (void );const struct tdsysinfo_struct *tdx_get_sysinfo (void ); +u32 tdx_get_global_keyid (void ); +int tdx_keyid_alloc (void ); +void tdx_keyid_free (int keyid);#else static inline void tdx_detect_cpu (struct cpuinfo_x86 *c) { }static inline int tdx_detect (void ) {return -ENODEV ; }
@@ -146,6 +149,9 @@ static inline int tdx_init(void) { return -ENODEV; }static inline bool platform_has_tdx (void ) {return false; }struct tdsysinfo_struct ;static inline const struct tdsysinfo_struct *tdx_get_sysinfo (void ) {return NULL ; } +static inline u32 tdx_get_global_keyid (void ) {return 0 ; }; +static inline int tdx_keyid_alloc (void ) {return -EOPNOTSUPP ; } +static inline void tdx_keyid_free (int keyid) { }#endif /* CONFIG_INTEL_TDX_HOST */ #endif /* !__ASSEMBLY__ */
arch/x86/virt/vmx/tdx.c +32 -1
@@ -113,7 +113,13 @@ static int tdx_cmr_num;static struct tdsysinfo_struct tdx_sysinfo;/* TDX global KeyID to protect TDX metadata */ -static u32 tdx_global_keyid; +static u32 __read_mostly tdx_global_keyid; + +u32 tdx_get_global_keyid (void ) +{ +return tdx_global_keyid; +} +EXPORT_SYMBOL_GPL (tdx_get_global_keyid);static bool enable_tdx_host;
@@ -189,6 +195,31 @@ static void detect_seam(struct cpuinfo_x86 *c)detect_seam_ap (c); } +/* TDX KeyID pool */ +static DEFINE_IDA (tdx_keyid_pool); + +int tdx_keyid_alloc (void ) +{ +if (WARN_ON_ONCE (!tdx_keyid_start || !tdx_keyid_num)) +return -EINVAL ; + +/* The first keyID is reserved for the global key. */ +return ida_alloc_range (&tdx_keyid_pool, tdx_keyid_start +1 , + tdx_keyid_start + tdx_keyid_num -1 , +GFP_KERNEL ); +} +EXPORT_SYMBOL_GPL (tdx_keyid_alloc); + +void tdx_keyid_free (int keyid) +{ +/* keyid = 0 is reserved. */ +if (!keyid || keyid <=0 ) +return ; + +ida_free (&tdx_keyid_pool, keyid); +} +EXPORT_SYMBOL_GPL (tdx_keyid_free); +static void detect_tdx_keyids_bsp (struct cpuinfo_x86 *c) {u64 keyid_part;
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 023/104] x86/cpu: Add helper functions to allocate/free MKTME keyid
Quoted 5 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
MKTME keyid is assigned to guest TD. The memory controller encrypts guest TD memory with key id. Add helper functions to allocate/free MKTME keyid so that TDX KVM assign keyid.
Using MKTME keyid is wrong, at least not accurate I think. We should use explicitly use "TDX private KeyID", which is clearly documented in the spec:
Also, description of IA32_MKTME_KEYID_PARTITIONING MSR clearly says TDX private KeyIDs span the range (NUM_MKTME_KIDS+1) through (NUM_MKTME_KIDS+NUM_TDX_PRIV_KIDS). So please just use TDX private KeyID here.
Quoted 3 lines
Also export MKTME global keyid that is used to encrypt TDX module and its memory.
This needs explanation why the global keyID needs to be exported.
Quoted 82 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/virt/vmx/tdx.c | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-)arch/x86/include/asm/tdx.h +6
-0@@ -139,6 +139,9 @@ int tdx_detect(void);int tdx_init (void );bool platform_has_tdx (void );const struct tdsysinfo_struct *tdx_get_sysinfo (void ); +u32 tdx_get_global_keyid (void ); +int tdx_keyid_alloc (void ); +void tdx_keyid_free (int keyid);#else static inline void tdx_detect_cpu (struct cpuinfo_x86 *c) { }static inline int tdx_detect (void ) {return -ENODEV ; }@@ -146,6 +149,9 @@ static inline int tdx_init(void) { return -ENODEV; }static inline bool platform_has_tdx (void ) {return false; }struct tdsysinfo_struct ;static inline const struct tdsysinfo_struct *tdx_get_sysinfo (void ) {return NULL ; } +static inline u32 tdx_get_global_keyid (void ) {return 0 ; }; +static inline int tdx_keyid_alloc (void ) {return -EOPNOTSUPP ; } +static inline void tdx_keyid_free (int keyid) { }#endif /* CONFIG_INTEL_TDX_HOST */ #endif /* !__ASSEMBLY__ */ arch/x86/virt/vmx/tdx.c +32
-1@@ -113,7 +113,13 @@ static int tdx_cmr_num;static struct tdsysinfo_struct tdx_sysinfo;/* TDX global KeyID to protect TDX metadata */ -static u32 tdx_global_keyid; +static u32 __read_mostly tdx_global_keyid; + +u32 tdx_get_global_keyid (void ) +{ +return tdx_global_keyid; +} +EXPORT_SYMBOL_GPL (tdx_get_global_keyid);static bool enable_tdx_host;@@ -189,6 +195,31 @@ static void detect_seam(struct cpuinfo_x86 *c)detect_seam_ap (c); } +/* TDX KeyID pool */ +static DEFINE_IDA (tdx_keyid_pool); + +int tdx_keyid_alloc (void ) +{ +if (WARN_ON_ONCE (!tdx_keyid_start || !tdx_keyid_num)) +return -EINVAL ; + +/* The first keyID is reserved for the global key. */ +return ida_alloc_range (&tdx_keyid_pool, tdx_keyid_start +1 , + tdx_keyid_start + tdx_keyid_num -1 , +GFP_KERNEL ); +} +EXPORT_SYMBOL_GPL (tdx_keyid_alloc); + +void tdx_keyid_free (int keyid) +{ +/* keyid = 0 is reserved. */ +if (!keyid || keyid <=0 ) +return ; + +ida_free (&tdx_keyid_pool, keyid); +} +EXPORT_SYMBOL_GPL (tdx_keyid_free); +static void detect_tdx_keyids_bsp (struct cpuinfo_x86 *c) {u64 keyid_part;
Re: [RFC PATCH v5 023/104] x86/cpu: Add helper functions to allocate/free MKTME keyid
Quoted 18 lines · Kai Huang
Quoted 5 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
MKTME keyid is assigned to guest TD. The memory controller encrypts guest TD memory with key id. Add helper functions to allocate/free MKTME keyid so that TDX KVM assign keyid.
Using MKTME keyid is wrong, at least not accurate I think. We should use explicitly use "TDX private KeyID", which is clearly documented in the spec:
Also, description of
IA32_MKTME_KEYID_PARTITIONINGMSR clearly says TDX private KeyIDs span the range (NUM_MKTME_KIDS+1) through (NUM_MKTME_KIDS+NUM_TDX_PRIV_KIDS). So please just use TDX private KeyID here.Quoted 1 line
Quoted 4 lines
Quoted 2 lines
Also export MKTME global keyid that is used to encrypt TDX module and its memory.
This needs explanation why the global keyID needs to be exported.
How about the followings?
TDX private host key id is assigned to guest TD. The memory controller encrypts guest TD memory with the assigned host key id (HIKD). Add helper functions to allocate/free TDX private host key id so that TDX KVM manage it.
Also export the global TDX private host key id that is used to encrypt TDX module, its memory and some dynamic data (e.g. TDR). When VMM releasing encrypted page to reuse it, the page needs to be flushed with the used host key id. VMM needs the global TDX private host key id to flush such pages TDX module accesses with the global TDX private host key id.
Thanks,
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH v5 023/104] x86/cpu: Add helper functions to allocate/free MKTME keyid
Quoted 24 lines · Isaku Yamahata
Quoted 21 lines · Kai Huang
Quoted 5 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
MKTME keyid is assigned to guest TD. The memory controller encrypts guest TD memory with key id. Add helper functions to allocate/free MKTME keyid so that TDX KVM assign keyid.
Using MKTME keyid is wrong, at least not accurate I think. We should use explicitly use "TDX private KeyID", which is clearly documented in the spec:
Also, description of
IA32_MKTME_KEYID_PARTITIONINGMSR clearly says TDX private KeyIDs span the range (NUM_MKTME_KIDS+1) through (NUM_MKTME_KIDS+NUM_TDX_PRIV_KIDS). So please just use TDX private KeyID here.Quoted 3 lines
Also export MKTME global keyid that is used to encrypt TDX module and its memory.
Quoted 8 lines
Quoted 1 line
This needs explanation why the global keyID needs to be exported.
How about the followings?
TDX private host key id is assigned to guest TD. The memory controller encrypts guest TD memory with the assigned host key id (HIKD). Add helper functions to allocate/free TDX private host key id so that TDX KVM manage it.
HIKD -> HKID.
You may also want to use KeyID in consistent way (KeyID, keyid, key id, etc). The spec uses KeyID.
Quoted 8 lines
Also export the global TDX private host key id that is used to encrypt TDX module, its memory and some dynamic data (e.g. TDR). When VMM releasing encrypted page to reuse it, the page needs to be flushed with the used host key id. VMM needs the global TDX private host key id to flush such pages TDX module accesses with the global TDX private host key id.
Find to me.
Re: [RFC PATCH v5 023/104] x86/cpu: Add helper functions to allocate/free MKTME keyid
Quoted 5 lines
Quoted 5 lines
Also export the global TDX private host key id that is used to encrypt TDX module, its memory and some dynamic data (e.g. TDR).
Sorry I was replying too quick.
This sentence is not correct. Hardware doesn't use global KeyID to encrypt TDX module itself. In current generation of TDX, global KeyID is used to encrypt TDX memory metadata (PAMTs) and TDRs.
Quoted 9 lines
Quoted 6 lines
When VMM releasing
encrypted page to reuse it, the page needs to be flushed with the used host key id. VMM needs the global TDX private host key id to flush such pages TDX module accesses with the global TDX private host key id.Find to me.
Re: [RFC PATCH v5 023/104] x86/cpu: Add helper functions to allocate/free MKTME keyid
In reply to [email protected]Quoted 80 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
MKTME keyid is assigned to guest TD. The memory controller encrypts guest TD memory with key id. Add helper functions to allocate/free MKTME keyid so that TDX KVM assign keyid.
Also export MKTME global keyid that is used to encrypt TDX module and its memory.
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/tdx.h | 6 ++++++ arch/x86/virt/vmx/tdx.c | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-)arch/x86/include/asm/tdx.h +6
-0@@ -139,6 +139,9 @@ int tdx_detect(void);int tdx_init (void );bool platform_has_tdx (void );const struct tdsysinfo_struct *tdx_get_sysinfo (void ); +u32 tdx_get_global_keyid (void ); +int tdx_keyid_alloc (void ); +void tdx_keyid_free (int keyid);#else static inline void tdx_detect_cpu (struct cpuinfo_x86 *c) { }static inline int tdx_detect (void ) {return -ENODEV ; }@@ -146,6 +149,9 @@ static inline int tdx_init(void) { return -ENODEV; }static inline bool platform_has_tdx (void ) {return false; }struct tdsysinfo_struct ;static inline const struct tdsysinfo_struct *tdx_get_sysinfo (void ) {return NULL ; } +static inline u32 tdx_get_global_keyid (void ) {return 0 ; }; +static inline int tdx_keyid_alloc (void ) {return -EOPNOTSUPP ; } +static inline void tdx_keyid_free (int keyid) { }#endif /* CONFIG_INTEL_TDX_HOST */ #endif /* !__ASSEMBLY__ */ arch/x86/virt/vmx/tdx.c +25
-1@@ -113,7 +113,13 @@ static int tdx_cmr_num;static struct tdsysinfo_struct tdx_sysinfo;/* TDX global KeyID to protect TDX metadata */ -static u32 tdx_global_keyid; +static u32 __read_mostly tdx_global_keyid; + +u32 tdx_get_global_keyid (void ) +{ +return tdx_global_keyid; +} +EXPORT_SYMBOL_GPL (tdx_get_global_keyid);static bool enable_tdx_host;@@ -189,6 +195,31 @@ static void detect_seam(struct cpuinfo_x86 *c)detect_seam_ap (c); } +/* TDX KeyID pool */ +static DEFINE_IDA (tdx_keyid_pool); + +int tdx_keyid_alloc (void ) +{ +if (WARN_ON_ONCE (!tdx_keyid_start || !tdx_keyid_num)) +return -EINVAL ; + +/* The first keyID is reserved for the global key. */ +return ida_alloc_range (&tdx_keyid_pool, tdx_keyid_start +1 , + tdx_keyid_start + tdx_keyid_num -1 , +GFP_KERNEL ); +} +EXPORT_SYMBOL_GPL (tdx_keyid_alloc); + +void tdx_keyid_free (int keyid) +{ +/* keyid = 0 is reserved. */
arch/x86/virt/vmx/tdx.c static void detect_seam(struct cpuinfo_x86 *c) · line 225
Quoted 10 lines
arch/x86/virt/vmx/tdx.c +7
-0+if (!keyid || keyid <=0 ) +return ; + +ida_free (&tdx_keyid_pool, keyid); +} +EXPORT_SYMBOL_GPL (tdx_keyid_free); +static void detect_tdx_keyids_bsp (struct cpuinfo_x86 *c) {u64 keyid_part;
RFC PATCH v5 24/104 KVM: TDX: create/destroy VM structure
In reply to [email protected]From: Sean Christopherson <[email protected]>
As the first step to create TDX guest, create/destroy VM struct. Assign Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.
Add a second kvm_x86_ops hook in kvm_arch_vm_destroy() to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resources, and finally free per-VM resources.
--- arch/x86/kvm/vmx/main.c | 16 +- arch/x86/kvm/vmx/tdx.c | 312 +++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 2 + arch/x86/kvm/vmx/tdx_errno.h | 2 +- arch/x86/kvm/vmx/tdx_ops.h | 8 + arch/x86/kvm/vmx/x86_ops.h | 8 + 6 files changed, 346 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/main.c +15 -1
@@ -39,12 +39,24 @@ static int vt_vm_init(struct kvm *kvm) ret =tdx_module_setup ();if (ret)return ret; -return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ +return tdx_vm_init (kvm); }return vmx_vm_init (kvm); } +static void vt_mmu_prezap (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_mmu_prezap (kvm); +} + +static void vt_vm_free (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_vm_free (kvm); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,
@@ -58,6 +70,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vt_vm_init, + .mmu_prezap = vt_mmu_prezap, + .vm_free = vt_vm_free, .vcpu_create = vmx_vcpu_create, .vcpu_free = vmx_vcpu_free,
arch/x86/kvm/vmx/tdx.c +312 -0
332 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +2 -0
@@ -19,6 +19,8 @@ struct kvm_tdx {struct tdx_td_page tdr;struct tdx_td_page *tdcs; + +int hkid; };struct vcpu_tdx {
arch/x86/kvm/vmx/tdx_errno.h +1 -1
@@ -12,11 +12,11 @@#define TDX_SUCCESS 0x0000000000000000ULL#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL -#define TDX_LIFECYCLE_STATE_INCORRECT 0xC000060700000000ULL#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL/*
arch/x86/kvm/vmx/tdx_ops.h +8 -0
@@ -6,6 +6,7 @@#include <linux/compiler.h> +#include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h>
@@ -15,8 +16,14 @@#ifdef CONFIG_INTEL_TDX_HOST +static inline void tdx_clflush_page (hpa_t addr) +{ +clflush_cache_range (__va (addr),PAGE_SIZE ); +} +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) { +tdx_clflush_page (addr);return kvm_seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); }
@@ -56,6 +63,7 @@ static inline u64 tdh_mng_key_config(hpa_t tdr)static inline u64 tdh_mng_create (hpa_t tdr,int hkid) { +tdx_clflush_page (tdr);return kvm_seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); }
arch/x86/kvm/vmx/x86_ops.h +8 -0
@@ -132,12 +132,20 @@ void __init tdx_pre_kvm_init(unsigned int *vcpu_size,bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void ); + +int tdx_vm_init (struct kvm *kvm); +void tdx_mmu_prezap (struct kvm *kvm); +void tdx_vm_free (struct kvm *kvm);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {}static inline void tdx_hardware_unsetup (void ) {} + +static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; } +static inline void tdx_mmu_prezap (struct kvm *kvm) {} +static inline void tdx_vm_free (struct kvm *kvm) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 024/104] KVM: TDX: create/destroy VM structure
Quoted 66 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Sean Christopherson <[email protected]>
As the first step to create TDX guest, create/destroy VM struct. Assign Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.
Add a second
kvm_x86_opshook inkvm_arch_vm_destroy()to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resources, and finally free per-VM resources.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 16 +- arch/x86/kvm/vmx/tdx.c | 312 +++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 2 + arch/x86/kvm/vmx/tdx_errno.h | 2 +- arch/x86/kvm/vmx/tdx_ops.h | 8 + arch/x86/kvm/vmx/x86_ops.h | 8 + 6 files changed, 346 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/main.c +15
-1@@ -39,12 +39,24 @@ static int vt_vm_init(struct kvm *kvm) ret =tdx_module_setup ();if (ret)return ret; -return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ +return tdx_vm_init (kvm); }return vmx_vm_init (kvm); } +static void vt_mmu_prezap (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_mmu_prezap (kvm); +} + +static void vt_vm_free (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_vm_free (kvm); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,@@ -58,6 +70,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vt_vm_init, + .mmu_prezap = vt_mmu_prezap, + .vm_free = vt_vm_free, .vcpu_create = vmx_vcpu_create, .vcpu_free = vmx_vcpu_free,arch/x86/kvm/vmx/tdx.c +0
-0
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 36
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +3
-0@@ -31,14 +31,324 @@ struct tdx_capabilities {struct tdx_cpuid_config cpuid_configs[TDX_MAX_NR_CPUID_CONFIGS ]; }; +/* KeyID used by TDX module */ +static u32 tdx_global_keyid__read_mostly ; +
It's really not clear why you need to know tdx_global_keyid in the context of creating/destroying a TD.
Quoted 29 lines
arch/x86/kvm/vmx/tdx.c +21
-0/* Capabilities of KVM + the TDX module. */ struct tdx_capabilities tdx_caps; +static DEFINE_MUTEX (tdx_lock);static struct mutex *tdx_mng_key_config_lock;static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ; +static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) +{ + pa &= ~hkid_mask; + pa |= (u64 )hkid << hkid_start_pos; + +return pa; +} + +static inline bool is_td_created (struct kvm_tdx *kvm_tdx) +{ +return kvm_tdx->tdr.added; +} + +static inline void tdx_hkid_free (struct kvm_tdx *kvm_tdx) +{ +tdx_keyid_free (kvm_tdx->hkid); + kvm_tdx->hkid =-1 ; +} + +static inline bool is_hkid_assigned (struct kvm_tdx *kvm_tdx)
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 75
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+{ +return kvm_tdx->hkid >0 ; +} + +static void tdx_clear_page (unsigned long page) +{ +const void *zero_page = (const void *)__va (page_to_phys (ZERO_PAGE (0 ))); +unsigned long i; + +/* Zeroing the page is only necessary for systems with MKTME-i. */
"only necessary for systems with MKTME-i" because of what?
Please be more clear that on MKTME-i system, when re-assign one page from old keyid to a new keyid, MOVDIR64B is required to clear/write the page with new keyid to prevent integrity error when read on the page with new keyid.
However, the new keyid is essentially 0, but in practice integrity check for keyid 0 is disabled in current generation of MKTME-i, so I guess we are also safe even we don't use MOVDIR64B to clear page for TDX here. But I agree it's better to do.
Quoted 11 lines
arch/x86/kvm/vmx/tdx.c +11
-0+if (!static_cpu_has (X86_FEATURE_MOVDIR64B )) +return ; + +for (i =0 ; i <4096 ; i +=64 ) +/* MOVDIR64B [rdx], es:rdi */ + asm (".byte 0x66, 0x0f, 0x38, 0xf8, 0x3a" + : :"d" (zero_page),"D" (page + i) :"memory" ); +} + +static int __tdx_reclaim_page (unsigned long va,hpa_t pa,bool do_wb,u16 hkid) +{
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 96
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+struct tdx_module_output out; +u64 err; + + err =tdh_phymem_page_reclaim (pa, &out); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_PAGE_RECLAIM , err, &out); +return -EIO ; + } + +if (do_wb) {
In the callers, please add some comments explaining why do_wb is needed, and why is not needed.
Quoted 76 lines
arch/x86/kvm/vmx/tdx.c +76
-0+ err =tdh_phymem_page_wbinvd (set_hkid_to_hpa (pa, hkid)); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_PAGE_WBINVD , err,NULL ); +return -EIO ; + } + } + +tdx_clear_page (va); +return 0 ; +} + +static int tdx_reclaim_page (unsigned long va,hpa_t pa) +{ +return __tdx_reclaim_page (va, pa, false,0 ); +} + +static int tdx_alloc_td_page (struct tdx_td_page *page) +{ + page->va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!page->va) +return -ENOMEM ; + + page->pa =__pa (page->va); +return 0 ; +} + +static void tdx_mark_td_page_added (struct tdx_td_page *page) +{ +WARN_ON_ONCE (page->added); + page->added = true; +} + +static void tdx_reclaim_td_page (struct tdx_td_page *page) +{ +if (page->added) { +if (tdx_reclaim_page (page->va, page->pa)) +return ; + + page->added = false; + } +free_page (page->va); +} + +static int tdx_do_tdh_phymem_cache_wb (void *param) +{ +u64 err =0 ; + +/* +* We can destroy multiple the guest TDs simultaneously. Prevent +* tdh_phymem_cache_wb from returning TDX_BUSY by serialization. +*/ +mutex_lock (&tdx_lock); +do { + err =tdh_phymem_cache_wb (!!err); + }while (err ==TDX_INTERRUPTED_RESUMABLE ); +mutex_unlock (&tdx_lock); + +/* Other thread may have done for us. */ +if (err ==TDX_NO_HKID_READY_TO_WBCACHE ) + err =TDX_SUCCESS ; +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_CACHE_WB , err,NULL ); +return -EIO ; + } + +return 0 ; +} + +void tdx_mmu_prezap (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +bool cpumask_allocated; +u64 err; +int ret; +int i;
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 182
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ +if (!is_hkid_assigned (kvm_tdx)) +return ; + +if (!is_td_created (kvm_tdx)) + goto free_hkid; + +mutex_lock (&tdx_lock); + err =tdh_mng_key_reclaimid (kvm_tdx->tdr.pa); +mutex_unlock (&tdx_lock);
Please add a comment explaining why the mutex is needed.
Quoted 55 lines
arch/x86/kvm/vmx/tdx.c +55
-0+if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_RECLAIMID , err,NULL ); +return ; + } + + cpumask_allocated =zalloc_cpumask_var (&packages,GFP_KERNEL ); +for_each_online_cpu (i) { +if (cpumask_allocated && +cpumask_test_and_set_cpu (topology_physical_package_id (i), + packages)) +continue ; + + ret =smp_call_on_cpu (i, tdx_do_tdh_phymem_cache_wb,NULL ,1 ); +if (ret) +break ; + } +free_cpumask_var (packages); + +mutex_lock (&tdx_lock); + err =tdh_mng_key_freeid (kvm_tdx->tdr.pa); +mutex_unlock (&tdx_lock); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_FREEID , err,NULL ); +return ; + } + +free_hkid: +tdx_hkid_free (kvm_tdx); +} + +void tdx_vm_free (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +int i; + +/* Can't reclaim or free TD pages if teardown failed. */ +if (is_hkid_assigned (kvm_tdx)) +return ; + +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) +tdx_reclaim_td_page (&kvm_tdx->tdcs[i]); +kfree (kvm_tdx->tdcs); + +if (kvm_tdx->tdr.added && +__tdx_reclaim_page (kvm_tdx->tdr.va, kvm_tdx->tdr.pa, true, + tdx_global_keyid)) +return ; + +free_page (kvm_tdx->tdr.va); +} + +static int tdx_do_tdh_mng_key_config (void *param) +{ +hpa_t *tdr_p = param; +int cpu, cur_pkg;
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 247
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+u64 err; + + cpu =raw_smp_processor_id (); + cur_pkg =topology_physical_package_id (cpu); + +mutex_lock (&tdx_mng_key_config_lock[cur_pkg]); +do { + err =tdh_mng_key_config (*tdr_p); + }while (err ==TDX_KEY_GENERATION_FAILED ); +mutex_unlock (&tdx_mng_key_config_lock[cur_pkg]);
Why not squashing patch 20 ("KVM: TDX: allocate per-package mutex") into this patch?
Quoted 30 lines
arch/x86/kvm/vmx/tdx.c +30
-0+ +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_CONFIG , err,NULL ); +return -EIO ; + } + +return 0 ; +} + +int tdx_vm_init (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +int ret, i; +u64 err; + +/* vCPUs can't be created until after KVM_TDX_INIT_VM. */ + kvm->max_vcpus =0 ; + + kvm_tdx->hkid =tdx_keyid_alloc (); +if (kvm_tdx->hkid <0 ) +return -EBUSY ; + + ret =tdx_alloc_td_page (&kvm_tdx->tdr); +if (ret) + goto free_hkid; + + kvm_tdx->tdcs =kcalloc (tdx_caps.tdcs_nr_pages,sizeof (*kvm_tdx->tdcs), +GFP_KERNEL_ACCOUNT ); +if (!kvm_tdx->tdcs)
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 287
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ goto free_tdr; +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) { + ret =tdx_alloc_td_page (&kvm_tdx->tdcs[i]); +if (ret) + goto free_tdcs; + } + +mutex_lock (&tdx_lock); + err =tdh_mng_create (kvm_tdx->tdr.pa, kvm_tdx->hkid); +mutex_unlock (&tdx_lock);
Please add comment explaining why locking is needed.
Quoted 68 lines
arch/x86/kvm/vmx/tdx.c +64
-0+if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_CREATE , err,NULL ); + ret = -EIO ; + goto free_tdcs; + } +tdx_mark_td_page_added (&kvm_tdx->tdr); + +if (!zalloc_cpumask_var (&packages,GFP_KERNEL )) { + ret = -ENOMEM ; + goto free_tdcs; + } +for_each_online_cpu (i) { +if (cpumask_test_and_set_cpu (topology_physical_package_id (i), + packages)) +continue ; + + ret =smp_call_on_cpu (i, tdx_do_tdh_mng_key_config, + &kvm_tdx->tdr.pa,1 ); +if (ret) +break ; + } +free_cpumask_var (packages); +if (ret) + goto teardown; + +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) { + err =tdh_mng_addcx (kvm_tdx->tdr.pa, kvm_tdx->tdcs[i].pa); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_ADDCX , err,NULL ); + ret = -EIO ; + goto teardown; + } +tdx_mark_td_page_added (&kvm_tdx->tdcs[i]); + } + +/* +* Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated +* ioctl() to define the configure CPUID values for the TD. +*/ +return 0 ; + +/* +* The sequence for freeing resources from a partially initialized TD +* varies based on where in the initialization flow failure occurred. +* Simply use the full teardown and destroy, which naturally play nice +* with partial initialization. +*/ +teardown: +tdx_mmu_prezap (kvm); +tdx_vm_free (kvm); +return ret; + +free_tdcs: +/* @i points at the TDCS page that failed allocation. */ +for (--i; i >=0 ; i--) +free_page (kvm_tdx->tdcs[i].va); +kfree (kvm_tdx->tdcs); +free_tdr: +free_page (kvm_tdx->tdr.va); +free_hkid: +tdx_hkid_free (kvm_tdx); +return ret; +} +static int __tdx_module_setup (void ) {const struct tdsysinfo_struct *tdsysinfo;
arch/x86/kvm/vmx/tdx.c static int __tdx_module_setup(void) · line 373
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +2
-0@@ -59,6 +369,8 @@ static int __tdx_module_setup(void)return ret; } + tdx_global_keyid =tdx_get_global_keyid (); +
Again, really confusing why this is needed.
Quoted 47 lines
arch/x86/kvm/vmx/tdx.c +0
-0tdsysinfo =tdx_get_sysinfo ();if (tdx_caps.nr_cpuid_configs >TDX_MAX_NR_CPUID_CONFIGS )return -EIO ;arch/x86/kvm/vmx/tdx.h +2
-0@@ -19,6 +19,8 @@ struct kvm_tdx {struct tdx_td_page tdr;struct tdx_td_page *tdcs; + +int hkid; };struct vcpu_tdx {arch/x86/kvm/vmx/tdx_errno.h +1
-1@@ -12,11 +12,11 @@#define TDX_SUCCESS 0x0000000000000000ULL#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL -#define TDX_LIFECYCLE_STATE_INCORRECT 0xC000060700000000ULL#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL/* arch/x86/kvm/vmx/tdx_ops.h +1
-0@@ -6,6 +6,7 @@#include <linux/compiler.h> +#include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h> @@ -15,8 +16,14 @@
arch/x86/kvm/vmx/tdx_ops.h line 26
Quoted 10 lines
arch/x86/kvm/vmx/tdx_ops.h +6
-0#ifdef CONFIG_INTEL_TDX_HOST +static inline void tdx_clflush_page (hpa_t addr) +{ +clflush_cache_range (__va (addr),PAGE_SIZE ); +} +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) { +tdx_clflush_page (addr);
Please add comment to explain why clflush is needed.
And you don't need the tdx_clflush_page() wrapper -- it's not a TDX specific ops. You can just use clflush_cache_range().
Quoted 36 lines
arch/x86/kvm/vmx/tdx_ops.h +1
-0return kvm_seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); }@@ -56,6 +63,7 @@ static inline u64 tdh_mng_key_config(hpa_t tdr)static inline u64 tdh_mng_create (hpa_t tdr,int hkid) { +tdx_clflush_page (tdr);return kvm_seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); }arch/x86/kvm/vmx/x86_ops.h +8
-0@@ -132,12 +132,20 @@ void __init tdx_pre_kvm_init(unsigned int *vcpu_size,bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void ); + +int tdx_vm_init (struct kvm *kvm); +void tdx_mmu_prezap (struct kvm *kvm); +void tdx_vm_free (struct kvm *kvm);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {}static inline void tdx_hardware_unsetup (void ) {} + +static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; } +static inline void tdx_mmu_prezap (struct kvm *kvm) {} +static inline void tdx_vm_free (struct kvm *kvm) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
Re: [RFC PATCH v5 024/104] KVM: TDX: create/destroy VM structure
Quoted 12 lines · Kai Huang
Quoted 11 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Sean Christopherson <[email protected]>
arch/x86/kvm/vmx/tdx.c +0
-0@@ -31,14 +31,324 @@ struct tdx_capabilities {struct tdx_cpuid_config cpuid_configs[TDX_MAX_NR_CPUID_CONFIGS ]; };
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 36
Quoted 3 lines
arch/x86/kvm/vmx/tdx.c +3
-0+/* KeyID used by TDX module */ +static u32 tdx_global_keyid__read_mostly ; +It's really not clear why you need to know
tdx_global_keyidin the context of creating/destroying a TD.
TDX module mpas TDR with TDX global key id. This page includes key id assigned to this TD. Then, TDX modules maps other TD-related pages with the HKID. TDR requires the TDX global key id for cache flush unlike other TD-related pages.
I'll add a comment.
Quoted 42 lines
Quoted 29 lines
arch/x86/kvm/vmx/tdx.c +21
-0/* Capabilities of KVM + the TDX module. */ struct tdx_capabilities tdx_caps; +static DEFINE_MUTEX (tdx_lock);static struct mutex *tdx_mng_key_config_lock;static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ; +static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) +{ + pa &= ~hkid_mask; + pa |= (u64 )hkid << hkid_start_pos; + +return pa; +} + +static inline bool is_td_created (struct kvm_tdx *kvm_tdx) +{ +return kvm_tdx->tdr.added; +} + +static inline void tdx_hkid_free (struct kvm_tdx *kvm_tdx) +{ +tdx_keyid_free (kvm_tdx->hkid); + kvm_tdx->hkid =-1 ; +} + +static inline bool is_hkid_assigned (struct kvm_tdx *kvm_tdx)arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 75
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+{ +return kvm_tdx->hkid >0 ; +} + +static void tdx_clear_page (unsigned long page) +{ +const void *zero_page = (const void *)__va (page_to_phys (ZERO_PAGE (0 ))); +unsigned long i; + +/* Zeroing the page is only necessary for systems with MKTME-i. */ "only necessary for systems with MKTME-i" because of what?
Quoted 3 lines
Please be more clear that on MKTME-i system, when re-assign one page from old keyid to a new keyid, MOVDIR64B is required to clear/write the page with new keyid to prevent integrity error when read on the page with new keyid.
Let me borrow this sentence as a comment on it.
Quoted 7 lines
Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +7
-0+static int __tdx_reclaim_page (unsigned long va,hpa_t pa,bool do_wb,u16 hkid) +{ +struct tdx_module_output out; +u64 err; + + err =tdh_phymem_page_reclaim (pa, &out); +if (WARN_ON_ONCE (err)) {
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 96
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+pr_tdx_error (TDH_PHYMEM_PAGE_RECLAIM , err, &out); +return -EIO ; + } + +if (do_wb) {In the callers, please add some comments explaining why
do_wbis needed, and why is not needed.
Will do.
Quoted 9 lines
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9
-0+static int tdx_do_tdh_mng_key_config (void *param) +{ +hpa_t *tdr_p = param; +int cpu, cur_pkg; +u64 err; + + cpu =raw_smp_processor_id (); + cur_pkg =topology_physical_package_id (cpu); +
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 247
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+mutex_lock (&tdx_mng_key_config_lock[cur_pkg]); +do { + err =tdh_mng_key_config (*tdr_p); + }while (err ==TDX_KEY_GENERATION_FAILED ); +mutex_unlock (&tdx_mng_key_config_lock[cur_pkg]);Why not squashing patch 20 ("KVM: TDX: allocate per-package mutex") into this patch?
Will do.
Quoted 34 lines
Quoted 34 lines
arch/x86/kvm/vmx/tdx.c +34
-0+ +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_CONFIG , err,NULL ); +return -EIO ; + } + +return 0 ; +} + +int tdx_vm_init (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +int ret, i; +u64 err; + +/* vCPUs can't be created until after KVM_TDX_INIT_VM. */ + kvm->max_vcpus =0 ; + + kvm_tdx->hkid =tdx_keyid_alloc (); +if (kvm_tdx->hkid <0 ) +return -EBUSY ; + + ret =tdx_alloc_td_page (&kvm_tdx->tdr); +if (ret) + goto free_hkid; + + kvm_tdx->tdcs =kcalloc (tdx_caps.tdcs_nr_pages,sizeof (*kvm_tdx->tdcs), +GFP_KERNEL_ACCOUNT ); +if (!kvm_tdx->tdcs) + goto free_tdr; +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) { + ret =tdx_alloc_td_page (&kvm_tdx->tdcs[i]); +if (ret)
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c struct tdx_capabilities { · line 287
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+ goto free_tdcs; + } + +mutex_lock (&tdx_lock); + err =tdh_mng_create (kvm_tdx->tdr.pa, kvm_tdx->hkid); +mutex_unlock (&tdx_lock);Please add comment explaining why locking is needed.
I'll add a comment on tdx_lock. Not each TDX seamcalls.
Quoted 27 lines
Quoted 14 lines
arch/x86/kvm/vmx/tdx_ops.h +1
-0@@ -6,6 +6,7 @@#include <linux/compiler.h> +#include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h> @@ -15,8 +16,14 @@arch/x86/kvm/vmx/tdx_ops.h line 26
Quoted 10 lines
arch/x86/kvm/vmx/tdx_ops.h +6
-0#ifdef CONFIG_INTEL_TDX_HOST +static inline void tdx_clflush_page (hpa_t addr) +{ +clflush_cache_range (__va (addr),PAGE_SIZE ); +} +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) { +tdx_clflush_page (addr);Please add comment to explain why clflush is needed.
Quoted 2 lines
And you don't need the
tdx_clflush_page()wrapper -- it's not a TDX specific ops. You can just useclflush_cache_range().
Will remove it.
The plan was to enhance tdx_cflush_page(addr, page_level) to support large page to avoid repeating page_level_to_size. Defer it to large page support.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH v5 024/104] KVM: TDX: create/destroy VM structure
Quoted 9 lines · Isaku Yamahata
Quoted 3 lines
It's really not clear why you need to know
tdx_global_keyidin the context of
creating/destroying a TD.TDX module mpas TDR with TDX global key id. This page includes key id assigned to this TD. Then, TDX modules maps other TD-related pages with the HKID. TDR requires the TDX global key id for cache flush unlike other TD-related pages.
I'll add a comment.
Sorry I forgot this. Thanks.
Signature · 2 lines
Thanks, -Kai
Paolo Bonzini
maintainerRe: [RFC PATCH v5 024/104] KVM: TDX: create/destroy VM structure
In reply to [email protected]Quoted 32 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
As the first step to create TDX guest, create/destroy VM struct. Assign Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.
Add a second
kvm_x86_opshook inkvm_arch_vm_destroy()to support TDX's destruction path, which needs to first put the VM into a teardown state, then free per-vCPU resources, and finally free per-VM resources.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/vmx/main.c | 16 +- arch/x86/kvm/vmx/tdx.c | 312 +++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 2 + arch/x86/kvm/vmx/tdx_errno.h | 2 +- arch/x86/kvm/vmx/tdx_ops.h | 8 + arch/x86/kvm/vmx/x86_ops.h | 8 + 6 files changed, 346 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/main.c +1
-1@@ -39,12 +39,24 @@ static int vt_vm_init(struct kvm *kvm) ret =tdx_module_setup ();if (ret)return ret; -return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ +return tdx_vm_init (kvm);
arch/x86/kvm/vmx/main.c static int vt_vm_init(struct kvm *kvm) · line 52
Quoted 10 lines
arch/x86/kvm/vmx/main.c +5
-0}return vmx_vm_init (kvm); } +static void vt_mmu_prezap (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_mmu_prezap (kvm); +}
Please rename the function to explain what it does, for example tdx_mmu_release_hkid.
Paolo
Quoted 447 lines
arch/x86/kvm/vmx/main.c +8
-0+static void vt_vm_free (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_vm_free (kvm); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,@@ -58,6 +70,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vt_vm_init, + .mmu_prezap = vt_mmu_prezap, + .vm_free = vt_vm_free, .vcpu_create = vmx_vcpu_create, .vcpu_free = vmx_vcpu_free,arch/x86/kvm/vmx/tdx.c +312
-0@@ -31,14 +31,324 @@ struct tdx_capabilities {struct tdx_cpuid_config cpuid_configs[TDX_MAX_NR_CPUID_CONFIGS ]; }; +/* KeyID used by TDX module */ +static u32 tdx_global_keyid__read_mostly ; +/* Capabilities of KVM + the TDX module. */ struct tdx_capabilities tdx_caps; +static DEFINE_MUTEX (tdx_lock);static struct mutex *tdx_mng_key_config_lock;static u64 hkid_mask__ro_after_init ;static u8 hkid_start_pos__ro_after_init ; +static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) +{ + pa &= ~hkid_mask; + pa |= (u64 )hkid << hkid_start_pos; + +return pa; +} + +static inline bool is_td_created (struct kvm_tdx *kvm_tdx) +{ +return kvm_tdx->tdr.added; +} + +static inline void tdx_hkid_free (struct kvm_tdx *kvm_tdx) +{ +tdx_keyid_free (kvm_tdx->hkid); + kvm_tdx->hkid =-1 ; +} + +static inline bool is_hkid_assigned (struct kvm_tdx *kvm_tdx) +{ +return kvm_tdx->hkid >0 ; +} + +static void tdx_clear_page (unsigned long page) +{ +const void *zero_page = (const void *)__va (page_to_phys (ZERO_PAGE (0 ))); +unsigned long i; + +/* Zeroing the page is only necessary for systems with MKTME-i. */ +if (!static_cpu_has (X86_FEATURE_MOVDIR64B )) +return ; + +for (i =0 ; i <4096 ; i +=64 ) +/* MOVDIR64B [rdx], es:rdi */ + asm (".byte 0x66, 0x0f, 0x38, 0xf8, 0x3a" + : :"d" (zero_page),"D" (page + i) :"memory" ); +} + +static int __tdx_reclaim_page (unsigned long va,hpa_t pa,bool do_wb,u16 hkid) +{ +struct tdx_module_output out; +u64 err; + + err =tdh_phymem_page_reclaim (pa, &out); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_PAGE_RECLAIM , err, &out); +return -EIO ; + } + +if (do_wb) { + err =tdh_phymem_page_wbinvd (set_hkid_to_hpa (pa, hkid)); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_PAGE_WBINVD , err,NULL ); +return -EIO ; + } + } + +tdx_clear_page (va); +return 0 ; +} + +static int tdx_reclaim_page (unsigned long va,hpa_t pa) +{ +return __tdx_reclaim_page (va, pa, false,0 ); +} + +static int tdx_alloc_td_page (struct tdx_td_page *page) +{ + page->va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!page->va) +return -ENOMEM ; + + page->pa =__pa (page->va); +return 0 ; +} + +static void tdx_mark_td_page_added (struct tdx_td_page *page) +{ +WARN_ON_ONCE (page->added); + page->added = true; +} + +static void tdx_reclaim_td_page (struct tdx_td_page *page) +{ +if (page->added) { +if (tdx_reclaim_page (page->va, page->pa)) +return ; + + page->added = false; + } +free_page (page->va); +} + +static int tdx_do_tdh_phymem_cache_wb (void *param) +{ +u64 err =0 ; + +/* +* We can destroy multiple the guest TDs simultaneously. Prevent +* tdh_phymem_cache_wb from returning TDX_BUSY by serialization. +*/ +mutex_lock (&tdx_lock); +do { + err =tdh_phymem_cache_wb (!!err); + }while (err ==TDX_INTERRUPTED_RESUMABLE ); +mutex_unlock (&tdx_lock); + +/* Other thread may have done for us. */ +if (err ==TDX_NO_HKID_READY_TO_WBCACHE ) + err =TDX_SUCCESS ; +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_CACHE_WB , err,NULL ); +return -EIO ; + } + +return 0 ; +} + +void tdx_mmu_prezap (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +bool cpumask_allocated; +u64 err; +int ret; +int i; + +if (!is_hkid_assigned (kvm_tdx)) +return ; + +if (!is_td_created (kvm_tdx)) + goto free_hkid; + +mutex_lock (&tdx_lock); + err =tdh_mng_key_reclaimid (kvm_tdx->tdr.pa); +mutex_unlock (&tdx_lock); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_RECLAIMID , err,NULL ); +return ; + } + + cpumask_allocated =zalloc_cpumask_var (&packages,GFP_KERNEL ); +for_each_online_cpu (i) { +if (cpumask_allocated && +cpumask_test_and_set_cpu (topology_physical_package_id (i), + packages)) +continue ; + + ret =smp_call_on_cpu (i, tdx_do_tdh_phymem_cache_wb,NULL ,1 ); +if (ret) +break ; + } +free_cpumask_var (packages); + +mutex_lock (&tdx_lock); + err =tdh_mng_key_freeid (kvm_tdx->tdr.pa); +mutex_unlock (&tdx_lock); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_FREEID , err,NULL ); +return ; + } + +free_hkid: +tdx_hkid_free (kvm_tdx); +} + +void tdx_vm_free (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +int i; + +/* Can't reclaim or free TD pages if teardown failed. */ +if (is_hkid_assigned (kvm_tdx)) +return ; + +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) +tdx_reclaim_td_page (&kvm_tdx->tdcs[i]); +kfree (kvm_tdx->tdcs); + +if (kvm_tdx->tdr.added && +__tdx_reclaim_page (kvm_tdx->tdr.va, kvm_tdx->tdr.pa, true, + tdx_global_keyid)) +return ; + +free_page (kvm_tdx->tdr.va); +} + +static int tdx_do_tdh_mng_key_config (void *param) +{ +hpa_t *tdr_p = param; +int cpu, cur_pkg; +u64 err; + + cpu =raw_smp_processor_id (); + cur_pkg =topology_physical_package_id (cpu); + +mutex_lock (&tdx_mng_key_config_lock[cur_pkg]); +do { + err =tdh_mng_key_config (*tdr_p); + }while (err ==TDX_KEY_GENERATION_FAILED ); +mutex_unlock (&tdx_mng_key_config_lock[cur_pkg]); + +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_CONFIG , err,NULL ); +return -EIO ; + } + +return 0 ; +} + +int tdx_vm_init (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +int ret, i; +u64 err; + +/* vCPUs can't be created until after KVM_TDX_INIT_VM. */ + kvm->max_vcpus =0 ; + + kvm_tdx->hkid =tdx_keyid_alloc (); +if (kvm_tdx->hkid <0 ) +return -EBUSY ; + + ret =tdx_alloc_td_page (&kvm_tdx->tdr); +if (ret) + goto free_hkid; + + kvm_tdx->tdcs =kcalloc (tdx_caps.tdcs_nr_pages,sizeof (*kvm_tdx->tdcs), +GFP_KERNEL_ACCOUNT ); +if (!kvm_tdx->tdcs) + goto free_tdr; +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) { + ret =tdx_alloc_td_page (&kvm_tdx->tdcs[i]); +if (ret) + goto free_tdcs; + } + +mutex_lock (&tdx_lock); + err =tdh_mng_create (kvm_tdx->tdr.pa, kvm_tdx->hkid); +mutex_unlock (&tdx_lock); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_CREATE , err,NULL ); + ret = -EIO ; + goto free_tdcs; + } +tdx_mark_td_page_added (&kvm_tdx->tdr); + +if (!zalloc_cpumask_var (&packages,GFP_KERNEL )) { + ret = -ENOMEM ; + goto free_tdcs; + } +for_each_online_cpu (i) { +if (cpumask_test_and_set_cpu (topology_physical_package_id (i), + packages)) +continue ; + + ret =smp_call_on_cpu (i, tdx_do_tdh_mng_key_config, + &kvm_tdx->tdr.pa,1 ); +if (ret) +break ; + } +free_cpumask_var (packages); +if (ret) + goto teardown; + +for (i =0 ; i < tdx_caps.tdcs_nr_pages; i++) { + err =tdh_mng_addcx (kvm_tdx->tdr.pa, kvm_tdx->tdcs[i].pa); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_ADDCX , err,NULL ); + ret = -EIO ; + goto teardown; + } +tdx_mark_td_page_added (&kvm_tdx->tdcs[i]); + } + +/* +* Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated +* ioctl() to define the configure CPUID values for the TD. +*/ +return 0 ; + +/* +* The sequence for freeing resources from a partially initialized TD +* varies based on where in the initialization flow failure occurred. +* Simply use the full teardown and destroy, which naturally play nice +* with partial initialization. +*/ +teardown: +tdx_mmu_prezap (kvm); +tdx_vm_free (kvm); +return ret; + +free_tdcs: +/* @i points at the TDCS page that failed allocation. */ +for (--i; i >=0 ; i--) +free_page (kvm_tdx->tdcs[i].va); +kfree (kvm_tdx->tdcs); +free_tdr: +free_page (kvm_tdx->tdr.va); +free_hkid: +tdx_hkid_free (kvm_tdx); +return ret; +} +static int __tdx_module_setup (void ) {const struct tdsysinfo_struct *tdsysinfo;@@ -59,6 +369,8 @@ static int __tdx_module_setup(void)return ret; } + tdx_global_keyid =tdx_get_global_keyid (); + tdsysinfo =tdx_get_sysinfo ();if (tdx_caps.nr_cpuid_configs >TDX_MAX_NR_CPUID_CONFIGS )return -EIO ;arch/x86/kvm/vmx/tdx.h +2
-0@@ -19,6 +19,8 @@ struct kvm_tdx {struct tdx_td_page tdr;struct tdx_td_page *tdcs; + +int hkid; };struct vcpu_tdx {arch/x86/kvm/vmx/tdx_errno.h +1
-1@@ -12,11 +12,11 @@#define TDX_SUCCESS 0x0000000000000000ULL#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL -#define TDX_LIFECYCLE_STATE_INCORRECT 0xC000060700000000ULL#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL/* arch/x86/kvm/vmx/tdx_ops.h +8
-0@@ -6,6 +6,7 @@#include <linux/compiler.h> +#include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h> @@ -15,8 +16,14 @@#ifdef CONFIG_INTEL_TDX_HOST +static inline void tdx_clflush_page (hpa_t addr) +{ +clflush_cache_range (__va (addr),PAGE_SIZE ); +} +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) { +tdx_clflush_page (addr);return kvm_seamcall (TDH_MNG_ADDCX , addr, tdr,0 ,0 ,0 ,NULL ); }@@ -56,6 +63,7 @@ static inline u64 tdh_mng_key_config(hpa_t tdr)static inline u64 tdh_mng_create (hpa_t tdr,int hkid) { +tdx_clflush_page (tdr);return kvm_seamcall (TDH_MNG_CREATE , tdr, hkid,0 ,0 ,0 ,NULL ); }arch/x86/kvm/vmx/x86_ops.h +8
-0@@ -132,12 +132,20 @@ void __init tdx_pre_kvm_init(unsigned int *vcpu_size,bool tdx_is_vm_type_supported (unsigned long type);void __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void ); + +int tdx_vm_init (struct kvm *kvm); +void tdx_mmu_prezap (struct kvm *kvm); +void tdx_vm_free (struct kvm *kvm);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }static inline void tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {}static inline void tdx_hardware_unsetup (void ) {} + +static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; } +static inline void tdx_mmu_prezap (struct kvm *kvm) {} +static inline void tdx_vm_free (struct kvm *kvm) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
Re: [RFC PATCH v5 024/104] KVM: TDX: create/destroy VM structure
Quoted 13 lines · Paolo Bonzini
Quoted 13 lines
arch/x86/kvm/vmx/main.c +1
-1@@ -39,12 +39,24 @@ static int vt_vm_init(struct kvm *kvm) ret =tdx_module_setup ();if (ret)return ret; -return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ +return tdx_vm_init (kvm); }return vmx_vm_init (kvm); }
Quoted 8 lines
arch/x86/kvm/vmx/main.c static int vt_vm_init(struct kvm *kvm) · line 52
Quoted 5 lines
arch/x86/kvm/vmx/main.c +5
-0+static void vt_mmu_prezap (struct kvm *kvm) +{ +if (is_td (kvm)) +return tdx_mmu_prezap (kvm); +}Please rename the function to explain what it does, for example
tdx_mmu_release_hkid.
In the patch 021/104, you suggested flush_shadow_all_private(). Which do you prefer? flush_shadow_all_private or tdx_mmu_release_hkid.
Signature · 1 line
Isaku Yamahata <[email protected]>
Paolo Bonzini
maintainerRe: [RFC PATCH v5 024/104] KVM: TDX: create/destroy VM structure
Quoted 4 lines · Isaku Yamahata
Quoted 2 lines
Please rename the function to explain what it does, for example
tdx_mmu_release_hkid.In the patch 021/104, you suggested
flush_shadow_all_private(). Which do you prefer?flush_shadow_all_privateortdx_mmu_release_hkid.
vt_mmu_prezap should become vt_flush_shadow_all_private.
tdx_mmu_prezap should be come tdx_mmu_release_hkid.
Paolo
RFC PATCH v5 25/104 KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add a place holder function for TDX specific VM-scoped ioctl as mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters.
KVM_MEMORY_ENCRYPT_OP was introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations under KVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurpose KVM_MEMORY_ENCRYPT_OP for TDX specific operations and define subcommands.
TDX requires VM-scoped, and VCPU-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization, and TDX-specific vCPU initialization. Which requires KVM vCPU-scoped operations in addition to the existing VM-scoped operations.
--- arch/x86/include/uapi/asm/kvm.h | 11 +++++++++++ arch/x86/kvm/vmx/main.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.c | 24 ++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ tools/arch/x86/include/uapi/asm/kvm.h | 11 +++++++++++ 5 files changed, 60 insertions(+)
arch/x86/include/uapi/asm/kvm.h +11 -0
@@ -528,4 +528,15 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_TDX_VM 1 +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +__u32 id; +__u32 metadata; +__u64 data; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/vmx/main.c +10 -0
@@ -57,6 +57,14 @@ static void vt_vm_free(struct kvm *kvm)return tdx_vm_free (kvm); } +static int vt_mem_enc_op (struct kvm *kvm,void __user *argp) +{ +if (!is_td (kvm)) +return -ENOTTY ; + +return tdx_vm_ioctl (kvm, argp); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,
@@ -195,6 +203,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .complete_emulated_msr = kvm_complete_insn_gp, .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, + + .mem_enc_op = vt_mem_enc_op, };struct kvm_x86_init_ops vt_init_ops__initdata = {
arch/x86/kvm/vmx/tdx.c +24 -0
@@ -349,6 +349,30 @@ int tdx_vm_init(struct kvm *kvm)return ret; } +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) +{ +struct kvm_tdx_cmd tdx_cmd; +int r; + +if (copy_from_user (&tdx_cmd, argp,sizeof (struct kvm_tdx_cmd ))) +return -EFAULT ; + +mutex_lock (&kvm->lock); + +switch (tdx_cmd.id) { +default : + r = -EINVAL ; + goto out; + } + +if (copy_to_user (argp, &tdx_cmd,sizeof (struct kvm_tdx_cmd ))) + r = -EFAULT ; + +out: +mutex_unlock (&kvm->lock); +return r; +} +static int __tdx_module_setup (void ) {const struct tdsysinfo_struct *tdsysinfo;
arch/x86/kvm/vmx/x86_ops.h +4 -0
@@ -136,6 +136,8 @@ void tdx_hardware_unsetup(void);int tdx_vm_init (struct kvm *kvm);void tdx_mmu_prezap (struct kvm *kvm);void tdx_vm_free (struct kvm *kvm); + +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}
@@ -146,6 +148,8 @@ static inline void tdx_hardware_unsetup(void) {}static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; }static inline void tdx_mmu_prezap (struct kvm *kvm) {}static inline void tdx_vm_free (struct kvm *kvm) {} + +static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
tools/arch/x86/include/uapi/asm/kvm.h +11 -0
@@ -528,4 +528,15 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_TDX_VM 1 +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +__u32 id; +__u32 metadata; +__u64 data; +}; +#endif /* _ASM_X86_KVM_H */
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 025/104] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl
Quoted 35 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a place holder function for TDX specific VM-scoped ioctl as
mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters.
KVM_MEMORY_ENCRYPT_OPwas introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations underKVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurposeKVM_MEMORY_ENCRYPT_OPfor TDX specific operations and define subcommands.TDX requires VM-scoped, and VCPU-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization, and TDX-specific vCPU initialization. Which requires KVM vCPU-scoped operations in addition to the existing VM-scoped operations.
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 11 +++++++++++ arch/x86/kvm/vmx/main.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.c | 24 ++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ tools/arch/x86/include/uapi/asm/kvm.h | 11 +++++++++++ 5 files changed, 60 insertions(+)arch/x86/include/uapi/asm/kvm.h +0
-0@@ -528,4 +528,15 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_TDX_VM 1
arch/x86/include/uapi/asm/kvm.h struct kvm_pmu_event_filter { · line 540
Quoted 10 lines
arch/x86/include/uapi/asm/kvm.h +10
-0+/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +__u32 id; +__u32 metadata; +__u64 data; +};
Please include some initial documentation here already, for example it is not clear what "metadata" is.
Also please add
u32 error; u32 unused;
for two reasons: 1) consistency with kvm_sev_cmd 2) error codes should be returned to userspace and not just sent through pr_tdx_error.
Paolo
Re: [RFC PATCH v5 025/104] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl
Quoted 49 lines · Paolo Bonzini
Quoted 34 lines · On 3/4/22 20:48, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add a place holder function for TDX specific VM-scoped ioctl as
mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters.
KVM_MEMORY_ENCRYPT_OPwas introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations underKVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurposeKVM_MEMORY_ENCRYPT_OPfor TDX specific operations and define subcommands.TDX requires VM-scoped, and VCPU-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization, and TDX-specific vCPU initialization. Which requires KVM vCPU-scoped operations in addition to the existing VM-scoped operations.
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 11 +++++++++++ arch/x86/kvm/vmx/main.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.c | 24 ++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ tools/arch/x86/include/uapi/asm/kvm.h | 11 +++++++++++ 5 files changed, 60 insertions(+)arch/x86/include/uapi/asm/kvm.h +0
-0@@ -528,4 +528,15 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_TDX_VM 1arch/x86/include/uapi/asm/kvm.h struct kvm_pmu_event_filter { · line 539
Quoted 10 lines
arch/x86/include/uapi/asm/kvm.h +10
-0+/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +__u32 id; +__u32 metadata; +__u64 data; +};Please include some initial documentation here already, for example it is not clear what "metadata" is.
Quoted 7 lines
Also please add
u32 error; u32 unused;for two reasons: 1) consistency with
kvm_sev_cmd2) error codes should be returned to userspace and not just sent throughpr_tdx_error.
Sure.
For now metadata is only used to specify flags specific to id. So I'll rename it to flags.
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH v5 26/104 KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Implement a VM-scoped subcomment to get system-wide parameters. Although this is system-wide parameters not per-VM, this subcomand is VM-scoped because
- Device model needs TDX system-wide parameters after creating KVM VM.
- This subcommands requires to initialize TDX module. For lazy initialization of the TDX module, vm-scope ioctl is better.
--- arch/x86/include/uapi/asm/kvm.h | 22 ++++++++++++++ arch/x86/kvm/vmx/tdx.c | 41 +++++++++++++++++++++++++++ tools/arch/x86/include/uapi/asm/kvm.h | 22 ++++++++++++++ 3 files changed, 85 insertions(+)
arch/x86/include/uapi/asm/kvm.h +22 -0
@@ -530,6 +530,8 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id { +KVM_TDX_CAPABILITIES =0 , +KVM_TDX_CMD_NR_MAX , };
@@ -539,4 +541,24 @@ struct kvm_tdx_cmd { __u64 data; }; +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; + +struct kvm_tdx_capabilities { +__u64 attrs_fixed0; +__u64 attrs_fixed1; +__u64 xfam_fixed0; +__u64 xfam_fixed1; + +__u32 nr_cpuid_configs; +__u32 padding; +struct kvm_tdx_cpuid_config cpuid_configs[0 ]; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/vmx/tdx.c +41 -0
53 lines not on this page · raw message
tools/arch/x86/include/uapi/asm/kvm.h +22 -0
@@ -530,6 +530,8 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id { +KVM_TDX_CAPABILITIES =0 , +KVM_TDX_CMD_NR_MAX , };
@@ -539,4 +541,24 @@ struct kvm_tdx_cmd { __u64 data; }; +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; + +struct kvm_tdx_capabilities { +__u64 attrs_fixed0; +__u64 attrs_fixed1; +__u64 xfam_fixed0; +__u64 xfam_fixed1; + +__u32 nr_cpuid_configs; +__u32 padding; +struct kvm_tdx_cpuid_config cpuid_configs[0 ]; +}; +#endif /* _ASM_X86_KVM_H */
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 026/104] KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Implement a VM-scoped subcomment to get system-wide parameters. Although this is system-wide parameters not per-VM, this subcomand is VM-scoped because
- Device model needs TDX system-wide parameters after creating KVM VM.
- This subcommands requires to initialize TDX module. For lazy initialization of the TDX module, vm-scope ioctl is better.
Since there was agreement to install the TDX module on load, please place this ioctl on the /dev/kvm file descriptor.
At least for SEV, there were cases where the system-wide parameters are needed outside KVM, so it's better to avoid requiring a VM file descriptor.
Thanks,
Paolo
Re: [RFC PATCH v5 026/104] KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Implement a VM-scoped subcomment to get system-wide parameters. Although this is system-wide parameters not per-VM, this subcomand is VM-scoped because
- Device model needs TDX system-wide parameters after creating KVM VM.
- This subcommands requires to initialize TDX module. For lazy initialization of the TDX module, vm-scope ioctl is better.Since there was agreement to install the TDX module on load, please place this ioctl on the
/dev/kvmfile descriptor.At least for SEV, there were cases where the system-wide parameters are needed outside KVM, so it's better to avoid requiring a VM file descriptor.
I don't have strong preference on KVM-scope ioctl or VM-scope.
Initially, we made it KVM-scope and change it to VM-scope in this version. Yes, it returns the info from TDX module, which doesn't vary per VM. However, what if we want to return different capabilities (software controlled capabilities) per VM? Part of the TDX capabilities serves like get_supported_cpuid, making it KVM wide lacks the flexibility to return differentiated capabilities for different TDs.
Quoted 4 lines
Thanks,
Paolo
Re: [RFC PATCH v5 026/104] KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters
Quoted 15 lines · Xiaoyao Li
Quoted 8 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Implement a VM-scoped subcomment to get system-wide parameters. Although this is system-wide parameters not per-VM, this subcomand is VM-scoped because
- Device model needs TDX system-wide parameters after creating KVM VM.
- This subcommands requires to initialize TDX module. For lazy initialization of the TDX module, vm-scope ioctl is better.Quoted 5 lines
Since there was agreement to install the TDX module on load, please place this ioctl on the
/dev/kvmfile descriptor.At least for SEV, there were cases where the system-wide parameters are needed outside KVM, so it's better to avoid requiring a VM file descriptor.
Quoted 7 lines
I don't have strong preference on KVM-scope ioctl or VM-scope.
Initially, we made it KVM-scope and change it to VM-scope in this version. Yes, it returns the info from TDX module, which doesn't vary per VM. However, what if we want to return different capabilities (software controlled capabilities) per VM?
In this case, you don't return different capabilities, instead, you return the same capabilities but control the capabilities on per-VM basis.
Quoted 10 lines
Part of the TDX capabilities
serves likeget_supported_cpuid, making it KVM wide lacks the flexibility to return differentiated capabilities for different TDs.Quoted 4 lines
Thanks,
Paolo
Re: [RFC PATCH v5 026/104] KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters
Quoted 18 lines · Kai Huang
Quoted 17 lines · Xiaoyao Li
Quoted 8 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Implement a VM-scoped subcomment to get system-wide parameters. Although this is system-wide parameters not per-VM, this subcomand is VM-scoped because
- Device model needs TDX system-wide parameters after creating KVM VM.
- This subcommands requires to initialize TDX module. For lazy initialization of the TDX module, vm-scope ioctl is better.Quoted 5 lines
Since there was agreement to install the TDX module on load, please place this ioctl on the
/dev/kvmfile descriptor.At least for SEV, there were cases where the system-wide parameters are needed outside KVM, so it's better to avoid requiring a VM file descriptor.
I don't have strong preference on KVM-scope ioctl or VM-scope.
Quoted 8 lines
Quoted 5 lines
Initially, we made it KVM-scope and change it to VM-scope in this version. Yes, it returns the info from TDX module, which doesn't vary per VM. However, what if we want to return different capabilities (software controlled capabilities) per VM?
In this case, you don't return different capabilities, instead, you return the same capabilities but control the capabilities on per-VM basis.
yes, so I'm not arguing it or insisting on per-VM.
I just speak out my concern since it's user ABI.
Quoted 11 lines
Quoted 10 lines
Part of the TDX capabilities
serves likeget_supported_cpuid, making it KVM wide lacks the flexibility to return differentiated capabilities for different TDs.Quoted 4 lines
Thanks,
Paolo
Re: [RFC PATCH v5 026/104] KVM: TDX: x86: Add vm ioctl to get TDX systemwide parameters
Quoted 25 lines · Xiaoyao Li
Quoted 24 lines · Kai Huang
Quoted 22 lines · Xiaoyao Li
Quoted 8 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
Implement a VM-scoped subcomment to get system-wide parameters. Although this is system-wide parameters not per-VM, this subcomand is VM-scoped because
- Device model needs TDX system-wide parameters after creating KVM VM.
- This subcommands requires to initialize TDX module. For lazy initialization of the TDX module, vm-scope ioctl is better.Quoted 5 lines
Since there was agreement to install the TDX module on load, please place this ioctl on the
/dev/kvmfile descriptor.At least for SEV, there were cases where the system-wide parameters are needed outside KVM, so it's better to avoid requiring a VM file descriptor.
I don't have strong preference on KVM-scope ioctl or VM-scope.
Initially, we made it KVM-scope and change it to VM-scope in this version. Yes, it returns the info from TDX module, which doesn't vary per VM. However, what if we want to return different capabilities (software controlled capabilities) per VM?
Quoted 6 lines
Quoted 2 lines
In this case, you don't return different capabilities, instead, you return the same capabilities but control the capabilities on per-VM basis.
yes, so I'm not arguing it or insisting on per-VM.
I just speak out my concern since it's user ABI.
The reason why I made this API to VM-scope API is to reduce the number of patch given qemu usage. Now Paolo requested it, I'll change it KVM-scope API.
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH v5 27/104 KVM: TDX: initialize VM with TDX specific parameters
In reply to [email protected]From: Xiaoyao Li <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect its confidentiality of its memory contents and its CPU state from any other software, including VMM. When creating guest TD VM before creating vcpu, the number of vcpu, TSC frequency (that is same among vcpus. and it can't be changed.) CPUIDs which is emulated by the TDX module. It means guest can trust those CPUIDs. and sha384 values for measurement.
Add new subcommand, KVM_TDX_INIT_VM, to pass parameters for TDX guest. It assigns encryption key to the TDX guest for memory encryption. TDX encrypts memory per-guest bases. It assigns Device model passes per-VM parameters for the TDX guest. The maximum number of vcpus, tsc frequency (TDX guest has fised VM-wide TSC frequency. not per-vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which is reflected into guest XCR0, IA32_XSS MSR), cpuids, sha384 measurements, and etc.
--- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/include/uapi/asm/kvm.h | 12 ++ arch/x86/kvm/vmx/tdx.c | 200 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 26 ++++ arch/x86/kvm/x86.c | 3 +- arch/x86/kvm/x86.h | 2 + tools/arch/x86/include/uapi/asm/kvm.h | 12 ++ 7 files changed, 256 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1234,6 +1234,8 @@ struct kvm_arch {hpa_t hv_root_tdp;spinlock_t hv_root_tdp_lock;#endif + +gfn_t gfn_shared_mask; };struct kvm_vm_stat {
arch/x86/include/uapi/asm/kvm.h +12 -0
@@ -531,6 +531,7 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM ,KVM_TDX_CMD_NR_MAX , };
@@ -561,4 +562,15 @@ struct kvm_tdx_capabilities { struct kvm_tdx_cpuid_config cpuid_configs[0]; }; +struct kvm_tdx_init_vm { +__u32 max_vcpus; +__u32 tsc_khz; +__u64 attributes; +__u64 cpuid; +__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha348 digest */ +__u64 reserved[43 ];/* must be zero for future extensibility */ +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/vmx/tdx.c +200 -0
212 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +26 -0
@@ -20,7 +20,15 @@ struct kvm_tdx {struct tdx_td_page tdr;struct tdx_td_page *tdcs; +u64 attributes; +u64 xfam;int hkid; + +int cpuid_nent; +struct kvm_cpuid_entry2 cpuid_entries[KVM_MAX_CPUID_ENTRIES ]; + +u64 tsc_offset; +unsigned long tsc_khz; };struct vcpu_tdx {
@@ -50,6 +58,11 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_tdx, vcpu); } +static inline bool is_td_initialized (struct kvm *kvm) +{ +return !!kvm->max_vcpus; +} + static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) { BUILD_BUG_ON_MSG(__builtin_constant_p(field) && (field) & 0x1,
@@ -135,6 +148,19 @@ TDX_BUILD_TDVPS_ACCESSORS(64, VMCS, vmcs);TDX_BUILD_TDVPS_ACCESSORS (64 ,STATE_NON_ARCH , state_non_arch);TDX_BUILD_TDVPS_ACCESSORS (8 ,MANAGEMENT , management); +static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) +{ +struct tdx_module_output out; +u64 err; + + err =tdh_mng_rd (kvm_tdx->tdr.pa,TDCS_EXEC (field), &out); +if (unlikely (err)) { +pr_err ("TDH_MNG_RD[EXEC.0x%x] failed: 0x%llx\n" , field, err); +return 0 ; + } +return out.r8; +} +#else static inline int tdx_module_setup (void ) {return -ENODEV ; };
arch/x86/kvm/x86.c +2 -1
@@ -2274,7 +2274,8 @@ static atomic_t kvm_guest_has_master_clock = ATOMIC_INIT(0);#endif static DEFINE_PER_CPU(unsigned long, cpu_tsc_khz); -static unsigned long max_tsc_khz; +unsigned long max_tsc_khz; +EXPORT_SYMBOL_GPL (max_tsc_khz);static u32 adjust_tsc_khz (u32 khz,s32 ppm) {
arch/x86/kvm/x86.h +2 -0
@@ -302,6 +302,8 @@ extern int pi_inject_timer;extern bool report_ignored_msrs; +extern unsigned long max_tsc_khz; +static inline u64 nsec_to_cycles (struct kvm_vcpu *vcpu,u64 nsec) { return pvclock_scale_delta(nsec, vcpu->arch.virtual_tsc_mult,
tools/arch/x86/include/uapi/asm/kvm.h +12 -0
@@ -531,6 +531,7 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM ,KVM_TDX_CMD_NR_MAX , };
@@ -561,4 +562,15 @@ struct kvm_tdx_capabilities { struct kvm_tdx_cpuid_config cpuid_configs[0]; }; +struct kvm_tdx_init_vm { +__u32 max_vcpus; +__u32 tsc_khz; +__u64 attributes; +__u64 cpuid; +__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha348 digest */ +__u64 reserved[43 ];/* must be zero for future extensibility */ +}; +#endif /* _ASM_X86_KVM_H */
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 57 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Xiaoyao Li <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect its confidentiality of its memory contents and its CPU state from any other software, including VMM. When creating guest TD VM before creating vcpu, the number of vcpu, TSC frequency (that is same among vcpus. and it can't be changed.) CPUIDs which is emulated by the TDX module. It means guest can trust those CPUIDs. and sha384 values for measurement.
Add new subcommand,
KVM_TDX_INIT_VM, to pass parameters for TDX guest. It assigns encryption key to the TDX guest for memory encryption. TDX encrypts memory per-guest bases. It assigns Device model passes per-VM parameters for the TDX guest. The maximum number of vcpus, tsc frequency (TDX guest has fised VM-wide TSC frequency. not per-vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which is reflected into guest XCR0,IA32_XSSMSR), cpuids, sha384 measurements, and etc.Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/include/uapi/asm/kvm.h | 12 ++ arch/x86/kvm/vmx/tdx.c | 200 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 26 ++++ arch/x86/kvm/x86.c | 3 +- arch/x86/kvm/x86.h | 2 + tools/arch/x86/include/uapi/asm/kvm.h | 12 ++ 7 files changed, 256 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -1234,6 +1234,8 @@ struct kvm_arch {hpa_t hv_root_tdp;spinlock_t hv_root_tdp_lock;#endif + +gfn_t gfn_shared_mask; };struct kvm_vm_stat {arch/x86/include/uapi/asm/kvm.h +1
-0@@ -531,6 +531,7 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM ,KVM_TDX_CMD_NR_MAX , };
arch/x86/include/uapi/asm/kvm.h struct kvm_tdx_capabilities { · line 569
Quoted 8 lines
arch/x86/include/uapi/asm/kvm.h +5
-0@@ -561,4 +562,15 @@ struct kvm_tdx_capabilities {struct kvm_tdx_cpuid_config cpuid_configs[0 ]; }; +struct kvm_tdx_init_vm { +__u32 max_vcpus; +__u32 tsc_khz; +__u64 attributes; +__u64 cpuid;
Is it better to append all CPUIDs directly into this structure, perhaps at end of this structure, to make it more consistent with TD_PARAMS?
Also, I think somewhere in commit message or comments we should explain why CPUIDs are passed here (why existing KVM_SET_CUPID2 is not sufficient).
Quoted 48 lines
arch/x86/include/uapi/asm/kvm.h +6
-0+__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha348 digest */ + __u64 reserved[43]; /* must be zero for future extensibility */ +}; +#endif /* _ASM_X86_KVM_H */ arch/x86/kvm/vmx/tdx.c +33
-0@@ -387,6 +387,203 @@ static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd)return 0 ; } +static struct kvm_cpuid_entry2 *tdx_find_cpuid_entry (struct kvm_tdx *kvm_tdx, +u32 function,u32 index) +{ +struct kvm_cpuid_entry2 *e; +int i; + +for (i =0 ; i < kvm_tdx->cpuid_nent; i++) { + e = &kvm_tdx->cpuid_entries[i]; + +if (e->function == function && (e->index == index || + !(e->flags &KVM_CPUID_FLAG_SIGNIFCANT_INDEX ))) +return e; + } +return NULL ; +} + +static int setup_tdparams (struct kvm *kvm,struct td_params *td_params, +struct kvm_tdx_init_vm *init_vm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +struct tdx_cpuid_config *config; +struct kvm_cpuid_entry2 *entry; +struct tdx_cpuid_value *value; +u64 guest_supported_xcr0; +u64 guest_supported_xss; +u32 guest_tsc_khz; +int max_pa; +int i; + +/* init_vm->reserved must be zero */ +if (find_first_bit ((unsigned long *)init_vm->reserved, +sizeof (init_vm->reserved) *8 ) != +sizeof (init_vm->reserved) *8 )
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 432
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+return -EINVAL ; + + td_params->max_vcpus = init_vm->max_vcpus; + + td_params->attributes = init_vm->attributes; +if (td_params->attributes &TDX_TD_ATTRIBUTE_PERFMON ) { +pr_warn ("TD doesn't support perfmon. KVM needs to save/restore " +"host perf registers properly.\n" ); +return -EOPNOTSUPP ; + }
PERFMON can be supported but it's not support in this series, so perhaps add a comment to explain it's a TODO?
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 434
Quoted 2 lines
arch/x86/kvm/vmx/tdx.c +2
-0+ +/* TODO: Enforce consistent CPUID features for all vCPUs. */
I guess you have to enforce when you do KVM_SET_CPUID2 after vcpu is created? Then I guess this comment shouldn't be here, because the enforcement isn't something you can do here in setup_tdparams().
Quoted 21 lines
arch/x86/kvm/vmx/tdx.c +21
-0+for (i =0 ; i < tdx_caps.nr_cpuid_configs; i++) { + config = &tdx_caps.cpuid_configs[i]; + + entry =tdx_find_cpuid_entry (kvm_tdx, config->leaf, + config->sub_leaf); +if (!entry) +continue ; + +/* +* Non-configurable bits must be '0', even if they are fixed to +* '1' by the TDX module, i.e. mask off non-configurable bits. +*/ + value = &td_params->cpuid_values[i]; + value->eax = entry->eax & config->eax; + value->ebx = entry->ebx & config->ebx; + value->ecx = entry->ecx & config->ecx; + value->edx = entry->edx & config->edx; + } + + max_pa =36 ; + entry =tdx_find_cpuid_entry (kvm_tdx,0x80000008 ,0 );
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 465
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+if (entry) + max_pa = entry->eax &0xff ; + + td_params->eptp_controls =VMX_EPTP_MT_WB ; +if (cpu_has_vmx_ept_5levels () && max_pa >48 ) { + td_params->eptp_controls |=VMX_EPTP_PWL_5 ; + td_params->exec_controls |=TDX_EXEC_CONTROL_MAX_GPAW ; + }else { + td_params->eptp_controls |=VMX_EPTP_PWL_4 ; + }
Not quite sure, but could we support >48 GPA with 4-level EPT?
Paolo Bonzini
maintainerRe: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 9 lines · Kai Huang
arch/x86/include/uapi/asm/kvm.h struct kvm_tdx_capabilities { · line 569
Quoted 6 lines
arch/x86/include/uapi/asm/kvm.h +5
-0+struct kvm_tdx_init_vm { +__u32 max_vcpus; +__u32 tsc_khz; +__u64 attributes; +__u64 cpuid;Is it better to append all CPUIDs directly into this structure, perhaps at end of this structure, to make it more consistent with
TD_PARAMS?
Quoted 3 lines
Also, I think somewhere in commit message or comments we should explain why CPUIDs are passed here (why existing
KVM_SET_CUPID2is not sufficient).
Indeed, it would be easier to use the existing cpuid data in struct kvm_vcpu, because right now there is no way to ensure that they are consistent.
Why is KVM_SET_CPUID2 not enough? Are there any modifications done by KVM that affect the measurement?
Thanks,
Paolo
Re: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 9 lines · Paolo Bonzini
Quoted 8 lines · Kai Huang
Quoted 5 lines
+struct
kvm_tdx_init_vm{
+__u32max_vcpus;
+__u32tsc_khz;
+__u64attributes;
+__u64cpuid;Is it better to append all CPUIDs directly into this structure, perhaps at end of this structure, to make it more consistent with
TD_PARAMS?
Quoted 7 lines
Quoted 3 lines
Also, I think somewhere in commit message or comments we should explain why CPUIDs are passed here (why existing
KVM_SET_CUPID2is not sufficient).Indeed, it would be easier to use the existing cpuid data in struct
kvm_vcpu, because right now there is no way to ensure that they are consistent.Why is
KVM_SET_CPUID2not enough? Are there any modifications done by KVM that affect the measurement?
Then we get the situation that KVM_TDX_INIT_VM must be called after 1 vcpu is created. It seems illogical that it has chance to fail the VM scope initialization after 1 vcpu is successfully created.
Quoted 4 lines
Thanks,
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 6 lines · Xiaoyao Li
Quoted 4 lines
Indeed, it would be easier to use the existing cpuid data in struct
kvm_vcpu, because right now there is no way to ensure that they are consistent.Why is
KVM_SET_CPUID2not enough? Are there any modifications done by KVM that affect the measurement?Then we get the situation that
KVM_TDX_INIT_VMmust be called after 1 vcpu is created. It seems illogical that it has chance to fail the VM scope initialization after 1 vcpu is successfully created.
I see. Yeah, it makes sense to have the CPUID in KVM_TDX_INIT_VM then.
Paolo
Re: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
In reply to Kai HuangQuoted 25 lines · Kai Huang
Quoted 13 lines
arch/x86/include/uapi/asm/kvm.h +1
-0@@ -531,6 +531,7 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM ,KVM_TDX_CMD_NR_MAX , };arch/x86/include/uapi/asm/kvm.h struct kvm_tdx_capabilities { · line 569
Quoted 8 lines
arch/x86/include/uapi/asm/kvm.h +5
-0@@ -561,4 +562,15 @@ struct kvm_tdx_capabilities {struct kvm_tdx_cpuid_config cpuid_configs[0 ]; }; +struct kvm_tdx_init_vm { +__u32 max_vcpus; +__u32 tsc_khz; +__u64 attributes; +__u64 cpuid;Is it better to append all CPUIDs directly into this structure, perhaps at end of this structure, to make it more consistent with
TD_PARAMS?
Quoted 2 lines
Also, I think somewhere in commit message or comments we should explain why CPUIDs are passed here (why existing
KVM_SET_CUPID2is not sufficient).
Ok, let's change the data structure to match more with TD_PARAMS.
Quoted 46 lines
Quoted 46 lines
arch/x86/kvm/vmx/tdx.c +38
-0@@ -387,6 +387,203 @@ static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd)return 0 ; } +static struct kvm_cpuid_entry2 *tdx_find_cpuid_entry (struct kvm_tdx *kvm_tdx, +u32 function,u32 index) +{ +struct kvm_cpuid_entry2 *e; +int i; + +for (i =0 ; i < kvm_tdx->cpuid_nent; i++) { + e = &kvm_tdx->cpuid_entries[i]; + +if (e->function == function && (e->index == index || + !(e->flags &KVM_CPUID_FLAG_SIGNIFCANT_INDEX ))) +return e; + } +return NULL ; +} + +static int setup_tdparams (struct kvm *kvm,struct td_params *td_params, +struct kvm_tdx_init_vm *init_vm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +struct tdx_cpuid_config *config; +struct kvm_cpuid_entry2 *entry; +struct tdx_cpuid_value *value; +u64 guest_supported_xcr0; +u64 guest_supported_xss; +u32 guest_tsc_khz; +int max_pa; +int i; + +/* init_vm->reserved must be zero */ +if (find_first_bit ((unsigned long *)init_vm->reserved, +sizeof (init_vm->reserved) *8 ) != +sizeof (init_vm->reserved) *8 ) +return -EINVAL ; + + td_params->max_vcpus = init_vm->max_vcpus; + + td_params->attributes = init_vm->attributes;
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 432
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+if (td_params->attributes &TDX_TD_ATTRIBUTE_PERFMON ) { +pr_warn ("TD doesn't support perfmon. KVM needs to save/restore " +"host perf registers properly.\n" ); +return -EOPNOTSUPP ; + }PERFMON can be supported but it's not support in this series, so perhaps add a comment to explain it's a TODO?
Yes, good idea. Will do.
Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+ max_pa =36 ; + entry =tdx_find_cpuid_entry (kvm_tdx,0x80000008 ,0 ); +if (entry) + max_pa = entry->eax &0xff ; + + td_params->eptp_controls =VMX_EPTP_MT_WB ;
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 465
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+if (cpu_has_vmx_ept_5levels () && max_pa >48 ) { + td_params->eptp_controls |=VMX_EPTP_PWL_5 ; + td_params->exec_controls |=TDX_EXEC_CONTROL_MAX_GPAW ; + }else { + td_params->eptp_controls |=VMX_EPTP_PWL_4 ; + }Not quite sure, but could we support >48 GPA with 4-level EPT?
No.
"5-level paging and 5-level EPT"
section 4.1 4-level EPT
"4-level EPT is limited to translating 48-bit guest-physical addresses."
Signature · 1 line
Isaku Yamahata <[email protected]>
Paolo Bonzini
maintainerRe: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
In reply to [email protected]arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 432
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +6
-0+ td_params->attributes = init_vm->attributes; +if (td_params->attributes &TDX_TD_ATTRIBUTE_PERFMON ) { +pr_warn ("TD doesn't support perfmon. KVM needs to save/restore " +"host perf registers properly.\n" ); +return -EOPNOTSUPP ; + }
Why does KVM have to hardcode this (and LBR/AMX below)? Is the level of hardware support available from tdx_caps, for example through the CPUID configs (0xA for this one, 0xD for LBR and AMX)?
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+/* PT can be exposed to TD guest regardless of KVM's XSS support */ + guest_supported_xss &= (supported_xss |XFEATURE_MASK_PT ); + td_params->xfam = guest_supported_xcr0 | guest_supported_xss; +if (td_params->xfam &TDX_TD_XFAM_LBR ) { +pr_warn ("TD doesn't support LBR. KVM needs to save/restore " +"IA32_LBR_DEPTH properly.\n" ); +return -EOPNOTSUPP ; + } + +if (td_params->xfam &TDX_TD_XFAM_AMX ) {
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 499
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +8
-0+pr_warn ("TD doesn't support AMX. KVM needs to save/restore " +"IA32_XFD, IA32_XFD_ERR properly.\n" ); +return -EOPNOTSUPP ; + } +if (init_vm->tsc_khz) + guest_tsc_khz = init_vm->tsc_khz; +else + guest_tsc_khz = max_tsc_khz;
You can just use kvm->arch.default_tsc_khz in the latest kvm/queue.
arch/x86/kvm/vmx/tdx.c static int tdx_capabilities(struct kvm *kvm, struct kvm_tdx_cmd *cmd) · line 511
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+#define BUILD_BUG_ON_MEMCPY (dst, src) \ + do { \ + BUILD_BUG_ON(sizeof(dst) != sizeof(src)); \ + memcpy((dst), (src), sizeof(dst)); \ + } while (0) + +BUILD_BUG_ON_MEMCPY (td_params->mrconfigid, init_vm->mrconfigid); +BUILD_BUG_ON_MEMCPY (td_params->mrowner, init_vm->mrowner); +BUILD_BUG_ON_MEMCPY (td_params->mrownerconfig, init_vm->mrownerconfig); +
Please rename to MEMCPY_SAME_SIZE.
Thanks,
Paolo
Re: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 9 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
+
td_params->attributes=init_vm->attributes;
+ if (td_params->attributes&TDX_TD_ATTRIBUTE_PERFMON) { + pr_warn("TD doesn't support perfmon. KVM needs to save/restore " + "host perf registers properly.\n");
+ return-EOPNOTSUPP;
+ }Why does KVM have to hardcode this (and LBR/AMX below)? Is the level of hardware support available from
tdx_caps, for example through the CPUID configs (0xAfor this one,0xDfor LBR and AMX)?
It's wrong code. PMU is allowed.
AMX and LBR are disallowed because and the time we wrote the codes they are not supported by KVM. Now AMX should be allowed, but (arch-)LBR should be still blocked until KVM merges arch-LBR support.
Quoted 16 lines
Quoted 16 lines
+ /* PT can be exposed to TD guest regardless of KVM's XSS support */ +
guest_supported_xss&= (supported_xss|XFEATURE_MASK_PT); +td_params->xfam=guest_supported_xcr0|guest_supported_xss; + if (td_params->xfam&TDX_TD_XFAM_LBR) {
+ pr_warn("TD doesn't support LBR. KVM needs to save/restore " + "IA32_LBR_DEPTHproperly.\n");
+ return-EOPNOTSUPP;
+ }
+
+ if (td_params->xfam&TDX_TD_XFAM_AMX) {
+ pr_warn("TD doesn't support AMX. KVM needs to save/restore " + "IA32_XFD,IA32_XFD_ERRproperly.\n");
+ return-EOPNOTSUPP;
+ }
Quoted 6 lines
Quoted 4 lines
+ if (
init_vm->tsc_khz)
+guest_tsc_khz=init_vm->tsc_khz;
+ else
+guest_tsc_khz=max_tsc_khz;You can just use
kvm->arch.default_tsc_khzin the latest kvm/queue.
yes. will change it.
Re: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 13 lines · Xiaoyao Li
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
+
td_params->attributes=init_vm->attributes;
+ if (td_params->attributes&TDX_TD_ATTRIBUTE_PERFMON) { + pr_warn("TD doesn't support perfmon. KVM needs to save/restore " + "host perf registers properly.\n");
+ return-EOPNOTSUPP;
+ }Why does KVM have to hardcode this (and LBR/AMX below)? Is the level of hardware support available from
tdx_caps, for example through the CPUID configs (0xAfor this one,0xDfor LBR and AMX)?
Quoted 5 lines
It's wrong code. PMU is allowed.
AMX and LBR are disallowed because and the time we wrote the codes they are not supported by KVM. Now AMX should be allowed, but (arch-)LBR should be still blocked until KVM merges arch-LBR support.
I think Isaku's idea is we don't support them in the first submission?
If so as I suggested, we should add a TODO in comment..
Re: [RFC PATCH v5 027/104] KVM: TDX: initialize VM with TDX specific parameters
Quoted 16 lines · Kai Huang
Quoted 15 lines · Xiaoyao Li
Quoted 11 lines · Paolo Bonzini
Quoted 6 lines · On 3/4/22 20:48, [email protected] wrote:
+
td_params->attributes=init_vm->attributes;
+ if (td_params->attributes&TDX_TD_ATTRIBUTE_PERFMON) { + pr_warn("TD doesn't support perfmon. KVM needs to save/restore " + "host perf registers properly.\n");
+ return-EOPNOTSUPP;
+ }Why does KVM have to hardcode this (and LBR/AMX below)? Is the level of hardware support available from
tdx_caps, for example through the CPUID configs (0xAfor this one,0xDfor LBR and AMX)?It's wrong code. PMU is allowed.
Quoted 7 lines
Quoted 3 lines
AMX and LBR are disallowed because and the time we wrote the codes they are not supported by KVM. Now AMX should be allowed, but (arch-)LBR should be still blocked until KVM merges arch-LBR support.
I think Isaku's idea is we don't support them in the first submission?
If so as I suggested, we should add a TODO in comment..
Sure, will add a TODO comment.
Signature · 1 line
Isaku Yamahata <[email protected]>
RFC PATCH v5 28/104 The start of TDX KVM patch series: TD vcpu creation/destruction
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of TD vcpu creation/destruction.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +3 -3
@@ -9,15 +9,15 @@ Layer status What qemu can do ---------------- - TDX VM TYPE is exposed to Qemu. -- Qemu can try to create VM of TDX VM type and then fails. +- Qemu can create/destroy guest of TDX vm type. Patch Layer status ------------------ Patch layer Status * TDX, VMX coexistence: Applied * TDX architectural definitions: Applied -* TD VM creation/destruction: Applying -* TD vcpu creation/destruction: Not yet +* TD VM creation/destruction: Applied +* TD vcpu creation/destruction: Applying * TDX EPT violation: Not yet * TD finalization: Not yet * TD vcpu enter/exit: Not yet
Signature · 1 line
2.25.1
RFC PATCH v5 29/104 KVM: TDX: allocate/free TDX vcpu structure
In reply to [email protected]From: Isaku Yamahata <[email protected]>
The next step of TDX guest creation is to create vcpu. Allocate TDX vcpu structures, initialize it. Allocate pages of TDX vcpu for the TDX module.
In the case of the conventional case, cpuid is empty at the initialization. and cpuid is configured after the vcpu initialization. Because TDX supports only X2APIC mode, cpuid is forcibly initialized to support X2APIC on the vcpu initialization.
--- arch/x86/kvm/vmx/main.c | 30 +++++++- arch/x86/kvm/vmx/tdx.c | 142 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx_ops.h | 2 + arch/x86/kvm/vmx/x86_ops.h | 8 +++ arch/x86/kvm/x86.c | 3 +- arch/x86/kvm/x86.h | 1 + 6 files changed, 182 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/main.c +27 -3
@@ -65,6 +65,30 @@ static int vt_mem_enc_op(struct kvm *kvm, void __user *argp)return tdx_vm_ioctl (kvm, argp); } +static int vt_vcpu_create (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return tdx_vcpu_create (vcpu); + +return vmx_vcpu_create (vcpu); +} + +static void vt_vcpu_free (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return tdx_vcpu_free (vcpu); + +return vmx_vcpu_free (vcpu); +} + +static void vt_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +{ +if (is_td_vcpu (vcpu)) +return tdx_vcpu_reset (vcpu, init_event); + +return vmx_vcpu_reset (vcpu, init_event); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,
@@ -81,9 +105,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .mmu_prezap = vt_mmu_prezap, .vm_free = vt_vm_free, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, + .vcpu_create = vt_vcpu_create, + .vcpu_free = vt_vcpu_free, + .vcpu_reset = vt_vcpu_reset, .prepare_guest_switch = vmx_prepare_switch_to_guest, .vcpu_load = vmx_vcpu_load,
arch/x86/kvm/vmx/tdx.c +142 -0
160 lines not on this page · raw message
arch/x86/kvm/vmx/tdx_ops.h +2 -0
@@ -41,6 +41,7 @@ static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page,static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) { +tdx_clflush_page (addr);return kvm_seamcall (TDH_VP_ADDCX , addr, tdvpr,0 ,0 ,0 ,NULL ); }
@@ -69,6 +70,7 @@ static inline u64 tdh_mng_create(hpa_t tdr, int hkid)static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) { +tdx_clflush_page (tdvpr);return kvm_seamcall (TDH_VP_CREATE , tdvpr, tdr,0 ,0 ,0 ,NULL ); }
arch/x86/kvm/vmx/x86_ops.h +8 -0
@@ -137,6 +137,10 @@ int tdx_vm_init(struct kvm *kvm);void tdx_mmu_prezap (struct kvm *kvm);void tdx_vm_free (struct kvm *kvm); +int tdx_vcpu_create (struct kvm_vcpu *vcpu); +void tdx_vcpu_free (struct kvm_vcpu *vcpu); +void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline void tdx_pre_kvm_init(
@@ -149,6 +153,10 @@ static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; }static inline void tdx_mmu_prezap (struct kvm *kvm) {}static inline void tdx_vm_free (struct kvm *kvm) {} +static inline int tdx_vcpu_create (struct kvm_vcpu *vcpu) {return -EOPNOTSUPP ; } +static inline void tdx_vcpu_free (struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {} +static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif
arch/x86/kvm/x86.c +2 -1
@@ -2322,7 +2322,7 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale)return 0 ; } -static int kvm_set_tsc_khz (struct kvm_vcpu *vcpu,u32 user_tsc_khz) +int kvm_set_tsc_khz (struct kvm_vcpu *vcpu,u32 user_tsc_khz) {u32 thresh_lo, thresh_hi;int use_scaling =0 ;
@@ -2354,6 +2354,7 @@ static int kvm_set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz) }return set_tsc_khz (vcpu, user_tsc_khz, use_scaling); } +EXPORT_SYMBOL_GPL (kvm_set_tsc_khz);static u64 compute_guest_tsc (struct kvm_vcpu *vcpu,s64 kernel_ns) {
arch/x86/kvm/x86.h +1 -0
@@ -303,6 +303,7 @@ extern int pi_inject_timer;extern bool report_ignored_msrs;extern unsigned long max_tsc_khz; +int kvm_set_tsc_khz (struct kvm_vcpu *vcpu,u32 user_tsc_khz);static inline u64 nsec_to_cycles (struct kvm_vcpu *vcpu,u64 nsec) {
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 029/104] KVM: TDX: allocate/free TDX vcpu structure
arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 443
Quoted 10 lines · On 3/4/22 20:48, [email protected] wrote:
arch/x86/kvm/vmx/tdx.c +10
-0+/* +* In TDX case, tsc frequency is per-VM and determined by the parameter +* tdh_mng_init(). Forcibly set it instead of max_tsc_khz set by +* kvm_arch_vcpu_create(). +* +* This function is called after kvm_arch_vcpu_create() calling +* kvm_set_tsc_khz(). +*/ +kvm_set_tsc_khz (vcpu, kvm_tdx->tsc_khz); +
I think this is not needed anymore, now that there is kvm->arch.default_tsc_khz. If so, exporting the function is not needed either.
Paolo
RFC PATCH v5 30/104 KVM: TDX: Do TDX specific vcpu initialization
In reply to [email protected]From: Sean Christopherson <[email protected]>
TD guest vcpu need to be configured before ready to run which requests addtional information from Device model (e.g. qemu), one 64bit value is passed to vcpu's RCX as an initial value. Repurpose KVM_MEMORY_ENCRYPT_OP to vcpu-scope and add new sub-commands KVM_TDX_INIT_VCPU under it for such additional vcpu configuration.
Add callback for kvm vCPU-scoped operations of KVM_MEMORY_ENCRYPT_OP and add a new subcommand, KVM_TDX_INIT_VCPU, for further vcpu initialization.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/vmx/main.c | 25 +++++++++++++------- arch/x86/kvm/vmx/tdx.c | 34 +++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 4 ++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ arch/x86/kvm/x86.c | 6 +++++ tools/arch/x86/include/uapi/asm/kvm.h | 1 + 8 files changed, 66 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1483,6 +1483,7 @@ struct kvm_x86_ops {void (*enable_smi_window)(struct kvm_vcpu *vcpu);int (*mem_enc_op)(struct kvm *kvm,void __user *argp); +int (*mem_enc_op_vcpu)(struct kvm_vcpu *vcpu,void __user *argp);int (*mem_enc_reg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*mem_enc_unreg_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*vm_copy_enc_context_from)(struct kvm *kvm,unsigned int source_fd);
arch/x86/include/uapi/asm/kvm.h +1 -0
@@ -532,6 +532,7 @@ struct kvm_pmu_event_filter {enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 ,KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU ,KVM_TDX_CMD_NR_MAX , };
arch/x86/kvm/vmx/main.c +17 -8
@@ -57,14 +57,6 @@ static void vt_vm_free(struct kvm *kvm)return tdx_vm_free (kvm); } -static int vt_mem_enc_op (struct kvm *kvm,void __user *argp) -{ -if (!is_td (kvm)) -return -ENOTTY ; - -return tdx_vm_ioctl (kvm, argp); -} -static int vt_vcpu_create (struct kvm_vcpu *vcpu) {if (is_td_vcpu (vcpu))
@@ -89,6 +81,22 @@ static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)return vmx_vcpu_reset (vcpu, init_event); } +static int vt_mem_enc_op (struct kvm *kvm,void __user *argp) +{ +if (!is_td (kvm)) +return -ENOTTY ; + +return tdx_vm_ioctl (kvm, argp); +} + +static int vt_mem_enc_op_vcpu (struct kvm_vcpu *vcpu,void __user *argp) +{ +if (!is_td_vcpu (vcpu)) +return -EINVAL ; + +return tdx_vcpu_ioctl (vcpu, argp); +} +struct kvm_x86_ops vt_x86_ops__initdata = { .name ="kvm_intel" ,
@@ -229,6 +237,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, .mem_enc_op = vt_mem_enc_op, + .mem_enc_op_vcpu = vt_mem_enc_op_vcpu, };struct kvm_x86_init_ops vt_init_ops__initdata = {
arch/x86/kvm/vmx/tdx.c +34 -0
@@ -73,6 +73,11 @@ static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)return kvm_tdx->hkid >0 ; } +static inline bool is_td_finalized (struct kvm_tdx *kvm_tdx) +{ +return kvm_tdx->finalized; +} +static void tdx_clear_page (unsigned long page) {const void *zero_page = (const void *)__va (page_to_phys (ZERO_PAGE (0 )));
@@ -756,6 +761,35 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)return r; } +int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); +struct vcpu_tdx *tdx =to_tdx (vcpu); +struct kvm_tdx_cmd cmd; +u64 err; + +if (tdx->initialized) +return -EINVAL ; + +if (!is_td_initialized (vcpu->kvm) ||is_td_finalized (kvm_tdx)) +return -EINVAL ; + +if (copy_from_user (&cmd, argp,sizeof (cmd))) +return -EFAULT ; + +if (cmd.metadata || cmd.id !=KVM_TDX_INIT_VCPU ) +return -EINVAL ; + + err =tdh_vp_init (tdx->tdvpr.pa, cmd.data); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_VP_INIT , err,NULL ); +return -EIO ; + } + + tdx->initialized = true; +return 0 ; +} +static int __tdx_module_setup (void ) {const struct tdsysinfo_struct *tdsysinfo;
arch/x86/kvm/vmx/tdx.h +4 -0
@@ -27,6 +27,8 @@ struct kvm_tdx {int cpuid_nent;struct kvm_cpuid_entry2 cpuid_entries[KVM_MAX_CPUID_ENTRIES ]; +bool finalized; +u64 tsc_offset;unsigned long tsc_khz; };
@@ -36,6 +38,8 @@ struct vcpu_tdx {struct tdx_td_page tdvpr;struct tdx_td_page *tdvpx; + +bool initialized; };static inline bool is_td (struct kvm *kvm)
arch/x86/kvm/vmx/x86_ops.h +2 -0
@@ -142,6 +142,7 @@ void tdx_vcpu_free(struct kvm_vcpu *vcpu);void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp); +int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp);#else static inline void tdx_pre_kvm_init (unsigned int *vcpu_size,unsigned int *vcpu_align,unsigned int *vm_size) {}
@@ -158,6 +159,7 @@ static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {}static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {}static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; } +static inline int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp) {return -EOPNOTSUPP ; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
arch/x86/kvm/x86.c +6 -0
@@ -5680,6 +5680,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_SET_DEVICE_ATTR : r =kvm_vcpu_ioctl_device_attr (vcpu, ioctl, argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -EINVAL ; +if (!kvm_x86_ops.mem_enc_op_vcpu) + goto out; + r = kvm_x86_ops.mem_enc_op_vcpu (vcpu, argp); +break ;default : r = -EINVAL ; }
tools/arch/x86/include/uapi/asm/kvm.h +1 -0
@@ -532,6 +532,7 @@ struct kvm_pmu_event_filter {enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 ,KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU ,KVM_TDX_CMD_NR_MAX , };
Signature · 1 line
2.25.1
RFC PATCH v5 31/104 The start of TDX KVM patch series: KVM MMU GPA stolen bits
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of KVM MMU GPA stolen bits.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +3 -2
@@ -10,6 +10,7 @@ What qemu can do ---------------- - TDX VM TYPE is exposed to Qemu. - Qemu can create/destroy guest of TDX vm type. +- Qemu can create/destroy vcpu of TDX vm type. Patch Layer status ------------------
@@ -17,13 +18,13 @@ Patch Layer status * TDX, VMX coexistence: Applied * TDX architectural definitions: Applied * TD VM creation/destruction: Applied -* TD vcpu creation/destruction: Applying +* TD vcpu creation/destruction: Applied * TDX EPT violation: Not yet * TD finalization: Not yet * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet -* KVM MMU GPA stolen bits: Not yet +* KVM MMU GPA stolen bits: Applying * KVM TDP refactoring for TDX: Not yet * KVM TDP MMU hooks: Not yet * KVM TDP MMU MapGPA: Not yet
Signature · 1 line
2.25.1
RFC PATCH v5 32/104 KVM: x86/mmu: introduce config for PRIVATE KVM MMU
In reply to [email protected]From: Isaku Yamahata <[email protected]>
To Keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for CONFIG_INTEL_TDX_HOST && CONFIG_KVM_INTEL. The new flag make it clear that the config is only for x86 KVM MMU.
--- arch/x86/kvm/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/kvm/Kconfig +4 -0
@@ -136,4 +136,8 @@ config KVM_MMU_AUDIT config KVM_EXTERNAL_WRITE_TRACKING bool +config KVM_MMU_PRIVATE + def_bool y + depends on INTEL_TDX_HOST && KVM_INTEL + endif # VIRTUALIZATION
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 032/104] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 17 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To Keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for
CONFIG_INTEL_TDX_HOST&&CONFIG_KVM_INTEL. The new flag make it clear that the config is only for x86 KVM MMU.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/Kconfig +0
-0
arch/x86/kvm/Kconfig config KVM_MMU_AUDIT · line 143
Quoted 8 lines
arch/x86/kvm/Kconfig +4
-0@@ -136,4 +136,8 @@ config KVM_MMU_AUDIT config KVM_EXTERNAL_WRITE_TRACKING bool +config KVM_MMU_PRIVATE + def_bool y + depends on INTEL_TDX_HOST && KVM_INTEL + endif # VIRTUALIZATION
I am really not sure why need this. Roughly looking at MMU related patches this new config option is hardly used. You have many code changes related to handling private/shared but they are not under this config option.
Signature · 2 lines
Thanks, -Kai
Re: [RFC PATCH v5 032/104] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 21 lines · Kai Huang
Quoted 20 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To Keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for
CONFIG_INTEL_TDX_HOST&&CONFIG_KVM_INTEL. The new flag make it clear that the config is only for x86 KVM MMU.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/Kconfig +0
-0@@ -136,4 +136,8 @@ config KVM_MMU_AUDIT config KVM_EXTERNAL_WRITE_TRACKING bool
Quoted 9 lines
arch/x86/kvm/Kconfig config KVM_MMU_AUDIT · line 143
Quoted 5 lines
arch/x86/kvm/Kconfig +4
-0+config KVM_MMU_PRIVATE + def_bool y + depends on INTEL_TDX_HOST && KVM_INTEL + endif # VIRTUALIZATIONI am really not sure why need this. Roughly looking at MMU related patches this new config option is hardly used. You have many code changes related to handling private/shared but they are not under this config option.
I don't want to use CONFIG_INTEL_TDX_HOST in KVM MMU code. I think the change to KVM MMU should be a sort of independent from TDX. But it seems failed based on your feedback.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [RFC PATCH v5 032/104] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 34 lines · Isaku Yamahata
Quoted 21 lines · Kai Huang
Quoted 20 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To Keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for
CONFIG_INTEL_TDX_HOST&&CONFIG_KVM_INTEL. The new flag make it clear that the config is only for x86 KVM MMU.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/Kconfig +0
-0@@ -136,4 +136,8 @@ config KVM_MMU_AUDIT config KVM_EXTERNAL_WRITE_TRACKING boolQuoted 9 lines
arch/x86/kvm/Kconfig config KVM_MMU_AUDIT · line 143
Quoted 5 lines
arch/x86/kvm/Kconfig +4
-0+config KVM_MMU_PRIVATE + def_bool y + depends on INTEL_TDX_HOST && KVM_INTEL + endif # VIRTUALIZATIONI am really not sure why need this. Roughly looking at MMU related patches this new config option is hardly used. You have many code changes related to handling private/shared but they are not under this config option.
Quoted 3 lines
I don't want to use
CONFIG_INTEL_TDX_HOSTin KVM MMU code. I think the change to KVM MMU should be a sort of independent from TDX. But it seems failed based on your feedback.
Why do you need to use any config? As I said majority of your changes to MMU are not under any config. But I'll leave this to maintainer/reviewers.
Signature · 2 lines
Thanks, -Kai
Paolo Bonzini
maintainerRe: [RFC PATCH v5 032/104] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 6 lines · Kai Huang
Quoted 3 lines
I don't want to use
CONFIG_INTEL_TDX_HOSTin KVM MMU code. I think the change to KVM MMU should be a sort of independent from TDX. But it seems failed based on your feedback.Why do you need to use any config? As I said majority of your changes to MMU are not under any config. But I'll leave this to maintainer/reviewers.
There are few uses, but the effect should be pretty large, because the config symbol replaces variable accesses with constants:
+static inline gfn_t kvm_gfn_stolen_mask(struct kvm *kvm) +{
+#ifdef CONFIG_KVM_MMU_PRIVATE
+ return kvm->arch.gfn_shared_mask;
+#else
+ return 0;
+#endif
+}
Please keep it.
Paolo
RFC PATCH v5 33/104 KVM: x86: Add infrastructure for stolen GPA bits
In reply to [email protected]From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA aliasing will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.
With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
- Specify physical memory by userspace via memslots
- Map GPAs to TDP PTEs via RMAP
- Specify dirty tracking and write protection
- Look up MTRR types
- Inject async page faults
Since there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately, this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN, kvm_handle_hva() based operations will be applied to both aliases.
In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the case of stolen bits.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 51 +++++++++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu.c | 19 ++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 25 +++++++++------- 4 files changed, 84 insertions(+), 13 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1235,7 +1235,9 @@ struct kvm_arch {spinlock_t hv_root_tdp_lock;#endif +#ifdef CONFIG_KVM_MMU_PRIVATE gfn_t gfn_shared_mask; +#endif };struct kvm_vm_stat {
arch/x86/kvm/mmu.h +51 -0
55 lines not on this page · raw message
arch/x86/kvm/mmu/mmu.c +16 -3
@@ -276,11 +276,24 @@ static inline bool kvm_available_flush_tlb_with_range(void)static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm,struct kvm_tlb_range *range) { -int ret = -ENOTSUPP ; +int ret = -EOPNOTSUPP ; +u64 gfn_stolen_mask; -if (range && kvm_x86_ops.tlb_remote_flush_with_range) +/* +* Fall back to the big hammer flush if there is more than one +* GPA alias that needs to be flushed. +*/ + gfn_stolen_mask =kvm_gfn_stolen_mask (kvm); +if (hweight64 (gfn_stolen_mask) >1 ) + goto generic_flush; + +if (range &&kvm_available_flush_tlb_with_range ()) { +/* Callback should flush both private GFN and shared GFN. */ + range->start_gfn =kvm_gfn_unalias (kvm, range->start_gfn); ret =static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, range); + } +generic_flush:if (ret)kvm_flush_remote_tlbs (kvm); }
@@ -4010,7 +4023,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultunsigned long mmu_seq;int r; - fault->gfn = fault->addr >>PAGE_SHIFT ; + fault->gfn =kvm_gfn_unalias (vcpu->kvm,gpa_to_gfn (fault->addr)); fault->slot =kvm_vcpu_gfn_to_memslot (vcpu, fault->gfn);if (page_fault_handle_page_track (vcpu, fault))
arch/x86/kvm/mmu/paging_tmpl.h +15 -10
@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~kvm_gpa_stolen_mask(vcpu->kvm) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS
@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS
@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page
@@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,
@@ -395,7 +396,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;
@@ -460,7 +461,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())
@@ -555,12 +556,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &kvm_gpa_stolen_mask (vcpu->kvm)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
@@ -656,6 +659,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,WARN_ON_ONCE (gw->gfn != base_gfn); direct_access = gw->pte_access; +WARN_ON (fault->addr &kvm_gpa_stolen_mask (vcpu->kvm)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;
@@ -1080,7 +1085,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; } - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 122 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA aliasing will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
- Specify physical memory by userspace via memslots
- Map GPAs to TDP PTEs via RMAP
- Specify dirty tracking and write protection
- Look up MTRR types
- Inject async page faultsSince there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately, this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN,
kvm_handle_hva()based operations will be applied to both aliases.In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the case of stolen bits.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
Signed-off-by Rick Edgecombe <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 51 +++++++++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu.c | 19 ++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 25 +++++++++------- 4 files changed, 84 insertions(+), 13 deletions(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -1235,7 +1235,9 @@ struct kvm_arch {spinlock_t hv_root_tdp_lock;#endif +#ifdef CONFIG_KVM_MMU_PRIVATE gfn_t gfn_shared_mask; +#endif };struct kvm_vm_stat {arch/x86/kvm/mmu.h +41
-0@@ -365,4 +365,55 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu,return gpa;return translate_nested_gpa (vcpu, gpa, access, exception); } + +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_MMU_PRIVATE +return kvm->arch.gfn_shared_mask; +#else +return 0 ; +#endif +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return gfn_to_gpa (kvm_gfn_stolen_mask (kvm)); +} + +static inline gpa_t kvm_gpa_unalias (struct kvm *kvm,gpa_t gpa) +{ +return gpa & ~kvm_gpa_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_unalias (struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_shared (struct kvm *kvm,gfn_t gfn) +{ +return gfn |kvm_gfn_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_private (struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_stolen_mask (kvm); +} + +static inline gpa_t kvm_gpa_private (struct kvm *kvm,gpa_t gpa) +{ +return gpa & ~kvm_gpa_stolen_mask (kvm); +} + +static inline bool kvm_is_private_gfn (struct kvm *kvm,gfn_t gfn)
arch/x86/kvm/mmu.h static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, · line 418
Quoted 10 lines
arch/x86/kvm/mmu.h +10
-0+{ +gfn_t mask =kvm_gfn_stolen_mask (kvm); + +return mask && !(gfn & mask); +} + +static inline bool kvm_is_private_gpa (struct kvm *kvm,gpa_t gpa) +{ +return kvm_is_private_gfn (kvm,gpa_to_gfn (gpa)); +}
The patch title and commit message say nothing about private/shared, but only mention stolen bits in general. It's weird to introduce those *private* related helpers here.
I think you can just ditch the concept of stolen bit infrastructure, but just adopt what TDX needs.
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 279
Quoted 11 lines
arch/x86/kvm/mmu.h +0
-0#endif arch/x86/kvm/mmu/mmu.c +1
-1@@ -276,11 +276,24 @@ static inline bool kvm_available_flush_tlb_with_range(void)static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm,struct kvm_tlb_range *range) { -int ret = -ENOTSUPP ; +int ret = -EOPNOTSUPP ;
Change doesn't belong to this patch.
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +2
-1+u64 gfn_stolen_mask; -if (range && kvm_x86_ops.tlb_remote_flush_with_range) +/*
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 292
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +10
-0+* Fall back to the big hammer flush if there is more than one +* GPA alias that needs to be flushed. +*/ + gfn_stolen_mask =kvm_gfn_stolen_mask (kvm); +if (hweight64 (gfn_stolen_mask) >1 ) + goto generic_flush; + +if (range &&kvm_available_flush_tlb_with_range ()) { +/* Callback should flush both private GFN and shared GFN. */ + range->start_gfn =kvm_gfn_unalias (kvm, range->start_gfn);
This seems wrong. It seems the intention of this function is to flush TLB for all aliases for a given GFN range. Here it seems you are unconditionally change to range to always exclude the stolen bits.
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 294
Quoted 2 lines
arch/x86/kvm/mmu/mmu.c +1
-0ret =static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, range); + }
And you always fall through to do big hammer flush, which is obviously not intended.
Quoted 111 lines
arch/x86/kvm/mmu/mmu.c +2
-1+generic_flush:if (ret)kvm_flush_remote_tlbs (kvm); }@@ -4010,7 +4023,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultunsigned long mmu_seq;int r; - fault->gfn = fault->addr >>PAGE_SHIFT ; + fault->gfn =kvm_gfn_unalias (vcpu->kvm,gpa_to_gfn (fault->addr)); fault->slot =kvm_vcpu_gfn_to_memslot (vcpu, fault->gfn);if (page_fault_handle_page_track (vcpu, fault))arch/x86/kvm/mmu/paging_tmpl.h +14
-9@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~kvm_gpa_stolen_mask(vcpu->kvm) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page @@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,@@ -395,7 +396,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;@@ -460,7 +461,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())@@ -555,12 +556,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &kvm_gpa_stolen_mask (vcpu->kvm)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);@@ -656,6 +659,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,WARN_ON_ONCE (gw->gfn != base_gfn); direct_access = gw->pte_access; +WARN_ON (fault->addr &kvm_gpa_stolen_mask (vcpu->kvm)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) · line 1091
Quoted 8 lines
arch/x86/kvm/mmu/paging_tmpl.h +1
-1@@ -1080,7 +1085,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; } - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);
In commit message you mentioned "Don't support stolen bits for shadow EPT" (you actually mean shadow MMU I suppose), yet there's bunch of code change to shadow MMU.
Re: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 190 lines · Kai Huang
Quoted 122 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA aliasing will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
- Specify physical memory by userspace via memslots
- Map GPAs to TDP PTEs via RMAP
- Specify dirty tracking and write protection
- Look up MTRR types
- Inject async page faultsSince there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately, this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN,
kvm_handle_hva()based operations will be applied to both aliases.In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the case of stolen bits.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
Signed-off-by Rick Edgecombe <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 51 +++++++++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu.c | 19 ++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 25 +++++++++------- 4 files changed, 84 insertions(+), 13 deletions(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -1235,7 +1235,9 @@ struct kvm_arch {spinlock_t hv_root_tdp_lock;#endif +#ifdef CONFIG_KVM_MMU_PRIVATE gfn_t gfn_shared_mask; +#endif };struct kvm_vm_stat {arch/x86/kvm/mmu.h +41
-0@@ -365,4 +365,55 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu,return gpa;return translate_nested_gpa (vcpu, gpa, access, exception); } + +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_MMU_PRIVATE +return kvm->arch.gfn_shared_mask; +#else +return 0 ; +#endif +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return gfn_to_gpa (kvm_gfn_stolen_mask (kvm)); +} + +static inline gpa_t kvm_gpa_unalias (struct kvm *kvm,gpa_t gpa) +{ +return gpa & ~kvm_gpa_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_unalias (struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_shared (struct kvm *kvm,gfn_t gfn) +{ +return gfn |kvm_gfn_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_private (struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_stolen_mask (kvm); +} + +static inline gpa_t kvm_gpa_private (struct kvm *kvm,gpa_t gpa) +{ +return gpa & ~kvm_gpa_stolen_mask (kvm); +} + +static inline bool kvm_is_private_gfn (struct kvm *kvm,gfn_t gfn)arch/x86/kvm/mmu.h static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, · line 418
Quoted 10 lines
arch/x86/kvm/mmu.h +10
-0+{ +gfn_t mask =kvm_gfn_stolen_mask (kvm); + +return mask && !(gfn & mask); +} + +static inline bool kvm_is_private_gpa (struct kvm *kvm,gpa_t gpa) +{ +return kvm_is_private_gfn (kvm,gpa_to_gfn (gpa)); +}The patch title and commit message say nothing about private/shared, but only mention stolen bits in general. It's weird to introduce those *private* related helpers here.
I think you can just ditch the concept of stolen bit infrastructure, but just adopt what TDX needs.
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 279
Quoted 11 lines
arch/x86/kvm/mmu.h +0
-0#endif arch/x86/kvm/mmu/mmu.c +1
-1@@ -276,11 +276,24 @@ static inline bool kvm_available_flush_tlb_with_range(void)static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm,struct kvm_tlb_range *range) { -int ret = -ENOTSUPP ; +int ret = -EOPNOTSUPP ;Change doesn't belong to this patch.
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +2
-1+u64 gfn_stolen_mask; -if (range && kvm_x86_ops.tlb_remote_flush_with_range) +/* arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 292
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +10
-0+* Fall back to the big hammer flush if there is more than one +* GPA alias that needs to be flushed. +*/ + gfn_stolen_mask =kvm_gfn_stolen_mask (kvm); +if (hweight64 (gfn_stolen_mask) >1 ) + goto generic_flush; + +if (range &&kvm_available_flush_tlb_with_range ()) { +/* Callback should flush both private GFN and shared GFN. */ + range->start_gfn =kvm_gfn_unalias (kvm, range->start_gfn);This seems wrong. It seems the intention of this function is to flush TLB for all aliases for a given GFN range. Here it seems you are unconditionally change to range to always exclude the stolen bits.
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 294
Quoted 2 lines
arch/x86/kvm/mmu/mmu.c +1
-0ret =static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, range); + }And you always fall through to do big hammer flush, which is obviously not intended.
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +1
-0+generic_flush:if (ret)kvm_flush_remote_tlbs (kvm); }@@ -4010,7 +4023,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultunsigned long mmu_seq;int r;
Quoted 5 lines
arch/x86/kvm/mmu/mmu.c static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault · line 4029
Quoted 5 lines
arch/x86/kvm/mmu/mmu.c +1
-1- fault->gfn = fault->addr >>PAGE_SHIFT ; + fault->gfn =kvm_gfn_unalias (vcpu->kvm,gpa_to_gfn (fault->addr)); fault->slot =kvm_vcpu_gfn_to_memslot (vcpu, fault->gfn);if (page_fault_handle_page_track (vcpu, fault))
Looking at code more, I think this patch is broken. There are couple of issues if I understand correctly:
- Rick's original patch has stolen_bits_mask encoded in 'struct kvm_mmu_page', so basically a new page table is allocated for different aliasing GPA. Sean suggested to use role.private instead of stolen_bits_mask so I changed but that was lost in this patch too. Therefore essentially, with this patch, all aliasing GFNs share the same page table and the same mapping. There's slight difference between TDP MMU and legacy MMU, that the former purely uses 'fault-
Quoted 1 line
gfn' (which doesn't have aliasing bit) to iterate page table and the latter
uses 'fault->addr' (which contains the aliasing bit), but this makes little difference. With this patch, all aliasing GFNs share page table and the mapping. This is not what we want, and this is wrong.
- The original change to get GFN w/o aliasing for MTRR check (below) is lost. And there are some other changes that are also lost (such as don't support aliasing for private (user-invisible, not TDX private) memory slot), but it's not immediately apparent to me whether this is an issue.
@@ -3833,7 +3865,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u32 error_code,
max_level > PG_LEVEL_4K;
max_level--) {
int page_num = KVM_PAGES_PER_HPAGE(max_level);
- gfn_t base = (gpa >> PAGE_SHIFT) & ~(page_num - 1);
+ gfn_t base = vcpu_gpa_to_gfn_unalias(vcpu, gpa) & ~(page_num -
1);
Another thing is above change to kvm_flush_remote_tlbs_with_range() to make it flush TLBs for mappings for all aliasing for a given GFN range doesn't fit for TDX. TDX private mapping and shared mapping cannot co-exist therefore when a page that has multiple aliasing mapped to it is taken out, only one mapping is valid (not to mention private page cannot be taken out). This is one of the reasons that I think this GPA stolen bits infrastructure isn't that mandatory for TDX. I think it's OK to ditch this infrastructure and adopt what TDX needs (the concept of private/shared mapping).
Signature · 2 lines
Thanks, -Kai
Re: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
In reply to Kai HuangAdded Peng Chao.
Quoted 138 lines · Kai Huang
Quoted 122 lines · On Fri, 2022-03-04 at 11:48 -0800, [email protected] wrote:
From: Rick Edgecombe <[email protected]>
Add support in KVM's MMU for aliasing multiple GPAs (from a hardware perspective) to a single GPA (from a memslot perspective). GPA aliasing will be used to repurpose GPA bits as attribute bits, e.g. to expose an execute-only permission bit to the guest. To keep the implementation simple (relatively speaking), GPA aliasing is only supported via TDP.
Today KVM assumes two things that are broken by GPA aliasing.
1. GPAs coming from hardware can be simply shifted to get the GFNs.
2. GPA bits 51:MAXPHYADDR are reserved to zero.With GPA aliasing, translating a GPA to GFN requires masking off the repurposed bit, and a repurposed bit may reside in 51:MAXPHYADDR.
To support GPA aliasing, introduce the concept of per-VM GPA stolen bits, that is, bits stolen from the GPA to act as new virtualized attribute bits. A bit in the mask will cause the MMU code to create aliases of the GPA. It can also be used to find the GFN out of a GPA coming from a tdp fault.
To handle case (1) from above, retain any stolen bits when passing a GPA in KVM's MMU code, but strip them when converting to a GFN so that the GFN contains only the "real" GFN, i.e. never has repurposed bits set.
GFNs (without stolen bits) continue to be used to:
- Specify physical memory by userspace via memslots
- Map GPAs to TDP PTEs via RMAP
- Specify dirty tracking and write protection
- Look up MTRR types
- Inject async page faultsSince there are now multiple aliases for the same aliased GPA, when userspace memory backing the memslots is paged out, both aliases need to be modified. Fortunately, this happens automatically. Since rmap supports multiple mappings for the same GFN for PTE shadowing based paging, by adding/removing each alias PTE with its GFN,
kvm_handle_hva()based operations will be applied to both aliases.In the case of the rmap being removed in the future, the needed information could be recovered by iterating over the stolen bits and walking the TDP page tables.
For TLB flushes that are address based, make sure to flush both aliases in the case of stolen bits.
Only support stolen bits in 64 bit guest paging modes (long, PAE). Features that use this infrastructure should restrict the stolen bits to exclude the other paging modes. Don't support stolen bits for shadow EPT.
Signed-off-by Rick Edgecombe <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 51 +++++++++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu.c | 19 ++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 25 +++++++++------- 4 files changed, 84 insertions(+), 13 deletions(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -1235,7 +1235,9 @@ struct kvm_arch {spinlock_t hv_root_tdp_lock;#endif +#ifdef CONFIG_KVM_MMU_PRIVATE gfn_t gfn_shared_mask; +#endif };struct kvm_vm_stat {arch/x86/kvm/mmu.h +41
-0@@ -365,4 +365,55 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu,return gpa;return translate_nested_gpa (vcpu, gpa, access, exception); } + +static inline gfn_t kvm_gfn_stolen_mask (struct kvm *kvm) +{ +#ifdef CONFIG_KVM_MMU_PRIVATE +return kvm->arch.gfn_shared_mask; +#else +return 0 ; +#endif +} + +static inline gpa_t kvm_gpa_stolen_mask (struct kvm *kvm) +{ +return gfn_to_gpa (kvm_gfn_stolen_mask (kvm)); +} + +static inline gpa_t kvm_gpa_unalias (struct kvm *kvm,gpa_t gpa) +{ +return gpa & ~kvm_gpa_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_unalias (struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_shared (struct kvm *kvm,gfn_t gfn) +{ +return gfn |kvm_gfn_stolen_mask (kvm); +} + +static inline gfn_t kvm_gfn_private (struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_stolen_mask (kvm); +} + +static inline gpa_t kvm_gpa_private (struct kvm *kvm,gpa_t gpa) +{ +return gpa & ~kvm_gpa_stolen_mask (kvm); +} + +static inline bool kvm_is_private_gfn (struct kvm *kvm,gfn_t gfn)arch/x86/kvm/mmu.h static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, · line 418
Quoted 10 lines
arch/x86/kvm/mmu.h +10
-0+{ +gfn_t mask =kvm_gfn_stolen_mask (kvm); + +return mask && !(gfn & mask); +} + +static inline bool kvm_is_private_gpa (struct kvm *kvm,gpa_t gpa) +{ +return kvm_is_private_gfn (kvm,gpa_to_gfn (gpa)); +}The patch title and commit message say nothing about private/shared, but only mention stolen bits in general. It's weird to introduce those *private* related helpers here.
Quoted 2 lines
I think you can just ditch the concept of stolen bit infrastructure, but just adopt what TDX needs.
Sure, this patch heavily changed from the original patch Now. One suggestion is that private/shared is characteristic to kvm page fault, not gpa/gfn. It's TDX specific.
- Add a helper function to check if KVM MMU is TD or VM. Right now kvm_gfn_stolen_mask() is used. Probably kvm_mmu_has_private_bit(). (any better name?)
- Let's keep address conversion functions: address => unalias/shared/private
- Add struct kvm_page_fault.is_private
see how kvm_is_private_{gpa, gfn}() can be removed (or reduced).
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +0
-0
Quoted 8 lines
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 279
Quoted 6 lines
arch/x86/kvm/mmu/mmu.c +1
-1@@ -276,11 +276,24 @@ static inline bool kvm_available_flush_tlb_with_range(void)static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm,struct kvm_tlb_range *range) { -int ret = -ENOTSUPP ; +int ret = -EOPNOTSUPP ;Change doesn't belong to this patch.
Will fix it.
Quoted 18 lines
Quoted 4 lines
arch/x86/kvm/mmu/mmu.c +2
-1+u64 gfn_stolen_mask; -if (range && kvm_x86_ops.tlb_remote_flush_with_range) +/* arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 292
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +10
-0+* Fall back to the big hammer flush if there is more than one +* GPA alias that needs to be flushed. +*/ + gfn_stolen_mask =kvm_gfn_stolen_mask (kvm); +if (hweight64 (gfn_stolen_mask) >1 ) + goto generic_flush; + +if (range &&kvm_available_flush_tlb_with_range ()) { +/* Callback should flush both private GFN and shared GFN. */ + range->start_gfn =kvm_gfn_unalias (kvm, range->start_gfn);This seems wrong. It seems the intention of this function is to flush TLB for all aliases for a given GFN range. Here it seems you are unconditionally change to range to always exclude the stolen bits.
Ooh, right. This alias knowledge is in TDX. This unalias should be dropped and put it in tdx.c. I'll fix it.
Quoted 5 lines
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 294
Quoted 2 lines
arch/x86/kvm/mmu/mmu.c +1
-0ret =static_call (kvm_x86_tlb_remote_flush_with_range)(kvm, range); + }And you always fall through to do big hammer flush, which is obviously not intended.
Please notice "if (ret)". If it succeeded, big hammer flush is skipped.
Quoted 113 lines
Quoted 113 lines
arch/x86/kvm/mmu/mmu.c +2
-1+generic_flush:if (ret)kvm_flush_remote_tlbs (kvm); }@@ -4010,7 +4023,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultunsigned long mmu_seq;int r; - fault->gfn = fault->addr >>PAGE_SHIFT ; + fault->gfn =kvm_gfn_unalias (vcpu->kvm,gpa_to_gfn (fault->addr)); fault->slot =kvm_vcpu_gfn_to_memslot (vcpu, fault->gfn);if (page_fault_handle_page_track (vcpu, fault))arch/x86/kvm/mmu/paging_tmpl.h +14
-9@@ -25,7 +25,8 @@#define guest_walker guest_walker64#define FNAME (name) paging##64_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) (~kvm_gpa_stolen_mask(vcpu->kvm) & \ + PT64_LVL_ADDR_MASK(lvl))#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -44,7 +45,7 @@#define guest_walker guest_walker32#define FNAME (name) paging##32_##name#define PT_BASE_ADDR_MASK PT32_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT32_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT32_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT32_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT32_INDEX(addr, level)#define PT_LEVEL_BITS PT32_LEVEL_BITS@@ -58,7 +59,7 @@#define guest_walker guest_walkerEPT#define FNAME (name) ept_##name#define PT_BASE_ADDR_MASK GUEST_PT64_BASE_ADDR_MASK -#define PT_LVL_ADDR_MASK (lvl) PT64_LVL_ADDR_MASK(lvl) +#define PT_LVL_ADDR_MASK (vcpu, lvl) PT64_LVL_ADDR_MASK(lvl)#define PT_LVL_OFFSET_MASK (lvl) PT64_LVL_OFFSET_MASK(lvl)#define PT_INDEX (addr, level) PT64_INDEX(addr, level)#define PT_LEVEL_BITS PT64_LEVEL_BITS@@ -75,7 +76,7 @@#define PT_GUEST_ACCESSED_MASK (1 << PT_GUEST_ACCESSED_SHIFT)#define gpte_to_gfn_lvl FNAME(gpte_to_gfn_lvl) -#define gpte_to_gfn (pte) gpte_to_gfn_lvl((pte), PG_LEVEL_4K) +#define gpte_to_gfn (vcpu, pte) gpte_to_gfn_lvl(vcpu, pte, PG_LEVEL_4K)/* * The guest_walker structure emulates the behavior of the hardware page @@ -96,9 +97,9 @@ struct guest_walker {struct x86_exception fault; }; -static gfn_t gpte_to_gfn_lvl (pt_element_t gpte,int lvl) +static gfn_t gpte_to_gfn_lvl (struct kvm_vcpu *vcpu,pt_element_t gpte,int lvl) { -return (gpte &PT_LVL_ADDR_MASK (lvl)) >>PAGE_SHIFT ; +return (gpte &PT_LVL_ADDR_MASK (vcpu, lvl)) >>PAGE_SHIFT ; } static inline void FNAME(protect_clean_gpte)(struct kvm_mmu *mmu, unsigned *access,@@ -395,7 +396,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker, --walker->level; index =PT_INDEX (addr, walker->level); - table_gfn =gpte_to_gfn (pte); + table_gfn =gpte_to_gfn (vcpu, pte); offset = index *sizeof (pt_element_t); pte_gpa =gfn_to_gpa (table_gfn) + offset;@@ -460,7 +461,7 @@ static int FNAME(walk_addr_generic)(struct guest_walker *walker,if (unlikely (errcode)) goto error; - gfn =gpte_to_gfn_lvl (pte, walker->level); + gfn =gpte_to_gfn_lvl (vcpu, pte, walker->level); gfn += (addr &PT_LVL_OFFSET_MASK (walker->level)) >>PAGE_SHIFT ;if (PTTYPE ==32 && walker->level >PG_LEVEL_4K &&is_cpuid_PSE36 ())@@ -555,12 +556,14 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp,gfn_t gfn;kvm_pfn_t pfn; +WARN_ON (gpte &kvm_gpa_stolen_mask (vcpu->kvm)); +if (FNAME (prefetch_invalid_gpte)(vcpu, sp, spte, gpte))return false;pgprintk ("%s: gpte %llx spte %p\n" , __func__, (u64 )gpte, spte); - gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access &FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);@@ -656,6 +659,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault,WARN_ON_ONCE (gw->gfn != base_gfn); direct_access = gw->pte_access; +WARN_ON (fault->addr &kvm_gpa_stolen_mask (vcpu->kvm)); + top_level = vcpu->arch.mmu->root_level;if (top_level ==PT32E_ROOT_LEVEL ) top_level =PT32_ROOT_LEVEL ;@@ -1080,7 +1085,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)continue ; }
Quoted 9 lines
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) · line 1091
Quoted 5 lines
arch/x86/kvm/mmu/paging_tmpl.h +1
-1- gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);In commit message you mentioned "Don't support stolen bits for shadow EPT" (you actually mean shadow MMU I suppose), yet there's bunch of code change to shadow MMU.
Those are not needed. I'll drop them.
Signature · 1 line
Isaku Yamahata <[email protected]>
Paolo Bonzini
maintainerRe: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 7 lines · Isaku Yamahata
Sure, this patch heavily changed from the original patch Now. One suggestion is that private/shared is characteristic to kvm page fault, not gpa/gfn. It's TDX specific.
- Add a helper function to check if KVM MMU is TD or VM. Right now
kvm_gfn_stolen_mask()is used. Probablykvm_mmu_has_private_bit(). (any better name?)
I think use of kvm_gfn_stolen_mask() should be minimized anyway. I would rename it to to kvm_{gfn,gpa}_private_mask and not return bool.
Quoted 1 line
- Let's keep address conversion functions: address => unalias/shared/private
unalias is the same as private. It doesn't seem to have a lot of uses.
I would just inline "x & ~gfn_private_mask", or "x &
~kvm_gfn_private_mask(kvm)"; or the same with gpa of course.
The shared and private conversion functions should remain.
Quoted 2 lines
- Add struct kvm_page_fault.is_private
see how kvm_is_private_{gpa, gfn}() can be removed (or reduced).
Agreed.
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
In reply to Isaku YamahataQuoted 7 lines · Isaku Yamahata
Sure, this patch heavily changed from the original patch Now. One suggestion is that private/shared is characteristic to kvm page fault, not gpa/gfn. It's TDX specific.
- Add a helper function to check if KVM MMU is TD or VM. Right now
kvm_gfn_stolen_mask()is used. Probablykvm_mmu_has_private_bit(). (any better name?)
I think use of kvm_gfn_stolen_mask() should be minimized anyway. I would rename it to to kvm_{gfn,gpa}_private_mask and not return bool.
Quoted 1 line
- Let's keep address conversion functions: address => unalias/shared/private
unalias is the same as private. It doesn't seem to have a lot of uses.
I would just inline "x & ~gfn_private_mask", or "x &
~kvm_gfn_private_mask(kvm)"; or the same with gpa of course.
The shared and private conversion functions should remain.
Quoted 2 lines
- Add struct kvm_page_fault.is_private
see how kvm_is_private_{gpa, gfn}() can be removed (or reduced).
Agreed.
Paolo
Paolo Bonzini
maintainerRe: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
In reply to Kai HuangQuoted 6 lines · Kai Huang
arch/x86/kvm/mmu/mmu.c static inline bool kvm_available_flush_tlb_with_range(void) · line 292
Quoted 3 lines
arch/x86/kvm/mmu/mmu.c +3
-0+if (range &&kvm_available_flush_tlb_with_range ()) { +/* Callback should flush both private GFN and shared GFN. */ + range->start_gfn =kvm_gfn_unalias (kvm, range->start_gfn);This seems wrong. It seems the intention of this function is to flush TLB for all aliases for a given GFN range. Here it seems you are unconditionally change to range to always exclude the stolen bits.
He passes the "low" range with bits cleared, and expects the callback to take care of both. That seems okay (apart from the incorrect fallthrough that you pointed out).
Quoted 11 lines
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) · line 1091
Quoted 7 lines
arch/x86/kvm/mmu/paging_tmpl.h +1
-1- gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);In commit message you mentioned "Don't support stolen bits for shadow EPT" (you actually mean shadow MMU I suppose), yet there's bunch of code change to shadow MMU.
It's a bit ugly, but it's uglier to keep two versions of gpte_to_gfn.
Perhaps the commit message can be rephrased to "Stolen bits are not supported in the shadow MMU; they will be used only for TDX which uses the TDP MMU exclusively as it does not support nested virtualization. Therefore, the gfn_shared_mask will always be zero in that case".
Paolo
Re: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 13 lines
Quoted 12 lines
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) · line 1091
Quoted 8 lines
arch/x86/kvm/mmu/paging_tmpl.h +1
-1- gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);In commit message you mentioned "Don't support stolen bits for shadow EPT" (you actually mean shadow MMU I suppose), yet there's bunch of code change to shadow MMU.
Quoted 1 line
It's a bit ugly, but it's uglier to keep two versions of
gpte_to_gfn.
gpte_to_gfn() is only used in paging_tmpl.h. Could you elaborate why we need to keep two versions of it?
Signature · 2 lines
Thanks, -Kai
Paolo Bonzini
maintainerRe: [RFC PATCH v5 033/104] KVM: x86: Add infrastructure for stolen GPA bits
Quoted 13 lines · Kai Huang
Quoted 13 lines
Quoted 12 lines
arch/x86/kvm/mmu/paging_tmpl.h static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) · line 1091
Quoted 8 lines
arch/x86/kvm/mmu/paging_tmpl.h +1
-1- gfn =gpte_to_gfn (gpte); + gfn =gpte_to_gfn (vcpu, gpte); pte_access = sp->role.access; pte_access &=FNAME (gpte_access)(gpte);FNAME (protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte);In commit message you mentioned "Don't support stolen bits for shadow EPT" (you actually mean shadow MMU I suppose), yet there's bunch of code change to shadow MMU.
Quoted 4 lines
Quoted 1 line
It's a bit ugly, but it's uglier to keep two versions of
gpte_to_gfn.
gpte_to_gfn()is only used inpaging_tmpl.h. Could you elaborate why we need to keep two versions of it?
You're right. Yeah, considering page table walks are not supported when private memory is available, it shouldn't be necessary to change paging_tmpl.h.
Paolo
RFC PATCH v5 34/104 The start of TDX KVM patch series: KVM TDP refactoring for TDX
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of KVM TDP refactoring for TDX.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2
@@ -24,7 +24,7 @@ Patch Layer status * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet -* KVM MMU GPA stolen bits: Applying -* KVM TDP refactoring for TDX: Not yet +* KVM MMU GPA stolen bits: Applied +* KVM TDP refactoring for TDX: Applying * KVM TDP MMU hooks: Not yet * KVM TDP MMU MapGPA: Not yet
Signature · 1 line
2.25.1
RFC PATCH v5 35/104 KVM: x86/mmu: Disallow dirty logging for x86 TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX doesn't support dirty logging. Report dirty logging isn't supported so that device model, for example qemu, can properly handle it.
--- arch/x86/kvm/x86.c | 5 +++++ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 15 ++++++++++++--- 3 files changed, 18 insertions(+), 3 deletions(-)
arch/x86/kvm/x86.c +5 -0
@@ -12876,6 +12876,11 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, }EXPORT_SYMBOL_GPL (kvm_sev_es_string_io); +bool kvm_arch_dirty_log_supported (struct kvm *kvm) +{ +return kvm->arch.vm_type !=KVM_X86_TDX_VM ; +} +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_entry);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);
include/linux/kvm_host.h +1 -0
@@ -1423,6 +1423,7 @@ bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu);int kvm_arch_post_init_vm (struct kvm *kvm);void kvm_arch_pre_destroy_vm (struct kvm *kvm);int kvm_arch_create_vm_debugfs (struct kvm *kvm); +bool kvm_arch_dirty_log_supported (struct kvm *kvm);#ifndef __KVM_HAVE_ARCH_VM_ALLOC/*
virt/kvm/kvm_main.c +12 -3
@@ -1423,9 +1423,18 @@ static void kvm_replace_memslot(struct kvm *kvm, } } -static int check_memory_region_flags (const struct kvm_userspace_memory_region *mem) +bool __weak kvm_arch_dirty_log_supported (struct kvm *kvm) { -u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ; +return true; +} + +static int check_memory_region_flags (struct kvm *kvm, +const struct kvm_userspace_memory_region *mem) +{ +u32 valid_flags =0 ; + +if (kvm_arch_dirty_log_supported (kvm)) + valid_flags |=KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEM valid_flags |=KVM_MEM_READONLY ;
@@ -1826,7 +1835,7 @@ int __kvm_set_memory_region(struct kvm *kvm,int as_id, id;int r; - r =check_memory_region_flags (mem); + r =check_memory_region_flags (kvm, mem);if (r)return r;
Signature · 1 line
2.25.1
Re: [RFC PATCH v5 035/104] KVM: x86/mmu: Disallow dirty logging for x86 TDX
Quoted 68 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TDX doesn't support dirty logging. Report dirty logging isn't supported so that device model, for example qemu, can properly handle it.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/x86.c | 5 +++++ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 15 ++++++++++++--- 3 files changed, 18 insertions(+), 3 deletions(-)arch/x86/kvm/x86.c +5
-0@@ -12876,6 +12876,11 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, }EXPORT_SYMBOL_GPL (kvm_sev_es_string_io); +bool kvm_arch_dirty_log_supported (struct kvm *kvm) +{ +return kvm->arch.vm_type !=KVM_X86_TDX_VM ; +} +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_entry);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_exit);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_fast_mmio);include/linux/kvm_host.h +1
-0@@ -1423,6 +1423,7 @@ bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu);int kvm_arch_post_init_vm (struct kvm *kvm);void kvm_arch_pre_destroy_vm (struct kvm *kvm);int kvm_arch_create_vm_debugfs (struct kvm *kvm); +bool kvm_arch_dirty_log_supported (struct kvm *kvm);#ifndef __KVM_HAVE_ARCH_VM_ALLOC/* virt/kvm/kvm_main.c +11
-2@@ -1423,9 +1423,18 @@ static void kvm_replace_memslot(struct kvm *kvm, } } -static int check_memory_region_flags (const struct kvm_userspace_memory_region *mem) +bool __weak kvm_arch_dirty_log_supported (struct kvm *kvm) { -u32 valid_flags =KVM_MEM_LOG_DIRTY_PAGES ; +return true; +} + +static int check_memory_region_flags (struct kvm *kvm, +const struct kvm_userspace_memory_region *mem) +{ +u32 valid_flags =0 ; + +if (kvm_arch_dirty_log_supported (kvm)) + valid_flags |=KVM_MEM_LOG_DIRTY_PAGES ;#ifdef __KVM_HAVE_READONLY_MEM valid_flags |=KVM_MEM_READONLY ;
virt/kvm/kvm_main.c int __kvm_set_memory_region(struct kvm *kvm, · line 1841
Quoted 8 lines
virt/kvm/kvm_main.c +1
-1@@ -1826,7 +1835,7 @@ int __kvm_set_memory_region(struct kvm *kvm,int as_id, id;int r; - r =check_memory_region_flags (mem); + r =check_memory_region_flags (kvm, mem);if (r)return r;
RFC PATCH v5 36/104 KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
In reply to [email protected]From: Sean Christopherson <[email protected]>
Explicitly check for an MMIO spte in the fast page fault flow. TDX will use a not-present entry for MMIO sptes, which can be mistaken for an access-tracked spte since both have SPTE_SPECIAL_MASK set.
The fast page fault handles the case of changing access bits without obtaining mmu_lock. For example, clear write protect bit for dirty page tracking. MMIO emulation is handled in a slow path. So it doesn't affect the default VM case.
--- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -3167,7 +3167,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)break ; sp =sptep_to_sp (sptep); -if (!is_last_spte (spte, sp->role.level)) +if (!is_last_spte (spte, sp->role.level) ||is_mmio_spte (spte))break ;/*
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [RFC PATCH v5 036/104] KVM: x86/mmu: Explicitly check for MMIO spte in fast page fault
Quoted 9 lines · On 3/4/22 20:48, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Explicitly check for an MMIO spte in the fast page fault flow. TDX will use a not-present entry for MMIO sptes, which can be mistaken for an access-tracked spte since both have
SPTE_SPECIAL_MASKset.The fast page fault handles the case of changing access bits without obtaining
mmu_lock. For example, clear write protect bit for dirty page tracking. MMIO emulation is handled in a slow path. So it doesn't affect
"MMIO sptes are handled in handle_mmio_page_fault for non-TDX VMs, so this patch does not affect them. TDX will handle MMIO emulation through a hypercall instead".
For this comment, it is not necessary to talk about the slow path, since that is just where MMIO sptes are installed. If the slow path is reached, fast_page_fault must not have seen is_mmio_spte(spte).
arch/x86/kvm/mmu/mmu.c static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) · line 3173
Quoted 9 lines
arch/x86/kvm/mmu/mmu.c +1
-1@@ -3167,7 +3167,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)break ; sp =sptep_to_sp (sptep); -if (!is_last_spte (spte, sp->role.level)) +if (!is_last_spte (spte, sp->role.level) ||is_mmio_spte (spte))break ;/*
I would include the check a couple lines before:
if (!is_shadow_present_pte(spte) || is_mmio_spte(spte))
This matches what is in the commit message: the problem is that MMIO SPTEs are present in the TDX case, so you need to check them even if is_shadow_present_pte(spte) returns true.
Paolo
RFC PATCH v5 37/104 KVM: x86/mmu: Allow non-zero init value for shadow PTE
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX will run with EPT violation #VEs enabled for shared EPT, which means KVM needs to set the "suppress #VE" bit in unused PTEs to avoid unintentionally reflecting not-present EPT violations into the guest.
Because guest memory is protected with TDX, VMM can't parse instructions in the guest memory. Instead, MMIO hypercall is used to pass necessary information to VMM.
To make unmodified device driver work, guest TD expects #VE on accessing shared GPA. The #VE handler converts MMIO access into MMIO hypercall with the EPT entry of enabled "#VE" by clearing "suppress #VE" bit. Before VMM enabling #VE, it needs to figure out the given GPA is for MMIO by EPT violation. So the execution flow looks like
- allocate unused shared EPT entry with suppress #VE bit set.
- EPT violation on that GPA.
- VMM figures out the faulted GPA is for MMIO.
- VMM clears the suppress #VE bit.
- Guest TD gets #VE, and converts MMIO access into MMIO hypercall.
--- arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 50 +++++++++++++++++++++++++++++++++++------ arch/x86/kvm/mmu/spte.c | 10 +++++++++ arch/x86/kvm/mmu/spte.h | 2 ++ 4 files changed, 56 insertions(+), 7 deletions(-)
arch/x86/kvm/mmu.h +1 -0
@@ -66,6 +66,7 @@ static __always_inline u64 rsvd_bits(int s, int e)void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask);void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only); +void kvm_mmu_set_spte_init_value (u64 init_value);void kvm_init_mmu (struct kvm_vcpu *vcpu); void kvm_init_shadow_npt_mmu(struct kvm_vcpu *vcpu, unsigned long cr0,
arch/x86/kvm/mmu/mmu.c +43 -7
87 lines not on this page · raw message
arch/x86/kvm/mmu/spte.c +10 -0
@@ -35,6 +35,7 @@ u64 __read_mostly shadow_mmio_access_mask;u64 __read_mostly shadow_present_mask;u64 __read_mostly shadow_me_mask;u64 __read_mostly shadow_acc_track_mask; +u64 __read_mostly shadow_init_value;u64 __read_mostly shadow_nonpresent_or_rsvd_mask;u64 __read_mostly shadow_nonpresent_or_rsvd_lower_gfn_mask;
@@ -223,6 +224,14 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn)return new_spte; } +void kvm_mmu_set_spte_init_value (u64 init_value) +{ +if (WARN_ON (!IS_ENABLED (CONFIG_X86_64 ) && init_value)) + init_value =0 ; + shadow_init_value = init_value; +} +EXPORT_SYMBOL_GPL (kvm_mmu_set_spte_init_value); +static u8 kvm_get_shadow_phys_bits (void ) {/*
@@ -367,6 +376,7 @@ void kvm_mmu_reset_all_pte_masks(void) shadow_present_mask =PT_PRESENT_MASK ; shadow_acc_track_mask =0 ; shadow_me_mask = sme_me_mask; + shadow_init_value =0 ; shadow_host_writable_mask =DEFAULT_SPTE_HOST_WRITEABLE ; shadow_mmu_writable_mask =DEFAULT_SPTE_MMU_WRITEABLE ;
arch/x86/kvm/mmu/spte.h +2 -0
@@ -171,6 +171,8 @@ extern u64 __read_mostly shadow_mmio_access_mask;extern u64 __read_mostly shadow_present_mask;extern u64 __read_mostly shadow_me_mask; +extern u64 __read_mostly shadow_init_value; +/* * SPTEs in MMUs without A/D bits are marked with SPTE_TDP_AD_DISABLED_MASK; * shadow_acc_track_mask is the set of bits to be cleared in non-accessed
Signature · 1 line
2.25.1