KVM TDX basic feature support
Subsystems (6)
v18 cover KVM TDX basic feature support
From: Isaku Yamahata <[email protected]>
KVM TDX basic feature support
Hello. This is v18 the patch series vof KVM TDX support. This is based on v6.8-rc1 + minor fixes that includes the tdx host initialization and the kvm guest memfd.
For unified KVM uAPI, I'll post independent patches and TDX support would be on top of this patch series.
The tree can be found at https://github.com/intel/tdx/tree/kvm-upstream
The corresponding qemu branch is found at
https://github.com/yamahata/qemu/tree/tdx/qemu-upm
How to run/test: It's describe at https://github.com/intel/tdx/wiki/TDX-KVM
More features tree is found at
https://github.com/intel/tdx/tree/kvm-upstream-next
This patch series supports only basic functionalities, TDX guest creation, execution and destruction. The following features are planned as follow up.
- vPMU
- Off-TD debug. qemu gdb stub support
- live migration
- etc
Isaku Yamahata
Cyhanges from v17:
- Changed tdx_seamcall() to use struct tdx_module_args
- use TDH.SYS.RD() instead of TDH.SYS.INFO()
- Drop workaround of pending interrupt
- move the initializatin of loaded_vmcss_on_cpu to vmx_init()
- More error handling on tdx 1.5 specific error code
- drop changes of tools/arch/x86/include/uapi/asm/kvm.h
- fixes typo, indent and tabs
Changes from v16:
- rebased v6.7-rc
- Switched to TDX module 1.5. Unsupport TDX module 1.0
Changes from v15:
- Added KVM_TDX_RELEASE_VM to reduce the destruction time
- Catch up the TDX module interface to use struct tdx_module_args instead of struct tdx_module_output
- Add tdh_mem_sept_rd() for SEPT_VE_DISABLE=1 and handle Secure-EPT violation with SEPT_VE_DISABLE case.
- Simplified tdx_reclaim_page()
- Reorganize the locking of tdx_release_hkid(), and use smp_call_mask() instead of smp_call_on_cpu() to hold spinlock to race with invalidation on releasing guest memfd
- Removed AMX check as the KVM upstream supports AMX.
- Added CET flag to guest supported xss
- add check if nr_pages isn't large with
(nr_page << PAGE_SHIFT) >> PAGE_SHIFT
- use __seamcall_saved_ret()
- As struct tdx_module_args doesn't match with vcpu.arch.regs, copy regs before/after calling __seamcall_saved_ret().
Changes from v14:
https://lore.kernel.org/all/[email protected]/
- rebased to v6.5-rc2, v11 KVM guest_memfd(), v11 TDX host kernel support
- ABI change to add reserved member for future compatibility, dropped unused member.
- handle EXIT_REASON_OTHER_SMI
- handle FEAT_CTL MSR access
Changes from v13:
- rbased to v6.4-rc3
- Make use of KVM gmem.
- Added check_cpuid callback for KVM_SET_CPUID2 as RFC patch.
- ABI change of KVM_TDX_VM_INIT as VM scoped KVM ioctl.
- Make TDX initialization non-depend on kvm hardware_enable. Use vmx_hardware_enable directly.
- Drop a patch to prohibit dirty logging as new KVM gmem code base
- Drop parameter only checking for some TDG.VP.VMCALL. Just default part
Changes from v12:
- ABI change of KVM_TDX_VM_INIT
- Rename kvm_gfn_{private, shared} to kvm_gfn_to_{private, shared}
- Move APIC BASE MSI initialization to KVM_TDX_VCPU_INIT
- Fix MTRR patch
- Make MapGpa hypercall always pass it to user space VMM
- Split hooks to TDP MMU into two part. populating and zapping.
Changes from v11:
- ABI change of KVM_TDX_VM_INIT
- Split the hook of TDP MMU to not modify handle_changed_spte()
- Enhanced commit message on mtrr patch
- Made KVM_CAP_MAX_VCPUS to x86 specific
Changes from v10:
- rebased to v6.2-rc3
- support mtrr with its own patches
- Integrated fd-based private page v10
- Integrated TDX host kernel support v8
- Integrated kvm_init rework v2
- removed struct tdx_td_page and its initialization logic
- cleaned up mmio spte and require enable_mmio_caching=true for TDX
- removed dubious WARN_ON_ONCE()
- split a patch adding methods as nop into several patches
Changes from v9:
- rebased to v6.1-rc2
- Integrated fd-based private page v9 as prerequisite.
- Integrated TDX host kernel support v6
- TDP MMU: Make handle_change_spte() return value.
- TDX: removed seamcall_lock and return -EAGAIN so that TDP MMU can retry
Changes from v8:
- rebased to v6.0-rc7
- Integrated with kvm hardware initialization. Check all packages has at least one online CPU when creating guest TD and refuse cpu offline during guest TDs are running.
- Integrated fd-based private page v8 as prerequisite.
- TDP MMU: Introduced more callbacks instead of single callback.
Changes from v7:
- Use xarray to track whether GFN is private or shared. Drop SPTE_SHARED_MASK. The complex state machine with SPTE_SHARED_MASK was ditched.
- Large page support is implemented. But will be posted as independent RFC patch.
- fd-based private page v7 is integrated. This is mostly same to Chao's patches. It's in github.
Changes from v6:
- rebased to v5.19
Changes from v5:
- export __seamcall and use it
- move mutex lock from callee function of smp_call_on_cpu to the caller.
- rename mmu_prezap => flush_shadow_all_private() and tdx_mmu_release_hkid
- updated comment
- drop the use of tdh_mng_key.reclaimid(): as the function is for backward compatibility to only return success
- struct kvm_tdx_cmd: metadata => flags, added __u64 error.
- make this ioctl systemwide ioctl
- ABI change to struct kvm_init_vm
- guest_tsc_khz: use kvm->arch.default_tsc_khz
- rename BUILD_BUG_ON_MEMCPY to MEMCPY_SAME_SIZE
- drop exporting kvm_set_tsc_khz().
- fix kvm_tdp_page_fault() for mtrr emulation
- rename it to kvm_gfn_shared_mask(), dropped kvm_gpa_shared_mask()
- drop kvm_is_private_gfn(), kept kvm_is_private_gpa()
keep kvm_{gfn, gpa}_private(), kvm_gpa_private()
- update commit message
- rename shadow_init_value => shadow_nonprsent_value
- added ept_violation_ve_test mode
- shadow_nonpresent_value => SHADOW_NONPRESENT_VALUE in tdp_mmu.c
- legacy MMU case
=> - mmu_topup_shadow_page_cache(), kvm_mmu_create()
- FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)
- #VE warning:
- rename: REMOVED_SPTE => __REMOVED_SPTE, SHADOW_REMOVED_SPTE => REMOVED_SPTE
- merge into Like we discussed, this patch should be merged with patch "KVM: x86/mmu: Allow non-zero init value for shadow PTE".
- fix pointed by Sagi. check !is_private check => (kvm_gfn_shared_mask && !is_private)
- introduce kvm_gfn_for_root(kvm, root, gfn)
- add only_shared argument to kvm_tdp_mmu_handle_gfn()
- use kvm_arch_dirty_log_supported()
- rename SPTE_PRIVATE_PROHIBIT to SPTE_SHARED_MASK.
- rename: is_private_prohibit_spte() => spte_shared_mask()
- fix: shadow_nonpresent_value => SHADOW_NONPRESENT_VALUE in comment
- dropped this patch as the change was merged into kvm/queue
- update vt_apicv_post_state_restore()
- use is_64_bit_hypercall()
- comment: expand MSMI -> Machine Check System Management Interrupt
- fixed TDX_SEPT_PFERR
- tdvmcall_p[1234]_{write, read}() => tdvmcall_a[0123]_{read,write}()
- rename tdmvcall_exit_readon() => tdvmcall_leaf()
- remove optional zero check of argument.
- do a check for static_call(kvm_x86_has_emulated_msr)(kvm, MSR_IA32_SMBASE) in kvm_vcpu_ioctl_smi and __apic_accept_irq.
- WARN_ON_ONCE in tdx_smi_allowed and tdx_enable_smi_window.
- introduce vcpu_deliver_init to x86_ops
- sprinkeled KVM_BUG_ON()
Changes from v4:
- rebased to TDX host kernel patch series.
- include all the patches to make this patch series working.
- add [MARKER] patches to mark the patch layer clear.
---
* What's TDX?
TDX stands for Trust Domain Extensions, which extends Intel Virtual Machines Extensions (VMX) to introduce a kind of virtual machine guest called a Trust Domain (TD) for confidential computing.
A TD runs in a CPU mode that is designed to protect the confidentiality of its memory contents and its CPU state from any other software, including the hosting Virtual Machine Monitor (VMM), unless explicitly shared by the TD itself.
We have more detailed explanations below (***). We have the high-level design of TDX KVM below (****).
In this patch series, we use "TD" or "guest TD" to differentiate it from the current "VM" (Virtual Machine), which is supported by KVM today.
* The organization of this patch series
This patch series is on top of the patches series "TDX host kernel support":
https://lore.kernel.org/lkml/[email protected]/
This patch series is available at
https://github.com/intel/tdx/tree/kvm-upstream
The related repositories (TDX qemu, TDX OVMF(tdvf) etc) are described at
https://github.com/intel/tdx/wiki/TDX-KVM
The relations of the layers are depicted as follows.
The arrows below show the order of patch reviews we would like to have.
The below layers are chosen so that the device model, for example, qemu can exercise each layering step by step. Check if TDX is supported, create TD VM, create TD vcpu, allow vcpu running, populate TD guest private memory, and handle vcpu exits/hypercalls/interrupts to run TD fully.
TDX vcpu
interrupt/exits/hypercall<------------\
^ |
| |
TD finalization |
^ |
| |
TDX EPT violation<------------\ |
^ | |
| | |
TD vcpu enter/exit | |
^ | |
| | |
TD vcpu creation/destruction | \-------KVM TDP MMU MapGPA
^ | ^
| | |
TD VM creation/destruction \---------------KVM TDP MMU hooks ^ ^
| |
TDX architectural definitions KVM TDP refactoring for TDX
^ ^
| |
TDX, VMX <--------TDX host kernel KVM MMU GPA stolen bits
coexistence support
The followings are explanations of each layer. Each layer has a dummy commit that starts with [MARKER] in subject. It is intended to help to identify where each layer starts.
TDX host kernel support:
https://lore.kernel.org/lkml/[email protected]/
The guts of system-wide initialization of TDX module. There is an independent patch series for host x86. TDX KVM patches call functions this patch series provides to initialize the TDX module.
TDX, VMX coexistence:
Infrastructure to allow TDX to coexist with VMX and trigger the initialization of the TDX module.
This layer starts with
"KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX"
TDX architectural definitions:
Add TDX architectural definitions and helper functions
This layer starts with
"[MARKER] The start of TDX KVM patch series: TDX architectural definitions".
TD VM creation/destruction:
Guest TD creation/destroy allocation and releasing of TDX specific vm and vcpu structure. Create an initial guest memory image with TDX measurement.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD VM creation/destruction".
TD vcpu creation/destruction:
guest TD creation/destroy Allocation and releasing of TDX specific vm and vcpu structure. Create an initial guest memory image with TDX measurement.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD vcpu creation/destruction"
TDX EPT violation:
Create an initial guest memory image with TDX measurement. Handle secure EPT violations to populate guest pages with TDX SEAMCALLs. This layer starts with
"[MARKER] The start of TDX KVM patch series: TDX EPT violation"
TD vcpu enter/exit:
Allow TDX vcpu to enter into TD and exit from TD. Save CPU state before entering into TD. Restore CPU state after exiting from TD.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD vcpu enter/exit"
TD vcpu interrupts/exit/hypercall:
Handle various exits/hypercalls and allow interrupts to be injected so that TD vcpu can continue running.
This layer starts with
"[MARKER] The start of TDX KVM patch series: TD vcpu exits/interrupts/hypercalls"
KVM MMU GPA shared bit:
Introduce framework to handle shared bit repurposed bit of GPA TDX repurposed a bit of GPA to indicate shared or private. If it's shared, it's the same as the conventional VMX EPT case. VMM can access shared guest pages. If it's private, it's handled by Secure-EPT and the guest page is encrypted.
This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM MMU GPA stolen bits"
KVM TDP refactoring for TDX:
TDX Secure EPT requires different constants. e.g. initial value EPT entry value etc. Various refactoring for those differences. This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM TDP refactoring for TDX"
KVM TDP MMU hooks:
Introduce framework to TDP MMU to add hooks in addition to direct EPT access TDX added Secure EPT which is an enhancement to VMX EPT. Unlike conventional VMX EPT, CPU can't directly read/write Secure EPT. Instead, use TDX SEAMCALLs to operate on Secure EPT.
This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM TDP MMU hooks"
KVM TDP MMU MapGPA:
Introduce framework to handle switching guest pages from private/shared to shared/private. For a given GPA, a guest page can be assigned to a private GPA or a shared GPA exclusively. With TDX MapGPA hypercall, guest TD converts GPA assignments from private (or shared) to shared (or private).
This layer starts with
"[MARKER] The start of TDX KVM patch series: KVM TDP MMU MapGPA "
KVM guest private memory: (not shown in the above diagram)
[PATCH v4 00/12] KVM: mm: fd-based approach for supporting KVM guest private
memory: https://lkml.org/lkml/2022/1/18/395
Guest private memory requires different memory management in KVM. The patch proposes a way for it. Integration with TDX KVM.
(***)
* TDX module
A CPU-attested software module called the "TDX module" is designed to implement the TDX architecture, and it is loaded by the UEFI firmware today. It can be loaded by the kernel or driver at runtime, but in this patch series we assume that the TDX module is already loaded and initialized.
The TDX module provides two main new logical modes of operation built upon the new SEAM (Secure Arbitration Mode) root and non-root CPU modes added to the VMX architecture. TDX root mode is mostly identical to the VMX root operation mode, and the TDX functions (described later) are triggered by the new SEAMCALL instruction with the desired interface function selected by an input operand (leaf number, in RAX). TDX non-root mode is used for TD guest operation. TDX non-root operation (i.e. "guest TD" mode) is similar to the VMX non-root operation (i.e. guest VM), with changes and restrictions to better assure that no other software or hardware has direct visibility of the TD memory and state.
TDX transitions between TDX root operation and TDX non-root operation include TD Entries, from TDX root to TDX non-root mode, and TD Exits from TDX non-root to TDX root mode. A TD Exit might be asynchronous, triggered by some external event (e.g., external interrupt or SMI) or an exception, or it might be synchronous, triggered by a TDCALL (TDG.VP.VMCALL) function.
TD VCPUs can be entered using SEAMCALL(TDH.VP.ENTER) by KVM. TDH.VP.ENTER is one of the TDX interface functions as mentioned above, and "TDH" stands for Trust Domain Host. Those host-side TDX interface functions are categorized into various areas just for better organization, such as SYS (TDX module management), MNG (TD management), VP (VCPU), PHYSMEM (physical memory), MEM (private memory), etc. For example, SEAMCALL(TDH.SYS.INFO) returns the TDX module information.
TDCS (Trust Domain Control Structure) is the main control structure of a guest TD, and encrypted (using the guest TD's ephemeral private key). At a high level, TDCS holds information for controlling TD operation as a whole, execution, EPTP, MSR bitmaps, etc that KVM needs to set it up. Note that MSR bitmaps are held as part of TDCS (unlike VMX) because they are meant to have the same value for all VCPUs of the same TD.
Trust Domain Virtual Processor State (TDVPS) is the root control structure of a TD VCPU. It helps the TDX module control the operation of the VCPU, and holds the VCPU state while the VCPU is not running. TDVPS is opaque to software and DMA access, accessible only by using the TDX module interface functions (such as TDH.VP.RD, TDH.VP.WR). TDVPS includes TD VMCS, and TD VMCS auxiliary structures, such as virtual APIC page, virtualization exception information, etc.
Several VMX control structures (such as Shared EPT and Posted interrupt descriptor) are directly managed and accessed by the host VMM. These control structures are pointed to by fields in the TD VMCS.
The above means that 1) KVM needs to allocate different data structures for TDs,
2) KVM can reuse the existing code for TDs for some operations, 3) it needs to define TD-specific handling for others. 3) Redirect operations to . 3) Redirect operations to the TDX specific callbacks, like "if (is_td_vcpu(vcpu)) tdx_callback() else vmx_callback();".
*TD Private Memory
TD private memory is designed to hold TD private content, encrypted by the CPU using the TD ephemeral key. An encryption engine holds a table of encryption keys, and an encryption key is selected for each memory transaction based on a Host Key Identifier (HKID). By design, the host VMM does not have access to the encryption keys.
In the first generation of MKTME, HKID is "stolen" from the physical address by allocating a configurable number of bits from the top of the physical address. The HKID space is partitioned into shared HKIDs for legacy MKTME accesses and private HKIDs for SEAM-mode-only accesses. We use 0 for the shared HKID on the host so that MKTME can be opaque or bypassed on the host.
During TDX non-root operation (i.e. guest TD), memory accesses can be qualified as either shared or private, based on the value of a new SHARED bit in the Guest Physical Address (GPA). The CPU translates shared GPAs using the usual VMX EPT (Extended Page Table) or "Shared EPT" (in this document), which resides in host VMM memory. The Shared EPT is directly managed by the host VMM - the same as with the current VMX. Since guest TDs usually require I/O, and the data exchange needs to be done via shared memory, thus KVM needs to use the current EPT functionality even for TDs.
* Secure EPT and Minoring using the TDP code
The CPU translates private GPAs using a separate Secure EPT. The Secure EPT pages are encrypted and integrity-protected with the TD's ephemeral private key. Secure EPT can be managed _indirectly_ by the host VMM, using the TDX interface functions, and thus conceptually Secure EPT is a subset of EPT (why "subset"). Since execution of such interface functions takes much longer time than accessing memory directly, in KVM we use the existing TDP code to minor the Secure EPT for the TD.
This way, we can effectively walk Secure EPT without using the TDX interface functions.
* VM life cycle and TDX specific operations
The userspace VMM, such as QEMU, needs to build and treat TDs differently. For example, a TD needs to boot in private memory, and the host software cannot copy the initial image to private memory.
* TSC Virtualization
The TDX module helps TDs maintain reliable TSC (Time Stamp Counter) values (e.g. consistent among the TD VCPUs) and the virtual TSC frequency is determined by TD configuration, i.e. when the TD is created, not per VCPU. The current KVM owns TSC virtualization for VMs, but the TDX module does for TDs.
* MCE support for TDs
The TDX module doesn't allow VMM to inject MCE. Instead PV way is needed for TD to communicate with VMM. For now, KVM silently ignores MCE request by VMM. MSRs related to MCE (e.g, MCE bank registers) can be naturally emulated by paravirtualizing MSR access.
[1] For details, the specifications, [2], [3], [4], [5], [6], [7], are available.
* Restrictions or future work
Some features are not included to reduce patch size. Those features are addressed as future independent patch series.
- large page (2M, 1G)
- qemu gdb stub
- guest PMU
- and more
* Prerequisites
It's required to load the TDX module and initialize it. It's out of the scope of this patch series. Another independent patch for the common x86 code is planned. It defines CONFIG_INTEL_TDX_HOST and this patch series uses CONFIG_INTEL_TDX_HOST. It's assumed that With CONFIG_INTEL_TDX_HOST=y, the TDX module is initialized and ready for KVM to use the TDX module APIs for TDX guest life cycle like tdh.mng.init are ready to use.
Concretely Global initialization, LP (Logical Processor) initialization, global configuration, the key configuration, and TDMR and PAMT initialization are done. The state of the TDX module is SYS_READY. Please refer to the TDX module specification, the chapter Intel TDX Module Lifecycle State Machine
** Detecting the TDX module readiness.
TDX host patch series implements the detection of the TDX module availability and its initialization so that KVM can use it. Also it manages Host KeyID (HKID) assigned to guest TD.
The assumed APIs the TDX host patch series provides are
- const struct tdsysinfo_struct *tdx_get_sysinfo(void);
Return the system wide information about the TDX module. NULL if the TDX isn't initialized.
- int tdx_enable(void);
Initialization of TDX module so that the TDX module is ready for KVM to use.
- extern u32 tdx_global_keyid __read_mostly;
global host key id that is used for the TDX module itself.
- u32 tdx_get_num_keyid(void);
return the number of available TDX private host key id.
- int tdx_keyid_alloc(void);
Allocate HKID for guest TD.
- void tdx_keyid_free(int keyid);
Free HKID for guest TD.
(****)
* TDX KVM high-level design
- Host key ID management
Host Key ID (HKID) needs to be assigned to each TDX guest for memory encryption. It is assumed The TDX host patch series implements necessary functions, u32 tdx_get_global_keyid(void), int tdx_keyid_alloc(void) and,
void tdx_keyid_free(int keyid).
- Data structures and VM type
Because TDX is different from VMX, define its own VM/VCPU structures, struct kvm_tdx and struct vcpu_tdx instead of struct kvm_vmx and struct vcpu_vmx. To identify the VM, introduce VM-type to specify which VM type, VMX (default) or TDX, is used.
- VM life cycle and TDX specific operations
Re-purpose the existing KVM_MEMORY_ENCRYPT_OP to add TDX specific operations. New commands are used to get the TDX system parameters, set TDX specific VM/VCPU parameters, set initial guest memory and measurement.
The creation of TDX VM requires five additional operations in addition to the conventional VM creation.
- Get KVM system capability to check if TDX VM type is supported
- VM creation (KVM_CREATE_VM)
- New: Get the TDX specific system parameters. KVM_TDX_GET_CAPABILITY.
- New: Set TDX specific VM parameters. KVM_TDX_INIT_VM.
- VCPU creation (KVM_CREATE_VCPU)
- New: Set TDX specific VCPU parameters. KVM_TDX_INIT_VCPU.
- New: Initialize guest memory as boot state and extend the measurement with the memory. KVM_TDX_INIT_MEM_REGION.
- New: Finalize VM. KVM_TDX_FINALIZE. Complete measurement of the initial TDX VM contents.
- VCPU RUN (KVM_VCPU_RUN)
- Protected guest state
Because the guest state (CPU state and guest memory) is protected, the KVM VMM can't operate on them. For example, accessing CPU registers, injecting exceptions, and accessing guest memory. Those operations are handled as silently ignored, returning zero or initial reset value when it's requested via KVM API ioctls.
VM/VCPU state and callbacks for TDX specific operations. Define tdx specific VM state and VCPU state instead of VMX ones. Redirect operations to TDX specific callbacks. "if (tdx) tdx_op() else vmx_op()".
Operations on the CPU state
silently ignore operations on the guest state. For example, the write to CPU registers is ignored and the read from CPU registers returns 0.
. ignore access to CPU registers except for allowed ones.
. TSC: add a check if tsc is immutable and return an error. Because the KVM
implementation updates the internal tsc state and it's difficult to back out those changes. Instead, skip the logic.
. dirty logging: add check if dirty logging is supported.
. exceptions/SMI/MCE/SIPI/INIT: silently ignore
Note: virtual external interrupt and NMI can be injected into TDX guests.
- KVM MMU integration
One bit of the guest physical address (bit 51 or 47) is repurposed to indicate if the guest physical address is private (the bit is cleared) or shared (the bit is set). The bits are called stolen bits.
- Stolen bits framework
systematically tracks which guest physical address, shared or private, is used.
- Shared EPT and secure EPT
There are two EPTs. Shared EPT (the conventional one) and Secure EPT(the new one). Shared EPT is handled the same for the stolen bit set. Secure EPT points to private guest pages. To resolve EPT violation, KVM walks one of two EPTs based on faulted GPA. Because it's costly to access secure EPT during walking EPTs with SEAMCALLs for the private guest physical address, another private EPT is used as a shadow of Secure-EPT with the existing logic at the cost of extra memory.
The following depicts the relationship.
KVM | TDX module
| | |
-------------+---------- | |
| | | |
V V | |
shared GPA private GPA | |
CPU shared EPT pointer KVM private EPT pointer | CPU secure EPT pointer
| | | |
| | | |
V V | V
shared EPT private EPT--------mirror----->Secure EPT
| | | |
| \--------------------+------\ |
| | | |
V | V V
shared guest page | private guest page
|
|
non-encrypted memory | encrypted memory
| - Operating on Secure EPT
Use the TDX module APIs to operate on Secure EPT. To call the TDX API during resolving EPT violation, add hooks to additional operation and wiring it to TDX backend.
* References
[1] TDX specification
https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html
[2] Intel Trust Domain Extensions (Intel TDX)
https://cdrdv2.intel.com/v1/dl/getContent/726790
[3] Intel CPU Architectural Extensions Specification
https://www.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-cpu-architectural-specification.pdf
[4] Intel TDX Module 1.0 Specification
https://www.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1.0-public-spec-v0.931.pdf
[5] Intel TDX Loader Interface Specification
https://www.intel.com/content/dam/develop/external/us/en/documents-tps/intel-tdx-seamldr-interface-specification.pdf
[6] Intel TDX Guest-Hypervisor Communication Interface
https://cdrdv2.intel.com/v1/dl/getContent/726790
[7] Intel TDX Virtual Firmware Design Guide
https://www.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.01.pdf
[8] intel public github
kvm TDX branch: https://github.com/intel/tdx/tree/kvm
TDX guest branch: https://github.com/intel/tdx/tree/guest
qemu TDX https://github.com/intel/qemu-tdx
[9] TDVF
https://github.com/tianocore/edk2-staging/tree/TDVF
This was merged into EDK2 main branch. https://github.com/tianocore/edk2
Chao Gao (2):
KVM: x86/mmu: Assume guest MMIOs are shared
KVM: x86: Allow to update cached values in kvm_user_return_msrs w/o
wrmsr
Isaku Yamahata (96):
KVM: x86: Add is_vm_type_supported callback
KVM: x86/vmx: initialize loaded_vmcss_on_cpu in vmx_hardware_setup()
KVM: x86/vmx: Refactor KVM VMX module init/exit functions
KVM: VMX: Reorder vmx initialization with kvm vendor initialization
KVM: TDX: Initialize the TDX module when loading the KVM intel kernel
module
KVM: TDX: Add placeholders for TDX VM/vcpu structure
KVM: TDX: Make TDX VM type supported
[MARKER] The start of TDX KVM patch series: TDX architectural
definitions
KVM: TDX: Define TDX architectural definitions
KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module
KVM: TDX: Retry SEAMCALL on the lack of entropy error
KVM: TDX: Add helper functions to print TDX SEAMCALL error [MARKER] The start of TDX KVM patch series: TD VM creation/destruction
KVM: TDX: Add helper functions to allocate/free TDX private host key
id
KVM: TDX: Add helper function to read TDX metadata in array x86/virt/tdx: Get system-wide info about TDX module on initialization
KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl
KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
KVM: TDX: create/destroy VM structure
KVM: TDX: initialize VM with TDX specific parameters
KVM: TDX: Make pmu_intel.c ignore guest TD case
KVM: TDX: Refuse to unplug the last cpu on the package [MARKER] The start of TDX KVM patch series: TD vcpu
creation/destruction
KVM: TDX: create/free TDX vcpu structure
KVM: TDX: Do TDX specific vcpu initialization
[MARKER] The start of TDX KVM patch series: KVM MMU GPA shared bits
KVM: x86/mmu: introduce config for PRIVATE KVM MMU
KVM: x86/mmu: Add address conversion functions for TDX shared bit of
GPA
[MARKER] The start of TDX KVM patch series: KVM TDP refactoring for
TDX
KVM: x86/mmu: Replace hardcoded value 0 for the initial value for SPTE
KVM: x86/mmu: Add Suppress VE bit to
shadow_mmio_mask/shadow_present_mask
KVM: x86/mmu: Track shadow MMIO value on a per-VM basis
KVM: x86/mmu: Disallow fast page fault on private GPA
KVM: VMX: Introduce test mode related to EPT violation VE [MARKER] The start of TDX KVM patch series: KVM TDP MMU hooks
KVM: x86/tdp_mmu: Init role member of struct kvm_mmu_page at
allocation
KVM: x86/mmu: Add a new is_private member for union kvm_mmu_page_role
KVM: x86/mmu: Add a private pointer to struct kvm_mmu_page
KVM: x86/tdp_mmu: Apply mmu notifier callback to only shared GPA
KVM: x86/tdp_mmu: Sprinkle __must_check
KVM: x86/tdp_mmu: Support TDX private mapping for TDP MMU [MARKER] The start of TDX KVM patch series: TDX EPT violation
KVM: TDX: Add accessors VMX VMCS helpers
KVM: TDX: Require TDP MMU and mmio caching for TDX
KVM: TDX: TDP MMU TDX support
KVM: TDX: MTRR: implement get_mt_mask() for TDX
[MARKER] The start of TDX KVM patch series: TD finalization
KVM: TDX: Create initial guest memory
KVM: TDX: Finalize VM initialization
[MARKER] The start of TDX KVM patch series: TD vcpu enter/exit
KVM: TDX: Implement TDX vcpu enter/exit path
KVM: TDX: vcpu_run: save/restore host state(host kernel gs)
KVM: TDX: restore host xsave state when exit from the guest TD
KVM: TDX: restore user ret MSRs
[MARKER] The start of TDX KVM patch series: TD vcpu
exits/interrupts/hypercalls
KVM: TDX: complete interrupts after tdexit
KVM: TDX: restore debug store when TD exit
KVM: TDX: handle vcpu migration over logical processor
KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched
behavior
KVM: TDX: remove use of struct vcpu_vmx from posted_interrupt.c
KVM: TDX: Implement interrupt injection
KVM: TDX: Implements vcpu request_immediate_exit
KVM: TDX: Implement methods to inject NMI
KVM: TDX: Add a place holder to handle TDX VM exit
KVM: TDX: handle EXIT_REASON_OTHER_SMI
KVM: TDX: handle ept violation/misconfig exit
KVM: TDX: handle EXCEPTION_NMI and EXTERNAL_INTERRUPT
KVM: TDX: Handle EXIT_REASON_OTHER_SMI with MSMI
KVM: TDX: Add a place holder for handler of TDX hypercalls
(TDG.VP.VMCALL)
KVM: TDX: handle KVM hypercall with TDG.VP.VMCALL
KVM: TDX: Add KVM Exit for TDX TDG.VP.VMCALL
KVM: TDX: Handle TDX PV CPUID hypercall
KVM: TDX: Handle TDX PV HLT hypercall
KVM: TDX: Handle TDX PV port io hypercall
KVM: TDX: Implement callbacks for MSR operations for TDX
KVM: TDX: Handle TDX PV rdmsr/wrmsr hypercall
KVM: TDX: Handle MSR MTRRCap and MTRRDefType access
KVM: TDX: Handle MSR IA32_FEAT_CTL MSR and IA32_MCG_EXT_CTL
KVM: TDX: Handle TDG.VP.VMCALL<GetTdVmCallInfo> hypercall
KVM: TDX: Silently discard SMI request
KVM: TDX: Silently ignore INIT/SIPI
KVM: TDX: Add methods to ignore accesses to CPU state
KVM: TDX: Add methods to ignore guest instruction emulation
KVM: TDX: Add a method to ignore dirty logging
KVM: TDX: Add methods to ignore VMX preemption timer
KVM: TDX: Add methods to ignore accesses to TSC
KVM: TDX: Ignore setting up mce
KVM: TDX: Add a method to ignore for TDX to ignore hypercall patch
KVM: TDX: Add methods to ignore virtual apic related operation
KVM: TDX: Inhibit APICv for TDX guest Documentation/virt/kvm: Document on Trust Domain Extensions(TDX)
KVM: x86: design documentation on TDX support of x86 KVM TDP MMU
KVM: TDX: Add hint TDX ioctl to release Secure-EPT
RFC: KVM: x86: Add x86 callback to check cpuid
RFC: KVM: x86, TDX: Add check for KVM_SET_CPUID2 [MARKER] the end of (the first phase of) TDX KVM patch series
Kai Huang (2):
x86/virt/tdx: Export TDX KeyID information
x86/virt/tdx: Export SEAMCALL functions
Sean Christopherson (17):
KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
KVM: TDX: Add TDX "architectural" error codes
KVM: TDX: x86: Add ioctl to get TDX systemwide parameters
KVM: Allow page-sized MMU caches to be initialized with custom 64-bit
values
KVM: x86/mmu: Allow non-zero value for non-present SPTE and removed
SPTE
KVM: x86/mmu: Allow per-VM override of the TDP max page level
KVM: x86/tdp_mmu: Don't zap private pages for unsupported cases
KVM: VMX: Split out guts of EPT violation to common/exposed function
KVM: VMX: Move setting of EPT MMU masks to common VT-x code
KVM: TDX: Add load_mmu_pgd method for TDX
KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX
KVM: TDX: Add support for find pending IRQ in a protected local APIC
KVM: x86: Assume timer IRQ was injected if APIC state is proteced
KVM: VMX: Modify NMI and INTR handlers to take intr_info as function
argument
KVM: VMX: Move NMI/exception handler to common helper
KVM: x86: Split core of hypercall emulation to helper function
KVM: TDX: Handle TDX PV MMIO hypercall
Yan Zhao (1):
KVM: x86/mmu: TDX: Do not enable page track for TD guest
Yang Weijiang (1):
KVM: TDX: Add TSX_CTRL msr into uret_msrs list
Yao Yuan (1):
KVM: TDX: Handle vmentry failure for INTEL TD guest
Yuan Yao (1):
KVM: TDX: Retry seamcall when TDX_OPERAND_BUSY with operand SEPT
Documentation/virt/kvm/api.rst | 9 +- Documentation/virt/kvm/index.rst | 1 + Documentation/virt/kvm/x86/index.rst | 2 + Documentation/virt/kvm/x86/intel-tdx.rst | 362 +++ Documentation/virt/kvm/x86/tdx-tdp-mmu.rst | 443 +++ arch/x86/events/intel/ds.c | 1 + arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/include/asm/kvm-x86-ops.h | 18 +- arch/x86/include/asm/kvm_host.h | 85 +- arch/x86/include/asm/tdx.h | 5 + arch/x86/include/asm/vmx.h | 14 + arch/x86/include/uapi/asm/kvm.h | 95 + arch/x86/include/uapi/asm/vmx.h | 5 +- arch/x86/kvm/Kconfig | 7 +- arch/x86/kvm/Makefile | 3 +- arch/x86/kvm/cpuid.c | 27 +- arch/x86/kvm/cpuid.h | 2 + arch/x86/kvm/irq.c | 3 + arch/x86/kvm/lapic.c | 33 +- arch/x86/kvm/lapic.h | 2 + arch/x86/kvm/mmu.h | 31 + arch/x86/kvm/mmu/mmu.c | 200 +- arch/x86/kvm/mmu/mmu_internal.h | 109 +- arch/x86/kvm/mmu/page_track.c | 3 + arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.c | 17 +- arch/x86/kvm/mmu/spte.h | 27 +- arch/x86/kvm/mmu/tdp_iter.h | 14 +- arch/x86/kvm/mmu/tdp_mmu.c | 442 ++- arch/x86/kvm/mmu/tdp_mmu.h | 7 +- arch/x86/kvm/smm.h | 7 +- arch/x86/kvm/svm/svm.c | 8 + arch/x86/kvm/vmx/common.h | 166 + arch/x86/kvm/vmx/main.c | 1246 ++++++++ arch/x86/kvm/vmx/pmu_intel.c | 46 +- arch/x86/kvm/vmx/pmu_intel.h | 28 + arch/x86/kvm/vmx/posted_intr.c | 43 +- arch/x86/kvm/vmx/posted_intr.h | 13 + arch/x86/kvm/vmx/tdx.c | 3321 ++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 266 ++ arch/x86/kvm/vmx/tdx_arch.h | 277 ++ arch/x86/kvm/vmx/tdx_errno.h | 44 + arch/x86/kvm/vmx/tdx_error.c | 21 + arch/x86/kvm/vmx/tdx_ops.h | 408 +++ arch/x86/kvm/vmx/vmcs.h | 5 + arch/x86/kvm/vmx/vmx.c | 661 ++-- arch/x86/kvm/vmx/vmx.h | 52 +- arch/x86/kvm/vmx/x86_ops.h | 257 ++ arch/x86/kvm/x86.c | 129 +- arch/x86/kvm/x86.h | 4 + arch/x86/virt/vmx/tdx/seamcall.S | 4 + arch/x86/virt/vmx/tdx/tdx.c | 11 +- include/linux/kvm_host.h | 1 + include/linux/kvm_types.h | 1 + include/uapi/linux/kvm.h | 89 + virt/kvm/kvm_main.c | 31 +- 56 files changed, 8407 insertions(+), 702 deletions(-) create mode 100644 Documentation/virt/kvm/x86/intel-tdx.rst create mode 100644 Documentation/virt/kvm/x86/tdx-tdp-mmu.rst create mode 100644 arch/x86/kvm/vmx/common.h create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/pmu_intel.h create mode 100644 arch/x86/kvm/vmx/tdx.c create mode 100644 arch/x86/kvm/vmx/tdx.h create mode 100644 arch/x86/kvm/vmx/tdx_arch.h create mode 100644 arch/x86/kvm/vmx/tdx_errno.h create mode 100644 arch/x86/kvm/vmx/tdx_error.c create mode 100644 arch/x86/kvm/vmx/tdx_ops.h create mode 100644 arch/x86/kvm/vmx/x86_ops.h
base-commit: 6613476e225e090cc9aad49be7fa504e290dd33d
prerequisite-patch-id: 5e29e9200c65fb7f6213b4aa85254f128a4fc49f
prerequisite-patch-id: 39908082e873a3828568cc6c626e734d4ccb279a
prerequisite-patch-id: 01d54029211a041370ee12d58825c42f2255d3f8
prerequisite-patch-id: ec2e5dc132d37f2ec76f56172fe82e0f30998a50
prerequisite-patch-id: b9438c767dbd45f4dda5287e104a54fe8f3c516f
prerequisite-patch-id: ef4b52b28c7459a217b5fc4bd4dc592354fb9a46
prerequisite-patch-id: bc425a71343a9fed0ad12e5b85a1c614f8eff934
prerequisite-patch-id: cb538a88ed4c5d7614d3e6fcd10b991cce605a0d
prerequisite-patch-id: 6019835e0581c2749e8e353f2045a2e94040a2b0
prerequisite-patch-id: 4dd00540050377ff852c0a939682d5894513444c
Signature · 1 line
2.25.1
PATCH v18 1/121 x86/virt/tdx: Export TDX KeyID information
From: Kai Huang <[email protected]>
Each TDX guest must be protected by its own unique TDX KeyID. KVM will need to tell the TDX module the unique KeyID for a TDX guest when KVM creates it.
Export the TDX KeyID range that can be used by TDX guests for KVM to use. KVM can then manage these KeyIDs and assign one for each TDX guest when it is created.
Each TDX guest has a root control structure called "Trust Domain Root" (TDR). Unlike the rest of the TDX guest, the TDR is protected by the TDX global KeyID. When tearing down the TDR, KVM will need to pass the TDX global KeyID explicitly to the TDX module to flush cache associated to the TDR.
Also export the TDX global KeyID for KVM to tear down the TDR.
--- arch/x86/include/asm/tdx.h | 5 +++++ arch/x86/virt/vmx/tdx/tdx.c | 11 ++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-)
arch/x86/include/asm/tdx.h +5 -0
@@ -88,6 +88,11 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,#endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */ #ifdef CONFIG_INTEL_TDX_HOST + +extern u32 tdx_global_keyid; +extern u32 tdx_guest_keyid_start; +extern u32 tdx_nr_guest_keyids; +u64 __seamcall (u64 fn,struct tdx_module_args *args);u64 __seamcall_ret (u64 fn,struct tdx_module_args *args);u64 __seamcall_saved_ret (u64 fn,struct tdx_module_args *args);
arch/x86/virt/vmx/tdx/tdx.c +8 -3
@@ -39,9 +39,14 @@#include <asm/mce.h> #include "tdx.h" -static u32 tdx_global_keyid__ro_after_init ; -static u32 tdx_guest_keyid_start__ro_after_init ; -static u32 tdx_nr_guest_keyids__ro_after_init ; +u32 tdx_global_keyid__ro_after_init ; +EXPORT_SYMBOL_GPL (tdx_global_keyid); + +u32 tdx_guest_keyid_start__ro_after_init ; +EXPORT_SYMBOL_GPL (tdx_guest_keyid_start); + +u32 tdx_nr_guest_keyids__ro_after_init ; +EXPORT_SYMBOL_GPL (tdx_nr_guest_keyids); static DEFINE_PER_CPU(bool, tdx_lp_initialized);
Signature · 1 line
2.25.1
Re: [PATCH v18 001/121] x86/virt/tdx: Export TDX KeyID information
Quoted 16 lines · On Mon, Jan 22, 2024 at 03:52:37PM -0800, [email protected] wrote:
From: Kai Huang <[email protected]>
Each TDX guest must be protected by its own unique TDX KeyID. KVM will need to tell the TDX module the unique KeyID for a TDX guest when KVM creates it.
Export the TDX KeyID range that can be used by TDX guests for KVM to use. KVM can then manage these KeyIDs and assign one for each TDX guest when it is created.
Each TDX guest has a root control structure called "Trust Domain Root" (TDR). Unlike the rest of the TDX guest, the TDR is protected by the TDX global KeyID. When tearing down the TDR, KVM will need to pass the TDX global KeyID explicitly to the TDX module to flush cache associated to the TDR.
Quoted 3 lines
Also export the TDX global KeyID for KVM to tear down the TDR.
Signed-off-by Kai Huang <[email protected]>
The variables exported by this patch are used first time in patch 18 IIUC... So how about move this one just before the patch 18 ?
Quoted 47 lines
--- arch/x86/include/asm/tdx.h | 5 +++++ arch/x86/virt/vmx/tdx/tdx.c | 11 ++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-)arch/x86/include/asm/tdx.h +5
-0@@ -88,6 +88,11 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,#endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */ #ifdef CONFIG_INTEL_TDX_HOST + +extern u32 tdx_global_keyid; +extern u32 tdx_guest_keyid_start; +extern u32 tdx_nr_guest_keyids; +u64 __seamcall (u64 fn,struct tdx_module_args *args);u64 __seamcall_ret (u64 fn,struct tdx_module_args *args);u64 __seamcall_saved_ret (u64 fn,struct tdx_module_args *args);arch/x86/virt/vmx/tdx/tdx.c +8
-4@@ -39,9 +39,14 @@#include <asm/mce.h> #include "tdx.h" -static u32 tdx_global_keyid__ro_after_init ; -static u32 tdx_guest_keyid_start__ro_after_init ; -static u32 tdx_nr_guest_keyids__ro_after_init ; +u32 tdx_global_keyid__ro_after_init ; +EXPORT_SYMBOL_GPL (tdx_global_keyid); + +u32 tdx_guest_keyid_start__ro_after_init ; +EXPORT_SYMBOL_GPL (tdx_guest_keyid_start); + +u32 tdx_nr_guest_keyids__ro_after_init ; +EXPORT_SYMBOL_GPL (tdx_nr_guest_keyids); static DEFINE_PER_CPU(bool, tdx_lp_initialized); --2.25.1
Re: [PATCH v18 001/121] x86/virt/tdx: Export TDX KeyID information
Quoted 17 lines · Yuan Yao
Quoted 16 lines · On Mon, Jan 22, 2024 at 03:52:37PM -0800, [email protected] wrote:
From: Kai Huang <[email protected]>
Each TDX guest must be protected by its own unique TDX KeyID. KVM will need to tell the TDX module the unique KeyID for a TDX guest when KVM creates it.
Export the TDX KeyID range that can be used by TDX guests for KVM to use. KVM can then manage these KeyIDs and assign one for each TDX guest when it is created.
Each TDX guest has a root control structure called "Trust Domain Root" (TDR). Unlike the rest of the TDX guest, the TDR is protected by the TDX global KeyID. When tearing down the TDR, KVM will need to pass the TDX global KeyID explicitly to the TDX module to flush cache associated to the TDR.
Quoted 6 lines
Quoted 3 lines
Also export the TDX global KeyID for KVM to tear down the TDR.
Signed-off-by Kai Huang <[email protected]>The variables exported by this patch are used first time in patch 18 IIUC... So how about move this one just before the patch 18 ?
With v19, I put those tdx host patches in the first part.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 2/121 x86/virt/tdx: Export SEAMCALL functions
In reply to [email protected]From: Kai Huang <[email protected]>
KVM will need to make SEAMCALLs to create and run TDX guests. Export SEAMCALL functions for KVM to use.
Also add declaration of SEAMCALL functions to <asm/asm-prototypes.h> to support CONFIG_MODVERSIONS=y.
--- arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/virt/vmx/tdx/seamcall.S | 4 ++++ 2 files changed, 5 insertions(+)
arch/x86/include/asm/asm-prototypes.h +1 -0
@@ -13,6 +13,7 @@#include <asm/preempt.h> #include <asm/asm.h> #include <asm/gsseg.h> +#include <asm/tdx.h> #ifndef CONFIG_X86_CMPXCHG64 extern void cmpxchg8b_emu (void );
arch/x86/virt/vmx/tdx/seamcall.S +4 -0
@@ -1,5 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include <linux/linkage.h> +#include <linux/export.h> #include <asm/frame.h> #include "tdxcall.S"
@@ -21,6 +22,7 @@ SYM_FUNC_START(__seamcall) TDX_MODULE_CALL host=1 SYM_FUNC_END(__seamcall) +EXPORT_SYMBOL_GPL(__seamcall); /* * __seamcall_ret() - Host-side interface functions to SEAM software
@@ -40,6 +42,7 @@ SYM_FUNC_END(__seamcall) SYM_FUNC_START(__seamcall_ret) TDX_MODULE_CALL host=1 ret=1 SYM_FUNC_END(__seamcall_ret) +EXPORT_SYMBOL_GPL(__seamcall_ret); /* * __seamcall_saved_ret() - Host-side interface functions to SEAM software
@@ -59,3 +62,4 @@ SYM_FUNC_END(__seamcall_ret) SYM_FUNC_START(__seamcall_saved_ret) TDX_MODULE_CALL host=1 ret=1 saved=1 SYM_FUNC_END(__seamcall_saved_ret) +EXPORT_SYMBOL_GPL(__seamcall_saved_ret);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [PATCH v18 002/121] x86/virt/tdx: Export SEAMCALL functions
Quoted 10 lines · On Tue, Jan 23, 2024 at 12:54 AM <[email protected]> wrote:
From: Kai Huang <[email protected]>
KVM will need to make SEAMCALLs to create and run TDX guests. Export SEAMCALL functions for KVM to use.
Also add declaration of SEAMCALL functions to <
asm/asm-prototypes.h> to supportCONFIG_MODVERSIONS=y.Signed-off-by Kai Huang <[email protected]>
I thought you were going to introduce builtin functions for SEAMCALLs needed by KVM, instead?
In any case, missing Signed-off-by.
Paolo
Quoted 52 lines
--- arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/virt/vmx/tdx/seamcall.S | 4 ++++ 2 files changed, 5 insertions(+)arch/x86/include/asm/asm-prototypes.h +1
-0@@ -13,6 +13,7 @@#include <asm/preempt.h> #include <asm/asm.h> #include <asm/gsseg.h> +#include <asm/tdx.h> #ifndef CONFIG_X86_CMPXCHG64 extern void cmpxchg8b_emu (void );arch/x86/virt/vmx/tdx/seamcall.S +4
-1@@ -1,5 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include <linux/linkage.h> +#include <linux/export.h> #include <asm/frame.h> #include "tdxcall.S"@@ -21,6 +22,7 @@ SYM_FUNC_START(__seamcall) TDX_MODULE_CALL host=1 SYM_FUNC_END(__seamcall) +EXPORT_SYMBOL_GPL(__seamcall); /* * __seamcall_ret() - Host-side interface functions to SEAM software@@ -40,6 +42,7 @@ SYM_FUNC_END(__seamcall) SYM_FUNC_START(__seamcall_ret) TDX_MODULE_CALL host=1 ret=1 SYM_FUNC_END(__seamcall_ret) +EXPORT_SYMBOL_GPL(__seamcall_ret); /* * __seamcall_saved_ret() - Host-side interface functions to SEAM software@@ -59,3 +62,4 @@ SYM_FUNC_END(__seamcall_ret) SYM_FUNC_START(__seamcall_saved_ret) TDX_MODULE_CALL host=1 ret=1 saved=1 SYM_FUNC_END(__seamcall_saved_ret) +EXPORT_SYMBOL_GPL(__seamcall_saved_ret); --2.25.1
Re: [PATCH v18 002/121] x86/virt/tdx: Export SEAMCALL functions
Quoted 7 lines · Paolo Bonzini
Quoted 6 lines · On Tue, Jan 23, 2024 at 12:54 AM <[email protected]> wrote:
From: Kai Huang <[email protected]>
KVM will need to make SEAMCALLs to create and run TDX guests. Export SEAMCALL functions for KVM to use.
Quoted 7 lines
Quoted 4 lines
Also add declaration of SEAMCALL functions to <
asm/asm-prototypes.h> to supportCONFIG_MODVERSIONS=y.Signed-off-by Kai Huang <[email protected]>I thought you were going to introduce builtin functions for SEAMCALLs needed by KVM, instead?
After I talked with Kai, I concluded to use the common function. Probably for TDH.VP.ENTER, we'd like to use customized version to avoid shuffling registers between struct tdx_module_args and KVM regs. We can do it later as optimization.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 3/121 KVM: x86: Add is_vm_type_supported callback
In reply to [email protected]From: Isaku Yamahata <[email protected]>
For TDX, allow the backend can override the supported vm type. Add KVM_X86_TDX_VM to reserve the bit.
---
v18:
- include into TDX KVM patch series v18
Changes v3 -> v4:
- Added KVM_X86_SNP_VM
Changes v2 -> v3:
- no change
- didn't bother to rename KVM_X86_PROTECTED_VM to KVM_X86_SW_PROTECTED_VM
Changes v1 -> v2
- no change
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 2 ++ arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 12 +++++++++++- arch/x86/kvm/x86.h | 2 ++ 7 files changed, 31 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -20,6 +20,7 @@ KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1601,6 +1601,7 @@ struct kvm_x86_ops {bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); +bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);
arch/x86/include/uapi/asm/kvm.h +2 -0
@@ -564,5 +564,7 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_SW_PROTECTED_VM 1 +#define KVM_X86_TDX_VM 2 +#define KVM_X86_SNP_VM 3#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/svm/svm.c +7 -0
@@ -4886,6 +4886,12 @@ static void svm_vm_destroy(struct kvm *kvm)sev_vm_destroy (kvm); } +static bool svm_is_vm_type_supported (unsigned long type) +{ +/* FIXME: Check if CPU is capable of SEV-SNP. */ +return __kvm_is_vm_type_supported (type); +} +static int svm_vm_init (struct kvm *kvm) {if (!pause_filter_count || !pause_filter_thresh)
@@ -4914,6 +4920,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_vcpu_free, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy,
arch/x86/kvm/vmx/vmx.c +7 -0
@@ -7541,6 +7541,12 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } +static bool vmx_is_vm_type_supported (unsigned long type) +{ +/* TODO: Check if TDX is supported. */ +return __kvm_is_vm_type_supported (type); +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
@@ -8263,6 +8269,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init, .vm_destroy = vmx_vm_destroy,
arch/x86/kvm/x86.c +11 -1
@@ -4577,12 +4577,18 @@ static int kvm_ioctl_get_supported_hv_cpuid(struct kvm_vcpu *vcpu, }#endif -static bool kvm_is_vm_type_supported (unsigned long type) +bool __kvm_is_vm_type_supported (unsigned long type) {return type ==KVM_X86_DEFAULT_VM || (type ==KVM_X86_SW_PROTECTED_VM &&IS_ENABLED (CONFIG_KVM_SW_PROTECTED_VM ) && tdp_enabled); } +EXPORT_SYMBOL_GPL (__kvm_is_vm_type_supported); + +static bool kvm_is_vm_type_supported (unsigned long type) +{ +return static_call (kvm_x86_is_vm_type_supported)(type); +}int kvm_vm_ioctl_check_extension (struct kvm *kvm,long ext) {
@@ -4784,6 +4790,10 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r =BIT (KVM_X86_DEFAULT_VM );if (kvm_is_vm_type_supported (KVM_X86_SW_PROTECTED_VM )) r |=BIT (KVM_X86_SW_PROTECTED_VM ); +if (kvm_is_vm_type_supported (KVM_X86_TDX_VM )) + r |=BIT (KVM_X86_TDX_VM ); +if (kvm_is_vm_type_supported (KVM_X86_SNP_VM )) + r |=BIT (KVM_X86_SNP_VM );break ;default :break ;
arch/x86/kvm/x86.h +2 -0
@@ -9,6 +9,8 @@#include "kvm_cache_regs.h" #include "kvm_emulate.h" +bool __kvm_is_vm_type_supported (unsigned long type); +struct kvm_caps {/* control of guest tsc rate supported? */ bool has_tsc_control;
Signature · 1 line
2.25.1
Re: [PATCH v18 003/121] KVM: x86: Add is_vm_type_supported callback
Quoted 4 lines · On Mon, Jan 22, 2024 at 03:52:39PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
For TDX, allow the backend can override the supported vm type. Add
KVM_X86_TDX_VMto reserve the bit.
Please highlight that KVM_X86_SNP_VM is also defined for AMD SEV-SNP in this patch.
Quoted 164 lines
Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- include into TDX KVM patch series v18Changes v3 -> v4:
- AddedKVM_X86_SNP_VMChanges v2 -> v3:
- no change
- didn't bother to renameKVM_X86_PROTECTED_VMtoKVM_X86_SW_PROTECTED_VMChanges v1 -> v2
- no change--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 2 ++ arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 12 +++++++++++- arch/x86/kvm/x86.h | 2 ++ 7 files changed, 31 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -20,6 +20,7 @@ KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)arch/x86/include/asm/kvm_host.h +1
-0@@ -1601,6 +1601,7 @@ struct kvm_x86_ops {bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); +bool (*is_vm_type_supported)(unsigned long vm_type);unsigned int vm_size;int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);arch/x86/include/uapi/asm/kvm.h +2
-0@@ -564,5 +564,7 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_SW_PROTECTED_VM 1 +#define KVM_X86_TDX_VM 2 +#define KVM_X86_SNP_VM 3#endif /* _ASM_X86_KVM_H */ arch/x86/kvm/svm/svm.c +7
-0@@ -4886,6 +4886,12 @@ static void svm_vm_destroy(struct kvm *kvm)sev_vm_destroy (kvm); } +static bool svm_is_vm_type_supported (unsigned long type) +{ +/* FIXME: Check if CPU is capable of SEV-SNP. */ +return __kvm_is_vm_type_supported (type); +} +static int svm_vm_init (struct kvm *kvm) {if (!pause_filter_count || !pause_filter_thresh)@@ -4914,6 +4920,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_vcpu_free, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy,arch/x86/kvm/vmx/vmx.c +7
-0@@ -7541,6 +7541,12 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } +static bool vmx_is_vm_type_supported (unsigned long type) +{ +/* TODO: Check if TDX is supported. */ +return __kvm_is_vm_type_supported (type); +} +#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"@@ -8263,6 +8269,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init, .vm_destroy = vmx_vm_destroy,arch/x86/kvm/x86.c +11
-1@@ -4577,12 +4577,18 @@ static int kvm_ioctl_get_supported_hv_cpuid(struct kvm_vcpu *vcpu, }#endif -static bool kvm_is_vm_type_supported (unsigned long type) +bool __kvm_is_vm_type_supported (unsigned long type) {return type ==KVM_X86_DEFAULT_VM || (type ==KVM_X86_SW_PROTECTED_VM &&IS_ENABLED (CONFIG_KVM_SW_PROTECTED_VM ) && tdp_enabled); } +EXPORT_SYMBOL_GPL (__kvm_is_vm_type_supported); + +static bool kvm_is_vm_type_supported (unsigned long type) +{ +return static_call (kvm_x86_is_vm_type_supported)(type); +}int kvm_vm_ioctl_check_extension (struct kvm *kvm,long ext) {@@ -4784,6 +4790,10 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r =BIT (KVM_X86_DEFAULT_VM );if (kvm_is_vm_type_supported (KVM_X86_SW_PROTECTED_VM )) r |=BIT (KVM_X86_SW_PROTECTED_VM ); +if (kvm_is_vm_type_supported (KVM_X86_TDX_VM )) + r |=BIT (KVM_X86_TDX_VM ); +if (kvm_is_vm_type_supported (KVM_X86_SNP_VM )) + r |=BIT (KVM_X86_SNP_VM );break ;default :break ;arch/x86/kvm/x86.h +2
-1@@ -9,6 +9,8 @@#include "kvm_cache_regs.h" #include "kvm_emulate.h" +bool __kvm_is_vm_type_supported (unsigned long type); +struct kvm_caps {/* control of guest tsc rate supported? */ bool has_tsc_control; --2.25.1
PATCH v18 4/121 KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
In reply to [email protected]From: Sean Christopherson <[email protected]>
KVM accesses Virtual Machine Control Structure (VMCS) with VMX instructions to operate on VM. TDX doesn't allow VMM to operate VMCS directly. Instead, TDX has its own data structures, and TDX SEAMCALL APIs for VMM to indirectly operate those data structures. This means we must have a TDX version of kvm_x86_ops.
The existing global struct kvm_x86_ops already defines an interface which fits with TDX. But kvm_x86_ops is system-wide, not per-VM structure. To allow VMX to coexist with TDs, the kvm_x86_ops callbacks will have wrappers "if (tdx) tdx_op() else vmx_op()" to switch VMX or TDX at run time.
To split the runtime switch, the VMX implementation, and the TDX implementation, add main.c, and move out the vmx_x86_ops hooks in preparation for adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.
The current code looks as follows.
In vmx.c
static vmx_op() { ... }
static struct kvm_x86_ops vmx_x86_ops = {
.op = vmx_op,initialization code
The eventually converted code will look like In vmx.c, keep the VMX operations.
vmx_op() { ... }
VMX initialization
In tdx.c, define the TDX operations.
tdx_op() { ... }
TDX initialization
In x86_ops.h, declare the VMX and TDX operations.
vmx_op();
tdx_op();
In main.c, define common wrappers for VMX and TDX.
static vt_ops() { if (tdx) tdx_ops() else vmx_ops() }
static struct kvm_x86_ops vt_x86_ops = {
.op = vt_op,initialization to call VMX and TDX initialization
Opportunistically, fix the name inconsistency from vmx_create_vcpu() and vmx_free_vcpu() to vmx_vcpu_create() and vmx_vcpu_free().
---
v18:
- Add Reviewed-by: Binbin Wu
- fix indent alignments pointed by Binbin Wu
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 169 +++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 378 ++++++++++--------------------------- arch/x86/kvm/vmx/x86_ops.h | 125 ++++++++++++ 4 files changed, 398 insertions(+), 276 deletions(-) create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/x86_ops.h
arch/x86/kvm/Makefile +1 -1
@@ -20,7 +20,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-$(CONFIG_KVM_SMM) += smm.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/nested.o vmx/posted_intr.o + vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o
arch/x86/kvm/vmx/main.c new file +169 -0
169 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.c +103 -275
933 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h new file +125 -0
125 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH v18 004/121] KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
Quoted 45 lines · On Mon, Jan 22, 2024 at 03:52:40PM -0800, [email protected] wrote:
From: Sean Christopherson <[email protected]>
KVM accesses Virtual Machine Control Structure (VMCS) with VMX instructions to operate on VM. TDX doesn't allow VMM to operate VMCS directly. Instead, TDX has its own data structures, and TDX SEAMCALL APIs for VMM to indirectly operate those data structures. This means we must have a TDX version of
kvm_x86_ops.The existing global struct
kvm_x86_opsalready defines an interface which fits with TDX. Butkvm_x86_opsis system-wide, not per-VM structure. To allow VMX to coexist with TDs, thekvm_x86_opscallbacks will have wrappers "if (tdx)tdx_op()elsevmx_op()" to switch VMX or TDX at run time.To split the runtime switch, the VMX implementation, and the TDX implementation, add
main.c, and move out thevmx_x86_opshooks in preparation for adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.The current code looks as follows.
Invmx.c
staticvmx_op(){ ... }static struct kvm_x86_ops vmx_x86_ops = { .op = vmx_op,initialization code
The eventually converted code will look like In
vmx.c, keep the VMX operations.
vmx_op(){ ... }
VMX initialization
Intdx.c, define the TDX operations.
tdx_op(){ ... }
TDX initialization
Inx86_ops.h, declare the VMX and TDX operations.
vmx_op();
tdx_op();
Inmain.c, define common wrappers for VMX and TDX.
staticvt_ops(){ if (tdx)tdx_ops()elsevmx_ops()}static struct kvm_x86_ops vt_x86_ops = { .op = vt_op,initialization to call VMX and TDX initialization
Opportunistically, fix the name inconsistency from
vmx_create_vcpu()andvmx_free_vcpu()tovmx_vcpu_create()andvmx_vcpu_free().
Quoted 6 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>Reviewed-by Binbin Wu <[email protected]>
Quoted 1,358 lines
---
v18:
- Add Reviewed-by: Binbin Wu
- fix indent alignments pointed by Binbin Wu--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 169 +++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 378 ++++++++++--------------------------- arch/x86/kvm/vmx/x86_ops.h | 125 ++++++++++++ 4 files changed, 398 insertions(+), 276 deletions(-) create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/x86_ops.harch/x86/kvm/Makefile +1
-1@@ -20,7 +20,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-$(CONFIG_KVM_SMM) += smm.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/nested.o vmx/posted_intr.o + vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.oarch/x86/kvm/vmx/main.c new file +169
-0@@ -0,0 +1,169 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/moduleparam.h> + +#include "x86_ops.h" +#include "vmx.h" +#include "nested.h" +#include "pmu.h" + +#define VMX_REQUIRED_APICV_INHIBITS \ + (BIT(APICV_INHIBIT_REASON_DISABLE)| \ + BIT(APICV_INHIBIT_REASON_ABSENT) | \ + BIT(APICV_INHIBIT_REASON_HYPERV) | \ + BIT(APICV_INHIBIT_REASON_BLOCKIRQ) | \ + BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) | \ + BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ + BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED)) + +struct kvm_x86_ops vt_x86_ops__initdata = { + .name =KBUILD_MODNAME , + + .check_processor_compatibility = vmx_check_processor_compat, + + .hardware_unsetup = vmx_hardware_unsetup, + + .hardware_enable = vmx_hardware_enable, + .hardware_disable = vmx_hardware_disable, + .has_emulated_msr = vmx_has_emulated_msr, + + .is_vm_type_supported = vmx_is_vm_type_supported, + .vm_size =sizeof (struct kvm_vmx ), + .vm_init = vmx_vm_init, + .vm_destroy = vmx_vm_destroy, + + .vcpu_precreate = vmx_vcpu_precreate, + .vcpu_create = vmx_vcpu_create, + .vcpu_free = vmx_vcpu_free, + .vcpu_reset = vmx_vcpu_reset, + + .prepare_switch_to_guest = vmx_prepare_switch_to_guest, + .vcpu_load = vmx_vcpu_load, + .vcpu_put = vmx_vcpu_put, + + .update_exception_bitmap = vmx_update_exception_bitmap, + .get_msr_feature = vmx_get_msr_feature, + .get_msr = vmx_get_msr, + .set_msr = vmx_set_msr, + .get_segment_base = vmx_get_segment_base, + .get_segment = vmx_get_segment, + .set_segment = vmx_set_segment, + .get_cpl = vmx_get_cpl, + .get_cs_db_l_bits = vmx_get_cs_db_l_bits, + .is_valid_cr0 = vmx_is_valid_cr0, + .set_cr0 = vmx_set_cr0, + .is_valid_cr4 = vmx_is_valid_cr4, + .set_cr4 = vmx_set_cr4, + .set_efer = vmx_set_efer, + .get_idt = vmx_get_idt, + .set_idt = vmx_set_idt, + .get_gdt = vmx_get_gdt, + .set_gdt = vmx_set_gdt, + .set_dr7 = vmx_set_dr7, + .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, + .cache_reg = vmx_cache_reg, + .get_rflags = vmx_get_rflags, + .set_rflags = vmx_set_rflags, + .get_if_flag = vmx_get_if_flag, + + .flush_tlb_all = vmx_flush_tlb_all, + .flush_tlb_current = vmx_flush_tlb_current, + .flush_tlb_gva = vmx_flush_tlb_gva, + .flush_tlb_guest = vmx_flush_tlb_guest, + + .vcpu_pre_run = vmx_vcpu_pre_run, + .vcpu_run = vmx_vcpu_run, + .handle_exit = vmx_handle_exit, + .skip_emulated_instruction = vmx_skip_emulated_instruction, + .update_emulated_instruction = vmx_update_emulated_instruction, + .set_interrupt_shadow = vmx_set_interrupt_shadow, + .get_interrupt_shadow = vmx_get_interrupt_shadow, + .patch_hypercall = vmx_patch_hypercall, + .inject_irq = vmx_inject_irq, + .inject_nmi = vmx_inject_nmi, + .inject_exception = vmx_inject_exception, + .cancel_injection = vmx_cancel_injection, + .interrupt_allowed = vmx_interrupt_allowed, + .nmi_allowed = vmx_nmi_allowed, + .get_nmi_mask = vmx_get_nmi_mask, + .set_nmi_mask = vmx_set_nmi_mask, + .enable_nmi_window = vmx_enable_nmi_window, + .enable_irq_window = vmx_enable_irq_window, + .update_cr8_intercept = vmx_update_cr8_intercept, + .set_virtual_apic_mode = vmx_set_virtual_apic_mode, + .set_apic_access_page_addr = vmx_set_apic_access_page_addr, + .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, + .load_eoi_exitmap = vmx_load_eoi_exitmap, + .apicv_pre_state_restore = vmx_apicv_pre_state_restore, + .required_apicv_inhibits =VMX_REQUIRED_APICV_INHIBITS , + .hwapic_irr_update = vmx_hwapic_irr_update, + .hwapic_isr_update = vmx_hwapic_isr_update, + .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, + .sync_pir_to_irr = vmx_sync_pir_to_irr, + .deliver_interrupt = vmx_deliver_interrupt, + .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, + + .set_tss_addr = vmx_set_tss_addr, + .set_identity_map_addr = vmx_set_identity_map_addr, + .get_mt_mask = vmx_get_mt_mask, + + .get_exit_info = vmx_get_exit_info, + + .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, + + .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, + + .get_l2_tsc_offset = vmx_get_l2_tsc_offset, + .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, + .write_tsc_offset = vmx_write_tsc_offset, + .write_tsc_multiplier = vmx_write_tsc_multiplier, + + .load_mmu_pgd = vmx_load_mmu_pgd, + + .check_intercept = vmx_check_intercept, + .handle_exit_irqoff = vmx_handle_exit_irqoff, + + .request_immediate_exit = vmx_request_immediate_exit, + + .sched_in = vmx_sched_in, + + .cpu_dirty_log_size =PML_ENTITY_NUM , + .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, + + .nested_ops = &vmx_nested_ops, + + .pi_update_irte = vmx_pi_update_irte, + .pi_start_assignment = vmx_pi_start_assignment, + +#ifdef CONFIG_X86_64 + .set_hv_timer = vmx_set_hv_timer, + .cancel_hv_timer = vmx_cancel_hv_timer, +#endif + + .setup_mce = vmx_setup_mce, + +#ifdef CONFIG_KVM_SMM + .smi_allowed = vmx_smi_allowed, + .enter_smm = vmx_enter_smm, + .leave_smm = vmx_leave_smm, + .enable_smi_window = vmx_enable_smi_window, +#endif + + .check_emulate_instruction = vmx_check_emulate_instruction, + .apic_init_signal_blocked = vmx_apic_init_signal_blocked, + .migrate_timers = vmx_migrate_timers, + + .msr_filter_changed = vmx_msr_filter_changed, + .complete_emulated_msr = kvm_complete_insn_gp, + + .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, + + .get_untagged_addr = vmx_get_untagged_addr, +}; + +struct kvm_x86_init_ops vt_init_ops__initdata = { + .hardware_setup = vmx_hardware_setup, + .handle_intel_pt_intr =NULL , + + .runtime_ops = &vt_x86_ops, + .pmu_ops = &intel_pmu_ops, +};arch/x86/kvm/vmx/vmx.c +103
-275@@ -65,6 +65,7 @@#include "vmcs12.h" #include "vmx.h" #include "x86.h" +#include "x86_ops.h" #include "smm.h" #include "vmx_onhyperv.h" @@ -516,8 +517,6 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx)static unsigned long host_idt_base;#if IS_ENABLED (CONFIG_HYPERV ) -static struct kvm_x86_ops vmx_x86_ops__initdata ; -static bool __read_mostly enlightened_vmcs = true;module_param (enlightened_vmcs, bool,0444 );@@ -567,9 +566,8 @@ static __init void hv_init_evmcs(void) }if (ms_hyperv.nested_features &HV_X64_NESTED_DIRECT_FLUSH ) - vmx_x86_ops.enable_l2_tlb_flush + vt_x86_ops.enable_l2_tlb_flush = hv_enable_l2_tlb_flush; - }else { enlightened_vmcs = false; }@@ -1474,7 +1472,7 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu,* Switches to specified vcpu, until a matching vcpu_put(), but assumes * vcpu mutex is already taken. */ -static void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu) +void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -1485,7 +1483,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vmx->host_debugctlmsr =get_debugctlmsr (); } -static void vmx_vcpu_put (struct kvm_vcpu *vcpu) +void vmx_vcpu_put (struct kvm_vcpu *vcpu) {vmx_vcpu_pi_put (vcpu);@@ -1544,7 +1542,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) vmx->emulation_required =vmx_emulation_required (vcpu); } -static bool vmx_get_if_flag (struct kvm_vcpu *vcpu) +bool vmx_get_if_flag (struct kvm_vcpu *vcpu) {return vmx_get_rflags (vcpu) &X86_EFLAGS_IF ; }@@ -1650,8 +1648,8 @@ static int vmx_rtit_ctl_check(struct kvm_vcpu *vcpu, u64 data)return 0 ; } -static int vmx_check_emulate_instruction (struct kvm_vcpu *vcpu,int emul_type, -void *insn,int insn_len) +int vmx_check_emulate_instruction (struct kvm_vcpu *vcpu,int emul_type, +void *insn,int insn_len) {/* * Emulation of instructions in SGX enclaves is impossible as RIP does @@ -1735,7 +1733,7 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu)* Recognizes a pending MTF VM-exit and records the nested state for later * delivery. */ -static void vmx_update_emulated_instruction (struct kvm_vcpu *vcpu) +void vmx_update_emulated_instruction (struct kvm_vcpu *vcpu) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -1766,7 +1764,7 @@ static void vmx_update_emulated_instruction(struct kvm_vcpu *vcpu) } } -static int vmx_skip_emulated_instruction (struct kvm_vcpu *vcpu) +int vmx_skip_emulated_instruction (struct kvm_vcpu *vcpu) {vmx_update_emulated_instruction (vcpu);return skip_emulated_instruction (vcpu);@@ -1785,7 +1783,7 @@ static void vmx_clear_hlt(struct kvm_vcpu *vcpu)vmcs_write32 (GUEST_ACTIVITY_STATE ,GUEST_ACTIVITY_ACTIVE ); } -static void vmx_inject_exception (struct kvm_vcpu *vcpu) +void vmx_inject_exception (struct kvm_vcpu *vcpu) {struct kvm_queued_exception *ex = &vcpu->arch.exception;u32 intr_info = ex->vector |INTR_INFO_VALID_MASK ;@@ -1906,12 +1904,12 @@ u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu)return kvm_caps.default_tsc_scaling_ratio; } -static void vmx_write_tsc_offset (struct kvm_vcpu *vcpu) +void vmx_write_tsc_offset (struct kvm_vcpu *vcpu) {vmcs_write64 (TSC_OFFSET , vcpu->arch.tsc_offset); } -static void vmx_write_tsc_multiplier (struct kvm_vcpu *vcpu) +void vmx_write_tsc_multiplier (struct kvm_vcpu *vcpu) {vmcs_write64 (TSC_MULTIPLIER , vcpu->arch.tsc_scaling_ratio); }@@ -1954,7 +1952,7 @@ static inline bool is_vmx_feature_control_msr_valid(struct vcpu_vmx *vmx,return !(msr->data & ~valid_bits); } -static int vmx_get_msr_feature (struct kvm_msr_entry *msr) +int vmx_get_msr_feature (struct kvm_msr_entry *msr) {switch (msr->index) { case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR:@@ -1971,7 +1969,7 @@ static int vmx_get_msr_feature(struct kvm_msr_entry *msr)* Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -static int vmx_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) +int vmx_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) {struct vcpu_vmx *vmx =to_vmx (vcpu);struct vmx_uret_msr *msr;@@ -2152,7 +2150,7 @@ static u64 vmx_get_supported_debugctl(struct kvm_vcpu *vcpu, bool host_initiated* Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -static int vmx_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) +int vmx_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info) {struct vcpu_vmx *vmx =to_vmx (vcpu);struct vmx_uret_msr *msr;@@ -2455,7 +2453,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)return ret; } -static void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) +void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) {unsigned long guest_owned_bits;@@ -2756,7 +2754,7 @@ static bool kvm_is_vmx_supported(void)return supported; } -static int vmx_check_processor_compat (void ) +int vmx_check_processor_compat (void ) {int cpu =raw_smp_processor_id ();struct vmcs_config vmcs_conf;@@ -2798,7 +2796,7 @@ static int kvm_cpu_vmxon(u64 vmxon_pointer)return -EFAULT ; } -static int vmx_hardware_enable (void ) +int vmx_hardware_enable (void ) {int cpu =raw_smp_processor_id ();u64 phys_addr =__pa (per_cpu (vmxarea, cpu));@@ -2838,7 +2836,7 @@ static void vmclear_local_loaded_vmcss(void)__loaded_vmcs_clear (v); } -static void vmx_hardware_disable (void ) +void vmx_hardware_disable (void ) {vmclear_local_loaded_vmcss ();@@ -3152,7 +3150,7 @@ static void exit_lmode(struct kvm_vcpu *vcpu)#endif -static void vmx_flush_tlb_all (struct kvm_vcpu *vcpu) +void vmx_flush_tlb_all (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -3182,7 +3180,7 @@ static inline int vmx_get_current_vpid(struct kvm_vcpu *vcpu)return to_vmx (vcpu)->vpid; } -static void vmx_flush_tlb_current (struct kvm_vcpu *vcpu) +void vmx_flush_tlb_current (struct kvm_vcpu *vcpu) {struct kvm_mmu *mmu = vcpu->arch.mmu;u64 root_hpa = mmu->root.hpa;@@ -3198,7 +3196,7 @@ static void vmx_flush_tlb_current(struct kvm_vcpu *vcpu)vpid_sync_context (vmx_get_current_vpid (vcpu)); } -static void vmx_flush_tlb_gva (struct kvm_vcpu *vcpu,gva_t addr) +void vmx_flush_tlb_gva (struct kvm_vcpu *vcpu,gva_t addr) {/* * vpid_sync_vcpu_addr() is a nop if vpid==0, see the comment in @@ -3207,7 +3205,7 @@ static void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr)vpid_sync_vcpu_addr (vmx_get_current_vpid (vcpu), addr); } -static void vmx_flush_tlb_guest (struct kvm_vcpu *vcpu) +void vmx_flush_tlb_guest (struct kvm_vcpu *vcpu) {/* * vpid_sync_context() is a nop if vpid==0, e.g. if enable_vpid==0 or a @@ -3252,7 +3250,7 @@ void ept_save_pdptrs(struct kvm_vcpu *vcpu)#define CR3_EXITING_BITS (CPU_BASED_CR3_LOAD_EXITING | \ CPU_BASED_CR3_STORE_EXITING) -static bool vmx_is_valid_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) +bool vmx_is_valid_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) {if (is_guest_mode (vcpu))return nested_guest_cr0_valid (vcpu, cr0);@@ -3373,8 +3371,7 @@ u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level)return eptp; } -static void vmx_load_mmu_pgd (struct kvm_vcpu *vcpu,hpa_t root_hpa, -int root_level) +void vmx_load_mmu_pgd (struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level) {struct kvm *kvm = vcpu->kvm;bool update_guest_cr3 = true;@@ -3403,8 +3400,7 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,vmcs_writel (GUEST_CR3 , guest_cr3); } - -static bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) +bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) {/* * We operate under the default treatment of SMM, so VMX cannot be @@ -3520,7 +3516,7 @@ void vmx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) var->g = (ar >>15 ) &1 ; } -static u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg) +u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg) {struct kvm_segment s;@@ -3597,14 +3593,14 @@ void __vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg)vmcs_write32 (sf->ar_bytes,vmx_segment_access_rights (var)); } -static void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg) +void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg) {__vmx_set_segment (vcpu, var, seg);to_vmx (vcpu)->emulation_required =vmx_emulation_required (vcpu); } -static void vmx_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l) +void vmx_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l) {u32 ar =vmx_read_guest_seg_ar (to_vmx (vcpu),VCPU_SREG_CS );@@ -3612,25 +3608,25 @@ static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *l = (ar >>13 ) &1 ; } -static void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { dt->size =vmcs_read32 (GUEST_IDTR_LIMIT ); dt->address =vmcs_readl (GUEST_IDTR_BASE ); } -static void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_IDTR_LIMIT , dt->size);vmcs_writel (GUEST_IDTR_BASE , dt->address); } -static void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) { dt->size =vmcs_read32 (GUEST_GDTR_LIMIT ); dt->address =vmcs_readl (GUEST_GDTR_BASE ); } -static void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) +void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt) {vmcs_write32 (GUEST_GDTR_LIMIT , dt->size);vmcs_writel (GUEST_GDTR_BASE , dt->address);@@ -4102,7 +4098,7 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu) } } -static bool vmx_guest_apic_has_interrupt (struct kvm_vcpu *vcpu) +bool vmx_guest_apic_has_interrupt (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);void *vapic_page;@@ -4122,7 +4118,7 @@ static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu)return ((rvi &0xf0 ) > (vppr &0xf0 )); } -static void vmx_msr_filter_changed (struct kvm_vcpu *vcpu) +void vmx_msr_filter_changed (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);u32 i;@@ -4263,8 +4259,8 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector)return 0 ; } -static void vmx_deliver_interrupt (struct kvm_lapic *apic,int delivery_mode, -int trig_mode,int vector) +void vmx_deliver_interrupt (struct kvm_lapic *apic,int delivery_mode, +int trig_mode,int vector) {struct kvm_vcpu *vcpu = apic->vcpu;@@ -4426,7 +4422,7 @@ static u32 vmx_vmexit_ctrl(void) ~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |VM_EXIT_LOAD_IA32_EFER ); } -static void vmx_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu) +void vmx_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -4690,7 +4686,7 @@ static int vmx_alloc_ipiv_pid_table(struct kvm *kvm)return 0 ; } -static int vmx_vcpu_precreate (struct kvm *kvm) +int vmx_vcpu_precreate (struct kvm *kvm) {return vmx_alloc_ipiv_pid_table (kvm); }@@ -4845,7 +4841,7 @@ static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu) vmx->pi_desc.sn =1 ; } -static void vmx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +void vmx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -4904,12 +4900,12 @@ static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)vmx_update_fb_clear_dis (vcpu, vmx); } -static void vmx_enable_irq_window (struct kvm_vcpu *vcpu) +void vmx_enable_irq_window (struct kvm_vcpu *vcpu) {exec_controls_setbit (to_vmx (vcpu),CPU_BASED_INTR_WINDOW_EXITING ); } -static void vmx_enable_nmi_window (struct kvm_vcpu *vcpu) +void vmx_enable_nmi_window (struct kvm_vcpu *vcpu) {if (!enable_vnmi ||vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) &GUEST_INTR_STATE_STI ) {@@ -4920,7 +4916,7 @@ static void vmx_enable_nmi_window(struct kvm_vcpu *vcpu)exec_controls_setbit (to_vmx (vcpu),CPU_BASED_NMI_WINDOW_EXITING ); } -static void vmx_inject_irq (struct kvm_vcpu *vcpu,bool reinjected) +void vmx_inject_irq (struct kvm_vcpu *vcpu,bool reinjected) {struct vcpu_vmx *vmx =to_vmx (vcpu);uint32_t intr;@@ -4948,7 +4944,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu, bool reinjected)vmx_clear_hlt (vcpu); } -static void vmx_inject_nmi (struct kvm_vcpu *vcpu) +void vmx_inject_nmi (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -5026,7 +5022,7 @@ bool vmx_nmi_blocked(struct kvm_vcpu *vcpu) GUEST_INTR_STATE_NMI)); } -static int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection) +int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection) {if (to_vmx (vcpu)->nested.nested_run_pending)return -EBUSY ;@@ -5048,7 +5044,7 @@ bool vmx_interrupt_blocked(struct kvm_vcpu *vcpu) (GUEST_INTR_STATE_STI | GUEST_INTR_STATE_MOV_SS)); } -static int vmx_interrupt_allowed (struct kvm_vcpu *vcpu,bool for_injection) +int vmx_interrupt_allowed (struct kvm_vcpu *vcpu,bool for_injection) {if (to_vmx (vcpu)->nested.nested_run_pending)return -EBUSY ;@@ -5063,7 +5059,7 @@ static int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection)return !vmx_interrupt_blocked (vcpu); } -static int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr) +int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr) {void __user *ret;@@ -5083,7 +5079,7 @@ static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr)return init_rmode_tss (kvm, ret); } -static int vmx_set_identity_map_addr (struct kvm *kvm,u64 ident_addr) +int vmx_set_identity_map_addr (struct kvm *kvm,u64 ident_addr) {to_kvm_vmx (kvm)->ept_identity_map_addr = ident_addr;return 0 ;@@ -5369,8 +5365,7 @@ static int handle_io(struct kvm_vcpu *vcpu)return kvm_fast_pio (vcpu, size, port, in); } -static void -vmx_patch_hypercall (struct kvm_vcpu *vcpu,unsigned char *hypercall) +void vmx_patch_hypercall (struct kvm_vcpu *vcpu,unsigned char *hypercall) {/* * Patch in the VMCALL instruction: @@ -5579,7 +5574,7 @@ static int handle_dr(struct kvm_vcpu *vcpu)return kvm_complete_insn_gp (vcpu, err); } -static void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu) +void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu) {get_debugreg (vcpu->arch.db[0 ],0 );get_debugreg (vcpu->arch.db[1 ],1 );@@ -5598,7 +5593,7 @@ static void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu)set_debugreg (DR6_RESERVED ,6 ); } -static void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val) +void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val) {vmcs_writel (GUEST_DR7 , val); }@@ -5869,7 +5864,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu)return 1 ; } -static int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu) +int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu) {if (vmx_emulation_required_with_pending_exception (vcpu)) {kvm_prepare_emulation_failure_exit (vcpu);@@ -6133,9 +6128,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = {static const int kvm_vmx_max_exit_handlers =ARRAY_SIZE (kvm_vmx_exit_handlers); -static void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, -u64 *info1,u64 *info2, -u32 *intr_info,u32 *error_code) +void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *intr_info,u32 *error_code) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6578,7 +6572,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)return 0 ; } -static int vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath) +int vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath) {int ret =__vmx_handle_exit (vcpu, exit_fastpath);@@ -6666,7 +6660,7 @@ static noinstr void vmx_l1d_flush(struct kvm_vcpu *vcpu) : "eax", "ebx", "ecx", "edx"); } -static void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr) +void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);int tpr_threshold;@@ -6736,7 +6730,7 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu)vmx_update_msr_bitmap_x2apic (vcpu); } -static void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu) +void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu) {const gfn_t gfn =APIC_DEFAULT_PHYS_BASE >>PAGE_SHIFT ;struct kvm *kvm = vcpu->kvm;@@ -6805,7 +6799,7 @@ static void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu)kvm_release_pfn_clean (pfn); } -static void vmx_hwapic_isr_update (int max_isr) +void vmx_hwapic_isr_update (int max_isr) {u16 status;u8 old;@@ -6839,7 +6833,7 @@ static void vmx_set_rvi(int vector) } } -static void vmx_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr) +void vmx_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr) {/* * When running L2, updating RVI is only relevant when @@ -6853,7 +6847,7 @@ static void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr)vmx_set_rvi (max_irr); } -static int vmx_sync_pir_to_irr (struct kvm_vcpu *vcpu) +int vmx_sync_pir_to_irr (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);int max_irr;@@ -6899,7 +6893,7 @@ static int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu)return max_irr; } -static void vmx_load_eoi_exitmap (struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap) +void vmx_load_eoi_exitmap (struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap) {if (!kvm_vcpu_apicv_active (vcpu))return ;@@ -6910,7 +6904,7 @@ static void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap)vmcs_write64 (EOI_EXIT_BITMAP3 , eoi_exit_bitmap[3 ]); } -static void vmx_apicv_pre_state_restore (struct kvm_vcpu *vcpu) +void vmx_apicv_pre_state_restore (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6973,7 +6967,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu) vcpu->arch.at_instruction_boundary = true; } -static void vmx_handle_exit_irqoff (struct kvm_vcpu *vcpu) +void vmx_handle_exit_irqoff (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -6990,7 +6984,7 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu)* The kvm parameter can be NULL (module initialization, or invocation before * VM creation). Be sure to check the kvm parameter before using it. */ -static bool vmx_has_emulated_msr (struct kvm *kvm,u32 index) +bool vmx_has_emulated_msr (struct kvm *kvm,u32 index) {switch (index) {case MSR_IA32_SMBASE :@@ -7113,7 +7107,7 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx) IDT_VECTORING_ERROR_CODE); } -static void vmx_cancel_injection (struct kvm_vcpu *vcpu) +void vmx_cancel_injection (struct kvm_vcpu *vcpu) { __vmx_complete_interrupts(vcpu, vmcs_read32(VM_ENTRY_INTR_INFO_FIELD),@@ -7268,7 +7262,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,guest_state_exit_irqoff (); } -static fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu) +fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);unsigned long cr3, cr4;@@ -7424,7 +7418,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)return vmx_exit_handlers_fastpath (vcpu); } -static void vmx_vcpu_free (struct kvm_vcpu *vcpu) +void vmx_vcpu_free (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -7435,7 +7429,7 @@ static void vmx_vcpu_free(struct kvm_vcpu *vcpu)free_loaded_vmcs (vmx->loaded_vmcs); } -static int vmx_vcpu_create (struct kvm_vcpu *vcpu) +int vmx_vcpu_create (struct kvm_vcpu *vcpu) {struct vmx_uret_msr *tsx_ctrl;struct vcpu_vmx *vmx;@@ -7541,7 +7535,7 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } -static bool vmx_is_vm_type_supported (unsigned long type) +bool vmx_is_vm_type_supported (unsigned long type) {/* TODO: Check if TDX is supported. */ return __kvm_is_vm_type_supported (type);@@ -7550,7 +7544,7 @@ static bool vmx_is_vm_type_supported(unsigned long type)#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" -static int vmx_vm_init (struct kvm *kvm) +int vmx_vm_init (struct kvm *kvm) {if (!ple_gap) kvm->arch.pause_in_guest = true;@@ -7581,7 +7575,7 @@ static int vmx_vm_init(struct kvm *kvm)return 0 ; } -static u8 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio) +u8 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio) {/* We wanted to honor guest CD/MTRR/PAT, but doing so could result in * memory aliases with conflicting memory types and sometimes MCEs. @@ -7753,7 +7747,7 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i *4 )); } -static void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu) +void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -7907,7 +7901,7 @@ static __init void vmx_set_cpu_caps(void)kvm_cpu_cap_check_and_set (X86_FEATURE_WAITPKG ); } -static void vmx_request_immediate_exit (struct kvm_vcpu *vcpu) +void vmx_request_immediate_exit (struct kvm_vcpu *vcpu) {to_vmx (vcpu)->req_immediate_exit = true; }@@ -7946,10 +7940,10 @@ static int vmx_check_intercept_io(struct kvm_vcpu *vcpu,return intercept ?X86EMUL_UNHANDLEABLE :X86EMUL_CONTINUE ; } -static int vmx_check_intercept (struct kvm_vcpu *vcpu, -struct x86_instruction_info *info, -enum x86_intercept_stage stage, -struct x86_exception *exception) +int vmx_check_intercept (struct kvm_vcpu *vcpu, +struct x86_instruction_info *info, +enum x86_intercept_stage stage, +struct x86_exception *exception) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);@@ -8029,8 +8023,8 @@ static inline int u64_shl_div_u64(u64 a, unsigned int shift,return 0 ; } -static int vmx_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, -bool *expired) +int vmx_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired) {struct vcpu_vmx *vmx;u64 tscl, guest_tscl, delta_tsc, lapic_timer_advance_cycles;@@ -8069,13 +8063,13 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, return 0; } -static void vmx_cancel_hv_timer (struct kvm_vcpu *vcpu) +void vmx_cancel_hv_timer (struct kvm_vcpu *vcpu) {to_vmx (vcpu)->hv_deadline_tsc =-1 ; }#endif -static void vmx_sched_in (struct kvm_vcpu *vcpu,int cpu) +void vmx_sched_in (struct kvm_vcpu *vcpu,int cpu) {if (!kvm_pause_in_guest (vcpu->kvm))shrink_ple_window (vcpu);@@ -8104,7 +8098,7 @@ void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu)secondary_exec_controls_clearbit (vmx,SECONDARY_EXEC_ENABLE_PML ); } -static void vmx_setup_mce (struct kvm_vcpu *vcpu) +void vmx_setup_mce (struct kvm_vcpu *vcpu) {if (vcpu->arch.mcg_cap &MCG_LMCE_P ) to_vmx(vcpu)->msr_ia32_feature_control_valid_bits |=@@ -8115,7 +8109,7 @@ static void vmx_setup_mce(struct kvm_vcpu *vcpu) }#ifdef CONFIG_KVM_SMM -static int vmx_smi_allowed (struct kvm_vcpu *vcpu,bool for_injection) +int vmx_smi_allowed (struct kvm_vcpu *vcpu,bool for_injection) {/* we need a nested vmexit to enter SMM, postpone if run is pending */ if (to_vmx (vcpu)->nested.nested_run_pending)@@ -8123,7 +8117,7 @@ static int vmx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection)return !is_smm (vcpu); } -static int vmx_enter_smm (struct kvm_vcpu *vcpu,union kvm_smram *smram) +int vmx_enter_smm (struct kvm_vcpu *vcpu,union kvm_smram *smram) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -8144,7 +8138,7 @@ static int vmx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram)return 0 ; } -static int vmx_leave_smm (struct kvm_vcpu *vcpu,const union kvm_smram *smram) +int vmx_leave_smm (struct kvm_vcpu *vcpu,const union kvm_smram *smram) {struct vcpu_vmx *vmx =to_vmx (vcpu);int ret;@@ -8165,18 +8159,18 @@ static int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) return 0; } -static void vmx_enable_smi_window (struct kvm_vcpu *vcpu) +void vmx_enable_smi_window (struct kvm_vcpu *vcpu) {/* RSM will cause a vmexit anyway. */ }#endif -static bool vmx_apic_init_signal_blocked (struct kvm_vcpu *vcpu) +bool vmx_apic_init_signal_blocked (struct kvm_vcpu *vcpu) {return to_vmx (vcpu)->nested.vmxon && !is_guest_mode (vcpu); } -static void vmx_migrate_timers (struct kvm_vcpu *vcpu) +void vmx_migrate_timers (struct kvm_vcpu *vcpu) {if (is_guest_mode (vcpu)) {struct hrtimer *timer = &to_vmx (vcpu)->nested.preemption_timer;@@ -8186,7 +8180,7 @@ static void vmx_migrate_timers(struct kvm_vcpu *vcpu) } } -static void vmx_hardware_unsetup (void ) +void vmx_hardware_unsetup (void ) {kvm_set_posted_intr_wakeup_handler (NULL );@@ -8196,18 +8190,7 @@ static void vmx_hardware_unsetup(void)free_kvm_area (); } -#define VMX_REQUIRED_APICV_INHIBITS \ -( \ - BIT(APICV_INHIBIT_REASON_DISABLE)| \ - BIT(APICV_INHIBIT_REASON_ABSENT) | \ - BIT(APICV_INHIBIT_REASON_HYPERV) | \ - BIT(APICV_INHIBIT_REASON_BLOCKIRQ) | \ - BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) | \ - BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ - BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) \ -) - -static void vmx_vm_destroy (struct kvm *kvm) +void vmx_vm_destroy (struct kvm *kvm) {struct kvm_vmx *kvm_vmx =to_kvm_vmx (kvm);@@ -8258,151 +8241,6 @@ gva_t vmx_get_untagged_addr(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flagsreturn (sign_extend64 (gva, lam_bit) & ~BIT_ULL (63 )) | (gva &BIT_ULL (63 )); } -static struct kvm_x86_ops vmx_x86_ops__initdata = { - .name =KBUILD_MODNAME , - - .check_processor_compatibility = vmx_check_processor_compat, - - .hardware_unsetup = vmx_hardware_unsetup, - - .hardware_enable = vmx_hardware_enable, - .hardware_disable = vmx_hardware_disable, - .has_emulated_msr = vmx_has_emulated_msr, - - .is_vm_type_supported = vmx_is_vm_type_supported, - .vm_size =sizeof (struct kvm_vmx ), - .vm_init = vmx_vm_init, - .vm_destroy = vmx_vm_destroy, - - .vcpu_precreate = vmx_vcpu_precreate, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, - - .prepare_switch_to_guest = vmx_prepare_switch_to_guest, - .vcpu_load = vmx_vcpu_load, - .vcpu_put = vmx_vcpu_put, - - .update_exception_bitmap = vmx_update_exception_bitmap, - .get_msr_feature = vmx_get_msr_feature, - .get_msr = vmx_get_msr, - .set_msr = vmx_set_msr, - .get_segment_base = vmx_get_segment_base, - .get_segment = vmx_get_segment, - .set_segment = vmx_set_segment, - .get_cpl = vmx_get_cpl, - .get_cs_db_l_bits = vmx_get_cs_db_l_bits, - .is_valid_cr0 = vmx_is_valid_cr0, - .set_cr0 = vmx_set_cr0, - .is_valid_cr4 = vmx_is_valid_cr4, - .set_cr4 = vmx_set_cr4, - .set_efer = vmx_set_efer, - .get_idt = vmx_get_idt, - .set_idt = vmx_set_idt, - .get_gdt = vmx_get_gdt, - .set_gdt = vmx_set_gdt, - .set_dr7 = vmx_set_dr7, - .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, - .cache_reg = vmx_cache_reg, - .get_rflags = vmx_get_rflags, - .set_rflags = vmx_set_rflags, - .get_if_flag = vmx_get_if_flag, - - .flush_tlb_all = vmx_flush_tlb_all, - .flush_tlb_current = vmx_flush_tlb_current, - .flush_tlb_gva = vmx_flush_tlb_gva, - .flush_tlb_guest = vmx_flush_tlb_guest, - - .vcpu_pre_run = vmx_vcpu_pre_run, - .vcpu_run = vmx_vcpu_run, - .handle_exit = vmx_handle_exit, - .skip_emulated_instruction = vmx_skip_emulated_instruction, - .update_emulated_instruction = vmx_update_emulated_instruction, - .set_interrupt_shadow = vmx_set_interrupt_shadow, - .get_interrupt_shadow = vmx_get_interrupt_shadow, - .patch_hypercall = vmx_patch_hypercall, - .inject_irq = vmx_inject_irq, - .inject_nmi = vmx_inject_nmi, - .inject_exception = vmx_inject_exception, - .cancel_injection = vmx_cancel_injection, - .interrupt_allowed = vmx_interrupt_allowed, - .nmi_allowed = vmx_nmi_allowed, - .get_nmi_mask = vmx_get_nmi_mask, - .set_nmi_mask = vmx_set_nmi_mask, - .enable_nmi_window = vmx_enable_nmi_window, - .enable_irq_window = vmx_enable_irq_window, - .update_cr8_intercept = vmx_update_cr8_intercept, - .set_virtual_apic_mode = vmx_set_virtual_apic_mode, - .set_apic_access_page_addr = vmx_set_apic_access_page_addr, - .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, - .load_eoi_exitmap = vmx_load_eoi_exitmap, - .apicv_pre_state_restore = vmx_apicv_pre_state_restore, - .required_apicv_inhibits =VMX_REQUIRED_APICV_INHIBITS , - .hwapic_irr_update = vmx_hwapic_irr_update, - .hwapic_isr_update = vmx_hwapic_isr_update, - .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, - .sync_pir_to_irr = vmx_sync_pir_to_irr, - .deliver_interrupt = vmx_deliver_interrupt, - .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, - - .set_tss_addr = vmx_set_tss_addr, - .set_identity_map_addr = vmx_set_identity_map_addr, - .get_mt_mask = vmx_get_mt_mask, - - .get_exit_info = vmx_get_exit_info, - - .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, - - .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, - - .get_l2_tsc_offset = vmx_get_l2_tsc_offset, - .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, - .write_tsc_offset = vmx_write_tsc_offset, - .write_tsc_multiplier = vmx_write_tsc_multiplier, - - .load_mmu_pgd = vmx_load_mmu_pgd, - - .check_intercept = vmx_check_intercept, - .handle_exit_irqoff = vmx_handle_exit_irqoff, - - .request_immediate_exit = vmx_request_immediate_exit, - - .sched_in = vmx_sched_in, - - .cpu_dirty_log_size =PML_ENTITY_NUM , - .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, - - .nested_ops = &vmx_nested_ops, - - .pi_update_irte = vmx_pi_update_irte, - .pi_start_assignment = vmx_pi_start_assignment, - -#ifdef CONFIG_X86_64 - .set_hv_timer = vmx_set_hv_timer, - .cancel_hv_timer = vmx_cancel_hv_timer, -#endif - - .setup_mce = vmx_setup_mce, - -#ifdef CONFIG_KVM_SMM - .smi_allowed = vmx_smi_allowed, - .enter_smm = vmx_enter_smm, - .leave_smm = vmx_leave_smm, - .enable_smi_window = vmx_enable_smi_window, -#endif - - .check_emulate_instruction = vmx_check_emulate_instruction, - .apic_init_signal_blocked = vmx_apic_init_signal_blocked, - .migrate_timers = vmx_migrate_timers, - - .msr_filter_changed = vmx_msr_filter_changed, - .complete_emulated_msr = kvm_complete_insn_gp, - - .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, - - .get_untagged_addr = vmx_get_untagged_addr, -}; -static unsigned int vmx_handle_intel_pt_intr (void ) {struct kvm_vcpu *vcpu =kvm_get_running_vcpu ();@@ -8468,9 +8306,7 @@ static void __init vmx_setup_me_spte_mask(void)kvm_mmu_set_me_spte_mask (0 , me_mask); } -static struct kvm_x86_init_ops vmx_init_ops__initdata ; - -static __init int hardware_setup (void ) +__init int vmx_hardware_setup (void ) {unsigned long host_bndcfgs;struct desc_ptr dt;@@ -8539,16 +8375,16 @@ static __init int hardware_setup(void)* using the APIC_ACCESS_ADDR VMCS field. */ if (!flexpriority_enabled) - vmx_x86_ops.set_apic_access_page_addr =NULL ; + vt_x86_ops.set_apic_access_page_addr =NULL ;if (!cpu_has_vmx_tpr_shadow ()) - vmx_x86_ops.update_cr8_intercept =NULL ; + vt_x86_ops.update_cr8_intercept =NULL ;#if IS_ENABLED (CONFIG_HYPERV )if (ms_hyperv.nested_features &HV_X64_NESTED_GUEST_MAPPING_FLUSH && enable_ept) { - vmx_x86_ops.flush_remote_tlbs = hv_flush_remote_tlbs; - vmx_x86_ops.flush_remote_tlbs_range = hv_flush_remote_tlbs_range; + vt_x86_ops.flush_remote_tlbs = hv_flush_remote_tlbs; + vt_x86_ops.flush_remote_tlbs_range = hv_flush_remote_tlbs_range; }#endif @@ -8563,7 +8399,7 @@ static __init int hardware_setup(void)if (!cpu_has_vmx_apicv ()) enable_apicv =0 ;if (!enable_apicv) - vmx_x86_ops.sync_pir_to_irr =NULL ; + vt_x86_ops.sync_pir_to_irr =NULL ;if (!enable_apicv || !cpu_has_vmx_ipiv ()) enable_ipiv = false;@@ -8599,7 +8435,7 @@ static __init int hardware_setup(void) enable_pml =0 ;if (!enable_pml) - vmx_x86_ops.cpu_dirty_log_size =0 ; + vt_x86_ops.cpu_dirty_log_size =0 ;if (!cpu_has_vmx_preemption_timer ()) enable_preemption_timer = false;@@ -8624,9 +8460,9 @@ static __init int hardware_setup(void) }if (!enable_preemption_timer) { - vmx_x86_ops.set_hv_timer =NULL ; - vmx_x86_ops.cancel_hv_timer =NULL ; - vmx_x86_ops.request_immediate_exit = __kvm_request_immediate_exit; + vt_x86_ops.set_hv_timer =NULL ; + vt_x86_ops.cancel_hv_timer =NULL ; + vt_x86_ops.request_immediate_exit = __kvm_request_immediate_exit; } kvm_caps.supported_mce_cap |=MCG_LMCE_P ;@@ -8637,9 +8473,9 @@ static __init int hardware_setup(void)if (!enable_ept || !enable_pmu || !cpu_has_vmx_intel_pt ()) pt_mode =PT_MODE_SYSTEM ;if (pt_mode ==PT_MODE_HOST_GUEST ) - vmx_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr; + vt_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr;else - vmx_init_ops.handle_intel_pt_intr =NULL ; + vt_init_ops.handle_intel_pt_intr =NULL ;setup_default_sgx_lepubkeyhash ();@@ -8662,14 +8498,6 @@ static __init int hardware_setup(void)return r; } -static struct kvm_x86_init_ops vmx_init_ops__initdata = { - .hardware_setup = hardware_setup, - .handle_intel_pt_intr =NULL , - - .runtime_ops = &vmx_x86_ops, - .pmu_ops = &intel_pmu_ops, -}; -static void vmx_cleanup_l1d_flush (void ) {if (vmx_l1d_flush_pages) {@@ -8711,7 +8539,7 @@ static int __init vmx_init(void)*/ hv_init_evmcs (); - r =kvm_x86_vendor_init (&vmx_init_ops); + r =kvm_x86_vendor_init (&vt_init_ops);if (r)return r;arch/x86/kvm/vmx/x86_ops.h new file +125
-1@@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_VMX_X86_OPS_H +#define __KVM_X86_VMX_X86_OPS_H + +#include <linux/kvm_host.h> + +#include "x86.h" + +__init int vmx_hardware_setup (void ); + +extern struct kvm_x86_ops vt_x86_ops__initdata ; +extern struct kvm_x86_init_ops vt_init_ops__initdata ; + +void vmx_hardware_unsetup (void ); +int vmx_check_processor_compat (void ); +int vmx_hardware_enable (void ); +void vmx_hardware_disable (void ); +bool vmx_is_vm_type_supported (unsigned long type); +int vmx_vm_init (struct kvm *kvm); +void vmx_vm_destroy (struct kvm *kvm); +int vmx_vcpu_precreate (struct kvm *kvm); +int vmx_vcpu_create (struct kvm_vcpu *vcpu); +int vmx_vcpu_pre_run (struct kvm_vcpu *vcpu); +fastpath_t vmx_vcpu_run (struct kvm_vcpu *vcpu); +void vmx_vcpu_free (struct kvm_vcpu *vcpu); +void vmx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); +void vmx_vcpu_load (struct kvm_vcpu *vcpu,int cpu); +void vmx_vcpu_put (struct kvm_vcpu *vcpu); +int vmx_handle_exit (struct kvm_vcpu *vcpu,fastpath_t exit_fastpath); +void vmx_handle_exit_irqoff (struct kvm_vcpu *vcpu); +int vmx_skip_emulated_instruction (struct kvm_vcpu *vcpu); +void vmx_update_emulated_instruction (struct kvm_vcpu *vcpu); +int vmx_set_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info); +#ifdef CONFIG_KVM_SMM +int vmx_smi_allowed (struct kvm_vcpu *vcpu,bool for_injection); +int vmx_enter_smm (struct kvm_vcpu *vcpu,union kvm_smram *smram); +int vmx_leave_smm (struct kvm_vcpu *vcpu,const union kvm_smram *smram); +void vmx_enable_smi_window (struct kvm_vcpu *vcpu); +#endif +int vmx_check_emulate_instruction (struct kvm_vcpu *vcpu,int emul_type, +void *insn,int insn_len); +int vmx_check_intercept (struct kvm_vcpu *vcpu, +struct x86_instruction_info *info, +enum x86_intercept_stage stage, +struct x86_exception *exception); +bool vmx_apic_init_signal_blocked (struct kvm_vcpu *vcpu); +void vmx_migrate_timers (struct kvm_vcpu *vcpu); +void vmx_set_virtual_apic_mode (struct kvm_vcpu *vcpu); +void vmx_apicv_pre_state_restore (struct kvm_vcpu *vcpu); +bool vmx_check_apicv_inhibit_reasons (enum kvm_apicv_inhibit reason); +void vmx_hwapic_irr_update (struct kvm_vcpu *vcpu,int max_irr); +void vmx_hwapic_isr_update (int max_isr); +bool vmx_guest_apic_has_interrupt (struct kvm_vcpu *vcpu); +int vmx_sync_pir_to_irr (struct kvm_vcpu *vcpu); +void vmx_deliver_interrupt (struct kvm_lapic *apic,int delivery_mode, +int trig_mode,int vector); +void vmx_vcpu_after_set_cpuid (struct kvm_vcpu *vcpu); +bool vmx_has_emulated_msr (struct kvm *kvm,u32 index); +void vmx_msr_filter_changed (struct kvm_vcpu *vcpu); +void vmx_prepare_switch_to_guest (struct kvm_vcpu *vcpu); +void vmx_update_exception_bitmap (struct kvm_vcpu *vcpu); +int vmx_get_msr_feature (struct kvm_msr_entry *msr); +int vmx_get_msr (struct kvm_vcpu *vcpu,struct msr_data *msr_info); +u64 vmx_get_segment_base (struct kvm_vcpu *vcpu,int seg); +void vmx_get_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg); +void vmx_set_segment (struct kvm_vcpu *vcpu,struct kvm_segment *var,int seg); +int vmx_get_cpl (struct kvm_vcpu *vcpu); +void vmx_get_cs_db_l_bits (struct kvm_vcpu *vcpu,int *db,int *l); +bool vmx_is_valid_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0); +void vmx_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0); +void vmx_load_mmu_pgd (struct kvm_vcpu *vcpu,hpa_t root_hpa,int root_level); +void vmx_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); +bool vmx_is_valid_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4); +int vmx_set_efer (struct kvm_vcpu *vcpu,u64 efer); +void vmx_get_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_set_idt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_get_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_set_gdt (struct kvm_vcpu *vcpu,struct desc_ptr *dt); +void vmx_set_dr7 (struct kvm_vcpu *vcpu,unsigned long val); +void vmx_sync_dirty_debug_regs (struct kvm_vcpu *vcpu); +void vmx_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg); +unsigned long vmx_get_rflags (struct kvm_vcpu *vcpu); +void vmx_set_rflags (struct kvm_vcpu *vcpu,unsigned long rflags); +bool vmx_get_if_flag (struct kvm_vcpu *vcpu); +void vmx_flush_tlb_all (struct kvm_vcpu *vcpu); +void vmx_flush_tlb_current (struct kvm_vcpu *vcpu); +void vmx_flush_tlb_gva (struct kvm_vcpu *vcpu,gva_t addr); +void vmx_flush_tlb_guest (struct kvm_vcpu *vcpu); +void vmx_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask); +u32 vmx_get_interrupt_shadow (struct kvm_vcpu *vcpu); +void vmx_patch_hypercall (struct kvm_vcpu *vcpu,unsigned char *hypercall); +void vmx_inject_irq (struct kvm_vcpu *vcpu,bool reinjected); +void vmx_inject_nmi (struct kvm_vcpu *vcpu); +void vmx_inject_exception (struct kvm_vcpu *vcpu); +void vmx_cancel_injection (struct kvm_vcpu *vcpu); +int vmx_interrupt_allowed (struct kvm_vcpu *vcpu,bool for_injection); +int vmx_nmi_allowed (struct kvm_vcpu *vcpu,bool for_injection); +bool vmx_get_nmi_mask (struct kvm_vcpu *vcpu); +void vmx_set_nmi_mask (struct kvm_vcpu *vcpu,bool masked); +void vmx_enable_nmi_window (struct kvm_vcpu *vcpu); +void vmx_enable_irq_window (struct kvm_vcpu *vcpu); +void vmx_update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr); +void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu); +void vmx_refresh_apicv_exec_ctrl (struct kvm_vcpu *vcpu); +void vmx_load_eoi_exitmap (struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap); +int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr); +int vmx_set_identity_map_addr (struct kvm *kvm,u64 ident_addr); +u8 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio); +void vmx_get_exit_info (struct kvm_vcpu *vcpu,u32 *reason, +u64 *info1,u64 *info2,u32 *intr_info,u32 *error_code); +u64 vmx_get_l2_tsc_offset (struct kvm_vcpu *vcpu); +u64 vmx_get_l2_tsc_multiplier (struct kvm_vcpu *vcpu); +void vmx_write_tsc_offset (struct kvm_vcpu *vcpu); +void vmx_write_tsc_multiplier (struct kvm_vcpu *vcpu); +void vmx_request_immediate_exit (struct kvm_vcpu *vcpu); +void vmx_sched_in (struct kvm_vcpu *vcpu,int cpu); +void vmx_update_cpu_dirty_logging (struct kvm_vcpu *vcpu); +#ifdef CONFIG_X86_64 +int vmx_set_hv_timer (struct kvm_vcpu *vcpu,u64 guest_deadline_tsc, +bool *expired); +void vmx_cancel_hv_timer (struct kvm_vcpu *vcpu); +#endif +void vmx_setup_mce (struct kvm_vcpu *vcpu); + +#endif /* __KVM_X86_VMX_X86_OPS_H */ --2.25.1
Re: [PATCH v18 004/121] KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX
In reply to [email protected]Quoted 45 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Sean Christopherson <[email protected]>
KVM accesses Virtual Machine Control Structure (VMCS) with VMX instructions to operate on VM. TDX doesn't allow VMM to operate VMCS directly. Instead, TDX has its own data structures, and TDX SEAMCALL APIs for VMM to indirectly operate those data structures. This means we must have a TDX version of
kvm_x86_ops.The existing global struct
kvm_x86_opsalready defines an interface which fits with TDX. Butkvm_x86_opsis system-wide, not per-VM structure. To allow VMX to coexist with TDs, thekvm_x86_opscallbacks will have wrappers "if (tdx)tdx_op()elsevmx_op()" to switch VMX or TDX at run time.To split the runtime switch, the VMX implementation, and the TDX implementation, add
main.c, and move out thevmx_x86_opshooks in preparation for adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx.The current code looks as follows.
Invmx.c
staticvmx_op(){ ... }static struct kvm_x86_ops vmx_x86_ops = { .op = vmx_op,initialization code
The eventually converted code will look like
Invmx.c, keep the VMX operations.
vmx_op(){ ... }
VMX initialization
Intdx.c, define the TDX operations.
tdx_op(){ ... }
TDX initialization
Inx86_ops.h, declare the VMX and TDX operations.vmx_op(); tdx_op(); In main.c, define common wrappers for VMX and TDX. static vt_ops() { if (tdx) tdx_ops() else vmx_ops() } static struct kvm_x86_ops vt_x86_ops = { .op = vt_op,initialization to call VMX and TDX initialization
Opportunistically, fix the name inconsistency from
vmx_create_vcpu()andvmx_free_vcpu()tovmx_vcpu_create()andvmx_vcpu_free().
Quoted 6 lines
Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>Reviewed-by Binbin Wu <[email protected]>
PATCH v18 5/121 KVM: x86/vmx: initialize loaded_vmcss_on_cpu in vmx_hardware_setup()
In reply to [email protected]From: Isaku Yamahata <[email protected]>
vmx_hardware_disable() accesses loaded_vmcss_on_cpu via hardware_disable_all(). To allow hardware_enable/disable_all() before kvm_init(), initialize it in before kvm_x86_vendor_init() in vmx_init() so that tdx module initialization, hardware_setup method, can reference the variable.
---
v18:
- Move the vmcss_on_cpu initialization from vmx_hardware_setup() to early point of vmx_init() by Binbin
--- arch/x86/kvm/vmx/vmx.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/vmx.c +5 -4
@@ -8539,6 +8539,10 @@ static int __init vmx_init(void)*/ hv_init_evmcs (); +/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ +for_each_possible_cpu (cpu) +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); + r =kvm_x86_vendor_init (&vt_init_ops);if (r)return r;
@@ -8554,11 +8558,8 @@ static int __init vmx_init(void)if (r) goto err_l1d_flush; -for_each_possible_cpu (cpu) { -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - +for_each_possible_cpu (cpu)pi_init_cpu (cpu); - }cpu_emergency_register_virt_callback (vmx_emergency_disable);
Signature · 1 line
2.25.1
Re: [PATCH v18 005/121] KVM: x86/vmx: initialize loaded_vmcss_on_cpu in vmx_hardware_setup()
Quoted 22 lines · On Mon, Jan 22, 2024 at 03:52:41PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
vmx_hardware_disable()accessesloaded_vmcss_on_cpuviahardware_disable_all(). To allow hardware_enable/disable_all() beforekvm_init(), initialize it in beforekvm_x86_vendor_init()invmx_init()so that tdx module initialization,hardware_setupmethod, can reference the variable.Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- Move thevmcss_on_cpuinitialization fromvmx_hardware_setup()to early point ofvmx_init()by Binbin--- arch/x86/kvm/vmx/vmx.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/vmx.c +0
-0
arch/x86/kvm/vmx/vmx.c static int __init vmx_init(void) · line 8545
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +4
-0@@ -8539,6 +8539,10 @@ static int __init vmx_init(void)*/ hv_init_evmcs (); +/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ +for_each_possible_cpu (cpu) +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); +
The subject "KVM: x86/vmx: initialize loaded_vmcss_on_cpu in vmx_hardware_setup()" should match the change here.
Quoted 20 lines
arch/x86/kvm/vmx/vmx.c +1
-5r =kvm_x86_vendor_init (&vt_init_ops);if (r)return r;@@ -8554,11 +8558,8 @@ static int __init vmx_init(void)if (r) goto err_l1d_flush; -for_each_possible_cpu (cpu) { -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - +for_each_possible_cpu (cpu)pi_init_cpu (cpu); - }cpu_emergency_register_virt_callback (vmx_emergency_disable); --2.25.1
PATCH v18 6/121 KVM: x86/vmx: Refactor KVM VMX module init/exit functions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well.
The current module initialization flow is,
0.) Check if VMX is supported,
1.) hyper-v specific initialization,
2.) system-wide x86 specific and vendor specific initialization, 3.) Final VMX specific system-wide initialization,
4.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 5.) report those sizes to the KVM common layer and KVM common
initialization
Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. Introduce a wrapper function for vmx_init().
The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures.
The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c.
---
v18:
- Move loaded_vmcss_on_cpu initialization to vt_init() before kvm_x86_vendor_init().
- added __init to an empty stub fucntion, hv_init_evmcs().
--- arch/x86/kvm/vmx/main.c | 54 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 60 +++++--------------------------------- arch/x86/kvm/vmx/x86_ops.h | 15 +++++++++- 3 files changed, 75 insertions(+), 54 deletions(-)
arch/x86/kvm/vmx/main.c +54 -0
@@ -167,3 +167,57 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .runtime_ops = &vt_x86_ops, .pmu_ops = &intel_pmu_ops, }; + +static int __init vt_init (void ) +{ +unsigned int vcpu_size, vcpu_align; +int cpu, r; + +if (!kvm_is_vmx_supported ()) +return -EOPNOTSUPP ; + +/* +* Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing +* to unwind if a later step fails. +*/ +hv_init_evmcs (); + +/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ +for_each_possible_cpu (cpu) +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); + + r =kvm_x86_vendor_init (&vt_init_ops); +if (r) +return r; + + r =vmx_init (); +if (r) + goto err_vmx_init; + +/* +* Common KVM initialization _must_ come last, after this, /dev/kvm is +* exposed to userspace! +*/ + vcpu_size =sizeof (struct vcpu_vmx ); + vcpu_align = __alignof__(struct vcpu_vmx ); + r =kvm_init (vcpu_size, vcpu_align,THIS_MODULE ); +if (r) + goto err_kvm_init; + +return 0 ; + +err_kvm_init: +vmx_exit (); +err_vmx_init: +kvm_x86_vendor_exit (); +return r; +} +module_init (vt_init); + +static void vt_exit (void ) +{ +kvm_exit (); +kvm_x86_vendor_exit (); +vmx_exit (); +} +module_exit (vt_exit);
arch/x86/kvm/vmx/vmx.c +7 -53
116 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +14 -1
@@ -6,11 +6,24 @@#include "x86.h" -__init int vmx_hardware_setup (void ); +#if IS_ENABLED (CONFIG_HYPERV ) +__init void hv_init_evmcs (void ); +void hv_reset_evmcs (void ); +#else /* IS_ENABLED(CONFIG_HYPERV) */ +static inline __init void hv_init_evmcs (void ) {} +static inline void hv_reset_evmcs (void ) {} +#endif /* IS_ENABLED(CONFIG_HYPERV) */ + +DECLARE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); + +bool kvm_is_vmx_supported (void ); +int __init vmx_init (void ); +void vmx_exit (void );extern struct kvm_x86_ops vt_x86_ops__initdata ;extern struct kvm_x86_init_ops vt_init_ops__initdata ; +__init int vmx_hardware_setup (void );void vmx_hardware_unsetup (void );int vmx_check_processor_compat (void );int vmx_hardware_enable (void );
Signature · 1 line
2.25.1
Re: [PATCH v18 006/121] KVM: x86/vmx: Refactor KVM VMX module init/exit functions
On 1/23/2024 7:52 AM, [email protected] wrote:
...
Quoted 9 lines
arch/x86/kvm/vmx/x86_ops.h +0
-1@@ -6,11 +6,24 @@#include "x86.h" -__init int vmx_hardware_setup (void );
...
Quoted 2 lines
+__init int
vmx_hardware_setup(void);
this change belongs to Patch 4
Quoted 3 lines
void vmx_hardware_unsetup(void); int vmx_check_processor_compat(void); int vmx_hardware_enable(void);
PATCH v18 7/121 KVM: VMX: Reorder vmx initialization with kvm vendor initialization
In reply to [email protected]From: Isaku Yamahata <[email protected]>
To match vmx_exit cleanup. Now vmx_init() is before kvm_x86_vendor_init(), vmx_init() can initialize loaded_vmcss_on_cpu. Oppertunistically move it back into vmx_init().
---
v18:
- move the loaded_vmcss_on_cpu initialization to vmx_init().
- fix error path of vt_init(). by Chao and Binbin
--- arch/x86/kvm/vmx/main.c | 17 +++++++---------- arch/x86/kvm/vmx/vmx.c | 6 ++++-- arch/x86/kvm/vmx/x86_ops.h | 2 -- 3 files changed, 11 insertions(+), 14 deletions(-)
arch/x86/kvm/vmx/main.c +7 -10
@@ -171,7 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {static int __init vt_init (void ) {unsigned int vcpu_size, vcpu_align; -int cpu, r; +int r;if (!kvm_is_vmx_supported ())return -EOPNOTSUPP ;
@@ -182,18 +182,14 @@ static int __init vt_init(void)*/ hv_init_evmcs (); -/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ -for_each_possible_cpu (cpu) -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - - r =kvm_x86_vendor_init (&vt_init_ops); -if (r) -return r; - r =vmx_init ();if (r) goto err_vmx_init; + r =kvm_x86_vendor_init (&vt_init_ops); +if (r) + goto err_vendor_init; +/* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace!
@@ -207,9 +203,10 @@ static int __init vt_init(void)return 0 ; err_kvm_init: +kvm_x86_vendor_exit (); +err_vendor_init:vmx_exit (); err_vmx_init: -kvm_x86_vendor_exit ();return r; }module_init (vt_init);
arch/x86/kvm/vmx/vmx.c +4 -2
@@ -477,7 +477,7 @@ DEFINE_PER_CPU(struct vmcs *, current_vmcs);* We maintain a per-CPU linked-list of VMCS loaded on that CPU. This is needed * when a CPU is brought down, and we need to VMCLEAR all VMCSs loaded on it. */ -DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); +static DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); static DECLARE_BITMAP(vmx_vpid_bitmap, VMX_NR_VPIDS);static DEFINE_SPINLOCK (vmx_vpid_lock);
@@ -8528,8 +8528,10 @@ int __init vmx_init(void)if (r)return r; -for_each_possible_cpu (cpu) +for_each_possible_cpu (cpu) { +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu));pi_init_cpu (cpu); + }cpu_emergency_register_virt_callback (vmx_emergency_disable);
arch/x86/kvm/vmx/x86_ops.h +0 -2
@@ -14,8 +14,6 @@ static inline __init void hv_init_evmcs(void) {}static inline void hv_reset_evmcs (void ) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ -DECLARE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); -bool kvm_is_vmx_supported (void );int __init vmx_init (void );void vmx_exit (void );
Signature · 1 line
2.25.1
Re: [PATCH v18 007/121] KVM: VMX: Reorder vmx initialization with kvm vendor initialization
Quoted 3 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To match
vmx_exitcleanup.
Do you mean vt_exit()?
Shouldn't vt_init() and vt_exit() be symmetric right from the beginning in the refactor patch (006/121)?
And also, since the reorder of kvm_x86_vendor_init() and vmx_init() is going to happen, can we just skip moving around the init of loaded_vmcss_on_cpu?
Quoted 101 lines
Now
vmx_init()is beforekvm_x86_vendor_init(),vmx_init()can initializeloaded_vmcss_on_cpu. Oppertunistically move it back intovmx_init().Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- move theloaded_vmcss_on_cpuinitialization tovmx_init().
- fix error path ofvt_init(). by Chao and Binbin--- arch/x86/kvm/vmx/main.c | 17 +++++++---------- arch/x86/kvm/vmx/vmx.c | 6 ++++-- arch/x86/kvm/vmx/x86_ops.h | 2 -- 3 files changed, 11 insertions(+), 14 deletions(-)arch/x86/kvm/vmx/main.c +7
-10@@ -171,7 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {static int __init vt_init (void ) {unsigned int vcpu_size, vcpu_align; -int cpu, r; +int r;if (!kvm_is_vmx_supported ())return -EOPNOTSUPP ;@@ -182,18 +182,14 @@ static int __init vt_init(void)*/ hv_init_evmcs (); -/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ -for_each_possible_cpu (cpu) -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - - r =kvm_x86_vendor_init (&vt_init_ops); -if (r) -return r; - r =vmx_init ();if (r) goto err_vmx_init; + r =kvm_x86_vendor_init (&vt_init_ops); +if (r) + goto err_vendor_init; +/* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! @@ -207,9 +203,10 @@ static int __init vt_init(void)return 0 ; err_kvm_init: +kvm_x86_vendor_exit (); +err_vendor_init:vmx_exit (); err_vmx_init: -kvm_x86_vendor_exit ();return r; }module_init (vt_init);arch/x86/kvm/vmx/vmx.c +4
-2@@ -477,7 +477,7 @@ DEFINE_PER_CPU(struct vmcs *, current_vmcs);* We maintain a per-CPU linked-list of VMCS loaded on that CPU. This is needed * when a CPU is brought down, and we need to VMCLEAR all VMCSs loaded on it. */ -DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); +static DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); static DECLARE_BITMAP(vmx_vpid_bitmap, VMX_NR_VPIDS);static DEFINE_SPINLOCK (vmx_vpid_lock);@@ -8528,8 +8528,10 @@ int __init vmx_init(void)if (r)return r; -for_each_possible_cpu (cpu) +for_each_possible_cpu (cpu) { +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu));pi_init_cpu (cpu); + }cpu_emergency_register_virt_callback (vmx_emergency_disable);arch/x86/kvm/vmx/x86_ops.h +0
-2@@ -14,8 +14,6 @@ static inline __init void hv_init_evmcs(void) {}static inline void hv_reset_evmcs (void ) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ -DECLARE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); -bool kvm_is_vmx_supported (void );int __init vmx_init (void );void vmx_exit (void );
Re: [PATCH v18 007/121] KVM: VMX: Reorder vmx initialization with kvm vendor initialization
In reply to [email protected]Quoted 37 lines · On Mon, Jan 22, 2024 at 03:52:43PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To match
vmx_exitcleanup. Nowvmx_init()is beforekvm_x86_vendor_init(),vmx_init()can initializeloaded_vmcss_on_cpu. Oppertunistically move it back intovmx_init().Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- move theloaded_vmcss_on_cpuinitialization tovmx_init().
- fix error path ofvt_init(). by Chao and Binbin--- arch/x86/kvm/vmx/main.c | 17 +++++++---------- arch/x86/kvm/vmx/vmx.c | 6 ++++-- arch/x86/kvm/vmx/x86_ops.h | 2 -- 3 files changed, 11 insertions(+), 14 deletions(-)arch/x86/kvm/vmx/main.c +1
-4@@ -171,7 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {static int __init vt_init (void ) {unsigned int vcpu_size, vcpu_align; -int cpu, r; +int r;if (!kvm_is_vmx_supported ())return -EOPNOTSUPP ;@@ -182,18 +182,14 @@ static int __init vt_init(void)*/ hv_init_evmcs (); -/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ -for_each_possible_cpu (cpu) -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu));
arch/x86/kvm/vmx/main.c static int __init vt_init(void) · line 189
Quoted 10 lines
arch/x86/kvm/vmx/main.c +1
-5- - r =kvm_x86_vendor_init (&vt_init_ops); -if (r) -return r; - r =vmx_init ();if (r) goto err_vmx_init; + r =kvm_x86_vendor_init (&vt_init_ops);
Do kvm_x86_vendor_init() *after* vmx_init() leads to "enable_ept" is used before set to 0 in some cases.
vmx_init() depends on "enable_ept" variable for below 2:
vmx_setup_l1d_flush()
allow_smaller_maxphyaddr = true;And "enable_ept" can be set to 0 in:kvm_x86_vendor_init()
vmx_hardware_setup()
Quoted 60 lines
arch/x86/kvm/vmx/main.c +5
-1+if (r) + goto err_vendor_init; +/* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! @@ -207,9 +203,10 @@ static int __init vt_init(void)return 0 ; err_kvm_init: +kvm_x86_vendor_exit (); +err_vendor_init:vmx_exit (); err_vmx_init: -kvm_x86_vendor_exit ();return r; }module_init (vt_init);arch/x86/kvm/vmx/vmx.c +4
-2@@ -477,7 +477,7 @@ DEFINE_PER_CPU(struct vmcs *, current_vmcs);* We maintain a per-CPU linked-list of VMCS loaded on that CPU. This is needed * when a CPU is brought down, and we need to VMCLEAR all VMCSs loaded on it. */ -DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); +static DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); static DECLARE_BITMAP(vmx_vpid_bitmap, VMX_NR_VPIDS);static DEFINE_SPINLOCK (vmx_vpid_lock);@@ -8528,8 +8528,10 @@ int __init vmx_init(void)if (r)return r; -for_each_possible_cpu (cpu) +for_each_possible_cpu (cpu) { +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu));pi_init_cpu (cpu); + }cpu_emergency_register_virt_callback (vmx_emergency_disable);arch/x86/kvm/vmx/x86_ops.h +0
-3@@ -14,8 +14,6 @@ static inline __init void hv_init_evmcs(void) {}static inline void hv_reset_evmcs (void ) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ -DECLARE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); -bool kvm_is_vmx_supported (void );int __init vmx_init (void );void vmx_exit (void ); --2.25.1
Re: [PATCH v18 007/121] KVM: VMX: Reorder vmx initialization with kvm vendor initialization
In reply to [email protected]Quoted 5 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To match
vmx_exitcleanup. Nowvmx_init()is beforekvm_x86_vendor_init(),vmx_init()can initializeloaded_vmcss_on_cpu. Oppertunistically move it back intovmx_init().
It sort of does a revert of Patch 05. Though I still don't get the reason why we need Patch 05, why not move this patch before patch 06, then we can drop Patch 05 and of course the revert part of this patch?
Quoted 97 lines
Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- move theloaded_vmcss_on_cpuinitialization tovmx_init().
- fix error path ofvt_init(). by Chao and Binbin--- arch/x86/kvm/vmx/main.c | 17 +++++++---------- arch/x86/kvm/vmx/vmx.c | 6 ++++-- arch/x86/kvm/vmx/x86_ops.h | 2 -- 3 files changed, 11 insertions(+), 14 deletions(-)arch/x86/kvm/vmx/main.c +7
-10@@ -171,7 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {static int __init vt_init (void ) {unsigned int vcpu_size, vcpu_align; -int cpu, r; +int r;if (!kvm_is_vmx_supported ())return -EOPNOTSUPP ;@@ -182,18 +182,14 @@ static int __init vt_init(void)*/ hv_init_evmcs (); -/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ -for_each_possible_cpu (cpu) -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - - r =kvm_x86_vendor_init (&vt_init_ops); -if (r) -return r; - r =vmx_init ();if (r) goto err_vmx_init; + r =kvm_x86_vendor_init (&vt_init_ops); +if (r) + goto err_vendor_init; +/* * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! @@ -207,9 +203,10 @@ static int __init vt_init(void)return 0 ; err_kvm_init: +kvm_x86_vendor_exit (); +err_vendor_init:vmx_exit (); err_vmx_init: -kvm_x86_vendor_exit ();return r; }module_init (vt_init);arch/x86/kvm/vmx/vmx.c +4
-2@@ -477,7 +477,7 @@ DEFINE_PER_CPU(struct vmcs *, current_vmcs);* We maintain a per-CPU linked-list of VMCS loaded on that CPU. This is needed * when a CPU is brought down, and we need to VMCLEAR all VMCSs loaded on it. */ -DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); +static DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); static DECLARE_BITMAP(vmx_vpid_bitmap, VMX_NR_VPIDS);static DEFINE_SPINLOCK (vmx_vpid_lock);@@ -8528,8 +8528,10 @@ int __init vmx_init(void)if (r)return r; -for_each_possible_cpu (cpu) +for_each_possible_cpu (cpu) { +INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu));pi_init_cpu (cpu); + }cpu_emergency_register_virt_callback (vmx_emergency_disable);arch/x86/kvm/vmx/x86_ops.h +0
-2@@ -14,8 +14,6 @@ static inline __init void hv_init_evmcs(void) {}static inline void hv_reset_evmcs (void ) {}#endif /* IS_ENABLED(CONFIG_HYPERV) */ -DECLARE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); -bool kvm_is_vmx_supported (void );int __init vmx_init (void );void vmx_exit (void );
Re: [PATCH v18 007/121] KVM: VMX: Reorder vmx initialization with kvm vendor initialization
In reply to [email protected]Quoted 40 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
To match
vmx_exitcleanup. Nowvmx_init()is beforekvm_x86_vendor_init(),vmx_init()can initializeloaded_vmcss_on_cpu. Oppertunistically move it back intovmx_init().Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- move theloaded_vmcss_on_cpuinitialization tovmx_init().
- fix error path ofvt_init(). by Chao and Binbin--- arch/x86/kvm/vmx/main.c | 17 +++++++---------- arch/x86/kvm/vmx/vmx.c | 6 ++++-- arch/x86/kvm/vmx/x86_ops.h | 2 -- 3 files changed, 11 insertions(+), 14 deletions(-)arch/x86/kvm/vmx/main.c +1
-7@@ -171,7 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {static int __init vt_init (void ) {unsigned int vcpu_size, vcpu_align; -int cpu, r; +int r;if (!kvm_is_vmx_supported ())return -EOPNOTSUPP ;@@ -182,18 +182,14 @@ static int __init vt_init(void)*/ hv_init_evmcs (); -/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ -for_each_possible_cpu (cpu) -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - - r =kvm_x86_vendor_init (&vt_init_ops); -if (r)
arch/x86/kvm/vmx/main.c static int __init vt_init(void) · line 192
Quoted 10 lines
arch/x86/kvm/vmx/main.c +4
-2-return r; - r =vmx_init ();if (r) goto err_vmx_init; + r =kvm_x86_vendor_init (&vt_init_ops); +if (r) + goto err_vendor_init; +
we cannot simply change the calling order of vmx_init() and kvm_x86_vendor_init(). There is dependency between them.
e.g.,
kvm_x86_vendor_init()
-> ops->hardware_setup()
-> vmx_hardware_setup()
will update 'enable_ept' based on hardware capability (e.g., if the hardware support EPT or not), while 'enable_ept' is used in vmx_init().
Re: [PATCH v18 007/121] KVM: VMX: Reorder vmx initialization with kvm vendor initialization
Quoted 34 lines · Xiaoyao Li
Quoted 20 lines
arch/x86/kvm/vmx/main.c +1
-6@@ -171,7 +171,7 @@ struct kvm_x86_init_ops vt_init_ops __initdata = {static int __init vt_init (void ) {unsigned int vcpu_size, vcpu_align; -int cpu, r; +int r;if (!kvm_is_vmx_supported ())return -EOPNOTSUPP ;@@ -182,18 +182,14 @@ static int __init vt_init(void)*/ hv_init_evmcs (); -/* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ -for_each_possible_cpu (cpu) -INIT_LIST_HEAD (&per_cpu (loaded_vmcss_on_cpu, cpu)); - - r =kvm_x86_vendor_init (&vt_init_ops);arch/x86/kvm/vmx/main.c static int __init vt_init(void) · line 190
Quoted 10 lines
arch/x86/kvm/vmx/main.c +4
-3-if (r) -return r; - r =vmx_init ();if (r) goto err_vmx_init; + r =kvm_x86_vendor_init (&vt_init_ops); +if (r) + goto err_vendor_init; +we cannot simply change the calling order of
vmx_init()andkvm_x86_vendor_init(). There is dependency between them.
Quoted 8 lines
e.g.,
kvm_x86_vendor_init()
->ops->hardware_setup()-> vmx_hardware_setup()will update '
enable_ept' based on hardware capability (e.g., if the hardware support EPT or not), while 'enable_ept' is used invmx_init().
I gave up this clean up to drop this patch with v19.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 8/121 KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX) on all online CPUs, detect the TDX module availability, initialize it and disable VMX.
To enable/disable VMX on all online CPUs, utilize vmx_hardware_enable/disable(). The method also initializes each CPU for TDX. TDX requires calling a TDX initialization function per logical processor (LP) before the LP uses TDX. When the CPU is becoming online, call the TDX LP initialization API. If it fails to initialize TDX, refuse CPU online for simplicity instead of TDX avoiding the failed LP.
There are several options on when to initialize the TDX module. A.) kernel module loading time, B.) the first guest TD creation time. A.) was chosen. With B.), a user may hit an error of the TDX initialization when trying to create the first guest TD. The machine that fails to initialize the TDX module can't boot any guest TD further. Such failure is undesirable and a surprise because the user expects that the machine can accommodate guest TD, but not. So A.) is better than B.).
Introduce a module parameter, kvm_intel.tdx, to explicitly enable TDX KVM support. It's off by default to keep the same behavior for those who don't use TDX. Implement hardware_setup method to detect TDX feature of CPU and initialize TDX module.
---
v18:
- Added comment in vt_hardware_enable() by Binbin.
--- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/main.c | 38 ++++++++++++++++- arch/x86/kvm/vmx/tdx.c | 84 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 8 ++++ 4 files changed, 129 insertions(+), 2 deletions(-) create mode 100644 arch/x86/kvm/vmx/tdx.c
arch/x86/kvm/Makefile +1 -0
@@ -24,6 +24,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ svm/sev.o
arch/x86/kvm/vmx/main.c +36 -2
@@ -6,6 +6,40 @@#include "nested.h" #include "pmu.h" +static bool enable_tdx__ro_after_init ; +module_param_named (tdx, enable_tdx, bool,0444 ); + +static int vt_hardware_enable (void ) +{ +int ret; + + ret =vmx_hardware_enable (); +if (ret || !enable_tdx) +return ret; + + ret =tdx_cpu_enable (); +if (ret) +/* +* In error case, we keep the CPU offline in error case. So +* need to revert VMXON. +*/ +vmx_hardware_disable (); +return ret; +} + +static __init int vt_hardware_setup (void ) +{ +int ret; + + ret =vmx_hardware_setup (); +if (ret) +return ret; + + enable_tdx = enable_tdx && !tdx_hardware_setup (&vt_x86_ops); + +return 0 ; +} +#define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \
@@ -22,7 +56,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .hardware_unsetup = vmx_hardware_unsetup, - .hardware_enable = vmx_hardware_enable, + .hardware_enable = vt_hardware_enable, .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr,
@@ -161,7 +195,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { };struct kvm_x86_init_ops vt_init_ops__initdata = { - .hardware_setup = vmx_hardware_setup, + .hardware_setup = vt_hardware_setup, .handle_intel_pt_intr =NULL , .runtime_ops = &vt_x86_ops,
arch/x86/kvm/vmx/tdx.c new file +84 -0
84 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +8 -0
@@ -18,6 +18,8 @@ bool kvm_is_vmx_supported(void);int __init vmx_init (void );void vmx_exit (void ); +__init int vmx_hardware_setup (void ); +extern struct kvm_x86_ops vt_x86_ops__initdata ;extern struct kvm_x86_init_ops vt_init_ops__initdata ;
@@ -133,4 +135,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);#endif void vmx_setup_mce (struct kvm_vcpu *vcpu); +#ifdef CONFIG_INTEL_TDX_HOST +int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +#else +static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; } +#endif +#endif /* __KVM_X86_VMX_X86_OPS_H */
Signature · 1 line
2.25.1
Re: [PATCH v18 008/121] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module
Quoted 61 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX) on all online CPUs, detect the TDX module availability, initialize it and disable VMX.
To enable/disable VMX on all online CPUs, utilize vmx_hardware_enable/disable(). The method also initializes each CPU for TDX. TDX requires calling a TDX initialization function per logical processor (LP) before the LP uses TDX. When the CPU is becoming online, call the TDX LP initialization API. If it fails to initialize TDX, refuse CPU online for simplicity instead of TDX avoiding the failed LP.
There are several options on when to initialize the TDX module. A.) kernel module loading time, B.) the first guest TD creation time. A.) was chosen. With B.), a user may hit an error of the TDX initialization when trying to create the first guest TD. The machine that fails to initialize the TDX module can't boot any guest TD further. Such failure is undesirable and a surprise because the user expects that the machine can accommodate guest TD, but not. So A.) is better than B.).
Introduce a module parameter, kvm_intel.tdx, to explicitly enable TDX KVM support. It's off by default to keep the same behavior for those who don't use TDX. Implement
hardware_setupmethod to detect TDX feature of CPU and initialize TDX module.Suggested-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- Added comment invt_hardware_enable()by Binbin.--- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/main.c | 38 ++++++++++++++++- arch/x86/kvm/vmx/tdx.c | 84 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 8 ++++ 4 files changed, 129 insertions(+), 2 deletions(-)create mode 100644
arch/x86/kvm/vmx/tdx.carch/x86/kvm/Makefile +1
-0@@ -24,6 +24,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ svm/sev.oarch/x86/kvm/vmx/main.c +2
-0@@ -6,6 +6,40 @@#include "nested.h" #include "pmu.h" +static bool enable_tdx__ro_after_init ; +module_param_named (tdx, enable_tdx, bool,0444 );
arch/x86/kvm/vmx/main.c line 20
Quoted 10 lines
arch/x86/kvm/vmx/main.c +10
-0+ +static int vt_hardware_enable (void ) +{ +int ret; + + ret =vmx_hardware_enable (); +if (ret || !enable_tdx) +return ret; + + ret =tdx_cpu_enable ();
What's the reason for it?
vt_hardware_setup()
-> tdx_hardware_setup()
-> on_each_cpu(vmx_tdx_on, &vmx_tdx, true);
-> vmx_tdx_on()
-> tdx_cpu_enable()ensures tdx_cpu_enable() is called once. No need to call it every vt_hardware_enable().
Quoted 65 lines
arch/x86/kvm/vmx/main.c +24
-2+if (ret) +/* +* In error case, we keep the CPU offline in error case. So +* need to revert VMXON. +*/ +vmx_hardware_disable (); +return ret; +} + +static __init int vt_hardware_setup (void ) +{ +int ret; + + ret =vmx_hardware_setup (); +if (ret) +return ret; + + enable_tdx = enable_tdx && !tdx_hardware_setup (&vt_x86_ops); + +return 0 ; +} +#define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \@@ -22,7 +56,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .hardware_unsetup = vmx_hardware_unsetup, - .hardware_enable = vmx_hardware_enable, + .hardware_enable = vt_hardware_enable, .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr,@@ -161,7 +195,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { };struct kvm_x86_init_ops vt_init_ops__initdata = { - .hardware_setup = vmx_hardware_setup, + .hardware_setup = vt_hardware_setup, .handle_intel_pt_intr =NULL , .runtime_ops = &vt_x86_ops,arch/x86/kvm/vmx/tdx.c new file +16
-0@@ -0,0 +1,84 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/cpu.h> + +#include <asm/tdx.h> + +#include "capabilities.h" +#include "x86_ops.h" +#include "x86.h" + +#undef pr_fmt +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt + +static int __init tdx_module_setup (void ) +{ +int ret; +
arch/x86/kvm/vmx/tdx.c line 26
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c new file +10
-0+ ret =tdx_enable (); +if (ret) { +pr_info ("Failed to initialize TDX module.\n" ); +return ret; + } + +return 0 ; +} + +struct vmx_tdx_enabled {
the name is so confusing at first glance, ...
arch/x86/kvm/vmx/tdx.c line 31
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c new file +5
-0+cpumask_var_t vmx_enabled; +atomic_t err; +}; + +static void __init vmx_tdx_on (void *_vmx_tdx)
so is this function name.
Quoted 58 lines
arch/x86/kvm/vmx/tdx.c new file +53
-0+{ +struct vmx_tdx_enabled *vmx_tdx = _vmx_tdx; +int r; + + r =vmx_hardware_enable (); +if (!r) { +cpumask_set_cpu (smp_processor_id (), vmx_tdx->vmx_enabled); + r =tdx_cpu_enable (); + } +if (r) +atomic_set (&vmx_tdx->err, r); +} + +static void __init vmx_off (void *_vmx_enabled) +{ +cpumask_var_t *vmx_enabled = (cpumask_var_t *)_vmx_enabled; + +if (cpumask_test_cpu (smp_processor_id (), *vmx_enabled)) +vmx_hardware_disable (); +} + +int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops) +{ +struct vmx_tdx_enabled vmx_tdx = { + .err =ATOMIC_INIT (0 ), + }; +int r =0 ; + +if (!enable_ept) { +pr_warn ("Cannot enable TDX with EPT disabled\n" ); +return -EINVAL ; + } + +if (!zalloc_cpumask_var (&vmx_tdx.vmx_enabled,GFP_KERNEL )) { + r = -ENOMEM ; + goto out; + } + +/* tdx_enable() in tdx_module_setup() requires cpus lock. */ +cpus_read_lock (); +on_each_cpu (vmx_tdx_on, &vmx_tdx, true);/* TDX requires vmxon. */ + r =atomic_read (&vmx_tdx.err); +if (!r) + r =tdx_module_setup (); +else + r = -EIO ; +on_each_cpu (vmx_off, &vmx_tdx.vmx_enabled, true); +cpus_read_unlock (); +free_cpumask_var (vmx_tdx.vmx_enabled); + +out: +return r; +}arch/x86/kvm/vmx/x86_ops.h +0
-0
arch/x86/kvm/vmx/x86_ops.h bool kvm_is_vmx_supported(void); · line 21
Quoted 4 lines
arch/x86/kvm/vmx/x86_ops.h +1
-0@@ -18,6 +18,8 @@ bool kvm_is_vmx_supported(void);int __init vmx_init (void );void vmx_exit (void ); +__init int vmx_hardware_setup (void );
superfluous declaration.
It's added in patch 4 already.
Quoted 14 lines
arch/x86/kvm/vmx/x86_ops.h +6
-0extern struct kvm_x86_ops vt_x86_ops__initdata ;extern struct kvm_x86_init_ops vt_init_ops__initdata ;@@ -133,4 +135,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu);#endif void vmx_setup_mce (struct kvm_vcpu *vcpu); +#ifdef CONFIG_INTEL_TDX_HOST +int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +#else +static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; } +#endif +#endif /* __KVM_X86_VMX_X86_OPS_H */
PATCH v18 9/121 KVM: TDX: Add placeholders for TDX VM/vcpu structure
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add placeholders TDX VM/vcpu structure that overlays with VMX VM/vcpu structures. Initialize VM structure size and vcpu size/align so that x86 KVM common code knows those size irrespective of VMX or TDX. Those structures will be populated as guest creation logic develops.
Add helper functions to check if the VM is guest TD and add conversion functions between KVM VM/VCPU and TDX VM/VCPU.
---
v14 -> v15:
- use KVM_X86_TDX_VM
--- arch/x86/kvm/vmx/main.c | 18 +++++++++++++-- arch/x86/kvm/vmx/tdx.c | 1 + arch/x86/kvm/vmx/tdx.h | 50 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 2 deletions(-) create mode 100644 arch/x86/kvm/vmx/tdx.h
arch/x86/kvm/vmx/main.c +16 -2
@@ -5,6 +5,7 @@#include "vmx.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 );
@@ -216,6 +217,21 @@ static int __init vt_init(void)*/ hv_init_evmcs (); +/* +* kvm_x86_ops is updated with vt_x86_ops. vt_x86_ops.vm_size must +* be set before kvm_x86_vendor_init(). +*/ + vcpu_size =sizeof (struct vcpu_vmx ); + vcpu_align = __alignof__(struct vcpu_vmx ); +if (enable_tdx) { + vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size, +sizeof (struct kvm_tdx )); + vcpu_size = max_t(unsigned int, vcpu_size, +sizeof (struct vcpu_tdx )); + vcpu_align = max_t(unsigned int, vcpu_align, + __alignof__(struct vcpu_tdx )); + } + r =vmx_init ();if (r) goto err_vmx_init;
@@ -228,8 +244,6 @@ static int __init vt_init(void)* Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! */ - vcpu_size =sizeof (struct vcpu_vmx ); - vcpu_align = __alignof__(struct vcpu_vmx ); r =kvm_init (vcpu_size, vcpu_align,THIS_MODULE );if (r) goto err_kvm_init;
arch/x86/kvm/vmx/tdx.c +1 -0
@@ -6,6 +6,7 @@#include "capabilities.h" #include "x86_ops.h" #include "x86.h" +#include "tdx.h" #undef pr_fmt#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt
arch/x86/kvm/vmx/tdx.h new file +50 -0
50 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH v18 009/121] KVM: TDX: Add placeholders for TDX VM/vcpu structure
Quoted 35 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add placeholders TDX VM/vcpu structure that overlays with VMX VM/vcpu structures. Initialize VM structure size and vcpu size/align so that x86 KVM common code knows those size irrespective of VMX or TDX. Those structures will be populated as guest creation logic develops.
Add helper functions to check if the VM is guest TD and add conversion functions between KVM VM/VCPU and TDX VM/VCPU.
Signed-off-by Isaku Yamahata <[email protected]>---
v14 -> v15:
- useKVM_X86_TDX_VM--- arch/x86/kvm/vmx/main.c | 18 +++++++++++++-- arch/x86/kvm/vmx/tdx.c | 1 + arch/x86/kvm/vmx/tdx.h | 50 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 2 deletions(-)create mode 100644
arch/x86/kvm/vmx/tdx.harch/x86/kvm/vmx/main.c +1
-0@@ -5,6 +5,7 @@#include "vmx.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 );
arch/x86/kvm/vmx/main.c static int __init vt_init(void) · line 226
Quoted 10 lines
arch/x86/kvm/vmx/main.c +7
-0@@ -216,6 +217,21 @@ static int __init vt_init(void)*/ hv_init_evmcs (); +/* +* kvm_x86_ops is updated with vt_x86_ops. vt_x86_ops.vm_size must +* be set before kvm_x86_vendor_init(). +*/ + vcpu_size =sizeof (struct vcpu_vmx ); + vcpu_align = __alignof__(struct vcpu_vmx ); +if (enable_tdx) {
until now, 'enable_tdx' is totally decided by module_param, which might change from Y to N if tdx fails enabling. In this case, the below should not be updated.
Quoted 88 lines
arch/x86/kvm/vmx/main.c +8
-2+ vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size, +sizeof (struct kvm_tdx )); + vcpu_size = max_t(unsigned int, vcpu_size, +sizeof (struct vcpu_tdx )); + vcpu_align = max_t(unsigned int, vcpu_align, + __alignof__(struct vcpu_tdx )); + } + r =vmx_init ();if (r) goto err_vmx_init;@@ -228,8 +244,6 @@ static int __init vt_init(void)* Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! */ - vcpu_size =sizeof (struct vcpu_vmx ); - vcpu_align = __alignof__(struct vcpu_vmx ); r =kvm_init (vcpu_size, vcpu_align,THIS_MODULE );if (r) goto err_kvm_init;arch/x86/kvm/vmx/tdx.c +1
-0@@ -6,6 +6,7 @@#include "capabilities.h" #include "x86_ops.h" #include "x86.h" +#include "tdx.h" #undef pr_fmt#define pr_fmt (fmt) KBUILD_MODNAME ": " fmtarch/x86/kvm/vmx/tdx.h new file +50
-0@@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_H +#define __KVM_X86_TDX_H + +#ifdef CONFIG_INTEL_TDX_HOST +struct kvm_tdx { +struct kvm kvm; +/* TDX specific members follow. */ +}; + +struct vcpu_tdx { +struct kvm_vcpu vcpu; +/* TDX specific members follow. */ +}; + +static inline bool is_td (struct kvm *kvm) +{ +return kvm->arch.vm_type ==KVM_X86_TDX_VM ; +} + +static inline bool is_td_vcpu (struct kvm_vcpu *vcpu) +{ +return is_td (vcpu->kvm); +} + +static inline struct kvm_tdx *to_kvm_tdx (struct kvm *kvm) +{ + return container_of(kvm, struct kvm_tdx, kvm); +} + +static inline struct vcpu_tdx *to_tdx (struct kvm_vcpu *vcpu) +{ + return container_of(vcpu, struct vcpu_tdx, vcpu); +} +#else +struct kvm_tdx { +struct kvm kvm; +}; + +struct vcpu_tdx { +struct kvm_vcpu vcpu; +}; + +static inline bool is_td (struct kvm *kvm) {return false; } +static inline bool is_td_vcpu (struct kvm_vcpu *vcpu) {return false; } +static inline struct kvm_tdx *to_kvm_tdx (struct kvm *kvm) {return NULL ; } +static inline struct vcpu_tdx *to_tdx (struct kvm_vcpu *vcpu) {return NULL ; } +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_X86_TDX_H */
PATCH v18 10/121 KVM: TDX: Make TDX VM type supported
In reply to [email protected]From: Isaku Yamahata <[email protected]>
NOTE: This patch is in position of the patch series for developers to be able to test codes during the middle of the patch series although this patch series doesn't provide functional features until the all the patches of this patch series. When merging this patch series, this patch can be moved to the end.
As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is vm_init callback for KVM_CREATE_VM.
--- arch/x86/kvm/vmx/main.c | 18 ++++++++++++++++-- arch/x86/kvm/vmx/tdx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.c | 6 ------ arch/x86/kvm/vmx/x86_ops.h | 3 ++- 4 files changed, 24 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/main.c +16 -2
@@ -10,6 +10,12 @@static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 ); +static bool vt_is_vm_type_supported (unsigned long type) +{ +return __kvm_is_vm_type_supported (type) || + (enable_tdx &&tdx_is_vm_type_supported (type)); +} +static int vt_hardware_enable (void ) {int ret;
@@ -41,6 +47,14 @@ static __init int vt_hardware_setup(void)return 0 ; } +static int vt_vm_init (struct kvm *kvm) +{ +if (is_td (kvm)) +return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ + +return vmx_vm_init (kvm); +} +#define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \
@@ -61,9 +75,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, - .is_vm_type_supported = vmx_is_vm_type_supported, + .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), - .vm_init = vmx_vm_init, + .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy, .vcpu_precreate = vmx_vcpu_precreate,
arch/x86/kvm/vmx/tdx.c +6 -0
@@ -24,6 +24,12 @@ static int __init tdx_module_setup(void)return 0 ; } +bool tdx_is_vm_type_supported (unsigned long type) +{ +/* enable_tdx check is done by the caller. */ +return type ==KVM_X86_TDX_VM ; +} +struct vmx_tdx_enabled {cpumask_var_t vmx_enabled;atomic_t err;
arch/x86/kvm/vmx/vmx.c +0 -6
@@ -7531,12 +7531,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)return err; } -bool vmx_is_vm_type_supported (unsigned long type) -{ -/* TODO: Check if TDX is supported. */ -return __kvm_is_vm_type_supported (type); -} -#define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"#define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n"
arch/x86/kvm/vmx/x86_ops.h +2 -1
@@ -28,7 +28,6 @@ void vmx_hardware_unsetup(void);int vmx_check_processor_compat (void );int vmx_hardware_enable (void );void vmx_hardware_disable (void ); -bool vmx_is_vm_type_supported (unsigned long type);int vmx_vm_init (struct kvm *kvm);void vmx_vm_destroy (struct kvm *kvm);int vmx_vcpu_precreate (struct kvm *kvm);
@@ -137,8 +136,10 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +bool tdx_is_vm_type_supported (unsigned long type);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; } +static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
Signature · 1 line
2.25.1
PATCH v18 11/121 The start of TDX KVM patch series: TDX architectural definitions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of TDX architectural definitions.
--- Documentation/virt/kvm/index.rst | 2 ++ .../virt/kvm/intel-tdx-layer-status.rst | 29 +++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 Documentation/virt/kvm/intel-tdx-layer-status.rst
Documentation/virt/kvm/index.rst +2 -0
@@ -19,3 +19,5 @@ KVM vcpu-requests halt-polling review-checklist + + intel-tdx-layer-status
Documentation/virt/kvm/intel-tdx-layer-status.rst new file +29 -0
@@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +Intel Trust Dodmain Extensions(TDX) +=================================== + +Layer status +============ +What qemu can do +---------------- +- TDX VM TYPE is exposed to Qemu. +- Qemu can try to create VM of TDX VM type and then fails. + +Patch Layer status +------------------ + Patch layer Status + +* TDX, VMX coexistence: Applied +* TDX architectural definitions: Applying +* TD VM creation/destruction: Not yet +* TD vcpu creation/destruction: Not yet +* TDX EPT violation: Not yet +* TD finalization: Not yet +* TD vcpu enter/exit: Not yet +* TD vcpu interrupts/exit/hypercall: Not yet + +* KVM MMU GPA shared bits: Not yet +* KVM TDP refactoring for TDX: Not yet +* KVM TDP MMU hooks: Not yet
Signature · 1 line
2.25.1
PATCH v18 12/121 KVM: TDX: Define TDX architectural definitions
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Define architectural definitions for KVM to issue the TDX SEAMCALLs.
Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference.
---
v18:
- Add metadata field id
--- arch/x86/kvm/vmx/tdx_arch.h | 269 ++++++++++++++++++++++++++++++++++++ 1 file changed, 269 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_arch.h
arch/x86/kvm/vmx/tdx_arch.h new file +269 -0
@@ -0,0 +1,269 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_RELOCATE 5 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_MEM_SEPT_RD 25 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_SEPT_REMOVE 30 +#define TDH_SYS_RD 34 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40 +#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_NON_ARCH BIT_ULL(63) +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define __BUILD_TDX_FIELD (non_arch, class, field) \ + (((non_arch) ? TDX_NON_ARCH : 0) | \ + ((u64)(class) << TDX_CLASS_SHIFT) | \ + ((u64)(field) & TDX_FIELD_MASK)) + +#define BUILD_TDX_FIELD (class, field) \ + __BUILD_TDX_FIELD(false, (class), (field)) + +#define BUILD_TDX_FIELD_NON_ARCH (class, field) \ + __BUILD_TDX_FIELD(true, (class), (field)) + + +/* Class code for TD */ +#define TD_CLASS_EXECUTION_CONTROLS 17ULL + +/* Class code for TDVPS */ +#define TDVPS_CLASS_VMCS 0ULL +#define TDVPS_CLASS_GUEST_GPR 16ULL +#define TDVPS_CLASS_OTHER_GUEST 17ULL +#define TDVPS_CLASS_MANAGEMENT 32ULL + +enum tdx_tdcs_execution_control { +TD_TDCS_EXEC_TSC_OFFSET =10 , +}; + +/* @field is any of enum tdx_tdcs_execution_control */ +#define TDCS_EXEC (field) BUILD_TDX_FIELD(TD_CLASS_EXECUTION_CONTROLS, (field)) + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS (field) BUILD_TDX_FIELD(TDVPS_CLASS_VMCS, (field)) + +enum tdx_vcpu_guest_other_state { +TD_VCPU_STATE_DETAILS_NON_ARCH =0x100 , +}; + +union tdx_vcpu_state_details { +struct { +u64 vmxip :1 ; +u64 reserved :63 ; + }; +u64 full; +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE (field) BUILD_TDX_FIELD(TDVPS_CLASS_OTHER_GUEST, (field)) +#define TDVPS_STATE_NON_ARCH (field) BUILD_TDX_FIELD_NON_ARCH(TDVPS_CLASS_OTHER_GUEST, (field)) + +/* Management class fields */ +enum tdx_vcpu_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_vcpu_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(TDVPS_CLASS_MANAGEMENT, (field)) + +#define TDX_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_value { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +#define TDX_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX_TD_ATTR_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +/* +* TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. +*/ +#define TDX_MAX_VCPUS (~(u16)0) + +struct td_params { +u64 attributes; +u64 xfam; +u16 max_vcpus; +u8 reserved0[6 ]; + +u64 eptp_controls; +u64 exec_controls; +u16 tsc_frequency; +u8 reserved1[38 ]; + +u64 mrconfigid[6 ]; +u64 mrowner[6 ]; +u64 mrownerconfig[6 ]; +u64 reserved2[4 ]; + +union { + DECLARE_FLEX_ARRAY(struct tdx_cpuid_value, cpuid_values); +u8 reserved3[768 ]; + }; +} __packed__aligned(1024) ; + +/* +* Guest uses MAX_PA for GPAW when set. +* 0: GPA.SHARED bit is GPA[47] +* 1: GPA.SHARED bit is GPA[51] +*/ +#define TDX_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* +* TDH.VP.ENTER, TDG.VP.VMCALL preserves RBP +* 0: RBP can be used for TDG.VP.VMCALL input. RBP is clobbered. +* 1: RBP can't be used for TDG.VP.VMCALL input. RBP is preserved. +*/ +#define TDX_CONTROL_FLAG_NO_RBP_MOD BIT_ULL(1) + + +/* +* TDX requires the frequency to be defined in units of 25MHz, which is the +* frequency of the core crystal clock on TDX-capable platforms, i.e. the TDX +* module can only program frequencies that are multiples of 25MHz. The +* frequency must be between 100mhz and 10ghz (inclusive). +*/ +#define TDX_TSC_KHZ_TO_25MHZ (tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX_TSC_25MHZ_TO_KHZ (tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +union tdx_sept_entry { +struct { +u64 r :1 ; +u64 w :1 ; +u64 x :1 ; +u64 mt :3 ; +u64 ipat :1 ; +u64 leaf :1 ; +u64 a :1 ; +u64 d :1 ; +u64 xu :1 ; +u64 ignored0 :1 ; +u64 pfn :40 ; +u64 reserved :5 ; +u64 vgp :1 ; +u64 pwa :1 ; +u64 ignored1 :1 ; +u64 sss :1 ; +u64 spp :1 ; +u64 ignored2 :1 ; +u64 sve :1 ; + }; +u64 raw; +}; + +enum tdx_sept_entry_state { +TDX_SEPT_FREE =0 , +TDX_SEPT_BLOCKED =1 , +TDX_SEPT_PENDING =2 , +TDX_SEPT_PENDING_BLOCKED =3 , +TDX_SEPT_PRESENT =4 , +}; + +union tdx_sept_level_state { +struct { +u64 level :3 ; +u64 reserved0 :5 ; +u64 state :8 ; +u64 reserved1 :48 ; + }; +u64 raw; +}; + +/* +* Global scope metadata field ID. +* See Table "Global Scope Metadata", TDX module 1.5 ABI spec. +*/ +#define MD_FIELD_ID_SYS_ATTRIBUTES 0x0A00000200000000ULL +#define MD_FIELD_ID_FEATURES0 0x0A00000300000008ULL +#define MD_FIELD_ID_ATTRS_FIXED0 0x1900000300000000ULL +#define MD_FIELD_ID_ATTRS_FIXED1 0x1900000300000001ULL +#define MD_FIELD_ID_XFAM_FIXED0 0x1900000300000002ULL +#define MD_FIELD_ID_XFAM_FIXED1 0x1900000300000003ULL + +#define MD_FIELD_ID_TDCS_BASE_SIZE 0x9800000100000100ULL +#define MD_FIELD_ID_TDVPS_BASE_SIZE 0x9800000100000200ULL + +#define MD_FIELD_ID_NUM_CPUID_CONFIG 0x9900000100000004ULL +#define MD_FIELD_ID_CPUID_CONFIG_LEAVES 0x9900000300000400ULL +#define MD_FIELD_ID_CPUID_CONFIG_VALUES 0x9900000300000500ULL + +#define TDX_MAX_NR_CPUID_CONFIGS 37 + +#define TDX_MD_ELEMENT_SIZE_8BITS 0 +#define TDX_MD_ELEMENT_SIZE_16BITS 1 +#define TDX_MD_ELEMENT_SIZE_32BITS 2 +#define TDX_MD_ELEMENT_SIZE_64BITS 3 + +union tdx_md_field_id { +struct { +u64 field :24 ; +u64 reserved0 :8 ; +u64 element_size_code :2 ; +u64 last_element_in_field :4 ; +u64 reserved1 :3 ; +u64 inc_size :1 ; +u64 write_mask_valid :1 ; +u64 context :3 ; +u64 reserved2 :1 ; +u64 class :6 ; +u64 reserved3 :1 ; +u64 non_arch :1 ; + }; +u64 raw; +}; + +#define TDX_MD_ELEMENT_SIZE_CODE (_field_id) \ + ({ union tdx_md_field_id _fid = { .raw = (_field_id)}; \ + _fid.element_size_code; }) + +#endif /* __KVM_X86_TDX_ARCH_H */
Signature · 1 line
2.25.1
Re: [PATCH v18 012/121] KVM: TDX: Define TDX architectural definitions
Quoted 11 lines · On Mon, Jan 22, 2024 at 03:52:48PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Define architectural definitions for KVM to issue the TDX SEAMCALLs.
Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference.
Co-developed-by Sean Christopherson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>
Quoted 287 lines
---
v18:
- Add metadata field id--- arch/x86/kvm/vmx/tdx_arch.h | 269 ++++++++++++++++++++++++++++++++++++ 1 file changed, 269 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_arch.harch/x86/kvm/vmx/tdx_arch.h new file +269
-1@@ -0,0 +1,269 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_RELOCATE 5 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_MEM_SEPT_RD 25 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_SEPT_REMOVE 30 +#define TDH_SYS_RD 34 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40 +#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_NON_ARCH BIT_ULL(63) +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define __BUILD_TDX_FIELD (non_arch, class, field) \ + (((non_arch) ? TDX_NON_ARCH : 0) | \ + ((u64)(class) << TDX_CLASS_SHIFT) | \ + ((u64)(field) & TDX_FIELD_MASK)) + +#define BUILD_TDX_FIELD (class, field) \ + __BUILD_TDX_FIELD(false, (class), (field)) + +#define BUILD_TDX_FIELD_NON_ARCH (class, field) \ + __BUILD_TDX_FIELD(true, (class), (field)) + + +/* Class code for TD */ +#define TD_CLASS_EXECUTION_CONTROLS 17ULL + +/* Class code for TDVPS */ +#define TDVPS_CLASS_VMCS 0ULL +#define TDVPS_CLASS_GUEST_GPR 16ULL +#define TDVPS_CLASS_OTHER_GUEST 17ULL +#define TDVPS_CLASS_MANAGEMENT 32ULL + +enum tdx_tdcs_execution_control { +TD_TDCS_EXEC_TSC_OFFSET =10 , +}; + +/* @field is any of enum tdx_tdcs_execution_control */ +#define TDCS_EXEC (field) BUILD_TDX_FIELD(TD_CLASS_EXECUTION_CONTROLS, (field)) + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS (field) BUILD_TDX_FIELD(TDVPS_CLASS_VMCS, (field)) + +enum tdx_vcpu_guest_other_state { +TD_VCPU_STATE_DETAILS_NON_ARCH =0x100 , +}; + +union tdx_vcpu_state_details { +struct { +u64 vmxip :1 ; +u64 reserved :63 ; + }; +u64 full; +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE (field) BUILD_TDX_FIELD(TDVPS_CLASS_OTHER_GUEST, (field)) +#define TDVPS_STATE_NON_ARCH (field) BUILD_TDX_FIELD_NON_ARCH(TDVPS_CLASS_OTHER_GUEST, (field)) + +/* Management class fields */ +enum tdx_vcpu_guest_management { +TD_VCPU_PEND_NMI =11 , +}; + +/* @field is any of enum tdx_vcpu_guest_management */ +#define TDVPS_MANAGEMENT (field) BUILD_TDX_FIELD(TDVPS_CLASS_MANAGEMENT, (field)) + +#define TDX_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_value { +u32 eax; +u32 ebx; +u32 ecx; +u32 edx; +} __packed; + +#define TDX_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX_TD_ATTR_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +/* +* TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. +*/ +#define TDX_MAX_VCPUS (~(u16)0) + +struct td_params { +u64 attributes; +u64 xfam; +u16 max_vcpus; +u8 reserved0[6 ]; + +u64 eptp_controls; +u64 exec_controls; +u16 tsc_frequency; +u8 reserved1[38 ]; + +u64 mrconfigid[6 ]; +u64 mrowner[6 ]; +u64 mrownerconfig[6 ]; +u64 reserved2[4 ]; + +union { + DECLARE_FLEX_ARRAY(struct tdx_cpuid_value, cpuid_values); +u8 reserved3[768 ]; + }; +} __packed__aligned(1024) ; + +/* +* Guest uses MAX_PA for GPAW when set. +* 0: GPA.SHARED bit is GPA[47] +* 1: GPA.SHARED bit is GPA[51] +*/ +#define TDX_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* +* TDH.VP.ENTER, TDG.VP.VMCALL preserves RBP +* 0: RBP can be used for TDG.VP.VMCALL input. RBP is clobbered. +* 1: RBP can't be used for TDG.VP.VMCALL input. RBP is preserved. +*/ +#define TDX_CONTROL_FLAG_NO_RBP_MOD BIT_ULL(1) + + +/* +* TDX requires the frequency to be defined in units of 25MHz, which is the +* frequency of the core crystal clock on TDX-capable platforms, i.e. the TDX +* module can only program frequencies that are multiples of 25MHz. The +* frequency must be between 100mhz and 10ghz (inclusive). +*/ +#define TDX_TSC_KHZ_TO_25MHZ (tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX_TSC_25MHZ_TO_KHZ (tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +union tdx_sept_entry { +struct { +u64 r :1 ; +u64 w :1 ; +u64 x :1 ; +u64 mt :3 ; +u64 ipat :1 ; +u64 leaf :1 ; +u64 a :1 ; +u64 d :1 ; +u64 xu :1 ; +u64 ignored0 :1 ; +u64 pfn :40 ; +u64 reserved :5 ; +u64 vgp :1 ; +u64 pwa :1 ; +u64 ignored1 :1 ; +u64 sss :1 ; +u64 spp :1 ; +u64 ignored2 :1 ; +u64 sve :1 ; + }; +u64 raw; +}; + +enum tdx_sept_entry_state { +TDX_SEPT_FREE =0 , +TDX_SEPT_BLOCKED =1 , +TDX_SEPT_PENDING =2 , +TDX_SEPT_PENDING_BLOCKED =3 , +TDX_SEPT_PRESENT =4 , +}; + +union tdx_sept_level_state { +struct { +u64 level :3 ; +u64 reserved0 :5 ; +u64 state :8 ; +u64 reserved1 :48 ; + }; +u64 raw; +}; + +/* +* Global scope metadata field ID. +* See Table "Global Scope Metadata", TDX module 1.5 ABI spec. +*/ +#define MD_FIELD_ID_SYS_ATTRIBUTES 0x0A00000200000000ULL +#define MD_FIELD_ID_FEATURES0 0x0A00000300000008ULL +#define MD_FIELD_ID_ATTRS_FIXED0 0x1900000300000000ULL +#define MD_FIELD_ID_ATTRS_FIXED1 0x1900000300000001ULL +#define MD_FIELD_ID_XFAM_FIXED0 0x1900000300000002ULL +#define MD_FIELD_ID_XFAM_FIXED1 0x1900000300000003ULL + +#define MD_FIELD_ID_TDCS_BASE_SIZE 0x9800000100000100ULL +#define MD_FIELD_ID_TDVPS_BASE_SIZE 0x9800000100000200ULL + +#define MD_FIELD_ID_NUM_CPUID_CONFIG 0x9900000100000004ULL +#define MD_FIELD_ID_CPUID_CONFIG_LEAVES 0x9900000300000400ULL +#define MD_FIELD_ID_CPUID_CONFIG_VALUES 0x9900000300000500ULL + +#define TDX_MAX_NR_CPUID_CONFIGS 37 + +#define TDX_MD_ELEMENT_SIZE_8BITS 0 +#define TDX_MD_ELEMENT_SIZE_16BITS 1 +#define TDX_MD_ELEMENT_SIZE_32BITS 2 +#define TDX_MD_ELEMENT_SIZE_64BITS 3 + +union tdx_md_field_id { +struct { +u64 field :24 ; +u64 reserved0 :8 ; +u64 element_size_code :2 ; +u64 last_element_in_field :4 ; +u64 reserved1 :3 ; +u64 inc_size :1 ; +u64 write_mask_valid :1 ; +u64 context :3 ; +u64 reserved2 :1 ; +u64 class :6 ; +u64 reserved3 :1 ; +u64 non_arch :1 ; + }; +u64 raw; +}; + +#define TDX_MD_ELEMENT_SIZE_CODE (_field_id) \ + ({ union tdx_md_field_id _fid = { .raw = (_field_id)}; \ + _fid.element_size_code; }) + +#endif /* __KVM_X86_TDX_ARCH_H */ --2.25.1
Re: [PATCH v18 012/121] KVM: TDX: Define TDX architectural definitions
In reply to [email protected]Quoted 65 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Define architectural definitions for KVM to issue the TDX SEAMCALLs.
Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference.
Co-developed-by Sean Christopherson <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>---
v18:
- Add metadata field id--- arch/x86/kvm/vmx/tdx_arch.h | 269 ++++++++++++++++++++++++++++++++++++ 1 file changed, 269 insertions(+)create mode 100644
arch/x86/kvm/vmx/tdx_arch.harch/x86/kvm/vmx/tdx_arch.h new file +40
-0@@ -0,0 +1,269 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include <linux/types.h> + +/* +* TDX SEAMCALL API function leaves +*/ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_RELOCATE 5 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_MEM_SEPT_RD 25 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_SEPT_REMOVE 30 +#define TDH_SYS_RD 34 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40
arch/x86/kvm/vmx/tdx_arch.h line 50
Quoted 10 lines
arch/x86/kvm/vmx/tdx_arch.h new file +10
-0+#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +#define TDG_VP_VMCALL_GET_TD_VM_CALL_INFO 0x10000 +#define TDG_VP_VMCALL_MAP_GPA 0x10001 +#define TDG_VP_VMCALL_GET_QUOTE 0x10002 +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003 +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004 +
these are definitions shared with TDX guest codes, and already defined in arch/x86/include/asm/shared/tdx.h. Please drop it.
Other than above,
PATCH v18 13/121 KVM: TDX: Add TDX "architectural" error codes
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed.
TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.
--- arch/x86/kvm/vmx/tdx_errno.h | 43 ++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_errno.h
arch/x86/kvm/vmx/tdx_errno.h new file +43 -0
@@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_OPERAND_INVALID 0xC000010000000000ULL +#define TDX_OPERAND_BUSY 0x8000020000000000ULL +#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL +#define TDX_FLUSHVP_NOT_DONE 0x8000082400000000ULL +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL +#define TDX_EPT_ENTRY_NOT_FREE 0xC0000B0200000000ULL +#define TDX_EPT_ENTRY_STATE_INCORRECT 0xC0000B0D00000000ULL + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_RETRY 0x0000000000000001ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001ULL + +/* +* TDX module operand ID, appears in 31:0 part of error code as +* detail information +*/ +#define TDX_OPERAND_ID_RCX 0x01 +#define TDX_OPERAND_ID_SEPT 0x92 +#define TDX_OPERAND_ID_TD_EPOCH 0xa9 + +#endif /* __KVM_X86_TDX_ERRNO_H */
Signature · 1 line
2.25.1
Re: [PATCH v18 013/121] KVM: TDX: Add TDX "architectural" error codes
Quoted 40 lines · On Mon, Jan 22, 2024 at 03:52:49PM -0800, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed.
TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>--- arch/x86/kvm/vmx/tdx_errno.h | 43 ++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_errno.harch/x86/kvm/vmx/tdx_errno.h new file +14
-0@@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_OPERAND_INVALID 0xC000010000000000ULL
arch/x86/kvm/vmx/tdx_errno.h line 24
Quoted 10 lines
arch/x86/kvm/vmx/tdx_errno.h new file +10
-0+#define TDX_OPERAND_BUSY 0x8000020000000000ULL +#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL +#define TDX_FLUSHVP_NOT_DONE 0x8000082400000000ULL +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL +#define TDX_EPT_ENTRY_NOT_FREE 0xC0000B0200000000ULL
Looks these 2 TDX_EPT_xx are not used, so can remove them.
Quoted 23 lines
arch/x86/kvm/vmx/tdx_errno.h new file +19
-1+#define TDX_EPT_ENTRY_STATE_INCORRECT 0xC0000B0D00000000ULL + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_RETRY 0x0000000000000001ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001ULL + +/* +* TDX module operand ID, appears in 31:0 part of error code as +* detail information +*/ +#define TDX_OPERAND_ID_RCX 0x01 +#define TDX_OPERAND_ID_SEPT 0x92 +#define TDX_OPERAND_ID_TD_EPOCH 0xa9 + +#endif /* __KVM_X86_TDX_ERRNO_H */ --2.25.1
Re: [PATCH v18 013/121] KVM: TDX: Add TDX "architectural" error codes
In reply to [email protected]Quoted 49 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Sean Christopherson <[email protected]>
Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed.
TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>--- arch/x86/kvm/vmx/tdx_errno.h | 43 ++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)create mode 100644
arch/x86/kvm/vmx/tdx_errno.harch/x86/kvm/vmx/tdx_errno.h new file +23
-0@@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* +* TDX SEAMCALL Status Codes (returned in RAX) +*/ +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_OPERAND_INVALID 0xC000010000000000ULL +#define TDX_OPERAND_BUSY 0x8000020000000000ULL +#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL +#define TDX_FLUSHVP_NOT_DONE 0x8000082400000000ULL +#define TDX_EPT_WALK_FAILED 0xC0000B0000000000ULL
arch/x86/kvm/vmx/tdx_errno.h line 33
Quoted 10 lines
arch/x86/kvm/vmx/tdx_errno.h new file +10
-0+#define TDX_EPT_ENTRY_NOT_FREE 0xC0000B0200000000ULL +#define TDX_EPT_ENTRY_STATE_INCORRECT 0xC0000B0D00000000ULL + +/* +* TDG.VP.VMCALL Status Codes (returned in R10) +*/ +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_RETRY 0x0000000000000001ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_TDREPORT_FAILED 0x8000000000000001ULL
Same to previous Patch:
These should be put in some shared header file, because they are shared with guest TD code.
Other than it,
Quoted 9 lines
arch/x86/kvm/vmx/tdx_errno.h new file +9
-0+/* +* TDX module operand ID, appears in 31:0 part of error code as +* detail information +*/ +#define TDX_OPERAND_ID_RCX 0x01 +#define TDX_OPERAND_ID_SEPT 0x92 +#define TDX_OPERAND_ID_TD_EPOCH 0xa9 + +#endif /* __KVM_X86_TDX_ERRNO_H */
PATCH v18 14/121 KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module
In reply to [email protected]From: Isaku Yamahata <[email protected]>
A VMM interacts with the TDX module using a new instruction (SEAMCALL). For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs.
Export __seamcall and define C wrapper functions for SEAMCALLs for readability.
Some SEAMCALL APIs donate host pages to TDX module or guest TD, and the donated pages are encrypted. Those require the VMM to flush the cache lines to avoid cache line alias.
---
Changes
v18:
- removed stub functions for __seamcall{,_ret}()
- Added Reviewed-by Binbin
- Make tdx_seamcall() use struct tdx_module_args instead of taking each inputs.
v15 -> v16:
- use struct tdx_module_args instead of struct tdx_module_output
- Add tdh_mem_sept_rd() for SEPT_VE_DISABLE=1.
--- arch/x86/kvm/vmx/tdx_ops.h | 360 +++++++++++++++++++++++++++++++++++++ 1 file changed, 360 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_ops.h
arch/x86/kvm/vmx/tdx_ops.h new file +360 -0
@@ -0,0 +1,360 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_OPS_H +#define __KVM_X86_TDX_OPS_H + +#include <linux/compiler.h> + +#include <asm/cacheflush.h> +#include <asm/asm.h> +#include <asm/kvm_host.h> + +#include "tdx_errno.h" +#include "tdx_arch.h" +#include "x86.h" + +static inline u64 tdx_seamcall (u64 op,struct tdx_module_args *in, +struct tdx_module_args *out) +{ +u64 ret; + +if (out) { + *out = *in; + ret =__seamcall_ret (op, out); + }else + ret =__seamcall (op, in); + +if (unlikely (ret ==TDX_SEAMCALL_UD )) { +/* +* SEAMCALLs fail with TDX_SEAMCALL_UD returned when VMX is off. +* This can happen when the host gets rebooted or live +* updated. In this case, the instruction execution is ignored +* as KVM is shut down, so the error code is suppressed. Other +* than this, the error is unexpected and the execution can't +* continue as the TDX features reply on VMX to be on. +*/ +kvm_spurious_fault (); +return 0 ; + } +return ret; +} + +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +struct tdx_module_args in = { + .rcx = addr, + .rdx = tdr, + }; + +clflush_cache_range (__va (addr),PAGE_SIZE ); +return tdx_seamcall (TDH_MNG_ADDCX , &in,NULL ); +} + +static inline u64 tdh_mem_page_add (hpa_t tdr,gpa_t gpa,hpa_t hpa,hpa_t source, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + .r9 = source, + }; + +clflush_cache_range (__va (hpa),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_PAGE_ADD , &in, out); +} + +static inline u64 tdh_mem_sept_add (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + .r8 = page, + }; + +clflush_cache_range (__va (page),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_SEPT_ADD , &in, out); +} + +static inline u64 tdh_mem_sept_rd (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_SEPT_RD , &in, out); +} + +static inline u64 tdh_mem_sept_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_SEPT_REMOVE , &in, out); +} + +static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) +{ +struct tdx_module_args in = { + .rcx = addr, + .rdx = tdvpr, + }; + +clflush_cache_range (__va (addr),PAGE_SIZE ); +return tdx_seamcall (TDH_VP_ADDCX , &in,NULL ); +} + +static inline u64 tdh_mem_page_relocate (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + }; + +clflush_cache_range (__va (hpa),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_PAGE_RELOCATE , &in, out); +} + +static inline u64 tdh_mem_page_aug (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + }; + +clflush_cache_range (__va (hpa),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_PAGE_AUG , &in, out); +} + +static inline u64 tdh_mem_range_block (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_RANGE_BLOCK , &in, out); +} + +static inline u64 tdh_mng_key_config (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_KEY_CONFIG , &in,NULL ); +} + +static inline u64 tdh_mng_create (hpa_t tdr,int hkid) +{ +struct tdx_module_args in = { + .rcx = tdr, + .rdx = hkid, + }; + +clflush_cache_range (__va (tdr),PAGE_SIZE ); +return tdx_seamcall (TDH_MNG_CREATE , &in,NULL ); +} + +static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = tdr, + }; + +clflush_cache_range (__va (tdvpr),PAGE_SIZE ); +return tdx_seamcall (TDH_VP_CREATE , &in,NULL ); +} + +static inline u64 tdh_mng_rd (hpa_t tdr,u64 field,struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdr, + .rdx = field, + }; + +return tdx_seamcall (TDH_MNG_RD , &in, out); +} + +static inline u64 tdh_mr_extend (hpa_t tdr,gpa_t gpa, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MR_EXTEND , &in, out); +} + +static inline u64 tdh_mr_finalize (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MR_FINALIZE , &in,NULL ); +} + +static inline u64 tdh_vp_flush (hpa_t tdvpr) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + }; + +return tdx_seamcall (TDH_VP_FLUSH , &in,NULL ); +} + +static inline u64 tdh_mng_vpflushdone (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_VPFLUSHDONE , &in,NULL ); +} + +static inline u64 tdh_mng_key_freeid (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_KEY_FREEID , &in,NULL ); +} + +static inline u64 tdh_mng_init (hpa_t tdr,hpa_t td_params, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdr, + .rdx = td_params, + }; + +return tdx_seamcall (TDH_MNG_INIT , &in, out); +} + +static inline u64 tdh_vp_init (hpa_t tdvpr,u64 rcx) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = rcx, + }; + +return tdx_seamcall (TDH_VP_INIT , &in,NULL ); +} + +static inline u64 tdh_vp_rd (hpa_t tdvpr,u64 field, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = field, + }; + +return tdx_seamcall (TDH_VP_RD , &in, out); +} + +static inline u64 tdh_mng_key_reclaimid (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_KEY_RECLAIMID , &in,NULL ); +} + +static inline u64 tdh_phymem_page_reclaim (hpa_t page, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = page, + }; + +return tdx_seamcall (TDH_PHYMEM_PAGE_RECLAIM , &in, out); +} + +static inline u64 tdh_mem_page_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_PAGE_REMOVE , &in, out); +} + +static inline u64 tdh_sys_lp_shutdown (void ) +{ +struct tdx_module_args in = { + }; + +return tdx_seamcall (TDH_SYS_LP_SHUTDOWN , &in,NULL ); +} + +static inline u64 tdh_mem_track (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MEM_TRACK , &in,NULL ); +} + +static inline u64 tdh_mem_range_unblock (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_RANGE_UNBLOCK , &in, out); +} + +static inline u64 tdh_phymem_cache_wb (bool resume) +{ +struct tdx_module_args in = { + .rcx = resume ?1 :0 , + }; + +return tdx_seamcall (TDH_PHYMEM_CACHE_WB , &in,NULL ); +} + +static inline u64 tdh_phymem_page_wbinvd (hpa_t page) +{ +struct tdx_module_args in = { + .rcx = page, + }; + +return tdx_seamcall (TDH_PHYMEM_PAGE_WBINVD , &in,NULL ); +} + +static inline u64 tdh_vp_wr (hpa_t tdvpr,u64 field,u64 val,u64 mask, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = field, + .r8 = val, + .r9 = mask, + }; + +return tdx_seamcall (TDH_VP_WR , &in, out); +} + +#endif /* __KVM_X86_TDX_OPS_H */
Signature · 1 line
2.25.1
Re: [PATCH v18 014/121] KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module
Quoted 8 lines · On Mon, Jan 22, 2024 at 03:52:50PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
A VMM interacts with the TDX module using a new instruction (SEAMCALL). For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs.
Export
__seamcalland define C wrapper functions for SEAMCALLs for
^^^^^^^^^^^^^^^^^
It's not exported by this patch.
Others LGTM.
Quoted 396 lines
readability.
Some SEAMCALL APIs donate host pages to TDX module or guest TD, and the donated pages are encrypted. Those require the VMM to flush the cache lines to avoid cache line alias.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Binbin Wu <[email protected]>---
Changes
v18:
- removed stub functions for __seamcall{,_ret}()
- Added Reviewed-by Binbin
- Maketdx_seamcall()use structtdx_module_argsinstead of taking each inputs.v15 -> v16:
- use structtdx_module_argsinstead of structtdx_module_output
- Addtdh_mem_sept_rd()for SEPT_VE_DISABLE=1.--- arch/x86/kvm/vmx/tdx_ops.h | 360 +++++++++++++++++++++++++++++++++++++ 1 file changed, 360 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_ops.harch/x86/kvm/vmx/tdx_ops.h new file +360
-1@@ -0,0 +1,360 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_OPS_H +#define __KVM_X86_TDX_OPS_H + +#include <linux/compiler.h> + +#include <asm/cacheflush.h> +#include <asm/asm.h> +#include <asm/kvm_host.h> + +#include "tdx_errno.h" +#include "tdx_arch.h" +#include "x86.h" + +static inline u64 tdx_seamcall (u64 op,struct tdx_module_args *in, +struct tdx_module_args *out) +{ +u64 ret; + +if (out) { + *out = *in; + ret =__seamcall_ret (op, out); + }else + ret =__seamcall (op, in); + +if (unlikely (ret ==TDX_SEAMCALL_UD )) { +/* +* SEAMCALLs fail with TDX_SEAMCALL_UD returned when VMX is off. +* This can happen when the host gets rebooted or live +* updated. In this case, the instruction execution is ignored +* as KVM is shut down, so the error code is suppressed. Other +* than this, the error is unexpected and the execution can't +* continue as the TDX features reply on VMX to be on. +*/ +kvm_spurious_fault (); +return 0 ; + } +return ret; +} + +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) +{ +struct tdx_module_args in = { + .rcx = addr, + .rdx = tdr, + }; + +clflush_cache_range (__va (addr),PAGE_SIZE ); +return tdx_seamcall (TDH_MNG_ADDCX , &in,NULL ); +} + +static inline u64 tdh_mem_page_add (hpa_t tdr,gpa_t gpa,hpa_t hpa,hpa_t source, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + .r9 = source, + }; + +clflush_cache_range (__va (hpa),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_PAGE_ADD , &in, out); +} + +static inline u64 tdh_mem_sept_add (hpa_t tdr,gpa_t gpa,int level,hpa_t page, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + .r8 = page, + }; + +clflush_cache_range (__va (page),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_SEPT_ADD , &in, out); +} + +static inline u64 tdh_mem_sept_rd (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_SEPT_RD , &in, out); +} + +static inline u64 tdh_mem_sept_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_SEPT_REMOVE , &in, out); +} + +static inline u64 tdh_vp_addcx (hpa_t tdvpr,hpa_t addr) +{ +struct tdx_module_args in = { + .rcx = addr, + .rdx = tdvpr, + }; + +clflush_cache_range (__va (addr),PAGE_SIZE ); +return tdx_seamcall (TDH_VP_ADDCX , &in,NULL ); +} + +static inline u64 tdh_mem_page_relocate (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + }; + +clflush_cache_range (__va (hpa),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_PAGE_RELOCATE , &in, out); +} + +static inline u64 tdh_mem_page_aug (hpa_t tdr,gpa_t gpa,hpa_t hpa, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + }; + +clflush_cache_range (__va (hpa),PAGE_SIZE ); +return tdx_seamcall (TDH_MEM_PAGE_AUG , &in, out); +} + +static inline u64 tdh_mem_range_block (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_RANGE_BLOCK , &in, out); +} + +static inline u64 tdh_mng_key_config (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_KEY_CONFIG , &in,NULL ); +} + +static inline u64 tdh_mng_create (hpa_t tdr,int hkid) +{ +struct tdx_module_args in = { + .rcx = tdr, + .rdx = hkid, + }; + +clflush_cache_range (__va (tdr),PAGE_SIZE ); +return tdx_seamcall (TDH_MNG_CREATE , &in,NULL ); +} + +static inline u64 tdh_vp_create (hpa_t tdr,hpa_t tdvpr) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = tdr, + }; + +clflush_cache_range (__va (tdvpr),PAGE_SIZE ); +return tdx_seamcall (TDH_VP_CREATE , &in,NULL ); +} + +static inline u64 tdh_mng_rd (hpa_t tdr,u64 field,struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdr, + .rdx = field, + }; + +return tdx_seamcall (TDH_MNG_RD , &in, out); +} + +static inline u64 tdh_mr_extend (hpa_t tdr,gpa_t gpa, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MR_EXTEND , &in, out); +} + +static inline u64 tdh_mr_finalize (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MR_FINALIZE , &in,NULL ); +} + +static inline u64 tdh_vp_flush (hpa_t tdvpr) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + }; + +return tdx_seamcall (TDH_VP_FLUSH , &in,NULL ); +} + +static inline u64 tdh_mng_vpflushdone (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_VPFLUSHDONE , &in,NULL ); +} + +static inline u64 tdh_mng_key_freeid (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_KEY_FREEID , &in,NULL ); +} + +static inline u64 tdh_mng_init (hpa_t tdr,hpa_t td_params, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdr, + .rdx = td_params, + }; + +return tdx_seamcall (TDH_MNG_INIT , &in, out); +} + +static inline u64 tdh_vp_init (hpa_t tdvpr,u64 rcx) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = rcx, + }; + +return tdx_seamcall (TDH_VP_INIT , &in,NULL ); +} + +static inline u64 tdh_vp_rd (hpa_t tdvpr,u64 field, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = field, + }; + +return tdx_seamcall (TDH_VP_RD , &in, out); +} + +static inline u64 tdh_mng_key_reclaimid (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MNG_KEY_RECLAIMID , &in,NULL ); +} + +static inline u64 tdh_phymem_page_reclaim (hpa_t page, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = page, + }; + +return tdx_seamcall (TDH_PHYMEM_PAGE_RECLAIM , &in, out); +} + +static inline u64 tdh_mem_page_remove (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_PAGE_REMOVE , &in, out); +} + +static inline u64 tdh_sys_lp_shutdown (void ) +{ +struct tdx_module_args in = { + }; + +return tdx_seamcall (TDH_SYS_LP_SHUTDOWN , &in,NULL ); +} + +static inline u64 tdh_mem_track (hpa_t tdr) +{ +struct tdx_module_args in = { + .rcx = tdr, + }; + +return tdx_seamcall (TDH_MEM_TRACK , &in,NULL ); +} + +static inline u64 tdh_mem_range_unblock (hpa_t tdr,gpa_t gpa,int level, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + +return tdx_seamcall (TDH_MEM_RANGE_UNBLOCK , &in, out); +} + +static inline u64 tdh_phymem_cache_wb (bool resume) +{ +struct tdx_module_args in = { + .rcx = resume ?1 :0 , + }; + +return tdx_seamcall (TDH_PHYMEM_CACHE_WB , &in,NULL ); +} + +static inline u64 tdh_phymem_page_wbinvd (hpa_t page) +{ +struct tdx_module_args in = { + .rcx = page, + }; + +return tdx_seamcall (TDH_PHYMEM_PAGE_WBINVD , &in,NULL ); +} + +static inline u64 tdh_vp_wr (hpa_t tdvpr,u64 field,u64 val,u64 mask, +struct tdx_module_args *out) +{ +struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = field, + .r8 = val, + .r9 = mask, + }; + +return tdx_seamcall (TDH_VP_WR , &in, out); +} + +#endif /* __KVM_X86_TDX_OPS_H */ --2.25.1
PATCH v18 15/121 KVM: TDX: Retry SEAMCALL on the lack of entropy error
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Some SEAMCALL may return TDX_RND_NO_ENTROPY error when the entropy is lacking. Retry SEAMCALL on the error following rdrand_long() to retry RDRAND_RETRY_LOOPS times.
---
v18:
- update to use struct tdx_modules_args for inputs.
--- arch/x86/kvm/vmx/tdx_errno.h | 1 + arch/x86/kvm/vmx/tdx_ops.h | 15 +++++++++++---- 2 files changed, 12 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/tdx_errno.h +1 -0
@@ -14,6 +14,7 @@#define TDX_OPERAND_INVALID 0xC000010000000000ULL#define TDX_OPERAND_BUSY 0x8000020000000000ULL#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL
arch/x86/kvm/vmx/tdx_ops.h +11 -4
@@ -6,6 +6,7 @@#include <linux/compiler.h> +#include <asm/archrandom.h> #include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h>
@@ -17,14 +18,20 @@static inline u64 tdx_seamcall (u64 op,struct tdx_module_args *in,struct tdx_module_args *out) { +struct tdx_module_args args; +int retry;u64 ret; -if (out) { +if (!out) + out = &args; + +/* Mimic the existing rdrand_long() to retry RDRAND_RETRY_LOOPS times. */ + retry =RDRAND_RETRY_LOOPS ; +do { +/* As __seamcall_ret() overwrites out, init out on each loop. */ *out = *in; ret =__seamcall_ret (op, out); - }else - ret =__seamcall (op, in); - + }while (unlikely (ret ==TDX_RND_NO_ENTROPY ) && --retry);if (unlikely (ret ==TDX_SEAMCALL_UD )) {/* * SEAMCALLs fail with TDX_SEAMCALL_UD returned when VMX is off.
Signature · 1 line
2.25.1
Re: [PATCH v18 015/121] KVM: TDX: Retry SEAMCALL on the lack of entropy error
Quoted 54 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Some SEAMCALL may return
TDX_RND_NO_ENTROPYerror when the entropy is lacking. Retry SEAMCALL on the error followingrdrand_long()to retryRDRAND_RETRY_LOOPStimes.Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- update to use structtdx_modules_argsfor inputs.--- arch/x86/kvm/vmx/tdx_errno.h | 1 + arch/x86/kvm/vmx/tdx_ops.h | 15 +++++++++++---- 2 files changed, 12 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/tdx_errno.h +1
-0@@ -14,6 +14,7 @@#define TDX_OPERAND_INVALID 0xC000010000000000ULL#define TDX_OPERAND_BUSY 0x8000020000000000ULL#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULLarch/x86/kvm/vmx/tdx_ops.h +9
-1@@ -6,6 +6,7 @@#include <linux/compiler.h> +#include <asm/archrandom.h> #include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h> @@ -17,14 +18,20 @@static inline u64 tdx_seamcall (u64 op,struct tdx_module_args *in,struct tdx_module_args *out) { +struct tdx_module_args args; +int retry;u64 ret; -if (out) { +if (!out) + out = &args; + +/* Mimic the existing rdrand_long() to retry RDRAND_RETRY_LOOPS times. */ + retry =RDRAND_RETRY_LOOPS ; + do {
arch/x86/kvm/vmx/tdx_ops.h line 37
Quoted 10 lines
arch/x86/kvm/vmx/tdx_ops.h +2
-3+/* As __seamcall_ret() overwrites out, init out on each loop. */ *out = *in; ret =__seamcall_ret (op, out); - }else - ret =__seamcall (op, in); - + }while (unlikely (ret ==TDX_RND_NO_ENTROPY ) && --retry);if (unlikely (ret ==TDX_SEAMCALL_UD )) {/* * SEAMCALLs fail with TDX_SEAMCALL_UD returned when VMX is off.
One question is when the return code is TDX_RND_NO_ENTROPY, does the input tdx_module_args value will be modified or not?
As mentioned by Kai in previous review, there are seamcall*() variants which handle TDX_RND_NO_ENTROPY already.
If the input tdx_module_args is not modified by a seamcall when the return code is TDX_RND_NO_ENTROPY, the seamcall_ret() can be used directly.
But if the input tdx_module_args could be modified by a seamcall when the return code is TDX_RND_NO_ENTROPY, the implementations of seamcall_ret() and seamcall_saved_ret() have problem then, because the input tdx_module_args is not re-initialized when retry.
They need to be fixed or deleted (no one is using them currently).
Paolo Bonzini
maintainerRe: [PATCH v18 015/121] KVM: TDX: Retry SEAMCALL on the lack of entropy error
In reply to [email protected]Quoted 8 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Isaku Yamahata <[email protected]>
Some SEAMCALL may return
TDX_RND_NO_ENTROPYerror when the entropy is lacking. Retry SEAMCALL on the error followingrdrand_long()to retryRDRAND_RETRY_LOOPStimes.Signed-off-by Isaku Yamahata <[email protected]>
This patch should not be needed anymore, just...
arch/x86/kvm/vmx/tdx_ops.h line 26
Quoted 8 lines
arch/x86/kvm/vmx/tdx_ops.h +4
-2+/* Mimic the existing rdrand_long() to retry RDRAND_RETRY_LOOPS times. */ + retry =RDRAND_RETRY_LOOPS ; + do { + /* As __seamcall_ret() overwrites out, init out on each loop. */ *out = *in; ret = __seamcall_ret(op, out); - }else - ret =__seamcall (op, in);
... use seamcall() and seamcall_ret() here instead of the "__" versions.
Paolo
PATCH v18 16/121 KVM: TDX: Add helper functions to print TDX SEAMCALL error
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add helper functions to print out errors from the TDX module in a uniform manner.
---
v18:
- Added Reviewed-by Binbin.
--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/tdx_error.c | 21 +++++++++++++++++++++ arch/x86/kvm/vmx/tdx_ops.h | 5 +++++ 3 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx_error.c
arch/x86/kvm/Makefile +1 -1
@@ -24,7 +24,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ svm/sev.o
arch/x86/kvm/vmx/tdx_error.c new file +21 -0
@@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +/* functions to record TDX SEAMCALL error */ + +#include <linux/kernel.h> +#include <linux/bug.h> + +#include "tdx_ops.h" + +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_args *out) +{ +if (!out) { +pr_err_ratelimited ("SEAMCALL (0x%016llx) failed: 0x%016llx\n" , + op, error_code); +return ; + } + +#define MSG \ + "SEAMCALL (0x%016llx) failed: 0x%016llx RCX 0x%016llx RDX 0x%016llx R8 0x%016llx R9 0x%016llx R10 0x%016llx R11 0x%016llx\n" +pr_err_ratelimited (MSG , op, error_code, out->rcx, out->rdx, out->r8, + out->r9, out->r10, out->r11); +}
arch/x86/kvm/vmx/tdx_ops.h +5 -0
@@ -10,6 +10,7 @@#include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h> +#include <asm/tdx.h> #include "tdx_errno.h" #include "tdx_arch.h"
@@ -47,6 +48,10 @@ static inline u64 tdx_seamcall(u64 op, struct tdx_module_args *in,return ret; } +#ifdef CONFIG_INTEL_TDX_HOST +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_args *out); +#endif +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) {struct tdx_module_args in = {
Signature · 1 line
2.25.1
Re: [PATCH v18 016/121] KVM: TDX: Add helper functions to print TDX SEAMCALL error
Quoted 62 lines · On Mon, Jan 22, 2024 at 03:52:52PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add helper functions to print out errors from the TDX module in a uniform manner.
Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Binbin Wu <[email protected]>---
v18:
- Added Reviewed-by Binbin.--- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/tdx_error.c | 21 +++++++++++++++++++++ arch/x86/kvm/vmx/tdx_ops.h | 5 +++++ 3 files changed, 27 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx_error.carch/x86/kvm/Makefile +1
-1@@ -24,7 +24,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ svm/sev.oarch/x86/kvm/vmx/tdx_error.c new file +21
-0@@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +/* functions to record TDX SEAMCALL error */ + +#include <linux/kernel.h> +#include <linux/bug.h> + +#include "tdx_ops.h" + +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_args *out) +{ +if (!out) { +pr_err_ratelimited ("SEAMCALL (0x%016llx) failed: 0x%016llx\n" , + op, error_code); +return ; + } + +#define MSG \ + "SEAMCALL (0x%016llx) failed: 0x%016llx RCX 0x%016llx RDX 0x%016llx R8 0x%016llx R9 0x%016llx R10 0x%016llx R11 0x%016llx\n" +pr_err_ratelimited (MSG , op, error_code, out->rcx, out->rdx, out->r8, + out->r9, out->r10, out->r11); +}arch/x86/kvm/vmx/tdx_ops.h +0
-0
arch/x86/kvm/vmx/tdx_ops.h line 13
Quoted 4 lines
arch/x86/kvm/vmx/tdx_ops.h +1
-0@@ -10,6 +10,7 @@#include <asm/cacheflush.h> #include <asm/asm.h> #include <asm/kvm_host.h> +#include <asm/tdx.h>
Just not sure the reason of this #include here, compile fine w/o it.
Quoted 18 lines
arch/x86/kvm/vmx/tdx_ops.h +4
-1#include "tdx_errno.h" #include "tdx_arch.h" @@ -47,6 +48,10 @@ static inline u64 tdx_seamcall(u64 op, struct tdx_module_args *in,return ret; } +#ifdef CONFIG_INTEL_TDX_HOST +void pr_tdx_error (u64 op,u64 error_code,const struct tdx_module_args *out); +#endif +static inline u64 tdh_mng_addcx (hpa_t tdr,hpa_t addr) {struct tdx_module_args in = { --2.25.1
PATCH v18 17/121 The start of TDX KVM patch series: TD VM creation/destruction
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of TD VM creation/destruction.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2
@@ -16,8 +16,8 @@ Patch Layer status Patch layer Status * TDX, VMX coexistence: Applied -* TDX architectural definitions: Applying -* TD VM creation/destruction: Not yet +* TDX architectural definitions: Applied +* TD VM creation/destruction: Applying * TD vcpu creation/destruction: Not yet * TDX EPT violation: Not yet * TD finalization: Not yet
Signature · 1 line
2.25.1
PATCH v18 18/121 KVM: TDX: Add helper functions to allocate/free TDX private host key id
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Add helper functions to allocate/free TDX private host key id (HKID).
The memory controller encrypts TDX memory with the assigned TDX HKIDs. The global TDX HKID is to encrypt the TDX module, its memory, and some dynamic data (TDR). The private TDX HKID is assigned to guest TD to encrypt guest memory and the related data. When VMM releases an encrypted page for reuse, the page needs a cache flush with the used HKID. VMM needs the global TDX HKID and the private TDX HKIDs to flush encrypted pages.
---
v18:
- Moved the functions to kvm tdx from arch/x86/virt/vmx/tdx/
- Drop exporting symbols as the host tdx does.
--- arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)
arch/x86/kvm/vmx/tdx.c +29 -0
@@ -11,6 +11,35 @@#undef pr_fmt#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt +/* +* Key id globally used by TDX module: TDX module maps TDR with this TDX global +* key id. TDR includes key id assigned to the TD. Then TDX module maps other +* TD-related pages with the assigned key id. TDR requires this TDX global key +* id for cache flush unlike other TD-related pages. +*/ +/* TDX KeyID pool */ +static DEFINE_IDA (tdx_guest_keyid_pool); + +static int __used tdx_guest_keyid_alloc (void ) +{ +if (WARN_ON_ONCE (!tdx_guest_keyid_start || !tdx_nr_guest_keyids)) +return -EINVAL ; + +/* The first keyID is reserved for the global key. */ +return ida_alloc_range (&tdx_guest_keyid_pool, tdx_guest_keyid_start, + tdx_guest_keyid_start + tdx_nr_guest_keyids -1 , +GFP_KERNEL ); +} + +static void __used tdx_guest_keyid_free (int keyid) +{ +/* keyid = 0 is reserved. */ +if (WARN_ON_ONCE (keyid <=0 )) +return ; + +ida_free (&tdx_guest_keyid_pool, keyid); +} +static int __init tdx_module_setup (void ) {int ret;
Signature · 1 line
2.25.1
Re: [PATCH v18 018/121] KVM: TDX: Add helper functions to allocate/free TDX private host key id
Quoted 33 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Add helper functions to allocate/free TDX private host key id (HKID).
The memory controller encrypts TDX memory with the assigned TDX HKIDs. The global TDX HKID is to encrypt the TDX module, its memory, and some dynamic data (TDR). The private TDX HKID is assigned to guest TD to encrypt guest memory and the related data. When VMM releases an encrypted page for reuse, the page needs a cache flush with the used HKID. VMM needs the global TDX HKID and the private TDX HKIDs to flush encrypted pages.
Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- Moved the functions to kvm tdx fromarch/x86/virt/vmx/tdx/
- Drop exporting symbols as the host tdx does.--- arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+)arch/x86/kvm/vmx/tdx.c +5
-0@@ -11,6 +11,35 @@#undef pr_fmt#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt +/* +* Key id globally used by TDX module: TDX module maps TDR with this TDX global +* key id. TDR includes key id assigned to the TD. Then TDX module maps other +* TD-related pages with the assigned key id. TDR requires this TDX global key +* id for cache flush unlike other TD-related pages.
arch/x86/kvm/vmx/tdx.c line 28
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+*/ +/* TDX KeyID pool */ +static DEFINE_IDA (tdx_guest_keyid_pool); + +static int __used tdx_guest_keyid_alloc (void ) +{ +if (WARN_ON_ONCE (!tdx_guest_keyid_start || !tdx_nr_guest_keyids)) +return -EINVAL ; + +/* The first keyID is reserved for the global key. */
Seems no need to add the comment here any more.
arch/x86/kvm/vmx/tdx.c line 37
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9
-0+return ida_alloc_range (&tdx_guest_keyid_pool, tdx_guest_keyid_start, + tdx_guest_keyid_start + tdx_nr_guest_keyids -1 , +GFP_KERNEL ); +} + +static void __used tdx_guest_keyid_free (int keyid) +{ +/* keyid = 0 is reserved. */ +if (WARN_ON_ONCE (keyid <=0 ))
Why not use tdx_guest_keyid_start and its range directly for the check?
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c +5
-0+return ; + +ida_free (&tdx_guest_keyid_pool, keyid); +} +static int __init tdx_module_setup (void ) {int ret;
PATCH v18 19/121 KVM: TDX: Add helper function to read TDX metadata in array
In reply to [email protected]From: Isaku Yamahata <[email protected]>
To read meta data in series, use table.
Instead of metadata_read(fid0, &data0); metadata_read(...); ... table = { {fid0, &data0}, ...}; metadata-read(tables).
TODO: Once the TDX host code introduces its framework to read TDX metadata, drop this patch and convert the code that uses this.
---
v18:
- newly added
--- arch/x86/kvm/vmx/tdx.c | 45 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+)
arch/x86/kvm/vmx/tdx.c +45 -0
@@ -6,6 +6,7 @@#include "capabilities.h" #include "x86_ops.h" #include "x86.h" +#include "tdx_arch.h" #include "tdx.h" #undef pr_fmt
@@ -40,6 +41,50 @@ static void __used tdx_guest_keyid_free(int keyid)ida_free (&tdx_guest_keyid_pool, keyid); } +#define TDX_MD_MAP (_fid, _ptr) \ + { .fid = MD_FIELD_ID_##_fid, \ + .ptr = (_ptr), } + +struct tdx_md_map { +u64 fid; +void *ptr; +}; + +static size_t tdx_md_element_size (u64 fid) +{ +switch (TDX_MD_ELEMENT_SIZE_CODE (fid)) { +case TDX_MD_ELEMENT_SIZE_8BITS : +return 1 ; +case TDX_MD_ELEMENT_SIZE_16BITS : +return 2 ; +case TDX_MD_ELEMENT_SIZE_32BITS : +return 4 ; +case TDX_MD_ELEMENT_SIZE_64BITS : +return 8 ; +default : +WARN_ON_ONCE (1 ); +return 0 ; + } +} + +int tdx_md_read (struct tdx_md_map *maps,int nr_maps) +{ +struct tdx_md_map *m; +int ret, i; +u64 tmp; + +for (i =0 ; i < nr_maps; i++) { + m = &maps[i]; + ret =tdx_sys_metadata_field_read (m->fid, &tmp); +if (ret) +return ret; + +memcpy (m->ptr, &tmp,tdx_md_element_size (m->fid)); + } + +return 0 ; +} +static int __init tdx_module_setup (void ) {int ret;
Signature · 1 line
2.25.1
PATCH v18 20/121 x86/virt/tdx: Get system-wide info about TDX module on initialization
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX KVM needs system-wide information about the TDX module, store it in struct tdx_info.
TODO: Once TDX host patch series introduces a framework to read TDX meta data, convert the code to it.
---
Change v18:
- Newly Added
--- arch/x86/include/uapi/asm/kvm.h | 11 +++++ arch/x86/kvm/vmx/main.c | 9 +++- arch/x86/kvm/vmx/tdx.c | 79 ++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/x86_ops.h | 2 + 4 files changed, 99 insertions(+), 2 deletions(-)
arch/x86/include/uapi/asm/kvm.h +11 -0
@@ -567,4 +567,15 @@ struct kvm_pmu_event_filter {#define KVM_X86_TDX_VM 2#define KVM_X86_SNP_VM 3 +#define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1) + +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/vmx/main.c +8 -1
@@ -47,6 +47,13 @@ static __init int vt_hardware_setup(void)return 0 ; } +static void vt_hardware_unsetup (void ) +{ +if (enable_tdx) +tdx_hardware_unsetup (); +vmx_hardware_unsetup (); +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))
@@ -69,7 +76,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_processor_compatibility = vmx_check_processor_compat, - .hardware_unsetup = vmx_hardware_unsetup, + .hardware_unsetup = vt_hardware_unsetup, .hardware_enable = vt_hardware_enable, .hardware_disable = vmx_hardware_disable,
arch/x86/kvm/vmx/tdx.c +78 -1
104 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +2 -0
@@ -136,9 +136,11 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; } +static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }#endif
Signature · 1 line
2.25.1
Re: [PATCH v18 020/121] x86/virt/tdx: Get system-wide info about TDX module on initialization
Quoted 127 lines · On Mon, Jan 22, 2024 at 03:52:56PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX KVM needs system-wide information about the TDX module, store it in struct
tdx_info.TODO: Once TDX host patch series introduces a framework to read TDX meta data, convert the code to it.
Signed-off-by Isaku Yamahata <[email protected]>---
Change v18:
- Newly Added--- arch/x86/include/uapi/asm/kvm.h | 11 +++++ arch/x86/kvm/vmx/main.c | 9 +++- arch/x86/kvm/vmx/tdx.c | 79 ++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/x86_ops.h | 2 + 4 files changed, 99 insertions(+), 2 deletions(-)arch/x86/include/uapi/asm/kvm.h +11
-0@@ -567,4 +567,15 @@ struct kvm_pmu_event_filter {#define KVM_X86_TDX_VM 2#define KVM_X86_SNP_VM 3 +#define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1) + +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; +#endif /* _ASM_X86_KVM_H */ arch/x86/kvm/vmx/main.c +8
-1@@ -47,6 +47,13 @@ static __init int vt_hardware_setup(void)return 0 ; } +static void vt_hardware_unsetup (void ) +{ +if (enable_tdx) +tdx_hardware_unsetup (); +vmx_hardware_unsetup (); +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))@@ -69,7 +76,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_processor_compatibility = vmx_check_processor_compat, - .hardware_unsetup = vmx_hardware_unsetup, + .hardware_unsetup = vt_hardware_unsetup, .hardware_enable = vt_hardware_enable, .hardware_disable = vmx_hardware_disable,arch/x86/kvm/vmx/tdx.c +35
-1@@ -67,7 +67,7 @@ static size_t tdx_md_element_size(u64 fid) } } -int tdx_md_read (struct tdx_md_map *maps,int nr_maps) +static int tdx_md_read (struct tdx_md_map *maps,int nr_maps) {struct tdx_md_map *m;int ret, i;@@ -85,9 +85,39 @@ int tdx_md_read(struct tdx_md_map *maps, int nr_maps)return 0 ; } +struct tdx_info { +u64 attributes_fixed0; +u64 attributes_fixed1; +u64 xfam_fixed0; +u64 xfam_fixed1; + +u16 num_cpuid_config; +/* This must the last member. */ + DECLARE_FLEX_ARRAY(struct kvm_tdx_cpuid_config, cpuid_configs); +}; + +/* Info about the TDX module. */ +static struct tdx_info *tdx_info; +static int __init tdx_module_setup (void ) { +u16 num_cpuid_config;int ret; +u32 i; + +struct tdx_md_map mds[] = { +TDX_MD_MAP (NUM_CPUID_CONFIG , &num_cpuid_config), + }; + +#define TDX_INFO_MAP (_field_id, _member) \ + TD_SYSINFO_MAP(_field_id, struct tdx_info, _member) + +struct tdx_metadata_field_mapping tdx_info_md[] = { +TDX_INFO_MAP (ATTRS_FIXED0 , attributes_fixed0), +TDX_INFO_MAP (ATTRS_FIXED1 , attributes_fixed1), +TDX_INFO_MAP (XFAM_FIXED0 , xfam_fixed0), +TDX_INFO_MAP (XFAM_FIXED1 , xfam_fixed1), + }; ret =tdx_enable ();if (ret) {@@ -95,7 +125,49 @@ static int __init tdx_module_setup(void)return ret; } + ret =tdx_md_read (mds,ARRAY_SIZE (mds)); +if (ret) +return ret; +
arch/x86/kvm/vmx/tdx.c static int __init tdx_module_setup(void) · line 141
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ tdx_info =kzalloc (sizeof (*tdx_info) + +sizeof (*tdx_info->cpuid_configs) * num_cpuid_config, +GFP_KERNEL ); +if (!tdx_info) +return -ENOMEM ; + tdx_info->num_cpuid_config = num_cpuid_config; + + ret =tdx_sys_metadata_read (tdx_info_md,ARRAY_SIZE (tdx_info_md), tdx_info); +if (ret) +return ret;
"goto error_sys_rd" as below to free the tdx_info ?
Quoted 61 lines
arch/x86/kvm/vmx/tdx.c +33
-0+ +for (i =0 ; i < num_cpuid_config; i++) { +struct kvm_tdx_cpuid_config *c = &tdx_info->cpuid_configs[i]; +u64 leaf, eax_ebx, ecx_edx; +struct tdx_md_map cpuids[] = { +TDX_MD_MAP (CPUID_CONFIG_LEAVES + i, &leaf), +TDX_MD_MAP (CPUID_CONFIG_VALUES + i *2 , &eax_ebx), +TDX_MD_MAP (CPUID_CONFIG_VALUES + i *2 +1 , &ecx_edx), + }; + + ret =tdx_md_read (cpuids,ARRAY_SIZE (cpuids)); +if (ret) + goto error_sys_rd; + + c->leaf = (u32 )leaf; + c->sub_leaf = leaf >>32 ; + c->eax = (u32 )eax_ebx; + c->ebx = eax_ebx >>32 ; + c->ecx = (u32 )ecx_edx; + c->edx = ecx_edx >>32 ; + } +return 0 ; + +error_sys_rd: + ret = -EIO ; +/* kfree() accepts NULL. */ +kfree (tdx_info); +return ret; }bool tdx_is_vm_type_supported (unsigned long type)@@ -163,3 +235,8 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) out:return r; } + +void tdx_hardware_unsetup (void ) +{ +kfree (tdx_info); +}arch/x86/kvm/vmx/x86_ops.h +2
-1@@ -136,9 +136,11 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);#ifdef CONFIG_INTEL_TDX_HOST int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops); +void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; } +static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; }#endif --2.25.1
PATCH v18 21/121 KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl
In reply to [email protected]From: Isaku Yamahata <[email protected]>
KVM_MEMORY_ENCRYPT_OP was introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations under KVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurpose KVM_MEMORY_ENCRYPT_OP for TDX specific operations and define subcommands.
TDX requires VM-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization.
Add a place holder function for TDX specific VM-scoped ioctl as mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters. Make mem_enc_ioctl non-optional as it's always filled.
---
v15:
- change struct kvm_tdx_cmd to drop unused member.
--- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/uapi/asm/kvm.h | 26 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/main.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.c | 26 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ arch/x86/kvm/x86.c | 4 ---- 6 files changed, 67 insertions(+), 5 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -1
@@ -123,7 +123,7 @@ KVM_X86_OP(enter_smm) KVM_X86_OP(leave_smm) KVM_X86_OP(enable_smi_window) #endif -KVM_X86_OP_OPTIONAL(mem_enc_ioctl) +KVM_X86_OP(mem_enc_ioctl) KVM_X86_OP_OPTIONAL(mem_enc_register_region) KVM_X86_OP_OPTIONAL(mem_enc_unregister_region) KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from)
arch/x86/include/uapi/asm/kvm.h +26 -0
@@ -567,6 +567,32 @@ struct kvm_pmu_event_filter {#define KVM_X86_TDX_VM 2#define KVM_X86_SNP_VM 3 +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CAPABILITIES =0 , + +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +/* enum kvm_tdx_cmd_id */ +__u32 id; +/* flags for sub-commend. If sub-command doesn't use this, set zero. */ +__u32 flags; +/* +* data for each sub-command. An immediate or a pointer to the actual +* data in process virtual address. If sub-command doesn't use it, +* set zero. +*/ +__u64 data; +/* +* Auxiliary error code. The sub-command may return TDX SEAMCALL +* status code in addition to -Exxx. +* Defined for consistency with struct kvm_sev_cmd. +*/ +__u64 error; +}; +#define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1)struct kvm_tdx_cpuid_config {
arch/x86/kvm/vmx/main.c +10 -0
@@ -62,6 +62,14 @@ static int vt_vm_init(struct kvm *kvm)return vmx_vm_init (kvm); } +static int vt_mem_enc_ioctl (struct kvm *kvm,void __user *argp) +{ +if (!is_td (kvm)) +return -ENOTTY ; + +return tdx_vm_ioctl (kvm, argp); +} +#define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \
@@ -214,6 +222,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, .get_untagged_addr = vmx_get_untagged_addr, + + .mem_enc_ioctl = vt_mem_enc_ioctl, };struct kvm_x86_init_ops vt_init_ops__initdata = {
arch/x86/kvm/vmx/tdx.c +26 -0
@@ -99,6 +99,32 @@ struct tdx_info {/* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) +{ +struct kvm_tdx_cmd tdx_cmd; +int r; + +if (copy_from_user (&tdx_cmd, argp,sizeof (struct kvm_tdx_cmd ))) +return -EFAULT ; +if (tdx_cmd.error) +return -EINVAL ; + +mutex_lock (&kvm->lock); + +switch (tdx_cmd.id) { +default : + r = -EINVAL ; + goto out; + } + +if (copy_to_user (argp, &tdx_cmd,sizeof (struct kvm_tdx_cmd ))) + r = -EFAULT ; + +out: +mutex_unlock (&kvm->lock); +return r; +} +static int __init tdx_module_setup (void ) {u16 num_cpuid_config;
arch/x86/kvm/vmx/x86_ops.h +4 -0
@@ -138,10 +138,14 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type); + +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; } + +static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
arch/x86/kvm/x86.c +0 -4
@@ -7218,10 +7218,6 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) goto out; }case KVM_MEMORY_ENCRYPT_OP : { - r = -ENOTTY ; -if (!kvm_x86_ops.mem_enc_ioctl) - goto out; - r =static_call (kvm_x86_mem_enc_ioctl)(kvm, argp);break ; }
Signature · 1 line
2.25.1
PATCH v18 22/121 KVM: TDX: x86: Add ioctl to get TDX systemwide parameters
In reply to [email protected]From: Sean Christopherson <[email protected]>
Implement an ioctl to get system-wide parameters for TDX. Although the function is systemwide, vm scoped mem_enc ioctl works for userspace VMM like qemu and device scoped version is not define, re-use vm scoped mem_enc.
---
v18:
- drop the use of tdhsysinfo_struct and TDH.SYS.INFO, use TDH.SYS.RD(). For that, dynamically allocate/free tdx_info.
- drop the change of tools/arch/x86/include/uapi/asm/kvm.h.
v14 -> v15:
- ABI change: added supported_gpaw and reserved area.
--- arch/x86/include/uapi/asm/kvm.h | 17 ++++++++++ arch/x86/kvm/vmx/tdx.c | 56 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 3 ++ 3 files changed, 76 insertions(+)
arch/x86/include/uapi/asm/kvm.h +17 -0
@@ -604,4 +604,21 @@ struct kvm_tdx_cpuid_config { __u32 edx; }; +/* supported_gpaw */ +#define TDX_CAP_GPAW_48 (1 << 0) +#define TDX_CAP_GPAW_52 (1 << 1) + +struct kvm_tdx_capabilities { +__u64 attrs_fixed0; +__u64 attrs_fixed1; +__u64 xfam_fixed0; +__u64 xfam_fixed1; +__u32 supported_gpaw; +__u32 padding; +__u64 reserved[251 ]; + +__u32 nr_cpuid_configs; +struct kvm_tdx_cpuid_config cpuid_configs[]; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/vmx/tdx.c +56 -0
74 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +3 -0
@@ -3,6 +3,9 @@#define __KVM_X86_TDX_H #ifdef CONFIG_INTEL_TDX_HOST + +#include "tdx_ops.h" + struct kvm_tdx { struct kvm kvm; /* TDX specific members follow. */
Signature · 1 line
2.25.1
PATCH v18 23/121 KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own KVM_ENABLE_CAP handler and implement TDX backend for KVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead of KVM_MAX_VCPUS.
---
v18:
- use TDX instead of "x86, tdx" in subject
- use min(max_vcpu, TDX_MAX_VCPU) instead of
min3(max_vcpu, KVM_MAX_VCPU, TDX_MAX_VCPU)
- make "if (KVM_MAX_VCPU) and if (TDX_MAX_VCPU)" into one if statement
--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/main.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 5 +++++ arch/x86/kvm/x86.c | 4 ++++ 6 files changed, 64 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +2 -0
@@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) +KVM_X86_OP_OPTIONAL(max_vcpus); +KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1602,7 +1602,9 @@ struct kvm_x86_ops {void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);bool (*is_vm_type_supported)(unsigned long vm_type); +int (*max_vcpus)(struct kvm *kvm);unsigned int vm_size; +int (*vm_enable_cap)(struct kvm *kvm,struct kvm_enable_cap *cap);int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);
arch/x86/kvm/vmx/main.c +22 -0
@@ -6,6 +6,7 @@#include "nested.h" #include "pmu.h" #include "tdx.h" +#include "tdx_arch.h" static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 );
@@ -16,6 +17,17 @@ static bool vt_is_vm_type_supported(unsigned long type) (enable_tdx &&tdx_is_vm_type_supported (type)); } +static int vt_max_vcpus (struct kvm *kvm) +{ +if (!kvm) +return KVM_MAX_VCPUS ; + +if (is_td (kvm)) +return min (kvm->max_vcpus,TDX_MAX_VCPUS ); + +return kvm->max_vcpus; +} +static int vt_hardware_enable (void ) {int ret;
@@ -54,6 +66,14 @@ static void vt_hardware_unsetup(void)vmx_hardware_unsetup (); } +static int vt_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +if (is_td (kvm)) +return tdx_vm_enable_cap (kvm, cap); + +return -EINVAL ; +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))
@@ -91,7 +111,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, + .max_vcpus = vt_max_vcpus, .vm_size = sizeof(struct kvm_vmx), + .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy,
arch/x86/kvm/vmx/tdx.c +29 -0
@@ -100,6 +100,35 @@ struct tdx_info {/* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +int r; + +switch (cap->cap) { +case KVM_CAP_MAX_VCPUS : { +if (cap->flags || cap->args[0 ] ==0 ) +return -EINVAL ; +if (cap->args[0 ] >KVM_MAX_VCPUS || + cap->args[0 ] >TDX_MAX_VCPUS ) +return -E2BIG ; + +mutex_lock (&kvm->lock); +if (kvm->created_vcpus) + r = -EBUSY ; +else { + kvm->max_vcpus = cap->args[0 ]; + r =0 ; + } +mutex_unlock (&kvm->lock); +break ; + } +default : + r = -EINVAL ; +break ; + } +return r; +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;
arch/x86/kvm/vmx/x86_ops.h +5 -0
@@ -139,12 +139,17 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type); +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; } +static inline int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +return -EINVAL ; +};static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif
arch/x86/kvm/x86.c +4 -0
@@ -4726,6 +4726,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)break ;case KVM_CAP_MAX_VCPUS : r =KVM_MAX_VCPUS ; +if (kvm_x86_ops.max_vcpus) + r =static_call (kvm_x86_max_vcpus)(kvm);break ;case KVM_CAP_MAX_VCPU_ID : r =KVM_MAX_VCPU_IDS ;
@@ -6683,6 +6685,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;default : r = -EINVAL ; +if (kvm_x86_ops.vm_enable_cap) + r =static_call (kvm_x86_vm_enable_cap)(kvm, cap);break ; }return r;
Signature · 1 line
2.25.1
Re: [PATCH v18 023/121] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
Quoted 6 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own
KVM_ENABLE_CAPhandler and implement TDX backend forKVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead ofKVM_MAX_VCPUS.
For legacy VM, KVM just provides the interface to query the max_vcpus. Why TD needs to provide a interface for userspace to set the limitation? What's the scenario?
Quoted 183 lines
Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- use TDX instead of "x86, tdx" in subject
- usemin(max_vcpu, TDX_MAX_VCPU)instead of
min3(max_vcpu, KVM_MAX_VCPU, TDX_MAX_VCPU)
- make "if (KVM_MAX_VCPU) and if (TDX_MAX_VCPU)" into one if statement--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/main.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 5 +++++ arch/x86/kvm/x86.c | 4 ++++ 6 files changed, 64 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +2
-0@@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) +KVM_X86_OP_OPTIONAL(max_vcpus); +KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)arch/x86/include/asm/kvm_host.h +2
-0@@ -1602,7 +1602,9 @@ struct kvm_x86_ops {void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);bool (*is_vm_type_supported)(unsigned long vm_type); +int (*max_vcpus)(struct kvm *kvm);unsigned int vm_size; +int (*vm_enable_cap)(struct kvm *kvm,struct kvm_enable_cap *cap);int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);arch/x86/kvm/vmx/main.c +22
-0@@ -6,6 +6,7 @@#include "nested.h" #include "pmu.h" #include "tdx.h" +#include "tdx_arch.h" static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 );@@ -16,6 +17,17 @@ static bool vt_is_vm_type_supported(unsigned long type) (enable_tdx &&tdx_is_vm_type_supported (type)); } +static int vt_max_vcpus (struct kvm *kvm) +{ +if (!kvm) +return KVM_MAX_VCPUS ; + +if (is_td (kvm)) +return min (kvm->max_vcpus,TDX_MAX_VCPUS ); + +return kvm->max_vcpus; +} +static int vt_hardware_enable (void ) {int ret;@@ -54,6 +66,14 @@ static void vt_hardware_unsetup(void)vmx_hardware_unsetup (); } +static int vt_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +if (is_td (kvm)) +return tdx_vm_enable_cap (kvm, cap); + +return -EINVAL ; +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))@@ -91,7 +111,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, + .max_vcpus = vt_max_vcpus, .vm_size = sizeof(struct kvm_vmx), + .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy,arch/x86/kvm/vmx/tdx.c +29
-0@@ -100,6 +100,35 @@ struct tdx_info {/* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +int r; + +switch (cap->cap) { +case KVM_CAP_MAX_VCPUS : { +if (cap->flags || cap->args[0 ] ==0 ) +return -EINVAL ; +if (cap->args[0 ] >KVM_MAX_VCPUS || + cap->args[0 ] >TDX_MAX_VCPUS ) +return -E2BIG ; + +mutex_lock (&kvm->lock); +if (kvm->created_vcpus) + r = -EBUSY ; +else { + kvm->max_vcpus = cap->args[0 ]; + r =0 ; + } +mutex_unlock (&kvm->lock); +break ; + } +default : + r = -EINVAL ; +break ; + } +return r; +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;arch/x86/kvm/vmx/x86_ops.h +5
-0@@ -139,12 +139,17 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type); +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; } +static inline int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +return -EINVAL ; +};static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif arch/x86/kvm/x86.c +4
-0@@ -4726,6 +4726,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)break ;case KVM_CAP_MAX_VCPUS : r =KVM_MAX_VCPUS ; +if (kvm_x86_ops.max_vcpus) + r =static_call (kvm_x86_max_vcpus)(kvm);break ;case KVM_CAP_MAX_VCPU_ID : r =KVM_MAX_VCPU_IDS ;@@ -6683,6 +6685,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;default : r = -EINVAL ; +if (kvm_x86_ops.vm_enable_cap) + r =static_call (kvm_x86_vm_enable_cap)(kvm, cap);break ; }return r;
Re: [PATCH v18 023/121] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
Quoted 5 lines · Binbin Wu
Quoted 2 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Quoted 7 lines
Quoted 4 lines
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own
KVM_ENABLE_CAPhandler and implement TDX backend forKVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead ofKVM_MAX_VCPUS.For legacy VM, KVM just provides the interface to query the
max_vcpus. Why TD needs to provide a interface for userspace to set the limitation? What's the scenario?
I think the reason is TDH.MNG.INIT needs it:
TD_PARAMS:
MAX_VCPUS:
offset: 16 bytes.
type: Unsigned 16b Integer.
size: 2.
Description: Maximum number of VCPUs.
May better to clarify this in the commit yet.
Quoted 176 lines
Quoted 174 lines
Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- use TDX instead of "x86, tdx" in subject
- usemin(max_vcpu, TDX_MAX_VCPU)instead of
min3(max_vcpu, KVM_MAX_VCPU, TDX_MAX_VCPU)
- make "if (KVM_MAX_VCPU) and if (TDX_MAX_VCPU)" into one if statement--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/main.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 5 +++++ arch/x86/kvm/x86.c | 4 ++++ 6 files changed, 64 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +2
-0@@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) +KVM_X86_OP_OPTIONAL(max_vcpus); +KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)arch/x86/include/asm/kvm_host.h +2
-0@@ -1602,7 +1602,9 @@ struct kvm_x86_ops {void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);bool (*is_vm_type_supported)(unsigned long vm_type); +int (*max_vcpus)(struct kvm *kvm);unsigned int vm_size; +int (*vm_enable_cap)(struct kvm *kvm,struct kvm_enable_cap *cap);int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);arch/x86/kvm/vmx/main.c +22
-0@@ -6,6 +6,7 @@#include "nested.h" #include "pmu.h" #include "tdx.h" +#include "tdx_arch.h" static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 );@@ -16,6 +17,17 @@ static bool vt_is_vm_type_supported(unsigned long type) (enable_tdx &&tdx_is_vm_type_supported (type)); } +static int vt_max_vcpus (struct kvm *kvm) +{ +if (!kvm) +return KVM_MAX_VCPUS ; + +if (is_td (kvm)) +return min (kvm->max_vcpus,TDX_MAX_VCPUS ); + +return kvm->max_vcpus; +} +static int vt_hardware_enable (void ) {int ret;@@ -54,6 +66,14 @@ static void vt_hardware_unsetup(void)vmx_hardware_unsetup (); } +static int vt_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +if (is_td (kvm)) +return tdx_vm_enable_cap (kvm, cap); + +return -EINVAL ; +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))@@ -91,7 +111,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, + .max_vcpus = vt_max_vcpus, .vm_size = sizeof(struct kvm_vmx), + .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy,arch/x86/kvm/vmx/tdx.c +29
-0@@ -100,6 +100,35 @@ struct tdx_info {/* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +int r; + +switch (cap->cap) { +case KVM_CAP_MAX_VCPUS : { +if (cap->flags || cap->args[0 ] ==0 ) +return -EINVAL ; +if (cap->args[0 ] >KVM_MAX_VCPUS || + cap->args[0 ] >TDX_MAX_VCPUS ) +return -E2BIG ; + +mutex_lock (&kvm->lock); +if (kvm->created_vcpus) + r = -EBUSY ; +else { + kvm->max_vcpus = cap->args[0 ]; + r =0 ; + } +mutex_unlock (&kvm->lock); +break ; + } +default : + r = -EINVAL ; +break ; + } +return r; +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;arch/x86/kvm/vmx/x86_ops.h +5
-0@@ -139,12 +139,17 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type); +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; } +static inline int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +return -EINVAL ; +};static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif arch/x86/kvm/x86.c +4
-0@@ -4726,6 +4726,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)break ;case KVM_CAP_MAX_VCPUS : r =KVM_MAX_VCPUS ; +if (kvm_x86_ops.max_vcpus) + r =static_call (kvm_x86_max_vcpus)(kvm);break ;case KVM_CAP_MAX_VCPU_ID : r =KVM_MAX_VCPU_IDS ;@@ -6683,6 +6685,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;default : r = -EINVAL ; +if (kvm_x86_ops.vm_enable_cap) + r =static_call (kvm_x86_vm_enable_cap)(kvm, cap);break ; }return r;
Re: [PATCH v18 023/121] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
Quoted 12 lines · Yuan Yao
Quoted 11 lines · Binbin Wu
Quoted 6 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own
KVM_ENABLE_CAPhandler and implement TDX backend forKVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead ofKVM_MAX_VCPUS.For legacy VM, KVM just provides the interface to query the
max_vcpus. Why TD needs to provide a interface for userspace to set the limitation? What's the scenario?
Quoted 8 lines
I think the reason is TDH.MNG.INIT needs it:
TD_PARAMS:
MAX_VCPUS:
offset: 16 bytes.
type: Unsigned 16b Integer.
size: 2.
Description: Maximum number of VCPUs.
Thanks for explanation.
I am also wondering if this info can be passed via KVM_TDX_INIT_VM. Because userspace is allowed to set the value no greater than min(KVM_MAX_VCPUS, TDX_MAX_VCPUS), providing the extra cap KVM_CAP_MAX_VCPUS doesn't make more restriction comparing to providing it in KVM_TDX_INIT_VM.
Quoted 176 lines
May better to clarify this in the commit yet.
Quoted 173 lines
Quoted 172 lines
Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- use TDX instead of "x86, tdx" in subject
- usemin(max_vcpu, TDX_MAX_VCPU)instead of
min3(max_vcpu, KVM_MAX_VCPU, TDX_MAX_VCPU)
- make "if (KVM_MAX_VCPU) and if (TDX_MAX_VCPU)" into one if statement--- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/main.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 5 +++++ arch/x86/kvm/x86.c | 4 ++++ 6 files changed, 64 insertions(+)arch/x86/include/asm/kvm-x86-ops.h +2
-0@@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) +KVM_X86_OP_OPTIONAL(max_vcpus); +KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate)arch/x86/include/asm/kvm_host.h +2
-0@@ -1602,7 +1602,9 @@ struct kvm_x86_ops {void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);bool (*is_vm_type_supported)(unsigned long vm_type); +int (*max_vcpus)(struct kvm *kvm);unsigned int vm_size; +int (*vm_enable_cap)(struct kvm *kvm,struct kvm_enable_cap *cap);int (*vm_init)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm);arch/x86/kvm/vmx/main.c +22
-0@@ -6,6 +6,7 @@#include "nested.h" #include "pmu.h" #include "tdx.h" +#include "tdx_arch.h" static bool enable_tdx__ro_after_init ;module_param_named (tdx, enable_tdx, bool,0444 );@@ -16,6 +17,17 @@ static bool vt_is_vm_type_supported(unsigned long type) (enable_tdx &&tdx_is_vm_type_supported (type)); } +static int vt_max_vcpus (struct kvm *kvm) +{ +if (!kvm) +return KVM_MAX_VCPUS ; + +if (is_td (kvm)) +return min (kvm->max_vcpus,TDX_MAX_VCPUS ); + +return kvm->max_vcpus; +} +static int vt_hardware_enable (void ) {int ret;@@ -54,6 +66,14 @@ static void vt_hardware_unsetup(void)vmx_hardware_unsetup (); } +static int vt_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +if (is_td (kvm)) +return tdx_vm_enable_cap (kvm, cap); + +return -EINVAL ; +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))@@ -91,7 +111,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, + .max_vcpus = vt_max_vcpus, .vm_size = sizeof(struct kvm_vmx), + .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy,arch/x86/kvm/vmx/tdx.c +29
-0@@ -100,6 +100,35 @@ struct tdx_info {/* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +int r; + +switch (cap->cap) { +case KVM_CAP_MAX_VCPUS : { +if (cap->flags || cap->args[0 ] ==0 ) +return -EINVAL ; +if (cap->args[0 ] >KVM_MAX_VCPUS || + cap->args[0 ] >TDX_MAX_VCPUS ) +return -E2BIG ; + +mutex_lock (&kvm->lock); +if (kvm->created_vcpus) + r = -EBUSY ; +else { + kvm->max_vcpus = cap->args[0 ]; + r =0 ; + } +mutex_unlock (&kvm->lock); +break ; + } +default : + r = -EINVAL ; +break ; + } +return r; +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;arch/x86/kvm/vmx/x86_ops.h +5
-0@@ -139,12 +139,17 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type); +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; } +static inline int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +return -EINVAL ; +};static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif arch/x86/kvm/x86.c +4
-0@@ -4726,6 +4726,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)break ;case KVM_CAP_MAX_VCPUS : r =KVM_MAX_VCPUS ; +if (kvm_x86_ops.max_vcpus) + r =static_call (kvm_x86_max_vcpus)(kvm);break ;case KVM_CAP_MAX_VCPU_ID : r =KVM_MAX_VCPU_IDS ;@@ -6683,6 +6685,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;default : r = -EINVAL ; +if (kvm_x86_ops.vm_enable_cap) + r =static_call (kvm_x86_vm_enable_cap)(kvm, cap);break ; }return r;
Re: [PATCH v18 023/121] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
Quoted 23 lines · Binbin Wu
Quoted 12 lines · Yuan Yao
Quoted 11 lines · Binbin Wu
Quoted 6 lines · On 1/23/2024 7:52 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own
KVM_ENABLE_CAPhandler and implement TDX backend forKVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead ofKVM_MAX_VCPUS.For legacy VM, KVM just provides the interface to query the
max_vcpus. Why TD needs to provide a interface for userspace to set the limitation? What's the scenario?Quoted 8 lines
I think the reason is TDH.MNG.INIT needs it:
TD_PARAMS:
MAX_VCPUS:
offset: 16 bytes.
type: Unsigned 16b Integer.
size: 2.
Description: Maximum number of VCPUs.
Quoted 6 lines
Thanks for explanation.
I am also wondering if this info can be passed via
KVM_TDX_INIT_VM. Because userspace is allowed to set the value no greater thanmin(KVM_MAX_VCPUS, TDX_MAX_VCPUS), providing the extra capKVM_CAP_MAX_VCPUSdoesn't make more restriction comparing to providing it inKVM_TDX_INIT_VM.
It's better for the API to be common, not specific to TDX. Also I don't want to play with max_vcpu in multiple places.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v18 023/121] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific
In reply to [email protected]arch/x86/kvm/vmx/main.c static bool vt_is_vm_type_supported(unsigned long type) · line 26
Quoted 7 lines
arch/x86/kvm/vmx/main.c +7
-0+static int vt_max_vcpus (struct kvm *kvm) +{ +if (!kvm) +return KVM_MAX_VCPUS ; + +if (is_td (kvm)) +return min (kvm->max_vcpus,TDX_MAX_VCPUS );
I suppose kvm->max_vcpus should be always smaller than TDX_MAX_VCPUS, right? if that's the case, the min() is pointless.
I don't get why kvm->max_vcpus is concerned. do you want to enable userspace to read back the max_vcpus configured last time? this looks useless because userspace can keep track of that value. how about:
/* * TDX module imposes additional restrictions on the maximum number of * vCPUs of a TD guest. */ if (kvm && is_td(kvm)) return min(TDX_MAX_VCPUS, KVM_MAX_VCPUS); else return KVM_MAX_VCPUS;
Quoted 45 lines
arch/x86/kvm/vmx/main.c +4
-0+ +return kvm->max_vcpus; +} +static int
vt_hardware_enable(void){ int ret;arch/x86/kvm/vmx/main.c +10
-0@@ -54,6 +66,14 @@ static void vt_hardware_unsetup(void)vmx_hardware_unsetup (); } +static int vt_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +if (is_td (kvm)) +return tdx_vm_enable_cap (kvm, cap); + +return -EINVAL ; +} +static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm))@@ -91,7 +111,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, + .max_vcpus = vt_max_vcpus, .vm_size = sizeof(struct kvm_vmx), + .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy,arch/x86/kvm/vmx/tdx.c +5
-0@@ -100,6 +100,35 @@ struct tdx_info {/* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) +{ +int r; + +switch (cap->cap) {
arch/x86/kvm/vmx/tdx.c struct tdx_info { · line 117
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+case KVM_CAP_MAX_VCPUS : { +if (cap->flags || cap->args[0 ] ==0 ) +return -EINVAL ; +if (cap->args[0 ] >KVM_MAX_VCPUS || + cap->args[0 ] >TDX_MAX_VCPUS ) +return -E2BIG ; + +mutex_lock (&kvm->lock); +if (kvm->created_vcpus) + r = -EBUSY ;
Curly brackets are missing.
And -EBUSY looks improper because it isn't a temporary error.
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+else { + kvm->max_vcpus = cap->args[0 ]; + r =0 ; + } +mutex_unlock (&kvm->lock); +break ;
PATCH v18 24/121 KVM: TDX: create/destroy VM structure
In reply to [email protected]From: Isaku Yamahata <[email protected]>
As the first step to create TDX guest, create/destroy VM struct. Assign TDX private Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. HKID must have been reclaimed, etc). Add mmu notifier release callback before tearing down private page tables for it.
Add vm_free() of kvm_x86_ops hook at the end of kvm_arch_destroy_vm() because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed.
---
v18:
- Use TDH.SYS.RD() instead of struct tdsysinfo_struct.
- Rename tdx_reclaim_td_page() to tdx_reclaim_control_page()
- return -EAGAIN on TDX_RND_NO_ENTROPY of TDH.MNG.CREATE(), TDH.MNG.ADDCX()
- fix comment to remove extra the.
- use true instead of 1 for boolean.
- remove an extra white line.
v16:
- Simplified tdx_reclaim_page()
- Reorganize the locking of tdx_release_hkid(), and use smp_call_mask() instead of smp_call_on_cpu() to hold spinlock to race with invalidation on releasing guest memfd
--- arch/x86/include/asm/kvm-x86-ops.h | 2 + arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/mmu/mmu.c | 7 + arch/x86/kvm/vmx/main.c | 26 +- arch/x86/kvm/vmx/tdx.c | 474 ++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 6 +- arch/x86/kvm/vmx/x86_ops.h | 6 + arch/x86/kvm/x86.c | 1 + 9 files changed, 519 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +2 -0
@@ -24,7 +24,9 @@ KVM_X86_OP(is_vm_type_supported)KVM_X86_OP_OPTIONAL (max_vcpus); KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) +KVM_X86_OP_OPTIONAL(flush_shadow_all_private) KVM_X86_OP_OPTIONAL(vm_destroy) +KVM_X86_OP_OPTIONAL(vm_free) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1606,7 +1606,9 @@ struct kvm_x86_ops {unsigned int vm_size;int (*vm_enable_cap)(struct kvm *kvm,struct kvm_enable_cap *cap);int (*vm_init)(struct kvm *kvm); +void (*flush_shadow_all_private)(struct kvm *kvm);void (*vm_destroy)(struct kvm *kvm); +void (*vm_free)(struct kvm *kvm);/* Create, but do not attach this VCPU */ int (*vcpu_precreate)(struct kvm *kvm);
arch/x86/kvm/Kconfig +2 -1
@@ -76,7 +76,6 @@ config KVM_WERROR config KVM_SW_PROTECTED_VM bool "Enable support for KVM software-protected VMs" - depends on EXPERT depends on KVM && X86_64 select KVM_GENERIC_PRIVATE_MEM help
@@ -89,6 +88,8 @@ config KVM_SW_PROTECTED_VM config KVM_INTEL tristate "KVM for Intel (and compatible) processors support" depends on KVM && IA32_FEAT_CTL + select KVM_SW_PROTECTED_VM if INTEL_TDX_HOST + select KVM_PRIVATE_MEM if INTEL_TDX_HOST help Provides support for KVM on processors equipped with Intel's VT extensions, a.k.a. Virtual Machine Extensions (VMX).
arch/x86/kvm/mmu/mmu.c +7 -0
@@ -6791,6 +6791,13 @@ static void kvm_mmu_zap_all(struct kvm *kvm)void kvm_arch_flush_shadow_all (struct kvm *kvm) { +/* +* kvm_mmu_zap_all() zaps both private and shared page tables. Before +* tearing down private page tables, TDX requires some TD resources to +* be destroyed (i.e. keyID must have been reclaimed, etc). Invoke +* kvm_x86_flush_shadow_all_private() for this. +*/ +static_call_cond (kvm_x86_flush_shadow_all_private)(kvm);kvm_mmu_zap_all (kvm); }
arch/x86/kvm/vmx/main.c +24 -2
@@ -77,11 +77,31 @@ static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)static int vt_vm_init (struct kvm *kvm) {if (is_td (kvm)) -return -EOPNOTSUPP ;/* Not ready to create guest TD yet. */ +return tdx_vm_init (kvm);return vmx_vm_init (kvm); } +static void vt_flush_shadow_all_private (struct kvm *kvm) +{ +if (is_td (kvm)) +tdx_mmu_release_hkid (kvm); +} + +static void vt_vm_destroy (struct kvm *kvm) +{ +if (is_td (kvm)) +return ; + +vmx_vm_destroy (kvm); +} + +static void vt_vm_free (struct kvm *kvm) +{ +if (is_td (kvm)) +tdx_vm_free (kvm); +} +static int vt_mem_enc_ioctl (struct kvm *kvm,void __user *argp) {if (!is_td (kvm))
@@ -115,7 +135,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vm_size = sizeof(struct kvm_vmx), .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, - .vm_destroy = vmx_vm_destroy, + .flush_shadow_all_private = vt_flush_shadow_all_private, + .vm_destroy = vt_vm_destroy, + .vm_free = vt_vm_free, .vcpu_precreate = vmx_vcpu_precreate, .vcpu_create = vmx_vcpu_create,
arch/x86/kvm/vmx/tdx.c +470 -4
547 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +5 -1
@@ -8,7 +8,11 @@struct kvm_tdx {struct kvm kvm; -/* TDX specific members follow. */ + +unsigned long tdr_pa; +unsigned long *tdcs_pa; + +int hkid; };struct vcpu_tdx {
arch/x86/kvm/vmx/x86_ops.h +6 -0
@@ -140,6 +140,9 @@ void tdx_hardware_unsetup(void);bool tdx_is_vm_type_supported (unsigned long type);int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap); +int tdx_vm_init (struct kvm *kvm); +void tdx_mmu_release_hkid (struct kvm *kvm); +void tdx_vm_free (struct kvm *kvm);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }
@@ -150,6 +153,9 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) {return -EINVAL ; }; +static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; } +static inline void tdx_mmu_release_hkid (struct kvm *kvm) {} +static inline void tdx_vm_free (struct kvm *kvm) {}static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif
arch/x86/kvm/x86.c +1 -0
@@ -12714,6 +12714,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_free)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot)
Signature · 1 line
2.25.1
Re: [PATCH v18 024/121] KVM: TDX: create/destroy VM structure
Quoted 46 lines · On Mon, Jan 22, 2024 at 03:53:00PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
As the first step to create TDX guest, create/destroy VM struct. Assign TDX private Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID.
Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. HKID must have been reclaimed, etc). Add mmu notifier release callback before tearing down private page tables for it.
Add
vm_free()ofkvm_x86_opshook at the end ofkvm_arch_destroy_vm()because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed.Co-developed-by Kai Huang <[email protected]>Signed-off-by Kai Huang <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- UseTDH.SYS.RD()instead of structtdsysinfo_struct.
- Renametdx_reclaim_td_page()totdx_reclaim_control_page()
- return-EAGAINonTDX_RND_NO_ENTROPYofTDH.MNG.CREATE(),TDH.MNG.ADDCX()
- fix comment to remove extra the.
- use true instead of 1 for boolean.
- remove an extra white line.v16:
- Simplifiedtdx_reclaim_page()
- Reorganize the locking oftdx_release_hkid(), and usesmp_call_mask()instead ofsmp_call_on_cpu()to hold spinlock to race with invalidation on releasing guest memfd--- arch/x86/include/asm/kvm-x86-ops.h | 2 + arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/mmu/mmu.c | 7 + arch/x86/kvm/vmx/main.c | 26 +- arch/x86/kvm/vmx/tdx.c | 474 ++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 6 +- arch/x86/kvm/vmx/x86_ops.h | 6 + arch/x86/kvm/x86.c | 1 + 9 files changed, 519 insertions(+), 8 deletions(-)
...
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+ +static int __tdx_reclaim_page (hpa_t pa) +{ +struct tdx_module_args out; +u64 err;
arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 203
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ +do { + err =tdh_phymem_page_reclaim (pa, &out); +/* +* TDH.PHYMEM.PAGE.RECLAIM is allowed only when TD is shutdown. +* state. i.e. destructing TD. +* TDH.PHYMEM.PAGE.RECLAIM requires TDR and target page. +* Because we're destructing TD, it's rare to contend with TDR. +*/ + }while (unlikely (err == (TDX_OPERAND_BUSY |TDX_OPERAND_ID_RCX )));
v16 changed to tdx module 1.5, so here should be TDX_OPERAND_ID_TDR, value 128ULL.
Quoted 257 lines
arch/x86/kvm/vmx/tdx.c +250
-0+if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_PAGE_RECLAIM , err, &out); +return -EIO ; + } + +return 0 ; +} + +static int tdx_reclaim_page (hpa_t pa) +{ +int r; + + r =__tdx_reclaim_page (pa); +if (!r) +tdx_clear_page (pa); +return r; +} + +static void tdx_reclaim_control_page (unsigned long td_page_pa) +{ +WARN_ON_ONCE (!td_page_pa); + +/* +* TDCX are being reclaimed. TDX module maps TDCX with HKID +* assigned to the TD. Here the cache associated to the TD +* was already flushed by TDH.PHYMEM.CACHE.WB before here, So +* cache doesn't need to be flushed again. +*/ +if (tdx_reclaim_page (td_page_pa)) +/* +* Leak the page on failure: +* tdx_reclaim_page() returns an error if and only if there's an +* unexpected, fatal error, e.g. a SEAMCALL with bad params, +* incorrect concurrency in KVM, a TDX Module bug, etc. +* Retrying at a later point is highly unlikely to be +* successful. +* No log here as tdx_reclaim_page() already did. +*/ +return ; +free_page ((unsigned long )__va (td_page_pa)); +} + +static void tdx_do_tdh_phymem_cache_wb (void *unused) +{ +u64 err =0 ; + +do { + err =tdh_phymem_cache_wb (!!err); + }while (err ==TDX_INTERRUPTED_RESUMABLE ); + +/* Other thread may have done for us. */ +if (err ==TDX_NO_HKID_READY_TO_WBCACHE ) + err =TDX_SUCCESS ; +if (WARN_ON_ONCE (err)) +pr_tdx_error (TDH_PHYMEM_CACHE_WB , err,NULL ); +} + +void tdx_mmu_release_hkid (struct kvm *kvm) +{ +bool packages_allocated, targets_allocated; +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages, targets; +u64 err; +int i; + +if (!is_hkid_assigned (kvm_tdx)) +return ; + +if (!is_td_created (kvm_tdx)) { +tdx_hkid_free (kvm_tdx); +return ; + } + + packages_allocated =zalloc_cpumask_var (&packages,GFP_KERNEL ); + targets_allocated =zalloc_cpumask_var (&targets,GFP_KERNEL ); +cpus_read_lock (); + +/* +* We can destroy multiple guest TDs simultaneously. Prevent +* tdh_phymem_cache_wb from returning TDX_BUSY by serialization. +*/ +mutex_lock (&tdx_lock); + +/* +* Go through multiple TDX HKID state transitions with three SEAMCALLs +* to make TDH.PHYMEM.PAGE.RECLAIM() usable. Make the transition atomic +* to other functions to operate private pages and Secure-EPT pages. +* +* Avoid race for kvm_gmem_release() to call kvm_mmu_unmap_gfn_range(). +* This function is called via mmu notifier, mmu_release(). +* kvm_gmem_release() is called via fput() on process exit. +*/ +write_lock (&kvm->mmu_lock); + +for_each_online_cpu (i) { +if (packages_allocated && +cpumask_test_and_set_cpu (topology_physical_package_id (i), + packages)) +continue ; +if (targets_allocated) +cpumask_set_cpu (i, targets); + } +if (targets_allocated) +on_each_cpu_mask (targets, tdx_do_tdh_phymem_cache_wb,NULL , true); +else +on_each_cpu (tdx_do_tdh_phymem_cache_wb,NULL , true); +/* +* In the case of error in tdx_do_tdh_phymem_cache_wb(), the following +* tdh_mng_key_freeid() will fail. +*/ + err =tdh_mng_key_freeid (kvm_tdx->tdr_pa); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_FREEID , err,NULL ); +pr_err ("tdh_mng_key_freeid() failed. HKID %d is leaked.\n" , + kvm_tdx->hkid); + }else +tdx_hkid_free (kvm_tdx); + +write_unlock (&kvm->mmu_lock); +mutex_unlock (&tdx_lock); +cpus_read_unlock (); +free_cpumask_var (targets); +free_cpumask_var (packages); +} + +void tdx_vm_free (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +u64 err; +int i; + +/* +* tdx_mmu_release_hkid() failed to reclaim HKID. Something went wrong +* heavily with TDX module. Give up freeing TD pages. As the function +* already warned, don't warn it again. +*/ +if (is_hkid_assigned (kvm_tdx)) +return ; + +if (kvm_tdx->tdcs_pa) { +for (i =0 ; i < tdx_info->nr_tdcs_pages; i++) { +if (kvm_tdx->tdcs_pa[i]) +tdx_reclaim_control_page (kvm_tdx->tdcs_pa[i]); + } +kfree (kvm_tdx->tdcs_pa); + kvm_tdx->tdcs_pa =NULL ; + } + +if (!kvm_tdx->tdr_pa) +return ; +if (__tdx_reclaim_page (kvm_tdx->tdr_pa)) +return ; +/* +* TDX module maps TDR with TDX global HKID. TDX module may access TDR +* while operating on TD (Especially reclaiming TDCS). Cache flush with +* TDX global HKID is needed. +*/ + err =tdh_phymem_page_wbinvd (set_hkid_to_hpa (kvm_tdx->tdr_pa, + tdx_global_keyid)); +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_PHYMEM_PAGE_WBINVD , err,NULL ); +return ; + } +tdx_clear_page (kvm_tdx->tdr_pa); + +free_page ((unsigned long )__va (kvm_tdx->tdr_pa)); + kvm_tdx->tdr_pa =0 ; +} + +static int tdx_do_tdh_mng_key_config (void *param) +{ +hpa_t *tdr_p = param; +u64 err; + +do { + err =tdh_mng_key_config (*tdr_p); + +/* +* If it failed to generate a random key, retry it because this +* is typically caused by an entropy error of the CPU's random +* number generator. +*/ + }while (err ==TDX_KEY_GENERATION_FAILED ); + +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_KEY_CONFIG , err,NULL ); +return -EIO ; + } + +return 0 ; +} + +static int __tdx_td_init (struct kvm *kvm); + +int tdx_vm_init (struct kvm *kvm) +{ +/* +* TDX has its own limit of the number of vcpus in addition to +* KVM_MAX_VCPUS. +*/ + kvm->max_vcpus =min (kvm->max_vcpus,TDX_MAX_VCPUS ); + +/* Place holder for TDX specific logic. */ +return __tdx_td_init (kvm); +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;@@ -181,6 +459,176 @@ static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd)return ret; } +static int __tdx_td_init (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +unsigned long *tdcs_pa =NULL ; +unsigned long tdr_pa =0 ; +unsigned long va; +int ret, i; +u64 err; + + ret =tdx_guest_keyid_alloc (); +if (ret <0 ) +return ret; + kvm_tdx->hkid = ret; + + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) + goto free_hkid; + tdr_pa =__pa (va); + + tdcs_pa =kcalloc (tdx_info->nr_tdcs_pages,sizeof (*kvm_tdx->tdcs_pa), +GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!tdcs_pa) + goto free_tdr; +for (i =0 ; i < tdx_info->nr_tdcs_pages; i++) { + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) + goto free_tdcs; + tdcs_pa[i] =__pa (va); + } + +if (!zalloc_cpumask_var (&packages,GFP_KERNEL )) { + ret = -ENOMEM ; + goto free_tdcs; + } +cpus_read_lock (); +/* +* Need at least one CPU of the package to be online in order to +* program all packages for host key id. Check it. +*/ +for_each_present_cpu (i) +cpumask_set_cpu (topology_physical_package_id (i), packages); +for_each_online_cpu (i) +cpumask_clear_cpu (topology_physical_package_id (i), packages);
arch/x86/kvm/vmx/tdx.c static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) · line 515
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+if (!cpumask_empty (packages)) { + ret = -EIO ; +/* +* Because it's hard for human operator to figure out the +* reason, warn it. +*/ +#define MSG_ALLPKG "All packages need to have online CPU to create TD. Online CPU and retry.\n" +pr_warn_ratelimited (MSG_ALLPKG ); + goto free_packages; + }
Generate/release hkid both requests to have "cpumask of at least 1 cpu per each node", how about add one helper for this ? The helper also checks the cpus_read_lock() is held and return the cpumask if at least 1 cpu is online per node, thus this init funciotn can be simplified and become more easy to review.
Quoted 234 lines
arch/x86/kvm/vmx/tdx.c +135
-1+ +/* +* Acquire global lock to avoid TDX_OPERAND_BUSY: +* TDH.MNG.CREATE and other APIs try to lock the global Key Owner +* Table (KOT) to track the assigned TDX private HKID. It doesn't spin +* to acquire the lock, returns TDX_OPERAND_BUSY instead, and let the +* caller to handle the contention. This is because of time limitation +* usable inside the TDX module and OS/VMM knows better about process +* scheduling. +* +* APIs to acquire the lock of KOT: +* TDH.MNG.CREATE, TDH.MNG.KEY.FREEID, TDH.MNG.VPFLUSHDONE, and +* TDH.PHYMEM.CACHE.WB. +*/ +mutex_lock (&tdx_lock); + err =tdh_mng_create (tdr_pa, kvm_tdx->hkid); +mutex_unlock (&tdx_lock); +if (err ==TDX_RND_NO_ENTROPY ) { + ret = -EAGAIN ; + goto free_packages; + } +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_CREATE , err,NULL ); + ret = -EIO ; + goto free_packages; + } + kvm_tdx->tdr_pa = tdr_pa; + +for_each_online_cpu (i) { +int pkg =topology_physical_package_id (i); + +if (cpumask_test_and_set_cpu (pkg, packages)) +continue ; + +/* +* Program the memory controller in the package with an +* encryption key associated to a TDX private host key id +* assigned to this TDR. Concurrent operations on same memory +* controller results in TDX_OPERAND_BUSY. Avoid this race by +* mutex. +*/ +mutex_lock (&tdx_mng_key_config_lock[pkg]); + ret =smp_call_on_cpu (i, tdx_do_tdh_mng_key_config, + &kvm_tdx->tdr_pa, true); +mutex_unlock (&tdx_mng_key_config_lock[pkg]); +if (ret) +break ; + } +cpus_read_unlock (); +free_cpumask_var (packages); +if (ret) { + i =0 ; + goto teardown; + } + + kvm_tdx->tdcs_pa = tdcs_pa; +for (i =0 ; i < tdx_info->nr_tdcs_pages; i++) { + err =tdh_mng_addcx (kvm_tdx->tdr_pa, tdcs_pa[i]); +if (err ==TDX_RND_NO_ENTROPY ) { +/* Here it's hard to allow userspace to retry. */ + ret = -EBUSY ; + goto teardown; + } +if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_ADDCX , err,NULL ); + ret = -EIO ; + goto teardown; + } + } + +/* +* Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated +* ioctl() to define the configure CPUID values for the TD. +*/ +return 0 ; + +/* +* The sequence for freeing resources from a partially initialized TD +* varies based on where in the initialization flow failure occurred. +* Simply use the full teardown and destroy, which naturally play nice +* with partial initialization. +*/ +teardown: +for (; i < tdx_info->nr_tdcs_pages; i++) { +if (tdcs_pa[i]) { +free_page ((unsigned long )__va (tdcs_pa[i])); + tdcs_pa[i] =0 ; + } + } +if (!kvm_tdx->tdcs_pa) +kfree (tdcs_pa); +tdx_mmu_release_hkid (kvm); +tdx_vm_free (kvm); +return ret; + +free_packages: +cpus_read_unlock (); +free_cpumask_var (packages); +free_tdcs: +for (i =0 ; i < tdx_info->nr_tdcs_pages; i++) { +if (tdcs_pa[i]) +free_page ((unsigned long )__va (tdcs_pa[i])); + } +kfree (tdcs_pa); + kvm_tdx->tdcs_pa =NULL ; + +free_tdr: +if (tdr_pa) +free_page ((unsigned long )__va (tdr_pa)); + kvm_tdx->tdr_pa =0 ; +free_hkid: +if (is_hkid_assigned (kvm_tdx)) +tdx_hkid_free (kvm_tdx); +return ret; +} +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {struct kvm_tdx_cmd tdx_cmd;@@ -212,12 +660,13 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)static int __init tdx_module_setup (void ) { -u16 num_cpuid_config; +u16 num_cpuid_config, tdcs_base_size;int ret;u32 i;struct tdx_md_map mds[] = {TDX_MD_MAP (NUM_CPUID_CONFIG , &num_cpuid_config), +TDX_MD_MAP (TDCS_BASE_SIZE , &tdcs_base_size), };#define TDX_INFO_MAP (_field_id, _member) \@@ -272,6 +721,8 @@ static int __init tdx_module_setup(void) c->edx = ecx_edx >>32 ; } + tdx_info->nr_tdcs_pages = tdcs_base_size /PAGE_SIZE ; +return 0 ; error_sys_rd:@@ -319,13 +770,27 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)struct vmx_tdx_enabled vmx_tdx = { .err =ATOMIC_INIT (0 ), }; +int max_pkgs;int r =0 ; +int i; +if (!cpu_feature_enabled (X86_FEATURE_MOVDIR64B )) { +pr_warn ("MOVDIR64B is reqiured for TDX\n" ); +return -EOPNOTSUPP ; + }if (!enable_ept) {pr_warn ("Cannot enable TDX with EPT disabled\n" );return -EINVAL ; } + max_pkgs =topology_max_packages (); + tdx_mng_key_config_lock =kcalloc (max_pkgs,sizeof (*tdx_mng_key_config_lock), +GFP_KERNEL ); +if (!tdx_mng_key_config_lock) +return -ENOMEM ; +for (i =0 ; i < max_pkgs; i++) +mutex_init (&tdx_mng_key_config_lock[i]); +if (!zalloc_cpumask_var (&vmx_tdx.vmx_enabled,GFP_KERNEL )) { r = -ENOMEM ; goto out;@@ -350,4 +815,5 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops)void tdx_hardware_unsetup (void ) {kfree (tdx_info); +kfree (tdx_mng_key_config_lock); }arch/x86/kvm/vmx/tdx.h +5
-1@@ -8,7 +8,11 @@struct kvm_tdx {struct kvm kvm; -/* TDX specific members follow. */ + +unsigned long tdr_pa; +unsigned long *tdcs_pa; + +int hkid; };struct vcpu_tdx {arch/x86/kvm/vmx/x86_ops.h +6
-0@@ -140,6 +140,9 @@ void tdx_hardware_unsetup(void);bool tdx_is_vm_type_supported (unsigned long type);int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap); +int tdx_vm_init (struct kvm *kvm); +void tdx_mmu_release_hkid (struct kvm *kvm); +void tdx_vm_free (struct kvm *kvm);int tdx_vm_ioctl (struct kvm *kvm,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }@@ -150,6 +153,9 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) {return -EINVAL ; }; +static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; } +static inline void tdx_mmu_release_hkid (struct kvm *kvm) {} +static inline void tdx_vm_free (struct kvm *kvm) {}static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; }#endif arch/x86/kvm/x86.c +1
-1@@ -12714,6 +12714,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)kvm_page_track_cleanup (kvm);kvm_xen_destroy_vm (kvm);kvm_hv_destroy_vm (kvm); +static_call_cond (kvm_x86_vm_free)(kvm); }static void memslot_rmap_free (struct kvm_memory_slot *slot) --2.25.1
Re: [PATCH v18 024/121] KVM: TDX: create/destroy VM structure
On Thu, Feb 01, 2024 at 04:32:27PM +0800,
Yuan Yao <[email protected]> wrote:
...
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c +8
-0+static int __tdx_reclaim_page (hpa_t pa) +{ +struct tdx_module_args out; +u64 err; + + do { + err = tdh_phymem_page_reclaim(pa, &out); + /*
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) · line 203
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+* TDH.PHYMEM.PAGE.RECLAIM is allowed only when TD is shutdown. +* state. i.e. destructing TD. +* TDH.PHYMEM.PAGE.RECLAIM requires TDR and target page. +* Because we're destructing TD, it's rare to contend with TDR. +*/ + }while (unlikely (err == (TDX_OPERAND_BUSY |TDX_OPERAND_ID_RCX )));v16 changed to tdx module 1.5, so here should be
TDX_OPERAND_ID_TDR, value 128ULL.
We should handle both RCX(SEPT) and TDR. So I make it err == RCX || err == TDR.
...
Quoted 52 lines
Quoted 52 lines
arch/x86/kvm/vmx/tdx.c +52
-0+static int __tdx_td_init (struct kvm *kvm) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +cpumask_var_t packages; +unsigned long *tdcs_pa =NULL ; +unsigned long tdr_pa =0 ; +unsigned long va; +int ret, i; +u64 err; + + ret =tdx_guest_keyid_alloc (); +if (ret <0 ) +return ret; + kvm_tdx->hkid = ret; + + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) + goto free_hkid; + tdr_pa =__pa (va); + + tdcs_pa =kcalloc (tdx_info->nr_tdcs_pages,sizeof (*kvm_tdx->tdcs_pa), +GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!tdcs_pa) + goto free_tdr; +for (i =0 ; i < tdx_info->nr_tdcs_pages; i++) { + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) + goto free_tdcs; + tdcs_pa[i] =__pa (va); + } + +if (!zalloc_cpumask_var (&packages,GFP_KERNEL )) { + ret = -ENOMEM ; + goto free_tdcs; + } +cpus_read_lock (); +/* +* Need at least one CPU of the package to be online in order to +* program all packages for host key id. Check it. +*/ +for_each_present_cpu (i) +cpumask_set_cpu (topology_physical_package_id (i), packages); +for_each_online_cpu (i) +cpumask_clear_cpu (topology_physical_package_id (i), packages); +if (!cpumask_empty (packages)) { + ret = -EIO ; +/* +* Because it's hard for human operator to figure out the +* reason, warn it. +*/ +#define MSG_ALLPKG "All packages need to have online CPU to create TD. Online CPU and retry.\n" +pr_warn_ratelimited (MSG_ALLPKG );
Quoted 8 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ goto free_packages; + }Generate/release hkid both requests to have "cpumask of at least 1 cpu per each node", how about add one helper for this ? The helper also checks the
cpus_read_lock()is held and return the cpumask if at least 1 cpu is online per node, thus this init funciotn can be simplified and become more easy to review.
We don't need cpumask to release hkid. So only tdx_td_init() needs cpumask allocation. So I didn't to bother create a helper function with v19.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 25/121 KVM: TDX: initialize VM with TDX specific parameters
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect the confidentiality of its memory contents and CPU state from any other software, including VMM. When creating a guest TD VM before creating vcpu, the number of vcpu, TSC frequency (the values are the same among vcpus, and it can't change.) CPUIDs which the TDX module emulates. Guest TDs can trust those CPUIDs and sha384 values for measurement.
Add a new subcommand, KVM_TDX_INIT_VM, to pass parameters for the TDX guest. It assigns an encryption key to the TDX guest for memory encryption. TDX encrypts memory per guest basis. The device model, say qemu, passes per-VM parameters for the TDX guest. The maximum number of vcpus, TSC frequency (TDX guest has fixed VM-wide TSC frequency, not per vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which configure guest XCR0, IA32_XSS MSR), CPUIDs, sha384 measurements, etc.
Call this subcommand before creating vcpu and KVM_SET_CPUID2, i.e. CPUID configurations aren't available yet. So CPUIDs configuration values need to be passed in struct kvm_tdx_init_vm. The device model's responsibility to make this CPUID config for KVM_TDX_INIT_VM and KVM_SET_CPUID2.
---
v18:
- remove the change of tools/arch/x86/include/uapi/asm/kvm.h
- typo in comment. sha348 => sha384
- updated comment in setup_tdparams_xfam()
- fix setup_tdparams_xfam() to use init_vm instead of td_params
v15 -> v16:
- Removed AMX check as the KVM upstream supports AMX.
- Added CET flag to guest supported xss
v14 -> v15:
- add check if the reserved area of init_vm is zero
--- arch/x86/include/uapi/asm/kvm.h | 27 ++++ arch/x86/kvm/cpuid.c | 7 + arch/x86/kvm/cpuid.h | 2 + arch/x86/kvm/vmx/tdx.c | 261 ++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.h | 18 +++ arch/x86/kvm/vmx/tdx_arch.h | 6 + 6 files changed, 311 insertions(+), 10 deletions(-)
arch/x86/include/uapi/asm/kvm.h +27 -0
@@ -570,6 +570,7 @@ struct kvm_pmu_event_filter {/* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM ,KVM_TDX_CMD_NR_MAX , };
@@ -621,4 +622,30 @@ struct kvm_tdx_capabilities { struct kvm_tdx_cpuid_config cpuid_configs[]; }; +struct kvm_tdx_init_vm { +__u64 attributes; +__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha384 digest */ +/* +* For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB. +* This should be enough given sizeof(TD_PARAMS) = 1024. +* 8KB was chosen given because +* sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB. +*/ +__u64 reserved[1004 ]; + +/* +* Call KVM_TDX_INIT_VM before vcpu creation, thus before +* KVM_SET_CPUID2. +* This configuration supersedes KVM_SET_CPUID2s for VCPUs because the +* TDX module directly virtualizes those CPUIDs without VMM. The user +* space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with +* those values. If it doesn't, KVM may have wrong idea of vCPUIDs of +* the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX +* module doesn't virtualize. +*/ +struct kvm_cpuid2 cpuid; +}; +#endif /* _ASM_X86_KVM_H */
arch/x86/kvm/cpuid.c +7 -0
@@ -1443,6 +1443,13 @@ int kvm_dev_ioctl_get_cpuid(struct kvm_cpuid2 *cpuid,return r; } +struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2 ( +struct kvm_cpuid_entry2 *entries,int nent,u32 function,u64 index) +{ +return cpuid_entry2_find (entries, nent, function, index); +} +EXPORT_SYMBOL_GPL (kvm_find_cpuid_entry2); +struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index (struct kvm_vcpu *vcpu,u32 function,u32 index) {
arch/x86/kvm/cpuid.h +2 -0
@@ -13,6 +13,8 @@ void kvm_set_cpu_caps(void);void kvm_update_cpuid_runtime (struct kvm_vcpu *vcpu);void kvm_update_pv_runtime (struct kvm_vcpu *vcpu); +struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2 (struct kvm_cpuid_entry2 *entries, +int nent,u32 function,u64 index);struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index (struct kvm_vcpu *vcpu,u32 function,u32 index); struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu,
arch/x86/kvm/vmx/tdx.c +251 -10
313 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +18 -0
@@ -12,7 +12,11 @@ struct kvm_tdx {unsigned long tdr_pa;unsigned long *tdcs_pa; +u64 attributes; +u64 xfam;int hkid; + +u64 tsc_offset; };struct vcpu_tdx {
@@ -39,6 +43,20 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_tdx, vcpu); } + +static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) +{ + struct tdx_module_args out; + u64 err; + + err = tdh_mng_rd(kvm_tdx->tdr_pa, TDCS_EXEC(field), &out); + if (unlikely(err)) { + pr_err("TDH_MNG_RD[EXEC.0x%x] failed: 0x%llx\n", field, err); + return 0; + } + return out.r8; +} + #else struct kvm_tdx { struct kvm kvm;
arch/x86/kvm/vmx/tdx_arch.h +6 -0
@@ -123,6 +123,12 @@ struct tdx_cpuid_value {#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31)#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) +/* +* TODO: Once XFEATURE_CET_{U, S} in arch/x86/include/asm/fpu/types.h is +* defined, Replace these with define ones. +*/ +#define TDX_TD_XFAM_CET (BIT(11) | BIT(12)) +/* * TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. */
Signature · 1 line
2.25.1
Re: [PATCH v18 025/121] KVM: TDX: initialize VM with TDX specific parameters
Quoted 48 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect the confidentiality of its memory contents and CPU state from any other software, including VMM. When creating a guest TD VM before creating vcpu, the number of vcpu, TSC frequency (the values are the same among vcpus, and it can't change.) CPUIDs which the TDX module emulates. Guest TDs can trust those CPUIDs and sha384 values for measurement.
Add a new subcommand,
KVM_TDX_INIT_VM, to pass parameters for the TDX guest. It assigns an encryption key to the TDX guest for memory encryption. TDX encrypts memory per guest basis. The device model, say qemu, passes per-VM parameters for the TDX guest. The maximum number of vcpus, TSC frequency (TDX guest has fixed VM-wide TSC frequency, not per vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which configure guest XCR0,IA32_XSSMSR), CPUIDs, sha384 measurements, etc.Call this subcommand before creating vcpu and
KVM_SET_CPUID2, i.e. CPUID configurations aren't available yet. So CPUIDs configuration values need to be passed in structkvm_tdx_init_vm. The device model's responsibility to make this CPUID config forKVM_TDX_INIT_VMandKVM_SET_CPUID2.Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- remove the change oftools/arch/x86/include/uapi/asm/kvm.h
- typo in comment. sha348 => sha384
- updated comment insetup_tdparams_xfam()
- fixsetup_tdparams_xfam()to useinit_vminstead oftd_paramsv15 -> v16:
- Removed AMX check as the KVM upstream supports AMX.
- Added CET flag to guest supported xssv14 -> v15:
- add check if the reserved area ofinit_vmis zero--- arch/x86/include/uapi/asm/kvm.h | 27 ++++ arch/x86/kvm/cpuid.c | 7 + arch/x86/kvm/cpuid.h | 2 + arch/x86/kvm/vmx/tdx.c | 261 ++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.h | 18 +++ arch/x86/kvm/vmx/tdx_arch.h | 6 + 6 files changed, 311 insertions(+), 10 deletions(-)
[...]
Quoted 15 lines
arch/x86/kvm/vmx/tdx.c +15
-0+ +static int setup_tdparams_xfam (struct kvm_cpuid2 *cpuid,struct td_params *td_params) +{ +const struct kvm_cpuid_entry2 *entry; +u64 guest_supported_xcr0; +u64 guest_supported_xss; + +/* Setup td_params.xfam */ + entry =kvm_find_cpuid_entry2 (cpuid->entries, cpuid->nent,0xd ,0 ); +if (entry) + guest_supported_xcr0 = (entry->eax | ((u64 )entry->edx <<32 )); +else + guest_supported_xcr0 =0 ; + guest_supported_xcr0 &= kvm_caps.supported_xcr0; +
arch/x86/kvm/vmx/tdx.c static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) · line 552
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ entry =kvm_find_cpuid_entry2 (cpuid->entries, cpuid->nent,0xd ,1 ); +if (entry) + guest_supported_xss = (entry->ecx | ((u64 )entry->edx <<32 )); +else + guest_supported_xss =0 ; + +/* +* PT can be exposed to TD guest regardless of KVM's XSS and CET +* support. +*/
According to the code below, it seems that both PT and CET can be exposed to TD guest regardless of KVM's XSS support?
Quoted 27 lines
arch/x86/kvm/vmx/tdx.c +27
-0+ guest_supported_xss &= + (kvm_caps.supported_xss |XFEATURE_MASK_PT |TDX_TD_XFAM_CET ); + + td_params->xfam = guest_supported_xcr0 | guest_supported_xss; +if (td_params->xfam &XFEATURE_MASK_LBR ) { +/* +* TODO: once KVM supports LBR(save/restore LBR related +* registers around TDENTER), remove this guard. +*/ +#define MSG_LBR "TD doesn't support LBR yet. KVM needs to save/restore IA32_LBR_DEPTH properly.\n" +pr_warn (MSG_LBR ); +return -EOPNOTSUPP ; + } + +return 0 ; +} + +static int setup_tdparams (struct kvm *kvm,struct td_params *td_params, +struct kvm_tdx_init_vm *init_vm) +{ +struct kvm_cpuid2 *cpuid = &init_vm->cpuid; +int ret; + +if (kvm->created_vcpus) +return -EBUSY ; + +if (init_vm->attributes &TDX_TD_ATTRIBUTE_PERFMON ) {
arch/x86/kvm/vmx/tdx.c static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) · line 589
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+/* +* TODO: save/restore PMU related registers around TDENTER. +* Once it's done, remove this guard. +*/ +#define MSG_PERFMON "TD doesn't support perfmon yet. KVM needs to save/restore host perf registers properly.\n" +pr_warn (MSG_PERFMON ); +return -EOPNOTSUPP ; + } + + td_params->max_vcpus = kvm->max_vcpus;
Can the max vcpu number be passed by KVM_TDX_INIT_VM? So that no need to add KVM_CAP_MAX_VCPUS in patch 23/121.
Quoted 209 lines
arch/x86/kvm/vmx/tdx.c +118
-4+ td_params->attributes = init_vm->attributes; + td_params->tsc_frequency =TDX_TSC_KHZ_TO_25MHZ (kvm->arch.default_tsc_khz); + + ret =setup_tdparams_eptp_controls (cpuid, td_params); +if (ret) +return ret; +setup_tdparams_cpuids (cpuid, td_params); + ret =setup_tdparams_xfam (cpuid, td_params); +if (ret) +return ret; + +#define MEMCPY_SAME_SIZE (dst, src) \ + do { \ + BUILD_BUG_ON(sizeof(dst) != sizeof(src)); \ + memcpy((dst), (src), sizeof(dst)); \ + } while (0) + +MEMCPY_SAME_SIZE (td_params->mrconfigid, init_vm->mrconfigid); +MEMCPY_SAME_SIZE (td_params->mrowner, init_vm->mrowner); +MEMCPY_SAME_SIZE (td_params->mrownerconfig, init_vm->mrownerconfig); + +return 0 ; +} + +static int __tdx_td_init (struct kvm *kvm,struct td_params *td_params, +u64 *seamcall_err) {struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +struct tdx_module_args out;cpumask_var_t packages;unsigned long *tdcs_pa =NULL ;unsigned long tdr_pa =0 ;@@ -469,6 +623,7 @@ static int __tdx_td_init(struct kvm *kvm)int ret, i;u64 err; + *seamcall_err =0 ; ret =tdx_guest_keyid_alloc ();if (ret <0 )return ret;@@ -583,10 +738,23 @@ static int __tdx_td_init(struct kvm *kvm) } } -/* -* Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated -* ioctl() to define the configure CPUID values for the TD. -*/ + err =tdh_mng_init (kvm_tdx->tdr_pa,__pa (td_params), &out); +if ((err &TDX_SEAMCALL_STATUS_MASK ) ==TDX_OPERAND_INVALID ) { +/* +* Because a user gives operands, don't warn. +* Return a hint to the user because it's sometimes hard for the +* user to figure out which operand is invalid. SEAMCALL status +* code includes which operand caused invalid operand error. +*/ + *seamcall_err = err; + ret = -EINVAL ; + goto teardown; + }else if (WARN_ON_ONCE (err)) { +pr_tdx_error (TDH_MNG_INIT , err, &out); + ret = -EIO ; + goto teardown; + } +return 0 ;/* @@ -629,6 +797,76 @@ static int __tdx_td_init(struct kvm *kvm)return ret; } +static int tdx_td_init (struct kvm *kvm,struct kvm_tdx_cmd *cmd) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (kvm); +struct kvm_tdx_init_vm *init_vm =NULL ; +struct td_params *td_params =NULL ; +int ret; + +BUILD_BUG_ON (sizeof (*init_vm) !=8 *1024 ); +BUILD_BUG_ON (sizeof (struct td_params ) !=1024 ); + +if (is_hkid_assigned (kvm_tdx)) +return -EINVAL ; + +if (cmd->flags) +return -EINVAL ; + + init_vm =kzalloc (sizeof (*init_vm) + +sizeof (init_vm->cpuid.entries[0 ]) *KVM_MAX_CPUID_ENTRIES , +GFP_KERNEL ); +if (!init_vm) +return -ENOMEM ; +if (copy_from_user (init_vm, (void __user *)cmd->data,sizeof (*init_vm))) { + ret = -EFAULT ; + goto out; + } +if (init_vm->cpuid.nent >KVM_MAX_CPUID_ENTRIES ) { + ret = -E2BIG ; + goto out; + } +if (copy_from_user (init_vm->cpuid.entries, + (void __user *)cmd->data +sizeof (*init_vm), +flex_array_size (init_vm, cpuid.entries, init_vm->cpuid.nent))) { + ret = -EFAULT ; + goto out; + } + +if (memchr_inv (init_vm->reserved,0 ,sizeof (init_vm->reserved))) { + ret = -EINVAL ; + goto out; + } +if (init_vm->cpuid.padding) { + ret = -EINVAL ; + goto out; + } + + td_params =kzalloc (sizeof (struct td_params ),GFP_KERNEL ); +if (!td_params) { + ret = -ENOMEM ; + goto out; + } + + ret =setup_tdparams (kvm, td_params, init_vm); +if (ret) + goto out; + + ret =__tdx_td_init (kvm, td_params, &cmd->error); +if (ret) + goto out; + + kvm_tdx->tsc_offset =td_tdcs_exec_read64 (kvm_tdx,TD_TDCS_EXEC_TSC_OFFSET ); + kvm_tdx->attributes = td_params->attributes; + kvm_tdx->xfam = td_params->xfam; + +out: +/* kfree() accepts NULL. */ +kfree (init_vm); +kfree (td_params); +return ret; +} +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {struct kvm_tdx_cmd tdx_cmd;@@ -645,6 +883,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)case KVM_TDX_CAPABILITIES : r =tdx_get_capabilities (&tdx_cmd);break ; +case KVM_TDX_INIT_VM : + r =tdx_td_init (kvm, &tdx_cmd); +break ;default : r = -EINVAL ; goto out;arch/x86/kvm/vmx/tdx.h +18
-0@@ -12,7 +12,11 @@ struct kvm_tdx {unsigned long tdr_pa;unsigned long *tdcs_pa; +u64 attributes; +u64 xfam;int hkid; + +u64 tsc_offset; };struct vcpu_tdx {@@ -39,6 +43,20 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_tdx, vcpu); } + +static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) +{ + struct tdx_module_args out; + u64 err; + + err = tdh_mng_rd(kvm_tdx->tdr_pa, TDCS_EXEC(field), &out); + if (unlikely(err)) { + pr_err("TDH_MNG_RD[EXEC.0x%x] failed: 0x%llx\n", field, err); + return 0; + } + return out.r8; +} + #else struct kvm_tdx { struct kvm kvm;arch/x86/kvm/vmx/tdx_arch.h +6
-0@@ -123,6 +123,12 @@ struct tdx_cpuid_value {#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31)#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) +/* +* TODO: Once XFEATURE_CET_{U, S} in arch/x86/include/asm/fpu/types.h is +* defined, Replace these with define ones. +*/ +#define TDX_TD_XFAM_CET (BIT(11) | BIT(12)) +/* * TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. */
Re: [PATCH v18 025/121] KVM: TDX: initialize VM with TDX specific parameters
Quoted 72 lines · Binbin Wu
Quoted 48 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX requires additional parameters for TDX VM for confidential execution to protect the confidentiality of its memory contents and CPU state from any other software, including VMM. When creating a guest TD VM before creating vcpu, the number of vcpu, TSC frequency (the values are the same among vcpus, and it can't change.) CPUIDs which the TDX module emulates. Guest TDs can trust those CPUIDs and sha384 values for measurement.
Add a new subcommand,
KVM_TDX_INIT_VM, to pass parameters for the TDX guest. It assigns an encryption key to the TDX guest for memory encryption. TDX encrypts memory per guest basis. The device model, say qemu, passes per-VM parameters for the TDX guest. The maximum number of vcpus, TSC frequency (TDX guest has fixed VM-wide TSC frequency, not per vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which configure guest XCR0,IA32_XSSMSR), CPUIDs, sha384 measurements, etc.Call this subcommand before creating vcpu and
KVM_SET_CPUID2, i.e. CPUID configurations aren't available yet. So CPUIDs configuration values need to be passed in structkvm_tdx_init_vm. The device model's responsibility to make this CPUID config forKVM_TDX_INIT_VMandKVM_SET_CPUID2.Signed-off-by Xiaoyao Li <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- remove the change oftools/arch/x86/include/uapi/asm/kvm.h
- typo in comment. sha348 => sha384
- updated comment insetup_tdparams_xfam()
- fixsetup_tdparams_xfam()to useinit_vminstead oftd_paramsv15 -> v16:
- Removed AMX check as the KVM upstream supports AMX.
- Added CET flag to guest supported xssv14 -> v15:
- add check if the reserved area ofinit_vmis zero--- arch/x86/include/uapi/asm/kvm.h | 27 ++++ arch/x86/kvm/cpuid.c | 7 + arch/x86/kvm/cpuid.h | 2 + arch/x86/kvm/vmx/tdx.c | 261 ++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.h | 18 +++ arch/x86/kvm/vmx/tdx_arch.h | 6 + 6 files changed, 311 insertions(+), 10 deletions(-)[...]
Quoted 20 lines
arch/x86/kvm/vmx/tdx.c +20
-0+ +static int setup_tdparams_xfam (struct kvm_cpuid2 *cpuid,struct td_params *td_params) +{ +const struct kvm_cpuid_entry2 *entry; +u64 guest_supported_xcr0; +u64 guest_supported_xss; + +/* Setup td_params.xfam */ + entry =kvm_find_cpuid_entry2 (cpuid->entries, cpuid->nent,0xd ,0 ); +if (entry) + guest_supported_xcr0 = (entry->eax | ((u64 )entry->edx <<32 )); +else + guest_supported_xcr0 =0 ; + guest_supported_xcr0 &= kvm_caps.supported_xcr0; + + entry =kvm_find_cpuid_entry2 (cpuid->entries, cpuid->nent,0xd ,1 ); +if (entry) + guest_supported_xss = (entry->ecx | ((u64 )entry->edx <<32 )); +else + guest_supported_xss =0 ;
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) · line 552
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+ +/* +* PT can be exposed to TD guest regardless of KVM's XSS and CET +* support. +*/ According to the code below, it seems that both PT and CET can be exposed to TD
guest regardless of KVM's XSS support?
Yes, updated the comment.
Quoted 31 lines
Quoted 31 lines
arch/x86/kvm/vmx/tdx.c +31
-0+ guest_supported_xss &= + (kvm_caps.supported_xss |XFEATURE_MASK_PT |TDX_TD_XFAM_CET ); + + td_params->xfam = guest_supported_xcr0 | guest_supported_xss; +if (td_params->xfam &XFEATURE_MASK_LBR ) { +/* +* TODO: once KVM supports LBR(save/restore LBR related +* registers around TDENTER), remove this guard. +*/ +#define MSG_LBR "TD doesn't support LBR yet. KVM needs to save/restore IA32_LBR_DEPTH properly.\n" +pr_warn (MSG_LBR ); +return -EOPNOTSUPP ; + } + +return 0 ; +} + +static int setup_tdparams (struct kvm *kvm,struct td_params *td_params, +struct kvm_tdx_init_vm *init_vm) +{ +struct kvm_cpuid2 *cpuid = &init_vm->cpuid; +int ret; + +if (kvm->created_vcpus) +return -EBUSY ; + +if (init_vm->attributes &TDX_TD_ATTRIBUTE_PERFMON ) { +/* +* TODO: save/restore PMU related registers around TDENTER. +* Once it's done, remove this guard. +*/
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) · line 589
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+#define MSG_PERFMON "TD doesn't support perfmon yet. KVM needs to save/restore host perf registers properly.\n" +pr_warn (MSG_PERFMON ); +return -EOPNOTSUPP ; + } + + td_params->max_vcpus = kvm->max_vcpus;Can the max vcpu number be passed by
KVM_TDX_INIT_VM? So that no need to addKVM_CAP_MAX_VCPUSin patch 23/121.
Please see the comment there.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 26/121 KVM: TDX: Make pmu_intel.c ignore guest TD case
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case.
---
v18:
- Removed unnecessary change to vmx.c which caused kernel warning.
--- arch/x86/kvm/vmx/pmu_intel.c | 46 +++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/pmu_intel.h | 28 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 8 ++++++- arch/x86/kvm/vmx/vmx.h | 32 +------------------------ 4 files changed, 81 insertions(+), 33 deletions(-) create mode 100644 arch/x86/kvm/vmx/pmu_intel.h
arch/x86/kvm/vmx/pmu_intel.c +45 -1
@@ -19,6 +19,7 @@#include "lapic.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" #define MSR_PMC_FULL_WIDTH_BIT (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)
@@ -68,6 +69,26 @@ static int fixed_pmc_events[] = { [2] = PSEUDO_ARCH_REFERENCE_CYCLES, }; +struct lbr_desc *vcpu_to_lbr_desc (struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_INTEL_TDX_HOST +if (is_td_vcpu (vcpu)) +return &to_tdx (vcpu)->lbr_desc; +#endif + +return &to_vmx (vcpu)->lbr_desc; +} + +struct x86_pmu_lbr *vcpu_to_lbr_records (struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_INTEL_TDX_HOST +if (is_td_vcpu (vcpu)) +return &to_tdx (vcpu)->lbr_desc.records; +#endif + +return &to_vmx (vcpu)->lbr_desc.records; +} +static void reprogram_fixed_counters (struct kvm_pmu *pmu,u64 data) {struct kvm_pmc *pmc;
@@ -179,6 +200,23 @@ static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr)return get_gp_pmc (pmu, msr,MSR_IA32_PMC0 ); } +bool intel_pmu_lbr_is_compatible (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return false; +return cpuid_model_is_consistent (vcpu); +} + +bool intel_pmu_lbr_is_enabled (struct kvm_vcpu *vcpu) +{ +struct x86_pmu_lbr *lbr =vcpu_to_lbr_records (vcpu); + +if (is_td_vcpu (vcpu)) +return false; + +return lbr->nr && (vcpu_get_perf_capabilities (vcpu) &PMU_CAP_LBR_FMT ); +} +static bool intel_pmu_is_valid_lbr_msr (struct kvm_vcpu *vcpu,u32 index) {struct x86_pmu_lbr *records =vcpu_to_lbr_records (vcpu);
@@ -244,6 +282,9 @@ static inline void intel_pmu_release_guest_lbr_event(struct kvm_vcpu *vcpu) {struct lbr_desc *lbr_desc =vcpu_to_lbr_desc (vcpu); +if (is_td_vcpu (vcpu)) +return ; +if (lbr_desc->event) {perf_event_release_kernel (lbr_desc->event); lbr_desc->event =NULL ;
@@ -285,6 +326,9 @@ int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu) PERF_SAMPLE_BRANCH_USER, }; +if (WARN_ON_ONCE (is_td_vcpu (vcpu))) +return 0 ; +if (unlikely (lbr_desc->event)) {__set_bit (INTEL_PMC_IDX_FIXED_VLBR , pmu->pmc_in_use);return 0 ;
@@ -578,7 +622,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) INTEL_PMC_MAX_GENERIC, pmu->nr_arch_fixed_counters); perf_capabilities =vcpu_get_perf_capabilities (vcpu); -if (cpuid_model_is_consistent (vcpu) && +if (intel_pmu_lbr_is_compatible (vcpu) && (perf_capabilities &PMU_CAP_LBR_FMT ))x86_perf_get_lbr (&lbr_desc->records);else
arch/x86/kvm/vmx/pmu_intel.h new file +28 -0
@@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_VMX_PMU_INTEL_H +#define __KVM_X86_VMX_PMU_INTEL_H + +struct lbr_desc *vcpu_to_lbr_desc (struct kvm_vcpu *vcpu); +struct x86_pmu_lbr *vcpu_to_lbr_records (struct kvm_vcpu *vcpu); + +bool intel_pmu_lbr_is_compatible (struct kvm_vcpu *vcpu); +bool intel_pmu_lbr_is_enabled (struct kvm_vcpu *vcpu); +int intel_pmu_create_guest_lbr_event (struct kvm_vcpu *vcpu); + +struct lbr_desc { +/* Basic info about guest LBR records. */ +struct x86_pmu_lbr records; + +/* +* Emulate LBR feature via passthrough LBR registers when the +* per-vcpu guest LBR event is scheduled on the current pcpu. +* +* The records may be inaccurate if the host reclaims the LBR. +*/ +struct perf_event *event; + +/* True if LBRs are marked as not intercepted in the MSR bitmap */ +bool msr_passthrough; +}; + +#endif /* __KVM_X86_VMX_PMU_INTEL_H */
arch/x86/kvm/vmx/tdx.h +7 -1
@@ -4,6 +4,7 @@#ifdef CONFIG_INTEL_TDX_HOST +#include "pmu_intel.h" #include "tdx_ops.h" struct kvm_tdx {
@@ -21,7 +22,12 @@ struct kvm_tdx {struct vcpu_tdx {struct kvm_vcpu vcpu; -/* TDX specific members follow. */ + +/* +* Dummy to make pmu_intel not corrupt memory. +* TODO: Support PMU for TDX. Future work. +*/ +struct lbr_desc lbr_desc; };static inline bool is_td (struct kvm *kvm)
arch/x86/kvm/vmx/vmx.h +1 -31
@@ -11,6 +11,7 @@#include "capabilities.h" #include "../kvm_cache_regs.h" #include "posted_intr.h" +#include "pmu_intel.h" #include "vmcs.h" #include "vmx_ops.h" #include "../cpuid.h"
@@ -93,22 +94,6 @@ union vmx_exit_reason {u32 full; }; -struct lbr_desc { -/* Basic info about guest LBR records. */ -struct x86_pmu_lbr records; - -/* -* Emulate LBR feature via passthrough LBR registers when the -* per-vcpu guest LBR event is scheduled on the current pcpu. -* -* The records may be inaccurate if the host reclaims the LBR. -*/ -struct perf_event *event; - -/* True if LBRs are marked as not intercepted in the MSR bitmap */ -bool msr_passthrough; -}; -/* * The nested_vmx structure is part of vcpu_vmx, and holds information we need * for correct emulation of VMX (i.e., nested VMX) on this vcpu.
@@ -659,21 +644,6 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); } -static inline struct lbr_desc *vcpu_to_lbr_desc (struct kvm_vcpu *vcpu) -{ -return &to_vmx (vcpu)->lbr_desc; -} - -static inline struct x86_pmu_lbr *vcpu_to_lbr_records (struct kvm_vcpu *vcpu) -{ -return &vcpu_to_lbr_desc (vcpu)->records; -} - -static inline bool intel_pmu_lbr_is_enabled (struct kvm_vcpu *vcpu) -{ -return !!vcpu_to_lbr_records (vcpu)->nr; -} -void intel_pmu_cross_mapped_check (struct kvm_pmu *pmu);int intel_pmu_create_guest_lbr_event (struct kvm_vcpu *vcpu);void vmx_passthrough_lbr_msrs (struct kvm_vcpu *vcpu);
Signature · 1 line
2.25.1
PATCH v18 27/121 KVM: TDX: Refuse to unplug the last cpu on the package
In reply to [email protected]From: Isaku Yamahata <[email protected]>
In order to reclaim TDX HKID, (i.e. when deleting guest TD), needs to call TDH.PHYMEM.PAGE.WBINVD on all packages. If we have active TDX HKID, refuse to offline the last online cpu to guarantee at least one CPU online per package. Add arch callback for cpu offline.
Because TDX doesn't support suspend, this also refuses suspend if TDs are running. If no TD is running, suspend is allowed.
---
v18:
- Added reviewed-by BinBin
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/main.c | 1 + arch/x86/kvm/vmx/tdx.c | 41 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ arch/x86/kvm/x86.c | 5 ++++ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 12 +++++++-- 8 files changed, 62 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -18,6 +18,7 @@ KVM_X86_OP(check_processor_compatibility) KVM_X86_OP(hardware_enable) KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) +KVM_X86_OP_OPTIONAL_RET0(offline_cpu) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1598,6 +1598,7 @@ struct kvm_x86_ops {int (*hardware_enable)(void );void (*hardware_disable)(void );void (*hardware_unsetup)(void ); +int (*offline_cpu)(void );bool (*has_emulated_msr)(struct kvm *kvm,u32 index);void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu);
arch/x86/kvm/vmx/main.c +1 -0
@@ -125,6 +125,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_processor_compatibility = vmx_check_processor_compat, .hardware_unsetup = vt_hardware_unsetup, + .offline_cpu = tdx_offline_cpu, .hardware_enable = vt_hardware_enable, .hardware_disable = vmx_hardware_disable,
arch/x86/kvm/vmx/tdx.c +41 -0
62 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +2 -0
@@ -138,6 +138,7 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu);int __init tdx_hardware_setup (struct kvm_x86_ops *x86_ops);void tdx_hardware_unsetup (void );bool tdx_is_vm_type_supported (unsigned long type); +int tdx_offline_cpu (void );int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap);int tdx_vm_init (struct kvm *kvm);
@@ -148,6 +149,7 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}static inline bool tdx_is_vm_type_supported (unsigned long type) {return false; } +static inline int tdx_offline_cpu (void ) {return 0 ; }static inline int tdx_vm_enable_cap (struct kvm *kvm,struct kvm_enable_cap *cap) {
arch/x86/kvm/x86.c +5 -0
@@ -12479,6 +12479,11 @@ void kvm_arch_hardware_disable(void)drop_user_return_notifiers (); } +int kvm_arch_offline_cpu (unsigned int cpu) +{ +return static_call (kvm_x86_offline_cpu)(); +} +bool kvm_vcpu_is_reset_bsp (struct kvm_vcpu *vcpu) {return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id;
include/linux/kvm_host.h +1 -0
@@ -1502,6 +1502,7 @@ static inline void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu) {}int kvm_arch_hardware_enable (void );void kvm_arch_hardware_disable (void );#endif +int kvm_arch_offline_cpu (unsigned int cpu);int kvm_arch_vcpu_runnable (struct kvm_vcpu *vcpu);bool kvm_arch_vcpu_in_kernel (struct kvm_vcpu *vcpu);int kvm_arch_vcpu_should_kick (struct kvm_vcpu *vcpu);
virt/kvm/kvm_main.c +10 -2
@@ -5586,13 +5586,21 @@ static void hardware_disable_nolock(void *junk)__this_cpu_write (hardware_enabled, false); } +__weak int kvm_arch_offline_cpu (unsigned int cpu) +{ +return 0 ; +} +static int kvm_offline_cpu (unsigned int cpu) { +int r =0 ; +mutex_lock (&kvm_lock); -if (kvm_usage_count) + r =kvm_arch_offline_cpu (cpu); +if (!r && kvm_usage_count)hardware_disable_nolock (NULL );mutex_unlock (&kvm_lock); -return 0 ; +return r; }static void hardware_disable_all_nolock (void )
Signature · 1 line
2.25.1
PATCH v18 28/121 The start of TDX KVM patch series: TD vcpu creation/destruction
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of TD vcpu creation/destruction.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +3 -3
@@ -9,7 +9,7 @@ Layer status What qemu can do ---------------- - TDX VM TYPE is exposed to Qemu. -- Qemu can try to create VM of TDX VM type and then fails. +- Qemu can create/destroy guest of TDX vm type. Patch Layer status ------------------
@@ -17,8 +17,8 @@ Patch Layer status * TDX, VMX coexistence: Applied * TDX architectural definitions: Applied -* TD VM creation/destruction: Applying -* TD vcpu creation/destruction: Not yet +* TD VM creation/destruction: Applied +* TD vcpu creation/destruction: Applying * TDX EPT violation: Not yet * TD finalization: Not yet * TD vcpu enter/exit: Not yet
Signature · 1 line
2.25.1
PATCH v18 29/121 KVM: TDX: create/free TDX vcpu structure
In reply to [email protected]From: Isaku Yamahata <[email protected]>
The next step of TDX guest creation is to create vcpu. Create TDX vcpu structures, initialize it that doesn't require TDX SEAMCALL. TDX specific vcpu initialization will be implemented as independent KVM_TDX_INIT_VCPU so that when error occurs it's easy to determine which component has the issue, KVM or TDX.
---
v18:
- update commit log to use create instead of allocate because the patch doesn't newly allocate memory for TDX vcpu.
v15 -> v16:
- Add AMX support as the KVM upstream supports it.
--- arch/x86/kvm/vmx/main.c | 44 ++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 49 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 10 ++++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 101 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx/main.c +40 -4
@@ -102,6 +102,42 @@ static void vt_vm_free(struct kvm *kvm)tdx_vm_free (kvm); } +static int vt_vcpu_precreate (struct kvm *kvm) +{ +if (is_td (kvm)) +return 0 ; + +return vmx_vcpu_precreate (kvm); +} + +static int vt_vcpu_create (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return tdx_vcpu_create (vcpu); + +return vmx_vcpu_create (vcpu); +} + +static void vt_vcpu_free (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) { +tdx_vcpu_free (vcpu); +return ; + } + +vmx_vcpu_free (vcpu); +} + +static void vt_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +{ +if (is_td_vcpu (vcpu)) { +tdx_vcpu_reset (vcpu, init_event); +return ; + } + +vmx_vcpu_reset (vcpu, init_event); +} +static int vt_mem_enc_ioctl (struct kvm *kvm,void __user *argp) {if (!is_td (kvm))
@@ -140,10 +176,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vm_destroy = vt_vm_destroy, .vm_free = vt_vm_free, - .vcpu_precreate = vmx_vcpu_precreate, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, + .vcpu_precreate = vt_vcpu_precreate, + .vcpu_create = vt_vcpu_create, + .vcpu_free = vt_vcpu_free, + .vcpu_reset = vt_vcpu_reset, .prepare_switch_to_guest = vmx_prepare_switch_to_guest, .vcpu_load = vmx_vcpu_load,
arch/x86/kvm/vmx/tdx.c +49 -0
55 lines not on this page · raw message
arch/x86/kvm/vmx/x86_ops.h +10 -0
@@ -144,7 +144,12 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);int tdx_vm_init (struct kvm *kvm);void tdx_mmu_release_hkid (struct kvm *kvm);void tdx_vm_free (struct kvm *kvm); +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp); + +int tdx_vcpu_create (struct kvm_vcpu *vcpu); +void tdx_vcpu_free (struct kvm_vcpu *vcpu); +void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}
@@ -158,7 +163,12 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; }static inline void tdx_mmu_release_hkid (struct kvm *kvm) {}static inline void tdx_vm_free (struct kvm *kvm) {} +static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; } + +static inline int tdx_vcpu_create (struct kvm_vcpu *vcpu) {return -EOPNOTSUPP ; } +static inline void tdx_vcpu_free (struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
arch/x86/kvm/x86.c +2 -0
@@ -502,6 +502,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)kvm_recalculate_apic_map (vcpu->kvm);return 0 ; } +EXPORT_SYMBOL_GPL (kvm_set_apic_base);/* * Handle a fault on a hardware virtualization (VMX or SVM) instruction.
@@ -12488,6 +12489,7 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) {return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id; } +EXPORT_SYMBOL_GPL (kvm_vcpu_is_reset_bsp);bool kvm_vcpu_is_bsp (struct kvm_vcpu *vcpu) {
Signature · 1 line
2.25.1
Re: [PATCH v18 029/121] KVM: TDX: create/free TDX vcpu structure
Quoted 90 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
The next step of TDX guest creation is to create vcpu. Create TDX vcpu structures, initialize it that doesn't require TDX SEAMCALL. TDX specific vcpu initialization will be implemented as independent
KVM_TDX_INIT_VCPUso that when error occurs it's easy to determine which component has the issue, KVM or TDX.Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- update commit log to use create instead of allocate because the patch doesn't newly allocate memory for TDX vcpu.v15 -> v16:
- Add AMX support as the KVM upstream supports it.--- arch/x86/kvm/vmx/main.c | 44 ++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 49 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 10 ++++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 101 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/main.c +40
-4@@ -102,6 +102,42 @@ static void vt_vm_free(struct kvm *kvm)tdx_vm_free (kvm); } +static int vt_vcpu_precreate (struct kvm *kvm) +{ +if (is_td (kvm)) +return 0 ; + +return vmx_vcpu_precreate (kvm); +} + +static int vt_vcpu_create (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return tdx_vcpu_create (vcpu); + +return vmx_vcpu_create (vcpu); +} + +static void vt_vcpu_free (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) { +tdx_vcpu_free (vcpu); +return ; + } + +vmx_vcpu_free (vcpu); +} + +static void vt_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +{ +if (is_td_vcpu (vcpu)) { +tdx_vcpu_reset (vcpu, init_event); +return ; + } + +vmx_vcpu_reset (vcpu, init_event); +} +static int vt_mem_enc_ioctl (struct kvm *kvm,void __user *argp) {if (!is_td (kvm))@@ -140,10 +176,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vm_destroy = vt_vm_destroy, .vm_free = vt_vm_free, - .vcpu_precreate = vmx_vcpu_precreate, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, + .vcpu_precreate = vt_vcpu_precreate, + .vcpu_create = vt_vcpu_create, + .vcpu_free = vt_vcpu_free, + .vcpu_reset = vt_vcpu_reset, .prepare_switch_to_guest = vmx_prepare_switch_to_guest, .vcpu_load = vmx_vcpu_load,arch/x86/kvm/vmx/tdx.c +0
-0
arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 420
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +7
-0@@ -411,6 +411,55 @@ int tdx_vm_init(struct kvm *kvm)return 0 ; } +int tdx_vcpu_create (struct kvm_vcpu *vcpu) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); + +/* +* On cpu creation, cpuid entry is blank. Forcibly enable +* X2APIC feature to allow X2APIC.
This comment is a bit confusing.
Do you mean force x2apic here or elsewhere?
So far, in this patch, x2apic is not forced yet.
arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 421
Quoted 1 line
arch/x86/kvm/vmx/tdx.c +1
-0+* Because vcpu_reset() can't return error, allocation is done here.
What do you mean "allocation" here?
Quoted 94 lines
arch/x86/kvm/vmx/tdx.c +41
-0+*/ +WARN_ON_ONCE (vcpu->arch.cpuid_entries); +WARN_ON_ONCE (vcpu->arch.cpuid_nent); + +/* TDX only supports x2APIC, which requires an in-kernel local APIC. */ +if (!vcpu->arch.apic) +return -EINVAL ; + +fpstate_set_confidential (&vcpu->arch.guest_fpu); + + vcpu->arch.efer =EFER_SCE |EFER_LME |EFER_LMA |EFER_NX ; + + vcpu->arch.cr0_guest_owned_bits =-1ul ; + vcpu->arch.cr4_guest_owned_bits =-1ul ; + + vcpu->arch.tsc_offset =to_kvm_tdx (vcpu->kvm)->tsc_offset; + vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset; + vcpu->arch.guest_state_protected = + !(to_kvm_tdx (vcpu->kvm)->attributes &TDX_TD_ATTRIBUTE_DEBUG ); + +if ((kvm_tdx->xfam &XFEATURE_MASK_XTILE ) ==XFEATURE_MASK_XTILE ) + vcpu->arch.xfd_no_write_intercept = true; + +return 0 ; +} + +void tdx_vcpu_free (struct kvm_vcpu *vcpu) +{ +/* This is stub for now. More logic will come. */ +} + +void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +{ + +/* Ignore INIT silently because TDX doesn't support INIT event. */ +if (init_event) +return ; + +/* This is stub for now. More logic will come here. */ +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;arch/x86/kvm/vmx/x86_ops.h +10
-0@@ -144,7 +144,12 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);int tdx_vm_init (struct kvm *kvm);void tdx_mmu_release_hkid (struct kvm *kvm);void tdx_vm_free (struct kvm *kvm); +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp); + +int tdx_vcpu_create (struct kvm_vcpu *vcpu); +void tdx_vcpu_free (struct kvm_vcpu *vcpu); +void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}@@ -158,7 +163,12 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; }static inline void tdx_mmu_release_hkid (struct kvm *kvm) {}static inline void tdx_vm_free (struct kvm *kvm) {} +static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; } + +static inline int tdx_vcpu_create (struct kvm_vcpu *vcpu) {return -EOPNOTSUPP ; } +static inline void tdx_vcpu_free (struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */ arch/x86/kvm/x86.c +2
-0@@ -502,6 +502,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)kvm_recalculate_apic_map (vcpu->kvm);return 0 ; } +EXPORT_SYMBOL_GPL (kvm_set_apic_base);/* * Handle a fault on a hardware virtualization (VMX or SVM) instruction. @@ -12488,6 +12489,7 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) {return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id; } +EXPORT_SYMBOL_GPL (kvm_vcpu_is_reset_bsp);bool kvm_vcpu_is_bsp (struct kvm_vcpu *vcpu) {
Re: [PATCH v18 029/121] KVM: TDX: create/free TDX vcpu structure
Quoted 19 lines · Binbin Wu
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +0
-0arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 419
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +7
-0@@ -411,6 +411,55 @@ int tdx_vm_init(struct kvm *kvm)return 0 ; } +int tdx_vcpu_create (struct kvm_vcpu *vcpu) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); + +/* +* On cpu creation, cpuid entry is blank. Forcibly enable +* X2APIC feature to allow X2APIC. This comment is a bit confusing.
Do you mean force x2apic here or elsewhere?
So far, in this patch, x2apic is not forced yet.
Quoted 3 lines
arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 421
Quoted 1 line
arch/x86/kvm/vmx/tdx.c +1
-0+* Because vcpu_reset() can't return error, allocation is done here. What do you mean "allocation" here?
Now this comment is stale, I removed it as Yuan pointed out.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v18 029/121] KVM: TDX: create/free TDX vcpu structure
In reply to [email protected]Quoted 92 lines · On Mon, Jan 22, 2024 at 03:53:05PM -0800, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
The next step of TDX guest creation is to create vcpu. Create TDX vcpu structures, initialize it that doesn't require TDX SEAMCALL. TDX specific vcpu initialization will be implemented as independent
KVM_TDX_INIT_VCPUso that when error occurs it's easy to determine which component has the issue, KVM or TDX.Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- update commit log to use create instead of allocate because the patch doesn't newly allocate memory for TDX vcpu.v15 -> v16:
- Add AMX support as the KVM upstream supports it.--- arch/x86/kvm/vmx/main.c | 44 ++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 49 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 10 ++++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 101 insertions(+), 4 deletions(-)arch/x86/kvm/vmx/main.c +40
-4@@ -102,6 +102,42 @@ static void vt_vm_free(struct kvm *kvm)tdx_vm_free (kvm); } +static int vt_vcpu_precreate (struct kvm *kvm) +{ +if (is_td (kvm)) +return 0 ; + +return vmx_vcpu_precreate (kvm); +} + +static int vt_vcpu_create (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) +return tdx_vcpu_create (vcpu); + +return vmx_vcpu_create (vcpu); +} + +static void vt_vcpu_free (struct kvm_vcpu *vcpu) +{ +if (is_td_vcpu (vcpu)) { +tdx_vcpu_free (vcpu); +return ; + } + +vmx_vcpu_free (vcpu); +} + +static void vt_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +{ +if (is_td_vcpu (vcpu)) { +tdx_vcpu_reset (vcpu, init_event); +return ; + } + +vmx_vcpu_reset (vcpu, init_event); +} +static int vt_mem_enc_ioctl (struct kvm *kvm,void __user *argp) {if (!is_td (kvm))@@ -140,10 +176,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vm_destroy = vt_vm_destroy, .vm_free = vt_vm_free, - .vcpu_precreate = vmx_vcpu_precreate, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, + .vcpu_precreate = vt_vcpu_precreate, + .vcpu_create = vt_vcpu_create, + .vcpu_free = vt_vcpu_free, + .vcpu_reset = vt_vcpu_reset, .prepare_switch_to_guest = vmx_prepare_switch_to_guest, .vcpu_load = vmx_vcpu_load,arch/x86/kvm/vmx/tdx.c +0
-0@@ -411,6 +411,55 @@ int tdx_vm_init(struct kvm *kvm)return 0 ; }
arch/x86/kvm/vmx/tdx.c int tdx_vm_init(struct kvm *kvm) · line 422
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +9
-0+int tdx_vcpu_create (struct kvm_vcpu *vcpu) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); + +/* +* On cpu creation, cpuid entry is blank. Forcibly enable +* X2APIC feature to allow X2APIC. +* Because vcpu_reset() can't return error, allocation is done here. +*/
These comments can be removed, the force x2apic now is done in KVM_TDX_INIT_VCPU, and no allocation inside this function, they happens also in KVM_TDX_INIT_VCPU IIUC.
Quoted 97 lines
arch/x86/kvm/vmx/tdx.c +40
-0+WARN_ON_ONCE (vcpu->arch.cpuid_entries); +WARN_ON_ONCE (vcpu->arch.cpuid_nent); + +/* TDX only supports x2APIC, which requires an in-kernel local APIC. */ +if (!vcpu->arch.apic) +return -EINVAL ; + +fpstate_set_confidential (&vcpu->arch.guest_fpu); + + vcpu->arch.efer =EFER_SCE |EFER_LME |EFER_LMA |EFER_NX ; + + vcpu->arch.cr0_guest_owned_bits =-1ul ; + vcpu->arch.cr4_guest_owned_bits =-1ul ; + + vcpu->arch.tsc_offset =to_kvm_tdx (vcpu->kvm)->tsc_offset; + vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset; + vcpu->arch.guest_state_protected = + !(to_kvm_tdx (vcpu->kvm)->attributes &TDX_TD_ATTRIBUTE_DEBUG ); + +if ((kvm_tdx->xfam &XFEATURE_MASK_XTILE ) ==XFEATURE_MASK_XTILE ) + vcpu->arch.xfd_no_write_intercept = true; + +return 0 ; +} + +void tdx_vcpu_free (struct kvm_vcpu *vcpu) +{ +/* This is stub for now. More logic will come. */ +} + +void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) +{ + +/* Ignore INIT silently because TDX doesn't support INIT event. */ +if (init_event) +return ; + +/* This is stub for now. More logic will come here. */ +} +static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd) {struct kvm_tdx_capabilities __user *user_caps;arch/x86/kvm/vmx/x86_ops.h +10
-0@@ -144,7 +144,12 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap);int tdx_vm_init (struct kvm *kvm);void tdx_mmu_release_hkid (struct kvm *kvm);void tdx_vm_free (struct kvm *kvm); +int tdx_vm_ioctl (struct kvm *kvm,void __user *argp); + +int tdx_vcpu_create (struct kvm_vcpu *vcpu); +void tdx_vcpu_free (struct kvm_vcpu *vcpu); +void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}@@ -158,7 +163,12 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)static inline int tdx_vm_init (struct kvm *kvm) {return -EOPNOTSUPP ; }static inline void tdx_mmu_release_hkid (struct kvm *kvm) {}static inline void tdx_vm_free (struct kvm *kvm) {} +static inline int tdx_vm_ioctl (struct kvm *kvm,void __user *argp) {return -EOPNOTSUPP ; } + +static inline int tdx_vcpu_create (struct kvm_vcpu *vcpu) {return -EOPNOTSUPP ; } +static inline void tdx_vcpu_free (struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {}#endif #endif /* __KVM_X86_VMX_X86_OPS_H */ arch/x86/kvm/x86.c +2
-1@@ -502,6 +502,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)kvm_recalculate_apic_map (vcpu->kvm);return 0 ; } +EXPORT_SYMBOL_GPL (kvm_set_apic_base);/* * Handle a fault on a hardware virtualization (VMX or SVM) instruction. @@ -12488,6 +12489,7 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) {return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id; } +EXPORT_SYMBOL_GPL (kvm_vcpu_is_reset_bsp);bool kvm_vcpu_is_bsp (struct kvm_vcpu *vcpu) { --2.25.1
PATCH v18 30/121 KVM: TDX: Do TDX specific vcpu initialization
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TD guest vcpu needs TDX specific initialization before running. Repurpose KVM_MEMORY_ENCRYPT_OP to vcpu-scope, add a new sub-command KVM_TDX_INIT_VCPU, and implement the callback for it.
---
v18:
- Use tdh_sys_rd() instead of struct tdsysinfo_struct.
- Rename tdx_reclaim_td_page() => tdx_reclaim_control_page()
- Remove the change of tools/arch/x86/include/uapi/asm/kvm.h.
--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/vmx/main.c | 9 ++ arch/x86/kvm/vmx/tdx.c | 184 ++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 8 ++ arch/x86/kvm/vmx/x86_ops.h | 4 + arch/x86/kvm/x86.c | 6 + 8 files changed, 211 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
@@ -129,6 +129,7 @@ KVM_X86_OP(leave_smm) KVM_X86_OP(enable_smi_window) #endif KVM_X86_OP(mem_enc_ioctl) +KVM_X86_OP_OPTIONAL(vcpu_mem_enc_ioctl) KVM_X86_OP_OPTIONAL(mem_enc_register_region) KVM_X86_OP_OPTIONAL(mem_enc_unregister_region) KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1776,6 +1776,7 @@ struct kvm_x86_ops {#endif int (*mem_enc_ioctl)(struct kvm *kvm,void __user *argp); +int (*vcpu_mem_enc_ioctl)(struct kvm_vcpu *vcpu,void __user *argp);int (*mem_enc_register_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*mem_enc_unregister_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*vm_copy_enc_context_from)(struct kvm *kvm,unsigned int source_fd);
arch/x86/include/uapi/asm/kvm.h +1 -0
@@ -571,6 +571,7 @@ struct kvm_pmu_event_filter {enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 ,KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU ,KVM_TDX_CMD_NR_MAX , };
arch/x86/kvm/vmx/main.c +9 -0
@@ -146,6 +146,14 @@ static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)return tdx_vm_ioctl (kvm, argp); } +static int vt_vcpu_mem_enc_ioctl (struct kvm_vcpu *vcpu,void __user *argp) +{ +if (!is_td_vcpu (vcpu)) +return -EINVAL ; + +return tdx_vcpu_ioctl (vcpu, argp); +} +#define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \
@@ -305,6 +313,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .get_untagged_addr = vmx_get_untagged_addr, .mem_enc_ioctl = vt_mem_enc_ioctl, + .vcpu_mem_enc_ioctl = vt_vcpu_mem_enc_ioctl, };struct kvm_x86_init_ops vt_init_ops__initdata = {
arch/x86/kvm/vmx/tdx.c +181 -3
235 lines not on this page · raw message
arch/x86/kvm/vmx/tdx.h +8 -0
@@ -17,12 +17,20 @@ struct kvm_tdx {u64 xfam;int hkid; +bool finalized; +u64 tsc_offset; };struct vcpu_tdx {struct kvm_vcpu vcpu; +unsigned long tdvpr_pa; +unsigned long *tdvpx_pa; +bool td_vcpu_created; + +bool initialized; +/* * Dummy to make pmu_intel not corrupt memory. * TODO: Support PMU for TDX. Future work.
arch/x86/kvm/vmx/x86_ops.h +4 -0
@@ -150,6 +150,8 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);int tdx_vcpu_create (struct kvm_vcpu *vcpu);void tdx_vcpu_free (struct kvm_vcpu *vcpu);void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); + +int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}
@@ -169,6 +171,8 @@ static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPstatic inline int tdx_vcpu_create (struct kvm_vcpu *vcpu) {return -EOPNOTSUPP ; }static inline void tdx_vcpu_free (struct kvm_vcpu *vcpu) {}static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {} + +static inline int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp) {return -EOPNOTSUPP ; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */
arch/x86/kvm/x86.c +6 -0
@@ -6248,6 +6248,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_SET_DEVICE_ATTR : r =kvm_vcpu_ioctl_device_attr (vcpu, ioctl, argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -ENOTTY ; +if (!kvm_x86_ops.vcpu_mem_enc_ioctl) + goto out; + r = kvm_x86_ops.vcpu_mem_enc_ioctl (vcpu, argp); +break ;default : r = -EINVAL ; }
Signature · 1 line
2.25.1
Re: [PATCH v18 030/121] KVM: TDX: Do TDX specific vcpu initialization
Quoted 222 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TD guest vcpu needs TDX specific initialization before running. Repurpose
KVM_MEMORY_ENCRYPT_OPto vcpu-scope, add a new sub-commandKVM_TDX_INIT_VCPU, and implement the callback for it.Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>---
v18:
- Usetdh_sys_rd()instead of structtdsysinfo_struct.
- Renametdx_reclaim_td_page()=>tdx_reclaim_control_page()
- Remove the change oftools/arch/x86/include/uapi/asm/kvm.h.--- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/vmx/main.c | 9 ++ arch/x86/kvm/vmx/tdx.c | 184 ++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 8 ++ arch/x86/kvm/vmx/x86_ops.h | 4 + arch/x86/kvm/x86.c | 6 + 8 files changed, 211 insertions(+), 3 deletions(-)arch/x86/include/asm/kvm-x86-ops.h +1
-0@@ -129,6 +129,7 @@ KVM_X86_OP(leave_smm) KVM_X86_OP(enable_smi_window) #endif KVM_X86_OP(mem_enc_ioctl) +KVM_X86_OP_OPTIONAL(vcpu_mem_enc_ioctl) KVM_X86_OP_OPTIONAL(mem_enc_register_region) KVM_X86_OP_OPTIONAL(mem_enc_unregister_region) KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from)arch/x86/include/asm/kvm_host.h +1
-0@@ -1776,6 +1776,7 @@ struct kvm_x86_ops {#endif int (*mem_enc_ioctl)(struct kvm *kvm,void __user *argp); +int (*vcpu_mem_enc_ioctl)(struct kvm_vcpu *vcpu,void __user *argp);int (*mem_enc_register_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*mem_enc_unregister_region)(struct kvm *kvm,struct kvm_enc_region *argp);int (*vm_copy_enc_context_from)(struct kvm *kvm,unsigned int source_fd);arch/x86/include/uapi/asm/kvm.h +1
-0@@ -571,6 +571,7 @@ struct kvm_pmu_event_filter {enum kvm_tdx_cmd_id {KVM_TDX_CAPABILITIES =0 ,KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU ,KVM_TDX_CMD_NR_MAX , };arch/x86/kvm/vmx/main.c +9
-0@@ -146,6 +146,14 @@ static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)return tdx_vm_ioctl (kvm, argp); } +static int vt_vcpu_mem_enc_ioctl (struct kvm_vcpu *vcpu,void __user *argp) +{ +if (!is_td_vcpu (vcpu)) +return -EINVAL ; + +return tdx_vcpu_ioctl (vcpu, argp); +} +#define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \@@ -305,6 +313,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .get_untagged_addr = vmx_get_untagged_addr, .mem_enc_ioctl = vt_mem_enc_ioctl, + .vcpu_mem_enc_ioctl = vt_vcpu_mem_enc_ioctl, };struct kvm_x86_init_ops vt_init_ops__initdata = {arch/x86/kvm/vmx/tdx.c +89
-2@@ -93,6 +93,7 @@ struct tdx_info {u64 xfam_fixed1;u8 nr_tdcs_pages; +u8 nr_tdvpx_pages;u16 num_cpuid_config;/* This must the last member. */ @@ -148,6 +149,11 @@ static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)return pa | ((hpa_t )hkid << boot_cpu_data.x86_phys_bits); } +static inline bool is_td_vcpu_created (struct vcpu_tdx *tdx) +{ +return tdx->td_vcpu_created; +} +static inline bool is_td_created (struct kvm_tdx *kvm_tdx) {return kvm_tdx->tdr_pa;@@ -165,6 +171,11 @@ static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx)return kvm_tdx->hkid >0 ; } +static inline bool is_td_finalized (struct kvm_tdx *kvm_tdx) +{ +return kvm_tdx->finalized; +} +static void tdx_clear_page (unsigned long page_pa) {const void *zero_page = (const void *)__va (page_to_phys (ZERO_PAGE (0 )));@@ -447,7 +458,32 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu)void tdx_vcpu_free (struct kvm_vcpu *vcpu) { -/* This is stub for now. More logic will come. */ +struct vcpu_tdx *tdx =to_tdx (vcpu); +int i; + +/* +* This methods can be called when vcpu allocation/initialization +* failed. So it's possible that hkid, tdvpx and tdvpr are not assigned +* yet. +*/ +if (is_hkid_assigned (to_kvm_tdx (vcpu->kvm))) { +WARN_ON_ONCE (tdx->tdvpx_pa); +WARN_ON_ONCE (tdx->tdvpr_pa); +return ; + } + +if (tdx->tdvpx_pa) { +for (i =0 ; i < tdx_info->nr_tdvpx_pages; i++) { +if (tdx->tdvpx_pa[i]) +tdx_reclaim_control_page (tdx->tdvpx_pa[i]); + } +kfree (tdx->tdvpx_pa); + tdx->tdvpx_pa =NULL ; + } +if (tdx->tdvpr_pa) { +tdx_reclaim_control_page (tdx->tdvpr_pa); + tdx->tdvpr_pa =0 ; + } }void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event)@@ -456,8 +492,13 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)/* Ignore INIT silently because TDX doesn't support INIT event. */ if (init_event)return ; +if (KVM_BUG_ON (is_td_vcpu_created (to_tdx (vcpu)), vcpu->kvm)) +return ; -/* This is stub for now. More logic will come here. */ +/* +* Don't update mp_state to runnable because more initialization +* is needed by TDX_VCPU_INIT. +*/ }static int tdx_get_capabilities (struct kvm_tdx_cmd *cmd)@@ -951,15 +992,147 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)return r; } +/* VMM can pass one 64bit auxiliary data to vcpu via RCX for guest BIOS. */ +static int tdx_td_vcpu_init (struct kvm_vcpu *vcpu,u64 vcpu_rcx) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); +struct vcpu_tdx *tdx =to_tdx (vcpu); +unsigned long *tdvpx_pa =NULL ; +unsigned long tdvpr_pa; +unsigned long va; +int ret, i; +u64 err; + +if (is_td_vcpu_created (tdx)) +return -EINVAL ; + +/* +* vcpu_free method frees allocated pages. Avoid partial setup so +* that the method can't handle it. +*/ + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) +return -ENOMEM ; + tdvpr_pa =__pa (va); + + tdvpx_pa =kcalloc (tdx_info->nr_tdvpx_pages,sizeof (*tdx->tdvpx_pa), +GFP_KERNEL_ACCOUNT ); +if (!tdvpx_pa) { + ret = -ENOMEM ; + goto free_tdvpr; + } +for (i =0 ; i < tdx_info->nr_tdvpx_pages; i++) { + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) { + ret = -ENOMEM ; + goto free_tdvpx; + } + tdvpx_pa[i] =__pa (va); + } + + err =tdh_vp_create (kvm_tdx->tdr_pa, tdvpr_pa); +if (KVM_BUG_ON (err, vcpu->kvm)) { + ret = -EIO ; +pr_tdx_error (TDH_VP_CREATE , err,NULL ); + goto free_tdvpx; + } + tdx->tdvpr_pa = tdvpr_pa; +
arch/x86/kvm/vmx/tdx.c int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) · line 1050
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ tdx->tdvpx_pa = tdvpx_pa; +for (i =0 ; i < tdx_info->nr_tdvpx_pages; i++) { + err =tdh_vp_addcx (tdx->tdvpr_pa, tdvpx_pa[i]); +if (KVM_BUG_ON (err, vcpu->kvm)) { +pr_tdx_error (TDH_VP_ADDCX , err,NULL ); +for (; i < tdx_info->nr_tdvpx_pages; i++) { +free_page ((unsigned long )__va (tdvpx_pa[i])); + tdvpx_pa[i] =0 ; + } +/* vcpu_free method frees TDVPX and TDR donated to TDX */
vcpu_free() interface is called by two sites.
One is the error handling path of kvm_vm_ioctl_create_vcpu() when vcpu creation.
The other is during kvm_destroy_vm().
What about the error occurs in KVM_TDX_INIT_VCPU?
Let's assume TDR and some of tdvpx pages are donated to TDX, and the next call of tdh_vp_addcx() failed. The comment says "vcpu_free method frees TDVPX and TDR donated to TDX", but if it happens, it seems that vcpu_free() would not be called? Memory leakage?
Quoted 165 lines
arch/x86/kvm/vmx/tdx.c +82
-1+return -EIO ; + } + } + + err =tdh_vp_init (tdx->tdvpr_pa, vcpu_rcx); +if (KVM_BUG_ON (err, vcpu->kvm)) { +pr_tdx_error (TDH_VP_INIT , err,NULL ); +return -EIO ; + } + + vcpu->arch.mp_state =KVM_MP_STATE_RUNNABLE ; + tdx->td_vcpu_created = true; +return 0 ; + +free_tdvpx: +for (i =0 ; i < tdx_info->nr_tdvpx_pages; i++) { +if (tdvpx_pa[i]) +free_page ((unsigned long )__va (tdvpx_pa[i])); + tdvpx_pa[i] =0 ; + } +kfree (tdvpx_pa); + tdx->tdvpx_pa =NULL ; +free_tdvpr: +if (tdvpr_pa) +free_page ((unsigned long )__va (tdvpr_pa)); + tdx->tdvpr_pa =0 ; + +return ret; +} + +int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp) +{ +struct msr_data apic_base_msr; +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); +struct vcpu_tdx *tdx =to_tdx (vcpu); +struct kvm_tdx_cmd cmd; +int ret; + +if (tdx->initialized) +return -EINVAL ; + +if (!is_hkid_assigned (kvm_tdx) ||is_td_finalized (kvm_tdx)) +return -EINVAL ; + +if (copy_from_user (&cmd, argp,sizeof (cmd))) +return -EFAULT ; + +if (cmd.error) +return -EINVAL ; + +/* Currently only KVM_TDX_INTI_VCPU is defined for vcpu operation. */ +if (cmd.flags || cmd.id !=KVM_TDX_INIT_VCPU ) +return -EINVAL ; + +/* +* As TDX requires X2APIC, set local apic mode to X2APIC. User space +* VMM, e.g. qemu, is required to set CPUID[0x1].ecx.X2APIC=1 by +* KVM_SET_CPUID2. Otherwise kvm_set_apic_base() will fail. +*/ + apic_base_msr = (struct msr_data ) { + .host_initiated = true, + .data =APIC_DEFAULT_PHYS_BASE |LAPIC_MODE_X2APIC | + (kvm_vcpu_is_reset_bsp (vcpu) ?MSR_IA32_APICBASE_BSP :0 ), + }; +if (kvm_set_apic_base (vcpu, &apic_base_msr)) +return -EINVAL ; + + ret =tdx_td_vcpu_init (vcpu, (u64 )cmd.data); +if (ret) +return ret; + + tdx->initialized = true; +return 0 ; +} +static int __init tdx_module_setup (void ) { -u16 num_cpuid_config, tdcs_base_size; +u16 num_cpuid_config, tdcs_base_size, tdvps_base_size;int ret;u32 i;struct tdx_md_map mds[] = {TDX_MD_MAP (NUM_CPUID_CONFIG , &num_cpuid_config),TDX_MD_MAP (TDCS_BASE_SIZE , &tdcs_base_size), +TDX_MD_MAP (TDVPS_BASE_SIZE , &tdvps_base_size), };#define TDX_INFO_MAP (_field_id, _member) \@@ -1015,6 +1188,11 @@ static int __init tdx_module_setup(void) } tdx_info->nr_tdcs_pages = tdcs_base_size /PAGE_SIZE ; +/* +* TDVPS = TDVPR(4K page) + TDVPX(multiple 4K pages). +* -1 for TDVPR. +*/ + tdx_info->nr_tdvpx_pages = tdvps_base_size /PAGE_SIZE -1 ;return 0 ;arch/x86/kvm/vmx/tdx.h +8
-0@@ -17,12 +17,20 @@ struct kvm_tdx {u64 xfam;int hkid; +bool finalized; +u64 tsc_offset; };struct vcpu_tdx {struct kvm_vcpu vcpu; +unsigned long tdvpr_pa; +unsigned long *tdvpx_pa; +bool td_vcpu_created; + +bool initialized; +/* * Dummy to make pmu_intel not corrupt memory. * TODO: Support PMU for TDX. Future work. arch/x86/kvm/vmx/x86_ops.h +4
-0@@ -150,6 +150,8 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);int tdx_vcpu_create (struct kvm_vcpu *vcpu);void tdx_vcpu_free (struct kvm_vcpu *vcpu);void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); + +int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp);#else static inline int tdx_hardware_setup (struct kvm_x86_ops *x86_ops) {return -EOPNOTSUPP ; }static inline void tdx_hardware_unsetup (void ) {}@@ -169,6 +171,8 @@ static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPstatic inline int tdx_vcpu_create (struct kvm_vcpu *vcpu) {return -EOPNOTSUPP ; }static inline void tdx_vcpu_free (struct kvm_vcpu *vcpu) {}static inline void tdx_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event) {} + +static inline int tdx_vcpu_ioctl (struct kvm_vcpu *vcpu,void __user *argp) {return -EOPNOTSUPP ; }#endif #endif /* __KVM_X86_VMX_X86_OPS_H */ arch/x86/kvm/x86.c +6
-0@@ -6248,6 +6248,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp,case KVM_SET_DEVICE_ATTR : r =kvm_vcpu_ioctl_device_attr (vcpu, ioctl, argp);break ; +case KVM_MEMORY_ENCRYPT_OP : + r = -ENOTTY ; +if (!kvm_x86_ops.vcpu_mem_enc_ioctl) + goto out; + r = kvm_x86_ops.vcpu_mem_enc_ioctl (vcpu, argp); +break ;default : r = -EINVAL ; }
Re: [PATCH v18 030/121] KVM: TDX: Do TDX specific vcpu initialization
Quoted 4 lines · Binbin Wu
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +0
-0
...
Quoted 64 lines
Quoted 49 lines
arch/x86/kvm/vmx/tdx.c +46
-0@@ -951,15 +992,147 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp)return r; } +/* VMM can pass one 64bit auxiliary data to vcpu via RCX for guest BIOS. */ +static int tdx_td_vcpu_init (struct kvm_vcpu *vcpu,u64 vcpu_rcx) +{ +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm); +struct vcpu_tdx *tdx =to_tdx (vcpu); +unsigned long *tdvpx_pa =NULL ; +unsigned long tdvpr_pa; +unsigned long va; +int ret, i; +u64 err; + +if (is_td_vcpu_created (tdx)) +return -EINVAL ; + +/* +* vcpu_free method frees allocated pages. Avoid partial setup so +* that the method can't handle it. +*/ + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) +return -ENOMEM ; + tdvpr_pa =__pa (va); + + tdvpx_pa =kcalloc (tdx_info->nr_tdvpx_pages,sizeof (*tdx->tdvpx_pa), +GFP_KERNEL_ACCOUNT ); +if (!tdvpx_pa) { + ret = -ENOMEM ; + goto free_tdvpr; + } +for (i =0 ; i < tdx_info->nr_tdvpx_pages; i++) { + va =__get_free_page (GFP_KERNEL_ACCOUNT ); +if (!va) { + ret = -ENOMEM ; + goto free_tdvpx; + } + tdvpx_pa[i] =__pa (va); + } + + err =tdh_vp_create (kvm_tdx->tdr_pa, tdvpr_pa); +if (KVM_BUG_ON (err, vcpu->kvm)) { + ret = -EIO ; +pr_tdx_error (TDH_VP_CREATE , err,NULL ); + goto free_tdvpx; + } + tdx->tdvpr_pa = tdvpr_pa; +arch/x86/kvm/vmx/tdx.c int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) · line 1049
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ tdx->tdvpx_pa = tdvpx_pa; +for (i =0 ; i < tdx_info->nr_tdvpx_pages; i++) { + err =tdh_vp_addcx (tdx->tdvpr_pa, tdvpx_pa[i]); +if (KVM_BUG_ON (err, vcpu->kvm)) { +pr_tdx_error (TDH_VP_ADDCX , err,NULL ); +for (; i < tdx_info->nr_tdvpx_pages; i++) { +free_page ((unsigned long )__va (tdvpx_pa[i])); + tdvpx_pa[i] =0 ; + } +/* vcpu_free method frees TDVPX and TDR donated to TDX */
vcpu_free()interface is called by two sites.
One is the error handling path ofkvm_vm_ioctl_create_vcpu()when vcpu creation.
The other is duringkvm_destroy_vm().
Quoted 6 lines
What about the error occurs in
KVM_TDX_INIT_VCPU?
Let's assume TDR and some of tdvpx pages are donated to TDX, and the next call oftdh_vp_addcx()failed. The comment says "vcpu_freemethod frees TDVPX
and TDR donated to TDX", but if it happens, it seems thatvcpu_free()would not be called? Memory leakage?
vcpu_free() is called because we already created vcpu with KVM_VCPU_CREATE irrelevant of the result of TDX_VCPU_INIT.tdx_vcpu_free() handles error case of TDX_VCPU_INIT. So no leakage.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 31/121 The start of TDX KVM patch series: KVM MMU GPA shared bits
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of KVM MMU GPA shared bits.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +3 -2
@@ -10,6 +10,7 @@ What qemu can do ---------------- - TDX VM TYPE is exposed to Qemu. - Qemu can create/destroy guest of TDX vm type. +- Qemu can create/destroy vcpu of TDX vm type. Patch Layer status ------------------
@@ -18,12 +19,12 @@ Patch Layer status * TDX, VMX coexistence: Applied * TDX architectural definitions: Applied * TD VM creation/destruction: Applied -* TD vcpu creation/destruction: Applying +* TD vcpu creation/destruction: Applied * TDX EPT violation: Not yet * TD finalization: Not yet * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet -* KVM MMU GPA shared bits: Not yet +* KVM MMU GPA shared bits: Applying * KVM TDP refactoring for TDX: Not yet * KVM TDP MMU hooks: Not yet
Signature · 1 line
2.25.1
PATCH v18 32/121 KVM: x86/mmu: introduce config for PRIVATE KVM MMU
In reply to [email protected]From: Isaku Yamahata <[email protected]>
To keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for CONFIG_INTEL_TDX_HOST && CONFIG_KVM_INTEL. The config makes it clear that the config is only for x86 KVM MMU.
--- arch/x86/kvm/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/kvm/Kconfig +4 -0
@@ -189,4 +189,8 @@ config KVM_MAX_NR_VCPUS the memory footprint of each KVM guest, regardless of how many vCPUs are created for a given VM. +config KVM_MMU_PRIVATE + def_bool y + depends on INTEL_TDX_HOST && KVM_INTEL + endif # VIRTUALIZATION
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [PATCH v18 032/121] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 7 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Isaku Yamahata <[email protected]>
To keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for
CONFIG_INTEL_TDX_HOST&&CONFIG_KVM_INTEL. The config makes it clear that the config is only for x86 KVM MMU.
Better, just put this as:
config KVM_MMU_PRIVATE
bool
but also add a reverse dependency to KVM_INTEL:
config KVM_INTEL
tristate "KVM for Intel (and compatible) processors support" depends on KVM && IA32_FEAT_CTL
select KVM_MMU_PRIVATE if INTEL_TDX_HOST
...
This matches the usage of kvm-intel-$(INTEL_TDX_HOST) in the Makefile.
Paolo
Quoted 21 lines
Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/kvm/Kconfig | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/Kconfig +4
-1@@ -189,4 +189,8 @@ config KVM_MAX_NR_VCPUS the memory footprint of each KVM guest, regardless of how many vCPUs are created for a given VM. +config KVM_MMU_PRIVATE + def_bool y + depends on INTEL_TDX_HOST && KVM_INTEL + endif # VIRTUALIZATION --2.25.1
Sean Christopherson
maintainerRe: [PATCH v18 032/121] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 16 lines · Paolo Bonzini
Quoted 7 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Isaku Yamahata <[email protected]>
To keep the case of non TDX intact, introduce a new config option for private KVM MMU support. At the moment, this is synonym for
CONFIG_INTEL_TDX_HOST&&CONFIG_KVM_INTEL. The config makes it clear that the config is only for x86 KVM MMU.Better, just put this as:
config
KVM_MMU_PRIVATE
boolbut also add a reverse dependency to
KVM_INTEL:
Quoted 7 lines
config
KVM_INTEL
tristate "KVM for Intel (and compatible) processors support" depends on KVM &&IA32_FEAT_CTL
selectKVM_MMU_PRIVATEifINTEL_TDX_HOST
...This matches the usage of kvm-intel-$(INTEL_TDX_HOST) in the
Makefile.
But why even bother with a Kconfig in the first place? Saving the is_private bit in the role adds no value whatsoever. In fact, it's probably a big net negative because it necessitates this ugly code:
if (private) kvm_mmu_page_role_set_private(&role);
which really should just be
role.private = private;
Ditto for kvm_mmu_page.private_spt.
The only thing that even so much as approaches being a hot path is kvm_gfn_shared_mask(), and if that needs to be optimized, then we'd probably be better off with a static_key, a la kvm_has_noapic_vcpu (though I'm *extremely* skeptical that that adds any measurable benefit).
Paolo Bonzini
maintainerRe: [PATCH v18 032/121] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 4 lines · Sean Christopherson
The only thing that even so much as approaches being a hot path is
kvm_gfn_shared_mask(), and if that needs to be optimized, then we'd probably be better off with astatic_key, a lakvm_has_noapic_vcpu(though I'm *extremely* skeptical that that adds any measurable benefit).
I'm okay with killing it altogether.
Paolo
Re: [PATCH v18 032/121] KVM: x86/mmu: introduce config for PRIVATE KVM MMU
Quoted 7 lines · Paolo Bonzini
Quoted 4 lines · Sean Christopherson
The only thing that even so much as approaches being a hot path is
kvm_gfn_shared_mask(), and if that needs to be optimized, then we'd probably be better off with astatic_key, a lakvm_has_noapic_vcpu(though I'm *extremely* skeptical that that adds any measurable benefit).I'm okay with killing it altogether.
I eliminated this config.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 33/121 KVM: x86/mmu: Add address conversion functions for TDX shared bit of GPA
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX repurposes one GPA bit (51 bit or 47 bit based on configuration) to indicate the GPA is private(if cleared) or shared (if set) with VMM. If GPA.shared is set, GPA is covered by the existing conventional EPT pointed by EPTP. If GPA.shared bit is cleared, GPA is covered by TDX module. VMM has to issue SEAMCALLs to operate.
Add a member to remember GPA shared bit for each guest TDs, add address conversion functions between private GPA and shared GPA and test if GPA is private.
Because struct kvm_arch (or struct kvm which includes struct kvm_arch. See kvm_arch_alloc_vm() that passes __GPF_ZERO) is zero-cleared when allocated, the new member to remember GPA shared bit is guaranteed to be zero with this patch unless it's initialized explicitly.
---
v18:
- Added Reviewed-by Binbin
--- arch/x86/include/asm/kvm_host.h | 4 ++++ arch/x86/kvm/mmu.h | 27 +++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 5 +++++ 3 files changed, 36 insertions(+)
arch/x86/include/asm/kvm_host.h +4 -0
@@ -1503,6 +1503,10 @@ struct kvm_arch {*/ #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1)struct kvm_mmu_memory_cache split_desc_cache; + +#ifdef CONFIG_KVM_MMU_PRIVATE +gfn_t gfn_shared_mask; +#endif };struct kvm_vm_stat {
arch/x86/kvm/mmu.h +27 -0
@@ -319,4 +319,31 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu,return gpa; return translate_nested_gpa(vcpu, gpa, access, exception); } + +static inline gfn_t kvm_gfn_shared_mask (const struct kvm *kvm) +{ +#ifdef CONFIG_KVM_MMU_PRIVATE +return kvm->arch.gfn_shared_mask; +#else +return 0 ; +#endif +} + +static inline gfn_t kvm_gfn_to_shared (const struct kvm *kvm,gfn_t gfn) +{ +return gfn |kvm_gfn_shared_mask (kvm); +} + +static inline gfn_t kvm_gfn_to_private (const struct kvm *kvm,gfn_t gfn) +{ +return gfn & ~kvm_gfn_shared_mask (kvm); +} + +static inline bool kvm_is_private_gpa (const struct kvm *kvm,gpa_t gpa) +{ +gfn_t mask =kvm_gfn_shared_mask (kvm); + +return mask && !(gpa_to_gfn (gpa) & mask); +} +#endif
arch/x86/kvm/vmx/tdx.c +5 -0
@@ -953,6 +953,11 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd) kvm_tdx->attributes = td_params->attributes; kvm_tdx->xfam = td_params->xfam; +if (td_params->exec_controls &TDX_EXEC_CONTROL_MAX_GPAW ) + kvm->arch.gfn_shared_mask =gpa_to_gfn (BIT_ULL (51 )); +else + kvm->arch.gfn_shared_mask =gpa_to_gfn (BIT_ULL (47 )); + out:/* kfree() accepts NULL. */ kfree (init_vm);
Signature · 1 line
2.25.1
PATCH v18 34/121 The start of TDX KVM patch series: KVM TDP refactoring for TDX
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of KVM TDP refactoring for TDX.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2
@@ -25,6 +25,6 @@ Patch Layer status * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet -* KVM MMU GPA shared bits: Applying -* KVM TDP refactoring for TDX: Not yet +* KVM MMU GPA shared bits: Applied +* KVM TDP refactoring for TDX: Applying * KVM TDP MMU hooks: Not yet
Signature · 1 line
2.25.1
PATCH v18 35/121 KVM: Allow page-sized MMU caches to be initialized with custom 64-bit values
In reply to [email protected]From: Sean Christopherson <[email protected]>
Add support to MMU caches for initializing a page with a custom 64-bit value, e.g. to pre-fill an entire page table with non-zero PTE values. The functionality will be used by x86 to support Intel's TDX, which needs to set bit 63 in all non-present PTEs in order to prevent !PRESENT page faults from getting reflected into the guest (Intel's EPT Violation #VE architecture made the less than brilliant decision of having the per-PTE behavior be opt-out instead of opt-in).
--- include/linux/kvm_types.h | 1 + virt/kvm/kvm_main.c | 16 ++++++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-)
include/linux/kvm_types.h +1 -0
@@ -94,6 +94,7 @@ struct gfn_to_pfn_cache {struct kvm_mmu_memory_cache {gfp_t gfp_zero;gfp_t gfp_custom; +u64 init_value;struct kmem_cache *kmem_cache;int capacity;int nobjs;
virt/kvm/kvm_main.c +14 -2
@@ -401,12 +401,17 @@ static void kvm_flush_shadow_all(struct kvm *kvm)static inline void *mmu_memory_cache_alloc_obj (struct kvm_mmu_memory_cache *mc,gfp_t gfp_flags) { +void *page; + gfp_flags |= mc->gfp_zero;if (mc->kmem_cache)return kmem_cache_alloc (mc->kmem_cache, gfp_flags); -else -return (void *)__get_free_page (gfp_flags); + + page = (void *)__get_free_page (gfp_flags); +if (page && mc->init_value) +memset64 (page, mc->init_value,PAGE_SIZE /sizeof (mc->init_value)); +return page; }int __kvm_mmu_topup_memory_cache (struct kvm_mmu_memory_cache *mc,int capacity,int min)
@@ -421,6 +426,13 @@ int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity,if (WARN_ON_ONCE (!capacity))return -EIO ; +/* +* Custom init values can be used only for page allocations, +* and obviously conflict with __GFP_ZERO. +*/ +if (WARN_ON_ONCE (mc->init_value && (mc->kmem_cache || mc->gfp_zero))) +return -EIO ; + mc->objects =kvmalloc_array (sizeof (void *), capacity, gfp);if (!mc->objects)return -ENOMEM ;
Signature · 1 line
2.25.1
PATCH v18 36/121 KVM: x86/mmu: Replace hardcoded value 0 for the initial value for SPTE
In reply to [email protected]From: Isaku Yamahata <[email protected]>
The TDX support will need the "suppress #VE" bit (bit 63) set as the initial value for SPTE. To reduce code change size, introduce a new macro SHADOW_NONPRESENT_VALUE for the initial value for the shadow page table entry (SPTE) and replace hard-coded value 0 for it. Initialize shadow page tables with their value.
The plan is to unconditionally set the "suppress #VE" bit for both AMD and Intel as: 1) AMD hardware uses the bit 63 as NX for present SPTE and ignored for non-present SPTE; 2) for conventional VMX guests, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware.
--- arch/x86/kvm/mmu/mmu.c | 20 +++++++++++++++----- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.h | 2 ++ arch/x86/kvm/mmu/tdp_mmu.c | 14 +++++++------- 4 files changed, 25 insertions(+), 13 deletions(-)
arch/x86/kvm/mmu/mmu.c +15 -5
@@ -567,9 +567,9 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep)if (!is_shadow_present_pte (old_spte) || !spte_has_volatile_bits (old_spte)) -__update_clear_spte_fast (sptep,0ull ); +__update_clear_spte_fast (sptep,SHADOW_NONPRESENT_VALUE );else - old_spte =__update_clear_spte_slow (sptep,0ull ); + old_spte =__update_clear_spte_slow (sptep,SHADOW_NONPRESENT_VALUE );if (!is_shadow_present_pte (old_spte))return old_spte;
@@ -603,7 +603,7 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep)*/ static void mmu_spte_clear_no_track (u64 *sptep) { -__update_clear_spte_fast (sptep,0ull ); +__update_clear_spte_fast (sptep,SHADOW_NONPRESENT_VALUE ); }static u64 mmu_spte_get_lockless (u64 *sptep)
@@ -1950,7 +1950,8 @@ static bool kvm_sync_page_check(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)static int kvm_sync_spte (struct kvm_vcpu *vcpu,struct kvm_mmu_page *sp,int i) { -if (!sp->spt[i]) +/* sp->spt[i] has initial value of shadow page table allocation */ +if (sp->spt[i] ==SHADOW_NONPRESENT_VALUE )return 0 ;return vcpu->arch.mmu->sync_spte (vcpu, sp, i);
@@ -6129,7 +6130,16 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu) vcpu->arch.mmu_page_header_cache.kmem_cache = mmu_page_header_cache; vcpu->arch.mmu_page_header_cache.gfp_zero = __GFP_ZERO; - vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO; +/* +* When X86_64, initial SEPT entries are initialized with +* SHADOW_NONPRESENT_VALUE. Otherwise zeroed. See +* mmu_memory_cache_alloc_obj(). +*/ +if (IS_ENABLED (CONFIG_X86_64 )) + vcpu->arch.mmu_shadow_page_cache.init_value = +SHADOW_NONPRESENT_VALUE ; +if (!vcpu->arch.mmu_shadow_page_cache.init_value) + vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO; vcpu->arch.mmu = &vcpu->arch.root_mmu; vcpu->arch.walk_mmu = &vcpu->arch.root_mmu;
arch/x86/kvm/mmu/paging_tmpl.h +1 -1
@@ -911,7 +911,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, intgpa_t pte_gpa;gfn_t gfn; -if (WARN_ON_ONCE (!sp->spt[i])) +if (WARN_ON_ONCE (sp->spt[i] ==SHADOW_NONPRESENT_VALUE ))return 0 ; first_pte_gpa =FNAME (get_level1_sp_gpa)(sp);
arch/x86/kvm/mmu/spte.h +2 -0
@@ -149,6 +149,8 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11);#define MMIO_SPTE_GEN_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0) +#define SHADOW_NONPRESENT_VALUE 0ULL +extern u64 __read_mostly shadow_host_writable_mask;extern u64 __read_mostly shadow_mmu_writable_mask;extern u64 __read_mostly shadow_nx_mask;
arch/x86/kvm/mmu/tdp_mmu.c +7 -7
@@ -570,7 +570,7 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm,* here since the SPTE is going from non-present to non-present. Use * the raw write helper to avoid an unnecessary check on volatile bits. */ -__kvm_tdp_mmu_write_spte (iter->sptep,0 ); +__kvm_tdp_mmu_write_spte (iter->sptep,SHADOW_NONPRESENT_VALUE );return 0 ; }
@@ -707,8 +707,8 @@ static void __tdp_mmu_zap_root(struct kvm *kvm, struct kvm_mmu_page *root,continue ;if (!shared) -tdp_mmu_iter_set_spte (kvm, &iter,0 ); -else if (tdp_mmu_set_spte_atomic (kvm, &iter,0 )) +tdp_mmu_iter_set_spte (kvm, &iter,SHADOW_NONPRESENT_VALUE ); +else if (tdp_mmu_set_spte_atomic (kvm, &iter,SHADOW_NONPRESENT_VALUE )) goto retry; } }
@@ -764,8 +764,8 @@ bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp)if (WARN_ON_ONCE (!is_shadow_present_pte (old_spte)))return false; -tdp_mmu_set_spte (kvm,kvm_mmu_page_as_id (sp), sp->ptep, old_spte,0 , - sp->gfn, sp->role.level +1 ); +tdp_mmu_set_spte (kvm,kvm_mmu_page_as_id (sp), sp->ptep, old_spte, +SHADOW_NONPRESENT_VALUE , sp->gfn, sp->role.level +1 );return true; }
@@ -799,7 +799,7 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, !is_last_spte(iter.old_spte, iter.level))continue ; -tdp_mmu_iter_set_spte (kvm, &iter,0 ); +tdp_mmu_iter_set_spte (kvm, &iter,SHADOW_NONPRESENT_VALUE ); flush = true; }
@@ -1226,7 +1226,7 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter,* invariant that the PFN of a present * leaf SPTE can never change. * See handle_changed_spte(). */ -tdp_mmu_iter_set_spte (kvm, iter,0 ); +tdp_mmu_iter_set_spte (kvm, iter,SHADOW_NONPRESENT_VALUE );if (!pte_write (range->arg.pte)) { new_spte = kvm_mmu_changed_pte_notifier_make_spte(iter->old_spte,
Signature · 1 line
2.25.1
PATCH v18 37/121 KVM: x86/mmu: Allow non-zero value for non-present SPTE and removed SPTE
In reply to [email protected]From: Sean Christopherson <[email protected]>
For TD guest, the current way to emulate MMIO doesn't work any more, as KVM is not able to access the private memory of TD guest and do the emulation. Instead, TD guest expects to receive #VE when it accesses the MMIO and then it can explicitly make hypercall to KVM to get the expected information.
To achieve this, the TDX module always enables "EPT-violation #VE" in the VMCS control. And accordingly, for the MMIO spte for the shared GPA,
1. KVM needs to set "suppress #VE" bit for the non-present SPTE so that EPT violation happens on TD accessing MMIO range. 2. On EPT violation, KVM sets the MMIO spte to clear "suppress #VE" bit so the TD guest can receive the #VE instead of EPT misconfigration unlike VMX case. For the shared GPA that is not populated yet, EPT violation need to be triggered when TD guest accesses such shared GPA. The non-present SPTE value for shared GPA should set "suppress #VE" bit.
Add "suppress #VE" bit (bit 63) to SHADOW_NONPRESENT_VALUE and REMOVED_SPTE. Unconditionally set the "suppress #VE" bit (which is bit 63) for both AMD and Intel as: 1) AMD hardware doesn't use this bit when present bit is off; 2) for normal VMX guest, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware.
--- arch/x86/kvm/mmu/spte.h | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/spte.h +14 -1
@@ -149,7 +149,20 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11);#define MMIO_SPTE_GEN_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0) +/* +* Non-present SPTE value for both VMX and SVM for TDP MMU. +* For SVM NPT, for non-present spte (bit 0 = 0), other bits are ignored. +* For VMX EPT, bit 63 is ignored if #VE is disabled. (EPT_VIOLATION_VE=0) +* bit 63 is #VE suppress if #VE is enabled. (EPT_VIOLATION_VE=1) +* For TDX: +* TDX module sets EPT_VIOLATION_VE for Secure-EPT and conventional EPT +*/ +#ifdef CONFIG_X86_64 +#define SHADOW_NONPRESENT_VALUE BIT_ULL(63) +static_assert (!(SHADOW_NONPRESENT_VALUE &SPTE_MMU_PRESENT_MASK )); +#else #define SHADOW_NONPRESENT_VALUE 0ULL +#endif extern u64 __read_mostly shadow_host_writable_mask;extern u64 __read_mostly shadow_mmu_writable_mask;
@@ -196,7 +209,7 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;* * Only used by the TDP MMU. */ -#define REMOVED_SPTE 0x5a0ULL +#define REMOVED_SPTE (SHADOW_NONPRESENT_VALUE | 0x5a0ULL)/* Removed SPTEs must not be misconstrued as shadow present PTEs. */ static_assert (!(REMOVED_SPTE &SPTE_MMU_PRESENT_MASK ));
Signature · 1 line
2.25.1
Re: [PATCH v18 037/121] KVM: x86/mmu: Allow non-zero value for non-present SPTE and removed SPTE
Quoted 7 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Sean Christopherson <[email protected]>
For TD guest, the current way to emulate MMIO doesn't work any more, as KVM is not able to access the private memory of TD guest and do the emulation. Instead, TD guest expects to receive #VE when it accesses the MMIO and then it can explicitly make hypercall to KVM to get the expected information.
Quoted 6 lines
To achieve this, the TDX module always enables "EPT-violation #VE" in the VMCS control. And accordingly, for the MMIO spte for the shared GPA,
1. KVM needs to set "suppress #VE" bit for the non-present SPTE so that EPT violation happens on TD accessing MMIO range. 2. On EPT violation, KVM sets the MMIO spte to clear "suppress #VE" bit so the TD guest can receive the #VE instead of EPT misconfigration unlike VMX case. For the shared GPA
s/misconfigration/misconfiguration
Quoted 11 lines
that is not populated yet, EPT violation need to be triggered when TD guest accesses such shared GPA. The non-present SPTE value for shared GPA should set "suppress #VE" bit.
Add "suppress #VE" bit (bit 63) to
SHADOW_NONPRESENT_VALUEandREMOVED_SPTE. Unconditionally set the "suppress #VE" bit (which is bit 63) for both AMD and Intel as: 1) AMD hardware doesn't use this bit when present bit is off; 2) for normal VMX guest, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware.
Quoted 2 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>
Nit: one typo above.
Quoted 38 lines
--- arch/x86/kvm/mmu/spte.h | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)arch/x86/kvm/mmu/spte.h +14
-1@@ -149,7 +149,20 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11);#define MMIO_SPTE_GEN_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0) +/* +* Non-present SPTE value for both VMX and SVM for TDP MMU. +* For SVM NPT, for non-present spte (bit 0 = 0), other bits are ignored. +* For VMX EPT, bit 63 is ignored if #VE is disabled. (EPT_VIOLATION_VE=0) +* bit 63 is #VE suppress if #VE is enabled. (EPT_VIOLATION_VE=1) +* For TDX: +* TDX module sets EPT_VIOLATION_VE for Secure-EPT and conventional EPT +*/ +#ifdef CONFIG_X86_64 +#define SHADOW_NONPRESENT_VALUE BIT_ULL(63) +static_assert (!(SHADOW_NONPRESENT_VALUE &SPTE_MMU_PRESENT_MASK )); +#else #define SHADOW_NONPRESENT_VALUE 0ULL +#endif extern u64 __read_mostly shadow_host_writable_mask;extern u64 __read_mostly shadow_mmu_writable_mask;@@ -196,7 +209,7 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask;* * Only used by the TDP MMU. */ -#define REMOVED_SPTE 0x5a0ULL +#define REMOVED_SPTE (SHADOW_NONPRESENT_VALUE | 0x5a0ULL)/* Removed SPTEs must not be misconstrued as shadow present PTEs. */ static_assert (!(REMOVED_SPTE &SPTE_MMU_PRESENT_MASK ));
PATCH v18 38/121 KVM: x86/mmu: Add Suppress VE bit to shadow_mmio_mask/shadow_present_mask
In reply to [email protected]From: Isaku Yamahata <[email protected]>
To make use of the same value of shadow_mmio_mask and shadow_present_mask for TDX and VMX, add Suppress-VE bit to shadow_mmio_mask and shadow_present_mask so that they can be common for both VMX and TDX.
TDX will require shadow_mmio_mask and shadow_present_mask to include VMX_SUPPRESS_VE for shared GPA so that EPT violation is triggered for shared GPA. For VMX, VMX_SUPPRESS_VE doesn't matter for MMIO because the spte value is required to cause EPT misconfig. the additional bit doesn't affect VMX logic to add the bit to shadow_mmio_{value, mask}.
--- arch/x86/include/asm/vmx.h | 1 + arch/x86/kvm/mmu/spte.c | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/include/asm/vmx.h +1 -0
@@ -513,6 +513,7 @@ enum vmcs_field {#define VMX_EPT_IPAT_BIT (1ull << 6)#define VMX_EPT_ACCESS_BIT (1ull << 8)#define VMX_EPT_DIRTY_BIT (1ull << 9) +#define VMX_EPT_SUPPRESS_VE_BIT (1ull << 63)#define VMX_EPT_RWX_MASK (VMX_EPT_READABLE_MASK | \ VMX_EPT_WRITABLE_MASK | \ VMX_EPT_EXECUTABLE_MASK)
arch/x86/kvm/mmu/spte.c +4 -2
@@ -429,7 +429,9 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only) shadow_dirty_mask = has_ad_bits ?VMX_EPT_DIRTY_BIT :0ull ; shadow_nx_mask =0ull ; shadow_x_mask =VMX_EPT_EXECUTABLE_MASK ; - shadow_present_mask = has_exec_only ?0ull :VMX_EPT_READABLE_MASK ; +/* VMX_EPT_SUPPRESS_VE_BIT is needed for W or X violation. */ + shadow_present_mask = + (has_exec_only ?0ull :VMX_EPT_READABLE_MASK ) |VMX_EPT_SUPPRESS_VE_BIT ;/* * EPT overrides the host MTRRs, and so KVM must program the desired * memtype directly into the SPTEs. Note, this mask is just the mask
@@ -446,7 +448,7 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only)* of an EPT paging-structure entry is 110b (write/execute). */ kvm_mmu_set_mmio_spte_mask (VMX_EPT_MISCONFIG_WX_VALUE , -VMX_EPT_RWX_MASK ,0 ); +VMX_EPT_RWX_MASK |VMX_EPT_SUPPRESS_VE_BIT ,0 ); }EXPORT_SYMBOL_GPL (kvm_mmu_set_ept_masks);
Signature · 1 line
2.25.1
PATCH v18 39/121 KVM: x86/mmu: Track shadow MMIO value on a per-VM basis
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX will use a different shadow PTE entry value for MMIO from VMX. Add members to kvm_arch and track value for MMIO per-VM instead of global variables. By using the per-VM EPT entry value for MMIO, the existing VMX logic is kept working. Introduce a separate setter function so that guest TD can override later.
Also require mmio spte cachcing for TDX. Actually this is true case because TDX require EPT and KVM EPT allows mmio spte caching.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 8 +++++--- arch/x86/kvm/mmu/spte.c | 10 ++++++++-- arch/x86/kvm/mmu/spte.h | 4 ++-- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- 6 files changed, 21 insertions(+), 10 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -1310,6 +1310,8 @@ struct kvm_arch {*/ spinlock_t mmu_unsync_pages_lock; +u64 shadow_mmio_value; +struct iommu_domain *iommu_domain;bool iommu_noncoherent;#define __KVM_HAVE_ARCH_NONCOHERENT_DMA
arch/x86/kvm/mmu.h +1 -0
@@ -101,6 +101,7 @@ static inline u8 kvm_get_shadow_phys_bits(void) }void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask); +void kvm_mmu_set_mmio_spte_value (struct kvm *kvm,u64 mmio_value);void kvm_mmu_set_me_spte_mask (u64 me_value,u64 me_mask);void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only);
arch/x86/kvm/mmu/mmu.c +5 -3
@@ -2515,7 +2515,7 @@ static int mmu_page_zap_pte(struct kvm *kvm, struct kvm_mmu_page *sp,return kvm_mmu_prepare_zap_page (kvm, child, invalid_list); } - }else if (is_mmio_spte (pte)) { + }else if (is_mmio_spte (kvm, pte)) {mmu_spte_clear_no_track (spte); }return 0 ;
@@ -4184,7 +4184,7 @@ static int handle_mmio_page_fault(struct kvm_vcpu *vcpu, u64 addr, bool direct)if (WARN_ON_ONCE (reserved))return -EINVAL ; -if (is_mmio_spte (spte)) { +if (is_mmio_spte (vcpu->kvm, spte)) {gfn_t gfn =get_mmio_spte_gfn (spte);unsigned int access =get_mmio_spte_access (spte);
@@ -4762,7 +4762,7 @@ EXPORT_SYMBOL_GPL(kvm_mmu_new_pgd);static bool sync_mmio_spte (struct kvm_vcpu *vcpu,u64 *sptep,gfn_t gfn,unsigned int access) { -if (unlikely (is_mmio_spte (*sptep))) { +if (unlikely (is_mmio_spte (vcpu->kvm, *sptep))) {if (gfn !=get_mmio_spte_gfn (*sptep)) {mmu_spte_clear_no_track (sptep);return true;
@@ -6282,6 +6282,8 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)void kvm_mmu_init_vm (struct kvm *kvm) { + + kvm->arch.shadow_mmio_value = shadow_mmio_value;INIT_LIST_HEAD (&kvm->arch.active_mmu_pages);INIT_LIST_HEAD (&kvm->arch.zapped_obsolete_pages);INIT_LIST_HEAD (&kvm->arch.possible_nx_huge_pages);
arch/x86/kvm/mmu/spte.c +8 -2
@@ -74,10 +74,10 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access)u64 spte =generation_mmio_spte_mask (gen);u64 gpa = gfn <<PAGE_SHIFT ; -WARN_ON_ONCE (!shadow_mmio_value); +WARN_ON_ONCE (!vcpu->kvm->arch.shadow_mmio_value); access &= shadow_mmio_access_mask; - spte |= shadow_mmio_value | access; + spte |= vcpu->kvm->arch.shadow_mmio_value | access; spte |= gpa | shadow_nonpresent_or_rsvd_mask; spte |= (gpa & shadow_nonpresent_or_rsvd_mask) <<SHADOW_NONPRESENT_OR_RSVD_MASK_LEN ;
@@ -411,6 +411,12 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) }EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_mask); +void kvm_mmu_set_mmio_spte_value (struct kvm *kvm,u64 mmio_value) +{ + kvm->arch.shadow_mmio_value = mmio_value; +} +EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_value); +void kvm_mmu_set_me_spte_mask (u64 me_value,u64 me_mask) {/* shadow_me_value must be a subset of shadow_me_mask */
arch/x86/kvm/mmu/spte.h +2 -2
@@ -264,9 +264,9 @@ static inline struct kvm_mmu_page *root_to_sp(hpa_t root)return spte_to_child_sp (root); } -static inline bool is_mmio_spte (u64 spte) +static inline bool is_mmio_spte (struct kvm *kvm,u64 spte) { -return (spte & shadow_mmio_mask) == shadow_mmio_value && +return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&likely (enable_mmio_caching); }
arch/x86/kvm/mmu/tdp_mmu.c +3 -3
@@ -462,8 +462,8 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,* impact the guest since both the former and current SPTEs * are nonpresent. */ -if (WARN_ON_ONCE (!is_mmio_spte (old_spte) && - !is_mmio_spte (new_spte) && +if (WARN_ON_ONCE (!is_mmio_spte (kvm, old_spte) && + !is_mmio_spte (kvm, new_spte) && !is_removed_spte (new_spte)))pr_err ("Unexpected SPTE change! Nonpresent SPTEs\n" "should not be replaced with another,\n"
@@ -978,7 +978,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, }/* If a MMIO SPTE is installed, the MMIO will need to be emulated. */ -if (unlikely (is_mmio_spte (new_spte))) { +if (unlikely (is_mmio_spte (vcpu->kvm, new_spte))) { vcpu->stat.pf_mmio_spte_created++;trace_mark_mmio_spte (rcu_dereference (iter->sptep), iter->gfn, new_spte);
Signature · 1 line
2.25.1
Re: [PATCH v18 039/121] KVM: x86/mmu: Track shadow MMIO value on a per-VM basis
Quoted 9 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX will use a different shadow PTE entry value for MMIO from VMX. Add members to
kvm_archand track value for MMIO per-VM instead of global variables. By using the per-VM EPT entry value for MMIO, the existing VMX logic is kept working. Introduce a separate setter function so that guest TD can override later.Also require mmio spte cachcing for TDX. Actually this is true case
s/cachcing/caching
Quoted 1 line
because TDX require EPT and KVM EPT allows mmio spte caching.
s/require/requires
Quoted 95 lines
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 8 +++++--- arch/x86/kvm/mmu/spte.c | 10 ++++++++-- arch/x86/kvm/mmu/spte.h | 4 ++-- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- 6 files changed, 21 insertions(+), 10 deletions(-)arch/x86/include/asm/kvm_host.h +2
-0@@ -1310,6 +1310,8 @@ struct kvm_arch {*/ spinlock_t mmu_unsync_pages_lock; +u64 shadow_mmio_value; +struct iommu_domain *iommu_domain;bool iommu_noncoherent;#define __KVM_HAVE_ARCH_NONCOHERENT_DMAarch/x86/kvm/mmu.h +1
-0@@ -101,6 +101,7 @@ static inline u8 kvm_get_shadow_phys_bits(void) }void kvm_mmu_set_mmio_spte_mask (u64 mmio_value,u64 mmio_mask,u64 access_mask); +void kvm_mmu_set_mmio_spte_value (struct kvm *kvm,u64 mmio_value);void kvm_mmu_set_me_spte_mask (u64 me_value,u64 me_mask);void kvm_mmu_set_ept_masks (bool has_ad_bits,bool has_exec_only);arch/x86/kvm/mmu/mmu.c +5
-3@@ -2515,7 +2515,7 @@ static int mmu_page_zap_pte(struct kvm *kvm, struct kvm_mmu_page *sp,return kvm_mmu_prepare_zap_page (kvm, child, invalid_list); } - }else if (is_mmio_spte (pte)) { + }else if (is_mmio_spte (kvm, pte)) {mmu_spte_clear_no_track (spte); }return 0 ;@@ -4184,7 +4184,7 @@ static int handle_mmio_page_fault(struct kvm_vcpu *vcpu, u64 addr, bool direct)if (WARN_ON_ONCE (reserved))return -EINVAL ; -if (is_mmio_spte (spte)) { +if (is_mmio_spte (vcpu->kvm, spte)) {gfn_t gfn =get_mmio_spte_gfn (spte);unsigned int access =get_mmio_spte_access (spte);@@ -4762,7 +4762,7 @@ EXPORT_SYMBOL_GPL(kvm_mmu_new_pgd);static bool sync_mmio_spte (struct kvm_vcpu *vcpu,u64 *sptep,gfn_t gfn,unsigned int access) { -if (unlikely (is_mmio_spte (*sptep))) { +if (unlikely (is_mmio_spte (vcpu->kvm, *sptep))) {if (gfn !=get_mmio_spte_gfn (*sptep)) {mmu_spte_clear_no_track (sptep);return true;@@ -6282,6 +6282,8 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm)void kvm_mmu_init_vm (struct kvm *kvm) { + + kvm->arch.shadow_mmio_value = shadow_mmio_value;INIT_LIST_HEAD (&kvm->arch.active_mmu_pages);INIT_LIST_HEAD (&kvm->arch.zapped_obsolete_pages);INIT_LIST_HEAD (&kvm->arch.possible_nx_huge_pages);arch/x86/kvm/mmu/spte.c +2
-2@@ -74,10 +74,10 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access)u64 spte =generation_mmio_spte_mask (gen);u64 gpa = gfn <<PAGE_SHIFT ; -WARN_ON_ONCE (!shadow_mmio_value); +WARN_ON_ONCE (!vcpu->kvm->arch.shadow_mmio_value); access &= shadow_mmio_access_mask; - spte |= shadow_mmio_value | access; + spte |= vcpu->kvm->arch.shadow_mmio_value | access; spte |= gpa | shadow_nonpresent_or_rsvd_mask; spte |= (gpa & shadow_nonpresent_or_rsvd_mask) <<SHADOW_NONPRESENT_OR_RSVD_MASK_LEN ;
arch/x86/kvm/mmu/spte.c void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) · line 415
Quoted 5 lines
arch/x86/kvm/mmu/spte.c +2
-0@@ -411,6 +411,12 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) }EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_mask); +void kvm_mmu_set_mmio_spte_value (struct kvm *kvm,u64 mmio_value) +{
Is it better to do some check on the mmio_value and warns if the value is illegal?
Quoted 47 lines
arch/x86/kvm/mmu/spte.c +4
-0+ kvm->arch.shadow_mmio_value = mmio_value; +} +EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_value); +void kvm_mmu_set_me_spte_mask (u64 me_value,u64 me_mask) {/* shadow_me_value must be a subset of shadow_me_mask */ arch/x86/kvm/mmu/spte.h +2
-2@@ -264,9 +264,9 @@ static inline struct kvm_mmu_page *root_to_sp(hpa_t root)return spte_to_child_sp (root); } -static inline bool is_mmio_spte (u64 spte) +static inline bool is_mmio_spte (struct kvm *kvm,u64 spte) { -return (spte & shadow_mmio_mask) == shadow_mmio_value && +return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&likely (enable_mmio_caching); }arch/x86/kvm/mmu/tdp_mmu.c +3
-3@@ -462,8 +462,8 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,* impact the guest since both the former and current SPTEs * are nonpresent. */ -if (WARN_ON_ONCE (!is_mmio_spte (old_spte) && - !is_mmio_spte (new_spte) && +if (WARN_ON_ONCE (!is_mmio_spte (kvm, old_spte) && + !is_mmio_spte (kvm, new_spte) && !is_removed_spte (new_spte)))pr_err ("Unexpected SPTE change! Nonpresent SPTEs\n" "should not be replaced with another,\n"@@ -978,7 +978,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, }/* If a MMIO SPTE is installed, the MMIO will need to be emulated. */ -if (unlikely (is_mmio_spte (new_spte))) { +if (unlikely (is_mmio_spte (vcpu->kvm, new_spte))) { vcpu->stat.pf_mmio_spte_created++;trace_mark_mmio_spte (rcu_dereference (iter->sptep), iter->gfn, new_spte);
Re: [PATCH v18 039/121] KVM: x86/mmu: Track shadow MMIO value on a per-VM basis
Quoted 15 lines · Binbin Wu
Quoted 15 lines
arch/x86/kvm/mmu/spte.c +2
-2@@ -74,10 +74,10 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access)u64 spte =generation_mmio_spte_mask (gen);u64 gpa = gfn <<PAGE_SHIFT ; -WARN_ON_ONCE (!shadow_mmio_value); +WARN_ON_ONCE (!vcpu->kvm->arch.shadow_mmio_value); access &= shadow_mmio_access_mask; - spte |= shadow_mmio_value | access; + spte |= vcpu->kvm->arch.shadow_mmio_value | access; spte |= gpa | shadow_nonpresent_or_rsvd_mask; spte |= (gpa & shadow_nonpresent_or_rsvd_mask) <<SHADOW_NONPRESENT_OR_RSVD_MASK_LEN ;
Quoted 8 lines
arch/x86/kvm/mmu/spte.c void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) · line 414
Quoted 5 lines
arch/x86/kvm/mmu/spte.c +2
-0@@ -411,6 +411,12 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) }EXPORT_SYMBOL_GPL (kvm_mmu_set_mmio_spte_mask); +void kvm_mmu_set_mmio_spte_value (struct kvm *kvm,u64 mmio_value) +{Is it better to do some check on the
mmio_valueand warns if the value is illegal?
I don't think so because the only caller is kvm_mmu_set_mmio_spte_value(kvm, 0) in tdx_vm_init(). I don't expect other caller.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 40/121 KVM: x86/mmu: Disallow fast page fault on private GPA
In reply to [email protected]From: Isaku Yamahata <[email protected]>
TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA.
--- arch/x86/kvm/mmu/mmu.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu/mmu.c +10 -2
@@ -3339,8 +3339,16 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; } -static bool page_fault_can_be_fast (struct kvm_page_fault *fault) +static bool page_fault_can_be_fast (struct kvm *kvm,struct kvm_page_fault *fault) { +/* +* TDX private mapping doesn't support fast page fault because the EPT +* entry is read/written with TDX SEAMCALLs instead of direct memory +* access. +*/ +if (kvm_is_private_gpa (kvm, fault->addr)) +return false; +/* * Page faults with reserved bits set, i.e. faults on MMIO SPTEs, only * reach the common page fault handler if the SPTE has an invalid MMIO
@@ -3450,7 +3458,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)u64 *sptep;uint retry_count =0 ; -if (!page_fault_can_be_fast (fault)) +if (!page_fault_can_be_fast (vcpu->kvm, fault))return ret;walk_shadow_page_lockless_begin (vcpu);
Signature · 1 line
2.25.1
Re: [PATCH v18 040/121] KVM: x86/mmu: Disallow fast page fault on private GPA
Quoted 8 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA.
Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>
Quoted 35 lines
--- arch/x86/kvm/mmu/mmu.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)arch/x86/kvm/mmu/mmu.c +10
-2@@ -3339,8 +3339,16 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; } -static bool page_fault_can_be_fast (struct kvm_page_fault *fault) +static bool page_fault_can_be_fast (struct kvm *kvm,struct kvm_page_fault *fault) { +/* +* TDX private mapping doesn't support fast page fault because the EPT +* entry is read/written with TDX SEAMCALLs instead of direct memory +* access. +*/ +if (kvm_is_private_gpa (kvm, fault->addr)) +return false; +/* * Page faults with reserved bits set, i.e. faults on MMIO SPTEs, only * reach the common page fault handler if the SPTE has an invalid MMIO @@ -3450,7 +3458,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)u64 *sptep;uint retry_count =0 ; -if (!page_fault_can_be_fast (fault)) +if (!page_fault_can_be_fast (vcpu->kvm, fault))return ret;walk_shadow_page_lockless_begin (vcpu);
Paolo Bonzini
maintainerRe: [PATCH v18 040/121] KVM: x86/mmu: Disallow fast page fault on private GPA
In reply to [email protected]Quoted 21 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Isaku Yamahata <[email protected]>
TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA.
Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)arch/x86/kvm/mmu/mmu.c +0
-0@@ -3339,8 +3339,16 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; }
arch/x86/kvm/mmu/mmu.c static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, · line 3350
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +8
-1-static bool page_fault_can_be_fast (struct kvm_page_fault *fault) +static bool page_fault_can_be_fast (struct kvm *kvm,struct kvm_page_fault *fault) { +/* +* TDX private mapping doesn't support fast page fault because the EPT +* entry is read/written with TDX SEAMCALLs instead of direct memory +* access. +*/ +if (kvm_is_private_gpa (kvm, fault->addr)) +return false;
I think this does not apply to SNP? If so, it would be better to check the SPTE against the shared-page mask inside the do...while loop.
Paolo
Quoted 15 lines
arch/x86/kvm/mmu/mmu.c +1
-2/* * Page faults with reserved bits set, i.e. faults on MMIO SPTEs, only * reach the common page fault handler if the SPTE has an invalid MMIO @@ -3450,7 +3458,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault)u64 *sptep;uint retry_count =0 ; -if (!page_fault_can_be_fast (fault)) +if (!page_fault_can_be_fast (vcpu->kvm, fault))return ret;walk_shadow_page_lockless_begin (vcpu); --2.25.1
Re: [PATCH v18 040/121] KVM: x86/mmu: Disallow fast page fault on private GPA
Quoted 22 lines · Paolo Bonzini
Quoted 21 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Isaku Yamahata <[email protected]>
TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA.
Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)arch/x86/kvm/mmu/mmu.c +0
-0@@ -3339,8 +3339,16 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; }
Quoted 13 lines
arch/x86/kvm/mmu/mmu.c static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, · line 3350
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +8
-1-static bool page_fault_can_be_fast (struct kvm_page_fault *fault) +static bool page_fault_can_be_fast (struct kvm *kvm,struct kvm_page_fault *fault) { +/* +* TDX private mapping doesn't support fast page fault because the EPT +* entry is read/written with TDX SEAMCALLs instead of direct memory +* access. +*/ +if (kvm_is_private_gpa (kvm, fault->addr)) +return false;I think this does not apply to SNP? If so, it would be better to check the SPTE against the shared-page mask inside the do...while loop.
No, this won't apply to SNP. Let me update the patch corresponding in v19.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v18 040/121] KVM: x86/mmu: Disallow fast page fault on private GPA
Quoted 39 lines · Isaku Yamahata
Quoted 22 lines · Paolo Bonzini
Quoted 21 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Isaku Yamahata <[email protected]>
TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA.
Signed-off-by Isaku Yamahata <[email protected]>Reviewed-by Paolo Bonzini <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)arch/x86/kvm/mmu/mmu.c +0
-0@@ -3339,8 +3339,16 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu,return RET_PF_CONTINUE ; }Quoted 13 lines
arch/x86/kvm/mmu/mmu.c static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, · line 3350
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +8
-1-static bool page_fault_can_be_fast (struct kvm_page_fault *fault) +static bool page_fault_can_be_fast (struct kvm *kvm,struct kvm_page_fault *fault) { +/* +* TDX private mapping doesn't support fast page fault because the EPT +* entry is read/written with TDX SEAMCALLs instead of direct memory +* access. +*/ +if (kvm_is_private_gpa (kvm, fault->addr)) +return false;I think this does not apply to SNP? If so, it would be better to check the SPTE against the shared-page mask inside the do...while loop.
Quoted 1 line
No, this won't apply to SNP. Let me update the patch corresponding in v19.
shared-page mask is against GPA or faulting address. Not SPTE unlike SNP. So it doesn't make sense to check inside the do..while loop.
Signature · 1 line
Isaku Yamahata <[email protected]>
PATCH v18 41/121 KVM: x86/mmu: Allow per-VM override of the TDP max page level
In reply to [email protected]From: Sean Christopherson <[email protected]>
TDX requires special handling to support large private page. For simplicity, only support 4K page for TD guest for now. Add per-VM maximum page level support to support different maximum page sizes for TD guest and conventional VMX guest.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 2 ++ arch/x86/kvm/mmu/mmu_internal.h | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1283,6 +1283,7 @@ struct kvm_arch {unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;unsigned int indirect_shadow_pages; +int tdp_max_page_level;u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages;
arch/x86/kvm/mmu/mmu.c +2 -0
@@ -6307,6 +6307,8 @@ void kvm_mmu_init_vm(struct kvm *kvm) kvm->arch.split_desc_cache.kmem_cache = pte_list_desc_cache; kvm->arch.split_desc_cache.gfp_zero = __GFP_ZERO; + + kvm->arch.tdp_max_page_level =KVM_MAX_HUGEPAGE_LEVEL ; }static void mmu_free_vm_memory_caches (struct kvm *kvm)
arch/x86/kvm/mmu/mmu_internal.h +1 -1
@@ -296,7 +296,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(vcpu->kvm), - .max_level = KVM_MAX_HUGEPAGE_LEVEL, + .max_level = vcpu->kvm->arch.tdp_max_page_level, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, };
Signature · 1 line
2.25.1
Re: [PATCH v18 041/121] KVM: x86/mmu: Allow per-VM override of the TDP max page level
Quoted 21 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Sean Christopherson <[email protected]>
TDX requires special handling to support large private page. For simplicity, only support 4K page for TD guest for now. Add per-VM maximum page level support to support different maximum page sizes for TD guest and conventional VMX guest.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Acked-by Kai Huang <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 2 ++ arch/x86/kvm/mmu/mmu_internal.h | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm_host.h +0
-0
arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1286
Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1
-0@@ -1283,6 +1283,7 @@ struct kvm_arch {unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;unsigned int indirect_shadow_pages; +int tdp_max_page_level;
Although only TDX need special handling for now, and TDX always use TDP, but it doesn't necessarily to be TDP, right?
When the value is assigned to kvm_page_fault.max_level, it is also used for non-TDP code path.
Quoted 29 lines
arch/x86/include/asm/kvm_host.h +0
-0u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages;arch/x86/kvm/mmu/mmu.c +2
-0@@ -6307,6 +6307,8 @@ void kvm_mmu_init_vm(struct kvm *kvm) kvm->arch.split_desc_cache.kmem_cache = pte_list_desc_cache; kvm->arch.split_desc_cache.gfp_zero = __GFP_ZERO; + + kvm->arch.tdp_max_page_level =KVM_MAX_HUGEPAGE_LEVEL ; }static void mmu_free_vm_memory_caches (struct kvm *kvm)arch/x86/kvm/mmu/mmu_internal.h +1
-1@@ -296,7 +296,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(vcpu->kvm), - .max_level = KVM_MAX_HUGEPAGE_LEVEL, + .max_level = vcpu->kvm->arch.tdp_max_page_level, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, };
Paolo Bonzini
maintainerRe: [PATCH v18 041/121] KVM: x86/mmu: Allow per-VM override of the TDP max page level
In reply to [email protected]Quoted 11 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Sean Christopherson <[email protected]>
TDX requires special handling to support large private page. For simplicity, only support 4K page for TD guest for now. Add per-VM maximum page level support to support different maximum page sizes for TD guest and conventional VMX guest.
Signed-off-by Sean Christopherson <[email protected]>Signed-off-by Isaku Yamahata <[email protected]>Acked-by Kai Huang <[email protected]>
Please reimplement this on top of "KVM: x86: Add gmem hook for determining max NPT mapping level" from the SEV-SNP series.
Paolo
Quoted 47 lines
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 2 ++ arch/x86/kvm/mmu/mmu_internal.h | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-)arch/x86/include/asm/kvm_host.h +1
-0@@ -1283,6 +1283,7 @@ struct kvm_arch {unsigned long n_requested_mmu_pages;unsigned long n_max_mmu_pages;unsigned int indirect_shadow_pages; +int tdp_max_page_level;u8 mmu_valid_gen;struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES ];struct list_head active_mmu_pages;arch/x86/kvm/mmu/mmu.c +2
-0@@ -6307,6 +6307,8 @@ void kvm_mmu_init_vm(struct kvm *kvm) kvm->arch.split_desc_cache.kmem_cache = pte_list_desc_cache; kvm->arch.split_desc_cache.gfp_zero = __GFP_ZERO; + + kvm->arch.tdp_max_page_level =KVM_MAX_HUGEPAGE_LEVEL ; }static void mmu_free_vm_memory_caches (struct kvm *kvm)arch/x86/kvm/mmu/mmu_internal.h +1
-2@@ -296,7 +296,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(vcpu->kvm), - .max_level = KVM_MAX_HUGEPAGE_LEVEL, + .max_level = vcpu->kvm->arch.tdp_max_page_level, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, }; --2.25.1
PATCH v18 42/121 KVM: VMX: Introduce test mode related to EPT violation VE
In reply to [email protected]From: Isaku Yamahata <[email protected]>
To support TDX, KVM is enhanced to operate with #VE. For TDX, KVM programs to inject #VE conditionally and set #VE suppress bit in EPT entry. For VMX case, #VE isn't used. If #VE happens for VMX, it's a bug. To be defensive (test that VMX case isn't broken), introduce option ept_violation_ve_test and when it's set, set error.
--- arch/x86/include/asm/vmx.h | 12 +++++++ arch/x86/kvm/vmx/vmcs.h | 5 +++ arch/x86/kvm/vmx/vmx.c | 69 +++++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 6 +++- 4 files changed, 90 insertions(+), 2 deletions(-)
arch/x86/include/asm/vmx.h +12 -0
@@ -70,6 +70,7 @@#define SECONDARY_EXEC_ENCLS_EXITING VMCS_CONTROL_BIT(ENCLS_EXITING)#define SECONDARY_EXEC_RDSEED_EXITING VMCS_CONTROL_BIT(RDSEED_EXITING)#define SECONDARY_EXEC_ENABLE_PML VMCS_CONTROL_BIT(PAGE_MOD_LOGGING) +#define SECONDARY_EXEC_EPT_VIOLATION_VE VMCS_CONTROL_BIT(EPT_VIOLATION_VE)#define SECONDARY_EXEC_PT_CONCEAL_VMX VMCS_CONTROL_BIT(PT_CONCEAL_VMX)#define SECONDARY_EXEC_ENABLE_XSAVES VMCS_CONTROL_BIT(XSAVES)#define SECONDARY_EXEC_MODE_BASED_EPT_EXEC VMCS_CONTROL_BIT(MODE_BASED_EPT_EXEC)
@@ -225,6 +226,8 @@ enum vmcs_field {VMREAD_BITMAP_HIGH =0x00002027 ,VMWRITE_BITMAP =0x00002028 ,VMWRITE_BITMAP_HIGH =0x00002029 , +VE_INFORMATION_ADDRESS =0x0000202A , +VE_INFORMATION_ADDRESS_HIGH =0x0000202B ,XSS_EXIT_BITMAP =0x0000202C ,XSS_EXIT_BITMAP_HIGH =0x0000202D , ENCLS_EXITING_BITMAP = 0x0000202E,
@@ -630,4 +633,13 @@ enum vmx_l1d_flush_state {extern enum vmx_l1d_flush_state l1tf_vmx_mitigation; +struct vmx_ve_information { +u32 exit_reason; +u32 delivery; +u64 exit_qualification; +u64 guest_linear_address; +u64 guest_physical_address; +u16 eptp_index; +}; +#endif
arch/x86/kvm/vmx/vmcs.h +5 -0
@@ -140,6 +140,11 @@ static inline bool is_nm_fault(u32 intr_info)return is_exception_n (intr_info,NM_VECTOR ); } +static inline bool is_ve_fault (u32 intr_info) +{ +return is_exception_n (intr_info,VE_VECTOR ); +} +/* Undocumented: icebp/int1 */ static inline bool is_icebp (u32 intr_info) {
arch/x86/kvm/vmx/vmx.c +68 -1
124 lines not on this page · raw message
arch/x86/kvm/vmx/vmx.h +5 -1
@@ -349,6 +349,9 @@ struct vcpu_vmx {DECLARE_BITMAP (read,MAX_POSSIBLE_PASSTHROUGH_MSRS );DECLARE_BITMAP (write,MAX_POSSIBLE_PASSTHROUGH_MSRS ); } shadow_msr_intercept; + +/* ve_info must be page aligned. */ +struct vmx_ve_information *ve_info; };struct kvm_vmx {
@@ -561,7 +564,8 @@ static inline u8 vmx_get_rvi(void) SECONDARY_EXEC_ENABLE_VMFUNC | \ SECONDARY_EXEC_BUS_LOCK_DETECTION | \ SECONDARY_EXEC_NOTIFY_VM_EXITING | \ - SECONDARY_EXEC_ENCLS_EXITING) + SECONDARY_EXEC_ENCLS_EXITING | \ + SECONDARY_EXEC_EPT_VIOLATION_VE)#define KVM_REQUIRED_VMX_TERTIARY_VM_EXEC_CONTROL 0#define KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL \
Signature · 1 line
2.25.1
PATCH v18 43/121 The start of TDX KVM patch series: KVM TDP MMU hooks
In reply to [email protected]From: Isaku Yamahata <[email protected]>
This empty commit is to mark the start of patch series of KVM TDP MMU hooks.
--- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/virt/kvm/intel-tdx-layer-status.rst +2 -2
@@ -26,5 +26,5 @@ Patch Layer status * TD vcpu interrupts/exit/hypercall: Not yet * KVM MMU GPA shared bits: Applied -* KVM TDP refactoring for TDX: Applying -* KVM TDP MMU hooks: Not yet +* KVM TDP refactoring for TDX: Applied +* KVM TDP MMU hooks: Applying
Signature · 1 line
2.25.1
PATCH v18 44/121 KVM: x86/mmu: Assume guest MMIOs are shared
In reply to [email protected]From: Chao Gao <[email protected]>
Guest TD doesn't necessarily invoke MAP_GPA to convert the virtual MMIO range to shared before accessing it. When TD tries to access the virtual device's MMIO as shared, an EPT violation is raised first. kvm_mem_is_private() checks whether the GFN is shared or private. If MAP_GPA is not called for the GPA, KVM thinks the GPA is private and refuses shared access, and doesn't set up shared EPT entry. The guest can't make progress.
Instead of requiring the guest to invoke MAP_GPA for regions of virtual MMIOs assume regions of virtual MMIOs are shared in KVM as well (i.e., GPAs either have no kvm_memory_slot or are backed by host MMIOs). So that guests can access those MMIO regions.
--- arch/x86/kvm/mmu/mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +6 -1
@@ -4371,7 +4371,12 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultreturn RET_PF_EMULATE ; } -if (fault->is_private !=kvm_mem_is_private (vcpu->kvm, fault->gfn)) { +/* +* !fault->slot means MMIO. Don't require explicit GPA conversion for +* MMIO because MMIO is assigned at the boot time. +*/ +if (fault->slot && + fault->is_private !=kvm_mem_is_private (vcpu->kvm, fault->gfn)) {if (vcpu->kvm->arch.vm_type ==KVM_X86_SW_PROTECTED_VM )return RET_PF_RETRY ;kvm_mmu_prepare_memory_fault_exit (vcpu, fault);
Signature · 1 line
2.25.1
Paolo Bonzini
maintainerRe: [PATCH v18 044/121] KVM: x86/mmu: Assume guest MMIOs are shared
Quoted 11 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Chao Gao <[email protected]>
Guest TD doesn't necessarily invoke
MAP_GPAto convert the virtual MMIO range to shared before accessing it. When TD tries to access the virtual device's MMIO as shared, an EPT violation is raised first.kvm_mem_is_private()checks whether the GFN is shared or private. IfMAP_GPAis not called for the GPA, KVM thinks the GPA is private and refuses shared access, and doesn't set up shared EPT entry. The guest can't make progress.
Quoted 4 lines
Instead of requiring the guest to invoke
MAP_GPAfor regions of virtual MMIOs assume regions of virtual MMIOs are shared in KVM as well (i.e., GPAs either have nokvm_memory_slotor are backed by host MMIOs). So that guests can access those MMIO regions.
I'm not sure how the patch below deals with host MMIOs?
Quoted 1 line
Signed-off-by Chao Gao <[email protected]>
Missing Signed-off-by.
Also, this patch conflicts with "[PATCH v11 09/35] KVM: x86: Determine shared/private faults based on vm_type". I think in general the logic in that patch (which forces an exit to userspace if needed, to convert the MMIO area to shared) can be applied to sw-protected and TDX guests as well. I'm preparing a set of common patches that can be applied for 6.9 and will include something after testing with sw-protected VMs.
Paolo
Quoted 25 lines
--- arch/x86/kvm/mmu/mmu.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)arch/x86/kvm/mmu/mmu.c +6
-2@@ -4371,7 +4371,12 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *faultreturn RET_PF_EMULATE ; } -if (fault->is_private !=kvm_mem_is_private (vcpu->kvm, fault->gfn)) { +/* +* !fault->slot means MMIO. Don't require explicit GPA conversion for +* MMIO because MMIO is assigned at the boot time. +*/ +if (fault->slot && + fault->is_private !=kvm_mem_is_private (vcpu->kvm, fault->gfn)) {if (vcpu->kvm->arch.vm_type ==KVM_X86_SW_PROTECTED_VM )return RET_PF_RETRY ;kvm_mmu_prepare_memory_fault_exit (vcpu, fault); --2.25.1
Re: [PATCH v18 044/121] KVM: x86/mmu: Assume guest MMIOs are shared
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Chao Gao <[email protected]>
Guest TD doesn't necessarily invoke
MAP_GPAto convert the virtual MMIO range to shared before accessing it. When TD tries to access the virtual device's MMIO as shared, an EPT violation is raised first.kvm_mem_is_private()checks whether the GFN is shared or private. IfMAP_GPAis not called for the GPA, KVM thinks the GPA is private and refuses shared access, and doesn't set up shared EPT entry. The guest can't make progress.
Quoted 6 lines
Quoted 4 lines
Instead of requiring the guest to invoke
MAP_GPAfor regions of virtual MMIOs assume regions of virtual MMIOs are shared in KVM as well (i.e., GPAs either have nokvm_memory_slotor are backed by host MMIOs). So that guests can access those MMIO regions.I'm not sure how the patch below deals with host MMIOs?
It falls back to shared case to hit KVM_PFN_NOSLOT. It will be handled as MMIO.
Anyway I found it breaks SW_PROTECTED case. So I came up with the following. I think we'd like to handle as
- SW_PROTECTED => KVM_EXIT_MEMORY_FAULT
- SNP, TDX => MMIO.
- if (fault->is_private != kvm_mem_is_private(vcpu->kvm, fault->gfn)) {
+ /*
+ * !fault->slot means MMIO for SNP and TDX. Don't require explicit GPA + * conversion for MMIO because MMIO is assigned at the boot time. Fall + * to !is_private case to get pfn = KVM_PFN_NOSLOT. + */
+ force_mmio = !slot &&
+ vcpu->kvm->arch.vm_type != KVM_X86_DEFAULT_VM &&
+ vcpu->kvm->arch.vm_type != KVM_X86_SW_PROTECTED_VM;
+ if (!force_mmio &&
+ fault->is_private != kvm_mem_is_private(vcpu->kvm, fault->gfn)) {
kvm_mmu_prepare_memory_fault_exit(vcpu, fault);
return -EFAULT;
}- if (fault->is_private)
+ if (!force_mmio && fault->is_private)
return kvm_faultin_pfn_private(vcpu, fault);
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v18 044/121] KVM: x86/mmu: Assume guest MMIOs are shared
Quoted 22 lines · Isaku Yamahata
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · On Tue, Jan 23, 2024 at 12:55 AM <[email protected]> wrote:
From: Chao Gao <[email protected]>
Guest TD doesn't necessarily invoke
MAP_GPAto convert the virtual MMIO range to shared before accessing it. When TD tries to access the virtual device's MMIO as shared, an EPT violation is raised first.kvm_mem_is_private()checks whether the GFN is shared or private. IfMAP_GPAis not called for the GPA, KVM thinks the GPA is private and refuses shared access, and doesn't set up shared EPT entry. The guest can't make progress.Quoted 6 lines
Quoted 4 lines
Instead of requiring the guest to invoke
MAP_GPAfor regions of virtual MMIOs assume regions of virtual MMIOs are shared in KVM as well (i.e., GPAs either have nokvm_memory_slotor are backed by host MMIOs). So that guests can access those MMIO regions.I'm not sure how the patch below deals with host MMIOs?
Quoted 8 lines
It falls back to shared case to hit
KVM_PFN_NOSLOT. It will be handled as MMIO.Anyway I found it breaks
SW_PROTECTEDcase. So I came up with the following. I think we'd like to handle as
-SW_PROTECTED=>KVM_EXIT_MEMORY_FAULT
- SNP, TDX => MMIO.
FFS. Stop lobbing patch bombs and start having actual conversations.
Seriously, the whole point of using mailing lists is to have *discussions* and to coordinate development. Throwing patches at kvm@ and then walking away DOES NOT WORK.
Putting a "TODO: Drop this patch once the common patch is merged." in the changelog[1] is not helpful.
Dropping a proposed common uAPI[2] into a 121 patch series without even *acknowledging* that you received the message DOES NOT WORK. You didn't even add a Suggested-by or Cc: the people who expressed interest. I can't read minds, and AFAIK no one else working on KVM is a telepath either.
I do not know to make it any clearer: for TDX support to go anywhere, there needs to be a _lot_ more communication.
[1] https://lore.kernel.org/all/b2e5c92fd66a0113b472dd602220346d3d435732.1708933498.git.isaku.yamahata@intel.com
[2] https://lore.kernel.org/all/8b7380f1b02f8e3995f18bebb085e43165d5d682.1708933498.git.isaku.yamahata@intel.com
Review
Quoted 7 lines
(quoted diff) +6
-1- if (fault->is_private != kvm_mem_is_private(vcpu->kvm, fault->gfn)) { + /* + * !fault->slot means MMIO for SNP and TDX. Don't require explicit GPA + * conversion for MMIO because MMIO is assigned at the boot time. Fall + * to !is_private case to get pfn = KVM_PFN_NOSLOT. + */ + force_mmio = !slot &&
NAK, this already got shot down.
Quoted 14 lines
(quoted diff) +5
-1+ vcpu->kvm->arch.vm_type != KVM_X86_DEFAULT_VM && + vcpu->kvm->arch.vm_type != KVM_X86_SW_PROTECTED_VM; + if (!force_mmio && + fault->is_private != kvm_mem_is_private(vcpu->kvm, fault->gfn)) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return -EFAULT; } - if (fault->is_private) + if (!force_mmio && fault->is_private) return kvm_faultin_pfn_private(vcpu, fault);Signature · 2 lines
-- Isaku Yamahata <[email protected]>
PATCH v18 45/121 KVM: x86/tdp_mmu: Init role member of struct kvm_mmu_page at allocation
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Refactor tdp_mmu_alloc_sp() and tdp_mmu_init_sp and eliminate tdp_mmu_init_child_sp(). Currently tdp_mmu_init_sp() (or tdp_mmu_init_child_sp()) sets kvm_mmu_page.role after tdp_mmu_alloc_sp() allocating struct kvm_mmu_page and its page table page. This patch makes tdp_mmu_alloc_sp() initialize kvm_mmu_page.role instead of tdp_mmu_init_sp().
To handle private page tables, argument of is_private needs to be passed down. Given that already page level is passed down, it would be cumbersome to add one more parameter about sp. Instead replace the level argument with union kvm_mmu_page_role. Thus the number of argument won't be increased and more info about sp can be passed down.
For private sp, secure page table will be also allocated in addition to struct kvm_mmu_page and page table (spt member). The allocation functions (tdp_mmu_alloc_sp() and __tdp_mmu_alloc_sp_for_split()) need to know if the allocation is for the conventional page table or private page table. Pass union kvm_mmu_role to those functions and initialize role member of struct kvm_mmu_page.
--- arch/x86/kvm/mmu/tdp_iter.h | 12 ++++++++++ arch/x86/kvm/mmu/tdp_mmu.c | 44 ++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 25 deletions(-)
arch/x86/kvm/mmu/tdp_iter.h +12 -0
@@ -135,4 +135,16 @@ void tdp_iter_start(struct tdp_iter *iter, struct kvm_mmu_page *root,void tdp_iter_next (struct tdp_iter *iter);void tdp_iter_restart (struct tdp_iter *iter); +static inline union kvm_mmu_page_role tdp_iter_child_role (struct tdp_iter *iter) +{ +union kvm_mmu_page_role child_role; +struct kvm_mmu_page *parent_sp; + + parent_sp =sptep_to_sp (rcu_dereference (iter->sptep)); + + child_role = parent_sp->role; + child_role.level--; +return child_role; +} +#endif /* __KVM_X86_MMU_TDP_ITER_H */
arch/x86/kvm/mmu/tdp_mmu.c +19 -25
119 lines not on this page · raw message
Signature · 1 line
2.25.1
PATCH v18 46/121 KVM: x86/mmu: Add a new is_private member for union kvm_mmu_page_role
In reply to [email protected]From: Isaku Yamahata <[email protected]>
Because TDX support introduces private mapping, add a new member in union kvm_mmu_page_role with access functions to check the member.
--- arch/x86/include/asm/kvm_host.h | 27 +++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu_internal.h | 5 +++++ arch/x86/kvm/mmu/spte.h | 6 ++++++ 3 files changed, 38 insertions(+)
arch/x86/include/asm/kvm_host.h +27 -0
@@ -349,7 +349,12 @@ union kvm_mmu_page_role { unsigned ad_disabled:1; unsigned guest_mode:1; unsigned passthrough:1; +#ifdef CONFIG_KVM_MMU_PRIVATE + unsigned is_private:1; + unsigned :4; +#else unsigned :5; +#endif /* * This is left at the top of the word so that
@@ -361,6 +366,28 @@ union kvm_mmu_page_role { }; }; +#ifdef CONFIG_KVM_MMU_PRIVATE +static inline bool kvm_mmu_page_role_is_private (union kvm_mmu_page_role role) +{ +return !!role.is_private; +} + +static inline void kvm_mmu_page_role_set_private (union kvm_mmu_page_role *role) +{ + role->is_private =1 ; +} +#else +static inline bool kvm_mmu_page_role_is_private (union kvm_mmu_page_role role) +{ +return false; +} + +static inline void kvm_mmu_page_role_set_private (union kvm_mmu_page_role *role) +{ +WARN_ON_ONCE (1 ); +} +#endif +/* * kvm_mmu_extended_role complements kvm_mmu_page_role, tracking properties * relevant to the current MMU configuration. When loading CR0, CR4, or EFER,
arch/x86/kvm/mmu/mmu_internal.h +5 -0
@@ -145,6 +145,11 @@ static inline int kvm_mmu_page_as_id(struct kvm_mmu_page *sp)return kvm_mmu_role_as_id (sp->role); } +static inline bool is_private_sp (const struct kvm_mmu_page *sp) +{ +return kvm_mmu_page_role_is_private (sp->role); +} +static inline bool kvm_mmu_page_ad_need_write_protect (struct kvm_mmu_page *sp) {/*
arch/x86/kvm/mmu/spte.h +6 -0
@@ -264,6 +264,12 @@ static inline struct kvm_mmu_page *root_to_sp(hpa_t root)return spte_to_child_sp (root); } +static inline bool is_private_sptep (u64 *sptep) +{ +WARN_ON_ONCE (!sptep); +return is_private_sp (sptep_to_sp (sptep)); +} +static inline bool is_mmio_spte (struct kvm *kvm,u64 spte) { return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&
Signature · 1 line
2.25.1
Re: [PATCH v18 046/121] KVM: x86/mmu: Add a new is_private member for union kvm_mmu_page_role
Quoted 79 lines · On 1/23/2024 7:53 AM, [email protected] wrote:
From: Isaku Yamahata <[email protected]>
Because TDX support introduces private mapping, add a new member in union
kvm_mmu_page_rolewith access functions to check the member.Signed-off-by Isaku Yamahata <[email protected]>--- arch/x86/include/asm/kvm_host.h | 27 +++++++++++++++++++++++++++ arch/x86/kvm/mmu/mmu_internal.h | 5 +++++ arch/x86/kvm/mmu/spte.h | 6 ++++++ 3 files changed, 38 insertions(+)arch/x86/include/asm/kvm_host.h +27
-0@@ -349,7 +349,12 @@ union kvm_mmu_page_role { unsigned ad_disabled:1; unsigned guest_mode:1; unsigned passthrough:1; +#ifdef CONFIG_KVM_MMU_PRIVATE + unsigned is_private:1; + unsigned :4; +#else unsigned :5; +#endif /* * This is left at the top of the word so that @@ -361,6 +366,28 @@ union kvm_mmu_page_role { }; }; +#ifdef CONFIG_KVM_MMU_PRIVATE +static inline bool kvm_mmu_page_role_is_private (union kvm_mmu_page_role role) +{ +return !!role.is_private; +} + +static inline void kvm_mmu_page_role_set_private (union kvm_mmu_page_role *role) +{ + role->is_private =1 ; +} +#else +static inline bool kvm_mmu_page_role_is_private (union kvm_mmu_page_role role) +{ +return false; +} + +static inline void kvm_mmu_page_role_set_private (union kvm_mmu_page_role *role) +{ +WARN_ON_ONCE (1 ); +} +#endif +/* * kvm_mmu_extended_role complements kvm_mmu_page_role, tracking properties * relevant to the current MMU configuration. When loading CR0, CR4, or EFER, arch/x86/kvm/mmu/mmu_internal.h +5
-0@@ -145,6 +145,11 @@ static inline int kvm_mmu_page_as_id(struct kvm_mmu_page *sp)return kvm_mmu_role_as_id (sp->role); } +static inline bool is_private_sp (const struct kvm_mmu_page *sp) +{ +return kvm_mmu_page_role_is_private (sp->role); +} +static inline bool kvm_mmu_page_ad_need_write_protect (struct kvm_mmu_page *sp) {/* arch/x86/kvm/mmu/spte.h +0
-0
arch/x86/kvm/mmu/spte.h static inline struct kvm_mmu_page *root_to_sp(hpa_t root) · line 269
Quoted 6 lines
arch/x86/kvm/mmu/spte.h +3
-0@@ -264,6 +264,12 @@ static inline struct kvm_mmu_page *root_to_sp(hpa_t root)return spte_to_child_sp (root); } +static inline bool is_private_sptep (u64 *sptep) +{ +WARN_ON_ONCE (!sptep);
If sptep is NULL, should return here, otherwise, the following code will de-reference a illegal pointer.
Quoted 6 lines
arch/x86/kvm/mmu/spte.h +3
-0+return is_private_sp (sptep_to_sp (sptep)); +} +static inline bool is_mmio_spte (struct kvm *kvm,u64 spte) { return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value &&