QEMU Guest memfd + QEMU TDX support
Subsystems (5)
v4 cover QEMU Guest memfd + QEMU TDX support
This v4 series still carry both QEMU guest memfd support and QEMU TDX support. Because TDX is going to be the first user of guest memfd in QEMU, put them together can provide us a full picture of how they work.
KVM provides guest memfd, which cannot be mapped, read, or written by userspace. It's designed to serve as private memory for confidential VMs, like Intel TDX and AMD sev-snp.
Patches 1 - 9 add support of guest memfd into QEMU, associating it with RAMBlock. For the VM types that require private memory (see tdx_kvm_init() in patch 14), QEMU will automatically create guest memfd for each RAM backend.
Follwoing patches 10 to 66, enables TDX support to allow creating and booting a TD (TDX VM) with QEMU.
This series needs to work with KVM TDX v18 series:
https://lore.kernel.org/all/[email protected]/T/#u
(TDX host kernel support and guest memfd KVM support lands in Linux v6.8-rc1.)
===
This series is also available in github:
https://github.com/intel/qemu-tdx/tree/tdx-qemu-upstream-v4
It's based on several patches that haven't get merged:
https://lore.kernel.org/qemu-devel/[email protected]/
https://lore.kernel.org/qemu-devel/[email protected]/
https://lore.kernel.org/all/[email protected]/
Luckily, the absence of them doesn't block applying this series nor affecting the functionality.
== Limitation and future work ==
- Readonly memslot
TDX only support readonly (write protection) memslot for shared memory, but not for private memory. For simplicity, just mark readonly memslot not supported entirely for TDX.
- CPU model
We cannot create a TD with arbitrary CPU model like what for non-TDX VMs, because only a subset of features can be configured for TD.
- It's recommended to use '-cpu host' to create TD;
- '+feature/-feature' might not work as expected;
future work: To introduce specific CPU model for TDs and enhance ±features
for TDs.
- gdb suppport
gdb support to debug a TD of off-debug mode is future work.
===
Main changes in v4:
guset memfd part:
- Drop the patch that supports KVM_GUEST_MEMFD_ALLOW_HUGEPAGE flag because KVM side leaves it as future work;
- send patch "physmem: replace function name with __func__ in ram_block_discard_range()" separately;
- drop ram_block_convert_range() and open code it instead.
- move the supported attribute bit check into kvm_set_memory_attributes() (per request from Wei)
- Clarify more for the desgin consideration in the commit messages
TDX part:
- Re-implement get-quote support by introduing TdxQuoteGenerator object and make the socket communication non-blocking; (per Daniel)
- Various code and doc refinement;
v3:
https://lore.kernel.org/qemu-devel/[email protected]/
Main changes in v3:
gmem memfd part:
- Since KVM side renamed gmem to guest_memfd in the uapi, this version renames it accordingly;
- Drop the 'private' property of memory backend. (see comment[1]) Now QEMU decides whether need to create guest memfd based on specific vm type (or specific VM implementation, please see patch *X* and *Y*);
- Drop sw_protected_vm implementation;
TDX part:
- improve the error report in various patches by utilizing 'errp';
- drop the vm-type interface;
- rename __tdx_ioctl() to tdx_ioctl_internal();
- refine the description of 'sept-ve-disable' in qom.json;
- use base64 for mrconfigif/mrowner/mrownerconfig instread of hex-string;
- use type SocketAddress for quote-generation-service;
[1] https://lore.kernel.org/qemu-devel/[email protected]/
Chao Peng (3):
kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
kvm: handle KVM_EXIT_MEMORY_FAULT
i386/tdx: register TDVF as private memory
Isaku Yamahata (14):
trace/kvm: Add trace for page convertion between shared and private i386/tdx: Make sept_ve_disable set by default
i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig kvm/tdx: Don't complain when converting vMMIO region to shared kvm/tdx: Ignore memory conversion to shared of unassigned region i386/tdvf: Introduce function to parse TDVF metadata i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt> i386/tdx: handle TDG.VP.VMCALL<GetQuote>
i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall pci-host/q35: Move PAM initialization above SMRAM initialization
q35: Introduce smm_ranges property for q35-pci-host hw/i386: add option to forcibly report edge trigger in acpi tables i386/tdx: Don't synchronize guest tsc for TDs
Sean Christopherson (2):
i386/kvm: Move architectural CPUID leaf generation to separate helper i386/tdx: Don't get/put guest state for TDX VMs
Xiaoyao Li (47):
linux-headers: Update to Linux v6.8-rc1
RAMBlock: Add support of KVM private guest memfd
HostMem: Add mechanism to opt in kvm guest memfd via MachineState trace/kvm: Split address space and slot id in
trace_kvm_set_user_memory()
kvm: Introduce support for memory_attributes
physmem: Introduce ram_block_discard_guest_memfd_range() *** HACK *** linux-headers: Update headers to pull in TDX API changes
i386: Introduce tdx-guest object
target/i386: Implement mc->kvm_type() to get VM type target/i386: Introduce kvm_confidential_guest_init() i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object i386/tdx: Adjust the supported CPUID based on TDX restrictions i386/tdx: Make Intel-PT unsupported for TD guest
i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by
tdx_caps.cpuid_config[]
i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup
kvm: Introduce kvm_arch_pre_create_vcpu()
i386/tdx: Initialize TDX before creating TD vcpus i386/tdx: Add property sept-ve-disable for tdx-guest object i386/tdx: Wire CPU features up with attributes of TD guest i386/tdx: Validate TD attributes
i386/tdx: Implement user specified tsc frequency
i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM kvm/memory: Introduce the infrastructure to set the default
shared/private value
i386/tdx: Make memory type private by default
i386/tdx: Parse TDVF metadata for TDX VM
i386/tdx: Skip BIOS shadowing setup
i386/tdx: Don't initialize pc.rom for TDX VMs
i386/tdx: Track mem_ptr for each firmware entry of TDVF i386/tdx: Track RAM entries for TDX VM
headers: Add definitions from UEFI spec for volumes, resources, etc... i386/tdx: Setup the TD HOB list
memory: Introduce memory_region_init_ram_guest_memfd() i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu i386/tdx: Finalize TDX VM
i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR> i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility i386/tdx: Disable SMM for TDX VMs
i386/tdx: Disable PIC for TDX VMs
i386/tdx: Don't allow system reset for TDX VMs
i386/tdx: LMCE is not supported for TDX
hw/i386: add eoi_intercept_unsupported member to X86MachineState i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs i386/tdx: Skip kvm_put_apicbase() for TDs
docs: Add TDX documentation
accel/kvm/kvm-all.c | 278 +++- accel/kvm/trace-events | 3 +- accel/stubs/kvm-stub.c | 5 + backends/hostmem-file.c | 1 + backends/hostmem-memfd.c | 1 + backends/hostmem-ram.c | 1 + backends/hostmem.c | 1 + configs/devices/i386-softmmu/default.mak | 1 + docs/system/confidential-guest-support.rst | 1 + docs/system/i386/tdx.rst | 113 ++ docs/system/target-i386.rst | 1 + hw/core/machine.c | 5 + hw/i386/Kconfig | 6 + hw/i386/acpi-build.c | 99 +- hw/i386/acpi-common.c | 50 +- hw/i386/meson.build | 1 + hw/i386/pc.c | 21 +- hw/i386/pc_q35.c | 2 + hw/i386/pc_sysfw.c | 7 + hw/i386/tdvf-hob.c | 147 ++ hw/i386/tdvf-hob.h | 24 + hw/i386/tdvf.c | 200 +++ hw/i386/x86.c | 51 +- hw/pci-host/q35.c | 61 +- include/exec/cpu-common.h | 2 + include/exec/memory.h | 33 +- include/exec/ram_addr.h | 2 +- include/exec/ramblock.h | 1 + include/hw/boards.h | 2 + include/hw/i386/pc.h | 1 + include/hw/i386/tdvf.h | 58 + include/hw/i386/x86.h | 2 + include/hw/pci-host/q35.h | 1 + include/standard-headers/drm/drm_fourcc.h | 10 +- include/standard-headers/linux/ethtool.h | 41 +- .../standard-headers/linux/virtio_config.h | 8 +- include/standard-headers/linux/virtio_pci.h | 68 + include/standard-headers/linux/virtio_pmem.h | 7 + include/standard-headers/uefi/uefi.h | 198 +++ include/sysemu/hostmem.h | 1 + include/sysemu/kvm.h | 8 + include/sysemu/kvm_int.h | 2 + linux-headers/asm-generic/unistd.h | 15 +- linux-headers/asm-loongarch/kvm.h | 1 + linux-headers/asm-mips/mman.h | 2 +- linux-headers/asm-mips/unistd_n32.h | 5 + linux-headers/asm-mips/unistd_n64.h | 5 + linux-headers/asm-mips/unistd_o32.h | 5 + linux-headers/asm-powerpc/unistd_32.h | 5 + linux-headers/asm-powerpc/unistd_64.h | 5 + linux-headers/asm-riscv/kvm.h | 13 + linux-headers/asm-s390/unistd_32.h | 5 + linux-headers/asm-s390/unistd_64.h | 5 + linux-headers/asm-x86/kvm.h | 94 ++ linux-headers/asm-x86/unistd_32.h | 5 + linux-headers/asm-x86/unistd_64.h | 5 + linux-headers/asm-x86/unistd_x32.h | 5 + linux-headers/linux/iommufd.h | 79 + linux-headers/linux/kvm.h | 229 +-- linux-headers/linux/userfaultfd.h | 29 +- linux-headers/linux/vfio.h | 1 + qapi/qom.json | 33 + qapi/run-state.json | 28 +- system/memory.c | 45 + system/physmem.c | 47 +- system/runstate.c | 54 + target/i386/cpu-internal.h | 9 + target/i386/cpu.c | 12 - target/i386/cpu.h | 21 + target/i386/kvm/kvm-cpu.c | 5 + target/i386/kvm/kvm.c | 611 +++++--- target/i386/kvm/kvm_i386.h | 6 + target/i386/kvm/meson.build | 2 + target/i386/kvm/tdx-quote-generator.c | 170 ++ target/i386/kvm/tdx-quote-generator.h | 95 ++ target/i386/kvm/tdx-stub.c | 23 + target/i386/kvm/tdx.c | 1373 +++++++++++++++++ target/i386/kvm/tdx.h | 82 + target/i386/sev.c | 1 - target/i386/sev.h | 2 + 80 files changed, 4156 insertions(+), 501 deletions(-) create mode 100644 docs/system/i386/tdx.rst create mode 100644 hw/i386/tdvf-hob.c create mode 100644 hw/i386/tdvf-hob.h create mode 100644 hw/i386/tdvf.c create mode 100644 include/hw/i386/tdvf.h create mode 100644 include/standard-headers/uefi/uefi.h create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.h create mode 100644 target/i386/kvm/tdx-stub.c create mode 100644 target/i386/kvm/tdx.c create mode 100644 target/i386/kvm/tdx.h
Signature · 1 line
2.34.1
PATCH v4 1/66 linux-headers: Update to Linux v6.8-rc1
Guest memfd support in QEMU requires corresponding KVM guest memfd APIs, which lands in Linux from v6.8-rc1.
--- include/standard-headers/drm/drm_fourcc.h | 10 +- include/standard-headers/linux/ethtool.h | 41 +++-- .../standard-headers/linux/virtio_config.h | 8 +- include/standard-headers/linux/virtio_pci.h | 68 +++++++++ include/standard-headers/linux/virtio_pmem.h | 7 + linux-headers/asm-generic/unistd.h | 15 +- linux-headers/asm-loongarch/kvm.h | 1 + linux-headers/asm-mips/mman.h | 2 +- linux-headers/asm-mips/unistd_n32.h | 5 + linux-headers/asm-mips/unistd_n64.h | 5 + linux-headers/asm-mips/unistd_o32.h | 5 + linux-headers/asm-powerpc/unistd_32.h | 5 + linux-headers/asm-powerpc/unistd_64.h | 5 + linux-headers/asm-riscv/kvm.h | 13 ++ linux-headers/asm-s390/unistd_32.h | 5 + linux-headers/asm-s390/unistd_64.h | 5 + linux-headers/asm-x86/kvm.h | 3 + linux-headers/asm-x86/unistd_32.h | 5 + linux-headers/asm-x86/unistd_64.h | 5 + linux-headers/asm-x86/unistd_x32.h | 5 + linux-headers/linux/iommufd.h | 79 ++++++++++ linux-headers/linux/kvm.h | 140 +++++++----------- linux-headers/linux/userfaultfd.h | 29 +++- linux-headers/linux/vfio.h | 1 + 24 files changed, 355 insertions(+), 112 deletions(-)
include/standard-headers/drm/drm_fourcc.h +5 -5
@@ -53,7 +53,7 @@ extern "C" {* Format modifiers may change any property of the buffer, including the number * of planes and/or the required allocation size. Format modifiers are * vendor-namespaced, and as such the relationship between a fourcc code and a -* modifier is specific to the modifer being used. For example, some modifiers +* modifier is specific to the modifier being used. For example, some modifiers * may preserve meaning - such as number of planes - from the fourcc code, * whereas others may not. *
@@ -78,7 +78,7 @@ extern "C" {* format. * - Higher-level programs interfacing with KMS/GBM/EGL/Vulkan/etc: these users * see modifiers as opaque tokens they can check for equality and intersect. -* These users musn't need to know to reason about the modifier value +* These users mustn't need to know to reason about the modifier value * (i.e. they are not expected to extract information out of the modifier). * * Vendors should document their modifier usage in as much detail as
@@ -539,7 +539,7 @@ extern "C" {* This is a tiled layout using 4Kb tiles in row-major layout. * Within the tile pixels are laid out in 16 256 byte units / sub-tiles which * are arranged in four groups (two wide, two high) with column-major layout. -* Each group therefore consits out of four 256 byte units, which are also laid +* Each group therefore consists out of four 256 byte units, which are also laid * out as 2x2 column-major. * 256 byte units are made out of four 64 byte blocks of pixels, producing * either a square block or a 2:1 unit.
@@ -1102,7 +1102,7 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)*/ /* -* The top 4 bits (out of the 56 bits alloted for specifying vendor specific +* The top 4 bits (out of the 56 bits allotted for specifying vendor specific * modifiers) denote the category for modifiers. Currently we have three * categories of modifiers ie AFBC, MISC and AFRC. We can have a maximum of * sixteen different categories.
@@ -1418,7 +1418,7 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)* Amlogic FBC Memory Saving mode * * Indicates the storage is packed when pixel size is multiple of word -* boudaries, i.e. 8bit should be stored in this mode to save allocation +* boundaries, i.e. 8bit should be stored in this mode to save allocation * memory. * * This mode reduces body layout to 3072 bytes per 64x32 superblock with
include/standard-headers/linux/ethtool.h +28 -13
71 lines not on this page · raw message
include/standard-headers/linux/virtio_config.h +7 -1
@@ -52,7 +52,7 @@* rest are per-device feature bits. */ #define VIRTIO_TRANSPORT_F_START 28 -#define VIRTIO_TRANSPORT_F_END 41 +#define VIRTIO_TRANSPORT_F_END 42#ifndef VIRTIO_CONFIG_NO_LEGACY /* Do we get callbacks when the ring is completely used, even if we've
@@ -112,4 +112,10 @@* This feature indicates that the driver can reset a queue individually. */ #define VIRTIO_F_RING_RESET 40 + +/* +* This feature indicates that the device support administration virtqueues. +*/ +#define VIRTIO_F_ADMIN_VQ 41 +#endif /* _LINUX_VIRTIO_CONFIG_H */
include/standard-headers/linux/virtio_pci.h +68 -0
81 lines not on this page · raw message
include/standard-headers/linux/virtio_pmem.h +7 -0
@@ -14,6 +14,13 @@#include "standard-headers/linux/virtio_ids.h" #include "standard-headers/linux/virtio_config.h" +/* Feature bits */ +/* guest physical address range will be indicated as shared memory region 0 */ +#define VIRTIO_PMEM_F_SHMEM_REGION 0 + +/* shmid of the shared memory region corresponding to the pmem */ +#define VIRTIO_PMEM_SHMEM_REGION_ID 0 +struct virtio_pmem_config {uint64_t start;uint64_t size;
linux-headers/asm-generic/unistd.h +14 -1
@@ -829,8 +829,21 @@ __SYSCALL(__NR_futex_wait, sys_futex_wait)#define __NR_futex_requeue 456 __SYSCALL(__NR_futex_requeue, sys_futex_requeue) +#define __NR_statmount 457 +__SYSCALL(__NR_statmount, sys_statmount) + +#define __NR_listmount 458 +__SYSCALL(__NR_listmount, sys_listmount) + +#define __NR_lsm_get_self_attr 459 +__SYSCALL(__NR_lsm_get_self_attr, sys_lsm_get_self_attr) +#define __NR_lsm_set_self_attr 460 +__SYSCALL(__NR_lsm_set_self_attr, sys_lsm_set_self_attr) +#define __NR_lsm_list_modules 461 +__SYSCALL(__NR_lsm_list_modules, sys_lsm_list_modules) +#undef __NR_syscalls -#define __NR_syscalls 457 +#define __NR_syscalls 462/* * 32 bit systems traditionally used different
linux-headers/asm-loongarch/kvm.h +1 -0
@@ -79,6 +79,7 @@ struct kvm_fpu {#define LOONGARCH_REG_64 (TYPE ,REG ) (TYPE | KVM_REG_SIZE_U64 | (REG << LOONGARCH_REG_SHIFT))#define KVM_IOC_CSRID (REG ) LOONGARCH_REG_64(KVM_REG_LOONGARCH_CSR, REG)#define KVM_IOC_CPUCFG (REG ) LOONGARCH_REG_64(KVM_REG_LOONGARCH_CPUCFG, REG) +#define KVM_LOONGARCH_VCPU_CPUCFG 0struct kvm_debug_exit_arch { };
linux-headers/asm-mips/mman.h +1 -1
@@ -88,7 +88,7 @@#define MADV_HUGEPAGE 14/* Worth backing with hugepages */ #define MADV_NOHUGEPAGE 15/* Not worth backing with hugepages */ -#define MADV_DONTDUMP 16/* Explicity exclude from the core dump, +#define MADV_DONTDUMP 16/* Explicitly exclude from core dump, overrides the coredump filter bits */ #define MADV_DODUMP 17/* Clear the MADV_NODUMP flag */
linux-headers/asm-mips/unistd_n32.h +5 -0
@@ -385,5 +385,10 @@#define __NR_futex_wake (__NR_Linux + 454)#define __NR_futex_wait (__NR_Linux + 455)#define __NR_futex_requeue (__NR_Linux + 456) +#define __NR_statmount (__NR_Linux + 457) +#define __NR_listmount (__NR_Linux + 458) +#define __NR_lsm_get_self_attr (__NR_Linux + 459) +#define __NR_lsm_set_self_attr (__NR_Linux + 460) +#define __NR_lsm_list_modules (__NR_Linux + 461)#endif /* _ASM_UNISTD_N32_H */
linux-headers/asm-mips/unistd_n64.h +5 -0
@@ -361,5 +361,10 @@#define __NR_futex_wake (__NR_Linux + 454)#define __NR_futex_wait (__NR_Linux + 455)#define __NR_futex_requeue (__NR_Linux + 456) +#define __NR_statmount (__NR_Linux + 457) +#define __NR_listmount (__NR_Linux + 458) +#define __NR_lsm_get_self_attr (__NR_Linux + 459) +#define __NR_lsm_set_self_attr (__NR_Linux + 460) +#define __NR_lsm_list_modules (__NR_Linux + 461)#endif /* _ASM_UNISTD_N64_H */
linux-headers/asm-mips/unistd_o32.h +5 -0
@@ -431,5 +431,10 @@#define __NR_futex_wake (__NR_Linux + 454)#define __NR_futex_wait (__NR_Linux + 455)#define __NR_futex_requeue (__NR_Linux + 456) +#define __NR_statmount (__NR_Linux + 457) +#define __NR_listmount (__NR_Linux + 458) +#define __NR_lsm_get_self_attr (__NR_Linux + 459) +#define __NR_lsm_set_self_attr (__NR_Linux + 460) +#define __NR_lsm_list_modules (__NR_Linux + 461)#endif /* _ASM_UNISTD_O32_H */
linux-headers/asm-powerpc/unistd_32.h +5 -0
@@ -438,6 +438,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_32_H */
linux-headers/asm-powerpc/unistd_64.h +5 -0
@@ -410,6 +410,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_64_H */
linux-headers/asm-riscv/kvm.h +13 -0
@@ -157,9 +157,16 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_EXPERIMENTAL ,KVM_RISCV_SBI_EXT_VENDOR ,KVM_RISCV_SBI_EXT_DBCN , +KVM_RISCV_SBI_EXT_STA , KVM_RISCV_SBI_EXT_MAX, }; +/* SBI STA extension registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_sbi_sta { +unsigned long shmem_lo; +unsigned long shmem_hi; +}; +/* Possible states for kvm_riscv_timer */ #define KVM_RISCV_TIMER_STATE_OFF 0#define KVM_RISCV_TIMER_STATE_ON 1
@@ -241,6 +248,12 @@ enum KVM_RISCV_SBI_EXT_ID {#define KVM_REG_RISCV_VECTOR_REG (n) \ ((n) + sizeof(struct __riscv_v_ext_state) / sizeof(unsigned long)) +/* Registers for specific SBI extensions are mapped as type 10 */ +#define KVM_REG_RISCV_SBI_STATE (0x0a << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_SBI_STA (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT) +#define KVM_REG_RISCV_SBI_STA_REG (name) \ + (offsetof(struct kvm_riscv_sbi_sta, name) / sizeof(unsigned long)) +/* Device Control API: RISC-V AIA */ #define KVM_DEV_RISCV_APLIC_ALIGN 0x1000#define KVM_DEV_RISCV_APLIC_SIZE 0x4000
linux-headers/asm-s390/unistd_32.h +5 -0
@@ -429,5 +429,10 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_S390_UNISTD_32_H */
linux-headers/asm-s390/unistd_64.h +5 -0
@@ -377,5 +377,10 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_S390_UNISTD_64_H */
linux-headers/asm-x86/kvm.h +3 -0
@@ -560,4 +560,7 @@ struct kvm_pmu_event_filter {/* x86-specific KVM_EXIT_HYPERCALL flags. */ #define KVM_EXIT_HYPERCALL_LONG_MODE BIT(0) +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_SW_PROTECTED_VM 1 +#endif /* _ASM_X86_KVM_H */
linux-headers/asm-x86/unistd_32.h +5 -0
@@ -447,6 +447,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_32_H */
linux-headers/asm-x86/unistd_64.h +5 -0
@@ -369,6 +369,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_64_H */
linux-headers/asm-x86/unistd_x32.h +5 -0
@@ -321,6 +321,11 @@#define __NR_futex_wake (__X32_SYSCALL_BIT + 454)#define __NR_futex_wait (__X32_SYSCALL_BIT + 455)#define __NR_futex_requeue (__X32_SYSCALL_BIT + 456) +#define __NR_statmount (__X32_SYSCALL_BIT + 457) +#define __NR_listmount (__X32_SYSCALL_BIT + 458) +#define __NR_lsm_get_self_attr (__X32_SYSCALL_BIT + 459) +#define __NR_lsm_set_self_attr (__X32_SYSCALL_BIT + 460) +#define __NR_lsm_list_modules (__X32_SYSCALL_BIT + 461)#define __NR_rt_sigaction (__X32_SYSCALL_BIT + 512)#define __NR_rt_sigreturn (__X32_SYSCALL_BIT + 513)#define __NR_ioctl (__X32_SYSCALL_BIT + 514)
linux-headers/linux/iommufd.h +79 -0
89 lines not on this page · raw message
linux-headers/linux/kvm.h +50 -90
218 lines not on this page · raw message
linux-headers/linux/userfaultfd.h +28 -1
@@ -41,7 +41,8 @@ UFFD_FEATURE_WP_HUGETLBFS_SHMEM | \ UFFD_FEATURE_WP_UNPOPULATED | \ UFFD_FEATURE_POISON | \ - UFFD_FEATURE_WP_ASYNC) + UFFD_FEATURE_WP_ASYNC | \ + UFFD_FEATURE_MOVE)#define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \
@@ -50,6 +51,7 @@ ((__u64 )1 << _UFFDIO_WAKE | \ (__u64 )1 << _UFFDIO_COPY | \ (__u64 )1 << _UFFDIO_ZEROPAGE | \ + (__u64 )1 << _UFFDIO_MOVE | \ (__u64 )1 << _UFFDIO_WRITEPROTECT | \ (__u64 )1 << _UFFDIO_CONTINUE | \ (__u64 )1 << _UFFDIO_POISON)
@@ -73,6 +75,7 @@#define _UFFDIO_WAKE (0x02)#define _UFFDIO_COPY (0x03)#define _UFFDIO_ZEROPAGE (0x04) +#define _UFFDIO_MOVE (0x05)#define _UFFDIO_WRITEPROTECT (0x06)#define _UFFDIO_CONTINUE (0x07)#define _UFFDIO_POISON (0x08)
@@ -92,6 +95,8 @@ struct uffdio_copy)#define UFFDIO_ZEROPAGE _IOWR(UFFDIO, _UFFDIO_ZEROPAGE, \ struct uffdio_zeropage) +#define UFFDIO_MOVE _IOWR(UFFDIO, _UFFDIO_MOVE, \ + struct uffdio_move)#define UFFDIO_WRITEPROTECT _IOWR(UFFDIO, _UFFDIO_WRITEPROTECT, \ struct uffdio_writeprotect)#define UFFDIO_CONTINUE _IOWR(UFFDIO, _UFFDIO_CONTINUE, \
@@ -222,6 +227,9 @@ struct uffdio_api {* asynchronous mode is supported in which the write fault is * automatically resolved and write-protection is un-set. * It implies UFFD_FEATURE_WP_UNPOPULATED. +* +* UFFD_FEATURE_MOVE indicates that the kernel supports moving an +* existing page contents from userspace. */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0)#define UFFD_FEATURE_EVENT_FORK (1<<1)
@@ -239,6 +247,7 @@ struct uffdio_api {#define UFFD_FEATURE_WP_UNPOPULATED (1<<13)#define UFFD_FEATURE_POISON (1<<14)#define UFFD_FEATURE_WP_ASYNC (1<<15) +#define UFFD_FEATURE_MOVE (1<<16)__u64 features;__u64 ioctls;
@@ -347,6 +356,24 @@ struct uffdio_poison {__s64 updated; }; +struct uffdio_move { +__u64 dst; +__u64 src; +__u64 len; +/* +* Especially if used to atomically remove memory from the +* address space the wake on the dst range is not needed. +*/ +#define UFFDIO_MOVE_MODE_DONTWAKE ((__u64)1<<0) +#define UFFDIO_MOVE_MODE_ALLOW_SRC_HOLES ((__u64)1<<1) +__u64 mode; +/* +* "move" is written by the ioctl and must be at the end: the +* copy_from_user will not read the last 8 bytes. +*/ +__s64 move; +}; +/* * Flags for the userfaultfd(2) system call itself. */
linux-headers/linux/vfio.h +1 -0
@@ -1219,6 +1219,7 @@ enum vfio_device_mig_state {VFIO_DEVICE_STATE_RUNNING_P2P =5 ,VFIO_DEVICE_STATE_PRE_COPY =6 ,VFIO_DEVICE_STATE_PRE_COPY_P2P =7 , + VFIO_DEVICE_STATE_NR, };/**
Signature · 1 line
2.34.1
PATCH v4 2/66 RAMBlock: Add support of KVM private guest memfd
In reply to Xiaoyao LiAdd KVM guest_memfd support to RAMBlock so both normal hva based memory and kvm guest memfd based private memory can be associated in one RAMBlock.
Introduce new flag RAM_GUEST_MEMFD. When it's set, it calls KVM ioctl to create private guest_memfd during RAMBlock setup.
Allocating a new RAM_GUEST_MEMFD flag to instruct the setup of guest memfd is more flexible and extensible than simply relying on the VM type because in the future we may have the case that not all the memory of a VM need guest memfd. As a benefit, it also avoid getting MachineState in memory subsystem.
Note, RAM_GUEST_MEMFD is supposed to be set for memory backends of confidential guests, such as TDX VM. How and when to set it for memory backends will be implemented in the following patches.
Introduce memory_region_has_guest_memfd() to query if the MemoryRegion has KVM guest_memfd allocated.
---
Changes in v4:
- Add clarification of RAM_GUEST_MEMFD in commit message; (David Hildenbrand)
- refine the return value and error message; (Daniel P. Berrangé)
- remove flags in ram_block_add(); (David Hildenbrand)
Changes in v3:
- rename gmem to guest_memfd;
- close(guest_memfd) when RAMBlock is released; (Daniel P. Berrangé)
- Suqash the patch that introduces memory_region_has_guest_memfd().
--- accel/kvm/kvm-all.c | 25 +++++++++++++++++++++++++ accel/stubs/kvm-stub.c | 5 +++++ include/exec/memory.h | 20 +++++++++++++++++--- include/exec/ram_addr.h | 2 +- include/exec/ramblock.h | 1 + include/sysemu/kvm.h | 2 ++ system/memory.c | 5 +++++ system/physmem.c | 24 +++++++++++++++++++++--- 8 files changed, 77 insertions(+), 7 deletions(-)
accel/kvm/kvm-all.c +25 -0
@@ -91,6 +91,7 @@ bool kvm_msi_use_devid;static bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit; +static bool kvm_guest_memfd_supported;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -2385,6 +2386,8 @@ static int kvm_init(MachineState *ms) } s->as = g_new0(struct KVMAs, s->nr_as); + kvm_guest_memfd_supported =kvm_check_extension (s,KVM_CAP_GUEST_MEMFD ); +if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",
@@ -4084,3 +4087,25 @@ void query_stats_schemas_cb(StatsSchemaList **result, Error **errp)query_stats_schema_vcpu (first_cpu, &stats_args); } } + +int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp) +{ +int fd; +struct kvm_create_guest_memfd guest_memfd = { + .size = size, + .flags = flags, + }; + +if (!kvm_guest_memfd_supported) { +error_setg (errp,"KVM doesn't support guest memfd\n" ); +return -1 ; + } + + fd =kvm_vm_ioctl (kvm_state,KVM_CREATE_GUEST_MEMFD , &guest_memfd); +if (fd <0 ) { +error_setg_errno (errp, errno,"Error creating kvm guest memfd" ); +return -1 ; + } + +return fd; +}
accel/stubs/kvm-stub.c +5 -0
@@ -124,3 +124,8 @@ uint32_t kvm_dirty_ring_size(void) {return 0 ; } + +int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp) +{ +return -ENOSYS ; +}
include/exec/memory.h +17 -3
@@ -243,6 +243,9 @@ typedef struct IOMMUTLBEvent {/* RAM FD is opened read-only */ #define RAM_READONLY_FD (1 << 11) +/* RAM can be private that has kvm guest memfd backend */ +#define RAM_GUEST_MEMFD (1 << 12) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
@@ -1307,7 +1310,8 @@ bool memory_region_init_ram_nomigrate(MemoryRegion *mr,* @name: Region name, becomes part of RAMBlock name used in migration stream * must be unique within any device * @size: size of the region. -* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE. +* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE, +* RAM_GUEST_MEMFD. * @errp: pointer to Error*, to store an error if it happens. * * Note that this function does not do anything to cause the data in the
@@ -1369,7 +1373,7 @@ bool memory_region_init_resizeable_ram(MemoryRegion *mr,* (getpagesize()) will be used. * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM, * RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY, -* RAM_READONLY_FD +* RAM_READONLY_FD, RAM_GUEST_MEMFD * @path: the path in which to allocate the RAM. * @offset: offset within the file referenced by path * @errp: pointer to Error*, to store an error if it happens.
@@ -1399,7 +1403,7 @@ bool memory_region_init_ram_from_file(MemoryRegion *mr,* @size: size of the region. * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM, * RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY, -* RAM_READONLY_FD +* RAM_READONLY_FD, RAM_GUEST_MEMFD * @fd: the fd to mmap. * @offset: offset within the file referenced by fd * @errp: pointer to Error*, to store an error if it happens.
@@ -1722,6 +1726,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)*/ bool memory_region_is_protected (MemoryRegion *mr); +/** +* memory_region_has_guest_memfd: check whether a memory region has guest_memfd +* associated +* +* Returns %true if a memory region's ram_block has valid guest_memfd assigned. +* +* @mr: the memory region being queried +*/ +bool memory_region_has_guest_memfd (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu *
include/exec/ram_addr.h +1 -1
@@ -109,7 +109,7 @@ long qemu_maxrampagesize(void);* @mr: the memory region where the ram block is * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM, * RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY, -* RAM_READONLY_FD +* RAM_READONLY_FD, RAM_GUEST_MEMFD * @mem_path or @fd: specify the backing file or device * @offset: Offset into target file * @errp: pointer to Error*, to store an error if it happens
include/exec/ramblock.h +1 -0
@@ -41,6 +41,7 @@ struct RAMBlock { QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers;int fd;uint64_t fd_offset; +int guest_memfd;size_t page_size;/* dirty bitmap used during migration */ unsigned long *bmap;
include/sysemu/kvm.h +2 -0
@@ -538,4 +538,6 @@ bool kvm_arch_cpu_check_are_resettable(void);bool kvm_dirty_ring_enabled (void );uint32_t kvm_dirty_ring_size (void ); + +int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp);#endif
system/memory.c +5 -0
@@ -1850,6 +1850,11 @@ bool memory_region_is_protected(MemoryRegion *mr)return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); } +bool memory_region_has_guest_memfd (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->guest_memfd >=0 ; +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
system/physmem.c +21 -3
@@ -1841,6 +1841,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) } } +if (kvm_enabled () && (new_block->flags &RAM_GUEST_MEMFD ) && + new_block->guest_memfd <0 ) { +/* TODO: to decide if KVM_GUEST_MEMFD_ALLOW_HUGEPAGE is supported */ + new_block->guest_memfd =kvm_create_guest_memfd (new_block->max_length, +0 , errp); +if (new_block->guest_memfd <0 ) { +qemu_mutex_unlock_ramlist (); +return ; + } + } + new_ram_size =MAX (old_ram_size, (new_block->offset + new_block->max_length) >>TARGET_PAGE_BITS );if (new_ram_size > old_ram_size) {
@@ -1903,7 +1914,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,/* Just support these ram flags by now. */ assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM |RAM_NORESERVE |RAM_PROTECTED |RAM_NAMED_FILE |RAM_READONLY | -RAM_READONLY_FD )) ==0 ); +RAM_READONLY_FD |RAM_GUEST_MEMFD )) ==0 );if (xen_enabled ()) {error_setg (errp,"-mem-path not supported with Xen" );
@@ -1938,6 +1949,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, new_block->used_length = size; new_block->max_length = size; new_block->flags = ram_flags; + new_block->guest_memfd =-1 ; new_block->host =file_ram_alloc (new_block, size, fd, !file_size, offset, errp);if (!new_block->host) {
@@ -2016,7 +2028,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,Error *local_err =NULL ;assert ((ram_flags & ~(RAM_SHARED |RAM_RESIZEABLE |RAM_PREALLOC | -RAM_NORESERVE )) ==0 ); +RAM_NORESERVE |RAM_GUEST_MEMFD )) ==0 );assert (!host ^ (ram_flags &RAM_PREALLOC )); size =HOST_PAGE_ALIGN (size);
@@ -2028,6 +2040,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, new_block->max_length = max_size;assert (max_size >= size); new_block->fd =-1 ; + new_block->guest_memfd =-1 ; new_block->page_size =qemu_real_host_page_size (); new_block->host = host; new_block->flags = ram_flags;
@@ -2050,7 +2063,7 @@ RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,RAMBlock *qemu_ram_alloc (ram_addr_t size,uint32_t ram_flags,MemoryRegion *mr,Error **errp) { -assert ((ram_flags & ~(RAM_SHARED |RAM_NORESERVE )) ==0 ); +assert ((ram_flags & ~(RAM_SHARED |RAM_NORESERVE |RAM_GUEST_MEMFD )) ==0 );return qemu_ram_alloc_internal (size, size,NULL ,NULL , ram_flags, mr, errp); }
@@ -2078,6 +2091,11 @@ static void reclaim_ramblock(RAMBlock *block) }else {qemu_anon_ram_free (block->host, block->max_length); } + +if (block->guest_memfd >=0 ) { +close (block->guest_memfd); + } +g_free (block); }
Signature · 1 line
2.34.1
Re: [PATCH v4 02/66] RAMBlock: Add support of KVM private guest memfd
system/physmem.c static void ram_block_add(RAMBlock *new_block, Error **errp) · line 1845
Quoted 13 lines
uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr) { uint8_t mask = mr->dirty_log_mask;system/physmem.c +2
-0@@ -1841,6 +1841,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) } } +if (kvm_enabled () && (new_block->flags &RAM_GUEST_MEMFD ) && + new_block->guest_memfd <0 ) {
How could we have a guest_memfd already at this point? Smells more like an assert(new_block->guest_memfd < 0);
system/physmem.c static void ram_block_add(RAMBlock *new_block, Error **errp) · line 1846
Quoted 1 line
system/physmem.c +1
-0+/* TODO: to decide if KVM_GUEST_MEMFD_ALLOW_HUGEPAGE is supported */
I suggest dropping that completely. As long as it's not upstream, not even the name of that thing is stable.
system/physmem.c static void ram_block_add(RAMBlock *new_block, Error **errp) · line 1854
Quoted 8 lines
system/physmem.c +8
-0+ new_block->guest_memfd =kvm_create_guest_memfd (new_block->max_length, +0 , errp); +if (new_block->guest_memfd <0 ) { +qemu_mutex_unlock_ramlist (); +return ; + } + } +
In general, LGTM. With the two nits above:
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v4 02/66] RAMBlock: Add support of KVM private guest memfd
Quoted 7 lines · David Hildenbrand
Quoted 7 lines
uint8_tmemory_region_get_dirty_log_mask(MemoryRegion *mr){
uint8_tmask =mr->dirty_log_mask;system/physmem.c +0
-0
Quoted 8 lines
system/physmem.c static void ram_block_add(RAMBlock *new_block, · line 1845
Quoted 6 lines
system/physmem.c +2
-0@@ -1841,6 +1841,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) } } + if (kvm_enabled() && (new_block->flags & RAM_GUEST_MEMFD) && + new_block->guest_memfd < 0) {How could we have a
guest_memfdalready at this point? Smells more like anassert(new_block->guest_memfd < 0);
you are right. I will change it to the assert()
Quoted 4 lines
Quoted 2 lines
system/physmem.c +1
-0+ /* TODO: to decide if KVM_GUEST_MEMFD_ALLOW_HUGEPAGE issupported */
I suggest dropping that completely. As long as it's not upstream, not even the name of that thing is stable.
OK
Quoted 11 lines
Quoted 9 lines
+
new_block->guest_memfd= kvm_create_guest_memfd(new_block->max_length,
+ 0, errp); + if (new_block->guest_memfd< 0) {
+qemu_mutex_unlock_ramlist();
+ return;
+ }
+ }
+
Quoted 4 lines
In general, LGTM. With the two nits above:
Reviewed-by David Hildenbrand <[email protected]>
Thanks!
PATCH v4 3/66 HostMem: Add mechanism to opt in kvm guest memfd via MachineState
In reply to Xiaoyao LiAdd a new member "guest_memfd" to memory backends. When it's set to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm guest_memfd will be allocated during RAMBlock allocation.
Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in phymem.c.
MachineState::require_guest_memfd is supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.
---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)
--- backends/hostmem-file.c | 1 + backends/hostmem-memfd.c | 1 + backends/hostmem-ram.c | 1 + backends/hostmem.c | 1 + hw/core/machine.c | 5 +++++ include/hw/boards.h | 2 ++ include/sysemu/hostmem.h | 1 + 7 files changed, 12 insertions(+)
backends/hostmem-file.c +1 -0
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) ram_flags |= fb->readonly ?RAM_READONLY_FD :0 ; ram_flags |= fb->rom ==ON_OFF_AUTO_ON ?RAM_READONLY :0 ; ram_flags |= backend->reserve ?0 :RAM_NORESERVE ; + ram_flags |= backend->guest_memfd ?RAM_GUEST_MEMFD :0 ; ram_flags |= fb->is_pmem ?RAM_PMEM :0 ; ram_flags |=RAM_NAMED_FILE ; return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-memfd.c +1 -0
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend); ram_flags = backend->share ?RAM_SHARED :0 ; ram_flags |= backend->reserve ?0 :RAM_NORESERVE ; + ram_flags |= backend->guest_memfd ?RAM_GUEST_MEMFD :0 ;return memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), name, backend->size, ram_flags, fd,0 , errp); }
backends/hostmem-ram.c +1 -0
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend); ram_flags = backend->share ?RAM_SHARED :0 ; ram_flags |= backend->reserve ?0 :RAM_NORESERVE ; + ram_flags |= backend->guest_memfd ?RAM_GUEST_MEMFD :0 ;return memory_region_init_ram_flags_nomigrate (&backend->mr,OBJECT (backend), name, backend->size, ram_flags, errp);
backends/hostmem.c +1 -0
@@ -276,6 +276,7 @@ static void host_memory_backend_init(Object *obj)/* TODO: convert access to globals to compat properties */ backend->merge =machine_mem_merge (machine); backend->dump =machine_dump_guest_core (machine); + backend->guest_memfd =machine_require_guest_memfd (machine); backend->reserve = true; backend->prealloc_threads = machine->smp.cpus; }
hw/core/machine.c +5 -0
@@ -1192,6 +1192,11 @@ bool machine_mem_merge(MachineState *machine)return machine->mem_merge; } +bool machine_require_guest_memfd (MachineState *machine) +{ +return machine->require_guest_memfd; +} +static char *cpu_slot_to_string (const CPUArchId *cpu) {GString *s =g_string_new (NULL );
include/hw/boards.h +2 -0
@@ -36,6 +36,7 @@ bool machine_usb(MachineState *machine);int machine_phandle_start (MachineState *machine);bool machine_dump_guest_core (MachineState *machine);bool machine_mem_merge (MachineState *machine); +bool machine_require_guest_memfd (MachineState *machine);HotpluggableCPUList *machine_query_hotpluggable_cpus (MachineState *machine); void machine_set_cpu_numa_node(MachineState *machine, const CpuInstanceProperties *props,
@@ -370,6 +371,7 @@ struct MachineState {char *dt_compatible;bool dump_guest_core;bool mem_merge; +bool require_guest_memfd;bool usb;bool usb_disabled;char *firmware;
include/sysemu/hostmem.h +1 -0
@@ -74,6 +74,7 @@ struct HostMemoryBackend {uint64_t size;bool merge, dump, use_canonical_path;bool prealloc, is_mapped, share, reserve; +bool guest_memfd;uint32_t prealloc_threads;ThreadContext *prealloc_context;DECLARE_BITMAP (host_nodes,MAX_NODES +1 );
Signature · 1 line
2.34.1
Re: [PATCH v4 03/66] HostMem: Add mechanism to opt in kvm guest memfd via MachineState
Quoted 4 lines · Xiaoyao Li
Add a new member "
guest_memfd" to memory backends. When it's set to true, it enablesRAM_GUEST_MEMFDinram_flags, thus private kvmguest_memfdwill be allocated during RAMBlock allocation.
Quoted 8 lines
Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in
phymem.c.
MachineState::require_guest_memfdis supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.Signed-off-by Xiaoyao Li <[email protected]>---
Signature · 3 lines
Cheers, David / dhildenb
PATCH v4 4/66 trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()
In reply to Xiaoyao LiThe upper 16 bits of kvm_userspace_memory_region::slot are address space id. Parse it separately in trace_kvm_set_user_memory().
--- accel/kvm/kvm-all.c | 5 +++-- accel/kvm/trace-events | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -2
@@ -304,8 +304,9 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); slot->old_flags = mem.flags; err: -trace_kvm_set_user_memory (mem.slot, mem.flags, mem.guest_phys_addr, - mem.memory_size, mem.userspace_addr, ret); +trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, + mem.guest_phys_addr, mem.memory_size, + mem.userspace_addr, ret);if (ret <0 ) { error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
accel/kvm/trace-events +1 -1
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"
Signature · 1 line
2.34.1
PATCH v4 5/66 kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.
With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backend'ed both by hva-based shared memory and guest memfd based private memory.
---
Changes in v4:
- update KVM_MEM_PRIVATE to KVM_MEM_GUEST_MEMFD; (Isaku)
--- accel/kvm/kvm-all.c | 56 ++++++++++++++++++++++++++++++++++------ accel/kvm/trace-events | 2 +- include/sysemu/kvm_int.h | 2 ++ 3 files changed, 51 insertions(+), 9 deletions(-)
accel/kvm/kvm-all.c +48 -8
@@ -283,35 +283,69 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,static int kvm_set_user_memory_region (KVMMemoryListener *kml,KVMSlot *slot,bool new) {KVMState *s = kvm_state; -struct kvm_userspace_memory_region mem; +struct kvm_userspace_memory_region2 mem; +static int cap_user_memory2 =-1 ;int ret; +if (cap_user_memory2 ==-1 ) { + cap_user_memory2 =kvm_check_extension (s,KVM_CAP_USER_MEMORY2 ); + } + +if (!cap_user_memory2 && slot->guest_memfd >=0 ) { +error_report ("%s, KVM doesn't support KVM_CAP_USER_MEMORY2," +" which is required by guest memfd!" , __func__); +exit (1 ); + } + mem.slot = slot->slot | (kml->as_id <<16 ); mem.guest_phys_addr = slot->start_addr; mem.userspace_addr = (unsigned long )slot->ram; mem.flags = slot->flags; + mem.guest_memfd = slot->guest_memfd; + mem.guest_memfd_offset = slot->guest_memfd_offset;if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) &KVM_MEM_READONLY ) {/* Set the slot size to 0 before setting the slot to the desired * value. This is needed based on KVM commit 75d61fbc. */ mem.memory_size =0 ; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + }if (ret <0 ) { goto err; } } mem.memory_size = slot->memory_size; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + } slot->old_flags = mem.flags; err:trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, mem.guest_phys_addr, mem.memory_size, - mem.userspace_addr, ret); + mem.userspace_addr, mem.guest_memfd, + mem.guest_memfd_offset, ret);if (ret <0 ) { -error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," -" start=0x%" PRIx64", size=0x%" PRIx64": %s" , - __func__, mem.slot, slot->start_addr, - (uint64_t )mem.memory_size,strerror (errno)); +if (cap_user_memory2) { +error_report ("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64"," +" flags=0x%" PRIx32", guest_memfd=%" PRId32"," +" guest_memfd_offset=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size, mem.flags, + mem.guest_memfd, (uint64_t )mem.guest_memfd_offset, +strerror (errno)); + }else { +error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size,strerror (errno)); + } }return ret; }
@@ -466,6 +500,9 @@ static int kvm_mem_flags(MemoryRegion *mr)if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; } +if (memory_region_has_guest_memfd (mr)) { + flags |=KVM_MEM_GUEST_MEMFD ; + }return flags; }
@@ -1353,6 +1390,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, mem->ram_start_offset = ram_start_offset; mem->ram = ram; mem->flags =kvm_mem_flags (mr); + mem->guest_memfd = mr->ram_block->guest_memfd; + mem->guest_memfd_offset = (uint8_t *)ram - mr->ram_block->host; +kvm_slot_init_dirty_bitmap (mem); err =kvm_set_user_memory_region (kml, mem, true);if (err) {
accel/kvm/trace-events +1 -1
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " guest_memfd=%d" " guest_memfd_offset=0x%" PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
@@ -30,6 +30,8 @@ typedef struct KVMSlotint as_id;/* Cache of the offset in ram address space */ ram_addr_t ram_start_offset; +int guest_memfd; +hwaddr guest_memfd_offset; } KVMSlot; typedef struct KVMMemoryUpdate {
Signature · 1 line
2.34.1
PATCH v4 6/66 kvm: Introduce support for memory_attributes
In reply to Xiaoyao LiIntroduce the helper functions to set the attributes of a range of memory to private or shared.
This is necessary to notify KVM the private/shared attribute of each gpa range. KVM needs the information to decide the GPA needs to be mapped at hva-based shared memory or guest_memfd based private memory.
---
Changes in v4:
- move the check of kvm_supported_memory_attributes to the common kvm_set_memory_attributes(); (Wang Wei)
- change warn_report() to error_report() in kvm_set_memory_attributes() and drop the __func__; (Daniel)
--- accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ include/sysemu/kvm.h | 3 +++ 2 files changed, 47 insertions(+)
accel/kvm/kvm-all.c +44 -0
@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit;static bool kvm_guest_memfd_supported; +static uint64_t kvm_supported_memory_attributes;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1294,6 +1295,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size) kvm_max_slot_size = max_slot_size; } +static int kvm_set_memory_attributes (hwaddr start,hwaddr size,uint64_t attr) +{ +struct kvm_memory_attributes attrs; +int r; + +if (kvm_supported_memory_attributes ==0 ) { +error_report ("No memory attribute supported by KVM\n" ); +return -EINVAL ; + } + +if ((attr & kvm_supported_memory_attributes) != attr) { +error_report ("memory attribute 0x%lx not supported by KVM," +" supported bits are 0x%lx\n" , + attr, kvm_supported_memory_attributes); +return -EINVAL ; + } + + attrs.attributes = attr; + attrs.address = start; + attrs.size = size; + attrs.flags =0 ; + + r =kvm_vm_ioctl (kvm_state,KVM_SET_MEMORY_ATTRIBUTES , &attrs); +if (r) { +error_report ("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'" , + start, size, attr,strerror (errno)); + } +return r; +} + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,KVM_MEMORY_ATTRIBUTE_PRIVATE ); +} + +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,0 ); +} +/* Called with KVMMemoryListener.slots_lock held */ static void kvm_set_phys_mem (KVMMemoryListener *kml,MemoryRegionSection *section,bool add)
@@ -2429,6 +2470,9 @@ static int kvm_init(MachineState *ms) kvm_guest_memfd_supported =kvm_check_extension (s,KVM_CAP_GUEST_MEMFD ); + ret =kvm_check_extension (s,KVM_CAP_MEMORY_ATTRIBUTES ); + kvm_supported_memory_attributes = ret >0 ? ret :0 ; +if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",
include/sysemu/kvm.h +3 -0
@@ -540,4 +540,7 @@ bool kvm_dirty_ring_enabled(void);uint32_t kvm_dirty_ring_size (void );int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp); + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size); +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size);#endif
Signature · 1 line
2.34.1
PATCH v4 7/66 physmem: Introduce ram_block_discard_guest_memfd_range()
In reply to Xiaoyao LiWhen memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.
Originally-from: Isaku Yamahata <[email protected]>
---
Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.
--- include/exec/cpu-common.h | 2 ++ system/physmem.c | 23 +++++++++++++++++++++++ 2 files changed, 25 insertions(+)
include/exec/cpu-common.h +2 -0
@@ -175,6 +175,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque);int ram_block_discard_range (RAMBlock *rb,uint64_t start,size_t length); +int ram_block_discard_guest_memfd_range (RAMBlock *rb,uint64_t start, +size_t length);#endif
system/physmem.c +23 -0
@@ -3626,6 +3626,29 @@ err:return ret; } +int ram_block_discard_guest_memfd_range (RAMBlock *rb,uint64_t start, +size_t length) +{ +int ret =-1 ; + +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE + ret =fallocate (rb->guest_memfd,FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE , + start, length); + +if (ret) { + ret = -errno; +error_report ("%s: Failed to fallocate %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); + } +#else + ret = -ENOSYS ; +error_report ("%s: fallocate not available %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); +#endif + +return ret; +} +bool ramblock_is_pmem (RAMBlock *rb) {return rb->flags &RAM_PMEM ;
Signature · 1 line
2.34.1
Re: [PATCH v4 07/66] physmem: Introduce ram_block_discard_guest_memfd_range()
Quoted 54 lines · Xiaoyao Li
When memory page is converted from private to shared, the original private memory is back'ed by
guest_memfd. Introduceram_block_discard_guest_memfd_range()for discarding memory inguest_memfd.Originally-from: Isaku Yamahata <[email protected]>
Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in in v4:
- Dropram_block_convert_range()and open code its implementation in the next Patch.--- include/exec/cpu-common.h | 2 ++ system/physmem.c | 23 +++++++++++++++++++++++ 2 files changed, 25 insertions(+)include/exec/cpu-common.h +2
-0@@ -175,6 +175,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque);int ram_block_discard_range (RAMBlock *rb,uint64_t start,size_t length); +int ram_block_discard_guest_memfd_range (RAMBlock *rb,uint64_t start, +size_t length);#endif system/physmem.c +16
-0@@ -3626,6 +3626,29 @@ err:return ret; } +int ram_block_discard_guest_memfd_range (RAMBlock *rb,uint64_t start, +size_t length) +{ +int ret =-1 ; + +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE + ret =fallocate (rb->guest_memfd,FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE , + start, length); + +if (ret) { + ret = -errno; +error_report ("%s: Failed to fallocate %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); + } +#else + ret = -ENOSYS ;
system/physmem.c err: · line 3654
Quoted 10 lines
system/physmem.c +7
-0+error_report ("%s: fallocate not available %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); +#endif + +return ret; +} +bool ramblock_is_pmem (RAMBlock *rb) {return rb->flags &RAM_PMEM ;
Signature · 3 lines
Cheers, David / dhildenb
PATCH v4 8/66 kvm: handle KVM_EXIT_MEMORY_FAULT
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.
Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens.
Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has guest_memfd memory backend.
Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is added.
When page is converted from shared to private, the original shared memory can be discarded via ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.
---
Changes in v4:
- open-coded ram_block_discard logic;
- change warn_report() to error_report(); (Daniel)
--- accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +84 -10
@@ -2893,6 +2893,68 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +ram_addr_t offset; +MemoryRegion *mr; +RAMBlock *rb; +void *addr; +int ret =-1 ; + +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || + !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) { +return -1 ; + } + +if (!size) { +return -1 ; + } + + section =memory_region_find (get_system_memory (), start, size); + mr = section.mr; +if (!mr) { +return -1 ; + } + +if (memory_region_has_guest_memfd (mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +memory_region_unref (section.mr); +return ret; + } + + addr =memory_region_get_ram_ptr (mr) + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); + +if (to_private) { +if (rb->page_size != qemu_host_page_size) { +/* +* shared memory is back'ed by hugetlb, which is supposed to be +* pre-allocated and doesn't need to be discarded +*/ +return 0 ; + }else { + ret =ram_block_discard_range (rb, offset, size); + } + }else { + ret =ram_block_discard_guest_memfd_range (rb, offset, size); + } + }else { +error_report ("Convert non guest_memfd backed memory region " +"(0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } + +memory_region_unref (section.mr); +return ret; +} +int kvm_cpu_exec (CPUState *cpu) {struct kvm_run *run = cpu->kvm_run;
@@ -2960,18 +3022,20 @@ int kvm_cpu_exec(CPUState *cpu) ret =EXCP_INTERRUPT ;break ; } -fprintf (stderr,"error: kvm run failed %s\n" , -strerror (-run_ret)); +if (!(run_ret == -EFAULT && run->exit_reason ==KVM_EXIT_MEMORY_FAULT )) { +fprintf (stderr,"error: kvm run failed %s\n" , +strerror (-run_ret));#ifdef TARGET_PPC -if (run_ret == -EBUSY ) { -fprintf (stderr, -"This is probably because your SMT is enabled.\n" -"VCPU can only run on primary threads with all " -"secondary threads offline.\n" ); - } +if (run_ret == -EBUSY ) { +fprintf (stderr, +"This is probably because your SMT is enabled.\n" +"VCPU can only run on primary threads with all " +"secondary threads offline.\n" ); + }#endif - ret =-1 ; -break ; + ret =-1 ; +break ; + } }trace_kvm_run_exit (cpu->cpu_index, run->exit_reason);
@@ -3054,6 +3118,16 @@ int kvm_cpu_exec(CPUState *cpu)break ; }break ; +case KVM_EXIT_MEMORY_FAULT : +if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE ) { +error_report ("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64, + (uint64_t )run->memory_fault.flags); + ret =-1 ; +break ; + } + ret =kvm_convert_memory (run->memory_fault.gpa, run->memory_fault.size, + run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE ); +break ;default : ret =kvm_arch_handle_exit (cpu, run);break ;
Signature · 1 line
2.34.1
PATCH v4 9/66 trace/kvm: Add trace for page convertion between shared and private
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
--- accel/kvm/kvm-all.c | 2 ++ accel/kvm/trace-events | 1 + 2 files changed, 3 insertions(+)
accel/kvm/kvm-all.c +2 -0
@@ -2902,6 +2902,8 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)void *addr;int ret =-1 ; +trace_kvm_convert_memory (start, size, to_private ?"shared_to_private" :"private_to_shared" ); +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) {return -1 ;
accel/kvm/trace-events +1 -0
@@ -31,3 +31,4 @@ kvm_cpu_exec(void) "" kvm_interrupt_exit_request(void) "" kvm_io_window_exit(void) "" kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32 +kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
Signature · 1 line
2.34.1
PATCH v4 10/66 *** HACK *** linux-headers: Update headers to pull in TDX API changes
In reply to Xiaoyao LiPull in recent TDX updates, which are not backwards compatible.
It's just to make this series runnable. It will be updated by script
scripts/update-linux-headers.sh
once TDX support is upstreamed in linux kernel
--- linux-headers/asm-x86/kvm.h | 91 +++++++++++++++++++++++++++++++++++++ linux-headers/linux/kvm.h | 89 ++++++++++++++++++++++++++++++++++++ 2 files changed, 180 insertions(+)
linux-headers/asm-x86/kvm.h +91 -0
@@ -562,5 +562,96 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_SW_PROTECTED_VM 1 +#define KVM_X86_TDX_VM 2 +#define KVM_X86_SNP_VM 3 + +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU , +KVM_TDX_INIT_MEM_REGION , +KVM_TDX_FINALIZE_VM , +KVM_TDX_RELEASE_VM , + +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +/* enum kvm_tdx_cmd_id */ +__u32 id; +/* flags for sub-commend. If sub-command doesn't use this, set zero. */ +__u32 flags; +/* +* data for each sub-command. An immediate or a pointer to the actual +* data in process virtual address. If sub-command doesn't use it, +* set zero. +*/ +__u64 data; +/* +* Auxiliary error code. The sub-command may return TDX SEAMCALL +* status code in addition to -Exxx. +* Defined for consistency with struct kvm_sev_cmd. +*/ +__u64 error; +}; + +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; + +struct kvm_tdx_capabilities { +__u64 attrs_fixed0; +__u64 attrs_fixed1; +__u64 xfam_fixed0; +__u64 xfam_fixed1; +#define TDX_CAP_GPAW_48 (1 << 0) +#define TDX_CAP_GPAW_52 (1 << 1) +__u32 supported_gpaw; +__u32 padding; +__u64 reserved[251 ]; + +__u32 nr_cpuid_configs; +struct kvm_tdx_cpuid_config cpuid_configs[]; +}; + +struct kvm_tdx_init_vm { +__u64 attributes; +__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha348 digest */ +/* +* For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB. +* This should be enough given sizeof(TD_PARAMS) = 1024. +* 8KB was chosen given because +* sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB. +*/ +__u64 reserved[1004 ]; + +/* +* Call KVM_TDX_INIT_VM before vcpu creation, thus before +* KVM_SET_CPUID2. +* This configuration supersedes KVM_SET_CPUID2s for VCPUs because the +* TDX module directly virtualizes those CPUIDs without VMM. The user +* space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with +* those values. If it doesn't, KVM may have wrong idea of vCPUIDs of +* the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX +* module doesn't virtualize. +*/ +struct kvm_cpuid2 cpuid; +}; + +#define KVM_TDX_MEASURE_MEMORY_REGION (1UL << 0) + +struct kvm_tdx_init_mem_region { +__u64 source_addr; +__u64 gpa; +__u64 nr_pages; +};#endif /* _ASM_X86_KVM_H */
linux-headers/linux/kvm.h +89 -0
107 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 11/66 i386: Introduce tdx-guest object
In reply to Xiaoyao LiIntroduce tdx-guest object which implements the interface of CONFIDENTIAL_GUEST_SUPPORT, and will be used to create TDX VMs (TDs) by
qemu -machine ...,confidential-guest-support=tdx0 \
-object tdx-guest,id=tdx0It has only one member 'attributes' with fixed value 0 and not configurable so far.
---
Changes in v4:
- update the new qapi since filed from 8.2 to 9.0
Changes in v1
- make @attributes not user-settable
--- configs/devices/i386-softmmu/default.mak | 1 + hw/i386/Kconfig | 5 +++ qapi/qom.json | 12 +++++++ target/i386/kvm/meson.build | 2 ++ target/i386/kvm/tdx.c | 40 ++++++++++++++++++++++++ target/i386/kvm/tdx.h | 19 +++++++++++ 6 files changed, 79 insertions(+) create mode 100644 target/i386/kvm/tdx.c create mode 100644 target/i386/kvm/tdx.h
configs/devices/i386-softmmu/default.mak +1 -0
@@ -18,6 +18,7 @@ #CONFIG_QXL=n #CONFIG_SEV=n #CONFIG_SGA=n +#CONFIG_TDX=n #CONFIG_TEST_DEVICES=n #CONFIG_TPM_CRB=n #CONFIG_TPM_TIS_ISA=n
hw/i386/Kconfig +5 -0
@@ -10,6 +10,10 @@ config SGX bool depends on KVM +config TDX + bool + depends on KVM + config PC bool imply APPLESMC
@@ -26,6 +30,7 @@ config PC imply QXL imply SEV imply SGX + imply TDX imply TEST_DEVICES imply TPM_CRB imply TPM_TIS_ISA
qapi/qom.json +12 -0
@@ -895,6 +895,16 @@ 'reduced-phys-bits': 'uint32', '*kernel-hashes': 'bool' } } +## +# @TdxGuestProperties: +# +# Properties for tdx-guest objects. +# +# Since: 9.0 +## +{ 'struct': 'TdxGuestProperties', + 'data': { }} + ## # @ThreadContextProperties: #
@@ -974,6 +984,7 @@ 'sev-guest', 'thread-context', 's390-pv-guest', + 'tdx-guest', 'throttle-group', 'tls-creds-anon', 'tls-creds-psk',
@@ -1041,6 +1052,7 @@ 'secret_keyring': { 'type': 'SecretKeyringProperties', 'if': 'CONFIG_SECRET_KEYRING' }, 'sev-guest': 'SevGuestProperties', + 'tdx-guest': 'TdxGuestProperties', 'thread-context': 'ThreadContextProperties', 'throttle-group': 'ThrottleGroupProperties', 'tls-creds-anon': 'TlsCredsAnonProperties',
target/i386/kvm/meson.build +2 -0
@@ -9,6 +9,8 @@ i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c')) i386_kvm_ss.add(when: 'CONFIG_SEV', if_false: files('sev-stub.c')) +i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c')) + i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c')) i386_system_ss.add_all(when: 'CONFIG_KVM', if_true: i386_kvm_ss)
target/i386/kvm/tdx.c new file +40 -0
@@ -0,0 +1,40 @@ +/* +* QEMU TDX support +* +* Copyright Intel +* +* Author: +* Xiaoyao Li <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory +* +*/ + +#include "qemu/osdep.h" +#include "qom/object_interfaces.h" + +#include "tdx.h" + +/* tdx guest */ +OBJECT_DEFINE_TYPE_WITH_INTERFACES (TdxGuest, + tdx_guest, +TDX_GUEST , +CONFIDENTIAL_GUEST_SUPPORT , + { TYPE_USER_CREATABLE }, + { NULL }) + +static void tdx_guest_init (Object *obj) +{ +TdxGuest *tdx =TDX_GUEST (obj); + + tdx->attributes =0 ; +} + +static void tdx_guest_finalize (Object *obj) +{ +} + +static void tdx_guest_class_init (ObjectClass *oc,void *data) +{ +}
target/i386/kvm/tdx.h new file +19 -0
@@ -0,0 +1,19 @@ +#ifndef QEMU_I386_TDX_H +#define QEMU_I386_TDX_H + +#include "exec/confidential-guest-support.h" + +#define TYPE_TDX_GUEST "tdx-guest" +#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST) + +typedef struct TdxGuestClass { +ConfidentialGuestSupportClass parent_class; +}TdxGuestClass ; + +typedef struct TdxGuest { +ConfidentialGuestSupport parent_obj; + +uint64_t attributes;/* TD attributes */ +}TdxGuest ; + +#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Re: [PATCH v4 11/66] i386: Introduce tdx-guest object
Quoted 8 lines · Xiaoyao Li
Introduce tdx-guest object which implements the interface of
CONFIDENTIAL_GUEST_SUPPORT, and will be used to create TDX VMs (TDs) byqemu -machine ...,confidential-guest-support=tdx0 \ -object tdx-guest,id=tdx0It has only one member 'attributes' with fixed value 0 and not configurable so far.
Really? Can't see it.
Suggest to add something like "Configuration properties will be added later in this series."
Quoted 3 lines
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>
[...]
Quoted 37 lines
qapi/qom.json +12
-0@@ -895,6 +895,16 @@ 'reduced-phys-bits': 'uint32', '*kernel-hashes': 'bool' } } +## +# @TdxGuestProperties: +# +# Properties for tdx-guest objects. +# +# Since: 9.0 +## +{ 'struct': 'TdxGuestProperties', + 'data': { }} + ## # @ThreadContextProperties: #@@ -974,6 +984,7 @@ 'sev-guest', 'thread-context', 's390-pv-guest', + 'tdx-guest', 'throttle-group', 'tls-creds-anon', 'tls-creds-psk',@@ -1041,6 +1052,7 @@ 'secret_keyring': { 'type': 'SecretKeyringProperties', 'if': 'CONFIG_SECRET_KEYRING' }, 'sev-guest': 'SevGuestProperties', + 'tdx-guest': 'TdxGuestProperties', 'thread-context': 'ThreadContextProperties', 'throttle-group': 'ThrottleGroupProperties', 'tls-creds-anon': 'TlsCredsAnonProperties',
[...]
Re: [PATCH v4 11/66] i386: Introduce tdx-guest object
Quoted 5 lines · Markus Armbruster
Quoted 3 lines · Xiaoyao Li
Introduce tdx-guest object which implements the interface of
CONFIDENTIAL_GUEST_SUPPORT, and will be used to create TDX VMs (TDs) by
Quoted 7 lines
Quoted 5 lines
qemu -machine ...,confidential-guest-support=tdx0 \ -object tdx-guest,id=tdx0It has only one member 'attributes' with fixed value 0 and not configurable so far.
Really? Can't see it.
The 'attributes' referred to is an internal struct field, rather than a QAPI declared member.
Quoted 41 lines
Suggest to add something like "Configuration properties will be added later in this series."
Quoted 3 lines
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>[...]
Quoted 31 lines
qapi/qom.json +11
-0@@ -895,6 +895,16 @@ 'reduced-phys-bits': 'uint32', '*kernel-hashes': 'bool' } } +## +# @TdxGuestProperties: +# +# Properties for tdx-guest objects. +# +# Since: 9.0 +## +{ 'struct': 'TdxGuestProperties', + 'data': { }} + ## # @ThreadContextProperties: #@@ -974,6 +984,7 @@ 'sev-guest', 'thread-context', 's390-pv-guest', + 'tdx-guest', 'throttle-group', 'tls-creds-anon', 'tls-creds-psk',@@ -1041,6 +1052,7 @@ 'secret_keyring': { 'type': 'SecretKeyringProperties',
Quoted 9 lines
Quoted 6 lines
qapi/qom.json +1
-0'if': 'CONFIG_SECRET_KEYRING' }, 'sev-guest': 'SevGuestProperties', + 'tdx-guest': 'TdxGuestProperties', 'thread-context': 'ThreadContextProperties', 'throttle-group': 'ThrottleGroupProperties', 'tls-creds-anon': 'TlsCredsAnonProperties',[...]
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v4 12/66 target/i386: Implement mc->kvm_type() to get VM type
In reply to Xiaoyao LiTDX VM requires VM type KVM_X86_TDX_VM to be passed to kvm_ioctl(KVM_CREATE_VM). Hence implement mc->kvm_type() for i386 architecture.
If tdx-guest object is specified to confidential-guest-support, like,
qemu -machine ...,confidential-guest-support=tdx0 \
-object tdx-guest,id=tdx0,...it parses VM type as KVM_X86_TDX_VM. Otherwise, it's KVM_X86_DEFAULT_VM.
Also store the vm_type in MachineState for other code to query what the VM type is.
---
Changes in v4:
- fix the build error of kvm_get_vm_type() when --disable-kvm;
--- hw/i386/x86.c | 12 ++++++++++++ include/hw/i386/x86.h | 1 + target/i386/kvm/kvm.c | 30 ++++++++++++++++++++++++++++++ target/i386/kvm/kvm_i386.h | 1 + 4 files changed, 44 insertions(+)
hw/i386/x86.c +12 -0
@@ -1386,6 +1386,17 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,qapi_free_SgxEPCList (list); } +static int x86_kvm_type (MachineState *ms,const char *vm_type) +{ +X86MachineState *x86ms =X86_MACHINE (ms); +int kvm_type; + + kvm_type =kvm_enabled () ?kvm_get_vm_type (ms, vm_type) :0 ; + x86ms->vm_type = kvm_type; + +return kvm_type; +} +static void x86_machine_initfn (Object *obj) {X86MachineState *x86ms =X86_MACHINE (obj);
@@ -1410,6 +1421,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) mc->cpu_index_to_instance_props = x86_cpu_index_to_props; mc->get_default_cpu_node_id = x86_get_default_cpu_node_id; mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids; + mc->kvm_type = x86_kvm_type; x86mc->save_tsc_khz = true; x86mc->fwcfg_dma_enabled = true; nc->nmi_monitor_handler = x86_nmi;
include/hw/i386/x86.h +1 -0
@@ -41,6 +41,7 @@ struct X86MachineState {MachineState parent;/*< public >*/ +unsigned int vm_type;/* Pointers to devices and objects: */ ISADevice *rtc;
target/i386/kvm/kvm.c +30 -0
@@ -32,6 +32,7 @@#include "sysemu/runstate.h" #include "kvm_i386.h" #include "sev.h" +#include "tdx.h" #include "xen-emu.h" #include "hyperv.h" #include "hyperv-proto.h"
@@ -161,6 +162,35 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];static RateLimit bus_lock_ratelimit_ctrl;static int kvm_get_one_msr (X86CPU *cpu,int index,uint64_t *value); +static const char *vm_type_name[] = { + [KVM_X86_DEFAULT_VM ] ="default" , + [KVM_X86_TDX_VM ] ="tdx" , +}; + +int kvm_get_vm_type (MachineState *ms,const char *vm_type) +{ +int kvm_type =KVM_X86_DEFAULT_VM ; + +if (ms->cgs &&object_dynamic_cast (OBJECT (ms->cgs),TYPE_TDX_GUEST )) { + kvm_type =KVM_X86_TDX_VM ; + } + +/* +* old KVM doesn't support KVM_CAP_VM_TYPES and KVM_X86_DEFAULT_VM +* is always supported +*/ +if (kvm_type ==KVM_X86_DEFAULT_VM ) { +return kvm_type; + } + +if (!(kvm_check_extension (KVM_STATE (ms->accelerator),KVM_CAP_VM_TYPES ) &BIT (kvm_type))) { +error_report ("vm-type %s not supported by KVM" , vm_type_name[kvm_type]); +exit (1 ); + } + +return kvm_type; +} +bool kvm_has_smm (void ) {return kvm_vm_check_extension (kvm_state,KVM_CAP_X86_SMM );
target/i386/kvm/kvm_i386.h +1 -0
@@ -37,6 +37,7 @@ bool kvm_hv_vpindex_settable(void);bool kvm_enable_sgx_provisioning (KVMState *s);bool kvm_hyperv_expand_features (X86CPU *cpu,Error **errp); +int kvm_get_vm_type (MachineState *ms,const char *vm_type);void kvm_arch_reset_vcpu (X86CPU *cs);void kvm_arch_after_reset_vcpu (X86CPU *cpu);void kvm_arch_do_init_vcpu (X86CPU *cs);
Signature · 1 line
2.34.1
PATCH v4 13/66 target/i386: Introduce kvm_confidential_guest_init()
In reply to Xiaoyao LiIntroduce a separate function kvm_confidential_guest_init(), which dispatches specific confidential guest initialization function by ms->cgs type.
--- target/i386/kvm/kvm.c | 11 ++++++++++- target/i386/sev.c | 1 - target/i386/sev.h | 2 ++ 3 files changed, 12 insertions(+), 2 deletions(-)
target/i386/kvm/kvm.c +10 -1
@@ -2541,6 +2541,15 @@ int kvm_arch_get_default_type(MachineState *ms)return 0 ; } +static int kvm_confidential_guest_init (MachineState *ms,Error **errp) +{ +if (object_dynamic_cast (OBJECT (ms->cgs),TYPE_SEV_GUEST )) { +return sev_kvm_init (ms->cgs, errp); + } + +return 0 ; +} +int kvm_arch_init (MachineState *ms,KVMState *s) {uint64_t identity_base =0xfffbc000 ;
@@ -2561,7 +2570,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)* mechanisms are supported in future (e.g. TDX), they'll need * their own initialization either here or elsewhere. */ - ret =sev_kvm_init (ms->cgs, &local_err); + ret =kvm_confidential_guest_init (ms, &local_err);if (ret <0 ) {error_report_err (local_err);return ret;
target/i386/sev.c +0 -1
@@ -39,7 +39,6 @@#include "hw/i386/pc.h" #include "exec/address-spaces.h" -#define TYPE_SEV_GUEST "sev-guest"OBJECT_DECLARE_SIMPLE_TYPE (SevGuestState,SEV_GUEST )
target/i386/sev.h +2 -0
@@ -20,6 +20,8 @@#include "exec/confidential-guest-support.h" +#define TYPE_SEV_GUEST "sev-guest" +#define SEV_POLICY_NODBG 0x1#define SEV_POLICY_NOKS 0x2#define SEV_POLICY_ES 0x4
Signature · 1 line
2.34.1
Re: [PATCH v4 13/66] target/i386: Introduce kvm_confidential_guest_init()
Quoted 19 lines · Xiaoyao Li
Introduce a separate function
kvm_confidential_guest_init(), which dispatches specific confidential guest initialization function byms->cgstype.Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>--- target/i386/kvm/kvm.c | 11 ++++++++++- target/i386/sev.c | 1 - target/i386/sev.h | 2 ++ 3 files changed, 12 insertions(+), 2 deletions(-)target/i386/kvm/kvm.c +0
-0@@ -2541,6 +2541,15 @@ int kvm_arch_get_default_type(MachineState *ms)return 0 ;
target/i386/kvm/kvm.c int kvm_arch_get_default_type(MachineState *ms) · line 2551
Quoted 10 lines
target/i386/kvm/kvm.c +8
-0} +static int kvm_confidential_guest_init (MachineState *ms,Error **errp) +{ +if (object_dynamic_cast (OBJECT (ms->cgs),TYPE_SEV_GUEST )) { +return sev_kvm_init (ms->cgs, errp); + } + +return 0 ; +}
if/else ladders checking object subclass type and then invoking a subclass specific method are quite an object oriented code anti-pattern. I think this suggests that ConfidentialGuestSupportClass should gain a member
void (*kvm_init)(ConfidentialGuestSpport *cgs, Error **errp);
and then an impl
void confidential_guest_kvm_init(ConfidentialGuestSupport *cgs, Error *errp) {
ConfidentialGuestSupportClass *klass = CONFIDENTIAL_GUEST_SUPPORT_GET_CLASS(cgs);
klass->kvm_init(cgs, errp)
}that way, kvm.c arch code doesn't get directly bound to the APIs for the different CVM technologies, the ConfidentialGuestSupport object will act as a proper isolation layer.
This is likely to apply in other parts of KVM code that need to call into SEV/TDX specific functions too.
Quoted 41 lines
target/i386/kvm/kvm.c +2
-1+int kvm_arch_init (MachineState *ms,KVMState *s) {uint64_t identity_base =0xfffbc000 ;@@ -2561,7 +2570,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)* mechanisms are supported in future (e.g. TDX), they'll need * their own initialization either here or elsewhere. */ - ret =sev_kvm_init (ms->cgs, &local_err); + ret =kvm_confidential_guest_init (ms, &local_err);if (ret <0 ) {error_report_err (local_err);return ret;target/i386/sev.c +0
-1@@ -39,7 +39,6 @@#include "hw/i386/pc.h" #include "exec/address-spaces.h" -#define TYPE_SEV_GUEST "sev-guest"OBJECT_DECLARE_SIMPLE_TYPE (SevGuestState,SEV_GUEST )target/i386/sev.h +2
-0@@ -20,6 +20,8 @@#include "exec/confidential-guest-support.h" +#define TYPE_SEV_GUEST "sev-guest" +#define SEV_POLICY_NODBG 0x1#define SEV_POLICY_NOKS 0x2#define SEV_POLICY_ES 0x4Signature · 1 line
2.34.1
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v4 14/66 i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context
In reply to Xiaoyao LiIntroduce tdx_kvm_init() and invoke it in kvm_confidential_guest_init() if it's a TDX VM.
Set ms->require_guest_memfd to require kvm guest memfd allocation for any memory backend. More TDX specific initialization will be added later.
--- target/i386/kvm/kvm.c | 15 ++++++--------- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-stub.c | 8 ++++++++ target/i386/kvm/tdx.c | 9 +++++++++ target/i386/kvm/tdx.h | 2 ++ 5 files changed, 26 insertions(+), 10 deletions(-) create mode 100644 target/i386/kvm/tdx-stub.c
target/i386/kvm/kvm.c +6 -9
@@ -62,6 +62,7 @@#include "migration/blocker.h" #include "exec/memattrs.h" #include "trace.h" +#include "tdx.h" #include CONFIG_DEVICES
@@ -2545,6 +2546,8 @@ static int kvm_confidential_guest_init(MachineState *ms, Error **errp) {if (object_dynamic_cast (OBJECT (ms->cgs),TYPE_SEV_GUEST )) {return sev_kvm_init (ms->cgs, errp); + }else if (object_dynamic_cast (OBJECT (ms->cgs),TYPE_TDX_GUEST )) { +return tdx_kvm_init (ms, errp); }return 0 ;
@@ -2559,16 +2562,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s)Error *local_err =NULL ;/* -* Initialize SEV context, if required +* Initialize confidential guest (SEV/TDX) context, if required * -* If no memory encryption is requested (ms->cgs == NULL) this is -* a no-op. -* -* It's also a no-op if a non-SEV confidential guest support -* mechanism is selected. SEV is the only mechanism available to -* select on x86 at present, so this doesn't arise, but if new -* mechanisms are supported in future (e.g. TDX), they'll need -* their own initialization either here or elsewhere. +* It's a no-op if a non-SEV/non-tdx confidential guest support +* mechanism is selected, i.e., ms->cgs == NULL */ ret =kvm_confidential_guest_init (ms, &local_err);if (ret <0 ) {
target/i386/kvm/meson.build +1 -1
@@ -9,7 +9,7 @@ i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c')) i386_kvm_ss.add(when: 'CONFIG_SEV', if_false: files('sev-stub.c')) -i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c')) +i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c')) i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
target/i386/kvm/tdx-stub.c new file +8 -0
@@ -0,0 +1,8 @@ +#include "qemu/osdep.h" + +#include "tdx.h" + +int tdx_kvm_init (MachineState *ms,Error **errp) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +9 -0
@@ -12,10 +12,19 @@*/ #include "qemu/osdep.h" +#include "qapi/error.h" #include "qom/object_interfaces.h" +#include "hw/i386/x86.h" #include "tdx.h" +int tdx_kvm_init (MachineState *ms,Error **errp) +{ + ms->require_guest_memfd = true; + +return 0 ; +} +/* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,
target/i386/kvm/tdx.h +2 -0
@@ -16,4 +16,6 @@ typedef struct TdxGuest { uint64_t attributes; /* TD attributes */ } TdxGuest; +int tdx_kvm_init (MachineState *ms,Error **errp); +#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 15/66 i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
In reply to Xiaoyao LiKVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.
Since there is no interface reporting how many cpuid configs contains in KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
Besides, introduce the interfaces to invoke TDX "ioctls" at different scope (KVM, VM and VCPU) in preparation.
---
Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;
Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();
changes in v2:
- Make the error message more clear;
changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
--- target/i386/kvm/kvm.c | 2 - target/i386/kvm/kvm_i386.h | 2 + target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++- 3 files changed, 92 insertions(+), 3 deletions(-)
target/i386/kvm/kvm.c +0 -2
@@ -1686,8 +1686,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)static Error *invtsc_mig_blocker; -#define KVM_MAX_CPUID_ENTRIES 100 -static void kvm_init_xsave (CPUX86State *env) {if (has_xsave2) {
target/i386/kvm/kvm_i386.h +2 -0
@@ -13,6 +13,8 @@#include "sysemu/kvm.h" +#define KVM_MAX_CPUID_ENTRIES 100 +#ifdef CONFIG_KVM #define kvm_pit_in_kernel () \
target/i386/kvm/tdx.c +90 -1
107 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 16/66 i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object
In reply to Xiaoyao LiIt will need special handling for TDX VMs all around the QEMU. Introduce is_tdx_vm() helper to query if it's a TDX VM.
Cache tdx_guest object thus no need to cast from ms->cgs every time.
---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();
--- target/i386/kvm/tdx.c | 15 ++++++++++++++- target/i386/kvm/tdx.h | 10 ++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +14 -1
@@ -21,8 +21,16 @@#include "kvm_i386.h" #include "tdx.h" +static TdxGuest *tdx_guest; +static struct kvm_tdx_capabilities *tdx_caps; +/* It's valid after kvm_confidential_guest_init()->kvm_tdx_init() */ +bool is_tdx_vm (void ) +{ +return !!tdx_guest; +} +enum tdx_ioctl_level {TDX_VM_IOCTL ,TDX_VCPU_IOCTL ,
@@ -103,15 +111,20 @@ static int get_tdx_capabilities(Error **errp)int tdx_kvm_init (MachineState *ms,Error **errp) { +TdxGuest *tdx =TDX_GUEST (OBJECT (ms->cgs));int r =0 ; ms->require_guest_memfd = true;if (!tdx_caps) { r =get_tdx_capabilities (errp); +if (r) { +return r; + } } -return r; + tdx_guest = tdx; +return 0 ; }/* tdx guest */
target/i386/kvm/tdx.h +10 -0
@@ -1,6 +1,10 @@#ifndef QEMU_I386_TDX_H #define QEMU_I386_TDX_H +#ifndef CONFIG_USER_ONLY +#include CONFIG_DEVICES /* CONFIG_TDX */ +#endif +#include "exec/confidential-guest-support.h" #define TYPE_TDX_GUEST "tdx-guest"
@@ -16,6 +20,12 @@ typedef struct TdxGuest { uint64_t attributes; /* TD attributes */ } TdxGuest; +#ifdef CONFIG_TDX +bool is_tdx_vm (void ); +#else +#define is_tdx_vm () 0 +#endif /* CONFIG_TDX */ +int tdx_kvm_init (MachineState *ms,Error **errp);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 17/66 i386/tdx: Adjust the supported CPUID based on TDX restrictions
In reply to Xiaoyao LiAccording to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types:
------------------------------------------------------------------------
1 | As configured | configurable by VMM, independent of native value;
------------------------------------------------------------------------
2 | As configured | configurable by VMM if the bit is supported natively
(if native) | Otherwise it equals as native(0).
------------------------------------------------------------------------
3 | Fixed | fixed to 0/1
------------------------------------------------------------------------
4 | Native | reflect the native value
------------------------------------------------------------------------
5 | Calculated | calculated by TDX module.
------------------------------------------------------------------------
6 | Inducing #VE | get #VE exception
------------------------------------------------------------------------
Note:
1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3.
2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec, TDX module injects #VE to TDs when those are queried. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM.
Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported CPUID bits for TDs. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions.
Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord.
Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields:
- tdx_fixed0/1: The bits that are fixed as 0/1;
- depends_on_vmm_cap: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them.
- inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM.
- supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs.
By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps:
- get the base of VMM supported feature set;
- if the leaf is not a FeatureWord just return VMM's value without modification;
- if the leaf is an inducing_ve type, applying supported_on_ve mask and return;
- include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.)
- apply fixed0/1 to it (it covers #3, and rectifies the previous step);
- add configurable bits (it covers the other part of type #1);
- fix the ones in vmm_fixup;
(Calculated type is ignored since it's determined at runtime).
--- target/i386/cpu.h | 16 +++ target/i386/kvm/kvm.c | 4 + target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 2 + 4 files changed, 285 insertions(+)
target/i386/cpu.h +16 -0
@@ -780,6 +780,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,/* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)/* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2)/* 1st Group of Advanced Bit Manipulation Extensions */
@@ -798,8 +800,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_EBX_INVPCID (1U << 10)/* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12)/* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15)/* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16)/* AVX-512 Doubleword & Quadword Instruction */
@@ -855,10 +861,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_AVX512VNNI (1U << 11)/* Support for VPOPCNT[B,W] and VPSHUFBITQMB */ #define CPUID_7_0_ECX_AVX512BITALG (1U << 12) +/* Intel Total Memory Encryption */ +#define CPUID_7_0_ECX_TME (1U << 13)/* POPCNT for vectors of DW/QW */ #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14) +/* Placeholder for bit 15 */ +#define CPUID_7_0_ECX_FZM (1U << 15)/* 5-level Page Tables */ #define CPUID_7_0_ECX_LA57 (1U << 16) +/* MAWAU for MPX */ +#define CPUID_7_0_ECX_MAWAU (31U << 17)/* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22)/* Bus Lock Debug Exception */
@@ -869,6 +881,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_MOVDIRI (1U << 27)/* Move 64 Bytes as Direct Store Instruction */ #define CPUID_7_0_ECX_MOVDIR64B (1U << 28) +/* ENQCMD and ENQCMDS instructions */ +#define CPUID_7_0_ECX_ENQCMD (1U << 29)/* Support SGX Launch Control */ #define CPUID_7_0_ECX_SGX_LC (1U << 30)/* Protection Keys for Supervisor-mode Pages */
@@ -886,6 +900,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_EDX_SERIALIZE (1U << 14)/* TSX Suspend Load Address Tracking instruction */ #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16) +/* PCONFIG instruction */ +#define CPUID_7_0_EDX_PCONFIG (1U << 18)/* Architectural LBRs */ #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)/* AMX_BF16 instruction */
target/i386/kvm/kvm.c +4 -0
@@ -521,6 +521,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function, ret |=1U <<KVM_HINTS_REALTIME ; } +if (is_tdx_vm ()) { +tdx_get_supported_cpuid (function, index, reg, &ret); + } +return ret; }
target/i386/kvm/tdx.c +263 -0
280 lines not on this page · raw message
target/i386/kvm/tdx.h +2 -0
@@ -27,5 +27,7 @@ bool is_tdx_vm(void);#endif /* CONFIG_TDX */ int tdx_kvm_init (MachineState *ms,Error **errp); +void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg, +uint32_t *ret);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 18/66 i386/tdx: Make Intel-PT unsupported for TD guest
In reply to Xiaoyao LiDue to the fact that Intel-PT virtualization support has been broken in QEMU since Sapphire Rapids generation[1], below warning is triggered when luanching TD guest:
warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]
Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD guest, to avoid the confusing warning.
[1] https://lore.kernel.org/qemu-devel/[email protected]/
---
Changes in v4:
- newly added patch;
--- target/i386/kvm/tdx.c | 5 +++++ 1 file changed, 5 insertions(+)
target/i386/kvm/tdx.c +5 -0
@@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,if (function ==1 && reg ==R_ECX && !enable_cpu_pm) { *ret &= ~CPUID_EXT_MONITOR ; } + +/* QEMU Intel-pt support is broken, don't advertise Intel-PT */ +if (function ==7 && reg ==R_EBX ) { + *ret &= ~CPUID_7_0_EBX_INTEL_PT ; + } }enum tdx_ioctl_level {
Signature · 1 line
2.34.1
PATCH v4 19/66 i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[]
In reply to Xiaoyao Litdx_cpuid_lookup[].tdx_fixed0/1 is QEMU maintained data which reflects TDX restrictions regrading what bits are fixed by TDX module.
It's retrieved from TDX spec and static. However, TDX may evolve and change some fixed fields to configurable in the future. Update tdx_cpuid.lookup[].tdx_fixed0/1 fields by removing the bits that reported from TDX module as configurable. This can adapt with the updated TDX (module) automatically.
--- target/i386/kvm/tdx.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
target/i386/kvm/tdx.c +34 -0
@@ -377,6 +377,38 @@ static int get_tdx_capabilities(Error **errp)return 0 ; } +static void update_tdx_cpuid_lookup_by_tdx_caps (void ) +{ +KvmTdxCpuidLookup *entry; +FeatureWordInfo *fi; +uint32_t config; +FeatureWord w; + +for (w =0 ; w <FEATURE_WORDS ; w++) { + fi = &feature_word_info[w]; + entry = &tdx_cpuid_lookup[w]; + +if (fi->type !=CPUID_FEATURE_WORD ) { +continue ; + } + + config =tdx_cap_cpuid_config (fi->cpuid.eax, + fi->cpuid.needs_ecx ? fi->cpuid.ecx : ~0u , + fi->cpuid.reg); + +if (!config) { +continue ; + } + +/* +* Remove the configurable bits from tdx_fixed0/1 in case QEMU +* maintained fixed0/1 values is outdated to TDX module. +*/ + entry->tdx_fixed0 &= ~config; + entry->tdx_fixed1 &= ~config; + } +} +int tdx_kvm_init (MachineState *ms,Error **errp) {TdxGuest *tdx =TDX_GUEST (OBJECT (ms->cgs));
@@ -391,6 +423,8 @@ int tdx_kvm_init(MachineState *ms, Error **errp) } } +update_tdx_cpuid_lookup_by_tdx_caps (); + tdx_guest = tdx;return 0 ; }
Signature · 1 line
2.34.1
PATCH v4 20/66 i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup
In reply to Xiaoyao LiKVM requires userspace to pass XFAM configuration via CPUID 0xD leaves.
Convert tdx_caps->xfam_fixed0/1 into corresponding tdx_cpuid_lookup[].tdx_fixed0/1 field of CPUID 0xD leaves. Thus the requirement can be applied naturally.
--- target/i386/cpu.c | 3 --- target/i386/cpu.h | 3 +++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 3 files changed, 27 insertions(+), 3 deletions(-)
target/i386/cpu.c +0 -3
@@ -1575,9 +1575,6 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = { };#undef REGISTER -/* CPUID feature bits available in XSS */ -#define CPUID_XSTATE_XSS_MASK (XSTATE_ARCH_LBR_MASK) -ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT ] = { [XSTATE_FP_BIT ] = {/* x87 FP state component is always enabled if XSAVE is supported */
target/i386/cpu.h +3 -0
@@ -588,6 +588,9 @@ typedef enum X86Seg { XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK | \ XSTATE_XTILE_CFG_MASK | XSTATE_XTILE_DATA_MASK) +/* CPUID feature bits available in XSS */ +#define CPUID_XSTATE_XSS_MASK (XSTATE_ARCH_LBR_MASK) +/* CPUID feature words */ typedef enum FeatureWord { FEAT_1_EDX, /* CPUID[1].EDX */
target/i386/kvm/tdx.c +24 -0
@@ -407,6 +407,30 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) entry->tdx_fixed0 &= ~config; entry->tdx_fixed1 &= ~config; } + +/* +* Because KVM gets XFAM settings via CPUID leaves 0xD, map +* tdx_caps->xfam_fixed{0, 1} into tdx_cpuid_lookup[].tdx_fixed{0, 1}. +* +* Then the enforment applies in tdx_get_configurable_cpuid() naturally. +*/ + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_LO ].tdx_fixed0 = + (uint32_t )~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XCR0_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_LO ].tdx_fixed1 = + (uint32_t )tdx_caps->xfam_fixed1 &CPUID_XSTATE_XCR0_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_HI ].tdx_fixed0 = + (~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XCR0_MASK ) >>32 ; + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_HI ].tdx_fixed1 = + (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XCR0_MASK ) >>32 ; + + tdx_cpuid_lookup[FEAT_XSAVE_XSS_LO ].tdx_fixed0 = + (uint32_t )~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XSS_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XSS_LO ].tdx_fixed1 = + (uint32_t )tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XSS_HI ].tdx_fixed0 = + (~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XSS_MASK ) >>32 ; + tdx_cpuid_lookup[FEAT_XSAVE_XSS_HI ].tdx_fixed1 = + (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; }int tdx_kvm_init (MachineState *ms,Error **errp)
Signature · 1 line
2.34.1
PATCH v4 21/66 i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup
In reply to Xiaoyao LiSome bits in TD attributes have corresponding CPUID feature bits. Reflect the fixed0/1 restriction on TD attributes to their corresponding CPUID bits in tdx_cpuid_lookup[] as well.
---
Changes in v4:
- reverse the meaning of tdx_caps->attr_fixed0, because value 0 of bit means the bit must be fixed 0.
--- target/i386/cpu-internal.h | 9 +++++++++ target/i386/cpu.c | 9 --------- target/i386/cpu.h | 2 ++ target/i386/kvm/tdx.c | 21 +++++++++++++++++++++ 4 files changed, 32 insertions(+), 9 deletions(-)
target/i386/cpu-internal.h +9 -0
@@ -20,6 +20,15 @@ #ifndef I386_CPU_INTERNAL_H #define I386_CPU_INTERNAL_H +typedef struct FeatureMask { + FeatureWord index; + uint64_t mask; +} FeatureMask; + +typedef struct FeatureDep { + FeatureMask from, to; +} FeatureDep; + typedef enum FeatureWordType { CPUID_FEATURE_WORD, MSR_FEATURE_WORD,
target/i386/cpu.c +0 -9
@@ -1442,15 +1442,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = { }, }; -typedef struct FeatureMask { -FeatureWord index; -uint64_t mask; -}FeatureMask ; - -typedef struct FeatureDep { -FeatureMask from, to; -}FeatureDep ; -static FeatureDep feature_dependencies[] = { { .from = {FEAT_7_0_EDX ,CPUID_7_0_EDX_ARCH_CAPABILITIES },
target/i386/cpu.h +2 -0
@@ -876,6 +876,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_MAWAU (31U << 17)/* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22) +/* KeyLocker */ +#define CPUID_7_0_ECX_KeyLocker (1U << 23)/* Bus Lock Debug Exception */ #define CPUID_7_0_ECX_BUS_LOCK_DETECT (1U << 24)/* Cache Line Demote Instruction */
target/i386/kvm/tdx.c +21 -0
@@ -32,6 +32,13 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_ATTRIBUTES_MAX_BITS 64 + +static FeatureMask tdx_attrs_ctrl_fields[TDX_ATTRIBUTES_MAX_BITS ] = { + [30 ] = { .index =FEAT_7_0_ECX , .mask =CPUID_7_0_ECX_PKS }, + [31 ] = { .index =FEAT_7_0_ECX , .mask = CPUID_7_0_ECX_KeyLocker}, +}; + typedef struct KvmTdxCpuidLookup { uint32_t tdx_fixed0; uint32_t tdx_fixed1;
@@ -383,6 +390,8 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void)FeatureWordInfo *fi;uint32_t config;FeatureWord w; +FeatureMask *fm; +int i;for (w =0 ; w <FEATURE_WORDS ; w++) { fi = &feature_word_info[w];
@@ -408,6 +417,18 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) entry->tdx_fixed1 &= ~config; } +for (i =0 ; i <ARRAY_SIZE (tdx_attrs_ctrl_fields); i++) { + fm = &tdx_attrs_ctrl_fields[i]; + +if (~(tdx_caps->attrs_fixed0 & (1ULL << i))) { + tdx_cpuid_lookup[fm->index].tdx_fixed0 |= fm->mask; + } + +if (tdx_caps->attrs_fixed1 & (1ULL << i)) { + tdx_cpuid_lookup[fm->index].tdx_fixed1 |= fm->mask; + } + } +/* * Because KVM gets XFAM settings via CPUID leaves 0xD, map * tdx_caps->xfam_fixed{0, 1} into tdx_cpuid_lookup[].tdx_fixed{0, 1}.
Signature · 1 line
2.34.1
PATCH v4 22/66 i386/kvm: Move architectural CPUID leaf generation to separate helper
In reply to Xiaoyao LiFrom: Sean Christopherson <[email protected]>
Move the architectural (for lack of a better term) CPUID leaf generation to a separate helper so that the generation code can be reused by TDX, which needs to generate a canonical VM-scoped configuration.
--- target/i386/kvm/kvm.c | 459 +++++++++++++++++++------------------ target/i386/kvm/kvm_i386.h | 3 + 2 files changed, 240 insertions(+), 222 deletions(-)
target/i386/kvm/kvm.c +237 -222
@@ -1732,6 +1732,241 @@ static void kvm_init_nested_state(CPUX86State *env) } } +uint32_t kvm_x86_arch_cpuid (CPUX86State *env,struct kvm_cpuid_entry2 *entries, +uint32_t cpuid_i) +{ +uint32_t limit, i, j; +uint32_t unused; +struct kvm_cpuid_entry2 *c; + +if (cpuid_i >KVM_MAX_CPUID_ENTRIES ) { +error_report ("exceeded cpuid index (%d) for entries[]" , cpuid_i); +abort (); + } + +cpu_x86_cpuid (env,0 ,0 , &limit, &unused, &unused, &unused); + +for (i =0 ; i <= limit; i++) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"unsupported level value: 0x%x\n" , limit); +abort (); + } + c = &entries[cpuid_i++]; + +switch (i) { +case 2 : { +/* Keep reading function 2 till all the input is received */ +int times; + + c->function = i; + c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC | +KVM_CPUID_FLAG_STATE_READ_NEXT ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + times = c->eax &0xff ; + +for (j =1 ; j < times; ++j) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:2):eax & 0xf = 0x%x\n" , times); +abort (); + } + c = &entries[cpuid_i++]; + c->function = i; + c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + } +break ; + } +case 0x1f : +if (env->nr_dies <2 ) { + cpuid_i--; +break ; + } +/* fallthrough */ +case 4 : +case 0xb : +case 0xd : +for (j =0 ; ; j++) { +if (i ==0xd && j ==64 ) { +break ; + } + + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (i ==4 && c->eax ==0 ) { +break ; + } +if (i ==0xb && !(c->ecx &0xff00 )) { +break ; + } +if (i ==0x1f && !(c->ecx &0xff00 )) { +break ; + } +if (i ==0xd && c->eax ==0 ) { +continue ; + } +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); +abort (); + } + c = &entries[cpuid_i++]; + } +break ; +case 0x12 : +for (j =0 ; ; j++) { + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (j >1 && (c->eax &0xf ) !=1 ) { +break ; + } + +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x12,ecx:0x%x)\n" , j); +abort (); + } + c = &entries[cpuid_i++]; + } +break ; +case 0x7 : +case 0x14 : +case 0x1d : +case 0x1e : { +uint32_t times; + + c->function = i; + c->index =0 ; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + times = c->eax; + +for (j =1 ; j <= times; ++j) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); +abort (); + } + c = &entries[cpuid_i++]; + c->function = i; + c->index = j; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + } +break ; + } +default : + c->function = i; + c->flags =0 ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); +if (!c->eax && !c->ebx && !c->ecx && !c->edx) { +/* +* KVM already returns all zeroes if a CPUID entry is missing, +* so we can omit it and avoid hitting KVM's 80-entry limit. +*/ + cpuid_i--; + } +break ; + } + } + +if (limit >=0x0a ) { +uint32_t eax, edx; + +cpu_x86_cpuid (env,0x0a ,0 , &eax, &unused, &unused, &edx); + + has_architectural_pmu_version = eax &0xff ; +if (has_architectural_pmu_version >0 ) { + num_architectural_pmu_gp_counters = (eax &0xff00 ) >>8 ; + +/* Shouldn't be more than 32, since that's the number of bits +* available in EBX to tell us _which_ counters are available. +* Play it safe. +*/ +if (num_architectural_pmu_gp_counters >MAX_GP_COUNTERS ) { + num_architectural_pmu_gp_counters =MAX_GP_COUNTERS ; + } + +if (has_architectural_pmu_version >1 ) { + num_architectural_pmu_fixed_counters = edx &0x1f ; + +if (num_architectural_pmu_fixed_counters >MAX_FIXED_COUNTERS ) { + num_architectural_pmu_fixed_counters =MAX_FIXED_COUNTERS ; + } + } + } + } + +cpu_x86_cpuid (env,0x80000000 ,0 , &limit, &unused, &unused, &unused); + +for (i =0x80000000 ; i <= limit; i++) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"unsupported xlevel value: 0x%x\n" , limit); +abort (); + } + c = &entries[cpuid_i++]; + +switch (i) { +case 0x8000001d : +/* Query for all AMD cache information leaves */ +for (j =0 ; ; j++) { + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (c->eax ==0 ) { +break ; + } +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); +abort (); + } + c = &entries[cpuid_i++]; + } +break ; +default : + c->function = i; + c->flags =0 ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); +if (!c->eax && !c->ebx && !c->ecx && !c->edx) { +/* +* KVM already returns all zeroes if a CPUID entry is missing, +* so we can omit it and avoid hitting KVM's 80-entry limit. +*/ + cpuid_i--; + } +break ; + } + } + +/* Call Centaur's CPUID instructions they are supported. */ +if (env->cpuid_xlevel2 >0 ) { +cpu_x86_cpuid (env,0xC0000000 ,0 , &limit, &unused, &unused, &unused); + +for (i =0xC0000000 ; i <= limit; i++) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"unsupported xlevel2 value: 0x%x\n" , limit); +abort (); + } + c = &entries[cpuid_i++]; + + c->function = i; + c->flags =0 ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + } + } + +return cpuid_i; +} +int kvm_arch_init_vcpu (CPUState *cs) {struct {
@@ -1748,8 +1983,7 @@ int kvm_arch_init_vcpu(CPUState *cs)X86CPU *cpu =X86_CPU (cs);CPUX86State *env = &cpu->env; -uint32_t limit, i, j, cpuid_i; -uint32_t unused; +uint32_t cpuid_i;struct kvm_cpuid_entry2 *c;uint32_t signature[3 ];int kvm_base =KVM_CPUID_SIGNATURE ;
@@ -1902,8 +2136,6 @@ int kvm_arch_init_vcpu(CPUState *cs) c->edx = env->features[FEAT_KVM_HINTS ]; } -cpu_x86_cpuid (env,0 ,0 , &limit, &unused, &unused, &unused); -if (cpu->kvm_pv_enforce_cpuid) { r =kvm_vcpu_enable_cap (cs,KVM_CAP_ENFORCE_PV_FEATURE_CPUID ,0 ,1 );if (r <0 ) {
@@ -1914,224 +2146,7 @@ int kvm_arch_init_vcpu(CPUState *cs) } } -for (i =0 ; i <= limit; i++) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"unsupported level value: 0x%x\n" , limit); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - -switch (i) { -case 2 : { -/* Keep reading function 2 till all the input is received */ -int times; - - c->function = i; - c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC | -KVM_CPUID_FLAG_STATE_READ_NEXT ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - times = c->eax &0xff ; - -for (j =1 ; j < times; ++j) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:2):eax & 0xf = 0x%x\n" , times); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - c->function = i; - c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - } -break ; - } -case 0x1f : -if (env->nr_dies <2 ) { - cpuid_i--; -break ; - } -/* fallthrough */ -case 4 : -case 0xb : -case 0xd : -for (j =0 ; ; j++) { -if (i ==0xd && j ==64 ) { -break ; - } - - c->function = i; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; - c->index = j; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - -if (i ==4 && c->eax ==0 ) { -break ; - } -if (i ==0xb && !(c->ecx &0xff00 )) { -break ; - } -if (i ==0x1f && !(c->ecx &0xff00 )) { -break ; - } -if (i ==0xd && c->eax ==0 ) { -continue ; - } -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - } -break ; -case 0x12 : -for (j =0 ; ; j++) { - c->function = i; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; - c->index = j; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - -if (j >1 && (c->eax &0xf ) !=1 ) { -break ; - } - -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x12,ecx:0x%x)\n" , j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - } -break ; -case 0x7 : -case 0x14 : -case 0x1d : -case 0x1e : { -uint32_t times; - - c->function = i; - c->index =0 ; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - times = c->eax; - -for (j =1 ; j <= times; ++j) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - c->function = i; - c->index = j; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - } -break ; - } -default : - c->function = i; - c->flags =0 ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); -if (!c->eax && !c->ebx && !c->ecx && !c->edx) { -/* -* KVM already returns all zeroes if a CPUID entry is missing, -* so we can omit it and avoid hitting KVM's 80-entry limit. -*/ - cpuid_i--; - } -break ; - } - } - -if (limit >=0x0a ) { -uint32_t eax, edx; - -cpu_x86_cpuid (env,0x0a ,0 , &eax, &unused, &unused, &edx); - - has_architectural_pmu_version = eax &0xff ; -if (has_architectural_pmu_version >0 ) { - num_architectural_pmu_gp_counters = (eax &0xff00 ) >>8 ; - -/* Shouldn't be more than 32, since that's the number of bits -* available in EBX to tell us _which_ counters are available. -* Play it safe. -*/ -if (num_architectural_pmu_gp_counters >MAX_GP_COUNTERS ) { - num_architectural_pmu_gp_counters =MAX_GP_COUNTERS ; - } - -if (has_architectural_pmu_version >1 ) { - num_architectural_pmu_fixed_counters = edx &0x1f ; - -if (num_architectural_pmu_fixed_counters >MAX_FIXED_COUNTERS ) { - num_architectural_pmu_fixed_counters =MAX_FIXED_COUNTERS ; - } - } - } - } - -cpu_x86_cpuid (env,0x80000000 ,0 , &limit, &unused, &unused, &unused); - -for (i =0x80000000 ; i <= limit; i++) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"unsupported xlevel value: 0x%x\n" , limit); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - -switch (i) { -case 0x8000001d : -/* Query for all AMD cache information leaves */ -for (j =0 ; ; j++) { - c->function = i; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; - c->index = j; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - -if (c->eax ==0 ) { -break ; - } -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - } -break ; -default : - c->function = i; - c->flags =0 ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); -if (!c->eax && !c->ebx && !c->ecx && !c->edx) { -/* -* KVM already returns all zeroes if a CPUID entry is missing, -* so we can omit it and avoid hitting KVM's 80-entry limit. -*/ - cpuid_i--; - } -break ; - } - } - -/* Call Centaur's CPUID instructions they are supported. */ -if (env->cpuid_xlevel2 >0 ) { -cpu_x86_cpuid (env,0xC0000000 ,0 , &limit, &unused, &unused, &unused); - -for (i =0xC0000000 ; i <= limit; i++) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"unsupported xlevel2 value: 0x%x\n" , limit); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - - c->function = i; - c->flags =0 ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - } - } - + cpuid_i =kvm_x86_arch_cpuid (env, cpuid_data.entries, cpuid_i); cpuid_data.cpuid.nent = cpuid_i; if (((env->cpuid_version >> 8)&0xF) >= 6
target/i386/kvm/kvm_i386.h +3 -0
@@ -24,6 +24,9 @@#define kvm_ioapic_in_kernel () \ (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split()) +uint32_t kvm_x86_arch_cpuid (CPUX86State *env,struct kvm_cpuid_entry2 *entries, +uint32_t cpuid_i); +#else #define kvm_pit_in_kernel () 0
Signature · 1 line
2.34.1
PATCH v4 23/66 kvm: Introduce kvm_arch_pre_create_vcpu()
In reply to Xiaoyao LiIntroduce kvm_arch_pre_create_vcpu(), to perform arch-dependent work prior to create any vcpu. This is for i386 TDX because it needs call TDX_INIT_VM before creating any vcpu.
---
Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)
--- accel/kvm/kvm-all.c | 10 ++++++++++ include/sysemu/kvm.h | 1 + 2 files changed, 11 insertions(+)
accel/kvm/kvm-all.c +10 -0
@@ -418,6 +418,11 @@ static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)return kvm_vm_ioctl (s,KVM_CREATE_VCPU , (void *)vcpu_id); } +int __attribute__ ((weak))kvm_arch_pre_create_vcpu (CPUState *cpu,Error **errp) +{ +return 0 ; +} +int kvm_init_vcpu (CPUState *cpu,Error **errp) {KVMState *s = kvm_state;
@@ -426,6 +431,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)trace_kvm_init_vcpu (cpu->cpu_index,kvm_arch_vcpu_id (cpu)); + ret =kvm_arch_pre_create_vcpu (cpu, errp); +if (ret <0 ) { + goto err; + } + ret =kvm_get_vcpu (s,kvm_arch_vcpu_id (cpu));if (ret <0 ) { error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
include/sysemu/kvm.h +1 -0
@@ -341,6 +341,7 @@ int kvm_arch_get_default_type(MachineState *ms);int kvm_arch_init (MachineState *ms,KVMState *s); +int kvm_arch_pre_create_vcpu (CPUState *cpu,Error **errp);int kvm_arch_init_vcpu (CPUState *cpu);int kvm_arch_destroy_vcpu (CPUState *cpu);
Signature · 1 line
2.34.1
PATCH v4 24/66 i386/tdx: Initialize TDX before creating TD vcpus
In reply to Xiaoyao LiInvoke KVM_TDX_INIT in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT configures global TD configurations, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs.
Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.
Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init().
---
Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
--- accel/kvm/kvm-all.c | 9 ++++++++- target/i386/kvm/kvm.c | 9 +++++++++ target/i386/kvm/tdx-stub.c | 5 +++++ target/i386/kvm/tdx.c | 41 ++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 4 ++++ 5 files changed, 67 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +8 -1
@@ -431,8 +431,15 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)trace_kvm_init_vcpu (cpu->cpu_index,kvm_arch_vcpu_id (cpu)); +/* +* tdx_pre_create_vcpu() may call cpu_x86_cpuid(). It in turn may call +* kvm_vm_ioctl(). Set cpu->kvm_state in advance to avoid NULL pointer +* dereference. +*/ + cpu->kvm_state = s; ret =kvm_arch_pre_create_vcpu (cpu, errp);if (ret <0 ) { + cpu->kvm_state =NULL ; goto err; }
@@ -440,11 +447,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)if (ret <0 ) {error_setg_errno (errp, -ret,"kvm_init_vcpu: kvm_get_vcpu failed (%lu)" ,kvm_arch_vcpu_id (cpu)); + cpu->kvm_state =NULL ; goto err; } cpu->kvm_fd = ret; - cpu->kvm_state = s; cpu->vcpu_dirty = true; cpu->dirty_pages =0 ; cpu->throttle_us_per_full =0 ;
target/i386/kvm/kvm.c +9 -0
@@ -2272,6 +2272,15 @@ int kvm_arch_init_vcpu(CPUState *cs)return r; } +int kvm_arch_pre_create_vcpu (CPUState *cpu,Error **errp) +{ +if (is_tdx_vm ()) { +return tdx_pre_create_vcpu (cpu, errp); + } + +return 0 ; +} +int kvm_arch_destroy_vcpu (CPUState *cs) {X86CPU *cpu =X86_CPU (cs);
target/i386/kvm/tdx-stub.c +5 -0
@@ -6,3 +6,8 @@ int tdx_kvm_init(MachineState *ms, Error **errp) {return -EINVAL ; } + +int tdx_pre_create_vcpu (CPUState *cpu,Error **errp) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +41 -0
53 lines not on this page · raw message
target/i386/kvm/tdx.h +4 -0
@@ -17,6 +17,9 @@ typedef struct TdxGuestClass {typedef struct TdxGuest {ConfidentialGuestSupport parent_obj; +QemuMutex lock; + +bool initialized;uint64_t attributes;/* TD attributes */ }TdxGuest ;
@@ -29,5 +32,6 @@ bool is_tdx_vm(void);int tdx_kvm_init (MachineState *ms,Error **errp);void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg,uint32_t *ret); +int tdx_pre_create_vcpu (CPUState *cpu,Error **errp);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 25/66 i386/tdx: Add property sept-ve-disable for tdx-guest object
In reply to Xiaoyao LiBit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
---
Changes in v4:
- collect Acked-by from Markus
Changes in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)
--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)
qapi/qom.json +6 -1
@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. # +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } } ## # @ThreadContextProperties:
target/i386/kvm/tdx.c +24 -0
@@ -32,6 +32,8 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_ATTRIBUTES_MAX_BITS 64static FeatureMask tdx_attrs_ctrl_fields[TDX_ATTRIBUTES_MAX_BITS ] = {
@@ -513,6 +515,24 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return 0 ; } +static bool tdx_guest_get_sept_ve_disable (Object *obj,Error **errp) +{ +TdxGuest *tdx =TDX_GUEST (obj); + +return !!(tdx->attributes &TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ); +} + +static void tdx_guest_set_sept_ve_disable (Object *obj,bool value,Error **errp) +{ +TdxGuest *tdx =TDX_GUEST (obj); + +if (value) { + tdx->attributes |=TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ; + }else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ; + } +} +/* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,
@@ -528,6 +548,10 @@ static void tdx_guest_init(Object *obj)qemu_mutex_init (&tdx->lock); tdx->attributes =0 ; + +object_property_add_bool (obj,"sept-ve-disable" , + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); }static void tdx_guest_finalize (Object *obj)
Signature · 1 line
2.34.1
PATCH v4 26/66 i386/tdx: Make sept_ve_disable set by default
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
For TDX KVM use case, Linux guest is the most major one. It requires sept_ve_disable set. Make it default for the main use case. For other use case, it can be enabled/disabled via qemu command line.
--- target/i386/kvm/tdx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/i386/kvm/tdx.c +1 -1
@@ -547,7 +547,7 @@ static void tdx_guest_init(Object *obj)qemu_mutex_init (&tdx->lock); - tdx->attributes =0 ; + tdx->attributes =TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ; object_property_add_bool(obj, "sept-ve-disable", tdx_guest_get_sept_ve_disable,
Signature · 1 line
2.34.1
PATCH v4 27/66 i386/tdx: Wire CPU features up with attributes of TD guest
In reply to Xiaoyao LiFor QEMU VMs, PKS is configured via CPUID_7_0_ECX_PKS and PMU is configured by x86cpu->enable_pmu. Reuse the existing configuration interface for TDX VMs.
--- target/i386/kvm/tdx.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
target/i386/kvm/tdx.c +13 -0
@@ -33,6 +33,8 @@ (1U << KVM_FEATURE_MSI_EXT_DEST_ID))#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTES_PERFMON BIT_ULL(63)#define TDX_ATTRIBUTES_MAX_BITS 64
@@ -476,6 +478,15 @@ int tdx_kvm_init(MachineState *ms, Error **errp)return 0 ; } +static void setup_td_guest_attributes (X86CPU *x86cpu) +{ +CPUX86State *env = &x86cpu->env; + + tdx_guest->attributes |= (env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_PKS ) ? +TDX_TD_ATTRIBUTES_PKS :0 ; + tdx_guest->attributes |= x86cpu->enable_pmu ?TDX_TD_ATTRIBUTES_PERFMON :0 ; +} +int tdx_pre_create_vcpu (CPUState *cpu,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ());
@@ -498,6 +509,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return r; } +setup_td_guest_attributes (x86cpu); + init_vm->cpuid.nent =kvm_x86_arch_cpuid (env, init_vm->cpuid.entries,0 ); init_vm->attributes = tdx_guest->attributes;
Signature · 1 line
2.34.1
PATCH v4 28/66 i386/tdx: Validate TD attributes
In reply to Xiaoyao LiValidate TD attributes with tdx_caps that fixed-0 bits must be zero and fixed-1 bits must be set.
Besides, sanity check the attribute bits that have not been supported by QEMU yet. e.g., debug bit, it will be allowed in the future when debug TD support lands in QEMU.
---
Changes in v3:
- using error_setg() for error report; (Daniel)
--- target/i386/kvm/tdx.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
target/i386/kvm/tdx.c +27 -2
@@ -32,6 +32,7 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_TD_ATTRIBUTES_DEBUG BIT_ULL(0)#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)#define TDX_TD_ATTRIBUTES_PERFMON BIT_ULL(63)
@@ -478,13 +479,34 @@ int tdx_kvm_init(MachineState *ms, Error **errp)return 0 ; } -static void setup_td_guest_attributes (X86CPU *x86cpu) +static int tdx_validate_attributes (TdxGuest *tdx,Error **errp) +{ +if (((tdx->attributes & tdx_caps->attrs_fixed0) | tdx_caps->attrs_fixed1) != + tdx->attributes) { +error_setg (errp,"Invalid attributes 0x%lx for TDX VM " +"(fixed0 0x%llx, fixed1 0x%llx)" , + tdx->attributes, tdx_caps->attrs_fixed0, + tdx_caps->attrs_fixed1); +return -1 ; + } + +if (tdx->attributes &TDX_TD_ATTRIBUTES_DEBUG ) { +error_setg (errp,"Current QEMU doesn't support attributes.debug[bit 0] for TDX VM" ); +return -1 ; + } + +return 0 ; +} + +static int setup_td_guest_attributes (X86CPU *x86cpu,Error **errp) {CPUX86State *env = &x86cpu->env; tdx_guest->attributes |= (env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_PKS ) ?TDX_TD_ATTRIBUTES_PKS :0 ; tdx_guest->attributes |= x86cpu->enable_pmu ?TDX_TD_ATTRIBUTES_PERFMON :0 ; + +return tdx_validate_attributes (tdx_guest, errp); }int tdx_pre_create_vcpu (CPUState *cpu,Error **errp)
@@ -509,7 +531,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return r; } -setup_td_guest_attributes (x86cpu); + r =setup_td_guest_attributes (x86cpu, errp); +if (r) { +return r; + } init_vm->cpuid.nent =kvm_x86_arch_cpuid (env, init_vm->cpuid.entries,0 );
Signature · 1 line
2.34.1
PATCH v4 29/66 i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/
Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
---
Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in _setter(); (Daniel)
Changes in v3:
- use base64 encoding instread of hex-string;
--- qapi/qom.json | 14 ++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 103 insertions(+), 1 deletion(-)
qapi/qom.json +13 -1
@@ -905,10 +905,22 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration. base64 encoded SHA384 digest. +# +# @mrowner: ID for the guest TD’s owner. base64 encoded SHA384 digest. +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS. +# base64 encoded SHA384 digest. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:
target/i386/kvm/tdx.c +87 -0
117 lines not on this page · raw message
target/i386/kvm/tdx.h +3 -0
@@ -21,6 +21,9 @@ typedef struct TdxGuest {bool initialized;uint64_t attributes;/* TD attributes */ +char *mrconfigid;/* base64 encoded sha348 digest */ +char *mrowner;/* base64 encoded sha348 digest */ +char *mrownerconfig;/* base64 encoded sha348 digest */ } TdxGuest;#ifdef CONFIG_TDX
Signature · 1 line
2.34.1
Re: [PATCH v4 29/66] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 38 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in_setter(); (Daniel)Changes in v3:
- use base64 encoding instread of hex-string;--- qapi/qom.json | 14 ++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 103 insertions(+), 1 deletion(-)qapi/qom.json +0
-0
qapi/qom.json line 909
Quoted 5 lines
qapi/qom.json +2
-0@@ -905,10 +905,22 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration. base64 encoded SHA384 digest.
"base64 encoded SHA384" is not a sentence.
Double-checking: the data being hashed here is the "non-owner-defined configuration of the guest TD", and the resulting hash is the "ID"?
qapi/qom.json line 911
Quoted 2 lines
qapi/qom.json +2
-0+# +# @mrowner: ID for the guest TD’s owner. base64 encoded SHA384 digest.
Likewise.
qapi/qom.json line 915
Quoted 4 lines
qapi/qom.json +4
-0+# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS. +# base64 encoded SHA384 digest.
Likewise.
qapi/qom.json line 923
Quoted 9 lines
qapi/qom.json +5
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } }
The new members are optional, but their description in the doc comment doesn't explain behavior when present vs. behavior when absent.
Quoted 3 lines
qapi/qom.json +0
-0## # @ThreadContextProperties:
[...]
Re: [PATCH v4 29/66] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 48 lines · Markus Armbruster
Quoted 38 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in_setter(); (Daniel)Changes in v3:
- use base64 encoding instread of hex-string;--- qapi/qom.json | 14 ++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 103 insertions(+), 1 deletion(-)qapi/qom.json +0
-0qapi/qom.json line 909
Quoted 5 lines
qapi/qom.json +2
-0@@ -905,10 +905,22 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration. base64 encoded SHA384 digest."base64 encoded SHA384" is not a sentence.
Quoted 2 lines
Double-checking: the data being hashed here is the "non-owner-defined configuration of the guest TD", and the resulting hash is the "ID"?
yes. The "ID" here means the resulting hash.
The reason to use "ID" here because in the TDX spec, it's description is
Software-defined ID for non-owner-defined configuration of the guest TD - e.g., run-time or OS configuration.
If ID is confusing, how about
SHA384 hash of non-owner-defined configuration of the guest TD, e.g., run-time of OS configuration. It's base64 encoded.
Quoted 32 lines
qapi/qom.json line 911
Quoted 2 lines
qapi/qom.json +2
-0+# +# @mrowner: ID for the guest TD’s owner. base64 encoded SHA384 digest.Likewise.
qapi/qom.json line 915
Quoted 4 lines
qapi/qom.json +4
-0+# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS. +# base64 encoded SHA384 digest.Likewise.
qapi/qom.json line 923
Quoted 9 lines
qapi/qom.json +5
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } }The new members are optional, but their description in the doc comment doesn't explain behavior when present vs. behavior when absent.
Quoted 3 lines
qapi/qom.json +0
-0## # @ThreadContextProperties:[...]
Re: [PATCH v4 29/66] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 56 lines · Xiaoyao Li
Quoted 48 lines · Markus Armbruster
Quoted 38 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in_setter(); (Daniel)Changes in v3:
- use base64 encoding instread of hex-string;--- qapi/qom.json | 14 ++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 103 insertions(+), 1 deletion(-)qapi/qom.json +0
-0qapi/qom.json line 909
Quoted 5 lines
qapi/qom.json +2
-0@@ -905,10 +905,22 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration. base64 encoded SHA384 digest."base64 encoded SHA384" is not a sentence.
Quoted 2 lines
Double-checking: the data being hashed here is the "non-owner-defined configuration of the guest TD", and the resulting hash is the "ID"?
yes. The "ID" here means the resulting hash.
The reason to use "ID" here because in the TDX spec, it's description is
Quoted 7 lines
Software-defined ID for non-owner-defined configuration of the guest TD - e.g., run-time or OS configuration.
If ID is confusing, how about
SHA384 hash of non-owner-defined configuration of the guest TD, e.g., run-time of OS configuration. It's base64 encoded.
I guess staying close to the TDX spec makes sense.
We still need to mention the base64 encoding.
What about something like
ID for non-owner-defined configuration of the guest TD, e.g., run-time or OS configuration (base64 encoded SHA384 digest)
or, if we decide that the fact it's SHA384 digest is irrelevant for QMP
ID for non-owner-defined configuration of the guest TD, e.g., run-time or OS configuration (base64 encoded)
Quoted 32 lines
Quoted 32 lines
qapi/qom.json line 911
Quoted 2 lines
qapi/qom.json +2
-0+# +# @mrowner: ID for the guest TD’s owner. base64 encoded SHA384 digest.Likewise.
qapi/qom.json line 915
Quoted 4 lines
qapi/qom.json +4
-0+# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS. +# base64 encoded SHA384 digest.Likewise.
qapi/qom.json line 923
Quoted 9 lines
qapi/qom.json +5
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } }The new members are optional, but their description in the doc comment doesn't explain behavior when present vs. behavior when absent.
Quoted 3 lines
qapi/qom.json +0
-0## # @ThreadContextProperties:[...]
PATCH v4 30/66 i386/tdx: Implement user specified tsc frequency
In reply to Xiaoyao LiReuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.
Besides, sanity check the tsc frequency to be in the legal range and legal granularity (required by TDX module).
---
Changes in v3:
- use @errp to report error info; (Daniel)
Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM side drop the @tsc_khz field in struct kvm_tdx_init_vm
--- target/i386/kvm/kvm.c | 9 +++++++++ target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++ 2 files changed, 34 insertions(+)
target/i386/kvm/kvm.c +9 -0
@@ -794,6 +794,15 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)int r, cur_freq;bool set_ioctl = false; +/* +* TSC of TD vcpu is immutable, it cannot be set/changed via vcpu scope +* VM_SET_TSC_KHZ, but only be initialized via VM scope VM_SET_TSC_KHZ +* before ioctl KVM_TDX_INIT_VM in tdx_pre_create_vcpu() +*/ +if (is_tdx_vm ()) { +return 0 ; + } +if (!env->tsc_khz) {return 0 ; }
target/i386/kvm/tdx.c +25 -0
@@ -33,6 +33,9 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) +#define TDX_TD_ATTRIBUTES_DEBUG BIT_ULL(0)#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)
@@ -565,6 +568,28 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return r; } +if (env->tsc_khz && (env->tsc_khz <TDX_MIN_TSC_FREQUENCY_KHZ || + env->tsc_khz >TDX_MAX_TSC_FREQUENCY_KHZ )) { +error_setg (errp,"Invalid TSC %ld KHz, must specify cpu_frequency between [%d, %d] kHz" , + env->tsc_khz,TDX_MIN_TSC_FREQUENCY_KHZ , +TDX_MAX_TSC_FREQUENCY_KHZ ); +return -EINVAL ; + } + +if (env->tsc_khz % (25 *1000 )) { +error_setg (errp,"Invalid TSC %ld KHz, it must be multiple of 25MHz" , + env->tsc_khz); +return -EINVAL ; + } + +/* it's safe even env->tsc_khz is 0. KVM uses host's tsc_khz in this case */ + r =kvm_vm_ioctl (kvm_state,KVM_SET_TSC_KHZ , env->tsc_khz); +if (r <0 ) { +error_setg_errno (errp, -r,"Unable to set TSC frequency to %" PRId64" kHz" , + env->tsc_khz); +return r; + } + r =setup_td_guest_attributes (x86cpu, errp);if (r) {return r;
Signature · 1 line
2.34.1
PATCH v4 31/66 i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM
In reply to Xiaoyao LiTDX only supports readonly for shared memory but not for private memory.
In the view of QEMU, it has no idea whether a memslot is used as shared memory of private. Thus just mark kvm_readonly_mem_enabled to false to TDX VM for simplicity.
--- target/i386/kvm/tdx.c | 9 +++++++++ 1 file changed, 9 insertions(+)
target/i386/kvm/tdx.c +9 -0
@@ -479,6 +479,15 @@ int tdx_kvm_init(MachineState *ms, Error **errp)update_tdx_cpuid_lookup_by_tdx_caps (); +/* +* Set kvm_readonly_mem_allowed to false, because TDX only supports readonly +* memory for shared memory but not for private memory. Besides, whether a +* memslot is private or shared is not determined by QEMU. +* +* Thus, just mark readonly memory not supported for simplicity. +*/ + kvm_readonly_mem_allowed = false; + tdx_guest = tdx;return 0 ; }
Signature · 1 line
2.34.1
PATCH v4 32/66 kvm/memory: Introduce the infrastructure to set the default shared/private value
In reply to Xiaoyao LiIntroduce new flag RAM_DEFAULT_PRIVATE for RAMBlock. It's used to indicate the default attribute, private or not.
Set the RAM range to private explicitly when it's default private.
Originated-from: Isaku Yamahata <[email protected]>
--- accel/kvm/kvm-all.c | 10 ++++++++++ include/exec/memory.h | 6 ++++++ system/memory.c | 13 +++++++++++++ 3 files changed, 29 insertions(+)
accel/kvm/kvm-all.c +10 -0
@@ -1458,6 +1458,16 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, strerror(-err));abort (); } + +if (memory_region_is_default_private (mr)) { + err =kvm_set_memory_attributes_private (start_addr, slot_size); +if (err) { +error_report ("%s: failed to set memory attribute private: %s\n" , + __func__,strerror (-err)); +exit (1 ); + } + } + start_addr += slot_size; ram_start_offset += slot_size; ram += slot_size;
include/exec/memory.h +6 -0
@@ -246,6 +246,9 @@ typedef struct IOMMUTLBEvent {/* RAM can be private that has kvm guest memfd backend */ #define RAM_GUEST_MEMFD (1 << 12) +/* RAM is default private */ +#define RAM_DEFAULT_PRIVATE (1 << 13) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
@@ -1736,6 +1739,9 @@ bool memory_region_is_protected(MemoryRegion *mr);*/ bool memory_region_has_guest_memfd (MemoryRegion *mr); +void memory_region_set_default_private (MemoryRegion *mr); +bool memory_region_is_default_private (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu *
system/memory.c +13 -0
@@ -1855,6 +1855,19 @@ bool memory_region_has_guest_memfd(MemoryRegion *mr)return mr->ram_block && mr->ram_block->guest_memfd >=0 ; } +bool memory_region_is_default_private (MemoryRegion *mr) +{ +return memory_region_has_guest_memfd (mr) && + (mr->ram_block->flags &RAM_DEFAULT_PRIVATE ); +} + +void memory_region_set_default_private (MemoryRegion *mr) +{ +if (memory_region_has_guest_memfd (mr)) { + mr->ram_block->flags |=RAM_DEFAULT_PRIVATE ; + } +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
Signature · 1 line
2.34.1
PATCH v4 33/66 i386/tdx: Make memory type private by default
In reply to Xiaoyao LiBy default (due to the recent UPM change), restricted memory attribute is shared. Convert the memory region from shared to private at the memory slot creation time.
add kvm region registering function to check the flag
and convert the region, and add memory listener to TDX guest code to set the flag to the possible memory region.
Without this patch
- Secure-EPT violation on private area
- KVM_MEMORY_FAULT EXIT (kvm -> qemu)
- qemu converts the 4K page from shared to private
- Resume VCPU execution
- Secure-EPT violation again
- KVM resolves EPT Violation
This also prevents huge page because page conversion is done at 4K granularity. Although it's possible to merge 4K private mapping into 2M large page, it slows guest boot.
With this patch
- After memory slot creation, convert the region from private to shared
- Secure-EPT violation on private area.
- KVM resolves EPT Violation
Originated-from: Isaku Yamahata <[email protected]>
--- include/exec/memory.h | 1 + target/i386/kvm/tdx.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+)
include/exec/memory.h +1 -0
@@ -850,6 +850,7 @@ struct IOMMUMemoryRegion {#define MEMORY_LISTENER_PRIORITY_MIN 0#define MEMORY_LISTENER_PRIORITY_ACCEL 10#define MEMORY_LISTENER_PRIORITY_DEV_BACKEND 10 +#define MEMORY_LISTENER_PRIORITY_ACCEL_HIGH 20/** * struct MemoryListener: callbacks structure for updates to the physical memory map
target/i386/kvm/tdx.c +20 -0
@@ -19,6 +19,7 @@#include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" #include "sysemu/sysemu.h" +#include "exec/address-spaces.h" #include "hw/i386/x86.h" #include "kvm_i386.h"
@@ -621,6 +622,19 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return 0 ; } +static void tdx_guest_region_add (MemoryListener *listener, +MemoryRegionSection *section) +{ +memory_region_set_default_private (section->mr); +} + +static MemoryListener tdx_memory_listener = { + .name =TYPE_TDX_GUEST , + .region_add = tdx_guest_region_add, +/* Higher than KVM memory listener = 10. */ + .priority =MEMORY_LISTENER_PRIORITY_ACCEL_HIGH , +}; +static bool tdx_guest_get_sept_ve_disable (Object *obj,Error **errp) {TdxGuest *tdx =TDX_GUEST (obj);
@@ -695,6 +709,12 @@ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,static void tdx_guest_init (Object *obj) {TdxGuest *tdx =TDX_GUEST (obj); +static bool memory_listener_registered = false; + +if (!memory_listener_registered) { +memory_listener_register (&tdx_memory_listener, &address_space_memory); + memory_listener_registered = true; + }qemu_mutex_init (&tdx->lock);
Signature · 1 line
2.34.1
Re: [PATCH v4 33/66] i386/tdx: Make memory type private by default
Quoted 56 lines · Xiaoyao Li
By default (due to the recent UPM change), restricted memory attribute is shared. Convert the memory region from shared to private at the memory slot creation time.
add kvm region registering function to check the flag
and convert the region, and add memory listener to TDX guest code to set the flag to the possible memory region.Without this patch
- Secure-EPT violation on private area
-KVM_MEMORY_FAULTEXIT (kvm -> qemu)
- qemu converts the 4K page from shared to private
- Resume VCPU execution
- Secure-EPT violation again
- KVM resolves EPT Violation
This also prevents huge page because page conversion is done at 4K granularity. Although it's possible to merge 4K private mapping into 2M large page, it slows guest boot.With this patch
- After memory slot creation, convert the region from private to shared
- Secure-EPT violation on private area.
- KVM resolves EPT ViolationOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>--- include/exec/memory.h | 1 + target/i386/kvm/tdx.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+)include/exec/memory.h +1
-0@@ -850,6 +850,7 @@ struct IOMMUMemoryRegion {#define MEMORY_LISTENER_PRIORITY_MIN 0#define MEMORY_LISTENER_PRIORITY_ACCEL 10#define MEMORY_LISTENER_PRIORITY_DEV_BACKEND 10 +#define MEMORY_LISTENER_PRIORITY_ACCEL_HIGH 20/** * struct MemoryListener: callbacks structure for updates to the physical memory map target/i386/kvm/tdx.c +1
-0@@ -19,6 +19,7 @@#include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" #include "sysemu/sysemu.h" +#include "exec/address-spaces.h" #include "hw/i386/x86.h" #include "kvm_i386.h"
target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 629
Quoted 8 lines
target/i386/kvm/tdx.c +5
-0@@ -621,6 +622,19 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return 0 ; } +static void tdx_guest_region_add (MemoryListener *listener, +MemoryRegionSection *section) +{ +memory_region_set_default_private (section->mr); +}
That looks fishy. Why is TDX to decide what happens to other memory regions it doesn't own?
We should define that behavior when creating these memory region, and TDX could sanity check that they have been setup properly.
Let me ask differently: For which memory region where we have RAM_GUEST_MEMFD set would we *not* want to set private as default right from the start?
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v4 33/66] i386/tdx: Make memory type private by default
Quoted 65 lines · David Hildenbrand
Quoted 54 lines · Xiaoyao Li
By default (due to the recent UPM change), restricted memory attribute is shared. Convert the memory region from shared to private at the memory slot creation time.
add kvm region registering function to check the flag
and convert the region, and add memory listener to TDX guest code to set the flag to the possible memory region.Without this patch
- Secure-EPT violation on private area
-KVM_MEMORY_FAULTEXIT (kvm -> qemu)
- qemu converts the 4K page from shared to private
- Resume VCPU execution
- Secure-EPT violation again
- KVM resolves EPT Violation
This also prevents huge page because page conversion is done at 4K granularity. Although it's possible to merge 4K private mapping into 2M large page, it slows guest boot.With this patch
- After memory slot creation, convert the region from private to shared
- Secure-EPT violation on private area.
- KVM resolves EPT ViolationOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>---
include/exec/memory.h| 1 +
target/i386/kvm/tdx.c| 20 ++++++++++++++++++++2 files changed, 21 insertions(+)include/exec/memory.h +1
-0@@ -850,6 +850,7 @@ struct IOMMUMemoryRegion { #define MEMORY_LISTENER_PRIORITY_MIN 0 #define MEMORY_LISTENER_PRIORITY_ACCEL 10 #define MEMORY_LISTENER_PRIORITY_DEV_BACKEND 10 +#define MEMORY_LISTENER_PRIORITY_ACCEL_HIGH 20 /*** struct MemoryListener: callbacks structure for updates to the physical memory map target/i386/kvm/tdx.c +1
-0@@ -19,6 +19,7 @@ #include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" #include "sysemu/sysemu.h" +#include "exec/address-spaces.h" #include "hw/i386/x86.h" #include "kvm_i386.h"target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 628
Quoted 7 lines
target/i386/kvm/tdx.c +5
-0@@ -621,6 +622,19 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) return 0; } +static void tdx_guest_region_add (MemoryListener *listener, + MemoryRegionSection *section) +{ + memory_region_set_default_private(section->mr); +}That looks fishy. Why is TDX to decide what happens to other memory regions it doesn't own?
Quoted 3 lines
We should define that behavior when creating these memory region, and TDX could sanity check that they have been setup properly.
Let me ask differently: For which memory region where we have
RAM_GUEST_MEMFDset would we *not* want to set private as default right from the start?
All memory regions have RAM_GUEST_MEMFD set will benefit from being private as default, for TDX guest.
I will update the implementation to set RAM_DEFAULT_PRIVATE flag when guest_memfd is created successfully, like
system/physmem.c +2 -0
@@ -1850,6 +1850,8 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)qemu_mutex_unlock_ramlist ();return ; } + + new_block->flags |=RAM_DEFAULT_PRIVATE ; }
then this patch can be dropped, and the calling of memory_region_set_default_private(mr) of Patch 45 can be dropped too.
I think this is what you suggested, right?
Re: [PATCH v4 33/66] i386/tdx: Make memory type private by default
Quoted 97 lines · Xiaoyao Li
Quoted 67 lines · David Hildenbrand
Quoted 55 lines · Xiaoyao Li
By default (due to the recent UPM change), restricted memory attribute is shared. Convert the memory region from shared to private at the memory slot creation time.
add kvm region registering function to check the flag
and convert the region, and add memory listener to TDX guest code to set the flag to the possible memory region.Without this patch
- Secure-EPT violation on private area
-KVM_MEMORY_FAULTEXIT (kvm -> qemu)
- qemu converts the 4K page from shared to private
- Resume VCPU execution
- Secure-EPT violation again
- KVM resolves EPT Violation
This also prevents huge page because page conversion is done at 4K granularity. Although it's possible to merge 4K private mapping into 2M large page, it slows guest boot.With this patch
- After memory slot creation, convert the region from private to shared
- Secure-EPT violation on private area.
- KVM resolves EPT ViolationOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>--- include/exec/memory.h | 1 + target/i386/kvm/tdx.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+)include/exec/memory.h +1
-0@@ -850,6 +850,7 @@ struct IOMMUMemoryRegion { #define MEMORY_LISTENER_PRIORITY_MIN 0 #define MEMORY_LISTENER_PRIORITY_ACCEL 10 #define MEMORY_LISTENER_PRIORITY_DEV_BACKEND 10 +#define MEMORY_LISTENER_PRIORITY_ACCEL_HIGH 20 /*** struct MemoryListener: callbacks structure for updates to the physical memory map
target/i386/kvm/tdx.c +1
-0@@ -19,6 +19,7 @@ #include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" #include "sysemu/sysemu.h" +#include "exec/address-spaces.h" #include "hw/i386/x86.h" #include "kvm_i386.h"target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 628
Quoted 7 lines
target/i386/kvm/tdx.c +5
-0@@ -621,6 +622,19 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) return 0; } +static void tdx_guest_region_add (MemoryListener *listener, + MemoryRegionSection *section) +{ + memory_region_set_default_private(section->mr); +}That looks fishy. Why is TDX to decide what happens to other memory regions it doesn't own?
Quoted 6 lines
We should define that behavior when creating these memory region, and TDX could sanity check that they have been setup properly.
Let me ask differently: For which memory region where we have
RAM_GUEST_MEMFDset would we *not* want to set private as default right from the start?All memory regions have
RAM_GUEST_MEMFDset will benefit from being private as default, for TDX guest.I will update the implementation to set
RAM_DEFAULT_PRIVATEflag whenguest_memfdis created successfully, likesystem/physmem.c +2
-0@@ -1850,6 +1850,8 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)qemu_mutex_unlock_ramlist ();return ; } + + new_block->flags |=RAM_DEFAULT_PRIVATE ; }then this patch can be dropped, and the calling of
memory_region_set_default_private(mr)of Patch 45 can be dropped too.
Quoted 1 line
I think this is what you suggested, right?
Yes, if that works, great!
Signature · 3 lines
Cheers, David / dhildenb
PATCH v4 34/66 kvm/tdx: Don't complain when converting vMMIO region to shared
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Because vMMIO region needs to be shared region, guest TD may explicitly convert such region from private to shared. Don't complain such conversion.
--- accel/kvm/kvm-all.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +16 -3
@@ -2975,9 +2975,22 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private) ret =ram_block_discard_guest_memfd_range (rb, offset, size); } }else { -error_report ("Convert non guest_memfd backed memory region " -"(0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , - start, size, to_private ?"private" :"shared" ); +/* +* Because vMMIO region must be shared, guest TD may convert vMMIO +* region to shared explicitly. Don't complain such case. See +* memory_region_type() for checking if the region is MMIO region. +*/ +if (!to_private && + !memory_region_is_ram (mr) && + !memory_region_is_ram_device (mr) && + !memory_region_is_rom (mr) && + !memory_region_is_romd (mr)) { + ret =0 ; + }else { +error_report ("Convert non guest_memfd backed memory region " +"(0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } }memory_region_unref (section.mr);
Signature · 1 line
2.34.1
PATCH v4 35/66 kvm/tdx: Ignore memory conversion to shared of unassigned region
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TDX requires vMMIO region to be shared. For KVM, MMIO region is the region which kvm memslot isn't assigned to (except in-kernel emulation). qemu has the memory region for vMMIO at each device level.
While OVMF issues MapGPA(to-shared) conservatively on 32bit PCI MMIO region, qemu doesn't find corresponding vMMIO region because it's before PCI device allocation and memory_region_find() finds the device region, not PCI bus region. It's safe to ignore MapGPA(to-shared) because when guest accesses those region they use GPA with shared bit set for vMMIO. Ignore memory conversion request of non-assigned region to shared and return success. Otherwise OVMF is confused and panics there.
--- accel/kvm/kvm-all.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
accel/kvm/kvm-all.c +12 -0
@@ -2943,6 +2943,18 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private) section =memory_region_find (get_system_memory (), start, size); mr = section.mr;if (!mr) { +/* +* Ignore converting non-assigned region to shared. +* +* TDX requires vMMIO region to be shared to inject #VE to guest. +* OVMF issues conservatively MapGPA(shared) on 32bit PCI MMIO region, +* and vIO-APIC 0xFEC00000 4K page. +* OVMF assigns 32bit PCI MMIO region to +* [top of low memory: typically 2GB=0xC000000, 0xFC00000) +*/ +if (!to_private) { +return 0 ; + }return -1 ; }
Signature · 1 line
2.34.1
PATCH v4 36/66 i386/tdvf: Introduce function to parse TDVF metadata
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.
A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function tdvf_parse_metadata() to parse the metadata from TDVF image and store the info of each TDVF section.
TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.
Select X86_FW_OVMF when TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.
---
Changes in v1:
- rename tdvf_parse_section_entry() to tdvf_parse_and_check_section_entry()
Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
--- hw/i386/Kconfig | 1 + hw/i386/meson.build | 1 + hw/i386/tdvf.c | 199 +++++++++++++++++++++++++++++++++++++++++ include/hw/i386/tdvf.h | 51 +++++++++++ 4 files changed, 252 insertions(+) create mode 100644 hw/i386/tdvf.c create mode 100644 include/hw/i386/tdvf.h
hw/i386/Kconfig +1 -0
@@ -12,6 +12,7 @@ config SGX config TDX bool + select X86_FW_OVMF depends on KVM config PC
hw/i386/meson.build +1 -0
@@ -27,6 +27,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'port92.c')) i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'), if_false: files('pc_sysfw_ovmf-stubs.c')) +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c')) subdir('kvm') subdir('xen')
hw/i386/tdvf.c new file +199 -0
199 lines not on this page · raw message
include/hw/i386/tdvf.h new file +51 -0
51 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 37/66 i386/tdx: Parse TDVF metadata for TDX VM
In reply to Xiaoyao LiTDX cannot support pflash device since it doesn't support read-only memslot and doesn't support emulation. Load TDVF(OVMF) with -bios option for TDs.
When boot a TD, besides loading TDVF to the address below 4G, it needs parse TDVF metadata.
--- hw/i386/pc_sysfw.c | 7 +++++++ hw/i386/x86.c | 3 ++- target/i386/kvm/tdx-stub.c | 5 +++++ target/i386/kvm/tdx.c | 5 +++++ target/i386/kvm/tdx.h | 4 ++++ 5 files changed, 23 insertions(+), 1 deletion(-)
hw/i386/pc_sysfw.c +7 -0
@@ -37,6 +37,7 @@#include "hw/block/flash.h" #include "sysemu/kvm.h" #include "sev.h" +#include "kvm/tdx.h" #define FLASH_SECTOR_SIZE 4096
@@ -265,5 +266,11 @@ void x86_firmware_configure(void *ptr, int size) }sev_encrypt_flash (ptr, size, &error_fatal); + }else if (is_tdx_vm ()) { + ret =tdx_parse_tdvf (ptr, size); +if (ret) { +error_report ("failed to parse TDVF for TDX VM" ); +exit (1 ); + } } }
hw/i386/x86.c +2 -1
@@ -47,6 +47,7 @@#include "hw/intc/i8259.h" #include "hw/rtc/mc146818rtc.h" #include "target/i386/sev.h" +#include "kvm/tdx.h" #include "hw/acpi/cpu_hotplug.h" #include "hw/irq.h"
@@ -1156,7 +1157,7 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware, } bios =g_malloc (sizeof (*bios));memory_region_init_ram (bios,NULL ,"pc.bios" , bios_size, &error_fatal); -if (sev_enabled ()) { +if (sev_enabled () ||is_tdx_vm ()) {/* * The concept of a "reset" simply doesn't exist for * confidential computing guests, we have to destroy and
target/i386/kvm/tdx-stub.c +5 -0
@@ -11,3 +11,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) {return -EINVAL ; } + +int tdx_parse_tdvf (void *flash_ptr,int size) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +5 -0
@@ -622,6 +622,11 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return 0 ; } +int tdx_parse_tdvf (void *flash_ptr,int size) +{ +return tdvf_parse_metadata (&tdx_guest->tdvf, flash_ptr, size); +} +static void tdx_guest_region_add (MemoryListener *listener,MemoryRegionSection *section) {
target/i386/kvm/tdx.h +4 -0
@@ -6,6 +6,7 @@#endif #include "exec/confidential-guest-support.h" +#include "hw/i386/tdvf.h" #define TYPE_TDX_GUEST "tdx-guest"#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -24,6 +25,8 @@ typedef struct TdxGuest {char *mrconfigid;/* base64 encoded sha348 digest */ char *mrowner;/* base64 encoded sha348 digest */ char *mrownerconfig;/* base64 encoded sha348 digest */ + +TdxFirmware tdvf; } TdxGuest;#ifdef CONFIG_TDX
@@ -36,5 +39,6 @@ int tdx_kvm_init(MachineState *ms, Error **errp);void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg,uint32_t *ret);int tdx_pre_create_vcpu (CPUState *cpu,Error **errp); +int tdx_parse_tdvf (void *flash_ptr,int size);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 38/66 i386/tdx: Skip BIOS shadowing setup
In reply to Xiaoyao LiTDX doesn't support map different GPAs to same private memory. Thus, aliasing top 128KB of BIOS as isa-bios is not supported.
On the other hand, TDX guest cannot go to real mode, it can work fine without isa-bios.
---
Changes in v1:
- update commit message and comment to clarify
--- hw/i386/x86.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
hw/i386/x86.c +14 -11
@@ -1179,17 +1179,20 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware, }g_free (filename); -/* map the last 128KB of the BIOS in ISA space */ - isa_bios_size =MIN (bios_size,128 * KiB); - isa_bios =g_malloc (sizeof (*isa_bios)); -memory_region_init_alias (isa_bios,NULL ,"isa-bios" , bios, - bios_size - isa_bios_size, isa_bios_size); -memory_region_add_subregion_overlap (rom_memory, -0x100000 - isa_bios_size, - isa_bios, -1 ); -if (!isapc_ram_fw) { -memory_region_set_readonly (isa_bios, true); +/* For TDX, alias different GPAs to same private memory is not supported */ +if (!is_tdx_vm ()) { +/* map the last 128KB of the BIOS in ISA space */ + isa_bios_size =MIN (bios_size,128 * KiB); + isa_bios =g_malloc (sizeof (*isa_bios)); +memory_region_init_alias (isa_bios,NULL ,"isa-bios" , bios, + bios_size - isa_bios_size, isa_bios_size); +memory_region_add_subregion_overlap (rom_memory, +0x100000 - isa_bios_size, + isa_bios, +1 ); +if (!isapc_ram_fw) { +memory_region_set_readonly (isa_bios, true); + } }/* map all the bios at the top of memory */
Signature · 1 line
2.34.1
PATCH v4 39/66 i386/tdx: Don't initialize pc.rom for TDX VMs
In reply to Xiaoyao LiFor TDX, the address below 1MB are entirely general RAM. No need to initialize pc.rom memory region for TDs.
---
This is more as a workaround of the issue that for q35 machine type, the real memslot update (which requires memslot deletion )for pc.rom happens after tdx_init_memory_region. It leads to the private memory ADD'ed before get lost. I haven't work out a good solution to resolve the order issue. So just skip the pc.rom setup to avoid memslot deletion.
--- hw/i386/pc.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-)
hw/i386/pc.c +12 -9
@@ -43,6 +43,7 @@#include "sysemu/xen.h" #include "sysemu/reset.h" #include "kvm/kvm_i386.h" +#include "kvm/tdx.h" #include "hw/xen/xen.h" #include "qapi/qmp/qlist.h" #include "qemu/error-report.h"
@@ -1039,16 +1040,18 @@ void pc_memory_init(PCMachineState *pcms,/* Initialize PC system firmware */ pc_system_firmware_init (pcms, rom_memory); - option_rom_mr =g_malloc (sizeof (*option_rom_mr)); -memory_region_init_ram (option_rom_mr,NULL ,"pc.rom" ,PC_ROM_SIZE , - &error_fatal); -if (pcmc->pci_enabled) { -memory_region_set_readonly (option_rom_mr, true); +if (!is_tdx_vm ()) { + option_rom_mr =g_malloc (sizeof (*option_rom_mr)); +memory_region_init_ram (option_rom_mr,NULL ,"pc.rom" ,PC_ROM_SIZE , + &error_fatal); +if (pcmc->pci_enabled) { +memory_region_set_readonly (option_rom_mr, true); + } +memory_region_add_subregion_overlap (rom_memory, +PC_ROM_MIN_VGA , + option_rom_mr, +1 ); } -memory_region_add_subregion_overlap (rom_memory, -PC_ROM_MIN_VGA , - option_rom_mr, -1 ); fw_cfg =fw_cfg_arch_create (machine, x86ms->boot_cpus, x86ms->apic_id_limit);
Signature · 1 line
2.34.1
PATCH v4 40/66 i386/tdx: Track mem_ptr for each firmware entry of TDVF
In reply to Xiaoyao LiFor each TDVF sections, QEMU needs to copy the content to guest private memory via KVM API (KVM_TDX_INIT_MEM_REGION).
Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory pointer of each TDVF sections. So that QEMU can add/copy them to guest private memory later.
TDVF sections can be classified into two groups:
- Firmware itself, e.g., TDVF BFV and CFV, that located separately from guest RAM. Its memory pointer is the bios pointer.
- Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB. mmap a new memory range for them.
Register a machine_init_done callback to do the stuff.
--- hw/i386/tdvf.c | 1 + include/hw/i386/tdvf.h | 7 +++++++ target/i386/kvm/tdx.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+)
hw/i386/tdvf.c +1 -0
@@ -189,6 +189,7 @@ int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size) }g_free (sections); + fw->mem_ptr = flash_ptr;return 0 ; err:
include/hw/i386/tdvf.h +7 -0
@@ -39,13 +39,20 @@ typedef struct TdxFirmwareEntry {uint64_t size;uint32_t type;uint32_t attributes; + + void *mem_ptr; } TdxFirmwareEntry;typedef struct TdxFirmware { +void *mem_ptr; +uint32_t nr_entries;TdxFirmwareEntry *entries; }TdxFirmware ; +#define for_each_tdx_fw_entry (fw, e) \ + for (e = (fw)->entries; e != (fw)->entries + (fw)->nr_entries; e++) +int tdvf_parse_metadata (TdxFirmware *fw,void *flash_ptr,int size);#endif /* HW_I386_TDVF_H */
target/i386/kvm/tdx.c +31 -0
@@ -14,6 +14,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" #include "qemu/base64.h" +#include "qemu/mmap-alloc.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h"
@@ -22,6 +23,7 @@#include "exec/address-spaces.h" #include "hw/i386/x86.h" +#include "hw/i386/tdvf.h" #include "kvm_i386.h" #include "tdx.h" #include "../cpu-internal.h"
@@ -464,6 +466,33 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; } +static void tdx_finalize_vm (Notifier *notifier,void *unused) +{ +TdxFirmware *tdvf = &tdx_guest->tdvf; +TdxFirmwareEntry *entry; + +for_each_tdx_fw_entry (tdvf, entry) { +switch (entry->type) { +case TDVF_SECTION_TYPE_BFV : +case TDVF_SECTION_TYPE_CFV : + entry->mem_ptr = tdvf->mem_ptr + entry->data_offset; +break ; +case TDVF_SECTION_TYPE_TD_HOB : +case TDVF_SECTION_TYPE_TEMP_MEM : + entry->mem_ptr =qemu_ram_mmap (-1 , entry->size, +qemu_real_host_page_size (),0 ,0 ); +break ; +default : +error_report ("Unsupported TDVF section %d" , entry->type); +exit (1 ); + } + } +} + +static Notifier tdx_machine_done_notify = { + .notify = tdx_finalize_vm, +}; +int tdx_kvm_init (MachineState *ms,Error **errp) {TdxGuest *tdx =TDX_GUEST (OBJECT (ms->cgs));
@@ -489,6 +518,8 @@ int tdx_kvm_init(MachineState *ms, Error **errp)*/ kvm_readonly_mem_allowed = false; +qemu_add_machine_init_done_notifier (&tdx_machine_done_notify); + tdx_guest = tdx;return 0 ; }
Signature · 1 line
2.34.1
PATCH v4 41/66 i386/tdx: Track RAM entries for TDX VM
In reply to Xiaoyao LiThe RAM of TDX VM can be classified into two types:
- TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be accepted by TDX guest before it can be used and will be all-zeros after being accepted.
- TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.
Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table and mark each RAM range as default type TDX_RAM_UNACCEPTED.
Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these ranges will be ADD'ed before TD runs and no need to be accepted runtime.
The TdxRamEntries[] are later used to setup the memory TD resource HOB that passes memory info from QEMU to TDVF.
---
Changes in v3:
- use enum TdxRamType in struct TdxRamEntry; (Isaku)
- Fix the indention; (Daniel)
Changes in v1:
- simplify the algorithm of tdx_accept_ram_range() (Suggested-by: Gerd Hoffman)
(1) Change the existing entry to cover the accepted ram range.
(2) If there is room before the accepted ram range add a TDX_RAM_UNACCEPTED entry for that.
(3) If there is room after the accepted ram range add a TDX_RAM_UNACCEPTED entry for that.
--- target/i386/kvm/tdx.c | 111 ++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 14 ++++++ 2 files changed, 125 insertions(+)
target/i386/kvm/tdx.c +111 -0
@@ -22,6 +22,7 @@#include "sysemu/sysemu.h" #include "exec/address-spaces.h" +#include "hw/i386/e820_memory_layout.h" #include "hw/i386/x86.h" #include "hw/i386/tdvf.h" #include "kvm_i386.h"
@@ -466,11 +467,117 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; } +static void tdx_add_ram_entry (uint64_t address,uint64_t length, +enum TdxRamType type) +{ +uint32_t nr_entries = tdx_guest->nr_ram_entries; + tdx_guest->ram_entries =g_renew (TdxRamEntry, tdx_guest->ram_entries, + nr_entries +1 ); + + tdx_guest->ram_entries[nr_entries].address = address; + tdx_guest->ram_entries[nr_entries].length = length; + tdx_guest->ram_entries[nr_entries].type = type; + tdx_guest->nr_ram_entries++; +} + +static int tdx_accept_ram_range (uint64_t address,uint64_t length) +{ +uint64_t head_start, tail_start, head_length, tail_length; +uint64_t tmp_address, tmp_length; +TdxRamEntry *e; +int i; + +for (i =0 ; i < tdx_guest->nr_ram_entries; i++) { + e = &tdx_guest->ram_entries[i]; + +if (address + length <= e->address || + e->address + e->length <= address) { +continue ; + } + +/* +* The to-be-accepted ram range must be fully contained by one +* RAM entry. +*/ +if (e->address > address || + e->address + e->length < address + length) { +return -EINVAL ; + } + +if (e->type ==TDX_RAM_ADDED ) { +return -EINVAL ; + } + +break ; + } + +if (i == tdx_guest->nr_ram_entries) { +return -1 ; + } + + tmp_address = e->address; + tmp_length = e->length; + + e->address = address; + e->length = length; + e->type =TDX_RAM_ADDED ; + + head_length = address - tmp_address; +if (head_length >0 ) { + head_start = tmp_address; +tdx_add_ram_entry (head_start, head_length,TDX_RAM_UNACCEPTED ); + } + + tail_start = address + length; +if (tail_start < tmp_address + tmp_length) { + tail_length = tmp_address + tmp_length - tail_start; +tdx_add_ram_entry (tail_start, tail_length,TDX_RAM_UNACCEPTED ); + } + +return 0 ; +} + +static int tdx_ram_entry_compare (const void *lhs_,const void * rhs_) +{ +const TdxRamEntry *lhs = lhs_; +const TdxRamEntry *rhs = rhs_; + +if (lhs->address == rhs->address) { +return 0 ; + } +if (le64_to_cpu (lhs->address) >le64_to_cpu (rhs->address)) { +return 1 ; + } +return -1 ; +} + +static void tdx_init_ram_entries (void ) +{ +unsigned i, j, nr_e820_entries; + + nr_e820_entries =e820_get_num_entries (); + tdx_guest->ram_entries =g_new (TdxRamEntry, nr_e820_entries); + +for (i =0 , j =0 ; i < nr_e820_entries; i++) { +uint64_t addr, len; + +if (e820_get_entry (i,E820_RAM , &addr, &len)) { + tdx_guest->ram_entries[j].address = addr; + tdx_guest->ram_entries[j].length = len; + tdx_guest->ram_entries[j].type =TDX_RAM_UNACCEPTED ; + j++; + } + } + tdx_guest->nr_ram_entries = j; +} +static void tdx_finalize_vm (Notifier *notifier,void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;TdxFirmwareEntry *entry; +tdx_init_ram_entries (); +for_each_tdx_fw_entry (tdvf, entry) {switch (entry->type) {case TDVF_SECTION_TYPE_BFV :
@@ -481,12 +588,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)case TDVF_SECTION_TYPE_TEMP_MEM : entry->mem_ptr =qemu_ram_mmap (-1 , entry->size,qemu_real_host_page_size (),0 ,0 ); +tdx_accept_ram_range (entry->address, entry->size);break ;default :error_report ("Unsupported TDVF section %d" , entry->type);exit (1 ); } } + +qsort (tdx_guest->ram_entries, tdx_guest->nr_ram_entries, +sizeof (TdxRamEntry), &tdx_ram_entry_compare); }static Notifier tdx_machine_done_notify = {
target/i386/kvm/tdx.h +14 -0
@@ -15,6 +15,17 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +enum TdxRamType { +TDX_RAM_UNACCEPTED , +TDX_RAM_ADDED , +}; + +typedef struct TdxRamEntry { +uint64_t address; +uint64_t length; +enum TdxRamType type; +}TdxRamEntry ; + typedef struct TdxGuest { ConfidentialGuestSupport parent_obj;
@@ -27,6 +38,9 @@ typedef struct TdxGuest {char *mrownerconfig;/* base64 encoded sha348 digest */ TdxFirmware tdvf; + +uint32_t nr_ram_entries; +TdxRamEntry *ram_entries; } TdxGuest;#ifdef CONFIG_TDX
Signature · 1 line
2.34.1
PATCH v4 42/66 headers: Add definitions from UEFI spec for volumes, resources, etc...
In reply to Xiaoyao LiAdd UEFI definitions for literals, enums, structs, GUIDs, etc... that will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed to the Trusted Domain Virtual Firmware (TDVF).
All values come from the UEFI specification [1], PI spec [2] and TDVF design guide[3].
[1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
[3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
--- include/standard-headers/uefi/uefi.h | 198 +++++++++++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 include/standard-headers/uefi/uefi.h
include/standard-headers/uefi/uefi.h new file +198 -0
@@ -0,0 +1,198 @@ +/* +* Copyright (C) 2020 Intel Corporation +* +* Author: Isaku Yamahata <isaku.yamahata at gmail.com> +* <isaku.yamahata at intel.com> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. + +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. + +* You should have received a copy of the GNU General Public License along +* with this program; if not, see <http://www.gnu.org/licenses/>. +* +*/ + +#ifndef HW_I386_UEFI_H +#define HW_I386_UEFI_H + +/***************************************************************************/ +/* +* basic EFI definitions +* supplemented with UEFI Specification Version 2.8 (Errata A) +* released February 2020 +*/ +/* UEFI integer is little endian */ + +typedef struct { +uint32_t Data1; +uint16_t Data2; +uint16_t Data3; +uint8_t Data4[8 ]; +}EFI_GUID ; + +typedef enum { + EfiReservedMemoryType, + EfiLoaderCode, + EfiLoaderData, + EfiBootServicesCode, + EfiBootServicesData, + EfiRuntimeServicesCode, + EfiRuntimeServicesData, + EfiConventionalMemory, + EfiUnusableMemory, + EfiACPIReclaimMemory, + EfiACPIMemoryNVS, + EfiMemoryMappedIO, + EfiMemoryMappedIOPortSpace, + EfiPalCode, + EfiPersistentMemory, + EfiUnacceptedMemoryType, + EfiMaxMemoryType +}EFI_MEMORY_TYPE ; + +#define EFI_HOB_HANDOFF_TABLE_VERSION 0x0009 + +#define EFI_HOB_TYPE_HANDOFF 0x0001 +#define EFI_HOB_TYPE_MEMORY_ALLOCATION 0x0002 +#define EFI_HOB_TYPE_RESOURCE_DESCRIPTOR 0x0003 +#define EFI_HOB_TYPE_GUID_EXTENSION 0x0004 +#define EFI_HOB_TYPE_FV 0x0005 +#define EFI_HOB_TYPE_CPU 0x0006 +#define EFI_HOB_TYPE_MEMORY_POOL 0x0007 +#define EFI_HOB_TYPE_FV2 0x0009 +#define EFI_HOB_TYPE_LOAD_PEIM_UNUSED 0x000A +#define EFI_HOB_TYPE_UEFI_CAPSULE 0x000B +#define EFI_HOB_TYPE_FV3 0x000C +#define EFI_HOB_TYPE_UNUSED 0xFFFE +#define EFI_HOB_TYPE_END_OF_HOB_LIST 0xFFFF + +typedef struct { +uint16_t HobType; +uint16_t HobLength; +uint32_t Reserved; +}EFI_HOB_GENERIC_HEADER ; + +typedef uint64_t EFI_PHYSICAL_ADDRESS ; +typedef uint32_t EFI_BOOT_MODE ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +uint32_t Version; +EFI_BOOT_MODE BootMode; +EFI_PHYSICAL_ADDRESS EfiMemoryTop; +EFI_PHYSICAL_ADDRESS EfiMemoryBottom; +EFI_PHYSICAL_ADDRESS EfiFreeMemoryTop; +EFI_PHYSICAL_ADDRESS EfiFreeMemoryBottom; +EFI_PHYSICAL_ADDRESS EfiEndOfHobList; +}EFI_HOB_HANDOFF_INFO_TABLE ; + +#define EFI_RESOURCE_SYSTEM_MEMORY 0x00000000 +#define EFI_RESOURCE_MEMORY_MAPPED_IO 0x00000001 +#define EFI_RESOURCE_IO 0x00000002 +#define EFI_RESOURCE_FIRMWARE_DEVICE 0x00000003 +#define EFI_RESOURCE_MEMORY_MAPPED_IO_PORT 0x00000004 +#define EFI_RESOURCE_MEMORY_RESERVED 0x00000005 +#define EFI_RESOURCE_IO_RESERVED 0x00000006 +#define EFI_RESOURCE_MEMORY_UNACCEPTED 0x00000007 +#define EFI_RESOURCE_MAX_MEMORY_TYPE 0x00000008 + +#define EFI_RESOURCE_ATTRIBUTE_PRESENT 0x00000001 +#define EFI_RESOURCE_ATTRIBUTE_INITIALIZED 0x00000002 +#define EFI_RESOURCE_ATTRIBUTE_TESTED 0x00000004 +#define EFI_RESOURCE_ATTRIBUTE_SINGLE_BIT_ECC 0x00000008 +#define EFI_RESOURCE_ATTRIBUTE_MULTIPLE_BIT_ECC 0x00000010 +#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_1 0x00000020 +#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_2 0x00000040 +#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTED 0x00000080 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTED 0x00000100 +#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTED 0x00000200 +#define EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE 0x00000400 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_COMBINEABLE 0x00000800 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_THROUGH_CACHEABLE 0x00001000 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_BACK_CACHEABLE 0x00002000 +#define EFI_RESOURCE_ATTRIBUTE_16_BIT_IO 0x00004000 +#define EFI_RESOURCE_ATTRIBUTE_32_BIT_IO 0x00008000 +#define EFI_RESOURCE_ATTRIBUTE_64_BIT_IO 0x00010000 +#define EFI_RESOURCE_ATTRIBUTE_UNCACHED_EXPORTED 0x00020000 +#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTED 0x00040000 +#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTABLE 0x00080000 +#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTABLE 0x00100000 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTABLE 0x00200000 +#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTABLE 0x00400000 +#define EFI_RESOURCE_ATTRIBUTE_PERSISTENT 0x00800000 +#define EFI_RESOURCE_ATTRIBUTE_PERSISTABLE 0x01000000 +#define EFI_RESOURCE_ATTRIBUTE_MORE_RELIABLE 0x02000000 + +typedef uint32_t EFI_RESOURCE_TYPE ; +typedef uint32_t EFI_RESOURCE_ATTRIBUTE_TYPE ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_GUID Owner; +EFI_RESOURCE_TYPE ResourceType; +EFI_RESOURCE_ATTRIBUTE_TYPE ResourceAttribute; +EFI_PHYSICAL_ADDRESS PhysicalStart; +uint64_t ResourceLength; +}EFI_HOB_RESOURCE_DESCRIPTOR ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_GUID Name; + +/* guid specific data follows */ +}EFI_HOB_GUID_TYPE ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +}EFI_HOB_FIRMWARE_VOLUME ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +EFI_GUID FvName; +EFI_GUID FileName; +}EFI_HOB_FIRMWARE_VOLUME2 ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +uint32_t AuthenticationStatus; +bool ExtractedFv; +EFI_GUID FvName; +EFI_GUID FileName; +}EFI_HOB_FIRMWARE_VOLUME3 ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +uint8_t SizeOfMemorySpace; +uint8_t SizeOfIoSpace; +uint8_t Reserved[6 ]; +}EFI_HOB_CPU ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +}EFI_HOB_MEMORY_POOL ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; + +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +}EFI_HOB_UEFI_CAPSULE ; + +#define EFI_HOB_OWNER_ZERO \ + ((EFI_GUID){ 0x00000000, 0x0000, 0x0000, \ + { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 } }) + +#endif
Signature · 1 line
2.34.1
PATCH v4 43/66 i386/tdx: Setup the TD HOB list
In reply to Xiaoyao LiThe TD HOB list is used to pass the information from VMM to TDVF. The TD HOB must include PHIT HOB and Resource Descriptor HOB. More details can be found in TDVF specification and PI specification.
Build the TD HOB in TDX's machine_init_done callback.
---
Changes in v1:
- drop the code of adding mmio resources since OVMF prepares all the MMIO hob itself.
--- hw/i386/meson.build | 2 +- hw/i386/tdvf-hob.c | 147 ++++++++++++++++++++++++++++++++++++++++++ hw/i386/tdvf-hob.h | 24 +++++++ target/i386/kvm/tdx.c | 16 +++++ 4 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 hw/i386/tdvf-hob.c create mode 100644 hw/i386/tdvf-hob.h
hw/i386/meson.build +1 -1
@@ -27,7 +27,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'port92.c')) i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'), if_false: files('pc_sysfw_ovmf-stubs.c')) -i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c')) +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c')) subdir('kvm') subdir('xen')
hw/i386/tdvf-hob.c new file +147 -0
147 lines not on this page · raw message
hw/i386/tdvf-hob.h new file +24 -0
@@ -0,0 +1,24 @@ +#ifndef HW_I386_TD_HOB_H +#define HW_I386_TD_HOB_H + +#include "hw/i386/tdvf.h" +#include "target/i386/kvm/tdx.h" + +void tdvf_hob_create (TdxGuest *tdx,TdxFirmwareEntry *td_hob); + +#define EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE \ + (EFI_RESOURCE_ATTRIBUTE_PRESENT | \ + EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \ + EFI_RESOURCE_ATTRIBUTE_TESTED) + +#define EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED \ + (EFI_RESOURCE_ATTRIBUTE_PRESENT | \ + EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \ + EFI_RESOURCE_ATTRIBUTE_TESTED) + +#define EFI_RESOURCE_ATTRIBUTE_TDVF_MMIO \ + (EFI_RESOURCE_ATTRIBUTE_PRESENT | \ + EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \ + EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE) + +#endif
target/i386/kvm/tdx.c +16 -0
@@ -25,6 +25,7 @@#include "hw/i386/e820_memory_layout.h" #include "hw/i386/x86.h" #include "hw/i386/tdvf.h" +#include "hw/i386/tdvf-hob.h" #include "kvm_i386.h" #include "tdx.h" #include "../cpu-internal.h"
@@ -467,6 +468,19 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; } +static TdxFirmwareEntry *tdx_get_hob_entry (TdxGuest *tdx) +{ +TdxFirmwareEntry *entry; + +for_each_tdx_fw_entry (&tdx->tdvf, entry) { +if (entry->type ==TDVF_SECTION_TYPE_TD_HOB ) { +return entry; + } + } +error_report ("TDVF metadata doesn't specify TD_HOB location." ); +exit (1 ); +} +static void tdx_add_ram_entry (uint64_t address,uint64_t length,enum TdxRamType type) {
@@ -598,6 +612,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)qsort (tdx_guest->ram_entries, tdx_guest->nr_ram_entries,sizeof (TdxRamEntry), &tdx_ram_entry_compare); + +tdvf_hob_create (tdx_guest,tdx_get_hob_entry (tdx_guest)); }static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
PATCH v4 44/66 i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TDVF firmware (CODE and VARS) needs to be added/copied to TD's private memory via KVM_TDX_INIT_MEM_REGION, as well as TD HOB and TEMP memory.
---
Changes in v1:
- rename variable @metadata to @flags
--- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
target/i386/kvm/tdx.c +24 -0
@@ -589,6 +589,7 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;TdxFirmwareEntry *entry; +int r;tdx_init_ram_entries ();
@@ -614,6 +615,29 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) sizeof(TdxRamEntry), &tdx_ram_entry_compare);tdvf_hob_create (tdx_guest,tdx_get_hob_entry (tdx_guest)); + +for_each_tdx_fw_entry (tdvf, entry) { +struct kvm_tdx_init_mem_region mem_region = { + .source_addr = (__u64 )entry->mem_ptr, + .gpa = entry->address, + .nr_pages = entry->size /4096 , + }; + +__u32 flags = entry->attributes &TDVF_SECTION_ATTRIBUTES_MR_EXTEND ? +KVM_TDX_MEASURE_MEMORY_REGION :0 ; + + r =tdx_vm_ioctl (KVM_TDX_INIT_MEM_REGION , flags, &mem_region); +if (r <0 ) { +error_report ("KVM_TDX_INIT_MEM_REGION failed %s" ,strerror (-r)); +exit (1 ); + } + +if (entry->type ==TDVF_SECTION_TYPE_TD_HOB || + entry->type ==TDVF_SECTION_TYPE_TEMP_MEM ) { +qemu_ram_munmap (-1 , entry->mem_ptr, entry->size); + entry->mem_ptr =NULL ; + } + } }static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
PATCH v4 45/66 memory: Introduce memory_region_init_ram_guest_memfd()
In reply to Xiaoyao LiIntroduce memory_region_init_ram_guest_memfd() to allocate private guset memfd on the MemoryRegion initialization. It's for the use case of TDVF, which must be private on TDX case.
--- include/exec/memory.h | 6 ++++++ system/memory.c | 27 +++++++++++++++++++++++++++ 2 files changed, 33 insertions(+)
include/exec/memory.h +6 -0
@@ -1607,6 +1607,12 @@ bool memory_region_init_ram(MemoryRegion *mr,uint64_t size, Error **errp); +bool memory_region_init_ram_guest_memfd (MemoryRegion *mr, +Object *owner, +const char *name, +uint64_t size, +Error **errp); +/** * memory_region_init_rom: Initialize a ROM memory region. *
system/memory.c +27 -0
@@ -3619,6 +3619,33 @@ bool memory_region_init_ram(MemoryRegion *mr,return true; } +bool memory_region_init_ram_guest_memfd (MemoryRegion *mr, +Object *owner, +const char *name, +uint64_t size, +Error **errp) +{ +DeviceState *owner_dev; + +if (!memory_region_init_ram_flags_nomigrate (mr, owner, name, size, +RAM_GUEST_MEMFD , errp)) { +return false; + } + +memory_region_set_default_private (mr); + +/* This will assert if owner is neither NULL nor a DeviceState. +* We only want the owner here for the purposes of defining a +* unique name for migration. TODO: Ideally we should implement +* a naming scheme for Objects which are not DeviceStates, in +* which case we can relax this restriction. +*/ + owner_dev =DEVICE (owner); +vmstate_register_ram (mr, owner_dev); + +return true; +} + bool memory_region_init_rom(MemoryRegion *mr, Object *owner, const char *name,
Signature · 1 line
2.34.1
Re: [PATCH v4 45/66] memory: Introduce memory_region_init_ram_guest_memfd()
Quoted 39 lines · Xiaoyao Li
Introduce
memory_region_init_ram_guest_memfd()to allocate private guset memfd on the MemoryRegion initialization. It's for the use case of TDVF, which must be private on TDX case.Signed-off-by Xiaoyao Li <[email protected]>--- include/exec/memory.h | 6 ++++++ system/memory.c | 27 +++++++++++++++++++++++++++ 2 files changed, 33 insertions(+)include/exec/memory.h +6
-0@@ -1607,6 +1607,12 @@ bool memory_region_init_ram(MemoryRegion *mr,uint64_t size, Error **errp); +bool memory_region_init_ram_guest_memfd (MemoryRegion *mr, +Object *owner, +const char *name, +uint64_t size, +Error **errp); +/** * memory_region_init_rom: Initialize a ROM memory region. * system/memory.c +4
-0@@ -3619,6 +3619,33 @@ bool memory_region_init_ram(MemoryRegion *mr,return true; } +bool memory_region_init_ram_guest_memfd(MemoryRegion *mr, + Object *owner, + const char *name, + uint64_t size,
system/memory.c bool memory_region_init_ram(MemoryRegion *mr, · line 3635
Quoted 10 lines
system/memory.c +10
-0+ Error **errp) +{ +DeviceState *owner_dev; + +if (!memory_region_init_ram_flags_nomigrate (mr, owner, name, size, +RAM_GUEST_MEMFD , errp)) { +return false; + } + +memory_region_set_default_private (mr);
It would be much cleaner to just pass the flag when initializing the memory region.
Ideally, we can get rid of memory_region_set_default_private().
What you do in patch #33 feels like a layer violation.
Signature · 3 lines
Cheers, David / dhildenb
PATCH v4 46/66 i386/tdx: register TDVF as private memory
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
Allocate private guest memfd memory for BIOS if it's TD VM.
--- hw/i386/x86.c | 10 +++++++++- target/i386/kvm/tdx.c | 18 ++++++++++++++++++ target/i386/kvm/tdx.h | 2 ++ 3 files changed, 29 insertions(+), 1 deletion(-)
hw/i386/x86.c +9 -1
@@ -1155,8 +1155,16 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware, (bios_size % 65536) != 0) { goto bios_error; } + bios =g_malloc (sizeof (*bios)); -memory_region_init_ram (bios,NULL ,"pc.bios" , bios_size, &error_fatal); +if (is_tdx_vm ()) { +memory_region_init_ram_guest_memfd (bios,NULL ,"pc.bios" , bios_size, + &error_fatal); +tdx_set_tdvf_region (bios); + }else { +memory_region_init_ram (bios,NULL ,"pc.bios" , bios_size, &error_fatal); + } +if (sev_enabled () ||is_tdx_vm ()) {/* * The concept of a "reset" simply doesn't exist for
target/i386/kvm/tdx.c +18 -0
@@ -21,6 +21,7 @@#include "sysemu/kvm.h" #include "sysemu/sysemu.h" #include "exec/address-spaces.h" +#include "exec/ramblock.h" #include "hw/i386/e820_memory_layout.h" #include "hw/i386/x86.h"
@@ -468,6 +469,12 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; } +void tdx_set_tdvf_region (MemoryRegion *tdvf_region) +{ +assert (!tdx_guest->tdvf_region); + tdx_guest->tdvf_region = tdvf_region; +} +static TdxFirmwareEntry *tdx_get_hob_entry (TdxGuest *tdx) {TdxFirmwareEntry *entry;
@@ -589,6 +596,7 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;TdxFirmwareEntry *entry; +RAMBlock *ram_block;int r;tdx_init_ram_entries ();
@@ -623,6 +631,12 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) .nr_pages = entry->size / 4096, }; + r =kvm_set_memory_attributes_private (entry->address, entry->size); +if (r <0 ) { +error_report ("Reserve initial private memory failed %s" ,strerror (-r)); +exit (1 ); + } +__u32 flags = entry->attributes &TDVF_SECTION_ATTRIBUTES_MR_EXTEND ?KVM_TDX_MEASURE_MEMORY_REGION :0 ;
@@ -638,6 +652,10 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) entry->mem_ptr =NULL ; } } + +/* Tdvf image was copied into private region above. It becomes unnecessary. */ + ram_block = tdx_guest->tdvf_region->ram_block; +ram_block_discard_range (ram_block,0 , ram_block->max_length); }static Notifier tdx_machine_done_notify = {
target/i386/kvm/tdx.h +2 -0
@@ -38,6 +38,7 @@ typedef struct TdxGuest {char *mrownerconfig;/* base64 encoded sha348 digest */ TdxFirmware tdvf; +MemoryRegion *tdvf_region;uint32_t nr_ram_entries;TdxRamEntry *ram_entries;
@@ -53,6 +54,7 @@ int tdx_kvm_init(MachineState *ms, Error **errp);void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg,uint32_t *ret);int tdx_pre_create_vcpu (CPUState *cpu,Error **errp); +void tdx_set_tdvf_region (MemoryRegion *tdvf_region);int tdx_parse_tdvf (void *flash_ptr,int size);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 47/66 i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu
In reply to Xiaoyao LiTDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.
KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for each vcpu after HOB list is created.
--- target/i386/kvm/tdx.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
target/i386/kvm/tdx.c +18 -0
@@ -592,6 +592,22 @@ static void tdx_init_ram_entries(void) tdx_guest->nr_ram_entries = j; } +static void tdx_post_init_vcpus (void ) +{ +TdxFirmwareEntry *hob; +CPUState *cpu; +int r; + + hob =tdx_get_hob_entry (tdx_guest); +CPU_FOREACH (cpu) { + r =tdx_vcpu_ioctl (cpu,KVM_TDX_INIT_VCPU ,0 , (void *)hob->address); +if (r <0 ) { +error_report ("KVM_TDX_INIT_VCPU failed %s" ,strerror (-r)); +exit (1 ); + } + } +} +static void tdx_finalize_vm (Notifier *notifier,void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;
@@ -624,6 +640,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)tdvf_hob_create (tdx_guest,tdx_get_hob_entry (tdx_guest)); +tdx_post_init_vcpus (); +for_each_tdx_fw_entry (tdvf, entry) {struct kvm_tdx_init_mem_region mem_region = { .source_addr = (__u64 )entry->mem_ptr,
Signature · 1 line
2.34.1
PATCH v4 48/66 i386/tdx: Finalize TDX VM
In reply to Xiaoyao LiInvoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make the TD vCPUs runnable once machine initialization is complete.
--- target/i386/kvm/tdx.c | 7 +++++++ 1 file changed, 7 insertions(+)
target/i386/kvm/tdx.c +7 -0
@@ -674,6 +674,13 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)/* Tdvf image was copied into private region above. It becomes unnecessary. */ ram_block = tdx_guest->tdvf_region->ram_block;ram_block_discard_range (ram_block,0 , ram_block->max_length); + + r =tdx_vm_ioctl (KVM_TDX_FINALIZE_VM ,0 ,NULL ); +if (r <0 ) { +error_report ("KVM_TDX_FINALIZE_VM failed %s" ,strerror (-r)); +exit (0 ); + } + tdx_guest->parent_obj.ready = true; }static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
PATCH v4 49/66 i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt>
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
For SetupEventNotifyInterrupt, record interrupt vector and the apic id of the vcpu that received this TDVMCALL.
Later it can inject interrupt with given vector to the specific vcpu that received SetupEventNotifyInterrupt.
--- target/i386/kvm/kvm.c | 8 ++++++ target/i386/kvm/tdx-stub.c | 5 ++++ target/i386/kvm/tdx.c | 53 ++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 14 ++++++++++ 4 files changed, 80 insertions(+)
target/i386/kvm/kvm.c +8 -0
@@ -5426,6 +5426,14 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) ret =kvm_xen_handle_exit (cpu, &run->xen);break ;#endif +case KVM_EXIT_TDX : +if (!is_tdx_vm ()) { +error_report ("KVM: get KVM_EXIT_TDX for a non-TDX VM." ); + ret =-1 ; +break ; + } + ret =tdx_handle_exit (cpu, &run->tdx); +break ;default :fprintf (stderr,"KVM: unknown exit reason %d\n" , run->exit_reason); ret =-1 ;
target/i386/kvm/tdx-stub.c +5 -0
@@ -16,3 +16,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size) {return -EINVAL ; } + +int tdx_handle_exit (X86CPU *cpu,struct kvm_tdx_exit *tdx_exit) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +53 -0
65 lines not on this page · raw message
target/i386/kvm/tdx.h +14 -0
@@ -7,6 +7,7 @@#include "exec/confidential-guest-support.h" #include "hw/i386/tdvf.h" +#include "sysemu/kvm.h" #define TYPE_TDX_GUEST "tdx-guest"#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -15,6 +16,14 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL + +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_RETRY 0x0000000000000001ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_GPA_INUSE 0x8000000000000001ULL +#define TDG_VP_VMCALL_ALIGN_ERROR 0x8000000000000002ULL +enum TdxRamType {TDX_RAM_UNACCEPTED ,TDX_RAM_ADDED ,
@@ -42,6 +51,10 @@ typedef struct TdxGuest {uint32_t nr_ram_entries;TdxRamEntry *ram_entries; + +/* runtime state */ +uint32_t event_notify_vector; +uint32_t event_notify_apicid; } TdxGuest;#ifdef CONFIG_TDX
@@ -56,5 +69,6 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,int tdx_pre_create_vcpu (CPUState *cpu,Error **errp);void tdx_set_tdvf_region (MemoryRegion *tdvf_region);int tdx_parse_tdvf (void *flash_ptr,int size); +int tdx_handle_exit (X86CPU *cpu,struct kvm_tdx_exit *tdx_exit);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v4 50/66 i386/tdx: handle TDG.VP.VMCALL<GetQuote>
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \
-object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
-machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
---
Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";
Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;
--- qapi/qom.json | 6 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 493 insertions(+), 2 deletions(-) create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.h
qapi/qom.json +5 -1
@@ -914,13 +914,17 @@ # e.g., specific to the workload rather than the run-time or OS. # base64 encoded SHA384 digest. # +# @quote-generation-socket: socket address for Quote Generation +# Service(QGS) +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } } ## # @ThreadContextProperties:
target/i386/kvm/meson.build +1 -1
@@ -9,7 +9,7 @@ i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c')) i386_kvm_ss.add(when: 'CONFIG_SEV', if_false: files('sev-stub.c')) -i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c')) +i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c', 'tdx-quote-generator.c'), if_false: files('tdx-stub.c')) i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
target/i386/kvm/tdx-quote-generator.c new file +170 -0
170 lines not on this page · raw message
target/i386/kvm/tdx-quote-generator.h new file +95 -0
95 lines not on this page · raw message
target/i386/kvm/tdx.c +216 -0
258 lines not on this page · raw message
target/i386/kvm/tdx.h +6 -0
@@ -9,6 +9,8 @@#include "hw/i386/tdvf.h" #include "sysemu/kvm.h" +#include "tdx-quote-generator.h" +#define TYPE_TDX_GUEST "tdx-guest"#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -16,6 +18,7 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
@@ -55,6 +58,9 @@ typedef struct TdxGuest {/* runtime state */ uint32_t event_notify_vector;uint32_t event_notify_apicid; + +/* GetQuote */ +TdxQuoteGenerator *quote_generator; } TdxGuest;#ifdef CONFIG_TDX
Signature · 1 line
2.34.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [PATCH v4 50/66] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 63 lines · Markus Armbruster
Quoted 53 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;--- qapi/qom.json | 6 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 493 insertions(+), 2 deletions(-) create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.hqapi/qom.json +0
-0qapi/qom.json line 918
Quoted 5 lines
qapi/qom.json +2
-0@@ -914,13 +914,17 @@ # e.g., specific to the workload rather than the run-time or OS. # base64 encoded SHA384 digest. # +# @quote-generation-socket: socket address for Quote Generation +# Service(QGS)Space between "Service" and "(QGS)", please.
Quoted 2 lines
The description feels too terse. What is the "Quote Generation Service", and why should I care?
The "Quote Generation Service" is a daemon running on the host. The reference implementation is at
If you don't provide this, then quests won't bet able to generate quotes needed for attestation. So although this is technically optional, in practice for a sane deployment, an admin should always provide this
Quoted 12 lines
Quoted 12 lines
qapi/qom.json +3
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } }
Quoted 5 lines
Quoted 2 lines
qapi/qom.json +0
-0## # @ThreadContextProperties:[...]
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v4 50/66] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 70 lines · Daniel P. Berrangé
Quoted 63 lines · Markus Armbruster
Quoted 53 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;--- qapi/qom.json | 6 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 493 insertions(+), 2 deletions(-) create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.hqapi/qom.json +0
-0qapi/qom.json line 918
Quoted 5 lines
qapi/qom.json +2
-0@@ -914,13 +914,17 @@ # e.g., specific to the workload rather than the run-time or OS. # base64 encoded SHA384 digest. # +# @quote-generation-socket: socket address for Quote Generation +# Service(QGS)Space between "Service" and "(QGS)", please.
Quoted 2 lines
The description feels too terse. What is the "Quote Generation Service", and why should I care?
The "Quote Generation Service" is a daemon running on the host. The reference implementation is at
Quoted 6 lines
If you don't provide this, then quests won't bet able to generate quotes needed for attestation. So although this is technically optional, in practice for a sane deployment, an admin should always provide this
Thanks. Care to work some of this information into the doc comment?
Quoted 19 lines
Quoted 11 lines
Quoted 11 lines
qapi/qom.json +3
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } }Quoted 5 lines
Quoted 2 lines
qapi/qom.json +0
-0## # @ThreadContextProperties:[...]
With regards,
Daniel
Re: [PATCH v4 50/66] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 72 lines · Markus Armbruster
Quoted 70 lines · Daniel P. Berrangé
Quoted 63 lines · Markus Armbruster
Quoted 53 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;--- qapi/qom.json | 6 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 493 insertions(+), 2 deletions(-)create mode 100644
target/i386/kvm/tdx-quote-generator.ccreate mode 100644target/i386/kvm/tdx-quote-generator.hqapi/qom.json +0
-0qapi/qom.json line 918
Quoted 5 lines
qapi/qom.json +2
-0@@ -914,13 +914,17 @@ # e.g., specific to the workload rather than the run-time or OS. # base64 encoded SHA384 digest. # +# @quote-generation-socket: socket address for Quote Generation +# Service(QGS)Space between "Service" and "(QGS)", please.
Quoted 2 lines
The description feels too terse. What is the "Quote Generation Service", and why should I care?
The "Quote Generation Service" is a daemon running on the host. The reference implementation is at
Quoted 8 lines
Quoted 6 lines
If you don't provide this, then quests won't bet able to generate quotes needed for attestation. So although this is technically optional, in practice for a sane deployment, an admin should always provide this
Thanks. Care to work some of this information into the doc comment?
Sure. Will add a new section of Attestation in the last patch.
Quoted 20 lines
Quoted 19 lines
Quoted 11 lines
Quoted 11 lines
qapi/qom.json +3
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } }Quoted 5 lines
Quoted 2 lines
qapi/qom.json +0
-0## # @ThreadContextProperties:[...]
With regards,
Daniel
Re: [PATCH v4 50/66] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
In reply to Xiaoyao LiQuoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 8 lines
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
Can you confirm again exactly what QGS impl you are testing against ?
I've tried the impl at
which supports UNIX sockets and VSOCK. In both cases, however, it appears to be speaking a different protocol than your QEMU impl below uses.
Specifically here:
it is reading 4 bytes of header, which are interpreted as the length of the payload which will then be read off the wire. IIUC the payload it expects is the TDREPORT struct.
Your QEMU patches here meanwhile are just sending the payload from the GetQuote hypercall which is the TDREPORT struct.
IOW, QEMU is not sending the 4 byte length header the QGS expects. and whole thing fails.
Quoted 11 lines
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v4 50/66] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
+ Feng Qiu,
Quoted 37 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 8 lines
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
Can you confirm again exactly what QGS impl you are testing against ?
Quoted 1 line
I've tried the impl at
which supports UNIX sockets and VSOCK. In both cases, however, it appears to be speaking a different protocol than your QEMU impl below uses.
Specifically here:
it is reading 4 bytes of header, which are interpreted as the length of the payload which will then be read off the wire. IIUC the payload it expects is the TDREPORT struct.
Quoted 5 lines
Your QEMU patches here meanwhile are just sending the payload from the GetQuote hypercall which is the TDREPORT struct.
IOW, QEMU is not sending the 4 byte length header the QGS expects. and whole thing fails.
I'm using the one provided by internal folks, which supports interpreting the payload without the header.
I don't know when will the updated implementation show up in public github. @Feng Liu can help on it.
Quoted 14 lines
Quoted 11 lines
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
With regards,
Daniel
Re: [PATCH v4 50/66] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Actually the 4 byte length header is provided by client library(https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/quote_wrapper/tdx_attest/tdx_attest.c#L295), not QEMU. QEMUjust treats the how payload including the header a whole blob.
BTW, in the latest stable kernel, the TDX guest driver changed to TSM based solution(https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/virt/coco/tdx-guest/tdx-guest.c?h=v6.7.5) and it will only send raw report without 4 byte length header and other stuff. Existing official QGS doesn't compatible with this change and we will deliver compatible QGS in the end of Q1.
Quoted 67 lines · Xiaoyao Li
+ Feng Qiu,
Quoted 39 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 8 lines
command line example:
qemu-system-x86_64 \
-object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
-machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
Can you confirm again exactly what QGS impl you are testing against ?
Quoted 1 line
I've tried the impl at
which supports UNIX sockets and VSOCK. In both cases, however, it appears to be speaking a different protocol than your QEMU impl below uses.
Specifically here:
it is reading 4 bytes of header, which are interpreted as the length of the payload which will then be read off the wire. IIUC the payload it expects is the TDREPORT struct.
Quoted 5 lines
Your QEMU patches here meanwhile are just sending the payload from the GetQuote hypercall which is the TDREPORT struct.
IOW, QEMU is not sending the 4 byte length header the QGS expects. and whole thing fails.
I'm using the one provided by internal folks, which supports interpreting the payload without the header.
I don't know when will the updated implementation show up in public github. @Feng Liu can help on it.
Quoted 14 lines
Quoted 11 lines
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
With regards,
Daniel
PATCH v4 51/66 i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
MapGPA is a hypercall to convert GPA from/to private GPA to/from shared GPA. As the conversion function is already implemented as kvm_convert_memory, wire it to TDX hypercall exit.
--- accel/kvm/kvm-all.c | 2 +- include/sysemu/kvm.h | 2 ++ target/i386/kvm/tdx.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 1 + 4 files changed, 58 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +1 -1
@@ -2920,7 +2920,7 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } -static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) {MemoryRegionSection section;ram_addr_t offset;
include/sysemu/kvm.h +2 -0
@@ -544,4 +544,6 @@ int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);int kvm_set_memory_attributes_private (hwaddr start,hwaddr size);int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size); + +int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private);#endif
target/i386/kvm/tdx.c +54 -0
66 lines not on this page · raw message
target/i386/kvm/tdx.h +1 -0
@@ -18,6 +18,7 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +#define TDG_VP_VMCALL_MAP_GPA 0x10001ULL#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL
Signature · 1 line
2.34.1
PATCH v4 52/66 i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>
In reply to Xiaoyao LiTD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request termination with error message encoded in GPRs.
Parse and print the error message, and terminate the TD guest in the handler.
--- target/i386/kvm/tdx.c | 39 +++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 1 + 2 files changed, 40 insertions(+)
target/i386/kvm/tdx.c +39 -0
@@ -1078,6 +1078,43 @@ static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)return 0 ; } +static int tdx_handle_report_fatal_error (X86CPU *cpu, +struct kvm_tdx_vmcall *vmcall) +{ +uint64_t error_code = vmcall->in_r12; +char *message =NULL ; + +if (error_code &0xffff ) { +error_report ("TDX: REPORT_FATAL_ERROR: invalid error code: " +"0x%lx\n" , error_code); +return -1 ; + } + +/* it has optional message */ +if (vmcall->in_r14) { +uint64_t * tmp; + +#define GUEST_PANIC_INFO_TDX_MESSAGE_MAX 64 + message =g_malloc0 (GUEST_PANIC_INFO_TDX_MESSAGE_MAX +1 ); + + tmp = (uint64_t *)message; +/* The order is defined in TDX GHCI spec */ + *(tmp++) =cpu_to_le64 (vmcall->in_r14); + *(tmp++) =cpu_to_le64 (vmcall->in_r15); + *(tmp++) =cpu_to_le64 (vmcall->in_rbx); + *(tmp++) =cpu_to_le64 (vmcall->in_rdi); + *(tmp++) =cpu_to_le64 (vmcall->in_rsi); + *(tmp++) =cpu_to_le64 (vmcall->in_r8); + *(tmp++) =cpu_to_le64 (vmcall->in_r9); + *(tmp++) =cpu_to_le64 (vmcall->in_rdx); + message[GUEST_PANIC_INFO_TDX_MESSAGE_MAX ] ='\0' ; +assert ((char *)tmp == message +GUEST_PANIC_INFO_TDX_MESSAGE_MAX ); + } + +error_report ("TD guest reports fatal error. %s\n" , message ? :"" ); +return -1 ; +} +static int tdx_handle_setup_event_notify_interrupt (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) {
@@ -1112,6 +1149,8 @@ static int tdx_handle_vmcall(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)return tdx_handle_map_gpa (cpu, vmcall);case TDG_VP_VMCALL_GET_QUOTE :return tdx_handle_get_quote (cpu, vmcall); +case TDG_VP_VMCALL_REPORT_FATAL_ERROR : +return tdx_handle_report_fatal_error (cpu, vmcall);case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT :return tdx_handle_setup_event_notify_interrupt (cpu, vmcall);default :
target/i386/kvm/tdx.h +1 -0
@@ -20,6 +20,7 @@ typedef struct TdxGuestClass {#define TDG_VP_VMCALL_MAP_GPA 0x10001ULL#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003ULL#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
Signature · 1 line
2.34.1
PATCH v4 53/66 i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
In reply to Xiaoyao LiIntegrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
Originated-from: Isaku Yamahata <[email protected]>
---
Changes in v4:
- refine the documentation; (Markus)
Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)
--- qapi/run-state.json | 28 ++++++++++++++++++++-- system/runstate.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++- 3 files changed, 103 insertions(+), 3 deletions(-)
qapi/run-state.json +26 -2
@@ -485,10 +485,12 @@ # # @s390: s390 guest panic information type (Since: 2.12) # +# @tdx: tdx guest panic information type (Since: 8.2) +# # Since: 2.9 ## { 'enum': 'GuestPanicInformationType', - 'data': [ 'hyper-v', 's390' ] } + 'data': [ 'hyper-v', 's390', 'tdx' ] } ## # @GuestPanicInformation:
@@ -503,7 +505,8 @@ 'base': {'type': 'GuestPanicInformationType'}, 'discriminator': 'type', 'data': {'hyper-v': 'GuestPanicInformationHyperV', - 's390': 'GuestPanicInformationS390'}} + 's390': 'GuestPanicInformationS390', + 'tdx' : 'GuestPanicInformationTdx'}} ## # @GuestPanicInformationHyperV:
@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}} +## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error code +# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string. +# +# @message: Human-readable error message provided by the guest. Not +# to be trusted. +# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}} + ## # @MEMORY_FAILURE: #
system/runstate.c +54 -0
67 lines not on this page · raw message
target/i386/kvm/tdx.c +23 -1
@@ -20,6 +20,7 @@#include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" +#include "sysemu/runstate.h" #include "sysemu/sysemu.h" #include "exec/address-spaces.h" #include "exec/ramblock.h"
@@ -1078,11 +1079,26 @@ static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)return 0 ; } +static void tdx_panicked_on_fatal_error (X86CPU *cpu,uint64_t error_code, +uint64_t gpa,char *message) +{ +GuestPanicInformation *panic_info; + + panic_info =g_new0 (GuestPanicInformation,1 ); + panic_info->type =GUEST_PANIC_INFORMATION_TYPE_TDX ; + panic_info->u.tdx.error_code = error_code; + panic_info->u.tdx.gpa = gpa; + panic_info->u.tdx.message = message; + +qemu_system_guest_panicked (panic_info); +} +static int tdx_handle_report_fatal_error (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) {uint64_t error_code = vmcall->in_r12;char *message =NULL ; +uint64_t gpa =-1ull ;if (error_code &0xffff ) { error_report("TDX: REPORT_FATAL_ERROR: invalid error code: "
@@ -1111,7 +1127,13 @@ static int tdx_handle_report_fatal_error(X86CPU *cpu,assert ((char *)tmp == message +GUEST_PANIC_INFO_TDX_MESSAGE_MAX ); } -error_report ("TD guest reports fatal error. %s\n" , message ? :"" ); +#define TDX_REPORT_FATAL_ERROR_GPA_VALID BIT_ULL(63) +if (error_code &TDX_REPORT_FATAL_ERROR_GPA_VALID ) { + gpa = vmcall->in_r13; + } + +tdx_panicked_on_fatal_error (cpu, error_code, gpa, message); +return -1 ; }
Signature · 1 line
2.34.1
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
Quoted 46 lines · Xiaoyao Li
Integrate TDX's
TDX_REPORT_FATAL_ERRORinto QEMU GuestPanic facilityOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- refine the documentation; (Markus)Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)--- qapi/run-state.json | 28 ++++++++++++++++++++-- system/runstate.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++- 3 files changed, 103 insertions(+), 3 deletions(-)qapi/run-state.json +5
-2@@ -485,10 +485,12 @@ # # @s390: s390 guest panic information type (Since: 2.12) # +# @tdx: tdx guest panic information type (Since: 8.2) +# # Since: 2.9 ## { 'enum': 'GuestPanicInformationType', - 'data': [ 'hyper-v', 's390' ] } + 'data': [ 'hyper-v', 's390', 'tdx' ] } ## # @GuestPanicInformation:@@ -503,7 +505,8 @@ 'base': {'type': 'GuestPanicInformationType'}, 'discriminator': 'type', 'data': {'hyper-v': 'GuestPanicInformationHyperV', - 's390': 'GuestPanicInformationS390'}} + 's390': 'GuestPanicInformationS390', + 'tdx' : 'GuestPanicInformationTdx'}} ## # @GuestPanicInformationHyperV:
qapi/run-state.json line 578
Quoted 10 lines
qapi/run-state.json +7
-0@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}} +## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error code
Where could a user find information on these error codes?
qapi/run-state.json line 581
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string.
"in the form of a zero-terminated string"
qapi/run-state.json line 584
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.
How is this message related to the one pointed to by @gpa?
Quoted 11 lines
qapi/run-state.json +8
-0+# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}} + ## # @MEMORY_FAILURE: #
[...]
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
Quoted 51 lines · Markus Armbruster
Quoted 49 lines · Xiaoyao Li
Integrate TDX's
TDX_REPORT_FATAL_ERRORinto QEMU GuestPanic facilityOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- refine the documentation; (Markus)Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)--- qapi/run-state.json | 28 ++++++++++++++++++++-- system/runstate.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++- 3 files changed, 103 insertions(+), 3 deletions(-)qapi/run-state.json +5
-2@@ -485,10 +485,12 @@ # # @s390: s390 guest panic information type (Since: 2.12) # +# @tdx: tdx guest panic information type (Since: 8.2) +# # Since: 2.9 ## { 'enum': 'GuestPanicInformationType', - 'data': [ 'hyper-v', 's390' ] } + 'data': [ 'hyper-v', 's390', 'tdx' ] } ## # @GuestPanicInformation:@@ -503,7 +505,8 @@ 'base': {'type': 'GuestPanicInformationType'}, 'discriminator': 'type', 'data': {'hyper-v': 'GuestPanicInformationHyperV', - 's390': 'GuestPanicInformationS390'}} + 's390': 'GuestPanicInformationS390', + 'tdx' : 'GuestPanicInformationTdx'}} ## # @GuestPanicInformationHyperV:@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}}
Quoted 9 lines
qapi/run-state.json line 578
Quoted 7 lines
qapi/run-state.json +7
-0+## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error codeWhere could a user find information on these error codes?
TDX GHCI (Guset-host-communication-Interface)spec. It defines all the TDVMCALL leaves.
0: panic;0x1 - 0xffffffff: reserved.
Quoted 5 lines
qapi/run-state.json line 581
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string."in the form of a zero-terminated string"
fixed.
Quoted 5 lines
qapi/run-state.json line 584
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.How is this message related to the one pointed to by @gpa?
In general, @message contains a brief message of the error. While @gpa (when valid) contains a verbose message.
The reason why we need both is because sometime when TD guest hits a fatal error, its memory may get corrupted so we cannot pass information via @gpa. Information in @message is passed through GPRs.
Quoted 14 lines
Quoted 11 lines
qapi/run-state.json +8
-0+# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}} + ## # @MEMORY_FAILURE: #[...]
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
Quoted 62 lines · Xiaoyao Li
Quoted 51 lines · Markus Armbruster
Quoted 49 lines · Xiaoyao Li
Integrate TDX's
TDX_REPORT_FATAL_ERRORinto QEMU GuestPanic facilityOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- refine the documentation; (Markus)Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)--- qapi/run-state.json | 28 ++++++++++++++++++++-- system/runstate.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++- 3 files changed, 103 insertions(+), 3 deletions(-)qapi/run-state.json +5
-2@@ -485,10 +485,12 @@ # # @s390: s390 guest panic information type (Since: 2.12) # +# @tdx: tdx guest panic information type (Since: 8.2) +# # Since: 2.9 ## { 'enum': 'GuestPanicInformationType', - 'data': [ 'hyper-v', 's390' ] } + 'data': [ 'hyper-v', 's390', 'tdx' ] } ## # @GuestPanicInformation:@@ -503,7 +505,8 @@ 'base': {'type': 'GuestPanicInformationType'}, 'discriminator': 'type', 'data': {'hyper-v': 'GuestPanicInformationHyperV', - 's390': 'GuestPanicInformationS390'}} + 's390': 'GuestPanicInformationS390', + 'tdx' : 'GuestPanicInformationTdx'}} ## # @GuestPanicInformationHyperV:@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}}Quoted 9 lines
qapi/run-state.json line 578
Quoted 7 lines
qapi/run-state.json +7
-0+## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error codeWhere could a user find information on these error codes?
Quoted 5 lines
TDX GHCI (Guset-host-communication-Interface)spec. It defines all the TDVMCALL leaves.
0: panic;
0x1-0xffffffff: reserved.
Would it make sense to add a reference?
Quoted 14 lines
Quoted 5 lines
qapi/run-state.json line 581
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string."in the form of a zero-terminated string"
fixed.
Quoted 5 lines
qapi/run-state.json line 584
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.How is this message related to the one pointed to by @gpa?
Quoted 6 lines
In general, @message contains a brief message of the error. While @gpa (when valid) contains a verbose message.
The reason why we need both is because sometime when TD guest hits a fatal error, its memory may get corrupted so we cannot pass information via @gpa. Information in @message is passed through GPRs.
Well, we do pass information via @gpa, always. I guess it page's contents can be corrupted.
Perhaps something like
# @message: Human-readable error message provided by the guest. Not
# to be trusted.
#
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Note that guest
# memory corruption can corrupt the page's contents.
Quoted 7 lines
Quoted 7 lines
qapi/run-state.json line 591
Quoted 7 lines
qapi/run-state.json +7
-0+# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}}
Note that my proposed doc string has the members in a different order. Recommend to use the same order here.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
qapi/run-state.json +1
-0+ ## # @MEMORY_FAILURE: #[...]
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
Quoted 64 lines · Markus Armbruster
Quoted 62 lines · Xiaoyao Li
Quoted 60 lines · Markus Armbruster
Quoted 46 lines · Xiaoyao Li
Integrate TDX's
TDX_REPORT_FATAL_ERRORinto QEMU GuestPanic facilityOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- refine the documentation; (Markus)Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)--- qapi/run-state.json | 28 ++++++++++++++++++++-- system/runstate.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++- 3 files changed, 103 insertions(+), 3 deletions(-)qapi/run-state.json +5
-2@@ -485,10 +485,12 @@ # # @s390: s390 guest panic information type (Since: 2.12) # +# @tdx: tdx guest panic information type (Since: 8.2) +# # Since: 2.9 ## { 'enum': 'GuestPanicInformationType', - 'data': [ 'hyper-v', 's390' ] } + 'data': [ 'hyper-v', 's390', 'tdx' ] } ## # @GuestPanicInformation:@@ -503,7 +505,8 @@ 'base': {'type': 'GuestPanicInformationType'}, 'discriminator': 'type', 'data': {'hyper-v': 'GuestPanicInformationHyperV', - 's390': 'GuestPanicInformationS390'}} + 's390': 'GuestPanicInformationS390', + 'tdx' : 'GuestPanicInformationTdx'}} ## # @GuestPanicInformationHyperV:qapi/run-state.json line 578
Quoted 10 lines
qapi/run-state.json +7
-0@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}} +## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error codeWhere could a user find information on these error codes?
Quoted 7 lines
Quoted 5 lines
TDX GHCI (Guset-host-communication-Interface)spec. It defines all the TDVMCALL leaves.
0: panic;
0x1-0xffffffff: reserved.Would it make sense to add a reference?
https://cdrdv2.intel.com/v1/dl/getContent/726792
Quoted 17 lines
Quoted 17 lines
Quoted 5 lines
qapi/run-state.json line 581
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string."in the form of a zero-terminated string"
fixed.
Quoted 5 lines
qapi/run-state.json line 584
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.How is this message related to the one pointed to by @gpa?
In general, @message contains a brief message of the error. While @gpa (when valid) contains a verbose message.
Quoted 6 lines
Quoted 3 lines
The reason why we need both is because sometime when TD guest hits a fatal error, its memory may get corrupted so we cannot pass information via @gpa. Information in @message is passed through GPRs.
Well, we do pass information via @gpa, always. I guess it page's contents can be corrupted.
No. It's not always. the bit 63 of the error code is "GPA valid" bit. @gpa is valid only when bit 63 of error code is 1.
And current Linux TD guest implementation doesn't use @gpa at all.
https://github.com/torvalds/linux/blob/45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663/arch/x86/coco/tdx/tdx.c#L131
Quoted 28 lines
Perhaps something like
# @message: Human-readable error message provided by the guest. Not
# to be trusted.
#
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Note that guest
# memory corruption can corrupt the page's contents.Quoted 7 lines
Quoted 7 lines
qapi/run-state.json line 591
Quoted 7 lines
qapi/run-state.json +7
-0+# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}}Note that my proposed doc string has the members in a different order. Recommend to use the same order here.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
qapi/run-state.json +1
-0+ ## # @MEMORY_FAILURE: #[...]
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
Quoted 27 lines · Xiaoyao Li
Quoted 26 lines · Markus Armbruster
Quoted 24 lines · Xiaoyao Li
Quoted 23 lines · Markus Armbruster
Quoted 21 lines · Xiaoyao Li
Integrate TDX's
TDX_REPORT_FATAL_ERRORinto QEMU GuestPanic facilityOriginated-from: Isaku Yamahata <[email protected]>
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- refine the documentation; (Markus)Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)--- qapi/run-state.json | 28 ++++++++++++++++++++-- system/runstate.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++- 3 files changed, 103 insertions(+), 3 deletions(-)qapi/run-state.json +0
-0
[...]
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 4 lines
Quoted 4 lines
qapi/run-state.json +0
-0@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}}Quoted 9 lines
qapi/run-state.json line 578
Quoted 7 lines
qapi/run-state.json +7
-0+## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error codeWhere could a user find information on these error codes?
TDX GHCI (Guset-host-communication-Interface)spec. It defines all the TDVMCALL leaves.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
0: panic;
0x1-0xffffffff: reserved.Would it make sense to add a reference?
URLs have this annoying tendency to rot.
What about
# @error-code: Error code as defined in "Guest-Hypervisor Communication
# Interface (GHCI) Specification for Intel TDX 1.5"
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 5 lines
qapi/run-state.json line 581
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string."in the form of a zero-terminated string"
fixed.
Quoted 5 lines
qapi/run-state.json line 584
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.How is this message related to the one pointed to by @gpa?
In general, @message contains a brief message of the error. While @gpa (when valid) contains a verbose message.
The reason why we need both is because sometime when TD guest hits a fatal error, its memory may get corrupted so we cannot pass information via @gpa. Information in @message is passed through GPRs.
Quoted 8 lines
Quoted 2 lines
Well, we do pass information via @gpa, always. I guess it page's contents can be corrupted.
No. It's not always. the bit 63 of the error code is "GPA valid" bit. @gpa is valid only when bit 63 of error code is 1.
And current Linux TD guest implementation doesn't use @gpa at all.
https://github.com/torvalds/linux/blob/45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663/arch/x86/coco/tdx/tdx.c#L131
Aha!
Why would we want to include @gpa when the "GPA valid" bit is off?
If we do want it, then
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Valid when the
# "GPA valid" bit is set in @error-code.
If we don't, then make @gpa optional, present when valid, and document it like this:
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Present when the
# "GPA valid" bit is set in @error-code.
Quoted 28 lines
Quoted 28 lines
Perhaps something like
# @message: Human-readable error message provided by the guest. Not
# to be trusted.
#
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Note that guest
# memory corruption can corrupt the page's contents.Quoted 7 lines
Quoted 7 lines
qapi/run-state.json line 591
Quoted 7 lines
qapi/run-state.json +7
-0+# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}}Note that my proposed doc string has the members in a different order. Recommend to use the same order here.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
qapi/run-state.json +1
-0+ ## # @MEMORY_FAILURE: #[...]
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
On 2/27/2024 9:09 PM, Markus Armbruster wrote:
[...]
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
qapi/run-state.json +0
-0@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}}Quoted 9 lines
qapi/run-state.json line 578
Quoted 7 lines
qapi/run-state.json +7
-0+## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error codeWhere could a user find information on these error codes?
Quoted 5 lines
TDX GHCI (Guset-host-communication-Interface)spec. It defines all the TDVMCALL leaves.
0: panic;
0x1-0xffffffff: reserved.Would it make sense to add a reference?
Quoted 8 lines
Quoted 1 line
URLs have this annoying tendency to rot.
What about
# @error-code: Error code as defined in "Guest-Hypervisor Communication
# Interface (GHCI) Specification for Intel TDX 1.5"
I think it gets mentioned at the beginning of @GuestPanicInformationTdx
TDX Guest panic information specific to TDX GHCI TDG.VP.VMCALL<ReportFatalError>.
Do we still to mention it in every single member?
Quoted 40 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 5 lines
qapi/run-state.json line 581
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string."in the form of a zero-terminated string"
fixed.
Quoted 5 lines
qapi/run-state.json line 584
Quoted 3 lines
qapi/run-state.json +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.How is this message related to the one pointed to by @gpa?
In general, @message contains a brief message of the error. While @gpa (when valid) contains a verbose message.
The reason why we need both is because sometime when TD guest hits a fatal error, its memory may get corrupted so we cannot pass information via @gpa. Information in @message is passed through GPRs.
Quoted 8 lines
Quoted 2 lines
Well, we do pass information via @gpa, always. I guess it page's contents can be corrupted.
No. It's not always. the bit 63 of the error code is "GPA valid" bit. @gpa is valid only when bit 63 of error code is 1.
And current Linux TD guest implementation doesn't use @gpa at all.
https://github.com/torvalds/linux/blob/45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663/arch/x86/coco/tdx/tdx.c#L131Aha!
Why would we want to include @gpa when the "GPA valid" bit is off?
If we do want it, then
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Valid when the
# "GPA valid" bit is set in @error-code.
Quoted 6 lines
If we don't, then make @gpa optional, present when valid, and document it like this:
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Present when the
# "GPA valid" bit is set in @error-code.
I will go this direction.
thanks!
Quoted 30 lines
Quoted 28 lines
Quoted 28 lines
Perhaps something like
# @message: Human-readable error message provided by the guest. Not
# to be trusted.
#
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Note that guest
# memory corruption can corrupt the page's contents.Quoted 7 lines
Quoted 7 lines
qapi/run-state.json line 591
Quoted 7 lines
qapi/run-state.json +7
-0+# +# Since: 9.0 +## +{'struct': 'GuestPanicInformationTdx', + 'data': {'error-code': 'uint64', + 'gpa': 'uint64', + 'message': 'str'}}Note that my proposed doc string has the members in a different order. Recommend to use the same order here.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
qapi/run-state.json +1
-0+ ## # @MEMORY_FAILURE: #[...]
Re: [PATCH v4 53/66] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
Quoted 33 lines · Xiaoyao Li
On 2/27/2024 9:09 PM, Markus Armbruster wrote:
[...]Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
(quoted diff) +0
-0@@ -566,6 +569,27 @@ 'psw-addr': 'uint64', 'reason': 'S390CrashReason'}}Quoted 9 lines
Review line 578
Quoted 7 lines
(quoted diff) +7
-0+## +# @GuestPanicInformationTdx: +# +# TDX Guest panic information specific to TDX GCHI +# TDG.VP.VMCALL<ReportFatalError>. +# +# @error-code: TD-specific error codeWhere could a user find information on these error codes?
Quoted 5 lines
TDX GHCI (Guset-host-communication-Interface)spec. It defines all the TDVMCALL leaves.
0: panic;
0x1-0xffffffff: reserved.Would it make sense to add a reference?
Quoted 8 lines
Quoted 1 line
URLs have this annoying tendency to rot.
What about
# @error-code: Error code as defined in "Guest-Hypervisor Communication
# Interface (GHCI) Specification for Intel TDX 1.5"
Quoted 6 lines
I think it gets mentioned at the beginning of @GuestPanicInformationTdx
TDX Guest panic information specific to TDX GHCI TDG.VP.VMCALL<ReportFatalError>.
Do we still to mention it in every single member?
No, I didn't recognize the alphabet soup there as a reference :)
Let me try again:
##
# @GuestPanicInformationTdx:
#
# TDX Guest panic information specific to TDX, as specified in the # "Guest-Hypervisor Communication Interface (GHCI) Specification",
# section TDG.VP.VMCALL<ReportFatalError>. # # @error-code: TD-specific error code # [...] # # Since: 9.0 ##
Quoted 43 lines
Quoted 43 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 5 lines
Review line 581
Quoted 3 lines
(quoted diff) +3
-0+# +# @gpa: guest-physical address of a page that contains additional +# error data, in forms of zero-terminated string."in the form of a zero-terminated string"
fixed.
Quoted 5 lines
Review line 584
Quoted 3 lines
(quoted diff) +3
-0+# +# @message: Human-readable error message provided by the guest. Not +# to be trusted.How is this message related to the one pointed to by @gpa?
In general, @message contains a brief message of the error. While @gpa (when valid) contains a verbose message.
The reason why we need both is because sometime when TD guest hits a fatal error, its memory may get corrupted so we cannot pass information via @gpa. Information in @message is passed through GPRs.
Quoted 8 lines
Quoted 2 lines
Well, we do pass information via @gpa, always. I guess it page's contents can be corrupted.
No. It's not always. the bit 63 of the error code is "GPA valid" bit. @gpa is valid only when bit 63 of error code is 1.
And current Linux TD guest implementation doesn't use @gpa at all.
https://github.com/torvalds/linux/blob/45ec2f5f6ed3ec3a79ba1329ad585497cdcbe663/arch/x86/coco/tdx/tdx.c#L131Aha!
Why would we want to include @gpa when the "GPA valid" bit is off?
If we do want it, then
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Valid when the
# "GPA valid" bit is set in @error-code.If we don't, then make @gpa optional, present when valid, and document it like this:
Quoted 7 lines
Quoted 3 lines
# @gpa: guest-physical address of a page that contains more verbose
# error information, as zero-terminated string. Present when the
# "GPA valid" bit is set in @error-code.I will go this direction.
thanks!
You're welcome!
[...]
PATCH v4 54/66 pci-host/q35: Move PAM initialization above SMRAM initialization
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
In mch_realize(), process PAM initialization before SMRAM initialization so that later patch can skill all the SMRAM related with a single check.
--- hw/pci-host/q35.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
hw/pci-host/q35.c +10 -9
@@ -568,6 +568,16 @@ static void mch_realize(PCIDevice *d, Error **errp)/* setup pci memory mapping */ pc_pci_as_mapping_init (mch->system_memory, mch->pci_address_space); +/* PAM */ +init_pam (&mch->pam_regions[0 ],OBJECT (mch), mch->ram_memory, + mch->system_memory, mch->pci_address_space, +PAM_BIOS_BASE ,PAM_BIOS_SIZE ); +for (i =0 ; i <ARRAY_SIZE (mch->pam_regions) -1 ; ++i) { +init_pam (&mch->pam_regions[i +1 ],OBJECT (mch), mch->ram_memory, + mch->system_memory, mch->pci_address_space, +PAM_EXPAN_BASE + i *PAM_EXPAN_SIZE ,PAM_EXPAN_SIZE ); + } +/* if *disabled* show SMRAM to all CPUs */ memory_region_init_alias(&mch->smram_region, OBJECT(mch), "smram-region", mch->pci_address_space, MCH_HOST_BRIDGE_SMRAM_C_BASE,
@@ -634,15 +644,6 @@ static void mch_realize(PCIDevice *d, Error **errp)object_property_add_const_link (qdev_get_machine (),"smram" ,OBJECT (&mch->smram)); - -init_pam (&mch->pam_regions[0 ],OBJECT (mch), mch->ram_memory, - mch->system_memory, mch->pci_address_space, -PAM_BIOS_BASE ,PAM_BIOS_SIZE ); -for (i =0 ; i <ARRAY_SIZE (mch->pam_regions) -1 ; ++i) { -init_pam (&mch->pam_regions[i +1 ],OBJECT (mch), mch->ram_memory, - mch->system_memory, mch->pci_address_space, -PAM_EXPAN_BASE + i *PAM_EXPAN_SIZE ,PAM_EXPAN_SIZE ); - } }uint64_t mch_mcfg_base (void )
Signature · 1 line
2.34.1
PATCH v4 55/66 q35: Introduce smm_ranges property for q35-pci-host
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Add a q35 property to check whether or not SMM ranges, e.g. SMRAM, TSEG, etc... exist for the target platform. TDX doesn't support SMM and doesn't play nice with QEMU modifying related guest memory ranges.
--- hw/i386/pc_q35.c | 2 ++ hw/pci-host/q35.c | 42 +++++++++++++++++++++++++++------------ include/hw/i386/pc.h | 1 + include/hw/pci-host/q35.h | 1 + 4 files changed, 33 insertions(+), 13 deletions(-)
hw/i386/pc_q35.c +2 -0
@@ -236,6 +236,8 @@ static void pc_q35_init(MachineState *machine) x86ms->above_4g_mem_size, NULL);object_property_set_bool (phb,PCI_HOST_BYPASS_IOMMU , pcms->default_bus_bypass_iommu,NULL ); +object_property_set_bool (phb,PCI_HOST_PROP_SMM_RANGES , +x86_machine_is_smm_enabled (x86ms),NULL );sysbus_realize_and_unref (SYS_BUS_DEVICE (phb), &error_fatal);/* pci */
hw/pci-host/q35.c +29 -13
82 lines not on this page · raw message
include/hw/i386/pc.h +1 -0
@@ -165,6 +165,7 @@ void pc_guest_info_init(PCMachineState *pcms);#define PCI_HOST_PROP_PCI_HOLE64_SIZE "pci-hole64-size"#define PCI_HOST_BELOW_4G_MEM_SIZE "below-4g-mem-size"#define PCI_HOST_ABOVE_4G_MEM_SIZE "above-4g-mem-size" +#define PCI_HOST_PROP_SMM_RANGES "smm-ranges" void pc_pci_as_mapping_init(MemoryRegion *system_memory,
include/hw/pci-host/q35.h +1 -0
@@ -50,6 +50,7 @@ struct MCHPCIState {MemoryRegion tseg_blackhole, tseg_window;MemoryRegion smbase_blackhole, smbase_window;bool has_smram_at_smbase; +bool has_smm_ranges;Range pci_hole;uint64_t below_4g_mem_size;uint64_t above_4g_mem_size;
Signature · 1 line
2.34.1
PATCH v4 56/66 i386/tdx: Disable SMM for TDX VMs
In reply to Xiaoyao LiTDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because VMM cannot manipulate TDX VM's memory.
Disable SMM for TDX VMs and error out if user requests to enable SMM.
--- target/i386/kvm/tdx.c | 8 ++++++++ 1 file changed, 8 insertions(+)
target/i386/kvm/tdx.c +8 -0
@@ -695,11 +695,19 @@ static Notifier tdx_machine_done_notify = {int tdx_kvm_init (MachineState *ms,Error **errp) { +X86MachineState *x86ms =X86_MACHINE (ms);TdxGuest *tdx =TDX_GUEST (OBJECT (ms->cgs));int r =0 ; ms->require_guest_memfd = true; +if (x86ms->smm ==ON_OFF_AUTO_AUTO ) { + x86ms->smm =ON_OFF_AUTO_OFF ; + }else if (x86ms->smm ==ON_OFF_AUTO_ON ) { +error_setg (errp,"TDX VM doesn't support SMM" ); +return -EINVAL ; + } +if (!tdx_caps) { r =get_tdx_capabilities (errp);if (r) {
Signature · 1 line
2.34.1
PATCH v4 57/66 i386/tdx: Disable PIC for TDX VMs
In reply to Xiaoyao LiLegacy PIC (8259) cannot be supported for TDX VMs since TDX module doesn't allow directly interrupt injection. Using posted interrupts for the PIC is not a viable option as the guest BIOS/kernel will not do EOI for PIC IRQs, i.e. will leave the vIRR bit set.
Hence disable PIC for TDX VMs and error out if user wants PIC.
--- target/i386/kvm/tdx.c | 7 +++++++ 1 file changed, 7 insertions(+)
target/i386/kvm/tdx.c +7 -0
@@ -708,6 +708,13 @@ int tdx_kvm_init(MachineState *ms, Error **errp)return -EINVAL ; } +if (x86ms->pic ==ON_OFF_AUTO_AUTO ) { + x86ms->pic =ON_OFF_AUTO_OFF ; + }else if (x86ms->pic ==ON_OFF_AUTO_ON ) { +error_setg (errp,"TDX VM doesn't support PIC" ); +return -EINVAL ; + } +if (!tdx_caps) { r =get_tdx_capabilities (errp);if (r) {
Signature · 1 line
2.34.1
PATCH v4 58/66 i386/tdx: Don't allow system reset for TDX VMs
In reply to Xiaoyao LiTDX CPU state is protected and thus vcpu state cann't be reset by VMM.
--- target/i386/kvm/kvm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/i386/kvm/kvm.c +1 -1
@@ -5686,7 +5686,7 @@ bool kvm_has_waitpkg(void)bool kvm_arch_cpu_check_are_resettable (void ) { -return !sev_es_enabled (); +return !sev_es_enabled () && !is_tdx_vm (); }#define ARCH_REQ_XCOMP_GUEST_PERM 0x1025
Signature · 1 line
2.34.1
PATCH v4 59/66 i386/tdx: LMCE is not supported for TDX
In reply to Xiaoyao LiLMCE is not supported TDX since KVM doesn't provide emulation for MSR_IA32_FEAT_CTL.
--- target/i386/kvm/kvm-cpu.c | 5 +++++ 1 file changed, 5 insertions(+)
target/i386/kvm/kvm-cpu.c +5 -0
@@ -15,6 +15,7 @@#include "sysemu/sysemu.h" #include "hw/boards.h" +#include "tdx.h" #include "kvm_i386.h" #include "hw/core/accel-cpu.h"
@@ -60,6 +61,10 @@ static bool lmce_supported(void)if (kvm_ioctl (kvm_state,KVM_X86_GET_MCE_CAP_SUPPORTED , &mce_cap) <0 ) {return false; } + +if (is_tdx_vm ()) +return false; +return !!(mce_cap &MCG_LMCE_P ); }
Signature · 1 line
2.34.1
PATCH v4 60/66 hw/i386: add eoi_intercept_unsupported member to X86MachineState
In reply to Xiaoyao LiAdd a new bool member, eoi_intercept_unsupported, to X86MachineState with default value false. Set true for TDX VM.
Inability to intercept eoi causes impossibility to emulate level triggered interrupt to be re-injected when level is still kept active. which affects interrupt controller emulation.
--- hw/i386/x86.c | 1 + include/hw/i386/x86.h | 1 + target/i386/kvm/tdx.c | 2 ++ 3 files changed, 4 insertions(+)
hw/i386/x86.c +1 -0
@@ -1422,6 +1422,7 @@ static void x86_machine_initfn(Object *obj) x86ms->oem_table_id =g_strndup (ACPI_BUILD_APPNAME8 ,8 ); x86ms->bus_lock_ratelimit =0 ; x86ms->above_4g_mem_start =4 * GiB; + x86ms->eoi_intercept_unsupported = false; }static void x86_machine_class_init (ObjectClass *oc,void *data)
include/hw/i386/x86.h +1 -0
@@ -59,6 +59,7 @@ struct X86MachineState {/* CPU and apic information: */ bool apic_xrupt_override; +bool eoi_intercept_unsupported;unsigned pci_irq_mask;unsigned apic_id_limit;uint16_t boot_cpus;
target/i386/kvm/tdx.c +2 -0
@@ -715,6 +715,8 @@ int tdx_kvm_init(MachineState *ms, Error **errp)return -EINVAL ; } + x86ms->eoi_intercept_unsupported = true; +if (!tdx_caps) { r =get_tdx_capabilities (errp);if (r) {
Signature · 1 line
2.34.1
PATCH v4 61/66 hw/i386: add option to forcibly report edge trigger in acpi tables
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
When level trigger isn't supported on x86 platform, forcibly report edge trigger in acpi tables.
--- hw/i386/acpi-build.c | 99 ++++++++++++++++++++++++++++--------------- hw/i386/acpi-common.c | 50 ++++++++++++++++------ 2 files changed, 104 insertions(+), 45 deletions(-)
hw/i386/acpi-build.c +66 -33
@@ -975,7 +975,8 @@ static void build_dbg_aml(Aml *table)aml_append (table, scope); } -static Aml *build_link_dev (const char *name,uint8_t uid,Aml *reg) +static Aml *build_link_dev (const char *name,uint8_t uid,Aml *reg, +bool level_trigger_unsupported) {Aml *dev;Aml *crs;
@@ -987,7 +988,10 @@ static Aml *build_link_dev(const char *name, uint8_t uid, Aml *reg)aml_append (dev,aml_name_decl ("_UID" ,aml_int (uid))); crs =aml_resource_template (); -aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_LEVEL ,AML_ACTIVE_HIGH , +aml_append (crs,aml_interrupt (AML_CONSUMER , + level_trigger_unsupported ? +AML_EDGE :AML_LEVEL , +AML_ACTIVE_HIGH ,AML_SHARED , irqs,ARRAY_SIZE (irqs)));aml_append (dev,aml_name_decl ("_PRS" , crs));
@@ -1011,7 +1015,8 @@ static Aml *build_link_dev(const char *name, uint8_t uid, Aml *reg)return dev; } -static Aml *build_gsi_link_dev (const char *name,uint8_t uid,uint8_t gsi) +static Aml *build_gsi_link_dev (const char *name,uint8_t uid, +uint8_t gsi,bool level_trigger_unsupported) {Aml *dev;Aml *crs;
@@ -1024,7 +1029,10 @@ static Aml *build_gsi_link_dev(const char *name, uint8_t uid, uint8_t gsi) crs =aml_resource_template (); irqs = gsi; -aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_LEVEL ,AML_ACTIVE_HIGH , +aml_append (crs,aml_interrupt (AML_CONSUMER , + level_trigger_unsupported ? +AML_EDGE :AML_LEVEL , +AML_ACTIVE_HIGH ,AML_SHARED , &irqs,1 ));aml_append (dev,aml_name_decl ("_PRS" , crs));
@@ -1043,7 +1051,7 @@ static Aml *build_gsi_link_dev(const char *name, uint8_t uid, uint8_t gsi) }/* _CRS method - get current settings */ -static Aml *build_iqcr_method (bool is_piix4) +static Aml *build_iqcr_method (bool is_piix4,bool level_trigger_unsupported) {Aml *if_ctx;uint32_t irqs;
@@ -1051,7 +1059,9 @@ static Aml *build_iqcr_method(bool is_piix4)Aml *crs =aml_resource_template (); irqs =0 ; -aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_LEVEL , +aml_append (crs,aml_interrupt (AML_CONSUMER , + level_trigger_unsupported ? +AML_EDGE :AML_LEVEL ,AML_ACTIVE_HIGH ,AML_SHARED , &irqs,1 ));aml_append (method,aml_name_decl ("PRR0" , crs));
@@ -1085,7 +1095,7 @@ static Aml *build_irq_status_method(void)return method; } -static void build_piix4_pci0_int (Aml *table) +static void build_piix4_pci0_int (Aml *table,bool level_trigger_unsupported) {Aml *dev;Aml *crs;
@@ -1098,12 +1108,16 @@ static void build_piix4_pci0_int(Aml *table)aml_append (sb_scope, pci0_scope);aml_append (sb_scope,build_irq_status_method ()); -aml_append (sb_scope,build_iqcr_method (true)); +aml_append (sb_scope,build_iqcr_method (true, level_trigger_unsupported)); -aml_append (sb_scope,build_link_dev ("LNKA" ,0 ,aml_name ("PRQ0" ))); -aml_append (sb_scope,build_link_dev ("LNKB" ,1 ,aml_name ("PRQ1" ))); -aml_append (sb_scope,build_link_dev ("LNKC" ,2 ,aml_name ("PRQ2" ))); -aml_append (sb_scope,build_link_dev ("LNKD" ,3 ,aml_name ("PRQ3" ))); +aml_append (sb_scope,build_link_dev ("LNKA" ,0 ,aml_name ("PRQ0" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKB" ,1 ,aml_name ("PRQ1" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKC" ,2 ,aml_name ("PRQ2" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKD" ,3 ,aml_name ("PRQ3" ), + level_trigger_unsupported)); dev =aml_device ("LNKS" ); {
@@ -1112,7 +1126,9 @@ static void build_piix4_pci0_int(Aml *table) crs =aml_resource_template (); irqs =9 ; -aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_LEVEL , +aml_append (crs,aml_interrupt (AML_CONSUMER , + level_trigger_unsupported ? +AML_EDGE :AML_LEVEL ,AML_ACTIVE_HIGH ,AML_SHARED , &irqs,1 ));aml_append (dev,aml_name_decl ("_PRS" , crs));
@@ -1198,7 +1214,7 @@ static Aml *build_q35_routing_table(const char *str)return pkg; } -static void build_q35_pci0_int (Aml *table) +static void build_q35_pci0_int (Aml *table,bool level_trigger_unsupported) {Aml *method;Aml *sb_scope =aml_scope ("_SB" );
@@ -1237,25 +1253,41 @@ static void build_q35_pci0_int(Aml *table)aml_append (sb_scope, pci0_scope);aml_append (sb_scope,build_irq_status_method ()); -aml_append (sb_scope,build_iqcr_method (false)); +aml_append (sb_scope,build_iqcr_method (false, level_trigger_unsupported)); -aml_append (sb_scope,build_link_dev ("LNKA" ,0 ,aml_name ("PRQA" ))); -aml_append (sb_scope,build_link_dev ("LNKB" ,1 ,aml_name ("PRQB" ))); -aml_append (sb_scope,build_link_dev ("LNKC" ,2 ,aml_name ("PRQC" ))); -aml_append (sb_scope,build_link_dev ("LNKD" ,3 ,aml_name ("PRQD" ))); -aml_append (sb_scope,build_link_dev ("LNKE" ,4 ,aml_name ("PRQE" ))); -aml_append (sb_scope,build_link_dev ("LNKF" ,5 ,aml_name ("PRQF" ))); -aml_append (sb_scope,build_link_dev ("LNKG" ,6 ,aml_name ("PRQG" ))); -aml_append (sb_scope,build_link_dev ("LNKH" ,7 ,aml_name ("PRQH" ))); +aml_append (sb_scope,build_link_dev ("LNKA" ,0 ,aml_name ("PRQA" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKB" ,1 ,aml_name ("PRQB" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKC" ,2 ,aml_name ("PRQC" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKD" ,3 ,aml_name ("PRQD" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKE" ,4 ,aml_name ("PRQE" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKF" ,5 ,aml_name ("PRQF" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKG" ,6 ,aml_name ("PRQG" ), + level_trigger_unsupported)); +aml_append (sb_scope,build_link_dev ("LNKH" ,7 ,aml_name ("PRQH" ), + level_trigger_unsupported)); -aml_append (sb_scope,build_gsi_link_dev ("GSIA" ,0x10 ,0x10 )); -aml_append (sb_scope,build_gsi_link_dev ("GSIB" ,0x11 ,0x11 )); -aml_append (sb_scope,build_gsi_link_dev ("GSIC" ,0x12 ,0x12 )); -aml_append (sb_scope,build_gsi_link_dev ("GSID" ,0x13 ,0x13 )); -aml_append (sb_scope,build_gsi_link_dev ("GSIE" ,0x14 ,0x14 )); -aml_append (sb_scope,build_gsi_link_dev ("GSIF" ,0x15 ,0x15 )); -aml_append (sb_scope,build_gsi_link_dev ("GSIG" ,0x16 ,0x16 )); -aml_append (sb_scope,build_gsi_link_dev ("GSIH" ,0x17 ,0x17 )); +aml_append (sb_scope,build_gsi_link_dev ("GSIA" ,0x10 ,0x10 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSIB" ,0x11 ,0x11 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSIC" ,0x12 ,0x12 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSID" ,0x13 ,0x13 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSIE" ,0x14 ,0x14 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSIF" ,0x15 ,0x15 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSIG" ,0x16 ,0x16 , + level_trigger_unsupported)); +aml_append (sb_scope,build_gsi_link_dev ("GSIH" ,0x17 ,0x17 , + level_trigger_unsupported));aml_append (table, sb_scope); }
@@ -1436,6 +1468,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,PCMachineState *pcms =PC_MACHINE (machine);PCMachineClass *pcmc =PC_MACHINE_GET_CLASS (machine);X86MachineState *x86ms =X86_MACHINE (machine); +bool level_trigger_unsupported = x86ms->eoi_intercept_unsupported;AcpiMcfgInfo mcfg;bool mcfg_valid = !!acpi_get_mcfg (&mcfg);uint32_t nr_mem = machine->ram_slots;
@@ -1468,7 +1501,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,if (pm->pcihp_bridge_en || pm->pcihp_root_en) {build_x86_acpi_pci_hotplug (dsdt, pm->pcihp_io_base); } -build_piix4_pci0_int (dsdt); +build_piix4_pci0_int (dsdt, level_trigger_unsupported); }else if (q35) { sb_scope =aml_scope ("_SB" ); dev =aml_device ("PCI0" );
@@ -1512,7 +1545,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,if (pm->pcihp_bridge_en) {build_x86_acpi_pci_hotplug (dsdt, pm->pcihp_io_base); } -build_q35_pci0_int (dsdt); +build_q35_pci0_int (dsdt, level_trigger_unsupported); }if (misc->has_hpet) {
hw/i386/acpi-common.c +38 -12
62 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v4 62/66 i386/tdx: Don't synchronize guest tsc for TDs
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TSC of TDs is not accessible and KVM doesn't allow access of MSR_IA32_TSC for TDs. To avoid the assert() in kvm_get_tsc, make kvm_synchronize_all_tsc() noop for TDs,
--- target/i386/kvm/kvm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/i386/kvm/kvm.c +1 -1
@@ -278,7 +278,7 @@ void kvm_synchronize_all_tsc(void) {CPUState *cpu; -if (kvm_enabled ()) { +if (kvm_enabled () && !is_tdx_vm ()) {CPU_FOREACH (cpu) {run_on_cpu (cpu, do_kvm_synchronize_tsc,RUN_ON_CPU_NULL ); }
Signature · 1 line
2.34.1
PATCH v4 63/66 i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs
In reply to Xiaoyao LiFor TDs, only MSR_IA32_UCODE_REV in kvm_init_msrs() can be configured by VMM, while the features enumerated/controlled by other MSRs except MSR_IA32_UCODE_REV in kvm_init_msrs() are not under control of VMM.
Only configure MSR_IA32_UCODE_REV for TDs.
--- target/i386/kvm/kvm.c | 44 ++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 21 deletions(-)
target/i386/kvm/kvm.c +23 -21
@@ -3283,32 +3283,34 @@ static void kvm_init_msrs(X86CPU *cpu)CPUX86State *env = &cpu->env;kvm_msr_buf_reset (cpu); -if (has_msr_arch_capabs) { -kvm_msr_entry_add (cpu,MSR_IA32_ARCH_CAPABILITIES , - env->features[FEAT_ARCH_CAPABILITIES ]); - } - -if (has_msr_core_capabs) { -kvm_msr_entry_add (cpu,MSR_IA32_CORE_CAPABILITY , - env->features[FEAT_CORE_CAPABILITY ]); - } - -if (has_msr_perf_capabs && cpu->enable_pmu) { -kvm_msr_entry_add_perf (cpu, env->features); + +if (!is_tdx_vm ()) { +if (has_msr_arch_capabs) { +kvm_msr_entry_add (cpu,MSR_IA32_ARCH_CAPABILITIES , + env->features[FEAT_ARCH_CAPABILITIES ]); + } + +if (has_msr_core_capabs) { +kvm_msr_entry_add (cpu,MSR_IA32_CORE_CAPABILITY , + env->features[FEAT_CORE_CAPABILITY ]); + } + +if (has_msr_perf_capabs && cpu->enable_pmu) { +kvm_msr_entry_add_perf (cpu, env->features); + } + +/* +* Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but +* all kernels with MSR features should have them. +*/ +if (kvm_feature_msrs &&cpu_has_vmx (env)) { +kvm_msr_entry_add_vmx (cpu, env->features); + } }if (has_msr_ucode_rev) {kvm_msr_entry_add (cpu,MSR_IA32_UCODE_REV , cpu->ucode_rev); } - -/* -* Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but -* all kernels with MSR features should have them. -*/ -if (kvm_feature_msrs &&cpu_has_vmx (env)) { -kvm_msr_entry_add_vmx (cpu, env->features); - } -assert (kvm_buf_set_msrs (cpu) ==0 ); }
Signature · 1 line
2.34.1
PATCH v4 64/66 i386/tdx: Skip kvm_put_apicbase() for TDs
In reply to Xiaoyao LiKVM doesn't allow wirting to MSR_IA32_APICBASE for TDs.
--- target/i386/kvm/kvm.c | 5 +++++ 1 file changed, 5 insertions(+)
target/i386/kvm/kvm.c +5 -0
@@ -3065,6 +3065,11 @@ void kvm_put_apicbase(X86CPU *cpu, uint64_t value) {int ret; +/* TODO: Allow accessing guest state for debug TDs. */ +if (is_tdx_vm ()) { +return ; + } + ret =kvm_put_one_msr (cpu,MSR_IA32_APICBASE , value);assert (ret ==1 ); }
Signature · 1 line
2.34.1
PATCH v4 65/66 i386/tdx: Don't get/put guest state for TDX VMs
In reply to Xiaoyao LiFrom: Sean Christopherson <[email protected]>
Don't get/put state of TDX VMs since accessing/mutating guest state of production TDs is not supported.
Note, it will be allowed for a debug TD. Corresponding support will be introduced when debug TD support is implemented in the future.
--- target/i386/kvm/kvm.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
target/i386/kvm/kvm.c +11 -0
@@ -4633,6 +4633,11 @@ int kvm_arch_put_registers(CPUState *cpu, int level)assert (cpu_is_stopped (cpu) ||qemu_cpu_is_self (cpu)); +/* TODO: Allow accessing guest state for debug TDs. */ +if (is_tdx_vm ()) { +return 0 ; + } +/* * Put MSR_IA32_FEATURE_CONTROL first, this ensures the VM gets out of VMX * root operation upon vCPU reset. kvm_put_msr_feature_control() should also
@@ -4733,6 +4738,12 @@ int kvm_arch_get_registers(CPUState *cs)if (ret <0 ) { goto out; } + +/* TODO: Allow accessing guest state for debug TDs. */ +if (is_tdx_vm ()) { +return 0 ; + } + ret =kvm_getput_regs (cpu,0 );if (ret <0 ) { goto out;
Signature · 1 line
2.34.1
PATCH v4 66/66 docs: Add TDX documentation
In reply to Xiaoyao LiAdd docs/system/i386/tdx.rst for TDX support, and add tdx in confidential-guest-support.rst
---
Changes since v1:
- Add prerequisite of private gmem;
- update example command to launch TD;
Changes since RFC v4:
- add the restriction that kernel-irqchip must be split
--- docs/system/confidential-guest-support.rst | 1 + docs/system/i386/tdx.rst | 113 +++++++++++++++++++++ docs/system/target-i386.rst | 1 + 3 files changed, 115 insertions(+) create mode 100644 docs/system/i386/tdx.rst
docs/system/confidential-guest-support.rst +1 -0
@@ -38,6 +38,7 @@ Supported mechanisms Currently supported confidential guest mechanisms are: * AMD Secure Encrypted Virtualization (SEV) (see :doc:`i386/amd-memory-encryption`) +* Intel Trust Domain Extension (TDX) (see :doc:`i386/tdx`) * POWER Protected Execution Facility (PEF) (see :ref:`power-papr-protected-execution-facility-pef`) * s390x Protected Virtualization (PV) (see :doc:`s390x/protvirt`)
docs/system/i386/tdx.rst new file +113 -0
113 lines not on this page · raw message
docs/system/target-i386.rst +1 -0
@@ -29,6 +29,7 @@ Architectural features i386/kvm-pv i386/sgx i386/amd-memory-encryption + i386/tdx OS requirements ~~~~~~~~~~~~~~~
Signature · 1 line
2.34.1