QEMU Guest memfd + QEMU TDX support
Subsystems (5)
v5 cover QEMU Guest memfd + QEMU TDX support
This v5 series matches with latest KVM v19 series:
https://lore.kernel.org/all/[email protected]/
This series is also available in github:
https://github.com/intel/qemu-tdx/tree/tdx-qemu-upstream-v5
It's based on one series that refacotrs cgs specific KVM functions:
https://lore.kernel.org/qemu-devel/[email protected]/
===
Patches 1 - 10 add support of guest memfd into QEMU, associating it with RAMBlock. For the VM types that require private memory (see tdx_kvm_init() in patch 14), QEMU will automatically create guest memfd for each RAM backend.
Follwoing patches 11 to 65, enables TDX support to allow creating and booting a TD (TDX VM) with QEMU.
== Limitation and future work ==
- Readonly memslot
TDX only support readonly (write protection) memslot for shared memory, but not for private memory. For simplicity, just mark readonly memslot not supported entirely for TDX.
- CPU model
We cannot create a TD with arbitrary CPU model like what for non-TDX VMs, because only a subset of features can be configured for TD.
- It's recommended to use '-cpu host' to create TD;
- '+feature/-feature' might not work as expected;
future work: To introduce specific CPU model for TDs and enhance ±features
for TDs.
- gdb suppport
gdb support to debug a TD of off-debug mode is future work.
===
Changes in v5:
guest memfd part:
- remove RAM_DEFAULT_PRIVATE that RAM_GUEST_MEMFD can serve its purpose. just make all the RAM that has guest memfd backend private by default;
- collect Reviewed-by from David;
TDX part:
- switch to use KVM_MEMORY_MAPPING to initialize TDVF into its private memory; (required by KVM v19 series);
- Add patch (28) to disable PMU for TD guest;
- various QAPI doc comment refine; (Thanks to Markus!)
- update Documentation to add section TD atttestation;
v4:
https://lore.kernel.org/qemu-devel/[email protected]/
Main changes in v4:
guest memfd part:
- Drop the patch that supports KVM_GUEST_MEMFD_ALLOW_HUGEPAGE flag because KVM side leaves it as future work;
- send patch "physmem: replace function name with __func__ in ram_block_discard_range()" separately;
- drop ram_block_convert_range() and open code it instead.
- move the supported attribute bit check into kvm_set_memory_attributes() (per request from Wei)
- Clarify more for the desgin consideration in the commit messages
TDX part:
- Re-implement get-quote support by introduing TdxQuoteGenerator object and make the socket communication non-blocking; (per Daniel)
- Various code and doc refinement;
v3:
https://lore.kernel.org/qemu-devel/[email protected]/
Main changes in v3:
gmem memfd part:
- Since KVM side renamed gmem to guest_memfd in the uapi, this version renames it accordingly;
- Drop the 'private' property of memory backend. (see comment[1]) Now QEMU decides whether need to create guest memfd based on specific vm type (or specific VM implementation, please see patch *X* and *Y*);
- Drop sw_protected_vm implementation;
TDX part:
- improve the error report in various patches by utilizing 'errp';
- drop the vm-type interface;
- rename __tdx_ioctl() to tdx_ioctl_internal();
- refine the description of 'sept-ve-disable' in qom.json;
- use base64 for mrconfigif/mrowner/mrownerconfig instread of hex-string;
- use type SocketAddress for quote-generation-service;
[1] https://lore.kernel.org/qemu-devel/[email protected]/
Chao Peng (3):
kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
kvm: handle KVM_EXIT_MEMORY_FAULT
i386/tdx: load TDVF for TD guest
Isaku Yamahata (14):
trace/kvm: Add trace for page convertion between shared and private i386/tdx: Make sept_ve_disable set by default
i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig kvm/tdx: Don't complain when converting vMMIO region to shared kvm/tdx: Ignore memory conversion to shared of unassigned region i386/tdvf: Introduce function to parse TDVF metadata i386/tdx: Populate TDVF private memory via KVM_MEMORY_MAPPING i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt> i386/tdx: handle TDG.VP.VMCALL<GetQuote>
i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall pci-host/q35: Move PAM initialization above SMRAM initialization
q35: Introduce smm_ranges property for q35-pci-host hw/i386: add option to forcibly report edge trigger in acpi tables i386/tdx: Don't synchronize guest tsc for TDs
Sean Christopherson (2):
i386/kvm: Move architectural CPUID leaf generation to separate helper i386/tdx: Don't get/put guest state for TDX VMs
Xiaoyao Li (46):
linux-headers: Update to Linux v6.8-rc5
RAMBlock: Add support of KVM private guest memfd
HostMem: Add mechanism to opt in kvm guest memfd via MachineState trace/kvm: Split address space and slot id in
trace_kvm_set_user_memory()
kvm: Introduce support for memory_attributes
physmem: Introduce ram_block_discard_guest_memfd_range() kvm/memory: Make memory type private by default if it has guest memfd
backend
*** HACK *** linux-headers: Update headers to pull in TDX API changes
i386: Introduce tdx-guest object
target/i386: Implement mc->kvm_type() to get VM type i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object i386/tdx: Adjust the supported CPUID based on TDX restrictions i386/tdx: Make Intel-PT unsupported for TD guest
i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by
tdx_caps.cpuid_config[]
i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup
kvm: Introduce kvm_arch_pre_create_vcpu()
i386/tdx: Initialize TDX before creating TD vcpus i386/tdx: Add property sept-ve-disable for tdx-guest object i386/tdx: Wire CPU features up with attributes of TD guest i386/tdx: Disable pmu for TD guest
i386/tdx: Validate TD attributes
i386/tdx: Implement user specified tsc frequency
i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM
memory: Introduce memory_region_init_ram_guest_memfd() i386/tdx: Parse TDVF metadata for TDX VM
i386/tdx: Skip BIOS shadowing setup
i386/tdx: Don't initialize pc.rom for TDX VMs
i386/tdx: Track mem_ptr for each firmware entry of TDVF i386/tdx: Track RAM entries for TDX VM
headers: Add definitions from UEFI spec for volumes, resources, etc... i386/tdx: Setup the TD HOB list
i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu i386/tdx: Finalize TDX VM
i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR> i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility i386/tdx: Disable SMM for TDX VMs
i386/tdx: Disable PIC for TDX VMs
i386/tdx: Don't allow system reset for TDX VMs
i386/tdx: LMCE is not supported for TDX
hw/i386: add eoi_intercept_unsupported member to X86MachineState i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs i386/tdx: Skip kvm_put_apicbase() for TDs
docs: Add TDX documentation
accel/kvm/kvm-all.c | 278 +++- accel/kvm/trace-events | 3 +- accel/stubs/kvm-stub.c | 5 + backends/hostmem-file.c | 1 + backends/hostmem-memfd.c | 1 + backends/hostmem-ram.c | 1 + backends/hostmem.c | 1 + configs/devices/i386-softmmu/default.mak | 1 + docs/system/confidential-guest-support.rst | 1 + docs/system/i386/tdx.rst | 143 ++ docs/system/target-i386.rst | 1 + hw/core/machine.c | 5 + hw/i386/Kconfig | 6 + hw/i386/acpi-build.c | 99 +- hw/i386/acpi-common.c | 45 +- hw/i386/meson.build | 1 + hw/i386/pc.c | 21 +- hw/i386/pc_q35.c | 2 + hw/i386/pc_sysfw.c | 7 + hw/i386/tdvf-hob.c | 147 ++ hw/i386/tdvf-hob.h | 24 + hw/i386/tdvf.c | 200 +++ hw/i386/x86.c | 51 +- hw/pci-host/q35.c | 61 +- include/exec/cpu-common.h | 2 + include/exec/memory.h | 26 +- include/exec/ram_addr.h | 2 +- include/exec/ramblock.h | 1 + include/hw/boards.h | 2 + include/hw/i386/pc.h | 1 + include/hw/i386/tdvf.h | 58 + include/hw/i386/x86.h | 2 + include/hw/pci-host/q35.h | 1 + include/standard-headers/drm/drm_fourcc.h | 10 +- include/standard-headers/linux/ethtool.h | 41 +- .../standard-headers/linux/virtio_config.h | 8 +- include/standard-headers/linux/virtio_pci.h | 68 + include/standard-headers/linux/virtio_pmem.h | 7 + include/standard-headers/uefi/uefi.h | 198 +++ include/sysemu/hostmem.h | 1 + include/sysemu/kvm.h | 8 + include/sysemu/kvm_int.h | 2 + linux-headers/asm-generic/unistd.h | 15 +- linux-headers/asm-mips/mman.h | 2 +- linux-headers/asm-mips/unistd_n32.h | 5 + linux-headers/asm-mips/unistd_n64.h | 5 + linux-headers/asm-mips/unistd_o32.h | 5 + linux-headers/asm-powerpc/unistd_32.h | 5 + linux-headers/asm-powerpc/unistd_64.h | 5 + linux-headers/asm-riscv/kvm.h | 40 + linux-headers/asm-s390/unistd_32.h | 5 + linux-headers/asm-s390/unistd_64.h | 5 + linux-headers/asm-x86/kvm.h | 89 ++ linux-headers/asm-x86/unistd_32.h | 5 + linux-headers/asm-x86/unistd_64.h | 5 + linux-headers/asm-x86/unistd_x32.h | 5 + linux-headers/linux/iommufd.h | 79 + linux-headers/linux/kvm.h | 240 +-- linux-headers/linux/userfaultfd.h | 29 +- linux-headers/linux/vfio.h | 1 + qapi/qom.json | 38 + qapi/run-state.json | 31 +- system/memory.c | 30 + system/physmem.c | 47 +- system/runstate.c | 58 + target/i386/cpu-internal.h | 9 + target/i386/cpu.c | 12 - target/i386/cpu.h | 21 + target/i386/kvm/kvm-cpu.c | 5 + target/i386/kvm/kvm.c | 596 ++++--- target/i386/kvm/kvm_i386.h | 6 + target/i386/kvm/meson.build | 2 + target/i386/kvm/tdx-quote-generator.c | 170 ++ target/i386/kvm/tdx-quote-generator.h | 95 ++ target/i386/kvm/tdx-stub.c | 18 + target/i386/kvm/tdx.c | 1371 +++++++++++++++++ target/i386/kvm/tdx.h | 81 + 77 files changed, 4180 insertions(+), 498 deletions(-) create mode 100644 docs/system/i386/tdx.rst create mode 100644 hw/i386/tdvf-hob.c create mode 100644 hw/i386/tdvf-hob.h create mode 100644 hw/i386/tdvf.c create mode 100644 include/hw/i386/tdvf.h create mode 100644 include/standard-headers/uefi/uefi.h create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.h create mode 100644 target/i386/kvm/tdx-stub.c create mode 100644 target/i386/kvm/tdx.c create mode 100644 target/i386/kvm/tdx.h
Signature · 1 line
2.34.1
PATCH v5 1/65 linux-headers: Update to Linux v6.8-rc5
Guest memfd support in QEMU requires corresponding KVM guest memfd APIs, which lands in Linux from v6.8-rc1.
--- include/standard-headers/drm/drm_fourcc.h | 10 +- include/standard-headers/linux/ethtool.h | 41 +++-- .../standard-headers/linux/virtio_config.h | 8 +- include/standard-headers/linux/virtio_pci.h | 68 +++++++++ include/standard-headers/linux/virtio_pmem.h | 7 + linux-headers/asm-generic/unistd.h | 15 +- linux-headers/asm-mips/mman.h | 2 +- linux-headers/asm-mips/unistd_n32.h | 5 + linux-headers/asm-mips/unistd_n64.h | 5 + linux-headers/asm-mips/unistd_o32.h | 5 + linux-headers/asm-powerpc/unistd_32.h | 5 + linux-headers/asm-powerpc/unistd_64.h | 5 + linux-headers/asm-riscv/kvm.h | 40 +++++ linux-headers/asm-s390/unistd_32.h | 5 + linux-headers/asm-s390/unistd_64.h | 5 + linux-headers/asm-x86/kvm.h | 3 + linux-headers/asm-x86/unistd_32.h | 5 + linux-headers/asm-x86/unistd_64.h | 5 + linux-headers/asm-x86/unistd_x32.h | 5 + linux-headers/linux/iommufd.h | 79 ++++++++++ linux-headers/linux/kvm.h | 140 +++++++----------- linux-headers/linux/userfaultfd.h | 29 +++- linux-headers/linux/vfio.h | 1 + 23 files changed, 381 insertions(+), 112 deletions(-)
include/standard-headers/drm/drm_fourcc.h +5 -5
@@ -53,7 +53,7 @@ extern "C" {* Format modifiers may change any property of the buffer, including the number * of planes and/or the required allocation size. Format modifiers are * vendor-namespaced, and as such the relationship between a fourcc code and a -* modifier is specific to the modifer being used. For example, some modifiers +* modifier is specific to the modifier being used. For example, some modifiers * may preserve meaning - such as number of planes - from the fourcc code, * whereas others may not. *
@@ -78,7 +78,7 @@ extern "C" {* format. * - Higher-level programs interfacing with KMS/GBM/EGL/Vulkan/etc: these users * see modifiers as opaque tokens they can check for equality and intersect. -* These users musn't need to know to reason about the modifier value +* These users mustn't need to know to reason about the modifier value * (i.e. they are not expected to extract information out of the modifier). * * Vendors should document their modifier usage in as much detail as
@@ -539,7 +539,7 @@ extern "C" {* This is a tiled layout using 4Kb tiles in row-major layout. * Within the tile pixels are laid out in 16 256 byte units / sub-tiles which * are arranged in four groups (two wide, two high) with column-major layout. -* Each group therefore consits out of four 256 byte units, which are also laid +* Each group therefore consists out of four 256 byte units, which are also laid * out as 2x2 column-major. * 256 byte units are made out of four 64 byte blocks of pixels, producing * either a square block or a 2:1 unit.
@@ -1102,7 +1102,7 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)*/ /* -* The top 4 bits (out of the 56 bits alloted for specifying vendor specific +* The top 4 bits (out of the 56 bits allotted for specifying vendor specific * modifiers) denote the category for modifiers. Currently we have three * categories of modifiers ie AFBC, MISC and AFRC. We can have a maximum of * sixteen different categories.
@@ -1418,7 +1418,7 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)* Amlogic FBC Memory Saving mode * * Indicates the storage is packed when pixel size is multiple of word -* boudaries, i.e. 8bit should be stored in this mode to save allocation +* boundaries, i.e. 8bit should be stored in this mode to save allocation * memory. * * This mode reduces body layout to 3072 bytes per 64x32 superblock with
include/standard-headers/linux/ethtool.h +28 -13
71 lines not on this page · raw message
include/standard-headers/linux/virtio_config.h +7 -1
@@ -52,7 +52,7 @@* rest are per-device feature bits. */ #define VIRTIO_TRANSPORT_F_START 28 -#define VIRTIO_TRANSPORT_F_END 41 +#define VIRTIO_TRANSPORT_F_END 42#ifndef VIRTIO_CONFIG_NO_LEGACY /* Do we get callbacks when the ring is completely used, even if we've
@@ -112,4 +112,10 @@* This feature indicates that the driver can reset a queue individually. */ #define VIRTIO_F_RING_RESET 40 + +/* +* This feature indicates that the device support administration virtqueues. +*/ +#define VIRTIO_F_ADMIN_VQ 41 +#endif /* _LINUX_VIRTIO_CONFIG_H */
include/standard-headers/linux/virtio_pci.h +68 -0
81 lines not on this page · raw message
include/standard-headers/linux/virtio_pmem.h +7 -0
@@ -14,6 +14,13 @@#include "standard-headers/linux/virtio_ids.h" #include "standard-headers/linux/virtio_config.h" +/* Feature bits */ +/* guest physical address range will be indicated as shared memory region 0 */ +#define VIRTIO_PMEM_F_SHMEM_REGION 0 + +/* shmid of the shared memory region corresponding to the pmem */ +#define VIRTIO_PMEM_SHMEM_REGION_ID 0 +struct virtio_pmem_config {uint64_t start;uint64_t size;
linux-headers/asm-generic/unistd.h +14 -1
@@ -829,8 +829,21 @@ __SYSCALL(__NR_futex_wait, sys_futex_wait)#define __NR_futex_requeue 456 __SYSCALL(__NR_futex_requeue, sys_futex_requeue) +#define __NR_statmount 457 +__SYSCALL(__NR_statmount, sys_statmount) + +#define __NR_listmount 458 +__SYSCALL(__NR_listmount, sys_listmount) + +#define __NR_lsm_get_self_attr 459 +__SYSCALL(__NR_lsm_get_self_attr, sys_lsm_get_self_attr) +#define __NR_lsm_set_self_attr 460 +__SYSCALL(__NR_lsm_set_self_attr, sys_lsm_set_self_attr) +#define __NR_lsm_list_modules 461 +__SYSCALL(__NR_lsm_list_modules, sys_lsm_list_modules) +#undef __NR_syscalls -#define __NR_syscalls 457 +#define __NR_syscalls 462/* * 32 bit systems traditionally used different
linux-headers/asm-mips/mman.h +1 -1
@@ -88,7 +88,7 @@#define MADV_HUGEPAGE 14/* Worth backing with hugepages */ #define MADV_NOHUGEPAGE 15/* Not worth backing with hugepages */ -#define MADV_DONTDUMP 16/* Explicity exclude from the core dump, +#define MADV_DONTDUMP 16/* Explicitly exclude from core dump, overrides the coredump filter bits */ #define MADV_DODUMP 17/* Clear the MADV_NODUMP flag */
linux-headers/asm-mips/unistd_n32.h +5 -0
@@ -385,5 +385,10 @@#define __NR_futex_wake (__NR_Linux + 454)#define __NR_futex_wait (__NR_Linux + 455)#define __NR_futex_requeue (__NR_Linux + 456) +#define __NR_statmount (__NR_Linux + 457) +#define __NR_listmount (__NR_Linux + 458) +#define __NR_lsm_get_self_attr (__NR_Linux + 459) +#define __NR_lsm_set_self_attr (__NR_Linux + 460) +#define __NR_lsm_list_modules (__NR_Linux + 461)#endif /* _ASM_UNISTD_N32_H */
linux-headers/asm-mips/unistd_n64.h +5 -0
@@ -361,5 +361,10 @@#define __NR_futex_wake (__NR_Linux + 454)#define __NR_futex_wait (__NR_Linux + 455)#define __NR_futex_requeue (__NR_Linux + 456) +#define __NR_statmount (__NR_Linux + 457) +#define __NR_listmount (__NR_Linux + 458) +#define __NR_lsm_get_self_attr (__NR_Linux + 459) +#define __NR_lsm_set_self_attr (__NR_Linux + 460) +#define __NR_lsm_list_modules (__NR_Linux + 461)#endif /* _ASM_UNISTD_N64_H */
linux-headers/asm-mips/unistd_o32.h +5 -0
@@ -431,5 +431,10 @@#define __NR_futex_wake (__NR_Linux + 454)#define __NR_futex_wait (__NR_Linux + 455)#define __NR_futex_requeue (__NR_Linux + 456) +#define __NR_statmount (__NR_Linux + 457) +#define __NR_listmount (__NR_Linux + 458) +#define __NR_lsm_get_self_attr (__NR_Linux + 459) +#define __NR_lsm_set_self_attr (__NR_Linux + 460) +#define __NR_lsm_list_modules (__NR_Linux + 461)#endif /* _ASM_UNISTD_O32_H */
linux-headers/asm-powerpc/unistd_32.h +5 -0
@@ -438,6 +438,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_32_H */
linux-headers/asm-powerpc/unistd_64.h +5 -0
@@ -410,6 +410,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_64_H */
linux-headers/asm-riscv/kvm.h +40 -0
@@ -139,6 +139,33 @@ enum KVM_RISCV_ISA_EXT_ID {KVM_RISCV_ISA_EXT_ZIHPM ,KVM_RISCV_ISA_EXT_SMSTATEEN ,KVM_RISCV_ISA_EXT_ZICOND , +KVM_RISCV_ISA_EXT_ZBC , +KVM_RISCV_ISA_EXT_ZBKB , +KVM_RISCV_ISA_EXT_ZBKC , +KVM_RISCV_ISA_EXT_ZBKX , +KVM_RISCV_ISA_EXT_ZKND , +KVM_RISCV_ISA_EXT_ZKNE , +KVM_RISCV_ISA_EXT_ZKNH , +KVM_RISCV_ISA_EXT_ZKR , +KVM_RISCV_ISA_EXT_ZKSED , +KVM_RISCV_ISA_EXT_ZKSH , +KVM_RISCV_ISA_EXT_ZKT , +KVM_RISCV_ISA_EXT_ZVBB , +KVM_RISCV_ISA_EXT_ZVBC , +KVM_RISCV_ISA_EXT_ZVKB , +KVM_RISCV_ISA_EXT_ZVKG , +KVM_RISCV_ISA_EXT_ZVKNED , +KVM_RISCV_ISA_EXT_ZVKNHA , +KVM_RISCV_ISA_EXT_ZVKNHB , +KVM_RISCV_ISA_EXT_ZVKSED , +KVM_RISCV_ISA_EXT_ZVKSH , +KVM_RISCV_ISA_EXT_ZVKT , +KVM_RISCV_ISA_EXT_ZFH , +KVM_RISCV_ISA_EXT_ZFHMIN , +KVM_RISCV_ISA_EXT_ZIHINTNTL , +KVM_RISCV_ISA_EXT_ZVFH , +KVM_RISCV_ISA_EXT_ZVFHMIN , +KVM_RISCV_ISA_EXT_ZFA , KVM_RISCV_ISA_EXT_MAX, };
@@ -157,9 +184,16 @@ enum KVM_RISCV_SBI_EXT_ID {KVM_RISCV_SBI_EXT_EXPERIMENTAL ,KVM_RISCV_SBI_EXT_VENDOR ,KVM_RISCV_SBI_EXT_DBCN , +KVM_RISCV_SBI_EXT_STA , KVM_RISCV_SBI_EXT_MAX, }; +/* SBI STA extension registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */ +struct kvm_riscv_sbi_sta { +unsigned long shmem_lo; +unsigned long shmem_hi; +}; +/* Possible states for kvm_riscv_timer */ #define KVM_RISCV_TIMER_STATE_OFF 0#define KVM_RISCV_TIMER_STATE_ON 1
@@ -241,6 +275,12 @@ enum KVM_RISCV_SBI_EXT_ID {#define KVM_REG_RISCV_VECTOR_REG (n) \ ((n) + sizeof(struct __riscv_v_ext_state) / sizeof(unsigned long)) +/* Registers for specific SBI extensions are mapped as type 10 */ +#define KVM_REG_RISCV_SBI_STATE (0x0a << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_SBI_STA (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT) +#define KVM_REG_RISCV_SBI_STA_REG (name) \ + (offsetof(struct kvm_riscv_sbi_sta, name) / sizeof(unsigned long)) +/* Device Control API: RISC-V AIA */ #define KVM_DEV_RISCV_APLIC_ALIGN 0x1000#define KVM_DEV_RISCV_APLIC_SIZE 0x4000
linux-headers/asm-s390/unistd_32.h +5 -0
@@ -429,5 +429,10 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_S390_UNISTD_32_H */
linux-headers/asm-s390/unistd_64.h +5 -0
@@ -377,5 +377,10 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_S390_UNISTD_64_H */
linux-headers/asm-x86/kvm.h +3 -0
@@ -560,4 +560,7 @@ struct kvm_pmu_event_filter {/* x86-specific KVM_EXIT_HYPERCALL flags. */ #define KVM_EXIT_HYPERCALL_LONG_MODE BIT(0) +#define KVM_X86_DEFAULT_VM 0 +#define KVM_X86_SW_PROTECTED_VM 1 +#endif /* _ASM_X86_KVM_H */
linux-headers/asm-x86/unistd_32.h +5 -0
@@ -447,6 +447,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_32_H */
linux-headers/asm-x86/unistd_64.h +5 -0
@@ -369,6 +369,11 @@#define __NR_futex_wake 454#define __NR_futex_wait 455#define __NR_futex_requeue 456 +#define __NR_statmount 457 +#define __NR_listmount 458 +#define __NR_lsm_get_self_attr 459 +#define __NR_lsm_set_self_attr 460 +#define __NR_lsm_list_modules 461#endif /* _ASM_UNISTD_64_H */
linux-headers/asm-x86/unistd_x32.h +5 -0
@@ -321,6 +321,11 @@#define __NR_futex_wake (__X32_SYSCALL_BIT + 454)#define __NR_futex_wait (__X32_SYSCALL_BIT + 455)#define __NR_futex_requeue (__X32_SYSCALL_BIT + 456) +#define __NR_statmount (__X32_SYSCALL_BIT + 457) +#define __NR_listmount (__X32_SYSCALL_BIT + 458) +#define __NR_lsm_get_self_attr (__X32_SYSCALL_BIT + 459) +#define __NR_lsm_set_self_attr (__X32_SYSCALL_BIT + 460) +#define __NR_lsm_list_modules (__X32_SYSCALL_BIT + 461)#define __NR_rt_sigaction (__X32_SYSCALL_BIT + 512)#define __NR_rt_sigreturn (__X32_SYSCALL_BIT + 513)#define __NR_ioctl (__X32_SYSCALL_BIT + 514)
linux-headers/linux/iommufd.h +79 -0
89 lines not on this page · raw message
linux-headers/linux/kvm.h +50 -90
218 lines not on this page · raw message
linux-headers/linux/userfaultfd.h +28 -1
@@ -41,7 +41,8 @@ UFFD_FEATURE_WP_HUGETLBFS_SHMEM | \ UFFD_FEATURE_WP_UNPOPULATED | \ UFFD_FEATURE_POISON | \ - UFFD_FEATURE_WP_ASYNC) + UFFD_FEATURE_WP_ASYNC | \ + UFFD_FEATURE_MOVE)#define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \
@@ -50,6 +51,7 @@ ((__u64 )1 << _UFFDIO_WAKE | \ (__u64 )1 << _UFFDIO_COPY | \ (__u64 )1 << _UFFDIO_ZEROPAGE | \ + (__u64 )1 << _UFFDIO_MOVE | \ (__u64 )1 << _UFFDIO_WRITEPROTECT | \ (__u64 )1 << _UFFDIO_CONTINUE | \ (__u64 )1 << _UFFDIO_POISON)
@@ -73,6 +75,7 @@#define _UFFDIO_WAKE (0x02)#define _UFFDIO_COPY (0x03)#define _UFFDIO_ZEROPAGE (0x04) +#define _UFFDIO_MOVE (0x05)#define _UFFDIO_WRITEPROTECT (0x06)#define _UFFDIO_CONTINUE (0x07)#define _UFFDIO_POISON (0x08)
@@ -92,6 +95,8 @@ struct uffdio_copy)#define UFFDIO_ZEROPAGE _IOWR(UFFDIO, _UFFDIO_ZEROPAGE, \ struct uffdio_zeropage) +#define UFFDIO_MOVE _IOWR(UFFDIO, _UFFDIO_MOVE, \ + struct uffdio_move)#define UFFDIO_WRITEPROTECT _IOWR(UFFDIO, _UFFDIO_WRITEPROTECT, \ struct uffdio_writeprotect)#define UFFDIO_CONTINUE _IOWR(UFFDIO, _UFFDIO_CONTINUE, \
@@ -222,6 +227,9 @@ struct uffdio_api {* asynchronous mode is supported in which the write fault is * automatically resolved and write-protection is un-set. * It implies UFFD_FEATURE_WP_UNPOPULATED. +* +* UFFD_FEATURE_MOVE indicates that the kernel supports moving an +* existing page contents from userspace. */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0)#define UFFD_FEATURE_EVENT_FORK (1<<1)
@@ -239,6 +247,7 @@ struct uffdio_api {#define UFFD_FEATURE_WP_UNPOPULATED (1<<13)#define UFFD_FEATURE_POISON (1<<14)#define UFFD_FEATURE_WP_ASYNC (1<<15) +#define UFFD_FEATURE_MOVE (1<<16)__u64 features;__u64 ioctls;
@@ -347,6 +356,24 @@ struct uffdio_poison {__s64 updated; }; +struct uffdio_move { +__u64 dst; +__u64 src; +__u64 len; +/* +* Especially if used to atomically remove memory from the +* address space the wake on the dst range is not needed. +*/ +#define UFFDIO_MOVE_MODE_DONTWAKE ((__u64)1<<0) +#define UFFDIO_MOVE_MODE_ALLOW_SRC_HOLES ((__u64)1<<1) +__u64 mode; +/* +* "move" is written by the ioctl and must be at the end: the +* copy_from_user will not read the last 8 bytes. +*/ +__s64 move; +}; +/* * Flags for the userfaultfd(2) system call itself. */
linux-headers/linux/vfio.h +1 -0
@@ -1219,6 +1219,7 @@ enum vfio_device_mig_state {VFIO_DEVICE_STATE_RUNNING_P2P =5 ,VFIO_DEVICE_STATE_PRE_COPY =6 ,VFIO_DEVICE_STATE_PRE_COPY_P2P =7 , + VFIO_DEVICE_STATE_NR, };/**
Signature · 1 line
2.34.1
PATCH v5 2/65 RAMBlock: Add support of KVM private guest memfd
In reply to Xiaoyao LiAdd KVM guest_memfd support to RAMBlock so both normal hva based memory and kvm guest memfd based private memory can be associated in one RAMBlock.
Introduce new flag RAM_GUEST_MEMFD. When it's set, it calls KVM ioctl to create private guest_memfd during RAMBlock setup.
Allocating a new RAM_GUEST_MEMFD flag to instruct the setup of guest memfd is more flexible and extensible than simply relying on the VM type because in the future we may have the case that not all the memory of a VM need guest memfd. As a benefit, it also avoid getting MachineState in memory subsystem.
Note, RAM_GUEST_MEMFD is supposed to be set for memory backends of confidential guests, such as TDX VM. How and when to set it for memory backends will be implemented in the following patches.
Introduce memory_region_has_guest_memfd() to query if the MemoryRegion has KVM guest_memfd allocated.
---
Changes in v5:
- Use assert(new_block->guest_memfd < 0) instead of condition check;
- Collect Reviewed-by tag from David;
Changes in v4:
- Add clarification of RAM_GUEST_MEMFD in commit message; (David Hildenbrand)
- refine the return value and error message; (Daniel P. Berrangé)
- remove flags in ram_block_add(); (David Hildenbrand)
Changes in v3:
- rename gmem to guest_memfd;
- close(guest_memfd) when RAMBlock is released; (Daniel P. Berrangé)
- Suqash the patch that introduces memory_region_has_guest_memfd().
--- accel/kvm/kvm-all.c | 25 +++++++++++++++++++++++++ accel/stubs/kvm-stub.c | 5 +++++ include/exec/memory.h | 20 +++++++++++++++++--- include/exec/ram_addr.h | 2 +- include/exec/ramblock.h | 1 + include/sysemu/kvm.h | 2 ++ system/memory.c | 5 +++++ system/physmem.c | 24 +++++++++++++++++++++--- 8 files changed, 77 insertions(+), 7 deletions(-)
accel/kvm/kvm-all.c +25 -0
@@ -91,6 +91,7 @@ bool kvm_msi_use_devid;static bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit; +static bool kvm_guest_memfd_supported;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -2395,6 +2396,8 @@ static int kvm_init(MachineState *ms) } s->as = g_new0(struct KVMAs, s->nr_as); + kvm_guest_memfd_supported =kvm_check_extension (s,KVM_CAP_GUEST_MEMFD ); +if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",
@@ -4094,3 +4097,25 @@ void query_stats_schemas_cb(StatsSchemaList **result, Error **errp)query_stats_schema_vcpu (first_cpu, &stats_args); } } + +int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp) +{ +int fd; +struct kvm_create_guest_memfd guest_memfd = { + .size = size, + .flags = flags, + }; + +if (!kvm_guest_memfd_supported) { +error_setg (errp,"KVM doesn't support guest memfd\n" ); +return -1 ; + } + + fd =kvm_vm_ioctl (kvm_state,KVM_CREATE_GUEST_MEMFD , &guest_memfd); +if (fd <0 ) { +error_setg_errno (errp, errno,"Error creating kvm guest memfd" ); +return -1 ; + } + +return fd; +}
accel/stubs/kvm-stub.c +5 -0
@@ -129,3 +129,8 @@ bool kvm_hwpoisoned_mem(void) {return false; } + +int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp) +{ +return -ENOSYS ; +}
include/exec/memory.h +17 -3
@@ -243,6 +243,9 @@ typedef struct IOMMUTLBEvent {/* RAM FD is opened read-only */ #define RAM_READONLY_FD (1 << 11) +/* RAM can be private that has kvm guest memfd backend */ +#define RAM_GUEST_MEMFD (1 << 12) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,
@@ -1307,7 +1310,8 @@ bool memory_region_init_ram_nomigrate(MemoryRegion *mr,* @name: Region name, becomes part of RAMBlock name used in migration stream * must be unique within any device * @size: size of the region. -* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE. +* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE, +* RAM_GUEST_MEMFD. * @errp: pointer to Error*, to store an error if it happens. * * Note that this function does not do anything to cause the data in the
@@ -1369,7 +1373,7 @@ bool memory_region_init_resizeable_ram(MemoryRegion *mr,* (getpagesize()) will be used. * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM, * RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY, -* RAM_READONLY_FD +* RAM_READONLY_FD, RAM_GUEST_MEMFD * @path: the path in which to allocate the RAM. * @offset: offset within the file referenced by path * @errp: pointer to Error*, to store an error if it happens.
@@ -1399,7 +1403,7 @@ bool memory_region_init_ram_from_file(MemoryRegion *mr,* @size: size of the region. * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM, * RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY, -* RAM_READONLY_FD +* RAM_READONLY_FD, RAM_GUEST_MEMFD * @fd: the fd to mmap. * @offset: offset within the file referenced by fd * @errp: pointer to Error*, to store an error if it happens.
@@ -1722,6 +1726,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)*/ bool memory_region_is_protected (MemoryRegion *mr); +/** +* memory_region_has_guest_memfd: check whether a memory region has guest_memfd +* associated +* +* Returns %true if a memory region's ram_block has valid guest_memfd assigned. +* +* @mr: the memory region being queried +*/ +bool memory_region_has_guest_memfd (MemoryRegion *mr); +/** * memory_region_get_iommu: check whether a memory region is an iommu *
include/exec/ram_addr.h +1 -1
@@ -109,7 +109,7 @@ long qemu_maxrampagesize(void);* @mr: the memory region where the ram block is * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM, * RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY, -* RAM_READONLY_FD +* RAM_READONLY_FD, RAM_GUEST_MEMFD * @mem_path or @fd: specify the backing file or device * @offset: Offset into target file * @errp: pointer to Error*, to store an error if it happens
include/exec/ramblock.h +1 -0
@@ -41,6 +41,7 @@ struct RAMBlock { QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers;int fd;uint64_t fd_offset; +int guest_memfd;size_t page_size;/* dirty bitmap used during migration */ unsigned long *bmap;
include/sysemu/kvm.h +2 -0
@@ -544,4 +544,6 @@ uint32_t kvm_dirty_ring_size(void);* reported for the VM. */ bool kvm_hwpoisoned_mem (void ); + +int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp);#endif
system/memory.c +5 -0
@@ -1850,6 +1850,11 @@ bool memory_region_is_protected(MemoryRegion *mr)return mr->ram && (mr->ram_block->flags &RAM_PROTECTED ); } +bool memory_region_has_guest_memfd (MemoryRegion *mr) +{ +return mr->ram_block && mr->ram_block->guest_memfd >=0 ; +} +uint8_t memory_region_get_dirty_log_mask (MemoryRegion *mr) {uint8_t mask = mr->dirty_log_mask;
system/physmem.c +21 -3
@@ -1841,6 +1841,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) } } +if (kvm_enabled () && (new_block->flags &RAM_GUEST_MEMFD )) { +assert (new_block->guest_memfd <0 ); + + new_block->guest_memfd =kvm_create_guest_memfd (new_block->max_length, +0 , errp); +if (new_block->guest_memfd <0 ) { +qemu_mutex_unlock_ramlist (); +return ; + } + } + new_ram_size =MAX (old_ram_size, (new_block->offset + new_block->max_length) >>TARGET_PAGE_BITS );if (new_ram_size > old_ram_size) {
@@ -1903,7 +1914,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,/* Just support these ram flags by now. */ assert ((ram_flags & ~(RAM_SHARED |RAM_PMEM |RAM_NORESERVE |RAM_PROTECTED |RAM_NAMED_FILE |RAM_READONLY | -RAM_READONLY_FD )) ==0 ); +RAM_READONLY_FD |RAM_GUEST_MEMFD )) ==0 );if (xen_enabled ()) {error_setg (errp,"-mem-path not supported with Xen" );
@@ -1938,6 +1949,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, new_block->used_length = size; new_block->max_length = size; new_block->flags = ram_flags; + new_block->guest_memfd =-1 ; new_block->host =file_ram_alloc (new_block, size, fd, !file_size, offset, errp);if (!new_block->host) {
@@ -2016,7 +2028,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,Error *local_err =NULL ;assert ((ram_flags & ~(RAM_SHARED |RAM_RESIZEABLE |RAM_PREALLOC | -RAM_NORESERVE )) ==0 ); +RAM_NORESERVE |RAM_GUEST_MEMFD )) ==0 );assert (!host ^ (ram_flags &RAM_PREALLOC )); size =HOST_PAGE_ALIGN (size);
@@ -2028,6 +2040,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, new_block->max_length = max_size;assert (max_size >= size); new_block->fd =-1 ; + new_block->guest_memfd =-1 ; new_block->page_size =qemu_real_host_page_size (); new_block->host = host; new_block->flags = ram_flags;
@@ -2050,7 +2063,7 @@ RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,RAMBlock *qemu_ram_alloc (ram_addr_t size,uint32_t ram_flags,MemoryRegion *mr,Error **errp) { -assert ((ram_flags & ~(RAM_SHARED |RAM_NORESERVE )) ==0 ); +assert ((ram_flags & ~(RAM_SHARED |RAM_NORESERVE |RAM_GUEST_MEMFD )) ==0 );return qemu_ram_alloc_internal (size, size,NULL ,NULL , ram_flags, mr, errp); }
@@ -2078,6 +2091,11 @@ static void reclaim_ramblock(RAMBlock *block) }else {qemu_anon_ram_free (block->host, block->max_length); } + +if (block->guest_memfd >=0 ) { +close (block->guest_memfd); + } +g_free (block); }
Signature · 1 line
2.34.1
PATCH v5 3/65 HostMem: Add mechanism to opt in kvm guest memfd via MachineState
In reply to Xiaoyao LiAdd a new member "guest_memfd" to memory backends. When it's set to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm guest_memfd will be allocated during RAMBlock allocation.
Memory backend's @guest_memfd is wired with @require_guest_memfd field of MachineState. It avoid looking up the machine in phymem.c.
MachineState::require_guest_memfd is supposed to be set by any VMs that requires KVM guest memfd as private memory, e.g., TDX VM.
---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)
--- backends/hostmem-file.c | 1 + backends/hostmem-memfd.c | 1 + backends/hostmem-ram.c | 1 + backends/hostmem.c | 1 + hw/core/machine.c | 5 +++++ include/hw/boards.h | 2 ++ include/sysemu/hostmem.h | 1 + 7 files changed, 12 insertions(+)
backends/hostmem-file.c +1 -0
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) ram_flags |= fb->readonly ?RAM_READONLY_FD :0 ; ram_flags |= fb->rom ==ON_OFF_AUTO_ON ?RAM_READONLY :0 ; ram_flags |= backend->reserve ?0 :RAM_NORESERVE ; + ram_flags |= backend->guest_memfd ?RAM_GUEST_MEMFD :0 ; ram_flags |= fb->is_pmem ?RAM_PMEM :0 ; ram_flags |=RAM_NAMED_FILE ; return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
backends/hostmem-memfd.c +1 -0
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend); ram_flags = backend->share ?RAM_SHARED :0 ; ram_flags |= backend->reserve ?0 :RAM_NORESERVE ; + ram_flags |= backend->guest_memfd ?RAM_GUEST_MEMFD :0 ;return memory_region_init_ram_from_fd (&backend->mr,OBJECT (backend), name, backend->size, ram_flags, fd,0 , errp); }
backends/hostmem-ram.c +1 -0
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) name =host_memory_backend_get_name (backend); ram_flags = backend->share ?RAM_SHARED :0 ; ram_flags |= backend->reserve ?0 :RAM_NORESERVE ; + ram_flags |= backend->guest_memfd ?RAM_GUEST_MEMFD :0 ;return memory_region_init_ram_flags_nomigrate (&backend->mr,OBJECT (backend), name, backend->size, ram_flags, errp);
backends/hostmem.c +1 -0
@@ -277,6 +277,7 @@ static void host_memory_backend_init(Object *obj)/* TODO: convert access to globals to compat properties */ backend->merge =machine_mem_merge (machine); backend->dump =machine_dump_guest_core (machine); + backend->guest_memfd =machine_require_guest_memfd (machine); backend->reserve = true; backend->prealloc_threads = machine->smp.cpus; }
hw/core/machine.c +5 -0
@@ -1192,6 +1192,11 @@ bool machine_mem_merge(MachineState *machine)return machine->mem_merge; } +bool machine_require_guest_memfd (MachineState *machine) +{ +return machine->require_guest_memfd; +} +static char *cpu_slot_to_string (const CPUArchId *cpu) {GString *s =g_string_new (NULL );
include/hw/boards.h +2 -0
@@ -36,6 +36,7 @@ bool machine_usb(MachineState *machine);int machine_phandle_start (MachineState *machine);bool machine_dump_guest_core (MachineState *machine);bool machine_mem_merge (MachineState *machine); +bool machine_require_guest_memfd (MachineState *machine);HotpluggableCPUList *machine_query_hotpluggable_cpus (MachineState *machine); void machine_set_cpu_numa_node(MachineState *machine, const CpuInstanceProperties *props,
@@ -370,6 +371,7 @@ struct MachineState {char *dt_compatible;bool dump_guest_core;bool mem_merge; +bool require_guest_memfd;bool usb;bool usb_disabled;char *firmware;
include/sysemu/hostmem.h +1 -0
@@ -74,6 +74,7 @@ struct HostMemoryBackend {uint64_t size;bool merge, dump, use_canonical_path;bool prealloc, is_mapped, share, reserve; +bool guest_memfd;uint32_t prealloc_threads;ThreadContext *prealloc_context;DECLARE_BITMAP (host_nodes,MAX_NODES +1 );
Signature · 1 line
2.34.1
PATCH v5 4/65 trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()
In reply to Xiaoyao LiThe upper 16 bits of kvm_userspace_memory_region::slot are address space id. Parse it separately in trace_kvm_set_user_memory().
--- accel/kvm/kvm-all.c | 5 +++-- accel/kvm/trace-events | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +3 -2
@@ -304,8 +304,9 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); slot->old_flags = mem.flags; err: -trace_kvm_set_user_memory (mem.slot, mem.flags, mem.guest_phys_addr, - mem.memory_size, mem.userspace_addr, ret); +trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, + mem.guest_phys_addr, mem.memory_size, + mem.userspace_addr, ret);if (ret <0 ) { error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
accel/kvm/trace-events +1 -1
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"
Signature · 1 line
2.34.1
PATCH v5 5/65 kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.
With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that backend'ed both by hva-based shared memory and guest memfd based private memory.
---
Changes in v4:
- update KVM_MEM_PRIVATE to KVM_MEM_GUEST_MEMFD; (Isaku)
--- accel/kvm/kvm-all.c | 56 ++++++++++++++++++++++++++++++++++------ accel/kvm/trace-events | 2 +- include/sysemu/kvm_int.h | 2 ++ 3 files changed, 51 insertions(+), 9 deletions(-)
accel/kvm/kvm-all.c +48 -8
@@ -283,35 +283,69 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,static int kvm_set_user_memory_region (KVMMemoryListener *kml,KVMSlot *slot,bool new) {KVMState *s = kvm_state; -struct kvm_userspace_memory_region mem; +struct kvm_userspace_memory_region2 mem; +static int cap_user_memory2 =-1 ;int ret; +if (cap_user_memory2 ==-1 ) { + cap_user_memory2 =kvm_check_extension (s,KVM_CAP_USER_MEMORY2 ); + } + +if (!cap_user_memory2 && slot->guest_memfd >=0 ) { +error_report ("%s, KVM doesn't support KVM_CAP_USER_MEMORY2," +" which is required by guest memfd!" , __func__); +exit (1 ); + } + mem.slot = slot->slot | (kml->as_id <<16 ); mem.guest_phys_addr = slot->start_addr; mem.userspace_addr = (unsigned long )slot->ram; mem.flags = slot->flags; + mem.guest_memfd = slot->guest_memfd; + mem.guest_memfd_offset = slot->guest_memfd_offset;if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) &KVM_MEM_READONLY ) {/* Set the slot size to 0 before setting the slot to the desired * value. This is needed based on KVM commit 75d61fbc. */ mem.memory_size =0 ; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + }if (ret <0 ) { goto err; } } mem.memory_size = slot->memory_size; - ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); +if (cap_user_memory2) { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION2 , &mem); + }else { + ret =kvm_vm_ioctl (s,KVM_SET_USER_MEMORY_REGION , &mem); + } slot->old_flags = mem.flags; err:trace_kvm_set_user_memory (mem.slot >>16 , (uint16_t )mem.slot, mem.flags, mem.guest_phys_addr, mem.memory_size, - mem.userspace_addr, ret); + mem.userspace_addr, mem.guest_memfd, + mem.guest_memfd_offset, ret);if (ret <0 ) { -error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," -" start=0x%" PRIx64", size=0x%" PRIx64": %s" , - __func__, mem.slot, slot->start_addr, - (uint64_t )mem.memory_size,strerror (errno)); +if (cap_user_memory2) { +error_report ("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64"," +" flags=0x%" PRIx32", guest_memfd=%" PRId32"," +" guest_memfd_offset=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size, mem.flags, + mem.guest_memfd, (uint64_t )mem.guest_memfd_offset, +strerror (errno)); + }else { +error_report ("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d," +" start=0x%" PRIx64", size=0x%" PRIx64": %s" , + __func__, mem.slot, slot->start_addr, + (uint64_t )mem.memory_size,strerror (errno)); + } }return ret; }
@@ -466,6 +500,9 @@ static int kvm_mem_flags(MemoryRegion *mr)if (readonly && kvm_readonly_mem_allowed) { flags |=KVM_MEM_READONLY ; } +if (memory_region_has_guest_memfd (mr)) { + flags |=KVM_MEM_GUEST_MEMFD ; + }return flags; }
@@ -1363,6 +1400,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, mem->ram_start_offset = ram_start_offset; mem->ram = ram; mem->flags =kvm_mem_flags (mr); + mem->guest_memfd = mr->ram_block->guest_memfd; + mem->guest_memfd_offset = (uint8_t *)ram - mr->ram_block->host; +kvm_slot_init_dirty_bitmap (mem); err =kvm_set_user_memory_region (kml, mem, true);if (err) {
accel/kvm/trace-events +1 -1
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d" kvm_irqchip_release_virq(int virq) "virq %d" kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d" kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d" -kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d" +kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " guest_memfd=%d" " guest_memfd_offset=0x%" PRIx64 " ret=%d" kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32 kvm_resample_fd_notify(int gsi) "gsi %d" kvm_dirty_ring_full(int id) "vcpu %d"
include/sysemu/kvm_int.h +2 -0
@@ -30,6 +30,8 @@ typedef struct KVMSlotint as_id;/* Cache of the offset in ram address space */ ram_addr_t ram_start_offset; +int guest_memfd; +hwaddr guest_memfd_offset; } KVMSlot; typedef struct KVMMemoryUpdate {
Signature · 1 line
2.34.1
PATCH v5 6/65 kvm: Introduce support for memory_attributes
In reply to Xiaoyao LiIntroduce the helper functions to set the attributes of a range of memory to private or shared.
This is necessary to notify KVM the private/shared attribute of each gpa range. KVM needs the information to decide the GPA needs to be mapped at hva-based shared memory or guest_memfd based private memory.
---
Changes in v4:
- move the check of kvm_supported_memory_attributes to the common kvm_set_memory_attributes(); (Wang Wei)
- change warn_report() to error_report() in kvm_set_memory_attributes() and drop the __func__; (Daniel)
--- accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ include/sysemu/kvm.h | 3 +++ 2 files changed, 47 insertions(+)
accel/kvm/kvm-all.c +44 -0
@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit;static bool kvm_guest_memfd_supported; +static uint64_t kvm_supported_memory_attributes;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size) kvm_max_slot_size = max_slot_size; } +static int kvm_set_memory_attributes (hwaddr start,hwaddr size,uint64_t attr) +{ +struct kvm_memory_attributes attrs; +int r; + +if (kvm_supported_memory_attributes ==0 ) { +error_report ("No memory attribute supported by KVM\n" ); +return -EINVAL ; + } + +if ((attr & kvm_supported_memory_attributes) != attr) { +error_report ("memory attribute 0x%lx not supported by KVM," +" supported bits are 0x%lx\n" , + attr, kvm_supported_memory_attributes); +return -EINVAL ; + } + + attrs.attributes = attr; + attrs.address = start; + attrs.size = size; + attrs.flags =0 ; + + r =kvm_vm_ioctl (kvm_state,KVM_SET_MEMORY_ATTRIBUTES , &attrs); +if (r) { +error_report ("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'" , + start, size, attr,strerror (errno)); + } +return r; +} + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,KVM_MEMORY_ATTRIBUTE_PRIVATE ); +} + +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,0 ); +} +/* Called with KVMMemoryListener.slots_lock held */ static void kvm_set_phys_mem (KVMMemoryListener *kml,MemoryRegionSection *section,bool add)
@@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms) kvm_guest_memfd_supported =kvm_check_extension (s,KVM_CAP_GUEST_MEMFD ); + ret =kvm_check_extension (s,KVM_CAP_MEMORY_ATTRIBUTES ); + kvm_supported_memory_attributes = ret >0 ? ret :0 ; +if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",
include/sysemu/kvm.h +3 -0
@@ -546,4 +546,7 @@ uint32_t kvm_dirty_ring_size(void);bool kvm_hwpoisoned_mem (void );int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp); + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size); +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size);#endif
Signature · 1 line
2.34.1
Re: [PATCH v5 06/65] kvm: Introduce support for memory_attributes
On 2/29/2024 14:36, Xiaoyao Li wrote:> Introduce the helper functions to set the attributes of a range of
Quoted 78 lines
memory to private or shared.
This is necessary to notify KVM the private/shared attribute of each gpa range. KVM needs the information to decide the GPA needs to be mapped at hva-based shared memory or
guest_memfdbased private memory.Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- move the check ofkvm_supported_memory_attributesto the commonkvm_set_memory_attributes(); (Wang Wei)
- changewarn_report()toerror_report()inkvm_set_memory_attributes()and drop the__func__; (Daniel)--- accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ include/sysemu/kvm.h | 3 +++ 2 files changed, 47 insertions(+)accel/kvm/kvm-all.c +41
-0@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit;static bool kvm_guest_memfd_supported; +static uint64_t kvm_supported_memory_attributes;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {@@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size) kvm_max_slot_size = max_slot_size; } +static int kvm_set_memory_attributes (hwaddr start,hwaddr size,uint64_t attr) +{ +struct kvm_memory_attributes attrs; +int r; + +if (kvm_supported_memory_attributes ==0 ) { +error_report ("No memory attribute supported by KVM\n" ); +return -EINVAL ; + } + +if ((attr & kvm_supported_memory_attributes) != attr) { +error_report ("memory attribute 0x%lx not supported by KVM," +" supported bits are 0x%lx\n" , + attr, kvm_supported_memory_attributes); +return -EINVAL ; + } + + attrs.attributes = attr; + attrs.address = start; + attrs.size = size; + attrs.flags =0 ; + + r =kvm_vm_ioctl (kvm_state,KVM_SET_MEMORY_ATTRIBUTES , &attrs); +if (r) { +error_report ("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'" , + start, size, attr,strerror (errno)); + } +return r; +} + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,KVM_MEMORY_ATTRIBUTE_PRIVATE ); +} + +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,0 ); +} +/* Called with KVMMemoryListener.slots_lock held */ static void kvm_set_phys_mem (KVMMemoryListener *kml,MemoryRegionSection *section,bool add)
accel/kvm/kvm-all.c static int kvm_init(MachineState *ms) · line 2484
Quoted 5 lines
accel/kvm/kvm-all.c +2
-0@@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms) kvm_guest_memfd_supported =kvm_check_extension (s,KVM_CAP_GUEST_MEMFD ); + ret =kvm_check_extension (s,KVM_CAP_MEMORY_ATTRIBUTES ); + kvm_supported_memory_attributes = ret >0 ? ret :0 ;
kvm_check_extension() only returns non-negative value, so we can just
kvm_supported_memory_attributes = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
Quoted 16 lines
accel/kvm/kvm-all.c +1
-0+if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",include/sysemu/kvm.h +3
-0@@ -546,4 +546,7 @@ uint32_t kvm_dirty_ring_size(void);bool kvm_hwpoisoned_mem (void );int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp); + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size); +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size);#endif
Re: [PATCH v5 06/65] kvm: Introduce support for memory_attributes
Quoted 80 lines · Lei
On 2/29/2024 14:36, Xiaoyao Li wrote:> Introduce the helper functions to set the attributes of a range of
Quoted 79 lines
memory to private or shared.
This is necessary to notify KVM the private/shared attribute of each gpa range. KVM needs the information to decide the GPA needs to be mapped at hva-based shared memory or
guest_memfdbased private memory.Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- move the check ofkvm_supported_memory_attributesto the commonkvm_set_memory_attributes(); (Wang Wei)
- changewarn_report()toerror_report()inkvm_set_memory_attributes()and drop the__func__; (Daniel)--- accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ include/sysemu/kvm.h | 3 +++ 2 files changed, 47 insertions(+)accel/kvm/kvm-all.c +41
-0@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;static int kvm_sstep_flags;static bool kvm_immediate_exit;static bool kvm_guest_memfd_supported; +static uint64_t kvm_supported_memory_attributes;static hwaddr kvm_max_slot_size = ~0 ;static const KVMCapabilityInfo kvm_required_capabilites[] = {@@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size) kvm_max_slot_size = max_slot_size; } +static int kvm_set_memory_attributes (hwaddr start,hwaddr size,uint64_t attr) +{ +struct kvm_memory_attributes attrs; +int r; + +if (kvm_supported_memory_attributes ==0 ) { +error_report ("No memory attribute supported by KVM\n" ); +return -EINVAL ; + } + +if ((attr & kvm_supported_memory_attributes) != attr) { +error_report ("memory attribute 0x%lx not supported by KVM," +" supported bits are 0x%lx\n" , + attr, kvm_supported_memory_attributes); +return -EINVAL ; + } + + attrs.attributes = attr; + attrs.address = start; + attrs.size = size; + attrs.flags =0 ; + + r =kvm_vm_ioctl (kvm_state,KVM_SET_MEMORY_ATTRIBUTES , &attrs); +if (r) { +error_report ("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'" , + start, size, attr,strerror (errno)); + } +return r; +} + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,KVM_MEMORY_ATTRIBUTE_PRIVATE ); +} + +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size) +{ +return kvm_set_memory_attributes (start, size,0 ); +} +/* Called with KVMMemoryListener.slots_lock held */ static void kvm_set_phys_mem (KVMMemoryListener *kml,MemoryRegionSection *section,bool add)@@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms)
Quoted 8 lines
accel/kvm/kvm-all.c static int kvm_init(MachineState *ms) · line 2484
Quoted 4 lines
accel/kvm/kvm-all.c +2
-0kvm_guest_memfd_supported =kvm_check_extension (s,KVM_CAP_GUEST_MEMFD ); + ret =kvm_check_extension (s,KVM_CAP_MEMORY_ATTRIBUTES ); + kvm_supported_memory_attributes = ret >0 ? ret :0 ;
kvm_check_extension()only returns non-negative value, so we can just
kvm_supported_memory_attributes=kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
Good catch, I will update it in next version.
Quoted 16 lines
Quoted 16 lines
accel/kvm/kvm-all.c +1
-0+if (object_property_find (OBJECT (current_machine),"kvm-type" )) { g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine), "kvm-type",include/sysemu/kvm.h +3
-0@@ -546,4 +546,7 @@ uint32_t kvm_dirty_ring_size(void);bool kvm_hwpoisoned_mem (void );int kvm_create_guest_memfd (uint64_t size,uint64_t flags,Error **errp); + +int kvm_set_memory_attributes_private (hwaddr start,hwaddr size); +int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size);#endif
PATCH v5 7/65 physmem: Introduce ram_block_discard_guest_memfd_range()
In reply to Xiaoyao LiWhen memory page is converted from private to shared, the original private memory is back'ed by guest_memfd. Introduce ram_block_discard_guest_memfd_range() for discarding memory in guest_memfd.
Originally-from: Isaku Yamahata <[email protected]>
---
Changes in v5:
- Collect Reviewed-by from David;
Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the next Patch.
--- include/exec/cpu-common.h | 2 ++ system/physmem.c | 23 +++++++++++++++++++++++ 2 files changed, 25 insertions(+)
include/exec/cpu-common.h +2 -0
@@ -166,6 +166,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque);int ram_block_discard_range (RAMBlock *rb,uint64_t start,size_t length); +int ram_block_discard_guest_memfd_range (RAMBlock *rb,uint64_t start, +size_t length);#endif
system/physmem.c +23 -0
@@ -3618,6 +3618,29 @@ err:return ret; } +int ram_block_discard_guest_memfd_range (RAMBlock *rb,uint64_t start, +size_t length) +{ +int ret =-1 ; + +#ifdef CONFIG_FALLOCATE_PUNCH_HOLE + ret =fallocate (rb->guest_memfd,FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE , + start, length); + +if (ret) { + ret = -errno; +error_report ("%s: Failed to fallocate %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); + } +#else + ret = -ENOSYS ; +error_report ("%s: fallocate not available %s:%" PRIx64" +%zx (%d)" , + __func__, rb->idstr, start, length, ret); +#endif + +return ret; +} +bool ramblock_is_pmem (RAMBlock *rb) {return rb->flags &RAM_PMEM ;
Signature · 1 line
2.34.1
PATCH v5 8/65 kvm: handle KVM_EXIT_MEMORY_FAULT
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.
Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when KVM_EXIT_MEMORY_FAULT happens.
Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has guest_memfd memory backend.
Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is added.
When page is converted from shared to private, the original shared memory can be discarded via ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.
---
Changes in v4:
- open-coded ram_block_discard logic;
- change warn_report() to error_report(); (Daniel)
--- accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 10 deletions(-)
accel/kvm/kvm-all.c +84 -10
@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +ram_addr_t offset; +MemoryRegion *mr; +RAMBlock *rb; +void *addr; +int ret =-1 ; + +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || + !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) { +return -1 ; + } + +if (!size) { +return -1 ; + } + + section =memory_region_find (get_system_memory (), start, size); + mr = section.mr; +if (!mr) { +return -1 ; + } + +if (memory_region_has_guest_memfd (mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +memory_region_unref (section.mr); +return ret; + } + + addr =memory_region_get_ram_ptr (mr) + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); + +if (to_private) { +if (rb->page_size != qemu_host_page_size) { +/* +* shared memory is back'ed by hugetlb, which is supposed to be +* pre-allocated and doesn't need to be discarded +*/ +return 0 ; + }else { + ret =ram_block_discard_range (rb, offset, size); + } + }else { + ret =ram_block_discard_guest_memfd_range (rb, offset, size); + } + }else { +error_report ("Convert non guest_memfd backed memory region " +"(0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } + +memory_region_unref (section.mr); +return ret; +} +int kvm_cpu_exec (CPUState *cpu) {struct kvm_run *run = cpu->kvm_run;
@@ -2970,18 +3032,20 @@ int kvm_cpu_exec(CPUState *cpu) ret =EXCP_INTERRUPT ;break ; } -fprintf (stderr,"error: kvm run failed %s\n" , -strerror (-run_ret)); +if (!(run_ret == -EFAULT && run->exit_reason ==KVM_EXIT_MEMORY_FAULT )) { +fprintf (stderr,"error: kvm run failed %s\n" , +strerror (-run_ret));#ifdef TARGET_PPC -if (run_ret == -EBUSY ) { -fprintf (stderr, -"This is probably because your SMT is enabled.\n" -"VCPU can only run on primary threads with all " -"secondary threads offline.\n" ); - } +if (run_ret == -EBUSY ) { +fprintf (stderr, +"This is probably because your SMT is enabled.\n" +"VCPU can only run on primary threads with all " +"secondary threads offline.\n" ); + }#endif - ret =-1 ; -break ; + ret =-1 ; +break ; + } }trace_kvm_run_exit (cpu->cpu_index, run->exit_reason);
@@ -3064,6 +3128,16 @@ int kvm_cpu_exec(CPUState *cpu)break ; }break ; +case KVM_EXIT_MEMORY_FAULT : +if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE ) { +error_report ("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64, + (uint64_t )run->memory_fault.flags); + ret =-1 ; +break ; + } + ret =kvm_convert_memory (run->memory_fault.gpa, run->memory_fault.size, + run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE ); +break ;default : ret =kvm_arch_handle_exit (cpu, run);break ;
Signature · 1 line
2.34.1
Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
Quoted 76 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
When geeting
KVM_EXIT_MEMORY_FAULTexit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock hasguest_memfdmemory backend.Note,
KVM_EXIT_MEMORY_FAULTreturns with-EFAULT, so special handling is added.When page is converted from shared to private, the original shared memory can be discarded via
ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.Signed-off-by Chao Peng <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- open-codedram_block_discardlogic;
- changewarn_report()toerror_report(); (Daniel)--- accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 10 deletions(-)accel/kvm/kvm-all.c +36
-0@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +ram_addr_t offset; +MemoryRegion *mr; +RAMBlock *rb; +void *addr; +int ret =-1 ; + +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || + !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) { +return -1 ; + } + +if (!size) { +return -1 ; + } + + section =memory_region_find (get_system_memory (), start, size); + mr = section.mr; +if (!mr) { +return -1 ; + } + +if (memory_region_has_guest_memfd (mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +memory_region_unref (section.mr); +return ret; + } +
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 2951
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+ addr =memory_region_get_ram_ptr (mr) + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); + +if (to_private) { +if (rb->page_size != qemu_host_page_size) { +/* +* shared memory is back'ed by hugetlb, which is supposed to be +* pre-allocated and doesn't need to be discarded +*/ +return 0 ;
The reference count leaks. Add memory_region_unref() is needed.
Otherwise looks good to me.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
Quoted 83 lines · Isaku Yamahata
Quoted 80 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
When geeting
KVM_EXIT_MEMORY_FAULTexit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock hasguest_memfdmemory backend.Note,
KVM_EXIT_MEMORY_FAULTreturns with-EFAULT, so special handling is added.When page is converted from shared to private, the original shared memory can be discarded via
ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.Signed-off-by Chao Peng <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- open-codedram_block_discardlogic;
- changewarn_report()toerror_report(); (Daniel)--- accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 10 deletions(-)accel/kvm/kvm-all.c +40
-0@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +ram_addr_t offset; +MemoryRegion *mr; +RAMBlock *rb; +void *addr; +int ret =-1 ; + +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || + !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) { +return -1 ; + } + +if (!size) { +return -1 ; + } + + section =memory_region_find (get_system_memory (), start, size); + mr = section.mr; +if (!mr) { +return -1 ; + } + +if (memory_region_has_guest_memfd (mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +memory_region_unref (section.mr); +return ret; + } + + addr =memory_region_get_ram_ptr (mr) + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); + +if (to_private) {
Quoted 8 lines
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 2951
Quoted 6 lines
accel/kvm/kvm-all.c +6
-0+if (rb->page_size != qemu_host_page_size) { +/* +* shared memory is back'ed by hugetlb, which is supposed to be +* pre-allocated and doesn't need to be discarded +*/ +return 0 ;The reference count leaks. Add
memory_region_unref()is needed.
thanks for catching it. Will fix it in next version.
Quoted 2 lines
Otherwise looks good to me.
Reviewed-by Isaku Yamahata <[email protected]>
Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
In reply to Xiaoyao LiQuoted 75 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
When geeting
KVM_EXIT_MEMORY_FAULTexit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock hasguest_memfdmemory backend.Note,
KVM_EXIT_MEMORY_FAULTreturns with-EFAULT, so special handling is added.When page is converted from shared to private, the original shared memory can be discarded via
ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.Signed-off-by Chao Peng <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- open-codedram_block_discardlogic;
- changewarn_report()toerror_report(); (Daniel)--- accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 10 deletions(-)accel/kvm/kvm-all.c +35
-0@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +ram_addr_t offset; +MemoryRegion *mr; +RAMBlock *rb; +void *addr; +int ret =-1 ; + +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || + !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) { +return -1 ; + } + +if (!size) { +return -1 ; + } + + section =memory_region_find (get_system_memory (), start, size); + mr = section.mr; +if (!mr) { +return -1 ; + } + +if (memory_region_has_guest_memfd (mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +memory_region_unref (section.mr); +return ret; + }
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 2950
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+ + addr =memory_region_get_ram_ptr (mr) + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); + +if (to_private) { +if (rb->page_size != qemu_host_page_size) { +/* +* shared memory is back'ed by hugetlb, which is supposed to be +* pre-allocated and doesn't need to be discarded +*/
Nit: comment indentation is broken here.
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 2960
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+return 0 ; + }else { + ret =ram_block_discard_range (rb, offset, size); + } + }else { + ret =ram_block_discard_guest_memfd_range (rb, offset, size); + } + }else { + error_report("Convert non guest_memfd backed memory region " + "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
Same as above.
Quoted 58 lines
accel/kvm/kvm-all.c +29
-10+ start, size, to_private ? "private" : "shared"); + } + +memory_region_unref (section.mr); +return ret; +} +int kvm_cpu_exec (CPUState *cpu) {struct kvm_run *run = cpu->kvm_run;@@ -2970,18 +3032,20 @@ int kvm_cpu_exec(CPUState *cpu) ret =EXCP_INTERRUPT ;break ; } -fprintf (stderr,"error: kvm run failed %s\n" , -strerror (-run_ret)); +if (!(run_ret == -EFAULT && run->exit_reason ==KVM_EXIT_MEMORY_FAULT )) { +fprintf (stderr,"error: kvm run failed %s\n" , +strerror (-run_ret));#ifdef TARGET_PPC -if (run_ret == -EBUSY ) { -fprintf (stderr, -"This is probably because your SMT is enabled.\n" -"VCPU can only run on primary threads with all " -"secondary threads offline.\n" ); - } +if (run_ret == -EBUSY ) { +fprintf (stderr, +"This is probably because your SMT is enabled.\n" +"VCPU can only run on primary threads with all " +"secondary threads offline.\n" ); + }#endif - ret =-1 ; -break ; + ret =-1 ; +break ; + } }trace_kvm_run_exit (cpu->cpu_index, run->exit_reason);@@ -3064,6 +3128,16 @@ int kvm_cpu_exec(CPUState *cpu)break ; }break ; +case KVM_EXIT_MEMORY_FAULT : +if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE ) { +error_report ("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64, + (uint64_t )run->memory_fault.flags); + ret =-1 ; +break ; + } + ret =kvm_convert_memory (run->memory_fault.gpa, run->memory_fault.size, + run->memory_fault.flags &KVM_MEMORY_EXIT_FLAG_PRIVATE ); +break ;default : ret =kvm_arch_handle_exit (cpu, run);break ;
Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
Quoted 100 lines · Lei
Quoted 75 lines · Xiaoyao Li
From: Chao Peng <[email protected]>
When geeting
KVM_EXIT_MEMORY_FAULTexit, it indicates userspace needs to do the memory conversion on the RAMBlock to turn the memory into desired attribute, i.e., private/shared.Currently only
KVM_MEMORY_EXIT_FLAG_PRIVATEin flags is valid whenKVM_EXIT_MEMORY_FAULThappens.Note,
KVM_EXIT_MEMORY_FAULTmakes sense only when the RAMBlock hasguest_memfdmemory backend.Note,
KVM_EXIT_MEMORY_FAULTreturns with-EFAULT, so special handling is added.When page is converted from shared to private, the original shared memory can be discarded via
ram_block_discard_range(). Note, shared memory can be discarded only when it's not back'ed by hugetlb because hugetlb is supposed to be pre-allocated and no need for discarding.Signed-off-by Chao Peng <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- open-codedram_block_discardlogic;
- changewarn_report()toerror_report(); (Daniel)--- accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 10 deletions(-)accel/kvm/kvm-all.c +35
-0@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } +static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +{ +MemoryRegionSection section; +ram_addr_t offset; +MemoryRegion *mr; +RAMBlock *rb; +void *addr; +int ret =-1 ; + +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || + !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) { +return -1 ; + } + +if (!size) { +return -1 ; + } + + section =memory_region_find (get_system_memory (), start, size); + mr = section.mr; +if (!mr) { +return -1 ; + } + +if (memory_region_has_guest_memfd (mr)) { +if (to_private) { + ret =kvm_set_memory_attributes_private (start, size); + }else { + ret =kvm_set_memory_attributes_shared (start, size); + } + +if (ret) { +memory_region_unref (section.mr); +return ret; + }accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 2950
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+ + addr =memory_region_get_ram_ptr (mr) + section.offset_within_region; + rb =qemu_ram_block_from_host (addr, false, &offset); + +if (to_private) { +if (rb->page_size != qemu_host_page_size) { +/* +* shared memory is back'ed by hugetlb, which is supposed to be +* pre-allocated and doesn't need to be discarded +*/ Nit: comment indentation is broken here.
accel/kvm/kvm-all.c static void kvm_eat_signals(CPUState *cpu) · line 2960
Quoted 10 lines
accel/kvm/kvm-all.c +10
-0+return 0 ; + }else { + ret =ram_block_discard_range (rb, offset, size); + } + }else { + ret =ram_block_discard_guest_memfd_range (rb, offset, size); + } + }else { + error_report("Convert non guest_memfd backed memory region " + "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
Quoted 2 lines
Same as above.
Fixed.
thanks!
PATCH v5 9/65 trace/kvm: Add trace for page convertion between shared and private
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
--- accel/kvm/kvm-all.c | 2 ++ accel/kvm/trace-events | 1 + 2 files changed, 3 insertions(+)
accel/kvm/kvm-all.c +2 -0
@@ -2912,6 +2912,8 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)void *addr;int ret =-1 ; +trace_kvm_convert_memory (start, size, to_private ?"shared_to_private" :"private_to_shared" ); +if (!QEMU_PTR_IS_ALIGNED (start, qemu_host_page_size) || !QEMU_PTR_IS_ALIGNED (size, qemu_host_page_size)) {return -1 ;
accel/kvm/trace-events +1 -0
@@ -31,3 +31,4 @@ kvm_cpu_exec(void) "" kvm_interrupt_exit_request(void) "" kvm_io_window_exit(void) "" kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32 +kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
Signature · 1 line
2.34.1
PATCH v5 10/65 kvm/memory: Make memory type private by default if it has guest memfd backend
In reply to Xiaoyao LiKVM side leaves the memory to shared by default, while may incur the overhead of paging conversion on the first visit of each page. Because the expectation is that page is likely to private for the VMs that require private memory (has guest memfd).
Explicitly set the memory to private when memory region has valid guest memfd backend.
--- accel/kvm/kvm-all.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
accel/kvm/kvm-all.c +10 -0
@@ -1451,6 +1451,16 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, strerror(-err));abort (); } + +if (memory_region_has_guest_memfd (mr)) { + err =kvm_set_memory_attributes_private (start_addr, slot_size); +if (err) { +error_report ("%s: failed to set memory attribute private: %s\n" , + __func__,strerror (-err)); +exit (1 ); + } + } + start_addr += slot_size; ram_start_offset += slot_size; ram += slot_size;
Signature · 1 line
2.34.1
PATCH v5 11/65 *** HACK *** linux-headers: Update headers to pull in TDX API changes
In reply to Xiaoyao LiPull in recent TDX updates, which are not backwards compatible.
It's just to make this series runnable. It will be updated by script
scripts/update-linux-headers.sh
once TDX support is upstreamed in linux kernel
--- linux-headers/asm-x86/kvm.h | 86 +++++++++++++++++++++++++++++++ linux-headers/linux/kvm.h | 100 ++++++++++++++++++++++++++++++++++++ 2 files changed, 186 insertions(+)
linux-headers/asm-x86/kvm.h +86 -0
@@ -562,5 +562,91 @@ struct kvm_pmu_event_filter {#define KVM_X86_DEFAULT_VM 0#define KVM_X86_SW_PROTECTED_VM 1 +#define KVM_X86_TDX_VM 2 +#define KVM_X86_SNP_VM 3 + +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { +KVM_TDX_CAPABILITIES =0 , +KVM_TDX_INIT_VM , +KVM_TDX_INIT_VCPU , +KVM_TDX_EXTEND_MEMORY , +KVM_TDX_FINALIZE_VM , + +KVM_TDX_CMD_NR_MAX , +}; + +struct kvm_tdx_cmd { +/* enum kvm_tdx_cmd_id */ +__u32 id; +/* flags for sub-commend. If sub-command doesn't use this, set zero. */ +__u32 flags; +/* +* data for each sub-command. An immediate or a pointer to the actual +* data in process virtual address. If sub-command doesn't use it, +* set zero. +*/ +__u64 data; +/* +* Auxiliary error code. The sub-command may return TDX SEAMCALL +* status code in addition to -Exxx. +* Defined for consistency with struct kvm_sev_cmd. +*/ +__u64 error; +}; + +#define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1) + +struct kvm_tdx_cpuid_config { +__u32 leaf; +__u32 sub_leaf; +__u32 eax; +__u32 ebx; +__u32 ecx; +__u32 edx; +}; + +/* supported_gpaw */ +#define TDX_CAP_GPAW_48 (1 << 0) +#define TDX_CAP_GPAW_52 (1 << 1) + +struct kvm_tdx_capabilities { +__u64 attrs_fixed0; +__u64 attrs_fixed1; +__u64 xfam_fixed0; +__u64 xfam_fixed1; +__u32 supported_gpaw; +__u32 padding; +__u64 reserved[251 ]; + +__u32 nr_cpuid_configs; +struct kvm_tdx_cpuid_config cpuid_configs[]; +}; + +struct kvm_tdx_init_vm { +__u64 attributes; +__u64 mrconfigid[6 ];/* sha384 digest */ +__u64 mrowner[6 ];/* sha384 digest */ +__u64 mrownerconfig[6 ];/* sha384 digest */ +/* +* For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB. +* This should be enough given sizeof(TD_PARAMS) = 1024. +* 8KB was chosen given because +* sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB. +*/ +__u64 reserved[1004 ]; + +/* +* Call KVM_TDX_INIT_VM before vcpu creation, thus before +* KVM_SET_CPUID2. +* This configuration supersedes KVM_SET_CPUID2s for VCPUs because the +* TDX module directly virtualizes those CPUIDs without VMM. The user +* space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with +* those values. If it doesn't, KVM may have wrong idea of vCPUIDs of +* the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX +* module doesn't virtualize. +*/ +struct kvm_cpuid2 cpuid; +};#endif /* _ASM_X86_KVM_H */
linux-headers/linux/kvm.h +100 -0
128 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v5 12/65 i386: Introduce tdx-guest object
In reply to Xiaoyao LiIntroduce tdx-guest object which inherits CONFIDENTIAL_GUEST_SUPPORT, and will be used to create TDX VMs (TDs) by
qemu -machine ...,confidential-guest-support=tdx0 \
-object tdx-guest,id=tdx0So far, it has no QAPI member/properety decleared and only one internal member 'attributes' with fixed value 0 that not configurable.
QAPI properties will be added later.
---
Changes in v4:
- update the new qapi since filed from 8.2 to 9.0
Changes in v1
- make @attributes not user-settable
--- configs/devices/i386-softmmu/default.mak | 1 + hw/i386/Kconfig | 5 +++ qapi/qom.json | 12 +++++++ target/i386/kvm/meson.build | 2 ++ target/i386/kvm/tdx.c | 40 ++++++++++++++++++++++++ target/i386/kvm/tdx.h | 19 +++++++++++ 6 files changed, 79 insertions(+) create mode 100644 target/i386/kvm/tdx.c create mode 100644 target/i386/kvm/tdx.h
configs/devices/i386-softmmu/default.mak +1 -0
@@ -18,6 +18,7 @@ #CONFIG_QXL=n #CONFIG_SEV=n #CONFIG_SGA=n +#CONFIG_TDX=n #CONFIG_TEST_DEVICES=n #CONFIG_TPM_CRB=n #CONFIG_TPM_TIS_ISA=n
hw/i386/Kconfig +5 -0
@@ -10,6 +10,10 @@ config SGX bool depends on KVM +config TDX + bool + depends on KVM + config PC bool imply APPLESMC
@@ -26,6 +30,7 @@ config PC imply QXL imply SEV imply SGX + imply TDX imply TEST_DEVICES imply TPM_CRB imply TPM_TIS_ISA
qapi/qom.json +12 -0
@@ -895,6 +895,16 @@ 'reduced-phys-bits': 'uint32', '*kernel-hashes': 'bool' } } +## +# @TdxGuestProperties: +# +# Properties for tdx-guest objects. +# +# Since: 9.0 +## +{ 'struct': 'TdxGuestProperties', + 'data': { }} + ## # @ThreadContextProperties: #
@@ -974,6 +984,7 @@ 'sev-guest', 'thread-context', 's390-pv-guest', + 'tdx-guest', 'throttle-group', 'tls-creds-anon', 'tls-creds-psk',
@@ -1041,6 +1052,7 @@ 'secret_keyring': { 'type': 'SecretKeyringProperties', 'if': 'CONFIG_SECRET_KEYRING' }, 'sev-guest': 'SevGuestProperties', + 'tdx-guest': 'TdxGuestProperties', 'thread-context': 'ThreadContextProperties', 'throttle-group': 'ThrottleGroupProperties', 'tls-creds-anon': 'TlsCredsAnonProperties',
target/i386/kvm/meson.build +2 -0
@@ -7,6 +7,8 @@ i386_kvm_ss.add(files( i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c')) +i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c')) + i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c')) i386_system_ss.add_all(when: 'CONFIG_KVM', if_true: i386_kvm_ss)
target/i386/kvm/tdx.c new file +40 -0
@@ -0,0 +1,40 @@ +/* +* QEMU TDX support +* +* Copyright Intel +* +* Author: +* Xiaoyao Li <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or later. +* See the COPYING file in the top-level directory +* +*/ + +#include "qemu/osdep.h" +#include "qom/object_interfaces.h" + +#include "tdx.h" + +/* tdx guest */ +OBJECT_DEFINE_TYPE_WITH_INTERFACES (TdxGuest, + tdx_guest, +TDX_GUEST , +CONFIDENTIAL_GUEST_SUPPORT , + { TYPE_USER_CREATABLE }, + { NULL }) + +static void tdx_guest_init (Object *obj) +{ +TdxGuest *tdx =TDX_GUEST (obj); + + tdx->attributes =0 ; +} + +static void tdx_guest_finalize (Object *obj) +{ +} + +static void tdx_guest_class_init (ObjectClass *oc,void *data) +{ +}
target/i386/kvm/tdx.h new file +19 -0
@@ -0,0 +1,19 @@ +#ifndef QEMU_I386_TDX_H +#define QEMU_I386_TDX_H + +#include "exec/confidential-guest-support.h" + +#define TYPE_TDX_GUEST "tdx-guest" +#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST) + +typedef struct TdxGuestClass { +ConfidentialGuestSupportClass parent_class; +}TdxGuestClass ; + +typedef struct TdxGuest { +ConfidentialGuestSupport parent_obj; + +uint64_t attributes;/* TD attributes */ +}TdxGuest ; + +#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Re: [PATCH v5 12/65] i386: Introduce tdx-guest object
Quoted 6 lines · Xiaoyao Li
Introduce tdx-guest object which inherits
CONFIDENTIAL_GUEST_SUPPORT, and will be used to create TDX VMs (TDs) byqemu -machine ...,confidential-guest-support=tdx0 \ -object tdx-guest,id=tdx0
Quoted 8 lines
So far, it has no QAPI member/properety decleared and only one internal member 'attributes' with fixed value 0 that not configurable.
QAPI properties will be added later.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>
I'm happy with the commit message now. Thanks!
PATCH v5 13/65 target/i386: Implement mc->kvm_type() to get VM type
In reply to Xiaoyao LiTDX VM requires VM type KVM_X86_TDX_VM to be passed to kvm_ioctl(KVM_CREATE_VM). Hence implement mc->kvm_type() for i386 architecture.
If tdx-guest object is specified to confidential-guest-support, like,
qemu -machine ...,confidential-guest-support=tdx0 \
-object tdx-guest,id=tdx0,...it parses VM type as KVM_X86_TDX_VM. Otherwise, it's KVM_X86_DEFAULT_VM.
Also store the vm_type in MachineState for other code to query what the VM type is.
---
Changes in v4:
- fix the build error of kvm_get_vm_type() when --disable-kvm;
--- hw/i386/x86.c | 12 ++++++++++++ include/hw/i386/x86.h | 1 + target/i386/kvm/kvm.c | 30 ++++++++++++++++++++++++++++++ target/i386/kvm/kvm_i386.h | 1 + 4 files changed, 44 insertions(+)
hw/i386/x86.c +12 -0
@@ -1389,6 +1389,17 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,qapi_free_SgxEPCList (list); } +static int x86_kvm_type (MachineState *ms,const char *vm_type) +{ +X86MachineState *x86ms =X86_MACHINE (ms); +int kvm_type; + + kvm_type =kvm_enabled () ?kvm_get_vm_type (ms, vm_type) :0 ; + x86ms->vm_type = kvm_type; + +return kvm_type; +} +static void x86_machine_initfn (Object *obj) {X86MachineState *x86ms =X86_MACHINE (obj);
@@ -1413,6 +1424,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) mc->cpu_index_to_instance_props = x86_cpu_index_to_props; mc->get_default_cpu_node_id = x86_get_default_cpu_node_id; mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids; + mc->kvm_type = x86_kvm_type; x86mc->save_tsc_khz = true; x86mc->fwcfg_dma_enabled = true; nc->nmi_monitor_handler = x86_nmi;
include/hw/i386/x86.h +1 -0
@@ -43,6 +43,7 @@ struct X86MachineState {MachineState parent;/*< public >*/ +unsigned int vm_type;/* Pointers to devices and objects: */ ISADevice *rtc;
target/i386/kvm/kvm.c +30 -0
@@ -32,6 +32,7 @@#include "sysemu/runstate.h" #include "kvm_i386.h" #include "sev.h" +#include "tdx.h" #include "xen-emu.h" #include "hyperv.h" #include "hyperv-proto.h"
@@ -161,6 +162,35 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];static RateLimit bus_lock_ratelimit_ctrl;static int kvm_get_one_msr (X86CPU *cpu,int index,uint64_t *value); +static const char *vm_type_name[] = { + [KVM_X86_DEFAULT_VM ] ="default" , + [KVM_X86_TDX_VM ] ="tdx" , +}; + +int kvm_get_vm_type (MachineState *ms,const char *vm_type) +{ +int kvm_type =KVM_X86_DEFAULT_VM ; + +if (ms->cgs &&object_dynamic_cast (OBJECT (ms->cgs),TYPE_TDX_GUEST )) { + kvm_type =KVM_X86_TDX_VM ; + } + +/* +* old KVM doesn't support KVM_CAP_VM_TYPES and KVM_X86_DEFAULT_VM +* is always supported +*/ +if (kvm_type ==KVM_X86_DEFAULT_VM ) { +return kvm_type; + } + +if (!(kvm_check_extension (KVM_STATE (ms->accelerator),KVM_CAP_VM_TYPES ) &BIT (kvm_type))) { +error_report ("vm-type %s not supported by KVM" , vm_type_name[kvm_type]); +exit (1 ); + } + +return kvm_type; +} +bool kvm_has_smm (void ) {return kvm_vm_check_extension (kvm_state,KVM_CAP_X86_SMM );
target/i386/kvm/kvm_i386.h +1 -0
@@ -37,6 +37,7 @@ bool kvm_hv_vpindex_settable(void);bool kvm_enable_sgx_provisioning (KVMState *s);bool kvm_hyperv_expand_features (X86CPU *cpu,Error **errp); +int kvm_get_vm_type (MachineState *ms,const char *vm_type);void kvm_arch_reset_vcpu (X86CPU *cs);void kvm_arch_after_reset_vcpu (X86CPU *cpu);void kvm_arch_do_init_vcpu (X86CPU *cs);
Signature · 1 line
2.34.1
PATCH v5 14/65 i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context
In reply to Xiaoyao LiImplement TDX specific ConfidentialGuestSupportClass::kvm_init() callback, tdx_kvm_init().
Set ms->require_guest_memfd to true to require private guest memfd allocation for any memory backend.
More TDX specific initialization will be added later.
---
Changes in v5:
- remove Acked-by from Gerd since the patch changed due to use ConfidentialGuestSupportClass::kvm_init();
--- target/i386/kvm/kvm.c | 11 +---------- target/i386/kvm/tdx.c | 13 +++++++++++++ 2 files changed, 14 insertions(+), 10 deletions(-)
target/i386/kvm/kvm.c +1 -10
@@ -2550,16 +2550,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)Error *local_err =NULL ;/* -* Initialize SEV context, if required -* -* If no memory encryption is requested (ms->cgs == NULL) this is -* a no-op. -* -* It's also a no-op if a non-SEV confidential guest support -* mechanism is selected. SEV is the only mechanism available to -* select on x86 at present, so this doesn't arise, but if new -* mechanisms are supported in future (e.g. TDX), they'll need -* their own initialization either here or elsewhere. +* Initialize confidential guest (SEV/TDX) context, if required */ if (ms->cgs) { ret =confidential_guest_kvm_init (ms->cgs, &local_err);
target/i386/kvm/tdx.c +13 -0
@@ -14,8 +14,18 @@#include "qemu/osdep.h" #include "qom/object_interfaces.h" +#include "hw/i386/x86.h" #include "tdx.h" +static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) +{ +MachineState *ms =MACHINE (qdev_get_machine ()); + + ms->require_guest_memfd = true; + +return 0 ; +} +/* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,
@@ -37,4 +47,7 @@ static void tdx_guest_finalize(Object *obj)static void tdx_guest_class_init (ObjectClass *oc,void *data) { +ConfidentialGuestSupportClass *klass =CONFIDENTIAL_GUEST_SUPPORT_CLASS (oc); + + klass->kvm_init = tdx_kvm_init; }
Signature · 1 line
2.34.1
PATCH v5 15/65 i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
In reply to Xiaoyao LiKVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.
Since there is no interface reporting how many cpuid configs contains in KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
Besides, introduce the interfaces to invoke TDX "ioctls" at different scope (KVM, VM and VCPU) in preparation.
---
Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;
Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();
changes in v2:
- Make the error message more clear;
changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
--- target/i386/kvm/kvm.c | 2 - target/i386/kvm/kvm_i386.h | 2 + target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++- 3 files changed, 92 insertions(+), 3 deletions(-)
target/i386/kvm/kvm.c +0 -2
@@ -1685,8 +1685,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)static Error *invtsc_mig_blocker; -#define KVM_MAX_CPUID_ENTRIES 100 -static void kvm_init_xsave (CPUX86State *env) {if (has_xsave2) {
target/i386/kvm/kvm_i386.h +2 -0
@@ -13,6 +13,8 @@#include "sysemu/kvm.h" +#define KVM_MAX_CPUID_ENTRIES 100 +#ifdef CONFIG_KVM #define kvm_pit_in_kernel () \
target/i386/kvm/tdx.c +90 -1
108 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
Quoted 105 lines · Xiaoyao Li
KVM provides TDX capabilities via sub command
KVM_TDX_CAPABILITIESofIOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.Since there is no interface reporting how many cpuid configs contains in
KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceedsKVM_MAX_CPUID_ENTRIES.Besides, introduce the interfaces to invoke TDX "ioctls" at different scope (KVM, VM and VCPU) in preparation.
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- use {} to initialize structkvm_tdx_cmd, to avoidmemset();
- removetdx_platform_ioctl()because no user;Changes in v3:
- rename__tdx_ioctl()totdx_ioctl_internal()
- Pass errp inget_tdx_capabilities();changes in v2:
- Make the error message more clear;changes in v1:
- start fromnr_cpuid_configs= 6 for the loop;
- stop the loop whennr_cpuid_configsexceedsKVM_MAX_CPUID_ENTRIES;--- target/i386/kvm/kvm.c | 2 - target/i386/kvm/kvm_i386.h | 2 + target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++- 3 files changed, 92 insertions(+), 3 deletions(-)target/i386/kvm/kvm.c +0
-2@@ -1685,8 +1685,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)static Error *invtsc_mig_blocker; -#define KVM_MAX_CPUID_ENTRIES 100 -static void kvm_init_xsave (CPUX86State *env) {if (has_xsave2) {target/i386/kvm/kvm_i386.h +2
-0@@ -13,6 +13,8 @@#include "sysemu/kvm.h" +#define KVM_MAX_CPUID_ENTRIES 100 +#ifdef CONFIG_KVM #define kvm_pit_in_kernel () \target/i386/kvm/tdx.c +33
-0@@ -12,18 +12,107 @@*/ #include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "qapi/error.h" #include "qom/object_interfaces.h" +#include "sysemu/kvm.h" #include "hw/i386/x86.h" +#include "kvm_i386.h" #include "tdx.h" +static struct kvm_tdx_capabilities *tdx_caps; + +enum tdx_ioctl_level { +TDX_VM_IOCTL , +TDX_VCPU_IOCTL , +}; + +static int tdx_ioctl_internal (void *state,enum tdx_ioctl_level level,int cmd_id, +__u32 flags,void *data) +{ +struct kvm_tdx_cmd tdx_cmd = {}; +int r; + + tdx_cmd.id = cmd_id; + tdx_cmd.flags = flags; + tdx_cmd.data = (__u64 )(unsigned long )data; + +switch (level) { +case TDX_VM_IOCTL : + r =kvm_vm_ioctl (kvm_state,KVM_MEMORY_ENCRYPT_OP , &tdx_cmd); +break ; +case TDX_VCPU_IOCTL : + r =kvm_vcpu_ioctl (state,KVM_MEMORY_ENCRYPT_OP , &tdx_cmd); +break ; +default : +error_report ("Invalid tdx_ioctl_level %d" , level); +exit (1 ); + } +
target/i386/kvm/tdx.c line 62
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+return r; +} + +static inline int tdx_vm_ioctl (int cmd_id,__u32 flags,void *data) +{ +return tdx_ioctl_internal (NULL ,TDX_VM_IOCTL , cmd_id, flags, data); +} + +static inline int tdx_vcpu_ioctl (void *vcpu_fd,int cmd_id,__u32 flags, +void *data)
As kvm_vcpu_ioctl(CPUState *cpu, int type, ...) takes CPUState *, this can be tdx_vcpu_ioctl(CPUState *cpu, ) instead of void *.
I struggled to fin my mistake to pass "int vcpu_fd" to this function.
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
In reply to Xiaoyao LiOn 2/29/2024 14:36, Xiaoyao Li wrote:> KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
Quoted 9 lines
IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.Since there is no interface reporting how many cpuid configs contains in
KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceedsKVM_MAX_CPUID_ENTRIES.Besides, introduce the interfaces to invoke TDX "ioctls" at different scope (KVM, VM and VCPU) in preparation.
tdx_platform_ioctl() is dropped because no user so suggest rephrasing this statement because no KVM scope ioctl interface is introduced in this patch.
Quoted 162 lines
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- use {} to initialize structkvm_tdx_cmd, to avoidmemset();
- removetdx_platform_ioctl()because no user;Changes in v3:
- rename__tdx_ioctl()totdx_ioctl_internal()
- Pass errp inget_tdx_capabilities();changes in v2:
- Make the error message more clear;changes in v1:
- start fromnr_cpuid_configs= 6 for the loop;
- stop the loop whennr_cpuid_configsexceedsKVM_MAX_CPUID_ENTRIES;--- target/i386/kvm/kvm.c | 2 - target/i386/kvm/kvm_i386.h | 2 + target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++- 3 files changed, 92 insertions(+), 3 deletions(-)target/i386/kvm/kvm.c +0
-2@@ -1685,8 +1685,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)static Error *invtsc_mig_blocker; -#define KVM_MAX_CPUID_ENTRIES 100 -static void kvm_init_xsave (CPUX86State *env) {if (has_xsave2) {target/i386/kvm/kvm_i386.h +2
-0@@ -13,6 +13,8 @@#include "sysemu/kvm.h" +#define KVM_MAX_CPUID_ENTRIES 100 +#ifdef CONFIG_KVM #define kvm_pit_in_kernel () \target/i386/kvm/tdx.c +90
-1@@ -12,18 +12,107 @@*/ #include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "qapi/error.h" #include "qom/object_interfaces.h" +#include "sysemu/kvm.h" #include "hw/i386/x86.h" +#include "kvm_i386.h" #include "tdx.h" +static struct kvm_tdx_capabilities *tdx_caps; + +enum tdx_ioctl_level { +TDX_VM_IOCTL , +TDX_VCPU_IOCTL , +}; + +static int tdx_ioctl_internal (void *state,enum tdx_ioctl_level level,int cmd_id, +__u32 flags,void *data) +{ +struct kvm_tdx_cmd tdx_cmd = {}; +int r; + + tdx_cmd.id = cmd_id; + tdx_cmd.flags = flags; + tdx_cmd.data = (__u64 )(unsigned long )data; + +switch (level) { +case TDX_VM_IOCTL : + r =kvm_vm_ioctl (kvm_state,KVM_MEMORY_ENCRYPT_OP , &tdx_cmd); +break ; +case TDX_VCPU_IOCTL : + r =kvm_vcpu_ioctl (state,KVM_MEMORY_ENCRYPT_OP , &tdx_cmd); +break ; +default : +error_report ("Invalid tdx_ioctl_level %d" , level); +exit (1 ); + } + +return r; +} + +static inline int tdx_vm_ioctl (int cmd_id,__u32 flags,void *data) +{ +return tdx_ioctl_internal (NULL ,TDX_VM_IOCTL , cmd_id, flags, data); +} + +static inline int tdx_vcpu_ioctl (void *vcpu_fd,int cmd_id,__u32 flags, +void *data) +{ +return tdx_ioctl_internal (vcpu_fd,TDX_VCPU_IOCTL , cmd_id, flags, data); +} + +static int get_tdx_capabilities (Error **errp) +{ +struct kvm_tdx_capabilities *caps; +/* 1st generation of TDX reports 6 cpuid configs */ +int nr_cpuid_configs =6 ; +size_t size; +int r; + +do { + size =sizeof (struct kvm_tdx_capabilities ) + + nr_cpuid_configs *sizeof (struct kvm_tdx_cpuid_config ); + caps =g_malloc0 (size); + caps->nr_cpuid_configs = nr_cpuid_configs; + + r =tdx_vm_ioctl (KVM_TDX_CAPABILITIES ,0 , caps); +if (r == -E2BIG ) { +g_free (caps); + nr_cpuid_configs *=2 ; +if (nr_cpuid_configs >KVM_MAX_CPUID_ENTRIES ) { +error_setg (errp,"%s: KVM TDX seems broken that number of CPUID " +"entries in kvm_tdx_capabilities exceeds limit %d" , + __func__,KVM_MAX_CPUID_ENTRIES ); +return r; + } + }else if (r <0 ) { +g_free (caps); +error_setg_errno (errp, -r,"%s: KVM_TDX_CAPABILITIES failed" , __func__); +return r; + } + } +while (r == -E2BIG ); + + tdx_caps = caps; + +return 0 ; +} +static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ()); +int r =0 ; ms->require_guest_memfd = true; -return 0 ; +if (!tdx_caps) { + r =get_tdx_capabilities (errp); + } + +return r; }/* tdx guest */
PATCH v5 16/65 i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object
In reply to Xiaoyao LiIt will need special handling for TDX VMs all around the QEMU. Introduce is_tdx_vm() helper to query if it's a TDX VM.
Cache tdx_guest object thus no need to cast from ms->cgs every time.
---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();
--- target/i386/kvm/tdx.c | 15 ++++++++++++++- target/i386/kvm/tdx.h | 10 ++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +14 -1
@@ -21,8 +21,16 @@#include "kvm_i386.h" #include "tdx.h" +static TdxGuest *tdx_guest; +static struct kvm_tdx_capabilities *tdx_caps; +/* It's valid after kvm_arch_init()->kvm_tdx_init() */ +bool is_tdx_vm (void ) +{ +return !!tdx_guest; +} +enum tdx_ioctl_level {TDX_VM_IOCTL ,TDX_VCPU_IOCTL ,
@@ -104,15 +112,20 @@ static int get_tdx_capabilities(Error **errp)static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ()); +TdxGuest *tdx =TDX_GUEST (cgs);int r =0 ; ms->require_guest_memfd = true;if (!tdx_caps) { r =get_tdx_capabilities (errp); +if (r) { +return r; + } } -return r; + tdx_guest = tdx; +return 0 ; }/* tdx guest */
target/i386/kvm/tdx.h +10 -0
@@ -1,6 +1,10 @@#ifndef QEMU_I386_TDX_H #define QEMU_I386_TDX_H +#ifndef CONFIG_USER_ONLY +#include CONFIG_DEVICES /* CONFIG_TDX */ +#endif +#include "exec/confidential-guest-support.h" #define TYPE_TDX_GUEST "tdx-guest"
@@ -16,4 +20,10 @@ typedef struct TdxGuest { uint64_t attributes; /* TD attributes */ } TdxGuest; +#ifdef CONFIG_TDX +bool is_tdx_vm (void ); +#else +#define is_tdx_vm () 0 +#endif /* CONFIG_TDX */ +#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v5 17/65 i386/tdx: Adjust the supported CPUID based on TDX restrictions
In reply to Xiaoyao LiAccording to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types:
------------------------------------------------------------------------
1 | As configured | configurable by VMM, independent of native value;
------------------------------------------------------------------------
2 | As configured | configurable by VMM if the bit is supported natively
(if native) | Otherwise it equals as native(0).
------------------------------------------------------------------------
3 | Fixed | fixed to 0/1
------------------------------------------------------------------------
4 | Native | reflect the native value
------------------------------------------------------------------------
5 | Calculated | calculated by TDX module.
------------------------------------------------------------------------
6 | Inducing #VE | get #VE exception
------------------------------------------------------------------------
Note:
1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3.
2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec, TDX module injects #VE to TDs when those are queried. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM.
Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported CPUID bits for TDs. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions.
Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord.
Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields:
- tdx_fixed0/1: The bits that are fixed as 0/1;
- depends_on_vmm_cap: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them.
- inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM.
- supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs.
By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps:
- get the base of VMM supported feature set;
- if the leaf is not a FeatureWord just return VMM's value without modification;
- if the leaf is an inducing_ve type, applying supported_on_ve mask and return;
- include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.)
- apply fixed0/1 to it (it covers #3, and rectifies the previous step);
- add configurable bits (it covers the other part of type #1);
- fix the ones in vmm_fixup;
(Calculated type is ignored since it's determined at runtime).
--- target/i386/cpu.h | 16 +++ target/i386/kvm/kvm.c | 4 + target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 + 4 files changed, 286 insertions(+)
target/i386/cpu.h +16 -0
@@ -787,6 +787,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,/* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)/* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2)/* 1st Group of Advanced Bit Manipulation Extensions */
@@ -805,8 +807,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_EBX_INVPCID (1U << 10)/* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12)/* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15)/* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16)/* AVX-512 Doubleword & Quadword Instruction */
@@ -862,10 +868,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_AVX512VNNI (1U << 11)/* Support for VPOPCNT[B,W] and VPSHUFBITQMB */ #define CPUID_7_0_ECX_AVX512BITALG (1U << 12) +/* Intel Total Memory Encryption */ +#define CPUID_7_0_ECX_TME (1U << 13)/* POPCNT for vectors of DW/QW */ #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14) +/* Placeholder for bit 15 */ +#define CPUID_7_0_ECX_FZM (1U << 15)/* 5-level Page Tables */ #define CPUID_7_0_ECX_LA57 (1U << 16) +/* MAWAU for MPX */ +#define CPUID_7_0_ECX_MAWAU (31U << 17)/* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22)/* Bus Lock Debug Exception */
@@ -876,6 +888,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_MOVDIRI (1U << 27)/* Move 64 Bytes as Direct Store Instruction */ #define CPUID_7_0_ECX_MOVDIR64B (1U << 28) +/* ENQCMD and ENQCMDS instructions */ +#define CPUID_7_0_ECX_ENQCMD (1U << 29)/* Support SGX Launch Control */ #define CPUID_7_0_ECX_SGX_LC (1U << 30)/* Protection Keys for Supervisor-mode Pages */
@@ -893,6 +907,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_EDX_SERIALIZE (1U << 14)/* TSX Suspend Load Address Tracking instruction */ #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16) +/* PCONFIG instruction */ +#define CPUID_7_0_EDX_PCONFIG (1U << 18)/* Architectural LBRs */ #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)/* AMX_BF16 instruction */
target/i386/kvm/kvm.c +4 -0
@@ -520,6 +520,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function, ret |=1U <<KVM_HINTS_REALTIME ; } +if (is_tdx_vm ()) { +tdx_get_supported_cpuid (function, index, reg, &ret); + } +return ret; }
target/i386/kvm/tdx.c +263 -0
280 lines not on this page · raw message
target/i386/kvm/tdx.h +3 -0
@@ -26,4 +26,7 @@ bool is_tdx_vm(void);#define is_tdx_vm () 0#endif /* CONFIG_TDX */ +void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg, +uint32_t *ret); +#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
Quoted 40 lines · Xiaoyao Li
According to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types:
------------------------------------------------------------------------
1 | As configured | configurable by VMM, independent of native value;
------------------------------------------------------------------------
2 | As configured | configurable by VMM if the bit is supported natively
(if native) | Otherwise it equals as native(0).
------------------------------------------------------------------------
3 | Fixed | fixed to 0/1
------------------------------------------------------------------------
4 | Native | reflect the native value
------------------------------------------------------------------------
5 | Calculated | calculated by TDX module.
------------------------------------------------------------------------
6 | Inducing #VE | get #VE exception
------------------------------------------------------------------------Note:
1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3.2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec, TDX module injects #VE to TDs when those are queried. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM.
Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via
KVM_GET_SUPPORTED_CPUIDdiverges from the supported CPUID bits for TDs. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions.Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord.
Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields:
Quoted 8 lines
- tdx_fixed0/1: The bits that are fixed as 0/1;
-
depends_on_vmm_cap: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them.
Previously I thought bits configurable for TDX module are emulated by TDX module,
it looks not. Just curious why doesn't those bits belong to "Inducing #VE" type?
Then guest can get KVM reported capabilities with tdvmcall directly.
Quoted 7 lines
-
inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM.-
supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs.
This is never used together with depends_on_vmm_cap, maybe one variable is enough?
Quoted 355 lines
By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps:
- get the base of VMM supported feature set;
- if the leaf is not a FeatureWord just return VMM's value without modification;
- if the leaf is an
inducing_vetype, applyingsupported_on_vemask and return;- include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.)
- apply fixed0/1 to it (it covers #3, and rectifies the previous step);
- add configurable bits (it covers the other part of type #1);
- fix the ones in
vmm_fixup;(Calculated type is ignored since it's determined at runtime).
Co-developed-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>--- target/i386/cpu.h | 16 +++ target/i386/kvm/kvm.c | 4 + target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 + 4 files changed, 286 insertions(+)target/i386/cpu.h +16
-0@@ -787,6 +787,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,/* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)/* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2)/* 1st Group of Advanced Bit Manipulation Extensions */ @@ -805,8 +807,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_EBX_INVPCID (1U << 10)/* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12)/* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15)/* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16)/* AVX-512 Doubleword & Quadword Instruction */ @@ -862,10 +868,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_AVX512VNNI (1U << 11)/* Support for VPOPCNT[B,W] and VPSHUFBITQMB */ #define CPUID_7_0_ECX_AVX512BITALG (1U << 12) +/* Intel Total Memory Encryption */ +#define CPUID_7_0_ECX_TME (1U << 13)/* POPCNT for vectors of DW/QW */ #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14) +/* Placeholder for bit 15 */ +#define CPUID_7_0_ECX_FZM (1U << 15)/* 5-level Page Tables */ #define CPUID_7_0_ECX_LA57 (1U << 16) +/* MAWAU for MPX */ +#define CPUID_7_0_ECX_MAWAU (31U << 17)/* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22)/* Bus Lock Debug Exception */ @@ -876,6 +888,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_MOVDIRI (1U << 27)/* Move 64 Bytes as Direct Store Instruction */ #define CPUID_7_0_ECX_MOVDIR64B (1U << 28) +/* ENQCMD and ENQCMDS instructions */ +#define CPUID_7_0_ECX_ENQCMD (1U << 29)/* Support SGX Launch Control */ #define CPUID_7_0_ECX_SGX_LC (1U << 30)/* Protection Keys for Supervisor-mode Pages */ @@ -893,6 +907,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_EDX_SERIALIZE (1U << 14)/* TSX Suspend Load Address Tracking instruction */ #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16) +/* PCONFIG instruction */ +#define CPUID_7_0_EDX_PCONFIG (1U << 18)/* Architectural LBRs */ #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)/* AMX_BF16 instruction */ target/i386/kvm/kvm.c +4
-0@@ -520,6 +520,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function, ret |=1U <<KVM_HINTS_REALTIME ; } +if (is_tdx_vm ()) { +tdx_get_supported_cpuid (function, index, reg, &ret); + } +return ret; }target/i386/kvm/tdx.c +224
-0@@ -15,11 +15,129 @@#include "qemu/error-report.h" #include "qapi/error.h" #include "qom/object_interfaces.h" +#include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" +#include "sysemu/sysemu.h" #include "hw/i386/x86.h" #include "kvm_i386.h" #include "tdx.h" +#include "../cpu-internal.h" + +#define TDX_SUPPORTED_KVM_FEATURES ((1U << KVM_FEATURE_NOP_IO_DELAY) | \ + (1U << KVM_FEATURE_PV_UNHALT) | \ + (1U << KVM_FEATURE_PV_TLB_FLUSH) | \ + (1U << KVM_FEATURE_PV_SEND_IPI) | \ + (1U << KVM_FEATURE_POLL_CONTROL) | \ + (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ + (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) + +typedef struct KvmTdxCpuidLookup { +uint32_t tdx_fixed0; +uint32_t tdx_fixed1; + +/* +* The CPUID bits that are configurable from the view of TDX module +* but require VMM's support when wanting to enable them. +* +* For those bits, they cannot be enabled if VMM (KVM/QEMU) doesn't support +* them. +*/ +uint32_t depends_on_vmm_cap; + +bool inducing_ve; +/* +* The maximum supported feature set for given inducing-#VE leaf. +* It's valid only when .inducing_ve is true. +*/ +uint32_t supported_value_on_ve; +}KvmTdxCpuidLookup ; + +/* +* QEMU maintained TDX CPUID lookup tables, which reflects how CPUIDs are +* virtualized for guest TDs based on "CPUID virtualization" of TDX spec. +* +* Note: +* +* This table will be updated runtime by tdx_caps reported by KVM. +* +*/ +static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS ] = { + [FEAT_1_EDX ] = { + .tdx_fixed0 = +BIT (10 )/* Reserved */ |BIT (20 )/* Reserved */ |CPUID_IA64 , + .tdx_fixed1 = +CPUID_MSR |CPUID_PAE |CPUID_MCE |CPUID_APIC | +CPUID_MTRR |CPUID_MCA |CPUID_CLFLUSH |CPUID_DTS , + .depends_on_vmm_cap = +CPUID_ACPI |CPUID_PBE , + }, + [FEAT_1_ECX ] = { + .tdx_fixed0 = +CPUID_EXT_VMX |CPUID_EXT_SMX |BIT (16 )/* Reserved */ , + .tdx_fixed1 = +CPUID_EXT_CX16 |CPUID_EXT_PDCM |CPUID_EXT_X2APIC | +CPUID_EXT_AES |CPUID_EXT_XSAVE |CPUID_EXT_RDRAND | +CPUID_EXT_HYPERVISOR , + .depends_on_vmm_cap = +CPUID_EXT_EST |CPUID_EXT_TM2 |CPUID_EXT_XTPR |CPUID_EXT_DCA , + }, + [FEAT_8000_0001_EDX ] = { + .tdx_fixed1 = +CPUID_EXT2_NX |CPUID_EXT2_PDPE1GB |CPUID_EXT2_RDTSCP | +CPUID_EXT2_LM , + }, + [FEAT_7_0_EBX ] = { + .tdx_fixed0 = +CPUID_7_0_EBX_TSC_ADJUST |CPUID_7_0_EBX_SGX |CPUID_7_0_EBX_MPX , + .tdx_fixed1 = +CPUID_7_0_EBX_FSGSBASE |CPUID_7_0_EBX_RTM | +CPUID_7_0_EBX_RDSEED |CPUID_7_0_EBX_SMAP | +CPUID_7_0_EBX_CLFLUSHOPT |CPUID_7_0_EBX_CLWB | +CPUID_7_0_EBX_SHA_NI , + .depends_on_vmm_cap = +CPUID_7_0_EBX_PQM |CPUID_7_0_EBX_RDT_A , + }, + [FEAT_7_0_ECX ] = { + .tdx_fixed0 = +CPUID_7_0_ECX_FZM |CPUID_7_0_ECX_MAWAU | +CPUID_7_0_ECX_ENQCMD |CPUID_7_0_ECX_SGX_LC , + .tdx_fixed1 = +CPUID_7_0_ECX_MOVDIR64B |CPUID_7_0_ECX_BUS_LOCK_DETECT , + .depends_on_vmm_cap = +CPUID_7_0_ECX_TME , + }, + [FEAT_7_0_EDX ] = { + .tdx_fixed1 = +CPUID_7_0_EDX_SPEC_CTRL |CPUID_7_0_EDX_ARCH_CAPABILITIES | +CPUID_7_0_EDX_CORE_CAPABILITY |CPUID_7_0_EDX_SPEC_CTRL_SSBD , + .depends_on_vmm_cap = +CPUID_7_0_EDX_PCONFIG , + }, + [FEAT_8000_0008_EBX ] = { + .tdx_fixed0 = + ~CPUID_8000_0008_EBX_WBNOINVD , + .tdx_fixed1 = +CPUID_8000_0008_EBX_WBNOINVD , + }, + [FEAT_XSAVE ] = { + .tdx_fixed1 = +CPUID_XSAVE_XSAVEOPT |CPUID_XSAVE_XSAVEC | +CPUID_XSAVE_XSAVES , + }, + [FEAT_6_EAX ] = { + .inducing_ve = true, + .supported_value_on_ve =CPUID_6_EAX_ARAT , + }, + [FEAT_8000_0007_EDX ] = { + .inducing_ve = true, + .supported_value_on_ve =-1U , + }, + [FEAT_KVM ] = { + .inducing_ve = true, + .supported_value_on_ve =TDX_SUPPORTED_KVM_FEATURES , + }, +};static TdxGuest *tdx_guest;@@ -31,6 +149,151 @@ bool is_tdx_vm(void)return !!tdx_guest; } +static inline uint32_t host_cpuid_reg (uint32_t function, +uint32_t index,int reg) +{ +uint32_t eax, ebx, ecx, edx; +uint32_t ret =0 ; + +host_cpuid (function, index, &eax, &ebx, &ecx, &edx); + +switch (reg) { +case R_EAX : + ret = eax; +break ; +case R_EBX : + ret = ebx; +break ; +case R_ECX : + ret = ecx; +break ; +case R_EDX : + ret = edx; +break ; + } +return ret; +} + +/* +* get the configurable cpuid bits (can be set to 0 or 1) reported by TDX module +* from tdx_caps. +*/ +static inline uint32_t tdx_cap_cpuid_config (uint32_t function, +uint32_t index,int reg) +{ +struct kvm_tdx_cpuid_config *cpuid_c; +int ret =0 ; +int i; + +if (tdx_caps->nr_cpuid_configs <=0 ) { +return ret; + } + +for (i =0 ; i < tdx_caps->nr_cpuid_configs; i++) { + cpuid_c = &tdx_caps->cpuid_configs[i]; +/* 0xffffffff in sub_leaf means the leaf doesn't require a sublesf */ +if (cpuid_c->leaf == function && + (cpuid_c->sub_leaf ==0xffffffff || cpuid_c->sub_leaf == index)) { +switch (reg) { +case R_EAX : + ret = cpuid_c->eax; +break ; +case R_EBX : + ret = cpuid_c->ebx; +break ; +case R_ECX : + ret = cpuid_c->ecx; +break ; +case R_EDX : + ret = cpuid_c->edx; +break ; +default : +return 0 ; + } + } + } +return ret; +} + +static FeatureWord get_cpuid_featureword_index (uint32_t function, +uint32_t index,int reg) +{ +FeatureWord w; + +for (w =0 ; w <FEATURE_WORDS ; w++) { +FeatureWordInfo *f = &feature_word_info[w]; + +if (f->type ==MSR_FEATURE_WORD || f->cpuid.eax != function || + f->cpuid.reg != reg || + (f->cpuid.needs_ecx && f->cpuid.ecx != index)) { +continue ; + } + +return w; + } + +return w; +} + +/* +* TDX supported CPUID varies from what KVM reports. Adjust the result by +* applying the TDX restrictions. +*/ +void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg, +uint32_t *ret) +{ +/* +* it's KVMM + QEMU 's capabilities of what CPUID bits is supported or +* can be emulated as supported. +*/ +uint32_t vmm_cap = *ret; +FeatureWord w; + +/* Only handle features leaves that recognized by feature_word_info[] */ + w =get_cpuid_featureword_index (function, index, reg); +if (w ==FEATURE_WORDS ) { +return ; + } +
target/i386/kvm/tdx.c bool is_tdx_vm(void) · line 267
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+if (tdx_cpuid_lookup[w].inducing_ve) { + *ret &= tdx_cpuid_lookup[w].supported_value_on_ve; +return ; + } + +/* +* Include all the native bits as first step. It covers types +* - As configured (if native) +* - Native +* - XFAM related and Attributes realted
s/realted/related
target/i386/kvm/tdx.c bool is_tdx_vm(void) · line 273
Quoted 6 lines
target/i386/kvm/tdx.c +6
-0+* +* It also has side effect to enable unsupported bits, e.g., the +* bits of "fixed0" type while present natively. It's safe because +* the unsupported bits will be masked off by .fixed0 later. +*/ + *ret |=host_cpuid_reg (function, index, reg);
Looks KVM capabilities are merged with native bits, is this intentional?
Thanks
Zhenzhong
Quoted 38 lines
target/i386/kvm/tdx.c +23
-0+ +/* Adjust according to "fixed" type in tdx_cpuid_lookup. */ + *ret |= tdx_cpuid_lookup[w].tdx_fixed1; + *ret &= ~tdx_cpuid_lookup[w].tdx_fixed0; + +/* +* Configurable cpuids are supported unconditionally. It's mainly to +* include those configurable regardless of native existence. +*/ + *ret |=tdx_cap_cpuid_config (function, index, reg); + +/* +* clear the configurable bits that require VMM emulation and VMM doesn't +* report the support. +*/ + *ret &= ~(tdx_cpuid_lookup[w].depends_on_vmm_cap & ~vmm_cap); + +/* special handling */ +if (function ==1 && reg ==R_ECX && !enable_cpu_pm) { + *ret &= ~CPUID_EXT_MONITOR ; + } +} +enum tdx_ioctl_level {TDX_VM_IOCTL ,TDX_VCPU_IOCTL ,target/i386/kvm/tdx.h +3
-0@@ -26,4 +26,7 @@ bool is_tdx_vm(void);#define is_tdx_vm () 0#endif /* CONFIG_TDX */ +void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg, +uint32_t *ret); +#endif /* QEMU_I386_TDX_H */
Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
Quoted 51 lines · Zhenzhong
Quoted 40 lines · Xiaoyao Li
According to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types:
------------------------------------------------------------------------
1 | As configured | configurable by VMM, independent of native value;
------------------------------------------------------------------------
2 | As configured | configurable by VMM if the bit is supported natively (if native) | Otherwise it equals as native(0).
------------------------------------------------------------------------
3 | Fixed | fixed to 0/1
------------------------------------------------------------------------
4 | Native | reflect the native value
------------------------------------------------------------------------
5 | Calculated | calculated by TDX module.
------------------------------------------------------------------------
6 | Inducing #VE | get #VE exception
------------------------------------------------------------------------Note:
1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3.2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec, TDX module injects #VE to TDs when those are queried. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM.
Due to TDX module has its own virtualization policy on CPUID bits, it leads to what reported via
KVM_GET_SUPPORTED_CPUIDdiverges from the supported CPUID bits for TDs. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions.Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord.
Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields:
Quoted 8 lines
- tdx_fixed0/1: The bits that are fixed as 0/1;
-
depends_on_vmm_cap: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them.
Quoted 3 lines
Previously I thought bits configurable for TDX module are emulated by TDX module,
it looks not. Just curious why doesn't those bits belong to "Inducing #VE" type?
Because when TD guest queries this type of CPUID leaf, it doesn't get #VE.
The bits in this category are free to be configured as on/off by VMM and passed to TDX module via TD_PARAM. Once they get configured, they are queried directly by TD guest without getting #VE.
The problem is whether VMM allows them to be configured freely. E.g., for features TME and PCONFIG, they are configurable. However when VMM configures them to 1, VMM needs to provide the support of related MSRs of them. If VMM cannot provide such support, VMM should be allow user to configured to 1.
That's why we have this kind of type.
BTW, we are going to abondan the solution that let QEMU to maintain the CPUID configurability table in this series. Next version we will come up with
Quoted 393 lines
Then guest can get KVM reported capabilities with tdvmcall directly.
Quoted 7 lines
-
inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM.-
supported_on_ve: It's valid only when @inducing_ve is true. It represents the maximum feature set supported that be emulated for TDs.This is never used together with
depends_on_vmm_cap, maybe one variable is enough?Quoted 372 lines
By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps:
- get the base of VMM supported feature set;
- if the leaf is not a FeatureWord just return VMM's value without modification;
- if the leaf is an
inducing_vetype, applyingsupported_on_vemask and return;- include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.)
- apply fixed0/1 to it (it covers #3, and rectifies the previous step);
- add configurable bits (it covers the other part of type #1);
- fix the ones in
vmm_fixup;(Calculated type is ignored since it's determined at runtime).
Co-developed-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
target/i386/cpu.h | 16 +++target/i386/kvm/kvm.c | 4 + target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 + 4 files changed, 286 insertions(+)target/i386/cpu.h +6
-0@@ -787,6 +787,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1) /* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2) /* 1st Group of Advanced Bit Manipulation Extensions */@@ -805,8 +807,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EBX_INVPCID (1U << 10) /* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12) /* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15) /* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16)/* AVX-512 Doubleword & Quadword Instruction */
target/i386/cpu.h +6
-0@@ -862,10 +868,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_ECX_AVX512VNNI (1U << 11) /* Support for VPOPCNT[B,W] and VPSHUFBITQMB */ #define CPUID_7_0_ECX_AVX512BITALG (1U << 12) +/* Intel Total Memory Encryption */ +#define CPUID_7_0_ECX_TME (1U << 13) /* POPCNT for vectors of DW/QW */ #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14) +/* Placeholder for bit 15 */ +#define CPUID_7_0_ECX_FZM (1U << 15) /* 5-level Page Tables */ #define CPUID_7_0_ECX_LA57 (1U << 16) +/* MAWAU for MPX */ +#define CPUID_7_0_ECX_MAWAU (31U << 17) /* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22)/* Bus Lock Debug Exception */
target/i386/cpu.h +2
-0@@ -876,6 +888,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_ECX_MOVDIRI (1U << 27) /* Move 64 Bytes as Direct Store Instruction */ #define CPUID_7_0_ECX_MOVDIR64B (1U << 28) +/* ENQCMD and ENQCMDS instructions */ +#define CPUID_7_0_ECX_ENQCMD (1U << 29) /* Support SGX Launch Control */ #define CPUID_7_0_ECX_SGX_LC (1U << 30)/* Protection Keys for Supervisor-mode Pages */
target/i386/cpu.h +2
-0@@ -893,6 +907,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EDX_SERIALIZE (1U << 14) /* TSX Suspend Load Address Tracking instruction */ #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16) +/* PCONFIG instruction */ +#define CPUID_7_0_EDX_PCONFIG (1U << 18) /* Architectural LBRs */ #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)/*
AMX_BF16instruction */target/i386/kvm/kvm.c +4
-0@@ -520,6 +520,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function, ret |= 1U << KVM_HINTS_REALTIME; } + if (is_tdx_vm()) { + tdx_get_supported_cpuid(function, index, reg, &ret); + } + return ret; }target/i386/kvm/tdx.c +11
-0@@ -15,11 +15,129 @@ #include "qemu/error-report.h" #include "qapi/error.h" #include "qom/object_interfaces.h" +#include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" +#include "sysemu/sysemu.h" #include "hw/i386/x86.h" #include "kvm_i386.h" #include "tdx.h" +#include "../cpu-internal.h" + +#define TDX_SUPPORTED_KVM_FEATURES ((1U << KVM_FEATURE_NOP_IO_DELAY) | \ + (1U << KVM_FEATURE_PV_UNHALT) | \ + (1U << KVM_FEATURE_PV_TLB_FLUSH) | \ + (1U << KVM_FEATURE_PV_SEND_IPI) | \ + (1U << KVM_FEATURE_POLL_CONTROL) | \ + (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ + (1U <<
KVM_FEATURE_MSI_EXT_DEST_ID))target/i386/kvm/tdx.c +10
-0+ +typedef struct KvmTdxCpuidLookup { + uint32_t tdx_fixed0; + uint32_t tdx_fixed1; + + /* + * The CPUID bits that are configurable from the view of TDX module + * but require VMM's support when wanting to enable them. + * + * For those bits, they cannot be enabled if VMM (KVM/QEMU)doesn't support
+ * them.
+ */
+uint32_tdepends_on_vmm_cap;target/i386/kvm/tdx.c +11
-0+ + bool inducing_ve; + /* + * The maximum supported feature set for given inducing-#VE leaf. + * It's valid only when .inducing_ve is true. + */ + uint32_t supported_value_on_ve; +} KvmTdxCpuidLookup; + +/* +* QEMU maintained TDX CPUID lookup tables, which reflects how CPUIDs are
+ * virtualized for guest TDs based on "CPUID virtualization" of TDX spec.
+ *
+ * Note:
+ *
+ * This table will be updated runtime bytdx_capsreported by KVM. + *
+ */
+static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS] = { + [FEAT_1_EDX] = {
+ .tdx_fixed0 =
+ BIT(10) /* Reserved */ | BIT(20) /* Reserved */ |CPUID_IA64,
+ .tdx_fixed1 =
+CPUID_MSR|CPUID_PAE|CPUID_MCE|CPUID_APIC| +CPUID_MTRR|CPUID_MCA|CPUID_CLFLUSH|CPUID_DTS, + .depends_on_vmm_cap =
+CPUID_ACPI|CPUID_PBE,
+ },
+ [FEAT_1_ECX] = {
+ .tdx_fixed0 =
+CPUID_EXT_VMX|CPUID_EXT_SMX| BIT(16) /* Reserved */, + .tdx_fixed1 =+ CPUID_EXT_CX16 | CPUID_EXT_PDCM | CPUID_EXT_X2APIC | + CPUID_EXT_AES | CPUID_EXT_XSAVE | CPUID_EXT_RDRAND |+
CPUID_EXT_HYPERVISOR,
+ .depends_on_vmm_cap =
+CPUID_EXT_EST|CPUID_EXT_TM2|CPUID_EXT_XTPR|CPUID_EXT_DCA,
+ },
+ [FEAT_8000_0001_EDX] = {
+ .tdx_fixed1 =
+CPUID_EXT2_NX|CPUID_EXT2_PDPE1GB|CPUID_EXT2_RDTSCP| +CPUID_EXT2_LM,
+ },
+ [FEAT_7_0_EBX] = {
+ .tdx_fixed0 =
+CPUID_7_0_EBX_TSC_ADJUST|CPUID_7_0_EBX_SGX|CPUID_7_0_EBX_MPX,
+ .tdx_fixed1 =
+CPUID_7_0_EBX_FSGSBASE|CPUID_7_0_EBX_RTM|
+CPUID_7_0_EBX_RDSEED|CPUID_7_0_EBX_SMAP|
+CPUID_7_0_EBX_CLFLUSHOPT|CPUID_7_0_EBX_CLWB| +CPUID_7_0_EBX_SHA_NI,
+ .depends_on_vmm_cap =
+CPUID_7_0_EBX_PQM|CPUID_7_0_EBX_RDT_A,
+ },
+ [FEAT_7_0_ECX] = {
+ .tdx_fixed0 =
+CPUID_7_0_ECX_FZM|CPUID_7_0_ECX_MAWAU|
+CPUID_7_0_ECX_ENQCMD|CPUID_7_0_ECX_SGX_LC,
+ .tdx_fixed1 =
+CPUID_7_0_ECX_MOVDIR64B|CPUID_7_0_ECX_BUS_LOCK_DETECT, + .depends_on_vmm_cap =
+CPUID_7_0_ECX_TME,
+ },
+ [FEAT_7_0_EDX] = {
+ .tdx_fixed1 =
+CPUID_7_0_EDX_SPEC_CTRL|CPUID_7_0_EDX_ARCH_CAPABILITIES| +CPUID_7_0_EDX_CORE_CAPABILITY|CPUID_7_0_EDX_SPEC_CTRL_SSBD,
+ .depends_on_vmm_cap =
+CPUID_7_0_EDX_PCONFIG,
+ },
+ [FEAT_8000_0008_EBX] = {
+ .tdx_fixed0 =
+ ~CPUID_8000_0008_EBX_WBNOINVD,
+ .tdx_fixed1 =
+CPUID_8000_0008_EBX_WBNOINVD,
+ },
+ [FEAT_XSAVE] = {
+ .tdx_fixed1 =
+CPUID_XSAVE_XSAVEOPT|CPUID_XSAVE_XSAVEC|
+CPUID_XSAVE_XSAVES,
+ },
+ [FEAT_6_EAX] = {
+ .inducing_ve = true,
+ .supported_value_on_ve =CPUID_6_EAX_ARAT,
+ },
+ [FEAT_8000_0007_EDX] = {
+ .inducing_ve = true,
+ .supported_value_on_ve = -1U,
+ },
+ [FEAT_KVM] = {
+ .inducing_ve = true,
+ .supported_value_on_ve =TDX_SUPPORTED_KVM_FEATURES, + },
+};
static TdxGuest*tdx_guest;target/i386/kvm/tdx.c +27
-0@@ -31,6 +149,151 @@ bool is_tdx_vm(void) return !!tdx_guest; } +static inline uint32_t host_cpuid_reg (uint32_t function, + uint32_t index, int reg) +{ + uint32_t eax, ebx, ecx, edx; + uint32_t ret = 0; + + host_cpuid(function, index, &eax, &ebx, &ecx, &edx); + + switch (reg) { + case R_EAX: + ret = eax; + break; + case R_EBX: + ret = ebx; + break; + case R_ECX: + ret = ecx; + break; + case R_EDX: + ret = edx; + break; + } + return ret; +} + +/* +* get the configurable cpuid bits (can be set to 0 or 1) reported by TDX module
+ * fromtdx_caps.
+ */target/i386/kvm/tdx.c +14
-0+static inline uint32_t tdx_cap_cpuid_config (uint32_t function, + uint32_t index, int reg) +{ + struct kvm_tdx_cpuid_config *cpuid_c; + int ret = 0; + int i; + + if (tdx_caps->nr_cpuid_configs <= 0) { + return ret; + } + + for (i = 0; i < tdx_caps->nr_cpuid_configs; i++) { + cpuid_c = &tdx_caps->cpuid_configs[i]; + /* 0xffffffff in sub_leaf means the leaf doesn't require asublesf */
+ if (cpuid_c->leaf== function &&
+ (cpuid_c->sub_leaf==0xffffffff||cpuid_c->sub_leaf== index)) {
+ switch (reg) {
+ caseR_EAX:
+ ret =cpuid_c->eax;
+ break;
+ caseR_EBX:
+ ret =cpuid_c->ebx;
+ break;
+ caseR_ECX:
+ ret =cpuid_c->ecx;
+ break;
+ caseR_EDX:
+ ret =cpuid_c->edx;
+ break;
+ default:
+ return 0;
+ }
+ }
+ }
+ return ret;target/i386/kvm/tdx.c +24
-0+} + +static FeatureWord get_cpuid_featureword_index (uint32_t function, + uint32_t index, int reg) +{ + FeatureWord w; + + for (w = 0; w < FEATURE_WORDS; w++) { + FeatureWordInfo *f = &feature_word_info[w]; + + if (f->type == MSR_FEATURE_WORD || f->cpuid.eax != function || + f->cpuid.reg != reg || + (f->cpuid.needs_ecx && f->cpuid.ecx != index)) { + continue; + } + + return w; + } + + return w; +} + +/* +* TDX supported CPUID varies from what KVM reports. Adjust the result by
+ * applying the TDX restrictions.
+ */target/i386/kvm/tdx.c +5
-0+void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg, + uint32_t *ret) +{ + /* +* it's KVMM + QEMU 's capabilities of what CPUID bits is supported or
+ * can be emulated as supported.
+ */
+uint32_tvmm_cap= *ret;
+ FeatureWord w;
+
+ /* Only handle features leaves that recognized by feature_word_info[] */
+ w =get_cpuid_featureword_index(function, index, reg); + if (w ==FEATURE_WORDS) {
+ return;
+ }target/i386/kvm/tdx.c +1
-0+target/i386/kvm/tdx.c bool is_tdx_vm(void) · line 267
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ if (tdx_cpuid_lookup[w].inducing_ve) { + *ret &= tdx_cpuid_lookup[w].supported_value_on_ve; + return; + } + + /* +* Include all the native bits as first step. It covers types +* - As configured (if native) +* - Native +* - XFAM related and Attributes realted s/realted/related
Quoted 8 lines
Quoted 6 lines
+ *
+ * It also has side effect to enable unsupported bits, e.g., the + * bits of "fixed0" type while present natively. It's safe because + * the unsupported bits will be masked off by .fixed0 later. + */
+ *ret |=host_cpuid_reg(function, index, reg);Looks KVM capabilities are merged with native bits, is this intentional?
yes, if we change the order, it would be more clear for you I guess.
host_cpuid_reg() | kvm_capabilities
The base is host's native value, while any bit that absent from native but KVM can emulate is also added to base.
Quoted 42 lines
Thanks
Zhenzhong
Quoted 38 lines
target/i386/kvm/tdx.c +13
-0+ + /* Adjust according to "fixed" type in tdx_cpuid_lookup. */ + *ret |= tdx_cpuid_lookup[w].tdx_fixed1; + *ret &= ~tdx_cpuid_lookup[w].tdx_fixed0; + + /* + * Configurable cpuids are supported unconditionally. It's mainly to + * include those configurable regardless of native existence. + */ + *ret |= tdx_cap_cpuid_config(function, index, reg); + + /* +* clear the configurable bits that require VMM emulation and VMM doesn't
+ * report the support.
+ */
+ *ret &= ~(tdx_cpuid_lookup[w].depends_on_vmm_cap & ~vmm_cap);target/i386/kvm/tdx.c +7
-0+ + /* special handling */ + if (function == 1 && reg == R_ECX && !enable_cpu_pm) { + *ret &= ~CPUID_EXT_MONITOR; + } +} +enum tdx_ioctl_level{
TDX_VM_IOCTL,
TDX_VCPU_IOCTL,target/i386/kvm/tdx.h +3
-0@@ -26,4 +26,7 @@ bool is_tdx_vm(void); #define is_tdx_vm() 0 #endif /* CONFIG_TDX */ +void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg, + uint32_t *ret); + #endif /* QEMU_I386_TDX_H */
RE: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
Quoted 495 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictionsQuoted 25 lines · Zhenzhong
Quoted 23 lines · Xiaoyao Li
According to Chapter "CPUID Virtualization" in TDX module spec, CPUID bits of TD can be classified into 6 types:
------------------------------------------------------------------------
1 | As configured | configurable by VMM, independent of native value;
------------------------------------------------------------------------
2 | As configured | configurable by VMM if the bit is supported natively (if native) | Otherwise it equals as native(0).
------------------------------------------------------------------------
3 | Fixed | fixed to 0/1
------------------------------------------------------------------------
4 | Native | reflect the native value
------------------------------------------------------------------------
5 | Calculated | calculated by TDX module.
------------------------------------------------------------------------
6 | Inducing #VE | get #VE exception
------------------------------------------------------------------------Note:
1. All the configurable XFAM related features and TD attributes related features fall into type #2. And fixed0/1 bits of XFAM and TD attributes fall into type #3.Quoted 8 lines
Quoted 8 lines
2. For CPUID leaves not listed in "CPUID virtualization Overview" table in TDX module spec, TDX module injects #VE to TDs when those are queried. For this case, TDs can request CPUID emulation from VMM via TDVMCALL and the values are fully controlled by VMM.
Due to TDX module has its own virtualization policy on CPUID bits, it leads
to what reported viaKVM_GET_SUPPORTED_CPUIDdiverges from thesupported
Quoted 19 lines
Quoted 10 lines
CPUID bits for TDs. In order to keep a consistent CPUID configuration between VMM and TDs. Adjust supported CPUID for TDs based on TDX restrictions.
Currently only focus on the CPUID leaves recognized by QEMU's feature_word_info[] that are indexed by a FeatureWord.
Introduce a TDX CPUID lookup table, which maintains 1 entry for each FeatureWord. Each entry has below fields:
Quoted 8 lines
- tdx_fixed0/1: The bits that are fixed as 0/1;
-
depends_on_vmm_cap: The bits that are configurable from the view of TDX module. But they requires emulation of VMM when configured as enabled. For those, they are not supported if VMM doesn't report them as supported. So they need be fixed up by checking if VMM supports them.Quoted 5 lines
Previously I thought bits configurable for TDX module are emulated by TDX module,
it looks not. Just curious why doesn't those bits belong to "Inducing #VE" type?
Because when TD guest queries this type of CPUID leaf, it doesn't get #VE.
The bits in this category are free to be configured as on/off by VMM and passed to TDX module via
TD_PARAM. Once they get configured, they are queried directly by TD guest without getting #VE.The problem is whether VMM allows them to be configured freely. E.g., for features TME and PCONFIG, they are configurable. However when VMM configures them to 1, VMM needs to provide the support of related MSRs of them. If VMM cannot provide such support, VMM should be allow user to configured to 1.
That's why we have this kind of type.
BTW, we are going to abondan the solution that let QEMU to maintain the CPUID configurability table in this series. Next version we will come up with
Quoted 11 lines
Then guest can get KVM reported capabilities with tdvmcall directly.
Quoted 8 lines
-
inducing_ve: TD gets #VE when querying this CPUID leaf. The result is totally configurable by VMM.-
supported_on_ve: It's valid only when @inducing_ve is true. It represents
the maximum feature set supported that be emulated for TDs.This is never used together with
depends_on_vmm_cap, maybe onevariable
Quoted 33 lines
is enough?
Quoted 32 lines
By applying TDX CPUID lookup table and TDX capabilities reported from TDX module, the supported CPUID for TDs can be obtained from following steps:
- get the base of VMM supported feature set;
- if the leaf is not a FeatureWord just return VMM's value without modification;
- if the leaf is an
inducing_vetype, applyingsupported_on_vemask and return;- include all native bits, it covers type #2, #4, and parts of type #1. (it also includes some unsupported bits. The following step will correct it.)
- apply fixed0/1 to it (it covers #3, and rectifies the previous step);
- add configurable bits (it covers the other part of type #1);
- fix the ones in
vmm_fixup;(Calculated type is ignored since it's determined at runtime).
Co-developed-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
target/i386/cpu.h | 16 +++
target/i386/kvm/kvm.c| 4 +
target/i386/kvm/tdx.c| 263++++++++++++++++++++++++++++++++++++++++++
Quoted 65 lines
Quoted 65 lines
target/i386/kvm/tdx.h| 3 +4 files changed, 286 insertions(+)target/i386/cpu.h +6
-0@@ -787,6 +787,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */ #define CPUID_7_0_EBX_FSGSBASE (1U << 0) +/* Support for TSC adjustment MSR 0x3B */ +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1) /* Support SGX */ #define CPUID_7_0_EBX_SGX (1U << 2) /* 1st Group of Advanced Bit Manipulation Extensions */@@ -805,8 +807,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EBX_INVPCID (1U << 10) /* Restricted Transactional Memory */ #define CPUID_7_0_EBX_RTM (1U << 11) +/* Cache QoS Monitoring */ +#define CPUID_7_0_EBX_PQM (1U << 12) /* Memory Protection Extension */ #define CPUID_7_0_EBX_MPX (1U << 14) +/* Resource Director Technology Allocation */ +#define CPUID_7_0_EBX_RDT_A (1U << 15) /* AVX-512 Foundation */ #define CPUID_7_0_EBX_AVX512F (1U << 16)/* AVX-512 Doubleword & Quadword Instruction */
target/i386/cpu.h +6
-0@@ -862,10 +868,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_ECX_AVX512VNNI (1U << 11) /* Support for VPOPCNT[B,W] and VPSHUFBITQMB */ #define CPUID_7_0_ECX_AVX512BITALG (1U << 12) +/* Intel Total Memory Encryption */ +#define CPUID_7_0_ECX_TME (1U << 13) /* POPCNT for vectors of DW/QW */ #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14) +/* Placeholder for bit 15 */ +#define CPUID_7_0_ECX_FZM (1U << 15) /* 5-level Page Tables */ #define CPUID_7_0_ECX_LA57 (1U << 16) +/* MAWAU for MPX */ +#define CPUID_7_0_ECX_MAWAU (31U << 17) /* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22)/* Bus Lock Debug Exception */
target/i386/cpu.h +2
-0@@ -876,6 +888,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_ECX_MOVDIRI (1U << 27) /* Move 64 Bytes as Direct Store Instruction */ #define CPUID_7_0_ECX_MOVDIR64B (1U << 28) +/* ENQCMD and ENQCMDS instructions */ +#define CPUID_7_0_ECX_ENQCMD (1U << 29) /* Support SGX Launch Control */ #define CPUID_7_0_ECX_SGX_LC (1U << 30)/* Protection Keys for Supervisor-mode Pages */
target/i386/cpu.h +2
-0@@ -893,6 +907,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w, #define CPUID_7_0_EDX_SERIALIZE (1U << 14) /* TSX Suspend Load Address Tracking instruction */ #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16) +/* PCONFIG instruction */ +#define CPUID_7_0_EDX_PCONFIG (1U << 18)Quoted 8 lines
target/i386/kvm/kvm.c uint32_t
Quoted 8 lines
target/i386/cpu.h +0
-0/* Architectural LBRs */ #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)/*
AMX_BF16instruction */target/i386/kvm/kvm.c +0
-0@@ -520,6 +520,10 @@ uint32_tkvm_arch_get_supported_cpuid(KVMState
Quoted 18 lines
Quoted 18 lines
target/i386/kvm/kvm.c +4
-0*s, uint32_t function, ret |= 1U << KVM_HINTS_REALTIME; } + if (is_tdx_vm()) { + tdx_get_supported_cpuid(function, index, reg, &ret); + } + return ret; }target/i386/kvm/tdx.c +1
-0@@ -15,11 +15,129 @@ #include "qemu/error-report.h" #include "qapi/error.h" #include "qom/object_interfaces.h" +#include "standard-headers/asm-x86/kvm_para.h" Quoted 8 lines
target/i386/kvm/tdx.c
Quoted 8 lines
target/i386/kvm/tdx.c +4
-0#include "sysemu/kvm.h" +#include "sysemu/sysemu.h" #include "hw/i386/x86.h" #include "kvm_i386.h" #include "tdx.h" +#include "../cpu-internal.h" + +#define TDX_SUPPORTED_KVM_FEATURES ((1U <<
KVM_FEATURE_NOP_IO_DELAY)Quoted 62 lines
Quoted 62 lines
target/i386/kvm/tdx.c +6
-0| \ + (1U << KVM_FEATURE_PV_UNHALT) | \ + (1U << KVM_FEATURE_PV_TLB_FLUSH) | \ + (1U << KVM_FEATURE_PV_SEND_IPI) | \ + (1U << KVM_FEATURE_POLL_CONTROL) | \ + (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ + (1U <<
KVM_FEATURE_MSI_EXT_DEST_ID))target/i386/kvm/tdx.c +10
-0+ +typedef struct KvmTdxCpuidLookup { + uint32_t tdx_fixed0; + uint32_t tdx_fixed1; + + /* + * The CPUID bits that are configurable from the view of TDX module + * but require VMM's support when wanting to enable them. + * + * For those bits, they cannot be enabled if VMM (KVM/QEMU)doesn't support
+ * them.
+ */
+uint32_tdepends_on_vmm_cap;target/i386/kvm/tdx.c +11
-0+ + bool inducing_ve; + /* + * The maximum supported feature set for given inducing-#VE leaf. + * It's valid only when .inducing_ve is true. + */ + uint32_t supported_value_on_ve; +} KvmTdxCpuidLookup; + +/* +* QEMU maintained TDX CPUID lookup tables, which reflects how CPUIDs are
+ * virtualized for guest TDs based on "CPUID virtualization" of TDX spec.
+ *
+ * Note:
+ *
+ * This table will be updated runtime bytdx_capsreported by KVM. + *
+ */
+static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS] = { + [FEAT_1_EDX] = {
+ .tdx_fixed0 =
+ BIT(10) /* Reserved */ | BIT(20) /* Reserved */ |CPUID_IA64,
+ .tdx_fixed1 =
+CPUID_MSR|CPUID_PAE|CPUID_MCE|CPUID_APIC| +CPUID_MTRR|CPUID_MCA|CPUID_CLFLUSH|CPUID_DTS, + .depends_on_vmm_cap =
+CPUID_ACPI|CPUID_PBE,
+ },
+ [FEAT_1_ECX] = {
+ .tdx_fixed0 =
+CPUID_EXT_VMX|CPUID_EXT_SMX| BIT(16) /* Reserved */, + .tdx_fixed1 =+ CPUID_EXT_CX16 | CPUID_EXT_PDCM | CPUID_EXT_X2APIC | + CPUID_EXT_AES | CPUID_EXT_XSAVE | CPUID_EXT_RDRAND |Quoted 8 lines
Quoted 8 lines
+
CPUID_EXT_HYPERVISOR,
+ .depends_on_vmm_cap =
+CPUID_EXT_EST|CPUID_EXT_TM2|CPUID_EXT_XTPR|CPUID_EXT_DCA,
+ },
+ [FEAT_8000_0001_EDX] = {
+ .tdx_fixed1 =
+CPUID_EXT2_NX|CPUID_EXT2_PDPE1GB|CPUID_EXT2_RDTSCP|
Quoted 12 lines
Quoted 12 lines
+
CPUID_EXT2_LM,
+ },
+ [FEAT_7_0_EBX] = {
+ .tdx_fixed0 =
+CPUID_7_0_EBX_TSC_ADJUST|CPUID_7_0_EBX_SGX|CPUID_7_0_EBX_MPX,
+ .tdx_fixed1 =
+CPUID_7_0_EBX_FSGSBASE|CPUID_7_0_EBX_RTM| +CPUID_7_0_EBX_RDSEED|CPUID_7_0_EBX_SMAP| +CPUID_7_0_EBX_CLFLUSHOPT|CPUID_7_0_EBX_CLWB| +CPUID_7_0_EBX_SHA_NI,
+ .depends_on_vmm_cap =Quoted 8 lines
Quoted 8 lines
+
CPUID_7_0_EBX_PQM|CPUID_7_0_EBX_RDT_A, + },
+ [FEAT_7_0_ECX] = {
+ .tdx_fixed0 =
+CPUID_7_0_ECX_FZM|CPUID_7_0_ECX_MAWAU| +CPUID_7_0_ECX_ENQCMD|CPUID_7_0_ECX_SGX_LC, + .tdx_fixed1 =
+CPUID_7_0_ECX_MOVDIR64B|
CPUID_7_0_ECX_BUS_LOCK_DETECT,Quoted 6 lines
Quoted 6 lines
+ .depends_on_vmm_cap =
+CPUID_7_0_ECX_TME,
+ },
+ [FEAT_7_0_EDX] = {
+ .tdx_fixed1 =
+CPUID_7_0_EDX_SPEC_CTRL|
CPUID_7_0_EDX_ARCH_CAPABILITIES|Quoted 120 lines
Quoted 120 lines
+
CPUID_7_0_EDX_CORE_CAPABILITY|CPUID_7_0_EDX_SPEC_CTRL_SSBD,
+ .depends_on_vmm_cap =
+CPUID_7_0_EDX_PCONFIG,
+ },
+ [FEAT_8000_0008_EBX] = {
+ .tdx_fixed0 =
+ ~CPUID_8000_0008_EBX_WBNOINVD,
+ .tdx_fixed1 =
+CPUID_8000_0008_EBX_WBNOINVD,
+ },
+ [FEAT_XSAVE] = {
+ .tdx_fixed1 =
+CPUID_XSAVE_XSAVEOPT|CPUID_XSAVE_XSAVEC| +CPUID_XSAVE_XSAVES,
+ },
+ [FEAT_6_EAX] = {
+ .inducing_ve = true,
+ .supported_value_on_ve =CPUID_6_EAX_ARAT, + },
+ [FEAT_8000_0007_EDX] = {
+ .inducing_ve = true,
+ .supported_value_on_ve = -1U,
+ },
+ [FEAT_KVM] = {
+ .inducing_ve = true,
+ .supported_value_on_ve =TDX_SUPPORTED_KVM_FEATURES, + },
+};
static TdxGuest*tdx_guest;target/i386/kvm/tdx.c +27
-0@@ -31,6 +149,151 @@ bool is_tdx_vm(void) return !!tdx_guest; } +static inline uint32_t host_cpuid_reg (uint32_t function, + uint32_t index, int reg) +{ + uint32_t eax, ebx, ecx, edx; + uint32_t ret = 0; + + host_cpuid(function, index, &eax, &ebx, &ecx, &edx); + + switch (reg) { + case R_EAX: + ret = eax; + break; + case R_EBX: + ret = ebx; + break; + case R_ECX: + ret = ecx; + break; + case R_EDX: + ret = edx; + break; + } + return ret; +} + +/* +* get the configurable cpuid bits (can be set to 0 or 1) reported by TDX module
+ * fromtdx_caps.
+ */target/i386/kvm/tdx.c +14
-0+static inline uint32_t tdx_cap_cpuid_config (uint32_t function, + uint32_t index, int reg) +{ + struct kvm_tdx_cpuid_config *cpuid_c; + int ret = 0; + int i; + + if (tdx_caps->nr_cpuid_configs <= 0) { + return ret; + } + + for (i = 0; i < tdx_caps->nr_cpuid_configs; i++) { + cpuid_c = &tdx_caps->cpuid_configs[i]; + /* 0xffffffff in sub_leaf means the leaf doesn't require asublesf */
+ if (cpuid_c->leaf== function &&
+ (cpuid_c->sub_leaf==0xffffffff||cpuid_c->sub_leaf== index)) {
+ switch (reg) {
+ caseR_EAX:
+ ret =cpuid_c->eax;
+ break;
+ caseR_EBX:
+ ret =cpuid_c->ebx;
+ break;
+ caseR_ECX:
+ ret =cpuid_c->ecx;
+ break;
+ caseR_EDX:
+ ret =cpuid_c->edx;
+ break;
+ default:
+ return 0;
+ }
+ }
+ }
+ return ret;target/i386/kvm/tdx.c +20
-0+} + +static FeatureWord get_cpuid_featureword_index (uint32_t function, + uint32_t index, int reg) +{ + FeatureWord w; + + for (w = 0; w < FEATURE_WORDS; w++) { + FeatureWordInfo *f = &feature_word_info[w]; + + if (f->type == MSR_FEATURE_WORD || f->cpuid.eax != function || + f->cpuid.reg != reg || + (f->cpuid.needs_ecx && f->cpuid.ecx != index)) { + continue; + } + + return w; + } + + return w;Quoted 8 lines
Quoted 8 lines
target/i386/kvm/tdx.c +4
-0+} + +/* +* TDX supported CPUID varies from what KVM reports. Adjust the result by
+ * applying the TDX restrictions.
+ */
+void tdx_get_supported_cpuid(uint32_t function,uint32_tindex, intreg,
Quoted 35 lines
Quoted 17 lines
+
uint32_t*ret)
+{
+ /*
+ * it's KVMM + QEMU 's capabilities of what CPUID bits is supported or
+ * can be emulated as supported.
+ */
+uint32_tvmm_cap= *ret;
+ FeatureWord w;
+
+ /* Only handle features leaves that recognized by feature_word_info[] */
+ w =get_cpuid_featureword_index(function, index, reg); + if (w ==FEATURE_WORDS) {
+ return;
+ }target/i386/kvm/tdx.c +1
-0+target/i386/kvm/tdx.c bool is_tdx_vm(void) · line 267
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ if (tdx_cpuid_lookup[w].inducing_ve) { + *ret &= tdx_cpuid_lookup[w].supported_value_on_ve; + return; + } + + /* +* Include all the native bits as first step. It covers types +* - As configured (if native) +* - Native +* - XFAM related and Attributes realted s/realted/related
Quoted 6 lines
+ *
+ * It also has side effect to enable unsupported bits, e.g., the + * bits of "fixed0" type while present natively. It's safe because + * the unsupported bits will be masked off by .fixed0 later. + */
+ *ret |=host_cpuid_reg(function, index, reg);
Quoted 8 lines
Quoted 1 line
Looks KVM capabilities are merged with native bits, is this intentional?
yes, if we change the order, it would be more clear for you I guess.
host_cpuid_reg()|kvm_capabilitiesThe base is host's native value, while any bit that absent from native but KVM can emulate is also added to base.
Imagine there is a 'type native' bit that's absent from native but KVM emulated, With above code we pass 1 to tdx module but it wants native 0, is it an issue?
Thanks
Zhenzhong
Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
Quoted 7 lines · Zhenzhong
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
+ *
+ * It also has side effect to enable unsupported bits, e.g., the + * bits of "fixed0" type while present natively. It's safe because + * the unsupported bits will be masked off by .fixed0 later. + */
+ *ret |=host_cpuid_reg(function, index, reg);Looks KVM capabilities are merged with native bits, is this intentional?
Quoted 8 lines
Quoted 6 lines
yes, if we change the order, it would be more clear for you I guess.
host_cpuid_reg()|kvm_capabilitiesThe base is host's native value, while any bit that absent from native but KVM can emulate is also added to base.
Imagine there is a 'type native' bit that's absent from native but KVM emulated, With above code we pass 1 to tdx module but it wants native 0, is it an issue?
yes, it will have issue but it's not "we pass 1 to tdx_module".
"Native" bit is not configurable in the view of TDX module, and QEMU/KVM cannot configure it. But it does causes mismatch in above case that QEMU sees the bit is supported while in the TD the bit is not supported.
This is one of the reason why we are going to drop the solution that QEMU maintains the CPUID configurability in this series.
PATCH v5 18/65 i386/tdx: Make Intel-PT unsupported for TD guest
In reply to Xiaoyao LiDue to the fact that Intel-PT virtualization support has been broken in QEMU since Sapphire Rapids generation[1], below warning is triggered when luanching TD guest:
warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]
Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD guest, to avoid the confusing warning.
[1] https://lore.kernel.org/qemu-devel/[email protected]/
---
Changes in v4:
- newly added patch;
--- target/i386/kvm/tdx.c | 5 +++++ 1 file changed, 5 insertions(+)
target/i386/kvm/tdx.c +5 -0
@@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,if (function ==1 && reg ==R_ECX && !enable_cpu_pm) { *ret &= ~CPUID_EXT_MONITOR ; } + +/* QEMU Intel-pt support is broken, don't advertise Intel-PT */ +if (function ==7 && reg ==R_EBX ) { + *ret &= ~CPUID_7_0_EBX_INTEL_PT ; + } }enum tdx_ioctl_level {
Signature · 1 line
2.34.1
Re: [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest
Quoted 8 lines · Xiaoyao Li
Due to the fact that Intel-PT virtualization support has been broken in QEMU since Sapphire Rapids generation[1], below warning is triggered when luanching TD guest:
warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]
Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD guest, to avoid the confusing warning.
I guess normal guest has same issue.
Thanks
Zhenzhong
Quoted 27 lines
[1] https://lore.kernel.org/qemu-devel/[email protected]/
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- newly added patch;--- target/i386/kvm/tdx.c | 5 +++++ 1 file changed, 5 insertions(+)target/i386/kvm/tdx.c +5
-0@@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,if (function ==1 && reg ==R_ECX && !enable_cpu_pm) { *ret &= ~CPUID_EXT_MONITOR ; } + +/* QEMU Intel-pt support is broken, don't advertise Intel-PT */ +if (function ==7 && reg ==R_EBX ) { + *ret &= ~CPUID_7_0_EBX_INTEL_PT ; + } }enum tdx_ioctl_level {
Re: [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest
Quoted 6 lines · Zhenzhong
Quoted 4 lines · Xiaoyao Li
Due to the fact that Intel-PT virtualization support has been broken in QEMU since Sapphire Rapids generation[1], below warning is triggered when luanching TD guest:
Quoted 6 lines
Quoted 4 lines
warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]
Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD guest, to avoid the confusing warning.
I guess normal guest has same issue.
yeah, just the bug referenced by [1]
Quoted 31 lines
Thanks
Zhenzhong
Quoted 27 lines
[1] https://lore.kernel.org/qemu-devel/[email protected]/
Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v4:
- newly added patch;
---
target/i386/kvm/tdx.c| 5 +++++1 file changed, 5 insertions(+)target/i386/kvm/tdx.c +5
-0@@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg, if (function == 1 && reg == R_ECX && !enable_cpu_pm) { *ret &= ~CPUID_EXT_MONITOR; } + + /* QEMU Intel-pt support is broken, don't advertise Intel-PT */ + if (function == 7 && reg == R_EBX) { + *ret &= ~CPUID_7_0_EBX_INTEL_PT; + } } enum tdx_ioctl_level{
PATCH v5 19/65 i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[]
In reply to Xiaoyao Litdx_cpuid_lookup[].tdx_fixed0/1 is QEMU maintained data which reflects TDX restrictions regrading what bits are fixed by TDX module.
It's retrieved from TDX spec and static. However, TDX may evolve and change some fixed fields to configurable in the future. Update tdx_cpuid.lookup[].tdx_fixed0/1 fields by removing the bits that reported from TDX module as configurable. This can adapt with the updated TDX (module) automatically.
--- target/i386/kvm/tdx.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
target/i386/kvm/tdx.c +34 -0
@@ -377,6 +377,38 @@ static int get_tdx_capabilities(Error **errp)return 0 ; } +static void update_tdx_cpuid_lookup_by_tdx_caps (void ) +{ +KvmTdxCpuidLookup *entry; +FeatureWordInfo *fi; +uint32_t config; +FeatureWord w; + +for (w =0 ; w <FEATURE_WORDS ; w++) { + fi = &feature_word_info[w]; + entry = &tdx_cpuid_lookup[w]; + +if (fi->type !=CPUID_FEATURE_WORD ) { +continue ; + } + + config =tdx_cap_cpuid_config (fi->cpuid.eax, + fi->cpuid.needs_ecx ? fi->cpuid.ecx : ~0u , + fi->cpuid.reg); + +if (!config) { +continue ; + } + +/* +* Remove the configurable bits from tdx_fixed0/1 in case QEMU +* maintained fixed0/1 values is outdated to TDX module. +*/ + entry->tdx_fixed0 &= ~config; + entry->tdx_fixed1 &= ~config; + } +} +static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ());
@@ -392,6 +424,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) } } +update_tdx_cpuid_lookup_by_tdx_caps (); + tdx_guest = tdx;return 0 ; }
Signature · 1 line
2.34.1
Re: [PATCH v5 19/65] i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[]
Quoted 8 lines · Xiaoyao Li
tdx_cpuid_lookup[].tdx_fixed0/1 is QEMU maintained data which reflects TDX restrictions regrading what bits are fixed by TDX module.
It's retrieved from TDX spec and static. However, TDX may evolve and change some fixed fields to configurable in the future. Update tdx_cpuid.lookup[].tdx_fixed0/1 fields by removing the bits that reported from TDX module as configurable. This can adapt with the updated TDX (module) automatically.
Can the fixed fields evolves to other type, i.e., native?
Quoted 21 lines
Signed-off-by Xiaoyao Li <[email protected]>--- target/i386/kvm/tdx.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)target/i386/kvm/tdx.c +7
-0@@ -377,6 +377,38 @@ static int get_tdx_capabilities(Error **errp)return 0 ; } +static void update_tdx_cpuid_lookup_by_tdx_caps (void ) +{ +KvmTdxCpuidLookup *entry; +FeatureWordInfo *fi; +uint32_t config; +FeatureWord w; +
target/i386/kvm/tdx.c static int get_tdx_capabilities(Error **errp) · line 396
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+for (w =0 ; w <FEATURE_WORDS ; w++) { + fi = &feature_word_info[w]; + entry = &tdx_cpuid_lookup[w]; + +if (fi->type !=CPUID_FEATURE_WORD ) { +continue ; + } + + config = tdx_cap_cpuid_config(fi->cpuid.eax, + fi->cpuid.needs_ecx ? fi->cpuid.ecx : ~0u,
So the check "cpuid_c->sub_leaf == 0xffffffff" in tdx_cap_cpuid_config() is unnecessary?
Thanks
Zhenzhong
Quoted 27 lines
target/i386/kvm/tdx.c +17
-0+ fi->cpuid.reg); + +if (!config) { +continue ; + } + +/* +* Remove the configurable bits from tdx_fixed0/1 in case QEMU +* maintained fixed0/1 values is outdated to TDX module. +*/ + entry->tdx_fixed0 &= ~config; + entry->tdx_fixed1 &= ~config; + } +} +static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ());@@ -392,6 +424,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) } } +update_tdx_cpuid_lookup_by_tdx_caps (); + tdx_guest = tdx;return 0 ; }
PATCH v5 20/65 i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup
In reply to Xiaoyao LiKVM requires userspace to pass XFAM configuration via CPUID 0xD leaves.
Convert tdx_caps->xfam_fixed0/1 into corresponding tdx_cpuid_lookup[].tdx_fixed0/1 field of CPUID 0xD leaves. Thus the requirement can be applied naturally.
--- target/i386/cpu.c | 3 --- target/i386/cpu.h | 3 +++ target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 3 files changed, 27 insertions(+), 3 deletions(-)
target/i386/cpu.c +0 -3
@@ -1576,9 +1576,6 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = { };#undef REGISTER -/* CPUID feature bits available in XSS */ -#define CPUID_XSTATE_XSS_MASK (XSTATE_ARCH_LBR_MASK) -ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT ] = { [XSTATE_FP_BIT ] = {/* x87 FP state component is always enabled if XSAVE is supported */
target/i386/cpu.h +3 -0
@@ -595,6 +595,9 @@ typedef enum X86Seg { XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK | \ XSTATE_XTILE_CFG_MASK | XSTATE_XTILE_DATA_MASK) +/* CPUID feature bits available in XSS */ +#define CPUID_XSTATE_XSS_MASK (XSTATE_ARCH_LBR_MASK) +/* CPUID feature words */ typedef enum FeatureWord { FEAT_1_EDX, /* CPUID[1].EDX */
target/i386/kvm/tdx.c +24 -0
@@ -407,6 +407,30 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) entry->tdx_fixed0 &= ~config; entry->tdx_fixed1 &= ~config; } + +/* +* Because KVM gets XFAM settings via CPUID leaves 0xD, map +* tdx_caps->xfam_fixed{0, 1} into tdx_cpuid_lookup[].tdx_fixed{0, 1}. +* +* Then the enforment applies in tdx_get_configurable_cpuid() naturally. +*/ + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_LO ].tdx_fixed0 = + (uint32_t )~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XCR0_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_LO ].tdx_fixed1 = + (uint32_t )tdx_caps->xfam_fixed1 &CPUID_XSTATE_XCR0_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_HI ].tdx_fixed0 = + (~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XCR0_MASK ) >>32 ; + tdx_cpuid_lookup[FEAT_XSAVE_XCR0_HI ].tdx_fixed1 = + (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XCR0_MASK ) >>32 ; + + tdx_cpuid_lookup[FEAT_XSAVE_XSS_LO ].tdx_fixed0 = + (uint32_t )~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XSS_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XSS_LO ].tdx_fixed1 = + (uint32_t )tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ; + tdx_cpuid_lookup[FEAT_XSAVE_XSS_HI ].tdx_fixed0 = + (~tdx_caps->xfam_fixed0 &CPUID_XSTATE_XSS_MASK ) >>32 ; + tdx_cpuid_lookup[FEAT_XSAVE_XSS_HI ].tdx_fixed1 = + (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; }static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp)
Signature · 1 line
2.34.1
PATCH v5 21/65 i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup
In reply to Xiaoyao LiSome bits in TD attributes have corresponding CPUID feature bits. Reflect the fixed0/1 restriction on TD attributes to their corresponding CPUID bits in tdx_cpuid_lookup[] as well.
---
Changes in v4:
- reverse the meaning of tdx_caps->attr_fixed0, because value 0 of bit means the bit must be fixed 0.
--- target/i386/cpu-internal.h | 9 +++++++++ target/i386/cpu.c | 9 --------- target/i386/cpu.h | 2 ++ target/i386/kvm/tdx.c | 21 +++++++++++++++++++++ 4 files changed, 32 insertions(+), 9 deletions(-)
target/i386/cpu-internal.h +9 -0
@@ -20,6 +20,15 @@ #ifndef I386_CPU_INTERNAL_H #define I386_CPU_INTERNAL_H +typedef struct FeatureMask { + FeatureWord index; + uint64_t mask; +} FeatureMask; + +typedef struct FeatureDep { + FeatureMask from, to; +} FeatureDep; + typedef enum FeatureWordType { CPUID_FEATURE_WORD, MSR_FEATURE_WORD,
target/i386/cpu.c +0 -9
@@ -1443,15 +1443,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = { }, }; -typedef struct FeatureMask { -FeatureWord index; -uint64_t mask; -}FeatureMask ; - -typedef struct FeatureDep { -FeatureMask from, to; -}FeatureDep ; -static FeatureDep feature_dependencies[] = { { .from = {FEAT_7_0_EDX ,CPUID_7_0_EDX_ARCH_CAPABILITIES },
target/i386/cpu.h +2 -0
@@ -883,6 +883,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_7_0_ECX_MAWAU (31U << 17)/* Read Processor ID */ #define CPUID_7_0_ECX_RDPID (1U << 22) +/* KeyLocker */ +#define CPUID_7_0_ECX_KeyLocker (1U << 23)/* Bus Lock Debug Exception */ #define CPUID_7_0_ECX_BUS_LOCK_DETECT (1U << 24)/* Cache Line Demote Instruction */
target/i386/kvm/tdx.c +21 -0
@@ -32,6 +32,13 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_ATTRIBUTES_MAX_BITS 64 + +static FeatureMask tdx_attrs_ctrl_fields[TDX_ATTRIBUTES_MAX_BITS ] = { + [30 ] = { .index =FEAT_7_0_ECX , .mask =CPUID_7_0_ECX_PKS }, + [31 ] = { .index =FEAT_7_0_ECX , .mask = CPUID_7_0_ECX_KeyLocker}, +}; + typedef struct KvmTdxCpuidLookup { uint32_t tdx_fixed0; uint32_t tdx_fixed1;
@@ -383,6 +390,8 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void)FeatureWordInfo *fi;uint32_t config;FeatureWord w; +FeatureMask *fm; +int i;for (w =0 ; w <FEATURE_WORDS ; w++) { fi = &feature_word_info[w];
@@ -408,6 +417,18 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) entry->tdx_fixed1 &= ~config; } +for (i =0 ; i <ARRAY_SIZE (tdx_attrs_ctrl_fields); i++) { + fm = &tdx_attrs_ctrl_fields[i]; + +if (~(tdx_caps->attrs_fixed0 & (1ULL << i))) { + tdx_cpuid_lookup[fm->index].tdx_fixed0 |= fm->mask; + } + +if (tdx_caps->attrs_fixed1 & (1ULL << i)) { + tdx_cpuid_lookup[fm->index].tdx_fixed1 |= fm->mask; + } + } +/* * Because KVM gets XFAM settings via CPUID leaves 0xD, map * tdx_caps->xfam_fixed{0, 1} into tdx_cpuid_lookup[].tdx_fixed{0, 1}.
Signature · 1 line
2.34.1
PATCH v5 22/65 i386/kvm: Move architectural CPUID leaf generation to separate helper
In reply to Xiaoyao LiFrom: Sean Christopherson <[email protected]>
Move the architectural (for lack of a better term) CPUID leaf generation to a separate helper so that the generation code can be reused by TDX, which needs to generate a canonical VM-scoped configuration.
--- target/i386/kvm/kvm.c | 459 +++++++++++++++++++------------------ target/i386/kvm/kvm_i386.h | 3 + 2 files changed, 240 insertions(+), 222 deletions(-)
target/i386/kvm/kvm.c +237 -222
@@ -1731,6 +1731,241 @@ static void kvm_init_nested_state(CPUX86State *env) } } +uint32_t kvm_x86_arch_cpuid (CPUX86State *env,struct kvm_cpuid_entry2 *entries, +uint32_t cpuid_i) +{ +uint32_t limit, i, j; +uint32_t unused; +struct kvm_cpuid_entry2 *c; + +if (cpuid_i >KVM_MAX_CPUID_ENTRIES ) { +error_report ("exceeded cpuid index (%d) for entries[]" , cpuid_i); +abort (); + } + +cpu_x86_cpuid (env,0 ,0 , &limit, &unused, &unused, &unused); + +for (i =0 ; i <= limit; i++) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"unsupported level value: 0x%x\n" , limit); +abort (); + } + c = &entries[cpuid_i++]; + +switch (i) { +case 2 : { +/* Keep reading function 2 till all the input is received */ +int times; + + c->function = i; + c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC | +KVM_CPUID_FLAG_STATE_READ_NEXT ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + times = c->eax &0xff ; + +for (j =1 ; j < times; ++j) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:2):eax & 0xf = 0x%x\n" , times); +abort (); + } + c = &entries[cpuid_i++]; + c->function = i; + c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + } +break ; + } +case 0x1f : +if (env->nr_dies <2 ) { + cpuid_i--; +break ; + } +/* fallthrough */ +case 4 : +case 0xb : +case 0xd : +for (j =0 ; ; j++) { +if (i ==0xd && j ==64 ) { +break ; + } + + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (i ==4 && c->eax ==0 ) { +break ; + } +if (i ==0xb && !(c->ecx &0xff00 )) { +break ; + } +if (i ==0x1f && !(c->ecx &0xff00 )) { +break ; + } +if (i ==0xd && c->eax ==0 ) { +continue ; + } +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); +abort (); + } + c = &entries[cpuid_i++]; + } +break ; +case 0x12 : +for (j =0 ; ; j++) { + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (j >1 && (c->eax &0xf ) !=1 ) { +break ; + } + +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x12,ecx:0x%x)\n" , j); +abort (); + } + c = &entries[cpuid_i++]; + } +break ; +case 0x7 : +case 0x14 : +case 0x1d : +case 0x1e : { +uint32_t times; + + c->function = i; + c->index =0 ; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + times = c->eax; + +for (j =1 ; j <= times; ++j) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); +abort (); + } + c = &entries[cpuid_i++]; + c->function = i; + c->index = j; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + } +break ; + } +default : + c->function = i; + c->flags =0 ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); +if (!c->eax && !c->ebx && !c->ecx && !c->edx) { +/* +* KVM already returns all zeroes if a CPUID entry is missing, +* so we can omit it and avoid hitting KVM's 80-entry limit. +*/ + cpuid_i--; + } +break ; + } + } + +if (limit >=0x0a ) { +uint32_t eax, edx; + +cpu_x86_cpuid (env,0x0a ,0 , &eax, &unused, &unused, &edx); + + has_architectural_pmu_version = eax &0xff ; +if (has_architectural_pmu_version >0 ) { + num_architectural_pmu_gp_counters = (eax &0xff00 ) >>8 ; + +/* Shouldn't be more than 32, since that's the number of bits +* available in EBX to tell us _which_ counters are available. +* Play it safe. +*/ +if (num_architectural_pmu_gp_counters >MAX_GP_COUNTERS ) { + num_architectural_pmu_gp_counters =MAX_GP_COUNTERS ; + } + +if (has_architectural_pmu_version >1 ) { + num_architectural_pmu_fixed_counters = edx &0x1f ; + +if (num_architectural_pmu_fixed_counters >MAX_FIXED_COUNTERS ) { + num_architectural_pmu_fixed_counters =MAX_FIXED_COUNTERS ; + } + } + } + } + +cpu_x86_cpuid (env,0x80000000 ,0 , &limit, &unused, &unused, &unused); + +for (i =0x80000000 ; i <= limit; i++) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"unsupported xlevel value: 0x%x\n" , limit); +abort (); + } + c = &entries[cpuid_i++]; + +switch (i) { +case 0x8000001d : +/* Query for all AMD cache information leaves */ +for (j =0 ; ; j++) { + c->function = i; + c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; + c->index = j; +cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); + +if (c->eax ==0 ) { +break ; + } +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"cpuid_data is full, no space for " +"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); +abort (); + } + c = &entries[cpuid_i++]; + } +break ; +default : + c->function = i; + c->flags =0 ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); +if (!c->eax && !c->ebx && !c->ecx && !c->edx) { +/* +* KVM already returns all zeroes if a CPUID entry is missing, +* so we can omit it and avoid hitting KVM's 80-entry limit. +*/ + cpuid_i--; + } +break ; + } + } + +/* Call Centaur's CPUID instructions they are supported. */ +if (env->cpuid_xlevel2 >0 ) { +cpu_x86_cpuid (env,0xC0000000 ,0 , &limit, &unused, &unused, &unused); + +for (i =0xC0000000 ; i <= limit; i++) { +if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { +fprintf (stderr,"unsupported xlevel2 value: 0x%x\n" , limit); +abort (); + } + c = &entries[cpuid_i++]; + + c->function = i; + c->flags =0 ; +cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); + } + } + +return cpuid_i; +} +int kvm_arch_init_vcpu (CPUState *cs) {struct {
@@ -1747,8 +1982,7 @@ int kvm_arch_init_vcpu(CPUState *cs)X86CPU *cpu =X86_CPU (cs);CPUX86State *env = &cpu->env; -uint32_t limit, i, j, cpuid_i; -uint32_t unused; +uint32_t cpuid_i;struct kvm_cpuid_entry2 *c;uint32_t signature[3 ];int kvm_base =KVM_CPUID_SIGNATURE ;
@@ -1901,8 +2135,6 @@ int kvm_arch_init_vcpu(CPUState *cs) c->edx = env->features[FEAT_KVM_HINTS ]; } -cpu_x86_cpuid (env,0 ,0 , &limit, &unused, &unused, &unused); -if (cpu->kvm_pv_enforce_cpuid) { r =kvm_vcpu_enable_cap (cs,KVM_CAP_ENFORCE_PV_FEATURE_CPUID ,0 ,1 );if (r <0 ) {
@@ -1913,224 +2145,7 @@ int kvm_arch_init_vcpu(CPUState *cs) } } -for (i =0 ; i <= limit; i++) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"unsupported level value: 0x%x\n" , limit); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - -switch (i) { -case 2 : { -/* Keep reading function 2 till all the input is received */ -int times; - - c->function = i; - c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC | -KVM_CPUID_FLAG_STATE_READ_NEXT ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - times = c->eax &0xff ; - -for (j =1 ; j < times; ++j) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:2):eax & 0xf = 0x%x\n" , times); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - c->function = i; - c->flags =KVM_CPUID_FLAG_STATEFUL_FUNC ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - } -break ; - } -case 0x1f : -if (env->nr_dies <2 ) { - cpuid_i--; -break ; - } -/* fallthrough */ -case 4 : -case 0xb : -case 0xd : -for (j =0 ; ; j++) { -if (i ==0xd && j ==64 ) { -break ; - } - - c->function = i; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; - c->index = j; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - -if (i ==4 && c->eax ==0 ) { -break ; - } -if (i ==0xb && !(c->ecx &0xff00 )) { -break ; - } -if (i ==0x1f && !(c->ecx &0xff00 )) { -break ; - } -if (i ==0xd && c->eax ==0 ) { -continue ; - } -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - } -break ; -case 0x12 : -for (j =0 ; ; j++) { - c->function = i; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; - c->index = j; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - -if (j >1 && (c->eax &0xf ) !=1 ) { -break ; - } - -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x12,ecx:0x%x)\n" , j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - } -break ; -case 0x7 : -case 0x14 : -case 0x1d : -case 0x1e : { -uint32_t times; - - c->function = i; - c->index =0 ; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - times = c->eax; - -for (j =1 ; j <= times; ++j) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - c->function = i; - c->index = j; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - } -break ; - } -default : - c->function = i; - c->flags =0 ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); -if (!c->eax && !c->ebx && !c->ecx && !c->edx) { -/* -* KVM already returns all zeroes if a CPUID entry is missing, -* so we can omit it and avoid hitting KVM's 80-entry limit. -*/ - cpuid_i--; - } -break ; - } - } - -if (limit >=0x0a ) { -uint32_t eax, edx; - -cpu_x86_cpuid (env,0x0a ,0 , &eax, &unused, &unused, &edx); - - has_architectural_pmu_version = eax &0xff ; -if (has_architectural_pmu_version >0 ) { - num_architectural_pmu_gp_counters = (eax &0xff00 ) >>8 ; - -/* Shouldn't be more than 32, since that's the number of bits -* available in EBX to tell us _which_ counters are available. -* Play it safe. -*/ -if (num_architectural_pmu_gp_counters >MAX_GP_COUNTERS ) { - num_architectural_pmu_gp_counters =MAX_GP_COUNTERS ; - } - -if (has_architectural_pmu_version >1 ) { - num_architectural_pmu_fixed_counters = edx &0x1f ; - -if (num_architectural_pmu_fixed_counters >MAX_FIXED_COUNTERS ) { - num_architectural_pmu_fixed_counters =MAX_FIXED_COUNTERS ; - } - } - } - } - -cpu_x86_cpuid (env,0x80000000 ,0 , &limit, &unused, &unused, &unused); - -for (i =0x80000000 ; i <= limit; i++) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"unsupported xlevel value: 0x%x\n" , limit); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - -switch (i) { -case 0x8000001d : -/* Query for all AMD cache information leaves */ -for (j =0 ; ; j++) { - c->function = i; - c->flags =KVM_CPUID_FLAG_SIGNIFCANT_INDEX ; - c->index = j; -cpu_x86_cpuid (env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); - -if (c->eax ==0 ) { -break ; - } -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"cpuid_data is full, no space for " -"cpuid(eax:0x%x,ecx:0x%x)\n" , i, j); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - } -break ; -default : - c->function = i; - c->flags =0 ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); -if (!c->eax && !c->ebx && !c->ecx && !c->edx) { -/* -* KVM already returns all zeroes if a CPUID entry is missing, -* so we can omit it and avoid hitting KVM's 80-entry limit. -*/ - cpuid_i--; - } -break ; - } - } - -/* Call Centaur's CPUID instructions they are supported. */ -if (env->cpuid_xlevel2 >0 ) { -cpu_x86_cpuid (env,0xC0000000 ,0 , &limit, &unused, &unused, &unused); - -for (i =0xC0000000 ; i <= limit; i++) { -if (cpuid_i ==KVM_MAX_CPUID_ENTRIES ) { -fprintf (stderr,"unsupported xlevel2 value: 0x%x\n" , limit); -abort (); - } - c = &cpuid_data.entries[cpuid_i++]; - - c->function = i; - c->flags =0 ; -cpu_x86_cpuid (env, i,0 , &c->eax, &c->ebx, &c->ecx, &c->edx); - } - } - + cpuid_i =kvm_x86_arch_cpuid (env, cpuid_data.entries, cpuid_i); cpuid_data.cpuid.nent = cpuid_i; if (((env->cpuid_version >> 8)&0xF) >= 6
target/i386/kvm/kvm_i386.h +3 -0
@@ -24,6 +24,9 @@#define kvm_ioapic_in_kernel () \ (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split()) +uint32_t kvm_x86_arch_cpuid (CPUX86State *env,struct kvm_cpuid_entry2 *entries, +uint32_t cpuid_i); +#else #define kvm_pit_in_kernel () 0
Signature · 1 line
2.34.1
PATCH v5 23/65 kvm: Introduce kvm_arch_pre_create_vcpu()
In reply to Xiaoyao LiIntroduce kvm_arch_pre_create_vcpu(), to perform arch-dependent work prior to create any vcpu. This is for i386 TDX because it needs call TDX_INIT_VM before creating any vcpu.
---
Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)
--- accel/kvm/kvm-all.c | 10 ++++++++++ include/sysemu/kvm.h | 1 + 2 files changed, 11 insertions(+)
accel/kvm/kvm-all.c +10 -0
@@ -418,6 +418,11 @@ static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)return kvm_vm_ioctl (s,KVM_CREATE_VCPU , (void *)vcpu_id); } +int __attribute__ ((weak))kvm_arch_pre_create_vcpu (CPUState *cpu,Error **errp) +{ +return 0 ; +} +int kvm_init_vcpu (CPUState *cpu,Error **errp) {KVMState *s = kvm_state;
@@ -426,6 +431,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)trace_kvm_init_vcpu (cpu->cpu_index,kvm_arch_vcpu_id (cpu)); + ret =kvm_arch_pre_create_vcpu (cpu, errp); +if (ret <0 ) { + goto err; + } + ret =kvm_get_vcpu (s,kvm_arch_vcpu_id (cpu));if (ret <0 ) { error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
include/sysemu/kvm.h +1 -0
@@ -341,6 +341,7 @@ int kvm_arch_get_default_type(MachineState *ms);int kvm_arch_init (MachineState *ms,KVMState *s); +int kvm_arch_pre_create_vcpu (CPUState *cpu,Error **errp);int kvm_arch_init_vcpu (CPUState *cpu);int kvm_arch_destroy_vcpu (CPUState *cpu);
Signature · 1 line
2.34.1
PATCH v5 24/65 i386/tdx: Initialize TDX before creating TD vcpus
In reply to Xiaoyao LiInvoke KVM_TDX_INIT in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT configures global TD configurations, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs.
Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.
Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init().
---
Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
--- accel/kvm/kvm-all.c | 9 +++++++- target/i386/kvm/kvm.c | 9 ++++++++ target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-stub.c | 8 ++++++++ target/i386/kvm/tdx.c | 41 +++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 4 ++++ 6 files changed, 71 insertions(+), 2 deletions(-) create mode 100644 target/i386/kvm/tdx-stub.c
accel/kvm/kvm-all.c +8 -1
@@ -431,8 +431,15 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)trace_kvm_init_vcpu (cpu->cpu_index,kvm_arch_vcpu_id (cpu)); +/* +* tdx_pre_create_vcpu() may call cpu_x86_cpuid(). It in turn may call +* kvm_vm_ioctl(). Set cpu->kvm_state in advance to avoid NULL pointer +* dereference. +*/ + cpu->kvm_state = s; ret =kvm_arch_pre_create_vcpu (cpu, errp);if (ret <0 ) { + cpu->kvm_state =NULL ; goto err; }
@@ -440,11 +447,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)if (ret <0 ) {error_setg_errno (errp, -ret,"kvm_init_vcpu: kvm_get_vcpu failed (%lu)" ,kvm_arch_vcpu_id (cpu)); + cpu->kvm_state =NULL ; goto err; } cpu->kvm_fd = ret; - cpu->kvm_state = s; cpu->vcpu_dirty = true; cpu->dirty_pages =0 ; cpu->throttle_us_per_full =0 ;
target/i386/kvm/kvm.c +9 -0
@@ -2271,6 +2271,15 @@ int kvm_arch_init_vcpu(CPUState *cs)return r; } +int kvm_arch_pre_create_vcpu (CPUState *cpu,Error **errp) +{ +if (is_tdx_vm ()) { +return tdx_pre_create_vcpu (cpu, errp); + } + +return 0 ; +} +int kvm_arch_destroy_vcpu (CPUState *cs) {X86CPU *cpu =X86_CPU (cs);
target/i386/kvm/meson.build +1 -1
@@ -7,7 +7,7 @@ i386_kvm_ss.add(files( i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c')) -i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c')) +i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c')) i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
target/i386/kvm/tdx-stub.c new file +8 -0
@@ -0,0 +1,8 @@ +#include "qemu/osdep.h" + +#include "tdx.h" + +int tdx_pre_create_vcpu (CPUState *cpu,Error **errp) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +41 -0
53 lines not on this page · raw message
target/i386/kvm/tdx.h +4 -0
@@ -17,6 +17,9 @@ typedef struct TdxGuestClass {typedef struct TdxGuest {ConfidentialGuestSupport parent_obj; +QemuMutex lock; + +bool initialized;uint64_t attributes;/* TD attributes */ }TdxGuest ;
@@ -28,5 +31,6 @@ bool is_tdx_vm(void);void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg,uint32_t *ret); +int tdx_pre_create_vcpu (CPUState *cpu,Error **errp);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v5 25/65 i386/tdx: Add property sept-ve-disable for tdx-guest object
In reply to Xiaoyao LiBit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
---
Changes in v4:
- collect Acked-by from Markus
Changes in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)
--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)
qapi/qom.json +6 -1
@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. # +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } } ## # @ThreadContextProperties:
target/i386/kvm/tdx.c +24 -0
@@ -32,6 +32,8 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_ATTRIBUTES_MAX_BITS 64static FeatureMask tdx_attrs_ctrl_fields[TDX_ATTRIBUTES_MAX_BITS ] = {
@@ -514,6 +516,24 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return 0 ; } +static bool tdx_guest_get_sept_ve_disable (Object *obj,Error **errp) +{ +TdxGuest *tdx =TDX_GUEST (obj); + +return !!(tdx->attributes &TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ); +} + +static void tdx_guest_set_sept_ve_disable (Object *obj,bool value,Error **errp) +{ +TdxGuest *tdx =TDX_GUEST (obj); + +if (value) { + tdx->attributes |=TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ; + }else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ; + } +} +/* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,
@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)qemu_mutex_init (&tdx->lock); tdx->attributes =0 ; + +object_property_add_bool (obj,"sept-ve-disable" , + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); }static void tdx_guest_finalize (Object *obj)
Signature · 1 line
2.34.1
Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 32 lines · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #
qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }
So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
qapi/qom.json
Quoted 11 lines
qapi/qom.json +11
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp) +{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}
If I look at the documentation for TD attributes
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',
For the proposed SEV-SNP patches, the same has been done again
https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg00536.html
'*policy': 'uint64',
The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWXEESYUA77DP7YIBP55T2OPSVKV5QW/
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
Quoted 18 lines
qapi/qom.json +5
-0+ /* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj) qemu_mutex_init(&tdx->lock); tdx->attributes = 0; + + object_property_add_bool(obj, "sept-ve-disable", + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); } static void tdx_guest_finalize(Object *obj)Signature · 1 line
2.34.1
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Quoted 100 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 32 lines · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
qapi/qom.json
Quoted 11 lines
qapi/qom.json +11
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp) +{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg00536.html
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
Quoted 8 lines
https://lists.libvirt.org/archives/list/[email protected]/message/WXWXEESYUA77DP7YIBP55T2OPSVKV5QW/
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't support
If allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
2. people need to know the exact bit position of each attribute. I don't think it is a user-friendly interface to require user to be aware of such details.
For example, if user wants to create a Debug TD, user just needs to set 'debug=on' for tdx-guest object. It's much more friendly than that user needs to set the bit 0 of the attribute.
Quoted 21 lines
Quoted 18 lines
qapi/qom.json +5
-0+ /* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj) qemu_mutex_init(&tdx->lock); tdx->attributes = 0; + + object_property_add_bool(obj, "sept-ve-disable", + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); } static void tdx_guest_finalize(Object *obj)Signature · 1 line
2.34.1With regards,
Daniel
RE: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Quoted 129 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 5 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 1 line · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, itdisables
Quoted 45 lines
Quoted 31 lines
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
Quoted 2 lines
qapi/qom.json line 913
Quoted 2 lines
qapi/qom.json +2
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error**errp)
Quoted 12 lines
qapi/qom.json
Quoted 9 lines
qapi/qom.json +9
-0+{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Quoted 1 line
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Quoted 8 lines
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?Quoted 8 lines
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
06/msg00536.html
Quoted 11 lines
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
Quoted 9 lines
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWX
EESYUA77DP7YIBP55T2OPSVKV5QW/Quoted 7 lines
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
Quoted 6 lines
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't supportIf allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
What about defining those bits associated with cpu properties reserved But other bits work as Daniel suggested way.
Thanks
Zhenzhong
Quoted 31 lines
2. people need to know the exact bit position of each attribute. I don't think it is a user-friendly interface to require user to be aware of such details.
For example, if user wants to create a Debug TD, user just needs to set 'debug=on' for tdx-guest object. It's much more friendly than that user needs to set the bit 0 of the attribute.
Quoted 21 lines
Quoted 16 lines
qapi/qom.json +5
-1+ /* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj) qemu_mutex_init(&tdx->lock); tdx->attributes = 0; + + object_property_add_bool(obj, "sept-ve-disable", + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); } static void tdx_guest_finalize(Object *obj) --Quoted 2 lines
2.34.1
With regards,
Daniel
Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Quoted 133 lines · Zhenzhong
Quoted 133 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 5 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 1 line · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, itdisables
Quoted 45 lines
Quoted 31 lines
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
Quoted 2 lines
qapi/qom.json line 913
Quoted 2 lines
qapi/qom.json +2
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error**errp)
Quoted 12 lines
qapi/qom.json
Quoted 9 lines
qapi/qom.json +9
-0+{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Quoted 1 line
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Quoted 8 lines
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?Quoted 8 lines
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
06/msg00536.html
Quoted 11 lines
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
Quoted 9 lines
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWX
EESYUA77DP7YIBP55T2OPSVKV5QW/Quoted 7 lines
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't support
Quoted 5 lines
Quoted 2 lines
If allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
What about defining those bits associated with cpu properties reserved But other bits work as Daniel suggested way.
I don't understand. Do you mean we provide the interface to configure raw 64 bit attributes while some bits of it are reserved?
Quoted 35 lines
Thanks
ZhenzhongQuoted 31 lines
2. people need to know the exact bit position of each attribute. I don't think it is a user-friendly interface to require user to be aware of such details.
For example, if user wants to create a Debug TD, user just needs to set 'debug=on' for tdx-guest object. It's much more friendly than that user needs to set the bit 0 of the attribute.
Quoted 21 lines
Quoted 16 lines
qapi/qom.json +5
-1+ /* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj) qemu_mutex_init(&tdx->lock); tdx->attributes = 0; + + object_property_add_bool(obj, "sept-ve-disable", + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); } static void tdx_guest_finalize(Object *obj) --Quoted 2 lines
2.34.1
With regards,
Daniel
RE: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Quoted 143 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 59 lines · Zhenzhong
Quoted 59 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 5 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 1 line · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, itdisables
Quoted 45 lines
Quoted 31 lines
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
qapi/qom.json line 913
Quoted 2 lines
qapi/qom.json +2
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value,Error
Quoted 47 lines
Quoted 47 lines
**errp)
Quoted 12 lines
qapi/qom.json
Quoted 9 lines
qapi/qom.json +9
-0+{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Quoted 1 line
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Quoted 8 lines
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?Quoted 8 lines
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
06/msg00536.html
Quoted 15 lines
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWX
Quoted 19 lines
Quoted 19 lines
EESYUA77DP7YIBP55T2OPSVKV5QW/
Quoted 7 lines
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't support
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
If allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
What about defining those bits associated with cpu properties reserved But other bits work as Daniel suggested way.
I don't understand. Do you mean we provide the interface to configure raw 64 bit attributes while some bits of it are reserved?
Yes, qemu provide raw 64bit attribute but ignore those cpuid related bits to avoid config conflict.
You can still provide sept-ve-disable and debug properties in qemu if you want, But Libvirt will not use them, it will use the raw 64bit attribute.
Thanks
Zhenzhong
Quoted 36 lines
Quoted 36 lines
Thanks
ZhenzhongQuoted 31 lines
2. people need to know the exact bit position of each attribute. I don't think it is a user-friendly interface to require user to be aware of such details.
For example, if user wants to create a Debug TD, user just needs to set 'debug=on' for tdx-guest object. It's much more friendly than that user needs to set the bit 0 of the attribute.
Quoted 21 lines
Quoted 16 lines
qapi/qom.json +5
-1+ /* tdx guest */ OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest, tdx_guest,@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj) qemu_mutex_init(&tdx->lock); tdx->attributes = 0; + + object_property_add_bool(obj, "sept-ve-disable", + tdx_guest_get_sept_ve_disable, + tdx_guest_set_sept_ve_disable); } static void tdx_guest_finalize(Object *obj) --Quoted 2 lines
2.34.1
With regards,
Daniel
Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
In reply to Xiaoyao LiQuoted 134 lines · Xiaoyao Li
Quoted 133 lines · Zhenzhong
Quoted 133 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 5 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 1 line · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, itdisables
Quoted 45 lines
Quoted 31 lines
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
Quoted 2 lines
qapi/qom.json line 913
Quoted 2 lines
qapi/qom.json +2
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error**errp)
Quoted 12 lines
qapi/qom.json
Quoted 9 lines
qapi/qom.json +9
-0+{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Quoted 1 line
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Quoted 8 lines
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?Quoted 8 lines
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
06/msg00536.html
Quoted 11 lines
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
Quoted 9 lines
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWX
EESYUA77DP7YIBP55T2OPSVKV5QW/Quoted 7 lines
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't support
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
If allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
What about defining those bits associated with cpu properties reserved But other bits work as Daniel suggested way.
I don't understand. Do you mean we provide the interface to configure raw 64 bit attributes while some bits of it are reserved?
Just have a mask of what bits are permitted to be set, and report an error if the user sets non-permitted bits.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Quoted 138 lines · Daniel P. Berrangé
Quoted 137 lines · Xiaoyao Li
Quoted 136 lines · Zhenzhong
Quoted 135 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 5 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 1 line · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, itdisables
Quoted 45 lines
Quoted 31 lines
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
Quoted 2 lines
qapi/qom.json line 913
Quoted 2 lines
qapi/qom.json +2
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error**errp)
Quoted 12 lines
qapi/qom.json
Quoted 9 lines
qapi/qom.json +9
-0+{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Quoted 1 line
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Quoted 8 lines
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?Quoted 8 lines
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
06/msg00536.html
Quoted 11 lines
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
Quoted 9 lines
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWX
EESYUA77DP7YIBP55T2OPSVKV5QW/Quoted 7 lines
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't supportIf allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
What about defining those bits associated with cpu properties reserved But other bits work as Daniel suggested way.
I don't understand. Do you mean we provide the interface to configure raw 64 bit attributes while some bits of it are reserved?
Just have a mask of what bits are permitted to be set, and report an error if the user sets non-permitted bits.
Looking at the IGVM patches, the CnofidentialGuestSupport class is gaining a "set_guest_policy" method, which takes a "uint64_t policy".
If (when) IGVM support is extended to cover TDX too, then we're going to need to accept the user providing the policy in integer format via the IGVM file metadata. This will require adding code to check for any inconsistency between the policy bitmask, and the CPU flags.
https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg04037.html
so to me this is another reason to just expose the policy as an integer in the QAPI/QOM structure too. Everywhere just wants to be working with policy in integer format.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
Quoted 151 lines · Daniel P. Berrangé
Quoted 138 lines · Daniel P. Berrangé
Quoted 137 lines · Xiaoyao Li
Quoted 136 lines · Zhenzhong
Quoted 135 lines
-----Original Message-----
From: Li, Xiaoyao <[email protected]>
Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest objectQuoted 5 lines · Daniel P. Berrangé
Copying Zhenzhong Duan as my point relates to the proposed libvirt TDX patches.
Quoted 1 line · Xiaoyao Li
Bit 28 of TD attribute, named
SEPT_VE_DISABLE. When set to 1, itdisables
Quoted 45 lines
Quoted 31 lines
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure this bit.
Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v4:
- collect Acked-by from MarkusChanges in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)--- qapi/qom.json | 7 ++++++- target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-)qapi/qom.json +0
-0@@ -900,10 +900,15 @@ # # Properties for tdx-guest objects. #qapi/qom.json line 911
Quoted 10 lines
qapi/qom.json +6
-1+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling +# of EPT violation conversion to #VE on guest TD access of PENDING +# pages. Some guest OS (e.g., Linux TD guest) may require this to +# be set, otherwise they refuse to boot. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { }} + 'data': { '*sept-ve-disable': 'bool' } }So this exposes a single boolean property that gets mapped into one specific bit in the TD attributes:
Quoted 2 lines
qapi/qom.json line 913
Quoted 2 lines
qapi/qom.json +2
-0+ +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error**errp)
Quoted 12 lines
qapi/qom.json
Quoted 9 lines
qapi/qom.json +9
-0+{ + TdxGuest *tdx = TDX_GUEST(obj); + + if (value) { + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } else { + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE; + } +}If I look at the documentation for TD attributes
Quoted 1 line
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Quoted 8 lines
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently defined "DEBUG", "
SEPT_VE_DISABLE", "PKS", "PL", "PERFMON", and the rest currently reserved for future use. This makes me wonder about our modelling approach into the future ?Quoted 8 lines
For the AMD SEV equivalent we've just directly exposed the whole field as an int:
'policy' : 'uint32',For the proposed SEV-SNP patches, the same has been done again
06/msg00536.html
Quoted 11 lines
'*policy': 'uint64',The advantage of exposing individual booleans is that it is self-documenting at the QAPI level, but the disadvantage is that every time we want to expose ability to control a new bit in the policy we have to modify QEMU, libvirt, the mgmt app above libvirt, and whatever tools the end user has to talk to the mgmt app.
Quoted 9 lines
If we expose a policy int, then newly defined bits only require a change in QEMU, and everything above QEMU will already be capable of setting it.
In fact if I look at the proposed libvirt patches, they have proposed just exposing a policy "int" field in the XML, which then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/[email protected]/message/WXWX
EESYUA77DP7YIBP55T2OPSVKV5QW/Quoted 7 lines
On balance, I think it would be better if QEMU just exposed the raw TD attributes policy as an uint64 at QAPI, instead of trying to unpack it to discrete bool fields. This gives consistency with SEV and SEV-SNP, and with what's proposed at the libvirt level, and minimizes future changes when more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties, e.g.,
perfmon -> pmu, pks -> pks, kl -> keylokcer feature that QEMU currently doesn't supportIf allowing configuring attribute directly, we need to deal with the inconsistence between attribute vs cpu property.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
What about defining those bits associated with cpu properties reserved But other bits work as Daniel suggested way.
I don't understand. Do you mean we provide the interface to configure raw 64 bit attributes while some bits of it are reserved?
Just have a mask of what bits are permitted to be set, and report an error if the user sets non-permitted bits.
Looking at the IGVM patches, the CnofidentialGuestSupport class is gaining a "
set_guest_policy" method, which takes a "uint64_tpolicy".
Quoted 7 lines
If (when) IGVM support is extended to cover TDX too, then we're going to need to accept the user providing the policy in integer format via the IGVM file metadata. This will require adding code to check for any inconsistency between the policy bitmask, and the CPU flags.
https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg04037.html
Just have a glance at the IGVM series (which is a news to me). The IGVM series looks specific to SEV-*. Though some of the callbacks introduced in ConfidentialGuestSupportClass can be tweaked for TDX, but the arguments introduced are specific to SEV-*.
Aside from above, are you going to map (sev-*)'s policy to TDX's attributes? I don't think it is a good idea. It mixes up things with limited benefit. To me, the common of them is only they are both 64-bit field.
Quoted 3 lines
so to me this is another reason to just expose the policy as an integer in the QAPI/QOM structure too. Everywhere just wants to be working with policy in integer format.
I'm not reluctant to expose TD's attribute as a raw 64-bit data in QAPI structure. I just don't like the idea of the permitted mask, which makes the 64-bit field incomplete, and makes things complicated. People need to learn that some bits are configured in attribute directly while other bits are configured via cpu properties indirectly.
Maybe we can allow the direct configurability of raw 64-bit attribute and give it highest priority. If inconsistent value is provided via cpu properties, warn it and let the attribute value overwrite CPU properties?
Quoted 2 lines
With regards,
Daniel
PATCH v5 26/65 i386/tdx: Make sept_ve_disable set by default
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
For TDX KVM use case, Linux guest is the most major one. It requires sept_ve_disable set. Make it default for the main use case. For other use case, it can be enabled/disabled via qemu command line.
--- target/i386/kvm/tdx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/i386/kvm/tdx.c +1 -1
@@ -548,7 +548,7 @@ static void tdx_guest_init(Object *obj)qemu_mutex_init (&tdx->lock); - tdx->attributes =0 ; + tdx->attributes =TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE ; object_property_add_bool(obj, "sept-ve-disable", tdx_guest_get_sept_ve_disable,
Signature · 1 line
2.34.1
PATCH v5 27/65 i386/tdx: Wire CPU features up with attributes of TD guest
In reply to Xiaoyao LiFor QEMU VMs, PKS is configured via CPUID_7_0_ECX_PKS and PMU is configured by x86cpu->enable_pmu. Reuse the existing configuration interface for TDX VMs.
--- target/i386/kvm/tdx.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
target/i386/kvm/tdx.c +13 -0
@@ -33,6 +33,8 @@ (1U << KVM_FEATURE_MSI_EXT_DEST_ID))#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTES_PERFMON BIT_ULL(63)#define TDX_ATTRIBUTES_MAX_BITS 64
@@ -477,6 +479,15 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)return 0 ; } +static void setup_td_guest_attributes (X86CPU *x86cpu) +{ +CPUX86State *env = &x86cpu->env; + + tdx_guest->attributes |= (env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_PKS ) ? +TDX_TD_ATTRIBUTES_PKS :0 ; + tdx_guest->attributes |= x86cpu->enable_pmu ?TDX_TD_ATTRIBUTES_PERFMON :0 ; +} +int tdx_pre_create_vcpu (CPUState *cpu,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ());
@@ -499,6 +510,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return r; } +setup_td_guest_attributes (x86cpu); + init_vm->cpuid.nent =kvm_x86_arch_cpuid (env, init_vm->cpuid.entries,0 ); init_vm->attributes = tdx_guest->attributes;
Signature · 1 line
2.34.1
PATCH v5 28/65 i386/tdx: Disable pmu for TD guest
In reply to Xiaoyao LiCurrent KVM doesn't support PMU for TD guest. It returns error if TD is created with PMU bit being set in attributes.
Disable PMU for TD guest on QEMU side.
--- target/i386/kvm/tdx.c | 2 ++ 1 file changed, 2 insertions(+)
target/i386/kvm/tdx.c +2 -0
@@ -496,6 +496,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) g_autofree struct kvm_tdx_init_vm *init_vm = NULL;int r =0 ; +object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort); +QEMU_LOCK_GUARD (&tdx_guest->lock);if (tdx_guest->initialized) {return r;
Signature · 1 line
2.34.1
Re: [PATCH v5 28/65] i386/tdx: Disable pmu for TD guest
Quoted 15 lines · Xiaoyao Li
Current KVM doesn't support PMU for TD guest. It returns error if TD is created with PMU bit being set in attributes.
Disable PMU for TD guest on QEMU side.
Signed-off-by Xiaoyao Li <[email protected]>--- target/i386/kvm/tdx.c | 2 ++ 1 file changed, 2 insertions(+)target/i386/kvm/tdx.c +0
-0
target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 499
Quoted 4 lines
target/i386/kvm/tdx.c +1
-0@@ -496,6 +496,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) g_autofree struct kvm_tdx_init_vm *init_vm = NULL;int r =0 ; +object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);
Is it necessary to output some prompt if the user wants to enable pmu by "-cpu host,pmu=on"? As in patch 27, it mentions PMU is configured by x86cpu->enable_pmu, but PMU is actually not support in this series and will be disabled silently.
Quoted 4 lines
target/i386/kvm/tdx.c +1
-0+QEMU_LOCK_GUARD (&tdx_guest->lock);if (tdx_guest->initialized) {return r;
Re: [PATCH v5 28/65] i386/tdx: Disable pmu for TD guest
Quoted 21 lines · Chenyi Qiang
Quoted 18 lines · Xiaoyao Li
Current KVM doesn't support PMU for TD guest. It returns error if TD is created with PMU bit being set in attributes.
Disable PMU for TD guest on QEMU side.
Signed-off-by Xiaoyao Li <[email protected]>--- target/i386/kvm/tdx.c | 2 ++ 1 file changed, 2 insertions(+)target/i386/kvm/tdx.c +0
-0@@ -496,6 +496,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) g_autofree struct kvm_tdx_init_vm *init_vm = NULL;int r =0 ;
Quoted 6 lines
target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 499
Quoted 1 line
target/i386/kvm/tdx.c +1
-0+object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);Is it necessary to output some prompt if the user wants to enable pmu by "-cpu host,pmu=on"? As in patch 27, it mentions PMU is configured by
x86cpu->enable_pmu, but PMU is actually not support in this series and will be disabled silently.
We do this in QEMU is mainly for KVM, because KVM will fail to init TD if ATTRIBUTE.PERFMON is set.
It's expected that KVM reports PERFMON in attributes.fixed0 when KVM cannot provide support of it. Then QEMU will print error message automatically when validate the attributes.
For QEMU part, next version is going to set the default value of "pmu" to false in kvm_cpu_max_instance_init(), so that "-cpu host" will not enable pmu for TDX VMs by default.
I suppose both the KVM and QEMU change will show up in the next version.
Quoted 4 lines
Quoted 4 lines
target/i386/kvm/tdx.c +1
-0+QEMU_LOCK_GUARD (&tdx_guest->lock);if (tdx_guest->initialized) {return r;
PATCH v5 29/65 i386/tdx: Validate TD attributes
In reply to Xiaoyao LiValidate TD attributes with tdx_caps that fixed-0 bits must be zero and fixed-1 bits must be set.
Besides, sanity check the attribute bits that have not been supported by QEMU yet. e.g., debug bit, it will be allowed in the future when debug TD support lands in QEMU.
---
Changes in v3:
- using error_setg() for error report; (Daniel)
--- target/i386/kvm/tdx.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
target/i386/kvm/tdx.c +27 -2
@@ -32,6 +32,7 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_TD_ATTRIBUTES_DEBUG BIT_ULL(0)#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)#define TDX_TD_ATTRIBUTES_PERFMON BIT_ULL(63)
@@ -479,13 +480,34 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)return 0 ; } -static void setup_td_guest_attributes (X86CPU *x86cpu) +static int tdx_validate_attributes (TdxGuest *tdx,Error **errp) +{ +if (((tdx->attributes & tdx_caps->attrs_fixed0) | tdx_caps->attrs_fixed1) != + tdx->attributes) { +error_setg (errp,"Invalid attributes 0x%lx for TDX VM " +"(fixed0 0x%llx, fixed1 0x%llx)" , + tdx->attributes, tdx_caps->attrs_fixed0, + tdx_caps->attrs_fixed1); +return -1 ; + } + +if (tdx->attributes &TDX_TD_ATTRIBUTES_DEBUG ) { +error_setg (errp,"Current QEMU doesn't support attributes.debug[bit 0] for TDX VM" ); +return -1 ; + } + +return 0 ; +} + +static int setup_td_guest_attributes (X86CPU *x86cpu,Error **errp) {CPUX86State *env = &x86cpu->env; tdx_guest->attributes |= (env->features[FEAT_7_0_ECX ] &CPUID_7_0_ECX_PKS ) ?TDX_TD_ATTRIBUTES_PKS :0 ; tdx_guest->attributes |= x86cpu->enable_pmu ?TDX_TD_ATTRIBUTES_PERFMON :0 ; + +return tdx_validate_attributes (tdx_guest, errp); }int tdx_pre_create_vcpu (CPUState *cpu,Error **errp)
@@ -512,7 +534,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return r; } -setup_td_guest_attributes (x86cpu); + r =setup_td_guest_attributes (x86cpu, errp); +if (r) { +return r; + } init_vm->cpuid.nent =kvm_x86_arch_cpuid (env, init_vm->cpuid.entries,0 );
Signature · 1 line
2.34.1
PATCH v5 30/65 i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/
Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
---
Changes in v5:
- refine the description of QAPI properties and add description of default value when not specified;
Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in _setter(); (Daniel)
Changes in v3:
- use base64 encoding instread of hex-string;
--- qapi/qom.json | 17 ++++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 106 insertions(+), 1 deletion(-)
qapi/qom.json +16 -1
@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:
target/i386/kvm/tdx.c +87 -0
117 lines not on this page · raw message
target/i386/kvm/tdx.h +3 -0
@@ -21,6 +21,9 @@ typedef struct TdxGuest {bool initialized;uint64_t attributes;/* TD attributes */ +char *mrconfigid;/* base64 encoded sha348 digest */ +char *mrowner;/* base64 encoded sha348 digest */ +char *mrownerconfig;/* base64 encoded sha348 digest */ } TdxGuest;#ifdef CONFIG_TDX
Signature · 1 line
2.34.1
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 42 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v5:
- refine the description of QAPI properties and add description of default value when not specified;Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in_setter(); (Daniel)Changes in v3:
- use base64 encoding instread of hex-string;--- qapi/qom.json | 17 ++++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 106 insertions(+), 1 deletion(-)qapi/qom.json +0
-0
qapi/qom.json line 910
Quoted 6 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).
Suggest to drop the parenthesis in the last sentence.
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000". More on this below.
Quoted 45 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +3
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48
target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 543
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }
When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6 __u64, remains all zero. How does the kernel treat that?
Quoted 24 lines
target/i386/kvm/tdx.c +21
-0+ +if (tdx_guest->mrowner) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner, +strlen (tdx_guest->mrowner), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrowner" ); +return -1 ; + } +memcpy (init_vm->mrowner, data, data_len); + } + +if (tdx_guest->mrownerconfig) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig, +strlen (tdx_guest->mrownerconfig), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrownerconfig" ); +return -1 ; + } +memcpy (init_vm->mrownerconfig, data, data_len); + } + r =kvm_vm_enable_cap (kvm_state,KVM_CAP_MAX_VCPUS ,0 , ms->smp.cpus);if (r <0 ) {error_setg (errp,"Unable to set MAX VCPUS to %d" , ms->smp.cpus);
[...]
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 53 lines · Markus Armbruster
Quoted 42 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v5:
- refine the description of QAPI properties and add description ofdefault value when not specified;Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in_setter(); (Daniel)Changes in v3:
- use base64 encoding instread of hex-string;--- qapi/qom.json | 17 ++++++++- target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 3 ++ 3 files changed, 106 insertions(+), 1 deletion(-)qapi/qom.json +0
-0qapi/qom.json line 910
Quoted 6 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
Quoted 46 lines
Quoted 44 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +2
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES );
Quoted 15 lines
target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 543
Quoted 11 lines
target/i386/kvm/tdx.c +11
-0+#define SHA384_DIGEST_SIZE 48 + +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Quoted 27 lines
Quoted 24 lines
target/i386/kvm/tdx.c +21
-0+ +if (tdx_guest->mrowner) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner, +strlen (tdx_guest->mrowner), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrowner" ); +return -1 ; + } +memcpy (init_vm->mrowner, data, data_len); + } + +if (tdx_guest->mrownerconfig) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig, +strlen (tdx_guest->mrownerconfig), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrownerconfig" ); +return -1 ; + } +memcpy (init_vm->mrownerconfig, data, data_len); + } + r =kvm_vm_enable_cap (kvm_state,KVM_CAP_MAX_VCPUS ,0 , ms->smp.cpus);if (r <0 ) {error_setg (errp,"Unable to set MAX VCPUS to %d" , ms->smp.cpus);[...]
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Markus Armbruster
Quoted 18 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>
[...]
Quoted 20 lines
Quoted 14 lines
qapi/qom.json line 910
Quoted 11 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
Quoted 3 lines
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface documentation to be. We work together to get there.
Quoted 58 lines
Quoted 49 lines
Quoted 47 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +9
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48 + +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" );Quoted 8 lines
target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 543
Quoted 4 lines
target/i386/kvm/tdx.c +4
-0+return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?
Quoted 3 lines
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
[...]
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 111 lines · Markus Armbruster
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Markus Armbruster
Quoted 18 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 20 lines
Quoted 14 lines
qapi/qom.json line 910
Quoted 11 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".Quoted 3 lines
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface documentation to be. We work together to get there.
Quoted 58 lines
Quoted 57 lines
Quoted 41 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +3
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 542
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?
Quoted 6 lines
Quoted 3 lines
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
KVM just copy what QEMU provides into its own data structure @td_params. The format @of td_params is defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
In fact, all the three SHA384 fields, will be hashed together with some other fields (in td_params and other content of TD) to compromise the initial measurement of TD.
TDX module doesn't care the value of td_params->mrconfigid.
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 117 lines · Xiaoyao Li
Quoted 110 lines · Markus Armbruster
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Markus Armbruster
Quoted 18 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 20 lines
Quoted 14 lines
qapi/qom.json line 910
Quoted 11 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".Quoted 3 lines
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface documentation to be. We work together to get there.
Quoted 58 lines
Quoted 57 lines
Quoted 41 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +3
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 542
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?Quoted 5 lines
Quoted 3 lines
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
Quoted 8 lines
KVM just copy what QEMU provides into its own data structure @td_params. The format @of
td_paramsis defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450In fact, all the three SHA384 fields, will be hashed together with some other fields (in
td_paramsand other content of TD) to compromise the initial measurement of TD.TDX module doesn't care the value of
td_params->mrconfigid.
My problem is that I don't understand when and why users would omit the optional @mrFOO.
I naively expected absent @mrFOO to mean something like "no attestation of FOO".
But I see that they default to "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
If this zero value is special and means "no attestation", then we accidentally get no attestation when whatever is being hashed happens to hash to this zero value. Unlikely, but possible.
If it's not special, then when and why is the ability to omit it useful?
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 124 lines · Markus Armbruster
Quoted 122 lines · Xiaoyao Li
Quoted 110 lines · Markus Armbruster
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Markus Armbruster
Quoted 18 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 20 lines
Quoted 14 lines
qapi/qom.json line 910
Quoted 11 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".Quoted 3 lines
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface documentation to be. We work together to get there.
Quoted 58 lines
Quoted 57 lines
Quoted 41 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +3
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 542
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?Quoted 5 lines
Quoted 3 lines
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
KVM just copy what QEMU provides into its own data structure @td_params. The format @of
td_paramsis defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
Quoted 6 lines
Quoted 3 lines
In fact, all the three SHA384 fields, will be hashed together with some other fields (in
td_paramsand other content of TD) to compromise the initial measurement of TD.TDX module doesn't care the value of
td_params->mrconfigid.My problem is that I don't understand when and why users would omit the optional @mrFOO.
When users don't care it and don't have an explicit value for them, they can omit it. Then a default all-zero value is used.
If making it mandatory field, then users have to explicit pass a all-zero value when they don't care it.
Quoted 12 lines
I naively expected absent @mrFOO to mean something like "no attestation of FOO".
But I see that they default to "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
If this zero value is special and means "no attestation", then we accidentally get no attestation when whatever is being hashed happens to hash to this zero value. Unlikely, but possible.
If it's not special, then when and why is the ability to omit it useful?
At some point, the zero value is special, because it is the default value if no explicit one provided by user. But for TDX point of view, it is not special. The field is a must for any TD, and whatever value it is, it will be hashed into MRTD (Build-time Measurement Register) for later attestation.
TDX architecture defines what fields are always hashed into measurement and also provide other mechanism to hash optional field into measurement. All this is known to users of TDX, and users can calculate the final measurement by itself and compare to what gets reported by TDX to see they are identical.
For these three fields, they are must-to-have fields to be hashed into measurement. For user's convenience, we don't want to make it mandatory input because not everyone cares it and have a specific value to input. What people needs to know is that, when no explicit value is provided for these three field, a all-zero value is used.
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 146 lines · Xiaoyao Li
Quoted 124 lines · Markus Armbruster
Quoted 122 lines · Xiaoyao Li
Quoted 110 lines · Markus Armbruster
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Markus Armbruster
Quoted 18 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 20 lines
Quoted 14 lines
qapi/qom.json line 910
Quoted 11 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".Quoted 3 lines
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface documentation to be. We work together to get there.
Quoted 58 lines
Quoted 57 lines
Quoted 41 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +3
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 542
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?Quoted 5 lines
Quoted 3 lines
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
KVM just copy what QEMU provides into its own data structure @td_params. The format @of
td_paramsis defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450Quoted 6 lines
Quoted 3 lines
In fact, all the three SHA384 fields, will be hashed together with some other fields (in
td_paramsand other content of TD) to compromise the initial measurement of TD.TDX module doesn't care the value of
td_params->mrconfigid.My problem is that I don't understand when and why users would omit the optional @mrFOO.
When users don't care it and don't have an explicit value for them, they can omit it. Then a default all-zero value is used.
If making it mandatory field, then users have to explicit pass a all-zero value when they don't care it.
Quoted 10 lines
I naively expected absent @mrFOO to mean something like "no attestation of FOO".
But I see that they default to "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
If this zero value is special and means "no attestation", then we accidentally get no attestation when whatever is being hashed happens to hash to this zero value. Unlikely, but possible.
Quoted 8 lines
Quoted 1 line
If it's not special, then when and why is the ability to omit it useful?
At some point, the zero value is special, because it is the default value if no explicit one provided by user. But for TDX point of view, it is not special. The field is a must for any TD, and whatever value it is, it will be hashed into MRTD (Build-time Measurement Register) for later attestation.
TDX architecture defines what fields are always hashed into measurement and also provide other mechanism to hash optional field into measurement. All this is known to users of TDX, and users can calculate the final measurement by itself and compare to what gets reported by TDX to see they are identical.
For these three fields, they are must-to-have fields to be hashed into measurement. For user's convenience, we don't want to make it mandatory input because not everyone cares it and have a specific value to input. What people needs to know is that, when no explicit value is provided for these three field, a all-zero value is used.
Alright, the doc comment is not the place to educate me about TDX. Perhaps we can go with
# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
# Defaults to all zeroes.
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 150 lines · Markus Armbruster
Quoted 148 lines · Xiaoyao Li
Quoted 124 lines · Markus Armbruster
Quoted 122 lines · Xiaoyao Li
Quoted 110 lines · Markus Armbruster
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Markus Armbruster
Quoted 18 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \ mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\ mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83vSigned-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 20 lines
Quoted 14 lines
qapi/qom.json line 910
Quoted 11 lines
qapi/qom.json +3
-0@@ -905,10 +905,25 @@ # pages. Some guest OS (e.g., Linux TD guest) may require this to # be set, otherwise they refuse to boot. # +# @mrconfigid: ID for non-owner-defined configuration of the guest TD, +# e.g., run-time or OS configuration (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent).Suggest to drop the parenthesis in the last sentence.
Quoted 5 lines
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".Quoted 3 lines
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface documentation to be. We work together to get there.
Quoted 58 lines
Quoted 57 lines
Quoted 41 lines
qapi/qom.json +13
-1+# +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest). +# (A default value 0 of SHA384 is used when absent). +# +# @mrownerconfig: ID for owner-defined configuration of the guest TD, +# e.g., specific to the workload rather than the run-time or OS +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is +# used when absent). +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', - 'data': { '*sept-ve-disable': 'bool' } } + 'data': { '*sept-ve-disable': 'bool', + '*mrconfigid': 'str', + '*mrowner': 'str', + '*mrownerconfig': 'str' } } ## # @ThreadContextProperties:target/i386/kvm/tdx.c +3
-0@@ -13,6 +13,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" +#include "qemu/base64.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h" @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)X86CPU *x86cpu =X86_CPU (cpu);CPUX86State *env = &x86cpu->env; g_autofree struct kvm_tdx_init_vm *init_vm = NULL; +size_t data_len;int r =0 ;object_property_set_bool (OBJECT (cpu),"pmu" , false, &error_abort);@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) init_vm =g_malloc0 (sizeof (struct kvm_tdx_init_vm ) +sizeof (struct kvm_cpuid_entry2 ) *KVM_MAX_CPUID_ENTRIES ); +#define SHA384_DIGEST_SIZE 48target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 542
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ +if (tdx_guest->mrconfigid) { + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid, +strlen (tdx_guest->mrconfigid), &data_len, errp); +if (!data || data_len !=SHA384_DIGEST_SIZE ) { +error_setg (errp,"TDX: failed to decode mrconfigid" ); +return -1 ; + } +memcpy (init_vm->mrconfigid, data, data_len); + }When @mrconfigid is absent, the property remains null, and this conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?Quoted 5 lines
Quoted 3 lines
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
KVM just copy what QEMU provides into its own data structure @td_params. The format @of
td_paramsis defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450Quoted 6 lines
Quoted 3 lines
In fact, all the three SHA384 fields, will be hashed together with some other fields (in
td_paramsand other content of TD) to compromise the initial measurement of TD.TDX module doesn't care the value of
td_params->mrconfigid.My problem is that I don't understand when and why users would omit the optional @mrFOO.
When users don't care it and don't have an explicit value for them, they can omit it. Then a default all-zero value is used.
If making it mandatory field, then users have to explicit pass a all-zero value when they don't care it.
Quoted 11 lines
I naively expected absent @mrFOO to mean something like "no attestation of FOO".
But I see that they default to "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
If this zero value is special and means "no attestation", then we accidentally get no attestation when whatever is being hashed happens to hash to this zero value. Unlikely, but possible.
If it's not special, then when and why is the ability to omit it useful?
Quoted 8 lines
Quoted 6 lines
At some point, the zero value is special, because it is the default value if no explicit one provided by user. But for TDX point of view, it is not special. The field is a must for any TD, and whatever value it is, it will be hashed into MRTD (Build-time Measurement Register) for later attestation.
TDX architecture defines what fields are always hashed into measurement and also provide other mechanism to hash optional field into measurement. All this is known to users of TDX, and users can calculate the final measurement by itself and compare to what gets reported by TDX to see they are identical.
For these three fields, they are must-to-have fields to be hashed into measurement. For user's convenience, we don't want to make it mandatory input because not everyone cares it and have a specific value to input. What people needs to know is that, when no explicit value is provided for these three field, a all-zero value is used.
Alright, the doc comment is not the place to educate me about TDX.
I'm pleased to help you (on or off the maillist) if you have any question on TDX. :)
Quoted 6 lines
Perhaps we can go with
# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
# Defaults to all zeroes.
I will update to this in next version.
Huge thanks for your help!
Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
Quoted 1 line · Xiaoyao Li
On 3/7/2024 9:56 PM, Markus Armbruster wrote:
[...]
Quoted 3 lines
Quoted 1 line
Alright, the doc comment is not the place to educate me about TDX.
I'm pleased to help you (on or off the maillist) if you have any question on TDX. :)
I can't possibly become knowledgable about QEMU's entire external interface, it's simply too much. Where I'm ignorant, I try to learn just enough to ensure the documentation is useful.
I appreciate your kind & patient help with that.
Quoted 8 lines
Quoted 5 lines
Perhaps we can go with
# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
# Defaults to all zeroes.I will update to this in next version.
Huge thanks for your help!
You're welcome!
PATCH v5 31/65 i386/tdx: Implement user specified tsc frequency
In reply to Xiaoyao LiReuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.
Besides, sanity check the tsc frequency to be in the legal range and legal granularity (required by TDX module).
---
Changes in v3:
- use @errp to report error info; (Daniel)
Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM side drop the @tsc_khz field in struct kvm_tdx_init_vm
--- target/i386/kvm/kvm.c | 9 +++++++++ target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++ 2 files changed, 34 insertions(+)
target/i386/kvm/kvm.c +9 -0
@@ -793,6 +793,15 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)int r, cur_freq;bool set_ioctl = false; +/* +* TSC of TD vcpu is immutable, it cannot be set/changed via vcpu scope +* VM_SET_TSC_KHZ, but only be initialized via VM scope VM_SET_TSC_KHZ +* before ioctl KVM_TDX_INIT_VM in tdx_pre_create_vcpu() +*/ +if (is_tdx_vm ()) { +return 0 ; + } +if (!env->tsc_khz) {return 0 ; }
target/i386/kvm/tdx.c +25 -0
@@ -33,6 +33,9 @@ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \ (1U << KVM_FEATURE_MSI_EXT_DEST_ID)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) +#define TDX_TD_ATTRIBUTES_DEBUG BIT_ULL(0)#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)
@@ -568,6 +571,28 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return r; } +if (env->tsc_khz && (env->tsc_khz <TDX_MIN_TSC_FREQUENCY_KHZ || + env->tsc_khz >TDX_MAX_TSC_FREQUENCY_KHZ )) { +error_setg (errp,"Invalid TSC %ld KHz, must specify cpu_frequency between [%d, %d] kHz" , + env->tsc_khz,TDX_MIN_TSC_FREQUENCY_KHZ , +TDX_MAX_TSC_FREQUENCY_KHZ ); +return -EINVAL ; + } + +if (env->tsc_khz % (25 *1000 )) { +error_setg (errp,"Invalid TSC %ld KHz, it must be multiple of 25MHz" , + env->tsc_khz); +return -EINVAL ; + } + +/* it's safe even env->tsc_khz is 0. KVM uses host's tsc_khz in this case */ + r =kvm_vm_ioctl (kvm_state,KVM_SET_TSC_KHZ , env->tsc_khz); +if (r <0 ) { +error_setg_errno (errp, -r,"Unable to set TSC frequency to %" PRId64" kHz" , + env->tsc_khz); +return r; + } + r =setup_td_guest_attributes (x86cpu, errp);if (r) {return r;
Signature · 1 line
2.34.1
PATCH v5 32/65 i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM
In reply to Xiaoyao LiTDX only supports readonly for shared memory but not for private memory.
In the view of QEMU, it has no idea whether a memslot is used as shared memory of private. Thus just mark kvm_readonly_mem_enabled to false to TDX VM for simplicity.
--- target/i386/kvm/tdx.c | 9 +++++++++ 1 file changed, 9 insertions(+)
target/i386/kvm/tdx.c +9 -0
@@ -480,6 +480,15 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)update_tdx_cpuid_lookup_by_tdx_caps (); +/* +* Set kvm_readonly_mem_allowed to false, because TDX only supports readonly +* memory for shared memory but not for private memory. Besides, whether a +* memslot is private or shared is not determined by QEMU. +* +* Thus, just mark readonly memory not supported for simplicity. +*/ + kvm_readonly_mem_allowed = false; + tdx_guest = tdx;return 0 ; }
Signature · 1 line
2.34.1
PATCH v5 33/65 kvm/tdx: Don't complain when converting vMMIO region to shared
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Because vMMIO region needs to be shared region, guest TD may explicitly convert such region from private to shared. Don't complain such conversion.
--- accel/kvm/kvm-all.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
accel/kvm/kvm-all.c +16 -3
@@ -2985,9 +2985,22 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private) ret =ram_block_discard_guest_memfd_range (rb, offset, size); } }else { -error_report ("Convert non guest_memfd backed memory region " -"(0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , - start, size, to_private ?"private" :"shared" ); +/* +* Because vMMIO region must be shared, guest TD may convert vMMIO +* region to shared explicitly. Don't complain such case. See +* memory_region_type() for checking if the region is MMIO region. +*/ +if (!to_private && + !memory_region_is_ram (mr) && + !memory_region_is_ram_device (mr) && + !memory_region_is_rom (mr) && + !memory_region_is_romd (mr)) { + ret =0 ; + }else { +error_report ("Convert non guest_memfd backed memory region " +"(0x%" HWADDR_PRIx" ,+ 0x%" HWADDR_PRIx") to %s" , + start, size, to_private ?"private" :"shared" ); + } }memory_region_unref (section.mr);
Signature · 1 line
2.34.1
PATCH v5 34/65 kvm/tdx: Ignore memory conversion to shared of unassigned region
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TDX requires vMMIO region to be shared. For KVM, MMIO region is the region which kvm memslot isn't assigned to (except in-kernel emulation). qemu has the memory region for vMMIO at each device level.
While OVMF issues MapGPA(to-shared) conservatively on 32bit PCI MMIO region, qemu doesn't find corresponding vMMIO region because it's before PCI device allocation and memory_region_find() finds the device region, not PCI bus region. It's safe to ignore MapGPA(to-shared) because when guest accesses those region they use GPA with shared bit set for vMMIO. Ignore memory conversion request of non-assigned region to shared and return success. Otherwise OVMF is confused and panics there.
--- accel/kvm/kvm-all.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
accel/kvm/kvm-all.c +12 -0
@@ -2953,6 +2953,18 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private) section =memory_region_find (get_system_memory (), start, size); mr = section.mr;if (!mr) { +/* +* Ignore converting non-assigned region to shared. +* +* TDX requires vMMIO region to be shared to inject #VE to guest. +* OVMF issues conservatively MapGPA(shared) on 32bit PCI MMIO region, +* and vIO-APIC 0xFEC00000 4K page. +* OVMF assigns 32bit PCI MMIO region to +* [top of low memory: typically 2GB=0xC000000, 0xFC00000) +*/ +if (!to_private) { +return 0 ; + }return -1 ; }
Signature · 1 line
2.34.1
PATCH v5 35/65 memory: Introduce memory_region_init_ram_guest_memfd()
In reply to Xiaoyao LiIntroduce memory_region_init_ram_guest_memfd() to allocate private guset memfd on the MemoryRegion initialization. It's for the use case of TDVF, which must be private on TDX case.
---
Changes in v5:
- drop memory_region_set_default_private() because this function is dropped in this v5 series;
--- include/exec/memory.h | 6 ++++++ system/memory.c | 25 +++++++++++++++++++++++++ 2 files changed, 31 insertions(+)
include/exec/memory.h +6 -0
@@ -1603,6 +1603,12 @@ bool memory_region_init_ram(MemoryRegion *mr,uint64_t size, Error **errp); +bool memory_region_init_ram_guest_memfd (MemoryRegion *mr, +Object *owner, +const char *name, +uint64_t size, +Error **errp); +/** * memory_region_init_rom: Initialize a ROM memory region. *
system/memory.c +25 -0
@@ -3606,6 +3606,31 @@ bool memory_region_init_ram(MemoryRegion *mr,return true; } +bool memory_region_init_ram_guest_memfd (MemoryRegion *mr, +Object *owner, +const char *name, +uint64_t size, +Error **errp) +{ +DeviceState *owner_dev; + +if (!memory_region_init_ram_flags_nomigrate (mr, owner, name, size, +RAM_GUEST_MEMFD , errp)) { +return false; + } + +/* This will assert if owner is neither NULL nor a DeviceState. +* We only want the owner here for the purposes of defining a +* unique name for migration. TODO: Ideally we should implement +* a naming scheme for Objects which are not DeviceStates, in +* which case we can relax this restriction. +*/ + owner_dev =DEVICE (owner); +vmstate_register_ram (mr, owner_dev); + +return true; +} + bool memory_region_init_rom(MemoryRegion *mr, Object *owner, const char *name,
Signature · 1 line
2.34.1
PATCH v5 36/65 i386/tdx: load TDVF for TD guest
In reply to Xiaoyao LiFrom: Chao Peng <[email protected]>
TDVF(OVMF) needs to run at private memory for TD guest. TDX cannot support pflash device since it doesn't support read-only private memory. Thus load TDVF(OVMF) with -bios option for TDs.
Use memory_region_init_ram_guest_memfd() to allocate the MemoryRegion for TDVF because it needs to be located at private memory.
Also store the MemoryRegion pointer of TDVF since the shared ramblock of it can be discared after it gets copied to private ramblock.
--- hw/i386/x86.c | 13 +++++++++++-- target/i386/kvm/tdx.c | 7 +++++++ target/i386/kvm/tdx.h | 3 +++ 3 files changed, 21 insertions(+), 2 deletions(-)
hw/i386/x86.c +11 -2
@@ -47,6 +47,7 @@#include "hw/intc/i8259.h" #include "hw/rtc/mc146818rtc.h" #include "target/i386/sev.h" +#include "kvm/tdx.h" #include "hw/acpi/cpu_hotplug.h" #include "hw/irq.h"
@@ -1157,9 +1158,17 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware, (bios_size % 65536) != 0) { goto bios_error; } + bios =g_malloc (sizeof (*bios)); -memory_region_init_ram (bios,NULL ,"pc.bios" , bios_size, &error_fatal); -if (sev_enabled ()) { +if (is_tdx_vm ()) { +memory_region_init_ram_guest_memfd (bios,NULL ,"pc.bios" , bios_size, + &error_fatal); +tdx_set_tdvf_region (bios); + }else { +memory_region_init_ram (bios,NULL ,"pc.bios" , bios_size, &error_fatal); + } + +if (sev_enabled () ||is_tdx_vm ()) {/* * The concept of a "reset" simply doesn't exist for * confidential computing guests, we have to destroy and
target/i386/kvm/tdx.c +7 -0
@@ -19,6 +19,7 @@#include "standard-headers/asm-x86/kvm_para.h" #include "sysemu/kvm.h" #include "sysemu/sysemu.h" +#include "exec/ramblock.h" #include "hw/i386/x86.h" #include "kvm_i386.h"
@@ -463,6 +464,12 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void) (tdx_caps->xfam_fixed1 &CPUID_XSTATE_XSS_MASK ) >>32 ; } +void tdx_set_tdvf_region (MemoryRegion *tdvf_mr) +{ +assert (!tdx_guest->tdvf_mr); + tdx_guest->tdvf_mr = tdvf_mr; +} +static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ());
target/i386/kvm/tdx.h +3 -0
@@ -24,6 +24,8 @@ typedef struct TdxGuest {char *mrconfigid;/* base64 encoded sha348 digest */ char *mrowner;/* base64 encoded sha348 digest */ char *mrownerconfig;/* base64 encoded sha348 digest */ + +MemoryRegion *tdvf_mr; } TdxGuest;#ifdef CONFIG_TDX
@@ -35,5 +37,6 @@ bool is_tdx_vm(void);void tdx_get_supported_cpuid (uint32_t function,uint32_t index,int reg,uint32_t *ret);int tdx_pre_create_vcpu (CPUState *cpu,Error **errp); +void tdx_set_tdvf_region (MemoryRegion *tdvf_mr);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v5 37/65 i386/tdvf: Introduce function to parse TDVF metadata
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.
A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function tdvf_parse_metadata() to parse the metadata from TDVF image and store the info of each TDVF section.
TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.
Select X86_FW_OVMF when TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.
---
Changes in v1:
- rename tdvf_parse_section_entry() to tdvf_parse_and_check_section_entry()
Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
--- hw/i386/Kconfig | 1 + hw/i386/meson.build | 1 + hw/i386/tdvf.c | 199 +++++++++++++++++++++++++++++++++++++++++ include/hw/i386/tdvf.h | 51 +++++++++++ 4 files changed, 252 insertions(+) create mode 100644 hw/i386/tdvf.c create mode 100644 include/hw/i386/tdvf.h
hw/i386/Kconfig +1 -0
@@ -12,6 +12,7 @@ config SGX config TDX bool + select X86_FW_OVMF depends on KVM config PC
hw/i386/meson.build +1 -0
@@ -28,6 +28,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'port92.c')) i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'), if_false: files('pc_sysfw_ovmf-stubs.c')) +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c')) subdir('kvm') subdir('xen')
hw/i386/tdvf.c new file +199 -0
199 lines not on this page · raw message
include/hw/i386/tdvf.h new file +51 -0
51 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v5 37/65] i386/tdvf: Introduce function to parse TDVF metadata
Quoted 228 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.
A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function
tdvf_parse_metadata()to parse the metadata from TDVF image and store the info of each TDVF section.TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.
Select
X86_FW_OVMFwhen TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.Signed-off-by Isaku Yamahata <[email protected]>Co-developed-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>Acked-by Gerd Hoffmann <[email protected]>---
Changes in v1:
- renametdvf_parse_section_entry()totdvf_parse_and_check_section_entry()
Changes in RFC v4:
- renameTDX_METADATA_GUIDtoTDX_METADATA_OFFSET_GUID--- hw/i386/Kconfig | 1 + hw/i386/meson.build | 1 + hw/i386/tdvf.c | 199 +++++++++++++++++++++++++++++++++++++++++ include/hw/i386/tdvf.h | 51 +++++++++++ 4 files changed, 252 insertions(+)create mode 100644
hw/i386/tdvf.c
create mode 100644include/hw/i386/tdvf.hhw/i386/Kconfig +1
-0@@ -12,6 +12,7 @@ config SGX config TDX bool + select X86_FW_OVMF depends on KVM config PChw/i386/meson.build +1
-0@@ -28,6 +28,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'port92.c')) i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'), if_false: files('pc_sysfw_ovmf-stubs.c')) +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c')) subdir('kvm') subdir('xen')hw/i386/tdvf.c new file +159
-0@@ -0,0 +1,199 @@ +/* +* SPDX-License-Identifier: GPL-2.0-or-later + +* Copyright (c) 2020 Intel Corporation +* Author: Isaku Yamahata <isaku.yamahata at gmail.com> +* <isaku.yamahata at intel.com> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. + +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. + +* You should have received a copy of the GNU General Public License along +* with this program; if not, see <http://www.gnu.org/licenses/>. +*/ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" + +#include "hw/i386/pc.h" +#include "hw/i386/tdvf.h" +#include "sysemu/kvm.h" + +#define TDX_METADATA_OFFSET_GUID "e47a6535-984a-4798-865e-4685a7bf8ec2" +#define TDX_METADATA_VERSION 1 +#define TDVF_SIGNATURE 0x46564454/* TDVF as little endian */ + +typedef struct { +uint32_t DataOffset; +uint32_t RawDataSize; +uint64_t MemoryAddress; +uint64_t MemoryDataSize; +uint32_t Type; +uint32_t Attributes; +}TdvfSectionEntry ; + +typedef struct { +uint32_t Signature; +uint32_t Length; +uint32_t Version; +uint32_t NumberOfSectionEntries; +TdvfSectionEntry SectionEntries[]; +}TdvfMetadata ; + +struct tdx_metadata_offset { +uint32_t offset; +}; + +static TdvfMetadata *tdvf_get_metadata (void *flash_ptr,int size) +{ +TdvfMetadata *metadata; +uint32_t offset =0 ; +uint8_t *data; + +if ((uint32_t ) size != size) { +return NULL ; + } + +if (pc_system_ovmf_table_find (TDX_METADATA_OFFSET_GUID , &data,NULL )) { + offset = size -le32_to_cpu (((struct tdx_metadata_offset *)data)->offset); + +if (offset +sizeof (*metadata) > size) { +return NULL ; + } + }else { +error_report ("Cannot find TDX_METADATA_OFFSET_GUID" ); +return NULL ; + } + + metadata = flash_ptr + offset; + +/* Finally, verify the signature to determine if this is a TDVF image. */ + metadata->Signature =le32_to_cpu (metadata->Signature); +if (metadata->Signature !=TDVF_SIGNATURE ) { +error_report ("Invalid TDVF signature in metadata!" ); +return NULL ; + } + +/* Sanity check that the TDVF doesn't overlap its own metadata. */ + metadata->Length =le32_to_cpu (metadata->Length); +if (offset + metadata->Length > size) { +return NULL ; + } + +/* Only version 1 is supported/defined. */ + metadata->Version =le32_to_cpu (metadata->Version); +if (metadata->Version !=TDX_METADATA_VERSION ) { +return NULL ; + } + +return metadata; +} + +static int tdvf_parse_and_check_section_entry (const TdvfSectionEntry *src, +TdxFirmwareEntry *entry) +{ + entry->data_offset =le32_to_cpu (src->DataOffset); + entry->data_len =le32_to_cpu (src->RawDataSize); + entry->address =le64_to_cpu (src->MemoryAddress); + entry->size =le64_to_cpu (src->MemoryDataSize); + entry->type =le32_to_cpu (src->Type); + entry->attributes =le32_to_cpu (src->Attributes); + +/* sanity check */ +if (entry->size < entry->data_len) { +error_report ("Broken metadata RawDataSize 0x%x MemoryDataSize 0x%lx" , + entry->data_len, entry->size); +return -1 ; + } +if (!QEMU_IS_ALIGNED (entry->address,TARGET_PAGE_SIZE )) { +error_report ("MemoryAddress 0x%lx not page aligned" , entry->address); +return -1 ; + } +if (!QEMU_IS_ALIGNED (entry->size,TARGET_PAGE_SIZE )) { +error_report ("MemoryDataSize 0x%lx not page aligned" , entry->size); +return -1 ; + } + +switch (entry->type) { +case TDVF_SECTION_TYPE_BFV : +case TDVF_SECTION_TYPE_CFV : +/* The sections that must be copied from firmware image to TD memory */ +if (entry->data_len ==0 ) { +error_report ("%d section with RawDataSize == 0" , entry->type); +return -1 ; + } +break ; +case TDVF_SECTION_TYPE_TD_HOB : +case TDVF_SECTION_TYPE_TEMP_MEM : +/* The sections that no need to be copied from firmware image */ +if (entry->data_len !=0 ) { +error_report ("%d section with RawDataSize 0x%x != 0" , + entry->type, entry->data_len); +return -1 ; + } +break ; +default : +error_report ("TDVF contains unsupported section type %d" , entry->type); +return -1 ; + } + +return 0 ; +} + +int tdvf_parse_metadata (TdxFirmware *fw,void *flash_ptr,int size) +{ +TdvfSectionEntry *sections; +TdvfMetadata *metadata; +ssize_t entries_size; +uint32_t len, i; + + metadata =tdvf_get_metadata (flash_ptr, size); +if (!metadata) { +return -EINVAL ;
hw/i386/tdvf.c line 169
Quoted 10 lines
hw/i386/tdvf.c new file +10
-0+ } + +//load and parse metadata entries + fw->nr_entries =le32_to_cpu (metadata->NumberOfSectionEntries); +if (fw->nr_entries <2 ) { +error_report ("Invalid number of fw entries (%u) in TDVF" , fw->nr_entries); +return -EINVAL ; + } + + len =le32_to_cpu (metadata->Length);
metadata->Length is already CPU endian, any reason to convert again?
Quoted 16 lines
hw/i386/tdvf.c new file +16
-0+ entries_size = fw->nr_entries *sizeof (TdvfSectionEntry); +if (len !=sizeof (*metadata) + entries_size) { +error_report ("TDVF metadata len (0x%x) mismatch, expected (0x%x)" , + len, (uint32_t )(sizeof (*metadata) + entries_size)); +return -EINVAL ; + } + + fw->entries =g_new (TdxFirmwareEntry, fw->nr_entries); + sections =g_new (TdvfSectionEntry, fw->nr_entries); + +if (!memcpy (sections, (void *)metadata +sizeof (*metadata), entries_size)) { +error_report ("Failed to read TDVF section entries" ); + goto err; + } + +for (i =0 ; i < fw->nr_entries; i++) {
hw/i386/tdvf.c line 195
Quoted 10 lines
hw/i386/tdvf.c new file +10
-0+if (tdvf_parse_and_check_section_entry (§ions[i], &fw->entries[i])) { + goto err; + } + } +g_free (sections); + +return 0 ; + +err: +g_free (sections);
This can be g_autofree.
Thanks
Zhenzhong
PATCH v5 38/65 i386/tdx: Parse TDVF metadata for TDX VM
In reply to Xiaoyao LiAfter TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.
--- hw/i386/pc_sysfw.c | 7 +++++++ target/i386/kvm/tdx-stub.c | 5 +++++ target/i386/kvm/tdx.c | 5 +++++ target/i386/kvm/tdx.h | 3 +++ 4 files changed, 20 insertions(+)
hw/i386/pc_sysfw.c +7 -0
@@ -37,6 +37,7 @@#include "hw/block/flash.h" #include "sysemu/kvm.h" #include "sev.h" +#include "kvm/tdx.h" #define FLASH_SECTOR_SIZE 4096
@@ -254,5 +255,11 @@ void x86_firmware_configure(void *ptr, int size) }sev_encrypt_flash (ptr, size, &error_fatal); + }else if (is_tdx_vm ()) { + ret =tdx_parse_tdvf (ptr, size); +if (ret) { +error_report ("failed to parse TDVF for TDX VM" ); +exit (1 ); + } } }
target/i386/kvm/tdx-stub.c +5 -0
@@ -6,3 +6,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) {return -EINVAL ; } + +int tdx_parse_tdvf (void *flash_ptr,int size) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +5 -0
@@ -631,6 +631,11 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)return 0 ; } +int tdx_parse_tdvf (void *flash_ptr,int size) +{ +return tdvf_parse_metadata (&tdx_guest->tdvf, flash_ptr, size); +} +static bool tdx_guest_get_sept_ve_disable (Object *obj,Error **errp) {TdxGuest *tdx =TDX_GUEST (obj);
target/i386/kvm/tdx.h +3 -0
@@ -6,6 +6,7 @@#endif #include "exec/confidential-guest-support.h" +#include "hw/i386/tdvf.h" #define TYPE_TDX_GUEST "tdx-guest"#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -26,6 +27,7 @@ typedef struct TdxGuest {char *mrownerconfig;/* base64 encoded sha348 digest */ MemoryRegion *tdvf_mr; +TdxFirmware tdvf; } TdxGuest;#ifdef CONFIG_TDX
@@ -38,5 +40,6 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg, uint32_t *ret);int tdx_pre_create_vcpu (CPUState *cpu,Error **errp);void tdx_set_tdvf_region (MemoryRegion *tdvf_mr); +int tdx_parse_tdvf (void *flash_ptr,int size);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v5 39/65 i386/tdx: Skip BIOS shadowing setup
In reply to Xiaoyao LiTDX doesn't support map different GPAs to same private memory. Thus, aliasing top 128KB of BIOS as isa-bios is not supported.
On the other hand, TDX guest cannot go to real mode, it can work fine without isa-bios.
---
Changes in v1:
- update commit message and comment to clarify
--- hw/i386/x86.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-)
hw/i386/x86.c +14 -11
@@ -1190,17 +1190,20 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware, }g_free (filename); -/* map the last 128KB of the BIOS in ISA space */ - isa_bios_size =MIN (bios_size,128 * KiB); - isa_bios =g_malloc (sizeof (*isa_bios)); -memory_region_init_alias (isa_bios,NULL ,"isa-bios" , bios, - bios_size - isa_bios_size, isa_bios_size); -memory_region_add_subregion_overlap (rom_memory, -0x100000 - isa_bios_size, - isa_bios, -1 ); -if (!isapc_ram_fw) { -memory_region_set_readonly (isa_bios, true); +/* For TDX, alias different GPAs to same private memory is not supported */ +if (!is_tdx_vm ()) { +/* map the last 128KB of the BIOS in ISA space */ + isa_bios_size =MIN (bios_size,128 * KiB); + isa_bios =g_malloc (sizeof (*isa_bios)); +memory_region_init_alias (isa_bios,NULL ,"isa-bios" , bios, + bios_size - isa_bios_size, isa_bios_size); +memory_region_add_subregion_overlap (rom_memory, +0x100000 - isa_bios_size, + isa_bios, +1 ); +if (!isapc_ram_fw) { +memory_region_set_readonly (isa_bios, true); + } }/* map all the bios at the top of memory */
Signature · 1 line
2.34.1
PATCH v5 40/65 i386/tdx: Don't initialize pc.rom for TDX VMs
In reply to Xiaoyao LiFor TDX, the address below 1MB are entirely general RAM. No need to initialize pc.rom memory region for TDs.
---
This is more as a workaround of the issue that for q35 machine type, the real memslot update (which requires memslot deletion )for pc.rom happens after tdx_init_memory_region. It leads to the private memory ADD'ed before get lost. I haven't work out a good solution to resolve the order issue. So just skip the pc.rom setup to avoid memslot deletion.
--- hw/i386/pc.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-)
hw/i386/pc.c +12 -9
@@ -43,6 +43,7 @@#include "sysemu/xen.h" #include "sysemu/reset.h" #include "kvm/kvm_i386.h" +#include "kvm/tdx.h" #include "hw/xen/xen.h" #include "qapi/qmp/qlist.h" #include "qemu/error-report.h"
@@ -1028,16 +1029,18 @@ void pc_memory_init(PCMachineState *pcms,/* Initialize PC system firmware */ pc_system_firmware_init (pcms, rom_memory); - option_rom_mr =g_malloc (sizeof (*option_rom_mr)); -memory_region_init_ram (option_rom_mr,NULL ,"pc.rom" ,PC_ROM_SIZE , - &error_fatal); -if (pcmc->pci_enabled) { -memory_region_set_readonly (option_rom_mr, true); +if (!is_tdx_vm ()) { + option_rom_mr =g_malloc (sizeof (*option_rom_mr)); +memory_region_init_ram (option_rom_mr,NULL ,"pc.rom" ,PC_ROM_SIZE , + &error_fatal); +if (pcmc->pci_enabled) { +memory_region_set_readonly (option_rom_mr, true); + } +memory_region_add_subregion_overlap (rom_memory, +PC_ROM_MIN_VGA , + option_rom_mr, +1 ); } -memory_region_add_subregion_overlap (rom_memory, -PC_ROM_MIN_VGA , - option_rom_mr, -1 ); fw_cfg =fw_cfg_arch_create (machine, x86ms->boot_cpus, x86ms->apic_id_limit);
Signature · 1 line
2.34.1
PATCH v5 41/65 i386/tdx: Track mem_ptr for each firmware entry of TDVF
In reply to Xiaoyao LiFor each TDVF sections, QEMU needs to copy the content to guest private memory via KVM API (KVM_TDX_INIT_MEM_REGION).
Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory pointer of each TDVF sections. So that QEMU can add/copy them to guest private memory later.
TDVF sections can be classified into two groups:
- Firmware itself, e.g., TDVF BFV and CFV, that located separately from guest RAM. Its memory pointer is the bios pointer.
- Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB. mmap a new memory range for them.
Register a machine_init_done callback to do the stuff.
--- hw/i386/tdvf.c | 1 + include/hw/i386/tdvf.h | 7 +++++++ target/i386/kvm/tdx.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+)
hw/i386/tdvf.c +1 -0
@@ -189,6 +189,7 @@ int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size) }g_free (sections); + fw->mem_ptr = flash_ptr;return 0 ; err:
include/hw/i386/tdvf.h +7 -0
@@ -39,13 +39,20 @@ typedef struct TdxFirmwareEntry {uint64_t size;uint32_t type;uint32_t attributes; + + void *mem_ptr; } TdxFirmwareEntry;typedef struct TdxFirmware { +void *mem_ptr; +uint32_t nr_entries;TdxFirmwareEntry *entries; }TdxFirmware ; +#define for_each_tdx_fw_entry (fw, e) \ + for (e = (fw)->entries; e != (fw)->entries + (fw)->nr_entries; e++) +int tdvf_parse_metadata (TdxFirmware *fw,void *flash_ptr,int size);#endif /* HW_I386_TDVF_H */
target/i386/kvm/tdx.c +31 -0
@@ -14,6 +14,7 @@#include "qemu/osdep.h" #include "qemu/error-report.h" #include "qemu/base64.h" +#include "qemu/mmap-alloc.h" #include "qapi/error.h" #include "qom/object_interfaces.h" #include "standard-headers/asm-x86/kvm_para.h"
@@ -22,6 +23,7 @@#include "exec/ramblock.h" #include "hw/i386/x86.h" +#include "hw/i386/tdvf.h" #include "kvm_i386.h" #include "tdx.h" #include "../cpu-internal.h"
@@ -470,6 +472,33 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr) tdx_guest->tdvf_mr = tdvf_mr; } +static void tdx_finalize_vm (Notifier *notifier,void *unused) +{ +TdxFirmware *tdvf = &tdx_guest->tdvf; +TdxFirmwareEntry *entry; + +for_each_tdx_fw_entry (tdvf, entry) { +switch (entry->type) { +case TDVF_SECTION_TYPE_BFV : +case TDVF_SECTION_TYPE_CFV : + entry->mem_ptr = tdvf->mem_ptr + entry->data_offset; +break ; +case TDVF_SECTION_TYPE_TD_HOB : +case TDVF_SECTION_TYPE_TEMP_MEM : + entry->mem_ptr =qemu_ram_mmap (-1 , entry->size, +qemu_real_host_page_size (),0 ,0 ); +break ; +default : +error_report ("Unsupported TDVF section %d" , entry->type); +exit (1 ); + } + } +} + +static Notifier tdx_machine_done_notify = { + .notify = tdx_finalize_vm, +}; +static int tdx_kvm_init (ConfidentialGuestSupport *cgs,Error **errp) {MachineState *ms =MACHINE (qdev_get_machine ());
@@ -496,6 +525,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)*/ kvm_readonly_mem_allowed = false; +qemu_add_machine_init_done_notifier (&tdx_machine_done_notify); + tdx_guest = tdx;return 0 ; }
Signature · 1 line
2.34.1
PATCH v5 42/65 i386/tdx: Track RAM entries for TDX VM
In reply to Xiaoyao LiThe RAM of TDX VM can be classified into two types:
- TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be accepted by TDX guest before it can be used and will be all-zeros after being accepted.
- TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.
Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table and mark each RAM range as default type TDX_RAM_UNACCEPTED.
Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these ranges will be ADD'ed before TD runs and no need to be accepted runtime.
The TdxRamEntries[] are later used to setup the memory TD resource HOB that passes memory info from QEMU to TDVF.
---
Changes in v3:
- use enum TdxRamType in struct TdxRamEntry; (Isaku)
- Fix the indention; (Daniel)
Changes in v1:
- simplify the algorithm of tdx_accept_ram_range() (Suggested-by: Gerd Hoffman)
(1) Change the existing entry to cover the accepted ram range.
(2) If there is room before the accepted ram range add a TDX_RAM_UNACCEPTED entry for that.
(3) If there is room after the accepted ram range add a TDX_RAM_UNACCEPTED entry for that.
--- target/i386/kvm/tdx.c | 111 ++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 14 ++++++ 2 files changed, 125 insertions(+)
target/i386/kvm/tdx.c +111 -0
@@ -22,6 +22,7 @@#include "sysemu/sysemu.h" #include "exec/ramblock.h" +#include "hw/i386/e820_memory_layout.h" #include "hw/i386/x86.h" #include "hw/i386/tdvf.h" #include "kvm_i386.h"
@@ -472,11 +473,117 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr) tdx_guest->tdvf_mr = tdvf_mr; } +static void tdx_add_ram_entry (uint64_t address,uint64_t length, +enum TdxRamType type) +{ +uint32_t nr_entries = tdx_guest->nr_ram_entries; + tdx_guest->ram_entries =g_renew (TdxRamEntry, tdx_guest->ram_entries, + nr_entries +1 ); + + tdx_guest->ram_entries[nr_entries].address = address; + tdx_guest->ram_entries[nr_entries].length = length; + tdx_guest->ram_entries[nr_entries].type = type; + tdx_guest->nr_ram_entries++; +} + +static int tdx_accept_ram_range (uint64_t address,uint64_t length) +{ +uint64_t head_start, tail_start, head_length, tail_length; +uint64_t tmp_address, tmp_length; +TdxRamEntry *e; +int i; + +for (i =0 ; i < tdx_guest->nr_ram_entries; i++) { + e = &tdx_guest->ram_entries[i]; + +if (address + length <= e->address || + e->address + e->length <= address) { +continue ; + } + +/* +* The to-be-accepted ram range must be fully contained by one +* RAM entry. +*/ +if (e->address > address || + e->address + e->length < address + length) { +return -EINVAL ; + } + +if (e->type ==TDX_RAM_ADDED ) { +return -EINVAL ; + } + +break ; + } + +if (i == tdx_guest->nr_ram_entries) { +return -1 ; + } + + tmp_address = e->address; + tmp_length = e->length; + + e->address = address; + e->length = length; + e->type =TDX_RAM_ADDED ; + + head_length = address - tmp_address; +if (head_length >0 ) { + head_start = tmp_address; +tdx_add_ram_entry (head_start, head_length,TDX_RAM_UNACCEPTED ); + } + + tail_start = address + length; +if (tail_start < tmp_address + tmp_length) { + tail_length = tmp_address + tmp_length - tail_start; +tdx_add_ram_entry (tail_start, tail_length,TDX_RAM_UNACCEPTED ); + } + +return 0 ; +} + +static int tdx_ram_entry_compare (const void *lhs_,const void * rhs_) +{ +const TdxRamEntry *lhs = lhs_; +const TdxRamEntry *rhs = rhs_; + +if (lhs->address == rhs->address) { +return 0 ; + } +if (le64_to_cpu (lhs->address) >le64_to_cpu (rhs->address)) { +return 1 ; + } +return -1 ; +} + +static void tdx_init_ram_entries (void ) +{ +unsigned i, j, nr_e820_entries; + + nr_e820_entries =e820_get_num_entries (); + tdx_guest->ram_entries =g_new (TdxRamEntry, nr_e820_entries); + +for (i =0 , j =0 ; i < nr_e820_entries; i++) { +uint64_t addr, len; + +if (e820_get_entry (i,E820_RAM , &addr, &len)) { + tdx_guest->ram_entries[j].address = addr; + tdx_guest->ram_entries[j].length = len; + tdx_guest->ram_entries[j].type =TDX_RAM_UNACCEPTED ; + j++; + } + } + tdx_guest->nr_ram_entries = j; +} +static void tdx_finalize_vm (Notifier *notifier,void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;TdxFirmwareEntry *entry; +tdx_init_ram_entries (); +for_each_tdx_fw_entry (tdvf, entry) {switch (entry->type) {case TDVF_SECTION_TYPE_BFV :
@@ -487,12 +594,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)case TDVF_SECTION_TYPE_TEMP_MEM : entry->mem_ptr =qemu_ram_mmap (-1 , entry->size,qemu_real_host_page_size (),0 ,0 ); +tdx_accept_ram_range (entry->address, entry->size);break ;default :error_report ("Unsupported TDVF section %d" , entry->type);exit (1 ); } } + +qsort (tdx_guest->ram_entries, tdx_guest->nr_ram_entries, +sizeof (TdxRamEntry), &tdx_ram_entry_compare); }static Notifier tdx_machine_done_notify = {
target/i386/kvm/tdx.h +14 -0
@@ -15,6 +15,17 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +enum TdxRamType { +TDX_RAM_UNACCEPTED , +TDX_RAM_ADDED , +}; + +typedef struct TdxRamEntry { +uint64_t address; +uint64_t length; +enum TdxRamType type; +}TdxRamEntry ; + typedef struct TdxGuest { ConfidentialGuestSupport parent_obj;
@@ -28,6 +39,9 @@ typedef struct TdxGuest {MemoryRegion *tdvf_mr;TdxFirmware tdvf; + +uint32_t nr_ram_entries; +TdxRamEntry *ram_entries; } TdxGuest;#ifdef CONFIG_TDX
Signature · 1 line
2.34.1
PATCH v5 43/65 headers: Add definitions from UEFI spec for volumes, resources, etc...
In reply to Xiaoyao LiAdd UEFI definitions for literals, enums, structs, GUIDs, etc... that will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed to the Trusted Domain Virtual Firmware (TDVF).
All values come from the UEFI specification [1], PI spec [2] and TDVF design guide[3].
[1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
[3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
--- include/standard-headers/uefi/uefi.h | 198 +++++++++++++++++++++++++++ 1 file changed, 198 insertions(+) create mode 100644 include/standard-headers/uefi/uefi.h
include/standard-headers/uefi/uefi.h new file +198 -0
@@ -0,0 +1,198 @@ +/* +* Copyright (C) 2020 Intel Corporation +* +* Author: Isaku Yamahata <isaku.yamahata at gmail.com> +* <isaku.yamahata at intel.com> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. + +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. + +* You should have received a copy of the GNU General Public License along +* with this program; if not, see <http://www.gnu.org/licenses/>. +* +*/ + +#ifndef HW_I386_UEFI_H +#define HW_I386_UEFI_H + +/***************************************************************************/ +/* +* basic EFI definitions +* supplemented with UEFI Specification Version 2.8 (Errata A) +* released February 2020 +*/ +/* UEFI integer is little endian */ + +typedef struct { +uint32_t Data1; +uint16_t Data2; +uint16_t Data3; +uint8_t Data4[8 ]; +}EFI_GUID ; + +typedef enum { + EfiReservedMemoryType, + EfiLoaderCode, + EfiLoaderData, + EfiBootServicesCode, + EfiBootServicesData, + EfiRuntimeServicesCode, + EfiRuntimeServicesData, + EfiConventionalMemory, + EfiUnusableMemory, + EfiACPIReclaimMemory, + EfiACPIMemoryNVS, + EfiMemoryMappedIO, + EfiMemoryMappedIOPortSpace, + EfiPalCode, + EfiPersistentMemory, + EfiUnacceptedMemoryType, + EfiMaxMemoryType +}EFI_MEMORY_TYPE ; + +#define EFI_HOB_HANDOFF_TABLE_VERSION 0x0009 + +#define EFI_HOB_TYPE_HANDOFF 0x0001 +#define EFI_HOB_TYPE_MEMORY_ALLOCATION 0x0002 +#define EFI_HOB_TYPE_RESOURCE_DESCRIPTOR 0x0003 +#define EFI_HOB_TYPE_GUID_EXTENSION 0x0004 +#define EFI_HOB_TYPE_FV 0x0005 +#define EFI_HOB_TYPE_CPU 0x0006 +#define EFI_HOB_TYPE_MEMORY_POOL 0x0007 +#define EFI_HOB_TYPE_FV2 0x0009 +#define EFI_HOB_TYPE_LOAD_PEIM_UNUSED 0x000A +#define EFI_HOB_TYPE_UEFI_CAPSULE 0x000B +#define EFI_HOB_TYPE_FV3 0x000C +#define EFI_HOB_TYPE_UNUSED 0xFFFE +#define EFI_HOB_TYPE_END_OF_HOB_LIST 0xFFFF + +typedef struct { +uint16_t HobType; +uint16_t HobLength; +uint32_t Reserved; +}EFI_HOB_GENERIC_HEADER ; + +typedef uint64_t EFI_PHYSICAL_ADDRESS ; +typedef uint32_t EFI_BOOT_MODE ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +uint32_t Version; +EFI_BOOT_MODE BootMode; +EFI_PHYSICAL_ADDRESS EfiMemoryTop; +EFI_PHYSICAL_ADDRESS EfiMemoryBottom; +EFI_PHYSICAL_ADDRESS EfiFreeMemoryTop; +EFI_PHYSICAL_ADDRESS EfiFreeMemoryBottom; +EFI_PHYSICAL_ADDRESS EfiEndOfHobList; +}EFI_HOB_HANDOFF_INFO_TABLE ; + +#define EFI_RESOURCE_SYSTEM_MEMORY 0x00000000 +#define EFI_RESOURCE_MEMORY_MAPPED_IO 0x00000001 +#define EFI_RESOURCE_IO 0x00000002 +#define EFI_RESOURCE_FIRMWARE_DEVICE 0x00000003 +#define EFI_RESOURCE_MEMORY_MAPPED_IO_PORT 0x00000004 +#define EFI_RESOURCE_MEMORY_RESERVED 0x00000005 +#define EFI_RESOURCE_IO_RESERVED 0x00000006 +#define EFI_RESOURCE_MEMORY_UNACCEPTED 0x00000007 +#define EFI_RESOURCE_MAX_MEMORY_TYPE 0x00000008 + +#define EFI_RESOURCE_ATTRIBUTE_PRESENT 0x00000001 +#define EFI_RESOURCE_ATTRIBUTE_INITIALIZED 0x00000002 +#define EFI_RESOURCE_ATTRIBUTE_TESTED 0x00000004 +#define EFI_RESOURCE_ATTRIBUTE_SINGLE_BIT_ECC 0x00000008 +#define EFI_RESOURCE_ATTRIBUTE_MULTIPLE_BIT_ECC 0x00000010 +#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_1 0x00000020 +#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_2 0x00000040 +#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTED 0x00000080 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTED 0x00000100 +#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTED 0x00000200 +#define EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE 0x00000400 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_COMBINEABLE 0x00000800 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_THROUGH_CACHEABLE 0x00001000 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_BACK_CACHEABLE 0x00002000 +#define EFI_RESOURCE_ATTRIBUTE_16_BIT_IO 0x00004000 +#define EFI_RESOURCE_ATTRIBUTE_32_BIT_IO 0x00008000 +#define EFI_RESOURCE_ATTRIBUTE_64_BIT_IO 0x00010000 +#define EFI_RESOURCE_ATTRIBUTE_UNCACHED_EXPORTED 0x00020000 +#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTED 0x00040000 +#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTABLE 0x00080000 +#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTABLE 0x00100000 +#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTABLE 0x00200000 +#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTABLE 0x00400000 +#define EFI_RESOURCE_ATTRIBUTE_PERSISTENT 0x00800000 +#define EFI_RESOURCE_ATTRIBUTE_PERSISTABLE 0x01000000 +#define EFI_RESOURCE_ATTRIBUTE_MORE_RELIABLE 0x02000000 + +typedef uint32_t EFI_RESOURCE_TYPE ; +typedef uint32_t EFI_RESOURCE_ATTRIBUTE_TYPE ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_GUID Owner; +EFI_RESOURCE_TYPE ResourceType; +EFI_RESOURCE_ATTRIBUTE_TYPE ResourceAttribute; +EFI_PHYSICAL_ADDRESS PhysicalStart; +uint64_t ResourceLength; +}EFI_HOB_RESOURCE_DESCRIPTOR ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_GUID Name; + +/* guid specific data follows */ +}EFI_HOB_GUID_TYPE ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +}EFI_HOB_FIRMWARE_VOLUME ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +EFI_GUID FvName; +EFI_GUID FileName; +}EFI_HOB_FIRMWARE_VOLUME2 ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +uint32_t AuthenticationStatus; +bool ExtractedFv; +EFI_GUID FvName; +EFI_GUID FileName; +}EFI_HOB_FIRMWARE_VOLUME3 ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +uint8_t SizeOfMemorySpace; +uint8_t SizeOfIoSpace; +uint8_t Reserved[6 ]; +}EFI_HOB_CPU ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; +}EFI_HOB_MEMORY_POOL ; + +typedef struct { +EFI_HOB_GENERIC_HEADER Header; + +EFI_PHYSICAL_ADDRESS BaseAddress; +uint64_t Length; +}EFI_HOB_UEFI_CAPSULE ; + +#define EFI_HOB_OWNER_ZERO \ + ((EFI_GUID){ 0x00000000, 0x0000, 0x0000, \ + { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 } }) + +#endif
Signature · 1 line
2.34.1
PATCH v5 44/65 i386/tdx: Setup the TD HOB list
In reply to Xiaoyao LiThe TD HOB list is used to pass the information from VMM to TDVF. The TD HOB must include PHIT HOB and Resource Descriptor HOB. More details can be found in TDVF specification and PI specification.
Build the TD HOB in TDX's machine_init_done callback.
---
Changes in v1:
- drop the code of adding mmio resources since OVMF prepares all the MMIO hob itself.
--- hw/i386/meson.build | 2 +- hw/i386/tdvf-hob.c | 147 ++++++++++++++++++++++++++++++++++++++++++ hw/i386/tdvf-hob.h | 24 +++++++ target/i386/kvm/tdx.c | 16 +++++ 4 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 hw/i386/tdvf-hob.c create mode 100644 hw/i386/tdvf-hob.h
hw/i386/meson.build +1 -1
@@ -28,7 +28,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files( 'port92.c')) i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'), if_false: files('pc_sysfw_ovmf-stubs.c')) -i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c')) +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c')) subdir('kvm') subdir('xen')
hw/i386/tdvf-hob.c new file +147 -0
147 lines not on this page · raw message
hw/i386/tdvf-hob.h new file +24 -0
@@ -0,0 +1,24 @@ +#ifndef HW_I386_TD_HOB_H +#define HW_I386_TD_HOB_H + +#include "hw/i386/tdvf.h" +#include "target/i386/kvm/tdx.h" + +void tdvf_hob_create (TdxGuest *tdx,TdxFirmwareEntry *td_hob); + +#define EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE \ + (EFI_RESOURCE_ATTRIBUTE_PRESENT | \ + EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \ + EFI_RESOURCE_ATTRIBUTE_TESTED) + +#define EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED \ + (EFI_RESOURCE_ATTRIBUTE_PRESENT | \ + EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \ + EFI_RESOURCE_ATTRIBUTE_TESTED) + +#define EFI_RESOURCE_ATTRIBUTE_TDVF_MMIO \ + (EFI_RESOURCE_ATTRIBUTE_PRESENT | \ + EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \ + EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE) + +#endif
target/i386/kvm/tdx.c +16 -0
@@ -25,6 +25,7 @@#include "hw/i386/e820_memory_layout.h" #include "hw/i386/x86.h" #include "hw/i386/tdvf.h" +#include "hw/i386/tdvf-hob.h" #include "kvm_i386.h" #include "tdx.h" #include "../cpu-internal.h"
@@ -473,6 +474,19 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr) tdx_guest->tdvf_mr = tdvf_mr; } +static TdxFirmwareEntry *tdx_get_hob_entry (TdxGuest *tdx) +{ +TdxFirmwareEntry *entry; + +for_each_tdx_fw_entry (&tdx->tdvf, entry) { +if (entry->type ==TDVF_SECTION_TYPE_TD_HOB ) { +return entry; + } + } +error_report ("TDVF metadata doesn't specify TD_HOB location." ); +exit (1 ); +} +static void tdx_add_ram_entry (uint64_t address,uint64_t length,enum TdxRamType type) {
@@ -604,6 +618,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)qsort (tdx_guest->ram_entries, tdx_guest->nr_ram_entries,sizeof (TdxRamEntry), &tdx_ram_entry_compare); + +tdvf_hob_create (tdx_guest,tdx_get_hob_entry (tdx_guest)); }static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
PATCH v5 45/65 i386/tdx: Populate TDVF private memory via KVM_MEMORY_MAPPING
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
TDVF firmware (CODE and VARS) needs to be copied to TD's private memory, as well as TD HOB and TEMP memory.
If the TDVF section has TDVF_SECTION_ATTRIBUTES_MR_EXTEND set in the flag, calling KVM_TDX_EXTEND_MEMORY to extend the measurement.
After populating the TDVF memory, the original image located in shared ramblock can be discarded.
---
Changes in v1:
- rename variable @metadata to @flags
--- target/i386/kvm/tdx.c | 47 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
target/i386/kvm/tdx.c +47 -0
@@ -595,6 +595,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;TdxFirmwareEntry *entry; +RAMBlock *ram_block; +int r;tdx_init_ram_entries ();
@@ -620,6 +622,51 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused) sizeof(TdxRamEntry), &tdx_ram_entry_compare);tdvf_hob_create (tdx_guest,tdx_get_hob_entry (tdx_guest)); + +for_each_tdx_fw_entry (tdvf, entry) { +struct kvm_memory_mapping mapping = { + .base_gfn = entry->address >>12 , + .nr_pages = entry->size >>12 , + .source = (__u64 )entry->mem_ptr, + }; + +do { + r =kvm_vcpu_ioctl (first_cpu,KVM_MEMORY_MAPPING , &mapping); + }while (r == -EAGAIN ); + +if (r <0 ) { +error_report ("KVM_MEMORY_MAPPING failed %s" ,strerror (-r)); +exit (1 ); + } + +if (entry->attributes &TDVF_SECTION_ATTRIBUTES_MR_EXTEND ) { + mapping = (struct kvm_memory_mapping ) { + .base_gfn = entry->address >>12 , + .nr_pages = entry->size >>12 , + }; + +do { + r =tdx_vm_ioctl (KVM_TDX_EXTEND_MEMORY ,0 , &mapping); + }while (r == -EAGAIN ); +if (r <0 ) { +error_report ("KVM_TDX_EXTEND_MEMORY failed %s" ,strerror (-r)); +exit (1 ); + } + } + +if (entry->type ==TDVF_SECTION_TYPE_TD_HOB || + entry->type ==TDVF_SECTION_TYPE_TEMP_MEM ) { +qemu_ram_munmap (-1 , entry->mem_ptr, entry->size); + entry->mem_ptr =NULL ; + } + } + +/* +* TDVF image has been copied into private region above via +* KVM_MEMORY_MAPPING. It becomes useless. +*/ + ram_block = tdx_guest->tdvf_mr->ram_block; +ram_block_discard_range (ram_block,0 , ram_block->max_length); }static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
PATCH v5 46/65 i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu
In reply to Xiaoyao LiTDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.
KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for each vcpu after HOB list is created.
--- target/i386/kvm/tdx.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
target/i386/kvm/tdx.c +18 -0
@@ -591,6 +591,22 @@ static void tdx_init_ram_entries(void) tdx_guest->nr_ram_entries = j; } +static void tdx_post_init_vcpus (void ) +{ +TdxFirmwareEntry *hob; +CPUState *cpu; +int r; + + hob =tdx_get_hob_entry (tdx_guest); +CPU_FOREACH (cpu) { + r =tdx_vcpu_ioctl (cpu,KVM_TDX_INIT_VCPU ,0 , (void *)hob->address); +if (r <0 ) { +error_report ("KVM_TDX_INIT_VCPU failed %s" ,strerror (-r)); +exit (1 ); + } + } +} +static void tdx_finalize_vm (Notifier *notifier,void *unused) {TdxFirmware *tdvf = &tdx_guest->tdvf;
@@ -623,6 +639,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)tdvf_hob_create (tdx_guest,tdx_get_hob_entry (tdx_guest)); +tdx_post_init_vcpus (); +for_each_tdx_fw_entry (tdvf, entry) {struct kvm_memory_mapping mapping = { .base_gfn = entry->address >>12 ,
Signature · 1 line
2.34.1
PATCH v5 47/65 i386/tdx: Finalize TDX VM
In reply to Xiaoyao LiInvoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make the TD vCPUs runnable once machine initialization is complete.
--- target/i386/kvm/tdx.c | 7 +++++++ 1 file changed, 7 insertions(+)
target/i386/kvm/tdx.c +7 -0
@@ -685,6 +685,13 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)*/ ram_block = tdx_guest->tdvf_mr->ram_block;ram_block_discard_range (ram_block,0 , ram_block->max_length); + + r =tdx_vm_ioctl (KVM_TDX_FINALIZE_VM ,0 ,NULL ); +if (r <0 ) { +error_report ("KVM_TDX_FINALIZE_VM failed %s" ,strerror (-r)); +exit (0 ); + } + tdx_guest->parent_obj.ready = true; }static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
PATCH v5 48/65 i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt>
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
For SetupEventNotifyInterrupt, record interrupt vector and the apic id of the vcpu that received this TDVMCALL.
Later it can inject interrupt with given vector to the specific vcpu that received SetupEventNotifyInterrupt.
--- target/i386/kvm/kvm.c | 8 ++++++ target/i386/kvm/tdx-stub.c | 5 ++++ target/i386/kvm/tdx.c | 53 ++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 14 ++++++++++ 4 files changed, 80 insertions(+)
target/i386/kvm/kvm.c +8 -0
@@ -5413,6 +5413,14 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) ret =kvm_xen_handle_exit (cpu, &run->xen);break ;#endif +case KVM_EXIT_TDX : +if (!is_tdx_vm ()) { +error_report ("KVM: get KVM_EXIT_TDX for a non-TDX VM." ); + ret =-1 ; +break ; + } + ret =tdx_handle_exit (cpu, &run->tdx); +break ;default :fprintf (stderr,"KVM: unknown exit reason %d\n" , run->exit_reason); ret =-1 ;
target/i386/kvm/tdx-stub.c +5 -0
@@ -11,3 +11,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size) {return -EINVAL ; } + +int tdx_handle_exit (X86CPU *cpu,struct kvm_tdx_exit *tdx_exit) +{ +return -EINVAL ; +}
target/i386/kvm/tdx.c +53 -0
65 lines not on this page · raw message
target/i386/kvm/tdx.h +14 -0
@@ -7,6 +7,7 @@#include "exec/confidential-guest-support.h" #include "hw/i386/tdvf.h" +#include "sysemu/kvm.h" #define TYPE_TDX_GUEST "tdx-guest"#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -15,6 +16,14 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL + +#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL +#define TDG_VP_VMCALL_RETRY 0x0000000000000001ULL +#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDG_VP_VMCALL_GPA_INUSE 0x8000000000000001ULL +#define TDG_VP_VMCALL_ALIGN_ERROR 0x8000000000000002ULL +enum TdxRamType {TDX_RAM_UNACCEPTED ,TDX_RAM_ADDED ,
@@ -42,6 +51,10 @@ typedef struct TdxGuest {uint32_t nr_ram_entries;TdxRamEntry *ram_entries; + +/* runtime state */ +uint32_t event_notify_vector; +uint32_t event_notify_apicid; } TdxGuest;#ifdef CONFIG_TDX
@@ -55,5 +68,6 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,int tdx_pre_create_vcpu (CPUState *cpu,Error **errp);void tdx_set_tdvf_region (MemoryRegion *tdvf_mr);int tdx_parse_tdvf (void *flash_ptr,int size); +int tdx_handle_exit (X86CPU *cpu,struct kvm_tdx_exit *tdx_exit);#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
PATCH v5 49/65 i386/tdx: handle TDG.VP.VMCALL<GetQuote>
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \
-object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
-machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
---
Changes in v5:
- add more decription of quote-generation-socket property;
Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";
Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;
--- qapi/qom.json | 8 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 495 insertions(+), 2 deletions(-) create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.h
qapi/qom.json +7 -1
@@ -917,13 +917,19 @@ # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is # used when absent). # +# @quote-generation-socket: socket address for Quote Generation +# Service (QGS). QGS is a daemon running on the host. User in +# TD guest cannot get TD quoting for attestation if QGS is not +# provided. So admin should always provide it. +# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } } ## # @ThreadContextProperties:
target/i386/kvm/meson.build +1 -1
@@ -7,7 +7,7 @@ i386_kvm_ss.add(files( i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c')) -i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c')) +i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c', 'tdx-quote-generator.c'), if_false: files('tdx-stub.c')) i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
target/i386/kvm/tdx-quote-generator.c new file +170 -0
170 lines not on this page · raw message
target/i386/kvm/tdx-quote-generator.h new file +95 -0
95 lines not on this page · raw message
target/i386/kvm/tdx.c +216 -0
258 lines not on this page · raw message
target/i386/kvm/tdx.h +6 -0
@@ -9,6 +9,8 @@#include "hw/i386/tdvf.h" #include "sysemu/kvm.h" +#include "tdx-quote-generator.h" +#define TYPE_TDX_GUEST "tdx-guest"#define TDX_GUEST (obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -16,6 +18,7 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
@@ -55,6 +58,9 @@ typedef struct TdxGuest {/* runtime state */ uint32_t event_notify_vector;uint32_t event_notify_apicid; + +/* GetQuote */ +TdxQuoteGenerator *quote_generator; } TdxGuest;#ifdef CONFIG_TDX
Signature · 1 line
2.34.1
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 28 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>
[...]
qapi/qom.json line 923
Quoted 12 lines
qapi/qom.json +4
-0@@ -917,13 +917,19 @@ # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is # used when absent). # +# @quote-generation-socket: socket address for Quote Generation +# Service (QGS). QGS is a daemon running on the host. User in +# TD guest cannot get TD quoting for attestation if QGS is not +# provided. So admin should always provide it.
This makes me wonder why it's optional. Can you describe a use case for *not* specifying @quote-generation-socket?
Quoted 13 lines
qapi/qom.json +3
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } } ## # @ThreadContextProperties:
[...]
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 33 lines · Markus Armbruster
Quoted 28 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 15 lines
qapi/qom.json line 923
Quoted 12 lines
qapi/qom.json +4
-0@@ -917,13 +917,19 @@ # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is # used when absent). # +# @quote-generation-socket: socket address for Quote Generation +# Service (QGS). QGS is a daemon running on the host. User in +# TD guest cannot get TD quoting for attestation if QGS is not +# provided. So admin should always provide it.This makes me wonder why it's optional. Can you describe a use case for *not* specifying @quote-generation-socket?
Maybe at last when all the TDX support lands on all the components, attestation will become a must for a TD guest to be usable.
However, at least for today, booting and running a TD guest don't require attestation. So not provide it, doesn't affect anything excepting cannot get a Quote.
Quoted 16 lines
Quoted 13 lines
qapi/qom.json +3
-1+# # Since: 9.0 ## { 'struct': 'TdxGuestProperties', 'data': { '*sept-ve-disable': 'bool', '*mrconfigid': 'str', '*mrowner': 'str', - '*mrownerconfig': 'str' } } + '*mrownerconfig': 'str', + '*quote-generation-socket': 'SocketAddress' } } ## # @ThreadContextProperties:[...]
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 50 lines · Xiaoyao Li
Quoted 33 lines · Markus Armbruster
Quoted 28 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 15 lines
qapi/qom.json line 923
Quoted 12 lines
qapi/qom.json +4
-0@@ -917,13 +917,19 @@ # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is # used when absent). # +# @quote-generation-socket: socket address for Quote Generation +# Service (QGS). QGS is a daemon running on the host. User in +# TD guest cannot get TD quoting for attestation if QGS is not +# provided. So admin should always provide it.This makes me wonder why it's optional. Can you describe a use case for *not* specifying @quote-generation-socket?
Quoted 3 lines
Maybe at last when all the TDX support lands on all the components, attestation will become a must for a TD guest to be usable.
However, at least for today, booting and running a TD guest don't require attestation. So not provide it, doesn't affect anything excepting cannot get a Quote.
Maybe
# @quote-generation-socket: Socket address for Quote Generation
# Service (QGS). QGS is a daemon running on the host. Without # it, the guest will not be able to get a TD quote for # attestation.
[...]
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 54 lines · Markus Armbruster
Quoted 52 lines · Xiaoyao Li
Quoted 33 lines · Markus Armbruster
Quoted 28 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>[...]
Quoted 15 lines
qapi/qom.json line 923
Quoted 12 lines
qapi/qom.json +4
-0@@ -917,13 +917,19 @@ # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is # used when absent). # +# @quote-generation-socket: socket address for Quote Generation +# Service (QGS). QGS is a daemon running on the host. User in +# TD guest cannot get TD quoting for attestation if QGS is not +# provided. So admin should always provide it.This makes me wonder why it's optional. Can you describe a use case for *not* specifying @quote-generation-socket?
Maybe at last when all the TDX support lands on all the components, attestation will become a must for a TD guest to be usable.
Quoted 8 lines
Quoted 1 line
However, at least for today, booting and running a TD guest don't require attestation. So not provide it, doesn't affect anything excepting cannot get a Quote.
Maybe
# @quote-generation-socket: Socket address for Quote Generation
# Service (QGS). QGS is a daemon running on the host. Without # it, the guest will not be able to get a TD quote for # attestation.
Thanks! will update to it.
Quoted 2 lines
[...]
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
In reply to Xiaoyao LiQuoted 5 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
Quoted 8 lines
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
Can you illustrate this with 'unix' sockets, not 'vsock'.
It makes no conceptual sense to be using vsock for two processes on the host to be using vsock to talk to each other. vsock is only needed for the guest to talk to the host.
Quoted 27 lines
-machine confidential-guest-support=tdx0
Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v5:
- add more decription of quote-generation-socket property;Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;
Quoted 10 lines
--- qapi/qom.json | 8 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 495 insertions(+), 2 deletions(-) create mode 100644 target/i386/kvm/tdx-quote-generator.c create mode 100644 target/i386/kvm/tdx-quote-generator.h
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 11 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 5 lines
Quoted 3 lines
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \Can you illustrate this with 'unix' sockets, not 'vsock'.
Are you suggesting only updating the commit message to an example of unix socket? Or you want the code to test with some unix socket QGS?
(It seems the QGS I got for testing, only supports vsock socket. Because at the time when it got developed, it was supposed to communicate with drivers inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to internal folks to see if any plan to support unix socket.)
Quoted 46 lines
It makes no conceptual sense to be using vsock for two processes on the host to be using vsock to talk to each other. vsock is only needed for the guest to talk to the host.
Quoted 35 lines
-machine confidential-guest-support=tdx0
Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>---
Changes in v5:
- add more decription of quote-generation-socket property;Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to SocketAddress;
- squash next patch into this one;--- qapi/qom.json | 8 +- target/i386/kvm/meson.build | 2 +- target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++ target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++ target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 6 + 6 files changed, 495 insertions(+), 2 deletions(-)Quoted 2 lines
create mode 100644
target/i386/kvm/tdx-quote-generator.ccreate mode 100644target/i386/kvm/tdx-quote-generator.hWith regards,
Daniel
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 18 lines · Xiaoyao Li
Quoted 11 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 5 lines
Quoted 3 lines
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \Can you illustrate this with 'unix' sockets, not 'vsock'.
Quoted 7 lines
Are you suggesting only updating the commit message to an example of unix socket? Or you want the code to test with some unix socket QGS?
(It seems the QGS I got for testing, only supports vsock socket. Because at the time when it got developed, it was supposed to communicate with drivers inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to internal folks to see if any plan to support unix socket.)
You can use small utility to proxy sockets for testing purpose. The famous one is socat or nmap ncat. They support vsock. At least their latest version does.
QGS <-vsock-> socat <-unix domain-> qemu
Signature · 1 line
Isaku Yamahata <[email protected]>
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
In reply to Xiaoyao LiQuoted 18 lines · Xiaoyao Li
Quoted 11 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 5 lines
Quoted 3 lines
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \Can you illustrate this with 'unix' sockets, not 'vsock'.
Quoted 7 lines
Are you suggesting only updating the commit message to an example of unix socket? Or you want the code to test with some unix socket QGS?
(It seems the QGS I got for testing, only supports vsock socket. Because at the time when it got developed, it was supposed to communicate with drivers inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to internal folks to see if any plan to support unix socket.)
The QGS provided as part of DCAP supports running with both UNIX sockets and VSOCK, and I would expect QEMU to be made to work with this, since its is Intel's OSS reference impl.
Exposing QGS to the guest when we only intend for it to be used by the host QEMU is needlessly expanding the attack surface.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 22 lines · Daniel P. Berrangé
Quoted 21 lines · Xiaoyao Li
Quoted 11 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
Quoted 5 lines
Quoted 3 lines
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \Can you illustrate this with 'unix' sockets, not 'vsock'.
Are you suggesting only updating the commit message to an example of unix socket? Or you want the code to test with some unix socket QGS?
Quoted 8 lines
Quoted 4 lines
(It seems the QGS I got for testing, only supports vsock socket. Because at the time when it got developed, it was supposed to communicate with drivers inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to internal folks to see if any plan to support unix socket.)
The QGS provided as part of DCAP supports running with both UNIX sockets and VSOCK, and I would expect QEMU to be made to work with this, since its is Intel's OSS reference impl.
After synced with internal folks, yes, the QGS I used does support unix socket. I tested it and it worked.
-object '{"qom-type":"tdx-guest","id":"tdx","quote-generation-socket":{"type":"unix", "path":"/var/run/tdx-qgs/qgs.socket"}}'
Quoted 6 lines
Exposing QGS to the guest when we only intend for it to be used by the host QEMU is needlessly expanding the attack surface.
With regards,
Daniel
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
In reply to Xiaoyao LiQuoted 125 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>target/i386/kvm/tdx.c +90
-0+static int tdx_handle_get_quote (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) +{ +struct tdx_generate_quote_task *task; +struct tdx_get_quote_header hdr; +hwaddr buf_gpa = vmcall->in_r12; +uint64_t buf_len = vmcall->in_r13; + +QEMU_BUILD_BUG_ON (sizeof (struct tdx_get_quote_header ) !=TDX_GET_QUOTE_HDR_SIZE ); + + vmcall->status_code =TDG_VP_VMCALL_INVALID_OPERAND ; + +if (buf_len ==0 ) { +return 0 ; + } + +/* GPA must be shared. */ +if (!(buf_gpa &tdx_shared_bit (cpu))) { +return 0 ; + } + buf_gpa &= ~tdx_shared_bit (cpu); + +if (!QEMU_IS_ALIGNED (buf_gpa,4096 ) || !QEMU_IS_ALIGNED (buf_len,4096 )) { + vmcall->status_code =TDG_VP_VMCALL_ALIGN_ERROR ; +return 0 ; + } + +if (address_space_read (&address_space_memory, buf_gpa,MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: get-quote: failed to read GetQuote header.\n" ); +return -1 ; + } + +if (le64_to_cpu (hdr.structure_version) !=TDX_GET_QUOTE_STRUCTURE_VERSION ) { +return 0 ; + } + +/* +* Paranoid: Guest should clear error_code and out_len to avoid information +* leak. Enforce it. The initial value of them doesn't matter for qemu to +* process the request. +*/ +if (le64_to_cpu (hdr.error_code) !=TDX_VP_GET_QUOTE_SUCCESS || +le32_to_cpu (hdr.out_len) !=0 ) { +return 0 ; + } + +/* Only safe-guard check to avoid too large buffer size. */ +if (buf_len >TDX_GET_QUOTE_MAX_BUF_LEN || +le32_to_cpu (hdr.in_len) > buf_len -TDX_GET_QUOTE_HDR_SIZE ) { +return 0 ; + } + + vmcall->status_code =TDG_VP_VMCALL_SUCCESS ; +if (!tdx_guest->quote_generator) { + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_QGS_UNAVAILABLE ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } +return 0 ; + } + +qemu_mutex_lock (&tdx_guest->quote_generator->lock); +if (tdx_guest->quote_generator->num >=TDX_MAX_GET_QUOTE_REQUEST ) { +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + vmcall->status_code =TDG_VP_VMCALL_RETRY ; +return 0 ; + } + tdx_guest->quote_generator->num++; +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + +/* Mark the buffer in-flight. */ + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_IN_FLIGHT ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } + + task =g_malloc (sizeof (*task)); + task->buf_gpa = buf_gpa; + task->payload_gpa = buf_gpa +TDX_GET_QUOTE_HDR_SIZE ; + task->payload_len = buf_len -TDX_GET_QUOTE_HDR_SIZE ; + task->hdr = hdr; + task->quote_gen = tdx_guest->quote_generator; + task->completion = tdx_get_quote_completion; +
target/i386/kvm/tdx.c
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ task->send_data_size =le32_to_cpu (hdr.in_len); + task->send_data =g_malloc (task->send_data_size); + task->send_data_sent =0 ; + +if (address_space_read (&address_space_memory, task->payload_gpa, +MEMTXATTRS_UNSPECIFIED , task->send_data, + task->send_data_size) !=MEMTX_OK ) { +g_free (task->send_data); +return -1 ; + }
In this method we've received "struct tdx_get_quote_header" from the guest OS, and the 'hdr.in_len' field in that struct tells us the payload to read from guest memory. This payload is treated as opaque by QEMU and sent over the UNIX socket directly to QGS with no validation of the payload.
The payload is supposed to be a raw TDX report, that QGS will turn into a quote.
Nothing guarantees that the guest OS has actually given QEMU a payload that represents a TDX report.
The only validation done in this patch is to check the 'hdr.in_len' was not ridiculously huge:
#define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024)
#define TDX_GET_QUOTE_HDR_SIZE 24
...
/* Only safe-guard check to avoid too large buffer size. */
if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
return 0;
}IOW, hdr.in_len can be any value between 0 and 131048, and the payload data read can contain arbitrary bytes.
Over in the QGS code, QGS historically had a socket protocol taking various messages from the libtdxattest library which were defined in this:
typedef enum _qgs_msg_type_t {
GET_QUOTE_REQ = 0,
GET_QUOTE_RESP = 1,
GET_COLLATERAL_REQ = 2,
GET_COLLATERAL_RESP = 3,
GET_PLATFORM_INFO_REQ = 4,
GET_PLATFORM_INFO_RESP = 5,
QGS_MSG_TYPE_MAX
} qgs_msg_type_t;
typedef struct _qgs_msg_header_t {
uint16_t major_version;
uint16_t minor_version;
uint32_t type;
uint32_t size; // size of the whole message, include this header, in byte
uint32_t error_code; // used in response only
} qgs_msg_header_t;such messages are processed by the 'get_resp' method in QGS:
The 1.21 release of DCAP introduced a new "raw" mode in QGS which just receives the raw 1024 byte packet from the client which is supposed to be a raw TDX report. This is what this QEMU patch is relying on IIUC.
The QGS daemon decides whether a client is speaking the formal protocol, or "raw" mode, by trying to interpret the incoming data as a 'qgs_msg_header_t' struct. If the header size looks wrong & it has exactly 1024 bytes, then QGS assumes it has got a raw TDX report:
This all works if the data QEMU gets from the guest is indeed a 1024 byte raw TDX report, but what happens if we face a malicious guest ?
AFAICT, the guest OS is able to send a "qgs_msg_header_t" packet to QEMU, which QEMU blindly passes on to QGS. This allows the guest OS to invoke any of the three QGS commands - GET_QUOTE_REQ, GET_COLLATERAL_REQ, or GET_PLATFORM_INFO_REQ. Fortunately I think those three messages are all safe to invoke, but none the less, this should not be permitted, as it leaves a wide open door for possible future exploits.
As mentioned before, I don't know why this raw mode was invented for QGS, when QEMU itself could just take the guest report and pack it into the 'GET_QUOTE_REQ' message format and send it to QGS. This prevents the guest OS from being able to exploit QEMU to invoke arbirtary QGS messages.
IMHO, QEMU should be made to pack & unpack the TDX report from the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and this "raw" mode should be removed to QGS as it is inherantly dangerous to have this magic protocol overloading.
Below is a patch on top of this one that illustrates how QEMU could use the GET_QUOTE_REQ / GET_QUOTE_RESP messages and avoid the "raw" mode of QGS.
target/i386/kvm/tdx.c
Quoted 8 lines
target/i386/kvm/tdx.c +8
-0+ + task->receive_buf =g_malloc0 (task->payload_len); + task->receive_buf_received =0 ; + +tdx_generate_quote (task); + +return 0 ; +}
target/i386/kvm/tdx-quote-generator.c +156 -1
@@ -24,6 +24,61 @@ OBJECT_DEFINE_TYPE(TdxQuoteGenerator, tdx_quote_generator, TDX_QUOTE_GENERATOR, OBJECT) +const uint32_t QGS_MSG_LIB_MAJOR_VER =1 ; +const uint32_t QGS_MSG_LIB_MINOR_VER =1 ; + +typedef enum _qgs_msg_type_t { +GET_QUOTE_REQ =0 , +GET_QUOTE_RESP =1 , +GET_COLLATERAL_REQ =2 , +GET_COLLATERAL_RESP =3 , +GET_PLATFORM_INFO_REQ =4 , +GET_PLATFORM_INFO_RESP =5 , +QGS_MSG_TYPE_MAX +}qgs_msg_type_t ; + +typedef struct _qgs_msg_header_t { +uint16_t major_version; +uint16_t minor_version; +uint32_t type; +uint32_t size;// size of the whole message, include this header, in byte +uint32_t error_code;// used in response only +}qgs_msg_header_t ; + +typedef struct _qgs_msg_get_quote_req_t { +qgs_msg_header_t header;// header.type = GET_QUOTE_REQ +uint32_t report_size;// cannot be 0 +uint32_t id_list_size;// length of id_list, in byte, can be 0 +}qgs_msg_get_quote_req_t ; + +typedef struct _qgs_msg_get_quote_resp_s { +qgs_msg_header_t header;// header.type = GET_QUOTE_RESP +uint32_t selected_id_size;// can be 0 in case only one id is sent in request +uint32_t quote_size;// length of quote_data, in byte +uint8_t id_quote[];// selected id followed by quote +}qgs_msg_get_quote_resp_t ; + +const unsigned HEADER_SIZE =4 ; + +static uint32_t decode_header (const char *buf,size_t len) { +if (len <HEADER_SIZE ) { +return 0 ; + } +uint32_t msg_size =0 ; +for (uint32_t i =0 ; i <HEADER_SIZE ; ++i) { + msg_size = msg_size *256 + (buf[i] &0xFF ); + } +return msg_size; +} + +static void encode_header (char *buf,size_t len,uint32_t size) { +assert (len >=HEADER_SIZE ); + buf[0 ] = ((size >>24 ) &0xFF ); + buf[1 ] = ((size >>16 ) &0xFF ); + buf[2 ] = ((size >>8 ) &0xFF ); + buf[3 ] = (size &0xFF ); +} +static void tdx_quote_generator_finalize (Object *obj) { }
@@ -70,9 +125,86 @@ goto end; } } + +if (ret ==0 ) { +error_report ("End of file before reply received" ); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } task->receive_buf_received += ret; -if (ret ==0 || task->receive_buf_received == task->payload_len) { +if (task->receive_buf_received >=HEADER_SIZE ) { +uint32_t len =decode_header (task->receive_buf, + task->receive_buf_received); +if (len ==0 || + len > (task->payload_len -HEADER_SIZE )) { +error_report ("Message len %u must be non-zero & less than %zu" , + len, (task->payload_len -HEADER_SIZE )); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + +/* Now we know the size, shrink to fit */ + task->payload_len =HEADER_SIZE + len; + task->receive_buf =g_renew (char, + task->receive_buf, + task->payload_len); + } + +if (task->receive_buf_received >= (sizeof (qgs_msg_header_t) +HEADER_SIZE )) { +qgs_msg_header_t *hdr = (qgs_msg_header_t *)(task->receive_buf +HEADER_SIZE ); +if (hdr->major_version !=QGS_MSG_LIB_MAJOR_VER || + hdr->minor_version !=QGS_MSG_LIB_MINOR_VER ) { +error_report ("Invalid QGS message header version %d.%d\n" , + hdr->major_version, + hdr->minor_version); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } +if (hdr->type !=GET_QUOTE_RESP ) { +error_report ("Invalid QGS message type %d\n" , + hdr->type); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } +if (hdr->size > (task->payload_len -HEADER_SIZE )) { +error_report ("QGS message size %d exceeds payload capacity %zu" , + hdr->size, task->payload_len); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } +if (hdr->error_code !=0 ) { +error_report ("QGS message error code %d" , + hdr->error_code); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + } +if (task->receive_buf_received >= (sizeof (qgs_msg_get_quote_resp_t) +HEADER_SIZE )) { +qgs_msg_get_quote_resp_t *msg = (qgs_msg_get_quote_resp_t *)(task->receive_buf +HEADER_SIZE ); +if (msg->selected_id_size !=0 ) { +error_report ("QGS message selected ID was %d not 0" , + msg->selected_id_size); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + +if ((task->payload_len -HEADER_SIZE -sizeof (qgs_msg_get_quote_resp_t)) != + msg->quote_size) { +error_report ("QGS quote size %d should be %zu" , + msg->quote_size, + (task->payload_len -sizeof (qgs_msg_get_quote_resp_t))); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + } + +if (task->receive_buf_received == task->payload_len) { +size_t strip =HEADER_SIZE +sizeof (qgs_msg_get_quote_resp_t); +memmove (task->receive_buf, + task->receive_buf + strip, + task->receive_buf_received - strip); + task->receive_buf_received -= strip; task->status_code =TDX_VP_GET_QUOTE_SUCCESS ; goto end; }
@@ -158,6 +290,29 @@ {struct TdxQuoteGenerator *quote_gen = task->quote_gen;QIOChannelSocket *sioc; +qgs_msg_get_quote_req_t msg; + +/* Prepare a QGS message prelude */ + msg.header.major_version =QGS_MSG_LIB_MAJOR_VER ; + msg.header.minor_version =QGS_MSG_LIB_MINOR_VER ; + msg.header.type =GET_QUOTE_REQ ; + msg.header.size =sizeof (msg) + task->send_data_size; + msg.header.error_code =0 ; + msg.report_size = task->send_data_size; + msg.id_list_size =0 ; + +/* Make room to add the QGS message prelude */ + task->send_data =g_renew (char, + task->send_data, + task->send_data_size +sizeof (msg) +HEADER_SIZE ); +memmove (task->send_data +sizeof (msg) +HEADER_SIZE , + task->send_data, + task->send_data_size); +memcpy (task->send_data +HEADER_SIZE , + &msg, +sizeof (msg)); +encode_header (task->send_data,HEADER_SIZE , task->send_data_size +sizeof (msg)); + task->send_data_size +=sizeof (msg) +HEADER_SIZE ; sioc =qio_channel_socket_new (); task->sioc = sioc;
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Sorry for the late response.
KVM part of TDX attestation support is submitting again. QEMU part will follow and we need to settle dowm this topic before QEMU patches submission.
Quoted 222 lines · Daniel P. Berrangé
Quoted 125 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>target/i386/kvm/tdx.c +90
-0+static int tdx_handle_get_quote (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) +{ +struct tdx_generate_quote_task *task; +struct tdx_get_quote_header hdr; +hwaddr buf_gpa = vmcall->in_r12; +uint64_t buf_len = vmcall->in_r13; + +QEMU_BUILD_BUG_ON (sizeof (struct tdx_get_quote_header ) !=TDX_GET_QUOTE_HDR_SIZE ); + + vmcall->status_code =TDG_VP_VMCALL_INVALID_OPERAND ; + +if (buf_len ==0 ) { +return 0 ; + } + +/* GPA must be shared. */ +if (!(buf_gpa &tdx_shared_bit (cpu))) { +return 0 ; + } + buf_gpa &= ~tdx_shared_bit (cpu); + +if (!QEMU_IS_ALIGNED (buf_gpa,4096 ) || !QEMU_IS_ALIGNED (buf_len,4096 )) { + vmcall->status_code =TDG_VP_VMCALL_ALIGN_ERROR ; +return 0 ; + } + +if (address_space_read (&address_space_memory, buf_gpa,MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: get-quote: failed to read GetQuote header.\n" ); +return -1 ; + } + +if (le64_to_cpu (hdr.structure_version) !=TDX_GET_QUOTE_STRUCTURE_VERSION ) { +return 0 ; + } + +/* +* Paranoid: Guest should clear error_code and out_len to avoid information +* leak. Enforce it. The initial value of them doesn't matter for qemu to +* process the request. +*/ +if (le64_to_cpu (hdr.error_code) !=TDX_VP_GET_QUOTE_SUCCESS || +le32_to_cpu (hdr.out_len) !=0 ) { +return 0 ; + } + +/* Only safe-guard check to avoid too large buffer size. */ +if (buf_len >TDX_GET_QUOTE_MAX_BUF_LEN || +le32_to_cpu (hdr.in_len) > buf_len -TDX_GET_QUOTE_HDR_SIZE ) { +return 0 ; + } + + vmcall->status_code =TDG_VP_VMCALL_SUCCESS ; +if (!tdx_guest->quote_generator) { + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_QGS_UNAVAILABLE ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } +return 0 ; + } + +qemu_mutex_lock (&tdx_guest->quote_generator->lock); +if (tdx_guest->quote_generator->num >=TDX_MAX_GET_QUOTE_REQUEST ) { +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + vmcall->status_code =TDG_VP_VMCALL_RETRY ; +return 0 ; + } + tdx_guest->quote_generator->num++; +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + +/* Mark the buffer in-flight. */ + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_IN_FLIGHT ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } + + task =g_malloc (sizeof (*task)); + task->buf_gpa = buf_gpa; + task->payload_gpa = buf_gpa +TDX_GET_QUOTE_HDR_SIZE ; + task->payload_len = buf_len -TDX_GET_QUOTE_HDR_SIZE ; + task->hdr = hdr; + task->quote_gen = tdx_guest->quote_generator; + task->completion = tdx_get_quote_completion; +target/i386/kvm/tdx.c
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ task->send_data_size =le32_to_cpu (hdr.in_len); + task->send_data =g_malloc (task->send_data_size); + task->send_data_sent =0 ; + +if (address_space_read (&address_space_memory, task->payload_gpa, +MEMTXATTRS_UNSPECIFIED , task->send_data, + task->send_data_size) !=MEMTX_OK ) { +g_free (task->send_data); +return -1 ; + }In this method we've received "struct
tdx_get_quote_header" from the guest OS, and the 'hdr.in_len' field in that struct tells us the payload to read from guest memory. This payload is treated as opaque by QEMU and sent over the UNIX socket directly to QGS with no validation of the payload.The payload is supposed to be a raw TDX report, that QGS will turn into a quote.
Nothing guarantees that the guest OS has actually given QEMU a payload that represents a TDX report.
The only validation done in this patch is to check the 'hdr.in_len' was not ridiculously huge:
#define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024) #define TDX_GET_QUOTE_HDR_SIZE 24 ... /* Only safe-guard check to avoid too large buffer size. */ if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN || le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) { return 0; }IOW, hdr.in_len can be any value between 0 and 131048, and the payload data read can contain arbitrary bytes.
Over in the QGS code, QGS historically had a socket protocol taking various messages from the libtdxattest library which were defined in this:
https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h typedef enum _qgs_msg_type_t { GET_QUOTE_REQ = 0, GET_QUOTE_RESP = 1, GET_COLLATERAL_REQ = 2, GET_COLLATERAL_RESP = 3, GET_PLATFORM_INFO_REQ = 4, GET_PLATFORM_INFO_RESP = 5, QGS_MSG_TYPE_MAX } qgs_msg_type_t; typedef struct _qgs_msg_header_t { uint16_t major_version; uint16_t minor_version; uint32_t type; uint32_t size; // size of the whole message, include this header, in byte uint32_t error_code; // used in response only } qgs_msg_header_t;such messages are processed by the '
get_resp' method in QGS:The 1.21 release of DCAP introduced a new "raw" mode in QGS which just receives the raw 1024 byte packet from the client which is supposed to be a raw TDX report. This is what this QEMU patch is relying on IIUC.
The QGS daemon decides whether a client is speaking the formal protocol, or "raw" mode, by trying to interpret the incoming data as a '
qgs_msg_header_t' struct. If the header size looks wrong & it has exactly 1024 bytes, then QGS assumes it has got a raw TDX report:This all works if the data QEMU gets from the guest is indeed a 1024 byte raw TDX report, but what happens if we face a malicious guest ?
AFAICT, the guest OS is able to send a "
qgs_msg_header_t" packet to QEMU, which QEMU blindly passes on to QGS. This allows the guest OS to invoke any of the three QGS commands -GET_QUOTE_REQ,GET_COLLATERAL_REQ, orGET_PLATFORM_INFO_REQ. Fortunately I think those three messages are all safe to invoke, but none the less, this should not be permitted, as it leaves a wide open door for possible future exploits.
Quoted 5 lines
As mentioned before, I don't know why this raw mode was invented for QGS, when QEMU itself could just take the guest report and pack it into the '
GET_QUOTE_REQ' message format and send it to QGS. This prevents the guest OS from being able to exploit QEMU to invoke arbirtary QGS messages.
I guess the raw mode was introduced due to the design was changed to let guest kernel to forward to TD report to host QGS via TDVMCALL instead of guest application communicates with host QGS via vsock, and Linux TD guest driver doesn't integrate any QGS protocol but just forward the raw TD report data to KVM.
Quoted 4 lines
IMHO, QEMU should be made to pack & unpack the TDX report from the guest into the
GET_QUOTE_REQ/GET_QUOTE_RESPmessages, and this "raw" mode should be removed to QGS as it is inherantly dangerous to have this magic protocol overloading.
There is no enforcement that the input data of TDVMCALL.GetQuote is the raw data of TD report. It is just the current Linux tdx-guest driver of tsm implementation send the raw data. For other TDX OS, or third-party driver, they might encapsulate the raw TD report data with QGS message header. For such cases, if QEMU adds another layer of package, it leads to the wrong result.
If we are going to pack the input data of GETQUOTE in QEMU, it becomes a hard requirement from QEMU that the input data of GETQUOTE must be raw data of TD report.
Quoted 199 lines
Below is a patch on top of this one that illustrates how QEMU could use the
GET_QUOTE_REQ/GET_QUOTE_RESPmessages and avoid the "raw" mode of QGS.Quoted 8 lines
target/i386/kvm/tdx.c +8
-0+ + task->receive_buf =g_malloc0 (task->payload_len); + task->receive_buf_received =0 ; + +tdx_generate_quote (task); + +return 0 ; +}target/i386/kvm/tdx-quote-generator.c +156
-1@@ -24,6 +24,61 @@ OBJECT_DEFINE_TYPE(TdxQuoteGenerator, tdx_quote_generator, TDX_QUOTE_GENERATOR, OBJECT) +const uint32_t QGS_MSG_LIB_MAJOR_VER =1 ; +const uint32_t QGS_MSG_LIB_MINOR_VER =1 ; + +typedef enum _qgs_msg_type_t { +GET_QUOTE_REQ =0 , +GET_QUOTE_RESP =1 , +GET_COLLATERAL_REQ =2 , +GET_COLLATERAL_RESP =3 , +GET_PLATFORM_INFO_REQ =4 , +GET_PLATFORM_INFO_RESP =5 , +QGS_MSG_TYPE_MAX +}qgs_msg_type_t ; + +typedef struct _qgs_msg_header_t { +uint16_t major_version; +uint16_t minor_version; +uint32_t type; +uint32_t size;// size of the whole message, include this header, in byte +uint32_t error_code;// used in response only +}qgs_msg_header_t ; + +typedef struct _qgs_msg_get_quote_req_t { +qgs_msg_header_t header;// header.type = GET_QUOTE_REQ +uint32_t report_size;// cannot be 0 +uint32_t id_list_size;// length of id_list, in byte, can be 0 +}qgs_msg_get_quote_req_t ; + +typedef struct _qgs_msg_get_quote_resp_s { +qgs_msg_header_t header;// header.type = GET_QUOTE_RESP +uint32_t selected_id_size;// can be 0 in case only one id is sent in request +uint32_t quote_size;// length of quote_data, in byte +uint8_t id_quote[];// selected id followed by quote +}qgs_msg_get_quote_resp_t ; + +const unsigned HEADER_SIZE =4 ; + +static uint32_t decode_header (const char *buf,size_t len) { +if (len <HEADER_SIZE ) { +return 0 ; + } +uint32_t msg_size =0 ; +for (uint32_t i =0 ; i <HEADER_SIZE ; ++i) { + msg_size = msg_size *256 + (buf[i] &0xFF ); + } +return msg_size; +} + +static void encode_header (char *buf,size_t len,uint32_t size) { +assert (len >=HEADER_SIZE ); + buf[0 ] = ((size >>24 ) &0xFF ); + buf[1 ] = ((size >>16 ) &0xFF ); + buf[2 ] = ((size >>8 ) &0xFF ); + buf[3 ] = (size &0xFF ); +} +static void tdx_quote_generator_finalize (Object *obj) { }@@ -70,9 +125,86 @@ goto end; } } + +if (ret ==0 ) { +error_report ("End of file before reply received" ); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } task->receive_buf_received += ret; -if (ret ==0 || task->receive_buf_received == task->payload_len) { +if (task->receive_buf_received >=HEADER_SIZE ) { +uint32_t len =decode_header (task->receive_buf, + task->receive_buf_received); +if (len ==0 || + len > (task->payload_len -HEADER_SIZE )) { +error_report ("Message len %u must be non-zero & less than %zu" , + len, (task->payload_len -HEADER_SIZE )); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + +/* Now we know the size, shrink to fit */ + task->payload_len =HEADER_SIZE + len; + task->receive_buf =g_renew (char, + task->receive_buf, + task->payload_len); + } + +if (task->receive_buf_received >= (sizeof (qgs_msg_header_t) +HEADER_SIZE )) { +qgs_msg_header_t *hdr = (qgs_msg_header_t *)(task->receive_buf +HEADER_SIZE ); +if (hdr->major_version !=QGS_MSG_LIB_MAJOR_VER || + hdr->minor_version !=QGS_MSG_LIB_MINOR_VER ) { +error_report ("Invalid QGS message header version %d.%d\n" , + hdr->major_version, + hdr->minor_version); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } +if (hdr->type !=GET_QUOTE_RESP ) { +error_report ("Invalid QGS message type %d\n" , + hdr->type); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } +if (hdr->size > (task->payload_len -HEADER_SIZE )) { +error_report ("QGS message size %d exceeds payload capacity %zu" , + hdr->size, task->payload_len); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } +if (hdr->error_code !=0 ) { +error_report ("QGS message error code %d" , + hdr->error_code); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + } +if (task->receive_buf_received >= (sizeof (qgs_msg_get_quote_resp_t) +HEADER_SIZE )) { +qgs_msg_get_quote_resp_t *msg = (qgs_msg_get_quote_resp_t *)(task->receive_buf +HEADER_SIZE ); +if (msg->selected_id_size !=0 ) { +error_report ("QGS message selected ID was %d not 0" , + msg->selected_id_size); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + +if ((task->payload_len -HEADER_SIZE -sizeof (qgs_msg_get_quote_resp_t)) != + msg->quote_size) { +error_report ("QGS quote size %d should be %zu" , + msg->quote_size, + (task->payload_len -sizeof (qgs_msg_get_quote_resp_t))); + task->status_code =TDX_VP_GET_QUOTE_ERROR ; + goto end; + } + } + +if (task->receive_buf_received == task->payload_len) { +size_t strip =HEADER_SIZE +sizeof (qgs_msg_get_quote_resp_t); +memmove (task->receive_buf, + task->receive_buf + strip, + task->receive_buf_received - strip); + task->receive_buf_received -= strip; task->status_code =TDX_VP_GET_QUOTE_SUCCESS ; goto end; }@@ -158,6 +290,29 @@ {struct TdxQuoteGenerator *quote_gen = task->quote_gen;QIOChannelSocket *sioc; +qgs_msg_get_quote_req_t msg; + +/* Prepare a QGS message prelude */ + msg.header.major_version =QGS_MSG_LIB_MAJOR_VER ; + msg.header.minor_version =QGS_MSG_LIB_MINOR_VER ; + msg.header.type =GET_QUOTE_REQ ; + msg.header.size =sizeof (msg) + task->send_data_size; + msg.header.error_code =0 ; + msg.report_size = task->send_data_size; + msg.id_list_size =0 ; + +/* Make room to add the QGS message prelude */ + task->send_data =g_renew (char, + task->send_data, + task->send_data_size +sizeof (msg) +HEADER_SIZE ); +memmove (task->send_data +sizeof (msg) +HEADER_SIZE , + task->send_data, + task->send_data_size); +memcpy (task->send_data +HEADER_SIZE , + &msg, +sizeof (msg)); +encode_header (task->send_data,HEADER_SIZE , task->send_data_size +sizeof (msg)); + task->send_data_size +=sizeof (msg) +HEADER_SIZE ; sioc =qio_channel_socket_new (); task->sioc = sioc;With regards,
Daniel
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 244 lines · Xiaoyao Li
Sorry for the late response.
KVM part of TDX attestation support is submitting again. QEMU part will follow and we need to settle dowm this topic before QEMU patches submission.
Quoted 221 lines · Daniel P. Berrangé
Quoted 125 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>target/i386/kvm/tdx.c +90
-0+static int tdx_handle_get_quote (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) +{ +struct tdx_generate_quote_task *task; +struct tdx_get_quote_header hdr; +hwaddr buf_gpa = vmcall->in_r12; +uint64_t buf_len = vmcall->in_r13; + +QEMU_BUILD_BUG_ON (sizeof (struct tdx_get_quote_header ) !=TDX_GET_QUOTE_HDR_SIZE ); + + vmcall->status_code =TDG_VP_VMCALL_INVALID_OPERAND ; + +if (buf_len ==0 ) { +return 0 ; + } + +/* GPA must be shared. */ +if (!(buf_gpa &tdx_shared_bit (cpu))) { +return 0 ; + } + buf_gpa &= ~tdx_shared_bit (cpu); + +if (!QEMU_IS_ALIGNED (buf_gpa,4096 ) || !QEMU_IS_ALIGNED (buf_len,4096 )) { + vmcall->status_code =TDG_VP_VMCALL_ALIGN_ERROR ; +return 0 ; + } + +if (address_space_read (&address_space_memory, buf_gpa,MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: get-quote: failed to read GetQuote header.\n" ); +return -1 ; + } + +if (le64_to_cpu (hdr.structure_version) !=TDX_GET_QUOTE_STRUCTURE_VERSION ) { +return 0 ; + } + +/* +* Paranoid: Guest should clear error_code and out_len to avoid information +* leak. Enforce it. The initial value of them doesn't matter for qemu to +* process the request. +*/ +if (le64_to_cpu (hdr.error_code) !=TDX_VP_GET_QUOTE_SUCCESS || +le32_to_cpu (hdr.out_len) !=0 ) { +return 0 ; + } + +/* Only safe-guard check to avoid too large buffer size. */ +if (buf_len >TDX_GET_QUOTE_MAX_BUF_LEN || +le32_to_cpu (hdr.in_len) > buf_len -TDX_GET_QUOTE_HDR_SIZE ) { +return 0 ; + } + + vmcall->status_code =TDG_VP_VMCALL_SUCCESS ; +if (!tdx_guest->quote_generator) { + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_QGS_UNAVAILABLE ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } +return 0 ; + } + +qemu_mutex_lock (&tdx_guest->quote_generator->lock); +if (tdx_guest->quote_generator->num >=TDX_MAX_GET_QUOTE_REQUEST ) { +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + vmcall->status_code =TDG_VP_VMCALL_RETRY ; +return 0 ; + } + tdx_guest->quote_generator->num++; +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + +/* Mark the buffer in-flight. */ + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_IN_FLIGHT ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } + + task =g_malloc (sizeof (*task)); + task->buf_gpa = buf_gpa; + task->payload_gpa = buf_gpa +TDX_GET_QUOTE_HDR_SIZE ; + task->payload_len = buf_len -TDX_GET_QUOTE_HDR_SIZE ; + task->hdr = hdr; + task->quote_gen = tdx_guest->quote_generator; + task->completion = tdx_get_quote_completion; +target/i386/kvm/tdx.c
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ task->send_data_size =le32_to_cpu (hdr.in_len); + task->send_data =g_malloc (task->send_data_size); + task->send_data_sent =0 ; + +if (address_space_read (&address_space_memory, task->payload_gpa, +MEMTXATTRS_UNSPECIFIED , task->send_data, + task->send_data_size) !=MEMTX_OK ) { +g_free (task->send_data); +return -1 ; + }In this method we've received "struct
tdx_get_quote_header" from the guest OS, and the 'hdr.in_len' field in that struct tells us the payload to read from guest memory. This payload is treated as opaque by QEMU and sent over the UNIX socket directly to QGS with no validation of the payload.The payload is supposed to be a raw TDX report, that QGS will turn into a quote.
Nothing guarantees that the guest OS has actually given QEMU a payload that represents a TDX report.
The only validation done in this patch is to check the 'hdr.in_len' was not ridiculously huge:
#define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024) #define TDX_GET_QUOTE_HDR_SIZE 24 ... /* Only safe-guard check to avoid too large buffer size. */ if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN || le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) { return 0; }IOW, hdr.in_len can be any value between 0 and 131048, and the payload data read can contain arbitrary bytes.
Over in the QGS code, QGS historically had a socket protocol taking various messages from the libtdxattest library which were defined in this:
https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h typedef enum _qgs_msg_type_t { GET_QUOTE_REQ = 0, GET_QUOTE_RESP = 1, GET_COLLATERAL_REQ = 2, GET_COLLATERAL_RESP = 3, GET_PLATFORM_INFO_REQ = 4, GET_PLATFORM_INFO_RESP = 5, QGS_MSG_TYPE_MAX } qgs_msg_type_t; typedef struct _qgs_msg_header_t { uint16_t major_version; uint16_t minor_version; uint32_t type; uint32_t size; // size of the whole message, include this header, in byte uint32_t error_code; // used in response only } qgs_msg_header_t;such messages are processed by the '
get_resp' method in QGS:The 1.21 release of DCAP introduced a new "raw" mode in QGS which just receives the raw 1024 byte packet from the client which is supposed to be a raw TDX report. This is what this QEMU patch is relying on IIUC.
The QGS daemon decides whether a client is speaking the formal protocol, or "raw" mode, by trying to interpret the incoming data as a '
qgs_msg_header_t' struct. If the header size looks wrong & it has exactly 1024 bytes, then QGS assumes it has got a raw TDX report:This all works if the data QEMU gets from the guest is indeed a 1024 byte raw TDX report, but what happens if we face a malicious guest ?
AFAICT, the guest OS is able to send a "
qgs_msg_header_t" packet to QEMU, which QEMU blindly passes on to QGS. This allows the guest OS to invoke any of the three QGS commands -GET_QUOTE_REQ,GET_COLLATERAL_REQ, orGET_PLATFORM_INFO_REQ. Fortunately I think those three messages are all safe to invoke, but none the less, this should not be permitted, as it leaves a wide open door for possible future exploits.Quoted 5 lines
As mentioned before, I don't know why this raw mode was invented for QGS, when QEMU itself could just take the guest report and pack it into the '
GET_QUOTE_REQ' message format and send it to QGS. This prevents the guest OS from being able to exploit QEMU to invoke arbirtary QGS messages.I guess the raw mode was introduced due to the design was changed to let guest kernel to forward to TD report to host QGS via TDVMCALL instead of guest application communicates with host QGS via vsock, and Linux TD guest driver doesn't integrate any QGS protocol but just forward the raw TD report data to KVM.
Quoted 4 lines
IMHO, QEMU should be made to pack & unpack the TDX report from the guest into the
GET_QUOTE_REQ/GET_QUOTE_RESPmessages, and this "raw" mode should be removed to QGS as it is inherantly dangerous to have this magic protocol overloading.
Quoted 6 lines
There is no enforcement that the input data of TDVMCALL.GetQuote is the raw data of TD report. It is just the current Linux tdx-guest driver of tsm implementation send the raw data. For other TDX OS, or third-party driver, they might encapsulate the raw TD report data with QGS message header. For such cases, if QEMU adds another layer of package, it leads to the wrong result.
If I look at the GHCI spec
In "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a "TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to send arbitrary other data as QGS protocol messages.
Quoted 3 lines
If we are going to pack the input data of GETQUOTE in QEMU, it becomes a hard requirement from QEMU that the input data of GETQUOTE must be raw data of TD report.
AFAICT it must be a raw TDREPORT_STRUCT per the spec and thus QEMU must not allow anything different, as that exposes a significantly larger security attack surface on the QGS daemon.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 252 lines · Daniel P. Berrangé
Quoted 244 lines · Xiaoyao Li
Sorry for the late response.
KVM part of TDX attestation support is submitting again. QEMU part will follow and we need to settle dowm this topic before QEMU patches submission.
Quoted 221 lines · Daniel P. Berrangé
Quoted 125 lines · Xiaoyao Li
From: Isaku Yamahata <[email protected]>
Add property "quote-generation-socket" to tdx-guest, which is a property of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request data from shared guest memory, send the request data to the QGS, and store the response into shared guest memory, at last notify TD guest by interrupt.
command line example:
qemu-system-x86_64 \ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \ -machine confidential-guest-support=tdx0Note, above example uses vsock type socket because the QGS we used implements the vsock socket. It can be other types, like UNIX socket, which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction. If timeout, make it an error and interrupt guest. Define the threshold of time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by Isaku Yamahata <[email protected]>Codeveloped-by Chenyi Qiang <[email protected]>Signed-off-by Chenyi Qiang <[email protected]>Codeveloped-by Xiaoyao Li <[email protected]>Signed-off-by Xiaoyao Li <[email protected]>target/i386/kvm/tdx.c +90
-0+static int tdx_handle_get_quote (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) +{ +struct tdx_generate_quote_task *task; +struct tdx_get_quote_header hdr; +hwaddr buf_gpa = vmcall->in_r12; +uint64_t buf_len = vmcall->in_r13; + +QEMU_BUILD_BUG_ON (sizeof (struct tdx_get_quote_header ) !=TDX_GET_QUOTE_HDR_SIZE ); + + vmcall->status_code =TDG_VP_VMCALL_INVALID_OPERAND ; + +if (buf_len ==0 ) { +return 0 ; + } + +/* GPA must be shared. */ +if (!(buf_gpa &tdx_shared_bit (cpu))) { +return 0 ; + } + buf_gpa &= ~tdx_shared_bit (cpu); + +if (!QEMU_IS_ALIGNED (buf_gpa,4096 ) || !QEMU_IS_ALIGNED (buf_len,4096 )) { + vmcall->status_code =TDG_VP_VMCALL_ALIGN_ERROR ; +return 0 ; + } + +if (address_space_read (&address_space_memory, buf_gpa,MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: get-quote: failed to read GetQuote header.\n" ); +return -1 ; + } + +if (le64_to_cpu (hdr.structure_version) !=TDX_GET_QUOTE_STRUCTURE_VERSION ) { +return 0 ; + } + +/* +* Paranoid: Guest should clear error_code and out_len to avoid information +* leak. Enforce it. The initial value of them doesn't matter for qemu to +* process the request. +*/ +if (le64_to_cpu (hdr.error_code) !=TDX_VP_GET_QUOTE_SUCCESS || +le32_to_cpu (hdr.out_len) !=0 ) { +return 0 ; + } + +/* Only safe-guard check to avoid too large buffer size. */ +if (buf_len >TDX_GET_QUOTE_MAX_BUF_LEN || +le32_to_cpu (hdr.in_len) > buf_len -TDX_GET_QUOTE_HDR_SIZE ) { +return 0 ; + } + + vmcall->status_code =TDG_VP_VMCALL_SUCCESS ; +if (!tdx_guest->quote_generator) { + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_QGS_UNAVAILABLE ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } +return 0 ; + } + +qemu_mutex_lock (&tdx_guest->quote_generator->lock); +if (tdx_guest->quote_generator->num >=TDX_MAX_GET_QUOTE_REQUEST ) { +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + vmcall->status_code =TDG_VP_VMCALL_RETRY ; +return 0 ; + } + tdx_guest->quote_generator->num++; +qemu_mutex_unlock (&tdx_guest->quote_generator->lock); + +/* Mark the buffer in-flight. */ + hdr.error_code =cpu_to_le64 (TDX_VP_GET_QUOTE_IN_FLIGHT ); +if (address_space_write (&address_space_memory, buf_gpa, +MEMTXATTRS_UNSPECIFIED , + &hdr,TDX_GET_QUOTE_HDR_SIZE ) !=MEMTX_OK ) { +error_report ("TDX: failed to update GetQuote header.\n" ); +return -1 ; + } + + task =g_malloc (sizeof (*task)); + task->buf_gpa = buf_gpa; + task->payload_gpa = buf_gpa +TDX_GET_QUOTE_HDR_SIZE ; + task->payload_len = buf_len -TDX_GET_QUOTE_HDR_SIZE ; + task->hdr = hdr; + task->quote_gen = tdx_guest->quote_generator; + task->completion = tdx_get_quote_completion; +target/i386/kvm/tdx.c
Quoted 10 lines
target/i386/kvm/tdx.c +10
-0+ task->send_data_size =le32_to_cpu (hdr.in_len); + task->send_data =g_malloc (task->send_data_size); + task->send_data_sent =0 ; + +if (address_space_read (&address_space_memory, task->payload_gpa, +MEMTXATTRS_UNSPECIFIED , task->send_data, + task->send_data_size) !=MEMTX_OK ) { +g_free (task->send_data); +return -1 ; + }In this method we've received "struct
tdx_get_quote_header" from the guest OS, and the 'hdr.in_len' field in that struct tells us the payload to read from guest memory. This payload is treated as opaque by QEMU and sent over the UNIX socket directly to QGS with no validation of the payload.The payload is supposed to be a raw TDX report, that QGS will turn into a quote.
Nothing guarantees that the guest OS has actually given QEMU a payload that represents a TDX report.
The only validation done in this patch is to check the 'hdr.in_len' was not ridiculously huge:
#define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024) #define TDX_GET_QUOTE_HDR_SIZE 24 ... /* Only safe-guard check to avoid too large buffer size. */ if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN || le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) { return 0; }IOW, hdr.in_len can be any value between 0 and 131048, and the payload data read can contain arbitrary bytes.
Over in the QGS code, QGS historically had a socket protocol taking various messages from the libtdxattest library which were defined in this:
https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h typedef enum _qgs_msg_type_t { GET_QUOTE_REQ = 0, GET_QUOTE_RESP = 1, GET_COLLATERAL_REQ = 2, GET_COLLATERAL_RESP = 3, GET_PLATFORM_INFO_REQ = 4, GET_PLATFORM_INFO_RESP = 5, QGS_MSG_TYPE_MAX } qgs_msg_type_t; typedef struct _qgs_msg_header_t { uint16_t major_version; uint16_t minor_version; uint32_t type; uint32_t size; // size of the whole message, include this header, in byte uint32_t error_code; // used in response only } qgs_msg_header_t;such messages are processed by the '
get_resp' method in QGS:The 1.21 release of DCAP introduced a new "raw" mode in QGS which just receives the raw 1024 byte packet from the client which is supposed to be a raw TDX report. This is what this QEMU patch is relying on IIUC.
The QGS daemon decides whether a client is speaking the formal protocol, or "raw" mode, by trying to interpret the incoming data as a '
qgs_msg_header_t' struct. If the header size looks wrong & it has exactly 1024 bytes, then QGS assumes it has got a raw TDX report:This all works if the data QEMU gets from the guest is indeed a 1024 byte raw TDX report, but what happens if we face a malicious guest ?
AFAICT, the guest OS is able to send a "
qgs_msg_header_t" packet to QEMU, which QEMU blindly passes on to QGS. This allows the guest OS to invoke any of the three QGS commands -GET_QUOTE_REQ,GET_COLLATERAL_REQ, orGET_PLATFORM_INFO_REQ. Fortunately I think those three messages are all safe to invoke, but none the less, this should not be permitted, as it leaves a wide open door for possible future exploits.Quoted 5 lines
As mentioned before, I don't know why this raw mode was invented for QGS, when QEMU itself could just take the guest report and pack it into the '
GET_QUOTE_REQ' message format and send it to QGS. This prevents the guest OS from being able to exploit QEMU to invoke arbirtary QGS messages.I guess the raw mode was introduced due to the design was changed to let guest kernel to forward to TD report to host QGS via TDVMCALL instead of guest application communicates with host QGS via vsock, and Linux TD guest driver doesn't integrate any QGS protocol but just forward the raw TD report data to KVM.
Quoted 4 lines
IMHO, QEMU should be made to pack & unpack the TDX report from the guest into the
GET_QUOTE_REQ/GET_QUOTE_RESPmessages, and this "raw" mode should be removed to QGS as it is inherantly dangerous to have this magic protocol overloading.Quoted 6 lines
There is no enforcement that the input data of TDVMCALL.GetQuote is the raw data of TD report. It is just the current Linux tdx-guest driver of tsm implementation send the raw data. For other TDX OS, or third-party driver, they might encapsulate the raw TD report data with QGS message header. For such cases, if QEMU adds another layer of package, it leads to the wrong result.
Quoted 7 lines
If I look at the GHCI spec
https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdfIn "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a "
TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to send arbitrary other data as QGS protocol messages.
In table 3-7, the description of R12 is
Shared GPA as input - the memory contains a TDREPORT_STRUCT. The same buffer is used as output - the memory contains a TD Quote.
table 3-10, describes the detailed format of the shared GPA:
starting from offset 24 bytes, it is the "Data"
On input, the data filled by TD with input length. The data should include TDREPORT_STRUCT. TD should zeroize the remaining buffer to avoid information leak if size of shared GPA (R13) > Input Length.
It uses the word "contains" and "include", but without "only". So it is not clear to me.
I will work with internal attestation folks to make it clearer that who (TD guest or host VMM) is responsible to encapsulate the raw TDERPORT_STRCUT with QGS MSG protocol, and update the spec accordingly.
Quoted 10 lines
Quoted 3 lines
If we are going to pack the input data of GETQUOTE in QEMU, it becomes a hard requirement from QEMU that the input data of GETQUOTE must be raw data of TD report.
AFAICT it must be a raw
TDREPORT_STRUCTper the spec and thus QEMU must not allow anything different, as that exposes a significantly larger security attack surface on the QGS daemon.With regards,
Daniel
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 42 lines · Xiaoyao Li
Quoted 20 lines · Daniel P. Berrangé
Quoted 12 lines · Xiaoyao Li
I guess the raw mode was introduced due to the design was changed to let guest kernel to forward to TD report to host QGS via TDVMCALL instead of guest application communicates with host QGS via vsock, and Linux TD guest driver doesn't integrate any QGS protocol but just forward the raw TD report data to KVM.
Quoted 4 lines
IMHO, QEMU should be made to pack & unpack the TDX report from the guest into the
GET_QUOTE_REQ/GET_QUOTE_RESPmessages, and this "raw" mode should be removed to QGS as it is inherantly dangerous to have this magic protocol overloading.Quoted 6 lines
There is no enforcement that the input data of TDVMCALL.GetQuote is the raw data of TD report. It is just the current Linux tdx-guest driver of tsm implementation send the raw data. For other TDX OS, or third-party driver, they might encapsulate the raw TD report data with QGS message header. For such cases, if QEMU adds another layer of package, it leads to the wrong result.
Quoted 7 lines
If I look at the GHCI spec
https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdfIn "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a "
TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to send arbitrary other data as QGS protocol messages.In table 3-7, the description of R12 is
Shared GPA as input - the memory contains a
TDREPORT_STRUCT. The same buffer is used as output - the memory contains a TD Quote.table 3-10, describes the detailed format of the shared GPA:
starting from offset 24 bytes, it is the "Data"
On input, the data filled by TD with input length. The data should include
TDREPORT_STRUCT. TD should zeroize the remaining buffer to avoid information leak if size of shared GPA (R13) > Input Length.
Quoted 6 lines
It uses the word "contains" and "include", but without "only". So it is not clear to me.
I will work with internal attestation folks to make it clearer that who (TD guest or host VMM) is responsible to encapsulate the raw
TDERPORT_STRCUTwith QGS MSG protocol, and update the spec accordingly.
To be clear, my strong preference is that the spec be updated to only permit the raw TDREPORT_STRUCT.
IMHO allowing arbitrary QGS MSGs would be a significant host security weakness, as it exposes a huge amount of the QGS codebase to direct attack from the guest. QEMU needs to be able to block that attack vector. Without that, the benefit/value of shuffling of TDREPORTs via the GetQuote hypercall is largely eliminated, and might as well have just exposed QGS over VSOCK.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
Quoted 50 lines · Daniel P. Berrangé
Quoted 42 lines · Xiaoyao Li
Quoted 20 lines · Daniel P. Berrangé
Quoted 12 lines · Xiaoyao Li
I guess the raw mode was introduced due to the design was changed to let guest kernel to forward to TD report to host QGS via TDVMCALL instead of guest application communicates with host QGS via vsock, and Linux TD guest driver doesn't integrate any QGS protocol but just forward the raw TD report data to KVM.
Quoted 4 lines
IMHO, QEMU should be made to pack & unpack the TDX report from the guest into the
GET_QUOTE_REQ/GET_QUOTE_RESPmessages, and this "raw" mode should be removed to QGS as it is inherantly dangerous to have this magic protocol overloading.Quoted 6 lines
There is no enforcement that the input data of TDVMCALL.GetQuote is the raw data of TD report. It is just the current Linux tdx-guest driver of tsm implementation send the raw data. For other TDX OS, or third-party driver, they might encapsulate the raw TD report data with QGS message header. For such cases, if QEMU adds another layer of package, it leads to the wrong result.
Quoted 7 lines
If I look at the GHCI spec
https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdfIn "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a "
TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to send arbitrary other data as QGS protocol messages.In table 3-7, the description of R12 is
Shared GPA as input - the memory contains a
TDREPORT_STRUCT. The same buffer is used as output - the memory contains a TD Quote.table 3-10, describes the detailed format of the shared GPA:
starting from offset 24 bytes, it is the "Data"
On input, the data filled by TD with input length. The data should include
TDREPORT_STRUCT. TD should zeroize the remaining buffer to avoid information leak if size of shared GPA (R13) > Input Length.Quoted 6 lines
It uses the word "contains" and "include", but without "only". So it is not clear to me.
I will work with internal attestation folks to make it clearer that who (TD guest or host VMM) is responsible to encapsulate the raw
TDERPORT_STRCUTwith QGS MSG protocol, and update the spec accordingly.
Quoted 6 lines
To be clear, my strong preference is that the spec be updated to only permit the raw
TDREPORT_STRUCT.IMHO allowing arbitrary QGS MSGs would be a significant host security weakness, as it exposes a huge amount of the QGS codebase to direct attack from the guest.
If I remember correctly, the QGS instance keeps the vsock interface so that TD guest can communicate with QGS directly without going through host VMM. (I'm not sure if latest QGS implementation still keeps it.) So QGS should know how to protect itself.
Regarding QEMU avoids from being exploited to forward arbitrary data from malicious TDX guest to QGS, QEMU can check the beginning of the data to be a valid QGS MSG header before handing it over to QGS socket.
Quoted 4 lines
QEMU needs to be able to block that attack
vector. Without that, the benefit/value of shuffling of TDREPORTs via the GetQuote hypercall is largely eliminated, and might as well have just exposed QGS over VSOCK.
If I remember correctly, one of the reason we changed from TD guest communicates with QGS directly with vsock to current TDVMCALL(GETQUOTE), is some of the TD guest environment might not have network stack for vsock to work.
Anyway, I don't have preference. Either is OK to me. Let's see what decision the attestation folks will make.
Quoted 2 lines
With regards,
Daniel
PATCH v5 50/65 i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall
In reply to Xiaoyao LiFrom: Isaku Yamahata <[email protected]>
MapGPA is a hypercall to convert GPA from/to private GPA to/from shared GPA. As the conversion function is already implemented as kvm_convert_memory, wire it to TDX hypercall exit.
--- accel/kvm/kvm-all.c | 2 +- include/sysemu/kvm.h | 2 ++ target/i386/kvm/tdx.c | 54 +++++++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 1 + 4 files changed, 58 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +1 -1
@@ -2930,7 +2930,7 @@ static void kvm_eat_signals(CPUState *cpu) }while (sigismember (&chkset,SIG_IPI )); } -static int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) +int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private) {MemoryRegionSection section;ram_addr_t offset;
include/sysemu/kvm.h +2 -0
@@ -550,4 +550,6 @@ int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);int kvm_set_memory_attributes_private (hwaddr start,hwaddr size);int kvm_set_memory_attributes_shared (hwaddr start,hwaddr size); + +int kvm_convert_memory (hwaddr start,hwaddr size,bool to_private);#endif
target/i386/kvm/tdx.c +54 -0
66 lines not on this page · raw message
target/i386/kvm/tdx.h +1 -0
@@ -18,6 +18,7 @@ typedef struct TdxGuestClass {ConfidentialGuestSupportClass parent_class; } TdxGuestClass; +#define TDG_VP_VMCALL_MAP_GPA 0x10001ULL#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL
Signature · 1 line
2.34.1
PATCH v5 51/65 i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>
In reply to Xiaoyao LiTD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request termination with error message encoded in GPRs.
Parse and print the error message, and terminate the TD guest in the handler.
--- target/i386/kvm/tdx.c | 39 +++++++++++++++++++++++++++++++++++++++ target/i386/kvm/tdx.h | 1 + 2 files changed, 40 insertions(+)
target/i386/kvm/tdx.c +39 -0
@@ -1092,6 +1092,43 @@ static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)return 0 ; } +static int tdx_handle_report_fatal_error (X86CPU *cpu, +struct kvm_tdx_vmcall *vmcall) +{ +uint64_t error_code = vmcall->in_r12; +char *message =NULL ; + +if (error_code &0xffff ) { +error_report ("TDX: REPORT_FATAL_ERROR: invalid error code: " +"0x%lx\n" , error_code); +return -1 ; + } + +/* it has optional message */ +if (vmcall->in_r14) { +uint64_t * tmp; + +#define GUEST_PANIC_INFO_TDX_MESSAGE_MAX 64 + message =g_malloc0 (GUEST_PANIC_INFO_TDX_MESSAGE_MAX +1 ); + + tmp = (uint64_t *)message; +/* The order is defined in TDX GHCI spec */ + *(tmp++) =cpu_to_le64 (vmcall->in_r14); + *(tmp++) =cpu_to_le64 (vmcall->in_r15); + *(tmp++) =cpu_to_le64 (vmcall->in_rbx); + *(tmp++) =cpu_to_le64 (vmcall->in_rdi); + *(tmp++) =cpu_to_le64 (vmcall->in_rsi); + *(tmp++) =cpu_to_le64 (vmcall->in_r8); + *(tmp++) =cpu_to_le64 (vmcall->in_r9); + *(tmp++) =cpu_to_le64 (vmcall->in_rdx); + message[GUEST_PANIC_INFO_TDX_MESSAGE_MAX ] ='\0' ; +assert ((char *)tmp == message +GUEST_PANIC_INFO_TDX_MESSAGE_MAX ); + } + +error_report ("TD guest reports fatal error. %s\n" , message ? :"" ); +return -1 ; +} +static int tdx_handle_setup_event_notify_interrupt (X86CPU *cpu,struct kvm_tdx_vmcall *vmcall) {
@@ -1126,6 +1163,8 @@ static int tdx_handle_vmcall(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)return tdx_handle_map_gpa (cpu, vmcall);case TDG_VP_VMCALL_GET_QUOTE :return tdx_handle_get_quote (cpu, vmcall); +case TDG_VP_VMCALL_REPORT_FATAL_ERROR : +return tdx_handle_report_fatal_error (cpu, vmcall);case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT :return tdx_handle_setup_event_notify_interrupt (cpu, vmcall);default :
target/i386/kvm/tdx.h +1 -0
@@ -20,6 +20,7 @@ typedef struct TdxGuestClass {#define TDG_VP_VMCALL_MAP_GPA 0x10001ULL#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL +#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003ULL#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
Signature · 1 line
2.34.1