Skip to contentSkip to conversation

QEMU TDX support

PATCH Superseded Xiaoyao Li 5 participants 161 messages Apr 1, 2025 – May 8, 2025

Subsystems (3)

v8 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Xiaoyao Li

author
#0 lore raw

v8 cover QEMU TDX support

This is the v8 series of TDX QEMU enabling and the series is also available at github:
https://github.com/intel-staging/qemu-tdx/tree/tdx-qemu-upstream-v8

To boot TD guest, please always use the latest TDX module (1.5) and OVMF available.

Note, this series has a dependency on
https://lore.kernel.org/qemu-devel/[email protected]/

=============
Changes in v8

- Address the comments from Francesco Lavra;
- Add a patch to check KVM_CAP_MAX_VCPUS at vm level;
- Make the default kernel_irqchip to split mode for TDX and refuse to boot when other mode requested; (Daniel)
- Drop the manually adjustment of supported CPUID for TDX, which turns to be not forward compatible since any newly enabled feature for common KVM will automatically be treated as supported for TDX as well. Instead, implement TDX's supported CPUID by combining the information from directly configurable bits, fixed1 bits, XFAM controlled bits and attributes controlled bits.

===========
future work

- CPU model

It now only supports booting TD VM with "-cpu host". It is the only case that not supposed to hit any warning/error.

When using named CPU model, even the same model as host, it likely hits warning like some feature not supported or some feature enforced on. It's a future work to decide if needs to introduce TDX specific named CPU models.

- Attestation support

Atttestation support will be submitted separately after KVM side patches being submitted.

- gdb support

gdb support to debug a TD in off-debug mode is left as future work.

=======
history

v7: https://lore.kernel.org/qemu-devel/[email protected]/

v6: https://lore.kernel.org/qemu-devel/[email protected]/

Chao Peng (1):
i386/tdx: load TDVF for TD guest

Isaku Yamahata (5):
i386/tdx: Make sept_ve_disable set by default
i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig i386/tdvf: Introduce function to parse TDVF metadata i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION i386/tdx: Don't synchronize guest tsc for TDs

Xiaoyao Li (49):
*** HACK *** linux-headers: Update headers to pull in TDX API changes
i386: Introduce tdx-guest object
i386/tdx: Implement tdx_kvm_type() for TDX
i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object
kvm: Introduce kvm_arch_pre_create_vcpu()
i386/tdx: Initialize TDX before creating TD vcpus i386/tdx: Add property sept-ve-disable for tdx-guest object i386/tdx: Wire CPU features up with attributes of TD guest i386/tdx: Validate TD attributes
i386/tdx: Set APIC bus rate to match with what TDX module enforces i386/tdx: Implement user specified tsc frequency
i386/tdx: Parse TDVF metadata for TDX VM
i386/tdx: Don't initialize pc.rom for TDX VMs
i386/tdx: Track mem_ptr for each firmware entry of TDVF i386/tdx: Track RAM entries for TDX VM
headers: Add definitions from UEFI spec for volumes, resources, etc... i386/tdx: Setup the TD HOB list
i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu i386/tdx: Finalize TDX VM
i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL
i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
kvm: Check KVM_CAP_MAX_VCPUS at vm level
i386/cpu: introduce x86_confidential_guest_cpu_instance_init() i386/tdx: implement tdx_cpu_instance_init()
i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f i386/tdx: Force exposing CPUID 0x1f
i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM i386/tdx: Disable SMM for TDX VMs
i386/tdx: Disable PIC for TDX VMs
i386/tdx: Set and check kernel_irqchip mode for TDX i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs i386/apic: Skip kvm_apic_put() for TDX
cpu: Don't set vcpu_dirty when guest_state_protected i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features() i386/tdx: Implement adjust_cpuid_features() for TDX i386/tdx: Add TDX fixed1 bits to supported CPUIDs i386/tdx: Add supported CPUID bits related to TD Attributes i386/tdx: Add supported CPUID bits relates to XFAM i386/tdx: Add XFD to supported bit of TDX
i386/tdx: Define supported KVM features for TDX
i386/cgs: Introduce x86_confidential_guest_check_features() i386/tdx: Fetch and validate CPUID of TD guest
i386/tdx: Don't treat SYSCALL as unavailable
i386/tdx: Make invtsc default on
i386/tdx: Validate phys_bits against host value
docs: Add TDX documentation

 accel/kvm/kvm-all.c                        |   11 +-
 configs/devices/i386-softmmu/default.mak   |    1 +
 docs/system/confidential-guest-support.rst |    1 +
 docs/system/i386/tdx.rst                   |  156 +++
 docs/system/target-i386.rst                |    1 +
 hw/i386/Kconfig                            |    6 +
 hw/i386/kvm/apic.c                         |    5 +
 hw/i386/meson.build                        |    1 +
 hw/i386/pc.c                               |   29 +-
 hw/i386/pc_sysfw.c                         |    7 +
 hw/i386/tdvf-hob.c                         |  130 ++
 hw/i386/tdvf-hob.h                         |   26 +
 hw/i386/tdvf.c                             |  184 +++
 hw/i386/x86-common.c                       |    6 +-
 include/hw/i386/tdvf.h                     |   45 +
 include/standard-headers/uefi/uefi.h       |  187 +++
 include/system/kvm.h                       |    1 +
 linux-headers/asm-x86/kvm.h                |   69 ++
 linux-headers/linux/kvm.h                  |    1 +
 qapi/qom.json                              |   35 +
 qapi/run-state.json                        |   31 +-
 system/runstate.c                          |   65 +
 target/arm/kvm.c                           |    5 +
 target/i386/confidential-guest.h           |   44 +-
 target/i386/cpu.c                          |   73 +-
 target/i386/cpu.h                          |   40 +
 target/i386/host-cpu.c                     |    2 +-
 target/i386/host-cpu.h                     |    1 +
 target/i386/kvm/kvm.c                      |  114 +-
 target/i386/kvm/kvm_i386.h                 |   15 +
 target/i386/kvm/meson.build                |    2 +
 target/i386/kvm/tdx-stub.c                 |   20 +
 target/i386/kvm/tdx.c                      | 1273 ++++++++++++++++++++
 target/i386/kvm/tdx.h                      |   65 +
 target/i386/sev.c                          |    9 +-
 target/loongarch/kvm/kvm.c                 |    4 +
 target/mips/kvm.c                          |    5 +
 target/ppc/kvm.c                           |    5 +
 target/riscv/kvm/kvm-cpu.c                 |    5 +
 target/s390x/kvm/kvm.c                     |    5 +
 40 files changed, 2592 insertions(+), 93 deletions(-)
 create mode 100644 docs/system/i386/tdx.rst
 create mode 100644 hw/i386/tdvf-hob.c
 create mode 100644 hw/i386/tdvf-hob.h
 create mode 100644 hw/i386/tdvf.c
 create mode 100644 include/hw/i386/tdvf.h
 create mode 100644 include/standard-headers/uefi/uefi.h
 create mode 100644 target/i386/kvm/tdx-stub.c
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v8 1/55 *** HACK *** linux-headers: Update headers to pull in TDX API changes

Pull in recent TDX updates, which are not backwards compatible.

It's just to make this series runnable. It will be updated by script

scripts/update-linux-headers.sh

once TDX support is upstreamed in linux kernel

---
 linux-headers/asm-x86/kvm.h | 69 +++++++++++++++++++++++++++++++++++++
 linux-headers/linux/kvm.h   |  1 +
 2 files changed, 70 insertions(+)
linux-headers/asm-x86/kvm.h +69 -0
index 86f2c34e7afa..baca2d479365 100644
@@ -925,4 +925,73 @@ struct kvm_hyperv_eventfd {
 #define KVM_X86_SNP_VM		4
 #define KVM_X86_TDX_VM		5
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
+	KVM_TDX_INIT_MEM_REGION,
+	KVM_TDX_FINALIZE_VM,
+	KVM_TDX_GET_CPUID,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
+	__u32 flags;
+	/*
+	 * data for each sub-command. An immediate or a pointer to the actual
+	 * data in process virtual address.  If sub-command doesn't use it,
+	 * set zero.
+	 */
+	__u64 data;
+	/*
+	 * Auxiliary error code.  The sub-command may return TDX SEAMCALL
+	 * status code in addition to -Exxx.
+	 * Defined for consistency with struct kvm_sev_cmd.
+	 */
+	__u64 hw_error;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 supported_attrs;
+	__u64 supported_xfam;
+	__u64 reserved[254];
+	struct kvm_cpuid2 cpuid;
+};
+
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u64 xfam;
+	__u64 mrconfigid[6];	/* sha384 digest */
+	__u64 mrowner[6];	/* sha384 digest */
+	__u64 mrownerconfig[6];	/* sha384 digest */
+
+	/* The total space for TD_PARAMS before the CPUIDs is 256 bytes */
+	__u64 reserved[12];
+
+	/*
+	 * Call KVM_TDX_INIT_VM before vcpu creation, thus before
+	 * KVM_SET_CPUID2.
+	 * This configuration supersedes KVM_SET_CPUID2s for VCPUs because the
+	 * TDX module directly virtualizes those CPUIDs without VMM.  The user
+	 * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with
+	 * those values.  If it doesn't, KVM may have wrong idea of vCPUIDs of
+	 * the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX
+	 * module doesn't virtualize.
+	 */
+	struct kvm_cpuid2 cpuid;
+};
+
+#define KVM_TDX_MEASURE_MEMORY_REGION   _BITULL(0)
+
+struct kvm_tdx_init_mem_region {
+	__u64 source_addr;
+	__u64 gpa;
+	__u64 nr_pages;
+};
+
 #endif /* _ASM_X86_KVM_H */
linux-headers/linux/kvm.h +1 -0
index 27181b3dd8fb..bf728042db0c 100644
@@ -369,6 +369,7 @@ struct kvm_run {
 #define KVM_SYSTEM_EVENT_WAKEUP         4
 #define KVM_SYSTEM_EVENT_SUSPEND        5
 #define KVM_SYSTEM_EVENT_SEV_TERM       6
+#define KVM_SYSTEM_EVENT_TDX_FATAL      7
 			__u32 type;
 			__u32 ndata;
 			union {
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v8 01/55] *** HACK *** linux-headers: Update headers to pull in TDX API changes

linux-headers/asm-x86/kvm.h struct kvm_hyperv_eventfd { · line 928

Quoted 10 lines
linux-headers/asm-x86/kvm.h +1 -0
index 86f2c34e7afa..baca2d479365 100644
@@ -925,4 +925,73 @@ struct kvm_hyperv_eventfd {
 #define KVM_X86_SNP_VM		4
 #define KVM_X86_TDX_VM		5
 
+/* Trust Domain eXtension sub-ioctl() commands. */

Typo? s/eXtension/Extension/

Quoted 5 lines
linux-headers/asm-x86/kvm.h +5 -0
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
+	KVM_TDX_INIT_MEM_REGION,

linux-headers/asm-x86/kvm.h struct kvm_hyperv_eventfd { · line 943

Quoted 10 lines
linux-headers/asm-x86/kvm.h +10 -0
+	KVM_TDX_FINALIZE_VM,
+	KVM_TDX_GET_CPUID,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */

Typo? s/sub-commend/sub-command/, this line have these 2 cases.

author
#3 lore raw

Re: [PATCH v8 01/55] *** HACK *** linux-headers: Update headers to pull in TDX API changes

Quoted 22 lines · Zhao Liu

linux-headers/asm-x86/kvm.h struct kvm_hyperv_eventfd { · line 928

Quoted 10 lines
linux-headers/asm-x86/kvm.h +1 -0
index 86f2c34e7afa..baca2d479365 100644
@@ -925,4 +925,73 @@ struct kvm_hyperv_eventfd {
  #define KVM_X86_SNP_VM		4
  #define KVM_X86_TDX_VM		5
 
+/* Trust Domain eXtension sub-ioctl() commands. */

Typo? s/eXtension/Extension/

Quoted 9 lines
linux-headers/asm-x86/kvm.h +9 -0
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
+	KVM_TDX_INIT_MEM_REGION,
+	KVM_TDX_FINALIZE_VM,
+	KVM_TDX_GET_CPUID,
+
+	KVM_TDX_CMD_NR_MAX,
Quoted 9 lines

linux-headers/asm-x86/kvm.h struct kvm_hyperv_eventfd { · line 943

Quoted 6 lines
linux-headers/asm-x86/kvm.h +6 -0
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */

Typo? s/sub-commend/sub-command/, this line have these 2 cases.

They (it and above) were synced from Linux headers. It needs to reported to KVM and fixed there.

author
#4 lore raw

PATCH v8 2/55 i386: Introduce tdx-guest object

In reply to Xiaoyao Li

Introduce tdx-guest object which inherits X86_CONFIDENTIAL_GUEST, and will be used to create TDX VMs (TDs) by

  qemu -machine ...,confidential-guest-support=tdx0	\
       -object tdx-guest,id=tdx0

It has one QAPI member 'attributes' defined, which allows user to set TD's attributes directly.

---
Changes in v7:
- update QAPI version to 10.0;
- update to use SPDX tags for license info;
- update copyright to 2025;

Chanegs in v6:
- Make tdx-guest inherits X86_CONFIDENTIAL_GUEST;
- set cgs->require_guest_memfd;
- allow attributes settable via QAPI;
- update QAPI version to since 9.2;

Changes in v4:
- update the new qapi since filed from 8.2 to 9.0

Changes in v1
- make @attributes not user-settable

---
 configs/devices/i386-softmmu/default.mak |  1 +
 hw/i386/Kconfig                          |  5 +++
 qapi/qom.json                            | 15 +++++++++
 target/i386/kvm/meson.build              |  2 ++
 target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
 target/i386/kvm/tdx.h                    | 21 ++++++++++++
 6 files changed, 87 insertions(+)
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h
configs/devices/i386-softmmu/default.mak +1 -0
index 4faf2f0315e2..bc0479a7e0a3 100644
@@ -18,6 +18,7 @@
 #CONFIG_QXL=n
 #CONFIG_SEV=n
 #CONFIG_SGA=n
+#CONFIG_TDX=n
 #CONFIG_TEST_DEVICES=n
 #CONFIG_TPM_CRB=n
 #CONFIG_TPM_TIS_ISA=n
hw/i386/Kconfig +5 -0
index d34ce07b215d..cce9521ba934 100644
@@ -10,6 +10,10 @@ config SGX
     bool
     depends on KVM
 
+config TDX
+    bool
+    depends on KVM
+
 config PC
     bool
     imply APPLESMC
@@ -26,6 +30,7 @@ config PC
     imply QXL
     imply SEV
     imply SGX
+    imply TDX
     imply TEST_DEVICES
     imply TPM_CRB
     imply TPM_TIS_ISA
qapi/qom.json +15 -0
index 28ce24cd8d08..c0b61df964ef 100644
@@ -1047,6 +1047,19 @@
             '*host-data': 'str',
             '*vcek-disabled': 'bool' } }
 
+##
+# @TdxGuestProperties:
+#
+# Properties for tdx-guest objects.
+#
+# @attributes: The 'attributes' of a TD guest that is passed to
+#     KVM_TDX_INIT_VM
+#
+# Since: 10.1
+##
+{ 'struct': 'TdxGuestProperties',
+  'data': { '*attributes': 'uint64' } }
+
 ##
 # @ThreadContextProperties:
 #
@@ -1132,6 +1145,7 @@
     'sev-snp-guest',
     'thread-context',
     's390-pv-guest',
+    'tdx-guest',
     'throttle-group',
     'tls-creds-anon',
     'tls-creds-psk',
@@ -1204,6 +1218,7 @@
                                       'if': 'CONFIG_SECRET_KEYRING' },
       'sev-guest':                  'SevGuestProperties',
       'sev-snp-guest':              'SevSnpGuestProperties',
+      'tdx-guest':                  'TdxGuestProperties',
       'thread-context':             'ThreadContextProperties',
       'throttle-group':             'ThrottleGroupProperties',
       'tls-creds-anon':             'TlsCredsAnonProperties',
target/i386/kvm/meson.build +2 -0
index 3996cafaf29f..466bccb9cb17 100644
@@ -8,6 +8,8 @@ i386_kvm_ss.add(files(
 
 i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
 
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'))
+
 i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
 
 i386_system_ss.add_all(when: 'CONFIG_KVM', if_true: i386_kvm_ss)
target/i386/kvm/tdx.c new file +43 -0

43 lines not on this page · raw message

target/i386/kvm/tdx.h new file +21 -0
new file mode 100644
index 000000000000..f3b725336161
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H
+
+#include "confidential-guest.h"
+
+#define TYPE_TDX_GUEST "tdx-guest"
+#define TDX_GUEST(obj)  OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
+
+typedef struct TdxGuestClass {
+    X86ConfidentialGuestClass parent_class;
+} TdxGuestClass;
+
+typedef struct TdxGuest {
+    X86ConfidentialGuest parent_obj;
+
+    uint64_t attributes;    /* TD attributes */
+} TdxGuest;
+
+#endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Reviewed-by on v8 2/55
#5 lore raw

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

Quoted 29 lines · Xiaoyao Li

Introduce tdx-guest object which inherits X86_CONFIDENTIAL_GUEST, and will be used to create TDX VMs (TDs) by

  qemu -machine ...,confidential-guest-support=tdx0	\
       -object tdx-guest,id=tdx0

It has one QAPI member 'attributes' defined, which allows user to set TD's attributes directly.

---
Changes in v7:
- update QAPI version to 10.0;
- update to use SPDX tags for license info;
- update copyright to 2025;

Chanegs in v6:
- Make tdx-guest inherits X86_CONFIDENTIAL_GUEST;
- set cgs->require_guest_memfd;
- allow attributes settable via QAPI;
- update QAPI version to since 9.2;

Changes in v4:
- update the new qapi since filed from 8.2 to 9.0

Changes in v1
- make @attributes not user-settable

Quoted 10 lines
---
 configs/devices/i386-softmmu/default.mak |  1 +
 hw/i386/Kconfig                          |  5 +++
 qapi/qom.json                            | 15 +++++++++
 target/i386/kvm/meson.build              |  2 ++
 target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
 target/i386/kvm/tdx.h                    | 21 ++++++++++++
 6 files changed, 87 insertions(+)
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 2/55
#6 lore raw

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

In reply to Xiaoyao Li
Quoted 6 lines
 configs/devices/i386-softmmu/default.mak |  1 +
 hw/i386/Kconfig                          |  5 +++
 qapi/qom.json                            | 15 +++++++++
 target/i386/kvm/meson.build              |  2 ++
 target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
 target/i386/kvm/tdx.h                    | 21 ++++++++++++

SEV.* and confidential-guest.* are all placed in target/i386/. It's best if all of these can be in the same place.

Quoted 3 lines
 6 files changed, 87 insertions(+)
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h

...

target/i386/kvm/tdx.h line 4

Quoted 10 lines
target/i386/kvm/tdx.h new file +4 -0
new file mode 100644
index 000000000000..f3b725336161
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H

I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole project.

Others look good,

author
#7 lore raw

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

Quoted 9 lines · Zhao Liu
Quoted 6 lines
  configs/devices/i386-softmmu/default.mak |  1 +
  hw/i386/Kconfig                          |  5 +++
  qapi/qom.json                            | 15 +++++++++
  target/i386/kvm/meson.build              |  2 ++
  target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
  target/i386/kvm/tdx.h                    | 21 ++++++++++++

SEV.* and confidential-guest.* are all placed in target/i386/. It's best if all of these can be in the same place.

I think it's more reasonable to put TDX code under KVM directory since TDX depends on KVM.

Regarding SEV, I think the same applies. But it was merged as is and I don't see it strongly that it has to move.

Quoted 6 lines
Quoted 3 lines
  6 files changed, 87 insertions(+)

create mode 100644 target/i386/kvm/tdx.c
create mode 100644 target/i386/kvm/tdx.h

...

Quoted 13 lines

target/i386/kvm/tdx.h line 4

Quoted 10 lines
target/i386/kvm/tdx.h new file +4 -0
new file mode 100644
index 000000000000..f3b725336161
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H

I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole project.

# git grep -r "#ifndef QEMU_" ./ | wc -l
# 329

I'm not sure if 329 means rarely?

I will just keep it unchanged unless someone objects it strongly.

Quoted 3 lines

Others look good,

Thanks!

#8 lore raw

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

In reply to Zhao Liu
Quoted 16 lines · Zhao Liu
Quoted 6 lines
 configs/devices/i386-softmmu/default.mak |  1 +
 hw/i386/Kconfig                          |  5 +++
 qapi/qom.json                            | 15 +++++++++
 target/i386/kvm/meson.build              |  2 ++
 target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
 target/i386/kvm/tdx.h                    | 21 ++++++++++++

SEV.* and confidential-guest.* are all placed in target/i386/. It's best if all of these can be in the same place.

Quoted 3 lines
 6 files changed, 87 insertions(+)
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h

...

Quoted 13 lines

target/i386/kvm/tdx.h line 4

Quoted 10 lines
target/i386/kvm/tdx.h new file +4 -0
new file mode 100644
index 000000000000..f3b725336161
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H

I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole project.

IMHO having a QEMU_ prefix here is "best practice", so don't remove it.

That lots of other QEMU code doesn't follow best practice is unfortunate.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#9 lore raw

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

Quoted 12 lines
Quoted 11 lines

target/i386/kvm/tdx.h line 4

Quoted 8 lines

index 000000000000..f3b725336161

target/i386/kvm/tdx.h new file +4 -0
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H

I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole project.

Quoted 3 lines

IMHO having a QEMU_ prefix here is "best practice", so don't remove it.

That lots of other QEMU code doesn't follow best practice is unfortunate.

Thanks, now I see!

author
#10 lore raw

PATCH v8 3/55 i386/tdx: Implement tdx_kvm_type() for TDX

In reply to Xiaoyao Li

TDX VM requires VM type to be KVM_X86_TDX_VM. Implement tdx_kvm_type() as X86ConfidentialGuestClass->kvm_type.

---
Changes in v6:
- new added patch;

---
 target/i386/kvm/kvm.c |  1 +
 target/i386/kvm/tdx.c | 12 ++++++++++++
 2 files changed, 13 insertions(+)
target/i386/kvm/kvm.c +1 -0
index 6c749d4ee812..4f1cfb529c19 100644
@@ -191,6 +191,7 @@ static const char *vm_type_name[] = {
     [KVM_X86_SEV_VM] = "SEV",
     [KVM_X86_SEV_ES_VM] = "SEV-ES",
     [KVM_X86_SNP_VM] = "SEV-SNP",
+    [KVM_X86_TDX_VM] = "TDX",
 };
 
 bool kvm_is_vm_type_supported(int type)
target/i386/kvm/tdx.c +12 -0
index ec84ae2947bb..d785c1f6d173 100644
@@ -12,8 +12,17 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 
+#include "kvm_i386.h"
 #include "tdx.h"
 
+static int tdx_kvm_type(X86ConfidentialGuest *cg)
+{
+    /* Do the object check */
+    TDX_GUEST(cg);
+
+    return KVM_X86_TDX_VM;
+}
+
 /* tdx guest */
 OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
                                    tdx_guest,
@@ -40,4 +49,7 @@ static void tdx_guest_finalize(Object *obj)
 
 static void tdx_guest_class_init(ObjectClass *oc, void *data)
 {
+    X86ConfidentialGuestClass *x86_klass = X86_CONFIDENTIAL_GUEST_CLASS(oc);
+
+    x86_klass->kvm_type = tdx_kvm_type;
 }
Signature · 1 line
2.34.1
Reviewed-by on v8 3/55
#11 lore raw

Re: [PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX

Quoted 11 lines · Xiaoyao Li

TDX VM requires VM type to be KVM_X86_TDX_VM. Implement tdx_kvm_type() as X86ConfidentialGuestClass->kvm_type.

---
Changes in v6:
- new added patch;

---
 target/i386/kvm/kvm.c |  1 +
 target/i386/kvm/tdx.c | 12 ++++++++++++
 2 files changed, 13 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 3/55
#12 lore raw

Re: [PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX

In reply to Xiaoyao Li
Quoted 8 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:13 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX
X-Mailer: git-send-email 2.34.1

TDX VM requires VM type to be KVM_X86_TDX_VM. Implement tdx_kvm_type() as X86ConfidentialGuestClass->kvm_type.

Quoted 8 lines

---
Changes in v6:
- new added patch;

---
 target/i386/kvm/kvm.c |  1 +
 target/i386/kvm/tdx.c | 12 ++++++++++++
 2 files changed, 13 insertions(+)
author
#13 lore raw

PATCH v8 4/55 i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context

In reply to Xiaoyao Li

Implement TDX specific ConfidentialGuestSupportClass::kvm_init() callback, tdx_kvm_init().

Mark guest state is proctected for TDX VM. More TDX specific initialization will be added later.

---
Changes in v6:
- remove Acked-by from Gerd since the patch changed due to use ConfidentialGuestSupportClass::kvm_init();

---
 target/i386/kvm/kvm.c | 11 +----------
 target/i386/kvm/tdx.c | 10 ++++++++++
 2 files changed, 11 insertions(+), 10 deletions(-)
target/i386/kvm/kvm.c +1 -10
index 4f1cfb529c19..1af4710556ad 100644
@@ -3206,16 +3206,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     Error *local_err = NULL;
 
     /*
-     * Initialize SEV context, if required
-     *
-     * If no memory encryption is requested (ms->cgs == NULL) this is
-     * a no-op.
-     *
-     * It's also a no-op if a non-SEV confidential guest support
-     * mechanism is selected.  SEV is the only mechanism available to
-     * select on x86 at present, so this doesn't arise, but if new
-     * mechanisms are supported in future (e.g. TDX), they'll need
-     * their own initialization either here or elsewhere.
+     * Initialize confidential guest (SEV/TDX) context, if required
      */
     if (ms->cgs) {
         ret = confidential_guest_kvm_init(ms->cgs, &local_err);
target/i386/kvm/tdx.c +10 -0
index d785c1f6d173..4ff94860815d 100644
@@ -12,9 +12,17 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 
+#include "hw/i386/x86.h"
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
+{
+    kvm_mark_guest_state_protected();
+
+    return 0;
+}
+
 static int tdx_kvm_type(X86ConfidentialGuest *cg)
 {
     /* Do the object check */
@@ -49,7 +57,9 @@ static void tdx_guest_finalize(Object *obj)
 
 static void tdx_guest_class_init(ObjectClass *oc, void *data)
 {
+    ConfidentialGuestSupportClass *klass = CONFIDENTIAL_GUEST_SUPPORT_CLASS(oc);
     X86ConfidentialGuestClass *x86_klass = X86_CONFIDENTIAL_GUEST_CLASS(oc);
 
+    klass->kvm_init = tdx_kvm_init;
     x86_klass->kvm_type = tdx_kvm_type;
 }
Signature · 1 line
2.34.1
Reviewed-by on v8 4/55
#14 lore raw

Re: [PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context

Quoted 7 lines · Xiaoyao Li

Implement TDX specific ConfidentialGuestSupportClass::kvm_init() callback, tdx_kvm_init().

Mark guest state is proctected for TDX VM. More TDX specific initialization will be added later.

Quoted 8 lines

---
Changes in v6:
- remove Acked-by from Gerd since the patch changed due to use ConfidentialGuestSupportClass::kvm_init();

---
 target/i386/kvm/kvm.c | 11 +----------
 target/i386/kvm/tdx.c | 10 ++++++++++
 2 files changed, 11 insertions(+), 10 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 4/55
#15 lore raw

Re: [PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context

In reply to Xiaoyao Li
Quoted 13 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:14 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize
TDX VM context
X-Mailer: git-send-email 2.34.1

Implement TDX specific ConfidentialGuestSupportClass::kvm_init() callback, tdx_kvm_init().

Mark guest state is proctected for TDX VM. More TDX specific initialization will be added later.

Quoted 8 lines

---
Changes in v6:
- remove Acked-by from Gerd since the patch changed due to use ConfidentialGuestSupportClass::kvm_init();

---
 target/i386/kvm/kvm.c | 11 +----------
 target/i386/kvm/tdx.c | 10 ++++++++++
 2 files changed, 11 insertions(+), 10 deletions(-)
author
#16 lore raw

PATCH v8 5/55 i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

In reply to Xiaoyao Li

KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.

Since there is no interface reporting how many cpuid configs contains in KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceeds KVM_MAX_CPUID_ENTRIES.

Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope in preparation.

---
Changes in v7:
- refine and unifiy the error handling; (Daniel)

Changes in v6:
- Pass CPUState * to tdx_vcpu_ioctl();
- update commit message to remove platform scope thing;
- dump hw_error when it's non-zero to help debug;

Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;

Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();

changes in v2:
- Make the error message more clear;

changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;

---
 target/i386/kvm/kvm.c      |   2 -
 target/i386/kvm/kvm_i386.h |   2 +
 target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
 3 files changed, 108 insertions(+), 3 deletions(-)
target/i386/kvm/kvm.c +0 -2
index 1af4710556ad..b4fa35405fe1 100644
@@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
 
 static Error *invtsc_mig_blocker;
 
-#define KVM_MAX_CPUID_ENTRIES  100
-
 static void kvm_init_xsave(CPUX86State *env)
 {
     if (has_xsave2) {
target/i386/kvm/kvm_i386.h +2 -0
index 88565e8dbac1..ed1e61fb8ba9 100644
@@ -13,6 +13,8 @@
 
 #include "system/kvm.h"
 
+#define KVM_MAX_CPUID_ENTRIES  100
+
 /* always false if !CONFIG_KVM */
 #define kvm_pit_in_kernel() \
     (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
target/i386/kvm/tdx.c +106 -1

123 lines not on this page · raw message

Signature · 1 line
2.34.1
#17 lore raw

Re: [PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

Quoted 158 lines · Xiaoyao Li

KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.

Since there is no interface reporting how many cpuid configs contains in KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceeds KVM_MAX_CPUID_ENTRIES.

Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope in preparation.

---
Changes in v7:
- refine and unifiy the error handling; (Daniel)

Changes in v6:
- Pass CPUState * to tdx_vcpu_ioctl();
- update commit message to remove platform scope thing;
- dump hw_error when it's non-zero to help debug;

Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;

Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();

changes in v2:
- Make the error message more clear;

changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;

---
 target/i386/kvm/kvm.c      |   2 -
 target/i386/kvm/kvm_i386.h |   2 +
 target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
 3 files changed, 108 insertions(+), 3 deletions(-)
target/i386/kvm/kvm.c +0 -2
index 1af4710556ad..b4fa35405fe1 100644
@@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
 
 static Error *invtsc_mig_blocker;
 
-#define KVM_MAX_CPUID_ENTRIES  100
-
 static void kvm_init_xsave(CPUX86State *env)
 {
     if (has_xsave2) {
target/i386/kvm/kvm_i386.h +2 -0
index 88565e8dbac1..ed1e61fb8ba9 100644
@@ -13,6 +13,8 @@
 
 #include "system/kvm.h"
 
+#define KVM_MAX_CPUID_ENTRIES  100
+
 /* always false if !CONFIG_KVM */
 #define kvm_pit_in_kernel() \
     (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
target/i386/kvm/tdx.c +77 -0
index 4ff94860815d..c67be5e618e2 100644
@@ -10,17 +10,122 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
 #include "qom/object_interfaces.h"
 
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static struct kvm_tdx_capabilities *tdx_caps;
+
+enum tdx_ioctl_level {
+    TDX_VM_IOCTL,
+    TDX_VCPU_IOCTL,
+};
+
+static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
+                              int cmd_id, __u32 flags, void *data,
+                              Error **errp)
+{
+    struct kvm_tdx_cmd tdx_cmd = {};
+    int r;
+
+    const char *tdx_ioctl_name[] = {
+        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
+        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
+        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
+        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
+        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
+        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
+    };
+
+    tdx_cmd.id = cmd_id;
+    tdx_cmd.flags = flags;
+    tdx_cmd.data = (__u64)(unsigned long)data;
+
+    switch (level) {
+    case TDX_VM_IOCTL:
+        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    case TDX_VCPU_IOCTL:
+        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    default:
+        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
+        return -EINVAL;
+    }
+
+    if (r < 0) {
+        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
+                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
+    }
+    return r;
+}
+
+static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
+                               Error **errp)
+{
+    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
+}
+
+static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
+                                 void *data, Error **errp)
+{
+    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
+}
+
+static int get_tdx_capabilities(Error **errp)
+{
+    struct kvm_tdx_capabilities *caps;
+    /* 1st generation of TDX reports 6 cpuid configs */
+    int nr_cpuid_configs = 6;
+    size_t size;
+    int r;
+
+    do {
+        Error *local_err = NULL;
+        size = sizeof(struct kvm_tdx_capabilities) +
+                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
+        caps = g_malloc0(size);
+        caps->cpuid.nent = nr_cpuid_configs;
+
+        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
+        if (r == -E2BIG) {

target/i386/kvm/tdx.c line 105

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+            g_free(caps);
+            nr_cpuid_configs *= 2;
+            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
+                error_report("KVM TDX seems broken that number of CPUID entries"
+                             " in kvm_tdx_capabilities exceeds limit: %d",
+                             KVM_MAX_CPUID_ENTRIES);
+                error_propagate(errp, local_err);
+                return r;
+            }
+            error_free(local_err);

IIRC, you'll need 'local_err = NULL' here, otherwise next time around the loop 'local_err' will be pointing to a free'd error object which 'error_setg' will think is still valid & won't overwrite.

target/i386/kvm/tdx.c line 117

Quoted 12 lines
target/i386/kvm/tdx.c +12 -0
+        } else if (r < 0) {
+            g_free(caps);
+            error_propagate(errp, local_err);
+            return r;
+        }
+    } while (r == -E2BIG);
+
+    tdx_caps = caps;
+
+    return 0;
+}
+

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#18 lore raw

Re: [PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

Quoted 165 lines · Daniel P. Berrangé
Quoted 164 lines · Xiaoyao Li

KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.

Since there is no interface reporting how many cpuid configs contains in KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceeds KVM_MAX_CPUID_ENTRIES.

Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope in preparation.

---
Changes in v7:
- refine and unifiy the error handling; (Daniel)

Changes in v6:
- Pass CPUState * to tdx_vcpu_ioctl();
- update commit message to remove platform scope thing;
- dump hw_error when it's non-zero to help debug;

Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;

Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();

changes in v2:
- Make the error message more clear;

changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;

---
  target/i386/kvm/kvm.c      |   2 -
  target/i386/kvm/kvm_i386.h |   2 +
  target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
  3 files changed, 108 insertions(+), 3 deletions(-)
target/i386/kvm/kvm.c +0 -2
index 1af4710556ad..b4fa35405fe1 100644
@@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
 
  static Error *invtsc_mig_blocker;
 
-#define KVM_MAX_CPUID_ENTRIES  100
-
  static void kvm_init_xsave(CPUX86State *env)
  {
      if (has_xsave2) {
target/i386/kvm/kvm_i386.h +2 -0
index 88565e8dbac1..ed1e61fb8ba9 100644
@@ -13,6 +13,8 @@
 
  #include "system/kvm.h"
 
+#define KVM_MAX_CPUID_ENTRIES  100
+
  /* always false if !CONFIG_KVM */
  #define kvm_pit_in_kernel() \
      (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
target/i386/kvm/tdx.c +83 -0
index 4ff94860815d..c67be5e618e2 100644
@@ -10,17 +10,122 @@
   */
 
  #include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
  #include "qom/object_interfaces.h"
 
  #include "hw/i386/x86.h"
  #include "kvm_i386.h"
  #include "tdx.h"
 
+static struct kvm_tdx_capabilities *tdx_caps;
+
+enum tdx_ioctl_level {
+    TDX_VM_IOCTL,
+    TDX_VCPU_IOCTL,
+};
+
+static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
+                              int cmd_id, __u32 flags, void *data,
+                              Error **errp)
+{
+    struct kvm_tdx_cmd tdx_cmd = {};
+    int r;
+
+    const char *tdx_ioctl_name[] = {
+        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
+        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
+        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
+        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
+        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
+        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
+    };
+
+    tdx_cmd.id = cmd_id;
+    tdx_cmd.flags = flags;
+    tdx_cmd.data = (__u64)(unsigned long)data;
+
+    switch (level) {
+    case TDX_VM_IOCTL:
+        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    case TDX_VCPU_IOCTL:
+        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    default:
+        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
+        return -EINVAL;
+    }
+
+    if (r < 0) {
+        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
+                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
+    }
+    return r;
+}
+
+static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
+                               Error **errp)
+{
+    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
+}
+
+static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
+                                 void *data, Error **errp)
+{
+    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
+}
+
+static int get_tdx_capabilities(Error **errp)
+{
+    struct kvm_tdx_capabilities *caps;
+    /* 1st generation of TDX reports 6 cpuid configs */
+    int nr_cpuid_configs = 6;
+    size_t size;
+    int r;
+
+    do {
+        Error *local_err = NULL;
+        size = sizeof(struct kvm_tdx_capabilities) +
+                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
+        caps = g_malloc0(size);
+        caps->cpuid.nent = nr_cpuid_configs;
+
+        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
+        if (r == -E2BIG) {
+            g_free(caps);
+            nr_cpuid_configs *= 2;
+            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
+                error_report("KVM TDX seems broken that number of CPUID entries"
+                             " in kvm_tdx_capabilities exceeds limit: %d",
+                             KVM_MAX_CPUID_ENTRIES);
Quoted 8 lines

target/i386/kvm/tdx.c line 105

Quoted 4 lines
target/i386/kvm/tdx.c +4 -0
+                error_propagate(errp, local_err);
+                return r;
+            }
+            error_free(local_err);

IIRC, you'll need 'local_err = NULL' here, otherwise next time around the loop 'local_err' will be pointing to a free'd error object which 'error_setg' will think is still valid & won't overwrite.

it's set to NULL at the beginning of the do loop.

(It looks I need to declare Error *local_err outside the do loop.)

Quoted 16 lines

target/i386/kvm/tdx.c line 117

Quoted 12 lines
target/i386/kvm/tdx.c +12 -0
+        } else if (r < 0) {
+            g_free(caps);
+            error_propagate(errp, local_err);
+            return r;
+        }
+    } while (r == -E2BIG);
+
+    tdx_caps = caps;
+
+    return 0;
+}
+

With regards,
Daniel

#19 lore raw

Re: [PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

Quoted 168 lines · Xiaoyao Li
Quoted 158 lines · Daniel P. Berrangé
Quoted 157 lines · Xiaoyao Li

KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing TDX context. It will be used to validate user's setting later.

Since there is no interface reporting how many cpuid configs contains in KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number and abort when it exceeds KVM_MAX_CPUID_ENTRIES.

Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope in preparation.

---
Changes in v7:
- refine and unifiy the error handling; (Daniel)

Changes in v6:
- Pass CPUState * to tdx_vcpu_ioctl();
- update commit message to remove platform scope thing;
- dump hw_error when it's non-zero to help debug;

Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;

Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();

changes in v2:
- Make the error message more clear;

changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;

---
  target/i386/kvm/kvm.c      |   2 -
  target/i386/kvm/kvm_i386.h |   2 +
  target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
  3 files changed, 108 insertions(+), 3 deletions(-)
target/i386/kvm/kvm.c +0 -2
index 1af4710556ad..b4fa35405fe1 100644
@@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
  static Error *invtsc_mig_blocker;
-#define KVM_MAX_CPUID_ENTRIES  100
-
  static void kvm_init_xsave(CPUX86State *env)
  {
      if (has_xsave2) {
target/i386/kvm/kvm_i386.h +2 -0
index 88565e8dbac1..ed1e61fb8ba9 100644
@@ -13,6 +13,8 @@
  #include "system/kvm.h"
+#define KVM_MAX_CPUID_ENTRIES  100
+
  /* always false if !CONFIG_KVM */
  #define kvm_pit_in_kernel() \
      (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
target/i386/kvm/tdx.c +83 -0
index 4ff94860815d..c67be5e618e2 100644
@@ -10,17 +10,122 @@
   */
  #include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
  #include "qom/object_interfaces.h"
  #include "hw/i386/x86.h"
  #include "kvm_i386.h"
  #include "tdx.h"
+static struct kvm_tdx_capabilities *tdx_caps;
+
+enum tdx_ioctl_level {
+    TDX_VM_IOCTL,
+    TDX_VCPU_IOCTL,
+};
+
+static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
+                              int cmd_id, __u32 flags, void *data,
+                              Error **errp)
+{
+    struct kvm_tdx_cmd tdx_cmd = {};
+    int r;
+
+    const char *tdx_ioctl_name[] = {
+        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
+        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
+        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
+        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
+        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
+        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
+    };
+
+    tdx_cmd.id = cmd_id;
+    tdx_cmd.flags = flags;
+    tdx_cmd.data = (__u64)(unsigned long)data;
+
+    switch (level) {
+    case TDX_VM_IOCTL:
+        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    case TDX_VCPU_IOCTL:
+        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    default:
+        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
+        return -EINVAL;
+    }
+
+    if (r < 0) {
+        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
+                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
+    }
+    return r;
+}
+
+static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
+                               Error **errp)
+{
+    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
+}
+
+static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
+                                 void *data, Error **errp)
+{
+    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
+}
+
+static int get_tdx_capabilities(Error **errp)
+{
+    struct kvm_tdx_capabilities *caps;
+    /* 1st generation of TDX reports 6 cpuid configs */
+    int nr_cpuid_configs = 6;
+    size_t size;
+    int r;
+
+    do {
+        Error *local_err = NULL;
+        size = sizeof(struct kvm_tdx_capabilities) +
+                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
+        caps = g_malloc0(size);
+        caps->cpuid.nent = nr_cpuid_configs;
+
+        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
+        if (r == -E2BIG) {
+            g_free(caps);
+            nr_cpuid_configs *= 2;
+            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
+                error_report("KVM TDX seems broken that number of CPUID entries"
+                             " in kvm_tdx_capabilities exceeds limit: %d",
+                             KVM_MAX_CPUID_ENTRIES);
Quoted 8 lines

target/i386/kvm/tdx.c line 105

Quoted 4 lines
target/i386/kvm/tdx.c +4 -0
+                error_propagate(errp, local_err);
+                return r;
+            }
+            error_free(local_err);

IIRC, you'll need 'local_err = NULL' here, otherwise next time around the loop 'local_err' will be pointing to a free'd error object which 'error_setg' will think is still valid & won't overwrite.

Quoted 1 line

it's set to NULL at the beginning of the do loop.

Oh yes, i missed that, your code is fine as is then.

Quoted 1 line

(It looks I need to declare Error *local_err outside the do loop.)

Nah, it is fine within the loop given nothing outside needs it.

Quoted 18 lines
Quoted 17 lines

target/i386/kvm/tdx.c line 117

Quoted 13 lines
target/i386/kvm/tdx.c +12 -0
+        } else if (r < 0) {
+            g_free(caps);
+            error_propagate(errp, local_err);
+            return r;
+        }
+    } while (r == -E2BIG);
+
+    tdx_caps = caps;
+
+    return 0;
+}
+

With regards,
Daniel

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#20 lore raw

PATCH v8 6/55 i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

In reply to Xiaoyao Li

It will need special handling for TDX VMs all around the QEMU. Introduce is_tdx_vm() helper to query if it's a TDX VM.

Cache tdx_guest object thus no need to cast from ms->cgs every time.

---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();

---
 target/i386/kvm/tdx.c | 15 ++++++++++++++-
 target/i386/kvm/tdx.h | 10 ++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +14 -1
index c67be5e618e2..16f67e18ae78 100644
@@ -18,8 +18,16 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static TdxGuest *tdx_guest;
+
 static struct kvm_tdx_capabilities *tdx_caps;
 
+/* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
+bool is_tdx_vm(void)
+{
+    return !!tdx_guest;
+}
+
 enum tdx_ioctl_level {
     TDX_VM_IOCTL,
     TDX_VCPU_IOCTL,
@@ -117,15 +125,20 @@ static int get_tdx_capabilities(Error **errp)
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
+    TdxGuest *tdx = TDX_GUEST(cgs);
     int r = 0;
 
     kvm_mark_guest_state_protected();
 
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
+        if (r) {
+            return r;
+        }
     }
 
-    return r;
+    tdx_guest = tdx;
+    return 0;
 }
 
 static int tdx_kvm_type(X86ConfidentialGuest *cg)
target/i386/kvm/tdx.h +10 -0
index f3b725336161..de8ae9196163 100644
@@ -3,6 +3,10 @@
 #ifndef QEMU_I386_TDX_H
 #define QEMU_I386_TDX_H
 
+#ifndef CONFIG_USER_ONLY
+#include CONFIG_DEVICES /* CONFIG_TDX */
+#endif
+
 #include "confidential-guest.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"
@@ -18,4 +22,10 @@ typedef struct TdxGuest {
     uint64_t attributes;    /* TD attributes */
 } TdxGuest;
 
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+
 #endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Reviewed-by on v8 6/55
#21 lore raw

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

Quoted 81 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:16 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache
tdx_guest object
X-Mailer: git-send-email 2.34.1

It will need special handling for TDX VMs all around the QEMU. Introduce is_tdx_vm() helper to query if it's a TDX VM.

Cache tdx_guest object thus no need to cast from ms->cgs every time.

---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();

---
 target/i386/kvm/tdx.c | 15 ++++++++++++++-
 target/i386/kvm/tdx.h | 10 ++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +14 -1
index c67be5e618e2..16f67e18ae78 100644
@@ -18,8 +18,16 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static TdxGuest *tdx_guest;
+
 static struct kvm_tdx_capabilities *tdx_caps;
 
+/* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
+bool is_tdx_vm(void)
+{
+    return !!tdx_guest;
+}
+
 enum tdx_ioctl_level {
     TDX_VM_IOCTL,
     TDX_VCPU_IOCTL,
@@ -117,15 +125,20 @@ static int get_tdx_capabilities(Error **errp)
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
+    TdxGuest *tdx = TDX_GUEST(cgs);
     int r = 0;
 
     kvm_mark_guest_state_protected();
 
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
+        if (r) {
+            return r;
+        }
     }
 
-    return r;
+    tdx_guest = tdx;
+    return 0;
 }
 
 static int tdx_kvm_type(X86ConfidentialGuest *cg)
target/i386/kvm/tdx.h +4 -0
index f3b725336161..de8ae9196163 100644
@@ -3,6 +3,10 @@
 #ifndef QEMU_I386_TDX_H
 #define QEMU_I386_TDX_H
 
+#ifndef CONFIG_USER_ONLY
+#include CONFIG_DEVICES /* CONFIG_TDX */
+#endif
+
 #include "confidential-guest.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"

target/i386/kvm/tdx.h typedef struct TdxGuest { · line 30

Quoted 9 lines
target/i386/kvm/tdx.h +6 -0
@@ -18,4 +22,10 @@ typedef struct TdxGuest {
     uint64_t attributes;    /* TD attributes */
 } TdxGuest;
 
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+

a little nit: could we rename it as "tdx_enabled"?

Then the cases like these would be neater?

if (sev_enabled() || is_tdx_vm()) {
    ...
}
if (sev_enabled()) {
    ...
} else if (is_tdx_vm()) {
    ...
}

Otherwise,

Thanks,
Zhao

author
#22 lore raw

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

Quoted 94 lines · Zhao Liu
Quoted 81 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:16 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache
tdx_guest object
X-Mailer: git-send-email 2.34.1

It will need special handling for TDX VMs all around the QEMU. Introduce is_tdx_vm() helper to query if it's a TDX VM.

Cache tdx_guest object thus no need to cast from ms->cgs every time.

---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();

---
  target/i386/kvm/tdx.c | 15 ++++++++++++++-
  target/i386/kvm/tdx.h | 10 ++++++++++
  2 files changed, 24 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +14 -1
index c67be5e618e2..16f67e18ae78 100644
@@ -18,8 +18,16 @@
  #include "kvm_i386.h"
  #include "tdx.h"
 
+static TdxGuest *tdx_guest;
+
  static struct kvm_tdx_capabilities *tdx_caps;
 
+/* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
+bool is_tdx_vm(void)
+{
+    return !!tdx_guest;
+}
+
  enum tdx_ioctl_level {
      TDX_VM_IOCTL,
      TDX_VCPU_IOCTL,
@@ -117,15 +125,20 @@ static int get_tdx_capabilities(Error **errp)
 
  static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
  {
+    TdxGuest *tdx = TDX_GUEST(cgs);
      int r = 0;
 
      kvm_mark_guest_state_protected();
 
      if (!tdx_caps) {
          r = get_tdx_capabilities(errp);
+        if (r) {
+            return r;
+        }
      }
 
-    return r;
+    tdx_guest = tdx;
+    return 0;
  }
 
  static int tdx_kvm_type(X86ConfidentialGuest *cg)
target/i386/kvm/tdx.h +4 -0
index f3b725336161..de8ae9196163 100644
@@ -3,6 +3,10 @@
  #ifndef QEMU_I386_TDX_H
  #define QEMU_I386_TDX_H
 
+#ifndef CONFIG_USER_ONLY
+#include CONFIG_DEVICES /* CONFIG_TDX */
+#endif
+
  #include "confidential-guest.h"
 
  #define TYPE_TDX_GUEST "tdx-guest"

target/i386/kvm/tdx.h typedef struct TdxGuest { · line 30

Quoted 9 lines
target/i386/kvm/tdx.h +6 -0
@@ -18,4 +22,10 @@ typedef struct TdxGuest {
      uint64_t attributes;    /* TD attributes */
  } TdxGuest;
 
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+

a little nit: could we rename it as "tdx_enabled"?

Quoted 1 line

Then the cases like these would be neater?

When sev support was added, it was seen as a feature for the VMs that are created on AMD platform. I think that's why it got called sev_enabled().

But for TDX, it is introduced as a different type of VM in contrast to the legacy/normal VMX VMs. We need to pass specific TDX vm type to KVM_CREATE_VM. Based on this, is_tdx_vm() was chosen.

I don't think the different name is a big issue, as nobody mentions it from the initial RFC to current v8 until you. So again, I will just keep it unchanged unless someone objects it strongly.

Quoted 11 lines
if (sev_enabled() || is_tdx_vm()) {
     ...
}
if (sev_enabled()) {
     ...
} else if (is_tdx_vm()) {
     ...
}
Quoted 3 lines

Otherwise,

Thanks!

Quoted 4 lines

Thanks,
Zhao

#23 lore raw

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

Quoted 8 lines
Quoted 8 lines

target/i386/kvm/tdx.h typedef struct TdxGuest { · line 30

Quoted 6 lines
target/i386/kvm/tdx.h +6 -0
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+

a little nit: could we rename it as "tdx_enabled"?

Quoted 8 lines
Quoted 1 line

Then the cases like these would be neater?

When sev support was added, it was seen as a feature for the VMs that are created on AMD platform. I think that's why it got called sev_enabled().

But for TDX, it is introduced as a different type of VM in contrast to the legacy/normal VMX VMs. We need to pass specific TDX vm type to KVM_CREATE_VM. Based on this, is_tdx_vm() was chosen.

But isn't AMD's SEV also defined as the VM type?

static const char *vm_type_name[] = {
    [KVM_X86_DEFAULT_VM] = "default",
    [KVM_X86_SEV_VM] = "SEV",
    [KVM_X86_SEV_ES_VM] = "SEV-ES",
    [KVM_X86_SNP_VM] = "SEV-SNP",
    [KVM_X86_TDX_VM] = "TDX",
};

Functionally, they are part of the coco functionality provided by different vendors, so it's better thatt both could be in the same place as much as possible, including file location, naming style. Of course, it's not a big deal, and it can be cleaned up after merge if needed.

Quoted 2 lines

I don't think the different name is a big issue, as nobody mentions it from the initial RFC to current v8 until you.

The Chinese saying: There are a thousand Hamlets in a thousand people's eyes :-).

author
#24 lore raw

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

Quoted 10 lines · Zhao Liu
Quoted 10 lines
Quoted 9 lines

target/i386/kvm/tdx.h typedef struct TdxGuest { · line 30

Quoted 6 lines
target/i386/kvm/tdx.h +6 -0
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+

a little nit: could we rename it as "tdx_enabled"?

Then the cases like these would be neater?

Quoted 8 lines
Quoted 6 lines

When sev support was added, it was seen as a feature for the VMs that are created on AMD platform. I think that's why it got called sev_enabled().

But for TDX, it is introduced as a different type of VM in contrast to the legacy/normal VMX VMs. We need to pass specific TDX vm type to KVM_CREATE_VM. Based on this, is_tdx_vm() was chosen.

But isn't AMD's SEV also defined as the VM type?

The initial SEV support that introduced sev_eanbled(), didn't introduce the VM type.

The specific type for SEV* was added later.

Quoted 8 lines
static const char *vm_type_name[] = {
     [KVM_X86_DEFAULT_VM] = "default",
     [KVM_X86_SEV_VM] = "SEV",
     [KVM_X86_SEV_ES_VM] = "SEV-ES",
     [KVM_X86_SNP_VM] = "SEV-SNP",
     [KVM_X86_TDX_VM] = "TDX",
};
Quoted 4 lines

Functionally, they are part of the coco functionality provided by different vendors, so it's better thatt both could be in the same place as much as possible, including file location, naming style. Of course, it's not a big deal, and it can be cleaned up after merge if needed.

yes, cleanup can be a separate work if Paolo doesn't dislike.

Quoted 6 lines
Quoted 2 lines

I don't think the different name is a big issue, as nobody mentions it from the initial RFC to current v8 until you.

The Chinese saying: There are a thousand Hamlets in a thousand people's eyes :-).

author
#25 lore raw

PATCH v8 7/55 kvm: Introduce kvm_arch_pre_create_vcpu()

In reply to Xiaoyao Li

Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent work prior to create any vcpu. This is for i386 TDX because it needs call TDX_INIT_VM before creating any vcpu.

The specific implemnet of i386 will be added in the future patch.

---
Changes in v7:
- Implement stub for all the ARCHes instead of defining it with weak attribute; (Philippe)

Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)

---
 accel/kvm/kvm-all.c        | 5 +++++
 include/system/kvm.h       | 1 +
 target/arm/kvm.c           | 5 +++++
 target/i386/kvm/kvm.c      | 5 +++++
 target/loongarch/kvm/kvm.c | 4 ++++
 target/mips/kvm.c          | 5 +++++
 target/ppc/kvm.c           | 5 +++++
 target/riscv/kvm/kvm-cpu.c | 5 +++++
 target/s390x/kvm/kvm.c     | 5 +++++
 9 files changed, 40 insertions(+)
accel/kvm/kvm-all.c +5 -0
index f89568bfa397..df9840e53a35 100644
@@ -540,6 +540,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
 
     trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
 
+    ret = kvm_arch_pre_create_vcpu(cpu, errp);
+    if (ret < 0) {
+        goto err;
+    }
+
     ret = kvm_create_vcpu(cpu);
     if (ret < 0) {
         error_setg_errno(errp, -ret,
include/system/kvm.h +1 -0
index ab17c09a551f..d7dfa25493a2 100644
@@ -374,6 +374,7 @@ int kvm_arch_get_default_type(MachineState *ms);
 
 int kvm_arch_init(MachineState *ms, KVMState *s);
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp);
 int kvm_arch_init_vcpu(CPUState *cpu);
 int kvm_arch_destroy_vcpu(CPUState *cpu);
 
target/arm/kvm.c +5 -0
index da30bdbb2349..93f1a7245b3f 100644
@@ -1874,6 +1874,11 @@ static int kvm_arm_sve_set_vls(ARMCPU *cpu)
 
 #define ARM_CPU_ID_MPIDR       3, 0, 0, 0, 5
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     int ret;
target/i386/kvm/kvm.c +5 -0
index b4fa35405fe1..1a4dd19e24ab 100644
@@ -2050,6 +2050,11 @@ full:
     abort();
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     struct {
target/loongarch/kvm/kvm.c +4 -0
index 7f63e7c8fe51..5e82dacc677b 100644
@@ -1071,7 +1071,11 @@ static int kvm_cpu_check_pv_features(CPUState *cs, Error **errp)
             env->pv_features |= BIT(KVM_FEATURE_VIRT_EXTIOI);
         }
     }
+    return 0;
+}
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
     return 0;
 }
 
target/mips/kvm.c +5 -0
index d67b7c1a8ecb..ec53acb51a1f 100644
@@ -61,6 +61,11 @@ int kvm_arch_irqchip_create(KVMState *s)
     return 0;
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     CPUMIPSState *env = cpu_env(cs);
target/ppc/kvm.c +5 -0
index 992356cb7593..20fabccecd54 100644
@@ -479,6 +479,11 @@ static void kvmppc_hw_debug_points_init(CPUPPCState *cenv)
     }
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
target/riscv/kvm/kvm-cpu.c +5 -0
index 4ffeeaa1c953..451c00f17c2f 100644
@@ -1389,6 +1389,11 @@ static int kvm_vcpu_enable_sbi_dbcn(RISCVCPU *cpu, CPUState *cs)
     return kvm_set_one_reg(cs, kvm_sbi_dbcn.kvm_reg_id, &reg);
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     int ret = 0;
target/s390x/kvm/kvm.c +5 -0
index 4d56e653ddf6..1f592733f4e2 100644
@@ -404,6 +404,11 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
     return cpu->cpu_index;
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
Signature · 1 line
2.34.1
Reviewed-by on v8 7/55
#26 lore raw

Re: [PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()

Quoted 5 lines · Xiaoyao Li

Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent work prior to create any vcpu. This is for i386 TDX because it needs call TDX_INIT_VM before creating any vcpu.

The specific implemnet of i386 will be added in the future patch.

s/implemnet of/implementation for'

Quoted 10 lines

---
Changes in v7:
- Implement stub for all the ARCHes instead of defining it with weak attribute; (Philippe)

Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)

Quoted 11 lines
---
 accel/kvm/kvm-all.c        | 5 +++++
 include/system/kvm.h       | 1 +
 target/arm/kvm.c           | 5 +++++
 target/i386/kvm/kvm.c      | 5 +++++
 target/loongarch/kvm/kvm.c | 4 ++++
 target/mips/kvm.c          | 5 +++++
 target/ppc/kvm.c           | 5 +++++
 target/riscv/kvm/kvm-cpu.c | 5 +++++
 target/s390x/kvm/kvm.c     | 5 +++++
 9 files changed, 40 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 7/55
#27 lore raw

Re: [PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()

In reply to Xiaoyao Li
Quoted 20 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:17 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()
X-Mailer: git-send-email 2.34.1

Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent work prior to create any vcpu. This is for i386 TDX because it needs call TDX_INIT_VM before creating any vcpu.

The specific implemnet of i386 will be added in the future patch.

---
Changes in v7:
- Implement stub for all the ARCHes instead of defining it with weak attribute; (Philippe)

Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)

Quoted 11 lines
---
 accel/kvm/kvm-all.c        | 5 +++++
 include/system/kvm.h       | 1 +
 target/arm/kvm.c           | 5 +++++
 target/i386/kvm/kvm.c      | 5 +++++
 target/loongarch/kvm/kvm.c | 4 ++++
 target/mips/kvm.c          | 5 +++++
 target/ppc/kvm.c           | 5 +++++
 target/riscv/kvm/kvm-cpu.c | 5 +++++
 target/s390x/kvm/kvm.c     | 5 +++++
 9 files changed, 40 insertions(+)
author
#28 lore raw

PATCH v8 8/55 i386/tdx: Initialize TDX before creating TD vcpus

In reply to Xiaoyao Li

Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs.

Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.

Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init().

---
Changes in v8:
- Drop the code that initializes cpu->kvm_state before kvm_arch_pre_create_vcpu() because it's not needed anymore.

Changes in v7:
- Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
- Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;

Changes in v6:
- setup xfam explicitly to fit with new uapi;
- use tdx_caps->cpuid to filter the input of cpuids because now KVM only allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;

Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)

---
 target/i386/kvm/kvm.c       |  16 +++---
 target/i386/kvm/kvm_i386.h  |   5 ++
 target/i386/kvm/meson.build |   2 +-
 target/i386/kvm/tdx-stub.c  |  10 ++++
 target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h       |   6 +++
 6 files changed, 137 insertions(+), 7 deletions(-)
 create mode 100644 target/i386/kvm/tdx-stub.c
target/i386/kvm/kvm.c +10 -6
index 1a4dd19e24ab..a537699bb7df 100644
@@ -38,6 +38,7 @@
 #include "kvm_i386.h"
 #include "../confidential-guest.h"
 #include "sev.h"
+#include "tdx.h"
 #include "xen-emu.h"
 #include "hyperv.h"
 #include "hyperv-proto.h"
@@ -414,9 +415,9 @@ static uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg)
 
 /* Find matching entry for function/index on kvm_cpuid2 struct
  */
-static struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
-                                                 uint32_t function,
-                                                 uint32_t index)
+struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
+                                          uint32_t function,
+                                          uint32_t index)
 {
     int i;
     for (i = 0; i < cpuid->nent; ++i) {
@@ -1821,9 +1822,8 @@ static void kvm_init_nested_state(CPUX86State *env)
     }
 }
 
-static uint32_t kvm_x86_build_cpuid(CPUX86State *env,
-                                    struct kvm_cpuid_entry2 *entries,
-                                    uint32_t cpuid_i)
+uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
+                             uint32_t cpuid_i)
 {
     uint32_t limit, i, j;
     uint32_t unused;
@@ -2052,6 +2052,10 @@ full:
 
 int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
 {
+    if (is_tdx_vm()) {
+        return tdx_pre_create_vcpu(cpu, errp);
+    }
+
     return 0;
 }
 
target/i386/kvm/kvm_i386.h +5 -0
index ed1e61fb8ba9..dc696cb7238a 100644
@@ -59,6 +59,11 @@ uint64_t kvm_swizzle_msi_ext_dest_id(uint64_t address);
 void kvm_update_msi_routes_all(void *private, bool global,
                                uint32_t index, uint32_t mask);
 
+struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
+                                          uint32_t function,
+                                          uint32_t index);
+uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
+                             uint32_t cpuid_i);
 #endif /* CONFIG_KVM */
 
 void kvm_pc_setup_irq_routing(bool pci_enabled);
target/i386/kvm/meson.build +1 -1
index 466bccb9cb17..3f44cdedb758 100644
@@ -8,7 +8,7 @@ i386_kvm_ss.add(files(
 
 i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
 
-i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'))
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c'))
 
 i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
 
target/i386/kvm/tdx-stub.c new file +10 -0
new file mode 100644
index 000000000000..2344433594ea
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+
+#include "tdx.h"
+
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return -EINVAL;
+}
target/i386/kvm/tdx.c +105 -0

117 lines not on this page · raw message

target/i386/kvm/tdx.h +6 -0
index de8ae9196163..4e2b5c61ff5b 100644
@@ -19,7 +19,11 @@ typedef struct TdxGuestClass {
 typedef struct TdxGuest {
     X86ConfidentialGuest parent_obj;
 
+    QemuMutex lock;
+
+    bool initialized;
     uint64_t attributes;    /* TD attributes */
+    uint64_t xfam;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
@@ -28,4 +32,6 @@ bool is_tdx_vm(void);
 #define is_tdx_vm() 0
 #endif /* CONFIG_TDX */
 
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
+
 #endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
#29 lore raw

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

Quoted 85 lines · Xiaoyao Li

Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs.

Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.

Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init().

---
Changes in v8:
- Drop the code that initializes cpu->kvm_state before kvm_arch_pre_create_vcpu() because it's not needed anymore.

Changes in v7:
- Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
- Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;

Changes in v6:
- setup xfam explicitly to fit with new uapi;
- use tdx_caps->cpuid to filter the input of cpuids because now KVM only allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;

Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)

---
 target/i386/kvm/kvm.c       |  16 +++---
 target/i386/kvm/kvm_i386.h  |   5 ++
 target/i386/kvm/meson.build |   2 +-
 target/i386/kvm/tdx-stub.c  |  10 ++++
 target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h       |   6 +++
 6 files changed, 137 insertions(+), 7 deletions(-)
 create mode 100644 target/i386/kvm/tdx-stub.c
target/i386/kvm/tdx.c +37 -0
index 16f67e18ae78..0afaf739c09f 100644
 
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+    Error *local_err = NULL;
+    int retry = 10000;
+    int r = 0;
+
+    QEMU_LOCK_GUARD(&tdx_guest->lock);
+    if (tdx_guest->initialized) {
+        return r;
+    }
+
+    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
+                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+
+    r = setup_td_xfam(x86cpu, errp);
+    if (r) {
+        return r;
+    }
+
+    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
+    tdx_filter_cpuid(&init_vm->cpuid);
+
+    init_vm->attributes = tdx_guest->attributes;
+    init_vm->xfam = tdx_guest->xfam;
+
+    /*
+     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
+     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
+     * RDSEED) is busy.
+     *
+     * Retry for the case.
+     */
+    do {
+        error_free(local_err);

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 244

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        local_err = NULL;
+        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
+    } while (r == -EAGAIN && --retry);
+
+    if (r < 0) {
+        if (!retry) {
+            error_report("Hardware RNG (Random Number Generator) is busy "
+                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
+                         "which leads to KVM_TDX_INIT_VM keeping failure "
+                         "due to lack of entropy.");

This needs to be

     error_append_hint(local_err, ....);

so that this message gets associated with the error object that is propagated, and the top level will print it all at once.

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 253

Quoted 9 lines
target/i386/kvm/tdx.c +9 -0
+        }
+        error_propagate(errp, local_err);
+        return r;
+    }
+
+    tdx_guest->initialized = true;
+
+    return 0;
+}

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#30 lore raw

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

Quoted 99 lines · Daniel P. Berrangé
Quoted 85 lines · Xiaoyao Li

Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs.

Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.

Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init().

---
Changes in v8:
- Drop the code that initializes cpu->kvm_state before kvm_arch_pre_create_vcpu() because it's not needed anymore.

Changes in v7:
- Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
- Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;

Changes in v6:
- setup xfam explicitly to fit with new uapi;
- use tdx_caps->cpuid to filter the input of cpuids because now KVM only allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;

Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)

---
  target/i386/kvm/kvm.c       |  16 +++---
  target/i386/kvm/kvm_i386.h  |   5 ++
  target/i386/kvm/meson.build |   2 +-
  target/i386/kvm/tdx-stub.c  |  10 ++++
  target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
  target/i386/kvm/tdx.h       |   6 +++
  6 files changed, 137 insertions(+), 7 deletions(-)

create mode 100644 target/i386/kvm/tdx-stub.c

target/i386/kvm/tdx.c +37 -0
index 16f67e18ae78..0afaf739c09f 100644
 
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+    Error *local_err = NULL;
+    int retry = 10000;
+    int r = 0;
+
+    QEMU_LOCK_GUARD(&tdx_guest->lock);
+    if (tdx_guest->initialized) {
+        return r;
+    }
+
+    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
+                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+
+    r = setup_td_xfam(x86cpu, errp);
+    if (r) {
+        return r;
+    }
+
+    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
+    tdx_filter_cpuid(&init_vm->cpuid);
+
+    init_vm->attributes = tdx_guest->attributes;
+    init_vm->xfam = tdx_guest->xfam;
+
+    /*
+     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
+     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
+     * RDSEED) is busy.
+     *
+     * Retry for the case.
+     */
+    do {
+        error_free(local_err);

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 244

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        local_err = NULL;
+        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
+    } while (r == -EAGAIN && --retry);
+
+    if (r < 0) {
+        if (!retry) {
+            error_report("Hardware RNG (Random Number Generator) is busy "
+                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
+                         "which leads to KVM_TDX_INIT_VM keeping failure "
+                         "due to lack of entropy.");

This needs to be

Quoted 4 lines
      error_append_hint(local_err, ....);

so that this message gets associated with the error object that is propagated, and the top level will print it all at once.

Good suggestion! Will change to it in the next version.

Quoted 12 lines

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 253

Quoted 9 lines
target/i386/kvm/tdx.c +9 -0
+        }
+        error_propagate(errp, local_err);
+        return r;
+    }
+
+    tdx_guest->initialized = true;
+
+    return 0;
+}

With regards,
Daniel

#31 lore raw

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

Quoted 50 lines
Quoted 50 lines
Quoted 37 lines
target/i386/kvm/tdx.c +37 -0
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+    Error *local_err = NULL;
+    int retry = 10000;
+    int r = 0;
+
+    QEMU_LOCK_GUARD(&tdx_guest->lock);
+    if (tdx_guest->initialized) {
+        return r;
+    }
+
+    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
+                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+
+    r = setup_td_xfam(x86cpu, errp);
+    if (r) {
+        return r;
+    }
+
+    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
+    tdx_filter_cpuid(&init_vm->cpuid);
+
+    init_vm->attributes = tdx_guest->attributes;
+    init_vm->xfam = tdx_guest->xfam;
+
+    /*
+     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
+     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
+     * RDSEED) is busy.
+     *
+     * Retry for the case.
+     */
+    do {
+        error_free(local_err);

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 244

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        local_err = NULL;
+        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
+    } while (r == -EAGAIN && --retry);
+
+    if (r < 0) {
+        if (!retry) {
+            error_report("Hardware RNG (Random Number Generator) is busy "
+                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
+                         "which leads to KVM_TDX_INIT_VM keeping failure "
+                         "due to lack of entropy.");

This needs to be

Quoted 6 lines
Quoted 4 lines
      error_append_hint(local_err, ....);

so that this message gets associated with the error object that is propagated, and the top level will print it all at once.

Good suggestion! Will change to it in the next version.

A little suggestion:

With error_append_hint(local_err, ...), you can add "ERRP_GUARD()" at the beginning of tdx_pre_create_vcpu(), just like the commit 95e9053a34ca.

author
#32 lore raw

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

Quoted 55 lines · Zhao Liu
Quoted 55 lines
Quoted 54 lines
Quoted 37 lines
target/i386/kvm/tdx.c +37 -0
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+    Error *local_err = NULL;
+    int retry = 10000;
+    int r = 0;
+
+    QEMU_LOCK_GUARD(&tdx_guest->lock);
+    if (tdx_guest->initialized) {
+        return r;
+    }
+
+    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
+                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+
+    r = setup_td_xfam(x86cpu, errp);
+    if (r) {
+        return r;
+    }
+
+    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
+    tdx_filter_cpuid(&init_vm->cpuid);
+
+    init_vm->attributes = tdx_guest->attributes;
+    init_vm->xfam = tdx_guest->xfam;
+
+    /*
+     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
+     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
+     * RDSEED) is busy.
+     *
+     * Retry for the case.
+     */
+    do {
+        error_free(local_err);

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 244

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        local_err = NULL;
+        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
+    } while (r == -EAGAIN && --retry);
+
+    if (r < 0) {
+        if (!retry) {
+            error_report("Hardware RNG (Random Number Generator) is busy "
+                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
+                         "which leads to KVM_TDX_INIT_VM keeping failure "
+                         "due to lack of entropy.");

This needs to be

       error_append_hint(local_err, ....);

so that this message gets associated with the error object that is propagated, and the top level will print it all at once.

Quoted 6 lines
Quoted 1 line

Good suggestion! Will change to it in the next version.

A little suggestion:

With error_append_hint(local_err, ...), you can add "ERRP_GUARD()" at the beginning of tdx_pre_create_vcpu(), just like the commit 95e9053a34ca.

I don't think ERRP_GUARD() is needed.

ERRP_GUARD() is used to guard @errp, while here error_append_hint() is used on @local_err.

#33 lore raw

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

Quoted 9 lines
Quoted 4 lines

A little suggestion:

With error_append_hint(local_err, ...), you can add "ERRP_GUARD()" at the beginning of tdx_pre_create_vcpu(), just like the commit 95e9053a34ca.

I don't think ERRP_GUARD() is needed.

ERRP_GUARD() is used to guard @errp, while here error_append_hint() is used on @local_err.

Yes, you're right. My bad and thanks for correction!

Reviewed-by on v8 8/55
#34 lore raw

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

In reply to Xiaoyao Li
Quoted 37 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:18 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus
X-Mailer: git-send-email 2.34.1

Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical CPUID config, and must be executed prior to creating vCPUs.

Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.

Note, this doesn't address the fact that QEMU may change the CPUID configuration when creating vCPUs, i.e. punts on refactoring QEMU to provide a stable CPUID config prior to kvm_arch_init().

---
Changes in v8:
- Drop the code that initializes cpu->kvm_state before kvm_arch_pre_create_vcpu() because it's not needed anymore.

Changes in v7:
- Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
- Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;

Changes in v6:
- setup xfam explicitly to fit with new uapi;
- use tdx_caps->cpuid to filter the input of cpuids because now KVM only allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;

Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)

Quoted 9 lines
---
 target/i386/kvm/kvm.c       |  16 +++---
 target/i386/kvm/kvm_i386.h  |   5 ++
 target/i386/kvm/meson.build |   2 +-
 target/i386/kvm/tdx-stub.c  |  10 ++++
 target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h       |   6 +++
 6 files changed, 137 insertions(+), 7 deletions(-)
 create mode 100644 target/i386/kvm/tdx-stub.c
author
#35 lore raw

PATCH v8 9/55 i386/tdx: Add property sept-ve-disable for tdx-guest object

In reply to Xiaoyao Li

Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.

Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.

Add sept-ve-disable property for tdx-guest object, for user to configure this bit.

---
Changes in v4:
- collect Acked-by from Markus

Changes in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)

---
 qapi/qom.json         |  8 +++++++-
 target/i386/kvm/tdx.c | 23 +++++++++++++++++++++++
 2 files changed, 30 insertions(+), 1 deletion(-)
qapi/qom.json +7 -1
index c0b61df964ef..f229bb07aaec 100644
@@ -1055,10 +1055,16 @@
 # @attributes: The 'attributes' of a TD guest that is passed to
 #     KVM_TDX_INIT_VM
 #
+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
+#     of EPT violation conversion to #VE on guest TD access of PENDING
+#     pages.  Some guest OS (e.g., Linux TD guest) may require this to
+#     be set, otherwise they refuse to boot.
+#
 # Since: 10.1
 ##
 { 'struct': 'TdxGuestProperties',
-  'data': { '*attributes': 'uint64' } }
+  'data': { '*attributes': 'uint64',
+            '*sept-ve-disable': 'bool' } }
 
 ##
 # @ThreadContextProperties:
target/i386/kvm/tdx.c +23 -0
index 0afaf739c09f..081094ae29db 100644
@@ -18,6 +18,8 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -252,6 +254,24 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     return 0;
 }
 
+static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    return !!(tdx->attributes & TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE);
+}
+
+static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    if (value) {
+        tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
+    } else {
+        tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
+    }
+}
+
 /* tdx guest */
 OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
                                    tdx_guest,
@@ -272,6 +292,9 @@ static void tdx_guest_init(Object *obj)
 
     object_property_add_uint64_ptr(obj, "attributes", &tdx->attributes,
                                    OBJ_PROP_FLAG_READWRITE);
+    object_property_add_bool(obj, "sept-ve-disable",
+                             tdx_guest_get_sept_ve_disable,
+                             tdx_guest_set_sept_ve_disable);
 }
 
 static void tdx_guest_finalize(Object *obj)
Signature · 1 line
2.34.1
Reviewed-by on v8 9/55
#36 lore raw

Re: [PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for tdx-guest object

Quoted 16 lines · Xiaoyao Li

Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.

Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.

Add sept-ve-disable property for tdx-guest object, for user to configure this bit.

---
Changes in v4:
- collect Acked-by from Markus

Quoted 7 lines

Changes in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)

---
 qapi/qom.json         |  8 +++++++-
 target/i386/kvm/tdx.c | 23 +++++++++++++++++++++++
 2 files changed, 30 insertions(+), 1 deletion(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 9/55
#37 lore raw

Re: [PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for tdx-guest object

In reply to Xiaoyao Li
Quoted 22 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:19 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for
tdx-guest object
X-Mailer: git-send-email 2.34.1

Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables EPT violation conversion to #VE on guest TD access of PENDING pages.

Some guest OS (e.g., Linux TD guest) may require this bit as 1. Otherwise refuse to boot.

Add sept-ve-disable property for tdx-guest object, for user to configure this bit.

---
Changes in v4:
- collect Acked-by from Markus

Quoted 7 lines

Changes in v3:
- update the comment of property @sept-ve-disable to make it more descriptive and use new format. (Daniel and Markus)

---
 qapi/qom.json         |  8 +++++++-
 target/i386/kvm/tdx.c | 23 +++++++++++++++++++++++
 2 files changed, 30 insertions(+), 1 deletion(-)
author
#38 lore raw

PATCH v8 10/55 i386/tdx: Make sept_ve_disable set by default

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

For TDX KVM use case, Linux guest is the most major one. It requires sept_ve_disable set. Make it default for the main use case. For other use case, it can be enabled/disabled via qemu command line.

---
 target/i386/kvm/tdx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/i386/kvm/tdx.c +1 -1
index 081094ae29db..592fb9cbbf0b 100644
@@ -288,7 +288,7 @@ static void tdx_guest_init(Object *obj)
     qemu_mutex_init(&tdx->lock);
 
     cgs->require_guest_memfd = true;
-    tdx->attributes = 0;
+    tdx->attributes = TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
 
     object_property_add_uint64_ptr(obj, "attributes", &tdx->attributes,
                                    OBJ_PROP_FLAG_READWRITE);
Signature · 1 line
2.34.1
Reviewed-by on v8 10/55
#39 lore raw

Re: [PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default

Quoted 11 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

For TDX KVM use case, Linux guest is the most major one. It requires sept_ve_disable set. Make it default for the main use case. For other use case, it can be enabled/disabled via qemu command line.

---
 target/i386/kvm/tdx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 10/55
#40 lore raw

Re: [PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default

In reply to Xiaoyao Li
Quoted 11 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:20 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default
X-Mailer: git-send-email 2.34.1

From: Isaku Yamahata <[email protected]>

For TDX KVM use case, Linux guest is the most major one. It requires sept_ve_disable set. Make it default for the main use case. For other use case, it can be enabled/disabled via qemu command line.

Quoted 5 lines
---
 target/i386/kvm/tdx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
author
#41 lore raw

PATCH v8 11/55 i386/tdx: Wire CPU features up with attributes of TD guest

In reply to Xiaoyao Li

For QEMU VMs,
- PKS is configured via CPUID_7_0_ECX_PKS, e.g., -cpu xxx,+pks and
- PMU is configured by x86cpu->enable_pmu, e.g., -cpu xxx,pmu=on

While the bit 30 (PKS) and bit 63 (PERFMON) of TD's attributes are also used to configure the PKS and PERFMON/PMU of TD, reuse the existing configuration interfaces of 'cpu' for TD's attributes.

---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
target/i386/kvm/tdx.c +13 -0
index 592fb9cbbf0b..1202b2111ba8 100644
@@ -19,6 +19,8 @@
 #include "tdx.h"
 
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
+#define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
+#define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
 
 static TdxGuest *tdx_guest;
 
@@ -151,6 +153,15 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
+static void setup_td_guest_attributes(X86CPU *x86cpu)
+{
+    CPUX86State *env = &x86cpu->env;
+
+    tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
+                             TDX_TD_ATTRIBUTES_PKS : 0;
+    tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+}
+
 static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
 {
     CPUX86State *env = &x86cpu->env;
@@ -214,6 +225,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    setup_td_guest_attributes(x86cpu);
+
     r = setup_td_xfam(x86cpu, errp);
     if (r) {
         return r;
Signature · 1 line
2.34.1
Reviewed-by on v8 11/55
#42 lore raw

Re: [PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of TD guest

Quoted 8 lines · Xiaoyao Li

For QEMU VMs,
- PKS is configured via CPUID_7_0_ECX_PKS, e.g., -cpu xxx,+pks and
- PMU is configured by x86cpu->enable_pmu, e.g., -cpu xxx,pmu=on

While the bit 30 (PKS) and bit 63 (PERFMON) of TD's attributes are also used to configure the PKS and PERFMON/PMU of TD, reuse the existing configuration interfaces of 'cpu' for TD's attributes.

Quoted 5 lines
---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 11/55
#43 lore raw

Re: [PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of TD guest

In reply to Xiaoyao Li
Quoted 14 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:21 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of
TD guest
X-Mailer: git-send-email 2.34.1

For QEMU VMs,
- PKS is configured via CPUID_7_0_ECX_PKS, e.g., -cpu xxx,+pks and
- PMU is configured by x86cpu->enable_pmu, e.g., -cpu xxx,pmu=on

While the bit 30 (PKS) and bit 63 (PERFMON) of TD's attributes are also used to configure the PKS and PERFMON/PMU of TD, reuse the existing configuration interfaces of 'cpu' for TD's attributes.

Quoted 5 lines
---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
author
#44 lore raw

PATCH v8 12/55 i386/tdx: Validate TD attributes

In reply to Xiaoyao Li

Validate TD attributes with tdx_caps that only supported bits are allowed by KVM.

Besides, sanity check the attribute bits that have not been supported by QEMU yet. e.g., debug bit, it will be allowed in the future when debug TD support lands in QEMU.

---
Changes in v8:
- Split the mrconfigid/mrowner/mrownerconfig part into a seperate next patch;

Changes in v7:
- Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates user's request. (Rick)

Changes in v3:
- using error_setg() for error report; (Daniel)

---
 target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)
target/i386/kvm/tdx.c +30 -2
index 1202b2111ba8..aa043acb1a88 100644
@@ -18,10 +18,15 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
 #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
 #define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
 
+#define TDX_SUPPORTED_TD_ATTRS  (TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE |\
+                                 TDX_TD_ATTRIBUTES_PKS | \
+                                 TDX_TD_ATTRIBUTES_PERFMON)
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -153,13 +158,33 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
-static void setup_td_guest_attributes(X86CPU *x86cpu)
+static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
+{
+    if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
+        error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
+                   "(KVM supported: 0x%llx)", tdx->attributes,
+                   tdx_caps->supported_attrs);
+        return -1;
+    }
+
+    if (tdx->attributes & ~TDX_SUPPORTED_TD_ATTRS) {
+        warn_report("Some QEMU unsupported TD attribute bits being requested:"
+                    "requested: 0x%lx QEMU supported: 0x%llx",
+                    tdx->attributes, TDX_SUPPORTED_TD_ATTRS);
+    }
+
+    return 0;
+}
+
+static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
 {
     CPUX86State *env = &x86cpu->env;
 
     tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
                              TDX_TD_ATTRIBUTES_PKS : 0;
     tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+
+    return tdx_validate_attributes(tdx_guest, errp);
 }
 
 static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
@@ -225,7 +250,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
-    setup_td_guest_attributes(x86cpu);
+    r = setup_td_guest_attributes(x86cpu, errp);
+    if (r) {
+        return r;
+    }
 
     r = setup_td_xfam(x86cpu, errp);
     if (r) {
Signature · 1 line
2.34.1
#45 lore raw

Re: [PATCH v8 12/55] i386/tdx: Validate TD attributes

Quoted 52 lines · Xiaoyao Li

Validate TD attributes with tdx_caps that only supported bits are allowed by KVM.

Besides, sanity check the attribute bits that have not been supported by QEMU yet. e.g., debug bit, it will be allowed in the future when debug TD support lands in QEMU.

---
Changes in v8:
- Split the mrconfigid/mrowner/mrownerconfig part into a seperate next patch;

Changes in v7:
- Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates user's request. (Rick)

Changes in v3:
- using error_setg() for error report; (Daniel)

---
 target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)
target/i386/kvm/tdx.c +8 -1
index 1202b2111ba8..aa043acb1a88 100644
@@ -18,10 +18,15 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
 #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
 #define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
 
+#define TDX_SUPPORTED_TD_ATTRS  (TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE |\
+                                 TDX_TD_ATTRIBUTES_PKS | \
+                                 TDX_TD_ATTRIBUTES_PERFMON)
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -153,13 +158,33 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
-static void setup_td_guest_attributes(X86CPU *x86cpu)
+static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
+{
+    if ((tdx->attributes & ~tdx_caps->supported_attrs)) {

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 173

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
+                   "(KVM supported: 0x%llx)", tdx->attributes,
+                   tdx_caps->supported_attrs);
+        return -1;
+    }
+
+    if (tdx->attributes & ~TDX_SUPPORTED_TD_ATTRS) {
+        warn_report("Some QEMU unsupported TD attribute bits being requested:"
+                    "requested: 0x%lx QEMU supported: 0x%llx",
+                    tdx->attributes, TDX_SUPPORTED_TD_ATTRS);

IIUC, this is an explicit user mis-configuration, and so we ought to use error_setg & return -1, not merely a warning.

Quoted 32 lines
target/i386/kvm/tdx.c +12 -1
+    }
+
+    return 0;
+}
+
+static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
 {
     CPUX86State *env = &x86cpu->env;
 
     tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
                              TDX_TD_ATTRIBUTES_PKS : 0;
     tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+
+    return tdx_validate_attributes(tdx_guest, errp);
 }
 
 static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
@@ -225,7 +250,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
-    setup_td_guest_attributes(x86cpu);
+    r = setup_td_guest_attributes(x86cpu, errp);
+    if (r) {
+        return r;
+    }
 
     r = setup_td_xfam(x86cpu, errp);
     if (r) {
Signature · 1 line
2.34.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#46 lore raw

Re: [PATCH v8 12/55] i386/tdx: Validate TD attributes

Quoted 58 lines · Daniel P. Berrangé
Quoted 57 lines · Xiaoyao Li

Validate TD attributes with tdx_caps that only supported bits are allowed by KVM.

Besides, sanity check the attribute bits that have not been supported by QEMU yet. e.g., debug bit, it will be allowed in the future when debug TD support lands in QEMU.

---
Changes in v8:
- Split the mrconfigid/mrowner/mrownerconfig part into a seperate next patch;

Changes in v7:
- Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates user's request. (Rick)

Changes in v3:
- using error_setg() for error report; (Daniel)

---
  target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
  1 file changed, 30 insertions(+), 2 deletions(-)
target/i386/kvm/tdx.c +13 -1
index 1202b2111ba8..aa043acb1a88 100644
@@ -18,10 +18,15 @@
  #include "kvm_i386.h"
  #include "tdx.h"
 
+#define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
  #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
  #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
  #define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
 
+#define TDX_SUPPORTED_TD_ATTRS  (TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE |\
+                                 TDX_TD_ATTRIBUTES_PKS | \
+                                 TDX_TD_ATTRIBUTES_PERFMON)
+
  static TdxGuest *tdx_guest;
 
  static struct kvm_tdx_capabilities *tdx_caps;
@@ -153,13 +158,33 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
      return KVM_X86_TDX_VM;
  }
 
-static void setup_td_guest_attributes(X86CPU *x86cpu)
+static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
+{
+    if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
+        error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
+                   "(KVM supported: 0x%llx)", tdx->attributes,
+                   tdx_caps->supported_attrs);
+        return -1;
+    }
Quoted 8 lines

target/i386/kvm/tdx.c static int tdx_kvm_type(X86ConfidentialGuest *cg) · line 173

Quoted 5 lines
target/i386/kvm/tdx.c +5 -0
+
+    if (tdx->attributes & ~TDX_SUPPORTED_TD_ATTRS) {
+        warn_report("Some QEMU unsupported TD attribute bits being requested:"
+                    "requested: 0x%lx QEMU supported: 0x%llx",
+                    tdx->attributes, TDX_SUPPORTED_TD_ATTRS);

IIUC, this is an explicit user mis-configuration, and so we ought to use error_setg & return -1, not merely a warning.

My thought was to allow user to enable some QEMU unsupported but KVM supported attribute bits, for testing purpose.

But now I agree with you. It's not good for production QEMU.

Quoted 35 lines
Quoted 32 lines
target/i386/kvm/tdx.c +12 -1
+    }
+
+    return 0;
+}
+
+static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
  {
      CPUX86State *env = &x86cpu->env;
 
      tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
                               TDX_TD_ATTRIBUTES_PKS : 0;
      tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+
+    return tdx_validate_attributes(tdx_guest, errp);
  }
 
  static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
@@ -225,7 +250,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
      init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                          sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
-    setup_td_guest_attributes(x86cpu);
+    r = setup_td_guest_attributes(x86cpu, errp);
+    if (r) {
+        return r;
+    }
 
      r = setup_td_xfam(x86cpu, errp);
      if (r) {
Signature · 1 line
2.34.1

With regards,
Daniel

Reviewed-by on v8 12/55
#47 lore raw

Re: [PATCH v8 12/55] i386/tdx: Validate TD attributes

In reply to Xiaoyao Li
Quoted 23 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:22 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 12/55] i386/tdx: Validate TD attributes
X-Mailer: git-send-email 2.34.1

Validate TD attributes with tdx_caps that only supported bits are allowed by KVM.

Besides, sanity check the attribute bits that have not been supported by QEMU yet. e.g., debug bit, it will be allowed in the future when debug TD support lands in QEMU.

---
Changes in v8:
- Split the mrconfigid/mrowner/mrownerconfig part into a seperate next patch;

Changes in v7:
- Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates user's request. (Rick)

Quoted 5 lines

Changes in v3:
- using error_setg() for error report; (Daniel)

---
 target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

Overall LGTM,

(with Daniel's comment fixed)

author
#48 lore raw

PATCH v8 13/55 i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/

Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.

example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v

---
Changes in v8:
- it gets squashed into previous patch in v7. So split it out in v8;

Changes in v6:
- refine the doc comment of QAPI properties;

Changes in v5:
- refine the description of QAPI properties and add description of default value when not specified;

Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in _setter(); (Daniel)

Changes in v3:
- use base64 encoding instread of hex-string;

---
 qapi/qom.json         | 16 +++++++-
 target/i386/kvm/tdx.c | 86 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h |  3 ++
 3 files changed, 104 insertions(+), 1 deletion(-)
qapi/qom.json +15 -1
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
 #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
 #     be set, otherwise they refuse to boot.
 #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.
+#
+# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
+#     Defaults to all zeros.
+#
+# @mrownerconfig: ID for owner-defined configuration of the guest TD,
+#     e.g., specific to the workload rather than the run-time or OS
+#     (base64 encoded SHA384 digest).  Defaults to all zeros.
+#
 # Since: 10.1
 ##
 { 'struct': 'TdxGuestProperties',
   'data': { '*attributes': 'uint64',
-            '*sept-ve-disable': 'bool' } }
+            '*sept-ve-disable': 'bool',
+            '*mrconfigid': 'str',
+            '*mrowner': 'str',
+            '*mrownerconfig': 'str' } }
 
 ##
 # @ThreadContextProperties:
target/i386/kvm/tdx.c +86 -0

118 lines not on this page · raw message

target/i386/kvm/tdx.h +3 -0
index 4e2b5c61ff5b..e472b11fb0dd 100644
@@ -24,6 +24,9 @@ typedef struct TdxGuest {
     bool initialized;
     uint64_t attributes;    /* TD attributes */
     uint64_t xfam;
+    char *mrconfigid;       /* base64 encoded sha348 digest */
+    char *mrowner;          /* base64 encoded sha348 digest */
+    char *mrownerconfig;    /* base64 encoded sha348 digest */
 } TdxGuest;
 
 #ifdef CONFIG_TDX
Signature · 1 line
2.34.1
#49 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

Quoted 67 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/

Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.

example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v

---
Changes in v8:
- it gets squashed into previous patch in v7. So split it out in v8;

Changes in v6:
- refine the doc comment of QAPI properties;

Changes in v5:
- refine the description of QAPI properties and add description of default value when not specified;

Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in _setter(); (Daniel)

Changes in v3:
- use base64 encoding instread of hex-string;

---
 qapi/qom.json         | 16 +++++++-
 target/i386/kvm/tdx.c | 86 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h |  3 ++
 3 files changed, 104 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +3 -0
index aa043acb1a88..77ddb2655c53 100644
@@ -11,8 +11,10 @@
 
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
+#include "qemu/base64.h"
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
+#include "crypto/hash.h"
 
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
@@ -239,6 +241,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     CPUX86State *env = &x86cpu->env;
     g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
     Error *local_err = NULL;
+    size_t data_len;
     int retry = 10000;
     int r = 0;
 

target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 262

Quoted 10 lines
target/i386/kvm/tdx.c +7 -0
@@ -250,6 +253,36 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    if (tdx_guest->mrconfigid) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
+                              strlen(tdx_guest->mrconfigid), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrconfigid");
+            return -1;
+        }

When '!data', qbase64_decode will have already filled 'errp' with details, so we must immediately 'return -1', as a repeated error_setg call is an programming error.

The error_setg call should only be done in response to failing the 'data_len' check and the message should specify the lengths eg more like this

  if (!data) {
      return -1;
  }

  if (data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
      error_setg(errp, "TDX mrconfigid len %d must match SHA384 digest len %d",
                 data_len, QCRYPTO_HASH_DIGEST_LEN_SHA384)
      return -1;
  }
Quoted 16 lines
target/i386/kvm/tdx.c +16 -0
+        memcpy(init_vm->mrconfigid, data, data_len);
+    }
+
+    if (tdx_guest->mrowner) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
+                              strlen(tdx_guest->mrowner), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrowner");
+            return -1;
+        }
+        memcpy(init_vm->mrowner, data, data_len);
+    }
+
+    if (tdx_guest->mrownerconfig) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
+                            strlen(tdx_guest->mrownerconfig), &data_len, errp);

target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 288

Quoted 10 lines
target/i386/kvm/tdx.c +7 -0
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrownerconfig");
+            return -1;
+        }
+        memcpy(init_vm->mrownerconfig, data, data_len);
+    }
+
     r = setup_td_guest_attributes(x86cpu, errp);
     if (r) {
         return r;

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#50 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

Quoted 87 lines · Daniel P. Berrangé
Quoted 67 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/

Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.

example

-object tdx-guest, \
   mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
   mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
   mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v

---
Changes in v8:
- it gets squashed into previous patch in v7. So split it out in v8;

Changes in v6:
- refine the doc comment of QAPI properties;

Changes in v5:
- refine the description of QAPI properties and add description of

    default value when not specified;

Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in

    _setter(); (Daniel)

Changes in v3:
- use base64 encoding instread of hex-string;

---
  qapi/qom.json         | 16 +++++++-
  target/i386/kvm/tdx.c | 86 +++++++++++++++++++++++++++++++++++++++++++
  target/i386/kvm/tdx.h |  3 ++
  3 files changed, 104 insertions(+), 1 deletion(-)
target/i386/kvm/tdx.c +3 -0
index aa043acb1a88..77ddb2655c53 100644
@@ -11,8 +11,10 @@
 
  #include "qemu/osdep.h"
  #include "qemu/error-report.h"
+#include "qemu/base64.h"
  #include "qapi/error.h"
  #include "qom/object_interfaces.h"
+#include "crypto/hash.h"
 
  #include "hw/i386/x86.h"
  #include "kvm_i386.h"
@@ -239,6 +241,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
      CPUX86State *env = &x86cpu->env;
      g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
      Error *local_err = NULL;
+    size_t data_len;
      int retry = 10000;
      int r = 0;
 

target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 262

Quoted 10 lines
target/i386/kvm/tdx.c +7 -0
@@ -250,6 +253,36 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
      init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                          sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    if (tdx_guest->mrconfigid) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
+                              strlen(tdx_guest->mrconfigid), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrconfigid");
+            return -1;
+        }

When '!data', qbase64_decode will have already filled 'errp' with details, so we must immediately 'return -1', as a repeated error_setg call is an programming error.

The error_setg call should only be done in response to failing the 'data_len' check and the message should specify the lengths eg more like this

Quoted 9 lines
   if (!data) {
       return -1;
   }

   if (data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
       error_setg(errp, "TDX mrconfigid len %d must match SHA384 digest len %d",
                  data_len, QCRYPTO_HASH_DIGEST_LEN_SHA384)
       return -1;
   }

Nice catch!

Quoted 30 lines
Quoted 17 lines
target/i386/kvm/tdx.c +16 -0
+        memcpy(init_vm->mrconfigid, data, data_len);
+    }
+
+    if (tdx_guest->mrowner) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
+                              strlen(tdx_guest->mrowner), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrowner");
+            return -1;
+        }
+        memcpy(init_vm->mrowner, data, data_len);
+    }
+
+    if (tdx_guest->mrownerconfig) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
+                            strlen(tdx_guest->mrownerconfig), &data_len, errp);

target/i386/kvm/tdx.c int tdx_pre_create_vcpu(CPUState *cpu, Error **errp) · line 288

Quoted 10 lines
target/i386/kvm/tdx.c +7 -0
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrownerconfig");
+            return -1;
+        }
+        memcpy(init_vm->mrownerconfig, data, data_len);
+    }
+
      r = setup_td_guest_attributes(x86cpu, errp);
      if (r) {
          return r;

With regards,
Daniel

Acked-by on v8 13/55
#51 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

In reply to Xiaoyao Li
Quoted 18 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/[email protected]/

Allow user to specify those values via property mrconfigid, mrowner and mrownerconfig. They are all in base64 format.

example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v

[...]

Quoted 20 lines
qapi/qom.json +11 -0
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
 #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
 #     be set, otherwise they refuse to boot.
 #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.
+#
+# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
+#     Defaults to all zeros.
+#
+# @mrownerconfig: ID for owner-defined configuration of the guest TD,
+#     e.g., specific to the workload rather than the run-time or OS
+#     (base64 encoded SHA384 digest).  Defaults to all zeros.
+#
 # Since: 10.1

qapi/qom.json line 1083

Quoted 10 lines
qapi/qom.json +4 -1
 ##
 { 'struct': 'TdxGuestProperties',
   'data': { '*attributes': 'uint64',
-            '*sept-ve-disable': 'bool' } }
+            '*sept-ve-disable': 'bool',
+            '*mrconfigid': 'str',
+            '*mrowner': 'str',
+            '*mrownerconfig': 'str' } }
 
 ##

[...]

Reviewed-by on v8 13/55
#52 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

In reply to Xiaoyao Li

qapi/qom.json line 1065

Quoted 11 lines
qapi/qom.json +3 -0
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
 #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
 #     be set, otherwise they refuse to boot.
 #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Maybe a typo? s/Defaults/Default/

qapi/qom.json line 1068

Quoted 3 lines
qapi/qom.json +3 -0
+#
+# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Ditto.

Quoted 5 lines
qapi/qom.json +5 -0
+#
+# @mrownerconfig: ID for owner-defined configuration of the guest TD,
+#     e.g., specific to the workload rather than the run-time or OS
+#     (base64 encoded SHA384 digest).  Defaults to all zeros.
+#

qapi/qom.json line 1082

Quoted 10 lines
qapi/qom.json +4 -1
 # Since: 10.1
 ##
 { 'struct': 'TdxGuestProperties',
   'data': { '*attributes': 'uint64',
-            '*sept-ve-disable': 'bool' } }
+            '*sept-ve-disable': 'bool',
+            '*mrconfigid': 'str',
+            '*mrowner': 'str',
+            '*mrownerconfig': 'str' } }
 

Overall LGTM, (with Daniel's comment fixed,)

author
#53 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

Quoted 5 lines · Zhao Liu
Quoted 5 lines
qapi/qom.json +0 -0
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
Quoted 8 lines

qapi/qom.json line 1065

Quoted 6 lines
qapi/qom.json +3 -0
  #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
  #     be set, otherwise they refuse to boot.
  #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Maybe a typo? s/Defaults/Default/

(It) defaults to all zeros.

If you grep the "defaults to", you can get a lot of it.

Quoted 27 lines

qapi/qom.json line 1068

Quoted 3 lines
qapi/qom.json +3 -0
+#
+# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Ditto.

Quoted 5 lines
qapi/qom.json +5 -0
+#
+# @mrownerconfig: ID for owner-defined configuration of the guest TD,
+#     e.g., specific to the workload rather than the run-time or OS
+#     (base64 encoded SHA384 digest).  Defaults to all zeros.
+#

qapi/qom.json line 1082

Quoted 10 lines
qapi/qom.json +4 -1
  # Since: 10.1
  ##
  { 'struct': 'TdxGuestProperties',
    'data': { '*attributes': 'uint64',
-            '*sept-ve-disable': 'bool' } }
+            '*sept-ve-disable': 'bool',
+            '*mrconfigid': 'str',
+            '*mrowner': 'str',
+            '*mrownerconfig': 'str' } }
 

Overall LGTM, (with Daniel's comment fixed,)

#54 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

Quoted 20 lines · Xiaoyao Li

Date: Wed, 23 Apr 2025 16:11:25 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v8 13/55] i386/tdx: Support user configurable
mrconfigid/mrowner/mrownerconfig

Quoted 5 lines · Zhao Liu
Quoted 5 lines
qapi/qom.json +0 -0
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
Quoted 8 lines

qapi/qom.json line 1065

Quoted 6 lines
qapi/qom.json +3 -0
  #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
  #     be set, otherwise they refuse to boot.
  #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Maybe a typo? s/Defaults/Default/

Quoted 3 lines

(It) defaults to all zeros.

If you grep the "defaults to", you can get a lot of it.

The comment can be relaxed, but please try to be precise with the doc. You can add the omitted "It".

author
#55 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

Quoted 21 lines · Zhao Liu
Quoted 20 lines · Xiaoyao Li

Date: Wed, 23 Apr 2025 16:11:25 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v8 13/55] i386/tdx: Support user configurable
mrconfigid/mrowner/mrownerconfig

Quoted 13 lines · Zhao Liu

qapi/qom.json line 1065

Quoted 11 lines
qapi/qom.json +3 -0
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
   #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
   #     be set, otherwise they refuse to boot.
   #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Maybe a typo? s/Defaults/Default/

Quoted 6 lines
Quoted 3 lines

(It) defaults to all zeros.

If you grep the "defaults to", you can get a lot of it.

The comment can be relaxed, but please try to be precise with the doc. You can add the omitted "It".

Actually, it came from Markus[*], the QAPI maintainer.

So I would leave it to him to make the decision.

[*] https://lore.kernel.org/qemu-devel/[email protected]/

#56 lore raw

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

In reply to Xiaoyao Li
Quoted 15 lines · Xiaoyao Li
Quoted 5 lines · Zhao Liu
Quoted 5 lines
qapi/qom.json +0 -0
index f229bb07aaec..a8379bac1719 100644
@@ -1060,11 +1060,25 @@
Quoted 8 lines

qapi/qom.json line 1065

Quoted 6 lines
qapi/qom.json +3 -0
  #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
  #     be set, otherwise they refuse to boot.
  #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.

Maybe a typo? s/Defaults/Default/

Quoted 3 lines

(It) defaults to all zeros.

If you grep the "defaults to", you can get a lot of it.

'Defaults to' is correct/acceptable terminology, IMHO, no need to change.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#57 lore raw

PATCH v8 14/55 i386/tdx: Set APIC bus rate to match with what TDX module enforces

In reply to Xiaoyao Li

TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD guests and it's unchangeable from VMM. As a result, TDX guest reads the APIC timer as the same frequency, 25MHz.

While KVM's default emulated frequency for APIC bus is 1GHz, set the APIC bus rate to match with TDX explicitly to ensure KVM provide correct emulated APIC timer for TD guest.

---
Changes in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 target/i386/kvm/tdx.h |  3 +++
 2 files changed, 16 insertions(+)
target/i386/kvm/tdx.c +13 -0
index 77ddb2655c53..1a99b677686e 100644
@@ -253,6 +253,19 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    if (!kvm_check_extension(kvm_state, KVM_CAP_X86_APIC_BUS_CYCLES_NS)) {
+        error_setg(errp, "KVM doesn't support KVM_CAP_X86_APIC_BUS_CYCLES_NS");
+        return -EOPNOTSUPP;
+    }
+
+    r = kvm_vm_enable_cap(kvm_state, KVM_CAP_X86_APIC_BUS_CYCLES_NS,
+                          0, TDX_APIC_BUS_CYCLES_NS);
+    if (r < 0) {
+        error_setg_errno(errp, -r,
+                         "Unable to set core crystal clock frequency to 25MHz");
+        return r;
+    }
+
     if (tdx_guest->mrconfigid) {
         g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
                               strlen(tdx_guest->mrconfigid), &data_len, errp);
target/i386/kvm/tdx.h +3 -0
index e472b11fb0dd..d39e733d9fcc 100644
@@ -16,6 +16,9 @@ typedef struct TdxGuestClass {
     X86ConfidentialGuestClass parent_class;
 } TdxGuestClass;
 
+/* TDX requires bus frequency 25MHz */
+#define TDX_APIC_BUS_CYCLES_NS 40
+
 typedef struct TdxGuest {
     X86ConfidentialGuest parent_obj;
 
Signature · 1 line
2.34.1
Reviewed-by on v8 14/55
#58 lore raw

Re: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

Quoted 3 lines · Xiaoyao Li

TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD guests and it's unchangeable from VMM. As a result, TDX guest reads the APIC timer as the same frequency, 25MHz.

Did you mean s/as/at/ ?

Quoted 4 lines

While KVM's default emulated frequency for APIC bus is 1GHz, set the APIC bus rate to match with TDX explicitly to ensure KVM provide correct emulated APIC timer for TD guest.

Quoted 8 lines

---
Changes in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 target/i386/kvm/tdx.h |  3 +++
 2 files changed, 16 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#59 lore raw

Re: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

Quoted 6 lines · Daniel P. Berrangé
Quoted 3 lines · Xiaoyao Li

TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD guests and it's unchangeable from VMM. As a result, TDX guest reads the APIC timer as the same frequency, 25MHz.

Did you mean s/as/at/ ?

yes, thanks for catching it!

Quoted 18 lines
Quoted 4 lines

While KVM's default emulated frequency for APIC bus is 1GHz, set the APIC bus rate to match with TDX explicitly to ensure KVM provide correct emulated APIC timer for TD guest.

Quoted 8 lines

---
Changes in v6:
- new patch;

---
  target/i386/kvm/tdx.c | 13 +++++++++++++
  target/i386/kvm/tdx.h |  3 +++
  2 files changed, 16 insertions(+)

With regards,
Daniel

Reviewed-by on v8 14/55
#60 lore raw

Re: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

In reply to Xiaoyao Li
Quoted 14 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:24 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what
TDX module enforces
X-Mailer: git-send-email 2.34.1

TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD guests and it's unchangeable from VMM. As a result, TDX guest reads the APIC timer as the same frequency, 25MHz.

While KVM's default emulated frequency for APIC bus is 1GHz, set the APIC bus rate to match with TDX explicitly to ensure KVM provide correct emulated APIC timer for TD guest.

Quoted 8 lines

---
Changes in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 target/i386/kvm/tdx.h |  3 +++
 2 files changed, 16 insertions(+)
author
#61 lore raw

PATCH v8 15/55 i386/tdx: Implement user specified tsc frequency

In reply to Xiaoyao Li

Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.

Besides, sanity check the tsc frequency to be in the legal range and legal granularity (required by TDX module).

---
Changes in v3:
- use @errp to report error info; (Daniel)

Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM side drop the @tsc_khz field in struct kvm_tdx_init_vm

---
 target/i386/kvm/kvm.c |  9 +++++++++
 target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
target/i386/kvm/kvm.c +9 -0
index a537699bb7df..7de5014051eb 100644
@@ -869,6 +869,15 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)
     int r, cur_freq;
     bool set_ioctl = false;
 
+    /*
+     * TSC of TD vcpu is immutable, it cannot be set/changed via vcpu scope
+     * VM_SET_TSC_KHZ, but only be initialized via VM scope VM_SET_TSC_KHZ
+     * before ioctl KVM_TDX_INIT_VM in tdx_pre_create_vcpu()
+     */
+    if (is_tdx_vm()) {
+        return 0;
+    }
+
     if (!env->tsc_khz) {
         return 0;
     }
target/i386/kvm/tdx.c +25 -0
index 1a99b677686e..d9e49b7444f8 100644
@@ -20,6 +20,9 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_MIN_TSC_FREQUENCY_KHZ   (100 * 1000)
+#define TDX_MAX_TSC_FREQUENCY_KHZ   (10 * 1000 * 1000)
+
 #define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
 #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
@@ -266,6 +269,28 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
         return r;
     }
 
+    if (env->tsc_khz && (env->tsc_khz < TDX_MIN_TSC_FREQUENCY_KHZ ||
+                         env->tsc_khz > TDX_MAX_TSC_FREQUENCY_KHZ)) {
+        error_setg(errp, "Invalid TSC %ld KHz, must specify cpu_frequency "
+                         "between [%d, %d] kHz", env->tsc_khz,
+                         TDX_MIN_TSC_FREQUENCY_KHZ, TDX_MAX_TSC_FREQUENCY_KHZ);
+       return -EINVAL;
+    }
+
+    if (env->tsc_khz % (25 * 1000)) {
+        error_setg(errp, "Invalid TSC %ld KHz, it must be multiple of 25MHz",
+                   env->tsc_khz);
+        return -EINVAL;
+    }
+
+    /* it's safe even env->tsc_khz is 0. KVM uses host's tsc_khz in this case */
+    r = kvm_vm_ioctl(kvm_state, KVM_SET_TSC_KHZ, env->tsc_khz);
+    if (r < 0) {
+        error_setg_errno(errp, -r, "Unable to set TSC frequency to %ld kHz",
+                         env->tsc_khz);
+        return r;
+    }
+
     if (tdx_guest->mrconfigid) {
         g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
                               strlen(tdx_guest->mrconfigid), &data_len, errp);
Signature · 1 line
2.34.1
Reviewed-by on v8 15/55
#62 lore raw

Re: [PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency

Quoted 12 lines · Xiaoyao Li

Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.

Besides, sanity check the tsc frequency to be in the legal range and legal granularity (required by TDX module).

---
Changes in v3:
- use @errp to report error info; (Daniel)

Quoted 7 lines

Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM side drop the @tsc_khz field in struct kvm_tdx_init_vm

---
 target/i386/kvm/kvm.c |  9 +++++++++
 target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 15/55
#63 lore raw

Re: [PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency

In reply to Xiaoyao Li
Quoted 17 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:25 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency
X-Mailer: git-send-email 2.34.1

Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.

Besides, sanity check the tsc frequency to be in the legal range and legal granularity (required by TDX module).

---
Changes in v3:
- use @errp to report error info; (Daniel)

Quoted 7 lines

Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM side drop the @tsc_khz field in struct kvm_tdx_init_vm

---
 target/i386/kvm/kvm.c |  9 +++++++++
 target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
author
#64 lore raw

PATCH v8 16/55 i386/tdx: load TDVF for TD guest

In reply to Xiaoyao Li

From: Chao Peng <[email protected]>

TDVF(OVMF) needs to run at private memory for TD guest. TDX cannot support pflash device since it doesn't support read-only private memory. Thus load TDVF(OVMF) with -bios option for TDs.

Use memory_region_init_ram_guest_memfd() to allocate the MemoryRegion for TDVF because it needs to be located at private memory.

Also store the MemoryRegion pointer of TDVF since the shared ramblock of it can be discared after it gets copied to private ramblock.

---
 hw/i386/x86-common.c  | 6 +++++-
 target/i386/kvm/tdx.c | 6 ++++++
 target/i386/kvm/tdx.h | 3 +++
 3 files changed, 14 insertions(+), 1 deletion(-)
hw/i386/x86-common.c +5 -1
index 1b0671c52397..b1b5f11e7396 100644
@@ -44,6 +44,7 @@
 #include "standard-headers/asm-x86/bootparam.h"
 #include CONFIG_DEVICES
 #include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
 
 #ifdef CONFIG_XEN_EMU
 #include "hw/xen/xen.h"
@@ -1035,11 +1036,14 @@ void x86_bios_rom_init(X86MachineState *x86ms, const char *default_firmware,
     if (machine_require_guest_memfd(MACHINE(x86ms))) {
         memory_region_init_ram_guest_memfd(&x86ms->bios, NULL, "pc.bios",
                                            bios_size, &error_fatal);
+        if (is_tdx_vm()) {
+            tdx_set_tdvf_region(&x86ms->bios);
+        }
     } else {
         memory_region_init_ram(&x86ms->bios, NULL, "pc.bios",
                                bios_size, &error_fatal);
     }
-    if (sev_enabled()) {
+    if (sev_enabled() || is_tdx_vm()) {
         /*
          * The concept of a "reset" simply doesn't exist for
          * confidential computing guests, we have to destroy and
target/i386/kvm/tdx.c +6 -0
index d9e49b7444f8..b7f6f3b708b0 100644
@@ -137,6 +137,12 @@ static int get_tdx_capabilities(Error **errp)
     return 0;
 }
 
+void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
+{
+    assert(!tdx_guest->tdvf_mr);
+    tdx_guest->tdvf_mr = tdvf_mr;
+}
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(cgs);
target/i386/kvm/tdx.h +3 -0
index d39e733d9fcc..b73461b8d8a3 100644
@@ -30,6 +30,8 @@ typedef struct TdxGuest {
     char *mrconfigid;       /* base64 encoded sha348 digest */
     char *mrowner;          /* base64 encoded sha348 digest */
     char *mrownerconfig;    /* base64 encoded sha348 digest */
+
+    MemoryRegion *tdvf_mr;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
@@ -39,5 +41,6 @@ bool is_tdx_vm(void);
 #endif /* CONFIG_TDX */
 
 int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
+void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
 
 #endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Reviewed-by on v8 16/55
#65 lore raw

Re: [PATCH v8 16/55] i386/tdx: load TDVF for TD guest

Quoted 17 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:26 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 16/55] i386/tdx: load TDVF for TD guest
X-Mailer: git-send-email 2.34.1

From: Chao Peng <[email protected]>

TDVF(OVMF) needs to run at private memory for TD guest. TDX cannot support pflash device since it doesn't support read-only private memory. Thus load TDVF(OVMF) with -bios option for TDs.

Use memory_region_init_ram_guest_memfd() to allocate the MemoryRegion for TDVF because it needs to be located at private memory.

Also store the MemoryRegion pointer of TDVF since the shared ramblock of it can be discared after it gets copied to private ramblock.

Quoted 8 lines
---
 hw/i386/x86-common.c  | 6 +++++-
 target/i386/kvm/tdx.c | 6 ++++++
 target/i386/kvm/tdx.h | 3 +++
 3 files changed, 14 insertions(+), 1 deletion(-)
author
#66 lore raw

PATCH v8 17/55 i386/tdvf: Introduce function to parse TDVF metadata

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.

A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function tdvf_parse_metadata() to parse the metadata from TDVF image and store the info of each TDVF section.

TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.

Select X86_FW_OVMF when TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.

---
Changes in v8:
- Drop the failure handling of memcpy() since it cannot fail;

Changes in v7:
- Update license info to only use SPDX tag;
- use g_autofree to avoid manually free;

Changes in v6:
- Drop the the data endianness change for metadata->Length;

Changes in v1:
- rename tdvf_parse_section_entry() to
tdvf_parse_and_check_section_entry()

Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID

---
 hw/i386/Kconfig        |   1 +
 hw/i386/meson.build    |   1 +
 hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
 include/hw/i386/tdvf.h |  38 +++++++++
 4 files changed, 223 insertions(+)
 create mode 100644 hw/i386/tdvf.c
 create mode 100644 include/hw/i386/tdvf.h
hw/i386/Kconfig +1 -0
index cce9521ba934..eb65bda6e071 100644
@@ -12,6 +12,7 @@ config SGX
 
 config TDX
     bool
+    select X86_FW_OVMF
     depends on KVM
 
 config PC
hw/i386/meson.build +1 -0
index 10bdfde27c69..3bc1da2b6eb4 100644
@@ -32,6 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
   'port92.c'))
 i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
                                         if_false: files('pc_sysfw_ovmf-stubs.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
 
 subdir('kvm')
 subdir('xen')
hw/i386/tdvf.c new file +183 -0

183 lines not on this page · raw message

include/hw/i386/tdvf.h new file +38 -0
new file mode 100644
index 000000000000..7ebcac42a36c
@@ -0,0 +1,38 @@
+/*
+ * Copyright (c) 2025 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_I386_TDVF_H
+#define HW_I386_TDVF_H
+
+#include "qemu/osdep.h"
+
+#define TDVF_SECTION_TYPE_BFV               0
+#define TDVF_SECTION_TYPE_CFV               1
+#define TDVF_SECTION_TYPE_TD_HOB            2
+#define TDVF_SECTION_TYPE_TEMP_MEM          3
+
+#define TDVF_SECTION_ATTRIBUTES_MR_EXTEND   (1U << 0)
+#define TDVF_SECTION_ATTRIBUTES_PAGE_AUG    (1U << 1)
+
+typedef struct TdxFirmwareEntry {
+    uint32_t data_offset;
+    uint32_t data_len;
+    uint64_t address;
+    uint64_t size;
+    uint32_t type;
+    uint32_t attributes;
+} TdxFirmwareEntry;
+
+typedef struct TdxFirmware {
+    uint32_t nr_entries;
+    TdxFirmwareEntry *entries;
+} TdxFirmware;
+
+int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size);
+
+#endif /* HW_I386_TDVF_H */
Signature · 1 line
2.34.1
#67 lore raw

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

Quoted 105 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.

A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function tdvf_parse_metadata() to parse the metadata from TDVF image and store the info of each TDVF section.

TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.

Select X86_FW_OVMF when TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.

---
Changes in v8:
- Drop the failure handling of memcpy() since it cannot fail;

Changes in v7:
- Update license info to only use SPDX tag;
- use g_autofree to avoid manually free;

Changes in v6:
- Drop the the data endianness change for metadata->Length;

Changes in v1:
- rename tdvf_parse_section_entry() to
tdvf_parse_and_check_section_entry()

Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID

---
 hw/i386/Kconfig        |   1 +
 hw/i386/meson.build    |   1 +
 hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
 include/hw/i386/tdvf.h |  38 +++++++++
 4 files changed, 223 insertions(+)
 create mode 100644 hw/i386/tdvf.c
 create mode 100644 include/hw/i386/tdvf.h
hw/i386/Kconfig +1 -0
index cce9521ba934..eb65bda6e071 100644
@@ -12,6 +12,7 @@ config SGX
 
 config TDX
     bool
+    select X86_FW_OVMF
     depends on KVM
 
 config PC
hw/i386/meson.build +1 -0
index 10bdfde27c69..3bc1da2b6eb4 100644
@@ -32,6 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
   'port92.c'))
 i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
                                         if_false: files('pc_sysfw_ovmf-stubs.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
 
 subdir('kvm')
 subdir('xen')
hw/i386/tdvf.c new file +26 -0
new file mode 100644
index 000000000000..328d1b7ffdf8
@@ -0,0 +1,183 @@
+/*
+ * Copyright (c) 2025 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *         Xiaoyao Li <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+
+#include "hw/i386/pc.h"
+#include "hw/i386/tdvf.h"
+#include "system/kvm.h"
+
+#define TDX_METADATA_OFFSET_GUID    "e47a6535-984a-4798-865e-4685a7bf8ec2"
+#define TDX_METADATA_VERSION        1
+#define TDVF_SIGNATURE              0x46564454 /* TDVF as little endian */
+
+typedef struct {
+    uint32_t DataOffset;
+    uint32_t RawDataSize;
+    uint64_t MemoryAddress;
+    uint64_t MemoryDataSize;
+    uint32_t Type;

hw/i386/tdvf.c line 36

Quoted 10 lines
hw/i386/tdvf.c new file +10 -0
+    uint32_t Attributes;
+} TdvfSectionEntry;
+
+typedef struct {
+    uint32_t Signature;
+    uint32_t Length;
+    uint32_t Version;
+    uint32_t NumberOfSectionEntries;
+    TdvfSectionEntry SectionEntries[];
+} TdvfMetadata;

struct field names starting with an initial capital is not the usual QEMU code style. Can this be all initial lowercase, with capital just for word separation.

hw/i386/tdvf.c line 41

Quoted 5 lines
hw/i386/tdvf.c new file +5 -0
+
+struct tdx_metadata_offset {
+    uint32_t offset;
+};
+

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#68 lore raw

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

Quoted 112 lines · Daniel P. Berrangé
Quoted 111 lines · Xiaoyao Li

From: Isaku Yamahata <[email protected]>

TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.

A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function tdvf_parse_metadata() to parse the metadata from TDVF image and store the info of each TDVF section.

TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.

Select X86_FW_OVMF when TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.

---
Changes in v8:
- Drop the failure handling of memcpy() since it cannot fail;

Changes in v7:
- Update license info to only use SPDX tag;
- use g_autofree to avoid manually free;

Changes in v6:
- Drop the the data endianness change for metadata->Length;

Changes in v1:
- rename tdvf_parse_section_entry() to
tdvf_parse_and_check_section_entry()

Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID

---
  hw/i386/Kconfig        |   1 +
  hw/i386/meson.build    |   1 +
  hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
  include/hw/i386/tdvf.h |  38 +++++++++
  4 files changed, 223 insertions(+)

create mode 100644 hw/i386/tdvf.c
create mode 100644 include/hw/i386/tdvf.h

hw/i386/Kconfig +1 -0
index cce9521ba934..eb65bda6e071 100644
@@ -12,6 +12,7 @@ config SGX
 
  config TDX
      bool
+    select X86_FW_OVMF
      depends on KVM
 
  config PC
hw/i386/meson.build +1 -0
index 10bdfde27c69..3bc1da2b6eb4 100644
@@ -32,6 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
    'port92.c'))
  i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
                                          if_false: files('pc_sysfw_ovmf-stubs.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
 
  subdir('kvm')
  subdir('xen')
hw/i386/tdvf.c new file +32 -0
new file mode 100644
index 000000000000..328d1b7ffdf8
@@ -0,0 +1,183 @@
+/*
+ * Copyright (c) 2025 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *         Xiaoyao Li <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+
+#include "hw/i386/pc.h"
+#include "hw/i386/tdvf.h"
+#include "system/kvm.h"
+
+#define TDX_METADATA_OFFSET_GUID    "e47a6535-984a-4798-865e-4685a7bf8ec2"
+#define TDX_METADATA_VERSION        1
+#define TDVF_SIGNATURE              0x46564454 /* TDVF as little endian */
+
+typedef struct {
+    uint32_t DataOffset;
+    uint32_t RawDataSize;
+    uint64_t MemoryAddress;
+    uint64_t MemoryDataSize;
+    uint32_t Type;
+    uint32_t Attributes;
+} TdvfSectionEntry;
+
+typedef struct {
+    uint32_t Signature;
+    uint32_t Length;
Quoted 8 lines

hw/i386/tdvf.c line 36

Quoted 4 lines
hw/i386/tdvf.c new file +4 -0
+    uint32_t Version;
+    uint32_t NumberOfSectionEntries;
+    TdvfSectionEntry SectionEntries[];
+} TdvfMetadata;

struct field names starting with an initial capital is not the usual QEMU code style. Can this be all initial lowercase, with capital just for word separation.

I think Isaku used such names because they are defined in TDVF Design Guide spec[1].

QEMU's internal data structure for metadata are below, which follows QEMU coding style.

	typedef struct TdxFirmwareEntry {
	    uint32_t data_offset;
	    uint32_t data_len;
	    uint64_t address;
	    uint64_t size;
	    uint32_t type;
	    uint32_t attributes;
	} TdxFirmwareEntry;

	typedef struct TdxFirmware {
	    uint32_t nr_entries;
	    TdxFirmwareEntry *entries;
	} TdxFirmware;

So if no strong preference, I would keep it as-is that the raw struct read from TDVF keeps the name convention of TDVF Design Guide spec and internal data struct uses QEMU's convention.

[1] https://cdrdv2.intel.com/v1/dl/getContent/733585

Quoted 9 lines

hw/i386/tdvf.c line 41

Quoted 6 lines
hw/i386/tdvf.c new file +5 -0
+
+struct tdx_metadata_offset {
+    uint32_t offset;
+};
+

With regards,
Daniel

#69 lore raw

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

Quoted 24 lines
Quoted 3 lines

struct field names starting with an initial capital is not the usual QEMU code style. Can this be all initial lowercase, with capital just for word separation.

I think Isaku used such names because they are defined in TDVF Design Guide spec[1].

QEMU's internal data structure for metadata are below, which follows QEMU coding style.

	typedef struct TdxFirmwareEntry {
	    uint32_t data_offset;
	    uint32_t data_len;
	    uint64_t address;
	    uint64_t size;
	    uint32_t type;
	    uint32_t attributes;
	} TdxFirmwareEntry;

	typedef struct TdxFirmware {
	    uint32_t nr_entries;
	    TdxFirmwareEntry *entries;
	} TdxFirmware;
Quoted 5 lines

So if no strong preference, I would keep it as-is that the raw struct read from TDVF keeps the name convention of TDVF Design Guide spec and internal data struct uses QEMU's convention.

[1] https://cdrdv2.intel.com/v1/dl/getContent/733585

Or you can add a simple comemnt about "the raw struct read from TDVF keeps the name convention of TDVF Design Guide spec", to remind other people that there is no need to clean up the style here. I have no objections; up to you.

Reviewed-by on v8 17/55
#70 lore raw

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

In reply to Xiaoyao Li
Quoted 46 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:27 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF
metadata
X-Mailer: git-send-email 2.34.1

From: Isaku Yamahata <[email protected]>

TDX VM needs to boot with its specialized firmware, Trusted Domain Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD guest memory prior to running the TDX VM.

A TDVF Metadata in TDVF image describes the structure of firmware. QEMU refers to it to setup memory for TDVF. Introduce function tdvf_parse_metadata() to parse the metadata from TDVF image and store the info of each TDVF section.

TDX metadata is located by a TDX metadata offset block, which is a GUID-ed structure. The data portion of the GUID structure contains only an 4-byte field that is the offset of TDX metadata to the end of firmware file.

Select X86_FW_OVMF when TDX is enable to leverage existing functions to parse and search OVMF's GUID-ed structures.

---
Changes in v8:
- Drop the failure handling of memcpy() since it cannot fail;

Changes in v7:
- Update license info to only use SPDX tag;
- use g_autofree to avoid manually free;

Changes in v6:
- Drop the the data endianness change for metadata->Length;

Changes in v1:
- rename tdvf_parse_section_entry() to
tdvf_parse_and_check_section_entry()

Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID

Quoted 8 lines
---
 hw/i386/Kconfig        |   1 +
 hw/i386/meson.build    |   1 +
 hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
 include/hw/i386/tdvf.h |  38 +++++++++
 4 files changed, 223 insertions(+)
 create mode 100644 hw/i386/tdvf.c
 create mode 100644 include/hw/i386/tdvf.h
author
#71 lore raw

PATCH v8 18/55 i386/tdx: Parse TDVF metadata for TDX VM

In reply to Xiaoyao Li

After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.

---
 hw/i386/pc_sysfw.c         | 7 +++++++
 target/i386/kvm/tdx-stub.c | 5 +++++
 target/i386/kvm/tdx.c      | 5 +++++
 target/i386/kvm/tdx.h      | 3 +++
 4 files changed, 20 insertions(+)
hw/i386/pc_sysfw.c +7 -0
index 1eeb58ab37f9..821396c16e91 100644
@@ -37,6 +37,7 @@
 #include "hw/block/flash.h"
 #include "system/kvm.h"
 #include "target/i386/sev.h"
+#include "kvm/tdx.h"
 
 #define FLASH_SECTOR_SIZE 4096
 
@@ -280,5 +281,11 @@ void x86_firmware_configure(hwaddr gpa, void *ptr, int size)
         }
 
         sev_encrypt_flash(gpa, ptr, size, &error_fatal);
+    } else if (is_tdx_vm()) {
+        ret = tdx_parse_tdvf(ptr, size);
+        if (ret) {
+            error_report("failed to parse TDVF for TDX VM");
+            exit(1);
+        }
     }
 }
target/i386/kvm/tdx-stub.c +5 -0
index 2344433594ea..7748b6d0a446 100644
@@ -8,3 +8,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
 {
     return -EINVAL;
 }
+
+int tdx_parse_tdvf(void *flash_ptr, int size)
+{
+    return -EINVAL;
+}
target/i386/kvm/tdx.c +5 -0
index b7f6f3b708b0..8d3475c6752c 100644
@@ -372,6 +372,11 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     return 0;
 }
 
+int tdx_parse_tdvf(void *flash_ptr, int size)
+{
+    return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
+}
+
 static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(obj);
target/i386/kvm/tdx.h +3 -0
index b73461b8d8a3..28a03c2a7b82 100644
@@ -8,6 +8,7 @@
 #endif
 
 #include "confidential-guest.h"
+#include "hw/i386/tdvf.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"
 #define TDX_GUEST(obj)  OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -32,6 +33,7 @@ typedef struct TdxGuest {
     char *mrownerconfig;    /* base64 encoded sha348 digest */
 
     MemoryRegion *tdvf_mr;
+    TdxFirmware tdvf;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
@@ -42,5 +44,6 @@ bool is_tdx_vm(void);
 
 int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
 void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
+int tdx_parse_tdvf(void *flash_ptr, int size);
 
 #endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Reviewed-by on v8 18/55
#72 lore raw

Re: [PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM

Quoted 10 lines · Xiaoyao Li

After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.

---
 hw/i386/pc_sysfw.c         | 7 +++++++
 target/i386/kvm/tdx-stub.c | 5 +++++
 target/i386/kvm/tdx.c      | 5 +++++
 target/i386/kvm/tdx.h      | 3 +++
 4 files changed, 20 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 18/55
#73 lore raw

Re: [PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM

In reply to Xiaoyao Li
Quoted 7 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:28 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM
X-Mailer: git-send-email 2.34.1

After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.

Quoted 8 lines
---
 hw/i386/pc_sysfw.c         | 7 +++++++
 target/i386/kvm/tdx-stub.c | 5 +++++
 target/i386/kvm/tdx.c      | 5 +++++
 target/i386/kvm/tdx.h      | 3 +++
 4 files changed, 20 insertions(+)
author
#74 lore raw

PATCH v8 19/55 i386/tdx: Don't initialize pc.rom for TDX VMs

In reply to Xiaoyao Li

For TDX, the address below 1MB are entirely general RAM. No need to initialize pc.rom memory region for TDs.

---
This is more as a workaround of the issue that for q35 machine type, the real memslot update (which requires memslot deletion )for pc.rom happens after tdx_init_memory_region. It leads to the private memory ADD'ed before get lost. I haven't work out a good solution to resolve the order issue. So just skip the pc.rom setup to avoid memslot deletion.

---
 hw/i386/pc.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)
hw/i386/pc.c +16 -13
index 01d0581f62a3..bcbbea235645 100644
@@ -43,6 +43,7 @@
 #include "system/xen.h"
 #include "system/reset.h"
 #include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
 #include "hw/xen/xen.h"
 #include "qobject/qlist.h"
 #include "qemu/error-report.h"
@@ -972,21 +973,23 @@ void pc_memory_init(PCMachineState *pcms,
     /* Initialize PC system firmware */
     pc_system_firmware_init(pcms, rom_memory);
 
-    option_rom_mr = g_malloc(sizeof(*option_rom_mr));
-    if (machine_require_guest_memfd(machine)) {
-        memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
-                                           PC_ROM_SIZE, &error_fatal);
-    } else {
-        memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
-                               &error_fatal);
-        if (pcmc->pci_enabled) {
-            memory_region_set_readonly(option_rom_mr, true);
+    if (!is_tdx_vm()) {
+        option_rom_mr = g_malloc(sizeof(*option_rom_mr));
+        if (machine_require_guest_memfd(machine)) {
+            memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
+                                            PC_ROM_SIZE, &error_fatal);
+        } else {
+            memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+                                &error_fatal);
+            if (pcmc->pci_enabled) {
+                memory_region_set_readonly(option_rom_mr, true);
+            }
         }
+        memory_region_add_subregion_overlap(rom_memory,
+                                            PC_ROM_MIN_VGA,
+                                            option_rom_mr,
+                                            1);
     }
-    memory_region_add_subregion_overlap(rom_memory,
-                                        PC_ROM_MIN_VGA,
-                                        option_rom_mr,
-                                        1);
 
     fw_cfg = fw_cfg_arch_create(machine,
                                 x86ms->boot_cpus, x86ms->apic_id_limit);
Signature · 1 line
2.34.1
Reviewed-by on v8 19/55
#75 lore raw

Re: [PATCH v8 19/55] i386/tdx: Don't initialize pc.rom for TDX VMs

Quoted 15 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:29 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 19/55] i386/tdx: Don't initialize pc.rom for TDX VMs
X-Mailer: git-send-email 2.34.1

For TDX, the address below 1MB are entirely general RAM. No need to initialize pc.rom memory region for TDs.

---
This is more as a workaround of the issue that for q35 machine type, the real memslot update (which requires memslot deletion )for pc.rom happens after tdx_init_memory_region. It leads to the private memory ADD'ed before get lost. I haven't work out a good solution to resolve the order issue. So just skip the pc.rom setup to avoid memslot deletion.

Quoted 3 lines
---
 hw/i386/pc.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

Fine for me,

author
#76 lore raw

PATCH v8 20/55 i386/tdx: Track mem_ptr for each firmware entry of TDVF

In reply to Xiaoyao Li

For each TDVF sections, QEMU needs to copy the content to guest private memory via KVM API (KVM_TDX_INIT_MEM_REGION).

Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory pointer of each TDVF sections. So that QEMU can add/copy them to guest private memory later.

TDVF sections can be classified into two groups:
- Firmware itself, e.g., TDVF BFV and CFV, that located separately from guest RAM. Its memory pointer is the bios pointer.

- Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB. mmap a new memory range for them.

Register a machine_init_done callback to do the stuff.

---
Changes in v8:
- Remove the duplicated header include;
- Add error handling for qemu_ram_mmap() failure;

---
 hw/i386/tdvf.c         |  1 +
 include/hw/i386/tdvf.h |  7 +++++++
 target/i386/kvm/tdx.c  | 37 +++++++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+)
hw/i386/tdvf.c +1 -0
index 328d1b7ffdf8..840c392c44ab 100644
@@ -174,6 +174,7 @@ int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size)
         }
     }
 
+    fw->mem_ptr = flash_ptr;
     return 0;
 
 err:
include/hw/i386/tdvf.h +7 -0
index 7ebcac42a36c..e75c8d1acc68 100644
@@ -26,13 +26,20 @@ typedef struct TdxFirmwareEntry {
     uint64_t size;
     uint32_t type;
     uint32_t attributes;
+
+    void *mem_ptr;
 } TdxFirmwareEntry;
 
 typedef struct TdxFirmware {
+    void *mem_ptr;
+
     uint32_t nr_entries;
     TdxFirmwareEntry *entries;
 } TdxFirmware;
 
+#define for_each_tdx_fw_entry(fw, e)    \
+    for (e = (fw)->entries; e != (fw)->entries + (fw)->nr_entries; e++)
+
 int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size);
 
 #endif /* HW_I386_TDVF_H */
target/i386/kvm/tdx.c +37 -0
index 8d3475c6752c..0885b99cfb88 100644
@@ -12,10 +12,13 @@
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
 #include "qemu/base64.h"
+#include "qemu/mmap-alloc.h"
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/system.h"
 
+#include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
 #include "tdx.h"
@@ -143,6 +146,38 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
     tdx_guest->tdvf_mr = tdvf_mr;
 }
 
+static void tdx_finalize_vm(Notifier *notifier, void *unused)
+{
+    TdxFirmware *tdvf = &tdx_guest->tdvf;
+    TdxFirmwareEntry *entry;
+
+    for_each_tdx_fw_entry(tdvf, entry) {
+        switch (entry->type) {
+        case TDVF_SECTION_TYPE_BFV:
+        case TDVF_SECTION_TYPE_CFV:
+            entry->mem_ptr = tdvf->mem_ptr + entry->data_offset;
+            break;
+        case TDVF_SECTION_TYPE_TD_HOB:
+        case TDVF_SECTION_TYPE_TEMP_MEM:
+            entry->mem_ptr = qemu_ram_mmap(-1, entry->size,
+                                           qemu_real_host_page_size(), 0, 0);
+            if (entry->mem_ptr == MAP_FAILED) {
+                error_report("Failed to mmap memory for TDVF section %d",
+                             entry->type);
+                exit(1);
+            }
+            break;
+        default:
+            error_report("Unsupported TDVF section %d", entry->type);
+            exit(1);
+        }
+    }
+}
+
+static Notifier tdx_machine_done_notify = {
+    .notify = tdx_finalize_vm,
+};
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(cgs);
@@ -157,6 +192,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         }
     }
 
+    qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
+
     tdx_guest = tdx;
     return 0;
 }
Signature · 1 line
2.34.1
Reviewed-by on v8 20/55
#77 lore raw

Re: [PATCH v8 20/55] i386/tdx: Track mem_ptr for each firmware entry of TDVF

Quoted 28 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:30 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 20/55] i386/tdx: Track mem_ptr for each firmware entry
of TDVF
X-Mailer: git-send-email 2.34.1

For each TDVF sections, QEMU needs to copy the content to guest private memory via KVM API (KVM_TDX_INIT_MEM_REGION).

Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory pointer of each TDVF sections. So that QEMU can add/copy them to guest private memory later.

TDVF sections can be classified into two groups:
- Firmware itself, e.g., TDVF BFV and CFV, that located separately from guest RAM. Its memory pointer is the bios pointer.

- Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB. mmap a new memory range for them.

Register a machine_init_done callback to do the stuff.

---
Changes in v8:
- Remove the duplicated header include;
- Add error handling for qemu_ram_mmap() failure;

Quoted 5 lines
---
 hw/i386/tdvf.c         |  1 +
 include/hw/i386/tdvf.h |  7 +++++++
 target/i386/kvm/tdx.c  | 37 +++++++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+)
author
#78 lore raw

PATCH v8 21/55 i386/tdx: Track RAM entries for TDX VM

In reply to Xiaoyao Li

The RAM of TDX VM can be classified into two types:

- TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be accepted by TDX guest before it can be used and will be all-zeros after being accepted.

- TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.

Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table and mark each RAM range as default type TDX_RAM_UNACCEPTED.

Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these ranges will be ADD'ed before TD runs and no need to be accepted runtime.

The TdxRamEntries[] are later used to setup the memory TD resource HOB that passes memory info from QEMU to TDVF.

---
Changes in v3:
- use enum TdxRamType in struct TdxRamEntry; (Isaku)
- Fix the indention; (Daniel)

Changes in v1:
- simplify the algorithm of tdx_accept_ram_range() (Suggested-by: Gerd Hoffman)
(1) Change the existing entry to cover the accepted ram range.
(2) If there is room before the accepted ram range add a TDX_RAM_UNACCEPTED entry for that.
(3) If there is room after the accepted ram range add a TDX_RAM_UNACCEPTED entry for that.

---
 target/i386/kvm/tdx.c | 111 ++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h |  14 ++++++
 2 files changed, 125 insertions(+)
target/i386/kvm/tdx.c +111 -0
index 0885b99cfb88..072ffcdfb41d 100644
@@ -18,6 +18,7 @@
 #include "crypto/hash.h"
 #include "system/system.h"
 
+#include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
@@ -146,11 +147,117 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
     tdx_guest->tdvf_mr = tdvf_mr;
 }
 
+static void tdx_add_ram_entry(uint64_t address, uint64_t length,
+                              enum TdxRamType type)
+{
+    uint32_t nr_entries = tdx_guest->nr_ram_entries;
+    tdx_guest->ram_entries = g_renew(TdxRamEntry, tdx_guest->ram_entries,
+                                     nr_entries + 1);
+
+    tdx_guest->ram_entries[nr_entries].address = address;
+    tdx_guest->ram_entries[nr_entries].length = length;
+    tdx_guest->ram_entries[nr_entries].type = type;
+    tdx_guest->nr_ram_entries++;
+}
+
+static int tdx_accept_ram_range(uint64_t address, uint64_t length)
+{
+    uint64_t head_start, tail_start, head_length, tail_length;
+    uint64_t tmp_address, tmp_length;
+    TdxRamEntry *e;
+    int i;
+
+    for (i = 0; i < tdx_guest->nr_ram_entries; i++) {
+        e = &tdx_guest->ram_entries[i];
+
+        if (address + length <= e->address ||
+            e->address + e->length <= address) {
+            continue;
+        }
+
+        /*
+         * The to-be-accepted ram range must be fully contained by one
+         * RAM entry.
+         */
+        if (e->address > address ||
+            e->address + e->length < address + length) {
+            return -EINVAL;
+        }
+
+        if (e->type == TDX_RAM_ADDED) {
+            return -EINVAL;
+        }
+
+        break;
+    }
+
+    if (i == tdx_guest->nr_ram_entries) {
+        return -1;
+    }
+
+    tmp_address = e->address;
+    tmp_length = e->length;
+
+    e->address = address;
+    e->length = length;
+    e->type = TDX_RAM_ADDED;
+
+    head_length = address - tmp_address;
+    if (head_length > 0) {
+        head_start = tmp_address;
+        tdx_add_ram_entry(head_start, head_length, TDX_RAM_UNACCEPTED);
+    }
+
+    tail_start = address + length;
+    if (tail_start < tmp_address + tmp_length) {
+        tail_length = tmp_address + tmp_length - tail_start;
+        tdx_add_ram_entry(tail_start, tail_length, TDX_RAM_UNACCEPTED);
+    }
+
+    return 0;
+}
+
+static int tdx_ram_entry_compare(const void *lhs_, const void* rhs_)
+{
+    const TdxRamEntry *lhs = lhs_;
+    const TdxRamEntry *rhs = rhs_;
+
+    if (lhs->address == rhs->address) {
+        return 0;
+    }
+    if (le64_to_cpu(lhs->address) > le64_to_cpu(rhs->address)) {
+        return 1;
+    }
+    return -1;
+}
+
+static void tdx_init_ram_entries(void)
+{
+    unsigned i, j, nr_e820_entries;
+
+    nr_e820_entries = e820_get_table(NULL);
+    tdx_guest->ram_entries = g_new(TdxRamEntry, nr_e820_entries);
+
+    for (i = 0, j = 0; i < nr_e820_entries; i++) {
+        uint64_t addr, len;
+
+        if (e820_get_entry(i, E820_RAM, &addr, &len)) {
+            tdx_guest->ram_entries[j].address = addr;
+            tdx_guest->ram_entries[j].length = len;
+            tdx_guest->ram_entries[j].type = TDX_RAM_UNACCEPTED;
+            j++;
+        }
+    }
+    tdx_guest->nr_ram_entries = j;
+}
+
 static void tdx_finalize_vm(Notifier *notifier, void *unused)
 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
     TdxFirmwareEntry *entry;
 
+    tdx_init_ram_entries();
+
     for_each_tdx_fw_entry(tdvf, entry) {
         switch (entry->type) {
         case TDVF_SECTION_TYPE_BFV:
@@ -166,12 +273,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
                              entry->type);
                 exit(1);
             }
+            tdx_accept_ram_range(entry->address, entry->size);
             break;
         default:
             error_report("Unsupported TDVF section %d", entry->type);
             exit(1);
         }
     }
+
+    qsort(tdx_guest->ram_entries, tdx_guest->nr_ram_entries,
+          sizeof(TdxRamEntry), &tdx_ram_entry_compare);
 }
 
 static Notifier tdx_machine_done_notify = {
target/i386/kvm/tdx.h +14 -0
index 28a03c2a7b82..36a7400e7480 100644
@@ -20,6 +20,17 @@ typedef struct TdxGuestClass {
 /* TDX requires bus frequency 25MHz */
 #define TDX_APIC_BUS_CYCLES_NS 40
 
+enum TdxRamType {
+    TDX_RAM_UNACCEPTED,
+    TDX_RAM_ADDED,
+};
+
+typedef struct TdxRamEntry {
+    uint64_t address;
+    uint64_t length;
+    enum TdxRamType type;
+} TdxRamEntry;
+
 typedef struct TdxGuest {
     X86ConfidentialGuest parent_obj;
 
@@ -34,6 +45,9 @@ typedef struct TdxGuest {
 
     MemoryRegion *tdvf_mr;
     TdxFirmware tdvf;
+
+    uint32_t nr_ram_entries;
+    TdxRamEntry *ram_entries;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
Signature · 1 line
2.34.1
Reviewed-by on v8 21/55
#79 lore raw

Re: [PATCH v8 21/55] i386/tdx: Track RAM entries for TDX VM

Quoted 26 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:31 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 21/55] i386/tdx: Track RAM entries for TDX VM
X-Mailer: git-send-email 2.34.1

The RAM of TDX VM can be classified into two types:

- TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be accepted by TDX guest before it can be used and will be all-zeros after being accepted.

- TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.

Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table and mark each RAM range as default type TDX_RAM_UNACCEPTED.

Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these ranges will be ADD'ed before TD runs and no need to be accepted runtime.

The TdxRamEntries[] are later used to setup the memory TD resource HOB that passes memory info from QEMU to TDVF.

---

[snip]

Quoted 11 lines

static void tdx_finalize_vm(Notifier *notifier, void *unused)

 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
     TdxFirmwareEntry *entry;
target/i386/kvm/tdx.c +2 -0
+    tdx_init_ram_entries();
+
     for_each_tdx_fw_entry(tdvf, entry) {
         switch (entry->type) {
         case TDVF_SECTION_TYPE_BFV:

target/i386/kvm/tdx.c static void tdx_finalize_vm(Notifier *notifier, void *unused) · line 276

Quoted 4 lines
target/i386/kvm/tdx.c +1 -0
@@ -166,12 +273,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
                              entry->type);
                 exit(1);
             }
+            tdx_accept_ram_range(entry->address, entry->size);

It's better to catch & report the failure case here.

Review static void tdx_finalize_vm(Notifier *notifier, void *unused) · line 277

Quoted 1 line
(quoted diff) +0 -0
             break;

Others LGTM,

author
#80 lore raw

PATCH v8 22/55 headers: Add definitions from UEFI spec for volumes, resources, etc...

In reply to Xiaoyao Li

Add UEFI definitions for literals, enums, structs, GUIDs, etc... that will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed to the Trusted Domain Virtual Firmware (TDVF).

All values come from the UEFI specification [1], PI spec [2] and TDVF design guide[3].

[1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
[3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf

---
Changes in v7:
- use SPDX tag;

---
 include/standard-headers/uefi/uefi.h | 187 +++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)
 create mode 100644 include/standard-headers/uefi/uefi.h
include/standard-headers/uefi/uefi.h new file +187 -0
new file mode 100644
index 000000000000..5256349ec0b6
@@ -0,0 +1,187 @@
+/*
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *         Xiaoyao Li <[email protected]>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_I386_UEFI_H
+#define HW_I386_UEFI_H
+
+/***************************************************************************/
+/*
+ * basic EFI definitions
+ * supplemented with UEFI Specification Version 2.8 (Errata A)
+ * released February 2020
+ */
+/* UEFI integer is little endian */
+
+typedef struct {
+    uint32_t Data1;
+    uint16_t Data2;
+    uint16_t Data3;
+    uint8_t Data4[8];
+} EFI_GUID;
+
+typedef enum {
+    EfiReservedMemoryType,
+    EfiLoaderCode,
+    EfiLoaderData,
+    EfiBootServicesCode,
+    EfiBootServicesData,
+    EfiRuntimeServicesCode,
+    EfiRuntimeServicesData,
+    EfiConventionalMemory,
+    EfiUnusableMemory,
+    EfiACPIReclaimMemory,
+    EfiACPIMemoryNVS,
+    EfiMemoryMappedIO,
+    EfiMemoryMappedIOPortSpace,
+    EfiPalCode,
+    EfiPersistentMemory,
+    EfiUnacceptedMemoryType,
+    EfiMaxMemoryType
+} EFI_MEMORY_TYPE;
+
+#define EFI_HOB_HANDOFF_TABLE_VERSION 0x0009
+
+#define EFI_HOB_TYPE_HANDOFF              0x0001
+#define EFI_HOB_TYPE_MEMORY_ALLOCATION    0x0002
+#define EFI_HOB_TYPE_RESOURCE_DESCRIPTOR  0x0003
+#define EFI_HOB_TYPE_GUID_EXTENSION       0x0004
+#define EFI_HOB_TYPE_FV                   0x0005
+#define EFI_HOB_TYPE_CPU                  0x0006
+#define EFI_HOB_TYPE_MEMORY_POOL          0x0007
+#define EFI_HOB_TYPE_FV2                  0x0009
+#define EFI_HOB_TYPE_LOAD_PEIM_UNUSED     0x000A
+#define EFI_HOB_TYPE_UEFI_CAPSULE         0x000B
+#define EFI_HOB_TYPE_FV3                  0x000C
+#define EFI_HOB_TYPE_UNUSED               0xFFFE
+#define EFI_HOB_TYPE_END_OF_HOB_LIST      0xFFFF
+
+typedef struct {
+    uint16_t HobType;
+    uint16_t HobLength;
+    uint32_t Reserved;
+} EFI_HOB_GENERIC_HEADER;
+
+typedef uint64_t EFI_PHYSICAL_ADDRESS;
+typedef uint32_t EFI_BOOT_MODE;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    uint32_t Version;
+    EFI_BOOT_MODE BootMode;
+    EFI_PHYSICAL_ADDRESS EfiMemoryTop;
+    EFI_PHYSICAL_ADDRESS EfiMemoryBottom;
+    EFI_PHYSICAL_ADDRESS EfiFreeMemoryTop;
+    EFI_PHYSICAL_ADDRESS EfiFreeMemoryBottom;
+    EFI_PHYSICAL_ADDRESS EfiEndOfHobList;
+} EFI_HOB_HANDOFF_INFO_TABLE;
+
+#define EFI_RESOURCE_SYSTEM_MEMORY          0x00000000
+#define EFI_RESOURCE_MEMORY_MAPPED_IO       0x00000001
+#define EFI_RESOURCE_IO                     0x00000002
+#define EFI_RESOURCE_FIRMWARE_DEVICE        0x00000003
+#define EFI_RESOURCE_MEMORY_MAPPED_IO_PORT  0x00000004
+#define EFI_RESOURCE_MEMORY_RESERVED        0x00000005
+#define EFI_RESOURCE_IO_RESERVED            0x00000006
+#define EFI_RESOURCE_MEMORY_UNACCEPTED      0x00000007
+#define EFI_RESOURCE_MAX_MEMORY_TYPE        0x00000008
+
+#define EFI_RESOURCE_ATTRIBUTE_PRESENT                  0x00000001
+#define EFI_RESOURCE_ATTRIBUTE_INITIALIZED              0x00000002
+#define EFI_RESOURCE_ATTRIBUTE_TESTED                   0x00000004
+#define EFI_RESOURCE_ATTRIBUTE_SINGLE_BIT_ECC           0x00000008
+#define EFI_RESOURCE_ATTRIBUTE_MULTIPLE_BIT_ECC         0x00000010
+#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_1           0x00000020
+#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_2           0x00000040
+#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTED           0x00000080
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTED          0x00000100
+#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTED      0x00000200
+#define EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE              0x00000400
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_COMBINEABLE        0x00000800
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_THROUGH_CACHEABLE  0x00001000
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_BACK_CACHEABLE     0x00002000
+#define EFI_RESOURCE_ATTRIBUTE_16_BIT_IO                0x00004000
+#define EFI_RESOURCE_ATTRIBUTE_32_BIT_IO                0x00008000
+#define EFI_RESOURCE_ATTRIBUTE_64_BIT_IO                0x00010000
+#define EFI_RESOURCE_ATTRIBUTE_UNCACHED_EXPORTED        0x00020000
+#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTED      0x00040000
+#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTABLE    0x00080000
+#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTABLE         0x00100000
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTABLE        0x00200000
+#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTABLE    0x00400000
+#define EFI_RESOURCE_ATTRIBUTE_PERSISTENT               0x00800000
+#define EFI_RESOURCE_ATTRIBUTE_PERSISTABLE              0x01000000
+#define EFI_RESOURCE_ATTRIBUTE_MORE_RELIABLE            0x02000000
+
+typedef uint32_t EFI_RESOURCE_TYPE;
+typedef uint32_t EFI_RESOURCE_ATTRIBUTE_TYPE;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_GUID Owner;
+    EFI_RESOURCE_TYPE ResourceType;
+    EFI_RESOURCE_ATTRIBUTE_TYPE ResourceAttribute;
+    EFI_PHYSICAL_ADDRESS PhysicalStart;
+    uint64_t ResourceLength;
+} EFI_HOB_RESOURCE_DESCRIPTOR;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_GUID Name;
+
+    /* guid specific data follows */
+} EFI_HOB_GUID_TYPE;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+} EFI_HOB_FIRMWARE_VOLUME;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+    EFI_GUID FvName;
+    EFI_GUID FileName;
+} EFI_HOB_FIRMWARE_VOLUME2;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+    uint32_t AuthenticationStatus;
+    bool ExtractedFv;
+    EFI_GUID FvName;
+    EFI_GUID FileName;
+} EFI_HOB_FIRMWARE_VOLUME3;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    uint8_t SizeOfMemorySpace;
+    uint8_t SizeOfIoSpace;
+    uint8_t Reserved[6];
+} EFI_HOB_CPU;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+} EFI_HOB_MEMORY_POOL;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+} EFI_HOB_UEFI_CAPSULE;
+
+#define EFI_HOB_OWNER_ZERO                                      \
+    ((EFI_GUID){ 0x00000000, 0x0000, 0x0000,                    \
+        { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 } })
+
+#endif
Signature · 1 line
2.34.1
Reviewed-by on v8 22/55
#81 lore raw

Re: [PATCH v8 22/55] headers: Add definitions from UEFI spec for volumes, resources, etc...

Quoted 19 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:32 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 22/55] headers: Add definitions from UEFI spec for
volumes, resources, etc...
X-Mailer: git-send-email 2.34.1

Add UEFI definitions for literals, enums, structs, GUIDs, etc... that will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed to the Trusted Domain Virtual Firmware (TDVF).

All values come from the UEFI specification [1], PI spec [2] and TDVF design guide[3].

[1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
[3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf

Quoted 7 lines

---
Changes in v7:
- use SPDX tag;

---
 include/standard-headers/uefi/uefi.h | 187 +++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)
 create mode 100644 include/standard-headers/uefi/uefi.h
author
#82 lore raw

PATCH v8 23/55 i386/tdx: Setup the TD HOB list

In reply to Xiaoyao Li

The TD HOB list is used to pass the information from VMM to TDVF. The TD HOB must include PHIT HOB and Resource Descriptor HOB. More details can be found in TDVF specification and PI specification.

Build the TD HOB in TDX's machine_init_done callback.

---
Changes in v7:
- use SPDX tag for license info;
- clean up the included headers;

Changes in v1:
- drop the code of adding mmio resources since OVMF prepares all the MMIO hob itself.

---
 hw/i386/meson.build   |   2 +-
 hw/i386/tdvf-hob.c    | 130 ++++++++++++++++++++++++++++++++++++++++++
 hw/i386/tdvf-hob.h    |  26 +++++++++
 target/i386/kvm/tdx.c |  16 ++++++
 4 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100644 hw/i386/tdvf-hob.c
 create mode 100644 hw/i386/tdvf-hob.h
hw/i386/meson.build +1 -1
index 3bc1da2b6eb4..7896f348cff8 100644
@@ -32,7 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
   'port92.c'))
 i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
                                         if_false: files('pc_sysfw_ovmf-stubs.c'))
-i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
 
 subdir('kvm')
 subdir('xen')
hw/i386/tdvf-hob.c new file +130 -0

130 lines not on this page · raw message

hw/i386/tdvf-hob.h new file +26 -0
new file mode 100644
index 000000000000..4fc6a3740a57
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef HW_I386_TD_HOB_H
+#define HW_I386_TD_HOB_H
+
+#include "hw/i386/tdvf.h"
+#include "target/i386/kvm/tdx.h"
+
+void tdvf_hob_create(TdxGuest *tdx, TdxFirmwareEntry *td_hob);
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE     \
+    (EFI_RESOURCE_ATTRIBUTE_PRESENT |           \
+     EFI_RESOURCE_ATTRIBUTE_INITIALIZED |       \
+     EFI_RESOURCE_ATTRIBUTE_TESTED)
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED  \
+    (EFI_RESOURCE_ATTRIBUTE_PRESENT |           \
+     EFI_RESOURCE_ATTRIBUTE_INITIALIZED |       \
+     EFI_RESOURCE_ATTRIBUTE_TESTED)
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_MMIO        \
+    (EFI_RESOURCE_ATTRIBUTE_PRESENT     |       \
+     EFI_RESOURCE_ATTRIBUTE_INITIALIZED |       \
+     EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE)
+
+#endif
target/i386/kvm/tdx.c +16 -0
index 072ffcdfb41d..2e8d32285d9a 100644
@@ -21,6 +21,7 @@
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
+#include "hw/i386/tdvf-hob.h"
 #include "kvm_i386.h"
 #include "tdx.h"
 
@@ -147,6 +148,19 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
     tdx_guest->tdvf_mr = tdvf_mr;
 }
 
+static TdxFirmwareEntry *tdx_get_hob_entry(TdxGuest *tdx)
+{
+    TdxFirmwareEntry *entry;
+
+    for_each_tdx_fw_entry(&tdx->tdvf, entry) {
+        if (entry->type == TDVF_SECTION_TYPE_TD_HOB) {
+            return entry;
+        }
+    }
+    error_report("TDVF metadata doesn't specify TD_HOB location.");
+    exit(1);
+}
+
 static void tdx_add_ram_entry(uint64_t address, uint64_t length,
                               enum TdxRamType type)
 {
@@ -283,6 +297,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
 
     qsort(tdx_guest->ram_entries, tdx_guest->nr_ram_entries,
           sizeof(TdxRamEntry), &tdx_ram_entry_compare);
+
+    tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
 }
 
 static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
Reviewed-by on v8 23/55
#83 lore raw

Re: [PATCH v8 23/55] i386/tdx: Setup the TD HOB list

Quoted 25 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:33 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 23/55] i386/tdx: Setup the TD HOB list
X-Mailer: git-send-email 2.34.1

The TD HOB list is used to pass the information from VMM to TDVF. The TD HOB must include PHIT HOB and Resource Descriptor HOB. More details can be found in TDVF specification and PI specification.

Build the TD HOB in TDX's machine_init_done callback.

---
Changes in v7:
- use SPDX tag for license info;
- clean up the included headers;

Changes in v1:
- drop the code of adding mmio resources since OVMF prepares all the MMIO hob itself.

Quoted 8 lines
---
 hw/i386/meson.build   |   2 +-
 hw/i386/tdvf-hob.c    | 130 ++++++++++++++++++++++++++++++++++++++++++
 hw/i386/tdvf-hob.h    |  26 +++++++++
 target/i386/kvm/tdx.c |  16 ++++++
 4 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100644 hw/i386/tdvf-hob.c
 create mode 100644 hw/i386/tdvf-hob.h
author
#84 lore raw

PATCH v8 24/55 i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

TDVF firmware (CODE and VARS) needs to be copied to TD's private memory via KVM_TDX_INIT_MEM_REGION, as well as TD HOB and TEMP memory.

If the TDVF section has TDVF_SECTION_ATTRIBUTES_MR_EXTEND set in the flag, calling KVM_TDX_EXTEND_MEMORY to extend the measurement.

After populating the TDVF memory, the original image located in shared ramblock can be discarded.

---
Changes in v6:
- switch back to use KVM_TDX_INIT_MEM_REGION according to KVM's change;

---
 target/i386/kvm/tdx.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
target/i386/kvm/tdx.c +42 -0
index 2e8d32285d9a..4cf0929d22aa 100644
@@ -17,6 +17,7 @@
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
 #include "system/system.h"
+#include "exec/ramblock.h"
 
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
@@ -269,6 +270,9 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
     TdxFirmwareEntry *entry;
+    RAMBlock *ram_block;
+    Error *local_err = NULL;
+    int r;
 
     tdx_init_ram_entries();
 
@@ -299,6 +303,44 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
           sizeof(TdxRamEntry), &tdx_ram_entry_compare);
 
     tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
+
+    for_each_tdx_fw_entry(tdvf, entry) {
+        struct kvm_tdx_init_mem_region region;
+        uint32_t flags;
+
+        region = (struct kvm_tdx_init_mem_region) {
+            .source_addr = (uint64_t)entry->mem_ptr,
+            .gpa = entry->address,
+            .nr_pages = entry->size >> 12,
+        };
+
+        flags = entry->attributes & TDVF_SECTION_ATTRIBUTES_MR_EXTEND ?
+                KVM_TDX_MEASURE_MEMORY_REGION : 0;
+
+        do {
+            error_free(local_err);
+            local_err = NULL;
+            r = tdx_vcpu_ioctl(first_cpu, KVM_TDX_INIT_MEM_REGION, flags,
+                               &region, &local_err);
+        } while (r == -EAGAIN || r == -EINTR);
+        if (r < 0) {
+            error_report_err(local_err);
+            exit(1);
+        }
+
+        if (entry->type == TDVF_SECTION_TYPE_TD_HOB ||
+            entry->type == TDVF_SECTION_TYPE_TEMP_MEM) {
+            qemu_ram_munmap(-1, entry->mem_ptr, entry->size);
+            entry->mem_ptr = NULL;
+        }
+    }
+
+    /*
+     * TDVF image has been copied into private region above via
+     * KVM_MEMORY_MAPPING. It becomes useless.
+     */
+    ram_block = tdx_guest->tdvf_mr->ram_block;
+    ram_block_discard_range(ram_block, 0, ram_block->max_length);
 }
 
 static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
Reviewed-by on v8 24/55
#85 lore raw

Re: [PATCH v8 24/55] i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION

Quoted 20 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:34 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 24/55] i386/tdx: Add TDVF memory via
KVM_TDX_INIT_MEM_REGION
X-Mailer: git-send-email 2.34.1

From: Isaku Yamahata <[email protected]>

TDVF firmware (CODE and VARS) needs to be copied to TD's private memory via KVM_TDX_INIT_MEM_REGION, as well as TD HOB and TEMP memory.

If the TDVF section has TDVF_SECTION_ATTRIBUTES_MR_EXTEND set in the flag, calling KVM_TDX_EXTEND_MEMORY to extend the measurement.

After populating the TDVF memory, the original image located in shared ramblock can be discarded.

Quoted 6 lines

---
Changes in v6:
- switch back to use KVM_TDX_INIT_MEM_REGION according to KVM's change;

---
 target/i386/kvm/tdx.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
author
#86 lore raw

PATCH v8 25/55 i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu

In reply to Xiaoyao Li

TDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.

KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for each vcpu after HOB list is created.

---
 target/i386/kvm/tdx.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
target/i386/kvm/tdx.c +14 -0
index 4cf0929d22aa..281fcba9f031 100644
@@ -266,6 +266,18 @@ static void tdx_init_ram_entries(void)
     tdx_guest->nr_ram_entries = j;
 }
 
+static void tdx_post_init_vcpus(void)
+{
+    TdxFirmwareEntry *hob;
+    CPUState *cpu;
+
+    hob = tdx_get_hob_entry(tdx_guest);
+    CPU_FOREACH(cpu) {
+        tdx_vcpu_ioctl(cpu, KVM_TDX_INIT_VCPU, 0, (void *)hob->address,
+                       &error_fatal);
+    }
+}
+
 static void tdx_finalize_vm(Notifier *notifier, void *unused)
 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
@@ -304,6 +316,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
 
     tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
 
+    tdx_post_init_vcpus();
+
     for_each_tdx_fw_entry(tdvf, entry) {
         struct kvm_tdx_init_mem_region region;
         uint32_t flags;
Signature · 1 line
2.34.1
Reviewed-by on v8 25/55
#87 lore raw

Re: [PATCH v8 25/55] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu

Quoted 9 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:35 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 25/55] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize
TDX vcpu
X-Mailer: git-send-email 2.34.1

TDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.

Quoted 8 lines

KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for each vcpu after HOB list is created.

---
 target/i386/kvm/tdx.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
author
#88 lore raw

PATCH v8 26/55 i386/tdx: Finalize TDX VM

In reply to Xiaoyao Li

Invoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make the TD vCPUs runnable once machine initialization is complete.

---
 target/i386/kvm/tdx.c | 3 +++
 1 file changed, 3 insertions(+)
target/i386/kvm/tdx.c +3 -0
index 281fcba9f031..78764b1d4be4 100644
@@ -355,6 +355,9 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
      */
     ram_block = tdx_guest->tdvf_mr->ram_block;
     ram_block_discard_range(ram_block, 0, ram_block->max_length);
+
+    tdx_vm_ioctl(KVM_TDX_FINALIZE_VM, 0, NULL, &error_fatal);
+    CONFIDENTIAL_GUEST_SUPPORT(tdx_guest)->ready = true;
 }
 
 static Notifier tdx_machine_done_notify = {
Signature · 1 line
2.34.1
Reviewed-by on v8 26/55
#89 lore raw

Re: [PATCH v8 26/55] i386/tdx: Finalize TDX VM

Quoted 5 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:36 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 26/55] i386/tdx: Finalize TDX VM
X-Mailer: git-send-email 2.34.1

Quoted 8 lines

Invoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make the TD vCPUs runnable once machine initialization is complete.

---
 target/i386/kvm/tdx.c | 3 +++
 1 file changed, 3 insertions(+)
author
#90 lore raw

PATCH v8 27/55 i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE

In reply to Xiaoyao Li

KVM translates TDG.VP.VMCALL<MapGPA> to KVM_HC_MAP_GPA_RANGE, and QEMU needs to enable user exit on KVM_HC_MAP_GPA_RANGE in order to handle the memory conversion requested by TD guest.

---
changes in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)
target/i386/kvm/tdx.c +7 -0
index 78764b1d4be4..f8953f598584 100644
@@ -19,6 +19,8 @@
 #include "system/system.h"
 #include "exec/ramblock.h"
 
+#include <linux/kvm_para.h>
+
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
@@ -378,6 +380,11 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         }
     }
 
+    /* TDX relies on KVM_HC_MAP_GPA_RANGE to handle TDG.VP.VMCALL<MapGPA> */
+    if (!kvm_enable_hypercall(BIT_ULL(KVM_HC_MAP_GPA_RANGE))) {
+        return -EOPNOTSUPP;
+    }
+
     qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
 
     tdx_guest = tdx;
Signature · 1 line
2.34.1
Reviewed-by on v8 27/55
#91 lore raw

Re: [PATCH v8 27/55] i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE

Quoted 9 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:37 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 27/55] i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE
X-Mailer: git-send-email 2.34.1

KVM translates TDG.VP.VMCALL<MapGPA> to KVM_HC_MAP_GPA_RANGE, and QEMU needs to enable user exit on KVM_HC_MAP_GPA_RANGE in order to handle the memory conversion requested by TD guest.

Quoted 7 lines

---
changes in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)
author
#92 lore raw

PATCH v8 28/55 i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL

In reply to Xiaoyao Li

TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request termination. KVM translates such request into KVM_EXIT_SYSTEM_EVENT with type of KVM_SYSTEM_EVENT_TDX_FATAL.

Add hanlder for such exit. Parse and print the error message, and terminate the TD guest in the handler.

---
Changes in v8:
- update to the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL;

Changes in v6:
- replace the patch " i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>" in v5;

---
 target/i386/kvm/kvm.c      | 10 +++++++++
 target/i386/kvm/tdx-stub.c |  5 +++++
 target/i386/kvm/tdx.c      | 45 ++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h      |  2 ++
 4 files changed, 62 insertions(+)
target/i386/kvm/kvm.c +10 -0
index 7de5014051eb..a76f34537908 100644
@@ -6128,6 +6128,16 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
     case KVM_EXIT_HYPERCALL:
         ret = kvm_handle_hypercall(run);
         break;
+    case KVM_EXIT_SYSTEM_EVENT:
+        switch (run->system_event.type) {
+        case KVM_SYSTEM_EVENT_TDX_FATAL:
+            ret = tdx_handle_report_fatal_error(cpu, run);
+            break;
+        default:
+            ret = -1;
+            break;
+        }
+        break;
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
target/i386/kvm/tdx-stub.c +5 -0
index 7748b6d0a446..720a4ff046ee 100644
@@ -13,3 +13,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
 {
     return -EINVAL;
 }
+
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    return -EINVAL;
+}
target/i386/kvm/tdx.c +45 -0

51 lines not on this page · raw message

target/i386/kvm/tdx.h +2 -0
index 36a7400e7480..04b5afe199f9 100644
@@ -8,6 +8,7 @@
 #endif
 
 #include "confidential-guest.h"
+#include "cpu.h"
 #include "hw/i386/tdvf.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"
@@ -59,5 +60,6 @@ bool is_tdx_vm(void);
 int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
 void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
 int tdx_parse_tdvf(void *flash_ptr, int size);
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run);
 
 #endif /* QEMU_I386_TDX_H */
Signature · 1 line
2.34.1
Reviewed-by on v8 28/55
#93 lore raw

Re: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL

Quoted 77 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:38 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL
X-Mailer: git-send-email 2.34.1

TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request termination. KVM translates such request into KVM_EXIT_SYSTEM_EVENT with type of KVM_SYSTEM_EVENT_TDX_FATAL.

Add hanlder for such exit. Parse and print the error message, and terminate the TD guest in the handler.

---
Changes in v8:
- update to the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL;

Changes in v6:
- replace the patch " i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>" in v5;

---
 target/i386/kvm/kvm.c      | 10 +++++++++
 target/i386/kvm/tdx-stub.c |  5 +++++
 target/i386/kvm/tdx.c      | 45 ++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h      |  2 ++
 4 files changed, 62 insertions(+)
target/i386/kvm/kvm.c +10 -0
index 7de5014051eb..a76f34537908 100644
@@ -6128,6 +6128,16 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
     case KVM_EXIT_HYPERCALL:
         ret = kvm_handle_hypercall(run);
         break;
+    case KVM_EXIT_SYSTEM_EVENT:
+        switch (run->system_event.type) {
+        case KVM_SYSTEM_EVENT_TDX_FATAL:
+            ret = tdx_handle_report_fatal_error(cpu, run);
+            break;
+        default:
+            ret = -1;
+            break;
+        }
+        break;
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
target/i386/kvm/tdx-stub.c +5 -0
index 7748b6d0a446..720a4ff046ee 100644
@@ -13,3 +13,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
 {
     return -EINVAL;
 }
+
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    return -EINVAL;
+}
target/i386/kvm/tdx.c +8 -0
index f8953f598584..74b7e3ac85fe 100644
@@ -607,6 +607,51 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
     return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
 }
 
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    uint64_t error_code = run->system_event.data[R_R12];
+    uint64_t reg_mask = run->system_event.data[R_ECX];
+    char *message = NULL;
+    uint64_t *tmp;
+
+    if (error_code & 0xffff) {

target/i386/kvm/tdx.c int tdx_parse_tdvf(void *flash_ptr, int size) · line 627

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
+                     error_code);
+        return -1;
+    }
+
+/*
+ * Only 8 registers can contain valid ASCII byte stream to form the fatal
+ * message, and their sequence is: R14, R15, RBX, RDI, RSI, R8, R9, RDX
+ */
+#define TDX_FATAL_MESSAGE_MAX        64

At least, for this macro, and TDX_REPORT_FATAL_ERROR_GPA_VALID in later patch, could we move these simple macro definitions out of the function?

This could improve the readability for this one function.

Quoted 17 lines
target/i386/kvm/tdx.c +17 -0
+    if (reg_mask) {
+        message = g_malloc0(TDX_FATAL_MESSAGE_MAX + 1);
+        tmp = (uint64_t *)message;
+
+#define COPY_REG(REG)                               \
+    do {                                            \
+        if (reg_mask & BIT_ULL(REG)) {              \
+            *(tmp++) = run->system_event.data[REG]; \
+        }                                           \
+    } while (0)
+
+        COPY_REG(R_R14);
+        COPY_REG(R_R15);
+        COPY_REG(R_EBX);
+        COPY_REG(R_EDI);
+        COPY_REG(R_ESI);
+        COPY_REG(R_R8);

target/i386/kvm/tdx.c int tdx_parse_tdvf(void *flash_ptr, int size) · line 654

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        COPY_REG(R_R9);
+        COPY_REG(R_EDX);
+        *((char *)tmp) = '\0';
+    }
+#undef COPY_REG
+
+    error_report("TD guest reports fatal error. %s", message ? : "");
+    return -1;
+}
+

Otherwise,

author
#94 lore raw

Re: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL

Quoted 92 lines · Zhao Liu
Quoted 77 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:38 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL
X-Mailer: git-send-email 2.34.1

TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request termination. KVM translates such request into KVM_EXIT_SYSTEM_EVENT with type of KVM_SYSTEM_EVENT_TDX_FATAL.

Add hanlder for such exit. Parse and print the error message, and terminate the TD guest in the handler.

---
Changes in v8:
- update to the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL;

Changes in v6:
- replace the patch " i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>"

    in v5;
---
  target/i386/kvm/kvm.c      | 10 +++++++++
  target/i386/kvm/tdx-stub.c |  5 +++++
  target/i386/kvm/tdx.c      | 45 ++++++++++++++++++++++++++++++++++++++
  target/i386/kvm/tdx.h      |  2 ++
  4 files changed, 62 insertions(+)
target/i386/kvm/kvm.c +10 -0
index 7de5014051eb..a76f34537908 100644
@@ -6128,6 +6128,16 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
      case KVM_EXIT_HYPERCALL:
          ret = kvm_handle_hypercall(run);
          break;
+    case KVM_EXIT_SYSTEM_EVENT:
+        switch (run->system_event.type) {
+        case KVM_SYSTEM_EVENT_TDX_FATAL:
+            ret = tdx_handle_report_fatal_error(cpu, run);
+            break;
+        default:
+            ret = -1;
+            break;
+        }
+        break;
      default:
          fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
          ret = -1;
target/i386/kvm/tdx-stub.c +5 -0
index 7748b6d0a446..720a4ff046ee 100644
@@ -13,3 +13,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
  {
      return -EINVAL;
  }
+
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    return -EINVAL;
+}
target/i386/kvm/tdx.c +8 -0
index f8953f598584..74b7e3ac85fe 100644
@@ -607,6 +607,51 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
      return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
  }
 
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    uint64_t error_code = run->system_event.data[R_R12];
+    uint64_t reg_mask = run->system_event.data[R_ECX];
+    char *message = NULL;
+    uint64_t *tmp;
+
+    if (error_code & 0xffff) {

target/i386/kvm/tdx.c int tdx_parse_tdvf(void *flash_ptr, int size) · line 627

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
+                     error_code);
+        return -1;
+    }
+
+/*
+ * Only 8 registers can contain valid ASCII byte stream to form the fatal
+ * message, and their sequence is: R14, R15, RBX, RDI, RSI, R8, R9, RDX
+ */
+#define TDX_FATAL_MESSAGE_MAX        64

At least, for this macro, and TDX_REPORT_FATAL_ERROR_GPA_VALID in later patch, could we move these simple macro definitions out of the function?

Quoted 1 line

This could improve the readability for this one function.

To me, it's common that put the one-off MACRO right before the line it is used.

Anyway, it's not a big deal to move them out of the function. So I will move them for your preference.

Quoted 32 lines
Quoted 17 lines
target/i386/kvm/tdx.c +17 -0
+    if (reg_mask) {
+        message = g_malloc0(TDX_FATAL_MESSAGE_MAX + 1);
+        tmp = (uint64_t *)message;
+
+#define COPY_REG(REG)                               \
+    do {                                            \
+        if (reg_mask & BIT_ULL(REG)) {              \
+            *(tmp++) = run->system_event.data[REG]; \
+        }                                           \
+    } while (0)
+
+        COPY_REG(R_R14);
+        COPY_REG(R_R15);
+        COPY_REG(R_EBX);
+        COPY_REG(R_EDI);
+        COPY_REG(R_ESI);
+        COPY_REG(R_R8);

target/i386/kvm/tdx.c int tdx_parse_tdvf(void *flash_ptr, int size) · line 654

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        COPY_REG(R_R9);
+        COPY_REG(R_EDX);
+        *((char *)tmp) = '\0';
+    }
+#undef COPY_REG
+
+    error_report("TD guest reports fatal error. %s", message ? : "");
+    return -1;
+}
+

Otherwise,

author
#95 lore raw

PATCH v8 29/55 i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility

In reply to Xiaoyao Li

Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility

Originated-from: Isaku Yamahata <[email protected]>

---
Changes in v8:
- use g_strdup() for copy string;
- use the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL to grab gpa info;

Changes in v6:
- change error_code of GuestPanicInformationTdx from uint64_t to uint32_t, to only contains the bit 31:0 returned in r12.

Changes in v5:
- mention additional error information in gpa when it presents;
- refine the documentation; (Markus)

Changes in v4:
- refine the documentation; (Markus)

Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)

---
 qapi/run-state.json   | 31 +++++++++++++++++++--
 system/runstate.c     | 65 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.c | 24 +++++++++++++++-
 3 files changed, 117 insertions(+), 3 deletions(-)
qapi/run-state.json +29 -2
index ce95cfa46b73..ee11adc50889 100644
@@ -501,10 +501,12 @@
 #
 # @s390: s390 guest panic information type (Since: 2.12)
 #
+# @tdx: tdx guest panic information type (Since: 10.1)
+#
 # Since: 2.9
 ##
 { 'enum': 'GuestPanicInformationType',
-  'data': [ 'hyper-v', 's390' ] }
+  'data': [ 'hyper-v', 's390', 'tdx' ] }
 
 ##
 # @GuestPanicInformation:
@@ -519,7 +521,8 @@
  'base': {'type': 'GuestPanicInformationType'},
  'discriminator': 'type',
  'data': {'hyper-v': 'GuestPanicInformationHyperV',
-          's390': 'GuestPanicInformationS390'}}
+          's390': 'GuestPanicInformationS390',
+          'tdx' : 'GuestPanicInformationTdx'}}
 
 ##
 # @GuestPanicInformationHyperV:
@@ -598,6 +601,30 @@
           'psw-addr': 'uint64',
           'reason': 'S390CrashReason'}}
 
+##
+# @GuestPanicInformationTdx:
+#
+# TDX Guest panic information specific to TDX, as specified in the
+# "Guest-Hypervisor Communication Interface (GHCI) Specification",
+# section TDG.VP.VMCALL<ReportFatalError>.
+#
+# @error-code: TD-specific error code
+#
+# @message: Human-readable error message provided by the guest. Not
+#     to be trusted.
+#
+# @gpa: guest-physical address of a page that contains more verbose
+#     error information, as zero-terminated string.  Present when the
+#     "GPA valid" bit (bit 63) is set in @error-code.
+#
+#
+# Since: 10.1
+##
+{'struct': 'GuestPanicInformationTdx',
+ 'data': {'error-code': 'uint32',
+          'message': 'str',
+          '*gpa': 'uint64'}}
+
 ##
 # @MEMORY_FAILURE:
 #
system/runstate.c +65 -0

78 lines not on this page · raw message

target/i386/kvm/tdx.c +23 -1
index 74b7e3ac85fe..282cdbd775c5 100644
@@ -16,6 +16,7 @@
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/runstate.h"
 #include "system/system.h"
 #include "exec/ramblock.h"
 
@@ -607,12 +608,27 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
     return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
 }
 
+static void tdx_panicked_on_fatal_error(X86CPU *cpu, uint64_t error_code,
+                                        char *message, uint64_t gpa)
+{
+    GuestPanicInformation *panic_info;
+
+    panic_info = g_new0(GuestPanicInformation, 1);
+    panic_info->type = GUEST_PANIC_INFORMATION_TYPE_TDX;
+    panic_info->u.tdx.error_code = (uint32_t) error_code;
+    panic_info->u.tdx.message = message;
+    panic_info->u.tdx.gpa = gpa;
+
+    qemu_system_guest_panicked(panic_info);
+}
+
 int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
 {
     uint64_t error_code = run->system_event.data[R_R12];
     uint64_t reg_mask = run->system_event.data[R_ECX];
     char *message = NULL;
     uint64_t *tmp;
+    uint64_t gpa = -1ull;
 
     if (error_code & 0xffff) {
         error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
@@ -648,7 +664,13 @@ int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
     }
 #undef COPY_REG
 
-    error_report("TD guest reports fatal error. %s", message ? : "");
+#define TDX_REPORT_FATAL_ERROR_GPA_VALID    BIT_ULL(63)
+    if (error_code & TDX_REPORT_FATAL_ERROR_GPA_VALID) {
+        gpa = run->system_event.data[R_R13];
+    }
+
+    tdx_panicked_on_fatal_error(cpu, error_code, message, gpa);
+
     return -1;
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v8 29/55
#96 lore raw

Re: [PATCH v8 29/55] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility

Quoted 27 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:39 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 29/55] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with
GuestPanic facility
X-Mailer: git-send-email 2.34.1

Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility

Originated-from: Isaku Yamahata <[email protected]>

---
Changes in v8:
- use g_strdup() for copy string;
- use the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL to grab gpa info;

Changes in v6:
- change error_code of GuestPanicInformationTdx from uint64_t to uint32_t, to only contains the bit 31:0 returned in r12.

Changes in v5:
- mention additional error information in gpa when it presents;
- refine the documentation; (Markus)

Changes in v4:
- refine the documentation; (Markus)

Quoted 8 lines

Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)

---
 qapi/run-state.json   | 31 +++++++++++++++++++--
 system/runstate.c     | 65 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.c | 24 +++++++++++++++-
 3 files changed, 117 insertions(+), 3 deletions(-)
author
#97 lore raw

PATCH v8 30/55 kvm: Check KVM_CAP_MAX_VCPUS at vm level

In reply to Xiaoyao Li

KVM with TDX support starts to report different KVM_CAP_MAX_VCPUS per different VM types. So switch to check the KVM_CAP_MAX_VCPUS at vm level.

KVM still returns the global KVM_CAP_MAX_VCPUS when the KVM is old that doesn't report different value at vm level.

---
 accel/kvm/kvm-all.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
accel/kvm/kvm-all.c +1 -1
index df9840e53a35..5835d840f3ad 100644
@@ -2425,7 +2425,7 @@ static int kvm_recommended_vcpus(KVMState *s)
 
 static int kvm_max_vcpus(KVMState *s)
 {
-    int ret = kvm_check_extension(s, KVM_CAP_MAX_VCPUS);
+    int ret = kvm_vm_check_extension(s, KVM_CAP_MAX_VCPUS);
     return (ret) ? ret : kvm_recommended_vcpus(s);
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v8 30/55
#98 lore raw

Re: [PATCH v8 30/55] kvm: Check KVM_CAP_MAX_VCPUS at vm level

Quoted 8 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:40 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 30/55] kvm: Check KVM_CAP_MAX_VCPUS at vm level
X-Mailer: git-send-email 2.34.1

KVM with TDX support starts to report different KVM_CAP_MAX_VCPUS per different VM types. So switch to check the KVM_CAP_MAX_VCPUS at vm level.

Quoted 7 lines

KVM still returns the global KVM_CAP_MAX_VCPUS when the KVM is old that doesn't report different value at vm level.

---
 accel/kvm/kvm-all.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
author
#99 lore raw

PATCH v8 31/55 i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

In reply to Xiaoyao Li

To allow execute confidential guest specific cpu init operations.

---
Changes in v6:
- new patch;

---
 target/i386/confidential-guest.h | 11 +++++++++++
 target/i386/cpu.c                | 10 ++++++++++
 2 files changed, 21 insertions(+)
target/i386/confidential-guest.h +11 -0
index 164be7633a20..a86c42a47558 100644
@@ -39,6 +39,7 @@ struct X86ConfidentialGuestClass {
 
     /* <public> */
     int (*kvm_type)(X86ConfidentialGuest *cg);
+    void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
     uint32_t (*mask_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
                                     int reg, uint32_t value);
 };
@@ -59,6 +60,16 @@ static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
     }
 }
 
+static inline void x86_confidential_guest_cpu_instance_init(X86ConfidentialGuest *cg,
+                                                            CPUState *cpu)
+{
+    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
+
+    if (klass->cpu_instance_init) {
+        klass->cpu_instance_init(cg, cpu);
+    }
+}
+
 /**
  * x86_confidential_guest_mask_cpuid_features:
  *
target/i386/cpu.c +10 -0
index add6430f7edd..5c69d1489365 100644
@@ -36,6 +36,7 @@
 #include "hw/qdev-properties.h"
 #include "hw/i386/topology.h"
 #ifndef CONFIG_USER_ONLY
+#include "confidential-guest.h"
 #include "system/reset.h"
 #include "qapi/qapi-commands-machine-target.h"
 #include "exec/address-spaces.h"
@@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
     }
 
     accel_cpu_instance_init(CPU(obj));
+
+#ifndef CONFIG_USER_ONLY
+    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
+                                                   TYPE_MACHINE));
+    if (ms && ms->cgs) {
+        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+                                                 (CPU(obj)));
+    }
+#endif
 }
 
 static void x86_cpu_init_default_topo(X86CPU *cpu)
Signature · 1 line
2.34.1
author
#100 lore raw

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

Hi Paolo,

On 4/1/2025 9:01 PM, Xiaoyao Li wrote:
...

Quoted 13 lines
target/i386/cpu.c +1 -0
index add6430f7edd..5c69d1489365 100644
@@ -36,6 +36,7 @@
  #include "hw/qdev-properties.h"
  #include "hw/i386/topology.h"
  #ifndef CONFIG_USER_ONLY
+#include "confidential-guest.h"
  #include "system/reset.h"
  #include "qapi/qapi-commands-machine-target.h"
  #include "exec/address-spaces.h"

target/i386/cpu.c static void x86_cpu_post_initfn(Object *obj) · line 8511

Quoted 7 lines
target/i386/cpu.c +4 -0
@@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
      }
 
      accel_cpu_instance_init(CPU(obj));
+
+#ifndef CONFIG_USER_ONLY
+    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
+                                                   TYPE_MACHINE));

It leads to

qemu-system-x86_64: ../hw/core/qdev.c:824: qdev_get_machine:
Assertion `dev' failed.
Aborted (core dumped)

for the case of "-cpu help" due to the assert(dev) in qdev_get_machine().

How do you want to resolve it? I can think of two:
1. remove the assert() in qdev_get_machine(). or
2. drop the callback introduce by this patch. Instead just do

   if (is_tdx_vm()) {
	tdx_cpu_instance_init();

}

Quoted 8 lines
target/i386/cpu.c +5 -0
+    if (ms && ms->cgs) {
+        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+                                                 (CPU(obj)));
+    }
+#endif
  }
 
  static void x86_cpu_init_default_topo(X86CPU *cpu)
#101 lore raw

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

Quoted 37 lines · Xiaoyao Li

Date: Thu, 24 Apr 2025 13:51:55 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v8 31/55] i386/cpu: introduce
x86_confidential_guest_cpu_instance_init()

Hi Paolo,

On 4/1/2025 9:01 PM, Xiaoyao Li wrote:
...

Quoted 13 lines
target/i386/cpu.c +1 -0
index add6430f7edd..5c69d1489365 100644
@@ -36,6 +36,7 @@
  #include "hw/qdev-properties.h"
  #include "hw/i386/topology.h"
  #ifndef CONFIG_USER_ONLY
+#include "confidential-guest.h"
  #include "system/reset.h"
  #include "qapi/qapi-commands-machine-target.h"
  #include "exec/address-spaces.h"

target/i386/cpu.c static void x86_cpu_post_initfn(Object *obj) · line 8510

Quoted 6 lines
target/i386/cpu.c +4 -0
@@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
      }
      accel_cpu_instance_init(CPU(obj));
+
+#ifndef CONFIG_USER_ONLY
+    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
+                                                   TYPE_MACHINE));

It leads to

qemu-system-x86_64: ../hw/core/qdev.c:824: qdev_get_machine: Assertion
`dev' failed.
Aborted (core dumped)

for the case of "-cpu help" due to the assert(dev) in qdev_get_machine().

Quoted 7 lines

How do you want to resolve it? I can think of two:
1. remove the assert() in qdev_get_machine(). or
2. drop the callback introduce by this patch. Instead just do

   if (is_tdx_vm()) {
	tdx_cpu_instance_init();
   }

Sorry I missed this mail when review this patch.

What about checking current_machine?

@@ -8541,10 +8541,8 @@ static void x86_cpu_post_initfn(Object *obj)

     accel_cpu_instance_init(CPU(obj));

#ifndef CONFIG_USER_ONLY
- MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
- TYPE_MACHINE));
- if (ms && ms->cgs) {
- x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+ if (current_machine && current_machine->cgs) {
+ x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(current_machine->cgs),

                                                  (CPU(obj)));
     }

#endif
---

"-cpu help" is processed before machine creation. The cpu-core (cpu_core_instance_init) also checks current_machine to avoid similar issue.

Regards,
Zhao

author
#102 lore raw

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

Quoted 65 lines · Zhao Liu
Quoted 37 lines · Xiaoyao Li

Date: Thu, 24 Apr 2025 13:51:55 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v8 31/55] i386/cpu: introduce
x86_confidential_guest_cpu_instance_init()

Hi Paolo,

On 4/1/2025 9:01 PM, Xiaoyao Li wrote:
...

Quoted 13 lines
target/i386/cpu.c +1 -0
index add6430f7edd..5c69d1489365 100644
@@ -36,6 +36,7 @@
   #include "hw/qdev-properties.h"
   #include "hw/i386/topology.h"
   #ifndef CONFIG_USER_ONLY
+#include "confidential-guest.h"
   #include "system/reset.h"
   #include "qapi/qapi-commands-machine-target.h"
   #include "exec/address-spaces.h"

target/i386/cpu.c static void x86_cpu_post_initfn(Object *obj) · line 8510

Quoted 6 lines
target/i386/cpu.c +4 -0
@@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
       }
       accel_cpu_instance_init(CPU(obj));
+
+#ifndef CONFIG_USER_ONLY
+    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
+                                                   TYPE_MACHINE));

It leads to

   qemu-system-x86_64: ../hw/core/qdev.c:824: qdev_get_machine: Assertion

`dev' failed.
Aborted (core dumped)

for the case of "-cpu help" due to the assert(dev) in qdev_get_machine().

Quoted 7 lines

How do you want to resolve it? I can think of two:
1. remove the assert() in qdev_get_machine(). or
2. drop the callback introduce by this patch. Instead just do

    if (is_tdx_vm()) {
	tdx_cpu_instance_init();
    }

Sorry I missed this mail when review this patch.

What about checking current_machine?

(quoted diff) +2 -5
@@ -8541,10 +8541,8 @@ static void x86_cpu_post_initfn(Object *obj)
      accel_cpu_instance_init(CPU(obj));
 
  #ifndef CONFIG_USER_ONLY
-    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
-                                                   TYPE_MACHINE));
-    if (ms && ms->cgs) {
-        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+    if (current_machine && current_machine->cgs) {
+        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(current_machine->cgs),
                                                   (CPU(obj)));
      }
  #endif
---
Quoted 3 lines

"-cpu help" is processed before machine creation. The cpu-core (cpu_core_instance_init) also checks current_machine to avoid similar issue.

This is a really good suggestion! I'll take it.

Thanks!

Quoted 4 lines

Regards,
Zhao

Reviewed-by on v8 31/55
#103 lore raw

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

In reply to Xiaoyao Li
Quoted 30 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:41 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 31/55] i386/cpu: introduce
x86_confidential_guest_cpu_instance_init()
X-Mailer: git-send-email 2.34.1

To allow execute confidential guest specific cpu init operations.

---
Changes in v6:
- new patch;

---
 target/i386/confidential-guest.h | 11 +++++++++++
 target/i386/cpu.c                | 10 ++++++++++
 2 files changed, 21 insertions(+)
target/i386/confidential-guest.h +1 -0
index 164be7633a20..a86c42a47558 100644
@@ -39,6 +39,7 @@ struct X86ConfidentialGuestClass {
 
     /* <public> */
     int (*kvm_type)(X86ConfidentialGuest *cg);
+    void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
     uint32_t (*mask_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
                                     int reg, uint32_t value);
 };

target/i386/confidential-guest.h static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg) · line 64

Quoted 5 lines
target/i386/confidential-guest.h +2 -0
@@ -59,6 +60,16 @@ static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
     }
 }
 
+static inline void x86_confidential_guest_cpu_instance_init(X86ConfidentialGuest *cg,
+                                                            CPUState *cpu)

Well, it's a so long function name.

Or maybe we can call it as "x86_confidential_guest_cpu_init" and rename the hook as "cpu_init"?

Quoted 24 lines
target/i386/confidential-guest.h +8 -0
+{
+    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
+
+    if (klass->cpu_instance_init) {
+        klass->cpu_instance_init(cg, cpu);
+    }
+}
+
 /**
  * x86_confidential_guest_mask_cpuid_features:
  *
target/i386/cpu.c +1 -0
index add6430f7edd..5c69d1489365 100644
@@ -36,6 +36,7 @@
 #include "hw/qdev-properties.h"
 #include "hw/i386/topology.h"
 #ifndef CONFIG_USER_ONLY
+#include "confidential-guest.h"
 #include "system/reset.h"
 #include "qapi/qapi-commands-machine-target.h"
 #include "exec/address-spaces.h"

target/i386/cpu.c static void x86_cpu_post_initfn(Object *obj) · line 8511

Quoted 7 lines
target/i386/cpu.c +4 -0
@@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
     }
 
     accel_cpu_instance_init(CPU(obj));
+
+#ifndef CONFIG_USER_ONLY
+    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
+                                                   TYPE_MACHINE));

There's no need to use object_dynamic_cast(), to cast to the basic machine type.

MACHINE(qdev_get_machine()) is enough.

target/i386/cpu.c static void x86_cpu_post_initfn(Object *obj) · line 8518

Quoted 7 lines
target/i386/cpu.c +5 -0
+    if (ms && ms->cgs) {
+        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+                                                 (CPU(obj)));
+    }
+#endif
 }
 

Others are fine for me,

author
#104 lore raw

PATCH v8 32/55 i386/tdx: implement tdx_cpu_instance_init()

In reply to Xiaoyao Li

Currently, pmu is not supported for TDX by KVM.

---
chanegs in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 6 ++++++
 1 file changed, 6 insertions(+)
target/i386/kvm/tdx.c +6 -0
index 282cdbd775c5..49e748af3949 100644
@@ -400,6 +400,11 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
+static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
+{
+    object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
+}
+
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
 {
     if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
@@ -781,4 +786,5 @@ static void tdx_guest_class_init(ObjectClass *oc, void *data)
 
     klass->kvm_init = tdx_kvm_init;
     x86_klass->kvm_type = tdx_kvm_type;
+    x86_klass->cpu_instance_init = tdx_cpu_instance_init;
 }
Signature · 1 line
2.34.1
Reviewed-by on v8 32/55
#105 lore raw

Re: [PATCH v8 32/55] i386/tdx: implement tdx_cpu_instance_init()

Quoted 7 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:42 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 32/55] i386/tdx: implement tdx_cpu_instance_init()
X-Mailer: git-send-email 2.34.1

Currently, pmu is not supported for TDX by KVM.

Quoted 7 lines

---
chanegs in v6:
- new patch;

---
 target/i386/kvm/tdx.c | 6 ++++++
 1 file changed, 6 insertions(+)
author
#106 lore raw

PATCH v8 33/55 i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f

In reply to Xiaoyao Li

Currently, QEMU exposes CPUID 0x1f to guest only when necessary, i.e., when topology level that cannot be enumerated by leaf 0xB, e.g., die or module level, are configured for the guest, e.g., -smp xx,dies=2.

However, TDX architecture forces to require CPUID 0x1f to configure CPU topology.

Introduce a bool flag, enable_cpuid_0x1f, in CPU for the case that requires CPUID leaf 0x1f to be exposed to guest.

Introduce a new function x86_has_cpuid_0x1f(), which is the warpper of cpu->enable_cpuid_0x1f and x86_has_extended_topo() to check if it needs to enable cpuid leaf 0x1f for the guest.

---
 target/i386/cpu.c     | 4 ++--
 target/i386/cpu.h     | 9 +++++++++
 target/i386/kvm/kvm.c | 2 +-
 3 files changed, 12 insertions(+), 3 deletions(-)
target/i386/cpu.c +2 -2
index 5c69d1489365..f219961b62cd 100644
@@ -7006,7 +7006,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         break;
     case 0x1F:
         /* V2 Extended Topology Enumeration Leaf */
-        if (!x86_has_extended_topo(env->avail_cpu_topo)) {
+        if (!x86_has_cpuid_0x1f(cpu)) {
             *eax = *ebx = *ecx = *edx = 0;
             break;
         }
@@ -7870,7 +7870,7 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
          * cpu->vendor_cpuid_only has been unset for compatibility with older
          * machine types.
          */
-        if (x86_has_extended_topo(env->avail_cpu_topo) &&
+        if (x86_has_cpuid_0x1f(cpu) &&
             (IS_INTEL_CPU(env) || !cpu->vendor_cpuid_only)) {
             x86_cpu_adjust_level(cpu, &env->cpuid_min_level, 0x1F);
         }
target/i386/cpu.h +9 -0
index 76f24446a55d..3910b488f775 100644
@@ -2251,6 +2251,9 @@ struct ArchCPU {
     /* Compatibility bits for old machine types: */
     bool enable_cpuid_0xb;
 
+    /* Force to enable cpuid 0x1f */
+    bool enable_cpuid_0x1f;
+
     /* Enable auto level-increase for all CPUID leaves */
     bool full_cpuid_auto_level;
 
@@ -2513,6 +2516,12 @@ void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 bool cpu_has_x2apic_feature(CPUX86State *env);
 
+static inline bool x86_has_cpuid_0x1f(X86CPU *cpu)
+{
+    return cpu->enable_cpuid_0x1f ||
+           x86_has_extended_topo(cpu->env.avail_cpu_topo);
+}
+
 /* helper.c */
 void x86_cpu_set_a20(X86CPU *cpu, int a20_state);
 void cpu_sync_avx_hflag(CPUX86State *env);
target/i386/kvm/kvm.c +1 -1
index a76f34537908..741b50181ed9 100644
@@ -1871,7 +1871,7 @@ uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
             break;
         }
         case 0x1f:
-            if (!x86_has_extended_topo(env->avail_cpu_topo)) {
+            if (!x86_has_cpuid_0x1f(env_archcpu(env))) {
                 cpuid_i--;
                 break;
             }
Signature · 1 line
2.34.1
Reviewed-by on v8 33/55
#107 lore raw

Re: [PATCH v8 33/55] i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f

Quoted 10 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:43 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 33/55] i386/cpu: Introduce enable_cpuid_0x1f to force
exposing CPUID 0x1f
X-Mailer: git-send-email 2.34.1

Currently, QEMU exposes CPUID 0x1f to guest only when necessary, i.e., when topology level that cannot be enumerated by leaf 0xB, e.g., die or module level, are configured for the guest, e.g., -smp xx,dies=2.

Quoted 7 lines

However, TDX architecture forces to require CPUID 0x1f to configure CPU topology.

Introduce a bool flag, enable_cpuid_0x1f, in CPU for the case that requires CPUID leaf 0x1f to be exposed to guest.

Introduce a new function x86_has_cpuid_0x1f(), which is the warpper of

s/warpper/wrapper/

(found by "scripts/checkpatch.pl --codespell *" :-))

Quoted 9 lines

cpu->enable_cpuid_0x1f and x86_has_extended_topo() to check if it needs to enable cpuid leaf 0x1f for the guest.

---
 target/i386/cpu.c     | 4 ++--
 target/i386/cpu.h     | 9 +++++++++
 target/i386/kvm/kvm.c | 2 +-
 3 files changed, 12 insertions(+), 3 deletions(-)
author
#108 lore raw

PATCH v8 34/55 i386/tdx: Force exposing CPUID 0x1f

In reply to Xiaoyao Li

TDX uses CPUID 0x1f to configure TD guest's CPU topology. So set enable_cpuid_0x1f for TDs.

---
 target/i386/kvm/tdx.c | 4 ++++
 1 file changed, 4 insertions(+)
target/i386/kvm/tdx.c +4 -0
index 49e748af3949..b0616eb3d371 100644
@@ -402,7 +402,11 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
 
 static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
 {
+    X86CPU *x86cpu = X86_CPU(cpu);
+
     object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
+
+    x86cpu->enable_cpuid_0x1f = true;
 }
 
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
Signature · 1 line
2.34.1
Reviewed-by on v8 34/55
#109 lore raw

Re: [PATCH v8 34/55] i386/tdx: Force exposing CPUID 0x1f

Quoted 5 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:44 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 34/55] i386/tdx: Force exposing CPUID 0x1f
X-Mailer: git-send-email 2.34.1

Quoted 7 lines

TDX uses CPUID 0x1f to configure TD guest's CPU topology. So set enable_cpuid_0x1f for TDs.

---
 target/i386/kvm/tdx.c | 4 ++++
 1 file changed, 4 insertions(+)
author
#110 lore raw

PATCH v8 35/55 i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM

In reply to Xiaoyao Li

TDX only supports readonly for shared memory but not for private memory.

In the view of QEMU, it has no idea whether a memslot is used as shared memory of private. Thus just mark kvm_readonly_mem_enabled to false to TDX VM for simplicity.

---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
target/i386/kvm/tdx.c +9 -0
index b0616eb3d371..a816f57043f6 100644
@@ -386,6 +386,15 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EOPNOTSUPP;
     }
 
+    /*
+     * Set kvm_readonly_mem_allowed to false, because TDX only supports readonly
+     * memory for shared memory but not for private memory. Besides, whether a
+     * memslot is private or shared is not determined by QEMU.
+     *
+     * Thus, just mark readonly memory not supported for simplicity.
+     */
+    kvm_readonly_mem_allowed = false;
+
     qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
 
     tdx_guest = tdx;
Signature · 1 line
2.34.1
Reviewed-by on v8 35/55
#111 lore raw

Re: [PATCH v8 35/55] i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM

Quoted 12 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:45 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 35/55] i386/tdx: Set kvm_readonly_mem_enabled to false
for TDX VM
X-Mailer: git-send-email 2.34.1

TDX only supports readonly for shared memory but not for private memory.

In the view of QEMU, it has no idea whether a memslot is used as shared memory of private. Thus just mark kvm_readonly_mem_enabled to false to TDX VM for simplicity.

Quoted 5 lines
---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
author
#112 lore raw

PATCH v8 36/55 i386/tdx: Disable SMM for TDX VMs

In reply to Xiaoyao Li

TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because VMM cannot manipulate TDX VM's memory.

Disable SMM for TDX VMs and error out if user requests to enable SMM.

---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
target/i386/kvm/tdx.c +9 -0
index a816f57043f6..0eefd058f7a2 100644
@@ -369,11 +369,20 @@ static Notifier tdx_machine_done_notify = {
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
+    MachineState *ms = MACHINE(qdev_get_machine());
+    X86MachineState *x86ms = X86_MACHINE(ms);
     TdxGuest *tdx = TDX_GUEST(cgs);
     int r = 0;
 
     kvm_mark_guest_state_protected();
 
+    if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+        x86ms->smm = ON_OFF_AUTO_OFF;
+    } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+        error_setg(errp, "TDX VM doesn't support SMM");
+        return -EINVAL;
+    }
+
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
Signature · 1 line
2.34.1
Reviewed-by on v8 36/55
#113 lore raw

Re: [PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs

Quoted 10 lines · Xiaoyao Li

TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because VMM cannot manipulate TDX VM's memory.

Disable SMM for TDX VMs and error out if user requests to enable SMM.

---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 36/55
#114 lore raw

Re: [PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs

In reply to Xiaoyao Li
Quoted 8 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:46 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs
X-Mailer: git-send-email 2.34.1

TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because VMM cannot manipulate TDX VM's memory.

Quoted 7 lines

Disable SMM for TDX VMs and error out if user requests to enable SMM.

---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)
author
#115 lore raw

PATCH v8 37/55 i386/tdx: Disable PIC for TDX VMs

In reply to Xiaoyao Li

Legacy PIC (8259) cannot be supported for TDX VMs since TDX module doesn't allow directly interrupt injection. Using posted interrupts for the PIC is not a viable option as the guest BIOS/kernel will not do EOI for PIC IRQs, i.e. will leave the vIRR bit set.

Hence disable PIC for TDX VMs and error out if user wants PIC.

---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)
target/i386/kvm/tdx.c +7 -0
index 0eefd058f7a2..0d30506c2021 100644
@@ -383,6 +383,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EINVAL;
     }
 
+    if (x86ms->pic == ON_OFF_AUTO_AUTO) {
+        x86ms->pic = ON_OFF_AUTO_OFF;
+    } else if (x86ms->pic == ON_OFF_AUTO_ON) {
+        error_setg(errp, "TDX VM doesn't support PIC");
+        return -EINVAL;
+    }
+
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
Signature · 1 line
2.34.1
Reviewed-by on v8 37/55
#116 lore raw

Re: [PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs

Quoted 5 lines · Xiaoyao Li

Legacy PIC (8259) cannot be supported for TDX VMs since TDX module doesn't allow directly interrupt injection. Using posted interrupts for the PIC is not a viable option as the guest BIOS/kernel will not do EOI for PIC IRQs, i.e. will leave the vIRR bit set.

Quoted 7 lines

Hence disable PIC for TDX VMs and error out if user wants PIC.

---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 37/55
#117 lore raw

Re: [PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs

In reply to Xiaoyao Li
Quoted 10 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:47 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs
X-Mailer: git-send-email 2.34.1

Legacy PIC (8259) cannot be supported for TDX VMs since TDX module doesn't allow directly interrupt injection. Using posted interrupts for the PIC is not a viable option as the guest BIOS/kernel will not do EOI for PIC IRQs, i.e. will leave the vIRR bit set.

Quoted 7 lines

Hence disable PIC for TDX VMs and error out if user wants PIC.

---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)
author
#118 lore raw

PATCH v8 38/55 i386/tdx: Set and check kernel_irqchip mode for TDX

In reply to Xiaoyao Li

KVM mandates kernel_irqchip to be split mode.

Set it to split mode automatically when users don't provide an explicit value, otherwise check it to be the split mode.

---
 target/i386/kvm/tdx.c | 8 ++++++++
 1 file changed, 8 insertions(+)
target/i386/kvm/tdx.c +8 -0
index 0d30506c2021..4128f27d6b64 100644
@@ -16,6 +16,7 @@
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/kvm_int.h"
 #include "system/runstate.h"
 #include "system/system.h"
 #include "exec/ramblock.h"
@@ -390,6 +391,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EINVAL;
     }
 
+    if (kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO ) {
+        kvm_state->kernel_irqchip_split = ON_OFF_AUTO_ON;
+    } else if(kvm_state->kernel_irqchip_split != ON_OFF_AUTO_ON) {
+        error_setg(errp, "TDX VM requires kernel_irqchip to be split");
+        return -EINVAL;
+    }
+
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
Signature · 1 line
2.34.1
Reviewed-by on v8 38/55
#119 lore raw

Re: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

Quoted 10 lines · Xiaoyao Li

KVM mandates kernel_irqchip to be split mode.

Set it to split mode automatically when users don't provide an explicit value, otherwise check it to be the split mode.

---
 target/i386/kvm/tdx.c | 8 ++++++++
 1 file changed, 8 insertions(+)
Quoted 13 lines
target/i386/kvm/tdx.c +1 -0
index 0d30506c2021..4128f27d6b64 100644
@@ -16,6 +16,7 @@
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/kvm_int.h"
 #include "system/runstate.h"
 #include "system/system.h"
 #include "exec/ramblock.h"

target/i386/kvm/tdx.c static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 394

Quoted 4 lines
target/i386/kvm/tdx.c +1 -0
@@ -390,6 +391,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EINVAL;
     }
 
+    if (kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO ) {

Nitpick, no need for a space before the ')' ...

target/i386/kvm/tdx.c static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 396

Quoted 2 lines
target/i386/kvm/tdx.c +2 -0
+        kvm_state->kernel_irqchip_split = ON_OFF_AUTO_ON;
+    } else if(kvm_state->kernel_irqchip_split != ON_OFF_AUTO_ON) {

..and need a space between 'if' and '('

target/i386/kvm/tdx.c static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 400

Quoted 4 lines
target/i386/kvm/tdx.c +4 -0
+        error_setg(errp, "TDX VM requires kernel_irqchip to be split");
+        return -EINVAL;
+    }
+

Note for self - 'tdx_kvm_init' is called by 'confidential_guest_kvm_init', which is called by 'kvm_arch_init', which is called by 'kvm_init' *before* it processes "kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO" to set the default. So this change is correctly taking priority over the default behaviour.

Quoted 6 lines
target/i386/kvm/tdx.c +0 -0
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
Signature · 1 line
2.34.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
author
#120 lore raw

Re: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

Quoted 31 lines · Daniel P. Berrangé
Quoted 10 lines · Xiaoyao Li

KVM mandates kernel_irqchip to be split mode.

Set it to split mode automatically when users don't provide an explicit value, otherwise check it to be the split mode.

---
  target/i386/kvm/tdx.c | 8 ++++++++
  1 file changed, 8 insertions(+)
Quoted 16 lines
target/i386/kvm/tdx.c +1 -0
index 0d30506c2021..4128f27d6b64 100644
@@ -16,6 +16,7 @@
  #include "qapi/error.h"
  #include "qom/object_interfaces.h"
  #include "crypto/hash.h"
+#include "system/kvm_int.h"
  #include "system/runstate.h"
  #include "system/system.h"
  #include "exec/ramblock.h"
@@ -390,6 +391,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
          return -EINVAL;
      }
 
Quoted 8 lines

target/i386/kvm/tdx.c static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 394

Quoted 1 line
target/i386/kvm/tdx.c +1 -0
+    if (kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO ) {

Nitpick, no need for a space before the ')' ...

target/i386/kvm/tdx.c static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 396

Quoted 2 lines
target/i386/kvm/tdx.c +2 -0
+        kvm_state->kernel_irqchip_split = ON_OFF_AUTO_ON;
+    } else if(kvm_state->kernel_irqchip_split != ON_OFF_AUTO_ON) {

..and need a space between 'if' and '('

Thanks for catching these coding style issue. Will fix them in the next version.

Quoted 20 lines

target/i386/kvm/tdx.c static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp) · line 400

Quoted 4 lines
target/i386/kvm/tdx.c +4 -0
+        error_setg(errp, "TDX VM requires kernel_irqchip to be split");
+        return -EINVAL;
+    }
+

Note for self - 'tdx_kvm_init' is called by 'confidential_guest_kvm_init', which is called by 'kvm_arch_init', which is called by 'kvm_init' *before* it processes "kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO" to set the default. So this change is correctly taking priority over the default behaviour.

Quoted 6 lines
target/i386/kvm/tdx.c +0 -0
      if (!tdx_caps) {
          r = get_tdx_capabilities(errp);
          if (r) {
Signature · 1 line
2.34.1

With regards,
Daniel

Reviewed-by on v8 38/55
#121 lore raw

Re: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

In reply to Xiaoyao Li
Quoted 8 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:48 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for
TDX
X-Mailer: git-send-email 2.34.1

KVM mandates kernel_irqchip to be split mode.

Quoted 8 lines

Set it to split mode automatically when users don't provide an explicit value, otherwise check it to be the split mode.

---
 target/i386/kvm/tdx.c | 8 ++++++++
 1 file changed, 8 insertions(+)
author
#122 lore raw

PATCH v8 39/55 i386/tdx: Don't synchronize guest tsc for TDs

In reply to Xiaoyao Li

From: Isaku Yamahata <[email protected]>

TSC of TDs is not accessible and KVM doesn't allow access of MSR_IA32_TSC for TDs. To avoid the assert() in kvm_get_tsc, make kvm_synchronize_all_tsc() noop for TDs,

---
 target/i386/kvm/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
target/i386/kvm/kvm.c +1 -1
index 741b50181ed9..ead1d0263385 100644
@@ -327,7 +327,7 @@ void kvm_synchronize_all_tsc(void)
 {
     CPUState *cpu;
 
-    if (kvm_enabled()) {
+    if (kvm_enabled() && !is_tdx_vm()) {
         CPU_FOREACH(cpu) {
             run_on_cpu(cpu, do_kvm_synchronize_tsc, RUN_ON_CPU_NULL);
         }
Signature · 1 line
2.34.1
Reviewed-by on v8 39/55
#123 lore raw

Re: [PATCH v8 39/55] i386/tdx: Don't synchronize guest tsc for TDs

Quoted 11 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:49 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 39/55] i386/tdx: Don't synchronize guest tsc for TDs
X-Mailer: git-send-email 2.34.1

From: Isaku Yamahata <[email protected]>

TSC of TDs is not accessible and KVM doesn't allow access of MSR_IA32_TSC for TDs. To avoid the assert() in kvm_get_tsc, make kvm_synchronize_all_tsc() noop for TDs,

Quoted 7 lines
---
 target/i386/kvm/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
author
#124 lore raw

PATCH v8 40/55 i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs

In reply to Xiaoyao Li

For TDs, only MSR_IA32_UCODE_REV in kvm_init_msrs() can be configured by VMM, while the features enumerated/controlled by other MSRs except MSR_IA32_UCODE_REV in kvm_init_msrs() are not under control of VMM.

Only configure MSR_IA32_UCODE_REV for TDs.

---
 target/i386/kvm/kvm.c | 44 ++++++++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
target/i386/kvm/kvm.c +23 -21
index ead1d0263385..4078ba40473e 100644
@@ -3863,32 +3863,34 @@ static void kvm_init_msrs(X86CPU *cpu)
     CPUX86State *env = &cpu->env;
 
     kvm_msr_buf_reset(cpu);
-    if (has_msr_arch_capabs) {
-        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
-                          env->features[FEAT_ARCH_CAPABILITIES]);
-    }
-
-    if (has_msr_core_capabs) {
-        kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
-                          env->features[FEAT_CORE_CAPABILITY]);
-    }
-
-    if (has_msr_perf_capabs && cpu->enable_pmu) {
-        kvm_msr_entry_add_perf(cpu, env->features);
+
+    if (!is_tdx_vm()) {
+        if (has_msr_arch_capabs) {
+            kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
+                                env->features[FEAT_ARCH_CAPABILITIES]);
+        }
+
+        if (has_msr_core_capabs) {
+            kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
+                                env->features[FEAT_CORE_CAPABILITY]);
+        }
+
+        if (has_msr_perf_capabs && cpu->enable_pmu) {
+            kvm_msr_entry_add_perf(cpu, env->features);
+        }
+
+        /*
+         * Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but
+         * all kernels with MSR features should have them.
+         */
+        if (kvm_feature_msrs && cpu_has_vmx(env)) {
+            kvm_msr_entry_add_vmx(cpu, env->features);
+        }
     }
 
     if (has_msr_ucode_rev) {
         kvm_msr_entry_add(cpu, MSR_IA32_UCODE_REV, cpu->ucode_rev);
     }
-
-    /*
-     * Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but
-     * all kernels with MSR features should have them.
-     */
-    if (kvm_feature_msrs && cpu_has_vmx(env)) {
-        kvm_msr_entry_add_vmx(cpu, env->features);
-    }
-
     assert(kvm_buf_set_msrs(cpu) == 0);
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v8 40/55
#125 lore raw

Re: [PATCH v8 40/55] i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs

Quoted 10 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:50 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 40/55] i386/tdx: Only configure MSR_IA32_UCODE_REV in
kvm_init_msrs() for TDs
X-Mailer: git-send-email 2.34.1

For TDs, only MSR_IA32_UCODE_REV in kvm_init_msrs() can be configured by VMM, while the features enumerated/controlled by other MSRs except MSR_IA32_UCODE_REV in kvm_init_msrs() are not under control of VMM.

Quoted 7 lines

Only configure MSR_IA32_UCODE_REV for TDs.

---
 target/i386/kvm/kvm.c | 44 ++++++++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 21 deletions(-)
author
#126 lore raw

PATCH v8 41/55 i386/apic: Skip kvm_apic_put() for TDX

In reply to Xiaoyao Li

KVM neithers allow writing to MSR_IA32_APICBASE for TDs, nor allow for KVM_SET_LAPIC[*].

Note, KVM_GET_LAPIC is also disallowed for TDX. It is called in the path

do_kvm_cpu_synchronize_state()
-> kvm_arch_get_registers()

     -> kvm_get_apic()

and it's already disllowed for confidential guest through guest_state_protected.

[*] https://lore.kernel.org/all/[email protected]/

---
Changes in v8:
- Fix the coding style; (Francesco)

---
 hw/i386/kvm/apic.c | 5 +++++
 1 file changed, 5 insertions(+)
hw/i386/kvm/apic.c +5 -0
index 757510600098..cb65fca49586 100644
@@ -17,6 +17,7 @@
 #include "system/hw_accel.h"
 #include "system/kvm.h"
 #include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
 
 static inline void kvm_apic_set_reg(struct kvm_lapic_state *kapic,
                                     int reg_id, uint32_t val)
@@ -141,6 +142,10 @@ static void kvm_apic_put(CPUState *cs, run_on_cpu_data data)
     struct kvm_lapic_state kapic;
     int ret;
 
+    if (is_tdx_vm()) {
+        return;
+    }
+
     kvm_put_apicbase(s->cpu, s->apicbase);
     kvm_put_apic_state(s, &kapic);
 
Signature · 1 line
2.34.1
Reviewed-by on v8 41/55
#127 lore raw

Re: [PATCH v8 41/55] i386/apic: Skip kvm_apic_put() for TDX

Quoted 19 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:51 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 41/55] i386/apic: Skip kvm_apic_put() for TDX
X-Mailer: git-send-email 2.34.1

KVM neithers allow writing to MSR_IA32_APICBASE for TDs, nor allow for KVM_SET_LAPIC[*].

Note, KVM_GET_LAPIC is also disallowed for TDX. It is called in the path

do_kvm_cpu_synchronize_state()
-> kvm_arch_get_registers()

     -> kvm_get_apic()

and it's already disllowed for confidential guest through guest_state_protected.

[*] https://lore.kernel.org/all/[email protected]/

Quoted 7 lines

---
Changes in v8:
- Fix the coding style; (Francesco)

---
 hw/i386/kvm/apic.c | 5 +++++
 1 file changed, 5 insertions(+)
author
#128 lore raw

PATCH v8 42/55 cpu: Don't set vcpu_dirty when guest_state_protected

In reply to Xiaoyao Li

QEMU calls kvm_arch_put_registers() when vcpu_dirty is true in kvm_vcpu_exec(). However, for confidential guest, like TDX, putting registers is disallowed due to guest state is protected.

Only set vcpu_dirty to true with guest state is not protected when creating the vcpu.

---
Changes in v7:
- new patch to replace "i386/tdx: Don't get/put guest state for TDX VMs" in v6;

---
 accel/kvm/kvm-all.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
accel/kvm/kvm-all.c +3 -1
index 5835d840f3ad..9862d8ff1d38 100644
@@ -466,7 +466,9 @@ int kvm_create_vcpu(CPUState *cpu)
 
     cpu->kvm_fd = kvm_fd;
     cpu->kvm_state = s;
-    cpu->vcpu_dirty = true;
+    if (!s->guest_state_protected) {
+        cpu->vcpu_dirty = true;
+    }
     cpu->dirty_pages = 0;
     cpu->throttle_us_per_full = 0;
 
Signature · 1 line
2.34.1
Reviewed-by on v8 42/55
#129 lore raw

Re: [PATCH v8 42/55] cpu: Don't set vcpu_dirty when guest_state_protected

Quoted 13 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:52 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 42/55] cpu: Don't set vcpu_dirty when
guest_state_protected
X-Mailer: git-send-email 2.34.1

QEMU calls kvm_arch_put_registers() when vcpu_dirty is true in kvm_vcpu_exec(). However, for confidential guest, like TDX, putting registers is disallowed due to guest state is protected.

Only set vcpu_dirty to true with guest state is not protected when creating the vcpu.

Quoted 8 lines

---
Changes in v7:
- new patch to replace "i386/tdx: Don't get/put guest state for TDX VMs" in v6;

---
 accel/kvm/kvm-all.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
author
#130 lore raw

PATCH v8 43/55 i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()

In reply to Xiaoyao Li

Because for TDX case, there are also fixed-1 bits that enfored by TDX module.

---
 target/i386/confidential-guest.h | 20 ++++++++++----------
 target/i386/kvm/kvm.c            |  2 +-
 target/i386/sev.c                |  4 ++--
 3 files changed, 13 insertions(+), 13 deletions(-)
target/i386/confidential-guest.h +10 -10
index a86c42a47558..777d43cc9688 100644
@@ -40,8 +40,8 @@ struct X86ConfidentialGuestClass {
     /* <public> */
     int (*kvm_type)(X86ConfidentialGuest *cg);
     void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
-    uint32_t (*mask_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
-                                    int reg, uint32_t value);
+    uint32_t (*adjust_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature,
+                                      uint32_t index, int reg, uint32_t value);
 };
 
 /**
@@ -71,21 +71,21 @@ static inline void x86_confidential_guest_cpu_instance_init(X86ConfidentialGuest
 }
 
 /**
- * x86_confidential_guest_mask_cpuid_features:
+ * x86_confidential_guest_adjust_cpuid_features:
  *
- * Removes unsupported features from a confidential guest's CPUID values, returns
- * the value with the bits removed.  The bits removed should be those that KVM
- * provides independent of host-supported CPUID features, but are not supported by
- * the confidential computing firmware.
+ * Adjust the supported features from a confidential guest's CPUID values,
+ * returns the adjusted value.  There are bits being removed that are not
+ * supported by the confidential computing firmware or bits being added that
+ * are forcibly exposed to guest by the confidential computing firmware.
  */
-static inline int x86_confidential_guest_mask_cpuid_features(X86ConfidentialGuest *cg,
+static inline int x86_confidential_guest_adjust_cpuid_features(X86ConfidentialGuest *cg,
                                                              uint32_t feature, uint32_t index,
                                                              int reg, uint32_t value)
 {
     X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
 
-    if (klass->mask_cpuid_features) {
-        return klass->mask_cpuid_features(cg, feature, index, reg, value);
+    if (klass->adjust_cpuid_features) {
+        return klass->adjust_cpuid_features(cg, feature, index, reg, value);
     } else {
         return value;
     }
target/i386/kvm/kvm.c +1 -1
index 4078ba40473e..fa46edaeac8d 100644
@@ -573,7 +573,7 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
     }
 
     if (current_machine->cgs) {
-        ret = x86_confidential_guest_mask_cpuid_features(
+        ret = x86_confidential_guest_adjust_cpuid_features(
             X86_CONFIDENTIAL_GUEST(current_machine->cgs),
             function, index, reg, ret);
     }
target/i386/sev.c +2 -2
index 0e1dbb6959ec..a6c0a697250b 100644
@@ -946,7 +946,7 @@ out:
 }
 
 static uint32_t
-sev_snp_mask_cpuid_features(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
+sev_snp_adjust_cpuid_features(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
                             int reg, uint32_t value)
 {
     switch (feature) {
@@ -2404,7 +2404,7 @@ sev_snp_guest_class_init(ObjectClass *oc, void *data)
     klass->launch_finish = sev_snp_launch_finish;
     klass->launch_update_data = sev_snp_launch_update_data;
     klass->kvm_init = sev_snp_kvm_init;
-    x86_klass->mask_cpuid_features = sev_snp_mask_cpuid_features;
+    x86_klass->adjust_cpuid_features = sev_snp_adjust_cpuid_features;
     x86_klass->kvm_type = sev_snp_kvm_type;
 
     object_class_property_add(oc, "policy", "uint64",
Signature · 1 line
2.34.1
Reviewed-by on v8 43/55
#131 lore raw

Re: [PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()

Quoted 9 lines · Xiaoyao Li

Because for TDX case, there are also fixed-1 bits that enfored by TDX module.

---
 target/i386/confidential-guest.h | 20 ++++++++++----------
 target/i386/kvm/kvm.c            |  2 +-
 target/i386/sev.c                |  4 ++--
 3 files changed, 13 insertions(+), 13 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 43/55
#132 lore raw

Re: [PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()

In reply to Xiaoyao Li
Quoted 9 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:53 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to
*adjust_cpuid_features()
X-Mailer: git-send-email 2.34.1

Because for TDX case, there are also fixed-1 bits that enfored by TDX module.

Quoted 6 lines
---
 target/i386/confidential-guest.h | 20 ++++++++++----------
 target/i386/kvm/kvm.c            |  2 +-
 target/i386/sev.c                |  4 ++--
 3 files changed, 13 insertions(+), 13 deletions(-)
author
#133 lore raw

PATCH v8 44/55 i386/tdx: Implement adjust_cpuid_features() for TDX

In reply to Xiaoyao Li

Maintain a TDX specific supported CPUID set, and use it to mask the common supported CPUID value of KVM. It can avoid newly added supported features (reported via KVM_GET_SUPPORTED_CPUID) for common VMs being falsely reported as supported for TDX.

As the first step, initialize the TDX supported CPUID set with all the configurable CPUID bits. It's not complete because there are other CPUID bits are supported for TDX but not reported as directly configurable. E.g. the XFAM related bits, attribute related bits and fixed-1 bits. They will be handled in the future.

Also, what matters are the CPUID bits related to QEMU's feature word. Only mask the CPUID leafs which are feature word leaf.

---
 target/i386/cpu.c          | 18 ++++++++++++++++++
 target/i386/cpu.h          |  1 +
 target/i386/kvm/kvm.c      |  2 +-
 target/i386/kvm/kvm_i386.h |  1 +
 target/i386/kvm/tdx.c      | 34 ++++++++++++++++++++++++++++++++++
 5 files changed, 55 insertions(+), 1 deletion(-)
target/i386/cpu.c +18 -0
index f219961b62cd..4061d38d3fbe 100644
@@ -1655,6 +1655,24 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     },
 };
 
+bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg)
+{
+    FeatureWordInfo *wi;
+    FeatureWord w;
+
+    for (w = 0; w < FEATURE_WORDS; w++)
+    {
+        wi = &feature_word_info[w];
+        if (wi->type == CPUID_FEATURE_WORD && wi->cpuid.eax == feature &&
+            (!wi->cpuid.needs_ecx || wi->cpuid.ecx == index) &&
+            wi->cpuid.reg == reg)
+        {
+            return true;
+        }
+    }
+    return false;
+}
+
 typedef struct FeatureMask {
     FeatureWord index;
     uint64_t mask;
target/i386/cpu.h +1 -0
index 3910b488f775..42ef77789ded 100644
@@ -2515,6 +2515,7 @@ void cpu_set_apic_feature(CPUX86State *env);
 void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 bool cpu_has_x2apic_feature(CPUX86State *env);
+bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg);
 
 static inline bool x86_has_cpuid_0x1f(X86CPU *cpu)
 {
target/i386/kvm/kvm.c +1 -1
index fa46edaeac8d..17d7bf6ae9aa 100644
@@ -393,7 +393,7 @@ static bool host_tsx_broken(void)
 
 /* Returns the value for a specific register on the cpuid entry
  */
-static uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg)
+uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg)
 {
     uint32_t ret = 0;
     switch (reg) {
target/i386/kvm/kvm_i386.h +1 -0
index dc696cb7238a..484a1de84d51 100644
@@ -62,6 +62,7 @@ void kvm_update_msi_routes_all(void *private, bool global,
 struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
                                           uint32_t function,
                                           uint32_t index);
+uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg);
 uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
                              uint32_t cpuid_i);
 #endif /* CONFIG_KVM */
target/i386/kvm/tdx.c +34 -0
index 4128f27d6b64..b9a96c2e392d 100644
@@ -45,6 +45,7 @@
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
+static struct kvm_cpuid2 *tdx_supported_cpuid;
 
 /* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
 bool is_tdx_vm(void)
@@ -368,6 +369,20 @@ static Notifier tdx_machine_done_notify = {
     .notify = tdx_finalize_vm,
 };
 
+static void tdx_setup_supported_cpuid(void)
+{
+    if (tdx_supported_cpuid) {
+        return;
+    }
+
+    tdx_supported_cpuid = g_malloc0(sizeof(*tdx_supported_cpuid) +
+                    KVM_MAX_CPUID_ENTRIES * sizeof(struct kvm_cpuid_entry2));
+
+    memcpy(tdx_supported_cpuid->entries, tdx_caps->cpuid.entries,
+           tdx_caps->cpuid.nent * sizeof(struct kvm_cpuid_entry2));
+    tdx_supported_cpuid->nent = tdx_caps->cpuid.nent;
+}
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
@@ -405,6 +420,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         }
     }
 
+    tdx_setup_supported_cpuid();
+
     /* TDX relies on KVM_HC_MAP_GPA_RANGE to handle TDG.VP.VMCALL<MapGPA> */
     if (!kvm_enable_hypercall(BIT_ULL(KVM_HC_MAP_GPA_RANGE))) {
         return -EOPNOTSUPP;
@@ -442,6 +459,22 @@ static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
     x86cpu->enable_cpuid_0x1f = true;
 }
 
+static uint32_t tdx_adjust_cpuid_features(X86ConfidentialGuest *cg,
+                                          uint32_t feature, uint32_t index,
+                                          int reg, uint32_t value)
+{
+    struct kvm_cpuid_entry2 *e;
+
+    if (is_feature_word_cpuid(feature, index, reg)) {
+        e = cpuid_find_entry(tdx_supported_cpuid, feature, index);
+        if (e) {
+            value &= cpuid_entry_get_reg(e, reg);
+        }
+    }
+
+    return value;
+}
+
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
 {
     if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
@@ -824,4 +857,5 @@ static void tdx_guest_class_init(ObjectClass *oc, void *data)
     klass->kvm_init = tdx_kvm_init;
     x86_klass->kvm_type = tdx_kvm_type;
     x86_klass->cpu_instance_init = tdx_cpu_instance_init;
+    x86_klass->adjust_cpuid_features = tdx_adjust_cpuid_features;
 }
Signature · 1 line
2.34.1
Reviewed-by on v8 44/55
#134 lore raw

Re: [PATCH v8 44/55] i386/tdx: Implement adjust_cpuid_features() for TDX

Quoted 20 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:54 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 44/55] i386/tdx: Implement adjust_cpuid_features() for
TDX
X-Mailer: git-send-email 2.34.1

Maintain a TDX specific supported CPUID set, and use it to mask the common supported CPUID value of KVM. It can avoid newly added supported features (reported via KVM_GET_SUPPORTED_CPUID) for common VMs being falsely reported as supported for TDX.

As the first step, initialize the TDX supported CPUID set with all the configurable CPUID bits. It's not complete because there are other CPUID bits are supported for TDX but not reported as directly configurable. E.g. the XFAM related bits, attribute related bits and fixed-1 bits. They will be handled in the future.

Also, what matters are the CPUID bits related to QEMU's feature word. Only mask the CPUID leafs which are feature word leaf.

Quoted 8 lines
---
 target/i386/cpu.c          | 18 ++++++++++++++++++
 target/i386/cpu.h          |  1 +
 target/i386/kvm/kvm.c      |  2 +-
 target/i386/kvm/kvm_i386.h |  1 +
 target/i386/kvm/tdx.c      | 34 ++++++++++++++++++++++++++++++++++
 5 files changed, 55 insertions(+), 1 deletion(-)
Quoted 5 lines
target/i386/cpu.c +0 -0
index f219961b62cd..4061d38d3fbe 100644

target/i386/cpu.c FeatureWordInfo feature_word_info[FEATURE_WORDS] = { · line 1664

Quoted 10 lines
target/i386/cpu.c +7 -0
@@ -1655,6 +1655,24 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     },
 };
 
+bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg)
+{
+    FeatureWordInfo *wi;
+    FeatureWord w;
+
+    for (w = 0; w < FEATURE_WORDS; w++)
+    {

(style nit?) "{" should be at the end of the previous line?

target/i386/cpu.c FeatureWordInfo feature_word_info[FEATURE_WORDS] = { · line 1669

Quoted 5 lines
target/i386/cpu.c +5 -0
+        wi = &feature_word_info[w];
+        if (wi->type == CPUID_FEATURE_WORD && wi->cpuid.eax == feature &&
+            (!wi->cpuid.needs_ecx || wi->cpuid.ecx == index) &&
+            wi->cpuid.reg == reg)
+        {

ditto.

Quoted 6 lines
target/i386/cpu.c +6 -0
+            return true;
+        }
+    }
+    return false;
+}
+
author
#135 lore raw

PATCH v8 45/55 i386/tdx: Add TDX fixed1 bits to supported CPUIDs

In reply to Xiaoyao Li

TDX architecture forcibly sets some CPUID bits for TD guest that VMM cannot disable it. They are fixed1 bits.

Fixed1 bits are not covered by tdx_caps.cpuid (which only contians the directly configurable bits), while fixed1 bits are supported for TD guest obviously.

Add fixed1 bits to tdx_supported_cpuid. Besides, set all the fixed1 bits to the initial set of KVM's support since KVM might not report them as supported.

---
 target/i386/cpu.h          |   2 +
 target/i386/kvm/kvm_i386.h |   7 ++
 target/i386/kvm/tdx.c      | 132 +++++++++++++++++++++++++++++++++++++
 target/i386/sev.c          |   5 --
 4 files changed, 141 insertions(+), 5 deletions(-)
target/i386/cpu.h +2 -0
index 42ef77789ded..115137279a1a 100644
@@ -924,6 +924,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_0_EDX_FSRM              (1U << 4)
 /* AVX512 Vector Pair Intersection to a Pair of Mask Registers */
 #define CPUID_7_0_EDX_AVX512_VP2INTERSECT (1U << 8)
+ /* "md_clear" VERW clears CPU buffers */
+#define CPUID_7_0_EDX_MD_CLEAR          (1U << 10)
 /* SERIALIZE instruction */
 #define CPUID_7_0_EDX_SERIALIZE         (1U << 14)
 /* TSX Suspend Load Address Tracking instruction */
target/i386/kvm/kvm_i386.h +7 -0
index 484a1de84d51..c1bafcfc9b63 100644
@@ -13,8 +13,15 @@
 
 #include "system/kvm.h"
 
+#include <linux/kvm.h>
+
 #define KVM_MAX_CPUID_ENTRIES  100
 
+typedef struct KvmCpuidInfo {
+    struct kvm_cpuid2 cpuid;
+    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
+} KvmCpuidInfo;
+
 /* always false if !CONFIG_KVM */
 #define kvm_pit_in_kernel() \
     (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
target/i386/kvm/tdx.c +132 -0

150 lines not on this page · raw message

target/i386/sev.c +0 -5
index a6c0a697250b..217b19ad7bc6 100644
@@ -214,11 +214,6 @@ static const char *const sev_fw_errlist[] = {
 /* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
 #define KVM_MAX_CPUID_ENTRIES 100
 
-typedef struct KvmCpuidInfo {
-    struct kvm_cpuid2 cpuid;
-    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
-} KvmCpuidInfo;
-
 #define SNP_CPUID_FUNCTION_MAXCOUNT 64
 #define SNP_CPUID_FUNCTION_UNKNOWN 0xFFFFFFFF
 
Signature · 1 line
2.34.1
#136 lore raw

Re: [PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs

Quoted 4 lines · Xiaoyao Li

TDX architecture forcibly sets some CPUID bits for TD guest that VMM cannot disable it. They are fixed1 bits.

Fixed1 bits are not covered by tdx_caps.cpuid (which only contians the

Typo s/contians/contains/

Quoted 224 lines

directly configurable bits), while fixed1 bits are supported for TD guest obviously.

Add fixed1 bits to tdx_supported_cpuid. Besides, set all the fixed1 bits to the initial set of KVM's support since KVM might not report them as supported.

---
 target/i386/cpu.h          |   2 +
 target/i386/kvm/kvm_i386.h |   7 ++
 target/i386/kvm/tdx.c      | 132 +++++++++++++++++++++++++++++++++++++
 target/i386/sev.c          |   5 --
 4 files changed, 141 insertions(+), 5 deletions(-)
target/i386/cpu.h +2 -0
index 42ef77789ded..115137279a1a 100644
@@ -924,6 +924,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_0_EDX_FSRM              (1U << 4)
 /* AVX512 Vector Pair Intersection to a Pair of Mask Registers */
 #define CPUID_7_0_EDX_AVX512_VP2INTERSECT (1U << 8)
+ /* "md_clear" VERW clears CPU buffers */
+#define CPUID_7_0_EDX_MD_CLEAR          (1U << 10)
 /* SERIALIZE instruction */
 #define CPUID_7_0_EDX_SERIALIZE         (1U << 14)
 /* TSX Suspend Load Address Tracking instruction */
target/i386/kvm/kvm_i386.h +7 -0
index 484a1de84d51..c1bafcfc9b63 100644
@@ -13,8 +13,15 @@
 
 #include "system/kvm.h"
 
+#include <linux/kvm.h>
+
 #define KVM_MAX_CPUID_ENTRIES  100
 
+typedef struct KvmCpuidInfo {
+    struct kvm_cpuid2 cpuid;
+    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
+} KvmCpuidInfo;
+
 /* always false if !CONFIG_KVM */
 #define kvm_pit_in_kernel() \
     (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
target/i386/kvm/tdx.c +132 -0
index b9a96c2e392d..49a94d8ffe7d 100644
@@ -369,6 +369,131 @@ static Notifier tdx_machine_done_notify = {
     .notify = tdx_finalize_vm,
 };
 
+/*
+ * Some CPUID bits change from fixed1 to configurable bits when TDX module
+ * supports TDX_FEATURES0.VE_REDUCTION. e.g., MCA/MCE/MTRR/CORE_CAPABILITY.
+ *
+ * To make QEMU work with all the versions of TDX module, keep the fixed1 bits
+ * here if they are ever fixed1 bits in any of the version though not fixed1 in
+ * the latest version. Otherwise, with the older version of TDX module, QEMU may
+ * treat the fixed1 bit as unsupported.
+ *
+ * For newer TDX module, it does no harm to keep them in tdx_fixed1_bits even
+ * though they changed to configurable bits. Because tdx_fixed1_bits is used to
+ * setup the supported bits.
+ */
+KvmCpuidInfo tdx_fixed1_bits = {
+    .cpuid.nent = 8,
+    .entries[0] = {
+        .function = 0x1,
+        .index = 0,
+        .ecx = CPUID_EXT_SSE3 | CPUID_EXT_PCLMULQDQ | CPUID_EXT_DTES64 |
+               CPUID_EXT_DSCPL | CPUID_EXT_SSSE3 | CPUID_EXT_CX16 |
+               CPUID_EXT_PDCM | CPUID_EXT_PCID | CPUID_EXT_SSE41 |
+               CPUID_EXT_SSE42 | CPUID_EXT_X2APIC | CPUID_EXT_MOVBE |
+               CPUID_EXT_POPCNT | CPUID_EXT_AES | CPUID_EXT_XSAVE |
+               CPUID_EXT_RDRAND | CPUID_EXT_HYPERVISOR,
+        .edx = CPUID_FP87 | CPUID_VME | CPUID_DE | CPUID_PSE | CPUID_TSC |
+               CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_CX8 | CPUID_APIC |
+               CPUID_SEP | CPUID_MTRR | CPUID_PGE | CPUID_MCA | CPUID_CMOV |
+               CPUID_PAT | CPUID_CLFLUSH | CPUID_DTS | CPUID_MMX | CPUID_FXSR |
+               CPUID_SSE | CPUID_SSE2,
+    },
+    .entries[1] = {
+        .function = 0x6,
+        .index = 0,
+        .eax = CPUID_6_EAX_ARAT,
+    },
+    .entries[2] = {
+        .function = 0x7,
+        .index = 0,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .ebx = CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_FDP_EXCPTN_ONLY |
+               CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_INVPCID |
+               CPUID_7_0_EBX_ZERO_FCS_FDS | CPUID_7_0_EBX_RDSEED |
+               CPUID_7_0_EBX_SMAP | CPUID_7_0_EBX_CLFLUSHOPT |
+               CPUID_7_0_EBX_CLWB | CPUID_7_0_EBX_SHA_NI,
+        .ecx = CPUID_7_0_ECX_BUS_LOCK_DETECT | CPUID_7_0_ECX_MOVDIRI |
+               CPUID_7_0_ECX_MOVDIR64B,
+        .edx = CPUID_7_0_EDX_MD_CLEAR | CPUID_7_0_EDX_SPEC_CTRL |
+               CPUID_7_0_EDX_STIBP | CPUID_7_0_EDX_FLUSH_L1D |
+               CPUID_7_0_EDX_ARCH_CAPABILITIES | CPUID_7_0_EDX_CORE_CAPABILITY |
+               CPUID_7_0_EDX_SPEC_CTRL_SSBD,
+    },
+    .entries[3] = {
+        .function = 0x7,
+        .index = 2,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .edx = (1U << 0) | (1U << 1) | (1U << 2) | (1U << 4),
+    },
+    .entries[4] = {
+        .function = 0xD,
+        .index = 0,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .eax = XSTATE_FP_MASK | XSTATE_SSE_MASK,
+    },
+    .entries[5] = {
+        .function = 0xD,
+        .index = 1,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .eax = CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC|
+               CPUID_XSAVE_XGETBV1 | CPUID_XSAVE_XSAVES,
+    },
+    .entries[6] = {
+        .function = 0x80000001,
+        .index = 0,
+        .ecx = CPUID_EXT3_LAHF_LM | CPUID_EXT3_ABM | CPUID_EXT3_3DNOWPREFETCH,
+        /* strictly speaking, SYSCALL is not fixed1 bit since it depends on
+         * the CPU to be in 64-bit mode. But here fixed1 is used to serve the
+         * purpose of supported bits for TDX. In this sense, SYACALL is always
+         * supported.
+         */
+        .edx = CPUID_EXT2_SYSCALL | CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB |
+               CPUID_EXT2_RDTSCP | CPUID_EXT2_LM,
+    },
+    .entries[7] = {
+        .function = 0x80000007,
+        .index = 0,
+        .edx = CPUID_APM_INVTSC,
+    },
+};
+
+static struct kvm_cpuid_entry2 *find_in_supported_entry(uint32_t function,
+                                                        uint32_t index)
+{
+    struct kvm_cpuid_entry2 *e;
+
+    e = cpuid_find_entry(tdx_supported_cpuid, function, index);
+    if (!e) {
+        if (tdx_supported_cpuid->nent >= KVM_MAX_CPUID_ENTRIES) {
+            error_report("tdx_supported_cpuid requries more space than %d entries",
+                          KVM_MAX_CPUID_ENTRIES);
+            exit(1);
+        }
+        e = &tdx_supported_cpuid->entries[tdx_supported_cpuid->nent++];
+        e->function = function;
+        e->index = index;
+    }
+
+    return e;
+}
+
+static void tdx_add_supported_cpuid_by_fixed1_bits(void)
+{
+    struct kvm_cpuid_entry2 *e, *e1;
+    int i;
+
+    for (i = 0; i < tdx_fixed1_bits.cpuid.nent; i++) {
+        e = &tdx_fixed1_bits.entries[i];
+
+        e1 = find_in_supported_entry(e->function, e->index);
+        e1->eax |= e->eax;
+        e1->ebx |= e->ebx;
+        e1->ecx |= e->ecx;
+        e1->edx |= e->edx;
+    }
+}
+
 static void tdx_setup_supported_cpuid(void)
 {
     if (tdx_supported_cpuid) {
@@ -381,6 +506,8 @@ static void tdx_setup_supported_cpuid(void)
     memcpy(tdx_supported_cpuid->entries, tdx_caps->cpuid.entries,
            tdx_caps->cpuid.nent * sizeof(struct kvm_cpuid_entry2));
     tdx_supported_cpuid->nent = tdx_caps->cpuid.nent;
+
+    tdx_add_supported_cpuid_by_fixed1_bits();
 }
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
@@ -465,6 +592,11 @@ static uint32_t tdx_adjust_cpuid_features(X86ConfidentialGuest *cg,
 {
     struct kvm_cpuid_entry2 *e;
 
+    e = cpuid_find_entry(&tdx_fixed1_bits.cpuid, feature, index);
+    if (e) {
+        value |= cpuid_entry_get_reg(e, reg);
+    }
+
     if (is_feature_word_cpuid(feature, index, reg)) {
         e = cpuid_find_entry(tdx_supported_cpuid, feature, index);
         if (e) {
target/i386/sev.c +0 -5
index a6c0a697250b..217b19ad7bc6 100644
@@ -214,11 +214,6 @@ static const char *const sev_fw_errlist[] = {
 /* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
 #define KVM_MAX_CPUID_ENTRIES 100
 
-typedef struct KvmCpuidInfo {
-    struct kvm_cpuid2 cpuid;
-    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
-} KvmCpuidInfo;
-
 #define SNP_CPUID_FUNCTION_MAXCOUNT 64
 #define SNP_CPUID_FUNCTION_UNKNOWN 0xFFFFFFFF
 
Signature · 1 line
2.34.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 45/55
#137 lore raw

Re: [PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs

In reply to Xiaoyao Li
Quoted 16 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:55 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs
X-Mailer: git-send-email 2.34.1

TDX architecture forcibly sets some CPUID bits for TD guest that VMM cannot disable it. They are fixed1 bits.

Fixed1 bits are not covered by tdx_caps.cpuid (which only contians the directly configurable bits), while fixed1 bits are supported for TD guest obviously.

Add fixed1 bits to tdx_supported_cpuid. Besides, set all the fixed1 bits to the initial set of KVM's support since KVM might not report them as supported.

Quoted 7 lines
---
 target/i386/cpu.h          |   2 +
 target/i386/kvm/kvm_i386.h |   7 ++
 target/i386/kvm/tdx.c      | 132 +++++++++++++++++++++++++++++++++++++
 target/i386/sev.c          |   5 --
 4 files changed, 141 insertions(+), 5 deletions(-)

LGTM, (only some nits)

target/i386/kvm/tdx.c static Notifier tdx_machine_done_notify = { · line 427

Quoted 5 lines
target/i386/kvm/tdx.c +5 -0
+    .entries[3] = {
+        .function = 0x7,
+        .index = 2,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .edx = (1U << 0) | (1U << 1) | (1U << 2) | (1U << 4),

Missed to use macro?

.edx = CPUID_7_2_EDX_PSFD | CPUID_7_2_EDX_IPRED_CTRL |

       CPUID_7_2_EDX_RRSBA_CTRL | CPUID_7_2_EDX_BHI_CTRL
Quoted 9 lines
target/i386/kvm/tdx.c +9 -0
+    },
+    .entries[4] = {
+        .function = 0xD,
+        .index = 0,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .eax = XSTATE_FP_MASK | XSTATE_SSE_MASK,
+    },
+    .entries[5] = {
+        .function = 0xD,

target/i386/kvm/tdx.c static Notifier tdx_machine_done_notify = { · line 446

Quoted 10 lines
target/i386/kvm/tdx.c +10 -0
+        .index = 1,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .eax = CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC|
+               CPUID_XSAVE_XGETBV1 | CPUID_XSAVE_XSAVES,
+    },
+    .entries[6] = {
+        .function = 0x80000001,
+        .index = 0,
+        .ecx = CPUID_EXT3_LAHF_LM | CPUID_EXT3_ABM | CPUID_EXT3_3DNOWPREFETCH,
+        /* strictly speaking, SYSCALL is not fixed1 bit since it depends on

style nit?

/*
* strictly ...
*/

Quoted 14 lines
target/i386/kvm/tdx.c +14 -0
+         * the CPU to be in 64-bit mode. But here fixed1 is used to serve the
+         * purpose of supported bits for TDX. In this sense, SYACALL is always
+         * supported.
+         */
+        .edx = CPUID_EXT2_SYSCALL | CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB |
+               CPUID_EXT2_RDTSCP | CPUID_EXT2_LM,
+    },
+    .entries[7] = {
+        .function = 0x80000007,
+        .index = 0,
+        .edx = CPUID_APM_INVTSC,
+    },
+};
+

...

target/i386/sev.c static const char *const sev_fw_errlist[] = { · line 215

Quoted 7 lines
target/i386/sev.c +0 -0
index a6c0a697250b..217b19ad7bc6 100644
@@ -214,11 +214,6 @@ static const char *const sev_fw_errlist[] = {
 /* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
 #define KVM_MAX_CPUID_ENTRIES 100

This macro can also be cleaned up in this patch or in your patch 5.

author
#138 lore raw

PATCH v8 46/55 i386/tdx: Add supported CPUID bits related to TD Attributes

In reply to Xiaoyao Li

For TDX, some CPUID feature bit is configured via TD attributes. They are not covered by tdx_caps.cpuid (which only contians the configurable bits), but they are actually supported when the related attributre bit is supported.

---
 target/i386/cpu.h     |  4 +++
 target/i386/kvm/tdx.c | 59 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+)
target/i386/cpu.h +4 -0
index 115137279a1a..0e984ec42bb6 100644
@@ -903,6 +903,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_0_ECX_LA57              (1U << 16)
 /* Read Processor ID */
 #define CPUID_7_0_ECX_RDPID             (1U << 22)
+/* KeyLocker */
+#define CPUID_7_0_ECX_KeyLocker         (1U << 23)
 /* Bus Lock Debug Exception */
 #define CPUID_7_0_ECX_BUS_LOCK_DETECT   (1U << 24)
 /* Cache Line Demote Instruction */
@@ -963,6 +965,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_1_EAX_AVX_VNNI          (1U << 4)
 /* AVX512 BFloat16 Instruction */
 #define CPUID_7_1_EAX_AVX512_BF16       (1U << 5)
+/* Linear address space separation */
+#define CPUID_7_1_EAX_LASS              (1U << 6)
 /* CMPCCXADD Instructions */
 #define CPUID_7_1_EAX_CMPCCXADD         (1U << 7)
 /* Fast Zero REP MOVS */
target/i386/kvm/tdx.c +59 -0

77 lines not on this page · raw message

Signature · 1 line
2.34.1
#139 lore raw

Re: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD Attributes

Quoted 31 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:56 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD
Attributes
X-Mailer: git-send-email 2.34.1

For TDX, some CPUID feature bit is configured via TD attributes. They are not covered by tdx_caps.cpuid (which only contians the configurable bits), but they are actually supported when the related attributre bit is supported.

---
 target/i386/cpu.h     |  4 +++
 target/i386/kvm/tdx.c | 59 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+)
target/i386/cpu.h +2 -0
index 115137279a1a..0e984ec42bb6 100644
@@ -903,6 +903,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_0_ECX_LA57              (1U << 16)
 /* Read Processor ID */
 #define CPUID_7_0_ECX_RDPID             (1U << 22)
+/* KeyLocker */
+#define CPUID_7_0_ECX_KeyLocker         (1U << 23)
 /* Bus Lock Debug Exception */
 #define CPUID_7_0_ECX_BUS_LOCK_DETECT   (1U << 24)
 /* Cache Line Demote Instruction */

target/i386/cpu.h uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w); · line 972

Quoted 8 lines
target/i386/cpu.h +2 -0
@@ -963,6 +965,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_1_EAX_AVX_VNNI          (1U << 4)
 /* AVX512 BFloat16 Instruction */
 #define CPUID_7_1_EAX_AVX512_BF16       (1U << 5)
+/* Linear address space separation */
+#define CPUID_7_1_EAX_LASS              (1U << 6)
 /* CMPCCXADD Instructions */
 #define CPUID_7_1_EAX_CMPCCXADD         (1U << 7)
 /* Fast Zero REP MOVS */

The whole framework is very good for me.

Since LASS/key locker are not yet supported, I think it's better to add the note (in the commit message or code) that they are helpful for future enabling. Of course, removing these two features would be best, as they cannot be covered by current testing.

Regards,
Zhao

author
#140 lore raw

Re: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD Attributes

Quoted 43 lines · Zhao Liu
Quoted 31 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:56 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD
Attributes
X-Mailer: git-send-email 2.34.1

For TDX, some CPUID feature bit is configured via TD attributes. They are not covered by tdx_caps.cpuid (which only contians the configurable bits), but they are actually supported when the related attributre bit is supported.

---
  target/i386/cpu.h     |  4 +++
  target/i386/kvm/tdx.c | 59 +++++++++++++++++++++++++++++++++++++++++++
  2 files changed, 63 insertions(+)
target/i386/cpu.h +2 -0
index 115137279a1a..0e984ec42bb6 100644
@@ -903,6 +903,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
  #define CPUID_7_0_ECX_LA57              (1U << 16)
  /* Read Processor ID */
  #define CPUID_7_0_ECX_RDPID             (1U << 22)
+/* KeyLocker */
+#define CPUID_7_0_ECX_KeyLocker         (1U << 23)
  /* Bus Lock Debug Exception */
  #define CPUID_7_0_ECX_BUS_LOCK_DETECT   (1U << 24)
  /* Cache Line Demote Instruction */

target/i386/cpu.h uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w); · line 972

Quoted 8 lines
target/i386/cpu.h +2 -0
@@ -963,6 +965,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
  #define CPUID_7_1_EAX_AVX_VNNI          (1U << 4)
  /* AVX512 BFloat16 Instruction */
  #define CPUID_7_1_EAX_AVX512_BF16       (1U << 5)
+/* Linear address space separation */
+#define CPUID_7_1_EAX_LASS              (1U << 6)
  /* CMPCCXADD Instructions */
  #define CPUID_7_1_EAX_CMPCCXADD         (1U << 7)
  /* Fast Zero REP MOVS */

The whole framework is very good for me.

Quoted 4 lines

Since LASS/key locker are not yet supported, I think it's better to add the note (in the commit message or code) that they are helpful for future enabling. Of course, removing these two features would be best, as they cannot be covered by current testing.

I will add note in the commit message and don't change the code.

Keep them in tdx_attrs_maps[] is helpful for the completeness of the existing attribute bits that are related to a CPUID feature.

Quoted 4 lines

Regards,
Zhao

author
#141 lore raw

PATCH v8 47/55 i386/tdx: Add supported CPUID bits relates to XFAM

In reply to Xiaoyao Li

Some CPUID bits are controlled by XFAM. They are not covered by tdx_caps.cpuid (which only contians the directly configurable bits), but they are actually supported when the related XFAM bit is supported.

Add these XFAM controlled bits to TDX supported CPUID bits based on the supported_xfam.

Besides, incorporate the supported_xfam into the supported CPUID leaf of 0xD.

---
 target/i386/cpu.c     | 12 -------
 target/i386/cpu.h     | 16 ++++++++++
 target/i386/kvm/tdx.c | 73 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 89 insertions(+), 12 deletions(-)
target/i386/cpu.c +0 -12
index 4061d38d3fbe..f09a1caac071 100644
@@ -1673,15 +1673,6 @@ bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg)
     return false;
 }
 
-typedef struct FeatureMask {
-    FeatureWord index;
-    uint64_t mask;
-} FeatureMask;
-
-typedef struct FeatureDep {
-    FeatureMask from, to;
-} FeatureDep;
-
 static FeatureDep feature_dependencies[] = {
     {
         .from = { FEAT_7_0_EDX,             CPUID_7_0_EDX_ARCH_CAPABILITIES },
@@ -1850,9 +1841,6 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = {
 };
 #undef REGISTER
 
-/* CPUID feature bits available in XSS */
-#define CPUID_XSTATE_XSS_MASK    (XSTATE_ARCH_LBR_MASK)
-
 ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
     [XSTATE_FP_BIT] = {
         /* x87 FP state component is always enabled if XSAVE is supported */
target/i386/cpu.h +16 -0
index 0e984ec42bb6..132312d70a54 100644
@@ -588,6 +588,7 @@ typedef enum X86Seg {
 #define XSTATE_OPMASK_BIT               5
 #define XSTATE_ZMM_Hi256_BIT            6
 #define XSTATE_Hi16_ZMM_BIT             7
+#define XSTATE_PT_BIT                   8
 #define XSTATE_PKRU_BIT                 9
 #define XSTATE_ARCH_LBR_BIT             15
 #define XSTATE_XTILE_CFG_BIT            17
@@ -601,6 +602,7 @@ typedef enum X86Seg {
 #define XSTATE_OPMASK_MASK              (1ULL << XSTATE_OPMASK_BIT)
 #define XSTATE_ZMM_Hi256_MASK           (1ULL << XSTATE_ZMM_Hi256_BIT)
 #define XSTATE_Hi16_ZMM_MASK            (1ULL << XSTATE_Hi16_ZMM_BIT)
+#define XSTATE_PT_MASK                  (1ULL << XSTATE_PT_BIT)
 #define XSTATE_PKRU_MASK                (1ULL << XSTATE_PKRU_BIT)
 #define XSTATE_ARCH_LBR_MASK            (1ULL << XSTATE_ARCH_LBR_BIT)
 #define XSTATE_XTILE_CFG_MASK           (1ULL << XSTATE_XTILE_CFG_BIT)
@@ -623,6 +625,11 @@ typedef enum X86Seg {
                                  XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK | \
                                  XSTATE_XTILE_CFG_MASK | XSTATE_XTILE_DATA_MASK)
 
+/* CPUID feature bits available in XSS */
+#define CPUID_XSTATE_XSS_MASK    (XSTATE_ARCH_LBR_MASK)
+
+#define CPUID_XSTATE_MASK       (CPUID_XSTATE_XCR0_MASK | CPUID_XSTATE_XSS_MASK)
+
 /* CPUID feature words */
 typedef enum FeatureWord {
     FEAT_1_EDX,         /* CPUID[1].EDX */
@@ -671,6 +678,15 @@ typedef enum FeatureWord {
     FEATURE_WORDS,
 } FeatureWord;
 
+typedef struct FeatureMask {
+    FeatureWord index;
+    uint64_t mask;
+} FeatureMask;
+
+typedef struct FeatureDep {
+    FeatureMask from, to;
+} FeatureDep;
+
 typedef uint64_t FeatureWordArray[FEATURE_WORDS];
 uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 
target/i386/kvm/tdx.c +73 -0

97 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#142 lore raw

PATCH v8 48/55 i386/tdx: Add XFD to supported bit of TDX

In reply to Xiaoyao Li

Just mark XFD as always supported for TDX. This simple solution relies on the fact KVM will report XFD as 0 when it's not supported by the hardware.

---
 target/i386/cpu.h     | 1 +
 target/i386/kvm/tdx.c | 5 +++++
 2 files changed, 6 insertions(+)
target/i386/cpu.h +1 -0
index 132312d70a54..a223e09a25c4 100644
@@ -1126,6 +1126,7 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_XSAVE_XSAVEC     (1U << 1)
 #define CPUID_XSAVE_XGETBV1    (1U << 2)
 #define CPUID_XSAVE_XSAVES     (1U << 3)
+#define CPUID_XSAVE_XFD        (1U << 4)
 
 #define CPUID_6_EAX_ARAT       (1U << 2)
 
target/i386/kvm/tdx.c +5 -0
index c7b4a098d12f..e07cd9a1ee15 100644
@@ -620,6 +620,11 @@ static void tdx_add_supported_cpuid_by_xfam(void)
     e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XCR0_MASK) >> 32;
 
     e = find_in_supported_entry(0xd, 1);
+    /* Mark XFD always support for TDX, it will be cleared finally in
+     * tdx_adjust_cpuid_features() if XFD is unavailable on the hardware
+     * because in this case the original data has it as 0.
+     */
+    e->eax |= CPUID_XSAVE_XFD;
     e->ecx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK);
     e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK) >> 32;
 }
Signature · 1 line
2.34.1
Reviewed-by on v8 48/55
#143 lore raw

Re: [PATCH v8 48/55] i386/tdx: Add XFD to supported bit of TDX

Quoted 9 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:58 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 48/55] i386/tdx: Add XFD to supported bit of TDX
X-Mailer: git-send-email 2.34.1

Just mark XFD as always supported for TDX. This simple solution relies on the fact KVM will report XFD as 0 when it's not supported by the hardware.

Quoted 5 lines
---
 target/i386/cpu.h     | 1 +
 target/i386/kvm/tdx.c | 5 +++++
 2 files changed, 6 insertions(+)
author
#144 lore raw

PATCH v8 49/55 i386/tdx: Define supported KVM features for TDX

In reply to Xiaoyao Li

For TDX, only limited KVM PV features are supported.

---
 target/i386/kvm/tdx.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
target/i386/kvm/tdx.c +20 -0
index e07cd9a1ee15..7382b53fcc51 100644
@@ -32,6 +32,8 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#include "standard-headers/asm-x86/kvm_para.h"
+
 #define TDX_MIN_TSC_FREQUENCY_KHZ   (100 * 1000)
 #define TDX_MAX_TSC_FREQUENCY_KHZ   (10 * 1000 * 1000)
 
@@ -44,6 +46,14 @@
                                  TDX_TD_ATTRIBUTES_PKS | \
                                  TDX_TD_ATTRIBUTES_PERFMON)
 
+#define TDX_SUPPORTED_KVM_FEATURES  ((1U << KVM_FEATURE_NOP_IO_DELAY) | \
+                                     (1U << KVM_FEATURE_PV_UNHALT) | \
+                                     (1U << KVM_FEATURE_PV_TLB_FLUSH) | \
+                                     (1U << KVM_FEATURE_PV_SEND_IPI) | \
+                                     (1U << KVM_FEATURE_POLL_CONTROL) | \
+                                     (1U << KVM_FEATURE_PV_SCHED_YIELD) | \
+                                     (1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -629,6 +639,14 @@ static void tdx_add_supported_cpuid_by_xfam(void)
     e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK) >> 32;
 }
 
+static void tdx_add_supported_kvm_features(void)
+{
+    struct kvm_cpuid_entry2 *e;
+
+    e = find_in_supported_entry(0x40000001, 0);
+    e->eax = TDX_SUPPORTED_KVM_FEATURES;
+}
+
 static void tdx_setup_supported_cpuid(void)
 {
     if (tdx_supported_cpuid) {
@@ -645,6 +663,8 @@ static void tdx_setup_supported_cpuid(void)
     tdx_add_supported_cpuid_by_fixed1_bits();
     tdx_add_supported_cpuid_by_attrs();
     tdx_add_supported_cpuid_by_xfam();
+
+    tdx_add_supported_kvm_features();
 }
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
Signature · 1 line
2.34.1
Reviewed-by on v8 49/55
#145 lore raw

Re: [PATCH v8 49/55] i386/tdx: Define supported KVM features for TDX

Quoted 11 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:01:59 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 49/55] i386/tdx: Define supported KVM features for TDX
X-Mailer: git-send-email 2.34.1

For TDX, only limited KVM PV features are supported.

---
 target/i386/kvm/tdx.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
author
#146 lore raw

PATCH v8 50/55 i386/cgs: Introduce x86_confidential_guest_check_features()

In reply to Xiaoyao Li

To do cgs specific feature checking. Note the feature checking in x86_cpu_filter_features() is valid for non-cgs VMs. For cgs VMs like TDX, what features can be supported has more restrictions.

---
 target/i386/confidential-guest.h | 13 +++++++++++++
 target/i386/kvm/kvm.c            |  8 ++++++++
 2 files changed, 21 insertions(+)
target/i386/confidential-guest.h +13 -0
index 777d43cc9688..48b88dbd3130 100644
@@ -42,6 +42,7 @@ struct X86ConfidentialGuestClass {
     void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
     uint32_t (*adjust_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature,
                                       uint32_t index, int reg, uint32_t value);
+    int (*check_features)(X86ConfidentialGuest *cg, CPUState *cs);
 };
 
 /**
@@ -91,4 +92,16 @@ static inline int x86_confidential_guest_adjust_cpuid_features(X86ConfidentialGu
     }
 }
 
+static inline int x86_confidential_guest_check_features(X86ConfidentialGuest *cg,
+                                                        CPUState *cs)
+{
+    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
+
+    if (klass->check_features) {
+        return klass->check_features(cg, cs);
+    }
+
+    return 0;
+}
+
 #endif
target/i386/kvm/kvm.c +8 -0
index 17d7bf6ae9aa..27b4a069d194 100644
@@ -2092,6 +2092,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
     int r;
     Error *local_err = NULL;
 
+    if (current_machine->cgs) {
+        r = x86_confidential_guest_check_features(
+                X86_CONFIDENTIAL_GUEST(current_machine->cgs), cs);
+        if (r < 0) {
+            return r;
+        }
+    }
+
     memset(&cpuid_data, 0, sizeof(cpuid_data));
 
     cpuid_i = 0;
Signature · 1 line
2.34.1
Reviewed-by on v8 50/55
#147 lore raw

Re: [PATCH v8 50/55] i386/cgs: Introduce x86_confidential_guest_check_features()

Quoted 10 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:02:00 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 50/55] i386/cgs: Introduce
x86_confidential_guest_check_features()
X-Mailer: git-send-email 2.34.1

To do cgs specific feature checking. Note the feature checking in x86_cpu_filter_features() is valid for non-cgs VMs. For cgs VMs like TDX, what features can be supported has more restrictions.

Quoted 5 lines
---
 target/i386/confidential-guest.h | 13 +++++++++++++
 target/i386/kvm/kvm.c            |  8 ++++++++
 2 files changed, 21 insertions(+)
author
#148 lore raw

PATCH v8 51/55 i386/tdx: Fetch and validate CPUID of TD guest

In reply to Xiaoyao Li

Use KVM_TDX_GET_CPUID to get the CPUIDs that are managed and enfored by TDX module for TD guest. Check QEMU's configuration against the fetched data.

Print wanring message when 1. a feature is not supported but requested by QEMU or 2. QEMU doesn't want to expose a feature while it is enforced enabled.

- If cpu->enforced_cpuid is not set, prints the warning message of both
1) and 2) and tweak QEMU's configuration.

- If cpu->enforced_cpuid is set, quit if any case of 1) or 2).

---
 target/i386/cpu.c     | 33 ++++++++++++++-
 target/i386/cpu.h     |  7 +++
 target/i386/kvm/tdx.c | 99 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 137 insertions(+), 2 deletions(-)
target/i386/cpu.c +31 -2
index f09a1caac071..41407c8a7248 100644
@@ -5772,8 +5772,8 @@ static bool x86_cpu_have_filtered_features(X86CPU *cpu)
     return false;
 }
 
-static void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
-                                      const char *verbose_prefix)
+void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                               const char *verbose_prefix)
 {
     CPUX86State *env = &cpu->env;
     FeatureWordInfo *f = &feature_word_info[w];
@@ -5800,6 +5800,35 @@ static void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
     }
 }
 
+void mark_forced_on_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                             const char *verbose_prefix)
+{
+    CPUX86State *env = &cpu->env;
+    FeatureWordInfo *f = &feature_word_info[w];
+    int i;
+
+    if (!cpu->force_features) {
+        env->features[w] |= mask;
+    }
+
+    cpu->forced_on_features[w] |= mask;
+
+    if (!verbose_prefix) {
+        return;
+    }
+
+    for (i = 0; i < 64; ++i) {
+        if ((1ULL << i) & mask) {
+            g_autofree char *feat_word_str = feature_word_description(f);
+            warn_report("%s: %s%s%s [bit %d]",
+                        verbose_prefix,
+                        feat_word_str,
+                        f->feat_names[i] ? "." : "",
+                        f->feat_names[i] ? f->feat_names[i] : "", i);
+        }
+    }
+}
+
 static void x86_cpuid_version_get_family(Object *obj, Visitor *v,
                                          const char *name, void *opaque,
                                          Error **errp)
target/i386/cpu.h +7 -0
index a223e09a25c4..1600e826f372 100644
@@ -2227,6 +2227,9 @@ struct ArchCPU {
     /* Features that were filtered out because of missing host capabilities */
     FeatureWordArray filtered_features;
 
+    /* Features that are forced enabled by underlying hypervisor, e.g., TDX */
+    FeatureWordArray forced_on_features;
+
     /* Enable PMU CPUID bits. This can't be enabled by default yet because
      * it doesn't have ABI stability guarantees, as it passes all PMU CPUID
      * bits returned by GET_SUPPORTED_CPUID (that depend on host CPU and kernel
@@ -2539,6 +2542,10 @@ void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 bool cpu_has_x2apic_feature(CPUX86State *env);
 bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg);
+void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                               const char *verbose_prefix);
+void mark_forced_on_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                             const char *verbose_prefix);
 
 static inline bool x86_has_cpuid_0x1f(X86CPU *cpu)
 {
target/i386/kvm/tdx.c +99 -0

109 lines not on this page · raw message

Signature · 1 line
2.34.1
author
#149 lore raw

PATCH v8 52/55 i386/tdx: Don't treat SYSCALL as unavailable

In reply to Xiaoyao Li

---
Changes in v7:
- fix CPUID_EXT2_SYSCALL by adding it to actual;

---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
target/i386/kvm/tdx.c +13 -0
index 58797470ba7e..cd8e96d476b8 100644
@@ -841,6 +841,19 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
             continue;
         }
 
+        /* Fixup for special cases */
+        switch (w) {
+        case FEAT_8000_0001_EDX:
+            /*
+             * Intel enumerates SYSCALL bit as 1 only when processor in 64-bit
+             * mode and before vcpu running it's not in 64-bit mode.
+             */
+            actual |= CPUID_EXT2_SYSCALL;
+            break;
+        default:
+            break;
+        }
+
         requested = env->features[w];
         unavailable = requested & ~actual;
         mark_unavailable_features(cpu, w, unavailable, unav_prefix);
Signature · 1 line
2.34.1
Reviewed-by on v8 52/55
#150 lore raw

Re: [PATCH v8 52/55] i386/tdx: Don't treat SYSCALL as unavailable

Quoted 4 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:02:02 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 52/55] i386/tdx: Don't treat SYSCALL as unavailable
X-Mailer: git-send-email 2.34.1

It's better to add the commit message (for example, your additional comment at v6 [1]).

[1]: https://lore.kernel.org/qemu-devel/[email protected]/

Quoted 7 lines

---
Changes in v7:
- fix CPUID_EXT2_SYSCALL by adding it to actual;

---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
author
#151 lore raw

PATCH v8 53/55 i386/tdx: Make invtsc default on

In reply to Xiaoyao Li

Because it's fixed1 bit that enforced by TDX module.

---
 target/i386/kvm/tdx.c | 3 +++
 1 file changed, 3 insertions(+)
target/i386/kvm/tdx.c +3 -0
index cd8e96d476b8..8f075ba5a4de 100644
@@ -740,6 +740,9 @@ static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
 
     object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
 
+    /* invtsc is fixed1 for TD guest */
+    object_property_set_bool(OBJECT(cpu), "invtsc", true, &error_abort);
+
     x86cpu->enable_cpuid_0x1f = true;
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v8 53/55
#152 lore raw

Re: [PATCH v8 53/55] i386/tdx: Make invtsc default on

Quoted 11 lines · Xiaoyao Li

Date: Tue, 1 Apr 2025 09:02:03 -0400
From: Xiaoyao Li <[email protected]>
Subject: [PATCH v8 53/55] i386/tdx: Make invtsc default on
X-Mailer: git-send-email 2.34.1

Because it's fixed1 bit that enforced by TDX module.

---
 target/i386/kvm/tdx.c | 3 +++
 1 file changed, 3 insertions(+)
author
#153 lore raw

PATCH v8 54/55 i386/tdx: Validate phys_bits against host value

In reply to Xiaoyao Li

For TDX guest, the phys_bits is not configurable and can only be host/native value.

Validate phys_bits inside tdx_check_features().

---
 target/i386/host-cpu.c | 2 +-
 target/i386/host-cpu.h | 1 +
 target/i386/kvm/tdx.c  | 8 ++++++++
 3 files changed, 10 insertions(+), 1 deletion(-)
target/i386/host-cpu.c +1 -1
index 3e4e85e729c8..8a15af458b05 100644
@@ -15,7 +15,7 @@
 #include "system/system.h"
 
 /* Note: Only safe for use on x86(-64) hosts */
-static uint32_t host_cpu_phys_bits(void)
+uint32_t host_cpu_phys_bits(void)
 {
     uint32_t eax;
     uint32_t host_phys_bits;
target/i386/host-cpu.h +1 -0
index 6a9bc918baa4..b97ec01c9bec 100644
@@ -10,6 +10,7 @@
 #ifndef HOST_CPU_H
 #define HOST_CPU_H
 
+uint32_t host_cpu_phys_bits(void);
 void host_cpu_instance_init(X86CPU *cpu);
 void host_cpu_max_instance_init(X86CPU *cpu);
 bool host_cpu_realizefn(CPUState *cs, Error **errp);
target/i386/kvm/tdx.c +8 -0
index 8f075ba5a4de..c8fffa94c7d2 100644
@@ -25,6 +25,7 @@
 
 #include "cpu.h"
 #include "cpu-internal.h"
+#include "host-cpu.h"
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
@@ -875,6 +876,13 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
         return -1;
     }
 
+    if (cpu->phys_bits != host_cpu_phys_bits()) {
+        error_report("TDX requires guest CPU physical bits (%u) "
+                     "to match host CPU physical bits (%u)",
+                     cpu->phys_bits, host_cpu_phys_bits());
+        exit(1);
+    }
+
     return 0;
 }
 
Signature · 1 line
2.34.1
Reviewed-by on v8 54/55
#154 lore raw

Re: [PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

Quoted 11 lines · Xiaoyao Li

For TDX guest, the phys_bits is not configurable and can only be host/native value.

Validate phys_bits inside tdx_check_features().

---
 target/i386/host-cpu.c | 2 +-
 target/i386/host-cpu.h | 1 +
 target/i386/kvm/tdx.c  | 8 ++++++++
 3 files changed, 10 insertions(+), 1 deletion(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v8 54/55
#155 lore raw

Re: [PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

In reply to Xiaoyao Li
Quoted 2 lines

@e -875,6 +876,13 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)

         return -1;

Here you has already used "return -1", so...

Review

Quoted 7 lines
     }
(quoted diff) +5 -0
+    if (cpu->phys_bits != host_cpu_phys_bits()) {
+        error_report("TDX requires guest CPU physical bits (%u) "
+                     "to match host CPU physical bits (%u)",
+                     cpu->phys_bits, host_cpu_phys_bits());
+        exit(1);

...what about "return -1"?

kvm_init_vcpu() missed a "goto err", but it is not a big deal:

accel/kvm/kvm-all.c +1 -1
index 9862d8ff1d38..05034f622f20 100644
@@ -595,6 +595,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
         error_setg_errno(errp, -ret,
                          "kvm_init_vcpu: kvm_arch_init_vcpu failed (%lu)",
                          kvm_arch_vcpu_id(cpu));
+        goto err;
     }
     cpu->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
---

Overall, LGTM,

Quoted 9 lines
(quoted diff) +2 -0
+    }
+
     return 0;
 }
Signature · 2 lines
-- 
2.34.1
author
#156 lore raw

Re: [PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

Quoted 7 lines · Zhao Liu
Quoted 2 lines

@e -875,6 +876,13 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)

          return -1;

Here you has already used "return -1", so...

Quoted 2 lines
      }
Quoted 7 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+    if (cpu->phys_bits != host_cpu_phys_bits()) {
+        error_report("TDX requires guest CPU physical bits (%u) "
+                     "to match host CPU physical bits (%u)",
+                     cpu->phys_bits, host_cpu_phys_bits());
+        exit(1);

...what about "return -1"?

I will change it to return -EINVAL and as well for the above. Since ...

accel/kvm/kvm-all.c int kvm_init_vcpu(CPUState *cpu, Error **errp) · line 597

Quoted 10 lines

kvm_init_vcpu() missed a "goto err", but it is not a big deal:

accel/kvm/kvm-all.c +0 -0
index 9862d8ff1d38..05034f622f20 100644
@@ -595,6 +595,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
          error_setg_errno(errp, -ret,
                           "kvm_init_vcpu: kvm_arch_init_vcpu failed (%lu)",
                           kvm_arch_vcpu_id(cpu));

... the handling here expects the @ret is a errno.

Quoted 18 lines
accel/kvm/kvm-all.c +1 -1
+        goto err;
      }
      cpu->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
---

Overall, LGTM,

Quoted 9 lines
(quoted diff) +2 -0
+    }
+
      return 0;
  }
Signature · 2 lines
-- 
2.34.1
author
#157 lore raw

PATCH v8 55/55 docs: Add TDX documentation

In reply to Xiaoyao Li

Add docs/system/i386/tdx.rst for TDX support, and add tdx in confidential-guest-support.rst

---
Changes in v6:
- Add more information of "Feature configuration"
- Mark TD Attestation as future work because KVM now drops the support of it.

Changes in v5:
- Add TD attestation section and update the QEMU parameter;

Changes since v1:
- Add prerequisite of private gmem;
- update example command to launch TD;

Changes since RFC v4:
- add the restriction that kernel-irqchip must be split

---
 docs/system/confidential-guest-support.rst |   1 +
 docs/system/i386/tdx.rst                   | 156 +++++++++++++++++++++
 docs/system/target-i386.rst                |   1 +
 3 files changed, 158 insertions(+)
 create mode 100644 docs/system/i386/tdx.rst
docs/system/confidential-guest-support.rst +1 -0
index 0c490dbda2b7..66129fbab64c 100644
@@ -38,6 +38,7 @@ Supported mechanisms
 Currently supported confidential guest mechanisms are:
 
 * AMD Secure Encrypted Virtualization (SEV) (see :doc:`i386/amd-memory-encryption`)
+* Intel Trust Domain Extension (TDX) (see :doc:`i386/tdx`)
 * POWER Protected Execution Facility (PEF) (see :ref:`power-papr-protected-execution-facility-pef`)
 * s390x Protected Virtualization (PV) (see :doc:`s390x/protvirt`)
 
docs/system/i386/tdx.rst new file +156 -0

156 lines not on this page · raw message

docs/system/target-i386.rst +1 -0
index ab7af1a75d6e..43b09c79d6be 100644
@@ -31,6 +31,7 @@ Architectural features
    i386/kvm-pv
    i386/sgx
    i386/amd-memory-encryption
+   i386/tdx
 
 OS requirements
 ~~~~~~~~~~~~~~~
Signature · 1 line
2.34.1
#158 lore raw

Re: [PATCH v8 55/55] docs: Add TDX documentation

CC libvirt / Jiri, for confirmation about whether the CPUID restrictions listed below will have any possible impact on libvirt CPUID handling...

Quoted 82 lines · Xiaoyao Li

Add docs/system/i386/tdx.rst for TDX support, and add tdx in confidential-guest-support.rst

---
Changes in v6:
- Add more information of "Feature configuration"
- Mark TD Attestation as future work because KVM now drops the support of it.

Changes in v5:
- Add TD attestation section and update the QEMU parameter;

Changes since v1:
- Add prerequisite of private gmem;
- update example command to launch TD;

Changes since RFC v4:
- add the restriction that kernel-irqchip must be split

---
 docs/system/confidential-guest-support.rst |   1 +
 docs/system/i386/tdx.rst                   | 156 +++++++++++++++++++++
 docs/system/target-i386.rst                |   1 +
 3 files changed, 158 insertions(+)
 create mode 100644 docs/system/i386/tdx.rst
docs/system/i386/tdx.rst new file +55 -0
+Feature Configuration
+---------------------
+
+Unlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD are not
+under full control of VMM. VMM can only configure part of features of a TD on
+``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl.
+
+The configurable features have three types:
+
+- Attributes:
+  - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD,
+  which determines related CPUID bit and CR4 bit;
+  - PERFMON (bit 63) controls whether PMU is exposed to TD.
+
+- XSAVE related features (XFAM):
+  XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It
+  determines the set of extended features available for use by the guest TD.
+
+- CPUID features:
+  Only some bits of some CPUID leaves are directly configurable by VMM.
+
+What features can be configured is reported via TDX capabilities.
+
+TDX capabilities
+~~~~~~~~~~~~~~~~
+
+The VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES``
+to get the TDX capabilities from KVM. It returns a data structure of
+``struct kvm_tdx_capabilities``, which tells the supported configuration of
+attributes, XFAM and CPUIDs.
+
+TD attributes
+~~~~~~~~~~~~~
+
+QEMU supports configuring raw 64-bit TD attributes directly via "attributes"
+property of "tdx-guest" object. Note, it's users' responsibility to provide a
+valid value because some bits may not supported by current QEMU or KVM yet.
+
+QEMU also supports the configuration of individual attribute bits that are
+supported by it, via properties of "tdx-guest" object.
+E.g., "sept-ve-disable" (bit 28).
+
+MSR based features
+~~~~~~~~~~~~~~~~~~
+
+Current KVM doesn't support MSR based feature (e.g., MSR_IA32_ARCH_CAPABILITIES)
+configuration for TDX, and it's a future work to enable it in QEMU when KVM adds
+support of it.
+
+Feature check
+~~~~~~~~~~~~~
+
+QEMU checks if the final (CPU) features, determined by given cpu model and
+explicit feature adjustment of "+featureA/-featureB", can be supported or not.
+It can produce feature not supported warning like

docs/system/i386/tdx.rst line 98

Quoted 10 lines
docs/system/i386/tdx.rst new file +10 -0
+
+  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
+
+It can also produce warning like
+
+  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
+
+if the fixed-1 feature is requested to be disabled explicitly. This is newly
+added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
+by TDX module and VMM cannot disable them.

This is where I'm wondering if libvirt has anything to be concerned about. Possibly when libvirt queries the actual CPUID after launching the guest it will just "do the right thing" ? Wondering if there's any need for libvirt to be aware of CPUID restrictions before that point though ?

docs/system/i386/tdx.rst line 111

Quoted 13 lines
docs/system/i386/tdx.rst new file +13 -0
+
+Launching a TD (TDX VM)
+-----------------------
+
+To launch a TD, the necessary command line options are tdx-guest object and
+split kernel-irqchip, as below:
+
+.. parsed-literal::
+
+    |qemu_system_x86| \\
+        -object tdx-guest,id=tdx0 \\
+        -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
+        -bios OVMF.fd \\

I don't think we need to show 'kernel-irqchip=split' now that we "do the right thing" by default

This surely also ought to include '-accel kvm', as IIUC there's no TCG support for TDX.

And presumably '-cpu host', since QEMU's default 'qemu64' CPU model is likely a terrible match for what TDX will force set.

docs/system/i386/tdx.rst line 116

Quoted 5 lines
docs/system/i386/tdx.rst new file +5 -0
+
+Restrictions
+------------
+
+ - kernel-irqchip must be split;

Can append

"This is set by default for TDX guests if kernel-irqchip is left on
its default 'auto' setting."

docs/system/i386/tdx.rst line 122

Quoted 6 lines
docs/system/i386/tdx.rst new file +6 -0
+
+ - No readonly support for private memory;
+
+ - No SMM support: SMM support requires manipulating the guest register states
+   which is not allowed;
+

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#159 lore raw

Re: [PATCH v8 55/55] docs: Add TDX documentation

Quoted 2 lines · Daniel P. Berrangé

CC libvirt / Jiri, for confirmation about whether the CPUID restrictions listed below will have any possible impact on libvirt CPUID handling...

..

Quoted 14 lines
Quoted 14 lines
docs/system/i386/tdx.rst new file +14 -0
+Feature check
+~~~~~~~~~~~~~
+
+QEMU checks if the final (CPU) features, determined by given cpu model and
+explicit feature adjustment of "+featureA/-featureB", can be supported or not.
+It can produce feature not supported warning like
+
+  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
+
+It can also produce warning like
+
+  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
+
+if the fixed-1 feature is requested to be disabled explicitly. This is newly
Quoted 8 lines

docs/system/i386/tdx.rst line 98

Quoted 2 lines
docs/system/i386/tdx.rst new file +2 -0
+added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
+by TDX module and VMM cannot disable them.

This is where I'm wondering if libvirt has anything to be concerned about. Possibly when libvirt queries the actual CPUID after launching the guest it will just "do the right thing" ? Wondering if there's any need for libvirt to be aware of CPUID restrictions before that point though ?

Yes, it will do the right thing.

Unless a user explicitly requests check='full'. Libvirt will use check='full' on its own only during migration and at that point the CPU definition is already modified according to what QEMU changed when the domain was started. So this will just work.

For explicit check='full' to be usable we'd need a way to tell users which features are required/forbidden with TDX. But to be honest I don't think this is needed.

Jirka

author
#160 lore raw

Re: [PATCH v8 55/55] docs: Add TDX documentation

In reply to Daniel P. Berrangé
Quoted 136 lines · Daniel P. Berrangé

CC libvirt / Jiri, for confirmation about whether the CPUID restrictions listed below will have any possible impact on libvirt CPUID handling...

Quoted 82 lines · Xiaoyao Li

Add docs/system/i386/tdx.rst for TDX support, and add tdx in confidential-guest-support.rst

---
Changes in v6:
- Add more information of "Feature configuration"
- Mark TD Attestation as future work because KVM now drops the support of it.

Changes in v5:
- Add TD attestation section and update the QEMU parameter;

Changes since v1:
- Add prerequisite of private gmem;
- update example command to launch TD;

Changes since RFC v4:
- add the restriction that kernel-irqchip must be split

---
  docs/system/confidential-guest-support.rst |   1 +
  docs/system/i386/tdx.rst                   | 156 +++++++++++++++++++++
  docs/system/target-i386.rst                |   1 +
  3 files changed, 158 insertions(+)

create mode 100644 docs/system/i386/tdx.rst

docs/system/i386/tdx.rst new file +55 -0
+Feature Configuration
+---------------------
+
+Unlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD are not
+under full control of VMM. VMM can only configure part of features of a TD on
+``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl.
+
+The configurable features have three types:
+
+- Attributes:
+  - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD,
+  which determines related CPUID bit and CR4 bit;
+  - PERFMON (bit 63) controls whether PMU is exposed to TD.
+
+- XSAVE related features (XFAM):
+  XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It
+  determines the set of extended features available for use by the guest TD.
+
+- CPUID features:
+  Only some bits of some CPUID leaves are directly configurable by VMM.
+
+What features can be configured is reported via TDX capabilities.
+
+TDX capabilities
+~~~~~~~~~~~~~~~~
+
+The VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES``
+to get the TDX capabilities from KVM. It returns a data structure of
+``struct kvm_tdx_capabilities``, which tells the supported configuration of
+attributes, XFAM and CPUIDs.
+
+TD attributes
+~~~~~~~~~~~~~
+
+QEMU supports configuring raw 64-bit TD attributes directly via "attributes"
+property of "tdx-guest" object. Note, it's users' responsibility to provide a
+valid value because some bits may not supported by current QEMU or KVM yet.
+
+QEMU also supports the configuration of individual attribute bits that are
+supported by it, via properties of "tdx-guest" object.
+E.g., "sept-ve-disable" (bit 28).
+
+MSR based features
+~~~~~~~~~~~~~~~~~~
+
+Current KVM doesn't support MSR based feature (e.g., MSR_IA32_ARCH_CAPABILITIES)
+configuration for TDX, and it's a future work to enable it in QEMU when KVM adds
+support of it.
+
+Feature check
+~~~~~~~~~~~~~
+
+QEMU checks if the final (CPU) features, determined by given cpu model and
+explicit feature adjustment of "+featureA/-featureB", can be supported or not.
+It can produce feature not supported warning like

docs/system/i386/tdx.rst line 98

Quoted 10 lines
docs/system/i386/tdx.rst new file +10 -0
+
+  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
+
+It can also produce warning like
+
+  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
+
+if the fixed-1 feature is requested to be disabled explicitly. This is newly
+added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
+by TDX module and VMM cannot disable them.

This is where I'm wondering if libvirt has anything to be concerned about. Possibly when libvirt queries the actual CPUID after launching the guest it will just "do the right thing" ? Wondering if there's any need for libvirt to be aware of CPUID restrictions before that point though ?

docs/system/i386/tdx.rst line 111

Quoted 13 lines
docs/system/i386/tdx.rst new file +13 -0
+
+Launching a TD (TDX VM)
+-----------------------
+
+To launch a TD, the necessary command line options are tdx-guest object and
+split kernel-irqchip, as below:
+
+.. parsed-literal::
+
+    |qemu_system_x86| \\
+        -object tdx-guest,id=tdx0 \\
+        -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
+        -bios OVMF.fd \\

I don't think we need to show 'kernel-irqchip=split' now that we "do the right thing" by default

This surely also ought to include '-accel kvm', as IIUC there's no TCG support for TDX.

And presumably '-cpu host', since QEMU's default 'qemu64' CPU model is likely a terrible match for what TDX will force set.

docs/system/i386/tdx.rst line 116

Quoted 5 lines
docs/system/i386/tdx.rst new file +5 -0
+
+Restrictions
+------------
+
+ - kernel-irqchip must be split;

Can append

Quoted 2 lines

"This is set by default for TDX guests if kernel-irqchip is left on
its default 'auto' setting."

This and above feedback are all good. Accept them all and will update in the next version.

Thanks!

Quoted 10 lines

docs/system/i386/tdx.rst line 122

Quoted 6 lines
docs/system/i386/tdx.rst new file +6 -0
+
+ - No readonly support for private memory;
+
+ - No SMM support: SMM support requires manipulating the guest register states
+   which is not allowed;
+

With regards,
Daniel