Skip to contentSkip to conversation

Enable DICE-based TDX Quoting Extension

PATCH In review Xu Yilun 11 participants 111 messages Jun 18 – Aug 11

Subsystems (5)
Documentation x86 KVM KVM x86 +1 more subsystems
lore
Share & tools
lore

Xu Yilun

author
#0 lore raw

v2 cover Enable DICE-based TDX Quoting Extension

This series adds infrastructure to enable TDX module extensions and then implements DICE-based TDX Quoting extension. This is the 2nd version and a significant change is that we want the quoting part to merge along with the basic TDX module extensions part, rather than serving as an example. So the quoting part drops RFC tags and requires initial review. The basic extensions part addresses v1 comments and needs more detailed review.

The quoting part contains some KVM patches, so we sorted the series for easier review and pick:

  Patches  1-6:  Enable the TDX module extensions support
  Patches  7-14: DICE-based TDX Quoting, x86/tdx part
  Patches  15-N: DICE-based TDX Quoting, KVM part

== Overview ==

To date, SEAMCALLs have been short lived routines that monopolize the CPU for their duration. This limits their utility for implementing higher order security protocols, or pushes complexity into Linux - such as by fragmenting a protocol setup service into several SEAMCALLs. The Linux appetite for ingesting complexity is low, so TDX now adds a new class of SEAMCALLs that are preemptible and resumable. This capability allows for higher-level API constructions - like "create a DICE-based quote" - which are more aligned to what is a good fit for Linux.

This new "extension SEAMCALL" capability is akin to ARM CCA's "Stateful RMI Operations (SRO)", and achieves similar externalized complexity relief as a dedicated hardware co-processor like AMD SEV-SNP. The mechanism is "give the service environment some memory", "invoke the service API", and "continue invoking until complete". All protocol state is internal to the service API.

TDX introduces "TDX module extensions" as the service environment for some add-on features - such as DICE-based quoting, TDISP, and live migration - to use "extension SEAMCALLs".

The extension SEAMCALLs are designed to be transparent to the host, using the same interface as normal SEAMCALLs, but the service environment should be initialized in several steps. First, configure/select (via TDH.SYS.CONFIG) add-on features during basic TDX initialization. Second, check if TDX module extensions are required to support these add-on features by reading TDX global metadata. Third, add extra memory to the TDX module via a SEAMCALL (TDH.EXT.MEM.ADD). Finally, use another SEAMCALL (TDH.EXT.INIT) to initialize the extensions.

== DICE-based Quoting extension ==

The first feature to use these extensions is the TDX Quoting extension [1], which converts guest launch attestation reports into a document that can be verified externally.

Today, the TDX host requires a separate software service to generate Quotes. The Quoting extension allows the TDX module to generate Quotes directly, without relying on a discrete Quoting engine. This simplifies the overall attestation flow: KVM no longer needs to return to userspace for Quote generation. Instead, Quote generation is handled directly by the TDX module through an extension SEAMCALL. See [2] for an overview of TDX attestation.

The Device Identifier Composition Engine ("DICE") provides a standardized framework for layering attestation evidence. It replaces SGX-based attestation and moves away from Intel-proprietary formats. It also eliminates the SGX requirement to contact an Intel service to obtain a certificate first. Instead, all attestation evidence is embedded in the Quote itself.

== The trade-off ==

The extensions create an extension instance for each feature that requires extension SEAMCALLs. More memory is consumed when more extension instances are created. There are 3 extensions (quoting, TDISP, Migration) in the foreseeable future. Turning on them all will require tens of megabytes. Note that the host can never reclaim the memory added to the extensions.

According to the TDX module design, basic TDX functionalities can run without the extensions. So theoretically the extensions don't need to be enabled at basic TDX initialization time. They could be lazily enabled right before the first extension SEAMCALL is issued.

However, Linux applies a simple policy for TDX: turn on all the features that Linux knows about all the time, unless and until any evidence makes this approach untenable. Enabling the extensions along with the basic TDX at boot time aligns with the policy, and offers several good reasons:

1. Simplify TDX state management, avoid runtime state transitions that could introduce race conditions or unexpected failure modes.

2. The kernel doesn't have to keep track of which SEAMCALLs need the extensions, as there is no HW/FW enumeration for this.

3. When no extension is configured, the extensions initialization is virtually skipped. So no impact on existing kernels.

4. A small trade-off is that eager initialization allocates memory (tens of megabytes) at boot time before any feature starts to work. However, these features provide critical security capabilities in confidential computing. They are expected to be enabled eventually when available. So this merely advances the timing of memory allocation.

== Restore the extensions after runtime TDX module update ==

Runtime TDX module update introduces a mechanism to update the module firmware while preserving and restoring TDX operations. As part of the restoration process, TDX module extensions must also be re-initialized to re-enable extension SEAMCALLs.

Similar to TDH.SYS.CONFIG, TDX module extends TDH.SYS.UPDATE with more parameters for the host to re-enable desired add-on features. Then host must re-execute all extensions initialization steps to restore extension SEAMCALL functionality.

However, Linux runs the update in stop_machine() context, which prevents memory allocation. This introduces a hard restriction that the updated TDX environment must not consume more memory for the extensions.

Fortunately, Linux applies another policy that no newer features should be added during runtime update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must enable the same features as the TDH.SYS.CONFIG. This policy mitigates the memory allocation problem a lot by minimizing the chance of increased memory demand. So now the restriction only affects the compatibility rule for choosing the update image.

The same memory constraint applies to the Quoting extension. A compatible runtime update must not increase the size limit of its Quotes, because the buffer used for Quote generation is allocated during TDX bringup. Otherwise, attestation could fail after the update if the TDX module requires a larger buffer for Quotes.

== Some history ==

The TDX module extensions support part was first posted along with TDX TDISP [3]. But quoting is the simplest consumer and is chosen as the lead vehicle over TDISP.

== Misc ==

This series is based on tip/x86/tdx [4], because we need the extensions play nice with runtime TDX module update.

== Changelog ==

v2:
- Support runtime TDX module update
- Refine quoting patches, drop RFC tag
- Change the patch order. (Xiaoyao & Tony)
- Fold metadata readings changes into patches that use them.
- Read the extensions metadata at init_tdx_ext() (Rick & Xiaoyao)
- Don't do get_tdx_sys_info() a 2nd time after TDH.SYS.CONFIG (Rick & Xiaoyao)
- Delete tdx_clflush_hpa_list() (Rick)
- s/TDX Module/TDX module (Sohil)
- s/Extensions/extensions (Dave)
- Change the data type of ext_required to bool (Rick)
- Change the data type of memory_pool_required_pages from u16 to u32, the Module team see this problem and promise the change (Sohil)
- s/init_tdx_ext()/init_tdx_module_extensions() to disambiguate from tdx_ext_init() (Kishen)
- Cover-letter & change log re-phrase (All reviewers)

v1: https://lore.kernel.org/all/[email protected]/

Peter Fang (11):
x86/virt/tdx: Initialize Quoting extension
x86/virt/tdx: Prepare Quote buffer during extension bringup x86/virt/tdx: Add interface to check Quoting availability x86/virt/tdx: Move tdx_tdr_pa() up in the file x86/virt/tdx: Add interface to generate a Quote x86/virt/tdx: Reinitialize the Quoting extension after TDX module
update
x86/virt/tdx: Enable Quoting extension
x86/tdx: Move and rename Quote request structure
KVM: TDX: Factor out userspace return path from tdx_get_quote()
KVM: TDX: Add in-kernel Quote generation
KVM: TDX: Support event-notify interrupts only with userspace Quoting

Xu Yilun (6):
x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions x86/virt/tdx: Configure add-on features on TDX module init and update x86/virt/tdx: Detect if the extensions initialization is required x86/virt/tdx: Add extra memory to TDX module for the extensions x86/virt/tdx: Make TDX module initialize the extensions x86/virt/tdx: Re-initialize the extensions on runtime TDX module
update

 Documentation/arch/x86/tdx.rst              |  19 +-
 Documentation/virt/kvm/api.rst              |   3 +
 arch/x86/include/asm/tdx.h                  |  35 ++
 arch/x86/include/asm/tdx_global_metadata.h  |   9 +
 arch/x86/kvm/vmx/tdx.h                      |   6 +
 arch/x86/virt/vmx/tdx/tdx.h                 |  33 +-
 arch/x86/kvm/vmx/tdx.c                      | 176 +++++++-
 arch/x86/virt/vmx/tdx/tdx.c                 | 465 +++++++++++++++++++-
 arch/x86/virt/vmx/tdx/tdx_global_metadata.c |  34 ++
 drivers/virt/coco/tdx-guest/tdx-guest.c     |  47 +-
 virt/kvm/kvm_main.c                         |   1 +
 11 files changed, 755 insertions(+), 73 deletions(-)

base-commit: 2b9ad7a6154e0938b9458691536296dd0224942d

Signature · 1 line
2.25.1
author
#1 lore raw

PATCH v2 1/17 x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions

Embed version information in SEAMCALL leaf function definitions rather than let the caller open code them. For now, only TDH.VP.INIT is involved.

Don't bother the caller to choose the SEAMCALL version if unnecessary. New version SEAMCALLs are guaranteed to be backward compatible, so ideally the kernel doesn't need to keep version history and only uses the latest version SEAMCALLs.

And in confidential computing world, system security requires us to stop using an older TDX module when there is a newer one. So don't burden the kernel with long-term supporting an older TDX module that doesn't understand newer version SEAMCALLs.

The only concern is there may be transitional periods when a new TDX module is not widely available, meaning the kernel may temporarily need to support multiple SEAMCALL versions. As time goes by, the old TDX modules deprecate and old version SEAMCALL definitions should disappear.

The old TDX modules that only support TDH.VP.INIT v0 are all deprecated, so only provide the latest (v1) definition.

---
 arch/x86/virt/vmx/tdx/tdx.h | 23 ++++++++++++++---------
 arch/x86/virt/vmx/tdx/tdx.c |  3 +--
 2 files changed, 15 insertions(+), 11 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +14 -9
index bdfd0e1e337a..fbb520704662 100644
@@ -2,6 +2,7 @@
 #ifndef _X86_VIRT_TDX_H
 #define _X86_VIRT_TDX_H
 
+#include <linux/bitfield.h>
 #include <linux/bits.h>
 
 /*
@@ -11,6 +12,18 @@
  * architectural definitions come first.
  */
 
+/*
+ * SEAMCALL leaf:
+ *
+ * Bit 15:0	Leaf number
+ * Bit 23:16	Version number
+ */
+#define SEAMCALL_LEAF			GENMASK(15, 0)
+#define SEAMCALL_VER			GENMASK(23, 16)
+
+#define SEAMCALL_LEAF_VER(l, v)		(FIELD_PREP(SEAMCALL_LEAF, l) | \
+					 FIELD_PREP(SEAMCALL_VER, v))
+
 /*
  * TDX module SEAMCALL leaf functions
  */
@@ -31,7 +44,7 @@
 #define TDH_VP_CREATE			10
 #define TDH_MNG_KEY_FREEID		20
 #define TDH_MNG_INIT			21
-#define TDH_VP_INIT			22
+#define TDH_VP_INIT			SEAMCALL_LEAF_VER(22, 1)
 #define TDH_PHYMEM_PAGE_RDMD		24
 #define TDH_VP_RD			26
 #define TDH_PHYMEM_PAGE_RECLAIM		28
@@ -50,14 +63,6 @@
 #define TDH_SYS_UPDATE			53
 #define TDH_SYS_DISABLE			69
 
-/*
- * SEAMCALL leaf:
- *
- * Bit 15:0	Leaf number
- * Bit 23:16	Version number
- */
-#define TDX_VERSION_SHIFT		16
-
 /* TDX page types */
 #define	PT_NDA		0x0
 #define	PT_RSVD		0x1
arch/x86/virt/vmx/tdx/tdx.c +1 -2
index b15269b5941d..2a03152796e6 100644
@@ -1903,8 +1903,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
 		.r8 = x2apicid,
 	};
 
-	/* apicid requires version == 1. */
-	return seamcall(TDH_VP_INIT | (1ULL << TDX_VERSION_SHIFT), &args);
+	return seamcall(TDH_VP_INIT, &args);
 }
 EXPORT_SYMBOL_FOR_KVM(tdh_vp_init);
 
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH v2 01/17] x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions

Quoted 3 lines · Xu Yilun

Embed version information in SEAMCALL leaf function definitions rather than let the caller open code them. For now, only TDH.VP.INIT is involved.

This is jumping the gun a bit in the changelog.

What is a SEAMCALL leaf function?

How does the version fit in?

Quoted 1 line

Don't bother the caller to choose the SEAMCALL version if unnecessary.

I think I see what you are trying to say here but it's more than that.

The question is whether there should be a base seamcall() API that takes an explicit version or whether the version should be passed in by callers.

One wrinkle is that the naming of all of these things is around "function", "func" and "fn":

u64 __seamcall(u64 fn, struct tdx_module_args *args);

A "function" is TDH.SYS.INIT or TDH.SYS.INFO, not 'TDH.SYS.INFO v123'.

But the low-level calls could be:

	u64 __seamcall(u64 fn, u64 version, ...);

or

	u64 __seamcall(u64 fn, ...);

Where 'fn' encodes the function *and* version.

Quoted 2 lines

The old TDX modules that only support TDH.VP.INIT v0 are all deprecated, so only provide the latest (v1) definition.

No, this isn't how this is going to work.

What do we *NEED* from v1? Why churn the code if we don't *NEED* something from v1 and can live with v0? It has *ZERO* to do with the TDX module being deprecated or whatever.

Linux stays on the old interface until we need a new interface. We are *not* going to bump version numbers just because.

Quoted 13 lines

/*
* TDX module SEAMCALL leaf functions
*/

arch/x86/virt/vmx/tdx/tdx.h +1 -1
@@ -31,7 +44,7 @@
 #define TDH_VP_CREATE			10
 #define TDH_MNG_KEY_FREEID		20
 #define TDH_MNG_INIT			21
-#define TDH_VP_INIT			22
+#define TDH_VP_INIT			SEAMCALL_LEAF_VER(22, 1)
 #define TDH_PHYMEM_PAGE_RDMD		24
 #define TDH_VP_RD			26
 #define TDH_PHYMEM_PAGE_RECLAIM		28

arch/x86/virt/vmx/tdx/tdx.h line 64

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.h +0 -0
@@ -50,14 +63,6 @@
 #define TDH_SYS_UPDATE			53
 #define TDH_SYS_DISABLE			69

That is unreadable and patterns can't be seen. This is better:

#define TDH_MNG_INIT SEAMCALL_LEAF_VER(21, 0)
#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1)
#define TDH_PHYMEM_PAGE_RDMD SEAMCALL_LEAF_VER(24, 0)

arch/x86/virt/vmx/tdx/tdx.c u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) · line 1908

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +1 -2
@@ -1903,8 +1903,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
 		.r8 = x2apicid,
 	};
 
-	/* apicid requires version == 1. */
-	return seamcall(TDH_VP_INIT | (1ULL << TDX_VERSION_SHIFT), &args);
+	return seamcall(TDH_VP_INIT, &args);
 }
 EXPORT_SYMBOL_FOR_KVM(tdh_vp_init);

But that whole scheme falls apart the first time the kernel needs functionality from v2. You'll need:

#define TDH_VP_INIT_V0 SEAMCALL_LEAF_VER(22, 0)
#define TDH_VP_INIT_V1 SEAMCALL_LEAF_VER(22, 1)

and then the calls will do:

	if (foo)
		return seamcall(TDH_VP_INIT_V0, &args);
	else
		return seamcall(TDH_VP_INIT_V1, &args);

So this 100% goes down the road of needing #defines for *EACH* version. That's the real implication here and the real choice.

That said, I don't particularly like:

	if (foo)
		return seamcall(TDH_VP_INIT, 0, &args);
	else
		return seamcall(TDH_VP_INIT, 1, &args);

all that much either because of the seemingly magic numbers.

The whole seamcall RAX thing is one step too clever. I think Linux did the right thing:

5 common open sys_open
288 common openat sys_openat
437 common openat2 sys_openat2

New ABI gets a new base number. No need for the other end of the ABI to know that 288 is arguably a later version of 5.

Ugh. But why is this patch even in here in the first place? Why is this even *ASSOCIATED* with DICE-based attestation? Isn't this completely orthogonal?

author
#3 lore raw

Re: [PATCH v2 01/17] x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions

Quoted 5 lines

This is jumping the gun a bit in the changelog.

What is a SEAMCALL leaf function?

How does the version fit in?

I think the word "leaf function" is confusing, maybe I should say "... SEAMCALL interface function definitions..."?

And I missed some more context explanation here:

SEAMCALL accepts parameters via CPU registers (RAX, RCX, RDX, ...), where RAX selects the specific TDX module function to execute. According to the TDX module SPEC, RAX is mainly composed of:

1. leaf number: selects the function.
2. version number: selects the function version. The newer version defines more operations. It also defines additional parameters for the rest of registers.

Newer version SEAMCALL uses the same leaf number as older versions, with only the version number increased. Newer version SEAMCALLs are guaranteed to be backward compatible.

[...]

Quoted 7 lines
Quoted 2 lines

The old TDX modules that only support TDH.VP.INIT v0 are all deprecated, so only provide the latest (v1) definition.

No, this isn't how this is going to work.

What do we *NEED* from v1? Why churn the code if we don't *NEED* something from v1 and can live with v0?

Sorry for the confusing words, for this TDH.VP.INIT, kernel is now using v1, we need v1 for virtual x2APIC. Maybe I should add "No functional change intended."

Quoted 1 line

It has *ZERO* to do with the TDX module being deprecated or whatever.

Since newer version SEAMCALLs are always backward compatible. TDH.VP.INIT v0 is only needed when we need to support legacy modules that don't understand TDH.VP.INIT v1. We don't support such legacy modules, so I meant to eliminate TDH.VP.INIT v0 definition and reduce histroy burdens.

We have some previous discussion that leads to "kernel don't keep SEAMCALL version at all":

https://lore.kernel.org/all/[email protected]/

But I have trouble for not keeping versions for TDH.SYS.CONFIG/UPDATE (in next patch). No public module supports TDH.SYS.CONFIG/UPDATE v1 so I can't eliminate TDH.SYS.CONFIG/UPDATE v0. I'm GOOD we are now considering keep aware of SEAMCALL versions at some level. Keeping in mind which module is published is another burden...

[...]

Quoted 6 lines

But that whole scheme falls apart the first time the kernel needs functionality from v2. You'll need:

#define TDH_VP_INIT_V0 SEAMCALL_LEAF_VER(22, 0)
#define TDH_VP_INIT_V1 SEAMCALL_LEAF_VER(22, 1)

Quoted 8 lines

and then the calls will do:

	if (foo)
		return seamcall(TDH_VP_INIT_V0, &args);
	else
		return seamcall(TDH_VP_INIT_V1, &args);

So this 100% goes down the road of needing #defines for *EACH* version.

TDH_VP_INIT_V0 has no use case in current code, so maybe not #define for EACH version.

And we have 100+ SEAMCALLs in SPEC, 30+ is being used in Linux, 20+ is comming, only 3 need versions now. I think making versioned definitions for EACH of them is overkill.

I see you have another suggestion in next patch which avoids the #defines. Let me continue in that patch.

Quoted 9 lines

The whole seamcall RAX thing is one step too clever. I think Linux did the right thing:

5 common open sys_open
288 common openat sys_openat
437 common openat2 sys_openat2

New ABI gets a new base number. No need for the other end of the ABI to know that 288 is arguably a later version of 5.

Yes, that's true.

Quoted 4 lines

Ugh. But why is this patch even in here in the first place? Why is this even *ASSOCIATED* with DICE-based attestation? Isn't this completely orthogonal?

TDH.SYS.UPDATE/CONFIG is the second user of the version stuff. So as required, fix the open code issue of the first existing user.

https://lore.kernel.org/all/[email protected]/

Thanks,
Yilun

author
#4 lore raw

PATCH v2 2/17 x86/virt/tdx: Configure add-on features on TDX module init and update

In reply to Xu Yilun

In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them. The kernel should explicitly configure these features at boot or post-update initialization time. Configuring an add-on feature, such as TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions. TDX Quoting is the target feature to enable but defer it for now until full kernel support is in place.

TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as TDX_FEATURES0.

For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1. So there is little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.

---
 arch/x86/virt/vmx/tdx/tdx.h |  6 ++++--
 arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++--
 2 files changed, 30 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +4 -2
index fbb520704662..a47e872480c7 100644
@@ -58,9 +58,11 @@
 #define TDH_PHYMEM_CACHE_WB		40
 #define TDH_PHYMEM_PAGE_WBINVD		41
 #define TDH_VP_WR			43
-#define TDH_SYS_CONFIG			45
+#define TDH_SYS_CONFIG_V0		45
+#define TDH_SYS_CONFIG			SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)
 #define TDH_SYS_SHUTDOWN		52
-#define TDH_SYS_UPDATE			53
+#define TDH_SYS_UPDATE_V0		53
+#define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
 #define TDH_SYS_DISABLE			69
 
 /* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +26 -2
index 2a03152796e6..92305b5ea90d 100644
@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;
 static u32 tdx_global_keyid __ro_after_init;
 static u32 tdx_guest_keyid_start __ro_after_init;
 static u32 tdx_nr_guest_keyids __ro_after_init;
+static u64 tdx_addon_feature0 __ro_after_init;
 
 static DEFINE_IDA(tdx_guest_keyid_pool);
 
@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
 	return ret;
 }
 
+static __init void set_tdx_addon_features(void)
+{
+	/*
+	 * To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when
+	 * kernel is ready.
+	 */
+}
+
 static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
 				    u64 global_keyid)
 {
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;
 	struct tdx_module_args args = {};
 	u64 *tdmr_pa_array;
 	size_t array_sz;
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
 	args.rcx = __pa(tdmr_pa_array);
 	args.rdx = tdmr_list->nr_consumed_tdmrs;
 	args.r8 = global_keyid;
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
+	set_tdx_addon_features();
+
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_CONFIG;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);
 
 	/* Free the array as it is not required anymore. */
 	kfree(tdmr_pa_array);
@@ -1314,10 +1332,16 @@ int tdx_module_shutdown(void)
 
 int tdx_module_run_update(void)
 {
+	u64 seamcall_fn = TDH_SYS_UPDATE_V0;
 	struct tdx_module_args args = {};
 	int ret;
 
-	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_UPDATE;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);
 	if (ret)
 		return ret;
 
Signature · 1 line
2.25.1
#5 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

arch/x86/virt/vmx/tdx/tdx.c int tdx_module_shutdown(void) · line 1342

Quoted 11 lines · Xu Yilun

int tdx_module_run_update(void)
{

arch/x86/virt/vmx/tdx/tdx.c +5 -1
+	u64 seamcall_fn = TDH_SYS_UPDATE_V0;
 	struct tdx_module_args args = {};
 	int ret;
 
-	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_UPDATE;
+	}

Heh, and it falls apart into craziness immediately. See how it immediately loses the logical information that there's a version 1 and a version 0? The "1" isn't even visible. It's hidden in "TDH_SYS_UPDATE".

Isn't this a million times more sane?

	struct tdx_module_args args = {};
	u64 version;

int ret;

	if (tdx_addon_feature0) {
		args.r9 = tdx_addon_feature0;
		version = 1;
	} else {
		version = 0;
	}

	ret = seamcall_prerr(TDH_SYS_UPDATE, version, &args);

There's also zero stopping us from putting version in args:

	struct tdx_module_args args = {};

int ret;

	if (tdx_addon_feature0) {
		args.r9 = tdx_addon_feature0;
		args.version = 1;
	}

	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);

Eh?

That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.

But this is *exactly* the kind of thing that shouldn't be a part of an attestation patch series. This could very much have been a separate discussion and happened a month or a year ago. But now it is blocking this DICE thing from getting done <grumble>.

author
#6 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

Quoted 5 lines

There's also zero stopping us from putting version in args:

	struct tdx_module_args args = {};

int ret;

Quoted 8 lines
	if (tdx_addon_feature0) {
		args.r9 = tdx_addon_feature0;
		args.version = 1;
	}

	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);

Eh?

I'm thinking the version is only needed for 3 SEAMCALLs. We don't have to make version common for all 100+ SEAMCALLs. Besides the layout of "struct tdx_module_args" correlates the fundamental assembly code of __seamcall() in tdcall.S.

Could we make dedicated SEAMCALL wrappers for TDH_SYS_UPDATE similar to other SEAMCALLs and wrapper the specific version handling there? I put the diff in the end.

Quoted 9 lines

That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.

But this is *exactly* the kind of thing that shouldn't be a part of an attestation patch series. This could very much have been a separate discussion and happened a month or a year ago. But now it is blocking this DICE thing from getting done <grumble>.

Sorry, I was thinking "don't keep version" was the conclusion...

--------8<--------

arch/x86/virt/vmx/tdx/tdx.c +18 -11
index 01fb01313077..b3b3540e431a 100644
@@ -1757,18 +1757,12 @@ int tdx_module_shutdown(void)
 
 int tdx_module_run_update(void)
 {
-       u64 seamcall_fn = TDH_SYS_UPDATE_V0;
-       struct tdx_module_args args = {};
+       u64 err;
        int ret;
 
-       if (tdx_addon_feature0) {
-               args.r9 = tdx_addon_feature0;
-               seamcall_fn = TDH_SYS_UPDATE;
-       }
-
-       ret = seamcall_prerr(seamcall_fn, &args);
-       if (ret)
-               return ret;
+       err = tdx_sys_update(tdx_addon_feature0);
+       if (err)
+               return -EIO;
 
        ret = get_tdx_sys_info_version(&tdx_sysinfo.version);
        /*
@@ -2351,7 +2345,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
                .r8 = x2apicid,
        };
 
-       return seamcall(TDH_VP_INIT, &args);
+       return seamcall(SEAMCALL_LEAF_VER(TDH_VP_INIT, 1), &args);
 }
 EXPORT_SYMBOL_FOR_KVM(tdh_vp_init);
 
@@ -2463,3 +2457,16 @@ void tdx_sys_disable(void)
        if (ret && (ret & TDX_SW_ERROR) != TDX_SW_ERROR)
                pr_err("TDH.SYS.DISABLE failed: 0x%016llx\n", ret);
 }
+
+u64 tdx_sys_update(u64 features_enable0)
+{
+       struct tdx_module_args args = {
+               .r9 = features_enable0,
+       };
+       u64 fn = TDH_SYS_UPDATE;
+
+       if (features_enable0)
+               fn = SEAMCALL_LEAF_VER(TDH_SYS_UPDATE, 1);
+
+       return seamcall(fn, &args);
+}
arch/x86/virt/vmx/tdx/tdx.h +2 -3
index 32b13b0c85f9..f07e12552bf9 100644
@@ -44,7 +44,7 @@
 #define TDH_VP_CREATE                  10
 #define TDH_MNG_KEY_FREEID             20
 #define TDH_MNG_INIT                   21
-#define TDH_VP_INIT                    SEAMCALL_LEAF_VER(22, 1)
+#define TDH_VP_INIT                    22
 #define TDH_PHYMEM_PAGE_RDMD           24
 #define TDH_VP_RD                      26
 #define TDH_PHYMEM_PAGE_RECLAIM                28
@@ -61,8 +61,7 @@
 #define TDH_SYS_SHUTDOWN               52
-#define TDH_SYS_UPDATE_V0              53
-#define TDH_SYS_UPDATE                 SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
+#define TDH_SYS_UPDATE                 53
 #define TDH_EXT_INIT                   60
 #define TDH_EXT_MEM_ADD                        61
 #define TDH_SYS_DISABLE                        69
 
author
#7 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

In reply to Dave Hansen
Quoted 12 lines

There's also zero stopping us from putting version in args:

	struct tdx_module_args args = {};

int ret;

	if (tdx_addon_feature0) {
		args.r9 = tdx_addon_feature0;
		args.version = 1;
	}

	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
Quoted 5 lines

Eh?

That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.

Ah, on 2nd reading, I'm pretty sure now I understand your logical argument in patch 1 and 2. It's good to me. I append my diff at the end.

Quoted 5 lines

But this is *exactly* the kind of thing that shouldn't be a part of an attestation patch series. This could very much have been a separate discussion and happened a month or a year ago. But now it is blocking this DICE thing from getting done <grumble>.

Sorry, I should have been more active in searching for the solution rather than sticking to "kernel never keeps versions", when I've found the problem that public modules are not available.

----8<----

arch/x86/include/asm/shared/tdx.h +2 -0
index f20e91d7ac35..972880910a5e 100644
@@ -143,6 +143,8 @@ struct tdx_module_args {
        u64 rbx;
        u64 rdi;
        u64 rsi;
+       /* for RAX encoding */
+       u8  version;
 };
 
 /* Used to communicate with the TDX module */
arch/x86/kernel/asm-offsets.c +1 -0
index 081816888f7a..b3c00ff4d819 100644
@@ -95,6 +95,7 @@ static void __used common(void)
        OFFSET(TDX_MODULE_rbx, tdx_module_args, rbx);
        OFFSET(TDX_MODULE_rdi, tdx_module_args, rdi);
        OFFSET(TDX_MODULE_rsi, tdx_module_args, rsi);
+       OFFSET(TDX_MODULE_version, tdx_module_args, version);
 
        BLANK();
        OFFSET(BP_scratch, boot_params, scratch);
arch/x86/virt/vmx/tdx/tdxcall.S +8 -0
index 016a2a1ec1d6..d1d3d40c5614 100644
@@ -48,6 +48,14 @@
        /* Move Leaf ID to RAX */
        mov %rdi, %rax
 
+       /*
+        * Extract the version from 'struct tdx_module_args', append it to
+        * RAX[23:16]
+        */
+       movzbl  TDX_MODULE_version(%rsi), %ecx
+       shll    $16, %ecx
+       orq     %rcx, %rax
+
        /* Move other input regs from 'struct tdx_module_args' */
        movq    TDX_MODULE_rcx(%rsi), %rcx
        movq    TDX_MODULE_rdx(%rsi), %rdx
arch/x86/virt/vmx/tdx/tdx.c +7 -8
index a6f8fd0a3df0..bc3aa1f78fc8 100644
@@ -1036,7 +1036,6 @@ static __init void set_tdx_addon_features(void)
 static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
                                    u64 global_keyid)
 {
-       u64 seamcall_fn = TDH_SYS_CONFIG_V0;
        struct tdx_module_args args = {};
        u64 *tdmr_pa_array;
        size_t array_sz;
@@ -1059,18 +1058,18 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
        for (i = 0; i < tdmr_list->nr_consumed_tdmrs; i++)
                tdmr_pa_array[i] = __pa(tdmr_entry(tdmr_list, i));
 
+       set_tdx_addon_features();
+
        args.rcx = __pa(tdmr_pa_array);
        args.rdx = tdmr_list->nr_consumed_tdmrs;
        args.r8 = global_keyid;
 
-       set_tdx_addon_features();
-
        if (tdx_addon_feature0) {
                args.r9 = tdx_addon_feature0;
-               seamcall_fn = TDH_SYS_CONFIG;
+               args.version = 1;
        }
 
-       ret = seamcall_prerr(seamcall_fn, &args);
+       ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
 
        /* Free the array as it is not required anymore. */
        kfree(tdmr_pa_array);
@@ -1761,16 +1760,15 @@ int tdx_module_shutdown(void)
 
 int tdx_module_run_update(void)
 {
-       u64 seamcall_fn = TDH_SYS_UPDATE_V0;
        struct tdx_module_args args = {};
        int ret;
 
        if (tdx_addon_feature0) {
                args.r9 = tdx_addon_feature0;
-               seamcall_fn = TDH_SYS_UPDATE;
+               args.version = 1;
        }
 
-       ret = seamcall_prerr(seamcall_fn, &args);
+       ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
        if (ret)
                return ret;
 
@@ -2353,6 +2351,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
                .rcx = vp->tdvpr_pa,
                .rdx = initial_rcx,
                .r8 = x2apicid,
+               .version = 1,
        };
 
        return seamcall(TDH_VP_INIT, &args);
arch/x86/virt/vmx/tdx/tdx.h +3 -5
index 32b13b0c85f9..018988c25caa 100644
@@ -44,7 +44,7 @@
 #define TDH_VP_CREATE                  10
 #define TDH_MNG_KEY_FREEID             20
 #define TDH_MNG_INIT                   21
-#define TDH_VP_INIT                    SEAMCALL_LEAF_VER(22, 1)
+#define TDH_VP_INIT                    22
 #define TDH_PHYMEM_PAGE_RDMD           24
 #define TDH_VP_RD                      26
 #define TDH_PHYMEM_PAGE_RECLAIM                28
@@ -58,11 +58,9 @@
 #define TDH_PHYMEM_CACHE_WB            40
 #define TDH_PHYMEM_PAGE_WBINVD         41
 #define TDH_VP_WR                      43
-#define TDH_SYS_CONFIG_V0              45
-#define TDH_SYS_CONFIG                 SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)
+#define TDH_SYS_CONFIG                 45
 #define TDH_SYS_SHUTDOWN               52
-#define TDH_SYS_UPDATE_V0              53
-#define TDH_SYS_UPDATE                 SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
+#define TDH_SYS_UPDATE                 53
 #define TDH_EXT_INIT                   60
 #define TDH_EXT_MEM_ADD                        61
 #define TDH_SYS_DISABLE                        69
#8 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

Quoted 21 lines · Xu Yilun
Quoted 12 lines

There's also zero stopping us from putting version in args:

	struct tdx_module_args args = {};

int ret;

	if (tdx_addon_feature0) {
		args.r9 = tdx_addon_feature0;
		args.version = 1;
	}

	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
Quoted 5 lines

Eh?

That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.

Ah, on 2nd reading, I'm pretty sure now I understand your logical argument in patch 1 and 2. It's good to me. I append my diff at the end.

[ ... ]

Quoted 9 lines
arch/x86/virt/vmx/tdx/tdxcall.S +1 -0
index 016a2a1ec1d6..d1d3d40c5614 100644
@@ -48,6 +48,14 @@
        /* Move Leaf ID to RAX */
        mov %rdi, %rax
 
+       /*

arch/x86/virt/vmx/tdx/tdxcall.S line 61

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdxcall.S +7 -0
+        * Extract the version from 'struct tdx_module_args', append it to
+        * RAX[23:16]
+        */
+       movzbl  TDX_MODULE_version(%rsi), %ecx
+       shll    $16, %ecx
+       orq     %rcx, %rax
+
        /* Move other input regs from 'struct tdx_module_args' */
        movq    TDX_MODULE_rcx(%rsi), %rcx
        movq    TDX_MODULE_rdx(%rsi), %rdx

This approach looks much cleaner to me. Would it be better to have a small C helper to encode the final RAX value instead of operating on RAX directly in asm? Looking at the May 2026 edition of the ABI spec, SEAMCALL RAX encoding is starting to get quite complex. Just thinking about this from a readability standpoint.

author
#9 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

Quoted 42 lines · Peter Fang
Quoted 21 lines · Xu Yilun
Quoted 12 lines

There's also zero stopping us from putting version in args:

	struct tdx_module_args args = {};

int ret;

	if (tdx_addon_feature0) {
		args.r9 = tdx_addon_feature0;
		args.version = 1;
	}

	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
Quoted 5 lines

Eh?

That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.

Ah, on 2nd reading, I'm pretty sure now I understand your logical argument in patch 1 and 2. It's good to me. I append my diff at the end.

[ ... ]

Quoted 17 lines
arch/x86/virt/vmx/tdx/tdxcall.S +8 -0
index 016a2a1ec1d6..d1d3d40c5614 100644
@@ -48,6 +48,14 @@
        /* Move Leaf ID to RAX */
        mov %rdi, %rax
 
+       /*
+        * Extract the version from 'struct tdx_module_args', append it to
+        * RAX[23:16]
+        */
+       movzbl  TDX_MODULE_version(%rsi), %ecx
+       shll    $16, %ecx
+       orq     %rcx, %rax
+
        /* Move other input regs from 'struct tdx_module_args' */
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdxcall.S line 61

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdxcall.S +0 -0
        movq    TDX_MODULE_rcx(%rsi), %rcx
        movq    TDX_MODULE_rdx(%rsi), %rdx

This approach looks much cleaner to me. Would it be better to have a small C helper to encode the final RAX value instead of operating on RAX directly in asm? Looking at the May 2026 edition of the ABI spec, SEAMCALL RAX encoding is starting to get quite complex. Just thinking about this from a readability standpoint.

I'm also good to it. I made some diff for your proposal, Some additional effort here is to update some comments and parameter names, to reflect the differences between "function/func/fn" (the unversioned number) and the final composite "fn_code" for RAX.

-----8<-------

arch/x86/include/asm/shared/tdx.h +2 -0
index f20e91d7ac35..c26eca18fded 100644
@@ -143,6 +143,8 @@ struct tdx_module_args {
 	u64 rbx;
 	u64 rdi;
 	u64 rsi;
+	/* for leaf encoding */
+	u8  version;
 };
 
 /* Used to communicate with the TDX module */
arch/x86/virt/vmx/tdx/seamcall.S +6 -6
index 6854c52c374b..5cf3993e98f4 100644
@@ -10,8 +10,8 @@
  *
  * __seamcall() function ABI:
  *
- * @fn   (RDI)  - SEAMCALL Leaf number, moved to RAX
- * @args (RSI)  - struct tdx_module_args for input
+ * @fn_code (RDI)  - SEAMCALL composite leaf code, moved to RAX
+ * @args    (RSI)  - struct tdx_module_args for input
  *
  * Only RCX/RDX/R8-R11 are used as input registers.
  *
@@ -29,8 +29,8 @@ SYM_FUNC_END(__seamcall)
  *
  * __seamcall_ret() function ABI:
  *
- * @fn   (RDI)  - SEAMCALL Leaf number, moved to RAX
- * @args (RSI)  - struct tdx_module_args for input and output
+ * @fn_code (RDI)  - SEAMCALL composite leaf code, moved to RAX
+ * @args    (RSI)  - struct tdx_module_args for input and output
  *
  * Only RCX/RDX/R8-R11 are used as input/output registers.
  *
@@ -51,8 +51,8 @@ SYM_FUNC_END(__seamcall_ret)
  *
  * __seamcall_saved_ret() function ABI:
  *
- * @fn   (RDI)  - SEAMCALL Leaf number, moved to RAX
- * @args (RSI)  - struct tdx_module_args for input and output
+ * @fn_code (RDI)  - SEAMCALL composite leaf code, moved to RAX
+ * @args    (RSI)  - struct tdx_module_args for input and output
  *
  * All registers in @args are used as input/output registers.
  *
arch/x86/virt/vmx/tdx/seamcall_internal.h +16 -5
index be5f446467df..bb17d965b453 100644
@@ -11,17 +11,28 @@
 #ifndef _X86_VIRT_SEAMCALL_INTERNAL_H
 #define _X86_VIRT_SEAMCALL_INTERNAL_H
 
+#include <linux/bitfield.h>
 #include <linux/printk.h>
 #include <linux/types.h>
 #include <asm/archrandom.h>
 #include <asm/processor.h>
 #include <asm/tdx.h>
 
-u64 __seamcall(u64 fn, struct tdx_module_args *args);
-u64 __seamcall_ret(u64 fn, struct tdx_module_args *args);
-u64 __seamcall_saved_ret(u64 fn, struct tdx_module_args *args);
+u64 __seamcall(u64 fn_code, struct tdx_module_args *args);
+u64 __seamcall_ret(u64 fn_code, struct tdx_module_args *args);
+u64 __seamcall_saved_ret(u64 fn_code, struct tdx_module_args *args);
 
-typedef u64 (*sc_func_t)(u64 fn, struct tdx_module_args *args);
+typedef u64 (*sc_func_t)(u64 fn_code, struct tdx_module_args *args);
+
+#define SEAMCALL_VERSION_MASK		GENMASK_U64(23, 16)
+
+static __always_inline u64 __seamcall_fn_encoding(sc_func_t func, u64 fn,
+						  struct tdx_module_args *args)
+{
+	FIELD_MODIFY(SEAMCALL_VERSION_MASK, &fn, args->version);
+
+	return func(fn, args);
+}
 
 static __always_inline u64 __seamcall_dirty_cache(sc_func_t func, u64 fn,
 						  struct tdx_module_args *args)
@@ -39,7 +50,7 @@ static __always_inline u64 __seamcall_dirty_cache(sc_func_t func, u64 fn,
 	 */
 	this_cpu_write(cache_state_incoherent, true);
 
-	return func(fn, args);
+	return __seamcall_fn_encoding(func, fn, args);
 }
 
 static __always_inline u64 sc_retry(sc_func_t func, u64 fn,
arch/x86/virt/vmx/tdx/tdxcall.S +1 -1
index 016a2a1ec1d6..b0f7867bcd1c 100644
@@ -24,7 +24,7 @@
  *-------------------------------------------------------------------------
  * Input Registers:
  *
- * RAX                        - TDCALL/SEAMCALL Leaf number.
+ * RAX                        - TDCALL/SEAMCALL composite Leaf code.
  * RCX,RDX,RDI,RSI,RBX,R8-R15 - TDCALL/SEAMCALL Leaf specific input registers.
  *
  * Output Registers:
arch/x86/virt/vmx/tdx/tdx.c +7 -8
index 6a1c4fe202bb..8c1a5b7f603a 100644
@@ -1019,7 +1019,6 @@ static __init void set_tdx_addon_features(void)
 static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
 				    u64 global_keyid)
 {
-	u64 seamcall_fn = TDH_SYS_CONFIG_V0;
 	struct tdx_module_args args = {};
 	u64 *tdmr_pa_array;
 	size_t array_sz;
@@ -1042,18 +1041,18 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
 	for (i = 0; i < tdmr_list->nr_consumed_tdmrs; i++)
 		tdmr_pa_array[i] = __pa(tdmr_entry(tdmr_list, i));
 
+	set_tdx_addon_features();
+
 	args.rcx = __pa(tdmr_pa_array);
 	args.rdx = tdmr_list->nr_consumed_tdmrs;
 	args.r8 = global_keyid;
 
-	set_tdx_addon_features();
-
 	if (tdx_addon_feature0) {
 		args.r9 = tdx_addon_feature0;
-		seamcall_fn = TDH_SYS_CONFIG;
+		args.version = 1;
 	}
 
-	ret = seamcall_prerr(seamcall_fn, &args);
+	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
 
 	/* Free the array as it is not required anymore. */
 	kfree(tdmr_pa_array);
@@ -1515,16 +1514,15 @@ int tdx_module_shutdown(void)
 
 int tdx_module_run_update(void)
 {
-	u64 seamcall_fn = TDH_SYS_UPDATE_V0;
 	struct tdx_module_args args = {};
 	int ret;
 
 	if (tdx_addon_feature0) {
 		args.r9 = tdx_addon_feature0;
-		seamcall_fn = TDH_SYS_UPDATE;
+		args.version = 1;
 	}
 
-	ret = seamcall_prerr(seamcall_fn, &args);
+	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
 	if (ret)
 		return ret;
 
@@ -2112,6 +2110,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
 		.rcx = vp->tdvpr_pa,
 		.rdx = initial_rcx,
 		.r8 = x2apicid,
+		.version = 1,
 	};
 
 	return seamcall(TDH_VP_INIT, &args);
arch/x86/virt/vmx/tdx/tdx.h +3 -18
index 2deb0a5c902e..1f43d2eb2345 100644
@@ -2,7 +2,6 @@
 #ifndef _X86_VIRT_TDX_H
 #define _X86_VIRT_TDX_H
 
-#include <linux/bitfield.h>
 #include <linux/bits.h>
 
 /*
@@ -12,18 +11,6 @@
  * architectural definitions come first.
  */
 
-/*
- * SEAMCALL leaf:
- *
- * Bit 15:0	Leaf number
- * Bit 23:16	Version number
- */
-#define SEAMCALL_LEAF			GENMASK(15, 0)
-#define SEAMCALL_VER			GENMASK(23, 16)
-
-#define SEAMCALL_LEAF_VER(l, v)		(FIELD_PREP(SEAMCALL_LEAF, l) | \
-					 FIELD_PREP(SEAMCALL_VER, v))
-
 /*
  * TDX module SEAMCALL leaf functions
  */
@@ -44,7 +31,7 @@
 #define TDH_VP_CREATE			10
 #define TDH_MNG_KEY_FREEID		20
 #define TDH_MNG_INIT			21
-#define TDH_VP_INIT			SEAMCALL_LEAF_VER(22, 1)
+#define TDH_VP_INIT			22
 #define TDH_PHYMEM_PAGE_RDMD		24
 #define TDH_VP_RD			26
 #define TDH_PHYMEM_PAGE_RECLAIM		28
@@ -58,11 +45,9 @@
 #define TDH_PHYMEM_CACHE_WB		40
 #define TDH_PHYMEM_PAGE_WBINVD		41
 #define TDH_VP_WR			43
-#define TDH_SYS_CONFIG_V0		45
-#define TDH_SYS_CONFIG			SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)
+#define TDH_SYS_CONFIG			45
 #define TDH_SYS_SHUTDOWN		52
-#define TDH_SYS_UPDATE_V0		53
-#define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
+#define TDH_SYS_UPDATE			53
 #define TDH_EXT_INIT			60
 #define TDH_EXT_MEM_ADD			61
 #define TDH_SYS_DISABLE			69
#10 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

In reply to Xu Yilun
Quoted 12 lines · Xu Yilun

In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them. The kernel should explicitly configure these features at boot or post-update initialization time. Configuring an add-on feature, such as TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions. TDX Quoting is the target feature to enable but defer it for now until full kernel support is in place.

TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as TDX_FEATURES0.

Quoted 5 lines

For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

Actually, we do not need another global variable here. tdx_features0 is cached and is not updated across a runtime update, so the derived add-on feature bitmap will be the same before and after the update.

Quoted 8 lines

static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
u64 global_keyid)
{

arch/x86/virt/vmx/tdx/tdx.c +1 -0
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;
	struct tdx_module_args args = {};
	u64 *tdmr_pa_array;
	size_t array_sz;

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1049

Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -1
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
 	args.rcx = __pa(tdmr_pa_array);
 	args.rdx = tdmr_list->nr_consumed_tdmrs;
 	args.r8 = global_keyid;
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
+	set_tdx_addon_features();
+
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;

How about moving this r9 assignment out of the if block and placing it next to 'args.r8 = global_keyid;'? There is no need to guard it, because args.r9 will be 0 when no add-on features are enabled, which is perfectly fine.

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+		seamcall_fn = TDH_SYS_CONFIG;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);
author
#11 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

Quoted 9 lines
Quoted 5 lines

For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

Actually, we do not need another global variable here. tdx_features0 is cached and is not updated across a runtime update, so the derived add-on feature bitmap will be the same before and after the update.

I think a global var "static u64 tdx_addon_feature0 *__ro_after_init*;" better illustrates the policy that add-on feature bitmap should be decided at boot up and never change later. It will also be used to decide if a specific add-on feature initialization is needed. We don't want to calculate the bitmap again and again, though the result must be the same.

Maybe I should strenghthen the commit message:

... both phases can use it. This actually mirrors a TDX module internal state so that kernel knows which add-on TDX operations (for example, quoting SEAMCALLs, which will be added in later patches) are valid.

Quoted 15 lines
Quoted 15 lines

static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
u64 global_keyid)
{

arch/x86/virt/vmx/tdx/tdx.c +1 -0
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;
	struct tdx_module_args args = {};
	u64 *tdmr_pa_array;
	size_t array_sz;
arch/x86/virt/vmx/tdx/tdx.c +1 -1
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
 	args.rcx = __pa(tdmr_pa_array);
 	args.rdx = tdmr_list->nr_consumed_tdmrs;
 	args.r8 = global_keyid;
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1049

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+	set_tdx_addon_features();
+
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;

How about moving this r9 assignment out of the if block and placing it next to 'args.r8 = global_keyid;'? There is no need to guard it, because args.r9 will be 0 when no add-on features are enabled, which is perfectly fine.

I tend to keep r9 assignment in the block, it clearly shows which SEAMCALL version needs what parameters, help people map the code to TDX module spec.

Quoted 5 lines
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+		seamcall_fn = TDH_SYS_CONFIG;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);
#12 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

In reply to Xu Yilun
Quoted 2 lines · Xu Yilun

In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them.

"add-on features" looks like a new term introduced by this sereis for TDX. So what is add-on features? all the features defined in TDX_FEATURE0/1? Or only the features needs to be explicitly enabled by R9 and R10 of TDH.SYS.CONFIG? ...

Quoted 2 lines

The kernel should explicitly configure these features at boot or post-update initialization time.

... So the answer is the latter. Then why only these bits are add-on features while the rest in TDX_FEATURES01 are not?

btw, s/should/needs/ is better?

Quoted 3 lines

Configuring an add-on feature, such as
TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions.

Though the statement is right, it leads to the impression that the reason we are configuring the add-on feature is to initialize the TDX module extensions.

However, the truth is kenrel wants to enable/use an add-on feature and the add-on feature requires the functionalities provided by some TDX module extension. So kernel needs to enable the TDX module extensions.

Quoted 77 lines

TDX Quoting is the target feature to
enable but defer it for now until full kernel support is in place.

TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as TDX_FEATURES0.

For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1. So there is little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.

---
  arch/x86/virt/vmx/tdx/tdx.h |  6 ++++--
  arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++--
  2 files changed, 30 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +4 -2
index fbb520704662..a47e872480c7 100644
@@ -58,9 +58,11 @@
  #define TDH_PHYMEM_CACHE_WB		40
  #define TDH_PHYMEM_PAGE_WBINVD		41
  #define TDH_VP_WR			43
-#define TDH_SYS_CONFIG			45
+#define TDH_SYS_CONFIG_V0		45
+#define TDH_SYS_CONFIG			SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)
  #define TDH_SYS_SHUTDOWN		52
-#define TDH_SYS_UPDATE			53
+#define TDH_SYS_UPDATE_V0		53
+#define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
  #define TDH_SYS_DISABLE			69
 
  /* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +10 -0
index 2a03152796e6..92305b5ea90d 100644
@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;
  static u32 tdx_global_keyid __ro_after_init;
  static u32 tdx_guest_keyid_start __ro_after_init;
  static u32 tdx_nr_guest_keyids __ro_after_init;
+static u64 tdx_addon_feature0 __ro_after_init;
 
  static DEFINE_IDA(tdx_guest_keyid_pool);
 
@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
  	return ret;
  }
 
+static __init void set_tdx_addon_features(void)
+{
+	/*
+	 * To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when
+	 * kernel is ready.
+	 */
+}
+
  static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
  				    u64 global_keyid)
  {
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;
  	struct tdx_module_args args = {};
  	u64 *tdmr_pa_array;
  	size_t array_sz;

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1046

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -1
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
  	args.rcx = __pa(tdmr_pa_array);
  	args.rdx = tdmr_list->nr_consumed_tdmrs;
  	args.r8 = global_keyid;
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
+	set_tdx_addon_features();

Maybe move the set_tdx_addon_features() out of config_tdx_module()? how about putting it after check_features(). config_tdx_module() looks like just a wrapper to invoke TDH.SYS.CONFIG, while the params of it are determined outside of it.

Just my feeling.

Quoted 28 lines
arch/x86/virt/vmx/tdx/tdx.c +14 -1
+
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_CONFIG;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);
 
  	/* Free the array as it is not required anymore. */
  	kfree(tdmr_pa_array);
@@ -1314,10 +1332,16 @@ int tdx_module_shutdown(void)
 
  int tdx_module_run_update(void)
  {
+	u64 seamcall_fn = TDH_SYS_UPDATE_V0;
  	struct tdx_module_args args = {};
  	int ret;
 
-	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_UPDATE;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);
  	if (ret)
  		return ret;
 
author
#13 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

Quoted 9 lines · Xiaoyao Li
Quoted 2 lines · Xu Yilun

In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them.

"add-on features" looks like a new term introduced by this sereis for TDX. So what is add-on features? all the features defined in TDX_FEATURE0/1? Or only the features needs to be explicitly enabled by R9 and R10 of TDH.SYS.CONFIG? ...

Quoted 5 lines
Quoted 2 lines

The kernel should explicitly configure these features at boot or post-update initialization time.

... So the answer is the latter. Then why only these bits are add-on features while the rest in TDX_FEATURES01 are not?

I think from TDX arch POV they impact the existing TDX system in some ways, either changes the behaviors of the existing features, or impact the performance.

Quoted 7 lines

btw, s/should/needs/ is better?

Quoted 3 lines

Configuring an add-on feature, such as
TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions.

Quoted 5 lines

Though the statement is right, it leads to the impression that the reason we are configuring the add-on feature is to initialize the TDX module extensions.

However, the truth is kenrel wants to enable/use an add-on feature and the

mm... yes it is somewhat misleading. Is it better we only talk about the add-on features' enabling in the first paragraph, then in next paragraph explain why we put the patch at the first one for TDX module Extension enabling?

Quoted 79 lines

add-on feature requires the functionalities provided by some TDX module extension. So kernel needs to enable the TDX module extensions.

Quoted 76 lines

TDX Quoting is the target feature to
enable but defer it for now until full kernel support is in place.

TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as TDX_FEATURES0.

For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1. So there is little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.

---
  arch/x86/virt/vmx/tdx/tdx.h |  6 ++++--
  arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++--
  2 files changed, 30 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +4 -2
index fbb520704662..a47e872480c7 100644
@@ -58,9 +58,11 @@
  #define TDH_PHYMEM_CACHE_WB		40
  #define TDH_PHYMEM_PAGE_WBINVD		41
  #define TDH_VP_WR			43
-#define TDH_SYS_CONFIG			45
+#define TDH_SYS_CONFIG_V0		45
+#define TDH_SYS_CONFIG			SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)
  #define TDH_SYS_SHUTDOWN		52
-#define TDH_SYS_UPDATE			53
+#define TDH_SYS_UPDATE_V0		53
+#define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
  #define TDH_SYS_DISABLE			69
  /* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +10 -0
index 2a03152796e6..92305b5ea90d 100644
@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;
  static u32 tdx_global_keyid __ro_after_init;
  static u32 tdx_guest_keyid_start __ro_after_init;
  static u32 tdx_nr_guest_keyids __ro_after_init;
+static u64 tdx_addon_feature0 __ro_after_init;
  static DEFINE_IDA(tdx_guest_keyid_pool);
@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
  	return ret;
  }
+static __init void set_tdx_addon_features(void)
+{
+	/*
+	 * To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when
+	 * kernel is ready.
+	 */
+}
+
  static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
  				    u64 global_keyid)
  {
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;
  	struct tdx_module_args args = {};
  	u64 *tdmr_pa_array;
  	size_t array_sz;
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
  	args.rcx = __pa(tdmr_pa_array);
  	args.rdx = tdmr_list->nr_consumed_tdmrs;
  	args.r8 = global_keyid;
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1046

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -1
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
+	set_tdx_addon_features();

Maybe move the set_tdx_addon_features() out of config_tdx_module()? how about putting it after check_features(). config_tdx_module() looks like just a wrapper to invoke TDH.SYS.CONFIG, while the params of it are determined outside of it.

I'm going to take Chao's advice [1] to remove the global var tdx_addon_feature0, so will also delete this global step - set_tdx_addon_features() - to setup this global var.

Instead I'll use a "static u64 get_tdx_addon_features0(void)" for TDH.SYS.CONFIG/UPDATE.

The reason is that I've found this extra TDX status - tdx_addon_feature0 - makes the feature supporting judgement unnecessary complex, so don't make it long live.

[1]: https://lore.kernel.org/all/[email protected]/

#14 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

In reply to Xu Yilun
Quoted 7 lines · Xu Yilun

In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them. The kernel should explicitly configure these features at boot or post-update initialization time. Configuring an add-on feature, such as TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions. TDX Quoting is the target feature to enable but defer it for now until full kernel support is in place.

I think this is too much project management type info for the commit log. This patch is generic and could support any type of extension. I'd just list a few as examples of extensions.

Quoted 4 lines

TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as TDX_FEATURES0.

Assuming the v3 seamcall version patch is the final one, this should be updated to talk about this with respect to how that patch exposes the seamcall version concept.

Quoted 2 lines

For runtime update, Linux applies a policy

"applies a policy" sticks out to me here. Can we put this in simple terms of why this isn't done. "Policy" makes it sound arbitrary rule that we are blindly following.

Quoted 1 line

that no newer features should

"newer" is ambiguous here. I think it means no features that were not previously enabled, but it could come off as saying no features that were released after a certain point.

Quoted 9 lines

be added after update to avoid disrupting live TDX operations.

To adhere
to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

Quoted 5 lines

TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1.

Ha, these are hard things to predict.

Quoted 3 lines

So there is
little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.

We can't deprecate it today, so what is this saying exactly?

Quoted 49 lines

---
 arch/x86/virt/vmx/tdx/tdx.h |  6 ++++-- arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++--

 2 files changed, 30 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +4 -2
index fbb520704662..a47e872480c7 100644
@@ -58,9 +58,11 @@
  #define TDH_PHYMEM_CACHE_WB		40
  #define TDH_PHYMEM_PAGE_WBINVD		41
  #define TDH_VP_WR			43
-#define TDH_SYS_CONFIG			45
+#define TDH_SYS_CONFIG_V0		45
+#define TDH_SYS_CONFIG			SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0,
 1)
  #define TDH_SYS_SHUTDOWN		52
-#define TDH_SYS_UPDATE			53
+#define TDH_SYS_UPDATE_V0		53
+#define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0,
 1)
  #define TDH_SYS_DISABLE			69


 /* TDX page types */

arch/x86/virt/vmx/tdx/tdx.c +1 -0
index 2a03152796e6..92305b5ea90d 100644
@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;
  static u32 tdx_global_keyid __ro_after_init;
  static u32 tdx_guest_keyid_start __ro_after_init;
  static u32 tdx_nr_guest_keyids __ro_after_init;
+static u64 tdx_addon_feature0 __ro_after_init;


 static DEFINE_IDA(tdx_guest_keyid_pool);

arch/x86/virt/vmx/tdx/tdx.c +0 -0
@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head
 *tmb_list,
  	return ret;
  }

arch/x86/virt/vmx/tdx/tdx.c +4 -0
+static __init void set_tdx_addon_features(void)
+{
+	/*
+	 * To add DICE-based TDX Quoting feature bit in tdx_addon_feature0

when

arch/x86/virt/vmx/tdx/tdx.c static __init int construct_tdmrs(struct list_head *tmb_list, · line 1013

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	 * kernel is ready.
+	 */

Can we not make this DICE specific?

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+}
+

 static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
 {

arch/x86/virt/vmx/tdx/tdx.c +1 -0
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;

  struct tdx_module_args args = {};
  u64 *tdmr_pa_array;
  size_t array_sz;

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct · line 1047

Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -1
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct
 tdmr_info_list *tdmr_list,
  	args.rcx = __pa(tdmr_pa_array);
  	args.rdx = tdmr_list->nr_consumed_tdmrs;
  	args.r8 = global_keyid;
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
+	set_tdx_addon_features();

This is a little opaque. Why not something like

tdx_addon_feature0 = TDX_KERNEL_SUPPORTED_ADDONS & tdx_sysinfo.features.tdx_features0;

tdx_addon_feature0 could probably be named better too. It means the features that the kernel has decided to use. HMm, any ideas?

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1051

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_CONFIG;
+	}

This kind of feels hacky to me. tdx_addon_feature0 is zero if there are none supported. It shouldn't interfere with the v0 call to set it unconditionally?

I think the connection between tdx_addon_feature0 and TDH_SYS_CONFIG is a bit tenuous too. It is set if a extension is found that the kernel supports.

What do you think about wrapping TDH_SYS_CONFIG. Have the wrapper take all the args including tdx_addon_feature0. Then have the wrapper decide which seamcall to use. Have a nice comment that says "Use version of seamcall that supports add ons if we have any, use the old one if none are supported for backward compatibility".

BTW, looking at the tdx module, if I'm reading this right, r9 used to be ignored. So did it need a new seamcall version? Some other VMM was passing garbage?

Quoted 23 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+
+	ret = seamcall_prerr(seamcall_fn, &args);


  /* Free the array as it is not required anymore. */ kfree(tdmr_pa_array);

arch/x86/virt/vmx/tdx/tdx.c +0 -0
@@ -1314,10 +1332,16 @@ int tdx_module_shutdown(void)


 int tdx_module_run_update(void)
 {

arch/x86/virt/vmx/tdx/tdx.c +1 -0
+	u64 seamcall_fn = TDH_SYS_UPDATE_V0;

  struct tdx_module_args args = {};
  int ret;

arch/x86/virt/vmx/tdx/tdx.c +6 -1
-	ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_UPDATE;
+	}
+
+	ret = seamcall_prerr(seamcall_fn, &args);

  if (ret)
  return ret;

author
#15 lore raw

Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update

Quoted 11 lines
Quoted 10 lines

To adhere
to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.

TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1.

Quoted 7 lines

Ha, these are hard things to predict.

Quoted 3 lines

So there is
little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.

We can't deprecate it today, so what is this saying exactly?

This paragraph is outdated, at that time the target is to always use the latest SEAMCALL version. I'll remove it.

I updated the changelog according to these comments, appended at the end.

[...]

Quoted 9 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int construct_tdmrs(struct list_head *tmb_list, · line 1013

Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+static __init void set_tdx_addon_features(void)
+{
+	/*
+	 * To add DICE-based TDX Quoting feature bit in tdx_addon_feature0

when

arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	 * kernel is ready.
+	 */

Can we not make this DICE specific?

Yes.

Quoted 21 lines
Quoted 11 lines
(quoted diff) +2 -0
+}
+

 static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
 {

arch/x86/virt/vmx/tdx/tdx.c +1 -0
+	u64 seamcall_fn = TDH_SYS_CONFIG_V0;

  struct tdx_module_args args = {};
  u64 *tdmr_pa_array;
  size_t array_sz;

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct · line 1047

Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -1
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct
 tdmr_info_list *tdmr_list,
  	args.rcx = __pa(tdmr_pa_array);
  	args.rdx = tdmr_list->nr_consumed_tdmrs;
  	args.r8 = global_keyid;
-	ret = seamcall_prerr(TDH_SYS_CONFIG, &args);
+
+	set_tdx_addon_features();

This is a little opaque. Why not something like

Quoted 2 lines

tdx_addon_feature0 = TDX_KERNEL_SUPPORTED_ADDONS & tdx_sysinfo.features.tdx_features0;

Good to me.

Quoted 3 lines

tdx_addon_feature0 could probably be named better too. It means the features that the kernel has decided to use. HMm, any ideas?

How about tdx_supported_features0?

BTW: I'm going to take Chao's advice [1] to remove the global var tdx_addon_feature0, and use a "static u64 get_tdx_supported_features0(void)" for TDH.SYS.CONFIG/UPDATE.

Quoted 10 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1051

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+
+	if (tdx_addon_feature0) {
+		args.r9 = tdx_addon_feature0;
+		seamcall_fn = TDH_SYS_CONFIG;
+	}

This kind of feels hacky to me. tdx_addon_feature0 is zero if there are none supported. It shouldn't interfere with the v0 call to set it unconditionally?

Quoted 8 lines

I think the connection between tdx_addon_feature0 and TDH_SYS_CONFIG is a bit tenuous too. It is set if a extension is found that the kernel supports.

What do you think about wrapping TDH_SYS_CONFIG. Have the wrapper take all the args including tdx_addon_feature0. Then have the wrapper decide which seamcall to use. Have a nice comment that says "Use version of seamcall that supports add ons if we have any, use the old one if none are supported for backward compatibility".

I'm good to add the wrapping for TDH_SYS_CONFIG and TDH_SYS_UPDATE.

Quoted 4 lines

BTW, looking at the tdx module, if I'm reading this right, r9 used to be ignored. So did it need a new seamcall version? Some other VMM was passing garbage?

Yes, I think it prevents possible garbage inputs suddenly take effect.

The updated changelog:

----8<----

x86/virt/tdx: Set configurable features on TDX module init and update

TDX module identifies some configurable features that must be explicitly enabled as the kernel supports them. These features affect existing TDX systems: they may change the behaviors of the existing features, reserve more memory, or impact the TDX initialization performance. The kernel needs to explicitly enable these features at boot or post-update initialization time.

Features such as TDISP, DICE-based quoting and TD migration are among those configurable features. They use preemptible SEAMCALLs so require extra memory and setup time to build a global service environment - the TDX module extensions. Add support for the configurable feature setting as the prerequisite for initializing TDX module extensions.

TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which features to enable. The bitmap uses the same definitions as TDX_FEATURES0. TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the enhancement. A previous change [1] adds the 'version' field to struct tdx_module_args to support versioned SEAMCALLs. Assign the version field to 1 if any configurable feature is required.

Add a get_tdx_addon_features0() helper to return the bitmap of the configurable features that the module & kernel both support. Initially, this helper returns 0. It will be updated to return specific feature bits as full kernel support lands. TDH.SYS.CONFIG consumes this bitmap during TDX initialization.

Runtime updates keep the reported features unchanged across updates, so that existing TDX users can continue to operate without disruption. To adhere to this, provide TDH.SYS.UPDATE with the same bitmap returned by get_tdx_addon_features0(). The bitmap is guaranteed to match the initial TDH.SYS.CONFIG input because tdx_sysinfo.features.tdx_features0 is cached and not refreshed after update.

author
#16 lore raw

PATCH v2 3/17 x86/virt/tdx: Detect if the extensions initialization is required

In reply to Xu Yilun

TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.

Add infrastructure to initialize TDX module extensions. Introduce the initial step of this process by detecting if the extensions are required by checking:

1. If the extensions are supported via TDX_FEATURES0_EXT.
2. If any TDX add-on feature needs the extensions via a boolean metadata field ext_required.

Currently all metadata fields are read at the very beginning of basic TDX initialization and stored in a global var. However, ext_required is only valid after the add-on feature configuration, making it incompatible with the existing metadata reading method.

To resolve this lifetime conflict, add a dedicated runtime metadata reading interface for the extensions, call it when the extensions initialization starts, and leave the field out of the global var. In this way, there is no confusion of when the metadata should be read.

---
 arch/x86/include/asm/tdx.h                  |  1 +
 arch/x86/include/asm/tdx_global_metadata.h  |  4 ++++
 arch/x86/virt/vmx/tdx/tdx.c                 | 25 +++++++++++++++++++++
 arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 14 ++++++++++++
 4 files changed, 44 insertions(+)
arch/x86/include/asm/tdx.h +1 -0
index e5a9cf656c07..5fbf89d5317c 100644
@@ -35,6 +35,7 @@
 /* Bit definitions of TDX_FEATURES0 metadata field */
 #define TDX_FEATURES0_TD_PRESERVING	BIT_ULL(1)
 #define TDX_FEATURES0_NO_RBP_MOD	BIT_ULL(18)
+#define TDX_FEATURES0_EXT		BIT_ULL(39)
 
 #ifndef __ASSEMBLER__
 
arch/x86/include/asm/tdx_global_metadata.h +4 -0
index 41150d546589..83fc657a438e 100644
@@ -52,4 +52,8 @@ struct tdx_sys_info {
 	struct tdx_sys_info_td_conf td_conf;
 };
 
+struct tdx_sys_info_ext {
+	bool ext_required;
+};
+
 #endif
arch/x86/virt/vmx/tdx/tdx.c +25 -0
index 92305b5ea90d..6f3596f11d25 100644
@@ -1166,6 +1166,27 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;
+
+	/* TODO: add the extensions enabling steps here */
+
+	return 0;
+}
+
 static __init int init_tdx_module(void)
 {
 	int ret;
@@ -1220,6 +1241,10 @@ static __init int init_tdx_module(void)
 	if (ret)
 		goto err_reset_pamts;
 
+	ret = init_tdx_module_extensions();
+	if (ret)
+		goto err_reset_pamts;
+
 	pr_info("%lu KB allocated for PAMT\n", tdmrs_count_pamt_kb(&tdx_tdmr_list));
 
 out_put_tdxmem:
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14 -0
index e49c300f23d4..b9e1c011a990 100644
@@ -131,3 +131,17 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
 
 	return ret;
 }
+
+static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
+{
+	int ret;
+	u64 val;
+
+	ret = read_sys_metadata_field(0x3100000000000001, &val);
+	if (ret)
+		return ret;
+
+	sysinfo_ext->ext_required = val;
+
+	return 0;
+}
Signature · 1 line
2.25.1
Reviewed-by on v2 3/17
#17 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 5 lines · Xu Yilun

TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.

How about "TDX module extension SEAMCALLs are preemptible and resumable..." above to make it easier to read?

Other than that:

author
#18 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 9 lines · Tony Lindgren
Quoted 5 lines · Xu Yilun

TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.

How about "TDX module extension SEAMCALLs are preemptible and resumable..." above to make it easier to read?

Included, thanks.

Quoted 4 lines

Other than that:

#19 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

In reply to Tony Lindgren
Quoted 9 lines · Tony Lindgren
Quoted 5 lines · Xu Yilun

TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.

How about "TDX module extension SEAMCALLs are preemptible and resumable..." above to make it easier to read?

Agreed, it is hard to read. Also the first question will be, what is a "TDX module extension SEAMCALL"

#20 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

In reply to Xu Yilun
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;

What would happen if the kernel doesn't do this 'ext_required' check and always does the extension initialization?

If TDH.EXT.INIT returns success when no extension is configured, we could drop this 'ext_required' from this series entirely.

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+
+	/* TODO: add the extensions enabling steps here */
+
+	return 0;
+}
author
#21 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 19 lines · Chao Gao
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;

What would happen if the kernel doesn't do this 'ext_required' check and always does the extension initialization?

Quoted 1 line

If TDH.EXT.INIT returns success when no extension is configured, we

No, TDH.EXT.INIT fails when ext_required==false

Quoted 1 line

could drop this 'ext_required' from this series entirely.

#22 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 20 lines · Xu Yilun
Quoted 19 lines · Chao Gao
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;

What would happen if the kernel doesn't do this 'ext_required' check and always does the extension initialization?

Quoted 3 lines
Quoted 1 line

If TDH.EXT.INIT returns success when no extension is configured, we

No, TDH.EXT.INIT fails when ext_required==false

What is the SEAMCALL return code in this case?

Quoted 1 line
Quoted 1 line

could drop this 'ext_required' from this series entirely.

author
#23 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Xu Yilun
Quoted 19 lines · Chao Gao
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;

What would happen if the kernel doesn't do this 'ext_required' check and always does the extension initialization?

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

If TDH.EXT.INIT returns success when no extension is configured, we

No, TDH.EXT.INIT fails when ext_required==false

What is the SEAMCALL return code in this case?

It will return TDX_EXT_MEMORY_POOL_NOT_READY, because TDX module's internal state machine stays at the first step of extensions' initialization.

Quoted 3 lines
Quoted 2 lines
Quoted 2 lines

could drop this 'ext_required' from this series entirely.

#24 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

In reply to Xu Yilun
Quoted 3 lines · Xu Yilun

TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU.

This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,

- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1

(There might be more. I didn't check all.)

Quoted 3 lines

This allows for higher-level API constructions,
so better supports some add-on features that implement higher order security protocols.

I think we don't need to explain this? Just that some add-on features require the functionalities of TDX module extensions is enough?

Quoted 64 lines

Add infrastructure to initialize TDX module extensions. Introduce the initial step of this process by detecting if the extensions are required by checking:

1. If the extensions are supported via TDX_FEATURES0_EXT.
2. If any TDX add-on feature needs the extensions via a boolean metadata field ext_required.

Currently all metadata fields are read at the very beginning of basic TDX initialization and stored in a global var. However, ext_required is only valid after the add-on feature configuration, making it incompatible with the existing metadata reading method.

To resolve this lifetime conflict, add a dedicated runtime metadata reading interface for the extensions, call it when the extensions initialization starts, and leave the field out of the global var. In this way, there is no confusion of when the metadata should be read.

---
  arch/x86/include/asm/tdx.h                  |  1 +
  arch/x86/include/asm/tdx_global_metadata.h  |  4 ++++
  arch/x86/virt/vmx/tdx/tdx.c                 | 25 +++++++++++++++++++++
  arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 14 ++++++++++++
  4 files changed, 44 insertions(+)
arch/x86/include/asm/tdx.h +1 -0
index e5a9cf656c07..5fbf89d5317c 100644
@@ -35,6 +35,7 @@
  /* Bit definitions of TDX_FEATURES0 metadata field */
  #define TDX_FEATURES0_TD_PRESERVING	BIT_ULL(1)
  #define TDX_FEATURES0_NO_RBP_MOD	BIT_ULL(18)
+#define TDX_FEATURES0_EXT		BIT_ULL(39)
 
  #ifndef __ASSEMBLER__
 
arch/x86/include/asm/tdx_global_metadata.h +4 -0
index 41150d546589..83fc657a438e 100644
@@ -52,4 +52,8 @@ struct tdx_sys_info {
  	struct tdx_sys_info_td_conf td_conf;
  };
 
+struct tdx_sys_info_ext {
+	bool ext_required;
+};
+
  #endif
arch/x86/virt/vmx/tdx/tdx.c +5 -0
index 92305b5ea90d..6f3596f11d25 100644
@@ -1166,6 +1166,27 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
  	return 0;
  }
 
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;

There was the discussion around it in [1]. I agree with the point to make kernel code simple. But it requires change/clarification from TDX spec, right? So will TDX spec change?

If so, you would need to mention it somewhere.

[1] https://lore.kernel.org/all/[email protected]/

Quoted 41 lines
arch/x86/virt/vmx/tdx/tdx.c +9 -0
+	/* TODO: add the extensions enabling steps here */
+
+	return 0;
+}
+
  static __init int init_tdx_module(void)
  {
  	int ret;
@@ -1220,6 +1241,10 @@ static __init int init_tdx_module(void)
  	if (ret)
  		goto err_reset_pamts;
 
+	ret = init_tdx_module_extensions();
+	if (ret)
+		goto err_reset_pamts;
+
  	pr_info("%lu KB allocated for PAMT\n", tdmrs_count_pamt_kb(&tdx_tdmr_list));
 
  out_put_tdxmem:
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14 -0
index e49c300f23d4..b9e1c011a990 100644
@@ -131,3 +131,17 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
 
  	return ret;
  }
+
+static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
+{
+	int ret;
+	u64 val;
+
+	ret = read_sys_metadata_field(0x3100000000000001, &val);
+	if (ret)
+		return ret;
+
+	sysinfo_ext->ext_required = val;
+
+	return 0;
+}
author
#25 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 11 lines · Xiaoyao Li
Quoted 3 lines · Xu Yilun

TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU.

This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,

- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1

No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.

Quoted 9 lines

(There might be more. I didn't check all.)

Quoted 3 lines

This allows for higher-level API constructions,
so better supports some add-on features that implement higher order security protocols.

I think we don't need to explain this? Just that some add-on features require the functionalities of TDX module extensions is enough?

mm.. a big part of the series is to bring up the TDX module extensions, and this is the first patch to start implementing TDX module extensions, a comment [1] said we could explain the context a little bit in general.

[1]: https://lore.kernel.org/all/[email protected]/

[...]

Quoted 10 lines
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+static __init int init_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+		return ret;
+
+	/* Skip if no feature requires TDX module extensions. */
+	if (!sysinfo_ext.ext_required)
+		return 0;

There was the discussion around it in [1]. I agree with the point to make kernel code simple. But it requires change/clarification from TDX spec,

I got the answer from TDX module team, that there is no use case ext memory is needed/the Extensions are enabled but TDH.EXT.INIT is not required. IOW ext_required == false exactly means TDX module extensions are not required and no extension SEAMCALLs should be invoked.

Quoted 1 line

right? So will TDX spec change?

No I didn't raise this nits to distract their main focus on blocking changes. But yes I can raise a ticket for this.

Quoted 4 lines

If so, you would need to mention it somewhere.

[1] https://lore.kernel.org/all/[email protected]/

#26 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 7 lines · Xu Yilun
Quoted 6 lines

This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,

- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1

Quoted 5 lines

No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.

We have RESUMABLE seamcalls today. They are guaranteed to save state and make forward progress. Xiaoyao is right, this is not a new concept from the VMM side. But also why highlight this for the patch here. Are you trying to explain why extension initialization needs some special step?

author
#27 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 8 lines · Rick P
Quoted 7 lines · Xu Yilun
Quoted 6 lines

This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,

- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1

Quoted 8 lines
Quoted 5 lines

No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.

We have RESUMABLE seamcalls today. They are guaranteed to save state and make forward progress. Xiaoyao is right, this is not a new concept from the VMM side.

OK. Let me de-emphasize resumable.

And I'm thinking of how to stop explaining new terms again and again, extension SEAMCALL is the one. I'm not sure if calling it "preemptible SEAMCALLs" helps, or is it still a new term?

Quoted 2 lines

But also why highlight this for the patch here. Are you trying to explain why extension initialization needs some special step?

I was following the suggestion [1] of explaining the context in general. But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.

[1]: https://lore.kernel.org/all/[email protected]/

Refactored the first paragraph:

----8<----

TDX adds a new class of SEAMCALLs which are preemptible and can cooperate with the host scheduling. Some features like TDISP, DICE-based quoting rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.

...

#28 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 27 lines · Xu Yilun
Quoted 15 lines · Rick P
Quoted 8 lines · Xu Yilun
Quoted 7 lines

This is not true. There are some normal (non-extension) SEAMCALLs also are
preemptible and resumable. For example,

- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1

Quoted 5 lines

No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.

Quoted 4 lines

We have RESUMABLE seamcalls today. They are guaranteed to save state and make
forward progress. Xiaoyao is right, this is not a new concept from the VMM side.

OK. Let me de-emphasize resumable.

And I'm thinking of how to stop explaining new terms again and again, extension SEAMCALL is the one. I'm not sure if calling it "preemptible SEAMCALLs" helps, or is it still a new term?

Quoted 8 lines
Quoted 3 lines

But also why highlight this for the patch here. Are you trying to explain why
extension initialization needs some special step?

I was following the suggestion [1] of explaining the context in general. But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.

Wait are all extension seamcalls resumable? Couldn't there be one associated with an extension that is not? We have so far explained extensions in terms of the TDX module needing more memory for the optional features. So "extension" probably be interpreted as "optional", not as enabling "preemptible" seamcalls.

Quoted 10 lines

[1]:
https://lore.kernel.org/all/[email protected]/

Refactored the first paragraph:

----8<----

  TDX adds a new class of SEAMCALLs which are preemptible and can cooperate with the host scheduling.

But these are not new is the point. Look for TDX_INTERRUPTED_RESUMABLE in Linux and TDX source.

Quoted 5 lines

Some features like TDISP, DICE-based quoting
  rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.

Why are we even talking about this in a patch that reads the metadata. Is it necessary?

Quoted 2 lines

  ...

author
#29 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 6 lines
Quoted 3 lines

But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.

Wait are all extension seamcalls resumable? Couldn't there be one associated with an extension that is not?

All extension SEAMCALLs must be preemptible and resumable. If a SEAMCALL doesn't need the ability, it is not backed by an extension. Note an extension may only serve part of the functionalities of a feature, for example, for TDX Connect, only SPDM related SEAMCALLs are extension SEAMCALLs, others are not cause they have short or simple routines.

Quoted 3 lines

We have so far explained extensions in terms of
the TDX module needing more memory for the optional features. So "extension" probably be interpreted as "optional", not as enabling "preemptible" seamcalls.

This is a long story and is explained in cover-letter. The need for extra memory is only a requirement to host, not the purpose for "extension".

Normal SEAMCALLs monopolize CPUs when execution. To prevent starving the host, SEAMCALL developers must count instructions at compile time, pre-define interrupt check-points and yield to host. This programming mode for TDX module is so complex that is hardly possible to handle complex tasks. As a result, TDX module had to push complexity into the host, such as by fragmenting a protocol setup service into several phases, let the host manage the overall flow and the SEAMCALLs only handle the secure parts which are short routines... This makes the SEAMCALL ABIs trivial, losing their semantic meanings. While AMD only calls a TIO_DEV_CONNECT and let its co-processor manage the whole thing.

A preemptible SEAMCALL behaves like an OS task, so a specific SEAMCALL developer could easily handle a complex routine, without caring about starving the host. A general underlying service environment is taking care of the hardware interrupt response, the context switch...

So IMHO a normal SEAMCALL may be called 'resumable' to some extent, but it is definitely not 'preemptible', it voluntarily yields. That's also why I think "preemptible SEAMCALLs" natually explain things than "extension SEAMCALLs"

[...]

Quoted 7 lines
Quoted 4 lines

----8<----

  TDX adds a new class of SEAMCALLs which are preemptible and can cooperate with the host scheduling.

But these are not new is the point. Look for TDX_INTERRUPTED_RESUMABLE in Linux and TDX source.

From host perspective, extension SEAMCALLs are just like normal 'resumable' SEAMCALLs. But the mechanism is new, and we have to explain the differences, otherwise there is no justification why we introduce the extensions.

Quoted 9 lines
Quoted 6 lines

Some features like TDISP, DICE-based quoting
  rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.

Why are we even talking about this in a patch that reads the metadata. Is it necessary?

OK, I can drop the introduction of the context here, it is not necessary. The cover-letter has the detailed explanation.

But should we still add the introduction in patch 5 ("x86/virt/tdx: Make TDX module initialize the extensions")?

#30 lore raw

Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required

Quoted 8 lines · Xu Yilun
Quoted 6 lines
Quoted 3 lines

But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.

Wait are all extension seamcalls resumable? Couldn't there be one associated with an extension that is not?

All extension SEAMCALLs must be preemptible and resumable.

My main issue is that I don't see why this need to be declared in a patch called "Detect if the extensions initialization is required".

But nothing I can see in the TDX docs says something like the above. It may be true from your reasoning, but I don't see what is gained by declaring it a hard rule either.

Quoted 25 lines

If a SEAMCALL
doesn't need the ability, it is not backed by an extension. Note an extension may only serve part of the functionalities of a feature, for example, for TDX Connect, only SPDM related SEAMCALLs are extension SEAMCALLs, others are not cause they have short or simple routines.

Quoted 4 lines

We have so far explained extensions in terms of
the TDX module needing more memory for the optional features. So "extension" probably be interpreted as "optional", not as enabling "preemptible" seamcalls.

This is a long story and is explained in cover-letter. The need for extra memory is only a requirement to host, not the purpose for "extension".

Normal SEAMCALLs monopolize CPUs when execution. To prevent starving the host, SEAMCALL developers must count instructions at compile time, pre-define interrupt check-points and yield to host. This programming mode for TDX module is so complex that is hardly possible to handle complex tasks. As a result, TDX module had to push complexity into the host, such as by fragmenting a protocol setup service into several phases, let the host manage the overall flow and the SEAMCALLs only handle the secure parts which are short routines...  This makes the SEAMCALL ABIs trivial, losing their semantic meanings. While AMD only calls a TIO_DEV_CONNECT and let its co-processor manage the whole thing.

Quoted 7 lines

A preemptible SEAMCALL behaves like an OS task, so a specific SEAMCALL developer could easily handle a complex routine, without caring about starving the host. A general underlying service environment is taking care of the hardware interrupt response, the context switch...

So IMHO a normal SEAMCALL may be called 'resumable' to some extent, but it is definitely not 'preemptible', it voluntarily yields.

No, some check for pending interrupts and exit back to the host if it sees one. They save their progress manually and Linux relies on this. So it's the same from the host perspective. Pre-empted, save state, resume where you left off.

Quoted 12 lines

That's also
why I think "preemptible SEAMCALLs" natually explain things than "extension SEAMCALLs"

[...]

Quoted 6 lines
Quoted 5 lines

----8<----

  TDX adds a new class of SEAMCALLs which are preemptible and can cooperate
  with the host scheduling.

Quoted 8 lines
Quoted 3 lines

But these are not new is the point. Look for TDX_INTERRUPTED_RESUMABLE in Linux
and TDX source.

From host perspective, extension SEAMCALLs are just like normal 'resumable' SEAMCALLs. But the mechanism is new, and we have to explain the differences, otherwise there is no justification why we introduce the extensions.

Sure. People are not saying to skip explaining what is new. People are saying that what you are saying is new (pre-emptible, resumable), isn't new.

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Some features like TDISP, DICE-based quoting
  rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.

Quoted 8 lines
Quoted 2 lines

Why are we even talking about this in a patch that reads the metadata. Is it necessary?

OK, I can drop the introduction of the context here, it is not necessary. The cover-letter has the detailed explanation.

But should we still add the introduction in patch 5 ("x86/virt/tdx: Make TDX module initialize the extensions")?

Maybe it fits better in patch 4, because it is adding memory to save the state. But I think we shouldn't dwell too much on the internal details all over the logs.

I think it's good context to help reason about what is going on. But the important parts are that these extensions are optional and need more memory to save state for their interruptible resumable operations.

author
#31 lore raw

PATCH v2 4/17 x86/virt/tdx: Add extra memory to TDX module for the extensions

In reply to Xu Yilun

TDX module extensions receive a one-time memory allocation at initialization time. The extensions use this memory as the baseline for their internal states and data required by the service APIs they offer.

Add a new memory feeding process backed by a new SEAMCALL TDH.EXT.MEM.ADD. The process is mostly the same as adding PAMT. The kernel queries TDX module how much memory needed by reading the memory_pool_required_pages, allocates it, hands it over to the module, and never gets it back.

TDH.EXT.MEM.ADD uses a new parameter type, HPA_LIST_INFO, to provide this memory. This type represents a list of pages for TDX module to access. It references an 'hpa_list page' which contains the list of target HPAs. It collapses the HPA of the hpa_list page and the number of valid target HPAs into a 64 bit raw value for SEAMCALL parameters. The hpa_list page is always a medium, TDX module never keeps the hpa_list page.

Don't CLFLUSH the pages handed to the TDX module, as is done for some other SEAMCALLs. The flushing operation is not expected to be needed for current and known future architectures. As more and more page feeding interfaces to come, the conservative flushing operation becomes a maintenance burden.

For now, TDX module extensions consume tens of megabytes memory that will never be returned to host. Use contiguous page allocation to isolate these large blocks entirely, avoiding permanent memory fragmentation and reducing buddy allocator efficiency. Print the allocation amount on TDX module extensions initialization for visibility.

---
 arch/x86/include/asm/tdx_global_metadata.h  |   1 +
 arch/x86/virt/vmx/tdx/tdx.h                 |   1 +
 arch/x86/virt/vmx/tdx/tdx.c                 | 107 +++++++++++++++++++-
 arch/x86/virt/vmx/tdx/tdx_global_metadata.c |   6 ++
 4 files changed, 112 insertions(+), 3 deletions(-)
arch/x86/include/asm/tdx_global_metadata.h +1 -0
index 83fc657a438e..b3442b7c88bb 100644
@@ -53,6 +53,7 @@ struct tdx_sys_info {
 };
 
 struct tdx_sys_info_ext {
+	u32 memory_pool_required_pages;
 	bool ext_required;
 };
 
arch/x86/virt/vmx/tdx/tdx.h +1 -0
index a47e872480c7..a100634087e7 100644
@@ -63,6 +63,7 @@
 #define TDH_SYS_SHUTDOWN		52
 #define TDH_SYS_UPDATE_V0		53
 #define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
+#define TDH_EXT_MEM_ADD			61
 #define TDH_SYS_DISABLE			69
 
 /* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +104 -3

125 lines not on this page · raw message

arch/x86/virt/vmx/tdx/tdx_global_metadata.c +6 -0
index b9e1c011a990..720cdaf76492 100644
@@ -137,6 +137,12 @@ static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
 	int ret;
 	u64 val;
 
+	ret = read_sys_metadata_field(0x3100000200000000, &val);
+	if (ret)
+		return ret;
+
+	sysinfo_ext->memory_pool_required_pages = val;
+
 	ret = read_sys_metadata_field(0x3100000000000001, &val);
 	if (ret)
 		return ret;
Signature · 1 line
2.25.1
#32 lore raw

Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] Hardware PFN mismatch due to unaligned slab allocation for `hpa_list` with debugging enabled.…

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Hardware PFN mismatch due to unaligned slab allocation for hpa_list with debugging enabled.
- [Medium] Resource leak of un-added contiguous host memory on TDX module error path.
- [Medium] Improper use of node_online_map instead of N_MEMORY node states for allocation.
- [Low] Unnecessary early boot memory allocation error handling returning -ENOMEM.
--

commit e910725d57b782da4b05f3bb30b4df0fa4da1da3
Author: Xu Yilun <[email protected]>

x86/virt/tdx: Add extra memory to TDX module for the extensions

This patch introduces a memory feeding process via SEAMCALL TDH.EXT.MEM.ADD to allocate one-time memory for TDX module extensions during initialization. It allocates contiguous memory and passes it to the module using a newly defined HPA_LIST_INFO format.

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 6f3596f11d250..dab17822c1c69 100644

[ ... ]

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int tdx_ext_mem_setup(unsigned int required_pages)
+{
+	struct tdx_hpa_list *hpa_list;
+	struct page *page;
+	unsigned int i;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1225

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	int ret;
+
+	/*
+	 * memory_pool_required_pages == 0 means no need to add pages,
+	 * skip the memory setup.
+	 */
+	if (!required_pages)
+		return 0;
+
+	hpa_list = kzalloc_obj(*hpa_list);

[Severity: High]
Does this code pass unaligned physical pages to the hardware if slab debugging is enabled?

When CONFIG_SLUB_DEBUG or KASAN is enabled, slab objects are padded with metadata, causing the object to start at a non-zero offset within a physical page. The virt_to_page(hpa_list) call below will grab the base page frame which the hardware will read from offset 0, leading it to consume SLUB metadata instead of the valid hpa_list entries.

Should this allocation use the page allocator directly (e.g., __get_free_page()) to guarantee strict page alignment?

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1230

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+	if (!hpa_list)
+		return -ENOMEM;
+
+	page = alloc_contig_pages(required_pages, GFP_KERNEL, numa_mem_id(),
+				  &node_online_map);

[Severity: Medium]
Can this allocation fall back to memoryless nodes?

The node_online_map can include nodes without memory. The mm allocation guidelines require using the N_MEMORY state mask (e.g., node_states[N_MEMORY]) to correctly constrain memory allocations to nodes that actually have memory.

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1232

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	if (!page) {
+		ret = -ENOMEM;

[Severity: Low]
Is this complex error handling necessary here?

Since tdx_ext_mem_setup() is only called from the __init path, physical memory should be available. If an allocation of this size fails during early boot, the system likely cannot boot anyway, making graceful error cleanup here largely dead code.

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+		goto out_free_hpa_list;
+	}
+
+	i = 0;
+	while (i < required_pages) {
+		unsigned int nents = min(required_pages - i,
+					 ARRAY_SIZE(hpa_list->phys));
+		unsigned int j;
+
+		for (j = 0; j < nents; j++)
+			hpa_list->phys[j] = page_to_phys(page + i + j);

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+
+		ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
+		/*
+		 * No SEAMCALLs to reclaim the added pages. For simple error
+		 * handling, leak all pages.
+		 */
+		WARN(ret, "Fatal: TDX module rejected (%d) memory for extensions, stranded all pages\n",
+		     ret);
+		if (ret)
+			break;

[Severity: Medium]
Does this intentionally leak un-added host memory on the error path?

While pages that have already been accepted by the hardware cannot be reclaimed, the remaining contiguous block of pages (from page + i to the end) are still entirely host-owned. Could these remaining un-added pages be safely freed via free_contig_range() instead of permanently leaking tens of megabytes?

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+
+		i += nents;
+	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author
#33 lore raw

Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions

Quoted 44 lines · On Thu, Jun 18, 2026 at 08:54:58AM +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Hardware PFN mismatch due to unaligned slab allocation for hpa_list with debugging enabled.
- [Medium] Resource leak of un-added contiguous host memory on TDX module error path.
- [Medium] Improper use of node_online_map instead of N_MEMORY node states for allocation.
- [Low] Unnecessary early boot memory allocation error handling returning -ENOMEM.
--

commit e910725d57b782da4b05f3bb30b4df0fa4da1da3
Author: Xu Yilun <[email protected]>

x86/virt/tdx: Add extra memory to TDX module for the extensions

This patch introduces a memory feeding process via SEAMCALL TDH.EXT.MEM.ADD to allocate one-time memory for TDX module extensions during initialization. It allocates contiguous memory and passes it to the module using a newly defined HPA_LIST_INFO format.

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 6f3596f11d250..dab17822c1c69 100644

[ ... ]

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int tdx_ext_mem_setup(unsigned int required_pages)
+{
+	struct tdx_hpa_list *hpa_list;
+	struct page *page;
+	unsigned int i;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1225

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	int ret;
+
+	/*
+	 * memory_pool_required_pages == 0 means no need to add pages,
+	 * skip the memory setup.
+	 */
+	if (!required_pages)
+		return 0;
+
+	hpa_list = kzalloc_obj(*hpa_list);

[Severity: High]
Does this code pass unaligned physical pages to the hardware if slab debugging is enabled?

Quoted 5 lines

When CONFIG_SLUB_DEBUG or KASAN is enabled, slab objects are padded with metadata, causing the object to start at a non-zero offset within a physical page. The virt_to_page(hpa_list) call below will grab the base page frame which the hardware will read from offset 0, leading it to consume SLUB metadata instead of the valid hpa_list entries.

I think this is false positive. "For sizes which are a power of two, the alignment is also guaranteed to be at least the respective size." according to:

59bb47985c1d ("mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two)")

Quoted 13 lines

Should this allocation use the page allocator directly (e.g., __get_free_page()) to guarantee strict page alignment?

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1230

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+	if (!hpa_list)
+		return -ENOMEM;
+
+	page = alloc_contig_pages(required_pages, GFP_KERNEL, numa_mem_id(),
+				  &node_online_map);

[Severity: Medium]
Can this allocation fall back to memoryless nodes?

Quoted 1 line

The node_online_map can include nodes without memory. The mm allocation

I think this is false positive, at least for x86. According to Documentation/mm/numa.rst, "For some architectures, such as x86, Linux will "hide" any node representing a physical cell that has no memory attached"

Quoted 9 lines

guidelines require using the N_MEMORY state mask (e.g., node_states[N_MEMORY]) to correctly constrain memory allocations to nodes that actually have memory.

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1232

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	if (!page) {
+		ret = -ENOMEM;

[Severity: Low]
Is this complex error handling necessary here?

Quoted 4 lines

Since tdx_ext_mem_setup() is only called from the __init path, physical memory should be available. If an allocation of this size fails during early boot, the system likely cannot boot anyway, making graceful error cleanup here largely dead code.

It executes at subsys_initcall(), the system can still go on booting. Free a previously allocated page is not complex.

Actually the error handling would be complex when memory is partially added for extensions but TDX error happens. In this case for simplicity the below code leaks all the memory.

Quoted 10 lines
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +9 -0
+		goto out_free_hpa_list;
+	}
+
+	i = 0;
+	while (i < required_pages) {
+		unsigned int nents = min(required_pages - i,
+					 ARRAY_SIZE(hpa_list->phys));
+		unsigned int j;
+
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1249

Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +8 -0
+		for (j = 0; j < nents; j++)
+			hpa_list->phys[j] = page_to_phys(page + i + j);
+
+		ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
+		/*
+		 * No SEAMCALLs to reclaim the added pages. For simple error
+		 * handling, leak all pages.
+		 */

Maybe the comment is not clear enough so sashiko raised the below concern, I'll make it more clear:

/*
* No SEAMCALLs to reclaim the added pages. For simple error
* handling, intentionally leak all pages, including un-added
* pages.
*/

Quoted 19 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+		WARN(ret, "Fatal: TDX module rejected (%d) memory for extensions, stranded all pages\n",
+		     ret);
+		if (ret)
+			break;

[Severity: Medium]
Does this intentionally leak un-added host memory on the error path?

While pages that have already been accepted by the hardware cannot be reclaimed, the remaining contiguous block of pages (from page + i to the end) are still entirely host-owned. Could these remaining un-added pages be safely freed via free_contig_range() instead of permanently leaking tens of megabytes?

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+
+		i += nents;
+	}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#34 lore raw

Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions

In reply to Xu Yilun
Quoted 16 lines
arch/x86/virt/vmx/tdx/tdx.c +16 -0
+#define HPA_LIST_INFO_FIRST_ENTRY	GENMASK_U64(11, 3)
+#define HPA_LIST_INFO_PFN		GENMASK_U64(51, 12)
+#define HPA_LIST_INFO_LAST_ENTRY	GENMASK_U64(63, 55)
+
+static __init u64 to_hpa_list_info(struct page *hpa_list_page,
+				   unsigned int nr_pages)
+{
+	return FIELD_PREP(HPA_LIST_INFO_FIRST_ENTRY, 0) |
+	       FIELD_PREP(HPA_LIST_INFO_PFN, page_to_pfn(hpa_list_page)) |
+	       FIELD_PREP(HPA_LIST_INFO_LAST_ENTRY, nr_pages - 1);
+}
+
+static __init int tdx_ext_mem_add(struct page *hpa_list_page,
+				  unsigned int nr_pages)
+{
+	struct tdx_module_args args = {

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1195

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+		.rcx = to_hpa_list_info(hpa_list_page, nr_pages),
+	};
+	u64 r;
+
+	do {
+		/*
+		 * TDH_EXT_MEM_ADD is designed to use output parameter RCX to
+		 * override/update input parameter RCX, so the caller doesn't
+		 * have to do manual parameter update on retry call.
+		 */

I am not sure about the "manual parameter update" part.

how about:
/*
* The TDX module overwrites RCX to track progress when
* this SEAMCALL is interrupted. Use seamcall_ret() to
* save and pass the updated value back on retry. */

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1200

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+		r = seamcall_ret(TDH_EXT_MEM_ADD, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

why -EFAULT?

In sc_retry_prerr(), most SEAMCALL errors are mapped to EIO. Maybe we should use EIO here.

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+
+	return 0;
+}
+
+struct tdx_hpa_list {
+	u64 phys[PAGE_SIZE / sizeof(u64)];
+};
+
+static_assert(sizeof(struct tdx_hpa_list) == PAGE_SIZE);
+
+static __init int tdx_ext_mem_setup(unsigned int required_pages)

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1221

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+{
+	struct tdx_hpa_list *hpa_list;
+	struct page *page;
+	unsigned int i;
+	int ret;
+
+	/*
+	 * memory_pool_required_pages == 0 means no need to add pages,
+	 * skip the memory setup.
+	 */

There is no "memory_pool_required_pages" here.

Quoted 20 lines
arch/x86/virt/vmx/tdx/tdx.c +20 -0
+	if (!required_pages)
+		return 0;
+
+	hpa_list = kzalloc_obj(*hpa_list);
+	if (!hpa_list)
+		return -ENOMEM;
+
+	page = alloc_contig_pages(required_pages, GFP_KERNEL, numa_mem_id(),
+				  &node_online_map);
+	if (!page) {
+		ret = -ENOMEM;
+		goto out_free_hpa_list;
+	}
+
+	i = 0;
+	while (i < required_pages) {
+		unsigned int nents = min(required_pages - i,
+					 ARRAY_SIZE(hpa_list->phys));
+		unsigned int j;
+

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1251

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+		for (j = 0; j < nents; j++)
+			hpa_list->phys[j] = page_to_phys(page + i + j);
+
+		ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
+		/*
+		 * No SEAMCALLs to reclaim the added pages. For simple error
+		 * handling, leak all pages.
+		 */
+		WARN(ret, "Fatal: TDX module rejected (%d) memory for extensions, stranded all pages\n",
+		     ret);

Printing 'ret' is useless since it's always -EFAULT.

The real reason to WARN here isn't "no SEAMCALL to reclaim". It is "this SEAMCALL shouldn't fail, and if it does, things are broken enough that complex error handling isn't worth it".

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1262

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+		if (ret)
+			break;
+
+		i += nents;
+	}
+
+	/*
+	 * Memory for extensions can't be reclaimed once added, print out the
+	 * amount, stop tracking it and free the hpa_list page, no matter
+	 * success or failure.
+	 */

This doesn't explain why it should be print. How about:

	/*
	 * Memory for TDX module extensions is never reclaimed and can be
	 * tens of megabytes. Print the amount so users know the cost.
	 */
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +8 -0
+	pr_info("%lu KB consumed for TDX module extensions\n",
+		required_pages * PAGE_SIZE / 1024);
+
+out_free_hpa_list:
+	kfree(hpa_list);
+
+	return ret;
+}
author
#35 lore raw

Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions

Quoted 5 lines
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+static __init int tdx_ext_mem_setup(unsigned int required_pages)
+{
+	struct tdx_hpa_list *hpa_list;
+	struct page *page;
+	unsigned int i;
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1221

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +6 -0
+	int ret;
+
+	/*
+	 * memory_pool_required_pages == 0 means no need to add pages,
+	 * skip the memory setup.
+	 */

There is no "memory_pool_required_pages" here.

OK.

BTW, sashiko says the comment restates what the if-statement. So I change to:

/*
* This required_pages comes from the metadata
* memory_pool_required_pages. TDX module uses it to indicate how much
* memory is still needed. This value decreases each time memory is
* added via TDH.EXT.MEM.ADD.
*
* On first time initialization, a value of 0 before any memory is
* added is unusual. But host makes no assumptions. Skip the memory
* setup and let subsequent steps catch any actual errors. */

Quoted 3 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1223

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	if (!required_pages)
+		return 0;

Other comments are all good, included.

#36 lore raw

Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions

In reply to Xu Yilun
Quoted 3 lines · Xu Yilun

TDX module extensions receive a one-time memory allocation at initialization time. The extensions use this memory as the baseline for their internal states and data required by the service APIs they offer.

Not sure what "baseline" means here. It is used for their internal state. So maybe just remove baseline. Unless it's important to clarify that it will need other memory later.

Quoted 10 lines

Add a new memory feeding process backed by a new SEAMCALL TDH.EXT.MEM.ADD. The process is mostly the same as adding PAMT. The kernel queries TDX module how much memory needed by reading the memory_pool_required_pages, allocates it, hands it over to the module, and never gets it back.

Quoted 7 lines

TDH.EXT.MEM.ADD uses a new parameter type, HPA_LIST_INFO, to provide this memory. This type represents a list of pages for TDX module to access. It references an 'hpa_list page' which contains the list of target HPAs. It collapses the HPA of the hpa_list page and the number of valid target HPAs into a 64 bit raw value for SEAMCALL parameters. The hpa_list page is always a medium, TDX module never keeps the hpa_list page.

I think you are going to need to put the basic concepts up front. Don't lead with the acronyms. The TDX module needs to get a list of pages. It takes them in a PFN array that is encoded like (...). Create a helper for this, and name it the same after the TDX module term for this format, HPA_LIST_INFO.

Basically there are no exotic concepts here. So you can explain it in normal kernel terms.

Quoted 6 lines

Don't CLFLUSH the pages handed to the TDX module, as is done for some other SEAMCALLs. The flushing operation is not expected to be needed for current and known future architectures. As more and more page feeding interfaces to come, the conservative flushing operation becomes a maintenance burden.

I think you can probably drop this paragraph.

Quoted 2 lines

For now,

Do we need the "for now"? Just adds more to think about and Im not sure it helps justify the patch.

Quoted 6 lines

TDX module extensions consume tens of megabytes memory that
will never be returned to host. Use contiguous page allocation to isolate these large blocks entirely, avoiding permanent memory fragmentation and reducing buddy allocator efficiency. Print the allocation amount on TDX module extensions initialization for visibility.

Hmm. I'm not sure the user will care about the specific category of extension used memory.

Quoted 73 lines

---
 arch/x86/include/asm/tdx_global_metadata.h  |   1 + arch/x86/virt/vmx/tdx/tdx.h                 |   1 + arch/x86/virt/vmx/tdx/tdx.c                 | 107 +++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx_global_metadata.c |   6 ++

 4 files changed, 112 insertions(+), 3 deletions(-)
arch/x86/include/asm/tdx_global_metadata.h +0 -0
index 83fc657a438e..b3442b7c88bb 100644
@@ -53,6 +53,7 @@ struct tdx_sys_info {
  };


 struct tdx_sys_info_ext {

arch/x86/include/asm/tdx_global_metadata.h +1 -0
+	u32 memory_pool_required_pages;

  bool ext_required;
 };

arch/x86/virt/vmx/tdx/tdx.h +1 -0
index a47e872480c7..a100634087e7 100644
@@ -63,6 +63,7 @@
  #define TDH_SYS_SHUTDOWN		52
  #define TDH_SYS_UPDATE_V0		53
  #define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0,
 1)
+#define TDH_EXT_MEM_ADD			61
  #define TDH_SYS_DISABLE			69


 /* TDX page types */

arch/x86/virt/vmx/tdx/tdx.c +1 -0
index 6f3596f11d25..dab17822c1c6 100644
@@ -31,6 +31,7 @@
  #include <linux/syscore_ops.h>
  #include <linux/idr.h>
  #include <linux/kvm_types.h>
+#include <linux/bitfield.h>
  #include <asm/page.h>
  #include <asm/special_insns.h>
  #include <asm/msr-index.h>
@@ -1166,6 +1167,108 @@ static __init int init_tdmrs(struct tdmr_info_list
 *tdmr_list)
  	return 0;
  }

arch/x86/virt/vmx/tdx/tdx.c +21 -0
+#define HPA_LIST_INFO_FIRST_ENTRY	GENMASK_U64(11, 3)
+#define HPA_LIST_INFO_PFN		GENMASK_U64(51, 12)
+#define HPA_LIST_INFO_LAST_ENTRY	GENMASK_U64(63, 55)
+
+static __init u64 to_hpa_list_info(struct page *hpa_list_page,
+				   unsigned int nr_pages)
+{
+	return FIELD_PREP(HPA_LIST_INFO_FIRST_ENTRY, 0) |
+	       FIELD_PREP(HPA_LIST_INFO_PFN, page_to_pfn(hpa_list_page)) |
+	       FIELD_PREP(HPA_LIST_INFO_LAST_ENTRY, nr_pages - 1);
+}
+
+static __init int tdx_ext_mem_add(struct page *hpa_list_page,
+				  unsigned int nr_pages)
+{
+	struct tdx_module_args args = {
+		.rcx = to_hpa_list_info(hpa_list_page, nr_pages),
+	};
+	u64 r;
+
+	do {

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1200

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+		/*
+		 * TDH_EXT_MEM_ADD is designed to use output parameter RCX to
+		 * override/update input parameter RCX, so the caller doesn't
+		 * have to do manual parameter update on retry call.
+		 */
+		r = seamcall_ret(TDH_EXT_MEM_ADD, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

Agree with Chao.

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1209

Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +9 -0
+
+	return 0;
+}
+
+struct tdx_hpa_list {
+	u64 phys[PAGE_SIZE / sizeof(u64)];
+};
+
+static_assert(sizeof(struct tdx_hpa_list) == PAGE_SIZE);

What kind of change are we guarding against here?

Quoted 20 lines
arch/x86/virt/vmx/tdx/tdx.c +20 -0
+
+static __init int tdx_ext_mem_setup(unsigned int required_pages)
+{
+	struct tdx_hpa_list *hpa_list;
+	struct page *page;
+	unsigned int i;
+	int ret;
+
+	/*
+	 * memory_pool_required_pages == 0 means no need to add pages,
+	 * skip the memory setup.
+	 */
+	if (!required_pages)
+		return 0;
+
+	hpa_list = kzalloc_obj(*hpa_list);
+	if (!hpa_list)
+		return -ENOMEM;
+
+	page = alloc_contig_pages(required_pages, GFP_KERNEL, numa_mem_id(),

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1239

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+				  &node_online_map);
+	if (!page) {
+		ret = -ENOMEM;
+		goto out_free_hpa_list;
+	}
+
+	i = 0;
+	while (i < required_pages) {
+		unsigned int nents = min(required_pages - i,
+					 ARRAY_SIZE(hpa_list->phys));

I think this whole loop could be made clearer. nents is the number of pages we will add for an execution of the seamcall. That is not clear enough either.

Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+		unsigned int j;
+
+		for (j = 0; j < nents; j++)
+			hpa_list->phys[j] = page_to_phys(page + i + j);
+
+		ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
+		/*
+		 * No SEAMCALLs to reclaim the added pages. For simple error
+		 * handling, leak all pages.
+		 */
+		WARN(ret, "Fatal: TDX module rejected (%d) memory for

extensions, stranded all pages\n",

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+		     ret);
+		if (ret)
+			break;

Can we combine the ret checks?

arch/x86/virt/vmx/tdx/tdx.c

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+
+		i += nents;

Maybe a full name for i?

Quoted 49 lines
arch/x86/virt/vmx/tdx/tdx.c +16 -0
+	}
+
+	/*
+	 * Memory for extensions can't be reclaimed once added, print out the
+	 * amount, stop tracking it and free the hpa_list page, no matter
+	 * success or failure.
+	 */
+	pr_info("%lu KB consumed for TDX module extensions\n",
+		required_pages * PAGE_SIZE / 1024);
+
+out_free_hpa_list:
+	kfree(hpa_list);
+
+	return ret;
+}
+

 static __init int init_tdx_module_extensions(void) {
  struct tdx_sys_info_ext sysinfo_ext;

arch/x86/virt/vmx/tdx/tdx.c +0 -0
@@ -1182,9 +1285,7 @@ static __init int init_tdx_module_extensions(void)
  	if (!sysinfo_ext.ext_required)
  		return 0;

arch/x86/virt/vmx/tdx/tdx.c +1 -3
-	/* TODO: add the extensions enabling steps here */
-
-	return 0;
+	return tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);

 }

 static __init int init_tdx_module(void)

arch/x86/virt/vmx/tdx/tdx_global_metadata.c +0 -0
index b9e1c011a990..720cdaf76492 100644
@@ -137,6 +137,12 @@ static __init int get_tdx_sys_info_ext(struct
 tdx_sys_info_ext *sysinfo_ext)
  	int ret;
  	u64 val;

arch/x86/virt/vmx/tdx/tdx_global_metadata.c +6 -0
+	ret = read_sys_metadata_field(0x3100000200000000, &val);
+	if (ret)
+		return ret;
+
+	sysinfo_ext->memory_pool_required_pages = val;
+

  ret = read_sys_metadata_field(0x3100000000000001, &val); if (ret)
  return ret;

author
#37 lore raw

Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions

Quoted 8 lines
Quoted 7 lines

TDH.EXT.MEM.ADD uses a new parameter type, HPA_LIST_INFO, to provide this memory. This type represents a list of pages for TDX module to access. It references an 'hpa_list page' which contains the list of target HPAs. It collapses the HPA of the hpa_list page and the number of valid target HPAs into a 64 bit raw value for SEAMCALL parameters. The hpa_list page is always a medium, TDX module never keeps the hpa_list page.

Quoted 7 lines

I think you are going to need to put the basic concepts up front. Don't lead with the acronyms. The TDX module needs to get a list of pages. It takes them in a PFN array that is encoded like (...). Create a helper for this, and name it the same after the TDX module term for this format, HPA_LIST_INFO.

Basically there are no exotic concepts here. So you can explain it in normal kernel terms.

OK. My refactored version:

When adding the memory, the TDX module needs to get a list of pages. It accepts these target pages in a PFN array that is encoded into a single 64-bit SEAMCALL parameter: the PFN of a container page that holds the PFN array, and the number of the pages packed into a single value. Create a helper to encode this format and name it after the TDX module term: HPA_LIST_INFO.

[...]

Quoted 9 lines
Quoted 6 lines

TDX module extensions consume tens of megabytes memory that
will never be returned to host. Use contiguous page allocation to isolate these large blocks entirely, avoiding permanent memory fragmentation and reducing buddy allocator efficiency. Print the allocation amount on TDX module extensions initialization for visibility.

Hmm. I'm not sure the user will care about the specific category of extension used memory.

Not sure if you suggest removing the print, but this is suggested by Dave:

https://lore.kernel.org/all/[email protected]/

[...]

Quoted 7 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1209

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5 -0
+struct tdx_hpa_list {
+	u64 phys[PAGE_SIZE / sizeof(u64)];
+};
+
+static_assert(sizeof(struct tdx_hpa_list) == PAGE_SIZE);

What kind of change are we guarding against here?

I don't expect changes of the format, but this is to ensure the container of the PFN array is a page, and also serves as the documentation.

[...]

Quoted 17 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1239

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+	i = 0;
+	while (i < required_pages) {
+		unsigned int nents = min(required_pages - i,
+					 ARRAY_SIZE(hpa_list->phys));

I think this whole loop could be made clearer. nents is the number of pages we will add for an execution of the seamcall. That is not clear enough either.

Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +9 -0
+		unsigned int j;
+
+		for (j = 0; j < nents; j++)
+			hpa_list->phys[j] = page_to_phys(page + i + j);
+
+		ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
+		/*
+		 * No SEAMCALLs to reclaim the added pages. For simple error
+		 * handling, leak all pages.
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253

Quoted 6 lines
(quoted diff) +2 -0
+		 */
+		WARN(ret, "Fatal: TDX module rejected (%d) memory for

extensions, stranded all pages\n",

arch/x86/virt/vmx/tdx/tdx.c +3 -0
+		     ret);
+		if (ret)
+			break;

Can we combine the ret checks?

Yes.

Quoted 5 lines

Review

Quoted 3 lines
(quoted diff) +2 -0
+
+		i += nents;

Maybe a full name for i?

Yes, I changed the name of i & nents.

author
#38 lore raw

PATCH v2 5/17 x86/virt/tdx: Make TDX module initialize the extensions

In reply to Xu Yilun

After providing all required memory to TDX module, initialize TDX module extensions via TDH.EXT.INIT, so extension SEAMCALLs can be used.

---
 arch/x86/virt/vmx/tdx/tdx.h |  1 +
 arch/x86/virt/vmx/tdx/tdx.c | 22 +++++++++++++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.h +1 -0
index a100634087e7..2deb0a5c902e 100644
@@ -63,6 +63,7 @@
 #define TDH_SYS_SHUTDOWN		52
 #define TDH_SYS_UPDATE_V0		53
 #define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)
+#define TDH_EXT_INIT			60
 #define TDH_EXT_MEM_ADD			61
 #define TDH_SYS_DISABLE			69
 
arch/x86/virt/vmx/tdx/tdx.c +21 -1
index dab17822c1c6..900928de373a 100644
@@ -1167,6 +1167,22 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+/* Initialize TDX module extensions for extension SEAMCALLs */
+static __init int tdx_ext_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_EXT_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;
+
+	return 0;
+}
+
 #define HPA_LIST_INFO_FIRST_ENTRY	GENMASK_U64(11, 3)
 #define HPA_LIST_INFO_PFN		GENMASK_U64(51, 12)
 #define HPA_LIST_INFO_LAST_ENTRY	GENMASK_U64(63, 55)
@@ -1285,7 +1301,11 @@ static __init int init_tdx_module_extensions(void)
 	if (!sysinfo_ext.ext_required)
 		return 0;
 
-	return tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
+	ret = tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
+	if (ret)
+		return ret;
+
+	return tdx_ext_init();
 }
 
 static __init int init_tdx_module(void)
Signature · 1 line
2.25.1
#39 lore raw

Re: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Medium] tdx_ext_init() (added in this patch) and tdx_ext_mem_add() silently discard the 64-bit hardware error code from their SEAMCALLs and return -EFAULT without logging.

Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but by the earlier commit 'x86/virt/tdx: Add extra memory to TDX module for the extensions'. The hpa_list structure is allocated via kzalloc_obj() which does not guarantee page-alignment. --

commit 555dbf36da39abb049edf2bd1291c474f12aac77
Author: Xu Yilun <[email protected]>

x86/virt/tdx: Make TDX module initialize the extensions

After providing all required memory to TDX module, initialize TDX module extensions via TDH.EXT.INIT, so extension SEAMCALLs can be used.

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index dab17822c1c69..900928de373ae 100644

[ ... ]

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
@@ -1167,6 +1167,22 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+/* Initialize TDX module extensions for extension SEAMCALLs */
+static __init int tdx_ext_init(void)

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1181

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_EXT_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

[Severity: Medium]
Does this code silently discard the 64-bit hardware error code?

When TDH_EXT_INIT fails, replacing the return value with -EFAULT means the caller only logs a generic initialization failure without details. Other TDX initialization functions use seamcall_prerr_ret() to preserve and log the hardware error.

Could this be updated to use seamcall_prerr_ret() to ensure the exact firmware failure reason is logged?

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+
+	return 0;
+}
+

[ ... ]

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1304

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +1 -1
@@ -1285,7 +1301,11 @@ static __init int init_tdx_module_extensions(void)
 	if (!sysinfo_ext.ext_required)
 		return 0;
 
-	return tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
+	ret = tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);

[Severity: High]
This isn't a bug introduced by this patch, but does the earlier introduction of tdx_ext_mem_setup() cause an alignment issue?

Looking at tdx_ext_mem_setup() in arch/x86/virt/vmx/tdx/tdx.c:

tdx_ext_mem_setup() {
    ...
    hpa_list = kzalloc_obj(*hpa_list);
    if (!hpa_list)
        return -ENOMEM;
    ...
    ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
    ...
}

The TDX module's TDH.EXT.MEM.ADD leaf expects a page-aligned PFN for the HPA list. Since kmalloc does not guarantee page-alignment (especially if CONFIG_SLUB_DEBUG is enabled), virt_to_page() drops the unaligned offset.

Will this result in the TDX module reading from offset 0 of the page, potentially consuming SLUB metadata instead of the actual hpa_list entries?

Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+	if (ret)
+		return ret;
+
+	return tdx_ext_init();
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
author
#40 lore raw

Re: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions

Quoted 16 lines

Review

Quoted 12 lines
(quoted diff) +12 -0
+/* Initialize TDX module extensions for extension SEAMCALLs */
+static __init int tdx_ext_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_EXT_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

[Severity: Medium]
Does this code silently discard the 64-bit hardware error code?

Quoted 7 lines

When TDH_EXT_INIT fails, replacing the return value with -EFAULT means the caller only logs a generic initialization failure without details. Other TDX initialization functions use seamcall_prerr_ret() to preserve and log the hardware error.

Could this be updated to use seamcall_prerr_ret() to ensure the exact firmware failure reason is logged?

No, it can't use seamcall_prerr_ret() directly. TDX_INTERRUPTED_RESUMABLE can't be handled in a unified way for all SEAMCALLs so can't embed in fundamental seamcall wrappers. I see 2 conventions:

1. Host should set a resume=1 parameter on retry, such as, TDH_PHYMEM_CACHE_WB, and several migration SEAMCALLs, TDH.EXPORT.MEM, TDH.EXPORT.STATE.XXX

2. Host just refills the previous args (may have been modified by seamcall output) on retry. Much more examples, TDH_EXT_INIT, TDH_EXT_MEM_ADD, TDH_QUOTE_INIT, TDH_QUOTE_GET...

So I think there is a need the callers have to:

1. call seamcall() or seamcall wrappers (tdh_xxx)
2. specific handling of raw SEAMCALL return code
3. do a standard TDX error logging and Linux errno conversion.

seamcall_prerr_ret() does 1 & 2 & 3, but can't customize step 2.

I tried to add some TDX error logging & errno conversion helpers, most of the code are abstracted from seamcall_prerr_ret(), see below.

But I'm not sure if it worths the churn.

----8<----

arch/x86/virt/vmx/tdx/seamcall_internal.h +18 -5
index be5f446467df..f838715c9446 100644
@@ -78,12 +78,10 @@ static inline void seamcall_err_ret(u64 fn, u64 err,
                        args->r9, args->r10, args->r11);
 }
 
-static __always_inline int sc_retry_prerr(sc_func_t func,
-                                         sc_err_func_t err_func,
-                                         u64 fn, struct tdx_module_args *args)
+static inline int sc_errno_prerr(sc_err_func_t err_func,
+                                u64 fn, u64 sret,
+                                struct tdx_module_args *args)
 {
-       u64 sret = sc_retry(func, fn, args);
-
        if (sret == TDX_SUCCESS)
                return 0;
 
@@ -100,10 +98,25 @@ static __always_inline int sc_retry_prerr(sc_func_t func,
        return -EIO;
 }
 
+static __always_inline int sc_retry_prerr(sc_func_t func,
+                                         sc_err_func_t err_func,
+                                         u64 fn, struct tdx_module_args *args)
+{
+       u64 sret = sc_retry(func, fn, args);
+
+       return sc_errno_prerr(err_func, fn, sret, args);
+}
+
 #define seamcall_prerr(__fn, __args)                                           \
        sc_retry_prerr(__seamcall, seamcall_err, (__fn), (__args))
 
 #define seamcall_prerr_ret(__fn, __args)                                       \
        sc_retry_prerr(__seamcall_ret, seamcall_err_ret, (__fn), (__args))
 
+#define seamcall_errno_prerr(__fn, __sret, __args)                             \
+       sc_errno_prerr(seamcall_err, (__fn), (__sret), (__args))
+
+#define seamcall_errno_prerr_ret(__fn, __sret, __args)                         \
+       sc_errno_prerr(seamcall_err_ret, (__fn), (__sret), (__args))
+
 #endif /* _X86_VIRT_SEAMCALL_INTERNAL_H */
arch/x86/virt/vmx/tdx/tdx.c +2 -8
index 826504fbcdc7..637ff01ed0f1 100644
@@ -1403,10 +1403,7 @@ static int tdx_ext_init(void)
                r = seamcall(TDH_EXT_INIT, &args);
        } while (r == TDX_INTERRUPTED_RESUMABLE);
 
-       if (r != TDX_SUCCESS)
-               return -EFAULT;
-
-       return 0;
+       return seamcall_errno_prerr(TDH_EXT_INIT, r, &args);
 }
 
 #define HPA_LIST_INFO_FIRST_ENTRY      GENMASK_U64(11, 3)
@@ -1438,10 +1435,7 @@ static __init int tdx_ext_mem_add(struct page *hpa_list_page,
                r = seamcall_ret(TDH_EXT_MEM_ADD, &args);
        } while (r == TDX_INTERRUPTED_RESUMABLE);
 
-       if (r != TDX_SUCCESS)
-               return -EFAULT;
-
-       return 0;
+       return seamcall_errno_prerr_ret(TDH_EXT_MEM_ADD, r, &args);
 }
#41 lore raw

Re: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions

Quoted 4 lines · Xu Yilun
Quoted 4 lines
Quoted 4 lines
(quoted diff) +4 -0
+/* Initialize TDX module extensions for extension SEAMCALLs */
+static __init int tdx_ext_init(void)
+{
+	struct tdx_module_args args = {};
Quoted 8 lines
Quoted 8 lines

Review

Quoted 8 lines
(quoted diff) +8 -0
+	u64 r;
+
+	do {
+		r = seamcall(TDH_EXT_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

As Chao caught in patch 04, we need to return -EIO here for consistence.

Quoted 30 lines
Quoted 11 lines

[Severity: Medium]
Does this code silently discard the 64-bit hardware error code?

When TDH_EXT_INIT fails, replacing the return value with -EFAULT means the caller only logs a generic initialization failure without details. Other TDX initialization functions use seamcall_prerr_ret() to preserve and log the hardware error.

Could this be updated to use seamcall_prerr_ret() to ensure the exact firmware failure reason is logged?

No, it can't use seamcall_prerr_ret() directly. TDX_INTERRUPTED_RESUMABLE can't be handled in a unified way for all SEAMCALLs so can't embed in fundamental seamcall wrappers. I see 2 conventions:

1. Host should set a resume=1 parameter on retry, such as, TDH_PHYMEM_CACHE_WB, and several migration SEAMCALLs, TDH.EXPORT.MEM, TDH.EXPORT.STATE.XXX

2. Host just refills the previous args (may have been modified by seamcall output) on retry. Much more examples, TDH_EXT_INIT, TDH_EXT_MEM_ADD, TDH_QUOTE_INIT, TDH_QUOTE_GET...

So I think there is a need the callers have to:

1. call seamcall() or seamcall wrappers (tdh_xxx)
2. specific handling of raw SEAMCALL return code
3. do a standard TDX error logging and Linux errno conversion.

Quoted 6 lines

seamcall_prerr_ret() does 1 & 2 & 3, but can't customize step 2.

I tried to add some TDX error logging & errno conversion helpers, most of the code are abstracted from seamcall_prerr_ret(), see below.

But I'm not sure if it worths the churn.

Though I have some comment, below refactor doesn't look so bad.

The only risk is existing SEAMCALL warppers is a mess and people may not want to add any more before re-work them. So the conservative option would to just add an ad-hoc pr_err() for !TDX_SUCCESS case?

Quoted 37 lines

----8<----

arch/x86/virt/vmx/tdx/seamcall_internal.h +12 -5
index be5f446467df..f838715c9446 100644
@@ -78,12 +78,10 @@ static inline void seamcall_err_ret(u64 fn, u64 err,
                         args->r9, args->r10, args->r11);
  }
 
-static __always_inline int sc_retry_prerr(sc_func_t func,
-                                         sc_err_func_t err_func,
-                                         u64 fn, struct tdx_module_args *args)
+static inline int sc_errno_prerr(sc_err_func_t err_func,
+                                u64 fn, u64 sret,
+                                struct tdx_module_args *args)
  {
-       u64 sret = sc_retry(func, fn, args);
-
         if (sret == TDX_SUCCESS)
                 return 0;
 
@@ -100,10 +98,25 @@ static __always_inline int sc_retry_prerr(sc_func_t func,
         return -EIO;
  }
 
+static __always_inline int sc_retry_prerr(sc_func_t func,
+                                         sc_err_func_t err_func,
+                                         u64 fn, struct tdx_module_args *args)
+{
+       u64 sret = sc_retry(func, fn, args);
+
+       return sc_errno_prerr(err_func, fn, sret, args);
+}
+
  #define seamcall_prerr(__fn, __args)                                           \

arch/x86/virt/vmx/tdx/seamcall_internal.h static __always_inline int sc_retry_prerr(sc_func_t func, · line 120

Quoted 10 lines
arch/x86/virt/vmx/tdx/seamcall_internal.h +5 -0
         sc_retry_prerr(__seamcall, seamcall_err, (__fn), (__args))
 
  #define seamcall_prerr_ret(__fn, __args)                                       \
         sc_retry_prerr(__seamcall_ret, seamcall_err_ret, (__fn), (__args))
 
+#define seamcall_errno_prerr(__fn, __sret, __args)                             \
+       sc_errno_prerr(seamcall_err, (__fn), (__sret), (__args))
+
+#define seamcall_errno_prerr_ret(__fn, __sret, __args)                         \
+       sc_errno_prerr(seamcall_err_ret, (__fn), (__sret), (__args))

We don't need these two wrappers.

The existing seamcall_prerr() and seamcall_prerr_ret() do two things:
1. call SEAMCALL;
2. handling the return code of SEAMCALL;

But the 2 new wrappers only do setp 2. So we can just call sc_errno_prerr() in the following cases ...

Quoted 6 lines
arch/x86/virt/vmx/tdx/seamcall_internal.h +0 -0
  #endif /* _X86_VIRT_SEAMCALL_INTERNAL_H */
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 826504fbcdc7..637ff01ed0f1 100644

arch/x86/virt/vmx/tdx/tdx.c static int tdx_ext_init(void) · line 1406

Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +1 -4
@@ -1403,10 +1403,7 @@ static int tdx_ext_init(void)
                 r = seamcall(TDH_EXT_INIT, &args);
         } while (r == TDX_INTERRUPTED_RESUMABLE);
 
-       if (r != TDX_SUCCESS)
-               return -EFAULT;
-
-       return 0;
+       return seamcall_errno_prerr(TDH_EXT_INIT, r, &args);

... return sc_errno_prerr(seamcall_err, TDH_EXT_INIT, r, &args);

and

arch/x86/virt/vmx/tdx/tdx.c static __init int tdx_ext_mem_add(struct page *hpa_list_page, · line 1438

Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +1 -4
  }
 
  #define HPA_LIST_INFO_FIRST_ENTRY      GENMASK_U64(11, 3)
@@ -1438,10 +1435,7 @@ static __init int tdx_ext_mem_add(struct page *hpa_list_page,
                 r = seamcall_ret(TDH_EXT_MEM_ADD, &args);
         } while (r == TDX_INTERRUPTED_RESUMABLE);
 
-       if (r != TDX_SUCCESS)
-               return -EFAULT;
-
-       return 0;
+       return seamcall_errno_prerr_ret(TDH_EXT_MEM_ADD, r, &args);

... return sc_errno_prerr(seamcall_err_ret, TDH_EXT_MEM_ADD, r, &args);

Quoted 1 line
arch/x86/virt/vmx/tdx/tdx.c +0 -0
  }
#42 lore raw

Re: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions

In reply to Xu Yilun
Quoted 2 lines · Xu Yilun

After providing all required memory to TDX module, initialize TDX module extensions via TDH.EXT.INIT, so extension SEAMCALLs can be used.

Pretty thin. Maybe some info that it is (apparently) a global init. Probably some very short background on what extensions are extension seamcalls are.

Quoted 64 lines

---
 arch/x86/virt/vmx/tdx/tdx.h |  1 + arch/x86/virt/vmx/tdx/tdx.c | 22 +++++++++++++++++++++-

 2 files changed, 22 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.h +1 -0
index a100634087e7..2deb0a5c902e 100644
@@ -63,6 +63,7 @@
  #define TDH_SYS_SHUTDOWN		52
  #define TDH_SYS_UPDATE_V0		53
  #define TDH_SYS_UPDATE			SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0,
 1)
+#define TDH_EXT_INIT			60
  #define TDH_EXT_MEM_ADD			61
  #define TDH_SYS_DISABLE			69

arch/x86/virt/vmx/tdx/tdx.c +0 -0
index dab17822c1c6..900928de373a 100644
@@ -1167,6 +1167,22 @@ static __init int init_tdmrs(struct tdmr_info_list
 *tdmr_list)
  	return 0;
  }

arch/x86/virt/vmx/tdx/tdx.c +16 -0
+/* Initialize TDX module extensions for extension SEAMCALLs */
+static __init int tdx_ext_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_EXT_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;
+
+	return 0;
+}
+

 #define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3)
 #define HPA_LIST_INFO_PFN GENMASK_U64(51, 12)
 #define HPA_LIST_INFO_LAST_ENTRY GENMASK_U64(63, 55)

arch/x86/virt/vmx/tdx/tdx.c +0 -0
@@ -1285,7 +1301,11 @@ static __init int init_tdx_module_extensions(void)
  	if (!sysinfo_ext.ext_required)
  		return 0;

arch/x86/virt/vmx/tdx/tdx.c +5 -1
-	return tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
+	ret = tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
+	if (ret)
+		return ret;
+
+	return tdx_ext_init();

 }

 static __init int init_tdx_module(void)

author
#43 lore raw

PATCH v2 6/17 x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

In reply to Xu Yilun

Runtime TDX module update introduces a mechanism to update the module firmware while preserving and restoring TDX operations. As part of the restoration process, the host must re-execute all extensions initialization steps to restore extension SEAMCALL functionality.

However, Linux runs the update in stop_machine() context, which prevents memory allocation. This introduces a hard restriction that the updated TDX environment must not consume more memory for the extensions. Consequently, the post-update initialization for the extensions is implemented as:

1. Detect if the extensions are supported and required.
2. Detect if the extensions require additional memory. If yes, fail the update.
3. Initialize the extensions via TDH.EXT.INIT.

The memory allocation problem is greatly mitigated since Linux applies a policy that configures the same add-on features for boot and for update. This policy minimizes the chance of increased memory demand. So now the restriction only affects the compatibility rule for choosing the update image.

---
 arch/x86/virt/vmx/tdx/tdx.c                 | 31 ++++++++++++++++++++-
 arch/x86/virt/vmx/tdx/tdx_global_metadata.c |  2 +-
 2 files changed, 31 insertions(+), 2 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +30 -1
index 900928de373a..4d2940f4538a 100644
@@ -1168,7 +1168,7 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 }
 
 /* Initialize TDX module extensions for extension SEAMCALLs */
-static __init int tdx_ext_init(void)
+static int tdx_ext_init(void)
 {
 	struct tdx_module_args args = {};
 	u64 r;
@@ -1308,6 +1308,31 @@ static __init int init_tdx_module_extensions(void)
 	return tdx_ext_init();
 }
 
+/*
+ * Mostly the same flow as init_tdx_module_extensions(), but rejects adding
+ * more memory.
+ */
+static int update_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	if (!sysinfo_ext.ext_required)
+		return 0;
+
+	if (sysinfo_ext.memory_pool_required_pages)
+		return -EFAULT;
+
+	return tdx_ext_init();
+}
+
 static __init int init_tdx_module(void)
 {
 	int ret;
@@ -1498,6 +1523,10 @@ int tdx_module_run_update(void)
 	 */
 	WARN_ON_ONCE(ret);
 
+	ret = update_tdx_module_extensions();
+	if (ret)
+		return ret;
+
 	tdx_module_state.initialized = true;
 	return 0;
 }
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +1 -1
index 720cdaf76492..84364da89649 100644
@@ -132,7 +132,7 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
 	return ret;
 }
 
-static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
+static int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
 {
 	int ret;
 	u64 val;
Signature · 1 line
2.25.1
#44 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [Critical] Infinite loop in tdx_ext_init() when an external interrupt becomes pending during stop_machine_cpuslocked().

Pre-existing issues:
- [High] Infinite loop in tdx_sys_disable() when an external interrupt becomes pending during a kernel panic or crash. --

commit 03507e2a23bf57aa355b6cf99d6b8107f41ea488
Author: Xu Yilun <[email protected]>

x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Runtime TDX module update introduces a mechanism to update the module firmware while preserving and restoring TDX operations. As part of the restoration process, the post-update initialization detects if extensions are supported, rejects updates requiring additional memory, and initializes the extensions via TDH.EXT.INIT.

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 900928de373ae..4d2940f4538a2 100644

[ ... ]

Quoted 17 lines
arch/x86/virt/vmx/tdx/tdx.c +13 -0
@@ -1308,6 +1308,31 @@ static __init int init_tdx_module_extensions(void)
 	return tdx_ext_init();
 }
 
+/*
+ * Mostly the same flow as init_tdx_module_extensions(), but rejects adding
+ * more memory.
+ */
+static int update_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1333

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	if (ret)
+		return ret;
+
+	if (!sysinfo_ext.ext_required)
+		return 0;
+
+	if (sysinfo_ext.memory_pool_required_pages)
+		return -EFAULT;
+
+	return tdx_ext_init();

[Severity: Critical]
Could this result in an infinite loop during a runtime update?

The tdx_module_run_update() function runs under stop_machine(), meaning local interrupts are explicitly disabled. When this calls tdx_ext_init(), it executes the following polling loop:

    do {
        r = seamcall(TDH_EXT_INIT, &args);
    } while (r == TDX_INTERRUPTED_RESUMABLE);

If an external hardware interrupt becomes pending during the update, the TDX module exits to the host and returns TDX_INTERRUPTED_RESUMABLE. Because the host CPU has interrupts disabled, it cannot service the interrupt and simply continues execution.

Does this mean the loop condition evaluates to true, immediately re-executing the seamcall, and permanently deadlocking the system?

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1337

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+}
+
 static __init int init_tdx_module(void)
 {

[ ... ]

[Severity: High]
This isn't a bug introduced by this patch, but while looking at how TDX_INTERRUPTED_RESUMABLE is handled, I noticed a similar pattern in tdx_sys_disable():

    do {
        ret = seamcall(TDH_SYS_DISABLE, &args);
    } while (ret == TDX_INTERRUPTED_RESUMABLE || ret == TDX_SYS_BUSY);

During a kernel panic or crash, machine_crash_shutdown() executes with local interrupts disabled. If an external interrupt becomes pending, the TDX module exits with TDX_INTERRUPTED_RESUMABLE.

With host interrupts permanently disabled in the panic context, will this loop endlessly and prevent the crash dump or kexec process from completing?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
author
#45 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Quoted 13 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1333

Quoted 1 line
arch/x86/virt/vmx/tdx/tdx.c +1 -0
+	return tdx_ext_init();

[Severity: Critical]
Could this result in an infinite loop during a runtime update?

The tdx_module_run_update() function runs under stop_machine(), meaning local interrupts are explicitly disabled. When this calls tdx_ext_init(), it executes the following polling loop:

    do {
        r = seamcall(TDH_EXT_INIT, &args);
    } while (r == TDX_INTERRUPTED_RESUMABLE);
Quoted 7 lines

If an external hardware interrupt becomes pending during the update, the TDX module exits to the host and returns TDX_INTERRUPTED_RESUMABLE. Because the host CPU has interrupts disabled, it cannot service the interrupt and simply continues execution.

Does this mean the loop condition evaluates to true, immediately re-executing the seamcall, and permanently deadlocking the system?

No, According to TDX module SPEC [1], 5.4.1. SEAMCALL Instruction (Common), RAX[bit 24] == 0 shows "Pending interrupts are detected only if the host VMM’s RFLAGS.IF is 1." Linux now doesn't set this bit for any SEAMCALL.

That means SEAMCALLs will keep on executing if host disabled interrupts so will not deadlock the system.

[1] https://cdrdv2.intel.com/v1/dl/getContent/795381

#46 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

In reply to Xu Yilun
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+/*
+ * Mostly the same flow as init_tdx_module_extensions(), but rejects adding
+ * more memory.
+ */
+static int update_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1331

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
+
+	if (!sysinfo_ext.ext_required)
+		return 0;
+
+	if (sysinfo_ext.memory_pool_required_pages)
+		return -EFAULT;

Will tdx_ext_init() return an error if more memory is needed?

If yes, we can leave this check to the module. And with ext_required removed (per my earlier comment), this function simplifies to:

int update_tdx_module_extensions(void)

{
	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
		return 0;

	return tdx_ext_init();
}
Quoted 34 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+
+	return tdx_ext_init();
+}
+

static __init int init_tdx_module(void)

{
	int ret;
arch/x86/virt/vmx/tdx/tdx.c +4 -0
@@ -1498,6 +1523,10 @@ int tdx_module_run_update(void)
 	 */
 	WARN_ON_ONCE(ret);
 
+	ret = update_tdx_module_extensions();
+	if (ret)
+		return ret;
+
 	tdx_module_state.initialized = true;
 	return 0;
 }
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +1 -1
index 720cdaf76492..84364da89649 100644
@@ -132,7 +132,7 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)
 	return ret;
 }
 
-static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
+static int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
 {
 	int ret;
 	u64 val;
Signature · 1 line
2.25.1
author
#47 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Quoted 15 lines · Chao Gao
Quoted 15 lines
arch/x86/virt/vmx/tdx/tdx.c +15 -0
+/*
+ * Mostly the same flow as init_tdx_module_extensions(), but rejects adding
+ * more memory.
+ */
+static int update_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1331

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +6 -0
+
+	if (!sysinfo_ext.ext_required)
+		return 0;
+
+	if (sysinfo_ext.memory_pool_required_pages)
+		return -EFAULT;

Will tdx_ext_init() return an error if more memory is needed?

Yes.

Quoted 2 lines

If yes, we can leave this check to the module.

Yes, we can. Although we can't remove ext_required check, we can remove memory_pool_required_pages check, making the function simpler.

#48 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

In reply to Chao Gao
Quoted 5 lines · Chao Gao

Will tdx_ext_init() return an error if more memory is needed?

If yes, we can leave this check to the module. And with ext_required removed (per my earlier comment), this function simplifies to:

Quoted 7 lines

int update_tdx_module_extensions(void)

{
	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
		return 0;

	return tdx_ext_init();
}

Oh, it can be simplified and not share so much with the init side. Please ignore my other question.

author
#49 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Quoted 11 lines · Rick P
Quoted 10 lines · Chao Gao

Will tdx_ext_init() return an error if more memory is needed?

If yes, we can leave this check to the module. And with ext_required removed (per my earlier comment), this function simplifies to:

int update_tdx_module_extensions(void)

{
	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
		return 0;
Quoted 5 lines
Quoted 2 lines
	return tdx_ext_init();
}

Oh, it can be simplified and not share so much with the init side. Please ignore my other question.

It can't be simplified like that. If extensions are supported but no extension feature is configured at boot up time. tdx_ext_init() would fail and the runtime update failed.

I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip tdx_ext_init() and the update succeed, but after that the extension features will break.

IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes. We should store the extension metadata in global tdx_sysinfo, and that eliminate the duplicate code.

----8<----

arch/x86/include/asm/tdx_global_metadata.h +6 -5
index b3442b7c88bb..43b8761c0854 100644
@@ -44,17 +44,18 @@ struct tdx_sys_info_handoff {
        u16 module_hv;
 };
 
+struct tdx_sys_info_ext {
+       u32 memory_pool_required_pages;
+       bool ext_required;
+};
+
 struct tdx_sys_info {
        struct tdx_sys_info_version version;
        struct tdx_sys_info_features features;
        struct tdx_sys_info_tdmr tdmr;
        struct tdx_sys_info_td_ctrl td_ctrl;
        struct tdx_sys_info_td_conf td_conf;
-};
-
-struct tdx_sys_info_ext {
-       u32 memory_pool_required_pages;
-       bool ext_required;
+       struct tdx_sys_info_ext ext;
 };
 
 #endif
arch/x86/virt/vmx/tdx/tdx.c +9 -18
index f03141f51840..90fbd22acb5e 100644
@@ -1304,13 +1304,13 @@ static int tdx_ext_init(void)
 
 static __init int init_tdx_module_extensions(void)
 {
-       struct tdx_sys_info_ext sysinfo_ext;
+       struct tdx_sys_info_ext *sysinfo_ext = &tdx_sysinfo.ext;
        int ret;
 
        if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
                return 0;
 
-       ret = get_tdx_sys_info_ext(&sysinfo_ext);
+       ret = get_tdx_sys_info_ext(sysinfo_ext);
        if (ret)
                return ret;
 
@@ -1319,10 +1319,10 @@ static __init int init_tdx_module_extensions(void)
         * extensions are configured via TDH.SYS.CONFIG. If none, skip the
         * initialization.
         */
-       if (!sysinfo_ext.ext_required)
+       if (!sysinfo_ext->ext_required)
                return 0;
 
-       ret = tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
+       ret = tdx_ext_mem_setup(sysinfo_ext->memory_pool_required_pages);
        if (ret)
                return ret;
 
@@ -1330,23 +1330,14 @@ static __init int init_tdx_module_extensions(void)
 }
 
 /*
- * Mostly the same flow as init_tdx_module_extensions(), but won't check and
- * add more memory. Let the extensions re-initialization fail when more memory
- * is actually needed.
+ * Don't update the extensions metadata, just follow the requirement originated
+ * on TDX module initialization. Let the extensions re-initialization fail when
+ * more memory is needed, or when the ext_required is accidentally flipped after
+ * update.
  */
 static int update_tdx_module_extensions(void)
 {
-       struct tdx_sys_info_ext sysinfo_ext;
-       int ret;
-
-       if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
-               return 0;
-
-       ret = get_tdx_sys_info_ext(&sysinfo_ext);
-       if (ret)
-               return ret;
-
-       if (!sysinfo_ext.ext_required)
+       if (!tdx_sysinfo.ext.ext_required)
                return 0;
 
        return tdx_ext_init();
author
#50 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Quoted 22 lines · Xu Yilun
Quoted 11 lines · Rick P
Quoted 10 lines · Chao Gao

Will tdx_ext_init() return an error if more memory is needed?

If yes, we can leave this check to the module. And with ext_required removed (per my earlier comment), this function simplifies to:

int update_tdx_module_extensions(void)

{
	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
		return 0;
Quoted 5 lines
Quoted 2 lines
	return tdx_ext_init();
}

Oh, it can be simplified and not share so much with the init side. Please ignore my other question.

It can't be simplified like that. If extensions are supported but no extension feature is configured at boot up time. tdx_ext_init() would fail and the runtime update failed.

Quoted 7 lines

I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip tdx_ext_init() and the update succeed, but after that the extension features will break.

IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes.

Sorry, I changed my mind. I remember TDX module architector said "the host should not assume any feature/SEAMCALL must be associated with any extension, it is hidden from the host. Today a SEAMCALL is backed by an extension, tomorrow it may not".

So in this case, ext_required may flip across updates and we should honor the runtime metadata.

I made another effort to combine common routines for ext init and update, but don't like how it turns out. If no one wants it that much, I'd rather stay unchanged:

----8<----

arch/x86/virt/vmx/tdx/tdx.c +12 -8
index cec266f76dc0..448d0ab76d05 100644
@@ -1302,15 +1302,22 @@ static int tdx_ext_init(void)
        return 0;
 }
 
+static int get_tdx_sys_info_extensions(struct tdx_sys_info_ext *sysinfo_ext)
+{
+       if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT)) {
+               memset(sysinfo_ext, 0, sizeof(*sysinfo_ext));
+               return 0;
+       }
+
+       return get_tdx_sys_info_ext(sysinfo_ext);
+}
+
 static __init int init_tdx_module_extensions(void)
 {
        struct tdx_sys_info_ext sysinfo_ext;
        int ret;
 
-       if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
-               return 0;
-
-       ret = get_tdx_sys_info_ext(&sysinfo_ext);
+       ret = get_tdx_sys_info_extensions(&sysinfo_ext);
        if (ret)
                return ret;
 
@@ -1339,10 +1346,7 @@ static int update_tdx_module_extensions(void)
        struct tdx_sys_info_ext sysinfo_ext;
        int ret;
 
-       if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
-               return 0;
-
-       ret = get_tdx_sys_info_ext(&sysinfo_ext);
+       ret = get_tdx_sys_info_extensions(&sysinfo_ext);
        if (ret)
                return ret;
 
#51 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Quoted 8 lines · Xu Yilun
Quoted 7 lines

I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip tdx_ext_init() and the update succeed, but after that the extension features will break.

IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes.

Quoted 7 lines

Sorry, I changed my mind. I remember TDX module architector said "the host should not assume any feature/SEAMCALL must be associated with any extension, it is hidden from the host. Today a SEAMCALL is backed by an extension, tomorrow it may not".

So in this case, ext_required may flip across updates and we should honor the runtime metadata.

Then this is an incompatible update, no? Linux doesn't support incompatible updates. We can keep it simpler.

Quoted 4 lines

I made another effort to combine common routines for ext init and update, but don't like how it turns out. If no one wants it that much, I'd rather stay unchanged:

author
#52 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

Quoted 14 lines · Rick P
Quoted 13 lines · Xu Yilun
Quoted 7 lines

I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip tdx_ext_init() and the update succeed, but after that the extension features will break.

IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes.

Sorry, I changed my mind. I remember TDX module architector said "the host should not assume any feature/SEAMCALL must be associated with any extension, it is hidden from the host. Today a SEAMCALL is backed by an extension, tomorrow it may not".

Quoted 5 lines
Quoted 2 lines

So in this case, ext_required may flip across updates and we should honor the runtime metadata.

Then this is an incompatible update, no? Linux doesn't support incompatible updates. We can keep it simpler.

I imagine if a SEAMCALL is no longer backed by an extension and we switched off extension after update, the SEAMCALL would still work. But I don't have use case and I'm not sure.

OK. I'm good this is defined as incompatible update now. Maybe I'm too worried about thing that is hardly possible. I'll make this set simple.

#53 lore raw

Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update

In reply to Xu Yilun

Chao,

Quoted 11 lines · Xu Yilun
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+static int update_tdx_module_extensions(void)
+{
+	struct tdx_sys_info_ext sysinfo_ext;
+	int ret;
+
+	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
+		return 0;
+
+	ret = get_tdx_sys_info_ext(&sysinfo_ext);
+	if (ret)
+		return ret;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1335

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+
+	if (!sysinfo_ext.ext_required)
+		return 0;
+
+	if (sysinfo_ext.memory_pool_required_pages)
+		return -EFAULT;
+
+	return tdx_ext_init();
+}
+

These update and init functions are looking very similar, any idea how the could be combined?

static __init int init_tdx_module_extensions(void)

{
	struct tdx_sys_info_ext sysinfo_ext;
	int ret;

	if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
		return 0;

	ret = get_tdx_sys_info_ext(&sysinfo_ext);
	if (ret)
		return ret;

	/* Skip if no feature requires TDX module extensions. */
	if (!sysinfo_ext.ext_required)
		return 0;

	ret = tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
	if (ret)
		return ret;

	return tdx_ext_init();
}
author
#54 lore raw

PATCH v2 7/17 x86/virt/tdx: Initialize Quoting extension

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Initialize the Quoting extension during TDX bringup, after enabling TDX module Extension.

Because Quoting is an optional TDX feature, do not let initialization failures cause TDX bringup to fail. In that case, TDX can fall back to the existing userspace flow via a KVM return code.

Only lay the groundwork for TDX Quoting support. Leave the opt-in portion of the initialization to a follow-up patch after fully implementing the feature.

---
 arch/x86/include/asm/tdx.h  |  1 +
 arch/x86/virt/vmx/tdx/tdx.h |  1 +
 arch/x86/virt/vmx/tdx/tdx.c | 34 +++++++++++++++++++++++++++++++++-
 3 files changed, 35 insertions(+), 1 deletion(-)
arch/x86/include/asm/tdx.h +1 -0
index 5fbf89d5317c..741fd97cc199 100644
@@ -36,6 +36,7 @@
 #define TDX_FEATURES0_TD_PRESERVING	BIT_ULL(1)
 #define TDX_FEATURES0_NO_RBP_MOD	BIT_ULL(18)
 #define TDX_FEATURES0_EXT		BIT_ULL(39)
+#define TDX_FEATURES0_QUOTE		BIT_ULL(50)
 
 #ifndef __ASSEMBLER__
 
arch/x86/virt/vmx/tdx/tdx.h +1 -0
index 2deb0a5c902e..1afa0b10dfc9 100644
@@ -66,6 +66,7 @@
 #define TDH_EXT_INIT			60
 #define TDH_EXT_MEM_ADD			61
 #define TDH_SYS_DISABLE			69
+#define TDH_QUOTE_INIT			100
 
 /* TDX page types */
 #define	PT_NDA		0x0
arch/x86/virt/vmx/tdx/tdx.c +33 -1
index 4d2940f4538a..06c42b86b05e 100644
@@ -1167,6 +1167,32 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+/* Initialize quoting extension */
+static __init int tdx_quote_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_QUOTE_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;
+
+	return 0;
+}
+
+static __init void init_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		ret = tdx_quote_init();
+		WARN_ON_ONCE(ret);
+	}
+}
+
 /* Initialize TDX module extensions for extension SEAMCALLs */
 static int tdx_ext_init(void)
 {
@@ -1305,7 +1331,13 @@ static __init int init_tdx_module_extensions(void)
 	if (ret)
 		return ret;
 
-	return tdx_ext_init();
+	ret = tdx_ext_init();
+	if (ret)
+		return ret;
+
+	init_tdx_quoting_extension();
+
+	return 0;
 }
 
 /*
Signature · 1 line
2.25.1
#55 lore raw

Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Using WARN_ON_ONCE() for an optional fallback feature causes kernel panics with panic_on_warn and…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Using WARN_ON_ONCE() for an optional fallback feature causes kernel panics with panic_on_warn and drops the actual hardware error code. --

commit cc6fec2434c274e836f0735b1b3ace996ebcb151
Author: Peter Fang <[email protected]>

x86/virt/tdx: Initialize Quoting extension

This commit initializes the Quoting extension during TDX bringup as an optional feature. It intends to allow TDX initialization to gracefully fall back to a userspace flow if the quoting initialization fails.

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 4d2940f4538a2..06c42b86b05e8 100644

[ ... ]

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1181

Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +12 -0
+/* Initialize quoting extension */
+static __init int tdx_quote_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_QUOTE_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

[Severity: High]
Does this discard the actual SEAMCALL error code? By returning -EFAULT here, the specific hardware error information from TDH_QUOTE_INIT is lost, which might make debugging initialization failures difficult.

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1192

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+
+	return 0;
+}
+
+static __init void init_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		ret = tdx_quote_init();
+		WARN_ON_ONCE(ret);

[Severity: High]
The commit message explicitly states that quoting is an optional feature and initialization failures shouldn't cause TDX bringup to fail.

Is WARN_ON_ONCE() too aggressive here? On systems configured with panic_on_warn=1 (which is common in many environments), this will trigger a kernel panic instead of gracefully falling back to the userspace flow.

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	}
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
#56 lore raw

Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension

Quoted 15 lines · On Thu, Jun 18, 2026 at 08:50:16AM +0000, [email protected] wrote:
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 4d2940f4538a2..06c42b86b05e8 100644

[ ... ]

Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +9 -0
+/* Initialize quoting extension */
+static __init int tdx_quote_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_QUOTE_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1181

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;

[Severity: High]
Does this discard the actual SEAMCALL error code? By returning -EFAULT here, the specific hardware error information from TDH_QUOTE_INIT is lost, which might make debugging initialization failures difficult.

Yilun did a pretty decent analysis of this in [1]. There really isn't much the host can do if this fails, so it seems like a reasonable tradeoff to just make sure there's a record of this in the kernel log.

Quoted 17 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1192

Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+
+	return 0;
+}
+
+static __init void init_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		ret = tdx_quote_init();
+		WARN_ON_ONCE(ret);

[Severity: High]
The commit message explicitly states that quoting is an optional feature and initialization failures shouldn't cause TDX bringup to fail.

Quoted 4 lines

Is WARN_ON_ONCE() too aggressive here? On systems configured with panic_on_warn=1 (which is common in many environments), this will trigger a kernel panic instead of gracefully falling back to the userspace flow.

Having a splat here is to draw the user's attention. Patch 13 sets the TDX_FEATURES0_QUOTE bit only if the TDX module reports support for it. Advertising such support but then failing to initialize it is very likely a TDX module bug. Since attestation is pretty important to TDX security, I think using WARN_ON_ONCE() here should be okay, and the backtrace should be enough to describe the failure.

Quoted 4 lines
Signature · 2 lines
-- 
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7

[1] https://lore.kernel.org/kvm/ajq8RFjv2YFghvQp@yilunxu-OptiPlex-7050/

#57 lore raw

Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension

In reply to Xu Yilun
Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>

Initialize the Quoting extension during TDX bringup, after enabling TDX module Extension.

This jumps into what the patch does without explaining what the Quoting extension is. A brief background would help reviewers who aren't familiar with it.

Quoted 46 lines

Because Quoting is an optional TDX feature, do not let initialization failures cause TDX bringup to fail. In that case, TDX can fall back to the existing userspace flow via a KVM return code.

Only lay the groundwork for TDX Quoting support. Leave the opt-in portion of the initialization to a follow-up patch after fully implementing the feature.

---

arch/x86/include/asm/tdx.h  |  1 +
arch/x86/virt/vmx/tdx/tdx.h |  1 +
arch/x86/virt/vmx/tdx/tdx.c | 34 +++++++++++++++++++++++++++++++++-
3 files changed, 35 insertions(+), 1 deletion(-)
arch/x86/include/asm/tdx.h +1 -0
index 5fbf89d5317c..741fd97cc199 100644
@@ -36,6 +36,7 @@
 #define TDX_FEATURES0_TD_PRESERVING	BIT_ULL(1)
 #define TDX_FEATURES0_NO_RBP_MOD	BIT_ULL(18)
 #define TDX_FEATURES0_EXT		BIT_ULL(39)
+#define TDX_FEATURES0_QUOTE		BIT_ULL(50)
 
 #ifndef __ASSEMBLER__
 
arch/x86/virt/vmx/tdx/tdx.h +1 -0
index 2deb0a5c902e..1afa0b10dfc9 100644
@@ -66,6 +66,7 @@
 #define TDH_EXT_INIT			60
 #define TDH_EXT_MEM_ADD			61
 #define TDH_SYS_DISABLE			69
+#define TDH_QUOTE_INIT			100
 
 /* TDX page types */
 #define	PT_NDA		0x0
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 4d2940f4538a..06c42b86b05e 100644

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1170

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +1 -0
@@ -1167,6 +1167,32 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+/* Initialize quoting extension */

This comment isn't quite helpful.

Quoted 13 lines
arch/x86/virt/vmx/tdx/tdx.c +13 -0
+static __init int tdx_quote_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_QUOTE_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;
+
+	return 0;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1193

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+}
+
+static __init void init_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		ret = tdx_quote_init();
+		WARN_ON_ONCE(ret);
+	}

nit: Reduce indentation of the main body.

	if (!(tdx_addon_feature0 & TDX_FEATURES0_QUOTE))
		return;
	
	ret = tdx_quote_init();
	WARN_ON_ONCE(ret);

Also, why two functions? Can we inline tdx_quote_init() here, or push the feature check into it.

Quoted 23 lines
arch/x86/virt/vmx/tdx/tdx.c +9 -1
+}
+
 /* Initialize TDX module extensions for extension SEAMCALLs */
 static int tdx_ext_init(void)
 {
@@ -1305,7 +1331,13 @@ static __init int init_tdx_module_extensions(void)
 	if (ret)
 		return ret;
 
-	return tdx_ext_init();
+	ret = tdx_ext_init();
+	if (ret)
+		return ret;
+
+	init_tdx_quoting_extension();
+
+	return 0;
 }
 
 /*
Signature · 1 line
2.25.1
#58 lore raw

Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension

Quoted 9 lines · Chao Gao
Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>

Initialize the Quoting extension during TDX bringup, after enabling TDX module Extension.

This jumps into what the patch does without explaining what the Quoting extension is. A brief background would help reviewers who aren't familiar with it.

Yep, I'll add more background in the changelog.

Quoted 9 lines
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 4d2940f4538a..06c42b86b05e 100644
@@ -1167,6 +1167,32 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
Quoted 3 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1170

Quoted 1 line
arch/x86/virt/vmx/tdx/tdx.c +1 -0
+/* Initialize quoting extension */

This comment isn't quite helpful.

Got it. It makes more sense to add it in patch 12.

Quoted 27 lines
Quoted 14 lines
arch/x86/virt/vmx/tdx/tdx.c +13 -0
+static __init int tdx_quote_init(void)
+{
+	struct tdx_module_args args = {};
+	u64 r;
+
+	do {
+		r = seamcall(TDH_QUOTE_INIT, &args);
+	} while (r == TDX_INTERRUPTED_RESUMABLE);
+
+	if (r != TDX_SUCCESS)
+		return -EFAULT;
+
+	return 0;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1193

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+}
+
+static __init void init_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		ret = tdx_quote_init();
+		WARN_ON_ONCE(ret);
+	}

nit: Reduce indentation of the main body.

Quoted 5 lines
	if (!(tdx_addon_feature0 & TDX_FEATURES0_QUOTE))
		return;
	
	ret = tdx_quote_init();
	WARN_ON_ONCE(ret);

That's better. I'll make this change.

Quoted 4 lines

Also, why two functions? Can we inline tdx_quote_init() here, or push the feature check into it.

tdx_quote_init() has a second call site in patch 12 (update_tdx_quoting_extension()), during runtime TDX module update.

Adding the "if (feature)" check inside a SEAMCALL helper doesn't seem like a common pattern. I'm a bit concerned about making this look inconsistent.

Quoted 1 line
author
#59 lore raw

PATCH v2 8/17 x86/virt/tdx: Prepare Quote buffer during extension bringup

In reply to Xu Yilun

From: Peter Fang <[email protected]>

During TDX attestation, the TDX guest asks the host to generate a signed, verifiable structure (a "Quote"). With the Quoting extension, the TDX module returns the Quote in pages that the host shares via an Extension-SEAMCALL.

The SEAMCALL accepts the host buffer pages as a linked list of 4KB "HPA_LINKED_LIST" nodes. Each entry holds the physical address of a 4KB data page, except for the last entry, which points to the next node. The TDX module reports the required Quote buffer size through a global metadata field. See [1] for details.

For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.

[1] Intel TDX Module ABI specification, Section "Physical Memory Management Types"

---
 arch/x86/include/asm/tdx_global_metadata.h  |   4 +
 arch/x86/virt/vmx/tdx/tdx.c                 | 115 +++++++++++++++++++-
 arch/x86/virt/vmx/tdx/tdx_global_metadata.c |  14 +++
 3 files changed, 129 insertions(+), 4 deletions(-)
arch/x86/include/asm/tdx_global_metadata.h +4 -0
index b3442b7c88bb..17cb13a1bb40 100644
@@ -57,4 +57,8 @@ struct tdx_sys_info_ext {
 	bool ext_required;
 };
 
+struct tdx_sys_info_quote {
+	u32 max_quote_size;
+};
+
 #endif
arch/x86/virt/vmx/tdx/tdx.c +111 -4

141 lines not on this page · raw message

arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14 -0
index 84364da89649..1eb2985307c6 100644
@@ -151,3 +151,17 @@ static int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
 
 	return 0;
 }
+
+static __init int get_tdx_sys_info_quote(struct tdx_sys_info_quote *sysinfo_quote)
+{
+	int ret;
+	u64 val;
+
+	ret = read_sys_metadata_field(0x2300000200000002, &val);
+	if (ret)
+		return ret;
+
+	sysinfo_quote->max_quote_size = val;
+
+	return 0;
+}
Signature · 1 line
2.25.1
#60 lore raw

Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup

Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>
For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.

To me it seems the pre-generated parts can be made into a template that can be copied to an allocated buffer looking at patch 11/17.

#61 lore raw

Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup

Quoted 8 lines · Tony Lindgren
Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>
For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.

To me it seems the pre-generated parts can be made into a template that can be copied to an allocated buffer looking at patch 11/17.

Thanks. I'm planning to switch to dynamic allocation in the next revision to make this simpler [1]. So the template will be the HPA_LINKED_LIST node pages plus the next pointers in these nodes.

[1] https://lore.kernel.org/linux-coco/20260626095833.GB1600180@pedri/

#62 lore raw

Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup

In reply to Xu Yilun
Quoted 3 lines · Xu Yilun

For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.

va-to-pa conversions are not expensive. Even for vmalloc() memory.If

I don't like the global buffer. It just generally seems like a bad idea. It needs locking, it wastes memory when nobody is doing quotes, etc..

Is there a way to do this without vmalloc()?

Could userspace provide the memory and then the kernel just does a gup to keep it in place while the TDX module is writing to it? Or, could it just be allocated:

	void *buffer = vmalloc(size);
	mutex_lock();
	// talk to TDX module
	mutex_unlock();
	vfree(buffer);

	return ret;

That actually has some nice properties. It doesn't require locking for the buffer. It also penalizes folks doing lots of quotes. If you hammer on quoting, you hammer on the memory allocator and TLB flushing code and slow yourself down more. If you hammer on it from multiple threads, it gets even worse.

That seems like a feature not a bug.

#63 lore raw

Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup

Quoted 7 lines · Dave Hansen
Quoted 3 lines · Xu Yilun

For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.

va-to-pa conversions are not expensive. Even for vmalloc() memory.If

Quoted 8 lines

I don't like the global buffer. It just generally seems like a bad idea. It needs locking, it wastes memory when nobody is doing quotes, etc..

Is there a way to do this without vmalloc()?

Could userspace provide the memory and then the kernel just does a gup to keep it in place while the TDX module is writing to it? Or, could it just be allocated:

Hi Dave,

The GUP idea makes a lot of sense at a high level I think. However, the shared buffer used by the GHCI call is not 4K-aligned because it has a header in front of it, while the SEAMCALL expects 4K-aligned HPAs (lesson learned...). This memory does come from userspace though:

        struct tdx_quote_req {  <-- 4K-aligned
                u64 version;
                u64 status;
                u32 in_len;
                u32 out_len;
                u8 data[];      <-- shared buffer
        };

So perhaps the simplest thing here is to just allocate a kernel buffer before the SEAMCALL. There's also another subtle wrinkle here: KVM is shifting from a FOLL_PIN-based API to an MMU-notifier-based API for guest page accesses [1], so I think that means more complex code to talk to the TDX module.

Quoted 9 lines
	void *buffer = vmalloc(size);
	mutex_lock();
	// talk to TDX module
	mutex_unlock();
	vfree(buffer);

	return ret;
Quoted 7 lines

That actually has some nice properties. It doesn't require locking for the buffer. It also penalizes folks doing lots of quotes. If you hammer on quoting, you hammer on the memory allocator and TLB flushing code and slow yourself down more. If you hammer on it from multiple threads, it gets even worse.

That seems like a feature not a bug.

This is a really good point... I'm a bit worried about the global impact of vfree() though. IIUC it eventually triggers an all-CPU TLB shootdown, so a guest abusing this path would not only throttle itself, but could also hurt everyone else. This might be more useful if the buffer was allocated in userspace? A free() in userspace usually triggers a more limited TLB shootdown.

I think I can use alloc_pages_bulk() instead to avoid the TLB shootdown cost altogether. It's a bit more work in KVM, but maybe worth it...

Thanks,
Peter

[1] https://lore.kernel.org/kvm/[email protected]/

#64 lore raw

Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup

In reply to Xu Yilun
Quoted 6 lines · Xu Yilun

From: Peter Fang <[email protected]>

During TDX attestation, the TDX guest asks the host to generate a signed, verifiable structure (a "Quote"). With the Quoting extension, the TDX module returns the Quote in pages that the host shares via an Extension-SEAMCALL.

nit: Just say the host provides the pages to the TDX module, no need to specify if it's via an ext seamcall or some otherway. One can infer this from the code itself.

Quoted 6 lines

The SEAMCALL accepts the host buffer pages as a linked list of 4KB "HPA_LINKED_LIST" nodes. Each entry holds the physical address of a 4KB data page, except for the last entry, which points to the next node. The TDX module reports the required Quote buffer size through a global metadata field. See [1] for details.

That HPA_LINKED_LIST is basically a linked list of arrays, each node into the linked list being the array, and each entry of that array being the HPA witht he last being a pointer to the next HPA_LINKED_LIST, so in a way a 2 level data structure. Can you be more explicit and just say something along the lines of :

"SEAMCALL accepts host buffer pages arranged in a custom data structure, consisting of nodes containing HPAs, arranged linearly, and each node is linked to the next one via the last entry"

Quoted 89 lines

For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.

[1] Intel TDX Module ABI specification, Section "Physical Memory Management Types"

---
  arch/x86/include/asm/tdx_global_metadata.h  |   4 +
  arch/x86/virt/vmx/tdx/tdx.c                 | 115 +++++++++++++++++++-
  arch/x86/virt/vmx/tdx/tdx_global_metadata.c |  14 +++
  3 files changed, 129 insertions(+), 4 deletions(-)
arch/x86/include/asm/tdx_global_metadata.h +4 -0
index b3442b7c88bb..17cb13a1bb40 100644
@@ -57,4 +57,8 @@ struct tdx_sys_info_ext {
  	bool ext_required;
  };
 
+struct tdx_sys_info_quote {
+	u32 max_quote_size;
+};
+
  #endif
arch/x86/virt/vmx/tdx/tdx.c +38 -0
index 06c42b86b05e..9716424a301f 100644
@@ -32,6 +32,7 @@
  #include <linux/idr.h>
  #include <linux/kvm_types.h>
  #include <linux/bitfield.h>
+#include <linux/vmalloc.h>
  #include <asm/page.h>
  #include <asm/special_insns.h>
  #include <asm/msr-index.h>
@@ -71,6 +72,24 @@ static LIST_HEAD(tdx_memlist);
 
  static struct tdx_sys_info tdx_sysinfo;
 
+/*
+ * Quote buffer shared with the TDX module for quote generation, in HPA linked
+ * list format.
+ *
+ * @buf: Virtual address of the quote buffer.
+ * @buf_len: Size of @buf in bytes.
+ * @hpa_entries: HPA entries, starting at the first list node.
+ * @hpa_entries_pa: Physical address for @hpa_entries.
+ */
+struct tdx_quote_data {
+	void		*buf;
+	u64		buf_len;
+	u64		*hpa_entries;
+	phys_addr_t	hpa_entries_pa;
+};
+
+static struct tdx_quote_data tdx_quote;
+
  static DEFINE_RAW_SPINLOCK(sysinit_lock);
 
  /*
@@ -1167,6 +1186,81 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
  	return 0;
  }
 
+static inline phys_addr_t tdx_vmalloc_to_pa(const void *addr)
+{
+	unsigned long pfn = vmalloc_to_pfn(addr);
+
+	return PFN_PHYS(pfn);
+}
+
+#define HPAS_PER_NODE			(PAGE_SIZE / sizeof(u64))
+
+/*
+ * Pass the quote buffer to the TDX module as an HPA linked list, where each
+ * node holds 4KB page HPAs and the last entry points to the next node.
+ */
+static __init int tdx_quote_create_buf(unsigned int npages,
+				       struct tdx_quote_data *qdata)
+{
+	unsigned int nnodes;
+	u64 *hpas;
+	void *qbuf;

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1217

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	int i, j;
+
+	if (!npages)
+		return -EINVAL;
+
+	/*
+	 * Each node holds up to (HPAS_PER_NODE - 1) 4KB page HPAs.
+	 * The last entry of the node points to the next node.
+	 */
+	nnodes = DIV_ROUND_UP(npages, HPAS_PER_NODE - 1);

nit: It's somewhat arbitrary but num_nodes seems more explicit, saving 3 chars is not worth it.

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1219

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+
+	hpas = vmalloc_array(nnodes, PAGE_SIZE);

hpas is basically the starting node, so name it "nodes". Also I'm slightly confused why you use vmalloc to allocate a contiguous address space when the nodes are linked in a list? I.e you keep referring to list in the changelog and the code seems to be working in an array chunks?

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
+	if (!hpas)
+		return -ENOMEM;
+
+	/*

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1233

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10 -0
+	 * ~0ULL is the list terminator for HPA_LINKED_LIST.
+	 *
+	 * Pre-fill the last node with 0xff bytes so that unused entries are
+	 * terminators. Overwrite populated entries later.
+	 */
+	memset((u8 *)hpas + (nnodes - 1) * PAGE_SIZE, 0xff, PAGE_SIZE);
+
+	qbuf = vcalloc(npages, PAGE_SIZE);
+	if (!qbuf)
+		goto out_nomem;

'qbuf' is rather arbitrary here. You simply pre-allocate a bunch of pages which you then initialize into the nodes. Can't this allocation be moved inside the loop itself, of course it will increase the number of allocs happening but this is during initialization so it's not performance critical, yes?

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1236

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+
+	/* Populate the linked list */
+	for (i = 0, j = 0; j < npages; i++) {

nit: This feedback was given before, but both variables can be defined in this loop.

Quoted 81 lines
arch/x86/virt/vmx/tdx/tdx.c +43 -4
+		if ((i % HPAS_PER_NODE) == HPAS_PER_NODE - 1) {
+			/*
+			 * The last node entry always points to the next node.
+			 * The address of the following entry must be on next
+			 * node's page boundary.
+			 */
+			hpas[i] = tdx_vmalloc_to_pa(&hpas[i + 1]);
+			continue;
+		}
+
+		hpas[i] = tdx_vmalloc_to_pa((u8 *)qbuf + j * PAGE_SIZE);
+		j++;
+	}
+> +	qdata->buf = qbuf;
+	qdata->buf_len = (u64)npages * PAGE_SIZE;
+	qdata->hpa_entries = hpas;
+	qdata->hpa_entries_pa = tdx_vmalloc_to_pa(hpas);
+
+	return 0;
+
+out_nomem:
+	vfree(hpas);
+
+	return -ENOMEM;
+}
+
  /* Initialize quoting extension */
  static __init int tdx_quote_init(void)
  {
@@ -1185,12 +1279,25 @@ static __init int tdx_quote_init(void)
 
  static __init void init_tdx_quoting_extension(void)
  {
-	int ret;
+	struct tdx_sys_info_quote sysinfo_quote;
+	unsigned int nr_quote_pages;
+
+	if (!(tdx_addon_feature0 & TDX_FEATURES0_QUOTE))
+		return;
 
-	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
-		ret = tdx_quote_init();
-		WARN_ON_ONCE(ret);
+	if (tdx_quote_init()) {
+		WARN_ON_ONCE(1);
+		return;
  	}
+
+	/* Quoting metadata is valid only after initialization */
+	if (get_tdx_sys_info_quote(&sysinfo_quote))
+		return;
+
+	nr_quote_pages = PAGE_ALIGN(sysinfo_quote.max_quote_size) /
+			 PAGE_SIZE;
+	if (tdx_quote_create_buf(nr_quote_pages, &tdx_quote))
+		pr_err("Failed to create quote buffer\n");
  }
 
  /* Initialize TDX module extensions for extension SEAMCALLs */
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14 -0
index 84364da89649..1eb2985307c6 100644
@@ -151,3 +151,17 @@ static int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)
 
  	return 0;
  }
+
+static __init int get_tdx_sys_info_quote(struct tdx_sys_info_quote *sysinfo_quote)
+{
+	int ret;
+	u64 val;
+
+	ret = read_sys_metadata_field(0x2300000200000002, &val);
+	if (ret)
+		return ret;
+
+	sysinfo_quote->max_quote_size = val;
+
+	return 0;
+}
#65 lore raw

Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup

Quoted 5 lines · Nikolay Borisov
Quoted 2 lines · Xu Yilun

From: Peter Fang <[email protected]>

Quoted 8 lines
Quoted 4 lines

During TDX attestation, the TDX guest asks the host to generate a signed, verifiable structure (a "Quote"). With the Quoting extension, the TDX module returns the Quote in pages that the host shares via an Extension-SEAMCALL.

nit: Just say the host provides the pages to the TDX module, no need to specify if it's via an ext seamcall or some otherway. One can infer this from the code itself.

Good point. Thanks for the simplification.

Quoted 14 lines
Quoted 7 lines

The SEAMCALL accepts the host buffer pages as a linked list of 4KB "HPA_LINKED_LIST" nodes. Each entry holds the physical address of a 4KB data page, except for the last entry, which points to the next node. The TDX module reports the required Quote buffer size through a global metadata field. See [1] for details.

That HPA_LINKED_LIST is basically a linked list of arrays, each node into the linked list being the array, and each entry of that array being the HPA witht he last being a pointer to the next HPA_LINKED_LIST, so in a way a 2 level data structure. Can you be more explicit and just say something along the lines of :

Quoted 3 lines

"SEAMCALL accepts host buffer pages arranged in a custom data structure, consisting of nodes containing HPAs, arranged linearly, and each node is linked to the next one via the last entry"

Oh this is much easier to read. Thanks for the suggestion! I can incorporate this.

Quoted 18 lines
Quoted 18 lines
arch/x86/virt/vmx/tdx/tdx.c +17 -0
+
+#define HPAS_PER_NODE			(PAGE_SIZE / sizeof(u64))
+
+/*
+ * Pass the quote buffer to the TDX module as an HPA linked list, where each
+ * node holds 4KB page HPAs and the last entry points to the next node.
+ */
+static __init int tdx_quote_create_buf(unsigned int npages,
+				       struct tdx_quote_data *qdata)
+{
+	unsigned int nnodes;
+	u64 *hpas;
+	void *qbuf;
+	int i, j;
+
+	if (!npages)
+		return -EINVAL;
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1217

Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +6 -0
+
+	/*
+	 * Each node holds up to (HPAS_PER_NODE - 1) 4KB page HPAs.
+	 * The last entry of the node points to the next node.
+	 */
+	nnodes = DIV_ROUND_UP(npages, HPAS_PER_NODE - 1);

nit: It's somewhat arbitrary but num_nodes seems more explicit, saving 3 chars is not worth it.

Sure, I can make this change.

Quoted 7 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+	hpas = vmalloc_array(nnodes, PAGE_SIZE);

hpas is basically the starting node, so name it "nodes". Also I'm slightly confused why you use vmalloc to allocate a contiguous address space when the nodes are linked in a list? I.e you keep referring to list in the changelog and the code seems to be working in an array chunks?

Hmm. I used the vmalloc family because I wanted to keep using hpas[i] without switching to the next node's va. It felt a bit simpler to just keep doing "i++". Maybe that actually hurts readability? I can replace that part of the for loop with something like:

        if (i == HPAS_PER_NODE - 1) {
                next_node = kmalloc(PAGE_SIZE, GFP_KERNEL);
                hpas[i] = virt_to_phys(next_node);
                hpas = next_node;
                i = 0;
        }

And "hpas[i]" feels a bit closer to what the code does than "nodes[i]"? It's filling this page with a bunch of HPAs.

Quoted 13 lines
Quoted 13 lines
arch/x86/virt/vmx/tdx/tdx.c +12 -0
+	if (!hpas)
+		return -ENOMEM;
+
+	/*
+	 * ~0ULL is the list terminator for HPA_LINKED_LIST.
+	 *
+	 * Pre-fill the last node with 0xff bytes so that unused entries are
+	 * terminators. Overwrite populated entries later.
+	 */
+	memset((u8 *)hpas + (nnodes - 1) * PAGE_SIZE, 0xff, PAGE_SIZE);
+
+	qbuf = vcalloc(npages, PAGE_SIZE);
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1233

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	if (!qbuf)
+		goto out_nomem;

'qbuf' is rather arbitrary here. You simply pre-allocate a bunch of pages which you then initialize into the nodes. Can't this allocation be moved inside the loop itself, of course it will increase the number of allocs happening but this is during initialization so it's not performance critical, yes?

I see your point. It’s a bit subtle here. In patch 11, there is a kvmemdup() that depends on this qbuf being virtually contiguous. Based on feedback from other folks, I think I might change this and use runtime quote buffers allocated with kmalloc(). That would also avoid the TLB flushing penalty of vfree(). The free path would be a bit more complicated, but that should be okay.

Quoted 7 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1236

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+
+	/* Populate the linked list */
+	for (i = 0, j = 0; j < npages; i++) {

nit: This feedback was given before, but both variables can be defined in this loop.

Great suggestion. I can do that. Thanks.

Quoted 2 lines
author
#66 lore raw

PATCH v2 9/17 x86/virt/tdx: Add interface to check Quoting availability

In reply to Xu Yilun

From: Peter Fang <[email protected]>

KVM needs to know if the Quoting extension is available to determine whether userspace must be involved in Quote generation.

Since the Quote buffer is always created during Quoting extension bringup, checking whether the buffer exists is sufficient.

---
 arch/x86/include/asm/tdx.h  |  2 ++
 arch/x86/virt/vmx/tdx/tdx.c | 10 ++++++++++
 2 files changed, 12 insertions(+)
arch/x86/include/asm/tdx.h +2 -0
index 741fd97cc199..9432a736855e 100644
@@ -147,6 +147,8 @@ struct tdx_vp {
 	struct page **tdcx_pages;
 };
 
+bool tdx_quote_enabled(void);
+
 static inline u64 mk_keyed_paddr(u16 hkid, struct page *page)
 {
 	u64 ret;
arch/x86/virt/vmx/tdx/tdx.c +10 -0
index 9716424a301f..da55c1aeeeb8 100644
@@ -1193,6 +1193,16 @@ static inline phys_addr_t tdx_vmalloc_to_pa(const void *addr)
 	return PFN_PHYS(pfn);
 }
 
+bool tdx_quote_enabled(void)
+{
+	/*
+	 * No need for locking here. The quote buffer is initialized as part of
+	 * core TDX bringup, which comes before KVM is ready for userspace.
+	 */
+	return !!tdx_quote.buf;
+}
+EXPORT_SYMBOL_FOR_KVM(tdx_quote_enabled);
+
 #define HPAS_PER_NODE			(PAGE_SIZE / sizeof(u64))
 
 /*
Signature · 1 line
2.25.1
Reviewed-by on v2 9/17
#67 lore raw

Re: [PATCH v2 09/17] x86/virt/tdx: Add interface to check Quoting availability

Quoted 7 lines · Xu Yilun

From: Peter Fang <[email protected]>

KVM needs to know if the Quoting extension is available to determine whether userspace must be involved in Quote generation.

Since the Quote buffer is always created during Quoting extension bringup, checking whether the buffer exists is sufficient.

author
#68 lore raw

PATCH v2 10/17 x86/virt/tdx: Move tdx_tdr_pa() up in the file

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Move tdx_tdr_pa() earlier in the file to prepare for upcoming changes that add a new Extension-SEAMCALL.

No functional change intended.

---
 arch/x86/virt/vmx/tdx/tdx.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +5 -5
index da55c1aeeeb8..1e2c7a33c7a9 100644
@@ -1186,6 +1186,11 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)
 	return 0;
 }
 
+static inline u64 tdx_tdr_pa(struct tdx_td *td)
+{
+	return page_to_phys(td->tdr_page);
+}
+
 static inline phys_addr_t tdx_vmalloc_to_pa(const void *addr)
 {
 	unsigned long pfn = vmalloc_to_pfn(addr);
@@ -1966,11 +1971,6 @@ void tdx_guest_keyid_free(unsigned int keyid)
 }
 EXPORT_SYMBOL_FOR_KVM(tdx_guest_keyid_free);
 
-static inline u64 tdx_tdr_pa(struct tdx_td *td)
-{
-	return page_to_phys(td->tdr_page);
-}
-
 /*
  * The TDX module exposes a CLFLUSH_BEFORE_ALLOC bit to specify whether
  * a CLFLUSH of pages is required before handing them to the TDX module.
Signature · 1 line
2.25.1
Reviewed-by on v2 10/17
#69 lore raw

Re: [PATCH v2 10/17] x86/virt/tdx: Move tdx_tdr_pa() up in the file

Quoted 6 lines · Xu Yilun

From: Peter Fang <[email protected]>

Move tdx_tdr_pa() earlier in the file to prepare for upcoming changes that add a new Extension-SEAMCALL.

No functional change intended.

author
#70 lore raw

PATCH v2 11/17 x86/virt/tdx: Add interface to generate a Quote

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Provide an interface to generate a Quote via the TDH.QUOTE.GET Extension-SEAMCALL. Although the TDX module may support concurrent Quote generation, use a single shared buffer for simplicity and serialize access with a mutex. TDX bringup code already prepares the buffer in the format required by the TDX module.

Return a per-call buffer containing the Quote so callers don't need to size the buffer themselves. The caller is responsible for freeing the returned buffer.

---
 arch/x86/include/asm/tdx.h  |  2 +
 arch/x86/virt/vmx/tdx/tdx.h |  1 +
 arch/x86/virt/vmx/tdx/tdx.c | 77 +++++++++++++++++++++++++++++++++++++
 3 files changed, 80 insertions(+)
arch/x86/include/asm/tdx.h +2 -0
index 9432a736855e..34764838f132 100644
@@ -148,6 +148,8 @@ struct tdx_vp {
 };
 
 bool tdx_quote_enabled(void);
+void *tdx_quote_generate(struct tdx_td *td, void *in_data, u32 in_data_len,
+			 u32 *quote_len);
 
 static inline u64 mk_keyed_paddr(u16 hkid, struct page *page)
 {
arch/x86/virt/vmx/tdx/tdx.h +1 -0
index 1afa0b10dfc9..32b13b0c85f9 100644
@@ -66,6 +66,7 @@
 #define TDH_EXT_INIT			60
 #define TDH_EXT_MEM_ADD			61
 #define TDH_SYS_DISABLE			69
+#define TDH_QUOTE_GET			98
 #define TDH_QUOTE_INIT			100
 
 /* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +77 -0

89 lines not on this page · raw message

Signature · 1 line
2.25.1
#71 lore raw

Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] tdx_quote_generate() uses an uninterruptible mutex lock for a guest-triggered hardware operation…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] tdx_quote_generate() uses an uninterruptible mutex lock for a guest-triggered hardware operation, creating a potential host-level denial of service. --

commit 3bb73b03a2480c47534841c05b531b06bda047a9
Author: Peter Fang <[email protected]>

x86/virt/tdx: Add interface to generate a Quote

This commit provides an interface to generate a Quote via the TDH.QUOTE.GET Extension-SEAMCALL. It introduces a global mutex to serialize access to a shared quote buffer.

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +0 -0

[ ... ]

arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+void *tdx_quote_generate(struct tdx_td *td, void *in_data, u32 in_data_len,
+			 u32 *quote_len)
+{
+	struct tdx_quote_data *qdata = &tdx_quote;
+	void *quote_dup = NULL;
+	u64 r, out_len;
+
+	if (!tdx_quote_enabled())
+		return NULL;
+
+	mutex_lock(&tdx_quote_lock);

[Severity: High]
Since this path can be triggered by a guest, could using an uninterruptible mutex_lock() here lead to a host-level denial of service?

If a guest rapidly requests quotes from multiple vCPUs, it could monopolize the global tdx_quote_lock. Other vCPU threads waiting for the lock would block in an uninterruptible sleep (D state). If an administrator attempts to kill the waiting VMs, the threads will not respond to fatal signals until they acquire the lock.

Would it be safer to use mutex_lock_killable() here to allow fatal signals to interrupt the wait and permit VM teardown?

Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +7 -0
+
+	/*
+	 * Use the first page of the quote buffer for input data. The buffer
+	 * must be at least one page in size. @in_data may not be page-aligned,
+	 * but TDH.QUOTE.GET expects page-aligned addresses.
+	 */
+	memcpy(qdata->buf, in_data, in_data_len);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
#72 lore raw

Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote

Quoted 20 lines · On Thu, Jun 18, 2026 at 08:49:40AM +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] tdx_quote_generate() uses an uninterruptible mutex lock for a guest-triggered hardware operation, creating a potential host-level denial of service. --

arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+void *tdx_quote_generate(struct tdx_td *td, void *in_data, u32 in_data_len,
+			 u32 *quote_len)
+{
+	struct tdx_quote_data *qdata = &tdx_quote;
+	void *quote_dup = NULL;
+	u64 r, out_len;
+
+	if (!tdx_quote_enabled())
+		return NULL;
+
+	mutex_lock(&tdx_quote_lock);

[Severity: High]
Since this path can be triggered by a guest, could using an uninterruptible mutex_lock() here lead to a host-level denial of service?

Quoted 8 lines

If a guest rapidly requests quotes from multiple vCPUs, it could monopolize the global tdx_quote_lock. Other vCPU threads waiting for the lock would block in an uninterruptible sleep (D state). If an administrator attempts to kill the waiting VMs, the threads will not respond to fatal signals until they acquire the lock.

Would it be safer to use mutex_lock_killable() here to allow fatal signals to interrupt the wait and permit VM teardown?

This is a good idea, but I think graceful teardown can be addressed later.

Quoted 4 lines
Signature · 2 lines
-- 
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
#73 lore raw

Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote

In reply to Xu Yilun
Quoted 3 lines · Xu Yilun

From: Peter Fang <[email protected]>

arch/x86/virt/vmx/tdx/tdx.c +0 -0

...

arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+void *tdx_quote_generate(struct tdx_td *td, void *in_data, u32 in_data_len,
+			 u32 *quote_len)
+{
+	struct tdx_quote_data *qdata = &tdx_quote;
+	void *quote_dup = NULL;
+	u64 r, out_len;
+
+	if (!tdx_quote_enabled())
+		return NULL;
+
+	mutex_lock(&tdx_quote_lock);

How about make the pre-generated static tdx_quote a template page that only gets read and copied to an allocated bufer here?

If the tdx_quote template is only read for copying here, seems you're not going to need the mutex at all? That is assuming tdx_quote template does not change after init.

#74 lore raw

Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote

Quoted 20 lines · Tony Lindgren
Quoted 3 lines · Xu Yilun

From: Peter Fang <[email protected]>

arch/x86/virt/vmx/tdx/tdx.c +0 -0

...

arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259

Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11 -0
+void *tdx_quote_generate(struct tdx_td *td, void *in_data, u32 in_data_len,
+			 u32 *quote_len)
+{
+	struct tdx_quote_data *qdata = &tdx_quote;
+	void *quote_dup = NULL;
+	u64 r, out_len;
+
+	if (!tdx_quote_enabled())
+		return NULL;
+
+	mutex_lock(&tdx_quote_lock);

How about make the pre-generated static tdx_quote a template page that only gets read and copied to an allocated bufer here?

Quoted 3 lines

If the tdx_quote template is only read for copying here, seems you're not going to need the mutex at all? That is assuming tdx_quote template does not change after init.

Hm, actually tdx_quote is an output buffer as well (in the form of a head pointer: qdata->hpa_entries_pa). Maybe this code needs better commenting...

author
#75 lore raw

PATCH v2 12/17 x86/virt/tdx: Reinitialize the Quoting extension after TDX module update

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Invoke TDH.QUOTE.INIT again after a runtime module update to trigger the necessary rekey procedure in the TDX module.

Keep the existing Quote buffer since memory allocation is not permitted during the update. Compatible TDX module updates must not increase the Quote buffer size, or an undersized buffer might cause Quote generation to fail. See [1] for module update details.

[1] Documentation/arch/x86/tdx.rst, Section "TDX module Runtime Update"

---
 arch/x86/virt/vmx/tdx/tdx.c | 31 ++++++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +28 -3
index ac0da4966697..81e7b6b1dacb 100644
@@ -1353,8 +1353,11 @@ static __init int tdx_quote_create_buf(unsigned int npages,
 	return -ENOMEM;
 }
 
-/* Initialize quoting extension */
-static __init int tdx_quote_init(void)
+/*
+ * Initialize quoting extension.
+ * It also rekeys the TDX module after a runtime module update.
+ */
+static int tdx_quote_init(void)
 {
 	struct tdx_module_args args = {};
 	u64 r;
@@ -1539,6 +1542,22 @@ static __init int init_tdx_module_extensions(void)
 	return 0;
 }
 
+static void update_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		/*
+		 * The TDH.QUOTE.INIT call renews the quoting keys.
+		 *
+		 * A module update must not increase the quote buffer size, or
+		 * quote generation may fail and break attestation.
+		 */
+		ret = tdx_quote_init();
+		WARN_ON(ret);
+	}
+}
+
 /*
  * Mostly the same flow as init_tdx_module_extensions(), but rejects adding
  * more memory.
@@ -1561,7 +1580,13 @@ static int update_tdx_module_extensions(void)
 	if (sysinfo_ext.memory_pool_required_pages)
 		return -EFAULT;
 
-	return tdx_ext_init();
+	ret = tdx_ext_init();
+	if (ret)
+		return ret;
+
+	update_tdx_quoting_extension();
+
+	return 0;
 }
 
 static __init int init_tdx_module(void)
Signature · 1 line
2.25.1
Reviewed-by on v2 12/17
#76 lore raw

Re: [PATCH v2 12/17] x86/virt/tdx: Reinitialize the Quoting extension after TDX module update

Quoted 11 lines · Xu Yilun

From: Peter Fang <[email protected]>

Invoke TDH.QUOTE.INIT again after a runtime module update to trigger the necessary rekey procedure in the TDX module.

Keep the existing Quote buffer since memory allocation is not permitted during the update. Compatible TDX module updates must not increase the Quote buffer size, or an undersized buffer might cause Quote generation to fail. See [1] for module update details.

[1] Documentation/arch/x86/tdx.rst, Section "TDX module Runtime Update"

#77 lore raw

Re: [PATCH v2 12/17] x86/virt/tdx: Reinitialize the Quoting extension after TDX module update

In reply to Xu Yilun
Quoted 50 lines · Xu Yilun

From: Peter Fang <[email protected]>

Invoke TDH.QUOTE.INIT again after a runtime module update to trigger the necessary rekey procedure in the TDX module.

Keep the existing Quote buffer since memory allocation is not permitted during the update. Compatible TDX module updates must not increase the Quote buffer size, or an undersized buffer might cause Quote generation to fail. See [1] for module update details.

[1] Documentation/arch/x86/tdx.rst, Section "TDX module Runtime Update"

---
 arch/x86/virt/vmx/tdx/tdx.c | 31 ++++++++++++++++++++++++++++---

 1 file changed, 28 insertions(+), 3 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index ac0da4966697..81e7b6b1dacb 100644
@@ -1353,8 +1353,11 @@ static __init int tdx_quote_create_buf(unsigned int
 npages,
  	return -ENOMEM;
  }

arch/x86/virt/vmx/tdx/tdx.c +5 -2
-/* Initialize quoting extension */
-static __init int tdx_quote_init(void)
+/*
+ * Initialize quoting extension.
+ * It also rekeys the TDX module after a runtime module update.
+ */
+static int tdx_quote_init(void)

 {
  struct tdx_module_args args = {};
  u64 r;

arch/x86/virt/vmx/tdx/tdx.c +0 -0
@@ -1539,6 +1542,22 @@ static __init int init_tdx_module_extensions(void)
  	return 0;
  }

arch/x86/virt/vmx/tdx/tdx.c +9 -0
+static void update_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		/*
+		 * The TDH.QUOTE.INIT call renews the quoting keys.
+		 *
+		 * A module update must not increase the quote buffer size,

or

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1555

Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+		 * quote generation may fail and break attestation.
+		 */

It makes sense to explain why to call this again, but I'm not sure how this comment helps.

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1558

Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+		ret = tdx_quote_init();
+		WARN_ON(ret);
+	}

Since the caller of this function has an return code. Please justify why not to return an error.

Quoted 20 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+}
+

 /*
  * Mostly the same flow as init_tdx_module_extensions(), but rejects adding
  * more memory.

arch/x86/virt/vmx/tdx/tdx.c +0 -0
@@ -1561,7 +1580,13 @@ static int update_tdx_module_extensions(void)
  	if (sysinfo_ext.memory_pool_required_pages)
  		return -EFAULT;

arch/x86/virt/vmx/tdx/tdx.c +7 -1
-	return tdx_ext_init();
+	ret = tdx_ext_init();
+	if (ret)
+		return ret;
+
+	update_tdx_quoting_extension();
+
+	return 0;

 }

 static __init int init_tdx_module(void)

#78 lore raw

Re: [PATCH v2 12/17] x86/virt/tdx: Reinitialize the Quoting extension after TDX module update

Quoted 3 lines · Rick P
Quoted 2 lines · Xu Yilun

From: Peter Fang <[email protected]>

[ ... ]

Quoted 8 lines
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c +7 -0
+static void update_tdx_quoting_extension(void)
+{
+	int ret;
+
+	if (tdx_addon_feature0 & TDX_FEATURES0_QUOTE) {
+		/*
+		 * The TDH.QUOTE.INIT call renews the quoting keys.
Quoted 8 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1555

Quoted 5 lines
(quoted diff) +2 -0
+		 *
+		 * A module update must not increase the quote buffer size,

or

arch/x86/virt/vmx/tdx/tdx.c +2 -0
+		 * quote generation may fail and break attestation.
+		 */

It makes sense to explain why to call this again, but I'm not sure how this comment helps.

OK, I can improve the "why" of this, and not just say what it does.

BTW the "must not increase the quote buffer size" comment will probably be irrelevant in the next version, since I plan to remove the static global quote buffer altogether.

Quoted 7 lines

arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1558

Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -0
+		ret = tdx_quote_init();
+		WARN_ON(ret);
+	}

Since the caller of this function has an return code. Please justify why not to return an error.

The original thought was to warn since there is no proper handling for this kind of error, and at the same time try to keep other parts of TDX going. But I think it's probably better to just fail TDX as a whole if this goes wrong. If this returns an error then maybe something more serious is wrong here.

Quoted 1 line
author
#79 lore raw

PATCH v2 13/17 x86/virt/tdx: Enable Quoting extension

In reply to Xu Yilun

From: Peter Fang <[email protected]>

The Quoting extension generates TDX attestation Quotes in the TDX module, without using a discrete Quoting engine. Enable this feature by requesting it in TDH.SYS.CONFIG and TDH.SYS.UPDATE.

---
 arch/x86/virt/vmx/tdx/tdx.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +2 -4
index 81e7b6b1dacb..01fb01313077 100644
@@ -1029,10 +1029,8 @@ static __init int construct_tdmrs(struct list_head *tmb_list,
 
 static __init void set_tdx_addon_features(void)
 {
-	/*
-	 * To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when
-	 * kernel is ready.
-	 */
+	if (tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_QUOTE)
+		tdx_addon_feature0 |= TDX_FEATURES0_QUOTE;
 }
 
 static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
Signature · 1 line
2.25.1
Reviewed-by on v2 13/17
#80 lore raw

Re: [PATCH v2 13/17] x86/virt/tdx: Enable Quoting extension

Quoted 5 lines · Xu Yilun

From: Peter Fang <[email protected]>

The Quoting extension generates TDX attestation Quotes in the TDX module, without using a discrete Quoting engine. Enable this feature by requesting it in TDH.SYS.CONFIG and TDH.SYS.UPDATE.

author
#81 lore raw

PATCH v2 14/17 x86/tdx: Move and rename Quote request structure

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Move struct tdx_quote_buf to tdx.h so it can be shared by the guest driver and core TDX code, as the host will also need the Quote buffer format for in-kernel Quote generation.

Rename the struct to tdx_quote_req to better reflect its purpose, and replace "quote_buf" with "quote_req" in tdx-guest.c.

---
 arch/x86/include/asm/tdx.h              | 20 +++++++++++
 drivers/virt/coco/tdx-guest/tdx-guest.c | 47 ++++++++-----------------
 2 files changed, 34 insertions(+), 33 deletions(-)
arch/x86/include/asm/tdx.h +20 -0
index 34764838f132..24bce7512de3 100644
@@ -66,6 +66,26 @@ struct ve_info {
 	u32 instr_info;
 };
 
+/**
+ * struct tdx_quote_req - Format of Quote request message
+ * @version: Quote format version, filled by TD.
+ * @status: Status code of Quote request, filled by VMM.
+ * @in_len: Length of TDREPORT, filled by TD.
+ * @out_len: Length of Quote data, filled by VMM.
+ * @data: Quote data on output or TDREPORT on input.
+ *
+ * More details of Quote request message can be found in TDX
+ * Guest-Host Communication Interface (GHCI) for Intel TDX 1.0,
+ * section titled "TDG.VP.VMCALL<GetQuote>"
+ */
+struct tdx_quote_req {
+	u64 version;
+	u64 status;
+	u32 in_len;
+	u32 out_len;
+	u8 data[];
+};
+
 #ifdef CONFIG_INTEL_TDX_GUEST
 
 void __init tdx_early_init(void);
drivers/virt/coco/tdx-guest/tdx-guest.c +14 -33

109 lines not on this page · raw message

Signature · 1 line
2.25.1
Reviewed-by on v2 14/17
#82 lore raw

Re: [PATCH v2 14/17] x86/tdx: Move and rename Quote request structure

Quoted 8 lines · Xu Yilun

From: Peter Fang <[email protected]>

Move struct tdx_quote_buf to tdx.h so it can be shared by the guest driver and core TDX code, as the host will also need the Quote buffer format for in-kernel Quote generation.

Rename the struct to tdx_quote_req to better reflect its purpose, and replace "quote_buf" with "quote_req" in tdx-guest.c.

author
#83 lore raw

PATCH v2 15/17 KVM: TDX: Factor out userspace return path from tdx_get_quote()

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Separate the logic that returns the GetQuote TDVMCALL exit to userspace so that tdx_get_quote() can be extended to support in-kernel Quote generation.

No functional change intended.

---
 arch/x86/kvm/vmx/tdx.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/tdx.c +16 -9
index ed12805bbb44..9f7c39e0d4b5 100644
@@ -1524,6 +1524,20 @@ static int tdx_complete_simple(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size)
+{
+	vcpu->run->exit_reason = KVM_EXIT_TDX;
+	vcpu->run->tdx.flags = 0;
+	vcpu->run->tdx.nr = TDVMCALL_GET_QUOTE;
+	vcpu->run->tdx.get_quote.ret = TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED;
+	vcpu->run->tdx.get_quote.gpa = gpa;
+	vcpu->run->tdx.get_quote.size = size;
+
+	vcpu->arch.complete_userspace_io = tdx_complete_simple;
+
+	return 0;
+}
+
 static int tdx_get_quote(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
@@ -1536,16 +1550,9 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
-	vcpu->run->exit_reason = KVM_EXIT_TDX;
-	vcpu->run->tdx.flags = 0;
-	vcpu->run->tdx.nr = TDVMCALL_GET_QUOTE;
-	vcpu->run->tdx.get_quote.ret = TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED;
-	vcpu->run->tdx.get_quote.gpa = gpa & ~gfn_to_gpa(kvm_gfn_direct_bits(tdx->vcpu.kvm));
-	vcpu->run->tdx.get_quote.size = size;
-
-	vcpu->arch.complete_userspace_io = tdx_complete_simple;
+	gpa &= ~gfn_to_gpa(kvm_gfn_direct_bits(vcpu->kvm));
 
-	return 0;
+	return tdx_get_quote_user(vcpu, gpa, size);
 }
 
 static int tdx_setup_event_notify_interrupt(struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
Reviewed-by on v2 15/17
#84 lore raw

Re: [PATCH v2 15/17] KVM: TDX: Factor out userspace return path from tdx_get_quote()

Quoted 7 lines · Xu Yilun

From: Peter Fang <[email protected]>

Separate the logic that returns the GetQuote TDVMCALL exit to userspace so that tdx_get_quote() can be extended to support in-kernel Quote generation.

No functional change intended.

author
#85 lore raw

PATCH v2 16/17 KVM: TDX: Add in-kernel Quote generation

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.

Use the core TDX API for Quote generation when the Quoting extension is available. For simplicity, KVM checks its availability once per guest during initialization. KVM does not handle Quoting service disruptions or switch between the in-kernel and userspace paths.

Update the KVM API and TDX documentation to describe this new Quoting capability.

---
 Documentation/arch/x86/tdx.rst |  19 ++---
 Documentation/virt/kvm/api.rst |   3 +
 arch/x86/include/asm/tdx.h     |   9 +++
 arch/x86/kvm/vmx/tdx.h         |   6 ++
 arch/x86/kvm/vmx/tdx.c         | 135 ++++++++++++++++++++++++++++++++-
 virt/kvm/kvm_main.c            |   1 +
 6 files changed, 163 insertions(+), 10 deletions(-)
Documentation/arch/x86/tdx.rst +10 -9
index 3303499ad4c6..f02bb6919d91 100644
@@ -522,15 +522,16 @@ provided by attestation service so the TDREPORT can be verified uniquely.
 More details about the TDREPORT can be found in Intel TDX Module
 specification, section titled "TDG.MR.REPORT Leaf".
 
-After getting the TDREPORT, the second step of the attestation process
-is to send it to the Quoting Enclave (QE) to generate the Quote. TDREPORT
-by design can only be verified on the local platform as the MAC key is
-bound to the platform. To support remote verification of the TDREPORT,
-TDX leverages Intel SGX Quoting Enclave to verify the TDREPORT locally
-and convert it to a remotely verifiable Quote. Method of sending TDREPORT
-to QE is implementation specific. Attestation software can choose
-whatever communication channel available (i.e. vsock or TCP/IP) to
-send the TDREPORT to QE and receive the Quote.
+After getting the TDREPORT, the second step of the attestation process is to
+convert it to a Quote. A TDREPORT by design can only be verified on the local
+platform, as the MAC key is bound to the platform. A Quote makes the TDREPORT
+remotely verifiable. It can be generated either through a Quoting Enclave
+(QE) in userspace or through the Quoting service in kernel space. In
+userspace, the Intel SGX Quoting Enclave verifies the TDREPORT locally and
+converts it to a Quote. The method of sending the TDREPORT to the QE and
+receiving the Quote is implementation-specific. If the TDX module supports the
+Quoting service, the kernel can convert a TDREPORT to a Quote directly through
+a SEAMCALL. In this case, the Quote is generated entirely by the TDX module.
 
 References
 ==========
Documentation/virt/kvm/api.rst +3 -0
index 52bbbb553ce1..4a3b69b2e602 100644
@@ -7335,6 +7335,9 @@ inputs and outputs of the TDVMCALL.  Currently the following values of
    queued successfully, the TDX guest can poll the status field in the
    shared-memory area to check whether the Quote generation is completed or
    not. When completed, the generated Quote is returned via the same buffer.
+   If the host kernel generates Quotes through the Quoting service provided by
+   the TDX module, KVM processes the GetQuote request and it will not appear in
+   userspace.
 
  * ``TDVMCALL_GET_TD_VM_CALL_INFO``: the guest has requested the support
    status of TDVMCALLs.  The output values for the given leaf should be
arch/x86/include/asm/tdx.h +9 -0
index 24bce7512de3..b9a24104415c 100644
@@ -86,6 +86,15 @@ struct tdx_quote_req {
 	u8 data[];
 };
 
+#define TDX_QUOTE_REQ_HDR_SIZE		(offsetof(struct tdx_quote_req, data))
+
+/*
+ * TDG.VP.VMCALL<GetQuote> Status Codes
+ */
+#define TDX_QUOTE_STATUS_SUCCESS	0x0000000000000000ULL
+#define TDX_QUOTE_STATUS_ERROR		0x8000000000000000ULL
+#define TDX_QUOTE_STATUS_UNAVAILABLE	0x8000000000000001ULL
+
 #ifdef CONFIG_INTEL_TDX_GUEST
 
 void __init tdx_early_init(void);
arch/x86/kvm/vmx/tdx.h +6 -0
index ac8323a68b16..5e4b3aee0577 100644
@@ -47,6 +47,12 @@ struct kvm_tdx {
 	 * Set/unset is protected with kvm->mmu_lock.
 	 */
 	bool wait_for_sept_zap;
+
+	/*
+	 * Whether to get the quote directly in kernel, without exiting to
+	 * userspace.
+	 */
+	bool get_quote_in_kernel;
 };
 
 /* TDX module vCPU states */
arch/x86/kvm/vmx/tdx.c +134 -1

158 lines not on this page · raw message

virt/kvm/kvm_main.c +1 -0
index 89489996fbc1..599f88a13071 100644
@@ -2461,6 +2461,7 @@ bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,
 
 	return true;
 }
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_range_has_memory_attributes);
 
 static __always_inline void kvm_handle_gfn_range(struct kvm *kvm,
 						 struct kvm_mmu_notifier_range *range)
Signature · 1 line
2.25.1
#86 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM incorrectly returns a hypercall error (TDVMCALL_STATUS_INVALID_OPERAND) when in-kernel quote…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM incorrectly returns a hypercall error (TDVMCALL_STATUS_INVALID_OPERAND) when in-kernel quote generation gracefully fails.
- [High] KVM fails to write the required buffer size back to the guest when the provided GetQuote buffer is too small. --

commit 482cde6e1c4a3f88b17ca20d55ddbc439d933981
Author: Peter Fang <[email protected]>

KVM: TDX: Add in-kernel Quote generation

Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.

Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +0 -0
index 9f7c39e0d4b59..20558b0185b63 100644

[ ... ]

Quoted 3 lines
arch/x86/kvm/vmx/tdx.c +3 -0
+static u64 get_quote_kernel(struct kvm_vcpu *vcpu, struct tdx_quote_req *req,
+			    gpa_t req_gpa, size_t total_len)
+{

[ ... ]

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1589

Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9 -0
+	/* Caller owns the requested quote */
+	void *quote_data __free(kvfree) =
+		tdx_quote_generate(td, req->data, req->in_len, &req->out_len);
+
+	if (!quote_data)
+		return TDX_QUOTE_STATUS_UNAVAILABLE;
+
+	if ((size_t)req->out_len + TDX_QUOTE_REQ_HDR_SIZE > total_len)
+		return TDX_QUOTE_STATUS_ERROR;

[Severity: High]
If the guest-provided buffer is too small, does this early return skip writing the required size back to the guest?

When tdx_quote_generate() fails due to an insufficient buffer, it populates req->out_len with the required size. By returning TDX_QUOTE_STATUS_ERROR immediately here, it seems we bypass the logic that writes req->out_len back to the guest.

Will this break a guest's ability to dynamically probe the required quote size and retry if its initial allocation is too small?

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+
+	if (!write_quote_to_guest(vcpu, quote_data, req->out_len, req_gpa))
+		return TDX_QUOTE_STATUS_ERROR;
+
+	return TDX_QUOTE_STATUS_SUCCESS;
+}

[ ... ]

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+static int tdx_get_quote_kernel(struct kvm_vcpu *vcpu, u64 gpa, u64 size)
+{
+	void *first_page = NULL;
+	u64 err, qerr;
+
+	err = tdx_get_quote_check_args(vcpu, gpa, size);
+	if (err != TDVMCALL_STATUS_SUCCESS)
+		goto out;
+
+	err = TDVMCALL_STATUS_INVALID_OPERAND;

[ ... ]

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1652

Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+	qerr = get_quote_kernel(vcpu, first_page, (gpa_t)gpa, size);
+
+	if (write_quote_status_to_guest(vcpu, qerr, (gpa_t)gpa) &&
+	    qerr == TDX_QUOTE_STATUS_SUCCESS)
+		err = TDVMCALL_STATUS_SUCCESS;

[Severity: High]
Should the hypercall return SUCCESS even if the quote generation itself failed gracefully?

If get_quote_kernel() encounters a quote service error (like TDX_QUOTE_STATUS_UNAVAILABLE), this condition prevents err from being updated to TDVMCALL_STATUS_SUCCESS. As a result, the hypercall returns the initialized value TDVMCALL_STATUS_INVALID_OPERAND in R10.

Since the hypercall arguments were structurally valid and we successfully wrote the error status into the shared memory buffer, shouldn't the hypercall return SUCCESS to let the guest parse the actual quote status?

Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +7 -0
+
+out:
+	kfree(first_page);
+	tdvmcall_set_return_code(vcpu, err);
+
+	return 1;
+}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
#87 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 19 lines · On Thu, Jun 18, 2026 at 09:03:09AM +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM incorrectly returns a hypercall error (TDVMCALL_STATUS_INVALID_OPERAND) when in-kernel quote generation gracefully fails.
- [High] KVM fails to write the required buffer size back to the guest when the provided GetQuote buffer is too small. --

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1589

Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9 -0
+	/* Caller owns the requested quote */
+	void *quote_data __free(kvfree) =
+		tdx_quote_generate(td, req->data, req->in_len, &req->out_len);
+
+	if (!quote_data)
+		return TDX_QUOTE_STATUS_UNAVAILABLE;
+
+	if ((size_t)req->out_len + TDX_QUOTE_REQ_HDR_SIZE > total_len)
+		return TDX_QUOTE_STATUS_ERROR;

[Severity: High]
If the guest-provided buffer is too small, does this early return skip writing the required size back to the guest?

Quoted 7 lines

When tdx_quote_generate() fails due to an insufficient buffer, it populates req->out_len with the required size. By returning TDX_QUOTE_STATUS_ERROR immediately here, it seems we bypass the logic that writes req->out_len back to the guest.

Will this break a guest's ability to dynamically probe the required quote size and retry if its initial allocation is too small?

Returning TDX_QUOTE_STATUS_ERROR immediately should be ok. The guest is expected to get the required size from a TD metadata field, not from the returned out_len here.

Quoted 20 lines
Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+
+	if (!write_quote_to_guest(vcpu, quote_data, req->out_len, req_gpa))
+		return TDX_QUOTE_STATUS_ERROR;
+
+	return TDX_QUOTE_STATUS_SUCCESS;
+}

[ ... ]

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1652

Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+	qerr = get_quote_kernel(vcpu, first_page, (gpa_t)gpa, size);
+
+	if (write_quote_status_to_guest(vcpu, qerr, (gpa_t)gpa) &&
+	    qerr == TDX_QUOTE_STATUS_SUCCESS)
+		err = TDVMCALL_STATUS_SUCCESS;

[Severity: High]
Should the hypercall return SUCCESS even if the quote generation itself failed gracefully?

Quoted 8 lines

If get_quote_kernel() encounters a quote service error (like TDX_QUOTE_STATUS_UNAVAILABLE), this condition prevents err from being updated to TDVMCALL_STATUS_SUCCESS. As a result, the hypercall returns the initialized value TDVMCALL_STATUS_INVALID_OPERAND in R10.

Since the hypercall arguments were structurally valid and we successfully wrote the error status into the shared memory buffer, shouldn't the hypercall return SUCCESS to let the guest parse the actual quote status?

Good point. The hypercall should return TDVMCALL_STATUS_SUCCESS on TDX_QUOTE_STATUS_UNAVAILABLE. I'll fix this in the next revision.

Quoted 4 lines
Signature · 2 lines
-- 
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
#88 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

In reply to Xu Yilun
Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>

Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.

Why?

#89 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 8 lines · Sean Christopherson
Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>

Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.

Why?

Hi Sean,

This is mainly to avoid a round trip to userspace for the GetQuote flow.

New TDX modules can now get a Quote directly via an "extension SEAMCALL" instead of exiting to userspace and using an SGX enclave. Exiting to userspace for GetQuote no longer seems worth the overhead/complexity.

The first half of the series enables extension SEAMCALLs. They implement simple APIs for higher-order security protocols that would otherwise need to be broken into smaller routines. For Quoting, this allows KVM to get a Quote directly through TDH.QUOTE.GET. The TDX module needs only the input data from TDG.VP.VMCALL<GetQuote> for that call.

Thanks,
Peter

#90 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 7 lines · Peter Fang
Quoted 6 lines · Sean Christopherson
Quoted 4 lines · Xu Yilun

From: Peter Fang <[email protected]>

Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.

Quoted 6 lines
Quoted 2 lines

Why?

Hi Sean,

This is mainly to avoid a round trip to userspace for the GetQuote flow.

Again, why?

Quoted 3 lines

New TDX modules can now get a Quote directly via an "extension SEAMCALL" instead of exiting to userspace and using an SGX enclave. Exiting to userspace for GetQuote no longer seems worth the overhead/complexity.

I dunno, from a kernel perspective, this is more complexity, not less:

 Documentation/arch/x86/tdx.rst |  19 ++---
 Documentation/virt/kvm/api.rst |   3 +
 arch/x86/include/asm/tdx.h     |   9 +++
 arch/x86/kvm/vmx/tdx.h         |   6 ++
 arch/x86/kvm/vmx/tdx.c         | 135 ++++++++++++++++++++++++++++++++-
 virt/kvm/kvm_main.c            |   1 +
 6 files changed, 163 insertions(+), 10 deletions(-)
Quoted 5 lines

The first half of the series enables extension SEAMCALLs. They implement simple APIs for higher-order security protocols that would otherwise need to be broken into smaller routines. For Quoting, this allows KVM to get a Quote directly through TDH.QUOTE.GET. The TDX module needs only the input data from TDG.VP.VMCALL<GetQuote> for that call.

Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.

If that's all there is too this, then why is KVM involved? I.e. why doesn't the TDX Module provide the quote directly to the guest?

#91 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 6 lines · Sean Christopherson

Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.

If that's all there is too this, then why is KVM involved?  I.e. why doesn't the TDX Module provide the quote directly to the guest?

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting, so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.

We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.

The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.

#92 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 17 lines · Rick P
Quoted 6 lines · Sean Christopherson

Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.

If that's all there is too this, then why is KVM involved?  I.e. why doesn't the TDX Module provide the quote directly to the guest?

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting, so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.

Quoted 8 lines

We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.

The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.

tl;dr: FWIW, host, if only because this is one of many blob transports across multiple archs that need host coordination.

While TDG calls to do the same helps a TDX problem it does not generally reduce the Linux problem of multiple archs having multiple blob protocols to shuffle data from host to guest in shared memory.

The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI attestation collateral) is shift the burden from arch/*/kvm/ to drivers/. There might be a later opportunity to consolidate some of those drivers' internals on a cross-arch generic transport (AF_VSOCK), but there remains a requirement to talk to the host.

#93 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 9 lines · Dan Williams (nvidia)
Quoted 8 lines · Rick P
Quoted 6 lines · Sean Christopherson

Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.

If that's all there is too this, then why is KVM involved?  I.e. why doesn't the TDX Module provide the quote directly to the guest?

Quoted 6 lines
Quoted 6 lines

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,

What is this magical resource?

Quoted 2 lines
Quoted 2 lines

so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.

At the risk of unintentionally causing effecitvely DoS by introducing a system- wide lock.

Quoted 3 lines
Quoted 3 lines

We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.

Simpler for what?

Quoted 11 lines
Quoted 4 lines

The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.

tl;dr: FWIW, host, if only because this is one of many blob transports across multiple archs that need host coordination.

While TDG calls to do the same helps a TDX problem it does not generally reduce the Linux problem of multiple archs having multiple blob protocols to shuffle data from host to guest in shared memory.

Why is that KVM's problem?

Quoted 5 lines

The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI attestation collateral) is shift the burden from arch/*/kvm/ to drivers/. There might be a later opportunity to consolidate some of those drivers' internals on a cross-arch generic transport (AF_VSOCK), but there remains a requirement to talk to the host.

Why?

Bluntly, this series is unreviewable. I've speed read through most of the patches, and I have *zero* clue as to *why* any of the design decisions were made. E.g. why is there a common buffer that's apparently shared by all CPUs and VMs, only for the API to immediately allocate per-quote memory and copy out the buffer?

The changelogs are all heavy on the "what", but explanations of why any of this is being done, and in the exact way it's being done, are non-existent.

So, back up to the beginning and explain why I should spend any time deciphering this series, because as-is I don't have the interest or the time.

#94 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,

What is this magical resource?

It's a HW crypto thing. I'll let Peter explain more.

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.

At the risk of unintentionally causing effecitvely DoS by introducing a system-wide lock.

The DoS that people are interested in trying to prevent is a TD getting starved of quotes forever. So the lock is supposed to give some eventual fairness. Eventually the guest gets its quote even if it has to wait.

If instead contention throws a BUSY code to the guest, the guest spins trying. That wouldn't have the guarantees.

So I think the two options are: have some lock in the TDX module, or have a lock of some sort in the host. I think actually a waiting lock in the TDX module is possible. But I think there are tradeoffs besides where the extra code is. If it's in Linux we get a lot more control, lockdep, etc.

BTW the "historical reasons" part I mentioned involves a past effort to create configurable host controlled policies around managing these resources. So part of the simplification is doing a simple eventually fair global lock instead of something more complicated.

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.

Simpler for what?

Linux of course.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.

Quoted 8 lines
Quoted 6 lines

tl;dr: FWIW, host, if only because this is one of many blob transports across multiple archs that need host coordination.

While TDG calls to do the same helps a TDX problem it does not generally reduce the Linux problem of multiple archs having multiple blob protocols to shuffle data from host to guest in shared memory.

Why is that KVM's problem?

I'm slightly lost on this too, so will leave it to Dan.

Quoted 7 lines
Quoted 6 lines

The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI attestation collateral) is shift the burden from arch/*/kvm/ to drivers/. There might be a later opportunity to consolidate some of those drivers' internals on a cross-arch generic transport (AF_VSOCK), but there remains a requirement to talk to the host.

Quoted 7 lines

Why?

Bluntly, this series is unreviewable.  I've speed read through most of the patches, and I have *zero* clue as to *why* any of the design decisions were made.  E.g. why is there a common buffer that's apparently shared by all CPUs and VMs, only for the API to immediately allocate per-quote memory and copy out the buffer?

We can keep discussing it, but...

Quoted 6 lines

The changelogs are all heavy on the "what", but explanations of why any of this is being done, and in the exact way it's being done, are non-existent.

So, back up to the beginning and explain why I should spend any time deciphering this series, because as-is I don't have the interest or the time.

...it sounds like it would be better if we tried again with a proper explanation, rather than try to back out of where the current presentation left you?

This should be a pretty simple feature. But there is a higher level theme of what is responsible for locks and scheduling in the overall TDX solution. This spills into a lot of future stuff. We can try to find the right level.

#95 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 11 lines · Rick P
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,

What is this magical resource?

It's a HW crypto thing. I'll let Peter explain more.

It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically provides a number of boot and security services through dedicated HW. Attestation is one of them. This is part of our plan to develop new attestation schemes without using SGX.

There was a lot of internal debate about how this HW should be managed. We decided to let the TDX module handle it and sort of hide it from the host kernel and only expose it as SEAMCALLs. It however impacted how the quoting SEAMCALLs look, so apologies for not calling out these factors.

Quoted 21 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.

At the risk of unintentionally causing effecitvely DoS by introducing a system-wide lock.

The DoS that people are interested in trying to prevent is a TD getting starved of quotes forever. So the lock is supposed to give some eventual fairness. Eventually the guest gets its quote even if it has to wait.

If instead contention throws a BUSY code to the guest, the guest spins trying. That wouldn't have the guarantees.

So I think the two options are: have some lock in the TDX module, or have a lock of some sort in the host. I think actually a waiting lock in the TDX module is possible. But I think there are tradeoffs besides where the extra code is. If it's in Linux we get a lot more control, lockdep, etc.

Quoted 4 lines

BTW the "historical reasons" part I mentioned involves a past effort to create configurable host controlled policies around managing these resources. So part of the simplification is doing a simple eventually fair global lock instead of something more complicated.

Yeah my bad for not including a lot more background. There were several design aspects that influenced this implementation. I'll do a better job of explaining them in the next version, but basically:

1. Performance requirements: For this first implementation, the priority is to not starve a legit guest from getting a quote, even when there's a malicious guest DoSing the host. A mutex is sort of a "fair enough queue" for this problem. A guest's authenticity can only be verified after the quote is generated.

2. HW limitation: The TDX module implements different attestation schemes, some of which use the S3M during TDH.QUOTE.GET. This limits the amount of concurrency this SEAMCALL supports and is also part of why we chose a mutex for TDH.QUOTE.GET. (The shared buffer in core TDX doesn't seem great, so I will improve it.)

3. Host policies: Preserving the GetQuote GHCI also allows the host to participate in the quoting process, e.g. add host-controlled policies or fetch host-side quote information. It's a bit similar to SEV-SNP's KVM_EXIT_SNP_REQ_CERTS [1]. This series does not implement host policies though.

Ultimately, there are ways to push more of this into the module, and the current design is kind of a tradeoff between the complexities of the work needed in the guest, the host, and the TDX module. Like Rick said, there were past versions that required way more work in the host and the guest, such as managing the S3M, new GHCI calls, etc.

[1] Commit fa9893fadbc2 ("KVM: Introduce KVM_EXIT_SNP_REQ_CERTS for SNP certificate-fetching"

Quoted 1 line
#96 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 54 lines · Peter Fang
Quoted 11 lines · Rick P
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,

What is this magical resource?

It's a HW crypto thing. I'll let Peter explain more.

It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically provides a number of boot and security services through dedicated HW. Attestation is one of them. This is part of our plan to develop new attestation schemes without using SGX.

There was a lot of internal debate about how this HW should be managed. We decided to let the TDX module handle it and sort of hide it from the host kernel and only expose it as SEAMCALLs. It however impacted how the quoting SEAMCALLs look, so apologies for not calling out these factors.

Quoted 21 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.

At the risk of unintentionally causing effecitvely DoS by introducing a system-wide lock.

The DoS that people are interested in trying to prevent is a TD getting starved of quotes forever. So the lock is supposed to give some eventual fairness. Eventually the guest gets its quote even if it has to wait.

If instead contention throws a BUSY code to the guest, the guest spins trying. That wouldn't have the guarantees.

So I think the two options are: have some lock in the TDX module, or have a lock of some sort in the host. I think actually a waiting lock in the TDX module is possible. But I think there are tradeoffs besides where the extra code is. If it's in Linux we get a lot more control, lockdep, etc.

Quoted 4 lines

BTW the "historical reasons" part I mentioned involves a past effort to create configurable host controlled policies around managing these resources. So part of the simplification is doing a simple eventually fair global lock instead of something more complicated.

Yeah my bad for not including a lot more background. There were several design aspects that influenced this implementation. I'll do a better job of explaining them in the next version, but basically:

Quoted 8 lines

1. Performance requirements: For this first implementation, the priority is to not starve a legit guest from getting a quote, even when there's a malicious guest DoSing the host. A mutex is sort of a "fair enough queue" for this problem. A guest's authenticity can only be verified after the quote is generated.

2. HW limitation: The TDX module implements different attestation schemes, some of which use the S3M during TDH.QUOTE.GET. This

What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core? How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at? What else uses the S3M? Do we have to worry about contending with non-TDX usage?

Quoted 21 lines

limits the amount of concurrency this SEAMCALL supports and is also part of why we chose a mutex for TDH.QUOTE.GET. (The shared buffer in core TDX doesn't seem great, so I will improve it.)

3. Host policies: Preserving the GetQuote GHCI also allows the host to participate in the quoting process, e.g. add host-controlled policies or fetch host-side quote information. It's a bit similar to SEV-SNP's KVM_EXIT_SNP_REQ_CERTS [1]. This series does not implement host policies though.

Ultimately, there are ways to push more of this into the module, and the current design is kind of a tradeoff between the complexities of the work needed in the guest, the host, and the TDX module. Like Rick said, there were past versions that required way more work in the host and the guest, such as managing the S3M, new GHCI calls, etc.

[1] Commit fa9893fadbc2 ("KVM: Introduce KVM_EXIT_SNP_REQ_CERTS for SNP certificate-fetching"

Quoted 1 line
#97 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 2 lines · Sean Christopherson

What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?

I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.

Quoted 2 lines

How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?

As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's no ioremap() because the I/O is hidden in the TDX module.

The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.

For TDX, though, the craziness is mostly hidden in the TDX module.

Quoted 2 lines

What else uses the S3M? Do we have to worry about contending with non-TDX usage?

There _are_ different users of S3M. But each of them should get their own I/O address and S3M firmware has to handle talking to those different users at the same time. The TDX I/O window is owned exclusively by the TDX module.

So, while S3M has and long and growing list of jobs, the random software (like the host kernel) poking at one I/O window doesn't have to know about the other piece of software (the TDX module) poking at another.

I'm sure I got a detail or two wrong in there, so folks that know this
better: please correct me. But I think that's a halfway-decent 10,000ft view.

#98 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 31 lines · Dave Hansen
Quoted 2 lines · Sean Christopherson

What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?

I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.

Quoted 2 lines

How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?

As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's no ioremap() because the I/O is hidden in the TDX module.

The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.

For TDX, though, the craziness is mostly hidden in the TDX module.

Quoted 2 lines

What else uses the S3M? Do we have to worry about contending with non-TDX usage?

There _are_ different users of S3M. But each of them should get their own I/O address and S3M firmware has to handle talking to those different users at the same time. The TDX I/O window is owned exclusively by the TDX module.

Quoted 7 lines

So, while S3M has and long and growing list of jobs, the random software (like the host kernel) poking at one I/O window doesn't have to know about the other piece of software (the TDX module) poking at another.

I'm sure I got a detail or two wrong in there, so folks that know this
better: please correct me. But I think that's a halfway-decent 10,000ft view.

Thank you, very helpful!

#99 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

In reply to Dave Hansen
Quoted 17 lines · Dave Hansen
Quoted 2 lines · Sean Christopherson

What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?

I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.

Quoted 2 lines

How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?

As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's no ioremap() because the I/O is hidden in the TDX module.

Quoted 6 lines

The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.

For TDX, though, the craziness is mostly hidden in the TDX module.

I'll just add that this design decision comes from the fact that we're looking at different kinds of future attestation stuff. Some of it is quite complex (like post-quantum crypto). Some of it may even require more complicated interactions with the S3M. Hiding these details allows all this crypto stuff to look like just another TDX module release. The host kernel shouldn't be burdened with attestation intricacies, and the TDX module should just make sure the new module "works".

Quoted 8 lines
Quoted 3 lines

What else uses the S3M? Do we have to worry about contending with non-TDX usage?

There _are_ different users of S3M. But each of them should get their own I/O address and S3M firmware has to handle talking to those different users at the same time. The TDX I/O window is owned exclusively by the TDX module.

Quoted 7 lines

So, while S3M has and long and growing list of jobs, the random software (like the host kernel) poking at one I/O window doesn't have to know about the other piece of software (the TDX module) poking at another.

I'm sure I got a detail or two wrong in there, so folks that know this
better: please correct me. But I think that's a halfway-decent 10,000ft view.

This is pretty spot on based on my understanding. Thanks Dave :)

#100 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 23 lines · Peter Fang
Quoted 22 lines · Dave Hansen
Quoted 2 lines · Sean Christopherson

What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?

I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.

Quoted 2 lines

How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?

As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's no ioremap() because the I/O is hidden in the TDX module.

The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.

Quoted 7 lines
Quoted 1 line

For TDX, though, the craziness is mostly hidden in the TDX module.

I'll just add that this design decision comes from the fact that we're looking at different kinds of future attestation stuff. Some of it is quite complex (like post-quantum crypto). Some of it may even require more complicated interactions with the S3M. Hiding these details allows all this crypto stuff to look like just another TDX module release.

IMO, that's not a good thing. Burying more code and more complexity in what is supposed to be a minimal TCB doesn't seem like a "win". I can certainly appreciate that interacting with the S3M microcontroller isn't something we want in KVM, but that's why drivers/ exists.

Quoted 1 line

The host kernel shouldn't be burdened with attestation intricacies,

Oh, but the kernel has to be, at least to the point where the kernel has enough information to guarantee backwards compatibility, enumerate choices, etc. If you're saying that the TDX module will always generate reports that are compatible across TDX Module updates, then we probably don't have anything to worry about, but it seems unlikely everything can Just Work while also updating the core crypto algorithms and whatnot.

Quoted 1 line

and the TDX module should just make sure the new module "works".

#101 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 10 lines · Sean Christopherson
Quoted 5 lines

I'll just add that this design decision comes from the fact that we're looking at different kinds of future attestation stuff. Some of it is quite complex (like post-quantum crypto). Some of it may even require more complicated interactions with the S3M. Hiding these details allows all this crypto stuff to look like just another TDX module release.

IMO, that's not a good thing.  Burying more code and more complexity in what is supposed to be a minimal TCB doesn't seem like a "win".  I can certainly appreciate that interacting with the S3M microcontroller isn't something we want in KVM, but that's why drivers/ exists.

This is an interesting take and makes me think even more that we need to be socializing the TDX arch stuff a lot more.

Quoted 9 lines
Quoted 2 lines

The host kernel shouldn't be burdened with attestation intricacies,

Oh, but the kernel has to be, at least to the point where the kernel has enough information to guarantee backwards compatibility, enumerate choices, etc.  If you're saying that the TDX module will always generate reports that are compatible across TDX Module updates, then we probably don't have anything to worry about, but it seems unlikely everything can Just Work while also updating the core crypto algorithms and whatnot.

There are absolutely some ABI sensitivity here that need to be considered, and unfortunately the analysis from the internal debates did not make into the public presentation of the feature. In fact the whole DICE part of this series is missing a lot of needed context.

I think I mentioned that we want to come to talk about migration in PUCK, it might help to bring this one in too. Despite being small, it actually took quite a bit for us to wrap our heads around it. Would you want to try that? Or we could fill out the logs and coverletter in v3. Or we could just try to slowly walk through it all here. I was thinking a call may shorten the loops.

#102 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

In reply to Peter Fang
Quoted 9 lines · Peter Fang
Quoted 8 lines · Rick P
Quoted 7 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

What is this magical resource?

It's a HW crypto thing. I'll let Peter explain more.

It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically

Are you referring to this: https://www.scribd.com/document/970439243/S3M-Intel-White-Paper ?

<snip>

#103 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 14 lines · Nikolay Borisov
Quoted 11 lines · Peter Fang
Quoted 10 lines · Rick P
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines

That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.

The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,

What is this magical resource?

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

It's a HW crypto thing. I'll let Peter explain more.

It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically

Are you referring to this:
https://www.scribd.com/document/970439243/S3M-Intel-White-Paper ?

Yep. The internet never forgets :)

Quoted 2 lines

<snip>

#104 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

In reply to Xu Yilun
Quoted 6 lines · Xu Yilun

void __init tdx_early_init(void);

arch/x86/kvm/vmx/tdx.h +0 -0
index ac8323a68b16..5e4b3aee0577 100644

arch/x86/kvm/vmx/tdx.h struct kvm_tdx { · line 55

Quoted 9 lines
arch/x86/kvm/vmx/tdx.h +6 -0
@@ -47,6 +47,12 @@ struct kvm_tdx {
 	 * Set/unset is protected with kvm->mmu_lock.
 	 */
 	bool wait_for_sept_zap;
+
+	/*
+	 * Whether to get the quote directly in kernel, without exiting to
+	 * userspace.
+	 */
+	bool get_quote_in_kernel;

Why is this cached per-VM? IIUC, tdx_quote_enabled() is constant after initial TDX bringup.

And is keying off tdx_quote_enabled() and only tdx_quote_enabled() backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?

Quoted 11 lines
arch/x86/kvm/vmx/tdx.h +0 -0
 };
 
 /* TDX module vCPU states */
arch/x86/kvm/vmx/tdx.c +0 -0
index 9f7c39e0d4b5..20558b0185b6 100644
@@ -1538,11 +1538,133 @@ static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size)
 	return 0;
 }
 

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1550

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+static bool write_quote_status_to_guest(struct kvm_vcpu *vcpu, u64 status,
+					gpa_t gpa)
+{
+	if (kvm_vcpu_write_guest(vcpu,
+				 gpa + offsetof(struct tdx_quote_req, status),
+				 &status, sizeof(status)))
+		return false;
+
+	return true;
+}

This is a completely useless wrapper. Actually, it's worse than useless, as it inverts the return value from the widely used kvm_vcpu_write_guest() API.

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1556

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+
+static bool write_quote_to_guest(struct kvm_vcpu *vcpu, void *quote_data,
+				 u32 quote_len, gpa_t gpa)
+{
+	if (kvm_vcpu_write_guest(vcpu,
+				 gpa + TDX_QUOTE_REQ_HDR_SIZE,

This is unnecessarily confusing. I had to look at the definition of TDX_QUOTE_REQ_HDR_SIZE to understand that this just writing to tdx_quote_req.data. I might not care if this was the only write, but then the very next statement writes to a field that isn't obviously in the header.

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1566

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+				 quote_data, quote_len))
+		return false;
+
+	if (kvm_vcpu_write_guest(vcpu,
+				 gpa + offsetof(struct tdx_quote_req, out_len),
+				 &quote_len, sizeof(quote_len)))
+		return false;
+
+	return true;
+}

So is this. There's literally one caller.

Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +7 -0
+
+static u64 get_quote_kernel(struct kvm_vcpu *vcpu, struct tdx_quote_req *req,
+			    gpa_t req_gpa, size_t total_len)
+{
+	struct tdx_td *td = &to_kvm_tdx(vcpu->kvm)->td;
+
+	/* Only support version 1 as defined in the GHCI spec */

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1583

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+	if (req->version != 1)
+		return TDX_QUOTE_STATUS_ERROR;
+
+	/* Header + input data must fit in the page read from guest memory */
+	if ((size_t)req->in_len + TDX_QUOTE_REQ_HDR_SIZE > PAGE_SIZE)
+		return TDX_QUOTE_STATUS_ERROR;
+
+	/* Caller owns the requested quote */
+	void *quote_data __free(kvfree) =
+		tdx_quote_generate(td, req->data, req->in_len, &req->out_len);

Am I the only person that finds the __free() stuff to be difficult to follow? I'm sure there are flows where it's a big net positive, but IMO this isn't one. If the *unnecesary* layers of single-use helpers are stripped away, I suspect we'll naturally end up with a goto sequence that is more obvious that using __free().

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1588

Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+
+	if (!quote_data)
+		return TDX_QUOTE_STATUS_UNAVAILABLE;
+
+	if ((size_t)req->out_len + TDX_QUOTE_REQ_HDR_SIZE > total_len)

Can we really not pre-check the size?

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1597

Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9 -0
+		return TDX_QUOTE_STATUS_ERROR;
+
+	if (!write_quote_to_guest(vcpu, quote_data, req->out_len, req_gpa))
+		return TDX_QUOTE_STATUS_ERROR;
+
+	return TDX_QUOTE_STATUS_SUCCESS;
+}
+
+static u64 tdx_get_quote_check_args(struct kvm_vcpu *vcpu, u64 gpa, u64 size)

Adding single use helpers does not make the code easier to read.

Quoted 12 lines
arch/x86/kvm/vmx/tdx.c +12 -0
+{
+	gfn_t gfn_start, gfn_end;
+	u64 end;
+
+	if (!size)
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	if (!PAGE_ALIGNED(gpa) || !PAGE_ALIGNED(size))
+		return TDVMCALL_STATUS_ALIGN_ERROR;
+
+	if (check_add_overflow(gpa, size, &end))
+		return TDVMCALL_STATUS_INVALID_OPERAND;

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+
+	gfn_start = gpa_to_gfn(gpa);
+	gfn_end = gpa_to_gfn(end);
+
+	/*
+	 * Reject if the guest didn't explicitly convert its quote pages to
+	 * shared.
+	 */
+	if (!kvm_range_has_memory_attributes(vcpu->kvm, gfn_start, gfn_end,
+					     KVM_MEMORY_ATTRIBUTE_PRIVATE, 0))

TOCTOU?

Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	return TDVMCALL_STATUS_SUCCESS;
+}
#105 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 11 lines · Sean Christopherson
Quoted 10 lines · Xu Yilun

void __init tdx_early_init(void);

arch/x86/kvm/vmx/tdx.h +1 -0
index ac8323a68b16..5e4b3aee0577 100644
@@ -47,6 +47,12 @@ struct kvm_tdx {
 	 * Set/unset is protected with kvm->mmu_lock.
 	 */
 	bool wait_for_sept_zap;
+
Quoted 8 lines

arch/x86/kvm/vmx/tdx.h struct kvm_tdx { · line 55

Quoted 5 lines
arch/x86/kvm/vmx/tdx.h +5 -0
+	/*
+	 * Whether to get the quote directly in kernel, without exiting to
+	 * userspace.
+	 */
+	bool get_quote_in_kernel;

Why is this cached per-VM? IIUC, tdx_quote_enabled() is constant after initial TDX bringup.

Ah you're right. It shouldn't change. I think I did this because there was an older version where quoting was not enabled during core TDX bringup. So this was an attempt to make sure a guest doesn't see tdx_quote_enabled() flip. It shouldn't be here. Sorry about that.

Quoted 4 lines

And is keying off tdx_quote_enabled() and only tdx_quote_enabled() backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?

Yeah this needs explaining:

The guest-observable difference between the two paths is that it sees quotes in different formats (the TDX module uses newer formats of course). And an old guest must be able to continue using its TDVMCALL<GetQuote> interface for quotes. The guest sends the quote to an external verifier to authenticate itself. The verifier (part of the cloud infrastructure) must know how to parse this format. So in other words, it's the verifier that must maintain backward compatibility if it wants to continue accepting old quotes. And it needs to be updated to support newer formats as well.

Quoted 13 lines
Quoted 13 lines
arch/x86/kvm/vmx/tdx.h +0 -0
 };
 
 /* TDX module vCPU states */
arch/x86/kvm/vmx/tdx.c +0 -0
index 9f7c39e0d4b5..20558b0185b6 100644
@@ -1538,11 +1538,133 @@ static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size)
 	return 0;
 }
 
Quoted 13 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1550

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+static bool write_quote_status_to_guest(struct kvm_vcpu *vcpu, u64 status,
+					gpa_t gpa)
+{
+	if (kvm_vcpu_write_guest(vcpu,
+				 gpa + offsetof(struct tdx_quote_req, status),
+				 &status, sizeof(status)))
+		return false;
+
+	return true;
+}

This is a completely useless wrapper. Actually, it's worse than useless, as it inverts the return value from the widely used kvm_vcpu_write_guest() API.

I'll remove this. I got feedback that tdx_get_quote_kernel() was full of kvm_vcpu_write_guest()s and was hard to read, so I tried to break it apart. Looks like it didn't help.

Quoted 12 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1556

Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+
+static bool write_quote_to_guest(struct kvm_vcpu *vcpu, void *quote_data,
+				 u32 quote_len, gpa_t gpa)
+{
+	if (kvm_vcpu_write_guest(vcpu,
+				 gpa + TDX_QUOTE_REQ_HDR_SIZE,

This is unnecessarily confusing. I had to look at the definition of TDX_QUOTE_REQ_HDR_SIZE to understand that this just writing to tdx_quote_req.data. I might not care if this was the only write, but then the very next statement writes to a field that isn't obviously in the header.

I'll fix this. I can just spell out offsetof() to make this consistent.

Quoted 13 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1566

Quoted 11 lines
arch/x86/kvm/vmx/tdx.c +10 -0
+				 quote_data, quote_len))
+		return false;
+
+	if (kvm_vcpu_write_guest(vcpu,
+				 gpa + offsetof(struct tdx_quote_req, out_len),
+				 &quote_len, sizeof(quote_len)))
+		return false;
+
+	return true;
+}

So is this. There's literally one caller.

Yeah breaking the kvm_vcpu_write_guest()s apart didn't help...

Quoted 15 lines
Quoted 15 lines
arch/x86/kvm/vmx/tdx.c +14 -0
+
+static u64 get_quote_kernel(struct kvm_vcpu *vcpu, struct tdx_quote_req *req,
+			    gpa_t req_gpa, size_t total_len)
+{
+	struct tdx_td *td = &to_kvm_tdx(vcpu->kvm)->td;
+
+	/* Only support version 1 as defined in the GHCI spec */
+	if (req->version != 1)
+		return TDX_QUOTE_STATUS_ERROR;
+
+	/* Header + input data must fit in the page read from guest memory */
+	if ((size_t)req->in_len + TDX_QUOTE_REQ_HDR_SIZE > PAGE_SIZE)
+		return TDX_QUOTE_STATUS_ERROR;
+
Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1583

Quoted 3 lines
arch/x86/kvm/vmx/tdx.c +3 -0
+	/* Caller owns the requested quote */
+	void *quote_data __free(kvfree) =
+		tdx_quote_generate(td, req->data, req->in_len, &req->out_len);

Am I the only person that finds the __free() stuff to be difficult to follow? I'm sure there are flows where it's a big net positive, but IMO this isn't one. If the *unnecesary* layers of single-use helpers are stripped away, I suspect we'll naturally end up with a goto sequence that is more obvious that using __free().

The main concern I had was that the guest could give random message lengths and mess with the host's memory allocator. Core TDX has a better idea because it knows the maximum possible quote size through a metadata field, so I just let the API return the final quote from the TDX module. It's a bit annoying that the caller now has to free it.

Several folks suggested letting KVM allocate and free the buffer, so I think I'll do that. It should make this easier to read, and this __free() can be replaced with a much simpler pattern.

Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1588

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +5 -0
+
+	if (!quote_data)
+		return TDX_QUOTE_STATUS_UNAVAILABLE;
+
+	if ((size_t)req->out_len + TDX_QUOTE_REQ_HDR_SIZE > total_len)

Can we really not pre-check the size?

Yeah like the response above, KVM doesn't know what a "sane" total_len is here. out_len comes from the TDX module so I used it to check against total_len. If we let KVM allocate+free the quote buffer, then I'll also let KVM get the maximum quote size from core TDX. I was trying to save some code in KVM but looks like it's not worth it.

Quoted 12 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1597

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +9 -0
+		return TDX_QUOTE_STATUS_ERROR;
+
+	if (!write_quote_to_guest(vcpu, quote_data, req->out_len, req_gpa))
+		return TDX_QUOTE_STATUS_ERROR;
+
+	return TDX_QUOTE_STATUS_SUCCESS;
+}
+
+static u64 tdx_get_quote_check_args(struct kvm_vcpu *vcpu, u64 gpa, u64 size)

Adding single use helpers does not make the code easier to read.

Yep. Will fix.

Quoted 17 lines
Quoted 17 lines
arch/x86/kvm/vmx/tdx.c +16 -0
+{
+	gfn_t gfn_start, gfn_end;
+	u64 end;
+
+	if (!size)
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	if (!PAGE_ALIGNED(gpa) || !PAGE_ALIGNED(size))
+		return TDVMCALL_STATUS_ALIGN_ERROR;
+
+	if (check_add_overflow(gpa, size, &end))
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	gfn_start = gpa_to_gfn(gpa);
+	gfn_end = gpa_to_gfn(end);
+
Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+	/*
+	 * Reject if the guest didn't explicitly convert its quote pages to
+	 * shared.
+	 */
+	if (!kvm_range_has_memory_attributes(vcpu->kvm, gfn_start, gfn_end,
+					     KVM_MEMORY_ATTRIBUTE_PRIVATE, 0))

TOCTOU?

I struggled a bit with this actually... I can't just grab kvm->slots_lock here, and dropping kvm->srcu seems bad because the quote will be written to guest memory later. I settled on the idea that the guest should make sure these pages are converted to shared first. If it tries to convert them back to private after this check, before this TDVMCALL returns, then it's kind of a self-inflicted race and I don't see KVM doing any self damage in this case. Not sure if this makes sense to you? Or maybe I should just drop this check and go straight to kvm_vcpu_read_guest()?

Thanks,
Peter

Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +4 -0
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	return TDVMCALL_STATUS_SUCCESS;
+}
#106 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 8 lines · Peter Fang
Quoted 4 lines · Sean Christopherson

On Thu, Jun 18, 2026, Xu Yilun wrote:
And is keying off tdx_quote_enabled() and only tdx_quote_enabled() backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?

Yeah this needs explaining:

Quoted 9 lines

The guest-observable difference between the two paths is that it sees quotes in different formats (the TDX module uses newer formats of course). And an old guest must be able to continue using its TDVMCALL<GetQuote> interface for quotes. The guest sends the quote to an external verifier to authenticate itself. The verifier (part of the cloud infrastructure) must know how to parse this format. So in other words, it's the verifier that must maintain backward compatibility if it wants to continue accepting old quotes. And it needs to be updated to support newer formats as well.

Uh, no, that's not how KVM's guest/host ABI works. Updating the host kernel and breaking existing guest is not acceptable.

Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
arch/x86/kvm/vmx/tdx.c +16 -0
+{
+	gfn_t gfn_start, gfn_end;
+	u64 end;
+
+	if (!size)
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	if (!PAGE_ALIGNED(gpa) || !PAGE_ALIGNED(size))
+		return TDVMCALL_STATUS_ALIGN_ERROR;
+
+	if (check_add_overflow(gpa, size, &end))
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	gfn_start = gpa_to_gfn(gpa);
+	gfn_end = gpa_to_gfn(end);
+
Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+	/*
+	 * Reject if the guest didn't explicitly convert its quote pages to
+	 * shared.
+	 */
+	if (!kvm_range_has_memory_attributes(vcpu->kvm, gfn_start, gfn_end,
+					     KVM_MEMORY_ATTRIBUTE_PRIVATE, 0))

TOCTOU?

Quoted 9 lines

I struggled a bit with this actually... I can't just grab kvm->slots_lock here, and dropping kvm->srcu seems bad because the quote will be written to guest memory later. I settled on the idea that the guest should make sure these pages are converted to shared first. If it tries to convert them back to private after this check, before this TDVMCALL returns, then it's kind of a self-inflicted race and I don't see KVM doing any self damage in this case. Not sure if this makes sense to you? Or maybe I should just drop this check and go straight to kvm_vcpu_read_guest()?

Yes, drop it and rely on uaccess to do the right thing.

#107 lore raw

Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation

Quoted 19 lines · Sean Christopherson
Quoted 8 lines · Peter Fang
Quoted 4 lines · Sean Christopherson

On Thu, Jun 18, 2026, Xu Yilun wrote:
And is keying off tdx_quote_enabled() and only tdx_quote_enabled() backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?

Yeah this needs explaining:

Quoted 9 lines

The guest-observable difference between the two paths is that it sees quotes in different formats (the TDX module uses newer formats of course). And an old guest must be able to continue using its TDVMCALL<GetQuote> interface for quotes. The guest sends the quote to an external verifier to authenticate itself. The verifier (part of the cloud infrastructure) must know how to parse this format. So in other words, it's the verifier that must maintain backward compatibility if it wants to continue accepting old quotes. And it needs to be updated to support newer formats as well.

Quoted 2 lines

Uh, no, that's not how KVM's guest/host ABI works. Updating the host kernel and breaking existing guest is not acceptable.

Yeah I agree there's ABI sensitivity here. I can provide a bit more context here:

- When you get a quote you don't check it yourself. You pass it to a verifier service which does the check. The verifier checks if the platform is genuine. It has platform specific knowledge for each new platform. So with a new platform these services need to be updated to know about the new platform's quote, or attestation will fail for these guests.
- There may be platforms that support TDX but have SGX off for various reasons. Meaning SGX based attestation won't work and they will only support this new type of quote.
- SGX based attestation is very complicated for users. It will likely be deprecated long term. This new type of quote simplifies a lot of steps.

A risk could be that someone makes a quote verifying service that supports a new DICE capable platform halfway (SGX only), then the user does a kernel upgrade and sees their verifications start failing. But these halfway services would probably come to life after the kernel support is upstream since DICE capable platforms are not available yet. So not sure if that's a major concern for regression.

I should call out this is not a zero-risk ABI change and the risks should have been enumerated. The approach this revision takes is to have less uABI on the assumption that it will not impact any real users. So we need to consider whether the simplicity is worth it vs going to userspace and back. Is it worth considering to have less uABI? Or would you rather punt to userspace? BTW, the quote SEAMCALL does it for a specifc TD. So putting this new ABI in the TDX host service would require adding KVM's TD life cycle knowledge. So the "and back" part probably would fit best as a new KVM TDX ioctl.

Quoted 36 lines
Quoted 26 lines
Quoted 17 lines
Quoted 17 lines
arch/x86/kvm/vmx/tdx.c +16 -0
+{
+	gfn_t gfn_start, gfn_end;
+	u64 end;
+
+	if (!size)
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	if (!PAGE_ALIGNED(gpa) || !PAGE_ALIGNED(size))
+		return TDVMCALL_STATUS_ALIGN_ERROR;
+
+	if (check_add_overflow(gpa, size, &end))
+		return TDVMCALL_STATUS_INVALID_OPERAND;
+
+	gfn_start = gpa_to_gfn(gpa);
+	gfn_end = gpa_to_gfn(end);
+
Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+	/*
+	 * Reject if the guest didn't explicitly convert its quote pages to
+	 * shared.
+	 */
+	if (!kvm_range_has_memory_attributes(vcpu->kvm, gfn_start, gfn_end,
+					     KVM_MEMORY_ATTRIBUTE_PRIVATE, 0))

TOCTOU?

Quoted 9 lines

I struggled a bit with this actually... I can't just grab kvm->slots_lock here, and dropping kvm->srcu seems bad because the quote will be written to guest memory later. I settled on the idea that the guest should make sure these pages are converted to shared first. If it tries to convert them back to private after this check, before this TDVMCALL returns, then it's kind of a self-inflicted race and I don't see KVM doing any self damage in this case. Not sure if this makes sense to you? Or maybe I should just drop this check and go straight to kvm_vcpu_read_guest()?

Quoted 1 line

Yes, drop it and rely on uaccess to do the right thing.

Got it, thanks Sean.

author
#108 lore raw

PATCH v2 17/17 KVM: TDX: Support event-notify interrupts only with userspace Quoting

In reply to Xu Yilun

From: Peter Fang <[email protected]>

Tie userspace SetupEventNotifyInterrupt support to userspace Quote generation. Delivering event-notify interrupts via userspace breaks if KVM never exits to userspace in the first place.

This is an optional capability to notify the guest when Quoting has completed. No known guest currently uses it, so defer adding in-kernel support for now. The Linux TDX guest relies on polling only.

---
 arch/x86/kvm/vmx/tdx.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/tdx.c +17 -3
index 20558b0185b6..25146da3933f 100644
@@ -185,7 +185,7 @@ static void td_init_cpuid_entry2(struct kvm_cpuid_entry2 *entry, unsigned char i
 	tdx_clear_unsupported_cpuid(entry);
 }
 
-#define TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT	BIT(1)
+#define TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT	BIT_ULL(1)
 
 static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf,
 			     struct kvm_tdx_capabilities *caps)
@@ -202,8 +202,15 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf,
 
 	caps->cpuid.nent = td_conf->num_cpuid_config;
 
-	caps->user_tdvmcallinfo_1_r11 =
-		TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT;
+	/*
+	 * Don't advertise userspace event-notify interrupt support if TDX
+	 * quoting service is enabled, as quote generation will be handled
+	 * entirely in the kernel. Support in the kernel can be added later.
+	 */
+	if (!tdx_quote_enabled()) {
+		caps->user_tdvmcallinfo_1_r11 |=
+			TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT;
+	}
 
 	for (i = 0; i < td_conf->num_cpuid_config; i++)
 		td_init_cpuid_entry2(&caps->cpuid.entries[i], i);
@@ -1684,9 +1691,16 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)
 
 static int tdx_setup_event_notify_interrupt(struct kvm_vcpu *vcpu)
 {
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 	u64 vector = tdx->vp_enter_args.r12;
 
+	/* See comment in init_kvm_tdx_caps() */
+	if (kvm_tdx->get_quote_in_kernel) {
+		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED);
+		return 1;
+	}
+
 	if (vector < 32 || vector > 255) {
 		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
 		return 1;
Signature · 1 line
2.25.1
#109 lore raw

Re: [PATCH v2 17/17] KVM: TDX: Support event-notify interrupts only with userspace Quoting

Quoted 8 lines · Xu Yilun

From: Peter Fang <[email protected]>

arch/x86/kvm/vmx/tdx.c +0 -1
@@ -202,8 +202,15 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf,
 
 	caps->cpuid.nent = td_conf->num_cpuid_config;
 
-	caps->user_tdvmcallinfo_1_r11 =

arch/x86/kvm/vmx/tdx.c static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, · line 213

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +9 -1
-		TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT;
+	/*
+	 * Don't advertise userspace event-notify interrupt support if TDX
+	 * quoting service is enabled, as quote generation will be handled
+	 * entirely in the kernel. Support in the kernel can be added later.
+	 */
+	if (!tdx_quote_enabled()) {
+		caps->user_tdvmcallinfo_1_r11 |=
+			TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT;
+	}

Can you use kvm_tdx->get_quote_in_kernel also above? Or should it maybe be initialized here if not used earlier?

Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +0 -0
@@ -1684,9 +1691,16 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)
 
 static int tdx_setup_event_notify_interrupt(struct kvm_vcpu *vcpu)
 {

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote(struct kvm_vcpu *vcpu) · line 1703

Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +7 -0
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 	u64 vector = tdx->vp_enter_args.r12;
 
+	/* See comment in init_kvm_tdx_caps() */
+	if (kvm_tdx->get_quote_in_kernel) {
+		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED);
+		return 1;
+	}
+

Since you're using kvm_tdx->get_quote_in_kernel here.

#110 lore raw

Re: [PATCH v2 17/17] KVM: TDX: Support event-notify interrupts only with userspace Quoting

Quoted 8 lines · Tony Lindgren
Quoted 7 lines · Xu Yilun

From: Peter Fang <[email protected]>

arch/x86/kvm/vmx/tdx.c +0 -0
@@ -202,8 +202,15 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf,
 
 	caps->cpuid.nent = td_conf->num_cpuid_config;
 
Quoted 14 lines

arch/x86/kvm/vmx/tdx.c static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, · line 213

Quoted 11 lines
arch/x86/kvm/vmx/tdx.c +9 -2
-	caps->user_tdvmcallinfo_1_r11 =
-		TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT;
+	/*
+	 * Don't advertise userspace event-notify interrupt support if TDX
+	 * quoting service is enabled, as quote generation will be handled
+	 * entirely in the kernel. Support in the kernel can be added later.
+	 */
+	if (!tdx_quote_enabled()) {
+		caps->user_tdvmcallinfo_1_r11 |=
+			TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT;
+	}

Can you use kvm_tdx->get_quote_in_kernel also above? Or should it maybe be initialized here if not used earlier?

This is a bit tricky. init_kvm_tdx_caps() is designed to be independent of a TDX VM instance (it reports the overall TDX capabilities), so there is no struct kvm_tdx available.

Quoted 9 lines
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +1 -0
@@ -1684,9 +1691,16 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)
 
 static int tdx_setup_event_notify_interrupt(struct kvm_vcpu *vcpu)
 {
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 	u64 vector = tdx->vp_enter_args.r12;
 
Quoted 8 lines

arch/x86/kvm/vmx/tdx.c static int tdx_get_quote(struct kvm_vcpu *vcpu) · line 1703

Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6 -0
+	/* See comment in init_kvm_tdx_caps() */
+	if (kvm_tdx->get_quote_in_kernel) {
+		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED);
+		return 1;
+	}
+

Since you're using kvm_tdx->get_quote_in_kernel here.

This is in per-vCPU context, so struct kvm_tdx is available.