Enable DICE-based TDX Quoting Extension
Subsystems (5)
v2 cover Enable DICE-based TDX Quoting Extension
This series adds infrastructure to enable TDX module extensions and then implements DICE-based TDX Quoting extension. This is the 2nd version and a significant change is that we want the quoting part to merge along with the basic TDX module extensions part, rather than serving as an example. So the quoting part drops RFC tags and requires initial review. The basic extensions part addresses v1 comments and needs more detailed review.
The quoting part contains some KVM patches, so we sorted the series for easier review and pick:
Patches 1-6: Enable the TDX module extensions support Patches 7-14: DICE-based TDX Quoting, x86/tdx part Patches 15-N: DICE-based TDX Quoting, KVM part
== Overview ==
To date, SEAMCALLs have been short lived routines that monopolize the CPU for their duration. This limits their utility for implementing higher order security protocols, or pushes complexity into Linux - such as by fragmenting a protocol setup service into several SEAMCALLs. The Linux appetite for ingesting complexity is low, so TDX now adds a new class of SEAMCALLs that are preemptible and resumable. This capability allows for higher-level API constructions - like "create a DICE-based quote" - which are more aligned to what is a good fit for Linux.
This new "extension SEAMCALL" capability is akin to ARM CCA's "Stateful RMI Operations (SRO)", and achieves similar externalized complexity relief as a dedicated hardware co-processor like AMD SEV-SNP. The mechanism is "give the service environment some memory", "invoke the service API", and "continue invoking until complete". All protocol state is internal to the service API.
TDX introduces "TDX module extensions" as the service environment for some add-on features - such as DICE-based quoting, TDISP, and live migration - to use "extension SEAMCALLs".
The extension SEAMCALLs are designed to be transparent to the host, using the same interface as normal SEAMCALLs, but the service environment should be initialized in several steps. First, configure/select (via TDH.SYS.CONFIG) add-on features during basic TDX initialization. Second, check if TDX module extensions are required to support these add-on features by reading TDX global metadata. Third, add extra memory to the TDX module via a SEAMCALL (TDH.EXT.MEM.ADD). Finally, use another SEAMCALL (TDH.EXT.INIT) to initialize the extensions.
== DICE-based Quoting extension ==
The first feature to use these extensions is the TDX Quoting extension [1], which converts guest launch attestation reports into a document that can be verified externally.
Today, the TDX host requires a separate software service to generate Quotes. The Quoting extension allows the TDX module to generate Quotes directly, without relying on a discrete Quoting engine. This simplifies the overall attestation flow: KVM no longer needs to return to userspace for Quote generation. Instead, Quote generation is handled directly by the TDX module through an extension SEAMCALL. See [2] for an overview of TDX attestation.
The Device Identifier Composition Engine ("DICE") provides a standardized framework for layering attestation evidence. It replaces SGX-based attestation and moves away from Intel-proprietary formats. It also eliminates the SGX requirement to contact an Intel service to obtain a certificate first. Instead, all attestation evidence is embedded in the Quote itself.
== The trade-off ==
The extensions create an extension instance for each feature that requires extension SEAMCALLs. More memory is consumed when more extension instances are created. There are 3 extensions (quoting, TDISP, Migration) in the foreseeable future. Turning on them all will require tens of megabytes. Note that the host can never reclaim the memory added to the extensions.
According to the TDX module design, basic TDX functionalities can run without the extensions. So theoretically the extensions don't need to be enabled at basic TDX initialization time. They could be lazily enabled right before the first extension SEAMCALL is issued.
However, Linux applies a simple policy for TDX: turn on all the features that Linux knows about all the time, unless and until any evidence makes this approach untenable. Enabling the extensions along with the basic TDX at boot time aligns with the policy, and offers several good reasons:
1. Simplify TDX state management, avoid runtime state transitions that could introduce race conditions or unexpected failure modes.
2. The kernel doesn't have to keep track of which SEAMCALLs need the extensions, as there is no HW/FW enumeration for this.
3. When no extension is configured, the extensions initialization is virtually skipped. So no impact on existing kernels.
4. A small trade-off is that eager initialization allocates memory (tens of megabytes) at boot time before any feature starts to work. However, these features provide critical security capabilities in confidential computing. They are expected to be enabled eventually when available. So this merely advances the timing of memory allocation.
== Restore the extensions after runtime TDX module update ==
Runtime TDX module update introduces a mechanism to update the module firmware while preserving and restoring TDX operations. As part of the restoration process, TDX module extensions must also be re-initialized to re-enable extension SEAMCALLs.
Similar to TDH.SYS.CONFIG, TDX module extends TDH.SYS.UPDATE with more parameters for the host to re-enable desired add-on features. Then host must re-execute all extensions initialization steps to restore extension SEAMCALL functionality.
However, Linux runs the update in stop_machine() context, which prevents memory allocation. This introduces a hard restriction that the updated TDX environment must not consume more memory for the extensions.
Fortunately, Linux applies another policy that no newer features should be added during runtime update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must enable the same features as the TDH.SYS.CONFIG. This policy mitigates the memory allocation problem a lot by minimizing the chance of increased memory demand. So now the restriction only affects the compatibility rule for choosing the update image.
The same memory constraint applies to the Quoting extension. A compatible runtime update must not increase the size limit of its Quotes, because the buffer used for Quote generation is allocated during TDX bringup. Otherwise, attestation could fail after the update if the TDX module requires a larger buffer for Quotes.
== Some history ==
The TDX module extensions support part was first posted along with TDX TDISP [3]. But quoting is the simplest consumer and is chosen as the lead vehicle over TDISP.
== Misc ==
This series is based on tip/x86/tdx [4], because we need the extensions play nice with runtime TDX module update.
== Changelog ==
v2:
- Support runtime TDX module update
- Refine quoting patches, drop RFC tag
- Change the patch order. (Xiaoyao & Tony)
- Fold metadata readings changes into patches that use them.
- Read the extensions metadata at init_tdx_ext() (Rick & Xiaoyao)
- Don't do get_tdx_sys_info() a 2nd time after TDH.SYS.CONFIG (Rick & Xiaoyao)
- Delete tdx_clflush_hpa_list() (Rick)
- s/TDX Module/TDX module (Sohil)
- s/Extensions/extensions (Dave)
- Change the data type of ext_required to bool (Rick)
- Change the data type of memory_pool_required_pages from u16 to u32, the Module team see this problem and promise the change (Sohil)
- s/init_tdx_ext()/init_tdx_module_extensions() to disambiguate from tdx_ext_init() (Kishen)
- Cover-letter & change log re-phrase (All reviewers)
v1: https://lore.kernel.org/all/[email protected]/
Peter Fang (11):
x86/virt/tdx: Initialize Quoting extension
x86/virt/tdx: Prepare Quote buffer during extension bringup x86/virt/tdx: Add interface to check Quoting availability x86/virt/tdx: Move tdx_tdr_pa() up in the file x86/virt/tdx: Add interface to generate a Quote x86/virt/tdx: Reinitialize the Quoting extension after TDX module
update
x86/virt/tdx: Enable Quoting extension
x86/tdx: Move and rename Quote request structure
KVM: TDX: Factor out userspace return path from tdx_get_quote()
KVM: TDX: Add in-kernel Quote generation
KVM: TDX: Support event-notify interrupts only with userspace Quoting
Xu Yilun (6):
x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions x86/virt/tdx: Configure add-on features on TDX module init and update x86/virt/tdx: Detect if the extensions initialization is required x86/virt/tdx: Add extra memory to TDX module for the extensions x86/virt/tdx: Make TDX module initialize the extensions x86/virt/tdx: Re-initialize the extensions on runtime TDX module
update
Documentation/arch/x86/tdx.rst | 19 +- Documentation/virt/kvm/api.rst | 3 + arch/x86/include/asm/tdx.h | 35 ++ arch/x86/include/asm/tdx_global_metadata.h | 9 + arch/x86/kvm/vmx/tdx.h | 6 + arch/x86/virt/vmx/tdx/tdx.h | 33 +- arch/x86/kvm/vmx/tdx.c | 176 +++++++- arch/x86/virt/vmx/tdx/tdx.c | 465 +++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 34 ++ drivers/virt/coco/tdx-guest/tdx-guest.c | 47 +- virt/kvm/kvm_main.c | 1 + 11 files changed, 755 insertions(+), 73 deletions(-)
base-commit: 2b9ad7a6154e0938b9458691536296dd0224942d
Signature · 1 line
2.25.1
PATCH v2 1/17 x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions
Embed version information in SEAMCALL leaf function definitions rather than let the caller open code them. For now, only TDH.VP.INIT is involved.
Don't bother the caller to choose the SEAMCALL version if unnecessary. New version SEAMCALLs are guaranteed to be backward compatible, so ideally the kernel doesn't need to keep version history and only uses the latest version SEAMCALLs.
And in confidential computing world, system security requires us to stop using an older TDX module when there is a newer one. So don't burden the kernel with long-term supporting an older TDX module that doesn't understand newer version SEAMCALLs.
The only concern is there may be transitional periods when a new TDX module is not widely available, meaning the kernel may temporarily need to support multiple SEAMCALL versions. As time goes by, the old TDX modules deprecate and old version SEAMCALL definitions should disappear.
The old TDX modules that only support TDH.VP.INIT v0 are all deprecated, so only provide the latest (v1) definition.
--- arch/x86/virt/vmx/tdx/tdx.h | 23 ++++++++++++++--------- arch/x86/virt/vmx/tdx/tdx.c | 3 +-- 2 files changed, 15 insertions(+), 11 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +14 -9
@@ -2,6 +2,7 @@#ifndef _X86_VIRT_TDX_H#define _X86_VIRT_TDX_H +#include <linux/bitfield.h> #include <linux/bits.h> /*
@@ -11,6 +12,18 @@* architectural definitions come first. */ +/* +* SEAMCALL leaf: +* +* Bit 15:0 Leaf number +* Bit 23:16 Version number +*/ +#define SEAMCALL_LEAF GENMASK(15, 0) +#define SEAMCALL_VER GENMASK(23, 16) + +#define SEAMCALL_LEAF_VER (l, v) (FIELD_PREP(SEAMCALL_LEAF, l) | \ + FIELD_PREP(SEAMCALL_VER, v)) +/* * TDX module SEAMCALL leaf functions */
@@ -31,7 +44,7 @@#define TDH_VP_CREATE 10#define TDH_MNG_KEY_FREEID 20#define TDH_MNG_INIT 21 -#define TDH_VP_INIT 22 +#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1)#define TDH_PHYMEM_PAGE_RDMD 24#define TDH_VP_RD 26#define TDH_PHYMEM_PAGE_RECLAIM 28
@@ -50,14 +63,6 @@#define TDH_SYS_UPDATE 53#define TDH_SYS_DISABLE 69 -/* -* SEAMCALL leaf: -* -* Bit 15:0 Leaf number -* Bit 23:16 Version number -*/ -#define TDX_VERSION_SHIFT 16 -/* TDX page types */ #define PT_NDA 0x0#define PT_RSVD 0x1
arch/x86/virt/vmx/tdx/tdx.c +1 -2
@@ -1903,8 +1903,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) .r8 = x2apicid, }; -/* apicid requires version == 1. */ -return seamcall (TDH_VP_INIT | (1ULL <<TDX_VERSION_SHIFT ), &args); +return seamcall (TDH_VP_INIT , &args); }EXPORT_SYMBOL_FOR_KVM (tdh_vp_init);
Signature · 1 line
2.25.1
Re: [PATCH v2 01/17] x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions
Quoted 3 lines · Xu Yilun
Embed version information in SEAMCALL leaf function definitions rather than let the caller open code them. For now, only TDH.VP.INIT is involved.
This is jumping the gun a bit in the changelog.
What is a SEAMCALL leaf function?
How does the version fit in?
Quoted 1 line
Don't bother the caller to choose the SEAMCALL version if unnecessary.
I think I see what you are trying to say here but it's more than that.
The question is whether there should be a base seamcall() API that takes an explicit version or whether the version should be passed in by callers.
One wrinkle is that the naming of all of these things is around "function", "func" and "fn":
u64 __seamcall(u64 fn, struct tdx_module_args *args);
A "function" is TDH.SYS.INIT or TDH.SYS.INFO, not 'TDH.SYS.INFO v123'.
But the low-level calls could be:
u64 __seamcall(u64 fn, u64 version, ...);
or
u64 __seamcall(u64 fn, ...);
Where 'fn' encodes the function *and* version.
Quoted 2 lines
The old TDX modules that only support TDH.VP.INIT v0 are all deprecated, so only provide the latest (v1) definition.
No, this isn't how this is going to work.
What do we *NEED* from v1? Why churn the code if we don't *NEED* something from v1 and can live with v0? It has *ZERO* to do with the TDX module being deprecated or whatever.
Linux stays on the old interface until we need a new interface. We are *not* going to bump version numbers just because.
Quoted 13 lines
/*
* TDX module SEAMCALL leaf functions
*/arch/x86/virt/vmx/tdx/tdx.h +1
-1@@ -31,7 +44,7 @@#define TDH_VP_CREATE 10#define TDH_MNG_KEY_FREEID 20#define TDH_MNG_INIT 21 -#define TDH_VP_INIT 22 +#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1)#define TDH_PHYMEM_PAGE_RDMD 24#define TDH_VP_RD 26#define TDH_PHYMEM_PAGE_RECLAIM 28
arch/x86/virt/vmx/tdx/tdx.h line 64
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.h +0
-0@@ -50,14 +63,6 @@#define TDH_SYS_UPDATE 53#define TDH_SYS_DISABLE 69
That is unreadable and patterns can't be seen. This is better:
#define TDH_MNG_INIT SEAMCALL_LEAF_VER(21, 0)
#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1)
#define TDH_PHYMEM_PAGE_RDMD SEAMCALL_LEAF_VER(24, 0)
arch/x86/virt/vmx/tdx/tdx.c u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) · line 1908
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +1
-2@@ -1903,8 +1903,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) .r8 = x2apicid, }; -/* apicid requires version == 1. */ -return seamcall (TDH_VP_INIT | (1ULL <<TDX_VERSION_SHIFT ), &args); +return seamcall (TDH_VP_INIT , &args); }EXPORT_SYMBOL_FOR_KVM (tdh_vp_init);
But that whole scheme falls apart the first time the kernel needs functionality from v2. You'll need:
#define TDH_VP_INIT_V0 SEAMCALL_LEAF_VER(22, 0)
#define TDH_VP_INIT_V1 SEAMCALL_LEAF_VER(22, 1)
and then the calls will do:
if (foo) return seamcall(TDH_VP_INIT_V0, &args); else return seamcall(TDH_VP_INIT_V1, &args);
So this 100% goes down the road of needing #defines for *EACH* version. That's the real implication here and the real choice.
That said, I don't particularly like:
if (foo) return seamcall(TDH_VP_INIT, 0, &args); else return seamcall(TDH_VP_INIT, 1, &args);
all that much either because of the seemingly magic numbers.
The whole seamcall RAX thing is one step too clever. I think Linux did the right thing:
5 common open sys_open
288 common openat sys_openat
437 common openat2 sys_openat2
New ABI gets a new base number. No need for the other end of the ABI to know that 288 is arguably a later version of 5.
Ugh. But why is this patch even in here in the first place? Why is this even *ASSOCIATED* with DICE-based attestation? Isn't this completely orthogonal?
Re: [PATCH v2 01/17] x86/virt/tdx: Embed version info in SEAMCALL leaf function definitions
Quoted 5 lines
This is jumping the gun a bit in the changelog.
What is a SEAMCALL leaf function?
How does the version fit in?
I think the word "leaf function" is confusing, maybe I should say "... SEAMCALL interface function definitions..."?
And I missed some more context explanation here:
SEAMCALL accepts parameters via CPU registers (RAX, RCX, RDX, ...), where RAX selects the specific TDX module function to execute. According to the TDX module SPEC, RAX is mainly composed of:
1. leaf number: selects the function.
2. version number: selects the function version. The newer version defines more operations. It also defines additional parameters for the rest of registers.
Newer version SEAMCALL uses the same leaf number as older versions, with only the version number increased. Newer version SEAMCALLs are guaranteed to be backward compatible.
[...]
Quoted 7 lines
Quoted 2 lines
The old TDX modules that only support TDH.VP.INIT v0 are all deprecated, so only provide the latest (v1) definition.
No, this isn't how this is going to work.
What do we *NEED* from v1? Why churn the code if we don't *NEED* something from v1 and can live with v0?
Sorry for the confusing words, for this TDH.VP.INIT, kernel is now using v1, we need v1 for virtual x2APIC. Maybe I should add "No functional change intended."
Quoted 1 line
It has *ZERO* to do with the TDX module being deprecated or whatever.
Since newer version SEAMCALLs are always backward compatible. TDH.VP.INIT v0 is only needed when we need to support legacy modules that don't understand TDH.VP.INIT v1. We don't support such legacy modules, so I meant to eliminate TDH.VP.INIT v0 definition and reduce histroy burdens.
We have some previous discussion that leads to "kernel don't keep SEAMCALL version at all":
https://lore.kernel.org/all/[email protected]/
But I have trouble for not keeping versions for TDH.SYS.CONFIG/UPDATE (in next patch). No public module supports TDH.SYS.CONFIG/UPDATE v1 so I can't eliminate TDH.SYS.CONFIG/UPDATE v0. I'm GOOD we are now considering keep aware of SEAMCALL versions at some level. Keeping in mind which module is published is another burden...
[...]
Quoted 6 lines
But that whole scheme falls apart the first time the kernel needs functionality from v2. You'll need:
#define
TDH_VP_INIT_V0SEAMCALL_LEAF_VER(22, 0)
#defineTDH_VP_INIT_V1SEAMCALL_LEAF_VER(22, 1)
Quoted 8 lines
and then the calls will do:
if (foo) return seamcall(TDH_VP_INIT_V0, &args); else return seamcall(TDH_VP_INIT_V1, &args);So this 100% goes down the road of needing #defines for *EACH* version.
TDH_VP_INIT_V0 has no use case in current code, so maybe not #define for EACH version.
And we have 100+ SEAMCALLs in SPEC, 30+ is being used in Linux, 20+ is comming, only 3 need versions now. I think making versioned definitions for EACH of them is overkill.
I see you have another suggestion in next patch which avoids the #defines. Let me continue in that patch.
Quoted 9 lines
The whole seamcall RAX thing is one step too clever. I think Linux did the right thing:
5 common open
sys_open
288 common openatsys_openat
437 common openat2sys_openat2New ABI gets a new base number. No need for the other end of the ABI to know that 288 is arguably a later version of 5.
Yes, that's true.
Quoted 4 lines
Ugh. But why is this patch even in here in the first place? Why is this even *ASSOCIATED* with DICE-based attestation? Isn't this completely orthogonal?
TDH.SYS.UPDATE/CONFIG is the second user of the version stuff. So as required, fix the open code issue of the first existing user.
https://lore.kernel.org/all/[email protected]/
Thanks,
Yilun
PATCH v2 2/17 x86/virt/tdx: Configure add-on features on TDX module init and update
In reply to Xu YilunIn addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them. The kernel should explicitly configure these features at boot or post-update initialization time. Configuring an add-on feature, such as TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions. TDX Quoting is the target feature to enable but defer it for now until full kernel support is in place.
TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as TDX_FEATURES0.
For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.
TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1. So there is little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.
--- arch/x86/virt/vmx/tdx/tdx.h | 6 ++++-- arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.h +4 -2
@@ -58,9 +58,11 @@#define TDH_PHYMEM_CACHE_WB 40#define TDH_PHYMEM_PAGE_WBINVD 41#define TDH_VP_WR 43 -#define TDH_SYS_CONFIG 45 +#define TDH_SYS_CONFIG_V0 45 +#define TDH_SYS_CONFIG SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)#define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE 53 +#define TDH_SYS_UPDATE_V0 53 +#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)#define TDH_SYS_DISABLE 69/* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +26 -2
@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;static u32 tdx_global_keyid__ro_after_init ;static u32 tdx_guest_keyid_start__ro_after_init ;static u32 tdx_nr_guest_keyids__ro_after_init ; +static u64 tdx_addon_feature0__ro_after_init ;static DEFINE_IDA (tdx_guest_keyid_pool);
@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list,return ret; } +static __init void set_tdx_addon_features (void ) +{ +/* +* To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when +* kernel is ready. +*/ +} +static __init int config_tdx_module (struct tdmr_info_list *tdmr_list,u64 global_keyid) { +u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {};u64 *tdmr_pa_array;size_t array_sz;
@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; - ret =seamcall_prerr (TDH_SYS_CONFIG , &args); + +set_tdx_addon_features (); + +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_CONFIG ; + } + + ret =seamcall_prerr (seamcall_fn, &args);/* Free the array as it is not required anymore. */ kfree (tdmr_pa_array);
@@ -1314,10 +1332,16 @@ int tdx_module_shutdown(void)int tdx_module_run_update (void ) { +u64 seamcall_fn =TDH_SYS_UPDATE_V0 ;struct tdx_module_args args = {};int ret; - ret =seamcall_prerr (TDH_SYS_UPDATE , &args); +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_UPDATE ; + } + + ret =seamcall_prerr (seamcall_fn, &args);if (ret)return ret;
Signature · 1 line
2.25.1
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
arch/x86/virt/vmx/tdx/tdx.c int tdx_module_shutdown(void) · line 1342
Quoted 11 lines · Xu Yilun
int
tdx_module_run_update(void)
{arch/x86/virt/vmx/tdx/tdx.c +5
-1+u64 seamcall_fn =TDH_SYS_UPDATE_V0 ;struct tdx_module_args args = {};int ret; - ret =seamcall_prerr (TDH_SYS_UPDATE , &args); +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_UPDATE ; + }
Heh, and it falls apart into craziness immediately. See how it immediately loses the logical information that there's a version 1 and a version 0? The "1" isn't even visible. It's hidden in "TDH_SYS_UPDATE".
Isn't this a million times more sane?
struct tdx_module_args args = {};
u64 version;int ret;
if (tdx_addon_feature0) {
args.r9 = tdx_addon_feature0;
version = 1;
} else {
version = 0;
}
ret = seamcall_prerr(TDH_SYS_UPDATE, version, &args);There's also zero stopping us from putting version in args:
struct tdx_module_args args = {};int ret;
if (tdx_addon_feature0) {
args.r9 = tdx_addon_feature0;
args.version = 1;
}
ret = seamcall_prerr(TDH_SYS_UPDATE, &args);Eh?
That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.
But this is *exactly* the kind of thing that shouldn't be a part of an attestation patch series. This could very much have been a separate discussion and happened a month or a year ago. But now it is blocking this DICE thing from getting done <grumble>.
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
Quoted 5 lines
There's also zero stopping us from putting version in args:
struct tdx_module_args args = {};int ret;
Quoted 8 lines
if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; args.version = 1; } ret = seamcall_prerr(TDH_SYS_UPDATE, &args);Eh?
I'm thinking the version is only needed for 3 SEAMCALLs. We don't have to make version common for all 100+ SEAMCALLs. Besides the layout of "struct tdx_module_args" correlates the fundamental assembly code of __seamcall() in tdcall.S.
Could we make dedicated SEAMCALL wrappers for TDH_SYS_UPDATE similar to other SEAMCALLs and wrapper the specific version handling there? I put the diff in the end.
Quoted 9 lines
That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.
But this is *exactly* the kind of thing that shouldn't be a part of an attestation patch series. This could very much have been a separate discussion and happened a month or a year ago. But now it is blocking this DICE thing from getting done <grumble>.
Sorry, I was thinking "don't keep version" was the conclusion...
--------8<--------
arch/x86/virt/vmx/tdx/tdx.c +18 -11
@@ -1757,18 +1757,12 @@ int tdx_module_shutdown(void)int tdx_module_run_update (void ) { -u64 seamcall_fn =TDH_SYS_UPDATE_V0 ; -struct tdx_module_args args = {}; +u64 err;int ret; -if (tdx_addon_feature0) { - args.r9 = tdx_addon_feature0; - seamcall_fn =TDH_SYS_UPDATE ; - } - - ret =seamcall_prerr (seamcall_fn, &args); -if (ret) -return ret; + err =tdx_sys_update (tdx_addon_feature0); +if (err) +return -EIO ; ret =get_tdx_sys_info_version (&tdx_sysinfo.version);/*
@@ -2351,7 +2345,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) .r8 = x2apicid, }; -return seamcall (TDH_VP_INIT , &args); +return seamcall (SEAMCALL_LEAF_VER (TDH_VP_INIT ,1 ), &args); }EXPORT_SYMBOL_FOR_KVM (tdh_vp_init);
@@ -2463,3 +2457,16 @@ void tdx_sys_disable(void)if (ret && (ret &TDX_SW_ERROR ) !=TDX_SW_ERROR )pr_err ("TDH.SYS.DISABLE failed: 0x%016llx\n" , ret); } + +u64 tdx_sys_update (u64 features_enable0) +{ +struct tdx_module_args args = { + .r9 = features_enable0, + }; +u64 fn =TDH_SYS_UPDATE ; + +if (features_enable0) + fn =SEAMCALL_LEAF_VER (TDH_SYS_UPDATE ,1 ); + +return seamcall (fn, &args); +}
arch/x86/virt/vmx/tdx/tdx.h +2 -3
@@ -44,7 +44,7 @@#define TDH_VP_CREATE 10#define TDH_MNG_KEY_FREEID 20#define TDH_MNG_INIT 21 -#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1) +#define TDH_VP_INIT 22#define TDH_PHYMEM_PAGE_RDMD 24#define TDH_VP_RD 26#define TDH_PHYMEM_PAGE_RECLAIM 28
@@ -61,8 +61,7 @@#define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE_V0 53 -#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_SYS_UPDATE 53#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
In reply to Dave HansenQuoted 12 lines
There's also zero stopping us from putting version in args:
struct tdx_module_args args = {};int ret;
if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; args.version = 1; } ret = seamcall_prerr(TDH_SYS_UPDATE, &args);
Quoted 5 lines
Eh?
That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.
Ah, on 2nd reading, I'm pretty sure now I understand your logical argument in patch 1 and 2. It's good to me. I append my diff at the end.
Quoted 5 lines
But this is *exactly* the kind of thing that shouldn't be a part of an attestation patch series. This could very much have been a separate discussion and happened a month or a year ago. But now it is blocking this DICE thing from getting done <grumble>.
Sorry, I should have been more active in searching for the solution rather than sticking to "kernel never keeps versions", when I've found the problem that public modules are not available.
----8<----
arch/x86/include/asm/shared/tdx.h +2 -0
@@ -143,6 +143,8 @@ struct tdx_module_args {u64 rbx;u64 rdi;u64 rsi; +/* for RAX encoding */ +u8 version; };/* Used to communicate with the TDX module */
arch/x86/kernel/asm-offsets.c +1 -0
@@ -95,6 +95,7 @@ static void __used common(void)OFFSET (TDX_MODULE_rbx, tdx_module_args, rbx);OFFSET (TDX_MODULE_rdi, tdx_module_args, rdi);OFFSET (TDX_MODULE_rsi, tdx_module_args, rsi); +OFFSET (TDX_MODULE_version, tdx_module_args, version);BLANK ();OFFSET (BP_scratch, boot_params, scratch);
arch/x86/virt/vmx/tdx/tdxcall.S +8 -0
@@ -48,6 +48,14 @@ /* Move Leaf ID to RAX */ mov %rdi, %rax + /* + * Extract the version from 'struct tdx_module_args', append it to + * RAX[23:16] + */ + movzbl TDX_MODULE_version(%rsi), %ecx + shll $16, %ecx + orq %rcx, %rax + /* Move other input regs from 'struct tdx_module_args' */ movq TDX_MODULE_rcx(%rsi), %rcx movq TDX_MODULE_rdx(%rsi), %rdx
arch/x86/virt/vmx/tdx/tdx.c +7 -8
@@ -1036,7 +1036,6 @@ static __init void set_tdx_addon_features(void)static __init int config_tdx_module (struct tdmr_info_list *tdmr_list,u64 global_keyid) { -u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {};u64 *tdmr_pa_array;size_t array_sz;
@@ -1059,18 +1058,18 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,for (i =0 ; i < tdmr_list->nr_consumed_tdmrs; i++) tdmr_pa_array[i] =__pa (tdmr_entry (tdmr_list, i)); +set_tdx_addon_features (); + args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; -set_tdx_addon_features (); -if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; - seamcall_fn =TDH_SYS_CONFIG ; + args.version =1 ; } - ret =seamcall_prerr (seamcall_fn, &args); + ret =seamcall_prerr (TDH_SYS_CONFIG , &args);/* Free the array as it is not required anymore. */ kfree (tdmr_pa_array);
@@ -1761,16 +1760,15 @@ int tdx_module_shutdown(void)int tdx_module_run_update (void ) { -u64 seamcall_fn =TDH_SYS_UPDATE_V0 ;struct tdx_module_args args = {};int ret;if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; - seamcall_fn =TDH_SYS_UPDATE ; + args.version =1 ; } - ret =seamcall_prerr (seamcall_fn, &args); + ret =seamcall_prerr (TDH_SYS_UPDATE , &args);if (ret)return ret;
@@ -2353,6 +2351,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) .rcx = vp->tdvpr_pa, .rdx = initial_rcx, .r8 = x2apicid, + .version = 1, };return seamcall (TDH_VP_INIT , &args);
arch/x86/virt/vmx/tdx/tdx.h +3 -5
@@ -44,7 +44,7 @@#define TDH_VP_CREATE 10#define TDH_MNG_KEY_FREEID 20#define TDH_MNG_INIT 21 -#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1) +#define TDH_VP_INIT 22#define TDH_PHYMEM_PAGE_RDMD 24#define TDH_VP_RD 26#define TDH_PHYMEM_PAGE_RECLAIM 28
@@ -58,11 +58,9 @@#define TDH_PHYMEM_CACHE_WB 40#define TDH_PHYMEM_PAGE_WBINVD 41#define TDH_VP_WR 43 -#define TDH_SYS_CONFIG_V0 45 -#define TDH_SYS_CONFIG SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1) +#define TDH_SYS_CONFIG 45#define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE_V0 53 -#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_SYS_UPDATE 53#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
Quoted 21 lines · Xu Yilun
Quoted 12 lines
There's also zero stopping us from putting version in args:
struct tdx_module_args args = {};int ret;
if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; args.version = 1; } ret = seamcall_prerr(TDH_SYS_UPDATE, &args);Quoted 5 lines
Eh?
That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.
Ah, on 2nd reading, I'm pretty sure now I understand your logical argument in patch 1 and 2. It's good to me. I append my diff at the end.
[ ... ]
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdxcall.S +1
-0@@ -48,6 +48,14 @@ /* Move Leaf ID to RAX */ mov %rdi, %rax + /*
arch/x86/virt/vmx/tdx/tdxcall.S line 61
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdxcall.S +7
-0+ * Extract the version from 'struct tdx_module_args', append it to + * RAX[23:16] + */ + movzbl TDX_MODULE_version(%rsi), %ecx + shll $16, %ecx + orq %rcx, %rax + /* Move other input regs from 'struct tdx_module_args' */ movq TDX_MODULE_rcx(%rsi), %rcx movq TDX_MODULE_rdx(%rsi), %rdx
This approach looks much cleaner to me. Would it be better to have a small C helper to encode the final RAX value instead of operating on RAX directly in asm? Looking at the May 2026 edition of the ABI spec, SEAMCALL RAX encoding is starting to get quite complex. Just thinking about this from a readability standpoint.
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
Quoted 42 lines · Peter Fang
Quoted 21 lines · Xu Yilun
Quoted 12 lines
There's also zero stopping us from putting version in args:
struct tdx_module_args args = {};int ret;
if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; args.version = 1; } ret = seamcall_prerr(TDH_SYS_UPDATE, &args);Quoted 5 lines
Eh?
That gives args.version==0 in all the normal cases which just happens to be the exact behavior we want. It also avoids having to plumb version through all the seamcall*() wrappers.
Ah, on 2nd reading, I'm pretty sure now I understand your logical argument in patch 1 and 2. It's good to me. I append my diff at the end.
[ ... ]
Quoted 17 lines
arch/x86/virt/vmx/tdx/tdxcall.S +8
-0@@ -48,6 +48,14 @@ /* Move Leaf ID to RAX */ mov %rdi, %rax + /* + * Extract the version from 'struct tdx_module_args', append it to + * RAX[23:16] + */ + movzbl TDX_MODULE_version(%rsi), %ecx + shll $16, %ecx + orq %rcx, %rax + /* Move other input regs from 'struct tdx_module_args' */
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdxcall.S line 61
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdxcall.S +0
-0movq TDX_MODULE_rcx(%rsi), %rcx movq TDX_MODULE_rdx(%rsi), %rdxThis approach looks much cleaner to me. Would it be better to have a small C helper to encode the final RAX value instead of operating on RAX directly in asm? Looking at the May 2026 edition of the ABI spec, SEAMCALL RAX encoding is starting to get quite complex. Just thinking about this from a readability standpoint.
I'm also good to it. I made some diff for your proposal, Some additional effort here is to update some comments and parameter names, to reflect the differences between "function/func/fn" (the unversioned number) and the final composite "fn_code" for RAX.
-----8<-------
arch/x86/include/asm/shared/tdx.h +2 -0
@@ -143,6 +143,8 @@ struct tdx_module_args {u64 rbx;u64 rdi;u64 rsi; +/* for leaf encoding */ +u8 version; };/* Used to communicate with the TDX module */
arch/x86/virt/vmx/tdx/seamcall.S +6 -6
@@ -10,8 +10,8 @@ * * __seamcall() function ABI: * - * @fn (RDI) - SEAMCALL Leaf number, moved to RAX - * @args (RSI) - struct tdx_module_args for input + * @fn_code (RDI) - SEAMCALL composite leaf code, moved to RAX + * @args (RSI) - struct tdx_module_args for input * * Only RCX/RDX/R8-R11 are used as input registers. *
@@ -29,8 +29,8 @@ SYM_FUNC_END(__seamcall) * * __seamcall_ret() function ABI: * - * @fn (RDI) - SEAMCALL Leaf number, moved to RAX - * @args (RSI) - struct tdx_module_args for input and output + * @fn_code (RDI) - SEAMCALL composite leaf code, moved to RAX + * @args (RSI) - struct tdx_module_args for input and output * * Only RCX/RDX/R8-R11 are used as input/output registers. *
@@ -51,8 +51,8 @@ SYM_FUNC_END(__seamcall_ret) * * __seamcall_saved_ret() function ABI: * - * @fn (RDI) - SEAMCALL Leaf number, moved to RAX - * @args (RSI) - struct tdx_module_args for input and output + * @fn_code (RDI) - SEAMCALL composite leaf code, moved to RAX + * @args (RSI) - struct tdx_module_args for input and output * * All registers in @args are used as input/output registers. *
arch/x86/virt/vmx/tdx/seamcall_internal.h +16 -5
@@ -11,17 +11,28 @@#ifndef _X86_VIRT_SEAMCALL_INTERNAL_H#define _X86_VIRT_SEAMCALL_INTERNAL_H +#include <linux/bitfield.h> #include <linux/printk.h> #include <linux/types.h> #include <asm/archrandom.h> #include <asm/processor.h> #include <asm/tdx.h> -u64 __seamcall (u64 fn,struct tdx_module_args *args); -u64 __seamcall_ret (u64 fn,struct tdx_module_args *args); -u64 __seamcall_saved_ret (u64 fn,struct tdx_module_args *args); +u64 __seamcall (u64 fn_code,struct tdx_module_args *args); +u64 __seamcall_ret (u64 fn_code,struct tdx_module_args *args); +u64 __seamcall_saved_ret (u64 fn_code,struct tdx_module_args *args); -typedef u64 (*sc_func_t )(u64 fn,struct tdx_module_args *args); +typedef u64 (*sc_func_t )(u64 fn_code,struct tdx_module_args *args); + +#define SEAMCALL_VERSION_MASK GENMASK_U64(23, 16) + +static __always_inline u64 __seamcall_fn_encoding(sc_func_t func, u64 fn, +struct tdx_module_args *args) +{ +FIELD_MODIFY (SEAMCALL_VERSION_MASK , &fn, args->version); + +return func (fn, args); +} static __always_inline u64 __seamcall_dirty_cache(sc_func_t func, u64 fn,struct tdx_module_args *args)
@@ -39,7 +50,7 @@ static __always_inline u64 __seamcall_dirty_cache(sc_func_t func, u64 fn,*/ this_cpu_write (cache_state_incoherent, true); -return func (fn, args); +return __seamcall_fn_encoding (func, fn, args); } static __always_inline u64 sc_retry(sc_func_t func, u64 fn,
arch/x86/virt/vmx/tdx/tdxcall.S +1 -1
@@ -24,7 +24,7 @@ *------------------------------------------------------------------------- * Input Registers: * - * RAX - TDCALL/SEAMCALL Leaf number. + * RAX - TDCALL/SEAMCALL composite Leaf code. * RCX,RDX,RDI,RSI,RBX,R8-R15 - TDCALL/SEAMCALL Leaf specific input registers. * * Output Registers:
arch/x86/virt/vmx/tdx/tdx.c +7 -8
@@ -1019,7 +1019,6 @@ static __init void set_tdx_addon_features(void)static __init int config_tdx_module (struct tdmr_info_list *tdmr_list,u64 global_keyid) { -u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {};u64 *tdmr_pa_array;size_t array_sz;
@@ -1042,18 +1041,18 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,for (i =0 ; i < tdmr_list->nr_consumed_tdmrs; i++) tdmr_pa_array[i] =__pa (tdmr_entry (tdmr_list, i)); +set_tdx_addon_features (); + args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; -set_tdx_addon_features (); -if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; - seamcall_fn =TDH_SYS_CONFIG ; + args.version =1 ; } - ret =seamcall_prerr (seamcall_fn, &args); + ret =seamcall_prerr (TDH_SYS_CONFIG , &args);/* Free the array as it is not required anymore. */ kfree (tdmr_pa_array);
@@ -1515,16 +1514,15 @@ int tdx_module_shutdown(void)int tdx_module_run_update (void ) { -u64 seamcall_fn =TDH_SYS_UPDATE_V0 ;struct tdx_module_args args = {};int ret;if (tdx_addon_feature0) { args.r9 = tdx_addon_feature0; - seamcall_fn =TDH_SYS_UPDATE ; + args.version =1 ; } - ret =seamcall_prerr (seamcall_fn, &args); + ret =seamcall_prerr (TDH_SYS_UPDATE , &args);if (ret)return ret;
@@ -2112,6 +2110,7 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid) .rcx = vp->tdvpr_pa, .rdx = initial_rcx, .r8 = x2apicid, + .version = 1, };return seamcall (TDH_VP_INIT , &args);
arch/x86/virt/vmx/tdx/tdx.h +3 -18
@@ -2,7 +2,6 @@#ifndef _X86_VIRT_TDX_H#define _X86_VIRT_TDX_H -#include <linux/bitfield.h> #include <linux/bits.h> /*
@@ -12,18 +11,6 @@* architectural definitions come first. */ -/* -* SEAMCALL leaf: -* -* Bit 15:0 Leaf number -* Bit 23:16 Version number -*/ -#define SEAMCALL_LEAF GENMASK(15, 0) -#define SEAMCALL_VER GENMASK(23, 16) - -#define SEAMCALL_LEAF_VER (l, v) (FIELD_PREP(SEAMCALL_LEAF, l) | \ - FIELD_PREP(SEAMCALL_VER, v)) -/* * TDX module SEAMCALL leaf functions */
@@ -44,7 +31,7 @@#define TDH_VP_CREATE 10#define TDH_MNG_KEY_FREEID 20#define TDH_MNG_INIT 21 -#define TDH_VP_INIT SEAMCALL_LEAF_VER(22, 1) +#define TDH_VP_INIT 22#define TDH_PHYMEM_PAGE_RDMD 24#define TDH_VP_RD 26#define TDH_PHYMEM_PAGE_RECLAIM 28
@@ -58,11 +45,9 @@#define TDH_PHYMEM_CACHE_WB 40#define TDH_PHYMEM_PAGE_WBINVD 41#define TDH_VP_WR 43 -#define TDH_SYS_CONFIG_V0 45 -#define TDH_SYS_CONFIG SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1) +#define TDH_SYS_CONFIG 45#define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE_V0 53 -#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_SYS_UPDATE 53#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
In reply to Xu YilunQuoted 12 lines · Xu Yilun
In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them. The kernel should explicitly configure these features at boot or post-update initialization time. Configuring an add-on feature, such as TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions. TDX Quoting is the target feature to enable but defer it for now until full kernel support is in place.
TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as
TDX_FEATURES0.
Quoted 5 lines
For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.
Actually, we do not need another global variable here. tdx_features0 is cached and is not updated across a runtime update, so the derived add-on feature bitmap will be the same before and after the update.
Quoted 8 lines
static
__initint config_tdx_module(structtdmr_info_list*tdmr_list,
u64global_keyid)
{arch/x86/virt/vmx/tdx/tdx.c +1
-0+u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {}; u64 *tdmr_pa_array; size_t array_sz;
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1049
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-1@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; - ret =seamcall_prerr (TDH_SYS_CONFIG , &args); + +set_tdx_addon_features (); + +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0;
How about moving this r9 assignment out of the if block and placing it next to 'args.r8 = global_keyid;'? There is no need to guard it, because args.r9 will be 0 when no add-on features are enabled, which is perfectly fine.
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+ seamcall_fn =TDH_SYS_CONFIG ; + } + + ret =seamcall_prerr (seamcall_fn, &args);
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
Quoted 9 lines
Quoted 5 lines
For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.
Actually, we do not need another global variable here.
tdx_features0is cached and is not updated across a runtime update, so the derived add-on feature bitmap will be the same before and after the update.
I think a global var "static u64 tdx_addon_feature0 *__ro_after_init*;" better illustrates the policy that add-on feature bitmap should be decided at boot up and never change later. It will also be used to decide if a specific add-on feature initialization is needed. We don't want to calculate the bitmap again and again, though the result must be the same.
Maybe I should strenghthen the commit message:
... both phases can use it. This actually mirrors a TDX module internal state so that kernel knows which add-on TDX operations (for example, quoting SEAMCALLs, which will be added in later patches) are valid.
Quoted 15 lines
Quoted 15 lines
static
__initint config_tdx_module(structtdmr_info_list*tdmr_list,
u64global_keyid)
{arch/x86/virt/vmx/tdx/tdx.c +1
-0+u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {}; u64 *tdmr_pa_array; size_t array_sz;arch/x86/virt/vmx/tdx/tdx.c +1
-1@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; - ret =seamcall_prerr (TDH_SYS_CONFIG , &args); +
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1049
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+set_tdx_addon_features (); + +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0;How about moving this r9 assignment out of the if block and placing it next to 'args.r8 =
global_keyid;'? There is no need to guard it, because args.r9 will be 0 when no add-on features are enabled, which is perfectly fine.
I tend to keep r9 assignment in the block, it clearly shows which SEAMCALL version needs what parameters, help people map the code to TDX module spec.
Quoted 5 lines
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+ seamcall_fn =TDH_SYS_CONFIG ; + } + + ret =seamcall_prerr (seamcall_fn, &args);
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
In reply to Xu YilunQuoted 2 lines · Xu Yilun
In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them.
"add-on features" looks like a new term introduced by this sereis for TDX. So what is add-on features? all the features defined in TDX_FEATURE0/1? Or only the features needs to be explicitly enabled by R9 and R10 of TDH.SYS.CONFIG? ...
Quoted 2 lines
The kernel should explicitly configure these features at boot or post-update initialization time.
... So the answer is the latter. Then why only these bits are add-on features while the rest in TDX_FEATURES01 are not?
btw, s/should/needs/ is better?
Quoted 3 lines
Configuring an add-on feature, such as
TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions.
Though the statement is right, it leads to the impression that the reason we are configuring the add-on feature is to initialize the TDX module extensions.
However, the truth is kenrel wants to enable/use an add-on feature and the add-on feature requires the functionalities provided by some TDX module extension. So kernel needs to enable the TDX module extensions.
Quoted 77 lines
TDX Quoting is the target feature to
enable but defer it for now until full kernel support is in place.TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as
TDX_FEATURES0.For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.
TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1. So there is little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.
Signed-off-by Xu Yilun <[email protected]>--- arch/x86/virt/vmx/tdx/tdx.h | 6 ++++-- arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 4 deletions(-)arch/x86/virt/vmx/tdx/tdx.h +4
-2@@ -58,9 +58,11 @@#define TDH_PHYMEM_CACHE_WB 40#define TDH_PHYMEM_PAGE_WBINVD 41#define TDH_VP_WR 43 -#define TDH_SYS_CONFIG 45 +#define TDH_SYS_CONFIG_V0 45 +#define TDH_SYS_CONFIG SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)#define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE 53 +#define TDH_SYS_UPDATE_V0 53 +#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)#define TDH_SYS_DISABLE 69/* TDX page types */ arch/x86/virt/vmx/tdx/tdx.c +10
-0@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;static u32 tdx_global_keyid__ro_after_init ;static u32 tdx_guest_keyid_start__ro_after_init ;static u32 tdx_nr_guest_keyids__ro_after_init ; +static u64 tdx_addon_feature0__ro_after_init ;static DEFINE_IDA (tdx_guest_keyid_pool);@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list,return ret; } +static __init void set_tdx_addon_features (void ) +{ +/* +* To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when +* kernel is ready. +*/ +} +static __init int config_tdx_module (struct tdmr_info_list *tdmr_list,u64 global_keyid) { +u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {};u64 *tdmr_pa_array;size_t array_sz;
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1046
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-1@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; - ret =seamcall_prerr (TDH_SYS_CONFIG , &args); + +set_tdx_addon_features ();
Maybe move the set_tdx_addon_features() out of config_tdx_module()? how about putting it after check_features(). config_tdx_module() looks like just a wrapper to invoke TDH.SYS.CONFIG, while the params of it are determined outside of it.
Just my feeling.
Quoted 28 lines
arch/x86/virt/vmx/tdx/tdx.c +14
-1+ +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_CONFIG ; + } + + ret =seamcall_prerr (seamcall_fn, &args);/* Free the array as it is not required anymore. */ kfree (tdmr_pa_array);@@ -1314,10 +1332,16 @@ int tdx_module_shutdown(void)int tdx_module_run_update (void ) { +u64 seamcall_fn =TDH_SYS_UPDATE_V0 ;struct tdx_module_args args = {};int ret; - ret =seamcall_prerr (TDH_SYS_UPDATE , &args); +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_UPDATE ; + } + + ret =seamcall_prerr (seamcall_fn, &args);if (ret)return ret;
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
Quoted 9 lines · Xiaoyao Li
Quoted 2 lines · Xu Yilun
In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them.
"add-on features" looks like a new term introduced by this sereis for TDX. So what is add-on features? all the features defined in TDX_FEATURE0/1? Or only the features needs to be explicitly enabled by R9 and R10 of TDH.SYS.CONFIG? ...
Quoted 5 lines
Quoted 2 lines
The kernel should explicitly configure these features at boot or post-update initialization time.
... So the answer is the latter. Then why only these bits are add-on features while the rest in
TDX_FEATURES01are not?
I think from TDX arch POV they impact the existing TDX system in some ways, either changes the behaviors of the existing features, or impact the performance.
Quoted 7 lines
btw, s/should/needs/ is better?
Quoted 3 lines
Configuring an add-on feature, such as
TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions.
Quoted 5 lines
Though the statement is right, it leads to the impression that the reason we are configuring the add-on feature is to initialize the TDX module extensions.
However, the truth is kenrel wants to enable/use an add-on feature and the
mm... yes it is somewhat misleading. Is it better we only talk about the add-on features' enabling in the first paragraph, then in next paragraph explain why we put the patch at the first one for TDX module Extension enabling?
Quoted 79 lines
add-on feature requires the functionalities provided by some TDX module extension. So kernel needs to enable the TDX module extensions.
Quoted 76 lines
TDX Quoting is the target feature to
enable but defer it for now until full kernel support is in place.TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as
TDX_FEATURES0.For runtime update, Linux applies a policy that no newer features should be added after update to avoid disrupting live TDX operations. To adhere to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.
TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1. So there is little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.
Signed-off-by Xu Yilun <[email protected]>--- arch/x86/virt/vmx/tdx/tdx.h | 6 ++++-- arch/x86/virt/vmx/tdx/tdx.c | 28 ++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 4 deletions(-)arch/x86/virt/vmx/tdx/tdx.h +4
-2@@ -58,9 +58,11 @@#define TDH_PHYMEM_CACHE_WB 40#define TDH_PHYMEM_PAGE_WBINVD 41#define TDH_VP_WR 43 -#define TDH_SYS_CONFIG 45 +#define TDH_SYS_CONFIG_V0 45 +#define TDH_SYS_CONFIG SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1)#define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE 53 +#define TDH_SYS_UPDATE_V0 53 +#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1)#define TDH_SYS_DISABLE 69/* TDX page types */ arch/x86/virt/vmx/tdx/tdx.c +10
-0@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state;static u32 tdx_global_keyid__ro_after_init ;static u32 tdx_guest_keyid_start__ro_after_init ;static u32 tdx_nr_guest_keyids__ro_after_init ; +static u64 tdx_addon_feature0__ro_after_init ;static DEFINE_IDA (tdx_guest_keyid_pool);@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list,return ret; } +static __init void set_tdx_addon_features (void ) +{ +/* +* To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when +* kernel is ready. +*/ +} +static __init int config_tdx_module (struct tdmr_info_list *tdmr_list,u64 global_keyid) { +u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct tdx_module_args args = {};u64 *tdmr_pa_array;size_t array_sz;@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, args.rcx =__pa (tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid;
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1046
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-1- ret =seamcall_prerr (TDH_SYS_CONFIG , &args); + +set_tdx_addon_features ();Maybe move the
set_tdx_addon_features()out ofconfig_tdx_module()? how about putting it aftercheck_features().config_tdx_module()looks like just a wrapper to invoke TDH.SYS.CONFIG, while the params of it are determined outside of it.
I'm going to take Chao's advice [1] to remove the global var tdx_addon_feature0, so will also delete this global step - set_tdx_addon_features() - to setup this global var.
Instead I'll use a "static u64 get_tdx_addon_features0(void)" for TDH.SYS.CONFIG/UPDATE.
The reason is that I've found this extra TDX status - tdx_addon_feature0 - makes the feature supporting judgement unnecessary complex, so don't make it long live.
Edgecombe, Rick P
maintainerRe: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
In reply to Xu YilunQuoted 7 lines · Xu Yilun
In addition to basic TDX functionalities, TDX module provides add-on features that can be progressively enabled as the kernel supports them. The kernel should explicitly configure these features at boot or post-update initialization time. Configuring an add-on feature, such as TDX Quoting, that uses extension SEAMCALLs is the prerequisite for initializing TDX module extensions. TDX Quoting is the target feature to enable but defer it for now until full kernel support is in place.
I think this is too much project management type info for the commit log. This patch is generic and could support any type of extension. I'd just list a few as examples of extensions.
Quoted 4 lines
TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which add-on features to configure. The bitmap uses the same definitions as
TDX_FEATURES0.
Assuming the v3 seamcall version patch is the final one, this should be updated to talk about this with respect to how that patch exposes the seamcall version concept.
Quoted 2 lines
For runtime update, Linux applies a policy
"applies a policy" sticks out to me here. Can we put this in simple terms of why this isn't done. "Policy" makes it sound arbitrary rule that we are blindly following.
Quoted 1 line
that no newer features should
"newer" is ambiguous here. I think it means no features that were not previously enabled, but it could come off as saying no features that were released after a certain point.
Quoted 9 lines
be added after update to avoid disrupting live TDX operations.
To adhere
to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.
Quoted 5 lines
TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1.
Ha, these are hard things to predict.
Quoted 3 lines
So there is
little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.
We can't deprecate it today, so what is this saying exactly?
Quoted 49 lines
Signed-off-by Xu Yilun <[email protected]>---
arch/x86/virt/vmx/tdx/tdx.h | 6 ++++--arch/x86/virt/vmx/tdx/tdx.c| 28 ++++++++++++++++++++++++++--2 files changed, 30 insertions(+), 4 deletions(-)arch/x86/virt/vmx/tdx/tdx.h +4
-2@@ -58,9 +58,11 @@ #define TDH_PHYMEM_CACHE_WB 40 #define TDH_PHYMEM_PAGE_WBINVD 41 #define TDH_VP_WR 43 -#define TDH_SYS_CONFIG 45 +#define TDH_SYS_CONFIG_V0 45 +#define TDH_SYS_CONFIG SEAMCALL_LEAF_VER(TDH_SYS_CONFIG_V0, 1) #define TDH_SYS_SHUTDOWN 52 -#define TDH_SYS_UPDATE 53 +#define TDH_SYS_UPDATE_V0 53 +#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) #define TDH_SYS_DISABLE 69
/* TDX page types */arch/x86/virt/vmx/tdx/tdx.c +1
-0@@ -57,6 +57,7 @@ static struct tdx_module_state tdx_module_state; static u32 tdx_global_keyid __ro_after_init; static u32 tdx_guest_keyid_start __ro_after_init; static u32 tdx_nr_guest_keyids __ro_after_init; +static u64 tdx_addon_feature0__ro_after_init ;
staticDEFINE_IDA(tdx_guest_keyid_pool);arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1004,9 +1005,18 @@ static __init int construct_tdmrs(struct list_head *tmb_list, return ret; }arch/x86/virt/vmx/tdx/tdx.c +4
-0+static __init void set_tdx_addon_features (void ) +{ +/* +* To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when
arch/x86/virt/vmx/tdx/tdx.c static __init int construct_tdmrs(struct list_head *tmb_list, · line 1013
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+* kernel is ready. +*/
Can we not make this DICE specific?
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+} +static
__initintconfig_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
{arch/x86/virt/vmx/tdx/tdx.c +1
-0+u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct
tdx_module_argsargs = {};
u64*tdmr_pa_array;
size_tarray_sz;
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct · line 1047
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-1@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(structtdmr_info_list *tdmr_list, args.rcx = __pa(tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; - ret =seamcall_prerr (TDH_SYS_CONFIG , &args); + +set_tdx_addon_features ();
This is a little opaque. Why not something like
tdx_addon_feature0 = TDX_KERNEL_SUPPORTED_ADDONS & tdx_sysinfo.features.tdx_features0;
tdx_addon_feature0 could probably be named better too. It means the features that the kernel has decided to use. HMm, any ideas?
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1051
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+ +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_CONFIG ; + }
This kind of feels hacky to me. tdx_addon_feature0 is zero if there are none supported. It shouldn't interfere with the v0 call to set it unconditionally?
I think the connection between tdx_addon_feature0 and TDH_SYS_CONFIG is a bit tenuous too. It is set if a extension is found that the kernel supports.
What do you think about wrapping TDH_SYS_CONFIG. Have the wrapper take all the args including tdx_addon_feature0. Then have the wrapper decide which seamcall to use. Have a nice comment that says "Use version of seamcall that supports add ons if we have any, use the old one if none are supported for backward compatibility".
BTW, looking at the tdx module, if I'm reading this right, r9 used to be ignored. So did it need a new seamcall version? Some other VMM was passing garbage?
Quoted 23 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+ + ret =seamcall_prerr (seamcall_fn, &args);
/* Free the array as it is not required anymore. */kfree(tdmr_pa_array);arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1314,10 +1332,16 @@ int tdx_module_shutdown(void)
inttdx_module_run_update(void)
{arch/x86/virt/vmx/tdx/tdx.c +1
-0+u64 seamcall_fn =TDH_SYS_UPDATE_V0 ;struct
tdx_module_argsargs = {};
int ret;arch/x86/virt/vmx/tdx/tdx.c +6
-1- ret =seamcall_prerr (TDH_SYS_UPDATE , &args); +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_UPDATE ; + } + + ret =seamcall_prerr (seamcall_fn, &args);if (ret)
return ret;
Re: [PATCH v2 02/17] x86/virt/tdx: Configure add-on features on TDX module init and update
Quoted 11 lines
Quoted 10 lines
To adhere
to this, TDH.SYS.UPDATE must configure the same features as the TDH.SYS.CONFIG. Record the kernel required add-on feature bitmap in a global var so that both phases can use it.TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the change, so use the latest version (v1) for add-on feature enabling. But supporting existing modules which only support v0 is still necessary until they are deprecated. In fact, it is unlikely that TDH.SYS.CONFIG ever needs to change again and the code would stay in v1.
Quoted 7 lines
Ha, these are hard things to predict.
Quoted 3 lines
So there is
little value in worrying about deprecating v0 to save a couple lines of code in 5-7 years when these original TDX platforms sunset.We can't deprecate it today, so what is this saying exactly?
This paragraph is outdated, at that time the target is to always use the latest SEAMCALL version. I'll remove it.
I updated the changelog according to these comments, appended at the end.
[...]
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int construct_tdmrs(struct list_head *tmb_list, · line 1013
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+static __init void set_tdx_addon_features (void ) +{ +/* +* To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when
arch/x86/virt/vmx/tdx/tdx.c +2
-0+* kernel is ready. +*/ Can we not make this DICE specific?
Yes.
Quoted 21 lines
Quoted 11 lines
(quoted diff) +2
-0+} +static
__initintconfig_tdx_module(struct tdmr_info_list *tdmr_list, u64 global_keyid)
{arch/x86/virt/vmx/tdx/tdx.c +1
-0+u64 seamcall_fn =TDH_SYS_CONFIG_V0 ;struct
tdx_module_argsargs = {};
u64*tdmr_pa_array;
size_tarray_sz;arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct · line 1047
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-1@@ -1032,7 +1042,15 @@ static __init int config_tdx_module(structtdmr_info_list *tdmr_list, args.rcx = __pa(tdmr_pa_array); args.rdx = tdmr_list->nr_consumed_tdmrs; args.r8 = global_keyid; - ret =seamcall_prerr (TDH_SYS_CONFIG , &args); + +set_tdx_addon_features ();This is a little opaque. Why not something like
Quoted 2 lines
tdx_addon_feature0=TDX_KERNEL_SUPPORTED_ADDONS& tdx_sysinfo.features.tdx_features0;
Good to me.
Quoted 3 lines
tdx_addon_feature0could probably be named better too. It means the features that the kernel has decided to use. HMm, any ideas?
How about tdx_supported_features0?
BTW: I'm going to take Chao's advice [1] to remove the global var tdx_addon_feature0, and use a "static u64 get_tdx_supported_features0(void)" for TDH.SYS.CONFIG/UPDATE.
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int config_tdx_module(struct tdmr_info_list *tdmr_list, · line 1051
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+ +if (tdx_addon_feature0) { + args.r9 = tdx_addon_feature0; + seamcall_fn =TDH_SYS_CONFIG ; + }This kind of feels hacky to me.
tdx_addon_feature0is zero if there are none supported. It shouldn't interfere with the v0 call to set it unconditionally?
Quoted 8 lines
I think the connection between
tdx_addon_feature0andTDH_SYS_CONFIGis a bit tenuous too. It is set if a extension is found that the kernel supports.What do you think about wrapping
TDH_SYS_CONFIG. Have the wrapper take all the args includingtdx_addon_feature0. Then have the wrapper decide which seamcall to use. Have a nice comment that says "Use version of seamcall that supports add ons if we have any, use the old one if none are supported for backward compatibility".
I'm good to add the wrapping for TDH_SYS_CONFIG and TDH_SYS_UPDATE.
Quoted 4 lines
BTW, looking at the tdx module, if I'm reading this right, r9 used to be ignored. So did it need a new seamcall version? Some other VMM was passing garbage?
Yes, I think it prevents possible garbage inputs suddenly take effect.
The updated changelog:
----8<----
x86/virt/tdx: Set configurable features on TDX module init and update
TDX module identifies some configurable features that must be explicitly enabled as the kernel supports them. These features affect existing TDX systems: they may change the behaviors of the existing features, reserve more memory, or impact the TDX initialization performance. The kernel needs to explicitly enable these features at boot or post-update initialization time.
Features such as TDISP, DICE-based quoting and TD migration are among those configurable features. They use preemptible SEAMCALLs so require extra memory and setup time to build a global service environment - the TDX module extensions. Add support for the configurable feature setting as the prerequisite for initializing TDX module extensions.
TDX module extends TDH.SYS.CONFIG and TDH.SYS.UPDATE with new bitmap input parameters to specify which features to enable. The bitmap uses the same definitions as TDX_FEATURES0. TDX module advances the version of TDH.SYS.CONFIG and TDH.SYS.UPDATE for the enhancement. A previous change [1] adds the 'version' field to struct tdx_module_args to support versioned SEAMCALLs. Assign the version field to 1 if any configurable feature is required.
Add a get_tdx_addon_features0() helper to return the bitmap of the configurable features that the module & kernel both support. Initially, this helper returns 0. It will be updated to return specific feature bits as full kernel support lands. TDH.SYS.CONFIG consumes this bitmap during TDX initialization.
Runtime updates keep the reported features unchanged across updates, so that existing TDX users can continue to operate without disruption. To adhere to this, provide TDH.SYS.UPDATE with the same bitmap returned by get_tdx_addon_features0(). The bitmap is guaranteed to match the initial TDH.SYS.CONFIG input because tdx_sysinfo.features.tdx_features0 is cached and not refreshed after update.
PATCH v2 3/17 x86/virt/tdx: Detect if the extensions initialization is required
In reply to Xu YilunTDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.
Add infrastructure to initialize TDX module extensions. Introduce the initial step of this process by detecting if the extensions are required by checking:
1. If the extensions are supported via TDX_FEATURES0_EXT.
2. If any TDX add-on feature needs the extensions via a boolean metadata field ext_required.
Currently all metadata fields are read at the very beginning of basic TDX initialization and stored in a global var. However, ext_required is only valid after the add-on feature configuration, making it incompatible with the existing metadata reading method.
To resolve this lifetime conflict, add a dedicated runtime metadata reading interface for the extensions, call it when the extensions initialization starts, and leave the field out of the global var. In this way, there is no confusion of when the metadata should be read.
--- arch/x86/include/asm/tdx.h | 1 + arch/x86/include/asm/tdx_global_metadata.h | 4 ++++ arch/x86/virt/vmx/tdx/tdx.c | 25 +++++++++++++++++++++ arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 14 ++++++++++++ 4 files changed, 44 insertions(+)
arch/x86/include/asm/tdx.h +1 -0
@@ -35,6 +35,7 @@/* Bit definitions of TDX_FEATURES0 metadata field */ #define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18) +#define TDX_FEATURES0_EXT BIT_ULL(39)#ifndef __ASSEMBLER__
arch/x86/include/asm/tdx_global_metadata.h +4 -0
@@ -52,4 +52,8 @@ struct tdx_sys_info { struct tdx_sys_info_td_conf td_conf; }; +struct tdx_sys_info_ext { +bool ext_required; +}; +#endif
arch/x86/virt/vmx/tdx/tdx.c +25 -0
@@ -1166,6 +1166,27 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ; + +/* TODO: add the extensions enabling steps here */ + +return 0 ; +} +static __init int init_tdx_module (void ) {int ret;
@@ -1220,6 +1241,10 @@ static __init int init_tdx_module(void)if (ret) goto err_reset_pamts; + ret =init_tdx_module_extensions (); +if (ret) + goto err_reset_pamts; +pr_info ("%lu KB allocated for PAMT\n" ,tdmrs_count_pamt_kb (&tdx_tdmr_list)); out_put_tdxmem:
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14 -0
@@ -131,3 +131,17 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)return ret; } + +static __init int get_tdx_sys_info_ext (struct tdx_sys_info_ext *sysinfo_ext) +{ +int ret; +u64 val; + + ret =read_sys_metadata_field (0x3100000000000001 , &val); +if (ret) +return ret; + + sysinfo_ext->ext_required = val; + +return 0 ; +}
Signature · 1 line
2.25.1
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 5 lines · Xu Yilun
TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.
How about "TDX module extension SEAMCALLs are preemptible and resumable..." above to make it easier to read?
Other than that:
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 9 lines · Tony Lindgren
Quoted 5 lines · Xu Yilun
TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.
How about "TDX module extension SEAMCALLs are preemptible and resumable..." above to make it easier to read?
Included, thanks.
Quoted 4 lines
Other than that:
Reviewed-by Tony Lindgren <[email protected]>
Edgecombe, Rick P
maintainerRe: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
In reply to Tony LindgrenQuoted 9 lines · Tony Lindgren
Quoted 5 lines · Xu Yilun
TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU. This allows for higher-level API constructions, so better supports some add-on features that implement higher order security protocols.
How about "TDX module extension SEAMCALLs are preemptible and resumable..." above to make it easier to read?
Agreed, it is hard to read. Also the first question will be, what is a "TDX module extension SEAMCALL"
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
In reply to Xu YilunQuoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; +
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ;
What would happen if the kernel doesn't do this 'ext_required' check and always does the extension initialization?
If TDH.EXT.INIT returns success when no extension is configured, we could drop this 'ext_required' from this series entirely.
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+ +/* TODO: add the extensions enabling steps here */ + +return 0 ; +}
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 19 lines · Chao Gao
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; +arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ;What would happen if the kernel doesn't do this '
ext_required' check and always does the extension initialization?
Quoted 1 line
If TDH.EXT.INIT returns success when no extension is configured, we
No, TDH.EXT.INIT fails when ext_required==false
Quoted 1 line
could drop this '
ext_required' from this series entirely.
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 20 lines · Xu Yilun
Quoted 19 lines · Chao Gao
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; +arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ;What would happen if the kernel doesn't do this '
ext_required' check and always does the extension initialization?
Quoted 3 lines
Quoted 1 line
If TDH.EXT.INIT returns success when no extension is configured, we
No, TDH.EXT.INIT fails when ext_required==false
What is the SEAMCALL return code in this case?
Quoted 1 line
Quoted 1 line
could drop this '
ext_required' from this series entirely.
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 21 lines · Xiaoyao Li
Quoted 20 lines · Xu Yilun
Quoted 19 lines · Chao Gao
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; +arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ;What would happen if the kernel doesn't do this '
ext_required' check and always does the extension initialization?
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
If TDH.EXT.INIT returns success when no extension is configured, we
No, TDH.EXT.INIT fails when ext_required==false
What is the SEAMCALL return code in this case?
It will return TDX_EXT_MEMORY_POOL_NOT_READY, because TDX module's internal state machine stays at the first step of extensions' initialization.
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
could drop this '
ext_required' from this series entirely.
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
In reply to Xu YilunQuoted 3 lines · Xu Yilun
TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU.
This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,
- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1
(There might be more. I didn't check all.)
Quoted 3 lines
This allows for higher-level API constructions,
so better supports some add-on features that implement higher order security protocols.
I think we don't need to explain this? Just that some add-on features require the functionalities of TDX module extensions is enough?
Quoted 64 lines
Add infrastructure to initialize TDX module extensions. Introduce the initial step of this process by detecting if the extensions are required by checking:
1. If the extensions are supported via
TDX_FEATURES0_EXT.
2. If any TDX add-on feature needs the extensions via a boolean metadata fieldext_required.Currently all metadata fields are read at the very beginning of basic TDX initialization and stored in a global var. However,
ext_requiredis only valid after the add-on feature configuration, making it incompatible with the existing metadata reading method.To resolve this lifetime conflict, add a dedicated runtime metadata reading interface for the extensions, call it when the extensions initialization starts, and leave the field out of the global var. In this way, there is no confusion of when the metadata should be read.
Signed-off-by Xu Yilun <[email protected]>--- arch/x86/include/asm/tdx.h | 1 + arch/x86/include/asm/tdx_global_metadata.h | 4 ++++ arch/x86/virt/vmx/tdx/tdx.c | 25 +++++++++++++++++++++ arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 14 ++++++++++++ 4 files changed, 44 insertions(+)arch/x86/include/asm/tdx.h +1
-0@@ -35,6 +35,7 @@/* Bit definitions of TDX_FEATURES0 metadata field */ #define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18) +#define TDX_FEATURES0_EXT BIT_ULL(39)#ifndef __ASSEMBLER__arch/x86/include/asm/tdx_global_metadata.h +4
-0@@ -52,4 +52,8 @@ struct tdx_sys_info { struct tdx_sys_info_td_conf td_conf; }; +struct tdx_sys_info_ext { +bool ext_required; +}; +#endif arch/x86/virt/vmx/tdx/tdx.c +5
-0@@ -1166,6 +1166,27 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; +
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ;
There was the discussion around it in [1]. I agree with the point to make kernel code simple. But it requires change/clarification from TDX spec, right? So will TDX spec change?
If so, you would need to mention it somewhere.
Quoted 41 lines
arch/x86/virt/vmx/tdx/tdx.c +9
-0+/* TODO: add the extensions enabling steps here */ + +return 0 ; +} +static __init int init_tdx_module (void ) {int ret;@@ -1220,6 +1241,10 @@ static __init int init_tdx_module(void)if (ret) goto err_reset_pamts; + ret =init_tdx_module_extensions (); +if (ret) + goto err_reset_pamts; +pr_info ("%lu KB allocated for PAMT\n" ,tdmrs_count_pamt_kb (&tdx_tdmr_list)); out_put_tdxmem:arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14
-0@@ -131,3 +131,17 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)return ret; } + +static __init int get_tdx_sys_info_ext (struct tdx_sys_info_ext *sysinfo_ext) +{ +int ret; +u64 val; + + ret =read_sys_metadata_field (0x3100000000000001 , &val); +if (ret) +return ret; + + sysinfo_ext->ext_required = val; + +return 0 ; +}
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 11 lines · Xiaoyao Li
Quoted 3 lines · Xu Yilun
TDX module extensions support extension SEAMCALLs that are preemptible and resumable, unlike normal SEAMCALLs that run to completion while monopolizing the CPU.
This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,
- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1
No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.
Quoted 9 lines
(There might be more. I didn't check all.)
Quoted 3 lines
This allows for higher-level API constructions,
so better supports some add-on features that implement higher order security protocols.I think we don't need to explain this? Just that some add-on features require the functionalities of TDX module extensions is enough?
mm.. a big part of the series is to bring up the TDX module extensions, and this is the first patch to start implementing TDX module extensions, a comment [1] said we could explain the context a little bit in general.
[1]: https://lore.kernel.org/all/[email protected]/
[...]
Quoted 10 lines
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+static __init int init_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret)
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1183
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+return ret; + +/* Skip if no feature requires TDX module extensions. */ +if (!sysinfo_ext.ext_required) +return 0 ;There was the discussion around it in [1]. I agree with the point to make kernel code simple. But it requires change/clarification from TDX spec,
I got the answer from TDX module team, that there is no use case ext memory is needed/the Extensions are enabled but TDH.EXT.INIT is not required. IOW ext_required == false exactly means TDX module extensions are not required and no extension SEAMCALLs should be invoked.
Quoted 1 line
right? So will TDX spec change?
No I didn't raise this nits to distract their main focus on blocking changes. But yes I can raise a ticket for this.
Quoted 4 lines
If so, you would need to mention it somewhere.
Edgecombe, Rick P
maintainerRe: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 7 lines · Xu Yilun
Quoted 6 lines
This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,
- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1
Quoted 5 lines
No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.
We have RESUMABLE seamcalls today. They are guaranteed to save state and make forward progress. Xiaoyao is right, this is not a new concept from the VMM side. But also why highlight this for the patch here. Are you trying to explain why extension initialization needs some special step?
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 8 lines · Rick P
Quoted 7 lines · Xu Yilun
Quoted 6 lines
This is not true. There are some normal (non-extension) SEAMCALLs also are preemptible and resumable. For example,
- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1
Quoted 8 lines
Quoted 5 lines
No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.
We have RESUMABLE seamcalls today. They are guaranteed to save state and make forward progress. Xiaoyao is right, this is not a new concept from the VMM side.
OK. Let me de-emphasize resumable.
And I'm thinking of how to stop explaining new terms again and again, extension SEAMCALL is the one. I'm not sure if calling it "preemptible SEAMCALLs" helps, or is it still a new term?
Quoted 2 lines
But also why highlight this for the patch here. Are you trying to explain why extension initialization needs some special step?
I was following the suggestion [1] of explaining the context in general. But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.
[1]: https://lore.kernel.org/all/[email protected]/
Refactored the first paragraph:
----8<----
TDX adds a new class of SEAMCALLs which are preemptible and can cooperate with the host scheduling. Some features like TDISP, DICE-based quoting rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.
...
Edgecombe, Rick P
maintainerRe: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 27 lines · Xu Yilun
Quoted 15 lines · Rick P
Quoted 8 lines · Xu Yilun
Quoted 7 lines
This is not true. There are some normal (non-extension) SEAMCALLs also are
preemptible and resumable. For example,- TDH.PHYMEM.CACHE.WB
- TDH.SYS.DISABLE
- TDH.MEM.SEPT.ADD with version 1Quoted 5 lines
No they are not preemptible and resumable. They just intentionally yield and don't have a generic way to save/resume their context. You can't rely on that to implement complex tasks in TDX module considering the complexity. That's the main reason why TDX module extensions are needed.
Quoted 4 lines
We have RESUMABLE seamcalls today. They are guaranteed to save state and make
forward progress. Xiaoyao is right, this is not a new concept from the VMM side.OK. Let me de-emphasize resumable.
And I'm thinking of how to stop explaining new terms again and again, extension SEAMCALL is the one. I'm not sure if calling it "preemptible SEAMCALLs" helps, or is it still a new term?
Quoted 8 lines
Quoted 3 lines
But also why highlight this for the patch here. Are you trying to explain why
extension initialization needs some special step?I was following the suggestion [1] of explaining the context in general. But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.
Wait are all extension seamcalls resumable? Couldn't there be one associated with an extension that is not? We have so far explained extensions in terms of the TDX module needing more memory for the optional features. So "extension" probably be interpreted as "optional", not as enabling "preemptible" seamcalls.
Quoted 10 lines
[1]:
https://lore.kernel.org/all/[email protected]/Refactored the first paragraph:
----8<----
TDX adds a new class of SEAMCALLs which are preemptible and can cooperate with the host scheduling.
But these are not new is the point. Look for TDX_INTERRUPTED_RESUMABLE in Linux and TDX source.
Quoted 5 lines
Some features like TDISP, DICE-based quoting
rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.
Why are we even talking about this in a patch that reads the metadata. Is it necessary?
Quoted 2 lines
...
Re: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 6 lines
Quoted 3 lines
But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.
Wait are all extension seamcalls resumable? Couldn't there be one associated with an extension that is not?
All extension SEAMCALLs must be preemptible and resumable. If a SEAMCALL doesn't need the ability, it is not backed by an extension. Note an extension may only serve part of the functionalities of a feature, for example, for TDX Connect, only SPDM related SEAMCALLs are extension SEAMCALLs, others are not cause they have short or simple routines.
Quoted 3 lines
We have so far explained extensions in terms of
the TDX module needing more memory for the optional features. So "extension" probably be interpreted as "optional", not as enabling "preemptible" seamcalls.
This is a long story and is explained in cover-letter. The need for extra memory is only a requirement to host, not the purpose for "extension".
Normal SEAMCALLs monopolize CPUs when execution. To prevent starving the host, SEAMCALL developers must count instructions at compile time, pre-define interrupt check-points and yield to host. This programming mode for TDX module is so complex that is hardly possible to handle complex tasks. As a result, TDX module had to push complexity into the host, such as by fragmenting a protocol setup service into several phases, let the host manage the overall flow and the SEAMCALLs only handle the secure parts which are short routines... This makes the SEAMCALL ABIs trivial, losing their semantic meanings. While AMD only calls a TIO_DEV_CONNECT and let its co-processor manage the whole thing.
A preemptible SEAMCALL behaves like an OS task, so a specific SEAMCALL developer could easily handle a complex routine, without caring about starving the host. A general underlying service environment is taking care of the hardware interrupt response, the context switch...
So IMHO a normal SEAMCALL may be called 'resumable' to some extent, but it is definitely not 'preemptible', it voluntarily yields. That's also why I think "preemptible SEAMCALLs" natually explain things than "extension SEAMCALLs"
[...]
Quoted 7 lines
Quoted 4 lines
----8<----
TDX adds a new class of SEAMCALLs which are preemptible and can cooperate with the host scheduling.
But these are not new is the point. Look for
TDX_INTERRUPTED_RESUMABLEin Linux and TDX source.
From host perspective, extension SEAMCALLs are just like normal 'resumable' SEAMCALLs. But the mechanism is new, and we have to explain the differences, otherwise there is no justification why we introduce the extensions.
Quoted 9 lines
Quoted 6 lines
Some features like TDISP, DICE-based quoting
rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.Why are we even talking about this in a patch that reads the metadata. Is it necessary?
OK, I can drop the introduction of the context here, it is not necessary. The cover-letter has the detailed explanation.
But should we still add the introduction in patch 5 ("x86/virt/tdx: Make TDX module initialize the extensions")?
Edgecombe, Rick P
maintainerRe: [PATCH v2 03/17] x86/virt/tdx: Detect if the extensions initialization is required
Quoted 8 lines · Xu Yilun
Quoted 6 lines
Quoted 3 lines
But yes, a brief explaination of why we need special initialization is good to me: A global service environment should be built to provide some general context switching mechanism inside TDX module.
Wait are all extension seamcalls resumable? Couldn't there be one associated with an extension that is not?
All extension SEAMCALLs must be preemptible and resumable.
My main issue is that I don't see why this need to be declared in a patch called "Detect if the extensions initialization is required".
But nothing I can see in the TDX docs says something like the above. It may be true from your reasoning, but I don't see what is gained by declaring it a hard rule either.
Quoted 25 lines
If a SEAMCALL
doesn't need the ability, it is not backed by an extension. Note an extension may only serve part of the functionalities of a feature, for example, for TDX Connect, only SPDM related SEAMCALLs are extension SEAMCALLs, others are not cause they have short or simple routines.Quoted 4 lines
We have so far explained extensions in terms of
the TDX module needing more memory for the optional features. So "extension" probably be interpreted as "optional", not as enabling "preemptible" seamcalls.This is a long story and is explained in cover-letter. The need for extra memory is only a requirement to host, not the purpose for "extension".
Normal SEAMCALLs monopolize CPUs when execution. To prevent starving the host, SEAMCALL developers must count instructions at compile time, pre-define interrupt check-points and yield to host. This programming mode for TDX module is so complex that is hardly possible to handle complex tasks. As a result, TDX module had to push complexity into the host, such as by fragmenting a protocol setup service into several phases, let the host manage the overall flow and the SEAMCALLs only handle the secure parts which are short routines... This makes the SEAMCALL ABIs trivial, losing their semantic meanings. While AMD only calls a
TIO_DEV_CONNECTand let its co-processor manage the whole thing.
Quoted 7 lines
A preemptible SEAMCALL behaves like an OS task, so a specific SEAMCALL developer could easily handle a complex routine, without caring about starving the host. A general underlying service environment is taking care of the hardware interrupt response, the context switch...
So IMHO a normal SEAMCALL may be called 'resumable' to some extent, but it is definitely not 'preemptible', it voluntarily yields.
No, some check for pending interrupts and exit back to the host if it sees one. They save their progress manually and Linux relies on this. So it's the same from the host perspective. Pre-empted, save state, resume where you left off.
Quoted 12 lines
That's also
why I think "preemptible SEAMCALLs" natually explain things than "extension SEAMCALLs"[...]
Quoted 6 lines
Quoted 5 lines
----8<----
TDX adds a new class of SEAMCALLs which are preemptible and can cooperate
with the host scheduling.
Quoted 8 lines
Quoted 3 lines
But these are not new is the point. Look for
TDX_INTERRUPTED_RESUMABLEin Linux
and TDX source.From host perspective, extension SEAMCALLs are just like normal 'resumable' SEAMCALLs. But the mechanism is new, and we have to explain the differences, otherwise there is no justification why we introduce the extensions.
Sure. People are not saying to skip explaining what is new. People are saying that what you are saying is new (pre-emptible, resumable), isn't new.
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Some features like TDISP, DICE-based quoting
rely on these preemptible SEAMCALLs to implement higher order security protocols with simple ABIs. To enable preemptible SEAMCALLs, a global service environment, called TDX module extensions, must first be initialized.
Quoted 8 lines
Quoted 2 lines
Why are we even talking about this in a patch that reads the metadata. Is it necessary?
OK, I can drop the introduction of the context here, it is not necessary. The cover-letter has the detailed explanation.
But should we still add the introduction in patch 5 ("x86/virt/tdx: Make TDX module initialize the extensions")?
Maybe it fits better in patch 4, because it is adding memory to save the state. But I think we shouldn't dwell too much on the internal details all over the logs.
I think it's good context to help reason about what is going on. But the important parts are that these extensions are optional and need more memory to save state for their interruptible resumable operations.
PATCH v2 4/17 x86/virt/tdx: Add extra memory to TDX module for the extensions
In reply to Xu YilunTDX module extensions receive a one-time memory allocation at initialization time. The extensions use this memory as the baseline for their internal states and data required by the service APIs they offer.
Add a new memory feeding process backed by a new SEAMCALL TDH.EXT.MEM.ADD. The process is mostly the same as adding PAMT. The kernel queries TDX module how much memory needed by reading the memory_pool_required_pages, allocates it, hands it over to the module, and never gets it back.
TDH.EXT.MEM.ADD uses a new parameter type, HPA_LIST_INFO, to provide this memory. This type represents a list of pages for TDX module to access. It references an 'hpa_list page' which contains the list of target HPAs. It collapses the HPA of the hpa_list page and the number of valid target HPAs into a 64 bit raw value for SEAMCALL parameters. The hpa_list page is always a medium, TDX module never keeps the hpa_list page.
Don't CLFLUSH the pages handed to the TDX module, as is done for some other SEAMCALLs. The flushing operation is not expected to be needed for current and known future architectures. As more and more page feeding interfaces to come, the conservative flushing operation becomes a maintenance burden.
For now, TDX module extensions consume tens of megabytes memory that will never be returned to host. Use contiguous page allocation to isolate these large blocks entirely, avoiding permanent memory fragmentation and reducing buddy allocator efficiency. Print the allocation amount on TDX module extensions initialization for visibility.
--- arch/x86/include/asm/tdx_global_metadata.h | 1 + arch/x86/virt/vmx/tdx/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.c | 107 +++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 6 ++ 4 files changed, 112 insertions(+), 3 deletions(-)
arch/x86/include/asm/tdx_global_metadata.h +1 -0
@@ -53,6 +53,7 @@ struct tdx_sys_info { };struct tdx_sys_info_ext { +u32 memory_pool_required_pages;bool ext_required; };
arch/x86/virt/vmx/tdx/tdx.h +1 -0
@@ -63,6 +63,7 @@#define TDH_SYS_SHUTDOWN 52#define TDH_SYS_UPDATE_V0 53#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69/* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +104 -3
125 lines not on this page · raw message
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +6 -0
@@ -137,6 +137,12 @@ static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)int ret;u64 val; + ret =read_sys_metadata_field (0x3100000200000000 , &val); +if (ret) +return ret; + + sysinfo_ext->memory_pool_required_pages = val; + ret =read_sys_metadata_field (0x3100000000000001 , &val);if (ret)return ret;
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] Hardware PFN mismatch due to unaligned slab allocation for `hpa_list` with debugging enabled.…
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Hardware PFN mismatch due to unaligned slab allocation for hpa_list with debugging enabled.
- [Medium] Resource leak of un-added contiguous host memory on TDX module error path.
- [Medium] Improper use of node_online_map instead of N_MEMORY node states for allocation.
- [Low] Unnecessary early boot memory allocation error handling returning -ENOMEM.
--
commit e910725d57b782da4b05f3bb30b4df0fa4da1da3
Author: Xu Yilun <[email protected]>
x86/virt/tdx: Add extra memory to TDX module for the extensions
This patch introduces a memory feeding process via SEAMCALL TDH.EXT.MEM.ADD to allocate one-time memory for TDX module extensions during initialization. It allocates contiguous memory and passes it to the module using a newly defined HPA_LIST_INFO format.
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0
[ ... ]
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int tdx_ext_mem_setup (unsigned int required_pages) +{ +struct tdx_hpa_list *hpa_list; +struct page *page; +unsigned int i;
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1225
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+int ret; + +/* +* memory_pool_required_pages == 0 means no need to add pages, +* skip the memory setup. +*/ +if (!required_pages) +return 0 ; + + hpa_list =kzalloc_obj (*hpa_list);
[Severity: High]
Does this code pass unaligned physical pages to the hardware if slab debugging is enabled?
When CONFIG_SLUB_DEBUG or KASAN is enabled, slab objects are padded with metadata, causing the object to start at a non-zero offset within a physical page. The virt_to_page(hpa_list) call below will grab the base page frame which the hardware will read from offset 0, leading it to consume SLUB metadata instead of the valid hpa_list entries.
Should this allocation use the page allocator directly (e.g., __get_free_page()) to guarantee strict page alignment?
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1230
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+if (!hpa_list) +return -ENOMEM ; + + page =alloc_contig_pages (required_pages,GFP_KERNEL ,numa_mem_id (), + &node_online_map);
[Severity: Medium]
Can this allocation fall back to memoryless nodes?
The node_online_map can include nodes without memory. The mm allocation guidelines require using the N_MEMORY state mask (e.g., node_states[N_MEMORY]) to correctly constrain memory allocations to nodes that actually have memory.
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1232
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+if (!page) { + ret = -ENOMEM ;
[Severity: Low]
Is this complex error handling necessary here?
Since tdx_ext_mem_setup() is only called from the __init path, physical memory should be available. If an allocation of this size fails during early boot, the system likely cannot boot anyway, making graceful error cleanup here largely dead code.
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+ goto out_free_hpa_list; + } + + i =0 ; +while (i < required_pages) { +unsigned int nents =min (required_pages - i, +ARRAY_SIZE (hpa_list->phys)); +unsigned int j; + +for (j =0 ; j < nents; j++) + hpa_list->phys[j] =page_to_phys (page + i + j);
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+ + ret =tdx_ext_mem_add (virt_to_page (hpa_list), nents); +/* +* No SEAMCALLs to reclaim the added pages. For simple error +* handling, leak all pages. +*/ +WARN (ret,"Fatal: TDX module rejected (%d) memory for extensions, stranded all pages\n" , + ret); +if (ret) +break ;
[Severity: Medium]
Does this intentionally leak un-added host memory on the error path?
While pages that have already been accepted by the hardware cannot be reclaimed, the remaining contiguous block of pages (from page + i to the end) are still entirely host-owned. Could these remaining un-added pages be safely freed via free_contig_range() instead of permanently leaking tens of megabytes?
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ + i += nents; + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions
Quoted 44 lines · On Thu, Jun 18, 2026 at 08:54:58AM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Hardware PFN mismatch due to unaligned slab allocation forhpa_listwith debugging enabled.
- [Medium] Resource leak of un-added contiguous host memory on TDX module error path.
- [Medium] Improper use ofnode_online_mapinstead ofN_MEMORYnode states for allocation.
- [Low] Unnecessary early boot memory allocation error handling returning-ENOMEM.
--commit
e910725d57b782da4b05f3bb30b4df0fa4da1da3
Author: Xu Yilun <[email protected]>x86/virt/tdx: Add extra memory to TDX module for the extensions
This patch introduces a memory feeding process via SEAMCALL TDH.EXT.MEM.ADD to allocate one-time memory for TDX module extensions during initialization. It allocates contiguous memory and passes it to the module using a newly defined
HPA_LIST_INFOformat.Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0[ ... ]
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int tdx_ext_mem_setup (unsigned int required_pages) +{ +struct tdx_hpa_list *hpa_list; +struct page *page; +unsigned int i;arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1225
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+int ret; + +/* +* memory_pool_required_pages == 0 means no need to add pages, +* skip the memory setup. +*/ +if (!required_pages) +return 0 ; + + hpa_list =kzalloc_obj (*hpa_list);[Severity: High]
Does this code pass unaligned physical pages to the hardware if slab debugging is enabled?
Quoted 5 lines
When
CONFIG_SLUB_DEBUGor KASAN is enabled, slab objects are padded with metadata, causing the object to start at a non-zero offset within a physical page. Thevirt_to_page(hpa_list)call below will grab the base page frame which the hardware will read from offset 0, leading it to consume SLUB metadata instead of the validhpa_listentries.
I think this is false positive. "For sizes which are a power of two, the alignment is also guaranteed to be at least the respective size." according to:
59bb47985c1d ("mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two)")
Quoted 13 lines
Should this allocation use the page allocator directly (e.g.,
__get_free_page()) to guarantee strict page alignment?arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1230
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+if (!hpa_list) +return -ENOMEM ; + + page =alloc_contig_pages (required_pages,GFP_KERNEL ,numa_mem_id (), + &node_online_map);[Severity: Medium]
Can this allocation fall back to memoryless nodes?
Quoted 1 line
The
node_online_mapcan include nodes without memory. The mm allocation
I think this is false positive, at least for x86. According to Documentation/mm/numa.rst, "For some architectures, such as x86, Linux will "hide" any node representing a physical cell that has no memory attached"
Quoted 9 lines
guidelines require using the
N_MEMORYstate mask (e.g., node_states[N_MEMORY]) to correctly constrain memory allocations to nodes that actually have memory.arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1232
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+if (!page) { + ret = -ENOMEM ;[Severity: Low]
Is this complex error handling necessary here?
Quoted 4 lines
Since
tdx_ext_mem_setup()is only called from the__initpath, physical memory should be available. If an allocation of this size fails during early boot, the system likely cannot boot anyway, making graceful error cleanup here largely dead code.
It executes at subsys_initcall(), the system can still go on booting. Free a previously allocated page is not complex.
Actually the error handling would be complex when memory is partially added for extensions but TDX error happens. In this case for simplicity the below code leaks all the memory.
Quoted 10 lines
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +9
-0+ goto out_free_hpa_list; + } + + i =0 ; +while (i < required_pages) { +unsigned int nents =min (required_pages - i, +ARRAY_SIZE (hpa_list->phys)); +unsigned int j; +
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1249
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +8
-0+for (j =0 ; j < nents; j++) + hpa_list->phys[j] =page_to_phys (page + i + j); + + ret =tdx_ext_mem_add (virt_to_page (hpa_list), nents); +/* +* No SEAMCALLs to reclaim the added pages. For simple error +* handling, leak all pages. +*/
Maybe the comment is not clear enough so sashiko raised the below concern, I'll make it more clear:
/*
* No SEAMCALLs to reclaim the added pages. For simple error
* handling, intentionally leak all pages, including un-added
* pages.
*/
Quoted 19 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+WARN (ret,"Fatal: TDX module rejected (%d) memory for extensions, stranded all pages\n" , + ret); +if (ret) +break ;[Severity: Medium]
Does this intentionally leak un-added host memory on the error path?While pages that have already been accepted by the hardware cannot be reclaimed, the remaining contiguous block of pages (from page + i to the end) are still entirely host-owned. Could these remaining un-added pages be safely freed via
free_contig_range()instead of permanently leaking tens of megabytes?Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ + i += nents; + }Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions
In reply to Xu YilunQuoted 16 lines
arch/x86/virt/vmx/tdx/tdx.c +16
-0+#define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3) +#define HPA_LIST_INFO_PFN GENMASK_U64(51, 12) +#define HPA_LIST_INFO_LAST_ENTRY GENMASK_U64(63, 55) + +static __init u64 to_hpa_list_info (struct page *hpa_list_page, +unsigned int nr_pages) +{ +return FIELD_PREP (HPA_LIST_INFO_FIRST_ENTRY ,0 ) | +FIELD_PREP (HPA_LIST_INFO_PFN ,page_to_pfn (hpa_list_page)) | +FIELD_PREP (HPA_LIST_INFO_LAST_ENTRY , nr_pages -1 ); +} + +static __init int tdx_ext_mem_add (struct page *hpa_list_page, +unsigned int nr_pages) +{ +struct tdx_module_args args = {
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1195
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+ .rcx = to_hpa_list_info(hpa_list_page, nr_pages), + }; +u64 r; + + do { + /* + * TDH_EXT_MEM_ADD is designed to use output parameter RCX to + * override/update input parameter RCX, so the caller doesn't + * have to do manual parameter update on retry call. + */
I am not sure about the "manual parameter update" part.
how about:
/*
* The TDX module overwrites RCX to track progress when
* this SEAMCALL is interrupted. Use seamcall_ret() to
* save and pass the updated value back on retry. */
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1200
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+ r =seamcall_ret (TDH_EXT_MEM_ADD , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ;
why -EFAULT?
In sc_retry_prerr(), most SEAMCALL errors are mapped to EIO. Maybe we should use EIO here.
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+ +return 0 ; +} + +struct tdx_hpa_list { +u64 phys[PAGE_SIZE /sizeof (u64)]; +}; + +static_assert (sizeof (struct tdx_hpa_list ) ==PAGE_SIZE ); + +static __init int tdx_ext_mem_setup (unsigned int required_pages)
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1221
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+{ +struct tdx_hpa_list *hpa_list; +struct page *page; +unsigned int i; +int ret; + +/* +* memory_pool_required_pages == 0 means no need to add pages, +* skip the memory setup. +*/
There is no "memory_pool_required_pages" here.
Quoted 20 lines
arch/x86/virt/vmx/tdx/tdx.c +20
-0+if (!required_pages) +return 0 ; + + hpa_list =kzalloc_obj (*hpa_list); +if (!hpa_list) +return -ENOMEM ; + + page =alloc_contig_pages (required_pages,GFP_KERNEL ,numa_mem_id (), + &node_online_map); +if (!page) { + ret = -ENOMEM ; + goto out_free_hpa_list; + } + + i =0 ; +while (i < required_pages) { +unsigned int nents =min (required_pages - i, +ARRAY_SIZE (hpa_list->phys)); +unsigned int j; +
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1251
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+for (j =0 ; j < nents; j++) + hpa_list->phys[j] =page_to_phys (page + i + j); + + ret =tdx_ext_mem_add (virt_to_page (hpa_list), nents); +/* +* No SEAMCALLs to reclaim the added pages. For simple error +* handling, leak all pages. +*/ +WARN (ret,"Fatal: TDX module rejected (%d) memory for extensions, stranded all pages\n" , + ret);
Printing 'ret' is useless since it's always -EFAULT.
The real reason to WARN here isn't "no SEAMCALL to reclaim". It is "this SEAMCALL shouldn't fail, and if it does, things are broken enough that complex error handling isn't worth it".
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1262
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+if (ret) +break ; + + i += nents; + } + +/* +* Memory for extensions can't be reclaimed once added, print out the +* amount, stop tracking it and free the hpa_list page, no matter +* success or failure. +*/
This doesn't explain why it should be print. How about:
/* * Memory for TDX module extensions is never reclaimed and can be * tens of megabytes. Print the amount so users know the cost. */
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +8
-0+pr_info ("%lu KB consumed for TDX module extensions\n" , + required_pages *PAGE_SIZE /1024 ); + +out_free_hpa_list: +kfree (hpa_list); + +return ret; +}
Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions
Quoted 5 lines
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+static __init int tdx_ext_mem_setup (unsigned int required_pages) +{ +struct tdx_hpa_list *hpa_list; +struct page *page; +unsigned int i;
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1221
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +6
-0+int ret; + +/* +* memory_pool_required_pages == 0 means no need to add pages, +* skip the memory setup. +*/ There is no "
memory_pool_required_pages" here.
OK.
BTW, sashiko says the comment restates what the if-statement. So I change to:
/*
* This required_pages comes from the metadata
* memory_pool_required_pages. TDX module uses it to indicate how much
* memory is still needed. This value decreases each time memory is
* added via TDH.EXT.MEM.ADD.
*
* On first time initialization, a value of 0 before any memory is
* added is unusual. But host makes no assumptions. Skip the memory
* setup and let subsequent steps catch any actual errors. */
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1223
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+if (!required_pages) +return 0 ;
Other comments are all good, included.
Edgecombe, Rick P
maintainerRe: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions
In reply to Xu YilunQuoted 3 lines · Xu Yilun
TDX module extensions receive a one-time memory allocation at initialization time. The extensions use this memory as the baseline for their internal states and data required by the service APIs they offer.
Not sure what "baseline" means here. It is used for their internal state. So maybe just remove baseline. Unless it's important to clarify that it will need other memory later.
Quoted 10 lines
Add a new memory feeding process backed by a new SEAMCALL TDH.EXT.MEM.ADD. The process is mostly the same as adding PAMT. The kernel queries TDX module how much memory needed by reading the
memory_pool_required_pages, allocates it, hands it over to the module, and never gets it back.
Quoted 7 lines
TDH.EXT.MEM.ADD uses a new parameter type,
HPA_LIST_INFO, to provide this memory. This type represents a list of pages for TDX module to access. It references an 'hpa_listpage' which contains the list of target HPAs. It collapses the HPA of thehpa_listpage and the number of valid target HPAs into a 64 bit raw value for SEAMCALL parameters. Thehpa_listpage is always a medium, TDX module never keeps thehpa_listpage.
I think you are going to need to put the basic concepts up front. Don't lead with the acronyms. The TDX module needs to get a list of pages. It takes them in a PFN array that is encoded like (...). Create a helper for this, and name it the same after the TDX module term for this format, HPA_LIST_INFO.
Basically there are no exotic concepts here. So you can explain it in normal kernel terms.
Quoted 6 lines
Don't CLFLUSH the pages handed to the TDX module, as is done for some other SEAMCALLs. The flushing operation is not expected to be needed for current and known future architectures. As more and more page feeding interfaces to come, the conservative flushing operation becomes a maintenance burden.
I think you can probably drop this paragraph.
Quoted 2 lines
For now,
Do we need the "for now"? Just adds more to think about and Im not sure it helps justify the patch.
Quoted 6 lines
TDX module extensions consume tens of megabytes memory that
will never be returned to host. Use contiguous page allocation to isolate these large blocks entirely, avoiding permanent memory fragmentation and reducing buddy allocator efficiency. Print the allocation amount on TDX module extensions initialization for visibility.
Hmm. I'm not sure the user will care about the specific category of extension used memory.
Quoted 73 lines
Signed-off-by Xu Yilun <[email protected]>---
arch/x86/include/asm/tdx_global_metadata.h | 1 + arch/x86/virt/vmx/tdx/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.c | 107 +++++++++++++++++++-arch/x86/virt/vmx/tdx/tdx_global_metadata.c| 6 ++4 files changed, 112 insertions(+), 3 deletions(-)arch/x86/include/asm/tdx_global_metadata.h +0
-0@@ -53,6 +53,7 @@ struct tdx_sys_info { };
structtdx_sys_info_ext{arch/x86/include/asm/tdx_global_metadata.h +1
-0+u32 memory_pool_required_pages;bool
ext_required;
};arch/x86/virt/vmx/tdx/tdx.h +1
-0@@ -63,6 +63,7 @@ #define TDH_SYS_SHUTDOWN 52 #define TDH_SYS_UPDATE_V0 53 #define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_EXT_MEM_ADD 61 #define TDH_SYS_DISABLE 69
/* TDX page types */arch/x86/virt/vmx/tdx/tdx.c +1
-0@@ -31,6 +31,7 @@ #include <linux/syscore_ops.h> #include <linux/idr.h> #include <linux/kvm_types.h> +#include <linux/bitfield.h> #include <asm/page.h> #include <asm/special_insns.h> #include <asm/msr-index.h>@@ -1166,6 +1167,108 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) return 0; }arch/x86/virt/vmx/tdx/tdx.c +21
-0+#define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3) +#define HPA_LIST_INFO_PFN GENMASK_U64(51, 12) +#define HPA_LIST_INFO_LAST_ENTRY GENMASK_U64(63, 55) + +static __init u64 to_hpa_list_info (struct page *hpa_list_page, + unsigned int nr_pages) +{ +return FIELD_PREP (HPA_LIST_INFO_FIRST_ENTRY ,0 ) | + FIELD_PREP(HPA_LIST_INFO_PFN, page_to_pfn(hpa_list_page)) | + FIELD_PREP(HPA_LIST_INFO_LAST_ENTRY, nr_pages - 1); +} + +static __init int tdx_ext_mem_add (struct page *hpa_list_page, + unsigned int nr_pages) +{ +struct tdx_module_args args = { + .rcx =to_hpa_list_info (hpa_list_page, nr_pages), + }; +u64 r; + + do {
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1200
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+/* +* TDH_EXT_MEM_ADD is designed to use output parameter RCX to +* override/update input parameter RCX, so the caller doesn't +* have to do manual parameter update on retry call. +*/ + r =seamcall_ret (TDH_EXT_MEM_ADD , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ;
Agree with Chao.
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1209
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +9
-0+ +return 0 ; +} + +struct tdx_hpa_list { +u64 phys[PAGE_SIZE /sizeof (u64)]; +}; + +static_assert (sizeof (struct tdx_hpa_list ) ==PAGE_SIZE );
What kind of change are we guarding against here?
Quoted 20 lines
arch/x86/virt/vmx/tdx/tdx.c +20
-0+ +static __init int tdx_ext_mem_setup (unsigned int required_pages) +{ +struct tdx_hpa_list *hpa_list; +struct page *page; +unsigned int i; +int ret; + +/* +* memory_pool_required_pages == 0 means no need to add pages, +* skip the memory setup. +*/ +if (!required_pages) +return 0 ; + + hpa_list =kzalloc_obj (*hpa_list); +if (!hpa_list) +return -ENOMEM ; + + page = alloc_contig_pages(required_pages, GFP_KERNEL, numa_mem_id(),
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1239
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+ &node_online_map); +if (!page) { + ret = -ENOMEM ; + goto out_free_hpa_list; + } + + i =0 ; +while (i < required_pages) { +unsigned int nents =min (required_pages - i, +ARRAY_SIZE (hpa_list->phys));
I think this whole loop could be made clearer. nents is the number of pages we will add for an execution of the seamcall. That is not clear enough either.
Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+unsigned int j; + +for (j =0 ; j < nents; j++) + hpa_list->phys[j] =page_to_phys (page + i + j); + + ret =tdx_ext_mem_add (virt_to_page (hpa_list), nents); +/* +* No SEAMCALLs to reclaim the added pages. For simple error +* handling, leak all pages. +*/ + WARN(ret, "Fatal: TDX module rejected (%d) memory forextensions, stranded all pages\n",
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ ret); +if (ret) +break ;
Can we combine the ret checks?
arch/x86/virt/vmx/tdx/tdx.c
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+ + i += nents;
Maybe a full name for i?
Quoted 49 lines
arch/x86/virt/vmx/tdx/tdx.c +16
-0+ } + +/* +* Memory for extensions can't be reclaimed once added, print out the +* amount, stop tracking it and free the hpa_list page, no matter +* success or failure. +*/ +pr_info ("%lu KB consumed for TDX module extensions\n" , + required_pages *PAGE_SIZE /1024 ); + +out_free_hpa_list: +kfree (hpa_list); + +return ret; +} +static
__initintinit_tdx_module_extensions(void){
structtdx_sys_info_extsysinfo_ext;arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1182,9 +1285,7 @@ static __init int init_tdx_module_extensions(void) if (!sysinfo_ext.ext_required) return 0;arch/x86/virt/vmx/tdx/tdx.c +1
-3-/* TODO: add the extensions enabling steps here */ - -return 0 ; +return tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages);}
static__initintinit_tdx_module(void)arch/x86/virt/vmx/tdx/tdx_global_metadata.c +0
-0@@ -137,6 +137,12 @@ static __init int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext) int ret; u64 val;arch/x86/virt/vmx/tdx/tdx_global_metadata.c +6
-0+ ret =read_sys_metadata_field (0x3100000200000000 , &val); +if (ret) +return ret; + + sysinfo_ext->memory_pool_required_pages = val; +ret =
read_sys_metadata_field(0x3100000000000001, &val); if (ret)
return ret;
Re: [PATCH v2 04/17] x86/virt/tdx: Add extra memory to TDX module for the extensions
Quoted 8 lines
Quoted 7 lines
TDH.EXT.MEM.ADD uses a new parameter type,
HPA_LIST_INFO, to provide this memory. This type represents a list of pages for TDX module to access. It references an 'hpa_listpage' which contains the list of target HPAs. It collapses the HPA of thehpa_listpage and the number of valid target HPAs into a 64 bit raw value for SEAMCALL parameters. Thehpa_listpage is always a medium, TDX module never keeps thehpa_listpage.
Quoted 7 lines
I think you are going to need to put the basic concepts up front. Don't lead with the acronyms. The TDX module needs to get a list of pages. It takes them in a PFN array that is encoded like (...). Create a helper for this, and name it the same after the TDX module term for this format,
HPA_LIST_INFO.Basically there are no exotic concepts here. So you can explain it in normal kernel terms.
OK. My refactored version:
When adding the memory, the TDX module needs to get a list of pages. It accepts these target pages in a PFN array that is encoded into a single 64-bit SEAMCALL parameter: the PFN of a container page that holds the PFN array, and the number of the pages packed into a single value. Create a helper to encode this format and name it after the TDX module term: HPA_LIST_INFO.
[...]
Quoted 9 lines
Quoted 6 lines
TDX module extensions consume tens of megabytes memory that
will never be returned to host. Use contiguous page allocation to isolate these large blocks entirely, avoiding permanent memory fragmentation and reducing buddy allocator efficiency. Print the allocation amount on TDX module extensions initialization for visibility.Hmm. I'm not sure the user will care about the specific category of extension used memory.
Not sure if you suggest removing the print, but this is suggested by Dave:
https://lore.kernel.org/all/[email protected]/
[...]
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1209
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +5
-0+struct tdx_hpa_list { +u64 phys[PAGE_SIZE /sizeof (u64)]; +}; + +static_assert (sizeof (struct tdx_hpa_list ) ==PAGE_SIZE );What kind of change are we guarding against here?
I don't expect changes of the format, but this is to ensure the container of the PFN array is a page, and also serves as the documentation.
[...]
Quoted 17 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1239
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+ i =0 ; +while (i < required_pages) { +unsigned int nents =min (required_pages - i, +ARRAY_SIZE (hpa_list->phys));I think this whole loop could be made clearer. nents is the number of pages we will add for an execution of the seamcall. That is not clear enough either.
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +9
-0+unsigned int j; + +for (j =0 ; j < nents; j++) + hpa_list->phys[j] =page_to_phys (page + i + j); + + ret =tdx_ext_mem_add (virt_to_page (hpa_list), nents); +/* +* No SEAMCALLs to reclaim the added pages. For simple error +* handling, leak all pages.
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1253
Quoted 6 lines
(quoted diff) +2
-0+ */ + WARN(ret, "Fatal: TDX module rejected (%d) memory forextensions, stranded all pages\n",
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ ret); +if (ret) +break ;Can we combine the ret checks?
Yes.
Quoted 5 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ + i += nents;Maybe a full name for i?
Yes, I changed the name of i & nents.
PATCH v2 5/17 x86/virt/tdx: Make TDX module initialize the extensions
In reply to Xu YilunAfter providing all required memory to TDX module, initialize TDX module extensions via TDH.EXT.INIT, so extension SEAMCALLs can be used.
--- arch/x86/virt/vmx/tdx/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.c | 22 +++++++++++++++++++++- 2 files changed, 22 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.h +1 -0
@@ -63,6 +63,7 @@#define TDH_SYS_SHUTDOWN 52#define TDH_SYS_UPDATE_V0 53#define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69
arch/x86/virt/vmx/tdx/tdx.c +21 -1
@@ -1167,6 +1167,22 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +/* Initialize TDX module extensions for extension SEAMCALLs */ +static __init int tdx_ext_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_EXT_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ; + +return 0 ; +} +#define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3)#define HPA_LIST_INFO_PFN GENMASK_U64(51, 12)#define HPA_LIST_INFO_LAST_ENTRY GENMASK_U64(63, 55)
@@ -1285,7 +1301,11 @@ static __init int init_tdx_module_extensions(void)if (!sysinfo_ext.ext_required)return 0 ; -return tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages); + ret =tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages); +if (ret) +return ret; + +return tdx_ext_init (); }static __init int init_tdx_module (void )
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [Medium] tdx_ext_init() (added in this patch) and tdx_ext_mem_add() silently discard the 64-bit hardware error code from their SEAMCALLs and return -EFAULT without logging.
Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but by the earlier commit 'x86/virt/tdx: Add extra memory to TDX module for the extensions'. The hpa_list structure is allocated via kzalloc_obj() which does not guarantee page-alignment. --
commit 555dbf36da39abb049edf2bd1291c474f12aac77
Author: Xu Yilun <[email protected]>
x86/virt/tdx: Make TDX module initialize the extensions
After providing all required memory to TDX module, initialize TDX module extensions via TDH.EXT.INIT, so extension SEAMCALLs can be used.
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0
[ ... ]
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0@@ -1167,6 +1167,22 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +/* Initialize TDX module extensions for extension SEAMCALLs */ +static __init int tdx_ext_init (void )
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1181
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_EXT_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ;
[Severity: Medium]
Does this code silently discard the 64-bit hardware error code?
When TDH_EXT_INIT fails, replacing the return value with -EFAULT means the caller only logs a generic initialization failure without details. Other TDX initialization functions use seamcall_prerr_ret() to preserve and log the hardware error.
Could this be updated to use seamcall_prerr_ret() to ensure the exact firmware failure reason is logged?
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+ +return 0 ; +} +
[ ... ]
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1304
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +1
-1@@ -1285,7 +1301,11 @@ static __init int init_tdx_module_extensions(void)if (!sysinfo_ext.ext_required)return 0 ; -return tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages); + ret =tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages);
[Severity: High]
This isn't a bug introduced by this patch, but does the earlier introduction of tdx_ext_mem_setup() cause an alignment issue?
Looking at tdx_ext_mem_setup() in arch/x86/virt/vmx/tdx/tdx.c:
tdx_ext_mem_setup() {
...
hpa_list = kzalloc_obj(*hpa_list);
if (!hpa_list)
return -ENOMEM;
...
ret = tdx_ext_mem_add(virt_to_page(hpa_list), nents);
...
}The TDX module's TDH.EXT.MEM.ADD leaf expects a page-aligned PFN for the HPA list. Since kmalloc does not guarantee page-alignment (especially if CONFIG_SLUB_DEBUG is enabled), virt_to_page() drops the unaligned offset.
Will this result in the TDX module reading from offset 0 of the page, potentially consuming SLUB metadata instead of the actual hpa_list entries?
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+if (ret) +return ret; + +return tdx_ext_init (); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions
Quoted 16 lines
Review
Quoted 12 lines
(quoted diff) +12
-0+/* Initialize TDX module extensions for extension SEAMCALLs */ +static __init int tdx_ext_init(void) +{ + struct tdx_module_args args = {}; + u64 r; + + do { + r = seamcall(TDH_EXT_INIT, &args); + } while (r == TDX_INTERRUPTED_RESUMABLE); + + if (r != TDX_SUCCESS) + return -EFAULT;[Severity: Medium]
Does this code silently discard the 64-bit hardware error code?
Quoted 7 lines
When
TDH_EXT_INITfails, replacing the return value with-EFAULTmeans the caller only logs a generic initialization failure without details. Other TDX initialization functions useseamcall_prerr_ret()to preserve and log the hardware error.Could this be updated to use
seamcall_prerr_ret()to ensure the exact firmware failure reason is logged?
No, it can't use seamcall_prerr_ret() directly. TDX_INTERRUPTED_RESUMABLE can't be handled in a unified way for all SEAMCALLs so can't embed in fundamental seamcall wrappers. I see 2 conventions:
1. Host should set a resume=1 parameter on retry, such as, TDH_PHYMEM_CACHE_WB, and several migration SEAMCALLs, TDH.EXPORT.MEM, TDH.EXPORT.STATE.XXX
2. Host just refills the previous args (may have been modified by seamcall output) on retry. Much more examples, TDH_EXT_INIT, TDH_EXT_MEM_ADD, TDH_QUOTE_INIT, TDH_QUOTE_GET...
So I think there is a need the callers have to:
1. call seamcall() or seamcall wrappers (tdh_xxx)
2. specific handling of raw SEAMCALL return code
3. do a standard TDX error logging and Linux errno conversion.
seamcall_prerr_ret() does 1 & 2 & 3, but can't customize step 2.
I tried to add some TDX error logging & errno conversion helpers, most of the code are abstracted from seamcall_prerr_ret(), see below.
But I'm not sure if it worths the churn.
----8<----
arch/x86/virt/vmx/tdx/seamcall_internal.h +18 -5
@@ -78,12 +78,10 @@ static inline void seamcall_err_ret(u64 fn, u64 err, args->r9, args->r10, args->r11); } -static __always_inline int sc_retry_prerr(sc_func_t func, -sc_err_func_t err_func, -u64 fn,struct tdx_module_args *args) +static inline int sc_errno_prerr (sc_err_func_t err_func, +u64 fn,u64 sret, +struct tdx_module_args *args) { -u64 sret =sc_retry (func, fn, args); -if (sret ==TDX_SUCCESS )return 0 ;
@@ -100,10 +98,25 @@ static __always_inline int sc_retry_prerr(sc_func_t func,return -EIO ; } +static __always_inline int sc_retry_prerr(sc_func_t func, +sc_err_func_t err_func, +u64 fn,struct tdx_module_args *args) +{ +u64 sret =sc_retry (func, fn, args); + +return sc_errno_prerr (err_func, fn, sret, args); +} +#define seamcall_prerr (__fn, __args) \ sc_retry_prerr(__seamcall, seamcall_err, (__fn), (__args))#define seamcall_prerr_ret (__fn, __args) \ sc_retry_prerr(__seamcall_ret, seamcall_err_ret, (__fn), (__args)) +#define seamcall_errno_prerr (__fn, __sret, __args) \ + sc_errno_prerr(seamcall_err, (__fn), (__sret), (__args)) + +#define seamcall_errno_prerr_ret (__fn, __sret, __args) \ + sc_errno_prerr(seamcall_err_ret, (__fn), (__sret), (__args)) +#endif /* _X86_VIRT_SEAMCALL_INTERNAL_H */
arch/x86/virt/vmx/tdx/tdx.c +2 -8
@@ -1403,10 +1403,7 @@ static int tdx_ext_init(void) r =seamcall (TDH_EXT_INIT , &args); }while (r ==TDX_INTERRUPTED_RESUMABLE ); -if (r !=TDX_SUCCESS ) -return -EFAULT ; - -return 0 ; +return seamcall_errno_prerr (TDH_EXT_INIT , r, &args); }#define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3)
@@ -1438,10 +1435,7 @@ static __init int tdx_ext_mem_add(struct page *hpa_list_page, r =seamcall_ret (TDH_EXT_MEM_ADD , &args); }while (r ==TDX_INTERRUPTED_RESUMABLE ); -if (r !=TDX_SUCCESS ) -return -EFAULT ; - -return 0 ; +return seamcall_errno_prerr_ret (TDH_EXT_MEM_ADD , r, &args); }
Re: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions
Quoted 4 lines · Xu Yilun
Quoted 4 lines
Quoted 4 lines
(quoted diff) +4
-0+/* Initialize TDX module extensions for extension SEAMCALLs */ +static __init int tdx_ext_init(void) +{ + struct tdx_module_args args = {};
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ u64 r; + + do { + r = seamcall(TDH_EXT_INIT, &args); + } while (r == TDX_INTERRUPTED_RESUMABLE); + + if (r != TDX_SUCCESS) + return -EFAULT;
As Chao caught in patch 04, we need to return -EIO here for consistence.
Quoted 30 lines
Quoted 11 lines
[Severity: Medium]
Does this code silently discard the 64-bit hardware error code?When
TDH_EXT_INITfails, replacing the return value with-EFAULTmeans the caller only logs a generic initialization failure without details. Other TDX initialization functions useseamcall_prerr_ret()to preserve and log the hardware error.Could this be updated to use
seamcall_prerr_ret()to ensure the exact firmware failure reason is logged?No, it can't use
seamcall_prerr_ret()directly.TDX_INTERRUPTED_RESUMABLEcan't be handled in a unified way for all SEAMCALLs so can't embed in fundamental seamcall wrappers. I see 2 conventions:1. Host should set a resume=1 parameter on retry, such as,
TDH_PHYMEM_CACHE_WB, and several migration SEAMCALLs, TDH.EXPORT.MEM, TDH.EXPORT.STATE.XXX2. Host just refills the previous args (may have been modified by seamcall output) on retry. Much more examples,
TDH_EXT_INIT,TDH_EXT_MEM_ADD,TDH_QUOTE_INIT,TDH_QUOTE_GET...So I think there is a need the callers have to:
1. call
seamcall()or seamcall wrappers (tdh_xxx)
2. specific handling of raw SEAMCALL return code
3. do a standard TDX error logging and Linux errno conversion.
Quoted 6 lines
seamcall_prerr_ret()does 1 & 2 & 3, but can't customize step 2.I tried to add some TDX error logging & errno conversion helpers, most of the code are abstracted from
seamcall_prerr_ret(), see below.But I'm not sure if it worths the churn.
Though I have some comment, below refactor doesn't look so bad.
The only risk is existing SEAMCALL warppers is a mess and people may not want to add any more before re-work them. So the conservative option would to just add an ad-hoc pr_err() for !TDX_SUCCESS case?
Quoted 37 lines
----8<----
arch/x86/virt/vmx/tdx/seamcall_internal.h +12
-5@@ -78,12 +78,10 @@ static inline void seamcall_err_ret(u64 fn, u64 err, args->r9, args->r10, args->r11); } -static __always_inline int sc_retry_prerr(sc_func_t func, -sc_err_func_t err_func, -u64 fn,struct tdx_module_args *args) +static inline int sc_errno_prerr (sc_err_func_t err_func, +u64 fn,u64 sret, +struct tdx_module_args *args) { -u64 sret =sc_retry (func, fn, args); -if (sret ==TDX_SUCCESS )return 0 ;@@ -100,10 +98,25 @@ static __always_inline int sc_retry_prerr(sc_func_t func,return -EIO ; } +static __always_inline int sc_retry_prerr(sc_func_t func, +sc_err_func_t err_func, +u64 fn,struct tdx_module_args *args) +{ +u64 sret =sc_retry (func, fn, args); + +return sc_errno_prerr (err_func, fn, sret, args); +} +#define seamcall_prerr (__fn, __args) \
arch/x86/virt/vmx/tdx/seamcall_internal.h static __always_inline int sc_retry_prerr(sc_func_t func, · line 120
Quoted 10 lines
arch/x86/virt/vmx/tdx/seamcall_internal.h +5
-0sc_retry_prerr(__seamcall, seamcall_err, (__fn), (__args))#define seamcall_prerr_ret (__fn, __args) \ sc_retry_prerr(__seamcall_ret, seamcall_err_ret, (__fn), (__args)) +#define seamcall_errno_prerr (__fn, __sret, __args) \ + sc_errno_prerr(seamcall_err, (__fn), (__sret), (__args)) + +#define seamcall_errno_prerr_ret (__fn, __sret, __args) \ + sc_errno_prerr(seamcall_err_ret, (__fn), (__sret), (__args))
We don't need these two wrappers.
The existing seamcall_prerr() and seamcall_prerr_ret() do two things:
1. call SEAMCALL;
2. handling the return code of SEAMCALL;
But the 2 new wrappers only do setp 2. So we can just call sc_errno_prerr() in the following cases ...
Quoted 6 lines
arch/x86/virt/vmx/tdx/seamcall_internal.h +0
-0#endif /* _X86_VIRT_SEAMCALL_INTERNAL_H */ arch/x86/virt/vmx/tdx/tdx.c +0
-0
arch/x86/virt/vmx/tdx/tdx.c static int tdx_ext_init(void) · line 1406
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +1
-4@@ -1403,10 +1403,7 @@ static int tdx_ext_init(void) r =seamcall (TDH_EXT_INIT , &args); }while (r ==TDX_INTERRUPTED_RESUMABLE ); -if (r !=TDX_SUCCESS ) -return -EFAULT ; - -return 0 ; +return seamcall_errno_prerr (TDH_EXT_INIT , r, &args);
... return sc_errno_prerr(seamcall_err, TDH_EXT_INIT, r, &args);
and
arch/x86/virt/vmx/tdx/tdx.c static __init int tdx_ext_mem_add(struct page *hpa_list_page, · line 1438
Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +1
-4}#define HPA_LIST_INFO_FIRST_ENTRY GENMASK_U64(11, 3)@@ -1438,10 +1435,7 @@ static __init int tdx_ext_mem_add(struct page *hpa_list_page, r =seamcall_ret (TDH_EXT_MEM_ADD , &args); }while (r ==TDX_INTERRUPTED_RESUMABLE ); -if (r !=TDX_SUCCESS ) -return -EFAULT ; - -return 0 ; +return seamcall_errno_prerr_ret (TDH_EXT_MEM_ADD , r, &args);
... return sc_errno_prerr(seamcall_err_ret, TDH_EXT_MEM_ADD, r, &args);
Quoted 1 line
arch/x86/virt/vmx/tdx/tdx.c +0
-0}
Edgecombe, Rick P
maintainerRe: [PATCH v2 05/17] x86/virt/tdx: Make TDX module initialize the extensions
In reply to Xu YilunQuoted 2 lines · Xu Yilun
After providing all required memory to TDX module, initialize TDX module extensions via TDH.EXT.INIT, so extension SEAMCALLs can be used.
Pretty thin. Maybe some info that it is (apparently) a global init. Probably some very short background on what extensions are extension seamcalls are.
Quoted 64 lines
Signed-off-by Xu Yilun <[email protected]>Reviewed-by Xiaoyao Li <[email protected]>Reviewed-by Tony Lindgren <[email protected]>Reviewed-by Adrian Hunter <[email protected]>---
arch/x86/virt/vmx/tdx/tdx.h | 1 +arch/x86/virt/vmx/tdx/tdx.c| 22 +++++++++++++++++++++-2 files changed, 22 insertions(+), 1 deletion(-)arch/x86/virt/vmx/tdx/tdx.h +1
-0@@ -63,6 +63,7 @@ #define TDH_SYS_SHUTDOWN 52 #define TDH_SYS_UPDATE_V0 53 #define TDH_SYS_UPDATE SEAMCALL_LEAF_VER(TDH_SYS_UPDATE_V0, 1) +#define TDH_EXT_INIT 60 #define TDH_EXT_MEM_ADD 61 #define TDH_SYS_DISABLE 69arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1167,6 +1167,22 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) return 0; }arch/x86/virt/vmx/tdx/tdx.c +16
-0+/* Initialize TDX module extensions for extension SEAMCALLs */ +static __init int tdx_ext_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_EXT_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ; + +return 0 ; +} +#define
HPA_LIST_INFO_FIRST_ENTRYGENMASK_U64(11, 3)
#defineHPA_LIST_INFO_PFNGENMASK_U64(51, 12)
#defineHPA_LIST_INFO_LAST_ENTRYGENMASK_U64(63, 55)arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1285,7 +1301,11 @@ static __init int init_tdx_module_extensions(void) if (!sysinfo_ext.ext_required) return 0;arch/x86/virt/vmx/tdx/tdx.c +5
-1-return tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages); + ret =tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages); +if (ret) +return ret; + +return tdx_ext_init ();}
static__initintinit_tdx_module(void)
PATCH v2 6/17 x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
In reply to Xu YilunRuntime TDX module update introduces a mechanism to update the module firmware while preserving and restoring TDX operations. As part of the restoration process, the host must re-execute all extensions initialization steps to restore extension SEAMCALL functionality.
However, Linux runs the update in stop_machine() context, which prevents memory allocation. This introduces a hard restriction that the updated TDX environment must not consume more memory for the extensions. Consequently, the post-update initialization for the extensions is implemented as:
1. Detect if the extensions are supported and required.
2. Detect if the extensions require additional memory. If yes, fail the update.
3. Initialize the extensions via TDH.EXT.INIT.
The memory allocation problem is greatly mitigated since Linux applies a policy that configures the same add-on features for boot and for update. This policy minimizes the chance of increased memory demand. So now the restriction only affects the compatibility rule for choosing the update image.
--- arch/x86/virt/vmx/tdx/tdx.c | 31 ++++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 2 +- 2 files changed, 31 insertions(+), 2 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +30 -1
@@ -1168,7 +1168,7 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) }/* Initialize TDX module extensions for extension SEAMCALLs */ -static __init int tdx_ext_init (void ) +static int tdx_ext_init (void ) {struct tdx_module_args args = {};u64 r;
@@ -1308,6 +1308,31 @@ static __init int init_tdx_module_extensions(void)return tdx_ext_init (); } +/* +* Mostly the same flow as init_tdx_module_extensions(), but rejects adding +* more memory. +*/ +static int update_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +if (!sysinfo_ext.ext_required) +return 0 ; + +if (sysinfo_ext.memory_pool_required_pages) +return -EFAULT ; + +return tdx_ext_init (); +} +static __init int init_tdx_module (void ) {int ret;
@@ -1498,6 +1523,10 @@ int tdx_module_run_update(void)*/ WARN_ON_ONCE (ret); + ret =update_tdx_module_extensions (); +if (ret) +return ret; + tdx_module_state.initialized = true;return 0 ; }
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +1 -1
@@ -132,7 +132,7 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)return ret; } -static __init int get_tdx_sys_info_ext (struct tdx_sys_info_ext *sysinfo_ext) +static int get_tdx_sys_info_ext (struct tdx_sys_info_ext *sysinfo_ext) {int ret;u64 val;
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [Critical] Infinite loop in tdx_ext_init() when an external interrupt becomes pending during stop_machine_cpuslocked().
Pre-existing issues:
- [High] Infinite loop in tdx_sys_disable() when an external interrupt becomes pending during a kernel panic or crash. --
commit 03507e2a23bf57aa355b6cf99d6b8107f41ea488
Author: Xu Yilun <[email protected]>
x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Runtime TDX module update introduces a mechanism to update the module firmware while preserving and restoring TDX operations. As part of the restoration process, the post-update initialization detects if extensions are supported, rejects updates requiring additional memory, and initializes the extensions via TDH.EXT.INIT.
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0
[ ... ]
Quoted 17 lines
arch/x86/virt/vmx/tdx/tdx.c +13
-0@@ -1308,6 +1308,31 @@ static __init int init_tdx_module_extensions(void)return tdx_ext_init (); } +/* +* Mostly the same flow as init_tdx_module_extensions(), but rejects adding +* more memory. +*/ +static int update_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext);
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1333
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+if (ret) +return ret; + +if (!sysinfo_ext.ext_required) +return 0 ; + +if (sysinfo_ext.memory_pool_required_pages) +return -EFAULT ; + +return tdx_ext_init ();
[Severity: Critical]
Could this result in an infinite loop during a runtime update?
The tdx_module_run_update() function runs under stop_machine(), meaning local interrupts are explicitly disabled. When this calls tdx_ext_init(), it executes the following polling loop:
do {
r = seamcall(TDH_EXT_INIT, &args);
} while (r == TDX_INTERRUPTED_RESUMABLE);If an external hardware interrupt becomes pending during the update, the TDX module exits to the host and returns TDX_INTERRUPTED_RESUMABLE. Because the host CPU has interrupts disabled, it cannot service the interrupt and simply continues execution.
Does this mean the loop condition evaluates to true, immediately re-executing the seamcall, and permanently deadlocking the system?
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1337
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+} +static __init int init_tdx_module (void ) {
[ ... ]
[Severity: High]
This isn't a bug introduced by this patch, but while looking at how TDX_INTERRUPTED_RESUMABLE is handled, I noticed a similar pattern in tdx_sys_disable():
do {
ret = seamcall(TDH_SYS_DISABLE, &args);
} while (ret == TDX_INTERRUPTED_RESUMABLE || ret == TDX_SYS_BUSY);During a kernel panic or crash, machine_crash_shutdown() executes with local interrupts disabled. If an external interrupt becomes pending, the TDX module exits with TDX_INTERRUPTED_RESUMABLE.
With host interrupts permanently disabled in the panic context, will this loop endlessly and prevent the crash dump or kexec process from completing?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Quoted 13 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1333
Quoted 1 line
arch/x86/virt/vmx/tdx/tdx.c +1
-0+return tdx_ext_init ();[Severity: Critical]
Could this result in an infinite loop during a runtime update?The
tdx_module_run_update()function runs understop_machine(), meaning local interrupts are explicitly disabled. When this callstdx_ext_init(), it executes the following polling loop:do { r = seamcall(TDH_EXT_INIT, &args); } while (r == TDX_INTERRUPTED_RESUMABLE);
Quoted 7 lines
If an external hardware interrupt becomes pending during the update, the TDX module exits to the host and returns
TDX_INTERRUPTED_RESUMABLE. Because the host CPU has interrupts disabled, it cannot service the interrupt and simply continues execution.Does this mean the loop condition evaluates to true, immediately re-executing the seamcall, and permanently deadlocking the system?
No, According to TDX module SPEC [1], 5.4.1. SEAMCALL Instruction (Common), RAX[bit 24] == 0 shows "Pending interrupts are detected only if the host VMM’s RFLAGS.IF is 1." Linux now doesn't set this bit for any SEAMCALL.
That means SEAMCALLs will keep on executing if host disabled interrupts so will not deadlock the system.
Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
In reply to Xu YilunQuoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+/* +* Mostly the same flow as init_tdx_module_extensions(), but rejects adding +* more memory. +*/ +static int update_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ;
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1331
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+ + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret; + +if (!sysinfo_ext.ext_required) +return 0 ; + +if (sysinfo_ext.memory_pool_required_pages) +return -EFAULT ;
Will tdx_ext_init() return an error if more memory is needed?
If yes, we can leave this check to the module. And with ext_required removed (per my earlier comment), this function simplifies to:
int update_tdx_module_extensions(void)
{
if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
return 0;
return tdx_ext_init();
}Quoted 34 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+ +return tdx_ext_init (); +} +static
__initintinit_tdx_module(void){ int ret;arch/x86/virt/vmx/tdx/tdx.c +4
-0@@ -1498,6 +1523,10 @@ int tdx_module_run_update(void)*/ WARN_ON_ONCE (ret); + ret =update_tdx_module_extensions (); +if (ret) +return ret; + tdx_module_state.initialized = true;return 0 ; }arch/x86/virt/vmx/tdx/tdx_global_metadata.c +1
-1@@ -132,7 +132,7 @@ static __init int get_tdx_sys_info(struct tdx_sys_info *sysinfo)return ret; } -static __init int get_tdx_sys_info_ext (struct tdx_sys_info_ext *sysinfo_ext) +static int get_tdx_sys_info_ext (struct tdx_sys_info_ext *sysinfo_ext) {int ret;u64 val;Signature · 1 line
2.25.1
Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Quoted 15 lines · Chao Gao
Quoted 15 lines
arch/x86/virt/vmx/tdx/tdx.c +15
-0+/* +* Mostly the same flow as init_tdx_module_extensions(), but rejects adding +* more memory. +*/ +static int update_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret;
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1331
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +6
-0+ +if (!sysinfo_ext.ext_required) +return 0 ; + +if (sysinfo_ext.memory_pool_required_pages) +return -EFAULT ;Will
tdx_ext_init()return an error if more memory is needed?
Yes.
Quoted 2 lines
If yes, we can leave this check to the module.
Yes, we can. Although we can't remove ext_required check, we can remove memory_pool_required_pages check, making the function simpler.
Edgecombe, Rick P
maintainerRe: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
In reply to Chao GaoQuoted 5 lines · Chao Gao
Will
tdx_ext_init()return an error if more memory is needed?If yes, we can leave this check to the module. And with
ext_requiredremoved (per my earlier comment), this function simplifies to:
Quoted 7 lines
int
update_tdx_module_extensions(void){ if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT)) return 0; return tdx_ext_init(); }
Oh, it can be simplified and not share so much with the init side. Please ignore my other question.
Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Quoted 11 lines · Rick P
Quoted 10 lines · Chao Gao
Will
tdx_ext_init()return an error if more memory is needed?If yes, we can leave this check to the module. And with
ext_requiredremoved (per my earlier comment), this function simplifies to:int
update_tdx_module_extensions(void){ if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT)) return 0;
Quoted 5 lines
Quoted 2 lines
return tdx_ext_init(); }Oh, it can be simplified and not share so much with the init side. Please ignore my other question.
It can't be simplified like that. If extensions are supported but no extension feature is configured at boot up time. tdx_ext_init() would fail and the runtime update failed.
I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip tdx_ext_init() and the update succeed, but after that the extension features will break.
IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes. We should store the extension metadata in global tdx_sysinfo, and that eliminate the duplicate code.
----8<----
arch/x86/include/asm/tdx_global_metadata.h +6 -5
@@ -44,17 +44,18 @@ struct tdx_sys_info_handoff { u16 module_hv; }; +struct tdx_sys_info_ext { +u32 memory_pool_required_pages; +bool ext_required; +}; +struct tdx_sys_info {struct tdx_sys_info_version version;struct tdx_sys_info_features features;struct tdx_sys_info_tdmr tdmr;struct tdx_sys_info_td_ctrl td_ctrl;struct tdx_sys_info_td_conf td_conf; -}; - -struct tdx_sys_info_ext { -u32 memory_pool_required_pages; -bool ext_required; +struct tdx_sys_info_ext ext; };#endif
arch/x86/virt/vmx/tdx/tdx.c +9 -18
@@ -1304,13 +1304,13 @@ static int tdx_ext_init(void)static __init int init_tdx_module_extensions (void ) { -struct tdx_sys_info_ext sysinfo_ext; +struct tdx_sys_info_ext *sysinfo_ext = &tdx_sysinfo.ext;int ret;if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT ))return 0 ; - ret =get_tdx_sys_info_ext (&sysinfo_ext); + ret =get_tdx_sys_info_ext (sysinfo_ext);if (ret)return ret;
@@ -1319,10 +1319,10 @@ static __init int init_tdx_module_extensions(void)* extensions are configured via TDH.SYS.CONFIG. If none, skip the * initialization. */ -if (!sysinfo_ext.ext_required) +if (!sysinfo_ext->ext_required)return 0 ; - ret =tdx_ext_mem_setup (sysinfo_ext.memory_pool_required_pages); + ret =tdx_ext_mem_setup (sysinfo_ext->memory_pool_required_pages);if (ret)return ret;
@@ -1330,23 +1330,14 @@ static __init int init_tdx_module_extensions(void) }/* -* Mostly the same flow as init_tdx_module_extensions(), but won't check and -* add more memory. Let the extensions re-initialization fail when more memory -* is actually needed. +* Don't update the extensions metadata, just follow the requirement originated +* on TDX module initialization. Let the extensions re-initialization fail when +* more memory is needed, or when the ext_required is accidentally flipped after +* update. */ static int update_tdx_module_extensions (void ) { -struct tdx_sys_info_ext sysinfo_ext; -int ret; - -if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) -return 0 ; - - ret =get_tdx_sys_info_ext (&sysinfo_ext); -if (ret) -return ret; - -if (!sysinfo_ext.ext_required) +if (!tdx_sysinfo.ext.ext_required)return 0 ;return tdx_ext_init ();
Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Quoted 22 lines · Xu Yilun
Quoted 11 lines · Rick P
Quoted 10 lines · Chao Gao
Will
tdx_ext_init()return an error if more memory is needed?If yes, we can leave this check to the module. And with
ext_requiredremoved (per my earlier comment), this function simplifies to:int
update_tdx_module_extensions(void){ if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT)) return 0;Quoted 5 lines
Quoted 2 lines
return tdx_ext_init(); }Oh, it can be simplified and not share so much with the init side. Please ignore my other question.
It can't be simplified like that. If extensions are supported but no extension feature is configured at boot up time.
tdx_ext_init()would fail and the runtime update failed.
Quoted 7 lines
I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip
tdx_ext_init()and the update succeed, but after that the extension features will break.IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes.
Sorry, I changed my mind. I remember TDX module architector said "the host should not assume any feature/SEAMCALL must be associated with any extension, it is hidden from the host. Today a SEAMCALL is backed by an extension, tomorrow it may not".
So in this case, ext_required may flip across updates and we should honor the runtime metadata.
I made another effort to combine common routines for ext init and update, but don't like how it turns out. If no one wants it that much, I'd rather stay unchanged:
----8<----
arch/x86/virt/vmx/tdx/tdx.c +12 -8
@@ -1302,15 +1302,22 @@ static int tdx_ext_init(void)return 0 ; } +static int get_tdx_sys_info_extensions (struct tdx_sys_info_ext *sysinfo_ext) +{ +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) { +memset (sysinfo_ext,0 ,sizeof (*sysinfo_ext)); +return 0 ; + } + +return get_tdx_sys_info_ext (sysinfo_ext); +} +static __init int init_tdx_module_extensions (void ) {struct tdx_sys_info_ext sysinfo_ext;int ret; -if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) -return 0 ; - - ret =get_tdx_sys_info_ext (&sysinfo_ext); + ret =get_tdx_sys_info_extensions (&sysinfo_ext);if (ret)return ret;
@@ -1339,10 +1346,7 @@ static int update_tdx_module_extensions(void)struct tdx_sys_info_ext sysinfo_ext;int ret; -if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) -return 0 ; - - ret =get_tdx_sys_info_ext (&sysinfo_ext); + ret =get_tdx_sys_info_extensions (&sysinfo_ext);if (ret)return ret;
Edgecombe, Rick P
maintainerRe: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Quoted 8 lines · Xu Yilun
Quoted 7 lines
I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip
tdx_ext_init()and the update succeed, but after that the extension features will break.IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes.
Quoted 7 lines
Sorry, I changed my mind. I remember TDX module architector said "the host should not assume any feature/SEAMCALL must be associated with any extension, it is hidden from the host. Today a SEAMCALL is backed by an extension, tomorrow it may not".
So in this case,
ext_requiredmay flip across updates and we should honor the runtime metadata.
Then this is an incompatible update, no? Linux doesn't support incompatible updates. We can keep it simpler.
Quoted 4 lines
I made another effort to combine common routines for ext init and update, but don't like how it turns out. If no one wants it that much, I'd rather stay unchanged:
Re: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
Quoted 14 lines · Rick P
Quoted 13 lines · Xu Yilun
Quoted 7 lines
I've found issue in my implementation. If extensions are required at boot up time but somehow not required after update, the code just skip
tdx_ext_init()and the update succeed, but after that the extension features will break.IOW, we should not update the metadata, we should follow the boot up time metadata to ensure no feature changes.
Sorry, I changed my mind. I remember TDX module architector said "the host should not assume any feature/SEAMCALL must be associated with any extension, it is hidden from the host. Today a SEAMCALL is backed by an extension, tomorrow it may not".
Quoted 5 lines
Quoted 2 lines
So in this case,
ext_requiredmay flip across updates and we should honor the runtime metadata.Then this is an incompatible update, no? Linux doesn't support incompatible updates. We can keep it simpler.
I imagine if a SEAMCALL is no longer backed by an extension and we switched off extension after update, the SEAMCALL would still work. But I don't have use case and I'm not sure.
OK. I'm good this is defined as incompatible update now. Maybe I'm too worried about thing that is hardly possible. I'll make this set simple.
Edgecombe, Rick P
maintainerRe: [PATCH v2 06/17] x86/virt/tdx: Re-initialize the extensions on runtime TDX module update
In reply to Xu YilunChao,
Quoted 11 lines · Xu Yilun
arch/x86/virt/vmx/tdx/tdx.c +11
-0+static int update_tdx_module_extensions (void ) +{ +struct tdx_sys_info_ext sysinfo_ext; +int ret; + +if (!(tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_EXT )) +return 0 ; + + ret =get_tdx_sys_info_ext (&sysinfo_ext); +if (ret) +return ret;
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1335
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+ +if (!sysinfo_ext.ext_required) +return 0 ; + +if (sysinfo_ext.memory_pool_required_pages) +return -EFAULT ; + +return tdx_ext_init (); +} +
These update and init functions are looking very similar, any idea how the could be combined?
static __init int init_tdx_module_extensions(void)
{
struct tdx_sys_info_ext sysinfo_ext;
int ret;
if (!(tdx_sysinfo.features.tdx_features0 & TDX_FEATURES0_EXT))
return 0;
ret = get_tdx_sys_info_ext(&sysinfo_ext);
if (ret)
return ret;
/* Skip if no feature requires TDX module extensions. */
if (!sysinfo_ext.ext_required)
return 0;
ret = tdx_ext_mem_setup(sysinfo_ext.memory_pool_required_pages);
if (ret)
return ret;
return tdx_ext_init();
}PATCH v2 7/17 x86/virt/tdx: Initialize Quoting extension
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Initialize the Quoting extension during TDX bringup, after enabling TDX module Extension.
Because Quoting is an optional TDX feature, do not let initialization failures cause TDX bringup to fail. In that case, TDX can fall back to the existing userspace flow via a KVM return code.
Only lay the groundwork for TDX Quoting support. Leave the opt-in portion of the initialization to a follow-up patch after fully implementing the feature.
--- arch/x86/include/asm/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.c | 34 +++++++++++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 1 deletion(-)
arch/x86/include/asm/tdx.h +1 -0
@@ -36,6 +36,7 @@#define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)#define TDX_FEATURES0_EXT BIT_ULL(39) +#define TDX_FEATURES0_QUOTE BIT_ULL(50)#ifndef __ASSEMBLER__
arch/x86/virt/vmx/tdx/tdx.h +1 -0
@@ -66,6 +66,7 @@#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69 +#define TDH_QUOTE_INIT 100/* TDX page types */ #define PT_NDA 0x0
arch/x86/virt/vmx/tdx/tdx.c +33 -1
@@ -1167,6 +1167,32 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +/* Initialize quoting extension */ +static __init int tdx_quote_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_QUOTE_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ; + +return 0 ; +} + +static __init void init_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { + ret =tdx_quote_init (); +WARN_ON_ONCE (ret); + } +} +/* Initialize TDX module extensions for extension SEAMCALLs */ static int tdx_ext_init (void ) {
@@ -1305,7 +1331,13 @@ static __init int init_tdx_module_extensions(void)if (ret)return ret; -return tdx_ext_init (); + ret =tdx_ext_init (); +if (ret) +return ret; + +init_tdx_quoting_extension (); + +return 0 ; }/*
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Using WARN_ON_ONCE() for an optional fallback feature causes kernel panics with panic_on_warn and…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Using WARN_ON_ONCE() for an optional fallback feature causes kernel panics with panic_on_warn and drops the actual hardware error code. --
commit cc6fec2434c274e836f0735b1b3ace996ebcb151
Author: Peter Fang <[email protected]>
x86/virt/tdx: Initialize Quoting extension
This commit initializes the Quoting extension during TDX bringup as an optional feature. It intends to allow TDX initialization to gracefully fall back to a userspace flow if the quoting initialization fails.
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0
[ ... ]
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1181
Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +12
-0+/* Initialize quoting extension */ +static __init int tdx_quote_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_QUOTE_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ;
[Severity: High]
Does this discard the actual SEAMCALL error code? By returning -EFAULT here, the specific hardware error information from TDH_QUOTE_INIT is lost, which might make debugging initialization failures difficult.
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1192
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+ +return 0 ; +} + +static __init void init_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { + ret =tdx_quote_init (); +WARN_ON_ONCE (ret);
[Severity: High]
The commit message explicitly states that quoting is an optional feature and initialization failures shouldn't cause TDX bringup to fail.
Is WARN_ON_ONCE() too aggressive here? On systems configured with panic_on_warn=1 (which is common in many environments), this will trigger a kernel panic instead of gracefully falling back to the userspace flow.
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+ } +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension
Quoted 15 lines · On Thu, Jun 18, 2026 at 08:50:16AM +0000, [email protected] wrote:
Quoted 5 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0[ ... ]
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +9
-0+/* Initialize quoting extension */ +static __init int tdx_quote_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_QUOTE_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE );
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1181
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ +if (r !=TDX_SUCCESS ) +return -EFAULT ;[Severity: High]
Does this discard the actual SEAMCALL error code? By returning-EFAULThere, the specific hardware error information fromTDH_QUOTE_INITis lost, which might make debugging initialization failures difficult.
Yilun did a pretty decent analysis of this in [1]. There really isn't much the host can do if this fails, so it seems like a reasonable tradeoff to just make sure there's a record of this in the kernel log.
Quoted 17 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1192
Quoted 12 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+ +return 0 ; +} + +static __init void init_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { + ret =tdx_quote_init (); +WARN_ON_ONCE (ret);[Severity: High]
The commit message explicitly states that quoting is an optional feature and initialization failures shouldn't cause TDX bringup to fail.
Quoted 4 lines
Is
WARN_ON_ONCE()too aggressive here? On systems configured with panic_on_warn=1 (which is common in many environments), this will trigger a kernel panic instead of gracefully falling back to the userspace flow.
Having a splat here is to draw the user's attention. Patch 13 sets the TDX_FEATURES0_QUOTE bit only if the TDX module reports support for it. Advertising such support but then failing to initialize it is very likely a TDX module bug. Since attestation is pretty important to TDX security, I think using WARN_ON_ONCE() here should be okay, and the backtrace should be enough to describe the failure.
Quoted 4 lines
Signature · 2 lines
-- Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
[1] https://lore.kernel.org/kvm/ajq8RFjv2YFghvQp@yilunxu-OptiPlex-7050/
Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension
In reply to Xu YilunQuoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
Initialize the Quoting extension during TDX bringup, after enabling TDX module Extension.
This jumps into what the patch does without explaining what the Quoting extension is. A brief background would help reviewers who aren't familiar with it.
Quoted 46 lines
Because Quoting is an optional TDX feature, do not let initialization failures cause TDX bringup to fail. In that case, TDX can fall back to the existing userspace flow via a KVM return code.
Only lay the groundwork for TDX Quoting support. Leave the opt-in portion of the initialization to a follow-up patch after fully implementing the feature.
Signed-off-by Peter Fang <[email protected]>Signed-off-by Xu Yilun <[email protected]>---
arch/x86/include/asm/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.c | 34 +++++++++++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 1 deletion(-)arch/x86/include/asm/tdx.h +1
-0@@ -36,6 +36,7 @@#define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)#define TDX_FEATURES0_EXT BIT_ULL(39) +#define TDX_FEATURES0_QUOTE BIT_ULL(50)#ifndef __ASSEMBLER__arch/x86/virt/vmx/tdx/tdx.h +1
-0@@ -66,6 +66,7 @@#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69 +#define TDH_QUOTE_INIT 100/* TDX page types */ #define PT_NDA 0x0arch/x86/virt/vmx/tdx/tdx.c +0
-0
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1170
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +1
-0@@ -1167,6 +1167,32 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +/* Initialize quoting extension */
This comment isn't quite helpful.
Quoted 13 lines
arch/x86/virt/vmx/tdx/tdx.c +13
-0+static __init int tdx_quote_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_QUOTE_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ; + +return 0 ;
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1193
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+} + +static __init void init_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { + ret =tdx_quote_init (); +WARN_ON_ONCE (ret); + }
nit: Reduce indentation of the main body.
if (!(tdx_addon_feature0 & TDX_FEATURES0_QUOTE)) return; ret = tdx_quote_init(); WARN_ON_ONCE(ret);
Also, why two functions? Can we inline tdx_quote_init() here, or push the feature check into it.
Quoted 23 lines
arch/x86/virt/vmx/tdx/tdx.c +9
-1+} +/* Initialize TDX module extensions for extension SEAMCALLs */ static int tdx_ext_init (void ) {@@ -1305,7 +1331,13 @@ static __init int init_tdx_module_extensions(void)if (ret)return ret; -return tdx_ext_init (); + ret =tdx_ext_init (); +if (ret) +return ret; + +init_tdx_quoting_extension (); + +return 0 ; }/* Signature · 1 line
2.25.1
Re: [PATCH v2 07/17] x86/virt/tdx: Initialize Quoting extension
Quoted 9 lines · Chao Gao
Quoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
Initialize the Quoting extension during TDX bringup, after enabling TDX module Extension.
This jumps into what the patch does without explaining what the Quoting extension is. A brief background would help reviewers who aren't familiar with it.
Yep, I'll add more background in the changelog.
Quoted 9 lines
Quoted 9 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1167,6 +1167,32 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; }
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1170
Quoted 1 line
arch/x86/virt/vmx/tdx/tdx.c +1
-0+/* Initialize quoting extension */ This comment isn't quite helpful.
Got it. It makes more sense to add it in patch 12.
Quoted 27 lines
Quoted 14 lines
arch/x86/virt/vmx/tdx/tdx.c +13
-0+static __init int tdx_quote_init (void ) +{ +struct tdx_module_args args = {}; +u64 r; + +do { + r =seamcall (TDH_QUOTE_INIT , &args); + }while (r ==TDX_INTERRUPTED_RESUMABLE ); + +if (r !=TDX_SUCCESS ) +return -EFAULT ; + +return 0 ;arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1193
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+} + +static __init void init_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { + ret =tdx_quote_init (); +WARN_ON_ONCE (ret); + }nit: Reduce indentation of the main body.
Quoted 5 lines
if (!(tdx_addon_feature0 & TDX_FEATURES0_QUOTE)) return; ret = tdx_quote_init(); WARN_ON_ONCE(ret);
That's better. I'll make this change.
Quoted 4 lines
Also, why two functions? Can we inline
tdx_quote_init()here, or push the feature check into it.
tdx_quote_init() has a second call site in patch 12 (update_tdx_quoting_extension()), during runtime TDX module update.
Adding the "if (feature)" check inside a SEAMCALL helper doesn't seem like a common pattern. I'm a bit concerned about making this look inconsistent.
Quoted 1 line
PATCH v2 8/17 x86/virt/tdx: Prepare Quote buffer during extension bringup
In reply to Xu YilunFrom: Peter Fang <[email protected]>
During TDX attestation, the TDX guest asks the host to generate a signed, verifiable structure (a "Quote"). With the Quoting extension, the TDX module returns the Quote in pages that the host shares via an Extension-SEAMCALL.
The SEAMCALL accepts the host buffer pages as a linked list of 4KB "HPA_LINKED_LIST" nodes. Each entry holds the physical address of a 4KB data page, except for the last entry, which points to the next node. The TDX module reports the required Quote buffer size through a global metadata field. See [1] for details.
For simplicity, let all guests share a global buffer. Build the buffer's HPA_LINKED_LIST at Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.
[1] Intel TDX Module ABI specification, Section "Physical Memory Management Types"
--- arch/x86/include/asm/tdx_global_metadata.h | 4 + arch/x86/virt/vmx/tdx/tdx.c | 115 +++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 14 +++ 3 files changed, 129 insertions(+), 4 deletions(-)
arch/x86/include/asm/tdx_global_metadata.h +4 -0
@@ -57,4 +57,8 @@ struct tdx_sys_info_ext { bool ext_required; }; +struct tdx_sys_info_quote { +u32 max_quote_size; +}; +#endif
arch/x86/virt/vmx/tdx/tdx.c +111 -4
141 lines not on this page · raw message
arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14 -0
@@ -151,3 +151,17 @@ static int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)return 0 ; } + +static __init int get_tdx_sys_info_quote (struct tdx_sys_info_quote *sysinfo_quote) +{ +int ret; +u64 val; + + ret =read_sys_metadata_field (0x2300000200000002 , &val); +if (ret) +return ret; + + sysinfo_quote->max_quote_size = val; + +return 0 ; +}
Signature · 1 line
2.25.1
Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup
Quoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
For simplicity, let all guests share a global buffer. Build the buffer'sHPA_LINKED_LISTat Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.
To me it seems the pre-generated parts can be made into a template that can be copied to an allocated buffer looking at patch 11/17.
Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup
Quoted 8 lines · Tony Lindgren
Quoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
For simplicity, let all guests share a global buffer. Build the buffer'sHPA_LINKED_LISTat Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.To me it seems the pre-generated parts can be made into a template that can be copied to an allocated buffer looking at patch 11/17.
Thanks. I'm planning to switch to dynamic allocation in the next revision to make this simpler [1]. So the template will be the HPA_LINKED_LIST node pages plus the next pointers in these nodes.
[1] https://lore.kernel.org/linux-coco/20260626095833.GB1600180@pedri/
Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup
In reply to Xu YilunQuoted 3 lines · Xu Yilun
For simplicity, let all guests share a global buffer. Build the buffer's
HPA_LINKED_LISTat Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.
va-to-pa conversions are not expensive. Even for vmalloc() memory.If
I don't like the global buffer. It just generally seems like a bad idea. It needs locking, it wastes memory when nobody is doing quotes, etc..
Is there a way to do this without vmalloc()?
Could userspace provide the memory and then the kernel just does a gup to keep it in place while the TDX module is writing to it? Or, could it just be allocated:
void *buffer = vmalloc(size); mutex_lock(); // talk to TDX module mutex_unlock(); vfree(buffer); return ret;
That actually has some nice properties. It doesn't require locking for the buffer. It also penalizes folks doing lots of quotes. If you hammer on quoting, you hammer on the memory allocator and TLB flushing code and slow yourself down more. If you hammer on it from multiple threads, it gets even worse.
That seems like a feature not a bug.
Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup
Quoted 7 lines · Dave Hansen
Quoted 3 lines · Xu Yilun
For simplicity, let all guests share a global buffer. Build the buffer's
HPA_LINKED_LISTat Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.va-to-pa conversions are not expensive. Even for
vmalloc()memory.If
Quoted 8 lines
I don't like the global buffer. It just generally seems like a bad idea. It needs locking, it wastes memory when nobody is doing quotes, etc..
Is there a way to do this without
vmalloc()?Could userspace provide the memory and then the kernel just does a gup to keep it in place while the TDX module is writing to it? Or, could it just be allocated:
Hi Dave,
The GUP idea makes a lot of sense at a high level I think. However, the shared buffer used by the GHCI call is not 4K-aligned because it has a header in front of it, while the SEAMCALL expects 4K-aligned HPAs (lesson learned...). This memory does come from userspace though:
struct tdx_quote_req { <-- 4K-aligned
u64 version;
u64 status;
u32 in_len;
u32 out_len;
u8 data[]; <-- shared buffer
};So perhaps the simplest thing here is to just allocate a kernel buffer before the SEAMCALL. There's also another subtle wrinkle here: KVM is shifting from a FOLL_PIN-based API to an MMU-notifier-based API for guest page accesses [1], so I think that means more complex code to talk to the TDX module.
Quoted 9 lines
void *buffer = vmalloc(size); mutex_lock(); // talk to TDX module mutex_unlock(); vfree(buffer); return ret;
Quoted 7 lines
That actually has some nice properties. It doesn't require locking for the buffer. It also penalizes folks doing lots of quotes. If you hammer on quoting, you hammer on the memory allocator and TLB flushing code and slow yourself down more. If you hammer on it from multiple threads, it gets even worse.
That seems like a feature not a bug.
This is a really good point... I'm a bit worried about the global impact of vfree() though. IIUC it eventually triggers an all-CPU TLB shootdown, so a guest abusing this path would not only throttle itself, but could also hurt everyone else. This might be more useful if the buffer was allocated in userspace? A free() in userspace usually triggers a more limited TLB shootdown.
I think I can use alloc_pages_bulk() instead to avoid the TLB shootdown cost altogether. It's a bit more work in KVM, but maybe worth it...
Thanks,
Peter
Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup
In reply to Xu YilunQuoted 6 lines · Xu Yilun
From: Peter Fang <[email protected]>
During TDX attestation, the TDX guest asks the host to generate a signed, verifiable structure (a "Quote"). With the Quoting extension, the TDX module returns the Quote in pages that the host shares via an Extension-SEAMCALL.
nit: Just say the host provides the pages to the TDX module, no need to specify if it's via an ext seamcall or some otherway. One can infer this from the code itself.
Quoted 6 lines
The SEAMCALL accepts the host buffer pages as a linked list of 4KB "
HPA_LINKED_LIST" nodes. Each entry holds the physical address of a 4KB data page, except for the last entry, which points to the next node. The TDX module reports the required Quote buffer size through a global metadata field. See [1] for details.
That HPA_LINKED_LIST is basically a linked list of arrays, each node into the linked list being the array, and each entry of that array being the HPA witht he last being a pointer to the next HPA_LINKED_LIST, so in a way a 2 level data structure. Can you be more explicit and just say something along the lines of :
"SEAMCALL accepts host buffer pages arranged in a custom data structure, consisting of nodes containing HPAs, arranged linearly, and each node is linked to the next one via the last entry"
Quoted 89 lines
For simplicity, let all guests share a global buffer. Build the buffer's
HPA_LINKED_LISTat Quoting extension bringup. This saves a bunch of va-to-pa conversions at runtime.[1] Intel TDX Module ABI specification, Section "Physical Memory Management Types"
Signed-off-by Peter Fang <[email protected]>Signed-off-by Xu Yilun <[email protected]>--- arch/x86/include/asm/tdx_global_metadata.h | 4 + arch/x86/virt/vmx/tdx/tdx.c | 115 +++++++++++++++++++- arch/x86/virt/vmx/tdx/tdx_global_metadata.c | 14 +++ 3 files changed, 129 insertions(+), 4 deletions(-)arch/x86/include/asm/tdx_global_metadata.h +4
-0@@ -57,4 +57,8 @@ struct tdx_sys_info_ext { bool ext_required; }; +struct tdx_sys_info_quote { +u32 max_quote_size; +}; +#endif arch/x86/virt/vmx/tdx/tdx.c +38
-0@@ -32,6 +32,7 @@#include <linux/idr.h> #include <linux/kvm_types.h> #include <linux/bitfield.h> +#include <linux/vmalloc.h> #include <asm/page.h> #include <asm/special_insns.h> #include <asm/msr-index.h> @@ -71,6 +72,24 @@ static LIST_HEAD(tdx_memlist);static struct tdx_sys_info tdx_sysinfo; +/* +* Quote buffer shared with the TDX module for quote generation, in HPA linked +* list format. +* +* @buf: Virtual address of the quote buffer. +* @buf_len: Size of @buf in bytes. +* @hpa_entries: HPA entries, starting at the first list node. +* @hpa_entries_pa: Physical address for @hpa_entries. +*/ +struct tdx_quote_data { +void *buf; +u64 buf_len; +u64 *hpa_entries; +phys_addr_t hpa_entries_pa; +}; + +static struct tdx_quote_data tdx_quote; +static DEFINE_RAW_SPINLOCK (sysinit_lock);/* @@ -1167,6 +1186,81 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +static inline phys_addr_t tdx_vmalloc_to_pa (const void *addr) +{ +unsigned long pfn =vmalloc_to_pfn (addr); + +return PFN_PHYS (pfn); +} + +#define HPAS_PER_NODE (PAGE_SIZE / sizeof(u64)) + +/* +* Pass the quote buffer to the TDX module as an HPA linked list, where each +* node holds 4KB page HPAs and the last entry points to the next node. +*/ +static __init int tdx_quote_create_buf (unsigned int npages, +struct tdx_quote_data *qdata) +{ +unsigned int nnodes; +u64 *hpas; +void *qbuf;
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1217
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+int i, j; + +if (!npages) +return -EINVAL ; + +/* +* Each node holds up to (HPAS_PER_NODE - 1) 4KB page HPAs. +* The last entry of the node points to the next node. +*/ + nnodes =DIV_ROUND_UP (npages,HPAS_PER_NODE -1 );
nit: It's somewhat arbitrary but num_nodes seems more explicit, saving 3 chars is not worth it.
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1219
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+ + hpas =vmalloc_array (nnodes,PAGE_SIZE );
hpas is basically the starting node, so name it "nodes". Also I'm slightly confused why you use vmalloc to allocate a contiguous address space when the nodes are linked in a list? I.e you keep referring to list in the changelog and the code seems to be working in an array chunks?
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +4
-0+if (!hpas) +return -ENOMEM ; + +/*
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1233
Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +10
-0+* ~0ULL is the list terminator for HPA_LINKED_LIST. +* +* Pre-fill the last node with 0xff bytes so that unused entries are +* terminators. Overwrite populated entries later. +*/ +memset ((u8 *)hpas + (nnodes -1 ) *PAGE_SIZE ,0xff ,PAGE_SIZE ); + + qbuf =vcalloc (npages,PAGE_SIZE ); +if (!qbuf) + goto out_nomem;
'qbuf' is rather arbitrary here. You simply pre-allocate a bunch of pages which you then initialize into the nodes. Can't this allocation be moved inside the loop itself, of course it will increase the number of allocs happening but this is during initialization so it's not performance critical, yes?
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1236
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ +/* Populate the linked list */ +for (i =0 , j =0 ; j < npages; i++) {
nit: This feedback was given before, but both variables can be defined in this loop.
Quoted 81 lines
arch/x86/virt/vmx/tdx/tdx.c +43
-4+if ((i %HPAS_PER_NODE ) ==HPAS_PER_NODE -1 ) { +/* +* The last node entry always points to the next node. +* The address of the following entry must be on next +* node's page boundary. +*/ + hpas[i] =tdx_vmalloc_to_pa (&hpas[i +1 ]); +continue ; + } + + hpas[i] =tdx_vmalloc_to_pa ((u8 *)qbuf + j *PAGE_SIZE ); + j++; + } +> + qdata->buf = qbuf; + qdata->buf_len = (u64 )npages *PAGE_SIZE ; + qdata->hpa_entries = hpas; + qdata->hpa_entries_pa =tdx_vmalloc_to_pa (hpas); + +return 0 ; + +out_nomem: +vfree (hpas); + +return -ENOMEM ; +} +/* Initialize quoting extension */ static __init int tdx_quote_init (void ) {@@ -1185,12 +1279,25 @@ static __init int tdx_quote_init(void)static __init void init_tdx_quoting_extension (void ) { -int ret; +struct tdx_sys_info_quote sysinfo_quote; +unsigned int nr_quote_pages; + +if (!(tdx_addon_feature0 &TDX_FEATURES0_QUOTE )) +return ; -if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { - ret =tdx_quote_init (); -WARN_ON_ONCE (ret); +if (tdx_quote_init ()) { +WARN_ON_ONCE (1 ); +return ; } + +/* Quoting metadata is valid only after initialization */ +if (get_tdx_sys_info_quote (&sysinfo_quote)) +return ; + + nr_quote_pages =PAGE_ALIGN (sysinfo_quote.max_quote_size) / +PAGE_SIZE ; +if (tdx_quote_create_buf (nr_quote_pages, &tdx_quote)) +pr_err ("Failed to create quote buffer\n" ); }/* Initialize TDX module extensions for extension SEAMCALLs */ arch/x86/virt/vmx/tdx/tdx_global_metadata.c +14
-0@@ -151,3 +151,17 @@ static int get_tdx_sys_info_ext(struct tdx_sys_info_ext *sysinfo_ext)return 0 ; } + +static __init int get_tdx_sys_info_quote (struct tdx_sys_info_quote *sysinfo_quote) +{ +int ret; +u64 val; + + ret =read_sys_metadata_field (0x2300000200000002 , &val); +if (ret) +return ret; + + sysinfo_quote->max_quote_size = val; + +return 0 ; +}
Re: [PATCH v2 08/17] x86/virt/tdx: Prepare Quote buffer during extension bringup
Quoted 5 lines · Nikolay Borisov
Quoted 2 lines · Xu Yilun
From: Peter Fang <[email protected]>
Quoted 8 lines
Quoted 4 lines
During TDX attestation, the TDX guest asks the host to generate a signed, verifiable structure (a "Quote"). With the Quoting extension, the TDX module returns the Quote in pages that the host shares via an Extension-SEAMCALL.
nit: Just say the host provides the pages to the TDX module, no need to specify if it's via an ext seamcall or some otherway. One can infer this from the code itself.
Good point. Thanks for the simplification.
Quoted 14 lines
Quoted 7 lines
The SEAMCALL accepts the host buffer pages as a linked list of 4KB "
HPA_LINKED_LIST" nodes. Each entry holds the physical address of a 4KB data page, except for the last entry, which points to the next node. The TDX module reports the required Quote buffer size through a global metadata field. See [1] for details.That
HPA_LINKED_LISTis basically a linked list of arrays, each node into the linked list being the array, and each entry of that array being the HPA witht he last being a pointer to the nextHPA_LINKED_LIST, so in a way a 2 level data structure. Can you be more explicit and just say something along the lines of :
Quoted 3 lines
"SEAMCALL accepts host buffer pages arranged in a custom data structure, consisting of nodes containing HPAs, arranged linearly, and each node is linked to the next one via the last entry"
Oh this is much easier to read. Thanks for the suggestion! I can incorporate this.
Quoted 18 lines
Quoted 18 lines
arch/x86/virt/vmx/tdx/tdx.c +17
-0+ +#define HPAS_PER_NODE (PAGE_SIZE / sizeof(u64)) + +/* +* Pass the quote buffer to the TDX module as an HPA linked list, where each +* node holds 4KB page HPAs and the last entry points to the next node. +*/ +static __init int tdx_quote_create_buf (unsigned int npages, +struct tdx_quote_data *qdata) +{ +unsigned int nnodes; +u64 *hpas; +void *qbuf; +int i, j; + +if (!npages) +return -EINVAL ;
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1217
Quoted 6 lines
arch/x86/virt/vmx/tdx/tdx.c +6
-0+ +/* +* Each node holds up to (HPAS_PER_NODE - 1) 4KB page HPAs. +* The last entry of the node points to the next node. +*/ + nnodes =DIV_ROUND_UP (npages,HPAS_PER_NODE -1 );nit: It's somewhat arbitrary but
num_nodesseems more explicit, saving 3 chars is not worth it.
Sure, I can make this change.
Quoted 7 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ + hpas = vmalloc_array(nnodes, PAGE_SIZE);hpas is basically the starting node, so name it "nodes". Also I'm slightly confused why you use vmalloc to allocate a contiguous address space when the nodes are linked in a list? I.e you keep referring to list in the changelog and the code seems to be working in an array chunks?
Hmm. I used the vmalloc family because I wanted to keep using hpas[i] without switching to the next node's va. It felt a bit simpler to just keep doing "i++". Maybe that actually hurts readability? I can replace that part of the for loop with something like:
if (i == HPAS_PER_NODE - 1) {
next_node = kmalloc(PAGE_SIZE, GFP_KERNEL);
hpas[i] = virt_to_phys(next_node);
hpas = next_node;
i = 0;
}And "hpas[i]" feels a bit closer to what the code does than "nodes[i]"? It's filling this page with a bunch of HPAs.
Quoted 13 lines
Quoted 13 lines
arch/x86/virt/vmx/tdx/tdx.c +12
-0+if (!hpas) +return -ENOMEM ; + +/* +* ~0ULL is the list terminator for HPA_LINKED_LIST. +* +* Pre-fill the last node with 0xff bytes so that unused entries are +* terminators. Overwrite populated entries later. +*/ +memset ((u8 *)hpas + (nnodes -1 ) *PAGE_SIZE ,0xff ,PAGE_SIZE ); + + qbuf =vcalloc (npages,PAGE_SIZE );
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1233
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+if (!qbuf) + goto out_nomem;'qbuf' is rather arbitrary here. You simply pre-allocate a bunch of pages which you then initialize into the nodes. Can't this allocation be moved inside the loop itself, of course it will increase the number of allocs happening but this is during initialization so it's not performance critical, yes?
I see your point. It’s a bit subtle here. In patch 11, there is a kvmemdup() that depends on this qbuf being virtually contiguous. Based on feedback from other folks, I think I might change this and use runtime quote buffers allocated with kmalloc(). That would also avoid the TLB flushing penalty of vfree(). The free path would be a bit more complicated, but that should be okay.
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdmrs(struct tdmr_info_list *tdmr_list) · line 1236
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ +/* Populate the linked list */ +for (i =0 , j =0 ; j < npages; i++) {nit: This feedback was given before, but both variables can be defined in this loop.
Great suggestion. I can do that. Thanks.
Quoted 2 lines
PATCH v2 9/17 x86/virt/tdx: Add interface to check Quoting availability
In reply to Xu YilunFrom: Peter Fang <[email protected]>
KVM needs to know if the Quoting extension is available to determine whether userspace must be involved in Quote generation.
Since the Quote buffer is always created during Quoting extension bringup, checking whether the buffer exists is sufficient.
--- arch/x86/include/asm/tdx.h | 2 ++ arch/x86/virt/vmx/tdx/tdx.c | 10 ++++++++++ 2 files changed, 12 insertions(+)
arch/x86/include/asm/tdx.h +2 -0
@@ -147,6 +147,8 @@ struct tdx_vp {struct page **tdcx_pages; }; +bool tdx_quote_enabled (void ); +static inline u64 mk_keyed_paddr (u16 hkid,struct page *page) {u64 ret;
arch/x86/virt/vmx/tdx/tdx.c +10 -0
@@ -1193,6 +1193,16 @@ static inline phys_addr_t tdx_vmalloc_to_pa(const void *addr)return PFN_PHYS (pfn); } +bool tdx_quote_enabled (void ) +{ +/* +* No need for locking here. The quote buffer is initialized as part of +* core TDX bringup, which comes before KVM is ready for userspace. +*/ +return !!tdx_quote.buf; +} +EXPORT_SYMBOL_FOR_KVM (tdx_quote_enabled); +#define HPAS_PER_NODE (PAGE_SIZE / sizeof(u64))/*
Signature · 1 line
2.25.1
Re: [PATCH v2 09/17] x86/virt/tdx: Add interface to check Quoting availability
Quoted 7 lines · Xu Yilun
From: Peter Fang <[email protected]>
KVM needs to know if the Quoting extension is available to determine whether userspace must be involved in Quote generation.
Since the Quote buffer is always created during Quoting extension bringup, checking whether the buffer exists is sufficient.
PATCH v2 10/17 x86/virt/tdx: Move tdx_tdr_pa() up in the file
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Move tdx_tdr_pa() earlier in the file to prepare for upcoming changes that add a new Extension-SEAMCALL.
No functional change intended.
--- arch/x86/virt/vmx/tdx/tdx.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +5 -5
@@ -1186,6 +1186,11 @@ static __init int init_tdmrs(struct tdmr_info_list *tdmr_list)return 0 ; } +static inline u64 tdx_tdr_pa (struct tdx_td *td) +{ +return page_to_phys (td->tdr_page); +} +static inline phys_addr_t tdx_vmalloc_to_pa (const void *addr) {unsigned long pfn =vmalloc_to_pfn (addr);
@@ -1966,11 +1971,6 @@ void tdx_guest_keyid_free(unsigned int keyid) }EXPORT_SYMBOL_FOR_KVM (tdx_guest_keyid_free); -static inline u64 tdx_tdr_pa (struct tdx_td *td) -{ -return page_to_phys (td->tdr_page); -} -/* * The TDX module exposes a CLFLUSH_BEFORE_ALLOC bit to specify whether * a CLFLUSH of pages is required before handing them to the TDX module.
Signature · 1 line
2.25.1
Re: [PATCH v2 10/17] x86/virt/tdx: Move tdx_tdr_pa() up in the file
Quoted 6 lines · Xu Yilun
From: Peter Fang <[email protected]>
Move
tdx_tdr_pa()earlier in the file to prepare for upcoming changes that add a new Extension-SEAMCALL.No functional change intended.
PATCH v2 11/17 x86/virt/tdx: Add interface to generate a Quote
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Provide an interface to generate a Quote via the TDH.QUOTE.GET Extension-SEAMCALL. Although the TDX module may support concurrent Quote generation, use a single shared buffer for simplicity and serialize access with a mutex. TDX bringup code already prepares the buffer in the format required by the TDX module.
Return a per-call buffer containing the Quote so callers don't need to size the buffer themselves. The caller is responsible for freeing the returned buffer.
--- arch/x86/include/asm/tdx.h | 2 + arch/x86/virt/vmx/tdx/tdx.h | 1 + arch/x86/virt/vmx/tdx/tdx.c | 77 +++++++++++++++++++++++++++++++++++++ 3 files changed, 80 insertions(+)
arch/x86/include/asm/tdx.h +2 -0
@@ -148,6 +148,8 @@ struct tdx_vp { };bool tdx_quote_enabled (void ); +void *tdx_quote_generate (struct tdx_td *td,void *in_data,u32 in_data_len, +u32 *quote_len);static inline u64 mk_keyed_paddr (u16 hkid,struct page *page) {
arch/x86/virt/vmx/tdx/tdx.h +1 -0
@@ -66,6 +66,7 @@#define TDH_EXT_INIT 60#define TDH_EXT_MEM_ADD 61#define TDH_SYS_DISABLE 69 +#define TDH_QUOTE_GET 98#define TDH_QUOTE_INIT 100/* TDX page types */
arch/x86/virt/vmx/tdx/tdx.c +77 -0
89 lines not on this page · raw message
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] tdx_quote_generate() uses an uninterruptible mutex lock for a guest-triggered hardware operation…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] tdx_quote_generate() uses an uninterruptible mutex lock for a guest-triggered hardware operation, creating a potential host-level denial of service. --
commit 3bb73b03a2480c47534841c05b531b06bda047a9
Author: Peter Fang <[email protected]>
x86/virt/tdx: Add interface to generate a Quote
This commit provides an interface to generate a Quote via the TDH.QUOTE.GET Extension-SEAMCALL. It introduces a global mutex to serialize access to a shared quote buffer.
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +0
-0
[ ... ]
arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+void *tdx_quote_generate (struct tdx_td *td,void *in_data,u32 in_data_len, +u32 *quote_len) +{ +struct tdx_quote_data *qdata = &tdx_quote; +void *quote_dup =NULL ; +u64 r, out_len; + +if (!tdx_quote_enabled ()) +return NULL ; + +mutex_lock (&tdx_quote_lock);
[Severity: High]
Since this path can be triggered by a guest, could using an uninterruptible mutex_lock() here lead to a host-level denial of service?
If a guest rapidly requests quotes from multiple vCPUs, it could monopolize the global tdx_quote_lock. Other vCPU threads waiting for the lock would block in an uninterruptible sleep (D state). If an administrator attempts to kill the waiting VMs, the threads will not respond to fatal signals until they acquire the lock.
Would it be safer to use mutex_lock_killable() here to allow fatal signals to interrupt the wait and permit VM teardown?
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c +7
-0+ +/* +* Use the first page of the quote buffer for input data. The buffer +* must be at least one page in size. @in_data may not be page-aligned, +* but TDH.QUOTE.GET expects page-aligned addresses. +*/ +memcpy (qdata->buf, in_data, in_data_len);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote
Quoted 20 lines · On Thu, Jun 18, 2026 at 08:49:40AM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High]tdx_quote_generate()uses an uninterruptible mutex lock for a guest-triggered hardware operation, creating a potential host-level denial of service. --arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+void *tdx_quote_generate (struct tdx_td *td,void *in_data,u32 in_data_len, +u32 *quote_len) +{ +struct tdx_quote_data *qdata = &tdx_quote; +void *quote_dup =NULL ; +u64 r, out_len; + +if (!tdx_quote_enabled ()) +return NULL ; + +mutex_lock (&tdx_quote_lock);[Severity: High]
Since this path can be triggered by a guest, could using an uninterruptiblemutex_lock()here lead to a host-level denial of service?
Quoted 8 lines
If a guest rapidly requests quotes from multiple vCPUs, it could monopolize the global
tdx_quote_lock. Other vCPU threads waiting for the lock would block in an uninterruptible sleep (D state). If an administrator attempts to kill the waiting VMs, the threads will not respond to fatal signals until they acquire the lock.Would it be safer to use
mutex_lock_killable()here to allow fatal signals to interrupt the wait and permit VM teardown?
This is a good idea, but I think graceful teardown can be addressed later.
Quoted 4 lines
Signature · 2 lines
-- Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote
In reply to Xu YilunQuoted 3 lines · Xu Yilun
From: Peter Fang <[email protected]>
arch/x86/virt/vmx/tdx/tdx.c +0
-0
...
arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+void *tdx_quote_generate (struct tdx_td *td,void *in_data,u32 in_data_len, +u32 *quote_len) +{ +struct tdx_quote_data *qdata = &tdx_quote; +void *quote_dup =NULL ; +u64 r, out_len; + +if (!tdx_quote_enabled ()) +return NULL ; + +mutex_lock (&tdx_quote_lock);
How about make the pre-generated static tdx_quote a template page that only gets read and copied to an allocated bufer here?
If the tdx_quote template is only read for copying here, seems you're not going to need the mutex at all? That is assuming tdx_quote template does not change after init.
Re: [PATCH v2 11/17] x86/virt/tdx: Add interface to generate a Quote
Quoted 20 lines · Tony Lindgren
Quoted 3 lines · Xu Yilun
From: Peter Fang <[email protected]>
arch/x86/virt/vmx/tdx/tdx.c +0
-0...
arch/x86/virt/vmx/tdx/tdx.c bool tdx_quote_enabled(void) · line 1259
Quoted 11 lines
arch/x86/virt/vmx/tdx/tdx.c +11
-0+void *tdx_quote_generate (struct tdx_td *td,void *in_data,u32 in_data_len, +u32 *quote_len) +{ +struct tdx_quote_data *qdata = &tdx_quote; +void *quote_dup =NULL ; +u64 r, out_len; + +if (!tdx_quote_enabled ()) +return NULL ; + +mutex_lock (&tdx_quote_lock);How about make the pre-generated static
tdx_quotea template page that only gets read and copied to an allocated bufer here?
Quoted 3 lines
If the
tdx_quotetemplate is only read for copying here, seems you're not going to need the mutex at all? That is assumingtdx_quotetemplate does not change after init.
Hm, actually tdx_quote is an output buffer as well (in the form of a head pointer: qdata->hpa_entries_pa). Maybe this code needs better commenting...
PATCH v2 12/17 x86/virt/tdx: Reinitialize the Quoting extension after TDX module update
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Invoke TDH.QUOTE.INIT again after a runtime module update to trigger the necessary rekey procedure in the TDX module.
Keep the existing Quote buffer since memory allocation is not permitted during the update. Compatible TDX module updates must not increase the Quote buffer size, or an undersized buffer might cause Quote generation to fail. See [1] for module update details.
[1] Documentation/arch/x86/tdx.rst, Section "TDX module Runtime Update"
--- arch/x86/virt/vmx/tdx/tdx.c | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +28 -3
@@ -1353,8 +1353,11 @@ static __init int tdx_quote_create_buf(unsigned int npages,return -ENOMEM ; } -/* Initialize quoting extension */ -static __init int tdx_quote_init (void ) +/* +* Initialize quoting extension. +* It also rekeys the TDX module after a runtime module update. +*/ +static int tdx_quote_init (void ) {struct tdx_module_args args = {};u64 r;
@@ -1539,6 +1542,22 @@ static __init int init_tdx_module_extensions(void)return 0 ; } +static void update_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { +/* +* The TDH.QUOTE.INIT call renews the quoting keys. +* +* A module update must not increase the quote buffer size, or +* quote generation may fail and break attestation. +*/ + ret =tdx_quote_init (); +WARN_ON (ret); + } +} +/* * Mostly the same flow as init_tdx_module_extensions(), but rejects adding * more memory.
@@ -1561,7 +1580,13 @@ static int update_tdx_module_extensions(void)if (sysinfo_ext.memory_pool_required_pages)return -EFAULT ; -return tdx_ext_init (); + ret =tdx_ext_init (); +if (ret) +return ret; + +update_tdx_quoting_extension (); + +return 0 ; }static __init int init_tdx_module (void )
Signature · 1 line
2.25.1
Re: [PATCH v2 12/17] x86/virt/tdx: Reinitialize the Quoting extension after TDX module update
Quoted 11 lines · Xu Yilun
From: Peter Fang <[email protected]>
Invoke TDH.QUOTE.INIT again after a runtime module update to trigger the necessary rekey procedure in the TDX module.
Keep the existing Quote buffer since memory allocation is not permitted during the update. Compatible TDX module updates must not increase the Quote buffer size, or an undersized buffer might cause Quote generation to fail. See [1] for module update details.
[1]
Documentation/arch/x86/tdx.rst, Section "TDX module Runtime Update"
Edgecombe, Rick P
maintainerRe: [PATCH v2 12/17] x86/virt/tdx: Reinitialize the Quoting extension after TDX module update
In reply to Xu YilunQuoted 50 lines · Xu Yilun
From: Peter Fang <[email protected]>
Invoke TDH.QUOTE.INIT again after a runtime module update to trigger the necessary rekey procedure in the TDX module.
Keep the existing Quote buffer since memory allocation is not permitted during the update. Compatible TDX module updates must not increase the Quote buffer size, or an undersized buffer might cause Quote generation to fail. See [1] for module update details.
[1]
Documentation/arch/x86/tdx.rst, Section "TDX module Runtime Update"Signed-off-by Peter Fang <[email protected]>---
arch/x86/virt/vmx/tdx/tdx.c | 31 ++++++++++++++++++++++++++++---1 file changed, 28 insertions(+), 3 deletions(-)arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1353,8 +1353,11 @@ static __init int tdx_quote_create_buf(unsigned int npages, return -ENOMEM; }arch/x86/virt/vmx/tdx/tdx.c +5
-2-/* Initialize quoting extension */ -static __init int tdx_quote_init (void ) +/* +* Initialize quoting extension. +* It also rekeys the TDX module after a runtime module update. +*/ +static int tdx_quote_init (void ){
structtdx_module_argsargs = {};
u64 r;arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1539,6 +1542,22 @@ static __init int init_tdx_module_extensions(void) return 0; }arch/x86/virt/vmx/tdx/tdx.c +9
-0+static void update_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { +/* +* The TDH.QUOTE.INIT call renews the quoting keys. +* +* A module update must not increase the quote buffer size, or
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1555
Quoted 2 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+* quote generation may fail and break attestation. +*/
It makes sense to explain why to call this again, but I'm not sure how this comment helps.
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1558
Quoted 3 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ ret =tdx_quote_init (); +WARN_ON (ret); + }
Since the caller of this function has an return code. Please justify why not to return an error.
Quoted 20 lines
arch/x86/virt/vmx/tdx/tdx.c +2
-0+} +/*
* Mostly the same flow asinit_tdx_module_extensions(), but rejects adding
* more memory.arch/x86/virt/vmx/tdx/tdx.c +0
-0@@ -1561,7 +1580,13 @@ static int update_tdx_module_extensions(void) if (sysinfo_ext.memory_pool_required_pages) return -EFAULT;arch/x86/virt/vmx/tdx/tdx.c +7
-1-return tdx_ext_init (); + ret =tdx_ext_init (); +if (ret) +return ret; + +update_tdx_quoting_extension (); + +return 0 ;}
static__initintinit_tdx_module(void)
Re: [PATCH v2 12/17] x86/virt/tdx: Reinitialize the Quoting extension after TDX module update
Quoted 3 lines · Rick P
Quoted 2 lines · Xu Yilun
From: Peter Fang <[email protected]>
[ ... ]
Quoted 8 lines
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +7
-0+static void update_tdx_quoting_extension (void ) +{ +int ret; + +if (tdx_addon_feature0 &TDX_FEATURES0_QUOTE ) { +/* +* The TDH.QUOTE.INIT call renews the quoting keys.
Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1555
Quoted 5 lines
(quoted diff) +2
-0+ * + * A module update must not increase the quote buffer size,or
arch/x86/virt/vmx/tdx/tdx.c +2
-0+* quote generation may fail and break attestation. +*/ It makes sense to explain why to call this again, but I'm not sure how this comment helps.
OK, I can improve the "why" of this, and not just say what it does.
BTW the "must not increase the quote buffer size" comment will probably be irrelevant in the next version, since I plan to remove the static global quote buffer altogether.
Quoted 7 lines
arch/x86/virt/vmx/tdx/tdx.c static __init int init_tdx_module_extensions(void) · line 1558
Quoted 4 lines
arch/x86/virt/vmx/tdx/tdx.c +3
-0+ ret =tdx_quote_init (); +WARN_ON (ret); + }Since the caller of this function has an return code. Please justify why not to return an error.
The original thought was to warn since there is no proper handling for this kind of error, and at the same time try to keep other parts of TDX going. But I think it's probably better to just fail TDX as a whole if this goes wrong. If this returns an error then maybe something more serious is wrong here.
Quoted 1 line
PATCH v2 13/17 x86/virt/tdx: Enable Quoting extension
In reply to Xu YilunFrom: Peter Fang <[email protected]>
The Quoting extension generates TDX attestation Quotes in the TDX module, without using a discrete Quoting engine. Enable this feature by requesting it in TDH.SYS.CONFIG and TDH.SYS.UPDATE.
--- arch/x86/virt/vmx/tdx/tdx.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
arch/x86/virt/vmx/tdx/tdx.c +2 -4
@@ -1029,10 +1029,8 @@ static __init int construct_tdmrs(struct list_head *tmb_list,static __init void set_tdx_addon_features (void ) { -/* -* To add DICE-based TDX Quoting feature bit in tdx_addon_feature0 when -* kernel is ready. -*/ +if (tdx_sysinfo.features.tdx_features0 &TDX_FEATURES0_QUOTE ) + tdx_addon_feature0 |=TDX_FEATURES0_QUOTE ; } static __init int config_tdx_module(struct tdmr_info_list *tdmr_list,
Signature · 1 line
2.25.1
Re: [PATCH v2 13/17] x86/virt/tdx: Enable Quoting extension
Quoted 5 lines · Xu Yilun
From: Peter Fang <[email protected]>
The Quoting extension generates TDX attestation Quotes in the TDX module, without using a discrete Quoting engine. Enable this feature by requesting it in TDH.SYS.CONFIG and TDH.SYS.UPDATE.
PATCH v2 14/17 x86/tdx: Move and rename Quote request structure
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Move struct tdx_quote_buf to tdx.h so it can be shared by the guest driver and core TDX code, as the host will also need the Quote buffer format for in-kernel Quote generation.
Rename the struct to tdx_quote_req to better reflect its purpose, and replace "quote_buf" with "quote_req" in tdx-guest.c.
--- arch/x86/include/asm/tdx.h | 20 +++++++++++ drivers/virt/coco/tdx-guest/tdx-guest.c | 47 ++++++++----------------- 2 files changed, 34 insertions(+), 33 deletions(-)
arch/x86/include/asm/tdx.h +20 -0
@@ -66,6 +66,26 @@ struct ve_info {u32 instr_info; }; +/** +* struct tdx_quote_req - Format of Quote request message +* @version: Quote format version, filled by TD. +* @status: Status code of Quote request, filled by VMM. +* @in_len: Length of TDREPORT, filled by TD. +* @out_len: Length of Quote data, filled by VMM. +* @data: Quote data on output or TDREPORT on input. +* +* More details of Quote request message can be found in TDX +* Guest-Host Communication Interface (GHCI) for Intel TDX 1.0, +* section titled "TDG.VP.VMCALL<GetQuote>" +*/ +struct tdx_quote_req { +u64 version; +u64 status; +u32 in_len; +u32 out_len; +u8 data[]; +}; +#ifdef CONFIG_INTEL_TDX_GUEST void __init tdx_early_init (void );
drivers/virt/coco/tdx-guest/tdx-guest.c +14 -33
109 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH v2 14/17] x86/tdx: Move and rename Quote request structure
Quoted 8 lines · Xu Yilun
From: Peter Fang <[email protected]>
Move struct
tdx_quote_buftotdx.hso it can be shared by the guest driver and core TDX code, as the host will also need the Quote buffer format for in-kernel Quote generation.Rename the struct to
tdx_quote_reqto better reflect its purpose, and replace "quote_buf" with "quote_req" intdx-guest.c.
PATCH v2 15/17 KVM: TDX: Factor out userspace return path from tdx_get_quote()
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Separate the logic that returns the GetQuote TDVMCALL exit to userspace so that tdx_get_quote() can be extended to support in-kernel Quote generation.
No functional change intended.
--- arch/x86/kvm/vmx/tdx.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/tdx.c +16 -9
@@ -1524,6 +1524,20 @@ static int tdx_complete_simple(struct kvm_vcpu *vcpu)return 1 ; } +static int tdx_get_quote_user (struct kvm_vcpu *vcpu,u64 gpa,u64 size) +{ + vcpu->run->exit_reason =KVM_EXIT_TDX ; + vcpu->run->tdx.flags =0 ; + vcpu->run->tdx.nr =TDVMCALL_GET_QUOTE ; + vcpu->run->tdx.get_quote.ret =TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED ; + vcpu->run->tdx.get_quote.gpa = gpa; + vcpu->run->tdx.get_quote.size = size; + + vcpu->arch.complete_userspace_io = tdx_complete_simple; + +return 0 ; +} +static int tdx_get_quote (struct kvm_vcpu *vcpu) {struct vcpu_tdx *tdx =to_tdx (vcpu);
@@ -1536,16 +1550,9 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)return 1 ; } - vcpu->run->exit_reason =KVM_EXIT_TDX ; - vcpu->run->tdx.flags =0 ; - vcpu->run->tdx.nr =TDVMCALL_GET_QUOTE ; - vcpu->run->tdx.get_quote.ret =TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED ; - vcpu->run->tdx.get_quote.gpa = gpa & ~gfn_to_gpa (kvm_gfn_direct_bits (tdx->vcpu.kvm)); - vcpu->run->tdx.get_quote.size = size; - - vcpu->arch.complete_userspace_io = tdx_complete_simple; + gpa &= ~gfn_to_gpa (kvm_gfn_direct_bits (vcpu->kvm)); -return 0 ; +return tdx_get_quote_user (vcpu, gpa, size); }static int tdx_setup_event_notify_interrupt (struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
Re: [PATCH v2 15/17] KVM: TDX: Factor out userspace return path from tdx_get_quote()
Quoted 7 lines · Xu Yilun
From: Peter Fang <[email protected]>
Separate the logic that returns the GetQuote TDVMCALL exit to userspace so that
tdx_get_quote()can be extended to support in-kernel Quote generation.No functional change intended.
PATCH v2 16/17 KVM: TDX: Add in-kernel Quote generation
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.
Use the core TDX API for Quote generation when the Quoting extension is available. For simplicity, KVM checks its availability once per guest during initialization. KVM does not handle Quoting service disruptions or switch between the in-kernel and userspace paths.
Update the KVM API and TDX documentation to describe this new Quoting capability.
--- Documentation/arch/x86/tdx.rst | 19 ++--- Documentation/virt/kvm/api.rst | 3 + arch/x86/include/asm/tdx.h | 9 +++ arch/x86/kvm/vmx/tdx.h | 6 ++ arch/x86/kvm/vmx/tdx.c | 135 ++++++++++++++++++++++++++++++++- virt/kvm/kvm_main.c | 1 + 6 files changed, 163 insertions(+), 10 deletions(-)
Documentation/arch/x86/tdx.rst +10 -9
@@ -522,15 +522,16 @@ provided by attestation service so the TDREPORT can be verified uniquely. More details about the TDREPORT can be found in Intel TDX Module specification, section titled "TDG.MR.REPORT Leaf". -After getting the TDREPORT, the second step of the attestation process -is to send it to the Quoting Enclave (QE) to generate the Quote. TDREPORT -by design can only be verified on the local platform as the MAC key is -bound to the platform. To support remote verification of the TDREPORT, -TDX leverages Intel SGX Quoting Enclave to verify the TDREPORT locally -and convert it to a remotely verifiable Quote. Method of sending TDREPORT -to QE is implementation specific. Attestation software can choose -whatever communication channel available (i.e. vsock or TCP/IP) to -send the TDREPORT to QE and receive the Quote. +After getting the TDREPORT, the second step of the attestation process is to +convert it to a Quote. A TDREPORT by design can only be verified on the local +platform, as the MAC key is bound to the platform. A Quote makes the TDREPORT +remotely verifiable. It can be generated either through a Quoting Enclave +(QE) in userspace or through the Quoting service in kernel space. In +userspace, the Intel SGX Quoting Enclave verifies the TDREPORT locally and +converts it to a Quote. The method of sending the TDREPORT to the QE and +receiving the Quote is implementation-specific. If the TDX module supports the +Quoting service, the kernel can convert a TDREPORT to a Quote directly through +a SEAMCALL. In this case, the Quote is generated entirely by the TDX module. References ==========
Documentation/virt/kvm/api.rst +3 -0
@@ -7335,6 +7335,9 @@ inputs and outputs of the TDVMCALL. Currently the following values of queued successfully, the TDX guest can poll the status field in the shared-memory area to check whether the Quote generation is completed or not. When completed, the generated Quote is returned via the same buffer. + If the host kernel generates Quotes through the Quoting service provided by + the TDX module, KVM processes the GetQuote request and it will not appear in + userspace. * ``TDVMCALL_GET_TD_VM_CALL_INFO``: the guest has requested the support status of TDVMCALLs. The output values for the given leaf should be
arch/x86/include/asm/tdx.h +9 -0
@@ -86,6 +86,15 @@ struct tdx_quote_req {u8 data[]; }; +#define TDX_QUOTE_REQ_HDR_SIZE (offsetof(struct tdx_quote_req, data)) + +/* +* TDG.VP.VMCALL<GetQuote> Status Codes +*/ +#define TDX_QUOTE_STATUS_SUCCESS 0x0000000000000000ULL +#define TDX_QUOTE_STATUS_ERROR 0x8000000000000000ULL +#define TDX_QUOTE_STATUS_UNAVAILABLE 0x8000000000000001ULL +#ifdef CONFIG_INTEL_TDX_GUEST void __init tdx_early_init (void );
arch/x86/kvm/vmx/tdx.h +6 -0
@@ -47,6 +47,12 @@ struct kvm_tdx {* Set/unset is protected with kvm->mmu_lock. */ bool wait_for_sept_zap; + +/* +* Whether to get the quote directly in kernel, without exiting to +* userspace. +*/ +bool get_quote_in_kernel; };/* TDX module vCPU states */
arch/x86/kvm/vmx/tdx.c +134 -1
158 lines not on this page · raw message
virt/kvm/kvm_main.c +1 -0
@@ -2461,6 +2461,7 @@ bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end,return true; } +EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_range_has_memory_attributes); static __always_inline void kvm_handle_gfn_range(struct kvm *kvm,struct kvm_mmu_notifier_range *range)
Signature · 1 line
2.25.1
[email protected]
AI reviewRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] KVM incorrectly returns a hypercall error (TDVMCALL_STATUS_INVALID_OPERAND) when in-kernel quote…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM incorrectly returns a hypercall error (TDVMCALL_STATUS_INVALID_OPERAND) when in-kernel quote generation gracefully fails.
- [High] KVM fails to write the required buffer size back to the guest when the provided GetQuote buffer is too small. --
commit 482cde6e1c4a3f88b17ca20d55ddbc439d933981
Author: Peter Fang <[email protected]>
KVM: TDX: Add in-kernel Quote generation
Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +0
-0
[ ... ]
Quoted 3 lines
arch/x86/kvm/vmx/tdx.c +3
-0+static u64 get_quote_kernel (struct kvm_vcpu *vcpu,struct tdx_quote_req *req, +gpa_t req_gpa,size_t total_len) +{
[ ... ]
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1589
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9
-0+/* Caller owns the requested quote */ + void *quote_data __free(kvfree) = +tdx_quote_generate (td, req->data, req->in_len, &req->out_len); + +if (!quote_data) +return TDX_QUOTE_STATUS_UNAVAILABLE ; + +if ((size_t )req->out_len +TDX_QUOTE_REQ_HDR_SIZE > total_len) +return TDX_QUOTE_STATUS_ERROR ;
[Severity: High]
If the guest-provided buffer is too small, does this early return skip writing the required size back to the guest?
When tdx_quote_generate() fails due to an insufficient buffer, it populates req->out_len with the required size. By returning TDX_QUOTE_STATUS_ERROR immediately here, it seems we bypass the logic that writes req->out_len back to the guest.
Will this break a guest's ability to dynamically probe the required quote size and retry if its initial allocation is too small?
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+ +if (!write_quote_to_guest (vcpu, quote_data, req->out_len, req_gpa)) +return TDX_QUOTE_STATUS_ERROR ; + +return TDX_QUOTE_STATUS_SUCCESS ; +}
[ ... ]
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+static int tdx_get_quote_kernel (struct kvm_vcpu *vcpu,u64 gpa,u64 size) +{ +void *first_page =NULL ; +u64 err, qerr; + + err =tdx_get_quote_check_args (vcpu, gpa, size); +if (err !=TDVMCALL_STATUS_SUCCESS ) + goto out; + + err =TDVMCALL_STATUS_INVALID_OPERAND ;
[ ... ]
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1652
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+ qerr =get_quote_kernel (vcpu, first_page, (gpa_t )gpa, size); + +if (write_quote_status_to_guest (vcpu, qerr, (gpa_t )gpa) && + qerr ==TDX_QUOTE_STATUS_SUCCESS ) + err =TDVMCALL_STATUS_SUCCESS ;
[Severity: High]
Should the hypercall return SUCCESS even if the quote generation itself failed gracefully?
If get_quote_kernel() encounters a quote service error (like TDX_QUOTE_STATUS_UNAVAILABLE), this condition prevents err from being updated to TDVMCALL_STATUS_SUCCESS. As a result, the hypercall returns the initialized value TDVMCALL_STATUS_INVALID_OPERAND in R10.
Since the hypercall arguments were structurally valid and we successfully wrote the error status into the shared memory buffer, shouldn't the hypercall return SUCCESS to let the guest parse the actual quote status?
Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +7
-0+ +out: +kfree (first_page); +tdvmcall_set_return_code (vcpu, err); + +return 1 ; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 19 lines · On Thu, Jun 18, 2026 at 09:03:09AM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] KVM incorrectly returns a hypercall error (TDVMCALL_STATUS_INVALID_OPERAND) when in-kernel quote generation gracefully fails.
- [High] KVM fails to write the required buffer size back to the guest when the provided GetQuote buffer is too small. --arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1589
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9
-0+/* Caller owns the requested quote */ + void *quote_data __free(kvfree) = +tdx_quote_generate (td, req->data, req->in_len, &req->out_len); + +if (!quote_data) +return TDX_QUOTE_STATUS_UNAVAILABLE ; + +if ((size_t )req->out_len +TDX_QUOTE_REQ_HDR_SIZE > total_len) +return TDX_QUOTE_STATUS_ERROR ;[Severity: High]
If the guest-provided buffer is too small, does this early return skip writing the required size back to the guest?
Quoted 7 lines
When
tdx_quote_generate()fails due to an insufficient buffer, it populatesreq->out_lenwith the required size. By returningTDX_QUOTE_STATUS_ERRORimmediately here, it seems we bypass the logic that writesreq->out_lenback to the guest.Will this break a guest's ability to dynamically probe the required quote size and retry if its initial allocation is too small?
Returning TDX_QUOTE_STATUS_ERROR immediately should be ok. The guest is expected to get the required size from a TD metadata field, not from the returned out_len here.
Quoted 20 lines
Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +6
-0+ +if (!write_quote_to_guest (vcpu, quote_data, req->out_len, req_gpa)) +return TDX_QUOTE_STATUS_ERROR ; + +return TDX_QUOTE_STATUS_SUCCESS ; +}[ ... ]
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1652
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+ qerr =get_quote_kernel (vcpu, first_page, (gpa_t )gpa, size); + +if (write_quote_status_to_guest (vcpu, qerr, (gpa_t )gpa) && + qerr ==TDX_QUOTE_STATUS_SUCCESS ) + err =TDVMCALL_STATUS_SUCCESS ;[Severity: High]
Should the hypercall return SUCCESS even if the quote generation itself failed gracefully?
Quoted 8 lines
If
get_quote_kernel()encounters a quote service error (likeTDX_QUOTE_STATUS_UNAVAILABLE), this condition prevents err from being updated toTDVMCALL_STATUS_SUCCESS. As a result, the hypercall returns the initialized valueTDVMCALL_STATUS_INVALID_OPERANDin R10.Since the hypercall arguments were structurally valid and we successfully wrote the error status into the shared memory buffer, shouldn't the hypercall return SUCCESS to let the guest parse the actual quote status?
Good point. The hypercall should return TDVMCALL_STATUS_SUCCESS on TDX_QUOTE_STATUS_UNAVAILABLE. I'll fix this in the next revision.
Quoted 4 lines
Signature · 2 lines
-- Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=16
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
In reply to Xu YilunQuoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.
Why?
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 8 lines · Sean Christopherson
Quoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.
Why?
Hi Sean,
This is mainly to avoid a round trip to userspace for the GetQuote flow.
New TDX modules can now get a Quote directly via an "extension SEAMCALL" instead of exiting to userspace and using an SGX enclave. Exiting to userspace for GetQuote no longer seems worth the overhead/complexity.
The first half of the series enables extension SEAMCALLs. They implement simple APIs for higher-order security protocols that would otherwise need to be broken into smaller routines. For Quoting, this allows KVM to get a Quote directly through TDH.QUOTE.GET. The TDX module needs only the input data from TDG.VP.VMCALL<GetQuote> for that call.
Thanks,
Peter
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 7 lines · Peter Fang
Quoted 6 lines · Sean Christopherson
Quoted 4 lines · Xu Yilun
From: Peter Fang <[email protected]>
Provide an in-kernel path for Quote generation when handling TDG.VP.VMCALL<GetQuote>, without requiring an exit to userspace.
Quoted 6 lines
Quoted 2 lines
Why?
Hi Sean,
This is mainly to avoid a round trip to userspace for the GetQuote flow.
Again, why?
Quoted 3 lines
New TDX modules can now get a Quote directly via an "extension SEAMCALL" instead of exiting to userspace and using an SGX enclave. Exiting to userspace for GetQuote no longer seems worth the overhead/complexity.
I dunno, from a kernel perspective, this is more complexity, not less:
Documentation/arch/x86/tdx.rst | 19 ++--- Documentation/virt/kvm/api.rst | 3 + arch/x86/include/asm/tdx.h | 9 +++ arch/x86/kvm/vmx/tdx.h | 6 ++ arch/x86/kvm/vmx/tdx.c | 135 ++++++++++++++++++++++++++++++++- virt/kvm/kvm_main.c | 1 + 6 files changed, 163 insertions(+), 10 deletions(-)
Quoted 5 lines
The first half of the series enables extension SEAMCALLs. They implement simple APIs for higher-order security protocols that would otherwise need to be broken into smaller routines. For Quoting, this allows KVM to get a Quote directly through TDH.QUOTE.GET. The TDX module needs only the input data from TDG.VP.VMCALL<GetQuote> for that call.
Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.
If that's all there is too this, then why is KVM involved? I.e. why doesn't the TDX Module provide the quote directly to the guest?
Edgecombe, Rick P
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 6 lines · Sean Christopherson
Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.
If that's all there is too this, then why is KVM involved? I.e. why doesn't the TDX Module provide the quote directly to the guest?
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting, so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.
We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.
The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 17 lines · Rick P
Quoted 6 lines · Sean Christopherson
Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.
If that's all there is too this, then why is KVM involved? I.e. why doesn't the TDX Module provide the quote directly to the guest?
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting, so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.
Quoted 8 lines
We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.
The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.
tl;dr: FWIW, host, if only because this is one of many blob transports across multiple archs that need host coordination.
While TDG calls to do the same helps a TDX problem it does not generally reduce the Linux problem of multiple archs having multiple blob protocols to shuffle data from host to guest in shared memory.
The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI attestation collateral) is shift the burden from arch/*/kvm/ to drivers/. There might be a later opportunity to consolidate some of those drivers' internals on a cross-arch generic transport (AF_VSOCK), but there remains a requirement to talk to the host.
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 9 lines · Dan Williams (nvidia)
Quoted 8 lines · Rick P
Quoted 6 lines · Sean Christopherson
Answering my own question (though probably poorly), IIUC the answer is that DICE-based quoting is done through the TDX Module, whereas existing quoting is done through an SGX enclave and so was routed through userspace.
If that's all there is too this, then why is KVM involved? I.e. why doesn't the TDX Module provide the quote directly to the guest?
Quoted 6 lines
Quoted 6 lines
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,
What is this magical resource?
Quoted 2 lines
Quoted 2 lines
so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.
At the risk of unintentionally causing effecitvely DoS by introducing a system- wide lock.
Quoted 3 lines
Quoted 3 lines
We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.
Simpler for what?
Quoted 11 lines
Quoted 4 lines
The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.
tl;dr: FWIW, host, if only because this is one of many blob transports across multiple archs that need host coordination.
While TDG calls to do the same helps a TDX problem it does not generally reduce the Linux problem of multiple archs having multiple blob protocols to shuffle data from host to guest in shared memory.
Why is that KVM's problem?
Quoted 5 lines
The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI attestation collateral) is shift the burden from
arch/*/kvm/todrivers/. There might be a later opportunity to consolidate some of those drivers' internals on a cross-arch generic transport (AF_VSOCK), but there remains a requirement to talk to the host.
Why?
Bluntly, this series is unreviewable. I've speed read through most of the patches, and I have *zero* clue as to *why* any of the design decisions were made. E.g. why is there a common buffer that's apparently shared by all CPUs and VMs, only for the API to immediately allocate per-quote memory and copy out the buffer?
The changelogs are all heavy on the "what", but explanations of why any of this is being done, and in the exact way it's being done, are non-existent.
So, back up to the beginning and explain why I should spend any time deciphering this series, because as-is I don't have the interest or the time.
Edgecombe, Rick P
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,
What is this magical resource?
It's a HW crypto thing. I'll let Peter explain more.
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.
At the risk of unintentionally causing effecitvely DoS by introducing a system-wide lock.
The DoS that people are interested in trying to prevent is a TD getting starved of quotes forever. So the lock is supposed to give some eventual fairness. Eventually the guest gets its quote even if it has to wait.
If instead contention throws a BUSY code to the guest, the guest spins trying. That wouldn't have the guarantees.
So I think the two options are: have some lock in the TDX module, or have a lock of some sort in the host. I think actually a waiting lock in the TDX module is possible. But I think there are tradeoffs besides where the extra code is. If it's in Linux we get a lot more control, lockdep, etc.
BTW the "historical reasons" part I mentioned involves a past effort to create configurable host controlled policies around managing these resources. So part of the simplification is doing a simple eventually fair global lock instead of something more complicated.
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
We should've explained this more, but TBH this solution is *way* simpler than the initial one that never saw the light of day. So this extra host work seemed quite small compared to what we have been staring at and we kinda overlooked it.
Simpler for what?
Linux of course.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
The other relevant tidbit is that the TDX module folks have some problems to solve before they can support TDG calls to TDX module extensions. I think we can get them to though. The question is probably really: do we want the guest trying/selecting multiple interfaces, or the host.
Quoted 8 lines
Quoted 6 lines
tl;dr: FWIW, host, if only because this is one of many blob transports across multiple archs that need host coordination.
While TDG calls to do the same helps a TDX problem it does not generally reduce the Linux problem of multiple archs having multiple blob protocols to shuffle data from host to guest in shared memory.
Why is that KVM's problem?
I'm slightly lost on this too, so will leave it to Dan.
Quoted 7 lines
Quoted 6 lines
The current direction for other archs (CCA, SEV-SNP) and blobs (e.g. PCI attestation collateral) is shift the burden from
arch/*/kvm/todrivers/. There might be a later opportunity to consolidate some of those drivers' internals on a cross-arch generic transport (AF_VSOCK), but there remains a requirement to talk to the host.
Quoted 7 lines
Why?
Bluntly, this series is unreviewable. I've speed read through most of the patches, and I have *zero* clue as to *why* any of the design decisions were made. E.g. why is there a common buffer that's apparently shared by all CPUs and VMs, only for the API to immediately allocate per-quote memory and copy out the buffer?
We can keep discussing it, but...
Quoted 6 lines
The changelogs are all heavy on the "what", but explanations of why any of this is being done, and in the exact way it's being done, are non-existent.
So, back up to the beginning and explain why I should spend any time deciphering this series, because as-is I don't have the interest or the time.
...it sounds like it would be better if we tried again with a proper explanation, rather than try to back out of where the current presentation left you?
This should be a pretty simple feature. But there is a higher level theme of what is responsible for locks and scheduling in the overall TDX solution. This spills into a lot of future stuff. We can try to find the right level.
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 11 lines · Rick P
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,
What is this magical resource?
It's a HW crypto thing. I'll let Peter explain more.
It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically provides a number of boot and security services through dedicated HW. Attestation is one of them. This is part of our plan to develop new attestation schemes without using SGX.
There was a lot of internal debate about how this HW should be managed. We decided to let the TDX module handle it and sort of hide it from the host kernel and only expose it as SEAMCALLs. It however impacted how the quoting SEAMCALLs look, so apologies for not calling out these factors.
Quoted 21 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.
At the risk of unintentionally causing effecitvely DoS by introducing a system-wide lock.
The DoS that people are interested in trying to prevent is a TD getting starved of quotes forever. So the lock is supposed to give some eventual fairness. Eventually the guest gets its quote even if it has to wait.
If instead contention throws a BUSY code to the guest, the guest spins trying. That wouldn't have the guarantees.
So I think the two options are: have some lock in the TDX module, or have a lock of some sort in the host. I think actually a waiting lock in the TDX module is possible. But I think there are tradeoffs besides where the extra code is. If it's in Linux we get a lot more control, lockdep, etc.
Quoted 4 lines
BTW the "historical reasons" part I mentioned involves a past effort to create configurable host controlled policies around managing these resources. So part of the simplification is doing a simple eventually fair global lock instead of something more complicated.
Yeah my bad for not including a lot more background. There were several design aspects that influenced this implementation. I'll do a better job of explaining them in the next version, but basically:
1. Performance requirements: For this first implementation, the priority is to not starve a legit guest from getting a quote, even when there's a malicious guest DoSing the host. A mutex is sort of a "fair enough queue" for this problem. A guest's authenticity can only be verified after the quote is generated.
2. HW limitation: The TDX module implements different attestation schemes, some of which use the S3M during TDH.QUOTE.GET. This limits the amount of concurrency this SEAMCALL supports and is also part of why we chose a mutex for TDH.QUOTE.GET. (The shared buffer in core TDX doesn't seem great, so I will improve it.)
3. Host policies: Preserving the GetQuote GHCI also allows the host to participate in the quoting process, e.g. add host-controlled policies or fetch host-side quote information. It's a bit similar to SEV-SNP's KVM_EXIT_SNP_REQ_CERTS [1]. This series does not implement host policies though.
Ultimately, there are ways to push more of this into the module, and the current design is kind of a tradeoff between the complexities of the work needed in the guest, the host, and the TDX module. Like Rick said, there were past versions that required way more work in the host and the guest, such as managing the S3M, new GHCI calls, etc.
[1] Commit fa9893fadbc2 ("KVM: Introduce KVM_EXIT_SNP_REQ_CERTS for SNP certificate-fetching"
Quoted 1 line
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 54 lines · Peter Fang
Quoted 11 lines · Rick P
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,
What is this magical resource?
It's a HW crypto thing. I'll let Peter explain more.
It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically provides a number of boot and security services through dedicated HW. Attestation is one of them. This is part of our plan to develop new attestation schemes without using SGX.
There was a lot of internal debate about how this HW should be managed. We decided to let the TDX module handle it and sort of hide it from the host kernel and only expose it as SEAMCALLs. It however impacted how the quoting SEAMCALLs look, so apologies for not calling out these factors.
Quoted 21 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
so we want to do these operations one at a time. Having a mutex on the host is the KISS way of accomplishing some level of fairness for DOS prevention.
At the risk of unintentionally causing effecitvely DoS by introducing a system-wide lock.
The DoS that people are interested in trying to prevent is a TD getting starved of quotes forever. So the lock is supposed to give some eventual fairness. Eventually the guest gets its quote even if it has to wait.
If instead contention throws a BUSY code to the guest, the guest spins trying. That wouldn't have the guarantees.
So I think the two options are: have some lock in the TDX module, or have a lock of some sort in the host. I think actually a waiting lock in the TDX module is possible. But I think there are tradeoffs besides where the extra code is. If it's in Linux we get a lot more control, lockdep, etc.
Quoted 4 lines
BTW the "historical reasons" part I mentioned involves a past effort to create configurable host controlled policies around managing these resources. So part of the simplification is doing a simple eventually fair global lock instead of something more complicated.
Yeah my bad for not including a lot more background. There were several design aspects that influenced this implementation. I'll do a better job of explaining them in the next version, but basically:
Quoted 8 lines
1. Performance requirements: For this first implementation, the priority is to not starve a legit guest from getting a quote, even when there's a malicious guest DoSing the host. A mutex is sort of a "fair enough queue" for this problem. A guest's authenticity can only be verified after the quote is generated.
2. HW limitation: The TDX module implements different attestation schemes, some of which use the S3M during TDH.QUOTE.GET. This
What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core? How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at? What else uses the S3M? Do we have to worry about contending with non-TDX usage?
Quoted 21 lines
limits the amount of concurrency this SEAMCALL supports and is also part of why we chose a mutex for TDH.QUOTE.GET. (The shared buffer in core TDX doesn't seem great, so I will improve it.)
3. Host policies: Preserving the GetQuote GHCI also allows the host to participate in the quoting process, e.g. add host-controlled policies or fetch host-side quote information. It's a bit similar to SEV-SNP's
KVM_EXIT_SNP_REQ_CERTS[1]. This series does not implement host policies though.Ultimately, there are ways to push more of this into the module, and the current design is kind of a tradeoff between the complexities of the work needed in the guest, the host, and the TDX module. Like Rick said, there were past versions that required way more work in the host and the guest, such as managing the S3M, new GHCI calls, etc.
[1] Commit
fa9893fadbc2("KVM: IntroduceKVM_EXIT_SNP_REQ_CERTSfor SNP certificate-fetching"Quoted 1 line
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 2 lines · Sean Christopherson
What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?
I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.
Quoted 2 lines
How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?
As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's no ioremap() because the I/O is hidden in the TDX module.
The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.
For TDX, though, the craziness is mostly hidden in the TDX module.
Quoted 2 lines
What else uses the S3M? Do we have to worry about contending with non-TDX usage?
There _are_ different users of S3M. But each of them should get their own I/O address and S3M firmware has to handle talking to those different users at the same time. The TDX I/O window is owned exclusively by the TDX module.
So, while S3M has and long and growing list of jobs, the random software (like the host kernel) poking at one I/O window doesn't have to know about the other piece of software (the TDX module) poking at another.
I'm sure I got a detail or two wrong in there, so folks that know this
better: please correct me. But I think that's a halfway-decent 10,000ft view.
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 31 lines · Dave Hansen
Quoted 2 lines · Sean Christopherson
What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?
I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.
Quoted 2 lines
How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?
As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with
ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's noioremap()because the I/O is hidden in the TDX module.The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.
For TDX, though, the craziness is mostly hidden in the TDX module.
Quoted 2 lines
What else uses the S3M? Do we have to worry about contending with non-TDX usage?
There _are_ different users of S3M. But each of them should get their own I/O address and S3M firmware has to handle talking to those different users at the same time. The TDX I/O window is owned exclusively by the TDX module.
Quoted 7 lines
So, while S3M has and long and growing list of jobs, the random software (like the host kernel) poking at one I/O window doesn't have to know about the other piece of software (the TDX module) poking at another.
I'm sure I got a detail or two wrong in there, so folks that know this
better: please correct me. But I think that's a halfway-decent 10,000ft view.
Thank you, very helpful!
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
In reply to Dave HansenQuoted 17 lines · Dave Hansen
Quoted 2 lines · Sean Christopherson
What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?
I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.
Quoted 2 lines
How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?
As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with
ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's noioremap()because the I/O is hidden in the TDX module.
Quoted 6 lines
The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.
For TDX, though, the craziness is mostly hidden in the TDX module.
I'll just add that this design decision comes from the fact that we're looking at different kinds of future attestation stuff. Some of it is quite complex (like post-quantum crypto). Some of it may even require more complicated interactions with the S3M. Hiding these details allows all this crypto stuff to look like just another TDX module release. The host kernel shouldn't be burdened with attestation intricacies, and the TDX module should just make sure the new module "works".
Quoted 8 lines
Quoted 3 lines
What else uses the S3M? Do we have to worry about contending with non-TDX usage?
There _are_ different users of S3M. But each of them should get their own I/O address and S3M firmware has to handle talking to those different users at the same time. The TDX I/O window is owned exclusively by the TDX module.
Quoted 7 lines
So, while S3M has and long and growing list of jobs, the random software (like the host kernel) poking at one I/O window doesn't have to know about the other piece of software (the TDX module) poking at another.
I'm sure I got a detail or two wrong in there, so folks that know this
better: please correct me. But I think that's a halfway-decent 10,000ft view.
This is pretty spot on based on my understanding. Thanks Dave :)
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 23 lines · Peter Fang
Quoted 22 lines · Dave Hansen
Quoted 2 lines · Sean Christopherson
What is "the S3M" though? Is it a separate chip a la AMD's PSP/ASP? Is it a per-package thing? Per-core?
I'll give you my rough software guy mental model of what it is: Each package has its own S3M. They are microcontrollers which are discrete from the CPU cores. Each S3M gets some CPU physical address space routed over to it.
Quoted 2 lines
How is it accessed, and what are the "rules" for for those accesses? What types of latencies are we looking at?
As far as I know, the latency for one round trip to/from S3M is on the order of a "real" device. It has a physical address and when the OS wants to talk to it, those addresses are mapped with
ioremap(). It's similar to any modern I/O device control plane. Note, though, that for TDX, there's noioremap()because the I/O is hidden in the TDX module.The real overhead comes because the I/O window is essentially 4 bytes wide (IIRC) and all the data that comes in and out of it has to be squeezed through that window. It reminds me of a UART, but with a slightly more arcane interface.
Quoted 7 lines
Quoted 1 line
For TDX, though, the craziness is mostly hidden in the TDX module.
I'll just add that this design decision comes from the fact that we're looking at different kinds of future attestation stuff. Some of it is quite complex (like post-quantum crypto). Some of it may even require more complicated interactions with the S3M. Hiding these details allows all this crypto stuff to look like just another TDX module release.
IMO, that's not a good thing. Burying more code and more complexity in what is supposed to be a minimal TCB doesn't seem like a "win". I can certainly appreciate that interacting with the S3M microcontroller isn't something we want in KVM, but that's why drivers/ exists.
Quoted 1 line
The host kernel shouldn't be burdened with attestation intricacies,
Oh, but the kernel has to be, at least to the point where the kernel has enough information to guarantee backwards compatibility, enumerate choices, etc. If you're saying that the TDX module will always generate reports that are compatible across TDX Module updates, then we probably don't have anything to worry about, but it seems unlikely everything can Just Work while also updating the core crypto algorithms and whatnot.
Quoted 1 line
and the TDX module should just make sure the new module "works".
Edgecombe, Rick P
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 10 lines · Sean Christopherson
Quoted 5 lines
I'll just add that this design decision comes from the fact that we're looking at different kinds of future attestation stuff. Some of it is quite complex (like post-quantum crypto). Some of it may even require more complicated interactions with the S3M. Hiding these details allows all this crypto stuff to look like just another TDX module release.
IMO, that's not a good thing. Burying more code and more complexity in what is supposed to be a minimal TCB doesn't seem like a "win". I can certainly appreciate that interacting with the S3M microcontroller isn't something we want in KVM, but that's why
drivers/exists.
This is an interesting take and makes me think even more that we need to be socializing the TDX arch stuff a lot more.
Quoted 9 lines
Quoted 2 lines
The host kernel shouldn't be burdened with attestation intricacies,
Oh, but the kernel has to be, at least to the point where the kernel has enough information to guarantee backwards compatibility, enumerate choices, etc. If you're saying that the TDX module will always generate reports that are compatible across TDX Module updates, then we probably don't have anything to worry about, but it seems unlikely everything can Just Work while also updating the core crypto algorithms and whatnot.
There are absolutely some ABI sensitivity here that need to be considered, and unfortunately the analysis from the internal debates did not make into the public presentation of the feature. In fact the whole DICE part of this series is missing a lot of needed context.
I think I mentioned that we want to come to talk about migration in PUCK, it might help to bring this one in too. Despite being small, it actually took quite a bit for us to wrap our heads around it. Would you want to try that? Or we could fill out the logs and coverletter in v3. Or we could just try to slowly walk through it all here. I was thinking a call may shorten the loops.
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
In reply to Peter FangQuoted 9 lines · Peter Fang
Quoted 8 lines · Rick P
Quoted 7 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
What is this magical resource?
It's a HW crypto thing. I'll let Peter explain more.
It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically
Are you referring to this: https://www.scribd.com/document/970439243/S3M-Intel-White-Paper ?
<snip>
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 14 lines · Nikolay Borisov
Quoted 11 lines · Peter Fang
Quoted 10 lines · Rick P
Quoted 8 lines · Sean Christopherson
Quoted 6 lines
Quoted 6 lines
That is a good question. The answer is partly historical reasons, but I think the pros/cons don’t really move the needle too much.
The main benefit of doing it with the host in the loop is that the guest side TDVMCALL quoting interface can stay the same. There is also a wrinkle in that there is a limited HW resource involved in the quoting,
What is this magical resource?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
It's a HW crypto thing. I'll let Peter explain more.
It's called S3M (Secured Startup Services). There was once a public document about it, but it was removed for some reason. It basically
Are you referring to this:
https://www.scribd.com/document/970439243/S3M-Intel-White-Paper ?
Yep. The internet never forgets :)
Quoted 2 lines
<snip>
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
In reply to Xu YilunQuoted 6 lines · Xu Yilun
void
__inittdx_early_init(void);arch/x86/kvm/vmx/tdx.h +0
-0
arch/x86/kvm/vmx/tdx.h struct kvm_tdx { · line 55
Quoted 9 lines
arch/x86/kvm/vmx/tdx.h +6
-0@@ -47,6 +47,12 @@ struct kvm_tdx {* Set/unset is protected with kvm->mmu_lock. */ bool wait_for_sept_zap; + +/* +* Whether to get the quote directly in kernel, without exiting to +* userspace. +*/ +bool get_quote_in_kernel;
Why is this cached per-VM? IIUC, tdx_quote_enabled() is constant after initial TDX bringup.
And is keying off tdx_quote_enabled() and only tdx_quote_enabled() backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?
Quoted 11 lines
arch/x86/kvm/vmx/tdx.h +0
-0};/* TDX module vCPU states */ arch/x86/kvm/vmx/tdx.c +0
-0@@ -1538,11 +1538,133 @@ static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size)return 0 ; }
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1550
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+static bool write_quote_status_to_guest (struct kvm_vcpu *vcpu,u64 status, +gpa_t gpa) +{ +if (kvm_vcpu_write_guest (vcpu, + gpa + offsetof(struct tdx_quote_req , status), + &status,sizeof (status))) +return false; + +return true; +}
This is a completely useless wrapper. Actually, it's worse than useless, as it inverts the return value from the widely used kvm_vcpu_write_guest() API.
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1556
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+ +static bool write_quote_to_guest (struct kvm_vcpu *vcpu,void *quote_data, +u32 quote_len,gpa_t gpa) +{ + if (kvm_vcpu_write_guest(vcpu, + gpa + TDX_QUOTE_REQ_HDR_SIZE,
This is unnecessarily confusing. I had to look at the definition of TDX_QUOTE_REQ_HDR_SIZE to understand that this just writing to tdx_quote_req.data. I might not care if this was the only write, but then the very next statement writes to a field that isn't obviously in the header.
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1566
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ quote_data, quote_len)) +return false; + +if (kvm_vcpu_write_guest (vcpu, + gpa + offsetof(struct tdx_quote_req , out_len), + "e_len,sizeof (quote_len))) +return false; + +return true; +}
So is this. There's literally one caller.
Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +7
-0+ +static u64 get_quote_kernel (struct kvm_vcpu *vcpu,struct tdx_quote_req *req, +gpa_t req_gpa,size_t total_len) +{ +struct tdx_td *td = &to_kvm_tdx (vcpu->kvm)->td; + +/* Only support version 1 as defined in the GHCI spec */
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1583
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+if (req->version !=1 ) +return TDX_QUOTE_STATUS_ERROR ; + +/* Header + input data must fit in the page read from guest memory */ +if ((size_t )req->in_len +TDX_QUOTE_REQ_HDR_SIZE >PAGE_SIZE ) +return TDX_QUOTE_STATUS_ERROR ; + +/* Caller owns the requested quote */ + void *quote_data __free(kvfree) = +tdx_quote_generate (td, req->data, req->in_len, &req->out_len);
Am I the only person that finds the __free() stuff to be difficult to follow? I'm sure there are flows where it's a big net positive, but IMO this isn't one. If the *unnecesary* layers of single-use helpers are stripped away, I suspect we'll naturally end up with a goto sequence that is more obvious that using __free().
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1588
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +5
-0+ +if (!quote_data) +return TDX_QUOTE_STATUS_UNAVAILABLE ; + +if ((size_t )req->out_len +TDX_QUOTE_REQ_HDR_SIZE > total_len)
Can we really not pre-check the size?
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1597
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +9
-0+return TDX_QUOTE_STATUS_ERROR ; + +if (!write_quote_to_guest (vcpu, quote_data, req->out_len, req_gpa)) +return TDX_QUOTE_STATUS_ERROR ; + +return TDX_QUOTE_STATUS_SUCCESS ; +} + +static u64 tdx_get_quote_check_args (struct kvm_vcpu *vcpu,u64 gpa,u64 size)
Adding single use helpers does not make the code easier to read.
Quoted 12 lines
arch/x86/kvm/vmx/tdx.c +12
-0+{ +gfn_t gfn_start, gfn_end; +u64 end; + +if (!size) +return TDVMCALL_STATUS_INVALID_OPERAND ; + +if (!PAGE_ALIGNED (gpa) || !PAGE_ALIGNED (size)) +return TDVMCALL_STATUS_ALIGN_ERROR ; + +if (check_add_overflow (gpa, size, &end)) +return TDVMCALL_STATUS_INVALID_OPERAND ;
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ + gfn_start =gpa_to_gfn (gpa); + gfn_end =gpa_to_gfn (end); + +/* +* Reject if the guest didn't explicitly convert its quote pages to +* shared. +*/ +if (!kvm_range_has_memory_attributes (vcpu->kvm, gfn_start, gfn_end, +KVM_MEMORY_ATTRIBUTE_PRIVATE ,0 ))
TOCTOU?
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +4
-0+return TDVMCALL_STATUS_INVALID_OPERAND ; + +return TDVMCALL_STATUS_SUCCESS ; +}
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 11 lines · Sean Christopherson
Quoted 10 lines · Xu Yilun
void
__inittdx_early_init(void);arch/x86/kvm/vmx/tdx.h +1
-0@@ -47,6 +47,12 @@ struct kvm_tdx {* Set/unset is protected with kvm->mmu_lock. */ bool wait_for_sept_zap; +
Quoted 8 lines
arch/x86/kvm/vmx/tdx.h struct kvm_tdx { · line 55
Quoted 5 lines
arch/x86/kvm/vmx/tdx.h +5
-0+/* +* Whether to get the quote directly in kernel, without exiting to +* userspace. +*/ +bool get_quote_in_kernel;Why is this cached per-VM? IIUC,
tdx_quote_enabled()is constant after initial TDX bringup.
Ah you're right. It shouldn't change. I think I did this because there was an older version where quoting was not enabled during core TDX bringup. So this was an attempt to make sure a guest doesn't see tdx_quote_enabled() flip. It shouldn't be here. Sorry about that.
Quoted 4 lines
And is keying off
tdx_quote_enabled()and onlytdx_quote_enabled()backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?
Yeah this needs explaining:
The guest-observable difference between the two paths is that it sees quotes in different formats (the TDX module uses newer formats of course). And an old guest must be able to continue using its TDVMCALL<GetQuote> interface for quotes. The guest sends the quote to an external verifier to authenticate itself. The verifier (part of the cloud infrastructure) must know how to parse this format. So in other words, it's the verifier that must maintain backward compatibility if it wants to continue accepting old quotes. And it needs to be updated to support newer formats as well.
Quoted 13 lines
Quoted 13 lines
arch/x86/kvm/vmx/tdx.h +0
-0};/* TDX module vCPU states */ arch/x86/kvm/vmx/tdx.c +0
-0@@ -1538,11 +1538,133 @@ static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size)return 0 ; }
Quoted 13 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1550
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +10
-0+static bool write_quote_status_to_guest (struct kvm_vcpu *vcpu,u64 status, +gpa_t gpa) +{ +if (kvm_vcpu_write_guest (vcpu, + gpa + offsetof(struct tdx_quote_req , status), + &status,sizeof (status))) +return false; + +return true; +}This is a completely useless wrapper. Actually, it's worse than useless, as it inverts the return value from the widely used
kvm_vcpu_write_guest()API.
I'll remove this. I got feedback that tdx_get_quote_kernel() was full of kvm_vcpu_write_guest()s and was hard to read, so I tried to break it apart. Looks like it didn't help.
Quoted 12 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1556
Quoted 7 lines
arch/x86/kvm/vmx/tdx.c +6
-0+ +static bool write_quote_to_guest (struct kvm_vcpu *vcpu,void *quote_data, +u32 quote_len,gpa_t gpa) +{ + if (kvm_vcpu_write_guest(vcpu, + gpa + TDX_QUOTE_REQ_HDR_SIZE,This is unnecessarily confusing. I had to look at the definition of
TDX_QUOTE_REQ_HDR_SIZEto understand that this just writing to tdx_quote_req.data. I might not care if this was the only write, but then the very next statement writes to a field that isn't obviously in the header.
I'll fix this. I can just spell out offsetof() to make this consistent.
Quoted 13 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1566
Quoted 11 lines
arch/x86/kvm/vmx/tdx.c +10
-0+ quote_data, quote_len)) +return false; + +if (kvm_vcpu_write_guest (vcpu, + gpa + offsetof(struct tdx_quote_req , out_len), + "e_len,sizeof (quote_len))) +return false; + +return true; +}So is this. There's literally one caller.
Yeah breaking the kvm_vcpu_write_guest()s apart didn't help...
Quoted 15 lines
Quoted 15 lines
arch/x86/kvm/vmx/tdx.c +14
-0+ +static u64 get_quote_kernel (struct kvm_vcpu *vcpu,struct tdx_quote_req *req, +gpa_t req_gpa,size_t total_len) +{ +struct tdx_td *td = &to_kvm_tdx (vcpu->kvm)->td; + +/* Only support version 1 as defined in the GHCI spec */ +if (req->version !=1 ) +return TDX_QUOTE_STATUS_ERROR ; + +/* Header + input data must fit in the page read from guest memory */ +if ((size_t )req->in_len +TDX_QUOTE_REQ_HDR_SIZE >PAGE_SIZE ) +return TDX_QUOTE_STATUS_ERROR ; +
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1583
Quoted 3 lines
arch/x86/kvm/vmx/tdx.c +3
-0+/* Caller owns the requested quote */ + void *quote_data __free(kvfree) = +tdx_quote_generate (td, req->data, req->in_len, &req->out_len);Am I the only person that finds the
__free()stuff to be difficult to follow? I'm sure there are flows where it's a big net positive, but IMO this isn't one. If the *unnecesary* layers of single-use helpers are stripped away, I suspect we'll naturally end up with a goto sequence that is more obvious that using__free().
The main concern I had was that the guest could give random message lengths and mess with the host's memory allocator. Core TDX has a better idea because it knows the maximum possible quote size through a metadata field, so I just let the API return the final quote from the TDX module. It's a bit annoying that the caller now has to free it.
Several folks suggested letting KVM allocate and free the buffer, so I think I'll do that. It should make this easier to read, and this __free() can be replaced with a much simpler pattern.
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1588
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +5
-0+ +if (!quote_data) +return TDX_QUOTE_STATUS_UNAVAILABLE ; + +if ((size_t )req->out_len +TDX_QUOTE_REQ_HDR_SIZE > total_len)Can we really not pre-check the size?
Yeah like the response above, KVM doesn't know what a "sane" total_len is here. out_len comes from the TDX module so I used it to check against total_len. If we let KVM allocate+free the quote buffer, then I'll also let KVM get the maximum quote size from core TDX. I was trying to save some code in KVM but looks like it's not worth it.
Quoted 12 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1597
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +9
-0+return TDX_QUOTE_STATUS_ERROR ; + +if (!write_quote_to_guest (vcpu, quote_data, req->out_len, req_gpa)) +return TDX_QUOTE_STATUS_ERROR ; + +return TDX_QUOTE_STATUS_SUCCESS ; +} + +static u64 tdx_get_quote_check_args (struct kvm_vcpu *vcpu,u64 gpa,u64 size)Adding single use helpers does not make the code easier to read.
Yep. Will fix.
Quoted 17 lines
Quoted 17 lines
arch/x86/kvm/vmx/tdx.c +16
-0+{ +gfn_t gfn_start, gfn_end; +u64 end; + +if (!size) +return TDVMCALL_STATUS_INVALID_OPERAND ; + +if (!PAGE_ALIGNED (gpa) || !PAGE_ALIGNED (size)) +return TDVMCALL_STATUS_ALIGN_ERROR ; + +if (check_add_overflow (gpa, size, &end)) +return TDVMCALL_STATUS_INVALID_OPERAND ; + + gfn_start =gpa_to_gfn (gpa); + gfn_end =gpa_to_gfn (end); +
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+/* +* Reject if the guest didn't explicitly convert its quote pages to +* shared. +*/ +if (!kvm_range_has_memory_attributes (vcpu->kvm, gfn_start, gfn_end, +KVM_MEMORY_ATTRIBUTE_PRIVATE ,0 ))TOCTOU?
I struggled a bit with this actually... I can't just grab kvm->slots_lock here, and dropping kvm->srcu seems bad because the quote will be written to guest memory later. I settled on the idea that the guest should make sure these pages are converted to shared first. If it tries to convert them back to private after this check, before this TDVMCALL returns, then it's kind of a self-inflicted race and I don't see KVM doing any self damage in this case. Not sure if this makes sense to you? Or maybe I should just drop this check and go straight to kvm_vcpu_read_guest()?
Thanks,
Peter
Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/vmx/tdx.c +4
-0+return TDVMCALL_STATUS_INVALID_OPERAND ; + +return TDVMCALL_STATUS_SUCCESS ; +}
Sean Christopherson
maintainerRe: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 8 lines · Peter Fang
Quoted 4 lines · Sean Christopherson
On Thu, Jun 18, 2026, Xu Yilun wrote:
And is keying offtdx_quote_enabled()and onlytdx_quote_enabled()backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?Yeah this needs explaining:
Quoted 9 lines
The guest-observable difference between the two paths is that it sees quotes in different formats (the TDX module uses newer formats of course). And an old guest must be able to continue using its TDVMCALL<GetQuote> interface for quotes. The guest sends the quote to an external verifier to authenticate itself. The verifier (part of the cloud infrastructure) must know how to parse this format. So in other words, it's the verifier that must maintain backward compatibility if it wants to continue accepting old quotes. And it needs to be updated to support newer formats as well.
Uh, no, that's not how KVM's guest/host ABI works. Updating the host kernel and breaking existing guest is not acceptable.
Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
arch/x86/kvm/vmx/tdx.c +16
-0+{ +gfn_t gfn_start, gfn_end; +u64 end; + +if (!size) +return TDVMCALL_STATUS_INVALID_OPERAND ; + +if (!PAGE_ALIGNED (gpa) || !PAGE_ALIGNED (size)) +return TDVMCALL_STATUS_ALIGN_ERROR ; + +if (check_add_overflow (gpa, size, &end)) +return TDVMCALL_STATUS_INVALID_OPERAND ; + + gfn_start =gpa_to_gfn (gpa); + gfn_end =gpa_to_gfn (end); +Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+/* +* Reject if the guest didn't explicitly convert its quote pages to +* shared. +*/ +if (!kvm_range_has_memory_attributes (vcpu->kvm, gfn_start, gfn_end, +KVM_MEMORY_ATTRIBUTE_PRIVATE ,0 ))TOCTOU?
Quoted 9 lines
I struggled a bit with this actually... I can't just grab
kvm->slots_lockhere, and droppingkvm->srcuseems bad because the quote will be written to guest memory later. I settled on the idea that the guest should make sure these pages are converted to shared first. If it tries to convert them back to private after this check, before this TDVMCALL returns, then it's kind of a self-inflicted race and I don't see KVM doing any self damage in this case. Not sure if this makes sense to you? Or maybe I should just drop this check and go straight tokvm_vcpu_read_guest()?
Yes, drop it and rely on uaccess to do the right thing.
Re: [PATCH v2 16/17] KVM: TDX: Add in-kernel Quote generation
Quoted 19 lines · Sean Christopherson
Quoted 8 lines · Peter Fang
Quoted 4 lines · Sean Christopherson
On Thu, Jun 18, 2026, Xu Yilun wrote:
And is keying offtdx_quote_enabled()and onlytdx_quote_enabled()backwards compatible? How do we know that cutting userspace out of the loop wont' break anything?Yeah this needs explaining:
Quoted 9 lines
The guest-observable difference between the two paths is that it sees quotes in different formats (the TDX module uses newer formats of course). And an old guest must be able to continue using its TDVMCALL<GetQuote> interface for quotes. The guest sends the quote to an external verifier to authenticate itself. The verifier (part of the cloud infrastructure) must know how to parse this format. So in other words, it's the verifier that must maintain backward compatibility if it wants to continue accepting old quotes. And it needs to be updated to support newer formats as well.
Quoted 2 lines
Uh, no, that's not how KVM's guest/host ABI works. Updating the host kernel and breaking existing guest is not acceptable.
Yeah I agree there's ABI sensitivity here. I can provide a bit more context here:
- When you get a quote you don't check it yourself. You pass it to a verifier service which does the check. The verifier checks if the platform is genuine. It has platform specific knowledge for each new platform. So with a new platform these services need to be updated to know about the new platform's quote, or attestation will fail for these guests.
- There may be platforms that support TDX but have SGX off for various reasons. Meaning SGX based attestation won't work and they will only support this new type of quote.
- SGX based attestation is very complicated for users. It will likely be deprecated long term. This new type of quote simplifies a lot of steps.
A risk could be that someone makes a quote verifying service that supports a new DICE capable platform halfway (SGX only), then the user does a kernel upgrade and sees their verifications start failing. But these halfway services would probably come to life after the kernel support is upstream since DICE capable platforms are not available yet. So not sure if that's a major concern for regression.
I should call out this is not a zero-risk ABI change and the risks should have been enumerated. The approach this revision takes is to have less uABI on the assumption that it will not impact any real users. So we need to consider whether the simplicity is worth it vs going to userspace and back. Is it worth considering to have less uABI? Or would you rather punt to userspace? BTW, the quote SEAMCALL does it for a specifc TD. So putting this new ABI in the TDX host service would require adding KVM's TD life cycle knowledge. So the "and back" part probably would fit best as a new KVM TDX ioctl.
Quoted 36 lines
Quoted 26 lines
Quoted 17 lines
Quoted 17 lines
arch/x86/kvm/vmx/tdx.c +16
-0+{ +gfn_t gfn_start, gfn_end; +u64 end; + +if (!size) +return TDVMCALL_STATUS_INVALID_OPERAND ; + +if (!PAGE_ALIGNED (gpa) || !PAGE_ALIGNED (size)) +return TDVMCALL_STATUS_ALIGN_ERROR ; + +if (check_add_overflow (gpa, size, &end)) +return TDVMCALL_STATUS_INVALID_OPERAND ; + + gfn_start =gpa_to_gfn (gpa); + gfn_end =gpa_to_gfn (end); +Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote_user(struct kvm_vcpu *vcpu, u64 gpa, u64 size) · line 1619
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+/* +* Reject if the guest didn't explicitly convert its quote pages to +* shared. +*/ +if (!kvm_range_has_memory_attributes (vcpu->kvm, gfn_start, gfn_end, +KVM_MEMORY_ATTRIBUTE_PRIVATE ,0 ))TOCTOU?
Quoted 9 lines
I struggled a bit with this actually... I can't just grab
kvm->slots_lockhere, and droppingkvm->srcuseems bad because the quote will be written to guest memory later. I settled on the idea that the guest should make sure these pages are converted to shared first. If it tries to convert them back to private after this check, before this TDVMCALL returns, then it's kind of a self-inflicted race and I don't see KVM doing any self damage in this case. Not sure if this makes sense to you? Or maybe I should just drop this check and go straight tokvm_vcpu_read_guest()?
Quoted 1 line
Yes, drop it and rely on uaccess to do the right thing.
Got it, thanks Sean.
PATCH v2 17/17 KVM: TDX: Support event-notify interrupts only with userspace Quoting
In reply to Xu YilunFrom: Peter Fang <[email protected]>
Tie userspace SetupEventNotifyInterrupt support to userspace Quote generation. Delivering event-notify interrupts via userspace breaks if KVM never exits to userspace in the first place.
This is an optional capability to notify the guest when Quoting has completed. No known guest currently uses it, so defer adding in-kernel support for now. The Linux TDX guest relies on polling only.
--- arch/x86/kvm/vmx/tdx.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/tdx.c +17 -3
@@ -185,7 +185,7 @@ static void td_init_cpuid_entry2(struct kvm_cpuid_entry2 *entry, unsigned char itdx_clear_unsupported_cpuid (entry); } -#define TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT BIT(1) +#define TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT BIT_ULL(1)static int init_kvm_tdx_caps (const struct tdx_sys_info_td_conf *td_conf,struct kvm_tdx_capabilities *caps)
@@ -202,8 +202,15 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, caps->cpuid.nent = td_conf->num_cpuid_config; - caps->user_tdvmcallinfo_1_r11 = -TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT ; +/* +* Don't advertise userspace event-notify interrupt support if TDX +* quoting service is enabled, as quote generation will be handled +* entirely in the kernel. Support in the kernel can be added later. +*/ +if (!tdx_quote_enabled ()) { + caps->user_tdvmcallinfo_1_r11 |= +TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT ; + }for (i =0 ; i < td_conf->num_cpuid_config; i++)td_init_cpuid_entry2 (&caps->cpuid.entries[i], i);
@@ -1684,9 +1691,16 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)static int tdx_setup_event_notify_interrupt (struct kvm_vcpu *vcpu) { +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm);struct vcpu_tdx *tdx =to_tdx (vcpu);u64 vector = tdx->vp_enter_args.r12; +/* See comment in init_kvm_tdx_caps() */ +if (kvm_tdx->get_quote_in_kernel) { +tdvmcall_set_return_code (vcpu,TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED ); +return 1 ; + } +if (vector <32 || vector >255 ) {tdvmcall_set_return_code (vcpu,TDVMCALL_STATUS_INVALID_OPERAND );return 1 ;
Signature · 1 line
2.25.1
Re: [PATCH v2 17/17] KVM: TDX: Support event-notify interrupts only with userspace Quoting
Quoted 8 lines · Xu Yilun
From: Peter Fang <[email protected]>
arch/x86/kvm/vmx/tdx.c +0
-1@@ -202,8 +202,15 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, caps->cpuid.nent = td_conf->num_cpuid_config; - caps->user_tdvmcallinfo_1_r11 =
arch/x86/kvm/vmx/tdx.c static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, · line 213
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +9
-1-TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT ; +/* +* Don't advertise userspace event-notify interrupt support if TDX +* quoting service is enabled, as quote generation will be handled +* entirely in the kernel. Support in the kernel can be added later. +*/ +if (!tdx_quote_enabled ()) { + caps->user_tdvmcallinfo_1_r11 |= +TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT ; + }
Can you use kvm_tdx->get_quote_in_kernel also above? Or should it maybe be initialized here if not used earlier?
Quoted 4 lines
arch/x86/kvm/vmx/tdx.c +0
-0@@ -1684,9 +1691,16 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)static int tdx_setup_event_notify_interrupt (struct kvm_vcpu *vcpu) {
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote(struct kvm_vcpu *vcpu) · line 1703
Quoted 10 lines
arch/x86/kvm/vmx/tdx.c +7
-0+struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm);struct vcpu_tdx *tdx =to_tdx (vcpu);u64 vector = tdx->vp_enter_args.r12; +/* See comment in init_kvm_tdx_caps() */ +if (kvm_tdx->get_quote_in_kernel) { +tdvmcall_set_return_code (vcpu,TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED ); +return 1 ; + } +
Since you're using kvm_tdx->get_quote_in_kernel here.
Re: [PATCH v2 17/17] KVM: TDX: Support event-notify interrupts only with userspace Quoting
Quoted 8 lines · Tony Lindgren
Quoted 7 lines · Xu Yilun
From: Peter Fang <[email protected]>
arch/x86/kvm/vmx/tdx.c +0
-0@@ -202,8 +202,15 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, caps->cpuid.nent = td_conf->num_cpuid_config;
Quoted 14 lines
arch/x86/kvm/vmx/tdx.c static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, · line 213
Quoted 11 lines
arch/x86/kvm/vmx/tdx.c +9
-2- caps->user_tdvmcallinfo_1_r11 = -TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT ; +/* +* Don't advertise userspace event-notify interrupt support if TDX +* quoting service is enabled, as quote generation will be handled +* entirely in the kernel. Support in the kernel can be added later. +*/ +if (!tdx_quote_enabled ()) { + caps->user_tdvmcallinfo_1_r11 |= +TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT ; + }Can you use
kvm_tdx->get_quote_in_kernelalso above? Or should it maybe be initialized here if not used earlier?
This is a bit tricky. init_kvm_tdx_caps() is designed to be independent of a TDX VM instance (it reports the overall TDX capabilities), so there is no struct kvm_tdx available.
Quoted 9 lines
Quoted 9 lines
arch/x86/kvm/vmx/tdx.c +1
-0@@ -1684,9 +1691,16 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu)static int tdx_setup_event_notify_interrupt (struct kvm_vcpu *vcpu) { +struct kvm_tdx *kvm_tdx =to_kvm_tdx (vcpu->kvm);struct vcpu_tdx *tdx =to_tdx (vcpu);u64 vector = tdx->vp_enter_args.r12;
Quoted 8 lines
arch/x86/kvm/vmx/tdx.c static int tdx_get_quote(struct kvm_vcpu *vcpu) · line 1703
Quoted 6 lines
arch/x86/kvm/vmx/tdx.c +6
-0+/* See comment in init_kvm_tdx_caps() */ +if (kvm_tdx->get_quote_in_kernel) { +tdvmcall_set_return_code (vcpu,TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED ); +return 1 ; + } +Since you're using
kvm_tdx->get_quote_in_kernelhere.
This is in per-vCPU context, so struct kvm_tdx is available.