Skip to contentSkip to conversation

x86/virt/tdx: Formalize SEAMCALL leaf version encoding support

PATCH Superseded Xu Yilun 5 participants 9 messages Sep 1 – Sep 10

Subsystems (2)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Xu Yilun

author
#0 lore raw

PATCH v5 1/1 x86/virt/tdx: Formalize SEAMCALL leaf version encoding support

The TDX architecture includes a syscall-like ABI for OSes to communicate with SEAM mode software. This ABI has the concept of a "leaf". Each leaf is roughly analogous to a Linux syscall: it has a set of register arguments and does one logical thing like adding a page of memory to a VM or running a VM. The TDX architecture refers to this interface function as a "SEAMCALL leaf".

Just like syscalls, the TDX architecture wants to evolve the ABI to extend functionality while keeping compatibility. But unlike syscalls, which do this by picking a totally new syscall number, the ABI encodes the "version" number directly into the bits of a register argument. So instead of openatN being whatever the next free number is, the SEAMCALL leaf number and version number are encoded into certain bits in how the RAX register is defined in the ABI.

In Linux, several seamcall*() wrappers have been introduced to invoke SEAMCALL leafs. They all take a u64 "fn" argument for the leaf number which eventually get set in the register. As above, the version number lives in the same register as the leaf number. So callers that want to select a specific version of the leaf, can jam it in the right place in the register by passing it in the "fn" argument. Today only the caller of TDH.VP.INIT does this hack, but future kernel changes will need to select versions for more SEAMCALL leafs, so a less hacky solution is needed.

Explicitly define the arguments for seamcall*() wrappers:

- Add a build-time assertion to ensure "fn" only contains valid SEAMCALL leaf number bits. Note the P-SEAMLDR selector bit (bit 63) is part of the leaf number for SEAM loader calls, so it is allowed. This also means the "fn" can't be a narrower type, such as u16, to exclude unrelated bits.

- Add a "version" field in struct tdx_module_args [1], so most existing callers get a default "version == 0" behavior without code churn. Update the TDH.VP.INIT caller to specify the version descriptively.

Encode the leaf number and tdx_module_args.version into RAX in assembly, because this is the place where the "fn" and struct tdx_module_args fields are marshaled into registers.

AI was used under supervision to review code and workshop logs. In particular, it helped evaluate the assembly changes.

---
Two alternative schemes were considered:

1. Define versioned macros like TDH_VP_INIT_V0, TDH_VP_INIT_V1, etc. Then callers could use them without encoding RAX bits themselves. However, this would break naming consistency unless all existing stable function macros are changed to TDH_XXX_V0.

2. Add an explicit "version" parameter to seamcall*() wrappers. This would unnecessarily force all stable seamcall*() wrappers to pass a meaningless '0' argument. Additionally, the magic '0' or '1' values at caller sites are not descriptive.

v5:
- Refactor the changelog. Retained the Reviewed-by tags, please let me know if you object.
- Rebase to v7.3-rc1

v4: https://lore.kernel.org/all/[email protected]/
- Add Reviewed-by tags

v3: https://lore.kernel.org/all/[email protected]/
- Move the RAX encoding in assembly code.
- Compile-time check that fn only contain base leaf number bits
- Add the full description of SEAMCALL leaf according to TDX module SPEC
- Fix the description of struct tdx_module_args.

v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

---
 arch/x86/include/asm/shared/tdx.h         | 10 +++++++---
 arch/x86/virt/vmx/tdx/seamcall_internal.h | 19 +++++++++++++++++++
 arch/x86/virt/vmx/tdx/tdx.h               |  8 --------
 arch/x86/virt/vmx/tdx/tdxcall.S           | 10 ++++++++--
 arch/x86/kernel/asm-offsets.c             |  1 +
 arch/x86/virt/vmx/tdx/tdx.c               |  5 +++--
 6 files changed, 38 insertions(+), 15 deletions(-)
arch/x86/include/asm/shared/tdx.h +7 -3
index f20e91d7ac35..12675c48402b 100644
@@ -122,9 +122,11 @@
 #include <linux/compiler_attributes.h>
 
 /*
- * Used in __tdcall*() to gather the input/output registers' values of the
- * TDCALL instruction when requesting services from the TDX module. This is a
- * software only structure and not part of the TDX module/VMM ABI
+ * Used in __tdcall*() to gather the input/output parameters of the TDCALL
+ * instruction when requesting services from the TDX module. "version" is
+ * encoded in RAX along with the leaf number. Others are raw input/output
+ * registers' values. This is a software only structure and not part of the TDX
+ * module/VMM ABI.
  */
 struct tdx_module_args {
 	/* callee-clobbered */
@@ -143,6 +145,8 @@ struct tdx_module_args {
 	u64 rbx;
 	u64 rdi;
 	u64 rsi;
+	/* leaf ABI version, encoded in rax */
+	u8  version;
 };
 
 /* Used to communicate with the TDX module */
arch/x86/virt/vmx/tdx/seamcall_internal.h +19 -0
index be5f446467df..051ad2d45cab 100644
@@ -11,6 +11,7 @@
 #ifndef _X86_VIRT_SEAMCALL_INTERNAL_H
 #define _X86_VIRT_SEAMCALL_INTERNAL_H
 
+#include <linux/bitfield.h>
 #include <linux/printk.h>
 #include <linux/types.h>
 #include <asm/archrandom.h>
@@ -23,9 +24,27 @@ u64 __seamcall_saved_ret(u64 fn, struct tdx_module_args *args);
 
 typedef u64 (*sc_func_t)(u64 fn, struct tdx_module_args *args);
 
+/*
+ * SEAMCALL leaf:
+ *
+ * Bit 15:0	Leaf number
+ * Bit 23:16	Leaf ABI version number
+ * Bit 24	Pending interrupts detection mode
+ * Bit 63	1 for P-SEAMLDR leaf, 0 for TDX module leaf
+ */
+#define SEAMCALL_LEAF_MASK		GENMASK_U64(15, 0)
+#define SEAMCALL_SEAMLDR_MASK		BIT_U64(63)
+
 static __always_inline u64 __seamcall_dirty_cache(sc_func_t func, u64 fn,
 						  struct tdx_module_args *args)
 {
+	/*
+	 * fn contains leaf number for TDX module calls and P-SEAMLDR calls.
+	 * Other fields in SEAMCALL leaf like leaf ABI version number are in
+	 * struct tdx_module_args.
+	 */
+	BUILD_BUG_ON(fn & ~(SEAMCALL_LEAF_MASK | SEAMCALL_SEAMLDR_MASK));
+
 	lockdep_assert_preemption_disabled();
 
 	/*
arch/x86/virt/vmx/tdx/tdx.h +0 -8
index bdfd0e1e337a..63e3acfb5d0c 100644
@@ -50,14 +50,6 @@
 #define TDH_SYS_UPDATE			53
 #define TDH_SYS_DISABLE			69
 
-/*
- * SEAMCALL leaf:
- *
- * Bit 15:0	Leaf number
- * Bit 23:16	Version number
- */
-#define TDX_VERSION_SHIFT		16
-
 /* TDX page types */
 #define	PT_NDA		0x0
 #define	PT_RSVD		0x1
arch/x86/virt/vmx/tdx/tdxcall.S +8 -2
index 016a2a1ec1d6..a194e83613e7 100644
@@ -45,8 +45,14 @@
 .macro TDX_MODULE_CALL host:req ret=0 saved=0
 	FRAME_BEGIN
 
-	/* Move Leaf ID to RAX */
-	mov %rdi, %rax
+	/* Leaf ABI version -> RAX[23:16]. Zero rest of RAX. */
+	movzbl	TDX_MODULE_version(%rsi), %eax
+	shl	$16, %eax
+	/*
+	 * Combine leaf number arg and leaf ABI version into RAX, they don't
+	 * overlap.
+	 */
+	or	%rdi, %rax
 
 	/* Move other input regs from 'struct tdx_module_args' */
 	movq	TDX_MODULE_rcx(%rsi), %rcx
arch/x86/kernel/asm-offsets.c +1 -0
index 081816888f7a..b3c00ff4d819 100644
@@ -95,6 +95,7 @@ static void __used common(void)
 	OFFSET(TDX_MODULE_rbx, tdx_module_args, rbx);
 	OFFSET(TDX_MODULE_rdi, tdx_module_args, rdi);
 	OFFSET(TDX_MODULE_rsi, tdx_module_args, rsi);
+	OFFSET(TDX_MODULE_version, tdx_module_args, version);
 
 	BLANK();
 	OFFSET(BP_scratch, boot_params, scratch);
arch/x86/virt/vmx/tdx/tdx.c +3 -2
index 1b9ff749dd8e..1668f8615607 100644
@@ -1912,10 +1912,11 @@ u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid)
 		.rcx = vp->tdvpr_pa,
 		.rdx = initial_rcx,
 		.r8 = x2apicid,
+		/* apicid requires version == 1. */
+		.version = 1,
 	};
 
-	/* apicid requires version == 1. */
-	return seamcall(TDH_VP_INIT | (1ULL << TDX_VERSION_SHIFT), &args);
+	return seamcall(TDH_VP_INIT, &args);
 }
 EXPORT_SYMBOL_FOR_KVM(tdh_vp_init);
 

base-commit: cee9395acd8043be0644b25c34bfa86623f2b935

Signature · 1 line
2.25.1
Reviewed-by on v5 1/1
#1 lore raw
Quoted 5 lines · Xu Yilun
arch/x86/include/asm/shared/tdx.h +0 -0
@@ -122,9 +122,11 @@
 #include <linux/compiler_attributes.h>
 

arch/x86/include/asm/shared/tdx.h line 130

Quoted 10 lines
arch/x86/include/asm/shared/tdx.h +5 -3
 /*
- * Used in __tdcall*() to gather the input/output registers' values of the
- * TDCALL instruction when requesting services from the TDX module. This is a
- * software only structure and not part of the TDX module/VMM ABI
+ * Used in __tdcall*() to gather the input/output parameters of the TDCALL
+ * instruction when requesting services from the TDX module. "version" is
+ * encoded in RAX along with the leaf number. Others are raw input/output
+ * registers' values. This is a software only structure and not part of the TDX
+ * module/VMM ABI.
  */

For better readability and a few less lines of changes, you could just add a one-liner to the existing comments:

Note that "version" gets encoded into RAX along with the leaf number.

Anyways, I'm happy with the current version without changes. The patch cleans up the version encoding issue nicely, so:

#2 lore raw
Quoted 10 lines · Tony Lindgren

/*
- * Used in __tdcall*() to gather the input/output registers' values of the
- * TDCALL instruction when requesting services from the TDX module. This is a
- * software only structure and not part of the TDX module/VMM ABI
+ * Used in __tdcall*() to gather the input/output parameters of the TDCALL
+ * instruction when requesting services from the TDX module. "version" is
+ * encoded in RAX along with the leaf number. Others are raw input/output
+ * registers' values. This is a software only structure and not part of the TDX
+ * module/VMM ABI.
*/

It's a wee bit silly to talk about "version" in here. If you need to comment on "version" specifically, then put a comment *on* that variable.

author
#3 lore raw
Quoted 12 lines · Dave Hansen
Quoted 10 lines · Tony Lindgren

/*
- * Used in __tdcall*() to gather the input/output registers' values of the
- * TDCALL instruction when requesting services from the TDX module. This is a
- * software only structure and not part of the TDX module/VMM ABI
+ * Used in __tdcall*() to gather the input/output parameters of the TDCALL
+ * instruction when requesting services from the TDX module. "version" is
+ * encoded in RAX along with the leaf number. Others are raw input/output
+ * registers' values. This is a software only structure and not part of the TDX
+ * module/VMM ABI.
*/

Quoted 2 lines

It's a wee bit silly to talk about "version" in here. If you need to comment on "version" specifically, then put a comment *on* that variable.

Yes. I'll recover the original structure description and comment on the "version".

And Thank Tony for the description for "version".

----8<----

/*
- * Used in __tdcall*() to gather the input/output parameters of the TDCALL
- * instruction when requesting services from the TDX module. "version" is
- * encoded in RAX along with the leaf number. Others are raw input/output
- * registers' values. This is a software only structure and not part of the TDX
- * module/VMM ABI.
+ * Used in __tdcall*() to gather the input/output registers' values of the
+ * TDCALL instruction when requesting services from the TDX module. This is a
+ * software only structure and not part of the TDX module/VMM ABI */

 struct tdx_module_args {
        /* callee-clobbered */
@@ -145,7 +143,10 @@ struct tdx_module_args {
        u64 rbx;
        u64 rdi;
        u64 rsi;
-       /* leaf ABI version, encoded in rax */
+       /*
+        * leaf ABI version. Note that it gets encoded into RAX along with the
+        * leaf number.
+        */
        u8  version;

};

#4 lore raw
In reply to Xu Yilun
Quoted 15 lines · Xu Yilun

The TDX architecture includes a syscall-like ABI for OSes to communicate with SEAM mode software. This ABI has the concept of a "leaf". Each leaf is roughly analogous to a Linux syscall: it has a set of register arguments and does one logical thing like adding a page of memory to a VM or running a VM. The TDX architecture refers to this interface function as a "SEAMCALL leaf".

Just like syscalls, the TDX architecture wants to evolve the ABI to extend functionality while keeping compatibility. But unlike syscalls, which do this by picking a totally new syscall number, the ABI encodes the "version" number directly into the bits of a register argument. So instead of openatN being whatever the next free number is, the SEAMCALL leaf number and version number are encoded into certain bits in how the RAX register is defined in the ABI.

Quoted 6 lines

In Linux, several seamcall*() wrappers have been introduced to invoke SEAMCALL leafs. They all take a u64 "fn" argument for the leaf number which eventually get set in the register. As above, the version number lives in the same register as the leaf number. So callers that want to select a specific version of the leaf, can jam it in the right place in the register by passing it in the "fn" argument.

Do you think it's worth explaining how callers determine the versions supported by the TDX module, and what happens if the specified version is not supported by the TDX module?

author
#5 lore raw
Quoted 23 lines · Binbin Wu
Quoted 15 lines · Xu Yilun

The TDX architecture includes a syscall-like ABI for OSes to communicate with SEAM mode software. This ABI has the concept of a "leaf". Each leaf is roughly analogous to a Linux syscall: it has a set of register arguments and does one logical thing like adding a page of memory to a VM or running a VM. The TDX architecture refers to this interface function as a "SEAMCALL leaf".

Just like syscalls, the TDX architecture wants to evolve the ABI to extend functionality while keeping compatibility. But unlike syscalls, which do this by picking a totally new syscall number, the ABI encodes the "version" number directly into the bits of a register argument. So instead of openatN being whatever the next free number is, the SEAMCALL leaf number and version number are encoded into certain bits in how the RAX register is defined in the ABI.

Quoted 6 lines

In Linux, several seamcall*() wrappers have been introduced to invoke SEAMCALL leafs. They all take a u64 "fn" argument for the leaf number which eventually get set in the register. As above, the version number lives in the same register as the leaf number. So callers that want to select a specific version of the leaf, can jam it in the right place in the register by passing it in the "fn" argument.

Quoted 2 lines

Do you think it's worth explaining how callers determine the versions supported by the TDX module,

Yeah I described this in v4 [1]. Generally it is "selecting the minimum workable version number for the required functionality".

But I feel it distracts people from the problem this patch wants to solve, so I removed it. This patch actually involves no runtime version selection. The future change for TDH.SYS.CONFIG does. Better we add the explaination in that patch?

[1]: https://lore.kernel.org/all/[email protected]/

Quoted 2 lines

and what happens if the specified version
is not supported by the TDX module?

I think it should not happen. This means the kernel is trying to enable a functionality that the module doesn't support. Either there is a problem on feature enumeration, or people are using a deprecated module. Anyway don't runtime check if a SEAMCALL version is supported.

Thanks,
Yilun

#6 lore raw
Quoted 27 lines · Xu Yilun
Quoted 23 lines · Binbin Wu
Quoted 15 lines · Xu Yilun

The TDX architecture includes a syscall-like ABI for OSes to communicate with SEAM mode software. This ABI has the concept of a "leaf". Each leaf is roughly analogous to a Linux syscall: it has a set of register arguments and does one logical thing like adding a page of memory to a VM or running a VM. The TDX architecture refers to this interface function as a "SEAMCALL leaf".

Just like syscalls, the TDX architecture wants to evolve the ABI to extend functionality while keeping compatibility. But unlike syscalls, which do this by picking a totally new syscall number, the ABI encodes the "version" number directly into the bits of a register argument. So instead of openatN being whatever the next free number is, the SEAMCALL leaf number and version number are encoded into certain bits in how the RAX register is defined in the ABI.

Quoted 6 lines

In Linux, several seamcall*() wrappers have been introduced to invoke SEAMCALL leafs. They all take a u64 "fn" argument for the leaf number which eventually get set in the register. As above, the version number lives in the same register as the leaf number. So callers that want to select a specific version of the leaf, can jam it in the right place in the register by passing it in the "fn" argument.

Quoted 2 lines

Do you think it's worth explaining how callers determine the versions supported by the TDX module,

Quoted 7 lines

Yeah I described this in v4 [1]. Generally it is "selecting the minimum workable version number for the required functionality".

But I feel it distracts people from the problem this patch wants to solve, so I removed it. This patch actually involves no runtime version selection. The future change for TDH.SYS.CONFIG does. Better we add the explaination in that patch?

It's OK for me.

I was wondering how did VMM know which version was supported. Now I know It's based on the feature enumeration.

Quoted 13 lines

[1]: https://lore.kernel.org/all/[email protected]/

Quoted 2 lines

and what happens if the specified version
is not supported by the TDX module?

I think it should not happen. This means the kernel is trying to enable a functionality that the module doesn't support. Either there is a problem on feature enumeration, or people are using a deprecated module. Anyway don't runtime check if a SEAMCALL version is supported.

Thanks,
Yilun

maintainer Reviewed-by on v5 1/1
#7 lore raw
In reply to Xu Yilun
Quoted 24 lines · Xu Yilun

The TDX architecture includes a syscall-like ABI for OSes to communicate with SEAM mode software. This ABI has the concept of a "leaf". Each leaf is roughly analogous to a Linux syscall: it has a set of register arguments and does one logical thing like adding a page of memory to a VM or running a VM. The TDX architecture refers to this interface function as a "SEAMCALL leaf".

Just like syscalls, the TDX architecture wants to evolve the ABI to extend functionality while keeping compatibility. But unlike syscalls, which do this by picking a totally new syscall number, the ABI encodes the "version" number directly into the bits of a register argument. So instead of openatN being whatever the next free number is, the SEAMCALL leaf number and version number are encoded into certain bits in how the RAX register is defined in the ABI.

In Linux, several seamcall*() wrappers have been introduced to invoke SEAMCALL leafs. They all take a u64 "fn" argument for the leaf number which eventually get set in the register. As above, the version number lives in the same register as the leaf number. So callers that want to select a specific version of the leaf, can jam it in the right place in the register by passing it in the "fn" argument.

Quoted 4 lines

Today only the caller
of TDH.VP.INIT does this hack, but future kernel changes will need to select versions for more SEAMCALL leafs, so a less hacky solution is needed.

Only nit is that this would probably read a little bit smoother as two sentences. Everything else looks good.

Quoted 25 lines

Explicitly define the arguments for seamcall*() wrappers:

  - Add a build-time assertion to ensure "fn" only contains valid SEAMCALL leaf number bits. Note the P-SEAMLDR selector bit (bit 63) is part of the leaf number for SEAM loader calls, so it is allowed. This also means the "fn" can't be a narrower type, such as u16, to exclude unrelated bits.

  - Add a "version" field in struct tdx_module_args [1], so most existing callers get a default "version == 0" behavior without code churn. Update the TDH.VP.INIT caller to specify the version descriptively.

Encode the leaf number and tdx_module_args.version into RAX in assembly, because this is the place where the "fn" and struct tdx_module_args fields are marshaled into registers.

AI was used under supervision to review code and workshop logs. In particular, it helped evaluate the assembly changes.

#8 lore raw
Quoted 4 lines · Rick Edgecombe

Only nit is that this would probably read a little bit smoother as two sentences. Everything else looks good.

Oops I replied to the wrong one after reviewing v6 on lore. Please ignore