Skip to contentSkip to conversation

x86/virt/tdx: accurately distinguishes TDX module loading situations

PATCH In review Jun Miao 3 participants 4 messages Jun 22, 2024 – Jul 19, 2024

Subsystems (2)
lore
Share & tools
lore

Jun Miao

author
#0 lore raw

PATCH 1/1 x86/virt/tdx: accurately distinguishes TDX module loading situations

The first SEAMCALL is important to response the state of TDX Module/BIOS.

In actual incorrect BIOS setup or deployment, the sysinit_ret will be -EOPNOTSUPP. But the message "module not loaded" isn`t enough to describe the accurate loading situation when module loaded but SEAMCALL failed for some BIOS wrong setting. So add the error return operation code number.

---
 arch/x86/virt/vmx/tdx/tdx.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.c +5 -1
index 4e2b2e2ac9f9..787dfaf44036 100644
@@ -122,10 +122,14 @@ static int try_init_module_global(void)
 	/*
 	 * The first SEAMCALL also detects the TDX module, thus
 	 * it can fail due to the TDX module is not loaded.
-	 * Dump message to let the user know.
+	 * Dump more detailed message to let the user know.
 	 */
 	if (sysinit_ret == -ENODEV)
 		pr_err("module not loaded\n");
+	else if (sysinit_ret)
+		pr_warn("module loaded error ret=%d\n",sysinit_ret);
+	else
+		pr_info("module loaded\n");
 
 	sysinit_done = true;
 out:
Signature · 1 line
2.32.0
#1 lore raw
Quoted 6 lines · Jun Miao

The first SEAMCALL is important to response the state of TDX Module/BIOS.

In actual incorrect BIOS setup or deployment, the sysinit_ret will be -EOPNOTSUPP. But the message "module not loaded" isn`t enough to describe the accurate loading situation when module loaded but SEAMCALL failed for some BIOS wrong setting. So add the error return operation code number.

Do you want to actually check for EOPNOTSUPP and print a user-understandable message for it?

Quoted 10 lines
---
 arch/x86/virt/vmx/tdx/tdx.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 4e2b2e2ac9f9..787dfaf44036 100644

arch/x86/virt/vmx/tdx/tdx.c static int try_init_module_global(void) · line 130

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -1
@@ -122,10 +122,14 @@ static int try_init_module_global(void)
 	/*
 	 * The first SEAMCALL also detects the TDX module, thus
 	 * it can fail due to the TDX module is not loaded.
-	 * Dump message to let the user know.
+	 * Dump more detailed message to let the user know.
 	 */
 	if (sysinit_ret == -ENODEV)
 		pr_err("module not loaded\n");
+	else if (sysinit_ret)
+		pr_warn("module loaded error ret=%d\n",sysinit_ret);

s/loaded/load/

Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	else
+		pr_info("module loaded\n");
 
 	sysinit_done = true;
 out:
Signature · 1 line
2.32.0
Signature · 2 lines
  Kiryl Shutse
  mau / Kirill A. Shutemov
author
#2 lore raw
Quoted 9 lines · Kirill A. Shutemov
Quoted 6 lines · Jun Miao

The first SEAMCALL is important to response the state of TDX Module/BIOS.

In actual incorrect BIOS setup or deployment, the sysinit_ret will be -EOPNOTSUPP. But the message "module not loaded" isn`t enough to describe the accurate loading situation when module loaded but SEAMCALL failed for some BIOS wrong setting. So add the error return operation code number.

Do you want to actually check for EOPNOTSUPP and print a user-understandable message for it?

It`s like this: I am a customer service support engineer, I awlays deploy TDX on site such as ByteDance/meituan TDX is not simple but full stack.If boot up a TD guest, the follwing three requirements must be met:
    - The IFWI includes the correct seamldr and module in the BIOS, as the same time have the ability to load the TDX module.
    - Setting the TDX series of options, such as TME/SGX in the bios.
    - At last, loading the kernel, check if it can be boot with below demsg(when loaded successfully)

dmesg | grep - i tdx

[0.303913] virt/tdx: BIOS enabled: private KeyID range [64, 128)
[0.303916] virt/tdx: Disable Kexec. Turn off TDX in the BIOS to use KEXEC.
[0.303918] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.
[4.846924] usb usb1: Manufacturer: Linux 6.6.0-tdx.1.0.v1 xhci-hcd
[4.847748] usb usb2: Manufacturer: Linux 6.6.0-tdx.1.0.v1 xhci-hcd
[6.577811] BOOT_IMAGE=(hd0,gpt2)/vmlinuz-6.6.0-tdx.1.0.v1
[17.723174] virt/tdx: 1575964 KB? allocated for PAMT
[17.723185] virt/tdx: module initialized

But this time,? It took a lot of effort to find the SEAMLDR.bin loading failed, since enable the bios debug need OEM cooperation Got the bios log below as:

[TdxDxe]: [TDX_LATE-HANDLE_SEAMLDR] LoadTdxSeamldr BEGIN FATAL ERROR - RaiseTpl withOldTpl(0x1F) > NewTpl(0x10) ASSERT [DxeCore]e:\w\r\MdeModulePkg\Core\Dxe\Event\Tpl.c(66): ((BOOLEAN)(0==1))

[TdxDxe]: [TDX_LATE-HANDLE_SEAMLDR LoadTdxSeamldr END (Load Error)

But In the kernel, there is too little log message to find the error like this(loading failed):
dmesg | grep - i tdx

[0.303913] virt/tdx: BIOS enabled: private KeyID range [64, 128)[0.303916] virt/tdx: Disable Kexec. Turn off TDX in the BIOS to use KEXEC.
[0.303918] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.

I add the printk in the kernel and find the sysinit_ret = -19 = -EOPNOTSUPP
From the current logic,? no ¡°module not loaded¡± always meaning module loaded.? But in fact, sysinit_ret is -EOPNOTSUPP(-19). I will believe the module block at next seamcall: TDH_SYS_LP_INIT, which is misleading.

Quoted 2 lines

    if (sysinit_ret == -ENODEV)
    pr_err("module not loaded\n");

... ...

Quoted 1 line

    ret = seamcall_prerr(TDH_SYS_LP_INIT,&args);?? ?

However I also use the old bkc kernel, there is a useful message to find such issues.¡± SEAMRR is not enabled by BIOS¡± to remind me where the problem lies. In order to more accurately find the problems, add some printing reminders here will help the On site deployment engineer like me. ?

The story is a bit long. Thank you again for reading it over. Looking forward to your suggestions.

---
Jun Miao

Quoted 31 lines
Quoted 11 lines
---
  arch/x86/virt/vmx/tdx/tdx.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.c +0 -0
index 4e2b2e2ac9f9..787dfaf44036 100644

arch/x86/virt/vmx/tdx/tdx.c static int try_init_module_global(void) · line 130

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +3 -1
@@ -122,10 +122,14 @@ static int try_init_module_global(void)
  	/*
  	 * The first SEAMCALL also detects the TDX module, thus
  	 * it can fail due to the TDX module is not loaded.
-	 * Dump message to let the user know.
+	 * Dump more detailed message to let the user know.
  	 */
  	if (sysinit_ret == -ENODEV)
  		pr_err("module not loaded\n");
+	else if (sysinit_ret)
+		pr_warn("module loaded error ret=%d\n",sysinit_ret);

s/loaded/load/

Quoted 8 lines
arch/x86/virt/vmx/tdx/tdx.c +2 -0
+	else
+		pr_info("module loaded\n");
 
  	sysinit_done = true;
  out:
Signature · 1 line
2.32.0
#3 lore raw
In reply to Jun Miao
Quoted 6 lines · Jun Miao

The first SEAMCALL is important to response the state of TDX Module/BIOS.

In actual incorrect BIOS setup or deployment, the sysinit_ret will be -EOPNOTSUPP. But the message "module not loaded" isn`t enough to describe the accurate loading situation when module loaded but SEAMCALL failed for some BIOS wrong setting. So add the error return operation code number.

(firstly, please CC me in the future, and please also CC x86 maintainers, [email protected], and [email protected]).

So this is the case where TDX is detected as enabled by the kernel (TDX KeyID reports valid info), but TDX actually isn't enabled by the BIOS.

This seems BIOS bug to me. The BIOS should never report valid TDX KeyID if TDX isn't enabled. I guess it happens on development machines, but I am not sure whether the production machine would fix this.

Anyway ...

Quoted 16 lines
---
 arch/x86/virt/vmx/tdx/tdx.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/virt/vmx/tdx/tdx.c +1 -1
index 4e2b2e2ac9f9..787dfaf44036 100644
@@ -122,10 +122,14 @@ static int try_init_module_global(void)
 	/*
 	 * The first SEAMCALL also detects the TDX module, thus
 	 * it can fail due to the TDX module is not loaded.
-	 * Dump message to let the user know.
+	 * Dump more detailed message to let the user know.

arch/x86/virt/vmx/tdx/tdx.c static int try_init_module_global(void) · line 135

Quoted 10 lines
arch/x86/virt/vmx/tdx/tdx.c +4 -0
 	 */
 	if (sysinit_ret == -ENODEV)
 		pr_err("module not loaded\n");
+	else if (sysinit_ret)
+		pr_warn("module loaded error ret=%d\n",sysinit_ret);
+	else
+		pr_info("module loaded\n");
 
 	sysinit_done = true;
 out:

... printing the error code won't help user a lot since it is not obvious to the user. I think we can do something below.

Hi Dave/Kirill,

Any comments?

arch/x86/virt/vmx/tdx/tdx.c +28 -6
index 49a1c6890b55..a8c273cfe17e 100644
@@ -119,13 +119,35 @@ static int try_init_module_global(void)
        args.rcx = 0;
        sysinit_ret = seamcall_prerr(TDH_SYS_INIT, &args);
 
-       /*
-        * The first SEAMCALL also detects the TDX module, thus
-        * it can fail due to the TDX module is not loaded.
-        * Dump message to let the user know.
-        */
-       if (sysinit_ret == -ENODEV)
+       switch (sysinit_ret) {
+       case -ENODEV:
+               /*
+                * The first SEAMCALL also detects the TDX module,
+                * thus it can fail due to the TDX module is not
+                * loaded.  Dump message to let the user know.
+                */
                pr_err("module not loaded\n");
+               break;
+       case -EOPNOTSUPP:
+               /*
+                * Some TDX-capable development machines may report
+                * valid TDX KeyID when TDX is actually not enabled
+                * by the BIOS (e.g., due to BIOS misconfiguration).
+                * Let the user know.
+                */
+               pr_err("[BIOS bug]: TDX isn't enabled.\n");
+               break;
+       case -EACCES:
+               /*
+                * -EACCES happens when this is called when CPU is
+                * not in post-VMXON state.  It's kernel issue, so
+                * don't dump error message.
+                */
+               break;
+       default:
+               /* Actual SEAMCALL failure, message already printed */
+               break;
+       }
 
        sysinit_done = true;
 out: