Skip to contentSkip to conversation

tboot: Disable CET at shutdown

PATCH In review Jason Andryuk 3 participants 3 messages Aug 15, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tboot: Disable CET at shutdown

tboot_shutdown() calls into tboot to perform the actual system shutdown. tboot isn't built with endbr annotations, and Xen has CET-IBT enabled on newer hardware. shutdown_entry isn't annotated with endbr and Xen faults:

Panic on CPU 0:
CONTROL-FLOW PROTECTION FAULT: #CP[0003] endbranch

And Xen hangs at this point.

Disabling CET-IBT let Xen and tboot power off, but reboot was perfoming a poweroff instead of a warm reboot. Disabling all of CET, i.e. shadow stacks as well, lets tboot reboot properly.

Fixes cdbe2b0a1aec ("x86: Enable CET Indirect Branch Tracking")

---
Without this fix, Xen subsequently hangs:

Reboot in five seconds...
[VT-D] IOMMU1: QI wait descriptor taking too long
IQA = 484897000
IQH = 0
IQT = 820

with no futher output.

---
 xen/arch/x86/tboot.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
xen/arch/x86/tboot.c +10 -0
index 90f6e805a9..86c4c22cac 100644
@@ -353,6 +353,16 @@ void tboot_shutdown(uint32_t shutdown_type)
         tboot_gen_xenheap_integrity(g_tboot_shared->s3_key, &xenheap_mac);
     }
 
+    /*
+     * Disable CET - tboot may not be built with endbr, and it doesn't support
+     * shadow stacks.
+     */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }
+
     /*
      * During early boot, we can be called by panic before idle_vcpu[0] is
      * setup, but in that case we don't need to change page tables.
Signature · 1 line
2.41.0
Acked-by on v1 1/1
#1 lore raw
Quoted 8 lines · Jason Andryuk

tboot_shutdown() calls into tboot to perform the actual system shutdown. tboot isn't built with endbr annotations, and Xen has CET-IBT enabled on newer hardware. shutdown_entry isn't annotated with endbr and Xen faults:

Panic on CPU 0:
CONTROL-FLOW PROTECTION FAULT: #CP[0003] endbranch

Quoted 8 lines

And Xen hangs at this point.

Disabling CET-IBT let Xen and tboot power off, but reboot was perfoming a poweroff instead of a warm reboot. Disabling all of CET, i.e. shadow stacks as well, lets tboot reboot properly.

Fixes cdbe2b0a1aec ("x86: Enable CET Indirect Branch Tracking")

:sadpanda:

I guess this is the least bad option going.

Reviewed-by on v1 1/1
#2 lore raw
In reply to Jason Andryuk
Quoted 41 lines · Jason Andryuk

tboot_shutdown() calls into tboot to perform the actual system shutdown. tboot isn't built with endbr annotations, and Xen has CET-IBT enabled on newer hardware. shutdown_entry isn't annotated with endbr and Xen faults:

Panic on CPU 0:
CONTROL-FLOW PROTECTION FAULT: #CP[0003] endbranch

And Xen hangs at this point.

Disabling CET-IBT let Xen and tboot power off, but reboot was perfoming a poweroff instead of a warm reboot. Disabling all of CET, i.e. shadow stacks as well, lets tboot reboot properly.

Fixes cdbe2b0a1aec ("x86: Enable CET Indirect Branch Tracking")

---
Without this fix, Xen subsequently hangs:

Reboot in five seconds...
[VT-D] IOMMU1: QI wait descriptor taking too long
IQA = 484897000
IQH = 0
IQT = 820

with no futher output.

---
  xen/arch/x86/tboot.c | 10 ++++++++++
  1 file changed, 10 insertions(+)
xen/arch/x86/tboot.c +3 -0
index 90f6e805a9..86c4c22cac 100644
@@ -353,6 +353,16 @@ void tboot_shutdown(uint32_t shutdown_type)
          tboot_gen_xenheap_integrity(g_tboot_shared->s3_key, &xenheap_mac);
      }
 
+    /*
+     * Disable CET - tboot may not be built with endbr, and it doesn't support
+     * shadow stacks.

xen/arch/x86/tboot.c void tboot_shutdown(uint32_t shutdown_type) · line 368

Quoted 10 lines
xen/arch/x86/tboot.c +7 -0
+     */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }
+
      /*
       * During early boot, we can be called by panic before idle_vcpu[0] is
       * setup, but in that case we don't need to change page tables.