x86: Improve CET tests
Sean Christopherson
reviewerv3 cover x86: Improve CET tests
I didn't want to miss out on the fun :-)
v3:
- Run the test if only one of SHSTK or IBT is supported (e.g. to test SHSTK on AMD).
- Rename the test from "intel_cet" to just "cet".
- Add an endbr64 in the user_mode trampoline (the test was getting false passes without ever reaching cet_shstk_far_ret() due to getting the expected #CP).
- Add testcases to verify KVM rejects emulation as expected.
- Add a comment explaining the SHSTK PTE magic (I forgot about the magic and spent a long time trying to figure out how the user_mode trampoline was succeeding if the SHSTK wasn't writable, *sigh*)
Chao Gao (7):
x86: cet: Remove unnecessary memory zeroing for shadow stack
x86: cet: Directly check for #CP exception in run_in_user()
x86: cet: Validate #CP error code
x86: cet: Use report_skip()
x86: cet: Drop unnecessary casting
x86: cet: Validate writing unaligned values to SSP MSR causes #GP
x86: cet: Validate CET states during VMX transitions
Mathias Krause (5):
x86: cet: Make shadow stack less fragile
x86: cet: Simplify IBT test
x86: cet: Use symbolic values for the #CP error codes
x86: cet: Test far returns too
x86: Avoid top-most page for vmalloc on x86-64
Sean Christopherson (4):
x86/run_in_user: Add an "end branch" marker on the user_mode
destination
x86/cet: Run SHSTK and IBT tests as appropriate if either feature is
supported
x86/cet: Drop the "intel_" prefix from the CET testcase x86/cet: Add testcases to verify KVM rejects emulation of CET
instructions
Yang Weijiang (1):
x86: cet: Pass virtual addresses to invlpg
lib/x86/msr.h | 1 + lib/x86/usermode.c | 7 ++ lib/x86/vm.c | 2 + x86/cet.c | 272 +++++++++++++++++++++++++++++++++++---------- x86/lam.c | 10 +- x86/unittests.cfg | 9 +- x86/vmx.h | 8 +- x86/vmx_tests.c | 81 ++++++++++++++ 8 files changed, 326 insertions(+), 64 deletions(-)
base-commit: af582a4ebaf7828c200dc7150aa0dbccb60b08a7
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 1/17 x86/run_in_user: Add an "end branch" marker on the user_mode destination
Add an endbr64 at the user_mode "entry point" so that run_in_user() can be used when CET's Indirect Branch Tracking is enabled.
--- lib/x86/usermode.c | 3 +++ 1 file changed, 3 insertions(+)
lib/x86/usermode.c +3 -0
@@ -68,6 +68,9 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector, "iretq\n" "user_mode:\n\t" +#ifdef __x86_64__ + "endbr64\n\t" +#endif/* Back up volatile registers before invoking func */ "push %%rcx\n\t" "push %%rdx\n\t"
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Re: [kvm-unit-tests PATCH v3 01/17] x86/run_in_user: Add an "end branch" marker on the user_mode destination
Quoted 2 lines · Sean Christopherson
Add an endbr64 at the
user_mode"entry point" so thatrun_in_user()can be used when CET's Indirect Branch Tracking is enabled.
I don't think that's needed, as 'user_mode' is branched to via IRETQ and that isn't covered by IBT -- nor is any other RET instruction.
Quoted 11 lines
Signed-off-by Sean Christopherson <[email protected]>--- lib/x86/usermode.c | 3 +++ 1 file changed, 3 insertions(+)lib/x86/usermode.c +0
-0
lib/x86/usermode.c uint64_t run_in_user(usermode_func func, unsigned int fault_vector, · line 73
Quoted 6 lines
lib/x86/usermode.c +3
-0@@ -68,6 +68,9 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,"iretq\n" "user_mode:\n\t" +#ifdef __x86_64__ +"endbr64\n\t" +#endif
The thing is, this ENDBR64 is actually masking a missing cleanup step in the IBT tests. The first failing IBT test will make the CET_U IBT tracker state get stuck in the WAIT_FOR_ENDBRANCH state. This means, every time we return to userland (and thereby implicitly switching to the CET_U state again), it wants to see an ENDBR64 first or it will directly trigger a #CP(3) again. This ENDBR64 will please that demand and make it transition to IDLE and allow executing the test. However, it's really the old test that should have fixed the tracker state and not a blanket ENDBR64 when entering usermode.
Attached is a patch on top of [1] that does that but is, admitted, a little hacky and evolved. However, it shows that above ENDBR64 is, in fact, not needed.
Thanks,
Mathias
lib/x86/usermode.c uint64_t run_in_user(usermode_func func, unsigned int fault_vector, · line 76
Quoted 3 lines
lib/x86/usermode.c +0
-0/* Back up volatile registers before invoking func */ "push %%rcx\n\t" "push %%rdx\n\t"
From 92f2a8000b2c6954825bde451f311eb513cef0e8 Mon Sep 17 00:00:00 2001
From: Mathias Krause <[email protected]>
Date: Fri, 14 Nov 2025 16:45:54 +0100
Subject: [kvm-unit-tests PATCH] x86: cet: Reset IBT tracker state on #CP
violations
Reset the IBT tracker state back to IDLE on #CP violations to not influence follow-up tests with a poisoned starting state.
--- lib/x86/usermode.h | 12 ++++++++++-- lib/x86/usermode.c | 13 ++++++++----- x86/cet.c | 25 ++++++++++++++++++++++--- 3 files changed, 40 insertions(+), 10 deletions(-)
lib/x86/usermode.h +10 -2
@@ -20,11 +20,19 @@ typedef uint64_t (*usermode_func)(void);* Supports running functions with up to 4 arguments. * fault_vector: exception vector that might get thrown during the function. * raised_vector: outputs true if exception occurred. +* ex_extra: additional handler to call when handling traps related to fault_vector * * returns: return value returned by function, or 0 if an exception occurred. */ +uint64_t run_in_user_ex (usermode_func func,unsigned int fault_vector, +uint64_t arg1,uint64_t arg2,uint64_t arg3, +uint64_t arg4,bool *raised_vector,handler ex_extra); + +static inline uint64_t run_in_user (usermode_func func,unsigned int fault_vector,uint64_t arg1,uint64_t arg2,uint64_t arg3, -uint64_t arg4,bool *raised_vector); - +uint64_t arg4,bool *raised_vector) +{ +return run_in_user_ex (func, fault_vector, arg1, arg2, arg3, arg4, raised_vector,NULL ); +}#endif
lib/x86/usermode.c +8 -5
@@ -21,12 +21,17 @@ static void restore_exec_to_jmpbuf(void)longjmp (jmpbuf,1 ); } +static handler extra_ex; +static void restore_exec_to_jmpbuf_exception_handler (struct ex_regs *regs) {this_cpu_write_exception_vector (regs->vector);this_cpu_write_exception_rflags_rf ((regs->rflags >>16 ) &1 );this_cpu_write_exception_error_code (regs->error_code); +if (extra_ex) +extra_ex (regs); +/* longjmp must happen after iret, so do not do it now. */ regs->rip = (unsigned long )&restore_exec_to_jmpbuf; regs->cs =KERNEL_CS ;
@@ -35,9 +40,9 @@ static void restore_exec_to_jmpbuf_exception_handler(struct ex_regs *regs)#endif } -uint64_t run_in_user (usermode_func func,unsigned int fault_vector, +uint64_t run_in_user_ex (usermode_func func,unsigned int fault_vector,uint64_t arg1,uint64_t arg2,uint64_t arg3, -uint64_t arg4,bool *raised_vector) +uint64_t arg4,bool *raised_vector,handler ex_extra) {extern char ret_to_kernel;volatile uint64_t rax =0 ;
@@ -45,6 +50,7 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,handler old_ex; *raised_vector =0 ; + extra_ex = ex_extra;set_idt_entry (RET_TO_KERNEL_IRQ , &ret_to_kernel,3 ); old_ex =handle_exception (fault_vector, restore_exec_to_jmpbuf_exception_handler);
@@ -72,9 +78,6 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,"iretq\n" "user_mode:\n\t" -#ifdef __x86_64__ - "endbr64\n\t" -#endif/* Back up volatile registers before invoking func */ "push %%rcx\n\t" "push %%rdx\n\t"
x86/cet.c +22 -3
@@ -1,4 +1,3 @@ -#include "libcflat.h" #include "x86/desc.h" #include "x86/processor.h"
@@ -192,6 +191,10 @@ static uint64_t cet_ibt_emulation(void)#define CET_ENABLE_SHSTK BIT(0)#define CET_ENABLE_IBT BIT(2)#define CET_ENABLE_NOTRACK BIT(4) +#define CET_IBT_SUPPRESS BIT(10) +#define CET_IBT_TRACKER_STATE BIT(11) +#define IBT_TRACKER_IDLE 0 +#define IBT_TRACKER_WAIT_FOR_ENDBRANCH BIT(11)static void test_shstk (void ) {
@@ -244,6 +247,22 @@ static void test_shstk(void)report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); } +static void ibt_tracker_fixup (struct ex_regs *regs) +{ +u64 cet_u =rdmsr (MSR_IA32_U_CET ); + +/* +* Switch the IBT tracker state to IDLE to have a clean state for +* following tests. +*/ +if ((cet_u &CET_IBT_TRACKER_STATE ) ==IBT_TRACKER_WAIT_FOR_ENDBRANCH ) { + cet_u &= ~IBT_TRACKER_WAIT_FOR_ENDBRANCH ; +printf ("CET: suppressing IBT WAIT_FOR_ENDBRANCH state at RIP: %lx\n" , + regs->rip); +wrmsr (MSR_IA32_U_CET , cet_u); + } +} +static void test_ibt (void ) {uint64_t l;
@@ -257,12 +276,12 @@ static void test_ibt(void)/* Enable indirect-branch tracking (notrack handling for jump tables) */ wrmsr (MSR_IA32_U_CET ,CET_ENABLE_IBT |CET_ENABLE_NOTRACK ); -run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); +run_in_user_ex (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc, ibt_tracker_fixup);report (rvc &&exception_error_code () ==CP_ERR_ENDBR ,"Indirect-branch tracking test" );if (is_fep_available && - ((l =run_in_user (cet_ibt_emulation,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc)) || rvc)) + ((l =run_in_user_ex (cet_ibt_emulation,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc, ibt_tracker_fixup)) || rvc))report_fail ("Forced emulation with IBT generated %s(%u) at line %lu" ,exception_mnemonic (exception_vector ()),exception_error_code (), l);
Signature · 1 line
2.51.0
Sean Christopherson
reviewerRe: [kvm-unit-tests PATCH v3 01/17] x86/run_in_user: Add an "end branch" marker on the user_mode destination
Quoted 35 lines · Mathias Krause
Quoted 2 lines · Sean Christopherson
Add an endbr64 at the
user_mode"entry point" so thatrun_in_user()can be used when CET's Indirect Branch Tracking is enabled.I don't think that's needed, as '
user_mode' is branched to via IRETQ and that isn't covered by IBT -- nor is any other RET instruction.Quoted 11 lines
Signed-off-by Sean Christopherson <[email protected]>--- lib/x86/usermode.c | 3 +++ 1 file changed, 3 insertions(+)lib/x86/usermode.c +0
-0lib/x86/usermode.c uint64_t run_in_user(usermode_func func, unsigned int fault_vector, · line 73
Quoted 6 lines
lib/x86/usermode.c +3
-0@@ -68,6 +68,9 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,"iretq\n" "user_mode:\n\t" +#ifdef __x86_64__ +"endbr64\n\t" +#endifThe thing is, this ENDBR64 is actually masking a missing cleanup step in the IBT tests. The first failing IBT test will make the
CET_UIBT tracker state get stuck in theWAIT_FOR_ENDBRANCHstate. This means, every time we return to userland (and thereby implicitly switching to theCET_Ustate again), it wants to see an ENDBR64 first or it will directly trigger a #CP(3) again. This ENDBR64 will please that demand and make it transition to IDLE and allow executing the test. However, it's really the old test that should have fixed the tracker state and not a blanket ENDBR64 when entering usermode.
Quoted 3 lines
Attached is a patch on top of [1] that does that but is, admitted, a little hacky and evolved. However, it shows that above ENDBR64 is, in fact, not needed.
You say hacky, I say clever and correct. :-)
Quoted 10 lines
x86/cet.c +0
-1@@ -1,4 +1,3 @@ -#include "libcflat.h" #include "x86/desc.h" #include "x86/processor.h"
x86/cet.c static uint64_t cet_ibt_emulation(void) · line 197
Quoted 7 lines
x86/cet.c +4
-0@@ -192,6 +191,10 @@ static uint64_t cet_ibt_emulation(void)#define CET_ENABLE_SHSTK BIT(0)#define CET_ENABLE_IBT BIT(2)#define CET_ENABLE_NOTRACK BIT(4) +#define CET_IBT_SUPPRESS BIT(10) +#define CET_IBT_TRACKER_STATE BIT(11) +#define IBT_TRACKER_IDLE 0 +#define IBT_TRACKER_WAIT_FOR_ENDBRANCH BIT(11)
For this, I think it makes sense to diverge slightly from the SDM and just do
#define CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH BIT(11)
because...
Quoted 6 lines
x86/cet.c +0
-0static void test_shstk (void ) {@@ -244,6 +247,22 @@ static void test_shstk(void)report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); }
x86/cet.c static void test_shstk(void) · line 259
Quoted 10 lines
x86/cet.c +10
-0+static void ibt_tracker_fixup (struct ex_regs *regs) +{ +u64 cet_u =rdmsr (MSR_IA32_U_CET ); + +/* +* Switch the IBT tracker state to IDLE to have a clean state for +* following tests. +*/ +if ((cet_u &CET_IBT_TRACKER_STATE ) ==IBT_TRACKER_WAIT_FOR_ENDBRANCH ) { + cet_u &= ~IBT_TRACKER_WAIT_FOR_ENDBRANCH ;
...this is quite weird/confusing. It relies on CET_IBT_TRACKER_STATE being a single bit, and "(x & y) == z)" is very un-idiomatic for a single bit.
Quoted 5 lines
x86/cet.c +5
-0+printf ("CET: suppressing IBT WAIT_FOR_ENDBRANCH state at RIP: %lx\n" , + regs->rip); +wrmsr (MSR_IA32_U_CET , cet_u); + } +}
Re: [kvm-unit-tests PATCH v3 01/17] x86/run_in_user: Add an "end branch" marker on the user_mode destination
Quoted 39 lines · Sean Christopherson
Quoted 5 lines · Mathias Krause
[...]
Attached is a patch on top of [1] that does that but is, admitted, a little hacky and evolved. However, it shows that above ENDBR64 is, in fact, not needed.
You say hacky, I say clever and correct. :-)
Quoted 10 lines
x86/cet.c +0
-1@@ -1,4 +1,3 @@ -#include "libcflat.h" #include "x86/desc.h" #include "x86/processor.h" x86/cet.c static uint64_t cet_ibt_emulation(void) · line 197
Quoted 7 lines
x86/cet.c +4
-0@@ -192,6 +191,10 @@ static uint64_t cet_ibt_emulation(void)#define CET_ENABLE_SHSTK BIT(0)#define CET_ENABLE_IBT BIT(2)#define CET_ENABLE_NOTRACK BIT(4) +#define CET_IBT_SUPPRESS BIT(10) +#define CET_IBT_TRACKER_STATE BIT(11) +#define IBT_TRACKER_IDLE 0 +#define IBT_TRACKER_WAIT_FOR_ENDBRANCH BIT(11)For this, I think it makes sense to diverge slightly from the SDM and just do
#define CET_IBT_TRACKER_WAIT_FOR_ENDBRANCH BIT(11)because...
Quoted 6 lines
x86/cet.c +0
-0static void test_shstk (void ) {@@ -244,6 +247,22 @@ static void test_shstk(void)report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); }
Quoted 13 lines
x86/cet.c static void test_shstk(void) · line 259
Quoted 10 lines
x86/cet.c +10
-0+static void ibt_tracker_fixup (struct ex_regs *regs) +{ +u64 cet_u =rdmsr (MSR_IA32_U_CET ); + +/* +* Switch the IBT tracker state to IDLE to have a clean state for +* following tests. +*/ +if ((cet_u &CET_IBT_TRACKER_STATE ) ==IBT_TRACKER_WAIT_FOR_ENDBRANCH ) { + cet_u &= ~IBT_TRACKER_WAIT_FOR_ENDBRANCH ;...this is quite weird/confusing. It relies on
CET_IBT_TRACKER_STATEbeing a single bit, and "(x & y) == z)" is very un-idiomatic for a single bit.
Well, it doesn't rely on IBT_TRACKER_WAIT_FOR_ENDBRANCH being a single bit, but a value with all ones.
Essentially what I wanted to do was (in pseudo-code):
if cet_u.tracker_state == WAIT_FOR_ENDBRANCH:
cet_u.tracker_state = IDLE
wrmsr(MSR_IA32_U_CET, cet_u)
I tried to make clear that it's the IBT tracker state, the code is interested in. But yeah, it's a single bit with only two states so just testing and toggling that single bit is fine.
Initially I thought that setting CET_IBT_SUPPRESS is needed was well, but it's not. It's related to the "legacy handling" which is yet another can of worms we haven't opened yet ;)
Quoted 6 lines
Quoted 6 lines
x86/cet.c +5
-0+printf ("CET: suppressing IBT WAIT_FOR_ENDBRANCH state at RIP: %lx\n" , + regs->rip); +wrmsr (MSR_IA32_U_CET , cet_u); + } +}
Sean Christopherson
reviewerPATCH v3 2/17 x86: cet: Pass virtual addresses to invlpg
In reply to Sean ChristophersonFrom: Yang Weijiang <[email protected]>
Correct the parameter passed to invlpg.
The invlpg instruction should take a virtual address instead of a physical address when flushing TLBs. Using shstk_phys results in TLBs associated with the virtual address (shstk_virt) not being flushed, and the virtual address may not be treated as a shadow stack address if there is a stale TLB. So, subsequent shadow stack accesses to shstk_virt may cause a #PF, which terminates the test unexpectedly.
--- x86/cet.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
x86/cet.c +1 -1
@@ -100,7 +100,7 @@ int main(int ac, char **av) *ptep |=PT_DIRTY_MASK ;/* Flush the paging cache. */ -invlpg ((void *)shstk_phys); +invlpg ((void *)shstk_virt);/* Enable shadow-stack protection */ wrmsr (MSR_IA32_U_CET ,ENABLE_SHSTK_BIT );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 3/17 x86: cet: Remove unnecessary memory zeroing for shadow stack
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
Skip mapping the shadow stack as a writable page and the redundant memory zeroing.
Currently, the shadow stack is allocated using alloc_page(), then mapped as a writable page, zeroed, and finally mapped as a shadow stack page. The memory zeroing is redundant as alloc_page() already does that.
This also eliminates the need for invlpg, as the shadow stack is no longer mapped writable.
--- x86/cet.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-)
x86/cet.c +7 -12
@@ -67,7 +67,6 @@ int main(int ac, char **av) {char *shstk_virt;unsigned long shstk_phys; -unsigned long *ptep;pteval_t pte =0 ;bool rvc;
@@ -89,18 +88,14 @@ int main(int ac, char **av) shstk_virt =alloc_vpage (); shstk_phys = (unsigned long )virt_to_phys (alloc_page ()); -/* Install the new page. */ - pte = shstk_phys |PT_PRESENT_MASK |PT_WRITABLE_MASK |PT_USER_MASK ; +/* +* Install a mapping for the shadow stack page. Shadow stack pages are +* denoted by an "impossible" combination of a !WRITABLE, DIRTY PTE +* (writes from CPU for shadow stack operations are allowed, but writes +* from software are not). +*/ + pte = shstk_phys |PT_PRESENT_MASK |PT_USER_MASK |PT_DIRTY_MASK ;install_pte (current_page_table (),1 , shstk_virt, pte,0 ); -memset (shstk_virt,0x0 ,PAGE_SIZE ); - -/* Mark it as shadow-stack page. */ - ptep =get_pte_level (current_page_table (), shstk_virt,1 ); - *ptep &= ~PT_WRITABLE_MASK ; - *ptep |=PT_DIRTY_MASK ; - -/* Flush the paging cache. */ -invlpg ((void *)shstk_virt);/* Enable shadow-stack protection */ wrmsr (MSR_IA32_U_CET ,ENABLE_SHSTK_BIT );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 4/17 x86: cet: Directly check for #CP exception in run_in_user()
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
Current CET tests validate if a #CP exception is raised by registering a #CP handler. This handler counts the #CP exceptions and raises a #GP exception, which is then caught by the run_in_user() infrastructure to switch back to the kernel. This is convoluted.
Catch the #CP exception directly by run_in_user() to avoid the manual counting of #CP exceptions and the #CP->#GP dance.
--- x86/cet.c | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-)
x86/cet.c +4 -19
@@ -8,9 +8,6 @@#include "alloc_page.h" #include "fault_test.h" -static int cp_count; -static unsigned long invalid_offset =0xffffffffffffff ; -static u64 cet_shstk_func (void ) {unsigned long *ret_addr, *ssp;
@@ -54,15 +51,6 @@ static u64 cet_ibt_func(void)#define ENABLE_SHSTK_BIT 0x1#define ENABLE_IBT_BIT 0x4 -static void handle_cp (struct ex_regs *regs) -{ - cp_count++; -printf ("In #CP exception handler, error_code = 0x%lx\n" , - regs->error_code); -/* Below jmp is expected to trigger #GP */ - asm("jmpq *%0" : :"m" (invalid_offset)); -} -int main (int ac,char **av) {char *shstk_virt;
@@ -70,7 +58,6 @@ int main(int ac, char **av)pteval_t pte =0 ;bool rvc; - cp_count =0 ;if (!this_cpu_has (X86_FEATURE_SHSTK )) {printf ("SHSTK not enabled\n" );return report_summary ();
@@ -82,7 +69,6 @@ int main(int ac, char **av) }setup_vm (); -handle_exception (CP_VECTOR , handle_cp);/* Allocate one page for shadow-stack. */ shstk_virt =alloc_vpage ();
@@ -107,15 +93,14 @@ int main(int ac, char **av)write_cr4 (read_cr4 () |X86_CR4_CET );printf ("Unit test for CET user mode...\n" ); -run_in_user ((usermode_func )cet_shstk_func,GP_VECTOR ,0 ,0 ,0 ,0 , &rvc); -report (cp_count ==1 ,"Completed shadow-stack protection test successfully." ); - cp_count =0 ; +run_in_user ((usermode_func )cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); +report (rvc,"Shadow-stack protection test." );/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT ); -run_in_user ((usermode_func )cet_ibt_func,GP_VECTOR ,0 ,0 ,0 ,0 , &rvc); -report (cp_count ==1 ,"Completed Indirect-branch tracking test successfully." ); +run_in_user ((usermode_func )cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); +report (rvc,"Indirect-branch tracking test." );write_cr4 (read_cr4 () & ~X86_CR4_CET );wrmsr (MSR_IA32_U_CET ,0 );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 5/17 x86: cet: Validate #CP error code
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
The #CP exceptions include an error code that provides additional information about how the exception occurred. Previously, CET tests simply printed these error codes without validation.
Enhance the CET tests to validate the #CP error code.
This requires the run_in_user() infrastructure to catch the exception vector, error code, and rflags, similar to what check_exception_table() does.
--- lib/x86/usermode.c | 4 ++++ x86/cet.c | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-)
lib/x86/usermode.c +4 -0
@@ -23,6 +23,10 @@ static void restore_exec_to_jmpbuf(void)static void restore_exec_to_jmpbuf_exception_handler (struct ex_regs *regs) { +this_cpu_write_exception_vector (regs->vector); +this_cpu_write_exception_rflags_rf ((regs->rflags >>16 ) &1 ); +this_cpu_write_exception_error_code (regs->error_code); +/* longjmp must happen after iret, so do not do it now. */ regs->rip = (unsigned long )&restore_exec_to_jmpbuf; regs->cs =KERNEL_CS ;
x86/cet.c +2 -2
@@ -94,13 +94,13 @@ int main(int ac, char **av)printf ("Unit test for CET user mode...\n" );run_in_user ((usermode_func )cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); -report (rvc,"Shadow-stack protection test." ); +report (rvc &&exception_error_code () ==1 ,"Shadow-stack protection test." );/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT );run_in_user ((usermode_func )cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); -report (rvc,"Indirect-branch tracking test." ); +report (rvc &&exception_error_code () ==3 ,"Indirect-branch tracking test." );write_cr4 (read_cr4 () & ~X86_CR4_CET );wrmsr (MSR_IA32_U_CET ,0 );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 6/17 x86: cet: Use report_skip()
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
report_skip() function is preferred for skipping inapplicable tests when the necessary hardware features are unavailable. For example, with this patch applied, the test output is as follows if IBT is not supported:
SKIP: IBT not enabled
SUMMARY: 1 tests, 1 skipped
Previously, it printed:
IBT not enabled
SUMMARY: 0 tests
--- x86/cet.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
x86/cet.c +2 -2
@@ -59,12 +59,12 @@ int main(int ac, char **av)bool rvc;if (!this_cpu_has (X86_FEATURE_SHSTK )) { -printf ("SHSTK not enabled\n" ); +report_skip ("SHSTK not enabled" );return report_summary (); }if (!this_cpu_has (X86_FEATURE_IBT )) { -printf ("IBT not enabled\n" ); +report_skip ("IBT not enabled" );return report_summary (); }
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 7/17 x86: cet: Drop unnecessary casting
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
cet_shstk_func() and cet_ibt_func() have the same type as usermode_func. So, remove the unnecessary casting.
--- x86/cet.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
x86/cet.c +4 -4
@@ -8,7 +8,7 @@#include "alloc_page.h" #include "fault_test.h" -static u64 cet_shstk_func (void ) +static uint64_t cet_shstk_func (void ) {unsigned long *ret_addr, *ssp;
@@ -31,7 +31,7 @@ static u64 cet_shstk_func(void)return 0 ; } -static u64 cet_ibt_func (void ) +static uint64_t cet_ibt_func (void ) {/* * In below assembly code, the first instruction at label 2 is not
@@ -93,13 +93,13 @@ int main(int ac, char **av)write_cr4 (read_cr4 () |X86_CR4_CET );printf ("Unit test for CET user mode...\n" ); -run_in_user ((usermode_func )cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); +run_in_user (cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==1 ,"Shadow-stack protection test." );/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT ); -run_in_user ((usermode_func )cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); +run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==3 ,"Indirect-branch tracking test." );write_cr4 (read_cr4 () & ~X86_CR4_CET );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 8/17 x86: cet: Validate writing unaligned values to SSP MSR causes #GP
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
Validate that writing invalid values to SSP MSRs triggers a #GP exception. This verifies that necessary validity checks are performed by the hardware or the underlying VMM.
--- x86/cet.c | 5 +++++ 1 file changed, 5 insertions(+)
x86/cet.c +5 -0
@@ -56,6 +56,7 @@ int main(int ac, char **av)char *shstk_virt;unsigned long shstk_phys;pteval_t pte =0 ; +u8 vector;bool rvc;if (!this_cpu_has (X86_FEATURE_SHSTK )) {
@@ -105,5 +106,9 @@ int main(int ac, char **av)write_cr4 (read_cr4 () & ~X86_CR4_CET );wrmsr (MSR_IA32_U_CET ,0 ); +/* SSP should be 4-Byte aligned */ + vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 ); +report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); +return report_summary (); }
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 9/17 x86: cet: Validate CET states during VMX transitions
In reply to Sean ChristophersonFrom: Chao Gao <[email protected]>
Add tests to verify that CET states are correctly handled during VMX transitions.
The following behaviors are verified:
1. Host states are loaded from VMCS iff "Load CET" VM-exit control is set
2. Guest states are loaded from VMCS iff "Load CET" VM-entry control is set
3. Guest states are saved to VMCS during VM exits unconditionally
4. Invalid guest or host CET states leads to VM entry failures.
--- lib/x86/msr.h | 1 + x86/unittests.cfg | 7 ++++ x86/vmx.h | 8 +++-- x86/vmx_tests.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 95 insertions(+), 2 deletions(-)
lib/x86/msr.h +1 -0
@@ -296,6 +296,7 @@#define MSR_IA32_FEATURE_CONTROL 0x0000003a#define MSR_IA32_TSC_ADJUST 0x0000003b#define MSR_IA32_U_CET 0x000006a0 +#define MSR_IA32_S_CET 0x000006a2#define MSR_IA32_PL3_SSP 0x000006a7#define MSR_IA32_PKRS 0x000006e1
x86/unittests.cfg +7 -0
@@ -453,6 +453,13 @@ arch = x86_64 groups = vmx nested_exception check = /sys/module/kvm_intel/parameters/allow_smaller_maxphyaddr=Y +[vmx_cet_test] +file = vmx.flat +extra_params = -cpu max,+vmx -append "vmx_cet_test" +arch = x86_64 +groups = vmx +timeout = 240 + [debug] file = debug.flat arch = x86_64
x86/vmx.h +6 -2
@@ -356,6 +356,7 @@ enum Encoding {GUEST_PENDING_DEBUG =0x6822ul ,GUEST_SYSENTER_ESP =0x6824ul ,GUEST_SYSENTER_EIP =0x6826ul , +GUEST_S_CET =0x6828ul ,/* Natural-Width Host State Fields */ HOST_CR0 = 0x6c00ul,
@@ -369,7 +370,8 @@ enum Encoding {HOST_SYSENTER_ESP =0x6c10ul ,HOST_SYSENTER_EIP =0x6c12ul ,HOST_RSP =0x6c14ul , - HOST_RIP = 0x6c16ul +HOST_RIP =0x6c16ul , + HOST_S_CET = 0x6c18ul, };#define VMX_ENTRY_FAILURE (1ul << 31)
@@ -449,6 +451,7 @@ enum Ctrl_exi {EXI_SAVE_EFER =1UL <<20 ,EXI_LOAD_EFER =1UL <<21 ,EXI_SAVE_PREEMPT =1UL <<22 , + EXI_LOAD_CET = 1UL << 28, }; enum Ctrl_ent {
@@ -457,7 +460,8 @@ enum Ctrl_ent {ENT_LOAD_PERF =1UL <<13 ,ENT_LOAD_PAT =1UL <<14 ,ENT_LOAD_EFER =1UL <<15 , -ENT_LOAD_BNDCFGS =1UL <<16 +ENT_LOAD_BNDCFGS =1UL <<16 , + ENT_LOAD_CET = 1UL << 20, }; enum Ctrl_pin {
x86/vmx_tests.c +81 -0
92 lines not on this page · raw message
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Re: [kvm-unit-tests PATCH v3 09/17] x86: cet: Validate CET states during VMX transitions
Quoted 7 lines · Sean Christopherson
[...]
x86/unittests.cfg +0
-0
x86/unittests.cfg arch = x86_64 · line 459
Quoted 7 lines
x86/unittests.cfg +3
-0@@ -453,6 +453,13 @@ arch = x86_64 groups = vmx nested_exception check = /sys/module/kvm_intel/parameters/allow_smaller_maxphyaddr=Y +[vmx_cet_test] +file = vmx.flat +extra_params = -cpu max,+vmx -append "vmx_cet_test"
This is missing the fixup from
https://lore.kernel.org/kvm/[email protected]/:
-extra_params = -cpu max,+vmx -append "vmx_cet_test" +test_args = "vmx_cet_test"
+qemu_params = -cpu max,+vmx
Quoted 8 lines
x86/unittests.cfg +4
-0+arch = x86_64 +groups = vmx +timeout = 240 + [debug] file = debug.flatarch =
x86_64
[...]
Thanks,
Mathias
Sean Christopherson
reviewerPATCH v3 10/17 x86: cet: Make shadow stack less fragile
In reply to Sean ChristophersonFrom: Mathias Krause <[email protected]>
The CET shadow stack test has certain assumptions about the code, namely that it was compiled with frame pointers enabled and the return address won't be 0xdeaddead.
Make the code less fragile by actually lifting these assumptions to (1) explicitly mention the dependency to the frame pointer by making us of __builtin_frame_address(0) and (2) modify the return address by toggling bits instead of writing a fixed value. Also ensure that write will actually be generated by the compiler by making it a 'volatile' write.
--- x86/cet.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
x86/cet.c +4 -4
@@ -10,14 +10,14 @@static uint64_t cet_shstk_func (void ) { -unsigned long *ret_addr, *ssp; +unsigned long *ret_addr =__builtin_frame_address (0 ) +sizeof (void *); +unsigned long *ssp;/* rdsspq %rax */ asmvolatile (".byte 0xf3, 0x48, 0x0f, 0x1e, 0xc8" :"=a" (ssp)); - asm("movq %%rbp,%0" :"=r" (ret_addr));printf ("The return-address in shadow-stack = 0x%lx, in normal stack = 0x%lx\n" , - *ssp, *(ret_addr +1 )); + *ssp, *ret_addr);/* * In below line, it modifies the return address, it'll trigger #CP
@@ -26,7 +26,7 @@ static uint64_t cet_shstk_func(void)* when HW detects the violation. */ printf ("Try to temper the return-address, this causes #CP on returning...\n" ); - *(ret_addr +1 ) =0xdeaddead ; + *(volatile unsigned long *)ret_addr ^=0xdeaddead ;return 0 ; }
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 11/17 x86: cet: Simplify IBT test
In reply to Sean ChristophersonFrom: Mathias Krause <[email protected]>
The inline assembly of cet_ibt_func() does unnecessary things and doesn't mention the clobbered registers.
Fix that by reducing the code to what's needed (an indirect jump to a target lacking the ENDBR instruction) and passing and output register variable for it.
--- x86/cet.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)
x86/cet.c +5 -6
@@ -33,18 +33,17 @@ static uint64_t cet_shstk_func(void)static uint64_t cet_ibt_func (void ) { +unsigned long tmp;/* * In below assembly code, the first instruction at label 2 is not * endbr64, it'll trigger #CP with error code 0x3, and the execution * is terminated when HW detects the violation. */ printf ("No endbr64 instruction at jmp target, this triggers #CP...\n" ); - asmvolatile ("movq $2, %rcx\n" -"dec %rcx\n" -"leaq 2f(%rip), %rax\n" -"jmp *%rax \n" -"2:\n" -"dec %rcx\n" ); + asmvolatile ("leaq 2f(%%rip), %0\n\t" +"jmpq *%0\n\t" +"2:" + :"=r" (tmp));return 0 ; }
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 12/17 x86: cet: Use symbolic values for the #CP error codes
In reply to Sean ChristophersonFrom: Mathias Krause <[email protected]>
Use symbolic names for the #CP exception error codes.
--- x86/cet.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
x86/cet.c +12 -3
@@ -47,6 +47,13 @@ static uint64_t cet_ibt_func(void)return 0 ; } +#define CP_ERR_NEAR_RET 0x0001 +#define CP_ERR_FAR_RET 0x0002 +#define CP_ERR_ENDBR 0x0003 +#define CP_ERR_RSTORSSP 0x0004 +#define CP_ERR_SETSSBSY 0x0005 +#define CP_ERR_ENCL BIT(15) +#define ENABLE_SHSTK_BIT 0x1#define ENABLE_IBT_BIT 0x4
@@ -92,15 +99,17 @@ int main(int ac, char **av)/* Enable CET master control bit in CR4. */ write_cr4 (read_cr4 () |X86_CR4_CET ); -printf ("Unit test for CET user mode...\n" ); +printf ("Unit tests for CET user mode...\n" );run_in_user (cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); -report (rvc &&exception_error_code () ==1 ,"Shadow-stack protection test." ); +report (rvc &&exception_error_code () ==CP_ERR_NEAR_RET , +"NEAR RET shadow-stack protection test" );/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT );run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); -report (rvc &&exception_error_code () ==3 ,"Indirect-branch tracking test." ); +report (rvc &&exception_error_code () ==CP_ERR_ENDBR , +"Indirect-branch tracking test" );write_cr4 (read_cr4 () & ~X86_CR4_CET );wrmsr (MSR_IA32_U_CET ,0 );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 13/17 x86: cet: Test far returns too
In reply to Sean ChristophersonFrom: Mathias Krause <[email protected]>
Add a test for far returns which has a dedicated error code.
--- x86/cet.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
x86/cet.c +32 -0
@@ -31,6 +31,34 @@ static uint64_t cet_shstk_func(void)return 0 ; } +static uint64_t cet_shstk_far_ret (void ) +{ +struct far_pointer32 fp = { + .offset = (uintptr_t )&&far_func, + .selector =USER_CS , + }; + +if (fp.offset != (uintptr_t )&&far_func) { +printf ("Code address too high.\n" ); +return -1 ; + } + +printf ("Try to temper the return-address of far-called function...\n" ); + +/* The NOP isn't superfluous, the called function tries to skip it. */ + asm goto ("lcall *%0; nop" : :"m" (fp) : : far_func); + +printf ("Uhm... how did we get here?! This should have #CP'ed!\n" ); + +return 0 ; +far_func: + asmvolatile (/* mess with the ret addr, make it point past the NOP */ +"incq (%rsp)\n\t" +/* 32-bit return, just as we have been called */ +"lretl" ); +__builtin_unreachable (); +} +static uint64_t cet_ibt_func (void ) {unsigned long tmp;
@@ -104,6 +132,10 @@ int main(int ac, char **av)report (rvc &&exception_error_code () ==CP_ERR_NEAR_RET ,"NEAR RET shadow-stack protection test" ); +run_in_user (cet_shstk_far_ret,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); +report (rvc &&exception_error_code () ==CP_ERR_FAR_RET , +"FAR RET shadow-stack protection test" ); +/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT );
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 14/17 x86: Avoid top-most page for vmalloc on x86-64
In reply to Sean ChristophersonFrom: Mathias Krause <[email protected]>
The x86-64 implementation if setup_mmu() doesn't initialize 'vfree_top' and leaves it at its zero-value. This isn't wrong per se, however, it leads to odd configurations when the first vmalloc/vmap page gets allocated. It'll be the very last page in the virtual address space -- which is an interesting corner case -- but its boundary will probably wrap. It does so, for CET's shadow stack, at least, which loads the shadow stack pointer with the base address of the mapped page plus its size, i.e. 0xffffffff_fffff000 + 4096, which wraps to 0x0.
The CPU seems to handle such configurations just fine. However, it feels odd to set the shadow stack pointer to "NULL".
To avoid the wrapping, ignore the top most page by initializing 'vfree_top' to just one page below.
--- lib/x86/vm.c | 2 ++ x86/lam.c | 10 +++++----- 2 files changed, 7 insertions(+), 5 deletions(-)
lib/x86/vm.c +2 -0
@@ -191,6 +191,8 @@ void *setup_mmu(phys_addr_t end_of_memory, void *opt_mask) end_of_memory = (1ul <<32 );/* map mmio 1:1 */ setup_mmu_range (cr3,0 , end_of_memory); +/* skip the last page for out-of-bound and wrap-around reasons */ +init_alloc_vpage ((void *)(~(PAGE_SIZE -1 )));#else setup_mmu_range (cr3,0 , (2ul <<30 ));setup_mmu_range (cr3,3ul <<30 , (1ul <<30 ));
x86/lam.c +5 -5
@@ -197,11 +197,11 @@ static void test_lam_sup(void)int vector;/* -* KUT initializes vfree_top to 0 for X86_64, and each virtual address -* allocation decreases the size from vfree_top. It's guaranteed that -* the return value of alloc_vpage() is considered as kernel mode -* address and canonical since only a small amount of virtual address -* range is allocated in this test. +* KUT initializes vfree_top to -PAGE_SIZE for X86_64, and each virtual +* address allocation decreases the size from vfree_top. It's +* guaranteed that the return value of alloc_vpage() is considered as +* kernel mode address and canonical since only a small amount of +* virtual address range is allocated in this test. */ vaddr =alloc_vpage (); vaddr_mmio =alloc_vpage ();
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 15/17 x86/cet: Run SHSTK and IBT tests as appropriate if either feature is supported
In reply to Sean ChristophersonRun the SHSTK and IBT tests if their respective feature is supported, as nothing in the architecture requires both features to be supported. Decoupling the two features allows running the SHSTK test on AMD CPUs, which support SHSTK but not IBT.
--- x86/cet.c | 50 +++++++++++++++++++++++++++++++++----------------- 1 file changed, 33 insertions(+), 17 deletions(-)
x86/cet.c +33 -17
@@ -85,7 +85,7 @@ static uint64_t cet_ibt_func(void)#define ENABLE_SHSTK_BIT 0x1#define ENABLE_IBT_BIT 0x4 -int main (int ac,char **av) +static void test_shstk (void ) {char *shstk_virt;unsigned long shstk_phys;
@@ -94,17 +94,10 @@ int main(int ac, char **av)bool rvc;if (!this_cpu_has (X86_FEATURE_SHSTK )) { -report_skip ("SHSTK not enabled" ); -return report_summary (); +report_skip ("SHSTK not supported" ); +return ; } -if (!this_cpu_has (X86_FEATURE_IBT )) { -report_skip ("IBT not enabled" ); -return report_summary (); - } - -setup_vm (); -/* Allocate one page for shadow-stack. */ shstk_virt =alloc_vpage (); shstk_phys = (unsigned long )virt_to_phys (alloc_page ());
@@ -124,9 +117,6 @@ int main(int ac, char **av)/* Store shadow-stack pointer. */ wrmsr (MSR_IA32_PL3_SSP , (u64)(shstk_virt +0x1000 )); -/* Enable CET master control bit in CR4. */ -write_cr4 (read_cr4 () |X86_CR4_CET ); -printf ("Unit tests for CET user mode...\n" );run_in_user (cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); report(rvc && exception_error_code() == CP_ERR_NEAR_RET,
@@ -136,19 +126,45 @@ int main(int ac, char **av)report (rvc &&exception_error_code () ==CP_ERR_FAR_RET ,"FAR RET shadow-stack protection test" ); +/* SSP should be 4-Byte aligned */ + vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 ); +report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); +} + +static void test_ibt (void ) +{ +bool rvc; + +if (!this_cpu_has (X86_FEATURE_IBT )) { +report_skip ("IBT not supported" ); +return ; + } +/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT );run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_ENDBR ,"Indirect-branch tracking test" ); +} + +int main (int ac,char **av) +{ +if (!this_cpu_has (X86_FEATURE_SHSTK ) && !this_cpu_has (X86_FEATURE_IBT )) { +report_skip ("No CET features supported" ); +return report_summary (); + } + +setup_vm (); + +/* Enable CET global control bit in CR4. */ +write_cr4 (read_cr4 () |X86_CR4_CET ); + +test_shstk (); +test_ibt ();write_cr4 (read_cr4 () & ~X86_CR4_CET );wrmsr (MSR_IA32_U_CET ,0 ); -/* SSP should be 4-Byte aligned */ - vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 ); -report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); -return report_summary (); }
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Re: [kvm-unit-tests PATCH v3 15/17] x86/cet: Run SHSTK and IBT tests as appropriate if either feature is supported
Quoted 93 lines · Sean Christopherson
Run the SHSTK and IBT tests if their respective feature is supported, as nothing in the architecture requires both features to be supported. Decoupling the two features allows running the SHSTK test on AMD CPUs, which support SHSTK but not IBT.
Signed-off-by Sean Christopherson <[email protected]>--- x86/cet.c | 50 +++++++++++++++++++++++++++++++++----------------- 1 file changed, 33 insertions(+), 17 deletions(-)x86/cet.c +33
-13@@ -85,7 +85,7 @@ static uint64_t cet_ibt_func(void)#define ENABLE_SHSTK_BIT 0x1#define ENABLE_IBT_BIT 0x4 -int main (int ac,char **av) +static void test_shstk (void ) {char *shstk_virt;unsigned long shstk_phys;@@ -94,17 +94,10 @@ int main(int ac, char **av)bool rvc;if (!this_cpu_has (X86_FEATURE_SHSTK )) { -report_skip ("SHSTK not enabled" ); -return report_summary (); +report_skip ("SHSTK not supported" ); +return ; } -if (!this_cpu_has (X86_FEATURE_IBT )) { -report_skip ("IBT not enabled" ); -return report_summary (); - } - -setup_vm (); -/* Allocate one page for shadow-stack. */ shstk_virt =alloc_vpage (); shstk_phys = (unsigned long )virt_to_phys (alloc_page ());@@ -124,9 +117,6 @@ int main(int ac, char **av)/* Store shadow-stack pointer. */ wrmsr (MSR_IA32_PL3_SSP , (u64)(shstk_virt +0x1000 )); -/* Enable CET master control bit in CR4. */ -write_cr4 (read_cr4 () |X86_CR4_CET ); -printf ("Unit tests for CET user mode...\n" );run_in_user (cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); report(rvc && exception_error_code() == CP_ERR_NEAR_RET,@@ -136,19 +126,45 @@ int main(int ac, char **av)report (rvc &&exception_error_code () ==CP_ERR_FAR_RET ,"FAR RET shadow-stack protection test" ); +/* SSP should be 4-Byte aligned */ + vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 ); +report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); +} + +static void test_ibt (void ) +{ +bool rvc; + +if (!this_cpu_has (X86_FEATURE_IBT )) { +report_skip ("IBT not supported" ); +return ; + } +/* Enable indirect-branch tracking */ wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT );run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_ENDBR ,"Indirect-branch tracking test" ); +} + +int main (int ac,char **av) +{ +if (!this_cpu_has (X86_FEATURE_SHSTK ) && !this_cpu_has (X86_FEATURE_IBT )) { +report_skip ("No CET features supported" ); +return report_summary (); + } + +setup_vm (); + +/* Enable CET global control bit in CR4. */ +write_cr4 (read_cr4 () |X86_CR4_CET ); + +test_shstk (); +test_ibt ();
x86/cet.c int main(int ac, char **av) · line 170
Quoted 10 lines
x86/cet.c +0
-4write_cr4 (read_cr4 () & ~X86_CR4_CET );wrmsr (MSR_IA32_U_CET ,0 ); -/* SSP should be 4-Byte aligned */ - vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 ); -report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." ); -return report_summary (); }
Looks good to me!
Successfully tested on Intel ADL, selectively disabling IBT (-cpu host,-ibt), shadow stacks (-cpu host,-shstk) or both (-cpu host,-ibt,-shstk), each doing "The Right Thing," therefore:
Thanks,
Mathias
Sean Christopherson
reviewerPATCH v3 16/17 x86/cet: Drop the "intel_" prefix from the CET testcase
In reply to Sean ChristophersonNow that the CET test supports both Intel and AMD, drop the "intel" prefix.
--- x86/unittests.cfg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
x86/unittests.cfg +1 -1
@@ -509,7 +509,7 @@ file = tsx-ctrl.flat qemu_params = -cpu max groups = tsx-ctrl -[intel_cet] +[cet] file = cet.flat arch = x86_64 smp = 2
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Sean Christopherson
reviewerPATCH v3 17/17 x86/cet: Add testcases to verify KVM rejects emulation of CET instructions
In reply to Sean ChristophersonAdd SHSTK and IBT testcases to verify that KVM rejects (forced) emulation of instructions that interact with SHSTK and/or IBT state, as KVM doesn't support emulating SHSTK or IBT (rejecting emulation is preferable to compromising guest security).
--- x86/cet.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 121 insertions(+), 2 deletions(-)
x86/cet.c +121 -2
@@ -75,6 +75,113 @@ static uint64_t cet_ibt_func(void)return 0 ; } +#define __CET_TEST_UNSUPPORTED_INSTRUCTION (insn) \ +({ \ + struct far_pointer32 fp = { \ + .offset = 0, \ + .selector = USER_CS, \ + }; \ + \ + asm volatile ("push %%rax\n" \ + ASM_TRY_FEP("1f") insn "\n\t" \ + "1:" \ + "pop %%rax\n" \ + : : "m" (fp), "a" (NONCANONICAL) : "memory"); \ + \ + exception_vector(); \ +}) + +#define SHSTK_TEST_UNSUPPORTED_INSTRUCTION (insn) \ +do { \ + uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn); \ + \ + report(vector == UD_VECTOR, "Wanted #UD on %s, got %s", \ + insn, exception_mnemonic(vector)); \ +} while (0) + +/* +* Treat IRET as unsupported with IBT even though the minimal interactions with +* IBT _could_ be easily emulated by KVM, as KVM doesn't support emulating IRET +* outside of Real Mode. +*/ +#define CET_TEST_UNSUPPORTED_INSTRUCTIONS (CET ) \ +do { \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("callq *%%rax"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("lcall *%0"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("syscall"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("sysenter"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("iretq"); \ +} while (0) + +static uint64_t cet_shstk_emulation (void ) +{ +CET_TEST_UNSUPPORTED_INSTRUCTIONS (SHSTK ); + +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("call 1f" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("retq" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("retq $10" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("lretq" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("lretq $10" ); + +/* Do a handful of JMPs to verify they aren't impacted by SHSTK. */ + asmvolatile (KVM_FEP "jmp 1f\n\t" +"1:\n\t" +KVM_FEP "lea 2f(%%rip), %%rax\n\t" +KVM_FEP "jmp *%%rax\n\t" +"2:\n\t" + KVM_FEP "push $" xstr(USER_CS) "\n\t" +KVM_FEP "lea 3f(%%rip), %%rax\n\t" +KVM_FEP "push %%rax\n\t" +/* Manually encode ljmpq, which gas doesn't recognize :-( */ +KVM_FEP ".byte 0x48\n\t" +"ljmpl *(%%rsp)\n\t" +"3:\n\t" +KVM_FEP "pop %%rax\n\t" +KVM_FEP "pop %%rax\n\t" + ::: "eax"); + +return 0 ; +} + +/* +* Don't invoke printf() or report() in the IBT testcase, as it will likely +* generate an indirect branch without an endbr64 annotation and thus #CP. +* Return the line number of the macro invocation to signal failure. +*/ +#define IBT_TEST_UNSUPPORTED_INSTRUCTION (insn) \ +do { \ + uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn); \ + \ + if (vector != UD_VECTOR) \ + return __LINE__; \ +} while (0) + +static uint64_t cet_ibt_emulation (void ) +{ +CET_TEST_UNSUPPORTED_INSTRUCTIONS (IBT ); + +IBT_TEST_UNSUPPORTED_INSTRUCTION ("jmp *%%rax" ); +IBT_TEST_UNSUPPORTED_INSTRUCTION ("ljmp *%0" ); + +/* Verify direct CALLs and JMPs, and all RETs aren't impacted by IBT. */ + asmvolatile (KVM_FEP "jmp 2f\n\t" +"1: " KVM_FEP " ret\n\t" +"2: " KVM_FEP " call 1b\n\t" + KVM_FEP "push $" xstr(USER_CS) "\n\t" +KVM_FEP "lea 3f(%%rip), %%rax\n\t" +KVM_FEP "push %%rax\n\t" +KVM_FEP "lretq\n\t" +"3:\n\t" +KVM_FEP "push $0x55555555\n\t" + KVM_FEP "push $" xstr(USER_CS) "\n\t" +KVM_FEP "lea 4f(%%rip), %%rax\n\t" +KVM_FEP "push %%rax\n\t" +KVM_FEP "lretq $8\n\t" +"4:\n\t" + ::: "eax"); +return 0 ; +} +#define CP_ERR_NEAR_RET 0x0001#define CP_ERR_FAR_RET 0x0002#define CP_ERR_ENDBR 0x0003
@@ -117,15 +224,20 @@ static void test_shstk(void)/* Store shadow-stack pointer. */ wrmsr (MSR_IA32_PL3_SSP , (u64)(shstk_virt +0x1000 )); -printf ("Unit tests for CET user mode...\n" ); +printf ("Running user mode Shadow Stack tests\n" );run_in_user (cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_NEAR_RET ,"NEAR RET shadow-stack protection test" ); -run_in_user (cet_shstk_far_ret,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_FAR_RET ,"FAR RET shadow-stack protection test" ); +if (is_fep_available && + (run_in_user (cet_shstk_emulation,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc) || rvc)) +report_fail ("Forced emulation with SHSTK generated %s(%u)" , +exception_mnemonic (exception_vector ()), +exception_error_code ()); +/* SSP should be 4-Byte aligned */ vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 );report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." );
@@ -133,6 +245,7 @@ static void test_shstk(void)static void test_ibt (void ) { +uint64_t l;bool rvc;if (!this_cpu_has (X86_FEATURE_IBT )) {
@@ -146,6 +259,12 @@ static void test_ibt(void)run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_ENDBR ,"Indirect-branch tracking test" ); + +if (is_fep_available && + ((l =run_in_user (cet_ibt_emulation,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc)) || rvc)) +report_fail ("Forced emulation with IBT generated %s(%u) at line %lu" , +exception_mnemonic (exception_vector ()), +exception_error_code (), l); }int main (int ac,char **av)
Signature · 1 line
2.52.0.rc1.455.g30608eb744-goog
Re: [kvm-unit-tests PATCH v3 17/17] x86/cet: Add testcases to verify KVM rejects emulation of CET instructions
Quoted 41 lines · Sean Christopherson
Add SHSTK and IBT testcases to verify that KVM rejects (forced) emulation of instructions that interact with SHSTK and/or IBT state, as KVM doesn't support emulating SHSTK or IBT (rejecting emulation is preferable to compromising guest security).
Signed-off-by Sean Christopherson <[email protected]>--- x86/cet.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 121 insertions(+), 2 deletions(-)x86/cet.c +23
-0@@ -75,6 +75,113 @@ static uint64_t cet_ibt_func(void)return 0 ; } +#define __CET_TEST_UNSUPPORTED_INSTRUCTION (insn) \ +({ \ + struct far_pointer32 fp = { \ + .offset = 0, \ + .selector = USER_CS, \ + }; \ + \ + asm volatile ("push %%rax\n" \ + ASM_TRY_FEP("1f") insn "\n\t" \ + "1:" \ + "pop %%rax\n" \ + : : "m" (fp), "a" (NONCANONICAL) : "memory"); \ + \ + exception_vector(); \ +}) + +#define SHSTK_TEST_UNSUPPORTED_INSTRUCTION (insn) \ +do { \ + uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn); \ + \ + report(vector == UD_VECTOR, "Wanted #UD on %s, got %s", \ + insn, exception_mnemonic(vector)); \ +} while (0)
x86/cet.c static uint64_t cet_ibt_func(void) · line 110
Quoted 10 lines
x86/cet.c +10
-0+ +/* +* Treat IRET as unsupported with IBT even though the minimal interactions with +* IBT _could_ be easily emulated by KVM, as KVM doesn't support emulating IRET +* outside of Real Mode. +*/ +#define CET_TEST_UNSUPPORTED_INSTRUCTIONS (CET ) \ +do { \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("callq *%%rax"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("lcall *%0"); \
Maybe spell out that this is a 32bit far call ("lcalll *%0"), as only these are supported on AMD *and* Intel? (Intel can do 64bit ones too.)
x86/cet.c static uint64_t cet_ibt_func(void) · line 113
Quoted 3 lines
x86/cet.c +3
-0+ CET##_TEST_UNSUPPORTED_INSTRUCTION("syscall"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("sysenter"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("iretq"); \
These, if emulated, would be rather disastrous to the test. :D
- SYSCALL messes badly with the register state.
- SYSENTER should #UD in long mode on AMD anyway but Intel allows it. However, it'll likely #GP early, prior to messing with register state, as IA32_SYSENTER_CS doesn't get initialized as the CPU expects it to be.
- The stack frame for IRETQ isn't properly set up (lacks CS, RFLAGS, SS, RSP) but even if it'll #GP early for the non-canonical address, it'll have modified RSP, making the recovery "pop %rax" pop an unrelated stack word, likely making the code crash soon after.
So these instructions really *rely* on KVM to #UD on these early. Dunno, but it feels like the tests should be made more robust wrt. possible future emulation by KVM?
Quoted 7 lines
x86/cet.c +7
-0+}while (0 ) + +static uint64_t cet_shstk_emulation(void) +{ +CET_TEST_UNSUPPORTED_INSTRUCTIONS (SHSTK ); + +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("call 1f" );
x86/cet.c static uint64_t cet_ibt_func(void) · line 130
Quoted 10 lines
x86/cet.c +10
-0+SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("retq" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("retq $10" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("lretq" ); +SHSTK_TEST_UNSUPPORTED_INSTRUCTION ("lretq $10" ); + +/* Do a handful of JMPs to verify they aren't impacted by SHSTK. */ + asmvolatile (KVM_FEP "jmp 1f\n\t" +"1:\n\t" +KVM_FEP "lea 2f(%%rip), %%rax\n\t" + KVM_FEP "jmp *%%rax\n\t"
^^^^^^^^^^
Fortunately, this indirect branch runs only with shadow stack enabled, no IBT. If it would, than below misses an ENDBR64.
vvv
x86/cet.c static uint64_t cet_ibt_func(void) · line 134
Quoted 4 lines
x86/cet.c +4
-0+ "2:\n\t"> + KVM_FEP "push $" xstr(USER_CS) "\n\t" +KVM_FEP "lea 3f(%%rip), %%rax\n\t" +KVM_FEP "push %%rax\n\t" +/* Manually encode ljmpq, which gas doesn't recognize :-( */
LJMPQ is actually only supported on Intel systems. AMD doesn't support it and one has to emulate it via "push $cs; push $rip; lretq". Dunno if it's a feature or bug of KVM to emulate LJMPQ fine on AMD -- if it does, that is!
Quoted 4 lines
x86/cet.c +4
-0+KVM_FEP ".byte 0x48\n\t" +"ljmpl *(%%rsp)\n\t" +"3:\n\t" +KVM_FEP "pop %%rax\n\t"
x86/cet.c static uint64_t cet_ibt_func(void) · line 148
Quoted 10 lines
x86/cet.c +9
-0+ KVM_FEP "pop %%rax\n\t" + ::: "eax"); + + return 0; +} + +/* +* Don't invoke printf() or report() in the IBT testcase, as it will likely +* generate an indirect branch without an endbr64 annotation and thus #CP.
Uhm, x86/Makefile.x86_64 has this:
fcf_protection_full := $(call cc-option, -fcf-protection=full,) COMMON_CFLAGS += -mno-red-zone -mno-sse -mno-sse2 $(fcf_protection_full) [...]
ifneq ($(fcf_protection_full),)
tests += $(TEST_DIR)/cet.$(exe)
endif
So all code that needs it should have a leading ENDBR64 or the cet test wouldn't be part of the test suite.
However, I also notice it isn't working...
[digging in, hours later...]
After tweaking the exception handling to be able to tweak the IBT tracker state (which would be stuck at WAIT_FOR_ENDBRANCH for a missing ENDBR64), it still wouldn't work for me. Bummer! Further instrumentation showed, the code triggered within exception_mnemonic() which caused only more question marks -- it's just a simple switch, right? Though, looking at the disassembly made it crystal clear:
000000000040707c <exception_mnemonic>:
40707c: endbr64
407080: cmp $0x1e,%edi
407083: ja 407117 <exception_mnemonic+0x9b>
407089: mov %edi,%edi
40708b: notrack jmp *0x4107e0(,%rdi,8)
::4070b1: mov $0x411c7c,%eax # <-- #CP(3) here
The switch block caused gcc to emit a jump table and an indirect jump for it (to 40708b). The jump is prefixed with 'notrack' to keep the size of the jump target stubs small, allowing for omitting the ENDBR64 instruction. However, the IBT test code doesn't enable that in the MSR, leading the CPU to raising a #CP(3) for this indirect jump. D'oh!
So we can either disable the generation of jump tables in gcc (-fno-jump-tables) or just allow the 'notrack' handling. I guess the latter is easier.
Attached patch gets me this (on some early CET series patch set, not v6.18-rcX):
Running user mode Shadow Stack tests
The return-address in shadow-stack = 0x4035ab, in normal stack = 0x4035ab Try to temper the return-address, this causes #CP on returning...
PASS: NEAR RET shadow-stack protection test
Try to temper the return-address of far-called function...
PASS: FAR RET shadow-stack protection test
PASS: SHSTK: Wanted #UD on callq *%%rax, got #UD
PASS: SHSTK: Wanted #UD on lcall *%0, got #UD
PASS: SHSTK: Wanted #UD on syscall, got #UD
PASS: SHSTK: Wanted #UD on sysenter, got #UD
PASS: SHSTK: Wanted #UD on iretq, got #UD
PASS: SHSTK: Wanted #UD on call 1f, got #UD
FAIL: SHSTK: Wanted #UD on retq, got #GP
FAIL: SHSTK: Wanted #UD on retq $10, got #GP
PASS: SHSTK: Wanted #UD on lretq, got #UD
PASS: SHSTK: Wanted #UD on lretq $10, got #UD
PASS: MSR_IA32_PL3_SSP alignment test.
No endbr64 instruction at jmp target, this triggers #CP...
PASS: Indirect-branch tracking test
PASS: IBT: Wanted #UD on callq *%%rax, got #UD
PASS: IBT: Wanted #UD on lcall *%0, got #UD
PASS: IBT: Wanted #UD on syscall, got #UD
PASS: IBT: Wanted #UD on sysenter, got #UD
PASS: IBT: Wanted #UD on iretq, got #UD
PASS: IBT: Wanted #UD on jmp *%%rax, got #UD
Unhandled exception 13 #GP(0) at ip 0000000000000000
error_code=0000 rflags=00010006 cs=0000004b
rax=aaaaaaaaaaaaaaaa rcx=00000000000003fd rdx=00000000000003f8 rbx=aaaaaaaaaaaaaaaa
rbp=0000000000713e70 rsi=0000000000411cb7 rdi=000000000000000a
r8=0000000000411cb7 r9=00000000000003f8 r10=000000000000000d
r11=0000000000000000
r12=0000000000000006 r13=0000000000000000 r14=0000000000000000 r15=0000000000000000
cr0=0000000080010011 cr2=0000000000000000 cr3=000000000107f000 cr4=0000000000800020
cr8=0000000000000000
STACK: @0 4035ab 400deb 4001bd
b'0x0000000000000000: ?? ??:0'
Progress, but still something to fix, namely the "ljmp *%0" which is, apparently, successfully branching to address 0 and then triggering a #GP(0). But maybe that's just because I'm on an old version of the KVM CET series, haven't build v6.18-rcX yet.
Anyhow, one less puzzle, so should be good?
Quoted 73 lines
x86/cet.c +44
-2+* Return the line number of the macro invocation to signal failure. +*/ +#define IBT_TEST_UNSUPPORTED_INSTRUCTION (insn) \ +do { \ + uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn); \ + \ + if (vector != UD_VECTOR) \ + return __LINE__; \ +} while (0) + +static uint64_t cet_ibt_emulation (void ) +{ +CET_TEST_UNSUPPORTED_INSTRUCTIONS (IBT ); + +IBT_TEST_UNSUPPORTED_INSTRUCTION ("jmp *%%rax" ); +IBT_TEST_UNSUPPORTED_INSTRUCTION ("ljmp *%0" ); + +/* Verify direct CALLs and JMPs, and all RETs aren't impacted by IBT. */ + asmvolatile (KVM_FEP "jmp 2f\n\t" +"1: " KVM_FEP " ret\n\t" +"2: " KVM_FEP " call 1b\n\t" + KVM_FEP "push $" xstr(USER_CS) "\n\t" +KVM_FEP "lea 3f(%%rip), %%rax\n\t" +KVM_FEP "push %%rax\n\t" +KVM_FEP "lretq\n\t" +"3:\n\t" +KVM_FEP "push $0x55555555\n\t" + KVM_FEP "push $" xstr(USER_CS) "\n\t" +KVM_FEP "lea 4f(%%rip), %%rax\n\t" +KVM_FEP "push %%rax\n\t" +KVM_FEP "lretq $8\n\t" +"4:\n\t" + ::: "eax"); +return 0 ; +} +#define CP_ERR_NEAR_RET 0x0001#define CP_ERR_FAR_RET 0x0002#define CP_ERR_ENDBR 0x0003@@ -117,15 +224,20 @@ static void test_shstk(void)/* Store shadow-stack pointer. */ wrmsr (MSR_IA32_PL3_SSP , (u64)(shstk_virt +0x1000 )); -printf ("Unit tests for CET user mode...\n" ); +printf ("Running user mode Shadow Stack tests\n" );run_in_user (cet_shstk_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_NEAR_RET ,"NEAR RET shadow-stack protection test" ); -run_in_user (cet_shstk_far_ret,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc);report (rvc &&exception_error_code () ==CP_ERR_FAR_RET ,"FAR RET shadow-stack protection test" ); +if (is_fep_available && + (run_in_user (cet_shstk_emulation,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc) || rvc)) +report_fail ("Forced emulation with SHSTK generated %s(%u)" , +exception_mnemonic (exception_vector ()), +exception_error_code ()); +/* SSP should be 4-Byte aligned */ vector =wrmsr_safe (MSR_IA32_PL3_SSP ,0x1 );report (vector ==GP_VECTOR ,"MSR_IA32_PL3_SSP alignment test." );@@ -133,6 +245,7 @@ static void test_shstk(void)static void test_ibt (void ) { +uint64_t l;bool rvc;if (!this_cpu_has (X86_FEATURE_IBT )) {@@ -146,6 +259,12 @@ static void test_ibt(void)run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); report(rvc && exception_error_code() == CP_ERR_ENDBR,
x86/cet.c static void test_ibt(void) · line 270
Quoted 10 lines
x86/cet.c +6
-0"Indirect-branch tracking test"); + +if (is_fep_available && + ((l =run_in_user (cet_ibt_emulation,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc)) || rvc)) +report_fail ("Forced emulation with IBT generated %s(%u) at line %lu" , +exception_mnemonic (exception_vector ()), +exception_error_code (), l); } int main(int ac, char **av)Reviewed-by: Mathias Krause <[email protected]>
Thanks,
Mathias
From 5cba4a9ccdff8e682ae53ee69849879edc326746 Mon Sep 17 00:00:00 2001
From: Mathias Krause <[email protected]>
Date: Fri, 14 Nov 2025 15:53:10 +0100
Subject: [kvm-unit-tests PATCH] x86: cet: Enable NOTRACK handling for IBT
tests
gcc's jump table handling makes use of 'notrack' indirect jumps, causing spurious #CP(3) exceptions.
Enable 'notrack' handling for the IBT tests instead of disabling jump tables as we may want to make use of 'notrack' ourselves in future tests.
With that fixed, we can make the IBT tests report() successfully, just as the shadow stack tests do.
--- x86/cet.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
x86/cet.c +9 -8
@@ -95,7 +95,7 @@ static uint64_t cet_ibt_func(void)do { \uint8_t vector =__CET_TEST_UNSUPPORTED_INSTRUCTION (insn); \ \ -report (vector ==UD_VECTOR ,"Wanted #UD on %s, got %s" , \ +report (vector ==UD_VECTOR ,"SHSTK: Wanted #UD on %s, got %s" , \ insn,exception_mnemonic (vector)); \ }while (0 )
@@ -152,8 +152,8 @@ static uint64_t cet_shstk_emulation(void)do { \uint8_t vector =__CET_TEST_UNSUPPORTED_INSTRUCTION (insn); \ \ -if (vector !=UD_VECTOR ) \ -return __LINE__; \ +report (vector ==UD_VECTOR ,"IBT: Wanted #UD on %s, got %s" , \ + insn,exception_mnemonic (vector)); \ } while (0)static uint64_t cet_ibt_emulation (void )
@@ -189,8 +189,9 @@ static uint64_t cet_ibt_emulation(void)#define CP_ERR_SETSSBSY 0x0005#define CP_ERR_ENCL BIT(15) -#define ENABLE_SHSTK_BIT 0x1 -#define ENABLE_IBT_BIT 0x4 +#define CET_ENABLE_SHSTK BIT(0) +#define CET_ENABLE_IBT BIT(2) +#define CET_ENABLE_NOTRACK BIT(4)static void test_shstk (void ) {
@@ -219,7 +220,7 @@ static void test_shstk(void)install_pte (current_page_table (),1 , shstk_virt, pte,0 );/* Enable shadow-stack protection */ -wrmsr (MSR_IA32_U_CET ,ENABLE_SHSTK_BIT ); +wrmsr (MSR_IA32_U_CET ,CET_ENABLE_SHSTK );/* Store shadow-stack pointer. */ wrmsr (MSR_IA32_PL3_SSP , (u64)(shstk_virt +0x1000 ));
@@ -253,8 +254,8 @@ static void test_ibt(void)return ; } -/* Enable indirect-branch tracking */ -wrmsr (MSR_IA32_U_CET ,ENABLE_IBT_BIT ); +/* Enable indirect-branch tracking (notrack handling for jump tables) */ +wrmsr (MSR_IA32_U_CET ,CET_ENABLE_IBT |CET_ENABLE_NOTRACK );run_in_user (cet_ibt_func,CP_VECTOR ,0 ,0 ,0 ,0 , &rvc); report(rvc && exception_error_code() == CP_ERR_ENDBR,
Signature · 1 line
2.51.0
Sean Christopherson
reviewerRe: [kvm-unit-tests PATCH v3 17/17] x86/cet: Add testcases to verify KVM rejects emulation of CET instructions
Quoted 37 lines · Mathias Krause
Quoted 7 lines · Sean Christopherson
(quoted diff) +7
-0+#define SHSTK_TEST_UNSUPPORTED_INSTRUCTION(insn) \ +do { \ + uint8_t vector = __CET_TEST_UNSUPPORTED_INSTRUCTION(insn); \ + \ + report(vector == UD_VECTOR, "Wanted #UD on %s, got %s", \ + insn, exception_mnemonic(vector)); \ +} while (0)Review
Quoted 10 lines
(quoted diff) +10
-0+ +/* + * Treat IRET as unsupported with IBT even though the minimal interactions with + * IBT _could_ be easily emulated by KVM, as KVM doesn't support emulating IRET + * outside of Real Mode. + */ +#define CET_TEST_UNSUPPORTED_INSTRUCTIONS(CET) \ +do { \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("callq *%%rax"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("lcall *%0"); \Maybe spell out that this is a 32bit far call ("lcalll *%0"), as only these are supported on AMD *and* Intel? (Intel can do 64bit ones too.)
Review
Quoted 3 lines
(quoted diff) +3
-0+ CET##_TEST_UNSUPPORTED_INSTRUCTION("syscall"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("sysenter"); \ + CET##_TEST_UNSUPPORTED_INSTRUCTION("iretq"); \These, if emulated, would be rather disastrous to the test. :D
- SYSCALL messes badly with the register state.
- SYSENTER should #UD in long mode on AMD anyway but Intel allows it. However, it'll likely #GP early, prior to messing with register state, asIA32_SYSENTER_CSdoesn't get initialized as the CPU expects it to be.
- The stack frame for IRETQ isn't properly set up (lacks CS, RFLAGS, SS, RSP) but even if it'll #GP early for the non-canonical address, it'll have modified RSP, making the recovery "pop %rax" pop an unrelated stack word, likely making the code crash soon after.
Quoted 3 lines
So these instructions really *rely* on KVM to #UD on these early. Dunno, but it feels like the tests should be made more robust wrt. possible future emulation by KVM?
I don't necessarily disagree, but I'm not convinced it'd be worth the effort to properly set everything up _and_ keep it functional. Unless KVM is buggy, the fallback will be completely untested. I hate user-hostile tests as much as the next person (probably more), but for this one my vote is to punt and cross our fingers that KVM never screws up.
Quoted 18 lines
Quoted 7 lines
(quoted diff) +7
-0+} while (0) + +static uint64_t cet_shstk_emulation(void) +{ + CET_TEST_UNSUPPORTED_INSTRUCTIONS(SHSTK); + + SHSTK_TEST_UNSUPPORTED_INSTRUCTION("call 1f");Review
Quoted 10 lines
(quoted diff) +10
-0+ SHSTK_TEST_UNSUPPORTED_INSTRUCTION("retq"); + SHSTK_TEST_UNSUPPORTED_INSTRUCTION("retq $10"); + SHSTK_TEST_UNSUPPORTED_INSTRUCTION("lretq"); + SHSTK_TEST_UNSUPPORTED_INSTRUCTION("lretq $10"); + + /* Do a handful of JMPs to verify they aren't impacted by SHSTK. */ + asm volatile(KVM_FEP "jmp 1f\n\t" + "1:\n\t" + KVM_FEP "lea 2f(%%rip), %%rax\n\t" + KVM_FEP "jmp *%%rax\n\t"^^^^^^^^^^
Quoted 2 lines
Fortunately, this indirect branch runs only with shadow stack enabled, no IBT. If it would, than below misses an ENDBR64.
50/50 as to whether I noticed that or just got lucky :-)
Quoted 8 lines
vvv
Review
Quoted 4 lines
(quoted diff) +4
-0+ "2:\n\t"> + KVM_FEP "push $" xstr(USER_CS) "\n\t" + KVM_FEP "lea 3f(%%rip), %%rax\n\t" + KVM_FEP "push %%rax\n\t" + /* Manually encode ljmpq, which gas doesn't recognize :-( */LJMPQ is actually only supported on Intel systems. AMD doesn't support it and one has to emulate it via "push $cs; push $rip; lretq".
Well that explains why gas wasn't being very nice to me. I'll tweak the comment.
Quoted 2 lines
Dunno if it's a feature or bug of KVM to emulate LJMPQ fine on AMD -- if it does, that is!
Bug? Sort of? KVM won't emulate it on #UD, so it's only very limited emulation. E.g. the guest would have to force emulation, put the jump source in emulated MMIO, or play TLB games to get KVM to emulate
Quoted 53 lines
Quoted 4 lines
(quoted diff) +4
-0+ KVM_FEP ".byte 0x48\n\t" + "ljmpl *(%%rsp)\n\t" + "3:\n\t" + KVM_FEP "pop %%rax\n\t"Review
Quoted 10 lines
(quoted diff) +9
-0+ KVM_FEP "pop %%rax\n\t" + ::: "eax"); + + return 0; +} + +/* + * Don't invoke printf() or report() in the IBT testcase, as it will likely + * generate an indirect branch without an endbr64 annotation and thus #CP.Uhm, x86/Makefile.x86_64 has this:
fcf_protection_full:= $(call cc-option, -fcf-protection=full,)COMMON_CFLAGS+= -mno-red-zone -mno-sse -mno-sse2 $(fcf_protection_full) [...]
ifneq ($(fcf_protection_full),)
tests += $(TEST_DIR)/cet.$(exe)
endifSo all code that needs it should have a leading ENDBR64 or the cet test wouldn't be part of the test suite.
However, I also notice it isn't working...
[digging in, hours later...]
After tweaking the exception handling to be able to tweak the IBT tracker state (which would be stuck at
WAIT_FOR_ENDBRANCHfor a missing ENDBR64), it still wouldn't work for me. Bummer! Further instrumentation showed, the code triggered withinexception_mnemonic()which caused only more question marks -- it's just a simple switch, right? Though, looking at the disassembly made it crystal clear:
000000000040707c<exception_mnemonic>:40707c: endbr64 407080: cmp $0x1e,%edi 407083: ja 407117 <exception_mnemonic+0x9b> 407089: mov %edi,%edi 40708b: notrack jmp *0x4107e0(,%rdi,8) ::4070b1: mov $0x411c7c,%eax # <-- #CP(3) here
The switch block caused gcc to emit a jump table and an indirect jump for it (to 40708b). The jump is prefixed with 'notrack' to keep the size of the jump target stubs small, allowing for omitting the ENDBR64 instruction. However, the IBT test code doesn't enable that in the MSR, leading the CPU to raising a #CP(3) for this indirect jump. D'oh!
Quoted 3 lines
So we can either disable the generation of jump tables in gcc (-fno-jump-tables) or just allow the 'notrack' handling. I guess the latter is easier.
And much more helpful for people like me, who know just enough about IBT to be dangerous, but don't fully understand how everything works the end-to-end.
Quoted 4 lines
Progress, but still something to fix, namely the "ljmp *%0" which is, apparently, successfully branching to address 0 and then triggering a #GP(0). But maybe that's just because I'm on an old version of the KVM CET series, haven't build v6.18-rcX yet.
I suspect it's your old version, it passes on my end.
Quoted 1 line
Anyhow, one less puzzle, so should be good?
Ya, I'll slot this in for the next version (instead of having it be a fixup).