Usermode Indirect Branch Tracking
Subsystems (5)
v2 cover Usermode Indirect Branch Tracking
Adds basic support for x86 userspace IBT.
IBT is part of Intel CET. It requires indirect call and jump targets to start with an endbr{32,64} instruction, otherwise throwing #CP.
In summary, this patch does 3 things:
- Config wiring ensuring supervisor XSAVE contains IBT state
- Allow userspace to enable IBT via prctl(PR_CFI_*) for an entire thread
- Enable IBT support (ENDBR instructions) in VDSO
Unlike the arm64 BTI API:
- does not support mixed usermode (all or nothing)
- does not touch page table code
- not enabled automatically (no ELF GNU note parsing)
- temporarily disables IBT enforcement when handling signals These can all be cleanly added later.
The main question is whether glibc is happy with this prctl syscall API.
Changes from v1:
- Removed all signal handler changes
- Removed all uapi changes
(v1 saved state via a redundant uc_flags bit, which was a complete
mess, and not compatible with sigframe_ia32)
- Don't allow unknown PR_CFI_* syscall flags
see RISC-V patch: https://lore.kernel.org/lkml/[email protected]/
- Added 32-bit support
The original usermode IBT patches were written by Yu Cheng years ago, but are quite different (no syscall API to enable IBT, no FRED support)
https://lore.kernel.org/all/[email protected]/
There is one notable gap in this patch series, to do with signals:
000a: mov rax, 0x100a
000f: jmp rax
*** signal occurs ***
*** signal handler runs, does sigreturn ***
100a: nop
The above sequence does not crash.
With IBT, it should crash at the nop (because an endr64 is expected there). The IBT state (WAIT_FOR_ENDBR in IA32_U_CET MSR) is not backed up to the signal frame though. So, when userland does a sigreturn, the CPU has forgotten that it was doing an indirect branch before the signal. (This specifically only occurs with signal handlers that sigreturn.)
This is because IA32_U_CET is part of XSAVE 'supervisor' state, so regular XSAVE/XRSTOR can't access it. Doing a manual backup is tricky.
A related problem is that the signal handler routine is not checked for endbr preamble.
Basic IBT is better than no IBT, though.
Richard Patel (5):
x86: add userspace IBT config option
x86: shstk: don't clobber IBT bits in U_CET MSR
x86: expose user IBT via PR_CFI_BRANCH_LANDING_PADS x86/entry/vdso: build with IBT support selftests/x86: test usermode IBT
arch/x86/Kconfig | 18 ++ arch/x86/entry/vdso/common/Makefile.include | 3 +- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/ibt.h | 14 ++ arch/x86/include/asm/processor.h | 5 + arch/x86/kernel/Makefile | 1 + arch/x86/kernel/cet.c | 3 +- arch/x86/kernel/cpu/common.c | 14 +- arch/x86/kernel/ibt.c | 98 ++++++++ arch/x86/kernel/process_64.c | 2 + arch/x86/kernel/shstk.c | 12 +- tools/arch/x86/include/asm/cpufeatures.h | 1 + tools/testing/selftests/x86/Makefile | 5 +- tools/testing/selftests/x86/user_ibt.c | 247 ++++++++++++++++++++ 14 files changed, 416 insertions(+), 8 deletions(-) create mode 100644 arch/x86/kernel/ibt.c create mode 100644 tools/testing/selftests/x86/user_ibt.c
--
2.47.3
PATCH v2 1/5 x86: add userspace IBT config option
Adds a X86_USER_IBT Kconfig option and "nouseribt" command-line option. Default disabled for now.
These prepare for userspace support for IBT (forward-edge control flow integrity protection).
User IBT works even if kernel IBT is disabled. However, ibt=off also disables user IBT.
Depends on usermode shadow stack, which enables XSAVES for the IA32_U_CET MSR, containing both SHSTK and IBT state.
--- arch/x86/Kconfig | 18 ++++++++++++++++++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cet.c | 3 ++- arch/x86/kernel/cpu/common.c | 14 ++++++++++++-- tools/arch/x86/include/asm/cpufeatures.h | 1 + 5 files changed, 34 insertions(+), 3 deletions(-)
arch/x86/Kconfig +18 -0
@@ -1901,6 +1901,24 @@ config X86_USER_SHADOW_STACK If unsure, say N. +config X86_USER_IBT + bool "X86 userspace indirect branch tracking" + depends on X86_64 + depends on X86_USER_SHADOW_STACK + select X86_CET + help + Support Indirect Branch Tracking protection for userspace + applications. IBT is a hardware-supported coarse-grained + forward-edge Control Flow Integrity protection feature. + It enforces that all indirect calls must land on an ENDBR + instruction. Applications must be enabled to use it, and old + userspace does not get protection "for free". Enables the + PR_CFI_BRANCH_LANDING_PADS prctl CFI option. + + CPUs supporting IBT were first released in 2021. + + If unsure, say N. + config INTEL_TDX_HOST bool "Intel Trust Domain Extensions (TDX) host support" depends on CPU_SUP_INTEL
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -516,6 +516,7 @@* and purposes if CLEAR_CPU_BUF_VM is set). */ #define X86_FEATURE_X2AVIC_EXT (21*32+20)/* AMD SVM x2AVIC support for 4k vCPUs */ +#define X86_FEATURE_USER_IBT (21*32+21)/* Indirect Branch Tracking for user mode applications */ /* * BUG word(s)
arch/x86/kernel/cet.c +2 -1
@@ -149,7 +149,8 @@ __setup("ibt=", ibt_setup);DEFINE_IDTENTRY_ERRORCODE (exc_control_protection) {if (user_mode (regs)) { -if (cpu_feature_enabled (X86_FEATURE_USER_SHSTK )) +if (cpu_feature_enabled (X86_FEATURE_USER_SHSTK ) || +cpu_feature_enabled (X86_FEATURE_USER_IBT ))do_user_cp_fault (regs, error_code);else do_unexpected_cp (regs, error_code);
arch/x86/kernel/cpu/common.c +12 -2
@@ -634,7 +634,7 @@ __noendbr void ibt_restore(u64 save) static __always_inline void setup_cet(struct cpuinfo_x86 *c) { -bool user_shstk, kernel_ibt; +bool user_shstk, kernel_ibt, user_ibt;if (!IS_ENABLED (CONFIG_X86_CET ))return ;
@@ -642,13 +642,19 @@ static __always_inline void setup_cet(struct cpuinfo_x86 *c) kernel_ibt =HAS_KERNEL_IBT &&cpu_feature_enabled (X86_FEATURE_IBT ); user_shstk =cpu_feature_enabled (X86_FEATURE_SHSTK ) &&IS_ENABLED (CONFIG_X86_USER_SHADOW_STACK ); +/* User IBT only needs hardware IBT, not kernel-enabled IBT. */ + user_ibt =cpu_has (c,X86_FEATURE_IBT ) && +IS_ENABLED (CONFIG_X86_USER_IBT ); -if (!kernel_ibt && !user_shstk) +if (!kernel_ibt && !user_shstk && !user_ibt)return ;if (user_shstk)set_cpu_cap (c,X86_FEATURE_USER_SHSTK ); +if (user_ibt) +set_cpu_cap (c,X86_FEATURE_USER_IBT ); +if (kernel_ibt)wrmsrq (MSR_IA32_S_CET ,CET_ENDBR_EN );else
@@ -666,6 +672,7 @@ static __always_inline void setup_cet(struct cpuinfo_x86 *c) __noendbr void cet_disable(void) {if (!(cpu_feature_enabled (X86_FEATURE_IBT ) || +cpu_feature_enabled (X86_FEATURE_USER_IBT ) ||cpu_feature_enabled (X86_FEATURE_SHSTK )))return ;
@@ -1760,6 +1767,9 @@ static void __init cpu_parse_early_param(void)if (cmdline_find_option_bool (boot_command_line,"nousershstk" ))setup_clear_cpu_cap (X86_FEATURE_USER_SHSTK ); +if (cmdline_find_option_bool (boot_command_line,"nouseribt" )) +setup_clear_cpu_cap (X86_FEATURE_USER_IBT ); +/* Minimize the gap between FRED is available and available but disabled. */ arglen =cmdline_find_option (boot_command_line,"fred" , arg,sizeof (arg));if (arglen ==3 && !strncmp (arg,"off" ,3 ))
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -515,6 +515,7 @@* and purposes if CLEAR_CPU_BUF_VM is set). */ #define X86_FEATURE_X2AVIC_EXT (21*32+20)/* AMD SVM x2AVIC support for 4k vCPUs */ +#define X86_FEATURE_USER_IBT (21*32+21)/* Indirect Branch Tracking for user mode applications */ /* * BUG word(s)
Signature · 1 line
2.47.3
PATCH v2 2/5 x86: shstk: don't clobber IBT bits in U_CET MSR
In reply to Richard PatelUpdates usermode shadow stack code to not set IBT-related bits in the U_CET MSR.
--- arch/x86/kernel/shstk.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/shstk.c +10 -2
@@ -150,6 +150,7 @@ static int shstk_setup(void) {struct thread_shstk *shstk = ¤t->thread.shstk;unsigned long addr, size; +u64 msrval;/* Already enabled */ if (features_enabled (ARCH_SHSTK_SHSTK ))
@@ -166,7 +167,10 @@ static int shstk_setup(void)fpregs_lock_and_load ();wrmsrq (MSR_IA32_PL3_SSP , addr + size); -wrmsrq (MSR_IA32_U_CET ,CET_SHSTK_EN ); +rdmsrq (MSR_IA32_U_CET , msrval); + msrval &= ~CET_WRSS_EN ; + msrval |=CET_SHSTK_EN ; +wrmsrq (MSR_IA32_U_CET , msrval);fpregs_unlock (); shstk->base = addr;
@@ -520,6 +524,8 @@ static int wrss_control(bool enable)static int shstk_disable (void ) { +u64 msrval; +if (!cpu_feature_enabled (X86_FEATURE_USER_SHSTK ))return -EOPNOTSUPP ;
@@ -528,8 +534,10 @@ static int shstk_disable(void)return 0 ;fpregs_lock_and_load (); +rdmsrq (MSR_IA32_U_CET , msrval);/* Disable WRSS too when disabling shadow stack */ -wrmsrq (MSR_IA32_U_CET ,0 ); + msrval &= ~(CET_SHSTK_EN |CET_WRSS_EN ); +wrmsrq (MSR_IA32_U_CET , msrval);wrmsrq (MSR_IA32_PL3_SSP ,0 );fpregs_unlock ();
Signature · 1 line
2.47.3
PATCH v2 3/5 x86: expose user IBT via PR_CFI_BRANCH_LANDING_PADS
In reply to Richard PatelAllows userspace applications to enable IBT (forward-edge control flow integrity protection) using the portable PR_CFI prctl API.
The name 'branch landing pads' is RISC-V specific, but the mechanism is nearly identical in x86.
This setting enables the following MSR_IA32_U_CET bits:
- CET_ENDBR_EN (enforce endbr as indirect branch target)
- CET_NOTRACK_EN (jump modifier to opt-out of IBT checking)
Kernel-mode IBT (as part of CFI) bans notrack. A future prctl flag could be introduced to ban notrack in usermode too.
--- arch/x86/include/asm/ibt.h | 14 +++++ arch/x86/include/asm/processor.h | 5 ++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/ibt.c | 98 ++++++++++++++++++++++++++++++++ arch/x86/kernel/process_64.c | 2 + 5 files changed, 120 insertions(+) create mode 100644 arch/x86/kernel/ibt.c
arch/x86/include/asm/ibt.h +14 -0
@@ -114,4 +114,18 @@ static inline void ibt_restore(u64 save) { }#define ENDBR_INSN_SIZE (4*HAS_KERNEL_IBT) +#ifndef __ASSEMBLER__ + +#include <linux/prctl.h> + +#define PR_CFI_SUPPORTED_STATUS_MASK (PR_CFI_ENABLE | PR_CFI_DISABLE | PR_CFI_LOCK) + +#ifdef CONFIG_X86_USER_IBT +void reset_thread_ibt (void ); +#else +static inline void reset_thread_ibt (void ) {} +#endif /* CONFIG_X86_USER_IBT */ + +#endif /* __ASSEMBLER__ */ +#endif /* _ASM_X86_IBT_H */
arch/x86/include/asm/processor.h +5 -0
@@ -504,6 +504,11 @@ struct thread_struct { unsigned int iopl_warn:1; +#ifdef CONFIG_X86_USER_IBT + unsigned int ibt:1; + unsigned int ibt_locked:1; +#endif +/* * Protection Keys Register for Userspace. Loaded immediately on * context switch. Store it in thread_struct to avoid a lookup in
arch/x86/kernel/Makefile +1 -0
@@ -169,6 +169,7 @@ obj-$(CONFIG_CALL_THUNKS) += callthunks.o obj-$(CONFIG_X86_CET) += cet.o obj-$(CONFIG_X86_USER_SHADOW_STACK) += shstk.o +obj-$(CONFIG_X86_USER_IBT) += ibt.o ### # 64 bit specific files
arch/x86/kernel/ibt.c new file +98 -0
98 lines not on this page · raw message
arch/x86/kernel/process_64.c +2 -0
@@ -59,6 +59,7 @@#include <asm/fsgsbase.h> #include <asm/fred.h> #include <asm/msr.h> +#include <asm/ibt.h> #ifdef CONFIG_IA32_EMULATION /* Not included via unistd.h */ #include <asm/unistd_32_ia32.h>
@@ -540,6 +541,7 @@ start_thread_common(struct pt_regs *regs, unsigned long new_ip, }reset_thread_features (); +reset_thread_ibt ();loadsegment (fs,0 );loadsegment (es, _ds);
Signature · 1 line
2.47.3
PATCH v2 4/5 x86/entry/vdso: build with IBT support
In reply to Richard PatelVDSO should expose ENDBR instructions now that usermode IBT is available.
--- arch/x86/entry/vdso/common/Makefile.include | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/entry/vdso/common/Makefile.include +1 -2
@@ -46,8 +46,7 @@ flags-y += -fasynchronous-unwind-tables # Reset cf protections enabled by compiler default flags-y += $(call cc-option, -fcf-protection=none) flags-$(X86_USER_SHADOW_STACK) += $(call cc-option, -fcf-protection=return) -# When user space IBT is supported, enable this. -# flags-$(CONFIG_USER_IBT) += $(call cc-option, -fcf-protection=branch) +flags-$(CONFIG_X86_USER_IBT) += $(call cc-option, -fcf-protection=branch) flags-$(CONFIG_MITIGATION_RETPOLINE) += $(RETPOLINE_VDSO_CFLAGS)
Signature · 1 line
2.47.3
PATCH v2 5/5 selftests/x86: test usermode IBT
In reply to Richard PatelAdds a basic selftest exercising a usermode IBT violation.
--- tools/testing/selftests/x86/Makefile | 5 +- tools/testing/selftests/x86/user_ibt.c | 247 +++++++++++++++++++++++++ 2 files changed, 251 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/x86/user_ibt.c
tools/testing/selftests/x86/Makefile +4 -1
@@ -13,7 +13,7 @@ CAN_BUILD_WITH_NOPIE := $(shell ./check_cc.sh "$(CC)" trivial_program.c -no-pie) TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt test_mremap_vdso \ check_initial_reg_state sigreturn iopl ioperm \ test_vsyscall mov_ss_trap sigtrap_loop \ - syscall_arg_fault fsgsbase_restore sigaltstack + syscall_arg_fault fsgsbase_restore sigaltstack user_ibt TARGETS_C_BOTHBITS += nx_stack TARGETS_C_32BIT_ONLY := entry_from_vm86 test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \
@@ -138,3 +138,6 @@ $(OUTPUT)/avx_64: CFLAGS += -mno-avx -mno-avx512f $(OUTPUT)/amx_64: EXTRA_FILES += xstate.c $(OUTPUT)/avx_64: EXTRA_FILES += xstate.c $(OUTPUT)/apx_64: EXTRA_FILES += xstate.c + +$(OUTPUT)/user_ibt_32: CFLAGS += -fcf-protection=branch +$(OUTPUT)/user_ibt_64: CFLAGS += -fcf-protection=branch
tools/testing/selftests/x86/user_ibt.c new file +247 -0
248 lines not on this page · raw message
Signature · 1 line
2.47.3
Quoted 10 lines · Richard Patel
The above sequence does not crash.
With IBT, it should crash at the nop (because an endr64 is expected there). The IBT state (
WAIT_FOR_ENDBRinIA32_U_CETMSR) is not backed up to the signal frame though. So, when userland does a sigreturn, the CPU has forgotten that it was doing an indirect branch before the signal. (This specifically only occurs with signal handlers that sigreturn.)This is because
IA32_U_CETis part of XSAVE 'supervisor' state, so regular XSAVE/XRSTOR can't access it. Doing a manual backup is tricky.
WAIT_FOR_ENDBR should be part of the exception frame with FRED, so if you're on FRED hardware, this should be fixed IIRC.
Re: [PATCH v2 4/5] x86/entry/vdso: build with IBT support
In reply to Richard Patel* Richard Patel:
Quoted 13 lines
VDSO should expose ENDBR instructions now that usermode IBT is available.
Signed-off-by Richard Patel <[email protected]>--- arch/x86/entry/vdso/common/Makefile.include | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)arch/x86/entry/vdso/common/Makefile.include +0
-0
arch/x86/entry/vdso/common/Makefile.include flags-y += -fasynchronous-unwind-tables · line 51
Quoted 8 lines
arch/x86/entry/vdso/common/Makefile.include +1
-2@@ -46,8 +46,7 @@ flags-y += -fasynchronous-unwind-tables # Reset cf protections enabled by compiler default flags-y += $(call cc-option, -fcf-protection=none) flags-$(X86_USER_SHADOW_STACK) += $(call cc-option, -fcf-protection=return) -# When user space IBT is supported, enable this. -# flags-$(CONFIG_USER_IBT) += $(call cc-option, -fcf-protection=branch) +flags-$(CONFIG_X86_USER_IBT) += $(call cc-option, -fcf-protection=branch) flags-$(CONFIG_MITIGATION_RETPOLINE) += $(RETPOLINE_VDSO_CFLAGS)
Surprisingly, the option is cumulative in GCC. I didn't expect that. So this in fact works with GCC.
However, it's not cumulative in Clang, and -fcf-protection=branch overrides the earlier -fcf-protection=return.
Thanks,
Florian
* Richard Patel:
Quoted 10 lines
Adds basic support for x86 userspace IBT.
IBT is part of Intel CET. It requires indirect call and jump targets to start with an endbr{32,64} instruction, otherwise throwing #CP.
In summary, this patch does 3 things:
- Config wiring ensuring supervisor XSAVE contains IBT state
- Allow userspace to enable IBT viaprctl(PR_CFI_*)for an entire thread
- Enable IBT support (ENDBR instructions) in VDSO
Quoted 6 lines
Unlike the arm64 BTI API:
- does not support mixed usermode (all or nothing)
- does not touch page table code
- not enabled automatically (no ELF GNU note parsing)
- temporarily disables IBT enforcement when handling signals These can all be cleanly added later.
Adding the ELF GNU note parsing can be added later, but perhaps not cleanly. I'm still a bit worried we might have to rev the markup because too many binaries are in circulation that claim compatibility, have never been tested, and are actually broken. If the kernel does not look at the ELF bits, things a slightly simpler.
How do you detect that handling a signal is complete and IBT can be re-enabled? Or is it re-enabled before entering the userspace signal handler?
Quoted 1 line
The main question is whether glibc is happy with this prctl syscall API.
As far as I can tell, the prctl works for glibc. Re-use of an arch_prctl constant might have been problematic, but the series is not doing that.
Quoted 10 lines
There is one notable gap in this patch series, to do with signals:
000a: mov rax,
0x100a
000f: jmp rax
*** signal occurs ***
*** signal handler runs, does sigreturn ***
100a: nopThe above sequence does not crash.
Quoted 8 lines
With IBT, it should crash at the nop (because an endr64 is expected there). The IBT state (
WAIT_FOR_ENDBRinIA32_U_CETMSR) is not backed up to the signal frame though. So, when userland does a sigreturn, the CPU has forgotten that it was doing an indirect branch before the signal. (This specifically only occurs with signal handlers that sigreturn.)This is because
IA32_U_CETis part of XSAVE 'supervisor' state, so regular XSAVE/XRSTOR can't access it. Doing a manual backup is tricky.
That's a bit annoying. Is this restricted to signal handlers, or does it apply to page faults, too?
Quoted 2 lines
A related problem is that the signal handler routine is not checked for endbr preamble.
That's not necessarily a problem because its address cannot be directly overwritten in userspace. Not all indirect branches need to be checked, only those that have tweakable targets. In fact, fewer ENDBR64 markers are better (although we wouldn't drop the marker from a signal handler specifically, of course).
Thanks,
Florian
Quoted 3 lines · Florian Weimer
How do you detect that handling a signal is complete and IBT can be re-enabled? Or is it re-enabled before entering the userspace signal handler?
Hi Florian,
In v1, we backed up the IBT CPU state into the (user-accessible) signal frame from FRED/XSAVE, then restored it:
https://lore.kernel.org/lkml/[email protected]/
In v2, when entering the signal handler, the kernel just context switches to the new user rip, bypassing IBT checks (continues executing if the signal handler does not begin with endbr).
IBT stays enabled in both designs, just the IBT state is preserved in v1, and lost in v2.
The same thing happens when doing a sigreturn in v2 (e.g. via trampoline), again IBT is not enforced. IBT stays enabled when doing a siglongjmp, though.
Some time in the future, ideally:
- signal handler is *required* to start with endbr (this is easy)
- sigreturn as in my asm example enforces endbr after returning from a signal handler to a in-progres indirect branc
- libc (sig)longjmp is made IBT-compatible
Btw, I had self-tests for the v1 design, and {signal handle,rt_sigreturn, siglongjmp} with {success case,violation} works flawlessly with Fedora 44 glibc amd64. With glibc i686 I ran into PLT issues, probably my fault.
It is quite surprised that siglongjmp was working, btw, since the glibc longjmp code uses 'jmp *reg' (without notrack prefix). I guess you do an endbr64 at the setjmp side?
Quoted 5 lines
Quoted 1 line
The main question is whether glibc is happy with this prctl syscall API.
As far as I can tell, the prctl works for glibc. Re-use of an
arch_prctlconstant might have been problematic, but the series is not doing that.
Nice :-)
The alternative would have been to bolt on stuff to ARCH_SHSTK, or create an entirely new arch_prctl. Open to any API.
Quoted 5 lines
Adding the ELF GNU note parsing can be added later, but perhaps not cleanly. I'm still a bit worried we might have to rev the markup because too many binaries are in circulation that claim compatibility, have never been tested, and are actually broken. If the kernel does not look at the ELF bits, things a slightly simpler.
Phew, I was hoping you'd say that.
If you want, I can sketch out glibc IBT enabling and test it on Debian and Fedora, which IIRC already emit compile with -fcf-protection=branch for all OS packages.
Quoted 16 lines
Quoted 16 lines
There is one notable gap in this patch series, to do with signals:
000a: mov rax,
0x100a
000f: jmp rax
*** signal occurs ***
*** signal handler runs, does sigreturn ***
100a: nopThe above sequence does not crash.
With IBT, it should crash at the nop (because an endr64 is expected there). The IBT state (
WAIT_FOR_ENDBRinIA32_U_CETMSR) is not backed up to the signal frame though. So, when userland does a sigreturn, the CPU has forgotten that it was doing an indirect branch before the signal. (This specifically only occurs with signal handlers that sigreturn.)
Quoted 5 lines
Quoted 2 lines
This is because
IA32_U_CETis part of XSAVE 'supervisor' state, so regular XSAVE/XRSTOR can't access it. Doing a manual backup is tricky.That's a bit annoying. Is this restricted to signal handlers, or does it apply to page faults, too?
Only signal handlers, page faults don't reset IBT.
Quoted 8 lines
Quoted 2 lines
A related problem is that the signal handler routine is not checked for endbr preamble.
That's not necessarily a problem because its address cannot be directly overwritten in userspace. Not all indirect branches need to be checked, only those that have tweakable targets. In fact, fewer ENDBR64 markers are better (although we wouldn't drop the marker from a signal handler specifically, of course).
Just one concern I have is that people start relying on signal handlers not requiring endbr64, and then a future kernel version breaking them once we enforce it.
Really appreciate your review,
-Richard
* Richard Patel:
Quoted 8 lines · Florian Weimer
Quoted 4 lines
How do you detect that handling a signal is complete and IBT can be re-enabled? Or is it re-enabled before entering the userspace signal handler?
Hi Florian,
Quoted 7 lines
In v1, we backed up the IBT CPU state into the (user-accessible) signal frame from FRED/XSAVE, then restored it:
https://lore.kernel.org/lkml/[email protected]/In v2, when entering the signal handler, the kernel just context switches to the new user rip, bypassing IBT checks (continues executing if the signal handler does not begin with endbr).
What's the reason for this?
Quoted 5 lines
Some time in the future, ideally:
- signal handler is *required* to start with endbr (this is easy)
- sigreturn as in my asm example enforces endbr after returning from a signal handler to a in-progres indirect branc
- libc (sig)longjmp is made IBT-compatible
I think the compiler already emits ENDBR markers for returns-twice functions, which is why longjmp does not use a no-track jump. Other architectures require such a proliferation of markers because they do not support no-track jumps at all. However, longjmp is arguable a corner case. It's not completely safe, like loading a function address from a RELRO GOT and jumping to it.
Quoted 3 lines
Btw, I had self-tests for the v1 design, and {signal handle,rt_sigreturn, siglongjmp} with {success case,violation} works flawlessly with Fedora 44 glibc amd64. With glibc i686 I ran into PLT issues, probably my fault.
There's no IBT support planned for i686, that's why we dropped all marker instructions in Fedora.
Quoted 3 lines
It is quite surprised that siglongjmp was working, btw, since the glibc longjmp code uses 'jmp *reg' (without notrack prefix). I guess you do an endbr64 at the setjmp side?
Yes, compilers generate landing pads for returns-twice functions. Not ideal, but it's the only way to get setjmp working on targets without NOTRACK.
Quoted 11 lines
Quoted 5 lines
Adding the ELF GNU note parsing can be added later, but perhaps not cleanly. I'm still a bit worried we might have to rev the markup because too many binaries are in circulation that claim compatibility, have never been tested, and are actually broken. If the kernel does not look at the ELF bits, things a slightly simpler.
Phew, I was hoping you'd say that.
If you want, I can sketch out glibc IBT enabling and test it on Debian and Fedora, which IIRC already emit compile with -fcf-protection=branch for all OS packages.
For Fedora, please coordinate with Arjun (Cc:ed), who is going through the motions of enabling SHSTK for real.
Quoted 9 lines
Quoted 5 lines
That's not necessarily a problem because its address cannot be directly overwritten in userspace. Not all indirect branches need to be checked, only those that have tweakable targets. In fact, fewer ENDBR64 markers are better (although we wouldn't drop the marker from a signal handler specifically, of course).
Just one concern I have is that people start relying on signal handlers not requiring endbr64, and then a future kernel version breaking them once we enforce it.
Would software enforcement be a possibility? The kernel could check if the landing pad is there.
Thanks,
Florian
Quoted 14 lines · Florian Weimer
* Richard Patel:
Quoted 12 lines · Florian Weimer
Quoted 4 lines
How do you detect that handling a signal is complete and IBT can be re-enabled? Or is it re-enabled before entering the userspace signal handler?
Hi Florian,
In v1, we backed up the IBT CPU state into the (user-accessible) signal frame from FRED/XSAVE, then restored it:
https://lore.kernel.org/lkml/[email protected]/
Quoted 5 lines
Quoted 3 lines
In v2, when entering the signal handler, the kernel just context switches to the new user rip, bypassing IBT checks (continues executing if the signal handler does not begin with endbr).
What's the reason for this?
Hi Florian,
We just don't have a nice way to include IBT state in the signal frame right now. v1 had an uabi change (adding a new bit in ucontext_t uc_flags), which was originally proposed by Intel years ago. My preferred way to add IBT state is to carve out an XSAVE area in fpstate, which works well with all the existing signal frame code.
But I figured it's better to just keep the first pass at user IBT super simple, in the hopes upstream is more inclined to accept that.
BTW, OpenBSD uses the v2 approach (don't preserve IBT state across signal handlers), presumably because it's also hard for them to restore IBT state on sigreturn.
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
That's not necessarily a problem because its address cannot be directly overwritten in userspace. Not all indirect branches need to be checked, only those that have tweakable targets. In fact, fewer ENDBR64 markers are better (although we wouldn't drop the marker from a signal handler specifically, of course).
Quoted 6 lines
Quoted 3 lines
Just one concern I have is that people start relying on signal handlers not requiring endbr64, and then a future kernel version breaking them once we enforce it.
Would software enforcement be a possibility? The kernel could check if the landing pad is there.
Enforcement is the easy part. I can trivially add back 'check if signal handler starts with endbr64'. Just the backup/restore of the pre-signal handler state ('do I expect an endbr64 after returning') is the tricky part.
Thank you,
-Richard