Enable Linear Address Space Separation support
Subsystems (5)
v3 cover Enable Linear Address Space Separation support
Changes from v2[5]:
- Added myself to the SoB chain
Changes from v1[1]:
- Emulate vsyscall violations in execute mode in the #GP fault handler
- Use inline memcpy and memset while patching alternatives
- Remove CONFIG_X86_LASS
- Make LASS depend on SMAP
- Dropped the minimal KVM enabling patch
Linear Address Space Separation (LASS) is a security feature that intends to prevent malicious virtual address space accesses across user/kernel mode.
Such mode based access protection already exists today with paging and features such as SMEP and SMAP. However, to enforce these protections, the processor must traverse the paging structures in memory. Malicious software can use timing information resulting from this traversal to determine details about the paging structures, and these details may also be used to determine the layout of the kernel memory.
The LASS mechanism provides the same mode-based protections as paging but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, software will not be able to derive paging-based timing information from the various caching structures such as the TLBs, mid-level caches, page walker, data caches, etc. LASS can avoid probing using double page faults, TLB flush and reload, and SW prefetch instructions. See [2], [3] and [4] for some research on the related attack vectors.
LASS enforcement relies on the typical kernel implemetation to divide the 64-bit virtual address space into two halves:
Addr[63]=0 -> User address space
Addr[63]=1 -> Kernel address space
Any data access or code execution across address spaces typically results in a #GP fault.
Kernel accesses usually only happen to the kernel address space. However, there are valid reasons for kernel to access memory in the user half. For these cases (such as text poking and EFI runtime accesses), the kernel can temporarily suspend the enforcement of LASS by toggling SMAP (Supervisor Mode Access Prevention) using the stac()/clac() instructions.
User space cannot access any kernel address while LASS is enabled. Unfortunately, legacy vsyscall functions are located in the address range 0xffffffffff600000 - 0xffffffffff601000 and emulated in kernel. To avoid breaking user applications when LASS is enabled, extend the vsyscall emulation in execute (XONLY) mode to the #GP fault handler.
In contrast, the vsyscall EMULATE mode is deprecated and not expected to be used by anyone. Supporting EMULATE mode with LASS would need complex intruction decoding in the #GP fault handler and is probably not worth the hassle. Disable LASS in this rare case when someone absolutely needs and enables vsyscall=emulate via the command line.
As of now there is no publicly available CPU supporting LASS. The first one to support LASS would be the Sierra Forest line. The Intel Simics® Simulator was used as software development and testing vehicle for this patch set.
[1] https://lore.kernel.org/lkml/[email protected]/
[2] “Practical Timing Side Channel Attacks against Kernel Space ASLR”,
https://www.ieee-security.org/TC/SP2013/papers/4977a191.pdf
[3] “Prefetch Side-Channel Attacks: Bypassing SMAP and Kernel ASLR”, http://doi.acm.org/10.1145/2976749.2978356
[4] “Harmful prefetch on Intel”, https://ioactive.com/harmful-prefetch-on-intel/ (H/T Anders)
[5] https://lore.kernel.org/all/[email protected]/
Alexander Shishkin (1):
x86/vsyscall: Document the fact that vsyscall=emulate disables LASS
Peter Zijlstra (1):
x86/asm: Introduce inline memcpy and memset
Sohil Mehta (9):
x86/cpu: Enumerate the LASS feature bits x86/alternatives: Disable LASS when patching kernel alternatives x86/cpu: Enable LASS during CPU initialization x86/cpu: Remove redundant comment during feature setup x86/vsyscall: Reorganize the #PF emulation code x86/traps: Consolidate user fixups in exc_general_protection() x86/vsyscall: Add vsyscall emulation for #GP x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE [RFC] x86/efi: Disable LASS enforcement when switching to EFI MM
Yian Chen (1):
x86/cpu: Set LASS CR4 bit as pinning sensitive
.../admin-guide/kernel-parameters.txt | 4 +- arch/x86/entry/vsyscall/vsyscall_64.c | 70 ++++++++++++++----- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 4 +- arch/x86/include/asm/smap.h | 4 ++ arch/x86/include/asm/string_32.h | 21 ++++++ arch/x86/include/asm/string_64.h | 21 ++++++ arch/x86/include/asm/vsyscall.h | 16 +++-- arch/x86/include/uapi/asm/processor-flags.h | 2 + arch/x86/kernel/alternative.c | 12 +++- arch/x86/kernel/cpu/common.c | 10 ++- arch/x86/kernel/cpu/cpuid-deps.c | 1 + arch/x86/kernel/traps.c | 12 ++-- arch/x86/mm/fault.c | 13 +--- arch/x86/platform/efi/efi_64.c | 6 ++ tools/arch/x86/include/asm/cpufeatures.h | 1 + 16 files changed, 153 insertions(+), 45 deletions(-)
Signature · 1 line
2.39.2
PATCH v3 1/12 x86/cpu: Enumerate the LASS feature bits
From: Sohil Mehta <[email protected]>
Linear Address Space Separation (LASS) is a security feature that intends to prevent malicious virtual address space accesses across user/kernel mode.
Such mode based access protection already exists today with paging and features such as SMEP and SMAP. However, to enforce these protections, the processor must traverse the paging structures in memory. Malicious software can use timing information resulting from this traversal to determine details about the paging structures, and these details may also be used to determine the layout of the kernel memory.
The LASS mechanism provides the same mode-based protections as paging but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, software will not be able to derive paging-based timing information from the various caching structures such as the TLBs, mid-level caches, page walker, data caches, etc.
LASS enforcement relies on the typical kernel implementation to divide the 64-bit virtual address space into two halves:
Addr[63]=0 -> User address space
Addr[63]=1 -> Kernel address space
Any data access or code execution across address spaces typically results in a #GP fault.
The LASS enforcement for kernel data access is dependent on CR4.SMAP being set. The enforcement can be disabled by toggling the RFLAGS.AC bit similar to SMAP.
Define the CPU feature bits to enumerate this feature and include feature dependencies to reflect the same.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 4 +++- arch/x86/include/asm/smap.h | 4 ++++ arch/x86/include/uapi/asm/processor-flags.h | 2 ++ arch/x86/kernel/cpu/cpuid-deps.c | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 6 files changed, 12 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -312,6 +312,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_LASS (12*32+ 6)/* Linear Address Space Separation */ #define X86_FEATURE_CMPCCXADD (12*32+ 7)/* "" CMPccXADD instructions */ #define X86_FEATURE_ARCH_PERFMON_EXT (12*32+ 8)/* "" Intel Architectural PerfMon Extension */ #define X86_FEATURE_FZRM (12*32+10)/* "" Fast zero-length REP MOVSB */
arch/x86/include/asm/disabled-features.h +3 -1
@@ -22,12 +22,14 @@# define DISABLE_CYRIX_ARR (1<<(X86_FEATURE_CYRIX_ARR & 31))# define DISABLE_CENTAUR_MCR (1<<(X86_FEATURE_CENTAUR_MCR & 31))# define DISABLE_PCID 0 +# define DISABLE_LASS 0#else # define DISABLE_VME 0# define DISABLE_K6_MTRR 0# define DISABLE_CYRIX_ARR 0# define DISABLE_CENTAUR_MCR 0# define DISABLE_PCID (1<<(X86_FEATURE_PCID & 31)) +# define DISABLE_LASS (1<<(X86_FEATURE_LASS & 31))#endif /* CONFIG_X86_64 */ #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
@@ -122,7 +124,7 @@#define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING)#define DISABLED_MASK12 (DISABLE_LAM) -#define DISABLED_MASK13 0 +#define DISABLED_MASK13 (DISABLE_LASS)#define DISABLED_MASK14 0#define DISABLED_MASK15 0#define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP| \
arch/x86/include/asm/smap.h +4 -0
@@ -27,6 +27,10 @@#else /* __ASSEMBLY__ */ +/* +* The CLAC/STAC instructions toggle enforcement of X86_FEATURE_SMAP as well as +* X86_FEATURE_LASS. +*/ static __always_inline void clac(void) {/* Note: a barrier is implicit in alternative() */
arch/x86/include/uapi/asm/processor-flags.h +2 -0
@@ -136,6 +136,8 @@#define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT)#define X86_CR4_CET_BIT 23/* enable Control-flow Enforcement Technology */ #define X86_CR4_CET _BITUL(X86_CR4_CET_BIT) +#define X86_CR4_LASS_BIT 27/* enable Linear Address Space Separation support */ +#define X86_CR4_LASS _BITUL(X86_CR4_LASS_BIT)#define X86_CR4_LAM_SUP_BIT 28/* LAM for supervisor pointers */ #define X86_CR4_LAM_SUP _BITUL(X86_CR4_LAM_SUP_BIT)
arch/x86/kernel/cpu/cpuid-deps.c +1 -0
@@ -81,6 +81,7 @@ static const struct cpuid_dep cpuid_deps[] = { { X86_FEATURE_XFD, X86_FEATURE_XSAVES }, { X86_FEATURE_XFD, X86_FEATURE_XGETBV1 }, { X86_FEATURE_AMX_TILE, X86_FEATURE_XFD }, + { X86_FEATURE_LASS, X86_FEATURE_SMAP }, {} };
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -311,6 +311,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_LASS (12*32+ 6)/* Linear Address Space Separation */ #define X86_FEATURE_CMPCCXADD (12*32+ 7)/* "" CMPccXADD instructions */ #define X86_FEATURE_LKGS (12*32+18)/* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */
Signature · 1 line
2.39.2
PATCH v3 2/12 x86/asm: Introduce inline memcpy and memset
In reply to Alexander ShishkinFrom: Peter Zijlstra <[email protected]>
Provide inline memcpy and memset functions that can be used instead of the GCC builtins whenever necessary.
Code posted by Peter Zijlstra <[email protected]>.
--- arch/x86/include/asm/string_32.h | 21 +++++++++++++++++++++ arch/x86/include/asm/string_64.h | 21 +++++++++++++++++++++ 2 files changed, 42 insertions(+)
arch/x86/include/asm/string_32.h +21 -0
@@ -151,6 +151,16 @@ extern void *memcpy(void *, const void *, size_t);#endif /* !CONFIG_FORTIFY_SOURCE */ +static __always_inline void *__inline_memcpy(void *to, const void *from, size_t len) +{ +void *ret = to; + + asmvolatile ("rep movsb" + :"+D" (to),"+S" (from),"+c" (len) + : :"memory" ); +return ret; +} +#define __HAVE_ARCH_MEMMOVEvoid *memmove (void *dest,const void *src,size_t n);
@@ -195,6 +205,17 @@ extern void *memset(void *, int, size_t);#define memset (s, c, count) __builtin_memset(s, c, count)#endif /* !CONFIG_FORTIFY_SOURCE */ +static __always_inline void *__inline_memset(void *s, int v, size_t n) +{ +void *ret = s; + + asmvolatile ("rep stosb" + :"+D" (s),"+c" (n) + :"a" ((uint8_t )v) + :"memory" ); +return ret; +} +#define __HAVE_ARCH_MEMSET16static inline void *memset16 (uint16_t *s,uint16_t v,size_t n) {
arch/x86/include/asm/string_64.h +21 -0
@@ -18,10 +18,31 @@extern void *memcpy (void *to,const void *from,size_t len);extern void *__memcpy (void *to,const void *from,size_t len); +static __always_inline void *__inline_memcpy(void *to, const void *from, size_t len) +{ +void *ret = to; + + asmvolatile ("rep movsb" + :"+D" (to),"+S" (from),"+c" (len) + : :"memory" ); +return ret; +} +#define __HAVE_ARCH_MEMSETvoid *memset (void *s,int c,size_t n);void *__memset (void *s,int c,size_t n); +static __always_inline void *__inline_memset(void *s, int v, size_t n) +{ +void *ret = s; + + asmvolatile ("rep stosb" + :"+D" (s),"+c" (n) + :"a" ((uint8_t )v) + :"memory" ); +return ret; +} +/* * KMSAN needs to instrument as much code as possible. Use C versions of * memsetXX() from lib/string.c under KMSAN.
Signature · 1 line
2.39.2
PATCH v3 3/12 x86/alternatives: Disable LASS when patching kernel alternatives
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
For patching, the kernel initializes a temporary mm area in the lower half of the address range. See commit 4fc19708b165 ("x86/alternatives:
Initialize temporary mm for patching").
Disable LASS enforcement during patching using the stac()/clac() instructions to avoid triggering a #GP fault.
The objtool warns due to a call to a non-allowed function that exists outside of the stac/clac guard, or references to any function with a dynamic function pointer inside the guard. See the Objtool warnings section #9 in the document tools/objtool/Documentation/objtool.txt.
Considering that patching is usually small, replace the memcpy and memset functions in the text poking functions with their inline versions respectively.
--- arch/x86/kernel/alternative.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/alternative.c +10 -2
@@ -1526,16 +1526,24 @@ static inline void unuse_temporary_mm(temp_mm_state_t prev_state)__ro_after_init struct mm_struct *poking_mm;__ro_after_init unsigned long poking_addr; +/* +* poking_init() initializes the text poking address from the lower half of the +* address space. Relax LASS enforcement when accessing the poking address. +*/ static void text_poke_memcpy (void *dst,const void *src,size_t len) { -memcpy (dst, src, len); +stac (); +__inline_memcpy (dst, src, len); +clac (); }static void text_poke_memset (void *dst,const void *src,size_t len) {int c = *(const int *)src; -memset (dst, c, len); +stac (); +__inline_memset (dst, c, len); +clac (); }typedef void text_poke_f (void *dst,const void *src,size_t len);
Signature · 1 line
2.39.2
Re: [PATCH v3 03/12] x86/alternatives: Disable LASS when patching kernel alternatives
Quoted 15 lines · Alexander Shishkin
+/*
+ *poking_init()initializes the text poking address from the lower half of the
+ * address space. Relax LASS enforcement when accessing the poking address.
+ */
static voidtext_poke_memcpy(void *dst, const void *src, size_t len){
- memcpy(dst, src, len);
+ stac();
+ __inline_memcpy(dst, src, len);
+ clac();
}
static voidtext_poke_memset(void *dst, const void *src, size_t len)
Quoted 8 lines
{
int c = *(const int *)src;
- memset(dst, c, len);
+ stac();
+ __inline_memset(dst, c, len);
+ clac();
}
Why not do stac/clac in a single place inside __text_poke()?
Re: [PATCH v3 03/12] x86/alternatives: Disable LASS when patching kernel alternatives
Quoted 1 line
Why not do stac/clac in a single place inside
__text_poke()?
It would mostly look something like this:
Quoted 5 lines
arch/x86/kernel/alternative.c +0
-0
arch/x86/kernel/alternative.c static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l · line 1789
Quoted 9 lines
arch/x86/kernel/alternative.c +2
-0@@ -1781,7 +1781,9 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l prev =use_temporary_mm (poking_mm);kasan_disable_current (); +stac ();func ((u8 *)poking_addr +offset_in_page (addr), src, len); +clac ();kasan_enable_current ();/*
Since, __text_poke() uses a dynamic function to call into text_poke_memcpy() and text_poke_memset(), objtool would still complain.
Quoted 1 line
arch/x86/kernel/alternative.o: warning: objtool:__text_poke+0x259: call to {dynamic}() with UACCESS enabled
We could change __text_poke() to not use the dynamic func but it might be a bit heavy handed to save a couple of lines of stac/clac calls. The current trade-off seems reasonable to me.
Did you have something different in mind?
Sohil
Re: [PATCH v3 03/12] x86/alternatives: Disable LASS when patching kernel alternatives
Quoted 27 lines · Sohil Mehta
Quoted 1 line
Why not do stac/clac in a single place inside
__text_poke()?It would mostly look something like this:
Quoted 9 lines
arch/x86/kernel/alternative.c +0
-0@@ -1781,7 +1781,9 @@ static void *__text_poke(text_poke_f func,void *addr, const void *src,
size_tl
prev =use_temporary_mm(poking_mm);Quoted 7 lines
kasan_disable_current();
+stac();
func((u8 *)poking_addr + offset_in_page(addr), src, len); +clac();
kasan_enable_current();/*
Since,
__text_poke()uses a dynamic function to call intotext_poke_memcpy()andtext_poke_memset(), objtool would still complain.Quoted 2 lines
arch/x86/kernel/alternative.o: warning: objtool:__text_poke+0x259:
call to {dynamic}() with UACCESS enabled
Quoted 7 lines
We could change
__text_poke()to not use the dynamic func but it might
be a bit heavy handed to save a couple of lines of stac/clac calls. The
current trade-off seems reasonable to me.Did you have something different in mind?
I wondered if it might be something like that. Yes, seems like an ok tradeoff.
PATCH v3 4/12 x86/cpu: Enable LASS during CPU initialization
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
Being a security feature, enable LASS by default if the platform supports it.
--- arch/x86/kernel/cpu/common.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/cpu/common.c +7 -0
@@ -413,6 +413,12 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)cr4_clear_bits (X86_CR4_UMIP ); } +static __always_inline void setup_lass(struct cpuinfo_x86 *c) +{ +if (cpu_feature_enabled (X86_FEATURE_LASS )) +cr4_set_bits (X86_CR4_LASS ); +} +/* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
@@ -1859,6 +1865,7 @@ static void identify_cpu(struct cpuinfo_x86 *c)setup_smep (c);setup_smap (c);setup_umip (c); +setup_lass (c);/* Enable FSGSBASE instructions if available. */ if (cpu_has (c,X86_FEATURE_FSGSBASE )) {
Signature · 1 line
2.39.2
PATCH v3 5/12 x86/cpu: Remove redundant comment during feature setup
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
The code below the comment is self explanatory. Instead of updating the comment with the newly added LASS feature, it is better to just remove it.
--- arch/x86/kernel/cpu/common.c | 1 - 1 file changed, 1 deletion(-)
arch/x86/kernel/cpu/common.c +0 -1
@@ -1861,7 +1861,6 @@ static void identify_cpu(struct cpuinfo_x86 *c)/* Disable the PN if appropriate */ squash_the_stupid_serial_number (c); -/* Set up SMEP/SMAP/UMIP */ setup_smep (c);setup_smap (c);setup_umip (c);
Signature · 1 line
2.39.2
PATCH v3 6/12 x86/vsyscall: Reorganize the #PF emulation code
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
Separate out the actual vsyscall emulation from the page fault specific handling in preparation for the upcoming #GP fault emulation.
Export is_vsyscall_vaddr() so that it can be reused later.
No functional change intended.
--- arch/x86/entry/vsyscall/vsyscall_64.c | 51 +++++++++++++++++---------- arch/x86/include/asm/vsyscall.h | 10 +++--- arch/x86/mm/fault.c | 13 ++----- 3 files changed, 41 insertions(+), 33 deletions(-)
arch/x86/entry/vsyscall/vsyscall_64.c +33 -18
@@ -82,6 +82,15 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs, regs->sp, regs->ax, regs->si, regs->di); } +/* +* The (legacy) vsyscall page is the long page in the kernel portion +* of the address space that has user-accessible permissions. +*/ +bool is_vsyscall_vaddr (unsigned long vaddr) +{ +return unlikely ((vaddr &PAGE_MASK ) ==VSYSCALL_ADDR ); +} +static int addr_to_vsyscall_nr (unsigned long addr) {int nr;
@@ -117,8 +126,7 @@ static bool write_ok_or_segv(unsigned long ptr, size_t size) } } -bool emulate_vsyscall (unsigned long error_code, -struct pt_regs *regs,unsigned long address) +static bool __emulate_vsyscall (struct pt_regs *regs,unsigned long address) {struct task_struct *tsk;unsigned long caller;
@@ -127,22 +135,6 @@ bool emulate_vsyscall(unsigned long error_code,long ret;unsigned long orig_dx; -/* Write faults or kernel-privilege faults never get fixed up. */ -if ((error_code & (X86_PF_WRITE |X86_PF_USER )) !=X86_PF_USER ) -return false; - -if (!(error_code &X86_PF_INSTR )) { -/* Failed vsyscall read */ -if (vsyscall_mode ==EMULATE ) -return false; - -/* -* User code tried and failed to read the vsyscall page. -*/ -warn_bad_vsyscall (KERN_INFO , regs,"vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround" ); -return false; - } -/* * No point in checking CS -- the only way to get here is a user mode * trap to a high address, which means that we're in 64-bit user code.
@@ -294,6 +286,29 @@ bool emulate_vsyscall(unsigned long error_code,return true; } +bool emulate_vsyscall_pf (unsigned long error_code,struct pt_regs *regs, +unsigned long address) +{ +/* Write faults or kernel-privilege faults never get fixed up. */ +if ((error_code & (X86_PF_WRITE |X86_PF_USER )) !=X86_PF_USER ) +return false; + +if (!(error_code &X86_PF_INSTR )) { +/* Failed vsyscall read */ +if (vsyscall_mode ==EMULATE ) +return false; + +/* +* User code tried and failed to read the vsyscall page. +*/ +warn_bad_vsyscall (KERN_INFO , regs, +"vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround" ); +return false; + } + +return __emulate_vsyscall (regs, address); +} +/* * A pseudo VMA to allow ptrace access for the vsyscall page. This only * covers the 64bit vsyscall page now. 32bit has a real VMA now and does
arch/x86/include/asm/vsyscall.h +6 -4
@@ -5,6 +5,8 @@#include <linux/seqlock.h> #include <uapi/asm/vsyscall.h> +extern bool is_vsyscall_vaddr (unsigned long vaddr); +#ifdef CONFIG_X86_VSYSCALL_EMULATION extern void map_vsyscall (void );extern void set_vsyscall_pgtable_user_bits (pgd_t *root);
@@ -13,12 +15,12 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root);* Called on instruction fetch fault in vsyscall page. * Returns true if handled. */ -extern bool emulate_vsyscall (unsigned long error_code, -struct pt_regs *regs,unsigned long address); +extern bool emulate_vsyscall_pf (unsigned long error_code, +struct pt_regs *regs,unsigned long address);#else static inline void map_vsyscall (void ) {} -static inline bool emulate_vsyscall (unsigned long error_code, -struct pt_regs *regs,unsigned long address) +static inline bool emulate_vsyscall_pf (unsigned long error_code, +struct pt_regs *regs,unsigned long address) {return false; }
arch/x86/mm/fault.c +2 -11
@@ -730,7 +730,7 @@ kernelmode_fixup_or_oops(struct pt_regs *regs, unsigned long error_code,* Per the above we're !in_interrupt(), aka. task context. * * In this case we need to make sure we're not recursively -* faulting through the emulate_vsyscall() logic. +* faulting through the emulate_vsyscall_pf() logic. */ if (current->thread.sig_on_uaccess_err && signal) {sanitize_error_code (address, &error_code);
@@ -798,15 +798,6 @@ show_signal_msg(struct pt_regs *regs, unsigned long error_code,show_opcodes (regs, loglvl); } -/* -* The (legacy) vsyscall page is the long page in the kernel portion -* of the address space that has user-accessible permissions. -*/ -static bool is_vsyscall_vaddr (unsigned long vaddr) -{ -return unlikely ((vaddr &PAGE_MASK ) ==VSYSCALL_ADDR ); -} -static void __bad_area_nosemaphore (struct pt_regs *regs,unsigned long error_code,unsigned long address,u32 pkey,int si_code)
@@ -1329,7 +1320,7 @@ void do_user_addr_fault(struct pt_regs *regs,* to consider the PF_PK bit. */ if (is_vsyscall_vaddr (address)) { -if (emulate_vsyscall (error_code, regs, address)) +if (emulate_vsyscall_pf (error_code, regs, address))return ; }#endif
Signature · 1 line
2.39.2
PATCH v3 7/12 x86/traps: Consolidate user fixups in exc_general_protection()
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
Move the UMIP exception fixup along with the other user mode fixups.
No functional change intended.
--- arch/x86/kernel/traps.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
arch/x86/kernel/traps.c +3 -5
@@ -736,11 +736,6 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)cond_local_irq_enable (regs); -if (static_cpu_has (X86_FEATURE_UMIP )) { -if (user_mode (regs) &&fixup_umip_exception (regs)) - goto exit; - } -if (v8086_mode (regs)) {local_irq_enable ();handle_vm86_fault ((struct kernel_vm86_regs *) regs, error_code);
@@ -755,6 +750,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)if (fixup_vdso_exception (regs,X86_TRAP_GP , error_code,0 )) goto exit; +if (cpu_feature_enabled (X86_FEATURE_UMIP ) &&fixup_umip_exception (regs)) + goto exit; +gp_user_force_sig_segv (regs,X86_TRAP_GP , error_code, desc); goto exit; }
Signature · 1 line
2.39.2
PATCH v3 8/12 x86/vsyscall: Add vsyscall emulation for #GP
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
The legacy vsyscall page is mapped at a fixed address in the kernel address range 0xffffffffff600000-0xffffffffff601000. Prior to LASS being introduced, a legacy vsyscall page access from userspace would always generate a page fault. The kernel emulates the execute (XONLY) accesses in the page fault handler and returns back to userspace with the appropriate register values.
Since LASS intercepts these accesses before the paging structures are traversed it generates a general protection fault instead of a page fault. The #GP fault doesn't provide much information in terms of the error code. So, use the faulting RIP which is preserved in the user registers to emulate the vsyscall access without going through complex instruction decoding.
--- arch/x86/entry/vsyscall/vsyscall_64.c | 11 ++++++++++- arch/x86/include/asm/vsyscall.h | 6 ++++++ arch/x86/kernel/traps.c | 4 ++++ 3 files changed, 20 insertions(+), 1 deletion(-)
arch/x86/entry/vsyscall/vsyscall_64.c +10 -1
@@ -23,7 +23,7 @@* soon be no new userspace code that will ever use a vsyscall. * * The code in this file emulates vsyscalls when notified of a page -* fault to a vsyscall address. +* fault or a general protection fault to a vsyscall address. */ #include <linux/kernel.h>
@@ -309,6 +309,15 @@ bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs,return __emulate_vsyscall (regs, address); } +bool emulate_vsyscall_gp (struct pt_regs *regs) +{ +/* Emulate only if the RIP points to the vsyscall address */ +if (!is_vsyscall_vaddr (regs->ip)) +return false; + +return __emulate_vsyscall (regs, regs->ip); +} +/* * A pseudo VMA to allow ptrace access for the vsyscall page. This only * covers the 64bit vsyscall page now. 32bit has a real VMA now and does
arch/x86/include/asm/vsyscall.h +6 -0
@@ -17,6 +17,7 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root);*/ extern bool emulate_vsyscall_pf (unsigned long error_code,struct pt_regs *regs,unsigned long address); +extern bool emulate_vsyscall_gp (struct pt_regs *regs);#else static inline void map_vsyscall (void ) {} static inline bool emulate_vsyscall_pf(unsigned long error_code,
@@ -24,6 +25,11 @@ static inline bool emulate_vsyscall_pf(unsigned long error_code, {return false; } + +static inline bool emulate_vsyscall_gp (struct pt_regs *regs) +{ +return false; +}#endif #endif /* _ASM_X86_VSYSCALL_H */
arch/x86/kernel/traps.c +4 -0
@@ -65,6 +65,7 @@#include <asm/vdso.h> #include <asm/tdx.h> #include <asm/cfi.h> +#include <asm/vsyscall.h> #ifdef CONFIG_X86_64 #include <asm/x86_init.h>
@@ -753,6 +754,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)if (cpu_feature_enabled (X86_FEATURE_UMIP ) &&fixup_umip_exception (regs)) goto exit; +if (cpu_feature_enabled (X86_FEATURE_LASS ) &&emulate_vsyscall_gp (regs)) + goto exit; +gp_user_force_sig_segv (regs,X86_TRAP_GP , error_code, desc); goto exit; }
Signature · 1 line
2.39.2
PATCH v3 9/12 x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
The EMULATE mode of vsyscall maps the vsyscall page into user address space which can be read directly by the user application. This mode has been deprecated recently and can only be enabled from a special command line parameter vsyscall=emulate. See commit bf00745e7791 ("x86/vsyscall:
Remove CONFIG_LEGACY_VSYSCALL_EMULATE")
Fixing the LASS violations during the EMULATE mode would need complex instruction decoding since the resulting #GP fault does not include any useful error information and the vsyscall address is not readily available in the RIP.
At this point, no one is expected to be using the insecure and deprecated EMULATE mode. The rare usages that need support probably don't care much about security anyway. Disable LASS when EMULATE mode is requested during command line parsing to avoid breaking user software. LASS will be supported if vsyscall mode is set to XONLY or NONE.
--- arch/x86/entry/vsyscall/vsyscall_64.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/entry/vsyscall/vsyscall_64.c +8 -0
@@ -36,6 +36,7 @@#include <asm/vsyscall.h> #include <asm/unistd.h> #include <asm/fixmap.h> +#include <asm/tlbflush.h> #include <asm/traps.h> #include <asm/paravirt.h>
@@ -63,6 +64,13 @@ static int __init vsyscall_setup(char *str)else return -EINVAL ; +if (cpu_feature_enabled (X86_FEATURE_LASS ) && + vsyscall_mode ==EMULATE ) { +cr4_clear_bits (X86_CR4_LASS ); +setup_clear_cpu_cap (X86_FEATURE_LASS ); +pr_info_once ("x86/cpu: Disabling LASS support due to vsyscall=emulate\n" ); + } +return 0 ; }
Signature · 1 line
2.39.2
PATCH v3 10/12 x86/vsyscall: Document the fact that vsyscall=emulate disables LASS
In reply to Alexander ShishkinSince EMULATE mode of vsyscall disables LASS, because fixing the LASS violations during the EMULATE mode would need complex instruction decoding, document this fact in kernel-parameters.txt.
--- Documentation/admin-guide/kernel-parameters.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/admin-guide/kernel-parameters.txt +3 -1
@@ -6850,7 +6850,9 @@ emulate Vsyscalls turn into traps and are emulated reasonably safely. The vsyscall page is - readable. + readable. This also disables the LASS + feature to allow userspace to poke around + the vsyscall page. xonly [default] Vsyscalls turn into traps and are emulated reasonably safely. The vsyscall
Signature · 1 line
2.39.2
PATCH v3 11/12 x86/cpu: Set LASS CR4 bit as pinning sensitive
In reply to Alexander ShishkinFrom: Yian Chen <[email protected]>
Security features such as LASS are not expected to be disabled once initialized. Add LASS to the CR4 pinned mask.
--- arch/x86/kernel/cpu/common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/common.c +1 -1
@@ -422,7 +422,7 @@ static __always_inline void setup_lass(struct cpuinfo_x86 *c)/* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask =X86_CR4_SMEP |X86_CR4_SMAP |X86_CR4_UMIP | -X86_CR4_FSGSBASE |X86_CR4_CET ; +X86_CR4_FSGSBASE |X86_CR4_CET |X86_CR4_LASS ;static DEFINE_STATIC_KEY_FALSE_RO (cr_pinning);static unsigned long cr4_pinned_bits__ro_after_init ;
Signature · 1 line
2.39.2
RFC v3 12/12 x86/efi: Disable LASS enforcement when switching to EFI MM
In reply to Alexander ShishkinFrom: Sohil Mehta <[email protected]>
[Code is experimental and not yet ready to be merged upstream]
PeterZ suggested that EFI memory can be mapped in user virtual address space which would trigger LASS violation upon access. It isn't exactly clear how and when these user address mapping happen. It may be possible this is related to EFI mixed mode.
stac()/clac() calls in the EFI MM enter and exit functions trigger objtool warnings due to switch_mm() not being classified as func_uaccess_safe. Refer Objtool warnings section #9 in the document tools/objtool/Documentation/objtool.txt. This would need to be resolved before even considering merging.
--- arch/x86/platform/efi/efi_64.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/x86/platform/efi/efi_64.c +6 -0
@@ -473,9 +473,14 @@ void __init efi_dump_pagetable(void)* while the EFI-mm is borrowed. mmgrab()/mmdrop() is not used because the mm * can not change under us. * It should be ensured that there are no concurrent calls to this function. +* +* Disable LASS enforcement temporarily when switching to EFI MM since it could +* be mapped into the low 64-bit virtual address space with address bit 63 set +* to 0. */ void efi_enter_mm (void ) { +stac (); efi_prev_mm = current->active_mm; current->active_mm = &efi_mm;switch_mm (efi_prev_mm, &efi_mm,NULL );
@@ -485,6 +490,7 @@ void efi_leave_mm(void) { current->active_mm = efi_prev_mm;switch_mm (&efi_mm, efi_prev_mm,NULL ); +clac (); }static DEFINE_SPINLOCK (efi_runtime_lock);
Signature · 1 line
2.39.2
Quoted 2 lines · Alexander Shishkin
Changes from v2[5]:
- Added myself to the SoB chain
Gentle ping.
Regards,
--
Alex
On Fri, 2023-06-09 at 21:36 +0300, Alexander Shishkin wrote:
What do NULL pointer de-references look like with LASS enabled? They will be a #GP instead of a #PF, right? Currently the kernel prints out several types of helpful messages:
- "BUG: kernel NULL pointer dereference, address: %lx"
- "BUG: unable to handle page fault for address: %px
- "unable to execute userspace code (SMEP?) (uid: %d)"
- etc
These will go away I guess, and you will get a more opaque "general protection fault" message?
Assuming that is all right, I don't know if it might be worth tweaking that #GP message, so people aren't confused when debugging. Something that explains to turn off LASS to get more debugging info.
Quoted 9 lines
Kernel accesses usually only happen to the kernel address space. However, there
are valid reasons for kernel to access memory in the user half. For these cases
(such as text poking and EFI runtime accesses), the kernel can temporarily
suspend the enforcement of LASS by toggling SMAP (Supervisor Mode Access
Prevention) using the stac()/clac() instructions.
CET introduces this unusual instruction called WRUSS. It allows you to make user memory accesses while executing in the kernel. Because of this special property, the CET shadow stack patches don't toggle stac/clac while executing this instruction. So I think LASS will need it to behave more like a normal userspace access from the kernel. Shadow stack is not upstream yet, so just something to keep in mind for the future.
Also, what is this series based on? I wasn't able to apply it.
Quoted 8 lines · Rick P
CET introduces this unusual instruction called WRUSS. It allows you to make user memory accesses while executing in the kernel. Because of this special property, the CET shadow stack patches don't toggle stac/clac while executing this instruction. So I think LASS will need it to behave more like a normal userspace access from the kernel. Shadow stack is not upstream yet, so just something to keep in mind for the future.
This is a good point. We should definitely test this out to confirm.
But, isn't WRUSS already defined as a user-mode access? So, in theory, a user-mode access to a user address space *should* not be blocked by LASS (even with CPL=0).
Are you suggesting that we might need to do something special for WRUSS with LASS enabled?
Sohil
Quoted 21 lines · Sohil Mehta
Quoted 11 lines · Rick P
CET introduces this unusual instruction called WRUSS. It allows you to
make user memory accesses while executing in the kernel. Because of this special property, the CET shadow stack patches don't toggle stac/clac while executing this instruction. So I think LASS will need
it to behave more like a normal userspace access from the kernel. Shadow stack is not upstream yet, so just something to keep in mind for
the future.This is a good point. We should definitely test this out to confirm.
But, isn't WRUSS already defined as a user-mode access? So, in theory, a
user-mode access to a user address space *should* not be blocked by LASS
(even with CPL=0).
Quoted 3 lines
Are you suggesting that we might need to do something special for WRUSS
with LASS enabled?
I was, but reading the docs, I think you are right. It looks like it will be treated like a user access as far as LASS is concerned. Thanks.
Quoted 12 lines · Rick Edgecombe
On Fri, 2023-06-09 at 21:36 +0300, Alexander Shishkin wrote:
What do NULL pointer de-references look like with LASS enabled? They will be a #GP instead of a #PF, right? Currently the kernel prints out
several types of helpful messages:
- "BUG: kernel NULL pointer dereference, address: %lx"
- "BUG: unable to handle page fault for address: %px
- "unable to execute userspace code (SMEP?) (uid: %d)"
- etc
Quoted 7 lines
These will go away I guess, and you will get a more opaque "general protection fault" message?
Assuming that is all right, I don't know if it might be worth tweaking
that #GP message, so people aren't confused when debugging. Something that explains to turn off LASS to get more debugging info.
Maybe get_kernel_gp_address() could be enhanced to give hints for some of those cases like it does for non-canonical addresses?
Separately, I think there is a tiny userspace visible change with this. If userspace tries to access the kernel half of the cannonical address space they will get a segfault. It seems previously the signal would have REG_TRAPNO as 14 (X86_TRAP_PF) in this case, but with LASS it will be 13 (X86_TRAP_GP).
I did a quick search and couldn't find any applications that seemed to be relying on this behavior (not surprising). Some are looking for REG_TRAPNO as 14, but none appeared to be relying on accesses to kernel memory so I guess this should be ok. Still, it is probably appropriate to call out the change and CC linux-api.
It makes me wonder if it should match for LASS and not LASS going forward though. Like maybe always do X86_TRAP_GP for user->kernel accesses instead of having it vary by whether LASS is used? Since there isn't enough information to do REG_TRAPNO X86_TRAP_PF when LASS is used.