Skip to contentSkip to conversation

Enable Linear Address Space Separation support

PATCH Superseded Alexander Shishkin 1 participant 14 messages May 30, 2023

Subsystems (5)
Documentation x86 x86 MM EFI +1 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Enable Linear Address Space Separation support

Changes from v1[1]:
- Emulate vsyscall violations in execute mode in the #GP fault handler
- Use inline memcpy and memset while patching alternatives
- Remove CONFIG_X86_LASS
- Make LASS depend on SMAP
- Dropped the minimal KVM enabling patch

Linear Address Space Separation (LASS) is a security feature that intends to prevent malicious virtual address space accesses across user/kernel mode.

Such mode based access protection already exists today with paging and features such as SMEP and SMAP. However, to enforce these protections, the processor must traverse the paging structures in memory. Malicious software can use timing information resulting from this traversal to determine details about the paging structures, and these details may also be used to determine the layout of the kernel memory.

The LASS mechanism provides the same mode-based protections as paging but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, software will not be able to derive paging-based timing information from the various caching structures such as the TLBs, mid-level caches, page walker, data caches, etc. LASS can avoid probing using double page faults, TLB flush and reload, and SW prefetch instructions. See [2], [3] and [4] for some research on the related attack vectors.

LASS enforcement relies on the typical kernel implemetation to divide the 64-bit virtual address space into two halves:
Addr[63]=0 -> User address space
Addr[63]=1 -> Kernel address space
Any data access or code execution across address spaces typically results in a #GP fault.

Kernel accesses usually only happen to the kernel address space. However, there are valid reasons for kernel to access memory in the user half. For these cases (such as text poking and EFI runtime accesses), the kernel can temporarily suspend the enforcement of LASS by toggling SMAP (Supervisor Mode Access Prevention) using the stac()/clac() instructions.

User space cannot access any kernel address while LASS is enabled. Unfortunately, legacy vsyscall functions are located in the address range 0xffffffffff600000 - 0xffffffffff601000 and emulated in kernel. To avoid breaking user applications when LASS is enabled, extend the vsyscall emulation in execute (XONLY) mode to the #GP fault handler.

In contrast, the vsyscall EMULATE mode is deprecated and not expected to be used by anyone. Supporting EMULATE mode with LASS would need complex intruction decoding in the #GP fault handler and is probably not worth the hassle. Disable LASS in this rare case when someone absolutely needs and enables vsyscall=emulate via the command line.

As of now there is no publicly available CPU supporting LASS. The first one to support LASS would be the Sierra Forest line. The Intel Simics® Simulator was used as software development and testing vehicle for this patch set.

[1] https://lore.kernel.org/lkml/[email protected]/
[2] “Practical Timing Side Channel Attacks against Kernel Space ASLR”,
https://www.ieee-security.org/TC/SP2013/papers/4977a191.pdf
[3] “Prefetch Side-Channel Attacks: Bypassing SMAP and Kernel ASLR”, http://doi.acm.org/10.1145/2976749.2978356
[4] “Harmful prefetch on Intel”, https://ioactive.com/harmful-prefetch-on-intel/ (H/T Anders)

Alexander Shishkin (1):
x86/vsyscall: Document the fact that vsyscall=emulate disables LASS

Peter Zijlstra (1):
x86/asm: Introduce inline memcpy and memset

Sohil Mehta (9):
x86/cpu: Enumerate the LASS feature bits x86/alternatives: Disable LASS when patching kernel alternatives x86/cpu: Enable LASS during CPU initialization x86/cpu: Remove redundant comment during feature setup x86/vsyscall: Reorganize the #PF emulation code x86/traps: Consolidate user fixups in exc_general_protection() x86/vsyscall: Add vsyscall emulation for #GP x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE [RFC] x86/efi: Disable LASS enforcement when switching to EFI MM

Yian Chen (1):
x86/cpu: Set LASS CR4 bit as pinning sensitive

 .../admin-guide/kernel-parameters.txt         |  4 +-
 arch/x86/entry/vsyscall/vsyscall_64.c         | 70 ++++++++++++++-----
 arch/x86/include/asm/cpufeatures.h            |  1 +
 arch/x86/include/asm/disabled-features.h      |  4 +-
 arch/x86/include/asm/smap.h                   |  4 ++
 arch/x86/include/asm/string_32.h              | 21 ++++++
 arch/x86/include/asm/string_64.h              | 21 ++++++
 arch/x86/include/asm/vsyscall.h               | 16 +++--
 arch/x86/include/uapi/asm/processor-flags.h   |  2 +
 arch/x86/kernel/alternative.c                 | 12 +++-
 arch/x86/kernel/cpu/common.c                  | 10 ++-
 arch/x86/kernel/cpu/cpuid-deps.c              |  1 +
 arch/x86/kernel/traps.c                       | 12 ++--
 arch/x86/mm/fault.c                           | 13 +---
 arch/x86/platform/efi/efi_64.c                |  6 ++
 tools/arch/x86/include/asm/cpufeatures.h      |  1 +
 16 files changed, 153 insertions(+), 45 deletions(-)
Signature · 1 line
2.39.2
#1 lore raw

PATCH v2 1/12 x86/cpu: Enumerate the LASS feature bits

From: Sohil Mehta <[email protected]>

Linear Address Space Separation (LASS) is a security feature that intends to prevent malicious virtual address space accesses across user/kernel mode.

Such mode based access protection already exists today with paging and features such as SMEP and SMAP. However, to enforce these protections, the processor must traverse the paging structures in memory. Malicious software can use timing information resulting from this traversal to determine details about the paging structures, and these details may also be used to determine the layout of the kernel memory.

The LASS mechanism provides the same mode-based protections as paging but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, software will not be able to derive paging-based timing information from the various caching structures such as the TLBs, mid-level caches, page walker, data caches, etc.

LASS enforcement relies on the typical kernel implementation to divide the 64-bit virtual address space into two halves:
Addr[63]=0 -> User address space
Addr[63]=1 -> Kernel address space

Any data access or code execution across address spaces typically results in a #GP fault.

The LASS enforcement for kernel data access is dependent on CR4.SMAP being set. The enforcement can be disabled by toggling the RFLAGS.AC bit similar to SMAP.

Define the CPU feature bits to enumerate this feature and include feature dependencies to reflect the same.

---
 arch/x86/include/asm/cpufeatures.h          | 1 +
 arch/x86/include/asm/disabled-features.h    | 4 +++-
 arch/x86/include/asm/smap.h                 | 4 ++++
 arch/x86/include/uapi/asm/processor-flags.h | 2 ++
 arch/x86/kernel/cpu/cpuid-deps.c            | 1 +
 tools/arch/x86/include/asm/cpufeatures.h    | 1 +
 6 files changed, 12 insertions(+), 1 deletion(-)
arch/x86/include/asm/cpufeatures.h +1 -0
index cb8ca46213be..47e775144a34 100644
@@ -312,6 +312,7 @@
 /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */
 #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* AVX VNNI instructions */
 #define X86_FEATURE_AVX512_BF16		(12*32+ 5) /* AVX512 BFLOAT16 instructions */
+#define X86_FEATURE_LASS		(12*32+ 6) /* Linear Address Space Separation */
 #define X86_FEATURE_CMPCCXADD           (12*32+ 7) /* "" CMPccXADD instructions */
 #define X86_FEATURE_ARCH_PERFMON_EXT	(12*32+ 8) /* "" Intel Architectural PerfMon Extension */
 #define X86_FEATURE_FZRM		(12*32+10) /* "" Fast zero-length REP MOVSB */
arch/x86/include/asm/disabled-features.h +3 -1
index fafe9be7a6f4..6535e5192082 100644
@@ -22,12 +22,14 @@
 # define DISABLE_CYRIX_ARR	(1<<(X86_FEATURE_CYRIX_ARR & 31))
 # define DISABLE_CENTAUR_MCR	(1<<(X86_FEATURE_CENTAUR_MCR & 31))
 # define DISABLE_PCID		0
+# define DISABLE_LASS		0
 #else
 # define DISABLE_VME		0
 # define DISABLE_K6_MTRR	0
 # define DISABLE_CYRIX_ARR	0
 # define DISABLE_CENTAUR_MCR	0
 # define DISABLE_PCID		(1<<(X86_FEATURE_PCID & 31))
+# define DISABLE_LASS		(1<<(X86_FEATURE_LASS & 31))
 #endif /* CONFIG_X86_64 */
 
 #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS
@@ -122,7 +124,7 @@
 #define DISABLED_MASK11	(DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \
 			 DISABLE_CALL_DEPTH_TRACKING)
 #define DISABLED_MASK12	(DISABLE_LAM)
-#define DISABLED_MASK13	0
+#define DISABLED_MASK13	(DISABLE_LASS)
 #define DISABLED_MASK14	0
 #define DISABLED_MASK15	0
 #define DISABLED_MASK16	(DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP| \
arch/x86/include/asm/smap.h +4 -0
index bab490379c65..776dce849a58 100644
@@ -27,6 +27,10 @@
 
 #else /* __ASSEMBLY__ */
 
+/*
+ * The CLAC/STAC instructions toggle enforcement of X86_FEATURE_SMAP as well as
+ * X86_FEATURE_LASS.
+ */
 static __always_inline void clac(void)
 {
 	/* Note: a barrier is implicit in alternative() */
arch/x86/include/uapi/asm/processor-flags.h +2 -0
index d898432947ff..1d2405869c7a 100644
@@ -136,6 +136,8 @@
 #define X86_CR4_PKE		_BITUL(X86_CR4_PKE_BIT)
 #define X86_CR4_CET_BIT		23 /* enable Control-flow Enforcement Technology */
 #define X86_CR4_CET		_BITUL(X86_CR4_CET_BIT)
+#define X86_CR4_LASS_BIT	27 /* enable Linear Address Space Separation support */
+#define X86_CR4_LASS		_BITUL(X86_CR4_LASS_BIT)
 #define X86_CR4_LAM_SUP_BIT	28 /* LAM for supervisor pointers */
 #define X86_CR4_LAM_SUP		_BITUL(X86_CR4_LAM_SUP_BIT)
 
arch/x86/kernel/cpu/cpuid-deps.c +1 -0
index f6748c8bd647..722020b2e837 100644
@@ -81,6 +81,7 @@ static const struct cpuid_dep cpuid_deps[] = {
 	{ X86_FEATURE_XFD,			X86_FEATURE_XSAVES    },
 	{ X86_FEATURE_XFD,			X86_FEATURE_XGETBV1   },
 	{ X86_FEATURE_AMX_TILE,			X86_FEATURE_XFD       },
+	{ X86_FEATURE_LASS,			X86_FEATURE_SMAP      },
 	{}
 };
 
tools/arch/x86/include/asm/cpufeatures.h +1 -0
index b89005819cd5..59d2880be0e0 100644
@@ -311,6 +311,7 @@
 /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */
 #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* AVX VNNI instructions */
 #define X86_FEATURE_AVX512_BF16		(12*32+ 5) /* AVX512 BFLOAT16 instructions */
+#define X86_FEATURE_LASS		(12*32+ 6) /* Linear Address Space Separation */
 #define X86_FEATURE_CMPCCXADD           (12*32+ 7) /* "" CMPccXADD instructions */
 #define X86_FEATURE_LKGS		(12*32+18) /* "" Load "kernel" (userspace) GS */
 #define X86_FEATURE_AMX_FP16		(12*32+21) /* "" AMX fp16 Support */
Signature · 1 line
2.39.2
#2 lore raw

PATCH v2 2/12 x86/asm: Introduce inline memcpy and memset

In reply to Alexander Shishkin

From: Peter Zijlstra <[email protected]>

Provide inline memcpy and memset functions that can be used instead of the GCC builtins whenever necessary.

Code posted by Peter Zijlstra <[email protected]>.

---
 arch/x86/include/asm/string_32.h | 21 +++++++++++++++++++++
 arch/x86/include/asm/string_64.h | 21 +++++++++++++++++++++
 2 files changed, 42 insertions(+)
arch/x86/include/asm/string_32.h +21 -0
index 32c0d981a82a..8896270e5eda 100644
@@ -151,6 +151,16 @@ extern void *memcpy(void *, const void *, size_t);
 
 #endif /* !CONFIG_FORTIFY_SOURCE */
 
+static __always_inline void *__inline_memcpy(void *to, const void *from, size_t len)
+{
+	void *ret = to;
+
+	asm volatile("rep movsb"
+		     : "+D" (to), "+S" (from), "+c" (len)
+		     : : "memory");
+	return ret;
+}
+
 #define __HAVE_ARCH_MEMMOVE
 void *memmove(void *dest, const void *src, size_t n);
 
@@ -195,6 +205,17 @@ extern void *memset(void *, int, size_t);
 #define memset(s, c, count) __builtin_memset(s, c, count)
 #endif /* !CONFIG_FORTIFY_SOURCE */
 
+static __always_inline void *__inline_memset(void *s, int v, size_t n)
+{
+	void *ret = s;
+
+	asm volatile("rep stosb"
+		     : "+D" (s), "+c" (n)
+		     : "a" ((uint8_t)v)
+		     : "memory");
+	return ret;
+}
+
 #define __HAVE_ARCH_MEMSET16
 static inline void *memset16(uint16_t *s, uint16_t v, size_t n)
 {
arch/x86/include/asm/string_64.h +21 -0
index 857d364b9888..ea51e2d73265 100644
@@ -18,10 +18,31 @@
 extern void *memcpy(void *to, const void *from, size_t len);
 extern void *__memcpy(void *to, const void *from, size_t len);
 
+static __always_inline void *__inline_memcpy(void *to, const void *from, size_t len)
+{
+	void *ret = to;
+
+	asm volatile("rep movsb"
+		     : "+D" (to), "+S" (from), "+c" (len)
+		     : : "memory");
+	return ret;
+}
+
 #define __HAVE_ARCH_MEMSET
 void *memset(void *s, int c, size_t n);
 void *__memset(void *s, int c, size_t n);
 
+static __always_inline void *__inline_memset(void *s, int v, size_t n)
+{
+	void *ret = s;
+
+	asm volatile("rep stosb"
+		     : "+D" (s), "+c" (n)
+		     : "a" ((uint8_t)v)
+		     : "memory");
+	return ret;
+}
+
 /*
  * KMSAN needs to instrument as much code as possible. Use C versions of
  * memsetXX() from lib/string.c under KMSAN.
Signature · 1 line
2.39.2
#3 lore raw

PATCH v2 3/12 x86/alternatives: Disable LASS when patching kernel alternatives

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

For patching, the kernel initializes a temporary mm area in the lower half of the address range. See commit 4fc19708b165 ("x86/alternatives:
Initialize temporary mm for patching").

Disable LASS enforcement during patching using the stac()/clac() instructions to avoid triggering a #GP fault.

The objtool warns due to a call to a non-allowed function that exists outside of the stac/clac guard, or references to any function with a dynamic function pointer inside the guard. See the Objtool warnings section #9 in the document tools/objtool/Documentation/objtool.txt.

Considering that patching is usually small, replace the memcpy and memset functions in the text poking functions with their inline versions respectively.

---
 arch/x86/kernel/alternative.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/alternative.c +10 -2
index f615e0cb6d93..eac6a5406d39 100644
@@ -1526,16 +1526,24 @@ static inline void unuse_temporary_mm(temp_mm_state_t prev_state)
 __ro_after_init struct mm_struct *poking_mm;
 __ro_after_init unsigned long poking_addr;
 
+/*
+ * poking_init() initializes the text poking address from the lower half of the
+ * address space. Relax LASS enforcement when accessing the poking address.
+ */
 static void text_poke_memcpy(void *dst, const void *src, size_t len)
 {
-	memcpy(dst, src, len);
+	stac();
+	__inline_memcpy(dst, src, len);
+	clac();
 }
 
 static void text_poke_memset(void *dst, const void *src, size_t len)
 {
 	int c = *(const int *)src;
 
-	memset(dst, c, len);
+	stac();
+	__inline_memset(dst, c, len);
+	clac();
 }
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
Signature · 1 line
2.39.2
#4 lore raw

PATCH v2 4/12 x86/cpu: Enable LASS during CPU initialization

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

Being a security feature, enable LASS by default if the platform supports it.

---
 arch/x86/kernel/cpu/common.c | 7 +++++++
 1 file changed, 7 insertions(+)
arch/x86/kernel/cpu/common.c +7 -0
index 80710a68ef7d..315cc67ba93a 100644
@@ -413,6 +413,12 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c)
 	cr4_clear_bits(X86_CR4_UMIP);
 }
 
+static __always_inline void setup_lass(struct cpuinfo_x86 *c)
+{
+	if (cpu_feature_enabled(X86_FEATURE_LASS))
+		cr4_set_bits(X86_CR4_LASS);
+}
+
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask =
 	X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
@@ -1859,6 +1865,7 @@ static void identify_cpu(struct cpuinfo_x86 *c)
 	setup_smep(c);
 	setup_smap(c);
 	setup_umip(c);
+	setup_lass(c);
 
 	/* Enable FSGSBASE instructions if available. */
 	if (cpu_has(c, X86_FEATURE_FSGSBASE)) {
Signature · 1 line
2.39.2
#5 lore raw

PATCH v2 5/12 x86/cpu: Remove redundant comment during feature setup

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

The code below the comment is self explanatory. Instead of updating the comment with the newly added LASS feature, it is better to just remove it.

---
 arch/x86/kernel/cpu/common.c | 1 -
 1 file changed, 1 deletion(-)
arch/x86/kernel/cpu/common.c +0 -1
index 315cc67ba93a..f26c56fe9963 100644
@@ -1861,7 +1861,6 @@ static void identify_cpu(struct cpuinfo_x86 *c)
 	/* Disable the PN if appropriate */
 	squash_the_stupid_serial_number(c);
 
-	/* Set up SMEP/SMAP/UMIP */
 	setup_smep(c);
 	setup_smap(c);
 	setup_umip(c);
Signature · 1 line
2.39.2
#6 lore raw

PATCH v2 6/12 x86/vsyscall: Reorganize the #PF emulation code

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

Separate out the actual vsyscall emulation from the page fault specific handling in preparation for the upcoming #GP fault emulation.

Export is_vsyscall_vaddr() so that it can be reused later.

No functional change intended.

---
 arch/x86/entry/vsyscall/vsyscall_64.c | 51 +++++++++++++++++----------
 arch/x86/include/asm/vsyscall.h       | 10 +++---
 arch/x86/mm/fault.c                   | 13 ++-----
 3 files changed, 41 insertions(+), 33 deletions(-)
arch/x86/entry/vsyscall/vsyscall_64.c +33 -18
index e0ca8120aea8..dd112e538992 100644
@@ -82,6 +82,15 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs,
 			   regs->sp, regs->ax, regs->si, regs->di);
 }
 
+/*
+ * The (legacy) vsyscall page is the long page in the kernel portion
+ * of the address space that has user-accessible permissions.
+ */
+bool is_vsyscall_vaddr(unsigned long vaddr)
+{
+	return unlikely((vaddr & PAGE_MASK) == VSYSCALL_ADDR);
+}
+
 static int addr_to_vsyscall_nr(unsigned long addr)
 {
 	int nr;
@@ -117,8 +126,7 @@ static bool write_ok_or_segv(unsigned long ptr, size_t size)
 	}
 }
 
-bool emulate_vsyscall(unsigned long error_code,
-		      struct pt_regs *regs, unsigned long address)
+static bool __emulate_vsyscall(struct pt_regs *regs, unsigned long address)
 {
 	struct task_struct *tsk;
 	unsigned long caller;
@@ -127,22 +135,6 @@ bool emulate_vsyscall(unsigned long error_code,
 	long ret;
 	unsigned long orig_dx;
 
-	/* Write faults or kernel-privilege faults never get fixed up. */
-	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
-		return false;
-
-	if (!(error_code & X86_PF_INSTR)) {
-		/* Failed vsyscall read */
-		if (vsyscall_mode == EMULATE)
-			return false;
-
-		/*
-		 * User code tried and failed to read the vsyscall page.
-		 */
-		warn_bad_vsyscall(KERN_INFO, regs, "vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround");
-		return false;
-	}
-
 	/*
 	 * No point in checking CS -- the only way to get here is a user mode
 	 * trap to a high address, which means that we're in 64-bit user code.
@@ -294,6 +286,29 @@ bool emulate_vsyscall(unsigned long error_code,
 	return true;
 }
 
+bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs,
+			 unsigned long address)
+{
+	/* Write faults or kernel-privilege faults never get fixed up. */
+	if ((error_code & (X86_PF_WRITE | X86_PF_USER)) != X86_PF_USER)
+		return false;
+
+	if (!(error_code & X86_PF_INSTR)) {
+		/* Failed vsyscall read */
+		if (vsyscall_mode == EMULATE)
+			return false;
+
+		/*
+		 * User code tried and failed to read the vsyscall page.
+		 */
+		warn_bad_vsyscall(KERN_INFO, regs,
+				  "vsyscall read attempt denied -- look up the vsyscall kernel parameter if you need a workaround");
+		return false;
+	}
+
+	return __emulate_vsyscall(regs, address);
+}
+
 /*
  * A pseudo VMA to allow ptrace access for the vsyscall page.  This only
  * covers the 64bit vsyscall page now. 32bit has a real VMA now and does
arch/x86/include/asm/vsyscall.h +6 -4
index ab60a71a8dcb..667b280afc1a 100644
@@ -5,6 +5,8 @@
 #include <linux/seqlock.h>
 #include <uapi/asm/vsyscall.h>
 
+extern bool is_vsyscall_vaddr(unsigned long vaddr);
+
 #ifdef CONFIG_X86_VSYSCALL_EMULATION
 extern void map_vsyscall(void);
 extern void set_vsyscall_pgtable_user_bits(pgd_t *root);
@@ -13,12 +15,12 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root);
  * Called on instruction fetch fault in vsyscall page.
  * Returns true if handled.
  */
-extern bool emulate_vsyscall(unsigned long error_code,
-			     struct pt_regs *regs, unsigned long address);
+extern bool emulate_vsyscall_pf(unsigned long error_code,
+				struct pt_regs *regs, unsigned long address);
 #else
 static inline void map_vsyscall(void) {}
-static inline bool emulate_vsyscall(unsigned long error_code,
-				    struct pt_regs *regs, unsigned long address)
+static inline bool emulate_vsyscall_pf(unsigned long error_code,
+				       struct pt_regs *regs, unsigned long address)
 {
 	return false;
 }
arch/x86/mm/fault.c +2 -11
index e4399983c50c..645eb3323f34 100644
@@ -730,7 +730,7 @@ kernelmode_fixup_or_oops(struct pt_regs *regs, unsigned long error_code,
 		 * Per the above we're !in_interrupt(), aka. task context.
 		 *
 		 * In this case we need to make sure we're not recursively
-		 * faulting through the emulate_vsyscall() logic.
+		 * faulting through the emulate_vsyscall_pf() logic.
 		 */
 		if (current->thread.sig_on_uaccess_err && signal) {
 			sanitize_error_code(address, &error_code);
@@ -798,15 +798,6 @@ show_signal_msg(struct pt_regs *regs, unsigned long error_code,
 	show_opcodes(regs, loglvl);
 }
 
-/*
- * The (legacy) vsyscall page is the long page in the kernel portion
- * of the address space that has user-accessible permissions.
- */
-static bool is_vsyscall_vaddr(unsigned long vaddr)
-{
-	return unlikely((vaddr & PAGE_MASK) == VSYSCALL_ADDR);
-}
-
 static void
 __bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,
 		       unsigned long address, u32 pkey, int si_code)
@@ -1329,7 +1320,7 @@ void do_user_addr_fault(struct pt_regs *regs,
 	 * to consider the PF_PK bit.
 	 */
 	if (is_vsyscall_vaddr(address)) {
-		if (emulate_vsyscall(error_code, regs, address))
+		if (emulate_vsyscall_pf(error_code, regs, address))
 			return;
 	}
 #endif
Signature · 1 line
2.39.2
#7 lore raw

PATCH v2 7/12 x86/traps: Consolidate user fixups in exc_general_protection()

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

Move the UMIP exception fixup along with the other user mode fixups.

No functional change intended.

---
 arch/x86/kernel/traps.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
arch/x86/kernel/traps.c +3 -5
index 58b1f208eff5..f3e619ce9fbd 100644
@@ -736,11 +736,6 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 
 	cond_local_irq_enable(regs);
 
-	if (static_cpu_has(X86_FEATURE_UMIP)) {
-		if (user_mode(regs) && fixup_umip_exception(regs))
-			goto exit;
-	}
-
 	if (v8086_mode(regs)) {
 		local_irq_enable();
 		handle_vm86_fault((struct kernel_vm86_regs *) regs, error_code);
@@ -755,6 +750,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 		if (fixup_vdso_exception(regs, X86_TRAP_GP, error_code, 0))
 			goto exit;
 
+		if (cpu_feature_enabled(X86_FEATURE_UMIP) && fixup_umip_exception(regs))
+			goto exit;
+
 		gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc);
 		goto exit;
 	}
Signature · 1 line
2.39.2
#8 lore raw

PATCH v2 8/12 x86/vsyscall: Add vsyscall emulation for #GP

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

The legacy vsyscall page is mapped at a fixed address in the kernel address range 0xffffffffff600000-0xffffffffff601000. Prior to LASS being introduced, a legacy vsyscall page access from userspace would always generate a page fault. The kernel emulates the execute (XONLY) accesses in the page fault handler and returns back to userspace with the appropriate register values.

Since LASS intercepts these accesses before the paging structures are traversed it generates a general protection fault instead of a page fault. The #GP fault doesn't provide much information in terms of the error code. So, use the faulting RIP which is preserved in the user registers to emulate the vsyscall access without going through complex instruction decoding.

---
 arch/x86/entry/vsyscall/vsyscall_64.c | 11 ++++++++++-
 arch/x86/include/asm/vsyscall.h       |  6 ++++++
 arch/x86/kernel/traps.c               |  4 ++++
 3 files changed, 20 insertions(+), 1 deletion(-)
arch/x86/entry/vsyscall/vsyscall_64.c +10 -1
index dd112e538992..76e1344997d2 100644
@@ -23,7 +23,7 @@
  * soon be no new userspace code that will ever use a vsyscall.
  *
  * The code in this file emulates vsyscalls when notified of a page
- * fault to a vsyscall address.
+ * fault or a general protection fault to a vsyscall address.
  */
 
 #include <linux/kernel.h>
@@ -309,6 +309,15 @@ bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs,
 	return __emulate_vsyscall(regs, address);
 }
 
+bool emulate_vsyscall_gp(struct pt_regs *regs)
+{
+	/* Emulate only if the RIP points to the vsyscall address */
+	if (!is_vsyscall_vaddr(regs->ip))
+		return false;
+
+	return __emulate_vsyscall(regs, regs->ip);
+}
+
 /*
  * A pseudo VMA to allow ptrace access for the vsyscall page.  This only
  * covers the 64bit vsyscall page now. 32bit has a real VMA now and does
arch/x86/include/asm/vsyscall.h +6 -0
index 667b280afc1a..7180a849143f 100644
@@ -17,6 +17,7 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root);
  */
 extern bool emulate_vsyscall_pf(unsigned long error_code,
 				struct pt_regs *regs, unsigned long address);
+extern bool emulate_vsyscall_gp(struct pt_regs *regs);
 #else
 static inline void map_vsyscall(void) {}
 static inline bool emulate_vsyscall_pf(unsigned long error_code,
@@ -24,6 +25,11 @@ static inline bool emulate_vsyscall_pf(unsigned long error_code,
 {
 	return false;
 }
+
+static inline bool emulate_vsyscall_gp(struct pt_regs *regs)
+{
+	return false;
+}
 #endif
 
 #endif /* _ASM_X86_VSYSCALL_H */
arch/x86/kernel/traps.c +4 -0
index f3e619ce9fbd..42d13e17e068 100644
@@ -65,6 +65,7 @@
 #include <asm/vdso.h>
 #include <asm/tdx.h>
 #include <asm/cfi.h>
+#include <asm/vsyscall.h>
 
 #ifdef CONFIG_X86_64
 #include <asm/x86_init.h>
@@ -753,6 +754,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection)
 		if (cpu_feature_enabled(X86_FEATURE_UMIP) && fixup_umip_exception(regs))
 			goto exit;
 
+		if (cpu_feature_enabled(X86_FEATURE_LASS) && emulate_vsyscall_gp(regs))
+			goto exit;
+
 		gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc);
 		goto exit;
 	}
Signature · 1 line
2.39.2
#9 lore raw

PATCH v2 9/12 x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

The EMULATE mode of vsyscall maps the vsyscall page into user address space which can be read directly by the user application. This mode has been deprecated recently and can only be enabled from a special command line parameter vsyscall=emulate. See commit bf00745e7791 ("x86/vsyscall:
Remove CONFIG_LEGACY_VSYSCALL_EMULATE")

Fixing the LASS violations during the EMULATE mode would need complex instruction decoding since the resulting #GP fault does not include any useful error information and the vsyscall address is not readily available in the RIP.

At this point, no one is expected to be using the insecure and deprecated EMULATE mode. The rare usages that need support probably don't care much about security anyway. Disable LASS when EMULATE mode is requested during command line parsing to avoid breaking user software. LASS will be supported if vsyscall mode is set to XONLY or NONE.

---
 arch/x86/entry/vsyscall/vsyscall_64.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/entry/vsyscall/vsyscall_64.c +8 -0
index 76e1344997d2..edd58eda8f50 100644
@@ -36,6 +36,7 @@
 #include <asm/vsyscall.h>
 #include <asm/unistd.h>
 #include <asm/fixmap.h>
+#include <asm/tlbflush.h>
 #include <asm/traps.h>
 #include <asm/paravirt.h>
 
@@ -63,6 +64,13 @@ static int __init vsyscall_setup(char *str)
 		else
 			return -EINVAL;
 
+		if (cpu_feature_enabled(X86_FEATURE_LASS) &&
+		    vsyscall_mode == EMULATE) {
+			cr4_clear_bits(X86_CR4_LASS);
+			setup_clear_cpu_cap(X86_FEATURE_LASS);
+			pr_info_once("x86/cpu: Disabling LASS support due to vsyscall=emulate\n");
+		}
+
 		return 0;
 	}
 
Signature · 1 line
2.39.2
#10 lore raw

PATCH v2 10/12 x86/vsyscall: Document the fact that vsyscall=emulate disables LASS

In reply to Alexander Shishkin

Since EMULATE mode of vsyscall disables LASS, because fixing the LASS violations during the EMULATE mode would need complex instruction decoding, document this fact in kernel-parameters.txt.

---
 Documentation/admin-guide/kernel-parameters.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/admin-guide/kernel-parameters.txt +3 -1
index 9e5bab29685f..efed9193107e 100644
@@ -6850,7 +6850,9 @@
 
 			emulate     Vsyscalls turn into traps and are emulated
 			            reasonably safely.  The vsyscall page is
-				    readable.
+				    readable.  This also disables the LASS
+				    feature to allow userspace to poke around
+				    the vsyscall page.
 
 			xonly       [default] Vsyscalls turn into traps and are
 			            emulated reasonably safely.  The vsyscall
Signature · 1 line
2.39.2
#11 lore raw

PATCH v2 11/12 x86/cpu: Set LASS CR4 bit as pinning sensitive

In reply to Alexander Shishkin

From: Yian Chen <[email protected]>

Security features such as LASS are not expected to be disabled once initialized. Add LASS to the CR4 pinned mask.

---
 arch/x86/kernel/cpu/common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/common.c +1 -1
index f26c56fe9963..9ddc19c8832d 100644
@@ -422,7 +422,7 @@ static __always_inline void setup_lass(struct cpuinfo_x86 *c)
 /* These bits should not change their value after CPU init is finished. */
 static const unsigned long cr4_pinned_mask =
 	X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP |
-	X86_CR4_FSGSBASE | X86_CR4_CET;
+	X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_LASS;
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
Signature · 1 line
2.39.2
#12 lore raw

RFC v2 12/12 x86/efi: Disable LASS enforcement when switching to EFI MM

In reply to Alexander Shishkin

From: Sohil Mehta <[email protected]>

[Code is experimental and not yet ready to be merged upstream]

PeterZ suggested that EFI memory can be mapped in user virtual address space which would trigger LASS violation upon access. It isn't exactly clear how and when these user address mapping happen. It may be possible this is related to EFI mixed mode.

stac()/clac() calls in the EFI MM enter and exit functions trigger objtool warnings due to switch_mm() not being classified as func_uaccess_safe. Refer Objtool warnings section #9 in the document tools/objtool/Documentation/objtool.txt. This would need to be resolved before even considering merging.

---
 arch/x86/platform/efi/efi_64.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/platform/efi/efi_64.c +6 -0
index 232acf418cfb..20966efcd87a 100644
@@ -473,9 +473,14 @@ void __init efi_dump_pagetable(void)
  * while the EFI-mm is borrowed. mmgrab()/mmdrop() is not used because the mm
  * can not change under us.
  * It should be ensured that there are no concurrent calls to this function.
+ *
+ * Disable LASS enforcement temporarily when switching to EFI MM since it could
+ * be mapped into the low 64-bit virtual address space with address bit 63 set
+ * to 0.
  */
 void efi_enter_mm(void)
 {
+	stac();
 	efi_prev_mm = current->active_mm;
 	current->active_mm = &efi_mm;
 	switch_mm(efi_prev_mm, &efi_mm, NULL);
@@ -485,6 +490,7 @@ void efi_leave_mm(void)
 {
 	current->active_mm = efi_prev_mm;
 	switch_mm(&efi_mm, efi_prev_mm, NULL);
+	clac();
 }
 
 static DEFINE_SPINLOCK(efi_runtime_lock);
Signature · 1 line
2.39.2
#13 lore raw
In reply to Alexander Shishkin
Quoted 6 lines · Alexander Shishkin

Changes from v1[1]:
- Emulate vsyscall violations in execute mode in the #GP fault handler
- Use inline memcpy and memset while patching alternatives
- Remove CONFIG_X86_LASS
- Make LASS depend on SMAP
- Dropped the minimal KVM enabling patch

Also, I've unfortunately overlooked adding my SoBs to the patches. This is a mistake, I'll fix it in the next round.

Regards,
--
Alex