Skip to contentSkip to conversation

Introduce a REQUIRE_NX Kconfig option

PATCH In review Alejandro Vallejo 3 participants 14 messages Jun 29, 2023 – Jul 19, 2023

lore
Share & tools
lore
#0 lore raw

v3 cover Introduce a REQUIRE_NX Kconfig option

v3:
* Fixed a Gitlab CI breakage on older toolchains (patch 1)
* Removed XD_DISABLE override logic from cpu/intel.c
* Various style fixes to patch 2 (from Andrew's feedback)

This option hardens Xen by forcing it to write secure (NX-enhanced) PTEs regardless of the runtime NX feature bit in boot_cpu_data. This prevents an attacker with partial write support from affecting Xen's PTE generation logic by overriding the NX feature flag. The patch asserts support for the NX bit in PTEs at boot time and if so short-circuits the cpu_has_nx macro to 1.

Alejandro Vallejo (3):
tools: Add __AC() macro to common-macros.h x86/boot: Clear XD_DISABLE from the early boot path
x86: Add Kconfig option to require NX bit support

 tools/include/xen-tools/common-macros.h |  3 ++
 tools/libs/light/libxl_internal.h       |  2 -
 tools/tests/x86_emulator/x86-emulate.h  |  3 --
 xen/arch/x86/Kconfig                    | 16 +++++++
 xen/arch/x86/boot/head.S                | 62 ++++++++++++++++++++++---
 xen/arch/x86/boot/trampoline.S          |  3 +-
 xen/arch/x86/cpu/intel.c                | 16 +++----
 xen/arch/x86/efi/efi-boot.h             |  9 ++++
 xen/arch/x86/include/asm/cpufeature.h   |  3 +-
 xen/arch/x86/include/asm/msr-index.h    |  2 +-
 xen/arch/x86/include/asm/x86-vendors.h  |  6 +--
 11 files changed, 98 insertions(+), 27 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v3 1/3 tools: Add __AC() macro to common-macros.h

Currently libxl and the x86-emulator tests carry their own versions. Factor those out into the common macros header so every library can make use of it. This is required so the following patch can add this macro to a header used both in Xen and tools/libs.

No functional change.

---
 tools/include/xen-tools/common-macros.h | 3 +++
 tools/libs/light/libxl_internal.h       | 2 --
 tools/tests/x86_emulator/x86-emulate.h  | 3 ---
 3 files changed, 3 insertions(+), 5 deletions(-)
tools/include/xen-tools/common-macros.h +3 -0
index 168691be0e..6d7de1bc0a 100644
@@ -84,4 +84,7 @@
     (type *)((char *)mptr__ - offsetof(type, member));  \
 })
 
+#define __AC(X,Y)   (X##Y)
+#define _AC(X,Y)    __AC(X,Y)
+
 #endif	/* __XEN_TOOLS_COMMON_MACROS__ */
tools/libs/light/libxl_internal.h +0 -2
index 61f4fe1dec..1cf3d400bf 100644
@@ -125,8 +125,6 @@
 #define PVSHIM_CMDLINE "pv-shim console=xen,pv"
 
 /* Size macros. */
-#define __AC(X,Y)   (X##Y)
-#define _AC(X,Y)    __AC(X,Y)
 #define MB(_mb)     (_AC(_mb, ULL) << 20)
 #define GB(_gb)     (_AC(_gb, ULL) << 30)
 
tools/tests/x86_emulator/x86-emulate.h +0 -3
index aa1ed75ec8..350d1a0abf 100644
@@ -59,9 +59,6 @@
 
 #define cf_check /* No Control Flow Integriy checking */
 
-#define AC_(n,t) (n##t)
-#define _AC(n,t) AC_(n,t)
-
 #ifdef __GCC_ASM_FLAG_OUTPUTS__
 # define ASM_FLAG_OUT(yes, no) yes
 #else
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v3 1/3] tools: Add __AC() macro to common-macros.h

Quoted 57 lines · Alejandro Vallejo

Currently libxl and the x86-emulator tests carry their own versions. Factor those out into the common macros header so every library can make use of it. This is required so the following patch can add this macro to a header used both in Xen and tools/libs.

No functional change.

---
 tools/include/xen-tools/common-macros.h | 3 +++
 tools/libs/light/libxl_internal.h       | 2 --
 tools/tests/x86_emulator/x86-emulate.h  | 3 ---
 3 files changed, 3 insertions(+), 5 deletions(-)
tools/include/xen-tools/common-macros.h +3 -0
index 168691be0e..6d7de1bc0a 100644
@@ -84,4 +84,7 @@
     (type *)((char *)mptr__ - offsetof(type, member));  \
 })
 
+#define __AC(X,Y)   (X##Y)
+#define _AC(X,Y)    __AC(X,Y)
+
 #endif /* __XEN_TOOLS_COMMON_MACROS__ */
tools/libs/light/libxl_internal.h +0 -2
index 61f4fe1dec..1cf3d400bf 100644
@@ -125,8 +125,6 @@
 #define PVSHIM_CMDLINE "pv-shim console=xen,pv"
 
 /* Size macros. */
-#define __AC(X,Y)   (X##Y)
-#define _AC(X,Y)    __AC(X,Y)
 #define MB(_mb)     (_AC(_mb, ULL) << 20)
 #define GB(_gb)     (_AC(_gb, ULL) << 30)
 
tools/tests/x86_emulator/x86-emulate.h +0 -4
index aa1ed75ec8..350d1a0abf 100644
@@ -59,9 +59,6 @@
 
 #define cf_check /* No Control Flow Integriy checking */
 
-#define AC_(n,t) (n##t)
-#define _AC(n,t) AC_(n,t)
-
 #ifdef __GCC_ASM_FLAG_OUTPUTS__
 # define ASM_FLAG_OUT(yes, no) yes
 #else
--
Quoted 2 lines

2.34.1

@mantainers

Reviewed-by on v3 1/3
#3 lore raw

Re: [PATCH v3 1/3] tools: Add __AC() macro to common-macros.h

In reply to Alejandro Vallejo
Quoted 8 lines · Alejandro Vallejo

Currently libxl and the x86-emulator tests carry their own versions. Factor those out into the common macros header so every library can make use of it. This is required so the following patch can add this macro to a header used both in Xen and tools/libs.

No functional change.

Quoted 11 lines
---
 tools/include/xen-tools/common-macros.h | 3 +++
 tools/libs/light/libxl_internal.h       | 2 --
 tools/tests/x86_emulator/x86-emulate.h  | 3 ---
 3 files changed, 3 insertions(+), 5 deletions(-)
tools/include/xen-tools/common-macros.h +0 -0
index 168691be0e..6d7de1bc0a 100644

tools/include/xen-tools/common-macros.h line 88

Quoted 5 lines
tools/include/xen-tools/common-macros.h +2 -0
@@ -84,4 +84,7 @@
     (type *)((char *)mptr__ - offsetof(type, member));  \
 })
 
+#define __AC(X,Y)   (X##Y)
+#define _AC(X,Y)    __AC(X,Y)

... I'll take the opportunity to do style fixes here.

~Andrew

#4 lore raw

PATCH v3 2/3 x86/boot: Clear XD_DISABLE from the early boot path

In reply to Alejandro Vallejo

Intel CPUs have a bit in MSR_IA32_MISC_ENABLE that may prevent the NX bit from being advertised. Clear it unconditionally if we can't find the NX feature right away on boot.

The conditions for the MSR being read on early boot are (in this order):

* Long Mode is supported
* NX isn't advertised
* The vendor is Intel

The order of checks has been chosen carefully so a virtualized Xen on a hypervisor that doesn't emulate that MSR (but supports NX) doesn't triple fault trying to access the non-existing MSR.

With that done, we can remove the XD_DISABLE checks in the intel-specific init path (as they are already done in early assembly). Keep a printk to highlight the fact that NX was forcefully enabled.

---
v3:
* In head.S: s/has_nx/got_nx and s/nx_bit/nx
* Style changes in assembly instructions (spaces + width modifiers)
* Big comment in head.S replaced
* Jump directly to .Lno_nx if NX not found and XD_DISABLE not present
* Restored rdmsrl (previously refactored into rdmsr_safe() in v2) and removed XD_DISABLE clearing in C (as it's now done in head.S).
* Moved printk in intel.c to highlight the XD_DISABLE override even when done in head.S

---
 xen/arch/x86/boot/head.S               | 49 ++++++++++++++++++++++----
 xen/arch/x86/cpu/intel.c               | 16 ++++-----
 xen/arch/x86/include/asm/msr-index.h   |  2 +-
 xen/arch/x86/include/asm/x86-vendors.h |  6 ++--
 4 files changed, 53 insertions(+), 20 deletions(-)
xen/arch/x86/boot/head.S +43 -6
index 9fbd602ea5..0e02c28f37 100644
@@ -652,16 +652,53 @@ trampoline_setup:
         cpuid
 1:      mov     %edx, CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM) + sym_esi(boot_cpu_data)
 
-        /* Check for NX. Adjust EFER setting if available. */
-        bt      $cpufeat_bit(X86_FEATURE_NX), %edx
-        jnc     1f
-        orb     $EFER_NXE >> 8, 1 + sym_esi(trampoline_efer)
-1:
-
         /* Check for availability of long mode. */
         bt      $cpufeat_bit(X86_FEATURE_LM),%edx
         jnc     .Lbad_cpu
 
+        /* Check for NX */
+        bt      $cpufeat_bit(X86_FEATURE_NX), %edx
+        jc     .Lgot_nx
+
+        /*
+         * NX appears to be unsupported, but it might be hidden.
+         *
+         * The feature is part of the AMD64 spec, but the very first Intel
+         * 64bit CPUs lacked the feature, and thereafter there was a
+         * firmware knob to disable the feature. Undo the disable if
+         * possible.
+         *
+         * All 64bit Intel CPUs support this MSR. If virtualised, expect
+         * the hypervisor to either emulate the MSR or give us NX.
+         */
+        xor     %eax, %eax
+        cpuid
+        cmp     $X86_VENDOR_INTEL_EBX, %ebx
+        jnz     .Lno_nx
+        cmp     $X86_VENDOR_INTEL_EDX, %edx
+        jnz     .Lno_nx
+        cmp     $X86_VENDOR_INTEL_ECX, %ecx
+        jnz     .Lno_nx
+
+        /* Clear the XD_DISABLE bit */
+        mov    $MSR_IA32_MISC_ENABLE, %ecx
+        rdmsr
+        btr     $2, %edx
+        jnc     .Lno_nx
+        wrmsr
+        orb     $MSR_IA32_MISC_ENABLE_XD_DISABLE >> 32, 4 + sym_esi(trampoline_misc_enable_off)
+
+        /* Check again for NX */
+        mov     $0x80000001, %eax
+        cpuid
+        bt      $cpufeat_bit(X86_FEATURE_NX), %edx
+        jnc     .Lno_nx
+
+.Lgot_nx:
+        /* Adjust EFER given that NX is present */
+        orb     $EFER_NXE >> 8, 1 + sym_esi(trampoline_efer)
+.Lno_nx:
+
         /* Stash TSC to calculate a good approximation of time-since-boot */
         rdtsc
         mov     %eax,     sym_esi(boot_tsc_stamp)
xen/arch/x86/cpu/intel.c +6 -10
index 168cd58f36..b2443b6831 100644
@@ -304,24 +304,20 @@ static void cf_check early_init_intel(struct cpuinfo_x86 *c)
 	if (c->x86 == 15 && c->x86_cache_alignment == 64)
 		c->x86_cache_alignment = 128;
 
+	if (bootsym(trampoline_misc_enable_off) &
+	    MSR_IA32_MISC_ENABLE_XD_DISABLE)
+		printk(KERN_INFO
+		       "re-enabled NX (Execute Disable) protection\n");
+
 	/* Unmask CPUID levels and NX if masked: */
 	rdmsrl(MSR_IA32_MISC_ENABLE, misc_enable);
 
-	disable = misc_enable & (MSR_IA32_MISC_ENABLE_LIMIT_CPUID |
-				 MSR_IA32_MISC_ENABLE_XD_DISABLE);
+	disable = misc_enable & MSR_IA32_MISC_ENABLE_LIMIT_CPUID;
 	if (disable) {
 		wrmsrl(MSR_IA32_MISC_ENABLE, misc_enable & ~disable);
 		bootsym(trampoline_misc_enable_off) |= disable;
-		bootsym(trampoline_efer) |= EFER_NXE;
-	}
-
-	if (disable & MSR_IA32_MISC_ENABLE_LIMIT_CPUID)
 		printk(KERN_INFO "revised cpuid level: %d\n",
 		       cpuid_eax(0));
-	if (disable & MSR_IA32_MISC_ENABLE_XD_DISABLE) {
-		write_efer(read_efer() | EFER_NXE);
-		printk(KERN_INFO
-		       "re-enabled NX (Execute Disable) protection\n");
 	}
 
 	/* CPUID workaround for Intel 0F33/0F34 CPU */
xen/arch/x86/include/asm/msr-index.h +1 -1
index 2749e433d2..4f861c0bb4 100644
@@ -502,7 +502,7 @@
 #define MSR_IA32_MISC_ENABLE_MONITOR_ENABLE (1<<18)
 #define MSR_IA32_MISC_ENABLE_LIMIT_CPUID  (1<<22)
 #define MSR_IA32_MISC_ENABLE_XTPR_DISABLE (1<<23)
-#define MSR_IA32_MISC_ENABLE_XD_DISABLE	(1ULL << 34)
+#define MSR_IA32_MISC_ENABLE_XD_DISABLE   (_AC(1, ULL) << 34)
 
 #define MSR_IA32_TSC_DEADLINE		0x000006E0
 #define MSR_IA32_ENERGY_PERF_BIAS	0x000001b0
xen/arch/x86/include/asm/x86-vendors.h +3 -3
index 0a37024cbd..9191da26d7 100644
@@ -12,9 +12,9 @@
 #define X86_VENDOR_UNKNOWN 0
 
 #define X86_VENDOR_INTEL (1 << 0)
-#define X86_VENDOR_INTEL_EBX 0x756e6547U /* "GenuineIntel" */
-#define X86_VENDOR_INTEL_ECX 0x6c65746eU
-#define X86_VENDOR_INTEL_EDX 0x49656e69U
+#define X86_VENDOR_INTEL_EBX _AC(0x756e6547, U) /* "GenuineIntel" */
+#define X86_VENDOR_INTEL_ECX _AC(0x6c65746e, U)
+#define X86_VENDOR_INTEL_EDX _AC(0x49656e69, U)
 
 #define X86_VENDOR_AMD (1 << 1)
 #define X86_VENDOR_AMD_EBX 0x68747541U /* "AuthenticAMD" */
Signature · 1 line
2.34.1
#5 lore raw

Re: [PATCH v3 2/3] x86/boot: Clear XD_DISABLE from the early boot path

Quoted 29 lines · Alejandro Vallejo

Intel CPUs have a bit in MSR_IA32_MISC_ENABLE that may prevent the NX bit from being advertised. Clear it unconditionally if we can't find the NX feature right away on boot.

The conditions for the MSR being read on early boot are (in this order):

* Long Mode is supported
* NX isn't advertised
* The vendor is Intel

The order of checks has been chosen carefully so a virtualized Xen on a hypervisor that doesn't emulate that MSR (but supports NX) doesn't triple fault trying to access the non-existing MSR.

With that done, we can remove the XD_DISABLE checks in the intel-specific init path (as they are already done in early assembly). Keep a printk to highlight the fact that NX was forcefully enabled.

---
v3:
* In head.S: s/has_nx/got_nx and s/nx_bit/nx
* Style changes in assembly instructions (spaces + width modifiers)
* Big comment in head.S replaced
* Jump directly to .Lno_nx if NX not found and XD_DISABLE not present
* Restored rdmsrl (previously refactored into rdmsr_safe() in v2) and removed XD_DISABLE clearing in C (as it's now done in head.S).
* Moved printk in intel.c to highlight the XD_DISABLE override even when done in head.S

Quoted 7 lines
---
 xen/arch/x86/boot/head.S               | 49 ++++++++++++++++++++++----
 xen/arch/x86/cpu/intel.c               | 16 ++++-----
 xen/arch/x86/include/asm/msr-index.h   |  2 +-
 xen/arch/x86/include/asm/x86-vendors.h |  6 ++--
 4 files changed, 53 insertions(+), 20 deletions(-)

@mantainers

Reviewed-by on v3 2/3
#6 lore raw

Re: [PATCH v3 2/3] x86/boot: Clear XD_DISABLE from the early boot path

In reply to Alejandro Vallejo
Quoted 14 lines · Alejandro Vallejo

Intel CPUs have a bit in MSR_IA32_MISC_ENABLE that may prevent the NX bit from being advertised. Clear it unconditionally if we can't find the NX feature right away on boot.

The conditions for the MSR being read on early boot are (in this order):

* Long Mode is supported
* NX isn't advertised
* The vendor is Intel

The order of checks has been chosen carefully so a virtualized Xen on a hypervisor that doesn't emulate that MSR (but supports NX) doesn't triple fault trying to access the non-existing MSR.

Quoted 5 lines

With that done, we can remove the XD_DISABLE checks in the intel-specific init path (as they are already done in early assembly). Keep a printk to highlight the fact that NX was forcefully enabled.

fixes.

Quoted 34 lines
xen/arch/x86/boot/head.S +16 -6
index 9fbd602ea5..0e02c28f37 100644
@@ -652,16 +652,53 @@ trampoline_setup:
         cpuid
 1:      mov     %edx, CPUINFO_FEATURE_OFFSET(X86_FEATURE_LM) + sym_esi(boot_cpu_data)
 
-        /* Check for NX. Adjust EFER setting if available. */
-        bt      $cpufeat_bit(X86_FEATURE_NX), %edx
-        jnc     1f
-        orb     $EFER_NXE >> 8, 1 + sym_esi(trampoline_efer)
-1:
-
         /* Check for availability of long mode. */
         bt      $cpufeat_bit(X86_FEATURE_LM),%edx
         jnc     .Lbad_cpu
 
+        /* Check for NX */
+        bt      $cpufeat_bit(X86_FEATURE_NX), %edx
+        jc     .Lgot_nx
+
+        /*
+         * NX appears to be unsupported, but it might be hidden.
+         *
+         * The feature is part of the AMD64 spec, but the very first Intel
+         * 64bit CPUs lacked the feature, and thereafter there was a
+         * firmware knob to disable the feature. Undo the disable if
+         * possible.
+         *
+         * All 64bit Intel CPUs support this MSR. If virtualised, expect
+         * the hypervisor to either emulate the MSR or give us NX.
+         */
+        xor     %eax, %eax

xen/arch/x86/boot/head.S trampoline_setup: · line 684

Quoted 10 lines
xen/arch/x86/boot/head.S +10 -0
+        cpuid
+        cmp     $X86_VENDOR_INTEL_EBX, %ebx
+        jnz     .Lno_nx
+        cmp     $X86_VENDOR_INTEL_EDX, %edx
+        jnz     .Lno_nx
+        cmp     $X86_VENDOR_INTEL_ECX, %ecx
+        jnz     .Lno_nx
+
+        /* Clear the XD_DISABLE bit */
+        mov    $MSR_IA32_MISC_ENABLE, %ecx

Parameter indention here.

Quoted 7 lines
xen/arch/x86/include/asm/x86-vendors.h +0 -0
index 0a37024cbd..9191da26d7 100644
@@ -12,9 +12,9 @@
 #define X86_VENDOR_UNKNOWN 0
 

xen/arch/x86/include/asm/x86-vendors.h line 20

Quoted 10 lines
xen/arch/x86/include/asm/x86-vendors.h +3 -3
 #define X86_VENDOR_INTEL (1 << 0)
-#define X86_VENDOR_INTEL_EBX 0x756e6547U /* "GenuineIntel" */
-#define X86_VENDOR_INTEL_ECX 0x6c65746eU
-#define X86_VENDOR_INTEL_EDX 0x49656e69U
+#define X86_VENDOR_INTEL_EBX _AC(0x756e6547, U) /* "GenuineIntel" */
+#define X86_VENDOR_INTEL_ECX _AC(0x6c65746e, U)
+#define X86_VENDOR_INTEL_EDX _AC(0x49656e69, U)
 
 #define X86_VENDOR_AMD (1 << 1)
 #define X86_VENDOR_AMD_EBX 0x68747541U /* "AuthenticAMD" */

And all vendors should get equivalent _AC() conversions.

Can fix both on commit.

~Andrew

#7 lore raw

Re: [PATCH v3 2/3] x86/boot: Clear XD_DISABLE from the early boot path

In reply to Alejandro Vallejo

xen/arch/x86/cpu/intel.c static void cf_check early_init_intel(struct cpuinfo_x86 *c) · line 310

Quoted 12 lines · Alejandro Vallejo
xen/arch/x86/cpu/intel.c +4 -0
index 168cd58f36..b2443b6831 100644
@@ -304,24 +304,20 @@ static void cf_check early_init_intel(struct cpuinfo_x86 *c)
 	if (c->x86 == 15 && c->x86_cache_alignment == 64)
 		c->x86_cache_alignment = 128;
 
+	if (bootsym(trampoline_misc_enable_off) &
+	    MSR_IA32_MISC_ENABLE_XD_DISABLE)
+		printk(KERN_INFO
+		       "re-enabled NX (Execute Disable) protection\n");

One other thing, which I'll also fix on commit.

This now prints per CPU on any system where we had to set XD_DISABLE. It want's a c == &boot_cpu_data to limit it to once-only.

~Andrew

#8 lore raw

PATCH v3 3/3 x86: Add Kconfig option to require NX bit support

In reply to Alejandro Vallejo

This option hardens Xen by forcing it to write secure (NX-enhanced) PTEs regardless of the runtime NX feature bit in boot_cpu_data. This prevents an attacker with partial write support from affecting Xen's PTE generation logic by overriding the NX feature flag. The patch asserts support for the NX bit in PTEs at boot time and if so short-circuits the cpu_has_nx macro to 1.

It has the nice benefit of replacing many instances of runtime checks with folded constants. This has several knock-on effects that improve codegen, saving 2.5KiB off the text section.

The config option defaults to OFF for compatibility with previous behaviour.

---
 xen/arch/x86/Kconfig                  | 16 ++++++++++++++++
 xen/arch/x86/boot/head.S              | 15 ++++++++++++++-
 xen/arch/x86/boot/trampoline.S        |  3 ++-
 xen/arch/x86/efi/efi-boot.h           |  9 +++++++++
 xen/arch/x86/include/asm/cpufeature.h |  3 ++-
 5 files changed, 43 insertions(+), 3 deletions(-)
xen/arch/x86/Kconfig +16 -0
index 406445a358..92f3a627da 100644
@@ -307,6 +307,22 @@ config MEM_SHARING
 	bool "Xen memory sharing support (UNSUPPORTED)" if UNSUPPORTED
 	depends on HVM
 
+config REQUIRE_NX
+	bool "Require NX (No eXecute) support"
+	help
+	  No-eXecute (also called XD "eXecute Disable" and DEP "Data
+	  Execution Prevention") is a security feature designed originally
+	  to combat buffer overflow attacks by marking regions of memory
+	  which the CPU must not interpret as instructions.
+
+	  The NX feature exists in every 64bit CPU except for some very
+	  early Pentium 4 Prescott machines.
+
+	  Enabling this option will improve Xen's security by removing
+	  cases where Xen could be tricked into thinking that the feature
+	  was unavailable. However, if enabled, Xen will no longer boot on
+	  any CPU which is lacking NX support.
+
 endmenu
 
 source "common/Kconfig"
xen/arch/x86/boot/head.S +14 -1
index 0e02c28f37..2e62d07f43 100644
@@ -123,6 +123,7 @@ multiboot2_header:
 .Lbad_ldr_nih: .asciz "ERR: EFI ImageHandle is not provided by bootloader!"
 .Lbad_efi_msg: .asciz "ERR: EFI IA-32 platforms are not supported!"
 .Lbag_alg_msg: .asciz "ERR: Xen must be loaded at a 2Mb boundary!"
+.Lno_nx_msg:   .asciz "ERR: Not an NX-capable CPU!"
 
         .section .init.data, "aw", @progbits
         .align 4
@@ -153,6 +154,11 @@ early_error: /* Here to improve the disassembly. */
 .Lnot_aligned:
         add     $sym_offs(.Lbag_alg_msg), %esi
         jmp     .Lget_vtb
+#ifdef CONFIG_REQUIRE_NX
+.Lno_nx:
+        add     $sym_offs(.Lno_nx_msg), %esi
+        jmp     .Lget_vtb
+#endif
 .Lmb2_no_st:
         /*
          * Here we are on EFI platform. vga_text_buffer was zapped earlier
@@ -656,7 +662,12 @@ trampoline_setup:
         bt      $cpufeat_bit(X86_FEATURE_LM),%edx
         jnc     .Lbad_cpu
 
-        /* Check for NX */
+        /*
+         * Check for NX
+         *   - If Xen was compiled requiring it simply assert it's
+         *     supported. The trampoline already has the right constant.
+         *   - Otherwise, update the trampoline EFER mask accordingly.
+         */
         bt      $cpufeat_bit(X86_FEATURE_NX), %edx
         jc     .Lgot_nx
 
@@ -695,9 +706,11 @@ trampoline_setup:
         jnc     .Lno_nx
 
 .Lgot_nx:
+#ifndef CONFIG_REQUIRE_NX
         /* Adjust EFER given that NX is present */
         orb     $EFER_NXE >> 8, 1 + sym_esi(trampoline_efer)
 .Lno_nx:
+#endif
 
         /* Stash TSC to calculate a good approximation of time-since-boot */
         rdtsc
xen/arch/x86/boot/trampoline.S +2 -1
index c6005fa33d..b8ab0ffdcb 100644
@@ -147,7 +147,8 @@ GLOBAL(trampoline_misc_enable_off)
 
 /* EFER OR-mask for boot paths.  SCE conditional on PV support, NX added when available. */
 GLOBAL(trampoline_efer)
-        .long   EFER_LME | (EFER_SCE * IS_ENABLED(CONFIG_PV))
+        .long   EFER_LME | (EFER_SCE * IS_ENABLED(CONFIG_PV)) | \
+                (EFER_NXE * IS_ENABLED(CONFIG_REQUIRE_NX))
 
 GLOBAL(trampoline_xen_phys_start)
         .long   0
xen/arch/x86/efi/efi-boot.h +9 -0
index c94e53d139..84700559bb 100644
@@ -751,6 +751,15 @@ static void __init efi_arch_cpu(void)
     {
         caps[FEATURESET_e1d] = cpuid_edx(0x80000001);
 
+        /*
+         * This check purposefully doesn't use cpu_has_nx because
+         * cpu_has_nx bypasses the boot_cpu_data read if Xen was compiled
+         * with CONFIG_REQUIRE_NX
+         */
+        if ( IS_ENABLED(CONFIG_REQUIRE_NX) &&
+             !boot_cpu_has(X86_FEATURE_NX) )
+            blexit(L"This Xen build requires NX bit support.");
+
         if ( cpu_has_nx )
             trampoline_efer |= EFER_NXE;
     }
xen/arch/x86/include/asm/cpufeature.h +2 -1
index e2cb8f3cc7..64e1dad225 100644
@@ -91,7 +91,8 @@ static inline bool boot_cpu_has(unsigned int feat)
 #define cpu_has_hypervisor      boot_cpu_has(X86_FEATURE_HYPERVISOR)
 
 /* CPUID level 0x80000001.edx */
-#define cpu_has_nx              boot_cpu_has(X86_FEATURE_NX)
+#define cpu_has_nx              (IS_ENABLED(CONFIG_REQUIRE_NX) || \
+                                 boot_cpu_has(X86_FEATURE_NX))
 #define cpu_has_page1gb         boot_cpu_has(X86_FEATURE_PAGE1GB)
 #define cpu_has_rdtscp          boot_cpu_has(X86_FEATURE_RDTSCP)
 #define cpu_has_3dnow_ext       boot_cpu_has(X86_FEATURE_3DNOWEXT)
Signature · 1 line
2.34.1
#9 lore raw

Re: [PATCH v3 3/3] x86: Add Kconfig option to require NX bit support

Quoted 16 lines · Alejandro Vallejo

This option hardens Xen by forcing it to write secure (NX-enhanced) PTEs regardless of the runtime NX feature bit in boot_cpu_data. This prevents an attacker with partial write support from affecting Xen's PTE generation logic by overriding the NX feature flag. The patch asserts support for the NX bit in PTEs at boot time and if so short-circuits the cpu_has_nx macro to 1.

It has the nice benefit of replacing many instances of runtime checks with folded constants. This has several knock-on effects that improve codegen, saving 2.5KiB off the text section.

The config option defaults to OFF for compatibility with previous behaviour.

Quoted 8 lines
---
 xen/arch/x86/Kconfig                  | 16 ++++++++++++++++
 xen/arch/x86/boot/head.S              | 15 ++++++++++++++-
 xen/arch/x86/boot/trampoline.S        |  3 ++-
 xen/arch/x86/efi/efi-boot.h           |  9 +++++++++
 xen/arch/x86/include/asm/cpufeature.h |  3 ++-
 5 files changed, 43 insertions(+), 3 deletions(-)

@mantainers

#10 lore raw

Re: [PATCH v3 3/3] x86: Add Kconfig option to require NX bit support

In reply to Alejandro Vallejo
Quoted 11 lines · Alejandro Vallejo
xen/arch/x86/boot/head.S +1 -0
@@ -123,6 +123,7 @@ multiboot2_header:
 .Lbad_ldr_nih: .asciz "ERR: EFI ImageHandle is not provided by bootloader!"
 .Lbad_efi_msg: .asciz "ERR: EFI IA-32 platforms are not supported!"
 .Lbag_alg_msg: .asciz "ERR: Xen must be loaded at a 2Mb boundary!"
+.Lno_nx_msg:   .asciz "ERR: Not an NX-capable CPU!"
 
         .section .init.data, "aw", @progbits
         .align 4

xen/arch/x86/boot/head.S early_error: /* Here to improve the disassembly. */ · line 161

Quoted 8 lines
xen/arch/x86/boot/head.S +5 -0
@@ -153,6 +154,11 @@ early_error: /* Here to improve the disassembly. */
 .Lnot_aligned:
         add     $sym_offs(.Lbag_alg_msg), %esi
         jmp     .Lget_vtb
+#ifdef CONFIG_REQUIRE_NX
+.Lno_nx:
+        add     $sym_offs(.Lno_nx_msg), %esi
+        jmp     .Lget_vtb
+#endif

Since I'm in the process of introducing more such paths (for the x86-64-v<N> series), I'm curious: Have you actually had success with getting any output from this code path? I see unreadable output come through serial (provided it's the normal com1 I/O port location where the serial port is), which likely is because baud rate wasn't configured yet, and hence I might have success by changing the config of the receiving side. And I see nothing at all on the screen. While kind of expected when in graphics mode, I wonder whether this ever worked, or whether this has simply bitrotted because of never actually coming into play.

Jan

#11 lore raw

Re: [PATCH v3 3/3] x86: Add Kconfig option to require NX bit support

Quoted 22 lines · Jan Beulich
Quoted 11 lines · Alejandro Vallejo
xen/arch/x86/boot/head.S +1 -0
@@ -123,6 +123,7 @@ multiboot2_header:
 .Lbad_ldr_nih: .asciz "ERR: EFI ImageHandle is not provided by bootloader!"
 .Lbad_efi_msg: .asciz "ERR: EFI IA-32 platforms are not supported!"
 .Lbag_alg_msg: .asciz "ERR: Xen must be loaded at a 2Mb boundary!"
+.Lno_nx_msg:   .asciz "ERR: Not an NX-capable CPU!"
 
         .section .init.data, "aw", @progbits
         .align 4

xen/arch/x86/boot/head.S early_error: /* Here to improve the disassembly. */ · line 161

Quoted 8 lines
xen/arch/x86/boot/head.S +5 -0
@@ -153,6 +154,11 @@ early_error: /* Here to improve the disassembly. */
 .Lnot_aligned:
         add     $sym_offs(.Lbag_alg_msg), %esi
         jmp     .Lget_vtb
+#ifdef CONFIG_REQUIRE_NX
+.Lno_nx:
+        add     $sym_offs(.Lno_nx_msg), %esi
+        jmp     .Lget_vtb
+#endif

Since I'm in the process of introducing more such paths (for the x86-64-v<N>

Quoted 8 lines

series), I'm curious: Have you actually had success with getting any output from this code path? I see unreadable output come through serial (provided it's the normal com1 I/O port location where the serial port is), which likely is because baud rate wasn't configured yet, and hence I might have success by changing the config of the receiving side. And I see nothing at all on the screen. While kind of expected when in graphics mode, I wonder whether this ever worked, or whether this has simply bitrotted because of never actually coming into play.

Pretty clearly this was broken in the course of adding MB2 support, by b28044226e1c using %esi as the "relocation base" after already having clobbered it. I'm working on a fix.

Jan

#12 lore raw

Re: [PATCH v3 3/3] x86: Add Kconfig option to require NX bit support

Quoted 23 lines · Jan Beulich
Quoted 22 lines · Jan Beulich
Quoted 11 lines · Alejandro Vallejo
xen/arch/x86/boot/head.S +1 -0
@@ -123,6 +123,7 @@ multiboot2_header:
 .Lbad_ldr_nih: .asciz "ERR: EFI ImageHandle is not provided by bootloader!"
 .Lbad_efi_msg: .asciz "ERR: EFI IA-32 platforms are not supported!"
 .Lbag_alg_msg: .asciz "ERR: Xen must be loaded at a 2Mb boundary!"
+.Lno_nx_msg:   .asciz "ERR: Not an NX-capable CPU!"
 
         .section .init.data, "aw", @progbits
         .align 4

xen/arch/x86/boot/head.S early_error: /* Here to improve the disassembly. */ · line 161

Quoted 8 lines
xen/arch/x86/boot/head.S +5 -0
@@ -153,6 +154,11 @@ early_error: /* Here to improve the disassembly. */
 .Lnot_aligned:
         add     $sym_offs(.Lbag_alg_msg), %esi
         jmp     .Lget_vtb
+#ifdef CONFIG_REQUIRE_NX
+.Lno_nx:
+        add     $sym_offs(.Lno_nx_msg), %esi
+        jmp     .Lget_vtb
+#endif

Since I'm in the process of introducing more such paths (for the x86-64-v<N>

Quoted 8 lines
Quoted 8 lines

series), I'm curious: Have you actually had success with getting any output from this code path? I see unreadable output come through serial (provided it's the normal com1 I/O port location where the serial port is), which likely is because baud rate wasn't configured yet, and hence I might have success by changing the config of the receiving side. And I see nothing at all on the screen. While kind of expected when in graphics mode, I wonder whether this ever worked, or whether this has simply bitrotted because of never actually coming into play.

I hacked the code to exercise the XD_DISABLE code path, but didn't try to exercise the failure paths, I'm afraid. Sorry.

Quoted 6 lines

Pretty clearly this was broken in the course of adding MB2 support, by b28044226e1c using %esi as the "relocation base" after already having clobbered it. I'm working on a fix.

Jan

Uh-oh. Good catch.

Alejandro

#13 lore raw
In reply to Alejandro Vallejo
Quoted 30 lines · Alejandro Vallejo

v3:
* Fixed a Gitlab CI breakage on older toolchains (patch 1)
* Removed XD_DISABLE override logic from cpu/intel.c
* Various style fixes to patch 2 (from Andrew's feedback)

This option hardens Xen by forcing it to write secure (NX-enhanced) PTEs regardless of the runtime NX feature bit in boot_cpu_data. This prevents an attacker with partial write support from affecting Xen's PTE generation logic by overriding the NX feature flag. The patch asserts support for the NX bit in PTEs at boot time and if so short-circuits the cpu_has_nx macro to 1.

Alejandro Vallejo (3):
tools: Add __AC() macro to common-macros.h x86/boot: Clear XD_DISABLE from the early boot path
x86: Add Kconfig option to require NX bit support

 tools/include/xen-tools/common-macros.h |  3 ++
 tools/libs/light/libxl_internal.h       |  2 -
 tools/tests/x86_emulator/x86-emulate.h  |  3 --
 xen/arch/x86/Kconfig                    | 16 +++++++
 xen/arch/x86/boot/head.S                | 62 ++++++++++++++++++++++---
 xen/arch/x86/boot/trampoline.S          |  3 +-
 xen/arch/x86/cpu/intel.c                | 16 +++----
 xen/arch/x86/efi/efi-boot.h             |  9 ++++
 xen/arch/x86/include/asm/cpufeature.h   |  3 +-
 xen/arch/x86/include/asm/msr-index.h    |  2 +-
 xen/arch/x86/include/asm/x86-vendors.h  |  6 +--
 11 files changed, 98 insertions(+), 27 deletions(-)
Quoted 4 lines

--
2.34.1

Adding CCs here because I forgot to run the add_maintainers.pl script

before sending. Ugh...

Alejandro