Skip to contentSkip to conversation

Add DEXCR support

RFC PATCH In review Benjamin Gray 3 participants 29 messages Nov 28, 2022 – Mar 21, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Add DEXCR support

This series is based on initial work by Chris Riedl that was not sent to the list.

Adds a kernel interface for userspace to interact with the DEXCR. The DEXCR is a SPR that allows control over various execution 'aspects', such as indirect branch prediction and enabling the hashst/hashchk instructions. Further details are in ISA 3.1B Book 3 chapter 12.

This RFC proposes an interface for users to interact with the DEXCR. It aims to support

* Querying supported aspects
* Getting/setting aspects on a per-process level
* Allowing global overrides across all processes

There are some parts that I'm not sure on the best way to approach (hence RFC):

* The feature names in arch/powerpc/kernel/dt_cpu_ftrs.c appear to be unimplemented in skiboot, so are being defined by this series. Is being so verbose fine?
* What aspects should be editable by a process? E.g., SBHE has
effects that potentially bleed into other processes. Should
it only be system wide configurable?
* Should configuring certain aspects for the process be non-privileged? E.g., Is there harm in always allowing configuration of IBRTPD, SRAPD? The *FORCE_SET* action prevents further process local changes regardless of privilege.
* The tests fail Patchwork CI because of the new prctl macros, and the CI doesn't run headers_install and add -isystem <buildpath>/usr/include to the make command.
* On handling an exception, I don't check if the NPHIE bit is enabled in the DEXCR. To do so would require reading both the DEXCR and HDEXCR, for little gain (it should only matter that the current instruction was a hashchk. If so, the only reason it would cause an exception is the failed check. If the instruction is rewritten between exception and check we'd be wrong anyway).

The series is based on the earlier selftest utils series[1], so the tests won't build at all without applying that first. The kernel side should build fine on ppc/next 247f34f7b80357943234f93f247a1ae6b6c3a740 though.

[1]: https://patchwork.ozlabs.org/project/linuxppc-dev/cover/[email protected]/

Benjamin Gray (13):
powerpc/book3s: Add missing <linux/sched.h> include
powerpc: Add initial Dynamic Execution Control Register (DEXCR)
support
powerpc/dexcr: Handle hashchk exception powerpc/dexcr: Support userspace ROP protection
prctl: Define PowerPC DEXCR interface
powerpc/dexcr: Add prctl implementation powerpc/dexcr: Add sysctl entry for SBHE system override powerpc/dexcr: Add enforced userspace ROP protection config selftests/powerpc: Add more utility macros selftests/powerpc: Add hashst/hashchk test selftests/powerpc: Add DEXCR prctl, sysctl interface test selftests/powerpc: Add DEXCR status utility lsdexcr
Documentation: Document PowerPC kernel DEXCR interface

 Documentation/powerpc/dexcr.rst               | 183 +++++++++++
 Documentation/powerpc/index.rst               |   1 +
 arch/powerpc/Kconfig                          |   5 +
 arch/powerpc/include/asm/book3s/64/kexec.h    |   6 +
 arch/powerpc/include/asm/book3s/64/kup.h      |   1 +
 arch/powerpc/include/asm/cputable.h           |   8 +-
 arch/powerpc/include/asm/ppc-opcode.h         |   1 +
 arch/powerpc/include/asm/processor.h          |  33 ++
 arch/powerpc/include/asm/reg.h                |   7 +
 arch/powerpc/kernel/Makefile                  |   1 +
 arch/powerpc/kernel/dexcr.c                   | 310 ++++++++++++++++++
 arch/powerpc/kernel/dt_cpu_ftrs.c             |   4 +
 arch/powerpc/kernel/process.c                 |  31 +-
 arch/powerpc/kernel/prom.c                    |   4 +
 arch/powerpc/kernel/traps.c                   |   6 +
 include/uapi/linux/prctl.h                    |  14 +
 kernel/sys.c                                  |  16 +
 tools/testing/selftests/powerpc/Makefile      |   1 +
 .../selftests/powerpc/dexcr/.gitignore        |   3 +
 .../testing/selftests/powerpc/dexcr/Makefile  |  11 +
 tools/testing/selftests/powerpc/dexcr/cap.c   |  72 ++++
 tools/testing/selftests/powerpc/dexcr/cap.h   |  18 +
 tools/testing/selftests/powerpc/dexcr/dexcr.c | 118 +++++++
 tools/testing/selftests/powerpc/dexcr/dexcr.h |  54 +++
 .../selftests/powerpc/dexcr/dexcr_test.c      | 241 ++++++++++++++
 .../selftests/powerpc/dexcr/hashchk_test.c    | 229 +++++++++++++
 .../testing/selftests/powerpc/dexcr/lsdexcr.c | 178 ++++++++++
 tools/testing/selftests/powerpc/include/reg.h |   4 +
 .../testing/selftests/powerpc/include/utils.h |  44 +++
 29 files changed, 1602 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/powerpc/dexcr.rst
 create mode 100644 arch/powerpc/kernel/dexcr.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/.gitignore
 create mode 100644 tools/testing/selftests/powerpc/dexcr/Makefile
 create mode 100644 tools/testing/selftests/powerpc/dexcr/cap.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/cap.h
 create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr.h
 create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr_test.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/hashchk_test.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/lsdexcr.c

base-commit: 9dc58a6040662faaf24c8932861f485670fce7ff --
2.38.1

#1 lore raw

RFC PATCH 1/13 powerpc/book3s: Add missing <linux/sched.h> include

The functions here use struct thread_struct fields, so need to import the full definition from <linux/sched.h>. The <asm/current.h> header that defines current only forward declares struct thread_struct.

Failing to include this <linux/sched.h> header leads to a compilation error when a translation unit does not also include <linux/sched.h> indirectly.

---
 arch/powerpc/include/asm/book3s/64/kup.h | 1 +
 1 file changed, 1 insertion(+)
arch/powerpc/include/asm/book3s/64/kup.h +1 -0
index 54cf46808157..84c09e546115 100644
@@ -194,6 +194,7 @@
 #else /* !__ASSEMBLY__ */
 
 #include <linux/jump_label.h>
+#include <linux/sched.h>
 
 DECLARE_STATIC_KEY_FALSE(uaccess_flush_key);
 
Signature · 1 line
2.38.1
reviewer Reviewed-by on v1 1/13
#2 lore raw

Re: [RFC PATCH 01/13] powerpc/book3s: Add missing <linux/sched.h> include

Quoted 9 lines · Benjamin Gray

The functions here use struct thread_struct fields, so need to import the full definition from <linux/sched.h>. The <asm/current.h> header that defines current only forward declares struct thread_struct.

Failing to include this <linux/sched.h> header leads to a compilation error when a translation unit does not also include <linux/sched.h> indirectly.

Quoted 18 lines
---
 arch/powerpc/include/asm/book3s/64/kup.h | 1 +
 1 file changed, 1 insertion(+)
arch/powerpc/include/asm/book3s/64/kup.h +1 -0
index 54cf46808157..84c09e546115 100644
@@ -194,6 +194,7 @@
 #else /* !__ASSEMBLY__ */
 
 #include <linux/jump_label.h>
+#include <linux/sched.h>
 
 DECLARE_STATIC_KEY_FALSE(uaccess_flush_key);
 
Signature · 1 line
2.38.1
#3 lore raw

RFC PATCH 2/13 powerpc: Add initial Dynamic Execution Control Register (DEXCR) support

In reply to Benjamin Gray

ISA 3.1B introduces the Dynamic Execution Control Register (DEXCR). It is a per-cpu register that allows control over various CPU behaviours including branch hint usage, indirect branch speculation, and hashst/hashchk support.

Though introduced in 3.1B, no CPUs using 3.1 were released, so CPU_FTR_ARCH_31 is used to determine support for the register itself. Support for each DEXCR bit (aspect) is reported separately by the firmware.

Add various definitions and basic support for the DEXCR in the kernel. Right now it just initialises and maintains the DEXCR on process creation/swap, and clears it in reset_sprs().

---
 arch/powerpc/include/asm/book3s/64/kexec.h |  3 +++
 arch/powerpc/include/asm/cputable.h        |  8 ++++++-
 arch/powerpc/include/asm/processor.h       | 13 +++++++++++
 arch/powerpc/include/asm/reg.h             |  6 ++++++
 arch/powerpc/kernel/Makefile               |  1 +
 arch/powerpc/kernel/dexcr.c                | 25 ++++++++++++++++++++++
 arch/powerpc/kernel/dt_cpu_ftrs.c          |  4 ++++
 arch/powerpc/kernel/process.c              | 13 ++++++++++-
 arch/powerpc/kernel/prom.c                 |  4 ++++
 9 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 arch/powerpc/kernel/dexcr.c
arch/powerpc/include/asm/book3s/64/kexec.h +3 -0
index d4b9d476ecba..563baf94a962 100644
@@ -21,6 +21,9 @@ static inline void reset_sprs(void)
 			plpar_set_ciabr(0);
 	}
 
+	if (cpu_has_feature(CPU_FTR_ARCH_31))
+		mtspr(SPRN_DEXCR, 0);
+
 	/*  Do we need isync()? We are going via a kexec reset */
 	isync();
 }
arch/powerpc/include/asm/cputable.h +7 -1
index 757dbded11dc..03bc192f2d8b 100644
@@ -192,6 +192,10 @@ static inline void cpu_feature_keys_init(void) { }
 #define CPU_FTR_P9_RADIX_PREFETCH_BUG	LONG_ASM_CONST(0x0002000000000000)
 #define CPU_FTR_ARCH_31			LONG_ASM_CONST(0x0004000000000000)
 #define CPU_FTR_DAWR1			LONG_ASM_CONST(0x0008000000000000)
+#define CPU_FTR_DEXCR_SBHE		LONG_ASM_CONST(0x0010000000000000)
+#define CPU_FTR_DEXCR_IBRTPD		LONG_ASM_CONST(0x0020000000000000)
+#define CPU_FTR_DEXCR_SRAPD		LONG_ASM_CONST(0x0040000000000000)
+#define CPU_FTR_DEXCR_NPHIE		LONG_ASM_CONST(0x0080000000000000)
 
 #ifndef __ASSEMBLY__
 
@@ -451,7 +455,9 @@ static inline void cpu_feature_keys_init(void) { }
 	    CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \
 	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_ARCH_207S | \
 	    CPU_FTR_ARCH_300 | CPU_FTR_ARCH_31 | \
-	    CPU_FTR_DAWR | CPU_FTR_DAWR1)
+	    CPU_FTR_DAWR | CPU_FTR_DAWR1 | \
+	    CPU_FTR_DEXCR_SBHE | CPU_FTR_DEXCR_IBRTPD | CPU_FTR_DEXCR_SRAPD | \
+	    CPU_FTR_DEXCR_NPHIE)
 #define CPU_FTRS_CELL	(CPU_FTR_LWSYNC | \
 	    CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
arch/powerpc/include/asm/processor.h +13 -0
index 631802999d59..0a8a793b8b8b 100644
@@ -446,6 +446,19 @@ int exit_vmx_usercopy(void);
 int enter_vmx_ops(void);
 void *exit_vmx_ops(void *dest);
 
+#ifdef CONFIG_PPC_BOOK3S_64
+
+unsigned long get_thread_dexcr(struct thread_struct const *t);
+
+#else
+
+static inline unsigned long get_thread_dexcr(struct thread_struct const *t)
+{
+	return 0;
+}
+
+#endif /* CONFIG_PPC_BOOK3S_64 */
+
 #endif /* __KERNEL__ */
 #endif /* __ASSEMBLY__ */
 #endif /* _ASM_POWERPC_PROCESSOR_H */
arch/powerpc/include/asm/reg.h +6 -0
index 1e8b2e04e626..cdd1f174c399 100644
@@ -385,6 +385,12 @@
 #define SPRN_HSRR0	0x13A	/* Hypervisor Save/Restore 0 */
 #define SPRN_HSRR1	0x13B	/* Hypervisor Save/Restore 1 */
 #define SPRN_ASDR	0x330	/* Access segment descriptor register */
+#define SPRN_DEXCR	0x33C	/* Dynamic execution control register */
+#define   DEXCR_PRO_MASK(aspect)	__MASK(63 - (32 + (aspect)))	/* Aspect number to problem state aspect mask */
+#define   DEXCR_PRO_SBHE		DEXCR_PRO_MASK(0)	/* Speculative Branch Hint Enable */
+#define   DEXCR_PRO_IBRTPD		DEXCR_PRO_MASK(3)	/* Indirect Branch Recurrent Target Prediction Disable */
+#define   DEXCR_PRO_SRAPD		DEXCR_PRO_MASK(4)	/* Subroutine Return Address Prediction Disable */
+#define   DEXCR_PRO_NPHIE		DEXCR_PRO_MASK(5)	/* Non-Privileged Hash Instruction Enable */
 #define SPRN_IC		0x350	/* Virtual Instruction Count */
 #define SPRN_VTB	0x351	/* Virtual Time Base */
 #define SPRN_LDBAR	0x352	/* LD Base Address Register */
arch/powerpc/kernel/Makefile +1 -0
index 9b6146056e48..b112315cfdc2 100644
@@ -79,6 +79,7 @@ obj-$(CONFIG_VDSO32)		+= vdso32_wrapper.o
 obj-$(CONFIG_PPC_WATCHDOG)	+= watchdog.o
 obj-$(CONFIG_HAVE_HW_BREAKPOINT)	+= hw_breakpoint.o
 obj-$(CONFIG_PPC_DAWR)		+= dawr.o
+obj-$(CONFIG_PPC_BOOK3S_64)	+= dexcr.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= cpu_setup_ppc970.o cpu_setup_pa6t.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= cpu_setup_power.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= mce.o mce_power.o
arch/powerpc/kernel/dexcr.c new file +25 -0
new file mode 100644
index 000000000000..32a0a69ff638
@@ -0,0 +1,25 @@
+#include <linux/cache.h>
+#include <linux/init.h>
+
+#include <asm/cpu_has_feature.h>
+#include <asm/cputable.h>
+#include <asm/processor.h>
+#include <asm/reg.h>
+
+#define DEFAULT_DEXCR	0
+
+static int __init dexcr_init(void)
+{
+	if (!early_cpu_has_feature(CPU_FTR_ARCH_31))
+		return 0;
+
+	mtspr(SPRN_DEXCR, DEFAULT_DEXCR);
+
+	return 0;
+}
+early_initcall(dexcr_init);
+
+unsigned long get_thread_dexcr(struct thread_struct const *t)
+{
+	return DEFAULT_DEXCR;
+}
arch/powerpc/kernel/dt_cpu_ftrs.c +4 -0
index c3fb9fdf5bd7..896a48211a37 100644
@@ -661,6 +661,10 @@ static struct dt_cpu_feature_match __initdata
 	{"prefix-instructions", feat_enable, 0},
 	{"matrix-multiply-assist", feat_enable_mma, 0},
 	{"debug-facilities-v31", feat_enable, CPU_FTR_DAWR1},
+	{"dexcr-speculative-branch-hint-enable", feat_enable, CPU_FTR_DEXCR_SBHE},
+	{"dexcr-indirect-branch-recurrent-target-prediction-disable", feat_enable, CPU_FTR_DEXCR_IBRTPD},
+	{"dexcr-subroutine-return-address-prediction-disable", feat_enable, CPU_FTR_DEXCR_SRAPD},
+	{"dexcr-non-privileged-hash-instruction-enable", feat_enable, CPU_FTR_DEXCR_NPHIE},
 };
 
 static bool __initdata using_dt_cpu_ftrs;
arch/powerpc/kernel/process.c +12 -1
index 67da147fe34d..17d26f652b80 100644
@@ -1228,6 +1228,13 @@ static inline void restore_sprs(struct thread_struct *old_thread,
 	if (cpu_has_feature(CPU_FTR_P9_TIDR) &&
 	    old_thread->tidr != new_thread->tidr)
 		mtspr(SPRN_TIDR, new_thread->tidr);
+
+	if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+		unsigned long new_dexcr = get_thread_dexcr(new_thread);
+
+		if (new_dexcr != get_thread_dexcr(old_thread))
+			mtspr(SPRN_DEXCR, new_dexcr);
+	}
 #endif
 
 }
@@ -1802,7 +1809,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 
 	setup_ksp_vsid(p, sp);
 
-#ifdef CONFIG_PPC64 
+#ifdef CONFIG_PPC64
 	if (cpu_has_feature(CPU_FTR_DSCR)) {
 		p->thread.dscr_inherit = current->thread.dscr_inherit;
 		p->thread.dscr = mfspr(SPRN_DSCR);
@@ -1939,6 +1946,10 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.tm_tfiar = 0;
 	current->thread.load_tm = 0;
 #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+#ifdef CONFIG_PPC_BOOK3S_64
+	if (cpu_has_feature(CPU_FTR_ARCH_31))
+		mtspr(SPRN_DEXCR, get_thread_dexcr(&current->thread));
+#endif /* CONFIG_PPC_BOOK3S_64 */
 }
 EXPORT_SYMBOL(start_thread);
 
arch/powerpc/kernel/prom.c +4 -0
index 1eed87d954ba..eff250e1ae9a 100644
@@ -180,6 +180,10 @@ static struct ibm_feature ibm_pa_features[] __initdata = {
 	  .cpu_user_ftrs2 = PPC_FEATURE2_HTM_COMP | PPC_FEATURE2_HTM_NOSC_COMP },
 
 	{ .pabyte = 64, .pabit = 0, .cpu_features = CPU_FTR_DAWR1 },
+	{ .pabyte = 68, .pabit = 0, .cpu_features = CPU_FTR_DEXCR_SBHE },
+	{ .pabyte = 68, .pabit = 3, .cpu_features = CPU_FTR_DEXCR_IBRTPD },
+	{ .pabyte = 68, .pabit = 4, .cpu_features = CPU_FTR_DEXCR_SRAPD },
+	{ .pabyte = 68, .pabit = 5, .cpu_features = CPU_FTR_DEXCR_NPHIE },
 };
 
 /*
Signature · 1 line
2.38.1
reviewer Reviewed-by on v1 2/13
#4 lore raw

Re: [RFC PATCH 02/13] powerpc: Add initial Dynamic Execution Control Register (DEXCR) support

Quoted 5 lines · Benjamin Gray

ISA 3.1B introduces the Dynamic Execution Control Register (DEXCR). It is a per-cpu register that allows control over various CPU behaviours including branch hint usage, indirect branch speculation, and hashst/hashchk support.

Quoted 9 lines

Though introduced in 3.1B, no CPUs using 3.1 were released, so CPU_FTR_ARCH_31 is used to determine support for the register itself. Support for each DEXCR bit (aspect) is reported separately by the firmware.

Add various definitions and basic support for the DEXCR in the kernel. Right now it just initialises and maintains the DEXCR on process creation/swap, and clears it in reset_sprs().

A couple of comments below, but it looks good:

Quoted 33 lines
---
 arch/powerpc/include/asm/book3s/64/kexec.h |  3 +++
 arch/powerpc/include/asm/cputable.h        |  8 ++++++-
 arch/powerpc/include/asm/processor.h       | 13 +++++++++++
 arch/powerpc/include/asm/reg.h             |  6 ++++++
 arch/powerpc/kernel/Makefile               |  1 +
 arch/powerpc/kernel/dexcr.c                | 25 ++++++++++++++++++++++
 arch/powerpc/kernel/dt_cpu_ftrs.c          |  4 ++++
 arch/powerpc/kernel/process.c              | 13 ++++++++++-
 arch/powerpc/kernel/prom.c                 |  4 ++++
 9 files changed, 75 insertions(+), 2 deletions(-)
 create mode 100644 arch/powerpc/kernel/dexcr.c
arch/powerpc/include/asm/book3s/64/kexec.h +3 -0
index d4b9d476ecba..563baf94a962 100644
@@ -21,6 +21,9 @@ static inline void reset_sprs(void)
 			plpar_set_ciabr(0);
 	}
 
+	if (cpu_has_feature(CPU_FTR_ARCH_31))
+		mtspr(SPRN_DEXCR, 0);
+
 	/*  Do we need isync()? We are going via a kexec reset */
 	isync();
 }
arch/powerpc/include/asm/cputable.h +0 -0
index 757dbded11dc..03bc192f2d8b 100644

arch/powerpc/include/asm/cputable.h static inline void cpu_feature_keys_init(void) { } · line 198

Quoted 7 lines
arch/powerpc/include/asm/cputable.h +4 -0
@@ -192,6 +192,10 @@ static inline void cpu_feature_keys_init(void) { }
 #define CPU_FTR_P9_RADIX_PREFETCH_BUG	LONG_ASM_CONST(0x0002000000000000)
 #define CPU_FTR_ARCH_31			LONG_ASM_CONST(0x0004000000000000)
 #define CPU_FTR_DAWR1			LONG_ASM_CONST(0x0008000000000000)
+#define CPU_FTR_DEXCR_SBHE		LONG_ASM_CONST(0x0010000000000000)
+#define CPU_FTR_DEXCR_IBRTPD		LONG_ASM_CONST(0x0020000000000000)
+#define CPU_FTR_DEXCR_SRAPD		LONG_ASM_CONST(0x0040000000000000)
+#define CPU_FTR_DEXCR_NPHIE		LONG_ASM_CONST(0x0080000000000000)

We potentially don't need to use CPU_FTR bits for each of these. We only really want them to use instruction patching and make feature tests fast. But we have been a bit liberal with using them and they are kind of tied into cpu feature parsing code so maybe it's easier to go with them for now.

Quoted 43 lines
arch/powerpc/include/asm/cputable.h +3 -1
 #ifndef __ASSEMBLY__
 
@@ -451,7 +455,9 @@ static inline void cpu_feature_keys_init(void) { }
 	    CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \
 	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_ARCH_207S | \
 	    CPU_FTR_ARCH_300 | CPU_FTR_ARCH_31 | \
-	    CPU_FTR_DAWR | CPU_FTR_DAWR1)
+	    CPU_FTR_DAWR | CPU_FTR_DAWR1 | \
+	    CPU_FTR_DEXCR_SBHE | CPU_FTR_DEXCR_IBRTPD | CPU_FTR_DEXCR_SRAPD | \
+	    CPU_FTR_DEXCR_NPHIE)
 #define CPU_FTRS_CELL	(CPU_FTR_LWSYNC | \
 	    CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
arch/powerpc/include/asm/processor.h +13 -0
index 631802999d59..0a8a793b8b8b 100644
@@ -446,6 +446,19 @@ int exit_vmx_usercopy(void);
 int enter_vmx_ops(void);
 void *exit_vmx_ops(void *dest);
 
+#ifdef CONFIG_PPC_BOOK3S_64
+
+unsigned long get_thread_dexcr(struct thread_struct const *t);
+
+#else
+
+static inline unsigned long get_thread_dexcr(struct thread_struct const *t)
+{
+	return 0;
+}
+
+#endif /* CONFIG_PPC_BOOK3S_64 */
+
 #endif /* __KERNEL__ */
 #endif /* __ASSEMBLY__ */
 #endif /* _ASM_POWERPC_PROCESSOR_H */
arch/powerpc/include/asm/reg.h +0 -0
index 1e8b2e04e626..cdd1f174c399 100644

arch/powerpc/include/asm/reg.h line 389

Quoted 5 lines
arch/powerpc/include/asm/reg.h +2 -0
@@ -385,6 +385,12 @@
 #define SPRN_HSRR0	0x13A	/* Hypervisor Save/Restore 0 */
 #define SPRN_HSRR1	0x13B	/* Hypervisor Save/Restore 1 */
 #define SPRN_ASDR	0x330	/* Access segment descriptor register */
+#define SPRN_DEXCR	0x33C	/* Dynamic execution control register */
+#define   DEXCR_PRO_MASK(aspect)	__MASK(63 - (32 + (aspect)))	/* Aspect number to problem state aspect mask */

I think PR is a better shorthand for problem state than PRO. It's just more commonly used.

We also have PPC_BIT and PPC_BITMASK, _BIT being used for single-bit mask. So this could be -

#define DEXCR_PR_BIT(aspect) PPC_BIT(32 + (aspect))

Or maybe DEXCR_PR_ASPECT_BIT.

Quoted 93 lines
arch/powerpc/include/asm/reg.h +4 -0
+#define   DEXCR_PRO_SBHE		DEXCR_PRO_MASK(0)	/* Speculative Branch Hint Enable */
+#define   DEXCR_PRO_IBRTPD		DEXCR_PRO_MASK(3)	/* Indirect Branch Recurrent Target Prediction Disable */
+#define   DEXCR_PRO_SRAPD		DEXCR_PRO_MASK(4)	/* Subroutine Return Address Prediction Disable */
+#define   DEXCR_PRO_NPHIE		DEXCR_PRO_MASK(5)	/* Non-Privileged Hash Instruction Enable */
 #define SPRN_IC		0x350	/* Virtual Instruction Count */
 #define SPRN_VTB	0x351	/* Virtual Time Base */
 #define SPRN_LDBAR	0x352	/* LD Base Address Register */
arch/powerpc/kernel/Makefile +1 -0
index 9b6146056e48..b112315cfdc2 100644
@@ -79,6 +79,7 @@ obj-$(CONFIG_VDSO32)		+= vdso32_wrapper.o
 obj-$(CONFIG_PPC_WATCHDOG)	+= watchdog.o
 obj-$(CONFIG_HAVE_HW_BREAKPOINT)	+= hw_breakpoint.o
 obj-$(CONFIG_PPC_DAWR)		+= dawr.o
+obj-$(CONFIG_PPC_BOOK3S_64)	+= dexcr.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= cpu_setup_ppc970.o cpu_setup_pa6t.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= cpu_setup_power.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= mce.o mce_power.o
arch/powerpc/kernel/dexcr.c new file +25 -0
new file mode 100644
index 000000000000..32a0a69ff638
@@ -0,0 +1,25 @@
+#include <linux/cache.h>
+#include <linux/init.h>
+
+#include <asm/cpu_has_feature.h>
+#include <asm/cputable.h>
+#include <asm/processor.h>
+#include <asm/reg.h>
+
+#define DEFAULT_DEXCR	0
+
+static int __init dexcr_init(void)
+{
+	if (!early_cpu_has_feature(CPU_FTR_ARCH_31))
+		return 0;
+
+	mtspr(SPRN_DEXCR, DEFAULT_DEXCR);
+
+	return 0;
+}
+early_initcall(dexcr_init);
+
+unsigned long get_thread_dexcr(struct thread_struct const *t)
+{
+	return DEFAULT_DEXCR;
+}
arch/powerpc/kernel/dt_cpu_ftrs.c +4 -0
index c3fb9fdf5bd7..896a48211a37 100644
@@ -661,6 +661,10 @@ static struct dt_cpu_feature_match __initdata
 	{"prefix-instructions", feat_enable, 0},
 	{"matrix-multiply-assist", feat_enable_mma, 0},
 	{"debug-facilities-v31", feat_enable, CPU_FTR_DAWR1},
+	{"dexcr-speculative-branch-hint-enable", feat_enable, CPU_FTR_DEXCR_SBHE},
+	{"dexcr-indirect-branch-recurrent-target-prediction-disable", feat_enable, CPU_FTR_DEXCR_IBRTPD},
+	{"dexcr-subroutine-return-address-prediction-disable", feat_enable, CPU_FTR_DEXCR_SRAPD},
+	{"dexcr-non-privileged-hash-instruction-enable", feat_enable, CPU_FTR_DEXCR_NPHIE},
 };
 
 static bool __initdata using_dt_cpu_ftrs;
arch/powerpc/kernel/process.c +8 -1
index 67da147fe34d..17d26f652b80 100644
@@ -1228,6 +1228,13 @@ static inline void restore_sprs(struct thread_struct *old_thread,
 	if (cpu_has_feature(CPU_FTR_P9_TIDR) &&
 	    old_thread->tidr != new_thread->tidr)
 		mtspr(SPRN_TIDR, new_thread->tidr);
+
+	if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+		unsigned long new_dexcr = get_thread_dexcr(new_thread);
+
+		if (new_dexcr != get_thread_dexcr(old_thread))
+			mtspr(SPRN_DEXCR, new_dexcr);
+	}
 #endif
 
 }
@@ -1802,7 +1809,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 
 	setup_ksp_vsid(p, sp);
 
-#ifdef CONFIG_PPC64 
+#ifdef CONFIG_PPC64
 	if (cpu_has_feature(CPU_FTR_DSCR)) {
 		p->thread.dscr_inherit = current->thread.dscr_inherit;
 		p->thread.dscr = mfspr(SPRN_DSCR);

arch/powerpc/kernel/process.c void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp) · line 1952

Quoted 7 lines
arch/powerpc/kernel/process.c +4 -0
@@ -1939,6 +1946,10 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.tm_tfiar = 0;
 	current->thread.load_tm = 0;
 #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+#ifdef CONFIG_PPC_BOOK3S_64
+	if (cpu_has_feature(CPU_FTR_ARCH_31))
+		mtspr(SPRN_DEXCR, get_thread_dexcr(&current->thread));
+#endif /* CONFIG_PPC_BOOK3S_64 */

You possibly don't need the ifdef here because CPU_FTR_ARCH_31 should fold away. Some of the others do because they're using open-coded access to struct members, but if you're using accessor functions to get and set such things, there may be no need to.

I think my preference is for your style.

Thanks,
Nick

Quoted 20 lines
arch/powerpc/kernel/process.c +0 -0
 }
 EXPORT_SYMBOL(start_thread);
 
arch/powerpc/kernel/prom.c +4 -0
index 1eed87d954ba..eff250e1ae9a 100644
@@ -180,6 +180,10 @@ static struct ibm_feature ibm_pa_features[] __initdata = {
 	  .cpu_user_ftrs2 = PPC_FEATURE2_HTM_COMP | PPC_FEATURE2_HTM_NOSC_COMP },
 
 	{ .pabyte = 64, .pabit = 0, .cpu_features = CPU_FTR_DAWR1 },
+	{ .pabyte = 68, .pabit = 0, .cpu_features = CPU_FTR_DEXCR_SBHE },
+	{ .pabyte = 68, .pabit = 3, .cpu_features = CPU_FTR_DEXCR_IBRTPD },
+	{ .pabyte = 68, .pabit = 4, .cpu_features = CPU_FTR_DEXCR_SRAPD },
+	{ .pabyte = 68, .pabit = 5, .cpu_features = CPU_FTR_DEXCR_NPHIE },
 };
 
 /*
Signature · 1 line
2.38.1
#5 lore raw

Re: [RFC PATCH 02/13] powerpc: Add initial Dynamic Execution Control Register (DEXCR) support

Quoted 21 lines · Nicholas Piggin
Quoted 20 lines · Benjamin Gray
arch/powerpc/include/asm/cputable.h +0 -0
index 757dbded11dc..03bc192f2d8b 100644
@@ -192,6 +192,10 @@ static inline void cpu_feature_keys_init(void)
 { }
  #define
 CPU_FTR_P9_RADIX_PREFETCH_BUG  LONG_ASM_CONST(0x0002000000000000)
  #define

CPU_FTR_ARCH_31                        LONG_ASM_CONST(0x00040000000 00000)
 #define
CPU_FTR_DAWR1                  LONG_ASM_CONST(0x0008000000000000) +#define
CPU_FTR_DEXCR_SBHE             LONG_ASM_CONST(0x0010000000000000) +#define
CPU_FTR_DEXCR_IBRTPD           LONG_ASM_CONST(0x0020000000000000) +#define
CPU_FTR_DEXCR_SRAPD            LONG_ASM_CONST(0x0040000000000000)

Quoted 8 lines
Quoted 2 lines

+#define
CPU_FTR_DEXCR_NPHIE            LONG_ASM_CONST(0x0080000000000000)

We potentially don't need to use CPU_FTR bits for each of these. We only really want them to use instruction patching and make feature tests fast. But we have been a bit liberal with using them and they are kind of tied into cpu feature parsing code so maybe it's easier to go with them for now.

For the static only DEXCR series I've only got CPU_FTR_DEXCR_NPHIE because that's needed for hashkey updates. The others don't really matter; they are only interesting for masking out unsupported bits. Masking itself seems to be unnecessary; the DEXCR will just ignore unsupported bits. Attempting to set all bits on a P10 showed the first 8 were set and the remainder stayed 0'd, and the kernel worked fine.

It's definitely easier to use CPU_FTR_* for feature detection from the PAPR specified blob though. Maybe it would be possible to support a callback on a match instead of setting a feature flag.
@@ -1802,7 +1809,7 @@ int copy_thread(struct task_struct *p, const

Quoted 23 lines
Quoted 23 lines
arch/powerpc/include/asm/cputable.h +0 -0
@@ -1802,7 +1809,7 @@ int copy_thread(struct task_struct *p, const
 struct kernel_clone_args *args)


        setup_ksp_vsid(p, sp);

arch/powerpc/include/asm/cputable.h +1 -1
-#ifdef CONFIG_PPC64 
+#ifdef CONFIG_PPC64

        if (cpu_has_feature(CPU_FTR_DSCR)) {
                p->thread.dscr_inherit = current-

Quoted 1 line

thread.dscr_inherit;

                p->thread.dscr = mfspr(SPRN_DSCR);

arch/powerpc/include/asm/cputable.h +3 -0
@@ -1939,6 +1946,10 @@ void start_thread(struct pt_regs *regs,
 unsigned long start, unsigned long sp)
         current->thread.tm_tfiar = 0;
         current->thread.load_tm = 0;
  #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+#ifdef CONFIG_PPC_BOOK3S_64
+       if (cpu_has_feature(CPU_FTR_ARCH_31))
+               mtspr(SPRN_DEXCR, get_thread_dexcr(&current-
 >thread));
Quoted 8 lines

arch/powerpc/include/asm/cputable.h void start_thread(struct pt_regs *regs, · line 1954

Quoted 1 line
arch/powerpc/include/asm/cputable.h +1 -0
+#endif /* CONFIG_PPC_BOOK3S_64 */

You possibly don't need the ifdef here because CPU_FTR_ARCH_31 should fold away. Some of the others do because they're using open-coded access to struct members, but if you're using accessor functions to get and set such things, there may be no need to.

I think my preference is for your style.

I've been revisiting where the DEXCR is initialised and updated. With the static DEXCR, the thread value is just a field on the task struct like the others.

#6 lore raw

RFC PATCH 3/13 powerpc/dexcr: Handle hashchk exception

In reply to Benjamin Gray

Recognise and pass the appropriate signal to the user program when a hashchk instruction triggers. This is independent of allowing configuration of DEXCR[NPHIE], as a hypervisor can enforce this aspect regardless of the kernel.

---
 arch/powerpc/include/asm/ppc-opcode.h |  1 +
 arch/powerpc/include/asm/processor.h  |  6 ++++++
 arch/powerpc/kernel/dexcr.c           | 22 ++++++++++++++++++++++
 arch/powerpc/kernel/traps.c           |  6 ++++++
 4 files changed, 35 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +1 -0
index 21e33e46f4b8..89b316466ed1 100644
@@ -215,6 +215,7 @@
 #define OP_31_XOP_STFSX	    663
 #define OP_31_XOP_STFSUX    695
 #define OP_31_XOP_STFDX     727
+#define OP_31_XOP_HASHCHK   754
 #define OP_31_XOP_STFDUX    759
 #define OP_31_XOP_LHBRX     790
 #define OP_31_XOP_LFIWAX    855
arch/powerpc/include/asm/processor.h +6 -0
index 0a8a793b8b8b..c17ec1e44c86 100644
@@ -448,10 +448,16 @@ void *exit_vmx_ops(void *dest);
 
 #ifdef CONFIG_PPC_BOOK3S_64
 
+bool is_hashchk_trap(struct pt_regs const *regs);
 unsigned long get_thread_dexcr(struct thread_struct const *t);
 
 #else
 
+static inline bool is_hashchk_trap(struct pt_regs const *regs)
+{
+	return false;
+}
+
 static inline unsigned long get_thread_dexcr(struct thread_struct const *t)
 {
 	return 0;
arch/powerpc/kernel/dexcr.c +22 -0
index 32a0a69ff638..11515e67afac 100644
@@ -3,6 +3,9 @@
 
 #include <asm/cpu_has_feature.h>
 #include <asm/cputable.h>
+#include <asm/disassemble.h>
+#include <asm/inst.h>
+#include <asm/ppc-opcode.h>
 #include <asm/processor.h>
 #include <asm/reg.h>
 
@@ -19,6 +22,25 @@ static int __init dexcr_init(void)
 }
 early_initcall(dexcr_init);
 
+bool is_hashchk_trap(struct pt_regs const *regs)
+{
+	ppc_inst_t insn;
+
+	if (!cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		return false;
+
+	if (get_user_instr(insn, (void __user *)regs->nip)) {
+		WARN_ON(1);
+		return false;
+	}
+
+	if (ppc_inst_primary_opcode(insn) == 31 &&
+	    get_xop(ppc_inst_val(insn)) == OP_31_XOP_HASHCHK)
+		return true;
+
+	return false;
+}
+
 unsigned long get_thread_dexcr(struct thread_struct const *t)
 {
 	return DEFAULT_DEXCR;
arch/powerpc/kernel/traps.c +6 -0
index 9bdd79aa51cf..b83f5b382f24 100644
@@ -1516,6 +1516,12 @@ static void do_program_check(struct pt_regs *regs)
 				return;
 			}
 		}
+
+		if (user_mode(regs) && is_hashchk_trap(regs)) {
+			_exception(SIGILL, regs, ILL_ILLOPN, regs->nip);
+			return;
+		}
+
 		_exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip);
 		return;
 	}
Signature · 1 line
2.38.1
#7 lore raw

Re: [RFC PATCH 03/13] powerpc/dexcr: Handle hashchk exception

Quoted 65 lines · Benjamin Gray

Recognise and pass the appropriate signal to the user program when a hashchk instruction triggers. This is independent of allowing configuration of DEXCR[NPHIE], as a hypervisor can enforce this aspect regardless of the kernel.

---
 arch/powerpc/include/asm/ppc-opcode.h |  1 +
 arch/powerpc/include/asm/processor.h  |  6 ++++++
 arch/powerpc/kernel/dexcr.c           | 22 ++++++++++++++++++++++
 arch/powerpc/kernel/traps.c           |  6 ++++++
 4 files changed, 35 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +1 -0
index 21e33e46f4b8..89b316466ed1 100644
@@ -215,6 +215,7 @@
 #define OP_31_XOP_STFSX	    663
 #define OP_31_XOP_STFSUX    695
 #define OP_31_XOP_STFDX     727
+#define OP_31_XOP_HASHCHK   754
 #define OP_31_XOP_STFDUX    759
 #define OP_31_XOP_LHBRX     790
 #define OP_31_XOP_LFIWAX    855
arch/powerpc/include/asm/processor.h +6 -0
index 0a8a793b8b8b..c17ec1e44c86 100644
@@ -448,10 +448,16 @@ void *exit_vmx_ops(void *dest);
 
 #ifdef CONFIG_PPC_BOOK3S_64
 
+bool is_hashchk_trap(struct pt_regs const *regs);
 unsigned long get_thread_dexcr(struct thread_struct const *t);
 
 #else
 
+static inline bool is_hashchk_trap(struct pt_regs const *regs)
+{
+	return false;
+}
+
 static inline unsigned long get_thread_dexcr(struct thread_struct const *t)
 {
 	return 0;
arch/powerpc/kernel/dexcr.c +4 -0
index 32a0a69ff638..11515e67afac 100644
@@ -3,6 +3,9 @@
 
 #include <asm/cpu_has_feature.h>
 #include <asm/cputable.h>
+#include <asm/disassemble.h>
+#include <asm/inst.h>
+#include <asm/ppc-opcode.h>
 #include <asm/processor.h>
 #include <asm/reg.h>
 
@@ -19,6 +22,25 @@ static int __init dexcr_init(void)
 }
 early_initcall(dexcr_init);
 
+bool is_hashchk_trap(struct pt_regs const *regs)

arch/powerpc/kernel/dexcr.c static int __init dexcr_init(void) · line 35

Quoted 10 lines
arch/powerpc/kernel/dexcr.c +10 -0
+{
+	ppc_inst_t insn;
+
+	if (!cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		return false;
+
+	if (get_user_instr(insn, (void __user *)regs->nip)) {
+		WARN_ON(1);
+		return false;
+	}

Nice series, just starting to have a look at it.

You probably don't want a WARN_ON() here because it's user triggerable and isn't necessarily even indiciating a problem or attack if the app is doing code unmapping in order to get faults.

Check some of the other instruction emulation for what to do in case of an EFAULT.

Quoted 16 lines
arch/powerpc/kernel/dexcr.c +8 -0
+
+	if (ppc_inst_primary_opcode(insn) == 31 &&
+	    get_xop(ppc_inst_val(insn)) == OP_31_XOP_HASHCHK)
+		return true;
+
+	return false;
+}
+
 unsigned long get_thread_dexcr(struct thread_struct const *t)
 {
 	return DEFAULT_DEXCR;
arch/powerpc/kernel/traps.c +0 -0
index 9bdd79aa51cf..b83f5b382f24 100644

arch/powerpc/kernel/traps.c static void do_program_check(struct pt_regs *regs) · line 1523

Quoted 8 lines
arch/powerpc/kernel/traps.c +5 -0
@@ -1516,6 +1516,12 @@ static void do_program_check(struct pt_regs *regs)
 				return;
 			}
 		}
+
+		if (user_mode(regs) && is_hashchk_trap(regs)) {
+			_exception(SIGILL, regs, ILL_ILLOPN, regs->nip);
+			return;
+		}

I guess ILLOPN makes sense. Do you know if any other archs do similar?

Thanks,
Nick

#8 lore raw

Re: [RFC PATCH 03/13] powerpc/dexcr: Handle hashchk exception

Quoted 85 lines · Nicholas Piggin
Quoted 71 lines · Benjamin Gray

Recognise and pass the appropriate signal to the user program when a
hashchk instruction triggers. This is independent of allowing configuration of DEXCR[NPHIE], as a hypervisor can enforce this aspect
regardless of the kernel.

---
 arch/powerpc/include/asm/ppc-opcode.h |  1 + arch/powerpc/include/asm/processor.h  |  6 ++++++ arch/powerpc/kernel/dexcr.c           | 22 ++++++++++++++++++++++ arch/powerpc/kernel/traps.c           |  6 ++++++

 4 files changed, 35 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +1 -0
index 21e33e46f4b8..89b316466ed1 100644
@@ -215,6 +215,7 @@
  #define OP_31_XOP_STFSX            663
  #define OP_31_XOP_STFSUX    695
  #define OP_31_XOP_STFDX     727
+#define OP_31_XOP_HASHCHK   754
  #define OP_31_XOP_STFDUX    759
  #define OP_31_XOP_LHBRX     790
  #define OP_31_XOP_LFIWAX    855
arch/powerpc/include/asm/processor.h +0 -0
index 0a8a793b8b8b..c17ec1e44c86 100644
@@ -448,10 +448,16 @@ void *exit_vmx_ops(void *dest);


 #ifdef CONFIG_PPC_BOOK3S_64

+bool is_hashchk_trap(struct pt_regs const *regs); unsigned long get_thread_dexcr(struct thread_struct const *t);
 #else

arch/powerpc/include/asm/processor.h +5 -0
+static inline bool is_hashchk_trap(struct pt_regs const *regs)
+{
+       return false;
+}
+

 static inline unsigned long get_thread_dexcr(struct thread_struct const *t)
 {
        return 0;

arch/powerpc/kernel/dexcr.c +0 -0
index 32a0a69ff638..11515e67afac 100644
@@ -3,6 +3,9 @@


 #include <asm/cpu_has_feature.h>
 #include <asm/cputable.h>

arch/powerpc/kernel/dexcr.c +3 -0
+#include <asm/disassemble.h>
+#include <asm/inst.h>
+#include <asm/ppc-opcode.h>

 #include <asm/processor.h>
 #include <asm/reg.h>

arch/powerpc/kernel/dexcr.c +0 -0
@@ -19,6 +22,25 @@ static int __init dexcr_init(void)
  }
  early_initcall(dexcr_init);

arch/powerpc/kernel/dexcr.c +1 -0
+bool is_hashchk_trap(struct pt_regs const *regs)

arch/powerpc/kernel/dexcr.c static int __init dexcr_init(void) · line 35

Quoted 10 lines
arch/powerpc/kernel/dexcr.c +10 -0
+{
+       ppc_inst_t insn;
+
+       if (!cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+               return false;
+
+       if (get_user_instr(insn, (void __user *)regs->nip)) {
+               WARN_ON(1);
+               return false;
+       }

Nice series, just starting to have a look at it.

Quoted 8 lines

You probably don't want a WARN_ON() here because it's user triggerable
and isn't necessarily even indiciating a problem or attack if the app is doing code unmapping in order to get faults.

Check some of the other instruction emulation for what to do in case of
an EFAULT.

Alright, I'll take a look

Quoted 24 lines
Quoted 24 lines
arch/powerpc/kernel/dexcr.c +8 -0
+
+       if (ppc_inst_primary_opcode(insn) == 31 &&
+           get_xop(ppc_inst_val(insn)) == OP_31_XOP_HASHCHK)
+               return true;
+
+       return false;
+}
+

 unsigned long get_thread_dexcr(struct thread_struct const *t) {
        return DEFAULT_DEXCR;

arch/powerpc/kernel/traps.c +3 -0
index 9bdd79aa51cf..b83f5b382f24 100644
@@ -1516,6 +1516,12 @@ static void do_program_check(struct pt_regs
 *regs)
                                 return;
                         }
                 }
+
+               if (user_mode(regs) && is_hashchk_trap(regs)) {
+                       _exception(SIGILL, regs, ILL_ILLOPN, regs-
Quoted 6 lines

arch/powerpc/kernel/traps.c static void do_program_check(struct pt_regs *regs) · line 1523

Quoted 3 lines
arch/powerpc/kernel/traps.c +2 -0
 >nip);
+                       return;
+               }

I guess ILLOPN makes sense. Do you know if any other archs do similar?

Ah sorry, when refactoring Chris' patches I forgot to put back in the commit message that this is how ARM reports their similar check failure. For example, their FPAC handler in arch/arm64/kernel/traps.c:518 does this.

#9 lore raw

RFC PATCH 4/13 powerpc/dexcr: Support userspace ROP protection

In reply to Benjamin Gray

The ISA 3.1B hashst and hashchk instructions use a per-cpu SPR HASHKEYR to hold a key used in the hash calculation. This key should be different for each process to make it harder for a malicious process to recreate valid hash values for a victim process.

Add support for storing a per-thread hash key, and setting/clearing HASHKEYR appropriately.

---
 arch/powerpc/include/asm/book3s/64/kexec.h |  3 +++
 arch/powerpc/include/asm/processor.h       |  1 +
 arch/powerpc/include/asm/reg.h             |  1 +
 arch/powerpc/kernel/process.c              | 12 ++++++++++++
 4 files changed, 17 insertions(+)
arch/powerpc/include/asm/book3s/64/kexec.h +3 -0
index 563baf94a962..163de935df28 100644
@@ -24,6 +24,9 @@ static inline void reset_sprs(void)
 	if (cpu_has_feature(CPU_FTR_ARCH_31))
 		mtspr(SPRN_DEXCR, 0);
 
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		mtspr(SPRN_HASHKEYR, 0);
+
 	/*  Do we need isync()? We are going via a kexec reset */
 	isync();
 }
arch/powerpc/include/asm/processor.h +1 -0
index c17ec1e44c86..2381217c95dc 100644
@@ -264,6 +264,7 @@ struct thread_struct {
 	unsigned long   mmcr3;
 	unsigned long   sier2;
 	unsigned long   sier3;
+	unsigned long	hashkeyr;
 
 #endif
 };
arch/powerpc/include/asm/reg.h +1 -0
index cdd1f174c399..854664cf844f 100644
@@ -384,6 +384,7 @@
 #define SPRN_HRMOR	0x139	/* Real mode offset register */
 #define SPRN_HSRR0	0x13A	/* Hypervisor Save/Restore 0 */
 #define SPRN_HSRR1	0x13B	/* Hypervisor Save/Restore 1 */
+#define SPRN_HASHKEYR	0x1D4	/* Non-privileged hashst/hashchk key register */
 #define SPRN_ASDR	0x330	/* Access segment descriptor register */
 #define SPRN_DEXCR	0x33C	/* Dynamic execution control register */
 #define   DEXCR_PRO_MASK(aspect)	__MASK(63 - (32 + (aspect)))	/* Aspect number to problem state aspect mask */
arch/powerpc/kernel/process.c +12 -0
index 17d26f652b80..4d7b0c7641d0 100644
@@ -1229,6 +1229,9 @@ static inline void restore_sprs(struct thread_struct *old_thread,
 	    old_thread->tidr != new_thread->tidr)
 		mtspr(SPRN_TIDR, new_thread->tidr);
 
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		mtspr(SPRN_HASHKEYR, new_thread->hashkeyr);
+
 	if (cpu_has_feature(CPU_FTR_ARCH_31)) {
 		unsigned long new_dexcr = get_thread_dexcr(new_thread);
 
@@ -1818,6 +1821,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 		childregs->ppr = DEFAULT_PPR;
 
 	p->thread.tidr = 0;
+#endif
+#ifdef CONFIG_PPC_BOOK3S_64
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		p->thread.hashkeyr = current->thread.hashkeyr;
 #endif
 	/*
 	 * Run with the current AMR value of the kernel
@@ -1947,6 +1954,11 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.load_tm = 0;
 #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 #ifdef CONFIG_PPC_BOOK3S_64
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE)) {
+		current->thread.hashkeyr = get_random_long();
+		mtspr(SPRN_HASHKEYR, current->thread.hashkeyr);
+	}
+
 	if (cpu_has_feature(CPU_FTR_ARCH_31))
 		mtspr(SPRN_DEXCR, get_thread_dexcr(&current->thread));
 #endif /* CONFIG_PPC_BOOK3S_64 */
Signature · 1 line
2.38.1
reviewer Reviewed-by on v1 4/13
#10 lore raw

Re: [RFC PATCH 04/13] powerpc/dexcr: Support userspace ROP protection

Quoted 59 lines · Benjamin Gray

The ISA 3.1B hashst and hashchk instructions use a per-cpu SPR HASHKEYR to hold a key used in the hash calculation. This key should be different for each process to make it harder for a malicious process to recreate valid hash values for a victim process.

Add support for storing a per-thread hash key, and setting/clearing HASHKEYR appropriately.

---
 arch/powerpc/include/asm/book3s/64/kexec.h |  3 +++
 arch/powerpc/include/asm/processor.h       |  1 +
 arch/powerpc/include/asm/reg.h             |  1 +
 arch/powerpc/kernel/process.c              | 12 ++++++++++++
 4 files changed, 17 insertions(+)
arch/powerpc/include/asm/book3s/64/kexec.h +3 -0
index 563baf94a962..163de935df28 100644
@@ -24,6 +24,9 @@ static inline void reset_sprs(void)
 	if (cpu_has_feature(CPU_FTR_ARCH_31))
 		mtspr(SPRN_DEXCR, 0);
 
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		mtspr(SPRN_HASHKEYR, 0);
+
 	/*  Do we need isync()? We are going via a kexec reset */
 	isync();
 }
arch/powerpc/include/asm/processor.h +1 -0
index c17ec1e44c86..2381217c95dc 100644
@@ -264,6 +264,7 @@ struct thread_struct {
 	unsigned long   mmcr3;
 	unsigned long   sier2;
 	unsigned long   sier3;
+	unsigned long	hashkeyr;
 
 #endif
 };
arch/powerpc/include/asm/reg.h +1 -0
index cdd1f174c399..854664cf844f 100644
@@ -384,6 +384,7 @@
 #define SPRN_HRMOR	0x139	/* Real mode offset register */
 #define SPRN_HSRR0	0x13A	/* Hypervisor Save/Restore 0 */
 #define SPRN_HSRR1	0x13B	/* Hypervisor Save/Restore 1 */
+#define SPRN_HASHKEYR	0x1D4	/* Non-privileged hashst/hashchk key register */
 #define SPRN_ASDR	0x330	/* Access segment descriptor register */
 #define SPRN_DEXCR	0x33C	/* Dynamic execution control register */
 #define   DEXCR_PRO_MASK(aspect)	__MASK(63 - (32 + (aspect)))	/* Aspect number to problem state aspect mask */
arch/powerpc/kernel/process.c +0 -0
index 17d26f652b80..4d7b0c7641d0 100644

arch/powerpc/kernel/process.c static inline void restore_sprs(struct thread_struct *old_thread, · line 1233

Quoted 5 lines
arch/powerpc/kernel/process.c +2 -0
@@ -1229,6 +1229,9 @@ static inline void restore_sprs(struct thread_struct *old_thread,
 	    old_thread->tidr != new_thread->tidr)
 		mtspr(SPRN_TIDR, new_thread->tidr);
 
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		mtspr(SPRN_HASHKEYR, new_thread->hashkeyr);

I wonder if we'd want to avoid switching it when switching to kernel threads, and from kernel thread back to the same user thread. Might want to optimise it to do that in future but for an initial enablement patch this is okay.

arch/powerpc/kernel/process.c int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) · line 1828

Quoted 13 lines
arch/powerpc/kernel/process.c +5 -0
+
 	if (cpu_has_feature(CPU_FTR_ARCH_31)) {
 		unsigned long new_dexcr = get_thread_dexcr(new_thread);
 
@@ -1818,6 +1821,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 		childregs->ppr = DEFAULT_PPR;
 
 	p->thread.tidr = 0;
+#endif
+#ifdef CONFIG_PPC_BOOK3S_64
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
+		p->thread.hashkeyr = current->thread.hashkeyr;
 #endif

Similar comment about your accessor style, if we had get/set_thread_hashkeyr() functions then no ifdef required.

I think it is not quite per-process? I don't actually know how the user toolchain side is put together, but I'm thinking we can not give it a new salt on fork(), but we could on exec(). I think we could actually give each thread their own salt within a process too, right?

I don't know off the top of my head whether that can be translated into a simple test at the copy_thread level. For now you're giving out a new salt on exec I think, which should be fine at least to start with.

Thanks,
Nick

Quoted 16 lines
arch/powerpc/kernel/process.c +5 -0
 	/*
 	 * Run with the current AMR value of the kernel
@@ -1947,6 +1954,11 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.load_tm = 0;
 #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 #ifdef CONFIG_PPC_BOOK3S_64
+	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE)) {
+		current->thread.hashkeyr = get_random_long();
+		mtspr(SPRN_HASHKEYR, current->thread.hashkeyr);
+	}
+
 	if (cpu_has_feature(CPU_FTR_ARCH_31))
 		mtspr(SPRN_DEXCR, get_thread_dexcr(&current->thread));
 #endif /* CONFIG_PPC_BOOK3S_64 */
Signature · 1 line
2.38.1
#11 lore raw

Re: [RFC PATCH 04/13] powerpc/dexcr: Support userspace ROP protection

Quoted 7 lines · Nicholas Piggin

I think it is not quite per-process? I don't actually know how the user
toolchain side is put together, but I'm thinking we can not give it a new
salt on fork(), but we could on exec(). I think we could actually give
each thread their own salt within a process too, right?

Yeah, the error case is we return further than we called in a given execution context. A forked child may return after the fork, meaning it needs the same key as the parent for the hashchk to work. Exec can get a new key because we can't return with any existing hashes. I haven't seen enough of kernel thread support to know if/how we can give threads their own key. I believe they go through the fork() call that copies the parent key currently.

#12 lore raw

Re: [RFC PATCH 04/13] powerpc/dexcr: Support userspace ROP protection

Quoted 9 lines · Benjamin Gray
Quoted 7 lines · Nicholas Piggin

I think it is not quite per-process? I don't actually know how the user
toolchain side is put together, but I'm thinking we can not give it a new
salt on fork(), but we could on exec(). I think we could actually give
each thread their own salt within a process too, right?

Quoted 7 lines

Yeah, the error case is we return further than we called in a given execution context. A forked child may return after the fork, meaning it needs the same key as the parent for the hashchk to work. Exec can get a new key because we can't return with any existing hashes. I haven't seen enough of kernel thread support to know if/how we can give threads their own key. I believe they go through the fork() call that copies the parent key currently.

Could look at possibly doing per-thread keys afterward but what you're doing makes sense so no problem.

Thanks,
Nick

#13 lore raw

RFC PATCH 5/13 prctl: Define PowerPC DEXCR interface

In reply to Benjamin Gray

Adds the definitions and generic handler for prctl control of the PowerPC Dynamic Execution Control Register (DEXCR).

---
 include/uapi/linux/prctl.h | 14 ++++++++++++++
 kernel/sys.c               | 16 ++++++++++++++++
 2 files changed, 30 insertions(+)
include/uapi/linux/prctl.h +14 -0
index a5e06dcbba13..b4720e8de6f3 100644
@@ -281,6 +281,20 @@ struct prctl_mm_map {
 # define PR_SME_VL_LEN_MASK		0xffff
 # define PR_SME_VL_INHERIT		(1 << 17) /* inherit across exec */
 
+/* PowerPC Dynamic Execution Control Register (DEXCR) controls */
+#define PR_PPC_GET_DEXCR		65
+#define PR_PPC_SET_DEXCR		66
+/* DEXCR aspect to act on */
+# define PR_PPC_DEXCR_SBHE		0 /* Speculative branch hint enable */
+# define PR_PPC_DEXCR_IBRTPD		1 /* Indirect branch recurrent target prediction disable */
+# define PR_PPC_DEXCR_SRAPD		2 /* Subroutine return address prediction disable */
+# define PR_PPC_DEXCR_NPHIE		3 /* Non-privileged hash instruction enable */
+/* Action to apply / return */
+# define PR_PPC_DEXCR_PRCTL		(1 << 0)
+# define PR_PPC_DEXCR_SET_ASPECT	(1 << 1)
+# define PR_PPC_DEXCR_FORCE_SET_ASPECT	(1 << 2)
+# define PR_PPC_DEXCR_CLEAR_ASPECT	(1 << 3)
+
 #define PR_SET_VMA		0x53564d41
 # define PR_SET_VMA_ANON_NAME		0
 
kernel/sys.c +16 -0
index 5fd54bf0e886..55b8f7369059 100644
@@ -139,6 +139,12 @@
 #ifndef GET_TAGGED_ADDR_CTRL
 # define GET_TAGGED_ADDR_CTRL()		(-EINVAL)
 #endif
+#ifndef PPC_GET_DEXCR_ASPECT
+# define PPC_GET_DEXCR_ASPECT(a, b)	(-EINVAL)
+#endif
+#ifndef PPC_SET_DEXCR_ASPECT
+# define PPC_SET_DEXCR_ASPECT(a, b, c)	(-EINVAL)
+#endif
 
 /*
  * this is where the system-wide overflow UID and GID are defined, for
@@ -2623,6 +2629,16 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
 		error = sched_core_share_pid(arg2, arg3, arg4, arg5);
 		break;
 #endif
+	case PR_PPC_GET_DEXCR:
+		if (arg3 || arg4 || arg5)
+			return -EINVAL;
+		error = PPC_GET_DEXCR_ASPECT(me, arg2);
+		break;
+	case PR_PPC_SET_DEXCR:
+		if (arg4 || arg5)
+			return -EINVAL;
+		error = PPC_SET_DEXCR_ASPECT(me, arg2, arg3);
+		break;
 	case PR_SET_VMA:
 		error = prctl_set_vma(arg2, arg3, arg4, arg5);
 		break;
Signature · 1 line
2.38.1
reviewer Reviewed-by on v1 5/13
#14 lore raw

Re: [RFC PATCH 05/13] prctl: Define PowerPC DEXCR interface

Quoted 2 lines · Benjamin Gray

Adds the definitions and generic handler for prctl control of the PowerPC Dynamic Execution Control Register (DEXCR).

Assuming we'd go with the later prctl patches, this prep patch is nice way to split out some of the mechanism.

Quoted 68 lines
---
 include/uapi/linux/prctl.h | 14 ++++++++++++++
 kernel/sys.c               | 16 ++++++++++++++++
 2 files changed, 30 insertions(+)
include/uapi/linux/prctl.h +14 -0
index a5e06dcbba13..b4720e8de6f3 100644
@@ -281,6 +281,20 @@ struct prctl_mm_map {
 # define PR_SME_VL_LEN_MASK		0xffff
 # define PR_SME_VL_INHERIT		(1 << 17) /* inherit across exec */
 
+/* PowerPC Dynamic Execution Control Register (DEXCR) controls */
+#define PR_PPC_GET_DEXCR		65
+#define PR_PPC_SET_DEXCR		66
+/* DEXCR aspect to act on */
+# define PR_PPC_DEXCR_SBHE		0 /* Speculative branch hint enable */
+# define PR_PPC_DEXCR_IBRTPD		1 /* Indirect branch recurrent target prediction disable */
+# define PR_PPC_DEXCR_SRAPD		2 /* Subroutine return address prediction disable */
+# define PR_PPC_DEXCR_NPHIE		3 /* Non-privileged hash instruction enable */
+/* Action to apply / return */
+# define PR_PPC_DEXCR_PRCTL		(1 << 0)
+# define PR_PPC_DEXCR_SET_ASPECT	(1 << 1)
+# define PR_PPC_DEXCR_FORCE_SET_ASPECT	(1 << 2)
+# define PR_PPC_DEXCR_CLEAR_ASPECT	(1 << 3)
+
 #define PR_SET_VMA		0x53564d41
 # define PR_SET_VMA_ANON_NAME		0
 
kernel/sys.c +16 -0
index 5fd54bf0e886..55b8f7369059 100644
@@ -139,6 +139,12 @@
 #ifndef GET_TAGGED_ADDR_CTRL
 # define GET_TAGGED_ADDR_CTRL()		(-EINVAL)
 #endif
+#ifndef PPC_GET_DEXCR_ASPECT
+# define PPC_GET_DEXCR_ASPECT(a, b)	(-EINVAL)
+#endif
+#ifndef PPC_SET_DEXCR_ASPECT
+# define PPC_SET_DEXCR_ASPECT(a, b, c)	(-EINVAL)
+#endif
 
 /*
  * this is where the system-wide overflow UID and GID are defined, for
@@ -2623,6 +2629,16 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
 		error = sched_core_share_pid(arg2, arg3, arg4, arg5);
 		break;
 #endif
+	case PR_PPC_GET_DEXCR:
+		if (arg3 || arg4 || arg5)
+			return -EINVAL;
+		error = PPC_GET_DEXCR_ASPECT(me, arg2);
+		break;
+	case PR_PPC_SET_DEXCR:
+		if (arg4 || arg5)
+			return -EINVAL;
+		error = PPC_SET_DEXCR_ASPECT(me, arg2, arg3);
+		break;
 	case PR_SET_VMA:
 		error = prctl_set_vma(arg2, arg3, arg4, arg5);
 		break;
Signature · 1 line
2.38.1
#15 lore raw

RFC PATCH 6/13 powerpc/dexcr: Add prctl implementation

In reply to Benjamin Gray

Adds an initial prctl interface implementation. Unprivileged processes can query the current prctl setting, including whether an aspect is implemented by the hardware or is permitted to be modified by a setter prctl. Editable aspects can be changed by a CAP_SYS_ADMIN privileged process.

The prctl setting represents what the process itself has requested, and does not account for any overrides. Either the kernel or a hypervisor may enforce a different setting for an aspect.

Userspace can access a readonly view of the current DEXCR via SPR 812, and a readonly view of the aspects enforced by the hypervisor via SPR 455. A bitwise OR of these two SPRs will give the effective DEXCR aspect state of the process.

---
 arch/powerpc/include/asm/processor.h |  13 +++
 arch/powerpc/kernel/dexcr.c          | 133 ++++++++++++++++++++++++++-
 arch/powerpc/kernel/process.c        |   6 ++
 3 files changed, 151 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/processor.h +13 -0
index 2381217c95dc..4c995258f668 100644
@@ -265,6 +265,9 @@ struct thread_struct {
 	unsigned long   sier2;
 	unsigned long   sier3;
 	unsigned long	hashkeyr;
+	unsigned int	dexcr_override;
+	unsigned int	dexcr_mask;
+	unsigned int	dexcr_forced;
 
 #endif
 };
@@ -338,6 +341,16 @@ extern int set_endian(struct task_struct *tsk, unsigned int val);
 extern int get_unalign_ctl(struct task_struct *tsk, unsigned long adr);
 extern int set_unalign_ctl(struct task_struct *tsk, unsigned int val);
 
+#ifdef CONFIG_PPC_BOOK3S_64
+
+#define PPC_GET_DEXCR_ASPECT(tsk, asp) dexcr_prctl_get((tsk), (asp))
+#define PPC_SET_DEXCR_ASPECT(tsk, asp, val) dexcr_prctl_set((tsk), (asp), (val))
+
+int dexcr_prctl_get(struct task_struct *tsk, unsigned long asp);
+int dexcr_prctl_set(struct task_struct *tsk, unsigned long asp, unsigned long val);
+
+#endif
+
 extern void load_fp_state(struct thread_fp_state *fp);
 extern void store_fp_state(struct thread_fp_state *fp);
 extern void load_vr_state(struct thread_vr_state *vr);
arch/powerpc/kernel/dexcr.c +132 -1

148 lines not on this page · raw message

arch/powerpc/kernel/process.c +6 -0
index 4d7b0c7641d0..a280842750f9 100644
@@ -1825,6 +1825,12 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 #ifdef CONFIG_PPC_BOOK3S_64
 	if (cpu_has_feature(CPU_FTR_DEXCR_NPHIE))
 		p->thread.hashkeyr = current->thread.hashkeyr;
+
+	if (cpu_has_feature(CPU_FTR_ARCH_31)) {
+		p->thread.dexcr_override = current->thread.dexcr_override;
+		p->thread.dexcr_mask = current->thread.dexcr_mask;
+		p->thread.dexcr_forced = current->thread.dexcr_forced;
+	}
 #endif
 	/*
 	 * Run with the current AMR value of the kernel
Signature · 1 line
2.38.1
#16 lore raw

Re: [RFC PATCH 06/13] powerpc/dexcr: Add prctl implementation

Quoted 6 lines · Benjamin Gray

Adds an initial prctl interface implementation. Unprivileged processes can query the current prctl setting, including whether an aspect is implemented by the hardware or is permitted to be modified by a setter prctl. Editable aspects can be changed by a CAP_SYS_ADMIN privileged process.

Quoted 8 lines

The prctl setting represents what the process itself has requested, and does not account for any overrides. Either the kernel or a hypervisor may enforce a different setting for an aspect.

Userspace can access a readonly view of the current DEXCR via SPR 812, and a readonly view of the aspects enforced by the hypervisor via SPR 455. A bitwise OR of these two SPRs will give the effective DEXCR aspect state of the process.

You said (offline) that you were looking at the PR_SPEC_* speculation control APIs but that this was different enough that you needed a different one.

It would be good to know what some of those issues were in the changelog, would be nice to have some docs (could we add something to spec_ctrl.rst maybe?). I assume at least one difference is that some of our bits are not speculative but architectural (e.g., the stack hash check).

I also wonder if we could implement some of the PR_SPEC controls APIs by mapping relevant DEXCR aspects to them instead of (or as well as) the DEXCR controls? Or would the PR_SPEC users be amenable to extensions that make our usage fit a bit better?

I'm just thinking if we can reduce reliance on arch specific APIs a bit would be nice.

Quoted 13 lines
---
 arch/powerpc/include/asm/processor.h |  13 +++
 arch/powerpc/kernel/dexcr.c          | 133 ++++++++++++++++++++++++++-
 arch/powerpc/kernel/process.c        |   6 ++
 3 files changed, 151 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/processor.h +0 -0
index 2381217c95dc..4c995258f668 100644

arch/powerpc/include/asm/processor.h struct thread_struct { · line 269

Quoted 5 lines
arch/powerpc/include/asm/processor.h +2 -0
@@ -265,6 +265,9 @@ struct thread_struct {
 	unsigned long   sier2;
 	unsigned long   sier3;
 	unsigned long	hashkeyr;
+	unsigned int	dexcr_override;
+	unsigned int	dexcr_mask;

Hmm, what's the mask doing here? It only gets bits set and never cleared AFAIKS. What is different between an initial state and a SET then CLEAR state?

Thanks,
Nick

#17 lore raw

RFC PATCH 7/13 powerpc/dexcr: Add sysctl entry for SBHE system override

In reply to Benjamin Gray

The DEXCR Speculative Branch Hint Enable (SBHE) aspect controls whether the hints provided by BO field of Branch instructions are obeyed during speculative execution.

SBHE behaviour per ISA 3.1B:

0: The hints provided by BO field of Branch instructions may be
ignored during speculative execution

1: The hints provided by BO field of Branch instructions are obeyed
during speculative execution

Add a sysctl entry to allow changing this aspect globally in the system at runtime:

/proc/sys/kernel/speculative_branch_hint_enable

Three values are supported:

-1: Disable DEXCR SBHE sysctl override
0: Override and set DEXCR[SBHE] aspect to 0
1: Override and set DEXCR[SBHE] aspect to 1

Internally, introduces a mechanism to apply arbitrary system wide overrides on top of the prctl() config.

---
 arch/powerpc/kernel/dexcr.c | 125 ++++++++++++++++++++++++++++++++++++
 1 file changed, 125 insertions(+)
arch/powerpc/kernel/dexcr.c +125 -0
index 9290beed722a..8239bcc92026 100644
@@ -1,8 +1,11 @@
 #include <linux/cache.h>
 #include <linux/capability.h>
+#include <linux/cpu.h>
 #include <linux/init.h>
 #include <linux/prctl.h>
 #include <linux/sched.h>
+#include <linux/spinlock.h>
+#include <linux/sysctl.h>
 
 #include <asm/cpu_has_feature.h>
 #include <asm/cputable.h>
@@ -18,6 +21,58 @@
 #define DEXCR_PRCTL_EDITABLE (DEXCR_PRO_SBHE | DEXCR_PRO_IBRTPD | \
 			      DEXCR_PRO_SRAPD | DEXCR_PRO_NPHIE)
 
+/*
+ * Lock to protect system DEXCR override from concurrent updates.
+ * RCU semantics: writers take lock, readers are unlocked.
+ * Writers ensure the memory update is atomic, readers read
+ * atomically.
+ */
+static DEFINE_SPINLOCK(dexcr_sys_enforced_write_lock);
+
+struct mask_override {
+	union {
+		struct {
+			unsigned int mask;
+			unsigned int override;
+		};
+
+		/* Raw access for atomic read/write */
+		unsigned long all;
+	};
+};
+
+static struct mask_override dexcr_sys_enforced;
+
+static int spec_branch_hint_enable = -1;
+
+static void update_userspace_system_dexcr(unsigned int pro_mask, int value)
+{
+	struct mask_override update = { .all = 0 };
+
+	switch (value) {
+	case -1:  /* Clear the mask bit, clear the override bit */
+		break;
+	case 0:  /* Set the mask bit, clear the override bit */
+		update.mask |= pro_mask;
+		break;
+	case 1:  /* Set the mask bit, set the override bit */
+		update.mask |= pro_mask;
+		update.override |= pro_mask;
+		break;
+	}
+
+	spin_lock(&dexcr_sys_enforced_write_lock);
+
+	/* Use the existing values for the non-updated bits */
+	update.mask |= dexcr_sys_enforced.mask & ~pro_mask;
+	update.override |= dexcr_sys_enforced.override & ~pro_mask;
+
+	/* Atomically update system enforced aspects */
+	WRITE_ONCE(dexcr_sys_enforced.all, update.all);
+
+	spin_unlock(&dexcr_sys_enforced_write_lock);
+}
+
 static int __init dexcr_init(void)
 {
 	if (!early_cpu_has_feature(CPU_FTR_ARCH_31))
@@ -25,6 +80,9 @@ static int __init dexcr_init(void)
 
 	mtspr(SPRN_DEXCR, DEFAULT_DEXCR);
 
+	if (early_cpu_has_feature(CPU_FTR_DEXCR_SBHE))
+		update_userspace_system_dexcr(DEXCR_PRO_SBHE, spec_branch_hint_enable);
+
 	return 0;
 }
 early_initcall(dexcr_init);
@@ -52,9 +110,15 @@ unsigned long get_thread_dexcr(struct thread_struct const *t)
 {
 	unsigned long dexcr = DEFAULT_DEXCR;
 
+	/* Atomically read enforced mask & override */
+	struct mask_override enforced = READ_ONCE(dexcr_sys_enforced);
+
 	/* Apply prctl overrides */
 	dexcr = (dexcr & ~t->dexcr_mask) | t->dexcr_override;
 
+	/* Apply system overrides */
+	dexcr = (dexcr & ~enforced.mask) | enforced.override;
+
 	return dexcr;
 }
 
@@ -176,3 +240,64 @@ int dexcr_prctl_set(struct task_struct *task, unsigned long which, unsigned long
 
 	return 0;
 }
+
+#ifdef CONFIG_SYSCTL
+
+static const int min_sysctl_val = -1;
+
+static int sysctl_dexcr_sbhe_handler(struct ctl_table *table, int write,
+				     void *buf, size_t *lenp, loff_t *ppos)
+{
+	int err;
+	int prev = spec_branch_hint_enable;
+
+	if (!capable(CAP_SYS_ADMIN))
+		return -EPERM;
+
+	if (!cpu_has_feature(CPU_FTR_DEXCR_SBHE))
+		return -ENODEV;
+
+	err = proc_dointvec_minmax(table, write, buf, lenp, ppos);
+	if (err)
+		return err;
+
+	if (prev != spec_branch_hint_enable && write) {
+		update_userspace_system_dexcr(DEXCR_PRO_SBHE, spec_branch_hint_enable);
+		cpus_read_lock();
+		on_each_cpu(update_dexcr_on_cpu, NULL, 1);
+		cpus_read_unlock();
+	}
+
+	return 0;
+}
+
+static struct ctl_table dexcr_sbhe_ctl_table[] = {
+	{
+		.procname	= "speculative_branch_hint_enable",
+		.data		= &spec_branch_hint_enable,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= sysctl_dexcr_sbhe_handler,
+		.extra1		= (void *)&min_sysctl_val,
+		.extra2		= SYSCTL_ONE,
+	},
+	{}
+};
+
+static struct ctl_table dexcr_sbhe_ctl_root[] = {
+	{
+		.procname	= "kernel",
+		.mode		= 0555,
+		.child		= dexcr_sbhe_ctl_table,
+	},
+	{}
+};
+
+static int __init register_dexcr_aspects_sysctl(void)
+{
+	register_sysctl_table(dexcr_sbhe_ctl_root);
+	return 0;
+}
+device_initcall(register_dexcr_aspects_sysctl);
+
+#endif /* CONFIG_SYSCTL */
Signature · 1 line
2.38.1
#18 lore raw

Re: [RFC PATCH 07/13] powerpc/dexcr: Add sysctl entry for SBHE system override

Quoted 17 lines · Benjamin Gray

The DEXCR Speculative Branch Hint Enable (SBHE) aspect controls whether the hints provided by BO field of Branch instructions are obeyed during speculative execution.

SBHE behaviour per ISA 3.1B:

0: The hints provided by BO field of Branch instructions may be
ignored during speculative execution

1: The hints provided by BO field of Branch instructions are obeyed
during speculative execution

Add a sysctl entry to allow changing this aspect globally in the system at runtime:

/proc/sys/kernel/speculative_branch_hint_enable

Quoted 8 lines

Three values are supported:

-1: Disable DEXCR SBHE sysctl override
0: Override and set DEXCR[SBHE] aspect to 0
1: Override and set DEXCR[SBHE] aspect to 1

Internally, introduces a mechanism to apply arbitrary system wide overrides on top of the prctl() config.

Why have an override for this, and not others?

Thanks,
Nick

#19 lore raw

Re: [RFC PATCH 07/13] powerpc/dexcr: Add sysctl entry for SBHE system override

Quoted 25 lines · Nicholas Piggin
Quoted 24 lines · Benjamin Gray

The DEXCR Speculative Branch Hint Enable (SBHE) aspect controls whether
the hints provided by BO field of Branch instructions are obeyed during
speculative execution.

SBHE behaviour per ISA 3.1B:

0:      The hints provided by BO field of Branch instructions may be
        ignored during speculative execution

1:      The hints provided by BO field of Branch instructions are obeyed
        during speculative execution

Add a sysctl entry to allow changing this aspect globally in the system
at runtime:

        /proc/sys/kernel/speculative_branch_hint_enable

Three values are supported:

Quoted 9 lines
Quoted 6 lines

-1:     Disable DEXCR SBHE sysctl override
 0:     Override and set DEXCR[SBHE] aspect to 0
 1:     Override and set DEXCR[SBHE] aspect to 1

Internally, introduces a mechanism to apply arbitrary system wide overrides on top of the prctl() config.

Why have an override for this, and not others?

Should be in the commit message of course, but this aspect bleeds over to other processes, so a user may wish to prevent all processes from changing the value. The other aspects are probably only relevant to their own process, though the implementation here should support arbitrary system wide overrides.

#20 lore raw

RFC PATCH 8/13 powerpc/dexcr: Add enforced userspace ROP protection config

In reply to Benjamin Gray

The DEXCR Non-Privileged Hash Instruction Enable (NPHIE) aspect controls whether the hashst and hashchk instructions are treated as no-ops by the CPU.

NPHIE behaviour per ISA 3.1B:

0: hashst and hashchk instructions are executed as no-ops
(even when allowed by PCR)

1: hashst and hashchk instructions are executed normally
(if allowed by PCR)

Currently this aspect may be set per-process by prctl() or enforced globally by the hypervisor.

Add a kernel config option PPC_USER_ROP_PROTECT to enforce DEXCR[NPHIE] globally regardless of prctl() or hypervisor. If set, don't report NPHIE as editable via prctl(), as the prctl() value can never take effect.

---
 arch/powerpc/Kconfig        |  5 +++++
 arch/powerpc/kernel/dexcr.c | 15 +++++++++++----
 2 files changed, 16 insertions(+), 4 deletions(-)
arch/powerpc/Kconfig +5 -0
index 699df27b0e2f..ba3458d07744 100644
@@ -434,6 +434,11 @@ config PGTABLE_LEVELS
 	default 2 if !PPC64
 	default 4
 
+config PPC_USER_ROP_PROTECT
+	bool
+	depends on PPC_BOOK3S_64
+	default y
+
 source "arch/powerpc/sysdev/Kconfig"
 source "arch/powerpc/platforms/Kconfig"
 
arch/powerpc/kernel/dexcr.c +11 -4
index 8239bcc92026..394140fc23aa 100644
@@ -2,6 +2,7 @@
 #include <linux/capability.h>
 #include <linux/cpu.h>
 #include <linux/init.h>
+#include <linux/kconfig.h>
 #include <linux/prctl.h>
 #include <linux/sched.h>
 #include <linux/spinlock.h>
@@ -18,8 +19,8 @@
 #define DEFAULT_DEXCR	0
 
 /* Allow process configuration of these by default */
-#define DEXCR_PRCTL_EDITABLE (DEXCR_PRO_SBHE | DEXCR_PRO_IBRTPD | \
-			      DEXCR_PRO_SRAPD | DEXCR_PRO_NPHIE)
+static unsigned long dexcr_prctl_editable __ro_after_init =
+	DEXCR_PRO_SBHE | DEXCR_PRO_IBRTPD | DEXCR_PRO_SRAPD | DEXCR_PRO_NPHIE;
 
 /*
  * Lock to protect system DEXCR override from concurrent updates.
@@ -83,6 +84,12 @@ static int __init dexcr_init(void)
 	if (early_cpu_has_feature(CPU_FTR_DEXCR_SBHE))
 		update_userspace_system_dexcr(DEXCR_PRO_SBHE, spec_branch_hint_enable);
 
+	if (early_cpu_has_feature(CPU_FTR_DEXCR_NPHIE) &&
+	    IS_ENABLED(CONFIG_PPC_USER_ROP_PROTECT)) {
+		update_userspace_system_dexcr(DEXCR_PRO_NPHIE, 1);
+		dexcr_prctl_editable &= ~DEXCR_PRO_NPHIE;
+	}
+
 	return 0;
 }
 early_initcall(dexcr_init);
@@ -131,7 +138,7 @@ static int dexcr_aspect_get(struct task_struct *task, unsigned int aspect)
 {
 	int ret = 0;
 
-	if (aspect & DEXCR_PRCTL_EDITABLE)
+	if (aspect & dexcr_prctl_editable)
 		ret |= PR_PPC_DEXCR_PRCTL;
 
 	if (aspect & task->thread.dexcr_mask) {
@@ -174,7 +181,7 @@ int dexcr_prctl_get(struct task_struct *task, unsigned long which)
 
 static int dexcr_aspect_set(struct task_struct *task, unsigned int aspect, unsigned long ctrl)
 {
-	if (!(aspect & DEXCR_PRCTL_EDITABLE))
+	if (!(aspect & dexcr_prctl_editable))
 		return -ENXIO;  /* Aspect is not allowed to be changed by prctl */
 
 	if (aspect & task->thread.dexcr_forced)
Signature · 1 line
2.38.1
#21 lore raw

RFC PATCH 9/13 selftests/powerpc: Add more utility macros

In reply to Benjamin Gray

Adds more assertion variants to provide more context behind why a failure occurred.

The SIGSAFE_FAIL_* variants are to allow safely asserting conditions in a signal handler (though we are about to exit, so it's unlikely to run into an issue with regular FAIL_IF_EXIT).

Also adds an ARRAY_SIZE macro.

These will be used by the following DEXCR selftests.

---
 .../testing/selftests/powerpc/include/utils.h | 44 +++++++++++++++++++
 1 file changed, 44 insertions(+)
tools/testing/selftests/powerpc/include/utils.h +44 -0
index 95f3a24a4569..b03d2192c6f6 100644
@@ -9,12 +9,19 @@
 #define __cacheline_aligned __attribute__((aligned(128)))
 
 #include <stdint.h>
+#include <stdio.h>
 #include <stdbool.h>
+#include <string.h>
+#include <unistd.h>
 #include <linux/auxvec.h>
 #include <linux/perf_event.h>
 #include <asm/cputable.h>
 #include "reg.h"
 
+#ifndef ARRAY_SIZE
+# define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
+#endif
+
 /* Avoid headaches with PRI?64 - just use %ll? always */
 typedef unsigned long long u64;
 typedef   signed long long s64;
@@ -111,6 +118,16 @@ do {								\
 	}							\
 } while (0)
 
+#define FAIL_IF_MSG(x, msg)					\
+do {								\
+	if ((x)) {						\
+		fprintf(stderr,					\
+		"[FAIL] Test FAILED on line %d: %s\n", 		\
+		__LINE__, msg);					\
+		return 1;					\
+	}							\
+} while (0)
+
 #define FAIL_IF_EXIT(x)						\
 do {								\
 	if ((x)) {						\
@@ -120,6 +137,16 @@ do {								\
 	}							\
 } while (0)
 
+#define FAIL_IF_EXIT_MSG(x, msg)				\
+do {								\
+	if ((x)) {						\
+		fprintf(stderr,					\
+		"[FAIL] Test FAILED on line %d: %s\n", 		\
+		__LINE__, msg);					\
+		_exit(1);					\
+	}							\
+} while (0)
+
 /* The test harness uses this, yes it's gross */
 #define MAGIC_SKIP_RETURN_VALUE	99
 
@@ -149,6 +176,23 @@ do {								\
 		ssize_t nbytes __attribute__((unused)); \
 		nbytes = write(STDERR_FILENO, msg, strlen(msg)); })
 
+#define SIGSAFE_FAIL_IF_EXIT(x)							\
+do {										\
+	if ((x)) {								\
+		sigsafe_err("[FAIL] Test FAILED on line " str(__LINE__) "\n");	\
+		_exit(1);							\
+	}									\
+} while (0)
+
+#define SIGSAFE_FAIL_IF_EXIT_MSG(x, msg)					\
+do {										\
+	if ((x)) {								\
+		sigsafe_err("[FAIL] Test FAILED on line " 			\
+			    str(__LINE__) ": " msg "\n");			\
+		_exit(1);							\
+	}									\
+} while (0)
+
 /* POWER9 feature */
 #ifndef PPC_FEATURE2_ARCH_3_00
 #define PPC_FEATURE2_ARCH_3_00 0x00800000
Signature · 1 line
2.38.1
#22 lore raw

RFC PATCH 10/13 selftests/powerpc: Add hashst/hashchk test

In reply to Benjamin Gray

Test the kernel DEXCR[NPHIE] interface and hashchk exception handling.

Introduces with it a DEXCR utils library for common DEXCR operations.

---
 tools/testing/selftests/powerpc/Makefile      |   1 +
 .../selftests/powerpc/dexcr/.gitignore        |   1 +
 .../testing/selftests/powerpc/dexcr/Makefile  |   9 +
 tools/testing/selftests/powerpc/dexcr/dexcr.c | 118 +++++++++
 tools/testing/selftests/powerpc/dexcr/dexcr.h |  52 ++++
 .../selftests/powerpc/dexcr/hashchk_test.c    | 229 ++++++++++++++++++
 tools/testing/selftests/powerpc/include/reg.h |   4 +
 7 files changed, 414 insertions(+)
 create mode 100644 tools/testing/selftests/powerpc/dexcr/.gitignore
 create mode 100644 tools/testing/selftests/powerpc/dexcr/Makefile
 create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr.h
 create mode 100644 tools/testing/selftests/powerpc/dexcr/hashchk_test.c
tools/testing/selftests/powerpc/Makefile +1 -0
index 6ba95cd19e42..00dbd000ee01 100644
@@ -17,6 +17,7 @@ SUB_DIRS = alignment		\
 	   benchmarks		\
 	   cache_shape		\
 	   copyloops		\
+	   dexcr		\
 	   dscr			\
 	   mm			\
 	   nx-gzip		\
tools/testing/selftests/powerpc/dexcr/.gitignore new file +1 -0
new file mode 100644
index 000000000000..37adb7f47832
@@ -0,0 +1 @@
+hashchk_user
tools/testing/selftests/powerpc/dexcr/Makefile new file +9 -0
new file mode 100644
index 000000000000..4b4380d4d986
@@ -0,0 +1,9 @@
+TEST_GEN_PROGS := hashchk_test
+
+TEST_FILES := settings
+top_srcdir = ../../../../..
+include ../../lib.mk
+
+HASHCHK_TEST_CFLAGS = -no-pie $(call cc-option,-mno-rop-protect)
+
+$(TEST_GEN_PROGS): ../harness.c ../utils.c ./dexcr.c
tools/testing/selftests/powerpc/dexcr/dexcr.c new file +118 -0

118 lines not on this page · raw message

tools/testing/selftests/powerpc/dexcr/dexcr.h new file +52 -0

52 lines not on this page · raw message

tools/testing/selftests/powerpc/dexcr/hashchk_test.c new file +229 -0

229 lines not on this page · raw message

tools/testing/selftests/powerpc/include/reg.h +4 -0
index d5a547f72669..cbb5979cb3e2 100644
@@ -19,6 +19,8 @@
 #define mb()		asm volatile("sync" : : : "memory");
 #define barrier()	asm volatile("" : : : "memory");
 
+#define SPRN_HDEXCR    455
+
 #define SPRN_MMCR2     769
 #define SPRN_MMCRA     770
 #define SPRN_MMCR0     779
@@ -47,6 +49,8 @@
 #define SPRN_SDAR      781
 #define SPRN_SIER      768
 
+#define SPRN_DEXCR     812
+
 #define SPRN_TEXASR     0x82    /* Transaction Exception and Status Register */
 #define SPRN_TFIAR      0x81    /* Transaction Failure Inst Addr    */
 #define SPRN_TFHAR      0x80    /* Transaction Failure Handler Addr */
Signature · 1 line
2.38.1
#23 lore raw

RFC PATCH 11/13 selftests/powerpc: Add DEXCR prctl, sysctl interface test

In reply to Benjamin Gray

Test the prctl and sysctl interfaces of the DEXCR.

This adds a new capabilities util for getting and setting CAP_SYS_ADMIN. Adding this avoids depending on an external libcap package. There is a similar implementation (and reason) in the tools/testing/selftests/bpf subtree but there's no obvious place to move it for sharing.

---
 .../selftests/powerpc/dexcr/.gitignore        |   1 +
 .../testing/selftests/powerpc/dexcr/Makefile  |   4 +-
 tools/testing/selftests/powerpc/dexcr/cap.c   |  72 ++++++
 tools/testing/selftests/powerpc/dexcr/cap.h   |  18 ++
 tools/testing/selftests/powerpc/dexcr/dexcr.h |   2 +
 .../selftests/powerpc/dexcr/dexcr_test.c      | 241 ++++++++++++++++++
 6 files changed, 336 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/powerpc/dexcr/cap.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/cap.h
 create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr_test.c
tools/testing/selftests/powerpc/dexcr/.gitignore +1 -0
index 37adb7f47832..035a1fcd8fb3 100644
@@ -1 +1,2 @@
+dexcr_test
 hashchk_user
tools/testing/selftests/powerpc/dexcr/Makefile +2 -2
index 4b4380d4d986..9814e72a4afa 100644
@@ -1,4 +1,4 @@
-TEST_GEN_PROGS := hashchk_test
+TEST_GEN_PROGS := dexcr_test hashchk_test
 
 TEST_FILES := settings
 top_srcdir = ../../../../..
@@ -6,4 +6,4 @@ include ../../lib.mk
 
 HASHCHK_TEST_CFLAGS = -no-pie $(call cc-option,-mno-rop-protect)
 
-$(TEST_GEN_PROGS): ../harness.c ../utils.c ./dexcr.c
+$(TEST_GEN_PROGS): ../harness.c ../utils.c ./dexcr.c ./cap.c
tools/testing/selftests/powerpc/dexcr/cap.c new file +72 -0

72 lines not on this page · raw message

tools/testing/selftests/powerpc/dexcr/cap.h new file +18 -0
new file mode 100644
index 000000000000..41f41dda9862
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Simple capabilities getter/setter
+ *
+ * This header file contains helper functions and macros
+ * required to get and set capabilities(7). Introduced so
+ * we aren't the first to rely on libcap.
+ */
+#ifndef _SELFTESTS_POWERPC_DEXCR_CAP_H
+#define _SELFTESTS_POWERPC_DEXCR_CAP_H
+
+#include <stdbool.h>
+
+bool check_cap_sysadmin(void);
+
+void drop_cap_sysadmin(void);
+
+#endif  /* _SELFTESTS_POWERPC_DEXCR_CAP_H */
tools/testing/selftests/powerpc/dexcr/dexcr.h +2 -0
index fb8007bf19f8..b90633ae49e9 100644
@@ -21,6 +21,8 @@
 #define DEXCR_PRO_SRAPD		DEXCR_PRO_MASK(4)
 #define DEXCR_PRO_NPHIE		DEXCR_PRO_MASK(5)
 
+#define SYSCTL_DEXCR_SBHE	"/proc/sys/kernel/speculative_branch_hint_enable"
+
 enum DexcrSource {
 	UDEXCR,		/* Userspace DEXCR value */
 	ENFORCED,	/* Enforced by hypervisor */
tools/testing/selftests/powerpc/dexcr/dexcr_test.c new file +241 -0

241 lines not on this page · raw message

Signature · 1 line
2.38.1
#24 lore raw

RFC PATCH 12/13 selftests/powerpc: Add DEXCR status utility lsdexcr

In reply to Benjamin Gray

Add a utility 'lsdexcr' to print the current DEXCR status. Useful for quickly checking the status when debugging test failures, using the sysctl interfaces manually, or just wanting to check it.

Example output:

          Requested: 84000000 (SBHE, NPHIE)

Hypervisor enforced: 00000000
Effective: 84000000 (SBHE, NPHIE)

SBHE * (0): set, prctl editable (Speculative branch hint enable)
IBRTPD (3): clear, prctl editable (Indirect branch recurrent target prediction disable)
SRAPD (4): clear, prctl editable (Subroutine return address prediction disable)
NPHIE * (5): set (Non-privileged hash instruction enable)

Global SBHE override: 1 (set)

---
 .../selftests/powerpc/dexcr/.gitignore        |   1 +
 .../testing/selftests/powerpc/dexcr/Makefile  |   2 +
 .../testing/selftests/powerpc/dexcr/lsdexcr.c | 178 ++++++++++++++++++
 3 files changed, 181 insertions(+)
 create mode 100644 tools/testing/selftests/powerpc/dexcr/lsdexcr.c
tools/testing/selftests/powerpc/dexcr/.gitignore +1 -0
index 035a1fcd8fb3..7dd2fad93732 100644
@@ -1,2 +1,3 @@
 dexcr_test
 hashchk_user
+lsdexcr
tools/testing/selftests/powerpc/dexcr/Makefile +2 -0
index 9814e72a4afa..8cb732cda7e7 100644
@@ -1,4 +1,5 @@
 TEST_GEN_PROGS := dexcr_test hashchk_test
+TEST_GEN_FILES := lsdexcr
 
 TEST_FILES := settings
 top_srcdir = ../../../../..
@@ -7,3 +8,4 @@ include ../../lib.mk
 HASHCHK_TEST_CFLAGS = -no-pie $(call cc-option,-mno-rop-protect)
 
 $(TEST_GEN_PROGS): ../harness.c ../utils.c ./dexcr.c ./cap.c
+$(TEST_GEN_FILES): ../utils.c ./dexcr.c
tools/testing/selftests/powerpc/dexcr/lsdexcr.c new file +178 -0

178 lines not on this page · raw message

Signature · 1 line
2.38.1
#25 lore raw

RFC PATCH 13/13 Documentation: Document PowerPC kernel DEXCR interface

In reply to Benjamin Gray

Describe the DEXCR and document how to interact with it via the prctl and sysctl interfaces.

---
 Documentation/powerpc/dexcr.rst | 183 ++++++++++++++++++++++++++++++++
 Documentation/powerpc/index.rst |   1 +
 2 files changed, 184 insertions(+)
 create mode 100644 Documentation/powerpc/dexcr.rst
Documentation/powerpc/dexcr.rst new file +183 -0
new file mode 100644
index 000000000000..3c995f4b9fe0
@@ -0,0 +1,183 @@
+==========================================
+DEXCR (Dynamic Execution Control Register)
+==========================================
+
+Overview
+========
+
+The DEXCR is a privileged special purpose register (SPR) introduced in
+PowerPC ISA 3.1B (Power10) that allows per-cpu control over several dynamic
+execution behaviours. These behaviours include speculation (e.g., indirect
+branch target prediction) and enabling return-oriented programming (ROP)
+protection instructions.
+
+The execution control is exposed in hardware as up to 32 bits ('aspects') in
+the DEXCR. Each aspect controls a certain behaviour, and can be set or cleared
+to enable/disable the aspect. There are several variants of the DEXCR for
+different purposes:
+
+DEXCR
+    A priviliged SPR that can control aspects for userspace and kernel space
+HDEXCR
+    A hypervisor-privileged SPR that can control aspects for the hypervisor and
+    enforce aspects for the kernel and userspace.
+UDEXCR
+    An optional ultravisor-privileged SPR that can control aspects for the ultravisor.
+
+Userspace can examine the current DEXCR state using a dedicated SPR that
+provides a non-privileged read-only view of the userspace DEXCR aspects.
+There is also an SPR that provides a read-only view of the hypervisor enforced
+aspects, which ORed with the userspace DEXCR view gives the effective DEXCR
+state for a process.
+
+
+User API
+========
+
+prctl()
+-------
+
+A process can control its own userspace DEXCR value using the
+``PR_PPC_GET_DEXCR`` and ``PR_PPC_SET_DEXCR`` pair of
+:manpage:`prctl(2)` commands. These calls have the form::
+
+    prctl(PR_PPC_GET_DEXCR, unsigned long aspect, 0, 0, 0);
+    prctl(PR_PPC_SET_DEXCR, unsigned long aspect, unsigned long flags, 0, 0);
+
+Where ``aspect`` (``arg1``) is a constant and ``flags`` (``arg2``) is a bifield.
+The possible aspect and flag values are as follows. Note there is no relation
+between aspect value and ``prctl()`` constant value.
+
+.. flat-table::
+   :header-rows: 1
+   :widths: 2 7 1
+
+   * - ``prctl()`` constant
+     - Aspect name
+     - Aspect bit
+
+   * - ``PR_PPC_DEXCR_SBHE``
+     - Speculative Branch Hint Enable (SBHE)
+     - 0
+
+   * - ``PR_PPC_DEXCR_IBRTPD``
+     - Indirect Branch Recurrent Target Prediction Disable (IBRTPD)
+     - 3
+
+   * - ``PR_PPC_DEXCR_SRAPD``
+     - Subroutine Return Address Prediction Disable (SRAPD)
+     - 4
+
+   * - ``PR_PPC_DEXCR_NPHIE``
+     - Non-Privileged Hash Instruction Enable (NPHIE)
+     - 5
+
+.. flat-table::
+   :header-rows: 1
+   :widths: 2 8
+
+   * - ``prctl()`` flag
+     - Meaning
+
+   * - ``PR_PPC_DEXCR_PRCTL``
+     - This aspect can be configured with ``prctl(PR_PPC_SET_DEXCR, ...)``
+
+   * - ``PR_PPC_DEXCR_SET_ASPECT``
+     - This aspect is set
+
+   * - ``PR_PPC_DEXCR_FORCE_SET_ASPECT``
+     - This aspect is set and cannot be undone. A subsequent
+       ``prctl(..., PR_PPC_DEXCR_CLEAR_ASPECT)`` will fail.
+
+   * - ``PR_PPC_DEXCR_CLEAR_ASPECT``
+     - This aspect is clear
+
+Note that
+
+* The ``*_SET_ASPECT`` / ``*_CLEAR_ASPECT`` refers to setting/clearing the bit in the DEXCR.
+  For example::
+
+      prctl(PR_PPC_SET_DEXCR, PR_PPC_DEXCR_IBRTPD, PR_PPC_DEXCR_SET_ASPECT, 0, 0);
+
+  will set the IBRTPD aspect bit in the DEXCR, causing indirect branch prediction
+  to be disabled.
+
+* The status returned by ``PR_PPC_GET_DEXCR`` does not include any alternative
+  config overrides. To see the true DEXCR state software should read the appropriate
+  SPRs directly.
+
+* A forced aspect will still report ``PR_PPC_DEXCR_PRCTL`` if it would
+  otherwise be editable.
+
+* The aspect state when starting a process is copied from the parent's
+  state on :manpage:`fork(2)` and :manpage:`execve(2)`. Aspects may also be set
+  or cleared by the kernel on process creation.
+
+Use ``PR_PPC_SET_DEXCR`` with one of ``PR_PPC_DEXCR_SET_ASPECT``,
+``PR_PPC_DEXCR_FORCE_SET_ASPECT``, or ``PR_PPC_DEXCR_CLEAR_ASPECT`` to edit a
+ given aspect.
+
+Common error codes for both getting and setting the DEXCR are as follows:
+
+.. flat-table::
+   :header-rows: 1
+   :widths: 2 8
+
+   * - Error
+     - Meaning
+
+   * - ``EINVAL``
+     - The DEXCR is not supported by the kernel.
+
+   * - ``ENODEV``
+     - The aspect is not recognised by the kernel or not supported by the hardware.
+
+``PR_PPC_SET_DEXCR`` may also report the following error codes:
+
+.. flat-table::
+   :header-rows: 1
+   :widths: 2 8
+
+   * - Error
+     - Meaning
+
+   * - ``ERANGE``
+     - ``arg2`` is incorrect. E.g., it does not select an action (set/clear),
+       or the flags are not recognised by the kernel.
+
+   * - ``ENXIO``
+     - The aspect is not editable via ``prctl()``.
+
+   * - ``EPERM``
+     - The process does not have sufficient privilege to modify this aspect,
+       or the aspect has been force set and cannot be modified.
+
+
+sysctl
+------
+
+Some aspects can be modified globally via :manpage:`sysctl(8)` entries. Such global
+modifications are applied after any process modifications. Any ``prctl()`` call to
+an overridden aspect this aspect may still report it as editable. The prctl setting
+will take effect again if the global override is restored to its default state.
+
+A global SBHE config is exposed in ``/proc/sys/kernel/speculative_branch_hint_enable``.
+Any process can read the current config value from it. Privileged processes can
+write to it to change the config. The new config is applied to all current and future
+processes (though note the kernel cannot override any hypervisor enforced aspects).
+
+.. flat-table::
+   :header-rows: 1
+   :widths: 2 8
+
+   * - Value
+     - Meaning
+
+   * - ``-1``
+     - Do not change from default or ``prctl()`` config.
+
+   * - ``0``
+     - Force clear aspect.
+
+   * - ``1``
+     - Force set aspect.
Documentation/powerpc/index.rst +1 -0
index 85e80e30160b..d33b554ca7ba 100644
@@ -15,6 +15,7 @@ powerpc
     cxl
     cxlflash
     dawr-power9
+    dexcr
     dscr
     eeh-pci-error-recovery
     elf_hwcaps
Signature · 1 line
2.38.1
#26 lore raw

Re: [RFC PATCH 13/13] Documentation: Document PowerPC kernel DEXCR interface

Quoted 2 lines · Benjamin Gray

Describe the DEXCR and document how to interact with it via the prctl and sysctl interfaces.

Oh you've got the docs here, sorry. Thanks for that. I don't know enough yet to give much useful feedback on the API. I think at least all the mechanism stuff up to the prctl API looks pretty straightfoward so would like to get that merged if we can.

Might need a bit more time and discussion on the API. Interestingly because the hashchk aspect is architectural, we may not be able to necessarily sanely enable that, because if it was disabled to start out with, our callchain up to the prctl call I think would have no return hashes set so we'd immediately fail on our first return.

Thanks,
Nick

#27 lore raw

Re: [RFC PATCH 13/13] Documentation: Document PowerPC kernel DEXCR interface

Quoted 5 lines · Nicholas Piggin

Might need a bit more time and discussion on the API. Interestingly because the hashchk aspect is architectural, we may not be able to necessarily sanely enable that, because if it was disabled to start out with, our callchain up to the prctl call I think would have no return hashes set so we'd immediately fail on our first return.

I assumed it could eventually be supported in whatever startup wrapper programs are built with, so either as one of the first things before any calls, or the compiler could skip putting hash instructions in the wrapper altogether. The ELF file itself might even be able to request bits be enabled, so the kernel would start the process correctly.

As it is inherited, it's also possible for a wrapper program to set a specific DEXCR before a child runs. E.g.,

	fork();
	prctl(...NPHIE...);
	exec();

I hadn't thought of the prctl call itself causing an unbalanced hashchk when it returns, but that should be solvable with an inline syscall.

#28 lore raw
In reply to Benjamin Gray
Quoted 16 lines · Benjamin Gray

This series is based on initial work by Chris Riedl that was not sent to the list.

Adds a kernel interface for userspace to interact with the DEXCR. The DEXCR is a SPR that allows control over various execution 'aspects', such as indirect branch prediction and enabling the hashst/hashchk instructions. Further details are in ISA 3.1B Book 3 chapter 12.

This RFC proposes an interface for users to interact with the DEXCR. It aims to support

* Querying supported aspects
* Getting/setting aspects on a per-process level
* Allowing global overrides across all processes

Quoted 7 lines

There are some parts that I'm not sure on the best way to approach (hence RFC):

* The feature names in arch/powerpc/kernel/dt_cpu_ftrs.c appear to be unimplemented
  in skiboot, so are being defined by this series. Is being so verbose fine?

These are going to need to be added to skiboot before they can be referenced in the kernel. Inclusion in skiboot makes them ABI, the kernel is just a consumer.

Quoted 3 lines

* What aspects should be editable by a process? E.g., SBHE has effects that potentially bleed into other processes. Should it only be system wide configurable?

For context, ISA 3.1B p1358 says:

In some micro-architectures, the execution behav- ior controlled by aspect 0 is difficult to change with any degree of timing precision. The change may also bleed over into other threads on the same pro- cessor. Any environment that has a dependence on the more secure setting of aspect 0 should not change the value, and ideally should share a pro- cessor only with similar threads. For other environ- ments, changes to the effective value of aspect 0 represent a relative risk tolerance for its aspect of execution behavior, with the understanding that there will be significant hysteresis in the execution behavior.

If a process sets SBHE for itself and all it takes is context switching from a process with SBHE unset to cause exposure, then yeah I think it should just be global. I doubt branch hints have enough impact for process granularity to be especially desirable anyway.

Quoted 6 lines

* Should configuring certain aspects for the process be non- privileged? E.g.,
  Is there harm in always allowing configuration of IBRTPD, SRAPD? The *FORCE_SET*
  action prevents further process local changes regardless of privilege.

I'm not aware of a reason why it would be a problem to allow unprivileged configuration as long as there's a way to prevent further changes. The concerning case is if a mitigation is set by a trusted process context, and then untrusted code is executed that manages to turn the mitigation off again.

Quoted 5 lines

* The tests fail Patchwork CI because of the new prctl macros, and the CI
  doesn't run headers_install and add -isystem <buildpath>/usr/include to
  the make command.

The CI runs on x86 and cross compiles the kernel and selftests, and boots are done in qemu tcg. Maybe we can skip the build if the symbols are undefined or do something like

	#ifndef PR_PPC_DEXCR_...
		return KSFT_SKIP;
	#endif

in the test itself?

Quoted 9 lines

* On handling an exception, I don't check if the NPHIE bit is enabled in the DEXCR.
  To do so would require reading both the DEXCR and HDEXCR, for little gain (it
  should only matter that the current instruction was a hashchk. If so, the only
  reason it would cause an exception is the failed check. If the instruction is
  rewritten between exception and check we'd be wrong anyway).

For context, the hashst and hashchk instructions are implemented using previously reserved nops. I'm not aware of any reason a nop could trap (i.e. we could check for a trap that came from hashchk even if NPHIE is not set), but afaik that'd be the only reason we would have to check.

Quoted 75 lines

The series is based on the earlier selftest utils series[1], so the tests won't build
at all without applying that first. The kernel side should build fine on ppc/next
247f34f7b80357943234f93f247a1ae6b6c3a740 though.

[1]:
https://patchwork.ozlabs.org/project/linuxppc-dev/cover/[email protected]/

Benjamin Gray (13):
  powerpc/book3s: Add missing <linux/sched.h> include
  powerpc: Add initial Dynamic Execution Control Register (DEXCR) support
  powerpc/dexcr: Handle hashchk exception powerpc/dexcr: Support userspace ROP protection
  prctl: Define PowerPC DEXCR interface
  powerpc/dexcr: Add prctl implementation powerpc/dexcr: Add sysctl entry for SBHE system override powerpc/dexcr: Add enforced userspace ROP protection config selftests/powerpc: Add more utility macros selftests/powerpc: Add hashst/hashchk test selftests/powerpc: Add DEXCR prctl, sysctl interface test selftests/powerpc: Add DEXCR status utility lsdexcr
  Documentation: Document PowerPC kernel DEXCR interface

 Documentation/powerpc/dexcr.rst               | 183 +++++++++++ Documentation/powerpc/index.rst               |   1 + arch/powerpc/Kconfig                          |   5 + arch/powerpc/include/asm/book3s/64/kexec.h    |   6 + arch/powerpc/include/asm/book3s/64/kup.h      |   1 + arch/powerpc/include/asm/cputable.h           |   8 +- arch/powerpc/include/asm/ppc-opcode.h         |   1 + arch/powerpc/include/asm/processor.h          |  33 ++ arch/powerpc/include/asm/reg.h                |   7 + arch/powerpc/kernel/Makefile                  |   1 + arch/powerpc/kernel/dexcr.c                   | 310
++++++++++++++++++
 arch/powerpc/kernel/dt_cpu_ftrs.c             |   4 + arch/powerpc/kernel/process.c                 |  31 +- arch/powerpc/kernel/prom.c                    |   4 + arch/powerpc/kernel/traps.c                   |   6 + include/uapi/linux/prctl.h                    |  14 + kernel/sys.c                                  |  16 + tools/testing/selftests/powerpc/Makefile      |   1 + .../selftests/powerpc/dexcr/.gitignore        |   3 + .../testing/selftests/powerpc/dexcr/Makefile  |  11 + tools/testing/selftests/powerpc/dexcr/cap.c   |  72 ++++ tools/testing/selftests/powerpc/dexcr/cap.h   |  18 + tools/testing/selftests/powerpc/dexcr/dexcr.c | 118 +++++++ tools/testing/selftests/powerpc/dexcr/dexcr.h |  54 +++ .../selftests/powerpc/dexcr/dexcr_test.c      | 241 ++++++++++++++ .../selftests/powerpc/dexcr/hashchk_test.c    | 229 +++++++++++++

 .../testing/selftests/powerpc/dexcr/lsdexcr.c | 178 ++++++++++
 tools/testing/selftests/powerpc/include/reg.h |   4 +
 .../testing/selftests/powerpc/include/utils.h |  44 +++
 29 files changed, 1602 insertions(+), 2 deletions(-)

 create mode 100644 Documentation/powerpc/dexcr.rst create mode 100644 arch/powerpc/kernel/dexcr.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/.gitignore create mode 100644 tools/testing/selftests/powerpc/dexcr/Makefile create mode 100644 tools/testing/selftests/powerpc/dexcr/cap.c create mode 100644 tools/testing/selftests/powerpc/dexcr/cap.h create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr.c create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr.h create mode 100644 tools/testing/selftests/powerpc/dexcr/dexcr_test.c
 create mode 100644 tools/testing/selftests/powerpc/dexcr/hashchk_test.c create mode 100644 tools/testing/selftests/powerpc/dexcr/lsdexcr.c

base-commit: 9dc58a6040662faaf24c8932861f485670fce7ff --
2.38.1