PKS: Add Protection Keys Supervisor (PKS) support RFC v3
Subsystems (13)
v3 cover PKS: Add Protection Keys Supervisor (PKS) support RFC v3
From: Ira Weiny <[email protected]>
This RFC series has been reviewed by Dave Hansen.
Introduce a new page protection mechanism for supervisor pages, Protection Key Supervisor (PKS).
2 use cases for PKS are being developed, trusted keys and PMEM. Trusted keys is a newer use case which is still being explored. PMEM was submitted as part of the RFC (v2) series[1]. However, since then it was found that some callers of kmap() require a global implementation of PKS. Specifically some users of kmap() expect mappings to be available to all kernel threads. While global use of PKS is rare it needs to be included for correctness. Unfortunately the kmap() updates required a large patch series to make the needed changes at the various kmap() call sites so that patch set has been split out. Because the global PKS feature is only required for that use case it will be deferred to that set as well.[2] This patch set is being submitted as a precursor to both of the use cases.
For an overview of the entire PKS ecosystem, a git tree including this series and the 2 use cases can be found here:
https://github.com/weiny2/linux-kernel/tree/pks-rfc-v3
PKS enables protections on 'domains' of supervisor pages to limit supervisor mode access to those pages beyond the normal paging protections. PKS works in a similar fashion to user space pkeys, PKU. As with PKU, supervisor pkeys are checked in addition to normal paging protections and Access or Writes can be disabled via a MSR update without TLB flushes when permissions change. Also like PKU, a page mapping is assigned to a domain by setting pkey bits in the page table entry for that mapping.
Access is controlled through a PKRS register which is updated via WRMSR/RDMSR.
XSAVE is not supported for the PKRS MSR. Therefore the implementation saves/restores the MSR across context switches and during exceptions. Nested exceptions are supported by each exception getting a new PKS state.
For consistent behavior with current paging protections, pkey 0 is reserved and configured to allow full access via the pkey mechanism, thus preserving the default paging protections on mappings with the default pkey value of 0.
Other keys, (1-15) are allocated by an allocator which prepares us for key contention from day one. Kernel users should be prepared for the allocator to fail either because of key exhaustion or due to PKS not being supported on the arch and/or CPU instance.
The following are key attributes of PKS.
1) Fast switching of permissions
1a) Prevents access without page table manipulations 1b) No TLB flushes required
2) Works on a per thread basis
PKS is available with 4 and 5 level paging. Like PKRU it consumes 4 bits from the PTE to store the pkey within the entry.
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://github.com/weiny2/linux-kernel/commit/f10abb0f0d7b4e14f03fc8890313a5830cde1e49
and a testing patch
https://github.com/weiny2/linux-kernel/commit/2a8e0fc7654a7c69b243d628f63b01ff26a5a797Fenghua Yu (3):
x86/fpu: Refactor arch_set_user_pkey_access() for PKS support x86/pks: Enable Protection Keys Supervisor (PKS) x86/pks: Add PKS kernel API
Ira Weiny (6):
x86/pkeys: Create pkeys_common.h x86/pks: Preserve the PKRS MSR on context switch x86/entry: Pass irqentry_state_t by reference x86/entry: Preserve PKRS MSR across exceptions x86/fault: Report the PKRS state on fault x86/pks: Add PKS test code
Documentation/core-api/protection-keys.rst | 102 ++- arch/x86/Kconfig | 1 + arch/x86/entry/common.c | 57 +- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/idtentry.h | 29 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/pgtable.h | 13 +- arch/x86/include/asm/pgtable_types.h | 12 + arch/x86/include/asm/pkeys.h | 15 + arch/x86/include/asm/pkeys_common.h | 36 + arch/x86/include/asm/processor.h | 13 + arch/x86/include/uapi/asm/processor-flags.h | 2 + arch/x86/kernel/cpu/common.c | 17 + arch/x86/kernel/cpu/mce/core.c | 4 + arch/x86/kernel/fpu/xstate.c | 22 +- arch/x86/kernel/kvm.c | 4 +- arch/x86/kernel/nmi.c | 7 +- arch/x86/kernel/process.c | 21 + arch/x86/kernel/traps.c | 21 +- arch/x86/mm/fault.c | 86 ++- arch/x86/mm/pkeys.c | 188 +++++- include/linux/entry-common.h | 19 +- include/linux/pgtable.h | 4 + include/linux/pkeys.h | 23 +- kernel/entry/common.c | 28 +- lib/Kconfig.debug | 12 + lib/Makefile | 3 + lib/pks/Makefile | 3 + lib/pks/pks_test.c | 690 ++++++++++++++++++++ mm/Kconfig | 2 + tools/testing/selftests/x86/Makefile | 3 +- tools/testing/selftests/x86/test_pks.c | 65 ++ 32 files changed, 1376 insertions(+), 128 deletions(-) create mode 100644 arch/x86/include/asm/pkeys_common.h create mode 100644 lib/pks/Makefile create mode 100644 lib/pks/pks_test.c create mode 100644 tools/testing/selftests/x86/test_pks.c
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
RFC PATCH v3 1/9 x86/pkeys: Create pkeys_common.h
From: Ira Weiny <[email protected]>
Protection Keys User (PKU) and Protection Keys Supervisor (PKS) work in similar fashions and can share common defines. Normally, these defines would be put in asm/pkeys.h to be used internally and externally to the arch code. However, the defines are required in pgtable.h and inclusion of pkeys.h in that header creates complex dependencies which are best resolved in a separate header.
Share these defines by moving those them into a new header, change their names to reflect the common use, and include the header where needed.
---
NOTE: The initialization of init_pkru_value cause checkpatch errors because of the space after the '(' in the macros. We leave this as is because it is more readable in this format. And it was existing code.
--- arch/x86/include/asm/pgtable.h | 13 ++++++------- arch/x86/include/asm/pkeys.h | 2 ++ arch/x86/include/asm/pkeys_common.h | 11 +++++++++++ arch/x86/kernel/fpu/xstate.c | 8 ++++---- arch/x86/mm/pkeys.c | 14 ++++++-------- 5 files changed, 29 insertions(+), 19 deletions(-) create mode 100644 arch/x86/include/asm/pkeys_common.h
arch/x86/include/asm/pgtable.h +6 -7
@@ -1361,9 +1361,7 @@ static inline pmd_t pmd_swp_clear_uffd_wp(pmd_t pmd) }#endif /* CONFIG_HAVE_ARCH_USERFAULTFD_WP */ -#define PKRU_AD_BIT 0x1 -#define PKRU_WD_BIT 0x2 -#define PKRU_BITS_PER_PKEY 2 +#include <asm/pkeys_common.h> #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS extern u32 init_pkru_value;
@@ -1373,18 +1371,19 @@ extern u32 init_pkru_value;static inline bool __pkru_allows_read (u32 pkru,u16 pkey) { -int pkru_pkey_bits = pkey *PKRU_BITS_PER_PKEY ; -return !(pkru & (PKRU_AD_BIT << pkru_pkey_bits)); +int pkru_pkey_bits = pkey *PKR_BITS_PER_PKEY ; + +return !(pkru & (PKR_AD_BIT << pkru_pkey_bits)); }static inline bool __pkru_allows_write (u32 pkru,u16 pkey) { -int pkru_pkey_bits = pkey *PKRU_BITS_PER_PKEY ; +int pkru_pkey_bits = pkey *PKR_BITS_PER_PKEY ;/* * Access-disable disables writes too so we need to check * both bits here. */ -return !(pkru & ((PKRU_AD_BIT |PKRU_WD_BIT ) << pkru_pkey_bits)); +return !(pkru & ((PKR_AD_BIT |PKR_WD_BIT ) << pkru_pkey_bits)); }static inline u16 pte_flags_pkey (unsigned long pte_flags)
arch/x86/include/asm/pkeys.h +2 -0
@@ -2,6 +2,8 @@#ifndef _ASM_X86_PKEYS_H#define _ASM_X86_PKEYS_H +#include <asm/pkeys_common.h> +#define ARCH_DEFAULT_PKEY 0/*
arch/x86/include/asm/pkeys_common.h new file +11 -0
@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_PKEYS_INTERNAL_H +#define _ASM_X86_PKEYS_INTERNAL_H + +#define PKR_AD_BIT 0x1 +#define PKR_WD_BIT 0x2 +#define PKR_BITS_PER_PKEY 2 + +#define PKR_AD_KEY (pkey) (PKR_AD_BIT << ((pkey) * PKR_BITS_PER_PKEY)) + +#endif /*_ASM_X86_PKEYS_INTERNAL_H */
arch/x86/kernel/fpu/xstate.c +4 -4
@@ -991,7 +991,7 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, unsigned long init_val) {u32 old_pkru; -int pkey_shift = (pkey *PKRU_BITS_PER_PKEY ); +int pkey_shift = (pkey *PKR_BITS_PER_PKEY );u32 new_pkru_bits =0 ;/*
@@ -1010,16 +1010,16 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,/* Set the bits we need in PKRU: */ if (init_val &PKEY_DISABLE_ACCESS ) - new_pkru_bits |=PKRU_AD_BIT ; + new_pkru_bits |=PKR_AD_BIT ;if (init_val &PKEY_DISABLE_WRITE ) - new_pkru_bits |=PKRU_WD_BIT ; + new_pkru_bits |=PKR_WD_BIT ;/* Shift the bits in to the correct place in PKRU for pkey: */ new_pkru_bits <<= pkey_shift;/* Get old PKRU and mask off any old bits in place: */ old_pkru =read_pkru (); - old_pkru &= ~((PKRU_AD_BIT |PKRU_WD_BIT ) << pkey_shift); + old_pkru &= ~((PKR_AD_BIT |PKR_WD_BIT ) << pkey_shift);/* Write old part along with new part: */ write_pkru (old_pkru | new_pkru_bits);
arch/x86/mm/pkeys.c +6 -8
@@ -111,19 +111,17 @@ int __arch_override_mprotect_pkey(struct vm_area_struct *vma, int prot, int pkeyreturn vma_pkey (vma); } -#define PKRU_AD_KEY (pkey) (PKRU_AD_BIT << ((pkey) * PKRU_BITS_PER_PKEY)) -/* * Make the default PKRU value (at execve() time) as restrictive * as possible. This ensures that any threads clone()'d early * in the process's lifetime will not accidentally get access * to data which is pkey-protected later on. */ -u32 init_pkru_value =PKRU_AD_KEY (1 ) |PKRU_AD_KEY (2 ) |PKRU_AD_KEY (3 ) | -PKRU_AD_KEY (4 ) |PKRU_AD_KEY (5 ) |PKRU_AD_KEY (6 ) | -PKRU_AD_KEY (7 ) |PKRU_AD_KEY (8 ) |PKRU_AD_KEY (9 ) | -PKRU_AD_KEY (10 ) |PKRU_AD_KEY (11 ) |PKRU_AD_KEY (12 ) | -PKRU_AD_KEY (13 ) |PKRU_AD_KEY (14 ) |PKRU_AD_KEY (15 ); +u32 init_pkru_value =PKR_AD_KEY (1 ) |PKR_AD_KEY (2 ) |PKR_AD_KEY (3 ) | +PKR_AD_KEY (4 ) |PKR_AD_KEY (5 ) |PKR_AD_KEY (6 ) | +PKR_AD_KEY (7 ) |PKR_AD_KEY (8 ) |PKR_AD_KEY (9 ) | +PKR_AD_KEY (10 ) |PKR_AD_KEY (11 ) |PKR_AD_KEY (12 ) | +PKR_AD_KEY (13 ) |PKR_AD_KEY (14 ) |PKR_AD_KEY (15 );/* * Called from the FPU code when creating a fresh set of FPU
@@ -173,7 +171,7 @@ static ssize_t init_pkru_write_file(struct file *file,* up immediately if someone attempts to disable access * or writes to pkey 0. */ -if (new_init_pkru & (PKRU_AD_BIT |PKRU_WD_BIT )) +if (new_init_pkru & (PKR_AD_BIT |PKR_WD_BIT ))return -EINVAL ;WRITE_ONCE (init_pkru_value, new_init_pkru);
Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
Re: [PATCH RFC V3 1/9] x86/pkeys: Create pkeys_common.h
Quoted 2 lines · On 10/9/20 12:42 PM, [email protected] wrote:
Protection Keys User (PKU) and Protection Keys Supervisor (PKS) work in similar fashions and can share common defines.
Could we be a bit less abstract? PKS and PKU each have:
1. A single control register
2. The same number of keys
3. The same number of bits in the register per key
4. Access and Write disable in the same bit locations
That means that we can share all the macros that synthesize and manipulate register values between the two features.
arch/x86/include/asm/pkeys_common.h line 9
Quoted 11 lines
+++
b/arch/x86/include/asm/pkeys_common.harch/x86/include/asm/pkeys_common.h new file +9
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_PKEYS_INTERNAL_H +#define _ASM_X86_PKEYS_INTERNAL_H + +#define PKR_AD_BIT 0x1 +#define PKR_WD_BIT 0x2 +#define PKR_BITS_PER_PKEY 2 + +#define PKR_AD_KEY (pkey) (PKR_AD_BIT << ((pkey) * PKR_BITS_PER_PKEY))
Now that this has moved away from its use-site, it's a bit less self-documenting. Let's add a comment:
/*
* Generate an Access-Disable mask for the given pkey. Several of these
* can be OR'd together to generate pkey register values.
*/
Once that's in place, along with the updated changelog:
Re: [PATCH RFC V3 1/9] x86/pkeys: Create pkeys_common.h
Quoted 4 lines · Dave Hansen
Quoted 2 lines · On 10/9/20 12:42 PM, [email protected] wrote:
Protection Keys User (PKU) and Protection Keys Supervisor (PKS) work in similar fashions and can share common defines.
Quoted 8 lines
Could we be a bit less abstract? PKS and PKU each have:
1. A single control register
2. The same number of keys
3. The same number of bits in the register per key
4. Access and Write disable in the same bit locationsThat means that we can share all the macros that synthesize and manipulate register values between the two features.
Sure. Done.
Quoted 16 lines
arch/x86/include/asm/pkeys_common.h line 9
Quoted 12 lines
+++
b/arch/x86/include/asm/pkeys_common.harch/x86/include/asm/pkeys_common.h new file +9
-0@@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_PKEYS_INTERNAL_H +#define _ASM_X86_PKEYS_INTERNAL_H + +#define PKR_AD_BIT 0x1 +#define PKR_WD_BIT 0x2 +#define PKR_BITS_PER_PKEY 2 + +#define PKR_AD_KEY (pkey) (PKR_AD_BIT << ((pkey) * PKR_BITS_PER_PKEY))Now that this has moved away from its use-site, it's a bit less self-documenting. Let's add a comment:
Quoted 4 lines
/*
* Generate an Access-Disable mask for the given pkey. Several of these
* can be OR'd together to generate pkey register values.
*/
Fair enough. done.
Quoted 4 lines
Once that's in place, along with the updated changelog:
Reviewed-by Dave Hansen <[email protected]>
Thanks,
Ira
RFC PATCH v3 2/9 x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
In reply to [email protected]From: Fenghua Yu <[email protected]>
Define a helper, update_pkey_val(), which will be used to support both Protection Key User (PKU) and the new Protection Key for Supervisor (PKS) in subsequent patches.
--- arch/x86/include/asm/pkeys.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 22 ++++------------------ arch/x86/mm/pkeys.c | 21 +++++++++++++++++++++ 3 files changed, 27 insertions(+), 18 deletions(-)
arch/x86/include/asm/pkeys.h +2 -0
@@ -136,4 +136,6 @@ static inline int vma_pkey(struct vm_area_struct *vma) return (vma->vm_flags & vma_pkey_mask) >> VM_PKEY_SHIFT; } +u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags); +#endif /*_ASM_X86_PKEYS_H */
arch/x86/kernel/fpu/xstate.c +4 -18
@@ -990,9 +990,7 @@ const void *get_xsave_field_ptr(int xfeature_nr)int arch_set_user_pkey_access (struct task_struct *tsk,int pkey,unsigned long init_val) { -u32 old_pkru; -int pkey_shift = (pkey *PKR_BITS_PER_PKEY ); -u32 new_pkru_bits =0 ; +u32 pkru;/* * This check implies XSAVE support. OSPKE only gets
@@ -1008,21 +1006,9 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,*/ WARN_ON_ONCE (pkey >=arch_max_pkey ()); -/* Set the bits we need in PKRU: */ -if (init_val &PKEY_DISABLE_ACCESS ) - new_pkru_bits |=PKR_AD_BIT ; -if (init_val &PKEY_DISABLE_WRITE ) - new_pkru_bits |=PKR_WD_BIT ; - -/* Shift the bits in to the correct place in PKRU for pkey: */ - new_pkru_bits <<= pkey_shift; - -/* Get old PKRU and mask off any old bits in place: */ - old_pkru =read_pkru (); - old_pkru &= ~((PKR_AD_BIT |PKR_WD_BIT ) << pkey_shift); - -/* Write old part along with new part: */ -write_pkru (old_pkru | new_pkru_bits); + pkru =read_pkru (); + pkru =update_pkey_val (pkru, pkey, init_val); +write_pkru (pkru);return 0 ; }
arch/x86/mm/pkeys.c +21 -0
@@ -208,3 +208,24 @@ static __init int setup_init_pkru(char *opt)return 1 ; }__setup ("init_pkru=" , setup_init_pkru); + +/* +* Update the pk_reg value and return it. +* +* Kernel users use the same flags as user space: +* PKEY_DISABLE_ACCESS +* PKEY_DISABLE_WRITE +*/ +u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags) +{ +int pkey_shift = pkey *PKR_BITS_PER_PKEY ; + + pk_reg &= ~(((1 <<PKR_BITS_PER_PKEY ) -1 ) << pkey_shift); + +if (flags &PKEY_DISABLE_ACCESS ) + pk_reg |=PKR_AD_BIT << pkey_shift; +if (flags &PKEY_DISABLE_WRITE ) + pk_reg |=PKR_WD_BIT << pkey_shift; + +return pk_reg; +}
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 2/9] x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
Quoted 2 lines · On 10/9/20 12:42 PM, [email protected] wrote:
+/*
+ * Update thepk_regvalue and return it.
How about:
Replace disable bits for @pkey with values from @flags.
Quoted 4 lines
+ * Kernel users use the same flags as user space:
+ *PKEY_DISABLE_ACCESS
+ *PKEY_DISABLE_WRITE
+ */
arch/x86/mm/pkeys.c static __init int setup_init_pkru(char *opt) · line 228
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags) +{ +int pkey_shift = pkey *PKR_BITS_PER_PKEY ; + + pk_reg &= ~(((1 <<PKR_BITS_PER_PKEY ) -1 ) << pkey_shift); + +if (flags &PKEY_DISABLE_ACCESS ) + pk_reg |=PKR_AD_BIT << pkey_shift; +if (flags &PKEY_DISABLE_WRITE ) + pk_reg |=PKR_WD_BIT << pkey_shift;
I still think this deserves two lines of comments:
/* Mask out old bit values */ /* Or in new values */
Re: [PATCH RFC V3 2/9] x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
Quoted 7 lines · Dave Hansen
Quoted 2 lines · On 10/9/20 12:42 PM, [email protected] wrote:
+/*
+ * Update thepk_regvalue and return it.How about:
Replace disable bits for @pkey with values from @flags.
Done.
Quoted 18 lines
Quoted 5 lines
+ * Kernel users use the same flags as user space:
+ *PKEY_DISABLE_ACCESS
+ *PKEY_DISABLE_WRITE
+ */arch/x86/mm/pkeys.c static __init int setup_init_pkru(char *opt) · line 228
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags) +{ +int pkey_shift = pkey *PKR_BITS_PER_PKEY ; + + pk_reg &= ~(((1 <<PKR_BITS_PER_PKEY ) -1 ) << pkey_shift); + +if (flags &PKEY_DISABLE_ACCESS ) + pk_reg |=PKR_AD_BIT << pkey_shift; +if (flags &PKEY_DISABLE_WRITE ) + pk_reg |=PKR_WD_BIT << pkey_shift;I still think this deserves two lines of comments:
Quoted 3 lines
/* Mask out old bit values */ /* Or in new values */
Sure, done.
Ira
Peter Zijlstra
maintainerRe: [PATCH RFC V3 2/9] x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
In reply to [email protected]Quoted 6 lines · On Fri, Oct 09, 2020 at 12:42:51PM -0700, [email protected] wrote:
From: Fenghua Yu <[email protected]>
Define a helper,
update_pkey_val(), which will be used to support both Protection Key User (PKU) and the new Protection Key for Supervisor (PKS) in subsequent patches.
Quoted 8 lines
Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Fenghua Yu <[email protected]>--- arch/x86/include/asm/pkeys.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 22 ++++------------------ arch/x86/mm/pkeys.c | 21 +++++++++++++++++++++ 3 files changed, 27 insertions(+), 18 deletions(-)
This is not from Fenghua.
https://lkml.kernel.org/r/[email protected]
This is your patch based on the code I wrote.
Quoted 32 lines
arch/x86/mm/pkeys.c +21
-0@@ -208,3 +208,24 @@ static __init int setup_init_pkru(char *opt)return 1 ; }__setup ("init_pkru=" , setup_init_pkru); + +/* +* Update the pk_reg value and return it. +* +* Kernel users use the same flags as user space: +* PKEY_DISABLE_ACCESS +* PKEY_DISABLE_WRITE +*/ +u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags) +{ +int pkey_shift = pkey *PKR_BITS_PER_PKEY ; + + pk_reg &= ~(((1 <<PKR_BITS_PER_PKEY ) -1 ) << pkey_shift); + +if (flags &PKEY_DISABLE_ACCESS ) + pk_reg |=PKR_AD_BIT << pkey_shift; +if (flags &PKEY_DISABLE_WRITE ) + pk_reg |=PKR_WD_BIT << pkey_shift; + +return pk_reg; +}Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
Re: [PATCH RFC V3 2/9] x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
Quoted 16 lines · Peter Zijlstra
Quoted 6 lines · On Fri, Oct 09, 2020 at 12:42:51PM -0700, [email protected] wrote:
From: Fenghua Yu <[email protected]>
Define a helper,
update_pkey_val(), which will be used to support both Protection Key User (PKU) and the new Protection Key for Supervisor (PKS) in subsequent patches.Quoted 8 lines
Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Fenghua Yu <[email protected]>--- arch/x86/include/asm/pkeys.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 22 ++++------------------ arch/x86/mm/pkeys.c | 21 +++++++++++++++++++++ 3 files changed, 27 insertions(+), 18 deletions(-)
Quoted 5 lines
This is not from Fenghua.
https://lkml.kernel.org/r/[email protected]
This is your patch based on the code I wrote.
Ok, I apologize. Yes the code below was all yours.
Is it ok to add?
?
Thanks,
Ira
Quoted 25 lines
Quoted 25 lines
arch/x86/mm/pkeys.c +16
-0@@ -208,3 +208,24 @@ static __init int setup_init_pkru(char *opt)return 1 ; }__setup ("init_pkru=" , setup_init_pkru); + +/* +* Update the pk_reg value and return it. +* +* Kernel users use the same flags as user space: +* PKEY_DISABLE_ACCESS +* PKEY_DISABLE_WRITE +*/ +u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags) +{ +int pkey_shift = pkey *PKR_BITS_PER_PKEY ; + + pk_reg &= ~(((1 <<PKR_BITS_PER_PKEY ) -1 ) << pkey_shift); + +if (flags &PKEY_DISABLE_ACCESS ) + pk_reg |=PKR_AD_BIT << pkey_shift;
Quoted 8 lines
Quoted 8 lines
arch/x86/mm/pkeys.c +5
-0+if (flags &PKEY_DISABLE_WRITE ) + pk_reg |=PKR_WD_BIT << pkey_shift; + +return pk_reg; +}Signature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Peter Zijlstra
maintainerRe: [PATCH RFC V3 2/9] x86/fpu: Refactor arch_set_user_pkey_access() for PKS support
Quoted 21 lines · Ira Weiny
Quoted 20 lines · Peter Zijlstra
Quoted 6 lines · On Fri, Oct 09, 2020 at 12:42:51PM -0700, [email protected] wrote:
From: Fenghua Yu <[email protected]>
Define a helper,
update_pkey_val(), which will be used to support both Protection Key User (PKU) and the new Protection Key for Supervisor (PKS) in subsequent patches.Quoted 8 lines
Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Fenghua Yu <[email protected]>--- arch/x86/include/asm/pkeys.h | 2 ++ arch/x86/kernel/fpu/xstate.c | 22 ++++------------------ arch/x86/mm/pkeys.c | 21 +++++++++++++++++++++ 3 files changed, 27 insertions(+), 18 deletions(-)This is not from Fenghua.
Quoted 9 lines
Quoted 1 line
This is your patch based on the code I wrote.
Ok, I apologize. Yes the code below was all yours.
Is it ok to add?
Co-developed-by Peter Zijlstra <[email protected]>Signed-off-by Peter Zijlstra <[email protected]>
Sure, thanks!
RFC PATCH v3 3/9 x86/pks: Enable Protection Keys Supervisor (PKS)
In reply to [email protected]From: Fenghua Yu <[email protected]>
Protection Keys for Supervisor pages (PKS) enables fast, hardware thread specific, manipulation of permission restrictions on supervisor page mappings. It uses the same mechanism of Protection Keys as those on User mappings but applies that mechanism to supervisor mappings using a supervisor specific MSR.
Kernel users can thus defines 'domains' of page mappings which have an extra level of protection beyond those specified in the supervisor page table entries.
Define ARCH_HAS_SUPERVISOR_PKEYS to distinguish this functionality from the existing ARCH_HAS_PKEYS and then enable PKS when configured and indicated by the CPU instance. While not strictly necessary in this patch, ARCH_HAS_SUPERVISOR_PKEYS separates this functionality through the patch series so it is introduced here.
--- arch/x86/Kconfig | 1 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/uapi/asm/processor-flags.h | 2 ++ arch/x86/kernel/cpu/common.c | 15 +++++++++++++++ mm/Kconfig | 2 ++ 5 files changed, 21 insertions(+)
arch/x86/Kconfig +1 -0
@@ -1873,6 +1873,7 @@ config X86_INTEL_MEMORY_PROTECTION_KEYS depends on X86_64 && (CPU_SUP_INTEL || CPU_SUP_AMD) select ARCH_USES_HIGH_VMA_FLAGS select ARCH_HAS_PKEYS + select ARCH_HAS_SUPERVISOR_PKEYS help Memory Protection Keys provides a mechanism for enforcing page-based protections, but without requiring modification of the
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -353,6 +353,7 @@#define X86_FEATURE_CLDEMOTE (16*32+25)/* CLDEMOTE instruction */ #define X86_FEATURE_MOVDIRI (16*32+27)/* MOVDIRI instruction */ #define X86_FEATURE_MOVDIR64B (16*32+28)/* MOVDIR64B instruction */ +#define X86_FEATURE_PKS (16*32+31)/* Protection Keys for Supervisor pages */ /* AMD-defined CPU features, CPUID level 0x80000007 (EBX), word 17 */ #define X86_FEATURE_OVERFLOW_RECOV (17*32+ 0)/* MCA overflow recovery support */
arch/x86/include/uapi/asm/processor-flags.h +2 -0
@@ -130,6 +130,8 @@#define X86_CR4_SMAP _BITUL(X86_CR4_SMAP_BIT)#define X86_CR4_PKE_BIT 22/* enable Protection Keys support */ #define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT) +#define X86_CR4_PKS_BIT 24/* enable Protection Keys for Supervisor */ +#define X86_CR4_PKS _BITUL(X86_CR4_PKS_BIT)/* * x86-64 Task Priority Register, CR8
arch/x86/kernel/cpu/common.c +15 -0
@@ -1447,6 +1447,20 @@ static void validate_apic_and_package_id(struct cpuinfo_x86 *c)#endif } +/* +* PKS is independent of PKU and either or both may be supported on a CPU. +* Configure PKS if the cpu supports the feature. +*/ +static void setup_pks (void ) +{ +if (!IS_ENABLED (CONFIG_ARCH_HAS_SUPERVISOR_PKEYS )) +return ; +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ; + +cr4_set_bits (X86_CR4_PKS ); +} +/* * This does the hard work of actually picking apart the CPU stuff... */
@@ -1544,6 +1558,7 @@ static void identify_cpu(struct cpuinfo_x86 *c)x86_init_rdrand (c);setup_pku (c); +setup_pks ();/* * Clear/Set all flags overridden by options, need do it
mm/Kconfig +2 -0
@@ -822,6 +822,8 @@ config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS bool +config ARCH_HAS_SUPERVISOR_PKEYS + bool config PERCPU_STATS bool "Collect percpu memory statistics"
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 3/9] x86/pks: Enable Protection Keys Supervisor (PKS)
Quoted 4 lines · On 10/9/20 12:42 PM, [email protected] wrote:
+/*
+ * PKS is independent of PKU and either or both may be supported on a CPU.
+ * Configure PKS if the cpu supports the feature.
+ */
Let's at least be consistent about CPU vs. cpu in a single comment. :)
arch/x86/kernel/cpu/common.c static void validate_apic_and_package_id(struct cpuinfo_x86 *c) · line 1459
Quoted 6 lines
arch/x86/kernel/cpu/common.c +6
-0+static void setup_pks (void ) +{ +if (!IS_ENABLED (CONFIG_ARCH_HAS_SUPERVISOR_PKEYS )) +return ; +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ;
If you put X86_FEATURE_PKS in disabled-features.h, you can get rid of the explicit CONFIG_ check.
Quoted 28 lines
arch/x86/kernel/cpu/common.c +4
-0+cr4_set_bits (X86_CR4_PKS ); +} +/* * This does the hard work of actually picking apart the CPU stuff... */ @@ -1544,6 +1558,7 @@ static void identify_cpu(struct cpuinfo_x86 *c)x86_init_rdrand (c);setup_pku (c); +setup_pks ();/* * Clear/Set all flags overridden by options, need do it mm/Kconfig +2
-0@@ -822,6 +822,8 @@ config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS bool +config ARCH_HAS_SUPERVISOR_PKEYS + bool config PERCPU_STATS bool "Collect percpu memory statistics"
Re: [PATCH RFC V3 3/9] x86/pks: Enable Protection Keys Supervisor (PKS)
Quoted 7 lines · Dave Hansen
Quoted 4 lines · On 10/9/20 12:42 PM, [email protected] wrote:
+/*
+ * PKS is independent of PKU and either or both may be supported on a CPU.
+ * Configure PKS if the cpu supports the feature.
+ */Let's at least be consistent about CPU vs. cpu in a single comment. :)
Sorry, done.
Quoted 10 lines
arch/x86/kernel/cpu/common.c static void validate_apic_and_package_id(struct cpuinfo_x86 *c) · line 1459
Quoted 7 lines
arch/x86/kernel/cpu/common.c +6
-0+static void setup_pks (void ) +{ +if (!IS_ENABLED (CONFIG_ARCH_HAS_SUPERVISOR_PKEYS )) +return ; +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ;If you put
X86_FEATURE_PKSindisabled-features.h, you can get rid of the explicitCONFIG_check.
Done.
Quoted 30 lines
Quoted 29 lines
arch/x86/kernel/cpu/common.c +4
-0+cr4_set_bits (X86_CR4_PKS ); +} +/* * This does the hard work of actually picking apart the CPU stuff... */ @@ -1544,6 +1558,7 @@ static void identify_cpu(struct cpuinfo_x86 *c)x86_init_rdrand (c);setup_pku (c); +setup_pks ();/* * Clear/Set all flags overridden by options, need do it mm/Kconfig +2
-0@@ -822,6 +822,8 @@ config ARCH_USES_HIGH_VMA_FLAGS bool config ARCH_HAS_PKEYS bool +config ARCH_HAS_SUPERVISOR_PKEYS + bool config PERCPU_STATS bool "Collect percpu memory statistics"
RFC PATCH v3 4/9 x86/pks: Preserve the PKRS MSR on context switch
In reply to [email protected]From: Ira Weiny <[email protected]>
The PKRS MSR is defined as a per-logical-processor register. This isolates memory access by logical CPU. Unfortunately, the MSR is not managed by XSAVE. Therefore, tasks must save/restore the MSR value on context switch.
Define a saved PKRS value in the task struct, as well as a cached per-logical-processor MSR value which mirrors the MSR value of the current CPU. Initialize all tasks with the default MSR value. Then, on schedule in, check the saved task MSR vs the per-cpu value. If different proceed to write the MSR. If not avoid the overhead of the MSR write and continue.
Follow on patches will update the saved PKRS as well as the MSR if needed.
--- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/pkeys_common.h | 20 ++++++++++++++++++++ arch/x86/include/asm/processor.h | 13 +++++++++++++ arch/x86/kernel/cpu/common.c | 2 ++ arch/x86/kernel/process.c | 21 +++++++++++++++++++++ arch/x86/mm/pkeys.c | 28 ++++++++++++++++++++++++++++ 6 files changed, 85 insertions(+)
arch/x86/include/asm/msr-index.h +1 -0
@@ -747,6 +747,7 @@#define MSR_IA32_TSC_DEADLINE 0x000006E0 +#define MSR_IA32_PKRS 0x000006E1#define MSR_TSX_FORCE_ABORT 0x0000010F
arch/x86/include/asm/pkeys_common.h +20 -0
@@ -8,4 +8,24 @@#define PKR_AD_KEY (pkey) (PKR_AD_BIT << ((pkey) * PKR_BITS_PER_PKEY)) +/* +* Define a default PKRS value for each task. +* +* Key 0 has no restriction. All other keys are set to the most restrictive +* value which is access disabled (AD=1). +* +* NOTE: This needs to be a macro to be used as part of the INIT_THREAD macro. +*/ +#define INIT_PKRS_VALUE (PKR_AD_KEY(1) | PKR_AD_KEY(2) | PKR_AD_KEY(3) | \ + PKR_AD_KEY(4) | PKR_AD_KEY(5) | PKR_AD_KEY(6) | \ + PKR_AD_KEY(7) | PKR_AD_KEY(8) | PKR_AD_KEY(9) | \ + PKR_AD_KEY(10) | PKR_AD_KEY(11) | PKR_AD_KEY(12) | \ + PKR_AD_KEY(13) | PKR_AD_KEY(14) | PKR_AD_KEY(15)) + +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +void write_pkrs (u32 new_pkrs); +#else +static inline void write_pkrs (u32 new_pkrs) { } +#endif +#endif /*_ASM_X86_PKEYS_INTERNAL_H */
arch/x86/include/asm/processor.h +13 -0
@@ -18,6 +18,7 @@ struct vm86;#include <asm/cpufeatures.h> #include <asm/page.h> #include <asm/pgtable_types.h> +#include <asm/pkeys_common.h> #include <asm/percpu.h> #include <asm/msr.h> #include <asm/desc_defs.h>
@@ -542,6 +543,11 @@ struct thread_struct { unsigned int sig_on_uaccess_err:1; +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +/* Saved Protection key register for supervisor mappings */ +u32 saved_pkrs; +#endif +/* Floating point and extended processor state */ struct fpu fpu;/*
@@ -840,8 +846,15 @@ static inline void spin_lock_prefetch(const void *x)#define STACK_TOP TASK_SIZE_LOW#define STACK_TOP_MAX TASK_SIZE_MAX +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +#define INIT_THREAD_PKRS .saved_pkrs = INIT_PKRS_VALUE +#else +#define INIT_THREAD_PKRS 0 +#endif +#define INIT_THREAD { \ .addr_limit = KERNEL_DS, \ + INIT_THREAD_PKRS, \ }extern unsigned long KSTK_ESP (struct task_struct *task);
arch/x86/kernel/cpu/common.c +2 -0
@@ -57,6 +57,7 @@#include <asm/intel-family.h> #include <asm/cpu_device_id.h> #include <asm/uv/uv.h> +#include <linux/pkeys.h> #include "cpu.h"
@@ -1458,6 +1459,7 @@ static void setup_pks(void)if (!cpu_feature_enabled (X86_FEATURE_PKS ))return ; +write_pkrs (INIT_PKRS_VALUE );cr4_set_bits (X86_CR4_PKS ); }
arch/x86/kernel/process.c +21 -0
@@ -43,6 +43,7 @@#include <asm/io_bitmap.h> #include <asm/proto.h> #include <asm/frame.h> +#include <asm/pkeys_common.h> #include "process.h"
@@ -187,6 +188,22 @@ int copy_thread(unsigned long clone_flags, unsigned long sp, unsigned long arg,return ret; } +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +DECLARE_PER_CPU (u32, pkrs_cache); +static inline void pks_init_task (struct task_struct *tsk) +{ +/* New tasks get the most restrictive PKRS value */ + tsk->thread.saved_pkrs =INIT_PKRS_VALUE ; +} +static inline void pks_sched_in (void ) +{ +write_pkrs (current->thread.saved_pkrs); +} +#else +static inline void pks_init_task (struct task_struct *tsk) { } +static inline void pks_sched_in (void ) { } +#endif +void flush_thread (void ) {struct task_struct *tsk = current;
@@ -195,6 +212,8 @@ void flush_thread(void)memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));fpu__clear_all (&tsk->thread.fpu); + +pks_init_task (tsk); }void disable_TSC (void )
@@ -644,6 +663,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p)if ((tifp ^ tifn) & _TIF_SLD)switch_to_sld (tifn); + +pks_sched_in (); }/*
arch/x86/mm/pkeys.c +28 -0
@@ -229,3 +229,31 @@ u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags)return pk_reg; } + +DEFINE_PER_CPU (u32, pkrs_cache); + +/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as +* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register. +*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; + +if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache); +if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs); + } +put_cpu_ptr (pkrs); +}
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
Quoted 7 lines · On 10/9/20 12:42 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
The PKRS MSR is defined as a per-logical-processor register. This isolates memory access by logical CPU. Unfortunately, the MSR is not managed by XSAVE. Therefore, tasks must save/restore the MSR value on context switch.
Quoted 6 lines
Define a saved PKRS value in the task struct, as well as a cached per-logical-processor MSR value which mirrors the MSR value of the current CPU. Initialize all tasks with the default MSR value. Then, on schedule in, check the saved task MSR vs the per-cpu value. If different proceed to write the MSR. If not avoid the overhead of the MSR write and continue.
It's probably nice to note how the WRMSR is special here, in addition to the comments below.
Quoted 14 lines
#endif /*_ASM_X86_PKEYS_INTERNAL_H */
arch/x86/include/asm/processor.h +1
-0@@ -18,6 +18,7 @@ struct vm86;#include <asm/cpufeatures.h> #include <asm/page.h> #include <asm/pgtable_types.h> +#include <asm/pkeys_common.h> #include <asm/percpu.h> #include <asm/msr.h> #include <asm/desc_defs.h>
arch/x86/include/asm/processor.h struct thread_struct { · line 549
Quoted 7 lines
arch/x86/include/asm/processor.h +4
-0@@ -542,6 +543,11 @@ struct thread_struct { unsigned int sig_on_uaccess_err:1; +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +/* Saved Protection key register for supervisor mappings */ +u32 saved_pkrs; +#endif
Could you take a look around thread_struct and see if there are some other MSRs near which you can stash this? This seems like a bit of a lonely place.
...
Quoted 48 lines
arch/x86/include/asm/processor.h +4
-0void flush_thread (void ) {struct task_struct *tsk = current;@@ -195,6 +212,8 @@ void flush_thread(void)memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));fpu__clear_all (&tsk->thread.fpu); + +pks_init_task (tsk); }void disable_TSC (void )@@ -644,6 +663,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p)if ((tifp ^ tifn) & _TIF_SLD)switch_to_sld (tifn); + +pks_sched_in (); }/* arch/x86/mm/pkeys.c +18
-0@@ -229,3 +229,31 @@ u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags)return pk_reg; } + +DEFINE_PER_CPU (u32, pkrs_cache); + +/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as +* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register. +*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; +
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 259
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache); +if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs); + } +put_cpu_ptr (pkrs); +}
It bugs me a *bit* that this is being called in a preempt-disabled region, but we still bother with the get/put_cpu jazz. Are there other future call-sites for this that aren't in preempt-disabled regions?
Re: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
Quoted 15 lines · Dave Hansen
Quoted 7 lines · On 10/9/20 12:42 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
The PKRS MSR is defined as a per-logical-processor register. This isolates memory access by logical CPU. Unfortunately, the MSR is not managed by XSAVE. Therefore, tasks must save/restore the MSR value on context switch.
Quoted 6 lines
Define a saved PKRS value in the task struct, as well as a cached per-logical-processor MSR value which mirrors the MSR value of the current CPU. Initialize all tasks with the default MSR value. Then, on schedule in, check the saved task MSR vs the per-cpu value. If different proceed to write the MSR. If not avoid the overhead of the MSR write and continue.
Quoted 2 lines
It's probably nice to note how the WRMSR is special here, in addition to the comments below.
Sure,
Quoted 18 lines
Quoted 18 lines
#endif /*_ASM_X86_PKEYS_INTERNAL_H */
arch/x86/include/asm/processor.h +1
-0@@ -18,6 +18,7 @@ struct vm86;#include <asm/cpufeatures.h> #include <asm/page.h> #include <asm/pgtable_types.h> +#include <asm/pkeys_common.h> #include <asm/percpu.h> #include <asm/msr.h> #include <asm/desc_defs.h> @@ -542,6 +543,11 @@ struct thread_struct { unsigned int sig_on_uaccess_err:1;
Quoted 8 lines
arch/x86/include/asm/processor.h struct thread_struct { · line 549
Quoted 4 lines
arch/x86/include/asm/processor.h +4
-0+#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +/* Saved Protection key register for supervisor mappings */ +u32 saved_pkrs; +#endif Could you take a look around
thread_structand see if there are some other MSRs near which you can stash this? This seems like a bit of a lonely place.
Are you more concerned with aesthetics or the in memory struct layout?
How about I put it after error_code?
unsigned long error_code;
+
+#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS
+ /* Saved Protection key register for supervisor mappings */
+ u32 saved_pkrs;
+#endif
+
?
Quoted 56 lines
...
Quoted 54 lines
void
flush_thread(void)
{arch/x86/include/asm/processor.h +4
-0struct task_struct *tsk = current;@@ -195,6 +212,8 @@ void flush_thread(void)memset (tsk->thread.tls_array,0 ,sizeof (tsk->thread.tls_array));fpu__clear_all (&tsk->thread.fpu); + +pks_init_task (tsk); }void disable_TSC (void )@@ -644,6 +663,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p)if ((tifp ^ tifn) & _TIF_SLD)switch_to_sld (tifn); + +pks_sched_in (); }/* arch/x86/mm/pkeys.c +25
-0@@ -229,3 +229,31 @@ u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags)return pk_reg; } + +DEFINE_PER_CPU (u32, pkrs_cache); + +/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as +* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register. +*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; + +if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache); +if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs);
Quoted 8 lines
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 259
Quoted 4 lines
arch/x86/mm/pkeys.c +3
-0+ } +put_cpu_ptr (pkrs); +}It bugs me a *bit* that this is being called in a preempt-disabled region, but we still bother with the get/put_cpu jazz. Are there other future call-sites for this that aren't in preempt-disabled regions?
I'm not specifically disabling preempt before calling write_pkrs except in the next patch (which is buggy because I meant to have it around the modification of thread.saved_pkrs as well). But that was to protect the thread variable not the percpu cache vs MSR.
My thought above was it is safer for this call to ensure the per-cpu variable is consistent with the register. The other calls to write_pkrs() may require preemption disable but for reasons unrelated to write_pkrs' state.
After some research I've now fully confused myself if this is needed in patch 7/9 where write_pkrs() is called from the exception handing code. But I think it is needed there. Isn't it?
Since preempt_disable() is nestable I think this is ok correct?
Ira
Peter Zijlstra
maintainerRe: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
In reply to Dave HansenQuoted 22 lines · Dave Hansen
Quoted 22 lines
arch/x86/mm/pkeys.c +22
-0+/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as +* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register. +*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; + +if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache); +if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs);
Quoted 8 lines
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 259
Quoted 4 lines
arch/x86/mm/pkeys.c +3
-0+ } +put_cpu_ptr (pkrs); +}It bugs me a *bit* that this is being called in a preempt-disabled region, but we still bother with the get/put_cpu jazz. Are there other future call-sites for this that aren't in preempt-disabled regions?
So the previous version had a useful comment that got lost. This stuff needs to fundamentally be preempt disabled, so it either needs to explicitly do so, or have an assertion that preemption is indeed disabled.
Re: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
Quoted 32 lines · Peter Zijlstra
Quoted 22 lines · Dave Hansen
Quoted 22 lines
arch/x86/mm/pkeys.c +22
-0+/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as +* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register. +*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; + +if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache); +if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs);Quoted 8 lines
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 259
Quoted 4 lines
arch/x86/mm/pkeys.c +3
-0+ } +put_cpu_ptr (pkrs); +}It bugs me a *bit* that this is being called in a preempt-disabled region, but we still bother with the get/put_cpu jazz. Are there other future call-sites for this that aren't in preempt-disabled regions?
Quoted 1 line
So the previous version had a useful comment that got lost.
Ok Looking back I see what happened... This comment...
/*
* PKRS is only temporarily changed during specific code paths.
* Only a preemption during these windows away from the default
* value would require updating the MSR.
*/
... was added to pks_sched_in() but that got simplified down because cleaning up write_pkrs() made the code there obsolete.
Quoted 2 lines
This stuff
needs to fundamentally be preempt disabled,
I agree, the update to the percpu cache value and MSR can not be torn.
Quoted 3 lines
so it either needs to
explicitly do so, or have an assertion that preemption is indeed disabled.
However, I don't think I understand clearly. Doesn't [get|put]_cpu_ptr() handle the preempt_disable() for us? Is it not sufficient to rely on that?
Dave's comment seems to be the opposite where we need to eliminate preempt disable before calling write_pkrs().
FWIW I think I'm mistaken in my response to Dave regarding the preempt_disable() in pks_update_protection().
Ira
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Peter Zijlstra
maintainerRe: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
Quoted 6 lines · Ira Weiny
Quoted 3 lines
so it either needs to
explicitly do so, or have an assertion that preemption is indeed disabled.However, I don't think I understand clearly. Doesn't [get|put]_cpu_ptr() handle the
preempt_disable()for us?
It does.
Quoted 1 line
Is it not sufficient to rely on that?
It is.
Quoted 5 lines
Dave's comment seems to be the opposite where we need to eliminate preempt disable before calling
write_pkrs().FWIW I think I'm mistaken in my response to Dave regarding the
preempt_disable()inpks_update_protection().
Dave's concern is that we're calling with with preemption already disabled so disabling it again is superfluous.
Re: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
Quoted 14 lines · Peter Zijlstra
Quoted 6 lines · Ira Weiny
Quoted 3 lines
so it either needs to
explicitly do so, or have an assertion that preemption is indeed disabled.However, I don't think I understand clearly. Doesn't [get|put]_cpu_ptr() handle the
preempt_disable()for us?It does.
Quoted 1 line
Is it not sufficient to rely on that?
It is.
Quoted 8 lines
Quoted 5 lines
Dave's comment seems to be the opposite where we need to eliminate preempt disable before calling
write_pkrs().FWIW I think I'm mistaken in my response to Dave regarding the
preempt_disable()inpks_update_protection().Dave's concern is that we're calling with with preemption already disabled so disabling it again is superfluous.
Ok, thanks, and after getting my head straight I think I agree with him, and you.
Thanks I've reworked the code to removed the superfluous calls. Sorry about being so dense... :-D
Ira
Peter Zijlstra
maintainerRe: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
In reply to [email protected]arch/x86/kernel/process.c void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p) · line 669
Quoted 8 lines · On Fri, Oct 09, 2020 at 12:42:53PM -0700, [email protected] wrote:
arch/x86/kernel/process.c +2
-0@@ -644,6 +663,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p)if ((tifp ^ tifn) & _TIF_SLD)switch_to_sld (tifn); + +pks_sched_in (); }
You seem to have lost the comment proposed here:
https://lkml.kernel.org/r/[email protected]
It is useful and important information that the wrmsr normally doesn't happen.
Quoted 11 lines
arch/x86/mm/pkeys.c +3
-0@@ -229,3 +229,31 @@ u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags)return pk_reg; } + +DEFINE_PER_CPU (u32, pkrs_cache); +
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 244
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as +* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register.
(whitespace damage; space followed by tabstop)
Quoted 5 lines
arch/x86/mm/pkeys.c +5
-0+*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; +
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 259
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache); +if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs); + } +put_cpu_ptr (pkrs); +}
looks familiar that... :-)
Re: [PATCH RFC V3 4/9] x86/pks: Preserve the PKRS MSR on context switch
Quoted 13 lines · Peter Zijlstra
arch/x86/kernel/process.c void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p) · line 669
Quoted 8 lines · On Fri, Oct 09, 2020 at 12:42:53PM -0700, [email protected] wrote:
arch/x86/kernel/process.c +2
-0@@ -644,6 +663,8 @@ void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p)if ((tifp ^ tifn) & _TIF_SLD)switch_to_sld (tifn); + +pks_sched_in (); }You seem to have lost the comment proposed here:
Quoted 4 lines
https://lkml.kernel.org/r/[email protected]
It is useful and important information that the wrmsr normally doesn't happen.
Added back in here.
Quoted 16 lines
Quoted 16 lines
arch/x86/mm/pkeys.c +7
-0@@ -229,3 +229,31 @@ u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags)return pk_reg; } + +DEFINE_PER_CPU (u32, pkrs_cache); + +/** +* It should also be noted that the underlying WRMSR(MSR_IA32_PKRS) is not +* serializing but still maintains ordering properties similar to WRPKRU. +* The current SDM section on PKRS needs updating but should be the same as
Quoted 8 lines
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 244
Quoted 6 lines
arch/x86/mm/pkeys.c +6
-0+* that of WRPKRU. So to quote from the WRPKRU text: +* +* WRPKRU will never execute transiently. Memory accesses +* affected by PKRU register will not execute (even transiently) +* until all prior executions of WRPKRU have completed execution +* and updated the PKRU register. (whitespace damage; space followed by tabstop)
Fixed thanks.
Quoted 10 lines
Quoted 10 lines
arch/x86/mm/pkeys.c +9
-0+*/ +void write_pkrs (u32 new_pkrs) +{ +u32 *pkrs; + +if (!static_cpu_has (X86_FEATURE_PKS )) +return ; + + pkrs =get_cpu_ptr (&pkrs_cache);
Quoted 8 lines
arch/x86/mm/pkeys.c u32 update_pkey_val(u32 pk_reg, int pkey, unsigned int flags) · line 259
Quoted 6 lines
arch/x86/mm/pkeys.c +6
-0+if (*pkrs != new_pkrs) { + *pkrs = new_pkrs; +wrmsrl (MSR_IA32_PKRS , new_pkrs); + } +put_cpu_ptr (pkrs); +}looks familiar that... :-)
Added you as a co-developer if that is ok?
Ira
RFC PATCH v3 5/9 x86/pks: Add PKS kernel API
In reply to [email protected]From: Fenghua Yu <[email protected]>
PKS allows kernel users to define domains of page mappings which have additional protections beyond the paging protections.
Add an API to allocate, use, and free a protection key which identifies such a domain. Export 5 new symbols pks_key_alloc(), pks_mknoaccess(), pks_mkread(), pks_mkrdwr(), and pks_key_free(). Add 2 new macros; PAGE_KERNEL_PKEY(key) and _PAGE_PKEY(pkey).
Update the protection key documentation to cover pkeys on supervisor pages.
--- Documentation/core-api/protection-keys.rst | 101 ++++++++++++++--- arch/x86/include/asm/pgtable_types.h | 12 ++ arch/x86/include/asm/pkeys.h | 11 ++ arch/x86/include/asm/pkeys_common.h | 4 + arch/x86/mm/pkeys.c | 125 +++++++++++++++++++++ include/linux/pgtable.h | 4 + include/linux/pkeys.h | 22 ++++ 7 files changed, 261 insertions(+), 18 deletions(-)
Documentation/core-api/protection-keys.rst +83 -18
@@ -4,25 +4,33 @@ Memory Protection Keys ====================== -Memory Protection Keys for Userspace (PKU aka PKEYs) is a feature -which is found on Intel's Skylake (and later) "Scalable Processor" -Server CPUs. It will be available in future non-server Intel parts -and future AMD processors. - -For anyone wishing to test or use this feature, it is available in -Amazon's EC2 C5 instances and is known to work there using an Ubuntu -17.04 image. - Memory Protection Keys provides a mechanism for enforcing page-based protections, but without requiring modification of the page tables -when an application changes protection domains. It works by -dedicating 4 previously ignored bits in each page table entry to a -"protection key", giving 16 possible keys. +when an application changes protection domains. + +PKeys Userspace (PKU) is a feature which is found on Intel's Skylake "Scalable +Processor" Server CPUs and later. And It will be available in future +non-server Intel parts and future AMD processors. + +Future Intel processors will support Protection Keys for Supervisor pages +(PKS). + +For anyone wishing to test or use user space pkeys, it is available in Amazon's +EC2 C5 instances and is known to work there using an Ubuntu 17.04 image. + +pkeys work by dedicating 4 previously Reserved bits in each page table entry to +a "protection key", giving 16 possible keys. User and Supervisor pages are +treated separately. + +Protections for each page are controlled with per CPU registers for each type +of page User and Supervisor. Each of these 32 bit register stores two separate +bits (Access Disable and Write Disable) for each key. -There is also a new user-accessible register (PKRU) with two separate -bits (Access Disable and Write Disable) for each key. Being a CPU -register, PKRU is inherently thread-local, potentially giving each -thread a different set of protections from every other thread. +For Userspace the register is user-accessible (rdpkru/wrpkru). For +Supervisor, the register (MSR_IA32_PKRS) is accessible only to the kernel. + +Being a CPU register, pkeys are inherently thread-local, potentially giving +each thread an independent set of protections from every other thread. There are two new instructions (RDPKRU/WRPKRU) for reading and writing to the new register. The feature is only available in 64-bit mode,
@@ -30,8 +38,11 @@ even though there is theoretically space in the PAE PTEs. These permissions are enforced on data access only and have no effect on instruction fetches. -Syscalls -======== +For kernel space rdmsr/wrmsr are used to access the kernel MSRs. + + +Syscalls for user space keys +============================ There are 3 system calls which directly interact with pkeys::
@@ -98,3 +109,57 @@ with a read():: The kernel will send a SIGSEGV in both cases, but si_code will be set to SEGV_PKERR when violating protection keys versus SEGV_ACCERR when the plain mprotect() permissions are violated. + + +Kernel API for PKS support +========================== + +The following interface is used to allocate, use, and free a pkey which defines +a 'protection domain' within the kernel. Setting a pkey value in a supervisor +mapping adds that mapping to the protection domain. + + int pks_key_alloc(const char * const pkey_user); + #define PAGE_KERNEL_PKEY(pkey) + #define _PAGE_KEY(pkey) + void pks_mknoaccess(int pkey); + void pks_mkread(int pkey); + void pks_mkrdwr(int pkey); + void pks_key_free(int pkey); + +pks_key_alloc() allocates keys dynamically to allow better use of the limited +key space. + +Callers of pks_key_alloc() _must_ be prepared for it to fail and take +appropriate action. This is due mainly to the fact that PKS may not be +available on all arch's. Failure to check the return of pks_key_alloc() and +using any of the rest of the API is undefined. + +Kernel users must set the PTE permissions in the page table entries for the +mappings they want to protect. This can be done with PAGE_KERNEL_PKEY() or +_PAGE_KEY(). + +The pks_mk*() family of calls allows kernel users the ability to change the +protections for the domain identified by the pkey specified. 3 states are +available pks_mknoaccess(), pks_mkread(), and pks_mkrdwr() which set the access +to none, read, and read/write respectively. + +Finally, pks_key_free() allows a user to return the key to the allocator for +use by others. + +The interface maintains pks_mknoaccess() (Access Disabled (AD=1)) for all keys +not currently allocated. Therefore, the user can depend on access being +disabled when pks_key_alloc() returns a key and the user should remove mappings +from the domain (remove the pkey from the PTE) prior to calling pks_key_free(). + +It should be noted that the underlying WRMSR(MSR_IA32_PKRS) is not serializing +but still maintains ordering properties similar to WRPKRU. Thus it is safe to +immediately use a mapping when the pks_mk*() functions returns. + +The current SDM section on PKRS needs updating but should be the same as that +of WRPKRU. So to quote from the WRPKRU text: + + WRPKRU will never execute transiently. Memory accesses + affected by PKRU register will not execute (even transiently) + until all prior executions of WRPKRU have completed execution + and updated the PKRU register. +
arch/x86/include/asm/pgtable_types.h +12 -0
@@ -73,6 +73,12 @@ _PAGE_PKEY_BIT2 | \ _PAGE_PKEY_BIT3) +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +#define _PAGE_PKEY (pkey) (_AT(pteval_t, pkey) << _PAGE_BIT_PKEY_BIT0) +#else +#define _PAGE_PKEY (pkey) (_AT(pteval_t, 0)) +#endif +#if defined(CONFIG_X86_64 ) || defined(CONFIG_X86_PAE )#define _PAGE_KNL_ERRATUM_MASK (_PAGE_DIRTY | _PAGE_ACCESSED)#else
@@ -229,6 +235,12 @@ enum page_cache_mode {#define PAGE_KERNEL_IO __pgprot_mask(__PAGE_KERNEL_IO)#define PAGE_KERNEL_IO_NOCACHE __pgprot_mask(__PAGE_KERNEL_IO_NOCACHE) +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +#define PAGE_KERNEL_PKEY (pkey) __pgprot_mask(__PAGE_KERNEL | _PAGE_PKEY(pkey)) +#else +#define PAGE_KERNEL_PKEY (pkey) PAGE_KERNEL +#endif +#endif /* __ASSEMBLY__ */ /* xwr */
arch/x86/include/asm/pkeys.h +11 -0
@@ -3,6 +3,7 @@#define _ASM_X86_PKEYS_H#include <asm/pkeys_common.h> +#include <asm-generic/mman-common.h> #define ARCH_DEFAULT_PKEY 0
@@ -138,4 +139,14 @@ static inline int vma_pkey(struct vm_area_struct *vma)u32 update_pkey_val (u32 pk_reg,int pkey,unsigned int flags); +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +int pks_key_alloc (const char *const pkey_user); +void pks_key_free (int pkey); + +void pks_mknoaccess (int pkey); +void pks_mkread (int pkey); +void pks_mkrdwr (int pkey); + +#endif /* CONFIG_ARCH_HAS_SUPERVISOR_PKEYS */ +#endif /*_ASM_X86_PKEYS_H */
arch/x86/include/asm/pkeys_common.h +4 -0
@@ -22,6 +22,10 @@ PKR_AD_KEY(10) | PKR_AD_KEY(11) | PKR_AD_KEY(12) | \ PKR_AD_KEY(13) | PKR_AD_KEY(14) | PKR_AD_KEY(15)) +/* PKS supports 16 keys. Key 0 is reserved for the kernel. */ +#define PKS_KERN_DEFAULT_KEY 0 +#define PKS_NUM_KEYS 16 +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS void write_pkrs (u32 new_pkrs);#else
arch/x86/mm/pkeys.c +125 -0
140 lines not on this page · raw message
include/linux/pgtable.h +4 -0
@@ -1374,6 +1374,10 @@ static inline bool arch_has_pfn_modify_check(void)# define PAGE_KERNEL_EXEC PAGE_KERNEL#endif +#ifndef PAGE_KERNEL_PKEY +#define PAGE_KERNEL_PKEY (pkey) PAGE_KERNEL +#endif +/* * Page Table Modification bits for pgtbl_mod_mask. *
include/linux/pkeys.h +22 -0
@@ -50,4 +50,26 @@ static inline void copy_init_pkru_to_fpregs(void)#endif /* ! CONFIG_ARCH_HAS_PKEYS */ +#ifndef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +static inline int pks_key_alloc (const char *const pkey_user) +{ +return -EINVAL ; +} +static inline void pks_key_free (int pkey) +{ +} +static inline void pks_mknoaccess (int pkey) +{ +WARN_ON_ONCE (1 ); +} +static inline void pks_mkread (int pkey) +{ +WARN_ON_ONCE (1 ); +} +static inline void pks_mkrdwr (int pkey) +{ +WARN_ON_ONCE (1 ); +} +#endif /* ! CONFIG_ARCH_HAS_SUPERVISOR_PKEYS */ +#endif /* _LINUX_PKEYS_H */
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 5/9] x86/pks: Add PKS kernel API
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 284
Quoted 8 lines
arch/x86/mm/pkeys.c +8
-0+static inline void pks_update_protection (int pkey,unsigned long protection) +{ + current->thread.saved_pkrs =update_pkey_val (current->thread.saved_pkrs, + pkey, protection); +preempt_disable (); +write_pkrs (current->thread.saved_pkrs); +preempt_enable (); +}
Why does this need preempt count manipulation in addition to the get/put_cpu_var() inside of write_pkrs()?
Quoted 13 lines
arch/x86/mm/pkeys.c +13
-0+/** +* PKS access control functions +* +* Change the access of the domain specified by the pkey. These are global +* updates. They only affects the current running thread. It is undefined and +* a bug for users to call this without having allocated a pkey and using it as +* pkey here. +* +* pks_mknoaccess() +* Disable all access to the domain +* pks_mkread() +* Make the domain Read only +* pks_mkrdwr()
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 308
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+* Make the domain Read/Write +* +* @pkey the pkey for which the access should change. +* +*/ +void pks_mknoaccess (int pkey) +{ +pks_update_protection (pkey,PKEY_DISABLE_ACCESS ); +} +EXPORT_SYMBOL_GPL (pks_mknoaccess);
These are named like PTE manipulation functions, which is kinda weird.
What's wrong with: pks_disable_access(pkey) ?
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 314
Quoted 5 lines
arch/x86/mm/pkeys.c +5
-0+void pks_mkread (int pkey) +{ +pks_update_protection (pkey,PKEY_DISABLE_WRITE ); +} +EXPORT_SYMBOL_GPL (pks_mkread);
I really don't like this name. It doesn't make readable, or even read-only, *especially* if it was already access-disabled.
Quoted 15 lines
arch/x86/mm/pkeys.c +15
-0+static const char pks_key_user0[] ="kernel" ; + +/* Store names of allocated keys for debug. Key 0 is reserved for the kernel. */ +static const char *pks_key_users[PKS_NUM_KEYS ] = { + pks_key_user0 +}; + +/* +* Each key is represented by a bit. Bit 0 is set for key 0 and reserved for +* its use. We use ulong for the bit operations but only 16 bits are used. +*/ +static unsigned long pks_key_allocation_map =1 <<PKS_KERN_DEFAULT_KEY ; + +/* +* pks_key_alloc - Allocate a PKS key
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 346
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+* +* @pkey_user: String stored for debugging of key exhaustion. The caller is +* responsible to maintain this memory until pks_key_free(). +*/ +int pks_key_alloc (const char *const pkey_user) +{ +int nr; + +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return -EINVAL ;
I'm not sure I like -EINVAL for this. I thought we returned -ENOSPC for this case for user pkeys.
Quoted 20 lines
arch/x86/mm/pkeys.c +20
-0+while (1 ) { + nr =find_first_zero_bit (&pks_key_allocation_map,PKS_NUM_KEYS ); +if (nr >=PKS_NUM_KEYS ) { +pr_info ("Cannot allocate supervisor key for %s.\n" , + pkey_user); +return -ENOSPC ; + } +if (!test_and_set_bit_lock (nr, &pks_key_allocation_map)) +break ; + } + +/* for debugging key exhaustion */ + pks_key_users[nr] = pkey_user; + +return nr; +} +EXPORT_SYMBOL_GPL (pks_key_alloc); + +/* +* pks_key_free - Free a previously allocate PKS key
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 377
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+* +* @pkey: Key to be free'ed +*/ +void pks_key_free (int pkey) +{ +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ; + +if (pkey >=PKS_NUM_KEYS || pkey <=PKS_KERN_DEFAULT_KEY ) +return ;
This seems worthy of a WARN_ON_ONCE() at least. It's essentially corrupt data coming into a kernel API.
Re: [PATCH RFC V3 5/9] x86/pks: Add PKS kernel API
Quoted 11 lines · Dave Hansen
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 284
Quoted 8 lines
arch/x86/mm/pkeys.c +8
-0+static inline void pks_update_protection (int pkey,unsigned long protection) +{ + current->thread.saved_pkrs =update_pkey_val (current->thread.saved_pkrs, + pkey, protection); +preempt_disable (); +write_pkrs (current->thread.saved_pkrs); +preempt_enable (); +}Why does this need preempt count manipulation in addition to the get/put_cpu_var() inside of
write_pkrs()?
This is a bug. The disable should be around the update_pkey_val().
Quoted 27 lines
Quoted 14 lines
arch/x86/mm/pkeys.c +13
-0+/** +* PKS access control functions +* +* Change the access of the domain specified by the pkey. These are global +* updates. They only affects the current running thread. It is undefined and +* a bug for users to call this without having allocated a pkey and using it as +* pkey here. +* +* pks_mknoaccess() +* Disable all access to the domain +* pks_mkread() +* Make the domain Read only +* pks_mkrdwr() arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 308
Quoted 10 lines
arch/x86/mm/pkeys.c +10
-0+* Make the domain Read/Write +* +* @pkey the pkey for which the access should change. +* +*/ +void pks_mknoaccess (int pkey) +{ +pks_update_protection (pkey,PKEY_DISABLE_ACCESS ); +} +EXPORT_SYMBOL_GPL (pks_mknoaccess);These are named like PTE manipulation functions, which is kinda weird.
Quoted 1 line
What's wrong with:
pks_disable_access(pkey)?
Internal review suggested these names. I'm not dead set on them.
FWIW I would rather they not get to wordy.
I was trying to get some consistency with pks_mk*() as meaning PKS 'make' X.
Do me 'disable' implies a state transition where 'make' implies we are 'setting' an absolute value. I think the later is a better name. And 'make' made more sense because 'set' is so overloaded IHO.
Quoted 9 lines
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 314
Quoted 6 lines
arch/x86/mm/pkeys.c +5
-0+void pks_mkread (int pkey) +{ +pks_update_protection (pkey,PKEY_DISABLE_WRITE ); +} +EXPORT_SYMBOL_GPL (pks_mkread);I really don't like this name. It doesn't make readable, or even read-only, *especially* if it was already access-disabled.
Ok.
But it does sense if going from access-disable to read, correct?. I could see this being better named pks_mkreadonly() so that going from RW to this would make more sense. Especially after thinking about it above 'read only' needs to be in the name.
Before I change anything I'd like to get consensus on naming.
How about the following?
pks_mk_noaccess()pks_mk_readonly()pks_mk_readwrite()
?
Quoted 21 lines
Quoted 21 lines
arch/x86/mm/pkeys.c +20
-0+static const char pks_key_user0[] ="kernel" ; + +/* Store names of allocated keys for debug. Key 0 is reserved for the kernel. */ +static const char *pks_key_users[PKS_NUM_KEYS ] = { + pks_key_user0 +}; + +/* +* Each key is represented by a bit. Bit 0 is set for key 0 and reserved for +* its use. We use ulong for the bit operations but only 16 bits are used. +*/ +static unsigned long pks_key_allocation_map =1 <<PKS_KERN_DEFAULT_KEY ; + +/* +* pks_key_alloc - Allocate a PKS key +* +* @pkey_user: String stored for debugging of key exhaustion. The caller is +* responsible to maintain this memory until pks_key_free(). +*/ +int pks_key_alloc (const char *const pkey_user)
Quoted 8 lines
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 346
Quoted 5 lines
arch/x86/mm/pkeys.c +5
-0+{ +int nr; + +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return -EINVAL ;I'm not sure I like
-EINVALfor this. I thought we returned-ENOSPCfor this case for user pkeys.
-ENOTSUP?
I'm not really sure anyone will need to know the difference between the platform not supporting the key vs running out of them. But they are 2 different error conditions.
Quoted 7 lines
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 353
Quoted 7 lines
arch/x86/mm/pkeys.c +6
-0+while (1 ) { + nr =find_first_zero_bit (&pks_key_allocation_map,PKS_NUM_KEYS ); +if (nr >=PKS_NUM_KEYS ) { +pr_info ("Cannot allocate supervisor key for %s.\n" , + pkey_user); +return -ENOSPC ;
We return -ENOSPC here when running out of keys.
Quoted 19 lines
Quoted 19 lines
arch/x86/mm/pkeys.c +19
-0+ } +if (!test_and_set_bit_lock (nr, &pks_key_allocation_map)) +break ; + } + +/* for debugging key exhaustion */ + pks_key_users[nr] = pkey_user; + +return nr; +} +EXPORT_SYMBOL_GPL (pks_key_alloc); + +/* +* pks_key_free - Free a previously allocate PKS key +* +* @pkey: Key to be free'ed +*/ +void pks_key_free (int pkey) +{
Quoted 8 lines
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 377
Quoted 5 lines
arch/x86/mm/pkeys.c +5
-0+if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ; + +if (pkey >=PKS_NUM_KEYS || pkey <=PKS_KERN_DEFAULT_KEY ) +return ;This seems worthy of a
WARN_ON_ONCE()at least. It's essentially corrupt data coming into a kernel API.
Ok, Done,
Ira
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Peter Zijlstra
maintainerRe: [PATCH RFC V3 5/9] x86/pks: Add PKS kernel API
In reply to [email protected]arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 284
Quoted 8 lines · On Fri, Oct 09, 2020 at 12:42:54PM -0700, [email protected] wrote:
arch/x86/mm/pkeys.c +8
-0+static inline void pks_update_protection (int pkey,unsigned long protection) +{ + current->thread.saved_pkrs =update_pkey_val (current->thread.saved_pkrs, + pkey, protection); +preempt_disable (); +write_pkrs (current->thread.saved_pkrs); +preempt_enable (); +}
write_pkrs() already disables preemption itself. Wrapping it in yet another layer is useless.
Re: [PATCH RFC V3 5/9] x86/pks: Add PKS kernel API
Quoted 12 lines · Peter Zijlstra
arch/x86/mm/pkeys.c void write_pkrs(u32 new_pkrs) · line 284
Quoted 8 lines · On Fri, Oct 09, 2020 at 12:42:54PM -0700, [email protected] wrote:
arch/x86/mm/pkeys.c +8
-0+static inline void pks_update_protection (int pkey,unsigned long protection) +{ + current->thread.saved_pkrs =update_pkey_val (current->thread.saved_pkrs, + pkey, protection); +preempt_disable (); +write_pkrs (current->thread.saved_pkrs); +preempt_enable (); +}
write_pkrs()already disables preemption itself. Wrapping it in yet another layer is useless.
I was thinking the update to saved_pkrs needed this protection as well and that was to be included in the preemption disable. But that too is incorrect.
I've removed this preemption disable.
Thanks,
Ira
RFC PATCH v3 6/9 x86/entry: Pass irqentry_state_t by reference
In reply to [email protected]From: Ira Weiny <[email protected]>
In preparation for adding PKS information to struct irqentry_state_t change all call sites and usages to pass the struct by reference instead of by value.
--- arch/x86/entry/common.c | 16 +++++++--------- arch/x86/include/asm/idtentry.h | 29 +++++++++++++++++------------ arch/x86/kernel/kvm.c | 4 ++-- arch/x86/kernel/nmi.c | 7 ++++--- arch/x86/kernel/traps.c | 21 +++++++++++++-------- arch/x86/mm/fault.c | 4 ++-- include/linux/entry-common.h | 7 ++++--- kernel/entry/common.c | 20 ++++++++------------ 8 files changed, 57 insertions(+), 51 deletions(-)
arch/x86/entry/common.c +7 -9
@@ -209,9 +209,9 @@ SYSCALL_DEFINE0(ni_syscall)return -ENOSYS ; } -noinstr bool idtentry_enter_nmi(struct pt_regs *regs) +noinstr void idtentry_enter_nmi(struct pt_regs *regs, irqentry_state_t *irq_state) { -bool irq_state =lockdep_hardirqs_enabled (); + irq_state->exit_rcu =lockdep_hardirqs_enabled ();__nmi_enter ();lockdep_hardirqs_off (CALLER_ADDR0 );
@@ -222,15 +222,13 @@ noinstr bool idtentry_enter_nmi(struct pt_regs *regs)trace_hardirqs_off_finish ();ftrace_nmi_enter ();instrumentation_end (); - -return irq_state; } -noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) +noinstr void idtentry_exit_nmi(struct pt_regs *regs, irqentry_state_t *irq_state) {instrumentation_begin ();ftrace_nmi_exit (); -if (restore) { +if (irq_state->exit_rcu) {trace_hardirqs_on_prepare ();lockdep_hardirqs_on_prepare (CALLER_ADDR0 ); }
@@ -238,7 +236,7 @@ noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore)rcu_nmi_exit ();lockdep_hardirq_exit (); -if (restore) +if (irq_state->exit_rcu)lockdep_hardirqs_on (CALLER_ADDR0 );__nmi_exit (); }
@@ -295,7 +293,7 @@ __visible noinstr void xen_pv_evtchn_do_upcall(struct pt_regs *regs)bool inhcall;irqentry_state_t state; - state =irqentry_enter (regs); +irqentry_enter (regs, &state); old_regs =set_irq_regs (regs);instrumentation_begin ();
@@ -311,7 +309,7 @@ __visible noinstr void xen_pv_evtchn_do_upcall(struct pt_regs *regs)instrumentation_end (); restore_inhcall(inhcall); }else { -irqentry_exit (regs, state); +irqentry_exit (regs, &state); } }#endif /* CONFIG_XEN_PV */
arch/x86/include/asm/idtentry.h +17 -12
@@ -11,8 +11,8 @@#include <asm/irq_stack.h> -bool idtentry_enter_nmi (struct pt_regs *regs); -void idtentry_exit_nmi (struct pt_regs *regs,bool irq_state); +void idtentry_enter_nmi (struct pt_regs *regs,irqentry_state_t *irq_state); +void idtentry_exit_nmi (struct pt_regs *regs,irqentry_state_t *irq_state);/** * DECLARE_IDTENTRY - Declare functions for simple IDT entry points
@@ -52,12 +52,13 @@ static __always_inline void __##func(struct pt_regs *regs); \ \ __visible noinstr void func(struct pt_regs *regs) \ { \ -irqentry_state_t state =irqentry_enter (regs); \ +irqentry_state_t state; \ \ +irqentry_enter (regs, &state); \instrumentation_begin (); \ __##func (regs); \ instrumentation_end(); \ - irqentry_exit(regs, state); \ + irqentry_exit(regs, &state); \ } \ \ static __always_inline void __##func(struct pt_regs *regs)
@@ -99,12 +100,13 @@ static __always_inline void __##func(struct pt_regs *regs, \ __visible noinstr void func(struct pt_regs *regs, \unsigned long error_code) \ { \ -irqentry_state_t state =irqentry_enter (regs); \ +irqentry_state_t state; \ \ +irqentry_enter (regs, &state); \instrumentation_begin (); \ __##func (regs, error_code); \ instrumentation_end(); \ - irqentry_exit(regs, state); \ + irqentry_exit(regs, &state); \ } \ \ static __always_inline void __##func(struct pt_regs *regs, \
@@ -195,15 +197,16 @@ static __always_inline void __##func(struct pt_regs *regs, u8 vector); \ __visible noinstr void func(struct pt_regs *regs, \unsigned long error_code) \ { \ -irqentry_state_t state =irqentry_enter (regs); \ +irqentry_state_t state; \ \ +irqentry_enter (regs, &state); \instrumentation_begin (); \irq_enter_rcu (); \kvm_set_cpu_l1tf_flush_l1d (); \ __##func (regs, (u8)error_code); \ irq_exit_rcu(); \ instrumentation_end(); \ - irqentry_exit(regs, state); \ + irqentry_exit(regs, &state); \ } \ \ static __always_inline void __##func(struct pt_regs *regs, u8 vector)
@@ -237,15 +240,16 @@ static void __##func(struct pt_regs *regs); \ \ __visible noinstr void func(struct pt_regs *regs) \ { \ -irqentry_state_t state =irqentry_enter (regs); \ +irqentry_state_t state; \ \ +irqentry_enter (regs, &state); \instrumentation_begin (); \irq_enter_rcu (); \kvm_set_cpu_l1tf_flush_l1d (); \ run_sysvec_on_irqstack_cond(__##func, regs); \ irq_exit_rcu(); \ instrumentation_end(); \ - irqentry_exit(regs, state); \ + irqentry_exit(regs, &state); \ } \ \ static noinline void __##func(struct pt_regs *regs)
@@ -266,15 +270,16 @@ static __always_inline void __##func(struct pt_regs *regs); \ \ __visible noinstr void func(struct pt_regs *regs) \ { \ -irqentry_state_t state =irqentry_enter (regs); \ +irqentry_state_t state; \ \ +irqentry_enter (regs, &state); \instrumentation_begin (); \__irq_enter_raw (); \kvm_set_cpu_l1tf_flush_l1d (); \ __##func (regs); \ __irq_exit_raw(); \ instrumentation_end(); \ - irqentry_exit(regs, state); \ + irqentry_exit(regs, &state); \ } \ \ static __always_inline void __##func(struct pt_regs *regs)
arch/x86/kernel/kvm.c +2 -2
@@ -241,7 +241,7 @@ noinstr bool __kvm_handle_async_pf(struct pt_regs *regs, u32 token)if (!flags)return false; - state =irqentry_enter (regs); +irqentry_enter (regs, &state);instrumentation_begin ();/*
@@ -262,7 +262,7 @@ noinstr bool __kvm_handle_async_pf(struct pt_regs *regs, u32 token) }instrumentation_end (); -irqentry_exit (regs, state); +irqentry_exit (regs, &state);return true; }
arch/x86/kernel/nmi.c +4 -3
@@ -33,6 +33,7 @@#include <asm/reboot.h> #include <asm/cache.h> #include <asm/nospec-branch.h> +#include <asm/idtentry.h> #define CREATE_TRACE_POINTS #include <trace/events/nmi.h>
@@ -475,7 +476,7 @@ static DEFINE_PER_CPU(unsigned long, nmi_dr7);DEFINE_IDTENTRY_RAW (exc_nmi) { -bool irq_state; +irqentry_state_t irq_state = { };if (IS_ENABLED (CONFIG_SMP ) &&arch_cpu_is_offline (smp_processor_id ()))return ;
@@ -490,14 +491,14 @@ DEFINE_IDTENTRY_RAW(exc_nmi)this_cpu_write (nmi_dr7,local_db_save ()); - irq_state =idtentry_enter_nmi (regs); +idtentry_enter_nmi (regs, &irq_state);inc_irq_stat (__nmi_count);if (!ignore_nmis)default_do_nmi (regs); -idtentry_exit_nmi (regs, irq_state); +idtentry_exit_nmi (regs, &irq_state);local_db_restore (this_cpu_read (nmi_dr7));
arch/x86/kernel/traps.c +13 -8
@@ -254,11 +254,11 @@ DEFINE_IDTENTRY_RAW(exc_invalid_op)if (!user_mode (regs) &&handle_bug (regs))return ; - state =irqentry_enter (regs); +irqentry_enter (regs, &state);instrumentation_begin ();handle_invalid_op (regs);instrumentation_end (); -irqentry_exit (regs, state); +irqentry_exit (regs, &state); }DEFINE_IDTENTRY (exc_coproc_segment_overrun)
@@ -342,6 +342,7 @@ __visible void __noreturn handle_stack_overflow(const char *message,*/ DEFINE_IDTENTRY_DF (exc_double_fault) { +irqentry_state_t irq_state;static const char str[] ="double fault" ;struct task_struct *tsk = current;
@@ -404,7 +405,7 @@ DEFINE_IDTENTRY_DF(exc_double_fault) }#endif -idtentry_enter_nmi (regs); +idtentry_enter_nmi (regs, &irq_state);instrumentation_begin ();notify_die (DIE_TRAP , str, regs, error_code,X86_TRAP_DF ,SIGSEGV );
@@ -650,12 +651,15 @@ DEFINE_IDTENTRY_RAW(exc_int3)instrumentation_end ();irqentry_exit_to_user_mode (regs); }else { -bool irq_state =idtentry_enter_nmi (regs); +irqentry_state_t irq_state; + +idtentry_enter_nmi (regs, &irq_state); +instrumentation_begin ();if (!do_int3 (regs))die ("int3" , regs,0 );instrumentation_end (); -idtentry_exit_nmi (regs, irq_state); +idtentry_exit_nmi (regs, &irq_state); } }
@@ -861,7 +865,9 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs,* includes the entry stack is excluded for everything. */ unsigned long dr7 =local_db_save (); -bool irq_state =idtentry_enter_nmi (regs); +irqentry_state_t irq_state; + +idtentry_enter_nmi (regs, &irq_state);instrumentation_begin ();/*
@@ -880,8 +886,7 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs,handle_debug (regs, dr6, false);instrumentation_end (); -idtentry_exit_nmi (regs, irq_state); - +idtentry_exit_nmi (regs, &irq_state);local_db_restore (dr7); }
arch/x86/mm/fault.c +2 -2
@@ -1476,11 +1476,11 @@ DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault)* code reenabled RCU to avoid subsequent wreckage which helps * debugability. */ - state =irqentry_enter (regs); +irqentry_enter (regs, &state);instrumentation_begin ();handle_page_fault (regs, error_code, address);instrumentation_end (); -irqentry_exit (regs, state); +irqentry_exit (regs, &state); }
include/linux/entry-common.h +4 -3
@@ -349,6 +349,7 @@ typedef struct irqentry_state {/** * irqentry_enter - Handle state tracking on ordinary interrupt entries * @regs: Pointer to pt_regs of interrupted context +* @state: Pointer to an object to store the irq state * * Invokes: * - lockdep irqflag state tracking as low level ASM entry disabled
@@ -377,7 +378,7 @@ typedef struct irqentry_state {* * Returns: An opaque object that must be passed to idtentry_exit() */ -irqentry_state_t noinstr irqentry_enter(struct pt_regs *regs); +void noinstr irqentry_enter(struct pt_regs *regs, irqentry_state_t *state);/** * irqentry_exit_cond_resched - Conditionally reschedule on return from interrupt
@@ -389,7 +390,7 @@ void irqentry_exit_cond_resched(void);/** * irqentry_exit - Handle return from exception that used irqentry_enter() * @regs: Pointer to pt_regs (exception entry regs) -* @state: Return value from matching call to irqentry_enter() +* @state: Reference to the value saved in irqentry_enter() * * Depending on the return target (kernel/user) this runs the necessary * preemption and work checks if possible and reguired and returns to
@@ -400,6 +401,6 @@ void irqentry_exit_cond_resched(void);* * Counterpart to irqentry_enter(). */ -void noinstr irqentry_exit(struct pt_regs *regs, irqentry_state_t state); +void noinstr irqentry_exit(struct pt_regs *regs, irqentry_state_t *state);#endif
kernel/entry/common.c +8 -12
@@ -282,15 +282,13 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs)exit_to_user_mode (); } -noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs) +noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state) { -irqentry_state_t ret = { - .exit_rcu = false, - }; + state->exit_rcu = false;if (user_mode (regs)) {irqentry_enter_from_user_mode (regs); -return ret; +return ; }/*
@@ -328,8 +326,8 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)trace_hardirqs_off_finish ();instrumentation_end (); - ret.exit_rcu = true; -return ret; + state->exit_rcu = true; +return ; }/*
@@ -343,8 +341,6 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs)/* Use the combo lockdep/tracing function */ trace_hardirqs_off ();instrumentation_end (); - -return ret; }void irqentry_exit_cond_resched (void )
@@ -359,7 +355,7 @@ void irqentry_exit_cond_resched(void) } } -noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state) +noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t *state) {lockdep_assert_irqs_disabled ();
@@ -372,7 +368,7 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state)* carefully and needs the same ordering of lockdep/tracing * and RCU as the return to user mode path. */ -if (state.exit_rcu) { +if (state->exit_rcu) {instrumentation_begin ();/* Tell the tracer that IRET will enable interrupts */ trace_hardirqs_on_prepare ();
@@ -394,7 +390,7 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t state)* IRQ flags state is correct already. Just tell RCU if it * was not watching on entry. */ -if (state.exit_rcu) +if (state->exit_rcu)rcu_irq_exit (); } }
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Peter Zijlstra
maintainerRe: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 27 lines · On Fri, Oct 09, 2020 at 12:42:55PM -0700, [email protected] wrote:
arch/x86/entry/common.c +4
-6-noinstr bool idtentry_enter_nmi(struct pt_regs *regs) +noinstr void idtentry_enter_nmi(struct pt_regs *regs, irqentry_state_t *irq_state) { -bool irq_state =lockdep_hardirqs_enabled (); + irq_state->exit_rcu =lockdep_hardirqs_enabled ();__nmi_enter ();lockdep_hardirqs_off (CALLER_ADDR0 );@@ -222,15 +222,13 @@ noinstr bool idtentry_enter_nmi(struct pt_regs *regs)trace_hardirqs_off_finish ();ftrace_nmi_enter ();instrumentation_end (); - -return irq_state; } -noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) +noinstr void idtentry_exit_nmi(struct pt_regs *regs, irqentry_state_t *irq_state) {instrumentation_begin ();ftrace_nmi_exit (); -if (restore) { +if (irq_state->exit_rcu) {trace_hardirqs_on_prepare ();lockdep_hardirqs_on_prepare (CALLER_ADDR0 ); }
arch/x86/entry/common.c noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) · line 242
Quoted 8 lines
arch/x86/entry/common.c +1
-1@@ -238,7 +236,7 @@ noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore)rcu_nmi_exit ();lockdep_hardirq_exit (); -if (restore) +if (irq_state->exit_rcu)lockdep_hardirqs_on (CALLER_ADDR0 );__nmi_exit (); }
That's not nice.. The NMI path is different from the IRQ path and has a different variable. Yes, this works, but *groan*.
Maybe union them if you want to avoid bloating the structure, but the above makes it really hard to read.
Re: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 14 lines · Peter Zijlstra
Review noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) · line 242
Quoted 9 lines · On Fri, Oct 09, 2020 at 12:42:55PM -0700, [email protected] wrote:
(quoted diff) +1
-1@@ -238,7 +236,7 @@ noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) rcu_nmi_exit(); lockdep_hardirq_exit(); - if (restore) + if (irq_state->exit_rcu) lockdep_hardirqs_on(CALLER_ADDR0); __nmi_exit(); }That's not nice.. The NMI path is different from the IRQ path and has a different variable. Yes, this works, but *groan*.
Quoted 2 lines
Maybe union them if you want to avoid bloating the structure, but the above makes it really hard to read.
Right, and also that nmi entry thing should not be in x86. Something like the untested below as first cleanup.
Thanks,
tglx
----
Subject: x86/entry: Move nmi entry/exit into common code
From: Thomas Gleixner <[email protected]>
Date: Fri, 11 Sep 2020 10:09:56 +0200
Add blurb here.
--- arch/x86/entry/common.c | 34 ---------------------------------- arch/x86/include/asm/idtentry.h | 3 --- arch/x86/kernel/cpu/mce/core.c | 6 +++--- arch/x86/kernel/nmi.c | 6 +++--- arch/x86/kernel/traps.c | 13 +++++++------ include/linux/entry-common.h | 20 ++++++++++++++++++++ kernel/entry/common.c | 36 ++++++++++++++++++++++++++++++++++++ 7 files changed, 69 insertions(+), 49 deletions(-)
arch/x86/entry/common.c +0 -34
@@ -209,40 +209,6 @@ SYSCALL_DEFINE0(ni_syscall)return -ENOSYS ; } -noinstr bool idtentry_enter_nmi(struct pt_regs *regs) -{ -bool irq_state =lockdep_hardirqs_enabled (); - -__nmi_enter (); -lockdep_hardirqs_off (CALLER_ADDR0 ); -lockdep_hardirq_enter (); -rcu_nmi_enter (); - -instrumentation_begin (); -trace_hardirqs_off_finish (); -ftrace_nmi_enter (); -instrumentation_end (); - -return irq_state; -} - -noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) -{ -instrumentation_begin (); -ftrace_nmi_exit (); -if (restore) { -trace_hardirqs_on_prepare (); -lockdep_hardirqs_on_prepare (CALLER_ADDR0 ); - } -instrumentation_end (); - -rcu_nmi_exit (); -lockdep_hardirq_exit (); -if (restore) -lockdep_hardirqs_on (CALLER_ADDR0 ); -__nmi_exit (); -} -#ifdef CONFIG_XEN_PV #ifndef CONFIG_PREEMPTION /*
arch/x86/include/asm/idtentry.h +0 -3
@@ -11,9 +11,6 @@#include <asm/irq_stack.h> -bool idtentry_enter_nmi (struct pt_regs *regs); -void idtentry_exit_nmi (struct pt_regs *regs,bool irq_state); -/** * DECLARE_IDTENTRY - Declare functions for simple IDT entry points * No error code pushed by hardware
arch/x86/kernel/cpu/mce/core.c +3 -3
@@ -1983,7 +1983,7 @@ void (*machine_check_vector)(struct pt_r static __always_inline void exc_machine_check_kernel(struct pt_regs *regs) { -bool irq_state; +irqentry_state_t irq_state;WARN_ON_ONCE (user_mode (regs));
@@ -1995,7 +1995,7 @@ static __always_inline void exc_machine_ mce_check_crashing_cpu())return ; - irq_state =idtentry_enter_nmi (regs); + irq_state =irqentry_nmi_enter (regs);/* * The call targets are marked noinstr, but objtool can't figure * that out because it's an indirect call. Annotate it.
@@ -2006,7 +2006,7 @@ static __always_inline void exc_machine_if (regs->flags &X86_EFLAGS_IF )trace_hardirqs_on_prepare ();instrumentation_end (); -idtentry_exit_nmi (regs, irq_state); +irqentry_nmi_exit (regs, irq_state); } static __always_inline void exc_machine_check_user(struct pt_regs *regs)
arch/x86/kernel/nmi.c +3 -3
@@ -475,7 +475,7 @@ static DEFINE_PER_CPU(unsigned long, nmiDEFINE_IDTENTRY_RAW (exc_nmi) { -bool irq_state; +irqentry_state_t irq_state;/* * Re-enable NMIs right here when running as an SEV-ES guest. This might
@@ -502,14 +502,14 @@ DEFINE_IDTENTRY_RAW(exc_nmi)this_cpu_write (nmi_dr7,local_db_save ()); - irq_state =idtentry_enter_nmi (regs); + irq_state =irqentry_nmi_enter (regs);inc_irq_stat (__nmi_count);if (!ignore_nmis)default_do_nmi (regs); -idtentry_exit_nmi (regs, irq_state); +irqentry_nmi_exit (regs, irq_state);local_db_restore (this_cpu_read (nmi_dr7));
arch/x86/kernel/traps.c +7 -6
@@ -405,7 +405,7 @@ DEFINE_IDTENTRY_DF(exc_double_fault) }#endif -idtentry_enter_nmi (regs); +irqentry_nmi_enter (regs);instrumentation_begin ();notify_die (DIE_TRAP , str, regs, error_code,X86_TRAP_DF ,SIGSEGV );
@@ -651,12 +651,13 @@ DEFINE_IDTENTRY_RAW(exc_int3)instrumentation_end ();irqentry_exit_to_user_mode (regs); }else { -bool irq_state =idtentry_enter_nmi (regs); +irqentry_state_t irq_state =irqentry_nmi_enter (regs); +instrumentation_begin ();if (!do_int3 (regs))die ("int3" , regs,0 );instrumentation_end (); -idtentry_exit_nmi (regs, irq_state); +irqentry_nmi_exit (regs, irq_state); } }
@@ -864,7 +865,7 @@ static __always_inline void exc_debug_ke* includes the entry stack is excluded for everything. */ unsigned long dr7 =local_db_save (); -bool irq_state =idtentry_enter_nmi (regs); +irqentry_state_t irq_state =irqentry_nmi_enter (regs);instrumentation_begin ();/*
@@ -907,7 +908,7 @@ static __always_inline void exc_debug_ke regs->flags &= ~X86_EFLAGS_TF ; out:instrumentation_end (); -idtentry_exit_nmi (regs, irq_state); +irqentry_nmi_exit (regs, irq_state);local_db_restore (dr7); }
@@ -925,7 +926,7 @@ static __always_inline void exc_debug_us/* * NB: We can't easily clear DR7 here because -* idtentry_exit_to_usermode() can invoke ptrace, schedule, access +* irqentry_exit_to_usermode() can invoke ptrace, schedule, access * user memory, etc. This means that a recursive #DB is possible. If * this happens, that #DB will hit exc_debug_kernel() and clear DR7. * Since we're not on the IST stack right now, everything will be
include/linux/entry-common.h +20 -0
@@ -343,6 +343,7 @@ void irqentry_exit_to_user_mode(struct p#ifndef irqentry_statetypedef struct irqentry_state {bool exit_rcu; +bool lockdep; }irqentry_state_t ;#endif
@@ -402,4 +403,23 @@ void irqentry_exit_cond_resched(void);*/ void noinstr irqentry_exit(struct pt_regs *regs, irqentry_state_t state); +/** +* irqentry_nmi_enter - Handle NMI entry +* @regs: Pointer to currents pt_regs +* +* Similar to irqentry_enter() but taking care of the NMI constraints. +*/ +irqentry_state_t noinstr irqentry_nmi_enter(struct pt_regs *regs); + +/** +* irqentry_nmi_exit - Handle return from NMI handling +* @regs: Pointer to pt_regs (NMI entry regs) +* @state: Return value from matching call to irqentry_nmi_enter() +* +* Last action before returning to the low level assmenbly code. +* +* Counterpart to irqentry_nmi_enter(). +*/ +void noinstr irqentry_nmi_exit(struct pt_regs *regs, irqentry_state_t irq_state); +#endif
kernel/entry/common.c +36 -0
@@ -398,3 +398,39 @@ noinstr void irqentry_exit(struct pt_regrcu_irq_exit (); } } + +irqentry_state_t noinstr irqentry_nmi_enter(struct pt_regs *regs) +{ +irqentry_state_t irq_state; + + irq_state.lockdep =lockdep_hardirqs_enabled (); + +__nmi_enter (); +lockdep_hardirqs_off (CALLER_ADDR0 ); +lockdep_hardirq_enter (); +rcu_nmi_enter (); + +instrumentation_begin (); +trace_hardirqs_off_finish (); +ftrace_nmi_enter (); +instrumentation_end (); + +return irq_state; +} + +void noinstr irqentry_nmi_exit(struct pt_regs *regs, irqentry_state_t irq_state) +{ +instrumentation_begin (); +ftrace_nmi_exit (); +if (irq_state.lockdep) { +trace_hardirqs_on_prepare (); +lockdep_hardirqs_on_prepare (CALLER_ADDR0 ); + } +instrumentation_end (); + +rcu_nmi_exit (); +lockdep_hardirq_exit (); +if (irq_state.lockdep) +lockdep_hardirqs_on (CALLER_ADDR0 ); +__nmi_exit (); +}
Re: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 15 lines · Thomas Gleixner
Quoted 14 lines · Peter Zijlstra
Review noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) · line 242
Quoted 9 lines · On Fri, Oct 09, 2020 at 12:42:55PM -0700, [email protected] wrote:
(quoted diff) +1
-1@@ -238,7 +236,7 @@ noinstr void idtentry_exit_nmi(struct pt_regs *regs, bool restore) rcu_nmi_exit(); lockdep_hardirq_exit(); - if (restore) + if (irq_state->exit_rcu) lockdep_hardirqs_on(CALLER_ADDR0); __nmi_exit(); }That's not nice.. The NMI path is different from the IRQ path and has a different variable. Yes, this works, but *groan*.
Quoted 5 lines
Quoted 2 lines
Maybe union them if you want to avoid bloating the structure, but the above makes it really hard to read.
Right, and also that nmi entry thing should not be in x86. Something like the untested below as first cleanup.
Ok, I see what Peter was talking about. I've added this patch to the series.
Quoted 10 lines
Thanks,
tglx
----
Subject: x86/entry: Move nmi entry/exit into common code
From: Thomas Gleixner <[email protected]>
Date: Fri, 11 Sep 2020 10:09:56 +0200Add blurb here.
How about:
To prepare for saving PKRS values across NMI's we lift the idtentry_[enter|exit]_nmi() to the common code. Rename them to irqentry_nmi_[enter|exit]() to reflect the new generic nature and store the state in the same irqentry_state_t structure as the other irqentry_*() functions. Finally, differentiate the state being stored between the NMI and IRQ path by adding 'lockdep' to irqentry_state_t.
?
Quoted 12 lines
Signed-off-by Thomas Gleixner <[email protected]>--- arch/x86/entry/common.c | 34 ---------------------------------- arch/x86/include/asm/idtentry.h | 3 --- arch/x86/kernel/cpu/mce/core.c | 6 +++--- arch/x86/kernel/nmi.c | 6 +++--- arch/x86/kernel/traps.c | 13 +++++++------ include/linux/entry-common.h | 20 ++++++++++++++++++++ kernel/entry/common.c | 36 ++++++++++++++++++++++++++++++++++++ 7 files changed, 69 insertions(+), 49 deletions(-)
[snip]
include/linux/entry-common.h void irqentry_exit_to_user_mode(struct p · line 347
Quoted 8 lines
include/linux/entry-common.h +1
-0@@ -343,6 +343,7 @@ void irqentry_exit_to_user_mode(struct p#ifndef irqentry_statetypedef struct irqentry_state {bool exit_rcu; +bool lockdep; }irqentry_state_t ;
Building on what Peter said do you agree this should be made into a union?
It may not be strictly necessary in this patch but I think it would reflect the mutual exclusivity better and could be changed easy enough in the follow on patch which adds the pkrs state.
Ira
Re: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 7 lines · Ira Weiny
Quoted 5 lines · Thomas Gleixner
Subject: x86/entry: Move nmi entry/exit into common code
From: Thomas Gleixner <[email protected]>
Date: Fri, 11 Sep 2020 10:09:56 +0200Add blurb here.
Quoted 8 lines
How about:
To prepare for saving PKRS values across NMI's we lift the idtentry_[enter|exit]_nmi() to the common code. Rename them to irqentry_nmi_[enter|exit]() to reflect the new generic nature and store the state in the same
irqentry_state_tstructure as the other irqentry_*() functions. Finally, differentiate the state being stored between the NMI and IRQ path by adding 'lockdep' toirqentry_state_t.
No. This has absolutely nothing to do with PKRS. It's a cleanup valuable by itself and that's how it should have been done right away.
So the proper changelog is:
Lockdep state handling on NMI enter and exit is nothing specific to X86. It's not any different on other architectures. Also the extra state type is not necessary, irqentry_state_t can carry the necessary information as well.
Move it to common code and extend irqentry_state_t to carry lockdep state.
Quoted 11 lines
include/linux/entry-common.h void irqentry_exit_to_user_mode(struct p · line 347
Quoted 8 lines
include/linux/entry-common.h +1
-0@@ -343,6 +343,7 @@ void irqentry_exit_to_user_mode(struct p#ifndef irqentry_statetypedef struct irqentry_state {bool exit_rcu; +bool lockdep; }irqentry_state_t ;Building on what Peter said do you agree this should be made into a union?
Quoted 3 lines
It may not be strictly necessary in this patch but I think it would reflect the mutual exclusivity better and could be changed easy enough in the follow on patch which adds the pkrs state.
Why the heck should it be changed in a patch which adds something completely different?
Either it's mutually exclusive or not and if so it want's to be done in this patch and not in a change which extends the struct for other reasons.
Thanks,
tglx
Re: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 22 lines · Thomas Gleixner
Quoted 7 lines · Ira Weiny
Quoted 5 lines · Thomas Gleixner
Subject: x86/entry: Move nmi entry/exit into common code
From: Thomas Gleixner <[email protected]>
Date: Fri, 11 Sep 2020 10:09:56 +0200Add blurb here.
Quoted 8 lines
How about:
To prepare for saving PKRS values across NMI's we lift the idtentry_[enter|exit]_nmi() to the common code. Rename them to irqentry_nmi_[enter|exit]() to reflect the new generic nature and store the state in the same
irqentry_state_tstructure as the other irqentry_*() functions. Finally, differentiate the state being stored between the NMI and IRQ path by adding 'lockdep' toirqentry_state_t.No. This has absolutely nothing to do with PKRS. It's a cleanup valuable by itself and that's how it should have been done right away.
So the proper changelog is:
Quoted 7 lines
Lockdep state handling on NMI enter and exit is nothing specific to X86. It's not any different on other architectures. Also the extra state type is not necessary,
irqentry_state_tcan carry the necessary information as well.Move it to common code and extend
irqentry_state_tto carry lockdep state.
Ok sounds good, thanks.
Quoted 12 lines
Quoted 12 lines
include/linux/entry-common.h void irqentry_exit_to_user_mode(struct p · line 347
Quoted 9 lines
include/linux/entry-common.h +1
-0@@ -343,6 +343,7 @@ void irqentry_exit_to_user_mode(struct p#ifndef irqentry_statetypedef struct irqentry_state {bool exit_rcu; +bool lockdep; }irqentry_state_t ;Building on what Peter said do you agree this should be made into a union?
Quoted 6 lines
Quoted 3 lines
It may not be strictly necessary in this patch but I think it would reflect the mutual exclusivity better and could be changed easy enough in the follow on patch which adds the pkrs state.
Why the heck should it be changed in a patch which adds something completely different?
Because the PKRS stuff is used in both NMI and IRQ state.
Quoted 4 lines
Either it's mutually exclusive or not and if so it want's to be done in this patch and not in a change which extends the struct for other reasons.
Sorry, let me clarify. After this patch we have.
typedef union irqentry_state {
bool exit_rcu;
bool lockdep;
} irqentry_state_t;Which reflects the mutual exclusion of the 2 variables.
But then when the pkrs stuff is added the union changes back to a structure and looks like this.
typedef struct irqentry_state {
#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS
u32 pkrs;
u32 thread_pkrs;
#endif
union {
bool exit_rcu;
bool lockdep;
};
} irqentry_state_t;Because the pkrs information is in addition to exit_rcu OR lockdep.
So this is what I meant by 'could be changed easy enough in the follow on patch'.
Is that clear?
Ira
Re: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 9 lines · Ira Weiny
On Mon, Oct 19, 2020 at 11:32:50AM +0200, Thomas Gleixner wrote:
Sorry, let me clarify. After this patch we have.typedef union irqentry_state { bool exit_rcu; bool lockdep; } irqentry_state_t;Which reflects the mutual exclusion of the 2 variables.
Huch? From the patch I gave you:
#ifndef irqentry_state
typedef struct irqentry_state {
bool exit_rcu;
+ bool lockdep;
} irqentry_state_t;
#endif
How is that a union?
Quoted 2 lines
But then when the pkrs stuff is added the union changes back to a structure and looks like this.
So you want:
1) Move stuff to struct irqentry_state (my patch)
2) Change it to a union and pass it as pointer at the same time
3) Change it back to struct to add PKRS
Quoted 1 line
Is that clear?
What's clear is that the above is nonsense. We can just do
#ifndef irqentry_state
typedef struct irqentry_state {
union {
bool exit_rcu;
bool lockdep;
}; } irqentry_state_t;
#endif
right in the patch which I gave you. Because that actually makes sense.
Thanks,
tglx
Re: [PATCH RFC V3 6/9] x86/entry: Pass irqentry_state_t by reference
Quoted 20 lines · Thomas Gleixner
Quoted 9 lines · Ira Weiny
On Mon, Oct 19, 2020 at 11:32:50AM +0200, Thomas Gleixner wrote:
Sorry, let me clarify. After this patch we have.typedef union irqentry_state { bool exit_rcu; bool lockdep; } irqentry_state_t;Which reflects the mutual exclusion of the 2 variables.
Huch? From the patch I gave you:
#ifndef
irqentry_state
typedef structirqentry_state{include/linux/entry-common.h +1
-0bool exit_rcu; + bool lockdep; } irqentry_state_t;#endif
Quoted 1 line
How is that a union?
I was proposing to make it a union.
Quoted 9 lines
Quoted 3 lines
But then when the pkrs stuff is added the union changes back to a structure and looks like this.
So you want:
1) Move stuff to struct
irqentry_state(my patch)2) Change it to a union and pass it as pointer at the same time
No, I would have made it a union in your patch.
Pass by reference would remain largely the same.
Quoted 2 lines
3) Change it back to struct to add PKRS
Yes. :-/
Quoted 7 lines
Quoted 2 lines
Is that clear?
What's clear is that the above is nonsense. We can just do
#ifndef
irqentry_state
typedef structirqentry_state{
Quoted 8 lines
union { bool exit_rcu; bool lockdep; };}
irqentry_state_t;
#endifright in the patch which I gave you. Because that actually makes sense.
Ok I'm very sorry. I was thinking that having a struct containing nothing but an anonymous union would be unacceptable as a stand alone item in your patch. In my experience other maintainers would have rejected such a change and would have asked; 'why not just make it a union'?
I'm very happy skipping the gymnastics on individual patches in favor of making the whole series work out in the end.
Thank you for your help again. :-)
Ira
RFC PATCH v3 7/9 x86/entry: Preserve PKRS MSR across exceptions
In reply to [email protected]From: Ira Weiny <[email protected]>
The PKRS MSR is not managed by XSAVE. It is preserved through a context switch but this support leaves exception handling code open to memory accesses during exceptions.
2 possible places for preserving this state were considered, irqentry_state_t or pt_regs.[1] pt_regs was much more complicated and was potentially fraught with unintended consequences.[2] irqentry_state_t was already an object being used in the exception handling and is straightforward. It is also easy for any number of nested states to be tracked and eventually can be enhanced to store the reference counting required to support PKS through kmap reentry
Preserve the current task's PKRS values in irqentry_state_t on exception entry and restoring them on exception exit.
Each nested exception is further saved allowing for any number of levels of exception handling.
Peter and Thomas both suggested parts of the patch, IDT and NMI respectively.
[1] https://lore.kernel.org/lkml/CALCETrVe1i5JdyzD_BcctxQJn+ZE3T38EFPgjxN1F577M36g+w@mail.gmail.com/
[2] https://lore.kernel.org/lkml/[email protected]/#t
--- arch/x86/entry/common.c | 43 +++++++++++++++++++++++++++++ arch/x86/include/asm/pkeys_common.h | 5 ++-- arch/x86/kernel/cpu/mce/core.c | 4 +++ arch/x86/mm/pkeys.c | 2 +- include/linux/entry-common.h | 12 ++++++++ kernel/entry/common.c | 12 ++++++-- 6 files changed, 73 insertions(+), 5 deletions(-)
arch/x86/entry/common.c +43 -0
@@ -19,6 +19,7 @@#include <linux/nospec.h> #include <linux/syscalls.h> #include <linux/uaccess.h> +#include <linux/pkeys.h> #ifdef CONFIG_XEN_PV #include <xen/xen-ops.h>
@@ -222,6 +223,8 @@ noinstr void idtentry_enter_nmi(struct pt_regs *regs, irqentry_state_t *irq_stattrace_hardirqs_off_finish ();ftrace_nmi_enter ();instrumentation_end (); + +irq_save_pkrs (irq_state); } noinstr void idtentry_exit_nmi(struct pt_regs *regs, irqentry_state_t *irq_state)
@@ -238,9 +241,47 @@ noinstr void idtentry_exit_nmi(struct pt_regs *regs, irqentry_state_t *irq_statelockdep_hardirq_exit ();if (irq_state->exit_rcu)lockdep_hardirqs_on (CALLER_ADDR0 ); + +irq_restore_pkrs (irq_state);__nmi_exit (); } +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +/* +* PKRS is a per-logical-processor MSR which overlays additional protection for +* pages which have been mapped with a protection key. +* +* The register is not maintained with XSAVE so we have to maintain the MSR +* value in software during context switch and exception handling. +* +* Context switches save the MSR in the task struct thus taking that value to +* other processors if necessary. +* +* To protect against exceptions having access to this memory we save the +* current running value and set the default PKRS value for the duration of the +* exception. Thus preventing exception handlers from having the elevated +* access of the interrupted task. +*/ +noinstr void irq_save_pkrs(irqentry_state_t *state) +{ +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ; + + state->thread_pkrs = current->thread.saved_pkrs; + state->pkrs =this_cpu_read (pkrs_cache); +write_pkrs (INIT_PKRS_VALUE ); +} + +noinstr void irq_restore_pkrs(irqentry_state_t *state) +{ +if (!cpu_feature_enabled (X86_FEATURE_PKS )) +return ; + +write_pkrs (state->pkrs); + current->thread.saved_pkrs = state->thread_pkrs; +} +#endif /* CONFIG_ARCH_HAS_SUPERVISOR_PKEYS */ +#ifdef CONFIG_XEN_PV #ifndef CONFIG_PREEMPTION /*
@@ -304,6 +345,8 @@ __visible noinstr void xen_pv_evtchn_do_upcall(struct pt_regs *regs) inhcall =get_and_clear_inhcall ();if (inhcall && !WARN_ON_ONCE (state.exit_rcu)) { +/* Normally called by irqentry_exit, we must restore pkrs here */ +irq_restore_pkrs (&state);instrumentation_begin ();irqentry_exit_cond_resched ();instrumentation_end ();
arch/x86/include/asm/pkeys_common.h +3 -2
@@ -27,9 +27,10 @@#define PKS_NUM_KEYS 16#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS -void write_pkrs (u32 new_pkrs); +DECLARE_PER_CPU (u32, pkrs_cache); +noinstr void write_pkrs(u32 new_pkrs);#else -static inline void write_pkrs (u32 new_pkrs) { } +static __always_inline void write_pkrs(u32 new_pkrs) { }#endif #endif /*_ASM_X86_PKEYS_INTERNAL_H */
arch/x86/kernel/cpu/mce/core.c +4 -0
@@ -1904,6 +1904,8 @@ void (*machine_check_vector)(struct pt_regs *) = unexpected_machine_check; static __always_inline void exc_machine_check_kernel(struct pt_regs *regs) { +irqentry_state_t irq_state = { }; +WARN_ON_ONCE (user_mode (regs));/*
@@ -1915,6 +1917,7 @@ static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)return ;nmi_enter (); +irq_save_pkrs (&irq_state);/* * The call targets are marked noinstr, but objtool can't figure * that out because it's an indirect call. Annotate it.
@@ -1925,6 +1928,7 @@ static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)if (regs->flags &X86_EFLAGS_IF )trace_hardirqs_on_prepare ();instrumentation_end (); +irq_restore_pkrs (&irq_state);nmi_exit (); }
arch/x86/mm/pkeys.c +1 -1
@@ -247,7 +247,7 @@ DEFINE_PER_CPU(u32, pkrs_cache);* until all prior executions of WRPKRU have completed execution * and updated the PKRU register. */ -void write_pkrs (u32 new_pkrs) +noinstr void write_pkrs(u32 new_pkrs) {u32 *pkrs;
include/linux/entry-common.h +12 -0
@@ -342,10 +342,22 @@ void irqentry_exit_to_user_mode(struct pt_regs *regs);#ifndef irqentry_statetypedef struct irqentry_state { +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +u32 pkrs; +u32 thread_pkrs; +#endif bool exit_rcu; }irqentry_state_t ;#endif +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +noinstr void irq_save_pkrs(irqentry_state_t *state); +noinstr void irq_restore_pkrs(irqentry_state_t *state); +#else +static __always_inline void irq_save_pkrs(irqentry_state_t *state) { } +static __always_inline void irq_restore_pkrs(irqentry_state_t *state) { } +#endif +/** * irqentry_enter - Handle state tracking on ordinary interrupt entries * @regs: Pointer to pt_regs of interrupted context
kernel/entry/common.c +10 -2
@@ -327,7 +327,7 @@ noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state)instrumentation_end (); state->exit_rcu = true; -return ; + goto done; }/*
@@ -341,6 +341,9 @@ noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state)/* Use the combo lockdep/tracing function */ trace_hardirqs_off ();instrumentation_end (); + +done: +irq_save_pkrs (state); }void irqentry_exit_cond_resched (void )
@@ -362,7 +365,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t *state)/* Check whether this returns to user mode */ if (user_mode (regs)) {irqentry_exit_to_user_mode (regs); - }else if (!regs_irqs_disabled (regs)) { +return ; + } + +irq_restore_pkrs (state); + +if (!regs_irqs_disabled (regs)) {/* * If RCU was not watching on entry this needs to be done * carefully and needs the same ordering of lockdep/tracing
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 7/9] x86/entry: Preserve PKRS MSR across exceptions
kernel/entry/common.c noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state) · line 347
Quoted 8 lines · On 10/9/20 12:42 PM, [email protected] wrote:
kernel/entry/common.c +3
-0@@ -341,6 +341,9 @@ noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state)/* Use the combo lockdep/tracing function */ trace_hardirqs_off ();instrumentation_end (); + +done: +irq_save_pkrs (state); }
One nit: This saves *and* sets PKRS. It's not obvious from the call here that PKRS is altered at this site. Seems like there could be a better name.
Even if we did:
irq_save_set_pkrs(state, INIT_VAL);
It would probably compile down to the same thing, but be *really* obvious what's going on.
Quoted 16 lines
kernel/entry/common.c +6
-1void irqentry_exit_cond_resched (void )@@ -362,7 +365,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t *state)/* Check whether this returns to user mode */ if (user_mode (regs)) {irqentry_exit_to_user_mode (regs); - }else if (!regs_irqs_disabled (regs)) { +return ; + } + +irq_restore_pkrs (state); + +if (!regs_irqs_disabled (regs)) {/* * If RCU was not watching on entry this needs to be done * carefully and needs the same ordering of lockdep/tracing
Re: [PATCH RFC V3 7/9] x86/entry: Preserve PKRS MSR across exceptions
Quoted 14 lines · Dave Hansen
kernel/entry/common.c noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state) · line 347
Quoted 8 lines · On 10/9/20 12:42 PM, [email protected] wrote:
kernel/entry/common.c +3
-0@@ -341,6 +341,9 @@ noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state)/* Use the combo lockdep/tracing function */ trace_hardirqs_off ();instrumentation_end (); + +done: +irq_save_pkrs (state); }One nit: This saves *and* sets PKRS. It's not obvious from the call here that PKRS is altered at this site. Seems like there could be a better name.
Quoted 6 lines
Even if we did:
irq_save_set_pkrs(state, INIT_VAL);It would probably compile down to the same thing, but be *really* obvious what's going on.
I suppose that is true. But I think it is odd having a parameter which is the same for every call site.
But I'm not going to quibble over something like this.
Changed,
Ira
Quoted 18 lines
Quoted 17 lines
kernel/entry/common.c +6
-1void irqentry_exit_cond_resched (void )@@ -362,7 +365,12 @@ noinstr void irqentry_exit(struct pt_regs *regs, irqentry_state_t *state)/* Check whether this returns to user mode */ if (user_mode (regs)) {irqentry_exit_to_user_mode (regs); - }else if (!regs_irqs_disabled (regs)) { +return ; + } + +irq_restore_pkrs (state); + +if (!regs_irqs_disabled (regs)) {/* * If RCU was not watching on entry this needs to be done * carefully and needs the same ordering of lockdep/tracing
Re: [PATCH RFC V3 7/9] x86/entry: Preserve PKRS MSR across exceptions
Quoted 14 lines · Ira Weiny
Quoted 13 lines · Dave Hansen
kernel/entry/common.c noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state) · line 347
Quoted 8 lines · On 10/9/20 12:42 PM, [email protected] wrote:
kernel/entry/common.c +3
-0@@ -341,6 +341,9 @@ noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state)/* Use the combo lockdep/tracing function */ trace_hardirqs_off ();instrumentation_end (); + +done: +irq_save_pkrs (state); }One nit: This saves *and* sets PKRS. It's not obvious from the call here that PKRS is altered at this site. Seems like there could be a better name.
Quoted 8 lines
Quoted 6 lines
Even if we did:
irq_save_set_pkrs(state, INIT_VAL);It would probably compile down to the same thing, but be *really* obvious what's going on.
I suppose that is true. But I think it is odd having a parameter which is the same for every call site.
Well, it depends on what you optimize for. I'm trying to optimize for the code being understood quickly the first time someone reads it. To me, that's more important than minimizing the number of function parameters (which are essentially free).
Re: [PATCH RFC V3 7/9] x86/entry: Preserve PKRS MSR across exceptions
Quoted 21 lines · Dave Hansen
Quoted 20 lines · Ira Weiny
Quoted 19 lines · Dave Hansen
kernel/entry/common.c noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state) · line 347
Quoted 8 lines · On 10/9/20 12:42 PM, [email protected] wrote:
kernel/entry/common.c +3
-0@@ -341,6 +341,9 @@ noinstr void irqentry_enter(struct pt_regs *regs, irqentry_state_t *state)/* Use the combo lockdep/tracing function */ trace_hardirqs_off ();instrumentation_end (); + +done: +irq_save_pkrs (state); }One nit: This saves *and* sets PKRS. It's not obvious from the call here that PKRS is altered at this site. Seems like there could be a better name.
Even if we did:
irq_save_set_pkrs(state, INIT_VAL);It would probably compile down to the same thing, but be *really* obvious what's going on.
Quoted 8 lines
Quoted 2 lines
I suppose that is true. But I think it is odd having a parameter which is the same for every call site.
Well, it depends on what you optimize for. I'm trying to optimize for the code being understood quickly the first time someone reads it. To me, that's more important than minimizing the number of function parameters (which are essentially free).
Agreed. Sorry I was not trying to be confrontational. There is just enough other things which are going to take me time to get right I need to focus on them... :-D
Sorry,
Ira
RFC PATCH v3 8/9 x86/fault: Report the PKRS state on fault
In reply to [email protected]From: Ira Weiny <[email protected]>
When only user space pkeys are enabled faulting within the kernel was an unexpected condition which should never happen, therefore a WARN_ON was added to the kernel fault handler to detect if it ever did. Now that PKS can be enabled this is no longer the case.
Report a Pkey fault with a normal splat and add the PKRS state to the fault splat text. Note the PKS register is reset during an exception therefore the saved PKRS value from before the beginning of the exception is passed down.
If PKS is not enabled, or not active, maintain the WARN_ON_ONCE() from before.
Because each fault has its own state the pkrs information will be correctly reported even if a fault 'faults'.
--- arch/x86/mm/fault.c | 59 ++++++++++++++++++++++++++------------------- 1 file changed, 34 insertions(+), 25 deletions(-)
arch/x86/mm/fault.c +34 -25
@@ -504,7 +504,8 @@ static void show_ldttss(const struct desc_ptr *gdt, const char *name, u16 index) }static void -show_fault_oops (struct pt_regs *regs,unsigned long error_code,unsigned long address) +show_fault_oops (struct pt_regs *regs,unsigned long error_code,unsigned long address, +irqentry_state_t *irq_state) {if (!oops_may_print ())return ;
@@ -548,6 +549,11 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad (error_code &X86_PF_PK ) ?"protection keys violation" : "permissions violation"); +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +if (irq_state && (error_code &X86_PF_PK )) +pr_alert ("PKRS: 0x%x\n" , irq_state->pkrs); +#endif +if (!(error_code &X86_PF_USER ) &&user_mode (regs)) {struct desc_ptr idt, gdt;u16 ldtr, tr;
@@ -626,7 +632,8 @@ static void set_signal_archinfo(unsigned long address, static noinline voidno_context (struct pt_regs *regs,unsigned long error_code, -unsigned long address,int signal,int si_code) +unsigned long address,int signal,int si_code, +irqentry_state_t *irq_state) {struct task_struct *tsk = current;unsigned long flags;
@@ -732,7 +739,7 @@ no_context(struct pt_regs *regs, unsigned long error_code,*/ flags =oops_begin (); -show_fault_oops (regs, error_code, address); +show_fault_oops (regs, error_code, address, irq_state);if (task_stack_end_corrupted (tsk))printk (KERN_EMERG "Thread overran stack, or stack corrupted\n" );
@@ -785,7 +792,8 @@ static bool is_vsyscall_vaddr(unsigned long vaddr)static void __bad_area_nosemaphore (struct pt_regs *regs,unsigned long error_code, -unsigned long address,u32 pkey,int si_code) +unsigned long address,u32 pkey,int si_code, +irqentry_state_t *state) {struct task_struct *tsk = current;
@@ -832,14 +840,14 @@ __bad_area_nosemaphore(struct pt_regs *regs, unsigned long error_code,if (is_f00f_bug (regs, address))return ; -no_context (regs, error_code, address,SIGSEGV , si_code); +no_context (regs, error_code, address,SIGSEGV , si_code, state); } static noinline voidbad_area_nosemaphore (struct pt_regs *regs,unsigned long error_code, -unsigned long address) +unsigned long address,irqentry_state_t *state) { -__bad_area_nosemaphore (regs, error_code, address,0 ,SEGV_MAPERR ); +__bad_area_nosemaphore (regs, error_code, address,0 ,SEGV_MAPERR , state); } static void
@@ -853,7 +861,7 @@ __bad_area(struct pt_regs *regs, unsigned long error_code,*/ mmap_read_unlock (mm); -__bad_area_nosemaphore (regs, error_code, address, pkey, si_code); +__bad_area_nosemaphore (regs, error_code, address, pkey, si_code,NULL ); }static noinline void
@@ -923,7 +931,7 @@ do_sigbus(struct pt_regs *regs, unsigned long error_code, unsigned long address, {/* Kernel mode? Handle exceptions or die: */ if (!(error_code &X86_PF_USER )) { -no_context (regs, error_code, address,SIGBUS ,BUS_ADRERR ); +no_context (regs, error_code, address,SIGBUS ,BUS_ADRERR ,NULL );return ; }
@@ -957,7 +965,7 @@ mm_fault_error(struct pt_regs *regs, unsigned long error_code, unsigned long address, vm_fault_t fault) { if (fatal_signal_pending(current) && !(error_code & X86_PF_USER)) { - no_context(regs, error_code, address, 0, 0); + no_context(regs, error_code, address, 0, 0, NULL);return ; }
@@ -965,7 +973,7 @@ mm_fault_error(struct pt_regs *regs, unsigned long error_code,/* Kernel mode? Handle exceptions or die: */ if (!(error_code &X86_PF_USER )) {no_context (regs, error_code, address, -SIGSEGV ,SEGV_MAPERR ); +SIGSEGV ,SEGV_MAPERR ,NULL );return ; }
@@ -980,7 +988,7 @@ mm_fault_error(struct pt_regs *regs, unsigned long error_code, VM_FAULT_HWPOISON_LARGE))do_sigbus (regs ,error_code ,address ,fault ); else if (fault & VM_FAULT_SIGSEGV) - bad_area_nosemaphore(regs, error_code, address); + bad_area_nosemaphore(regs, error_code, address, NULL); elseBUG (); }
@@ -1148,14 +1156,15 @@ static int fault_in_kernel_space(unsigned long address)*/ static void do_kern_addr_fault (struct pt_regs *regs,unsigned long hw_error_code, -unsigned long address) +unsigned long address,irqentry_state_t *irq_state) {/* -* Protection keys exceptions only happen on user pages. We -* have no user pages in the kernel portion of the address -* space, so do not expect them here. +* If protection keys are not enabled for kernel space +* do not expect Pkey errors here. */ -WARN_ON_ONCE (hw_error_code &X86_PF_PK ); +if (!IS_ENABLED (CONFIG_ARCH_HAS_SUPERVISOR_PKEYS ) || + !cpu_feature_enabled (X86_FEATURE_PKS )) +WARN_ON_ONCE (hw_error_code &X86_PF_PK );#ifdef CONFIG_X86_32 /*
@@ -1204,7 +1213,7 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code,* Don't take the mm semaphore here. If we fixup a prefetch * fault we could otherwise deadlock: */ -bad_area_nosemaphore (regs, hw_error_code, address); +bad_area_nosemaphore (regs, hw_error_code, address, irq_state); }NOKPROBE_SYMBOL (do_kern_addr_fault);
@@ -1245,7 +1254,7 @@ void do_user_addr_fault(struct pt_regs *regs, !(hw_error_code & X86_PF_USER) && !(regs->flags & X86_EFLAGS_AC))) { -bad_area_nosemaphore (regs, hw_error_code, address); +bad_area_nosemaphore (regs, hw_error_code, address,NULL );return ; }
@@ -1254,7 +1263,7 @@ void do_user_addr_fault(struct pt_regs *regs,* in a region with pagefaults disabled then we must not take the fault */ if (unlikely (faulthandler_disabled () || !mm)) { -bad_area_nosemaphore (regs, hw_error_code, address); +bad_area_nosemaphore (regs, hw_error_code, address,NULL );return ; }
@@ -1316,7 +1325,7 @@ void do_user_addr_fault(struct pt_regs *regs,* Fault from code in kernel from * which we do not expect faults. */ -bad_area_nosemaphore (regs, hw_error_code, address); +bad_area_nosemaphore (regs, hw_error_code, address,NULL );return ; } retry:
@@ -1375,7 +1384,7 @@ void do_user_addr_fault(struct pt_regs *regs,if (fault_signal_pending (fault, regs)) {if (!user_mode (regs))no_context (regs, hw_error_code, address,SIGBUS , -BUS_ADRERR ); +BUS_ADRERR ,NULL );return ; }
@@ -1415,7 +1424,7 @@ trace_page_fault_entries(struct pt_regs *regs, unsigned long error_code, static __always_inline voidhandle_page_fault (struct pt_regs *regs,unsigned long error_code, -unsigned long address) +unsigned long address,irqentry_state_t *irq_state) {trace_page_fault_entries (regs, error_code, address);
@@ -1424,7 +1433,7 @@ handle_page_fault(struct pt_regs *regs, unsigned long error_code,/* Was the fault on kernel-controlled part of the address space? */ if (unlikely (fault_in_kernel_space (address))) { -do_kern_addr_fault (regs, error_code, address); +do_kern_addr_fault (regs, error_code, address, irq_state); }else {do_user_addr_fault (regs, error_code, address);/*
@@ -1479,7 +1488,7 @@ DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault)irqentry_enter (regs, &state);instrumentation_begin (); -handle_page_fault (regs, error_code, address); +handle_page_fault (regs, error_code, address, &state);instrumentation_end ();irqentry_exit (regs, &state);
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 8/9] x86/fault: Report the PKRS state on fault
arch/x86/mm/fault.c show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad · line 555
Quoted 8 lines
arch/x86/mm/fault.c +4
-0@@ -548,6 +549,11 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad (error_code &X86_PF_PK ) ?"protection keys violation" : "permissions violation"); +#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +if (irq_state && (error_code &X86_PF_PK )) +pr_alert ("PKRS: 0x%x\n" , irq_state->pkrs); +#endif
This means everyone will see 'PKRS: 0x0', even if they're on non-PKS hardware. I think I'd rather have this only show PKRS when we're on cpu_feature_enabled(PKS) hardware.
...
Quoted 4 lines
arch/x86/mm/fault.c +0
-0@@ -1148,14 +1156,15 @@ static int fault_in_kernel_space(unsigned long address)*/ static void do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code,
arch/x86/mm/fault.c static int fault_in_kernel_space(unsigned long address) · line 1164
Quoted 10 lines
arch/x86/mm/fault.c +3
-4- unsigned long address) + unsigned long address, irqentry_state_t *irq_state) {/* -* Protection keys exceptions only happen on user pages. We -* have no user pages in the kernel portion of the address -* space, so do not expect them here. +* If protection keys are not enabled for kernel space +* do not expect Pkey errors here. */
Let's fix the double-negative:
/* * PF_PK is only expected on kernel addresses whenn * supervisor pkeys are enabled: */
arch/x86/mm/fault.c static int fault_in_kernel_space(unsigned long address) · line 1167
Quoted 4 lines
arch/x86/mm/fault.c +3
-1-WARN_ON_ONCE (hw_error_code &X86_PF_PK ); +if (!IS_ENABLED (CONFIG_ARCH_HAS_SUPERVISOR_PKEYS ) || + !cpu_feature_enabled (X86_FEATURE_PKS )) +WARN_ON_ONCE (hw_error_code &X86_PF_PK );
Yeah, please stick X86_FEATURE_PKS in disabled-features so you can use cpu_feature_enabled(X86_FEATURE_PKS) by itself here..
Re: [PATCH RFC V3 8/9] x86/fault: Report the PKRS state on fault
Quoted 4 lines · Dave Hansen
Quoted 4 lines
arch/x86/mm/fault.c +0
-0@@ -548,6 +549,11 @@ show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad (error_code &X86_PF_PK ) ?"protection keys violation" : "permissions violation");
Quoted 8 lines
arch/x86/mm/fault.c show_fault_oops(struct pt_regs *regs, unsigned long error_code, unsigned long ad · line 555
Quoted 4 lines
arch/x86/mm/fault.c +4
-0+#ifdef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS +if (irq_state && (error_code &X86_PF_PK )) +pr_alert ("PKRS: 0x%x\n" , irq_state->pkrs); +#endif This means everyone will see 'PKRS:
0x0', even if they're on non-PKS hardware. I think I'd rather have this only show PKRS when we're oncpu_feature_enabled(PKS)hardware.
Good catch, thanks.
Quoted 19 lines
...
Quoted 4 lines
arch/x86/mm/fault.c +0
-0@@ -1148,14 +1156,15 @@ static int fault_in_kernel_space(unsigned long address)*/ static void do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code,arch/x86/mm/fault.c static int fault_in_kernel_space(unsigned long address) · line 1164
Quoted 10 lines
arch/x86/mm/fault.c +3
-4- unsigned long address) + unsigned long address, irqentry_state_t *irq_state) {/* -* Protection keys exceptions only happen on user pages. We -* have no user pages in the kernel portion of the address -* space, so do not expect them here. +* If protection keys are not enabled for kernel space +* do not expect Pkey errors here. */ Let's fix the double-negative:
Quoted 4 lines
/* * PF_PK is only expected on kernel addresses whenn * supervisor pkeys are enabled: */
done. thanks.
Quoted 8 lines
arch/x86/mm/fault.c static int fault_in_kernel_space(unsigned long address) · line 1167
Quoted 5 lines
arch/x86/mm/fault.c +3
-1-WARN_ON_ONCE (hw_error_code &X86_PF_PK ); +if (!IS_ENABLED (CONFIG_ARCH_HAS_SUPERVISOR_PKEYS ) || + !cpu_feature_enabled (X86_FEATURE_PKS )) +WARN_ON_ONCE (hw_error_code &X86_PF_PK );Yeah, please stick
X86_FEATURE_PKSin disabled-features so you can usecpu_feature_enabled(X86_FEATURE_PKS)by itself here..
done.
thanks,
Ira
RFC PATCH v3 9/9 x86/pks: Add PKS test code
In reply to [email protected]From: Ira Weiny <[email protected]>
The core PKS functionality provides an interface for kernel users to reserve keys to their domains set up the page tables with those keys and control access to those domains when needed.
Define test code which exercises the core functionality of PKS via a debugfs entry. Basic checks can be triggered on boot with a kernel command line option while both basic and preemption checks can be triggered with separate debugfs values.
debugfs controls are:
'0' -- Run access tests with a single pkey
'1' -- Set up the pkey register with no access for the pkey allocated to
this fd
'2' -- Check that the pkey register updated in '1' is still the same.
(To be used after a forced context switch.)
'3' -- Allocate all pkeys possible and run tests on each pkey allocated.
DEFAULT when run at boot.
Closing the fd will cleanup and release the pkey, therefore to exercise context switch testing a user space program is provided in:
.../tools/testing/selftests/x86/test_pks.c
--- Documentation/core-api/protection-keys.rst | 1 + arch/x86/mm/fault.c | 23 + include/linux/pkeys.h | 1 - lib/Kconfig.debug | 12 + lib/Makefile | 3 + lib/pks/Makefile | 3 + lib/pks/pks_test.c | 690 +++++++++++++++++++++ tools/testing/selftests/x86/Makefile | 3 +- tools/testing/selftests/x86/test_pks.c | 65 ++ 9 files changed, 799 insertions(+), 2 deletions(-) create mode 100644 lib/pks/Makefile create mode 100644 lib/pks/pks_test.c create mode 100644 tools/testing/selftests/x86/test_pks.c
Documentation/core-api/protection-keys.rst +1 -0
@@ -163,3 +163,4 @@ of WRPKRU. So to quote from the WRPKRU text: until all prior executions of WRPKRU have completed execution and updated the PKRU register. +Example code can be found in lib/pks/pks_test.c
arch/x86/mm/fault.c +23 -0
@@ -18,6 +18,7 @@#include <linux/uaccess.h> /* faulthandler_disabled() */ #include <linux/efi.h> /* efi_recover_from_page_fault()*/ #include <linux/mm_types.h> +#include <linux/pkeys.h> #include <asm/cpufeature.h> /* boot_cpu_has, ... */ #include <asm/traps.h> /* dotraplinkage, ... */
@@ -1149,6 +1150,25 @@ static int fault_in_kernel_space(unsigned long address)return address >=TASK_SIZE_MAX ; } +#ifdef CONFIG_PKS_TESTING +bool pks_test_callback (irqentry_state_t *irq_state); +static bool handle_pks_testing (unsigned long hw_error_code,irqentry_state_t *irq_state) +{ +/* +* If we get a protection key exception it could be because we +* are running the PKS test. If so, pks_test_callback() will +* clear the protection mechanism and return true to indicate +* the fault was handled. +*/ +return (hw_error_code &X86_PF_PK ) &&pks_test_callback (irq_state); +} +#else +static bool handle_pks_testing (unsigned long hw_error_code,irqentry_state_t *irq_state) +{ +return false; +} +#endif +/* * Called for all faults where 'address' is part of the kernel address * space. Might get called for faults that originate from *code* that
@@ -1166,6 +1186,9 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code, !cpu_feature_enabled(X86_FEATURE_PKS))WARN_ON_ONCE (hw_error_code &X86_PF_PK ); +if (handle_pks_testing (hw_error_code, irq_state)) +return ; +#ifdef CONFIG_X86_32 /* * We can fault-in kernel-space virtual memory on-demand. The
include/linux/pkeys.h +0 -1
@@ -47,7 +47,6 @@ static inline bool arch_pkeys_enabled(void)static inline void copy_init_pkru_to_fpregs (void ) { } -#endif /* ! CONFIG_ARCH_HAS_PKEYS */ #ifndef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS
lib/Kconfig.debug +12 -0
@@ -2400,6 +2400,18 @@ config HYPERV_TESTING help Select this option to enable Hyper-V vmbus testing. +config PKS_TESTING + bool "PKey(S)upervisor testing" + default n + depends on ARCH_HAS_SUPERVISOR_PKEYS + help + Select this option to enable testing of PKS core software and + hardware. The PKS core provides a mechanism to allocate keys as well + as maintain the protection settings across context switches. + Answer N if you don't know what supervisor keys are. + + If unsure, say N. + endmenu # "Kernel Testing and Coverage" endmenu # Kernel hacking
lib/Makefile +3 -0
@@ -348,3 +348,6 @@ obj-$(CONFIG_PLDMFW) += pldmfw/ obj-$(CONFIG_LIST_KUNIT_TEST) += list-test.o obj-$(CONFIG_LINEAR_RANGES_TEST) += test_linear_ranges.o obj-$(CONFIG_BITS_TEST) += test_bits.o + +# PKS test +obj-y += pks/
lib/pks/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_PKS_TESTING) += pks_test.o
lib/pks/pks_test.c new file +690 -0
690 lines not on this page · raw message
tools/testing/selftests/x86/Makefile +2 -1
@@ -13,7 +13,8 @@ CAN_BUILD_WITH_NOPIE := $(shell ./check_cc.sh $(CC) trivial_program.c -no-pie) TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt test_mremap_vdso \ check_initial_reg_state sigreturn iopl ioperm \ test_vdso test_vsyscall mov_ss_trap \ - syscall_arg_fault fsgsbase_restore + syscall_arg_fault fsgsbase_restore test_pks + TARGETS_C_32BIT_ONLY := entry_from_vm86 test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \ vdso_restorer
tools/testing/selftests/x86/test_pks.c new file +65 -0
65 lines not on this page · raw message
Signature · 4 lines
2.28.0.rc0.12.gb6a658bd00c9 _______________________________________________ Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]
Re: [PATCH RFC V3 9/9] x86/pks: Add PKS test code
include/linux/pkeys.h static inline bool arch_pkeys_enabled(void) · line 50
Quoted 13 lines · On 10/9/20 12:42 PM, [email protected] wrote:
#ifdef
CONFIG_X86_32arch/x86/mm/fault.c +0
-0/* * We can fault-in kernel-space virtual memory on-demand. The include/linux/pkeys.h +0
-1@@ -47,7 +47,6 @@ static inline bool arch_pkeys_enabled(void)static inline void copy_init_pkru_to_fpregs (void ) { } -#endif /* ! CONFIG_ARCH_HAS_PKEYS */
^ Whitespace damage
lib/Kconfig.debug config HYPERV_TESTING · line 2404
Quoted 11 lines
include/linux/pkeys.h +0
-0#ifndef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS lib/Kconfig.debug +2
-0@@ -2400,6 +2400,18 @@ config HYPERV_TESTING help Select this option to enable Hyper-V vmbus testing. +config PKS_TESTING + bool "PKey(S)upervisor testing"
Seems like we need a space in there somewhere.
lib/Kconfig.debug config HYPERV_TESTING · line 2412
Quoted 8 lines
lib/Kconfig.debug +8
-0+ pid = fork(); + if (pid == 0) { + fd = open("/sys/kernel/debug/x86/run_pks", O_RDWR); + if (fd < 0) { + printf("cannot open file\n"); + return -1; + } +
Will this return code make anybody mad? Should we have a nicer return code for when this is running on non-PKS hardware?
I'm not going to be too picky about this. I'll just ask one question:
Has this found real bugs for you?
Re: [PATCH RFC V3 9/9] x86/pks: Add PKS test code
Quoted 16 lines · Dave Hansen
include/linux/pkeys.h static inline bool arch_pkeys_enabled(void) · line 50
Quoted 13 lines · On 10/9/20 12:42 PM, [email protected] wrote:
#ifdef
CONFIG_X86_32(quoted diff) +0
-0/* * We can fault-in kernel-space virtual memory on-demand. Theinclude/linux/pkeys.h +0
-1@@ -47,7 +47,6 @@ static inline bool arch_pkeys_enabled(void)static inline void copy_init_pkru_to_fpregs (void ) { } -#endif /* ! CONFIG_ARCH_HAS_PKEYS */ ^ Whitespace damage
Done.
Quoted 10 lines
Quoted 10 lines
include/linux/pkeys.h +0
-0#ifndef CONFIG_ARCH_HAS_SUPERVISOR_PKEYS lib/Kconfig.debug +0
-0@@ -2400,6 +2400,18 @@ config HYPERV_TESTING help Select this option to enable Hyper-V vmbus testing.
Quoted 4 lines
lib/Kconfig.debug config HYPERV_TESTING · line 2404
Quoted 2 lines
lib/Kconfig.debug +2
-0+config PKS_TESTING + bool "PKey(S)upervisor testing"Seems like we need a space in there somewhere.
heheh... yea...
Quoted 12 lines
lib/Kconfig.debug config HYPERV_TESTING · line 2412
Quoted 9 lines
lib/Kconfig.debug +8
-0+ pid = fork(); + if (pid == 0) { + fd = open("/sys/kernel/debug/x86/run_pks", O_RDWR); + if (fd < 0) { + printf("cannot open file\n"); + return -1; + } +Will this return code make anybody mad? Should we have a nicer return code for when this is running on non-PKS hardware?
I'm not sure it will matter much but I think it is better to report the missing file.[1]
Quoted 3 lines
I'm not going to be too picky about this. I'll just ask one question:
Has this found real bugs for you?
Many, especially regressions as things have changed.
Quoted 3 lines
Reviewed-by Dave Hansen <[email protected]>
Thanks,
Ira
[1]
tools/testing/selftests/x86/test_pks.c +6 -4
@@ -11,6 +11,8 @@#include <sys/stat.h> #include <fcntl.h> +#define PKS_TEST_FILE "/sys/kernel/debug/x86/run_pks" +int main (void ) {cpu_set_t cpuset;
@@ -25,9 +27,9 @@ int main(void) pid =fork ();if (pid ==0 ) { - fd =open ("/sys/kernel/debug/x86/run_pks" ,O_RDWR ); + fd =open (PKS_TEST_FILE ,O_RDWR );if (fd <0 ) { -printf ("cannot open file\n" ); +printf ("cannot open %s\n" ,PKS_TEST_FILE );return -1 ; }
@@ -45,9 +47,9 @@ int main(void) }else {sleep (2 ); - fd =open ("/sys/kernel/debug/x86/run_pks" ,O_RDWR ); + fd =open (PKS_TEST_FILE ,O_RDWR );if (fd <0 ) { -printf ("cannot open file\n" ); +printf ("cannot open %s\n" ,PKS_TEST_FILE );return -1 ; }
Quoted 8 lines · Ira Weiny
From: Ira Weiny <[email protected]>
This RFC series has been reviewed by Dave Hansen.
Introduce a new page protection mechanism for supervisor pages, Protection Key Supervisor (PKS).
2 use cases for PKS are being developed, trusted keys and PMEM.
RFC patch sets for these use cases have also been posted:
PMEM:
https://lore.kernel.org/lkml/[email protected]/
Trusted Keys:
https://lore.kernel.org/lkml/[email protected]/
Ira
Quoted 100 lines
Trusted keys
is a newer use case which is still being explored. PMEM was submitted as part of the RFC (v2) series[1]. However, since then it was found that some callers ofkmap()require a global implementation of PKS. Specifically some users ofkmap()expect mappings to be available to all kernel threads. While global use of PKS is rare it needs to be included for correctness. Unfortunately thekmap()updates required a large patch series to make the needed changes at the variouskmap()call sites so that patch set has been split out. Because the global PKS feature is only required for that use case it will be deferred to that set as well.[2] This patch set is being submitted as a precursor to both of the use cases.For an overview of the entire PKS ecosystem, a git tree including this series and the 2 use cases can be found here:
https://github.com/weiny2/linux-kernel/tree/pks-rfc-v3
PKS enables protections on 'domains' of supervisor pages to limit supervisor mode access to those pages beyond the normal paging protections. PKS works in a similar fashion to user space pkeys, PKU. As with PKU, supervisor pkeys are checked in addition to normal paging protections and Access or Writes can be disabled via a MSR update without TLB flushes when permissions change. Also like PKU, a page mapping is assigned to a domain by setting pkey bits in the page table entry for that mapping.
Access is controlled through a PKRS register which is updated via WRMSR/RDMSR.
XSAVE is not supported for the PKRS MSR. Therefore the implementation saves/restores the MSR across context switches and during exceptions. Nested exceptions are supported by each exception getting a new PKS state.
For consistent behavior with current paging protections, pkey 0 is reserved and configured to allow full access via the pkey mechanism, thus preserving the default paging protections on mappings with the default pkey value of 0.
Other keys, (1-15) are allocated by an allocator which prepares us for key contention from day one. Kernel users should be prepared for the allocator to fail either because of key exhaustion or due to PKS not being supported on the arch and/or CPU instance.
The following are key attributes of PKS.
1) Fast switching of permissions
1a) Prevents access without page table manipulations 1b) No TLB flushes required
2) Works on a per thread basisPKS is available with 4 and 5 level paging. Like PKRU it consumes 4 bits from the PTE to store the pkey within the entry.
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://github.com/weiny2/linux-kernel/commit/f10abb0f0d7b4e14f03fc8890313a5830cde1e49and a testing patch https://github.com/weiny2/linux-kernel/commit/2a8e0fc7654a7c69b243d628f63b01ff26a5a797Fenghua Yu (3):
x86/fpu: Refactorarch_set_user_pkey_access()for PKS support x86/pks: Enable Protection Keys Supervisor (PKS) x86/pks: Add PKS kernel APIIra Weiny (6):
x86/pkeys: Createpkeys_common.hx86/pks: Preserve the PKRS MSR on context switch x86/entry: Passirqentry_state_tby reference x86/entry: Preserve PKRS MSR across exceptions x86/fault: Report the PKRS state on fault x86/pks: Add PKS test codeDocumentation/core-api/protection-keys.rst | 102 ++- arch/x86/Kconfig | 1 + arch/x86/entry/common.c | 57 +- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/idtentry.h | 29 +- arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/pgtable.h | 13 +- arch/x86/include/asm/pgtable_types.h | 12 + arch/x86/include/asm/pkeys.h | 15 + arch/x86/include/asm/pkeys_common.h | 36 + arch/x86/include/asm/processor.h | 13 + arch/x86/include/uapi/asm/processor-flags.h | 2 + arch/x86/kernel/cpu/common.c | 17 + arch/x86/kernel/cpu/mce/core.c | 4 + arch/x86/kernel/fpu/xstate.c | 22 +- arch/x86/kernel/kvm.c | 4 +- arch/x86/kernel/nmi.c | 7 +- arch/x86/kernel/process.c | 21 + arch/x86/kernel/traps.c | 21 +- arch/x86/mm/fault.c | 86 ++- arch/x86/mm/pkeys.c | 188 +++++- include/linux/entry-common.h | 19 +- include/linux/pgtable.h | 4 + include/linux/pkeys.h | 23 +- kernel/entry/common.c | 28 +- lib/Kconfig.debug | 12 + lib/Makefile | 3 + lib/pks/Makefile | 3 + lib/pks/pks_test.c | 690 ++++++++++++++++++++
Quoted 12 lines
mm/Kconfig | 2 + tools/testing/selftests/x86/Makefile | 3 +- tools/testing/selftests/x86/test_pks.c | 65 ++ 32 files changed, 1376 insertions(+), 128 deletions(-) create mode 100644 arch/x86/include/asm/pkeys_common.h create mode 100644 lib/pks/Makefile create mode 100644 lib/pks/pks_test.c create mode 100644 tools/testing/selftests/x86/test_pks.cSignature · 1 line
2.28.0.rc0.12.gb6a658bd00c9
_______________________________________________
Linux-nvdimm mailing list -- [email protected] To unsubscribe send an email to [email protected]