powerpc/book3s/64/pkeys: Simplify the code
v6 cover powerpc/book3s/64/pkeys: Simplify the code
This patch series update the pkey subsystem with more documentation and rename variables so that it is easy to follow the code. We drop the changes to support KUAP/KUEP with hash translation in this update. The changes are adding 200 cycles to null syscalls benchmark and I want to look at that closely before requesting a merge. The rest of the patches are included in this series. This should avoid having to carry a large patchset across the upstream merge. Some of the changes in here make the hash KUEP/KUAP addition simpler.
Changes from v5:
* Address review feedback.
* Dropped patches moving kup to generic name.
* Dropped static key changes related to execute only support.
Changes from v4:
* Drop hash KUAP/KUEP changes.
Changes from v3:
* Fix build error reported by kernel test robot <[email protected]>
Changes from v2:
* Rebase to the latest kernel.
* Fixed a bug with disabling KUEP/KUAP on kernel command line
* Added a patch to make kup key dynamic.
Changes from v1:
* Rebased on latest kernel
Aneesh Kumar K.V (23):
powerpc/book3s64/pkeys: Use PVR check instead of cpu feature powerpc/book3s64/pkeys: Fixup bit numbering powerpc/book3s64/pkeys: pkeys are supported only on hash on book3s. powerpc/book3s64/pkeys: Move pkey related bits in the linux page table powerpc/book3s64/pkeys: Explain key 1 reservation details powerpc/book3s64/pkeys: Simplify the key initialization powerpc/book3s64/pkeys: Prevent key 1 modification from userspace. powerpc/book3s64/pkeys: kill cpu feature key CPU_FTR_PKEY powerpc/book3s64/pkeys: Simplify pkey disable branch powerpc/book3s64/pkeys: Convert pkey_total to num_pkey powerpc/book3s64/pkeys: Make initial_allocation_mask static powerpc/book3s64/pkeys: Mark all the pkeys above max pkey as reserved powerpc/book3s64/pkeys: Add MMU_FTR_PKEY powerpc/book3s64/kuep: Add MMU_FTR_KUEP powerpc/book3s64/pkeys: Use pkey_execute_disable_supported powerpc/book3s64/pkeys: Use MMU_FTR_PKEY instead of pkey_disabled
static key
powerpc/book3s64/keys: Print information during boot. powerpc/book3s64/keys/kuap: Reset AMR/IAMR values on kexec powerpc/book3s64/kuap: Move UAMOR setup to key init function powerpc/selftest/ptrave-pkey: Rename variables to make it easier to
follow code
powerpc/selftest/ptrace-pkey: Update the test to mark an invalid pkey
correctly
powerpc/selftest/ptrace-pkey: Don't update expected UAMOR value powerpc/book3s64/pkeys: Remove is_pkey_enabled()
arch/powerpc/include/asm/book3s/64/hash-4k.h | 21 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 12 +- .../powerpc/include/asm/book3s/64/hash-pkey.h | 32 ++ arch/powerpc/include/asm/book3s/64/kexec.h | 23 ++ arch/powerpc/include/asm/book3s/64/mmu-hash.h | 8 +- arch/powerpc/include/asm/book3s/64/mmu.h | 6 + arch/powerpc/include/asm/book3s/64/pgtable.h | 17 +- arch/powerpc/include/asm/book3s/64/pkeys.h | 27 ++ arch/powerpc/include/asm/cputable.h | 13 +- arch/powerpc/include/asm/kexec.h | 12 + arch/powerpc/include/asm/mmu.h | 17 + arch/powerpc/include/asm/pkeys.h | 65 +--- arch/powerpc/include/asm/processor.h | 1 - arch/powerpc/kernel/dt_cpu_ftrs.c | 6 - arch/powerpc/kernel/misc_64.S | 14 - arch/powerpc/kernel/prom.c | 5 + arch/powerpc/kernel/ptrace/ptrace-view.c | 27 +- arch/powerpc/kernel/smp.c | 1 + arch/powerpc/kexec/core_64.c | 2 + arch/powerpc/mm/book3s64/hash_utils.c | 4 + arch/powerpc/mm/book3s64/pgtable.c | 3 + arch/powerpc/mm/book3s64/pkeys.c | 294 ++++++++++-------- arch/powerpc/mm/book3s64/radix_pgtable.c | 8 +- .../selftests/powerpc/ptrace/ptrace-pkey.c | 55 ++-- 24 files changed, 404 insertions(+), 269 deletions(-) create mode 100644 arch/powerpc/include/asm/book3s/64/hash-pkey.h create mode 100644 arch/powerpc/include/asm/book3s/64/kexec.h create mode 100644 arch/powerpc/include/asm/book3s/64/pkeys.h
Signature · 1 line
2.26.2
PATCH v6 1/23 powerpc/book3s64/pkeys: Use PVR check instead of cpu feature
We are wrongly using CPU_FTRS_POWER8 to check for P8 support. Instead, we should use PVR value. Now considering we are using CPU_FTRS_POWER8, that implies we returned true for P9 with older firmware. Keep the same behavior by checking for P9 PVR value.
--- arch/powerpc/mm/book3s64/pkeys.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
arch/powerpc/mm/book3s64/pkeys.c +10 -6
@@ -83,13 +83,17 @@ static int pkey_initialize(void)scan_pkey_feature ();/* -* Let's assume 32 pkeys on P8 bare metal, if its not defined by device -* tree. We make this exception since skiboot forgot to expose this -* property on power8. +* Let's assume 32 pkeys on P8/P9 bare metal, if its not defined by device +* tree. We make this exception since some version of skiboot forgot to +* expose this property on power8/9. */ -if (!pkeys_devtree_defined && !firmware_has_feature (FW_FEATURE_LPAR ) && -cpu_has_feature (CPU_FTRS_POWER8 )) - pkeys_total =32 ; +if (!pkeys_devtree_defined && !firmware_has_feature (FW_FEATURE_LPAR )) { +unsigned long pvr =mfspr (SPRN_PVR ); + +if (PVR_VER (pvr) ==PVR_POWER8 ||PVR_VER (pvr) ==PVR_POWER8E || +PVR_VER (pvr) ==PVR_POWER8NVL ||PVR_VER (pvr) ==PVR_POWER9 ) + pkeys_total =32 ; + }/* * Adjust the upper limit, based on the number of bits supported by
Signature · 1 line
2.26.2
PATCH v6 2/23 powerpc/book3s64/pkeys: Fixup bit numbering
In reply to Aneesh Kumar K.VThis number the pkey bit such that it is easy to follow. PKEY_BIT0 is the lower order bit. This makes further changes easy to follow.
No functional change in this patch other than linux page table for hash translation now maps pkeys differently.
--- arch/powerpc/include/asm/book3s/64/hash-4k.h | 9 +++---- arch/powerpc/include/asm/book3s/64/hash-64k.h | 8 +++---- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 8 +++---- arch/powerpc/include/asm/pkeys.h | 24 +++++++++---------- 4 files changed, 25 insertions(+), 24 deletions(-)
arch/powerpc/include/asm/book3s/64/hash-4k.h +5 -4
@@ -57,11 +57,12 @@#define H_PMD_FRAG_NR (PAGE_SIZE >> H_PMD_FRAG_SIZE_SHIFT)/* memory key bits, only 8 keys supported */ -#define H_PTE_PKEY_BIT0 0 -#define H_PTE_PKEY_BIT1 0 +#define H_PTE_PKEY_BIT4 0 +#define H_PTE_PKEY_BIT3 0#define H_PTE_PKEY_BIT2 _RPAGE_RSV3 -#define H_PTE_PKEY_BIT3 _RPAGE_RSV4 -#define H_PTE_PKEY_BIT4 _RPAGE_RSV5 +#define H_PTE_PKEY_BIT1 _RPAGE_RSV4 +#define H_PTE_PKEY_BIT0 _RPAGE_RSV5 +/* * On all 4K setups, remap_4k_pfn() equates to remap_pfn_range()
arch/powerpc/include/asm/book3s/64/hash-64k.h +4 -4
@@ -36,11 +36,11 @@#define H_PAGE_HASHPTE _RPAGE_RPN43/* PTE has associated HPTE */ /* memory key bits. */ -#define H_PTE_PKEY_BIT0 _RPAGE_RSV1 -#define H_PTE_PKEY_BIT1 _RPAGE_RSV2 +#define H_PTE_PKEY_BIT4 _RPAGE_RSV1 +#define H_PTE_PKEY_BIT3 _RPAGE_RSV2#define H_PTE_PKEY_BIT2 _RPAGE_RSV3 -#define H_PTE_PKEY_BIT3 _RPAGE_RSV4 -#define H_PTE_PKEY_BIT4 _RPAGE_RSV5 +#define H_PTE_PKEY_BIT1 _RPAGE_RSV4 +#define H_PTE_PKEY_BIT0 _RPAGE_RSV5/* * We need to differentiate between explicit huge page and THP huge
arch/powerpc/include/asm/book3s/64/mmu-hash.h +4 -4
@@ -86,8 +86,8 @@#define HPTE_R_PP0 ASM_CONST(0x8000000000000000)#define HPTE_R_TS ASM_CONST(0x4000000000000000)#define HPTE_R_KEY_HI ASM_CONST(0x3000000000000000) -#define HPTE_R_KEY_BIT0 ASM_CONST(0x2000000000000000) -#define HPTE_R_KEY_BIT1 ASM_CONST(0x1000000000000000) +#define HPTE_R_KEY_BIT4 ASM_CONST(0x2000000000000000) +#define HPTE_R_KEY_BIT3 ASM_CONST(0x1000000000000000)#define HPTE_R_RPN_SHIFT 12#define HPTE_R_RPN ASM_CONST(0x0ffffffffffff000)#define HPTE_R_RPN_3_0 ASM_CONST(0x01fffffffffff000)
@@ -103,8 +103,8 @@#define HPTE_R_R ASM_CONST(0x0000000000000100)#define HPTE_R_KEY_LO ASM_CONST(0x0000000000000e00)#define HPTE_R_KEY_BIT2 ASM_CONST(0x0000000000000800) -#define HPTE_R_KEY_BIT3 ASM_CONST(0x0000000000000400) -#define HPTE_R_KEY_BIT4 ASM_CONST(0x0000000000000200) +#define HPTE_R_KEY_BIT1 ASM_CONST(0x0000000000000400) +#define HPTE_R_KEY_BIT0 ASM_CONST(0x0000000000000200)#define HPTE_R_KEY (HPTE_R_KEY_LO | HPTE_R_KEY_HI)#define HPTE_V_1TB_SEG ASM_CONST(0x4000000000000000)
arch/powerpc/include/asm/pkeys.h +12 -12
@@ -35,11 +35,11 @@ static inline u64 vmflag_to_pte_pkey_bits(u64 vm_flags)if (static_branch_likely (&pkey_disabled))return 0x0UL ; -return (((vm_flags &VM_PKEY_BIT0 ) ?H_PTE_PKEY_BIT4 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT1 ) ?H_PTE_PKEY_BIT3 :0x0UL ) | +return (((vm_flags &VM_PKEY_BIT0 ) ?H_PTE_PKEY_BIT0 :0x0UL ) | + ((vm_flags &VM_PKEY_BIT1 ) ?H_PTE_PKEY_BIT1 :0x0UL ) | ((vm_flags &VM_PKEY_BIT2 ) ?H_PTE_PKEY_BIT2 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT3 ) ?H_PTE_PKEY_BIT1 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT4 ) ?H_PTE_PKEY_BIT0 :0x0UL )); + ((vm_flags &VM_PKEY_BIT3 ) ?H_PTE_PKEY_BIT3 :0x0UL ) | + ((vm_flags &VM_PKEY_BIT4 ) ?H_PTE_PKEY_BIT4 :0x0UL )); }static inline int vma_pkey (struct vm_area_struct *vma)
@@ -53,20 +53,20 @@ static inline int vma_pkey(struct vm_area_struct *vma)static inline u64 pte_to_hpte_pkey_bits (u64 pteflags) { -return (((pteflags &H_PTE_PKEY_BIT0 ) ?HPTE_R_KEY_BIT0 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT1 ) ?HPTE_R_KEY_BIT1 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT2 ) ?HPTE_R_KEY_BIT2 :0x0UL ) | +return (((pteflags &H_PTE_PKEY_BIT4 ) ?HPTE_R_KEY_BIT4 :0x0UL ) | ((pteflags &H_PTE_PKEY_BIT3 ) ?HPTE_R_KEY_BIT3 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT4 ) ?HPTE_R_KEY_BIT4 :0x0UL )); + ((pteflags &H_PTE_PKEY_BIT2 ) ?HPTE_R_KEY_BIT2 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT1 ) ?HPTE_R_KEY_BIT1 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT0 ) ?HPTE_R_KEY_BIT0 :0x0UL )); }static inline u16 pte_to_pkey_bits (u64 pteflags) { -return (((pteflags &H_PTE_PKEY_BIT0 ) ?0x10 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT1 ) ?0x8 :0x0UL ) | +return (((pteflags &H_PTE_PKEY_BIT4 ) ?0x10 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT3 ) ?0x8 :0x0UL ) | ((pteflags &H_PTE_PKEY_BIT2 ) ?0x4 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT3 ) ?0x2 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT4 ) ?0x1 :0x0UL )); + ((pteflags &H_PTE_PKEY_BIT1 ) ?0x2 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT0 ) ?0x1 :0x0UL )); }#define pkey_alloc_mask (pkey) (0x1 << pkey)
Signature · 1 line
2.26.2
PATCH v6 3/23 powerpc/book3s64/pkeys: pkeys are supported only on hash on book3s.
In reply to Aneesh Kumar K.VMove them to hash specific file and add BUG() for radix path.
--- .../powerpc/include/asm/book3s/64/hash-pkey.h | 32 ++++++++++++++++ arch/powerpc/include/asm/book3s/64/pkeys.h | 25 +++++++++++++ arch/powerpc/include/asm/pkeys.h | 37 ++++--------------- 3 files changed, 64 insertions(+), 30 deletions(-) create mode 100644 arch/powerpc/include/asm/book3s/64/hash-pkey.h create mode 100644 arch/powerpc/include/asm/book3s/64/pkeys.h
arch/powerpc/include/asm/book3s/64/hash-pkey.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_POWERPC_BOOK3S_64_HASH_PKEY_H +#define _ASM_POWERPC_BOOK3S_64_HASH_PKEY_H + +static inline u64 hash__vmflag_to_pte_pkey_bits (u64 vm_flags) +{ +return (((vm_flags &VM_PKEY_BIT0 ) ?H_PTE_PKEY_BIT0 :0x0UL ) | + ((vm_flags &VM_PKEY_BIT1 ) ?H_PTE_PKEY_BIT1 :0x0UL ) | + ((vm_flags &VM_PKEY_BIT2 ) ?H_PTE_PKEY_BIT2 :0x0UL ) | + ((vm_flags &VM_PKEY_BIT3 ) ?H_PTE_PKEY_BIT3 :0x0UL ) | + ((vm_flags &VM_PKEY_BIT4 ) ?H_PTE_PKEY_BIT4 :0x0UL )); +} + +static inline u64 pte_to_hpte_pkey_bits (u64 pteflags) +{ +return (((pteflags &H_PTE_PKEY_BIT4 ) ?HPTE_R_KEY_BIT4 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT3 ) ?HPTE_R_KEY_BIT3 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT2 ) ?HPTE_R_KEY_BIT2 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT1 ) ?HPTE_R_KEY_BIT1 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT0 ) ?HPTE_R_KEY_BIT0 :0x0UL )); +} + +static inline u16 hash__pte_to_pkey_bits (u64 pteflags) +{ +return (((pteflags &H_PTE_PKEY_BIT4 ) ?0x10 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT3 ) ?0x8 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT2 ) ?0x4 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT1 ) ?0x2 :0x0UL ) | + ((pteflags &H_PTE_PKEY_BIT0 ) ?0x1 :0x0UL )); +} + +#endif
arch/powerpc/include/asm/book3s/64/pkeys.h new file +25 -0
@@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ + +#ifndef _ASM_POWERPC_BOOK3S_64_PKEYS_H +#define _ASM_POWERPC_BOOK3S_64_PKEYS_H + +#include <asm/book3s/64/hash-pkey.h> + +static inline u64 vmflag_to_pte_pkey_bits (u64 vm_flags) +{ +if (static_branch_likely (&pkey_disabled)) +return 0x0UL ; + +if (radix_enabled ()) +BUG (); +return hash__vmflag_to_pte_pkey_bits (vm_flags); +} + +static inline u16 pte_to_pkey_bits (u64 pteflags) +{ +if (radix_enabled ()) +BUG (); +return hash__pte_to_pkey_bits (pteflags); +} + +#endif /*_ASM_POWERPC_KEYS_H */
arch/powerpc/include/asm/pkeys.h +7 -30
@@ -25,23 +25,18 @@ extern u32 reserved_allocation_mask; /* bits set for reserved keys */ PKEY_DISABLE_WRITE | \ PKEY_DISABLE_EXECUTE) +#ifdef CONFIG_PPC_BOOK3S_64 +#include <asm/book3s/64/pkeys.h> +#else +#error "Not supported" +#endif + +static inline u64 pkey_to_vmflag_bits (u16 pkey) {return (((u64 )pkey <<VM_PKEY_SHIFT ) &ARCH_VM_PKEY_FLAGS ); } -static inline u64 vmflag_to_pte_pkey_bits (u64 vm_flags) -{ -if (static_branch_likely (&pkey_disabled)) -return 0x0UL ; - -return (((vm_flags &VM_PKEY_BIT0 ) ?H_PTE_PKEY_BIT0 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT1 ) ?H_PTE_PKEY_BIT1 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT2 ) ?H_PTE_PKEY_BIT2 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT3 ) ?H_PTE_PKEY_BIT3 :0x0UL ) | - ((vm_flags &VM_PKEY_BIT4 ) ?H_PTE_PKEY_BIT4 :0x0UL )); -} -static inline int vma_pkey (struct vm_area_struct *vma) {if (static_branch_likely (&pkey_disabled))
@@ -51,24 +46,6 @@ static inline int vma_pkey(struct vm_area_struct *vma)#define arch_max_pkey () pkeys_total -static inline u64 pte_to_hpte_pkey_bits (u64 pteflags) -{ -return (((pteflags &H_PTE_PKEY_BIT4 ) ?HPTE_R_KEY_BIT4 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT3 ) ?HPTE_R_KEY_BIT3 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT2 ) ?HPTE_R_KEY_BIT2 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT1 ) ?HPTE_R_KEY_BIT1 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT0 ) ?HPTE_R_KEY_BIT0 :0x0UL )); -} - -static inline u16 pte_to_pkey_bits (u64 pteflags) -{ -return (((pteflags &H_PTE_PKEY_BIT4 ) ?0x10 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT3 ) ?0x8 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT2 ) ?0x4 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT1 ) ?0x2 :0x0UL ) | - ((pteflags &H_PTE_PKEY_BIT0 ) ?0x1 :0x0UL )); -} -#define pkey_alloc_mask (pkey) (0x1 << pkey)#define mm_pkey_allocation_map (mm) (mm->context.pkey_allocation_map)
Signature · 1 line
2.26.2
PATCH v6 4/23 powerpc/book3s64/pkeys: Move pkey related bits in the linux page table
In reply to Aneesh Kumar K.VTo keep things simple, all the pkey related bits are kept together in linux page table for 64K config with hash translation. With hash-4k kernel requires 4 bits to store slots details. This is done by overloading some of the RPN bits for storing the slot details. Due to this PKEY_BIT0 on the 4K config is used for storing hash slot details.
64K before
|....|RSV1| RSV2| RSV3 | RSV4 | RPN44| RPN43 |.... | RSV5| |....| P4 | P3 | P2 | P1 | Busy | HASHPTE |.... | P0 |
after
|....|RSV1| RSV2| RSV3 | RSV4 | RPN44 | RPN43 |.... | RSV5 | |....| P4 | P3 | P2 | P1 | P0 | HASHPTE |.... | Busy |
4k before
|....| RSV1 | RSV2 | RSV3 | RSV4 | RPN44| RPN43.... | RSV5| |....| Busy | HASHPTE | P2 | P1 | F_SEC| F_GIX.... | P0 |
after
|....| RSV1 | RSV2| RSV3 | RSV4 | Free | RPN43.... | RSV5 | |....| HASHPTE | P2 | P1 | P0 | F_SEC| F_GIX.... | BUSY |
--- arch/powerpc/include/asm/book3s/64/hash-4k.h | 16 ++++++++-------- arch/powerpc/include/asm/book3s/64/hash-64k.h | 12 ++++++------ arch/powerpc/include/asm/book3s/64/pgtable.h | 17 ++++++++--------- 3 files changed, 22 insertions(+), 23 deletions(-)
arch/powerpc/include/asm/book3s/64/hash-4k.h +8 -8
@@ -34,11 +34,11 @@#define H_PUD_TABLE_SIZE (sizeof(pud_t) << H_PUD_INDEX_SIZE)#define H_PGD_TABLE_SIZE (sizeof(pgd_t) << H_PGD_INDEX_SIZE) -#define H_PAGE_F_GIX_SHIFT 53 -#define H_PAGE_F_SECOND _RPAGE_RPN44/* HPTE is in 2ndary HPTEG */ -#define H_PAGE_F_GIX (_RPAGE_RPN43 | _RPAGE_RPN42 | _RPAGE_RPN41) -#define H_PAGE_BUSY _RPAGE_RSV1/* software: PTE & hash are busy */ -#define H_PAGE_HASHPTE _RPAGE_RSV2/* software: PTE & hash are busy */ +#define H_PAGE_F_GIX_SHIFT _PAGE_PA_MAX +#define H_PAGE_F_SECOND _RPAGE_PKEY_BIT0/* HPTE is in 2ndary HPTEG */ +#define H_PAGE_F_GIX (_RPAGE_RPN43 | _RPAGE_RPN42 | _RPAGE_RPN41) +#define H_PAGE_BUSY _RPAGE_RSV1 +#define H_PAGE_HASHPTE _RPAGE_PKEY_BIT4/* PTE flags to conserve for HPTE identification */ #define _PAGE_HPTEFLAGS (H_PAGE_BUSY | H_PAGE_HASHPTE | \
@@ -59,9 +59,9 @@/* memory key bits, only 8 keys supported */ #define H_PTE_PKEY_BIT4 0#define H_PTE_PKEY_BIT3 0 -#define H_PTE_PKEY_BIT2 _RPAGE_RSV3 -#define H_PTE_PKEY_BIT1 _RPAGE_RSV4 -#define H_PTE_PKEY_BIT0 _RPAGE_RSV5 +#define H_PTE_PKEY_BIT2 _RPAGE_PKEY_BIT3 +#define H_PTE_PKEY_BIT1 _RPAGE_PKEY_BIT2 +#define H_PTE_PKEY_BIT0 _RPAGE_PKEY_BIT1/*
arch/powerpc/include/asm/book3s/64/hash-64k.h +6 -6
@@ -32,15 +32,15 @@*/ #define H_PAGE_COMBO _RPAGE_RPN0/* this is a combo 4k page */ #define H_PAGE_4K_PFN _RPAGE_RPN1/* PFN is for a single 4k page */ -#define H_PAGE_BUSY _RPAGE_RPN44/* software: PTE & hash are busy */ +#define H_PAGE_BUSY _RPAGE_RSV1/* software: PTE & hash are busy */ #define H_PAGE_HASHPTE _RPAGE_RPN43/* PTE has associated HPTE */ /* memory key bits. */ -#define H_PTE_PKEY_BIT4 _RPAGE_RSV1 -#define H_PTE_PKEY_BIT3 _RPAGE_RSV2 -#define H_PTE_PKEY_BIT2 _RPAGE_RSV3 -#define H_PTE_PKEY_BIT1 _RPAGE_RSV4 -#define H_PTE_PKEY_BIT0 _RPAGE_RSV5 +#define H_PTE_PKEY_BIT4 _RPAGE_PKEY_BIT4 +#define H_PTE_PKEY_BIT3 _RPAGE_PKEY_BIT3 +#define H_PTE_PKEY_BIT2 _RPAGE_PKEY_BIT2 +#define H_PTE_PKEY_BIT1 _RPAGE_PKEY_BIT1 +#define H_PTE_PKEY_BIT0 _RPAGE_PKEY_BIT0/* * We need to differentiate between explicit huge page and THP huge
arch/powerpc/include/asm/book3s/64/pgtable.h +8 -9
@@ -32,11 +32,13 @@#define _RPAGE_SW1 0x00800#define _RPAGE_SW2 0x00400#define _RPAGE_SW3 0x00200 -#define _RPAGE_RSV1 0x1000000000000000UL -#define _RPAGE_RSV2 0x0800000000000000UL -#define _RPAGE_RSV3 0x0400000000000000UL -#define _RPAGE_RSV4 0x0200000000000000UL -#define _RPAGE_RSV5 0x00040UL +#define _RPAGE_RSV1 0x00040UL + +#define _RPAGE_PKEY_BIT4 0x1000000000000000UL +#define _RPAGE_PKEY_BIT3 0x0800000000000000UL +#define _RPAGE_PKEY_BIT2 0x0400000000000000UL +#define _RPAGE_PKEY_BIT1 0x0200000000000000UL +#define _RPAGE_PKEY_BIT0 0x0100000000000000UL#define _PAGE_PTE 0x4000000000000000UL/* distinguishes PTEs from pointers */ #define _PAGE_PRESENT 0x8000000000000000UL/* pte contains a translation */
@@ -58,13 +60,12 @@*/ #define _RPAGE_RPN0 0x01000#define _RPAGE_RPN1 0x02000 -#define _RPAGE_RPN44 0x0100000000000000UL#define _RPAGE_RPN43 0x0080000000000000UL#define _RPAGE_RPN42 0x0040000000000000UL#define _RPAGE_RPN41 0x0020000000000000UL/* Max physical address bit as per radix table */ -#define _RPAGE_PA_MAX 57 +#define _RPAGE_PA_MAX 56/* * Max physical address bit we will use for now.
@@ -125,8 +126,6 @@ _PAGE_ACCESSED | _PAGE_SPECIAL | _PAGE_PTE | \ _PAGE_SOFT_DIRTY | _PAGE_DEVMAP) -#define H_PTE_PKEY (H_PTE_PKEY_BIT0 | H_PTE_PKEY_BIT1 | H_PTE_PKEY_BIT2 | \ - H_PTE_PKEY_BIT3 | H_PTE_PKEY_BIT4)/* * We define 2 sets of base prot bits, one for basic pages (ie, * cacheable kernel and user pages) and one for non cacheable
Signature · 1 line
2.26.2
PATCH v6 5/23 powerpc/book3s64/pkeys: Explain key 1 reservation details
In reply to Aneesh Kumar K.VThis explains the details w.r.t key 1.
--- arch/powerpc/mm/book3s64/pkeys.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/powerpc/mm/book3s64/pkeys.c +4 -1
@@ -128,7 +128,10 @@ static int pkey_initialize(void)#else os_reserved =0 ;#endif -/* Bits are in LE format. */ +/* +* key 1 is recommended not to be used. PowerISA(3.0) page 1015, +* programming note. +*/ reserved_allocation_mask = (0x1 <<1 ) | (0x1 << execute_only_key);/* register mask is in BE format */
Signature · 1 line
2.26.2
PATCH v6 6/23 powerpc/book3s64/pkeys: Simplify the key initialization
In reply to Aneesh Kumar K.VAdd documentation explaining the execute_only_key. The reservation and initialization mask details are also explained in this patch.
No functional change in this patch.
--- arch/powerpc/mm/book3s64/pkeys.c | 199 ++++++++++++++++++------------- 1 file changed, 116 insertions(+), 83 deletions(-)
arch/powerpc/mm/book3s64/pkeys.c +116 -83
@@ -15,48 +15,80 @@DEFINE_STATIC_KEY_TRUE (pkey_disabled);int pkeys_total;/* Total pkeys as per device tree */ u32 initial_allocation_mask;/* Bits set for the initially allocated keys */ -u32 reserved_allocation_mask;/* Bits set for reserved keys */ +/* +* Keys marked in the reservation list cannot be allocated by userspace +*/ +u32 reserved_allocation_mask;static bool pkey_execute_disable_supported; -static bool pkeys_devtree_defined;/* property exported by device tree */ -static u64 pkey_amr_mask;/* Bits in AMR not to be touched */ -static u64 pkey_iamr_mask;/* Bits in AMR not to be touched */ -static u64 pkey_uamor_mask;/* Bits in UMOR not to be touched */ +/* +* Even if we allocate keys with sys_pkey_alloc(), we need to make sure +* other thread still find the access denied using the same keys. +*/ +static u64 default_amr = ~0x0UL ; +static u64 default_iamr =0x5555555555555555UL ; + +/* Allow all keys to be modified by default */ +static u64 default_uamor = ~0x0UL ; +/* +* Key used to implement PROT_EXEC mmap. Denies READ/WRITE +* We pick key 2 because 0 is special key and 1 is reserved as per ISA. +*/ static int execute_only_key =2 ; +#define AMR_BITS_PER_PKEY 2#define AMR_RD_BIT 0x1UL#define AMR_WR_BIT 0x2UL#define IAMR_EX_BIT 0x1UL -#define PKEY_REG_BITS (sizeof(u64)*8) +#define PKEY_REG_BITS (sizeof(u64) * 8)#define pkeyshift (pkey) (PKEY_REG_BITS - ((pkey+1) * AMR_BITS_PER_PKEY)) -static void scan_pkey_feature (void ) +static int scan_pkey_feature (void ) {u32 vals[2 ]; +int pkeys_total =0 ;struct device_node *cpu; +/* +* Pkey is not supported with Radix translation. +*/ +if (radix_enabled ()) +return 0 ; + cpu =of_find_node_by_type (NULL ,"cpu" );if (!cpu) -return ; +return 0 ;if (of_property_read_u32_array (cpu, -"ibm,processor-storage-keys" , vals,2 )) -return ; +"ibm,processor-storage-keys" , vals,2 ) ==0 ) { +/* +* Since any pkey can be used for data or execute, we will +* just treat all keys as equal and track them as one entity. +*/ + pkeys_total = vals[0 ]; + }else { + +/* +* Let's assume 32 pkeys on P8/P9 bare metal, if its not defined by device +* tree. We make this exception since some version of skiboot forgot to +* expose this property on power8/9. +*/ +if (!firmware_has_feature (FW_FEATURE_LPAR )) { +unsigned long pvr =mfspr (SPRN_PVR ); + +if (PVR_VER (pvr) ==PVR_POWER8 ||PVR_VER (pvr) ==PVR_POWER8E || +PVR_VER (pvr) ==PVR_POWER8NVL ||PVR_VER (pvr) ==PVR_POWER9 ) + pkeys_total =32 ; + } + }/* -* Since any pkey can be used for data or execute, we will just treat -* all keys as equal and track them as one entity. +* Adjust the upper limit, based on the number of bits supported by +* arch-neutral code. */ - pkeys_total = vals[0 ]; - pkeys_devtree_defined = true; -} - -static inline bool pkey_mmu_enabled (void ) -{ -if (firmware_has_feature (FW_FEATURE_LPAR )) -return pkeys_total; -else -return cpu_has_feature (CPU_FTR_PKEY ); + pkeys_total =min_t (int, pkeys_total, + ((ARCH_VM_PKEY_FLAGS >>VM_PKEY_SHIFT ) +1 )); +return pkeys_total; }static int pkey_initialize (void )
@@ -80,35 +112,13 @@ static int pkey_initialize(void) != (sizeof(u64) * BITS_PER_BYTE));/* scan the device tree for pkey feature */ -scan_pkey_feature (); - -/* -* Let's assume 32 pkeys on P8/P9 bare metal, if its not defined by device -* tree. We make this exception since some version of skiboot forgot to -* expose this property on power8/9. -*/ -if (!pkeys_devtree_defined && !firmware_has_feature (FW_FEATURE_LPAR )) { -unsigned long pvr =mfspr (SPRN_PVR ); - -if (PVR_VER (pvr) ==PVR_POWER8 ||PVR_VER (pvr) ==PVR_POWER8E || -PVR_VER (pvr) ==PVR_POWER8NVL ||PVR_VER (pvr) ==PVR_POWER9 ) - pkeys_total =32 ; - } - -/* -* Adjust the upper limit, based on the number of bits supported by -* arch-neutral code. -*/ - pkeys_total =min_t (int, pkeys_total, - ((ARCH_VM_PKEY_FLAGS >>VM_PKEY_SHIFT )+1 )); - -if (!pkey_mmu_enabled () ||radix_enabled () || !pkeys_total) -static_branch_enable (&pkey_disabled); -else + pkeys_total =scan_pkey_feature (); +if (pkeys_total)static_branch_disable (&pkey_disabled); - -if (static_branch_likely (&pkey_disabled)) +else { +static_branch_enable (&pkey_disabled);return 0 ; + }/* * The device tree cannot be relied to indicate support for
@@ -122,48 +132,71 @@ static int pkey_initialize(void)#ifdef CONFIG_PPC_4K_PAGES /* * The OS can manage only 8 pkeys due to its inability to represent them -* in the Linux 4K PTE. +* in the Linux 4K PTE. Mark all other keys reserved. */ os_reserved = pkeys_total -8 ;#else os_reserved =0 ;#endif -/* -* key 1 is recommended not to be used. PowerISA(3.0) page 1015, -* programming note. -*/ - reserved_allocation_mask = (0x1 <<1 ) | (0x1 << execute_only_key); - -/* register mask is in BE format */ - pkey_amr_mask = ~0x0ul ; - pkey_amr_mask &= ~(0x3ul <<pkeyshift (0 )); - - pkey_iamr_mask = ~0x0ul ; - pkey_iamr_mask &= ~(0x3ul <<pkeyshift (0 )); - pkey_iamr_mask &= ~(0x3ul <<pkeyshift (execute_only_key)); - - pkey_uamor_mask = ~0x0ul ; - pkey_uamor_mask &= ~(0x3ul <<pkeyshift (0 )); - pkey_uamor_mask &= ~(0x3ul <<pkeyshift (execute_only_key)); - -/* mark the rest of the keys as reserved and hence unavailable */ -for (i = (pkeys_total - os_reserved); i < pkeys_total; i++) { - reserved_allocation_mask |= (0x1 << i); - pkey_uamor_mask &= ~(0x3ul <<pkeyshift (i)); - } - initial_allocation_mask = reserved_allocation_mask | (0x1 <<0 );if (unlikely ((pkeys_total - os_reserved) <= execute_only_key)) {/* * Insufficient number of keys to support * execute only key. Mark it unavailable. -* Any AMR, UAMOR, IAMR bit set for -* this key is irrelevant since this key -* can never be allocated. */ execute_only_key =-1 ; + }else { +/* +* Mark the execute_only_pkey as not available for +* user allocation via pkey_alloc. +*/ + reserved_allocation_mask |= (0x1 << execute_only_key); + +/* +* Deny READ/WRITE for execute_only_key. +* Allow execute in IAMR. +*/ + default_amr |= (0x3ul <<pkeyshift (execute_only_key)); + default_iamr &= ~(0x1ul <<pkeyshift (execute_only_key)); + +/* +* Clear the uamor bits for this key. +*/ + default_uamor &= ~(0x3ul <<pkeyshift (execute_only_key)); } +/* +* Allow access for only key 0. And prevent any other modification. +*/ + default_amr &= ~(0x3ul <<pkeyshift (0 )); + default_iamr &= ~(0x1ul <<pkeyshift (0 )); + default_uamor &= ~(0x3ul <<pkeyshift (0 )); +/* +* key 0 is special in that we want to consider it an allocated +* key which is preallocated. We don't allow changing AMR bits +* w.r.t key 0. But one can pkey_free(key0) +*/ + initial_allocation_mask |= (0x1 <<0 ); + +/* +* key 1 is recommended not to be used. PowerISA(3.0) page 1015, +* programming note. +*/ + reserved_allocation_mask |= (0x1 <<1 ); + +/* +* Prevent the usage of OS reserved the keys. Update UAMOR +* for those keys. +*/ +for (i = (pkeys_total - os_reserved); i < pkeys_total; i++) { + reserved_allocation_mask |= (0x1 << i); + default_uamor &= ~(0x3ul <<pkeyshift (i)); + } +/* +* Prevent the allocation of reserved keys too. +*/ + initial_allocation_mask |= reserved_allocation_mask; +return 0 ; }
@@ -305,13 +338,13 @@ void thread_pkey_regs_init(struct thread_struct *thread)if (static_branch_likely (&pkey_disabled))return ; - thread->amr = pkey_amr_mask; - thread->iamr = pkey_iamr_mask; - thread->uamor = pkey_uamor_mask; + thread->amr = default_amr; + thread->iamr = default_iamr; + thread->uamor = default_uamor; -write_uamor (pkey_uamor_mask); -write_amr (pkey_amr_mask); -write_iamr (pkey_iamr_mask); +write_amr (default_amr); +write_iamr (default_iamr); +write_uamor (default_uamor); }int __execute_only_pkey (struct mm_struct *mm)
Signature · 1 line
2.26.2
PATCH v6 7/23 powerpc/book3s64/pkeys: Prevent key 1 modification from userspace.
In reply to Aneesh Kumar K.VKey 1 is marked reserved by ISA. Setup uamor to prevent userspace modification of the same.
--- arch/powerpc/mm/book3s64/pkeys.c | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/mm/book3s64/pkeys.c +1 -0
@@ -183,6 +183,7 @@ static int pkey_initialize(void)* programming note. */ reserved_allocation_mask |= (0x1 <<1 ); + default_uamor &= ~(0x3ul <<pkeyshift (1 ));/* * Prevent the usage of OS reserved the keys. Update UAMOR
Signature · 1 line
2.26.2
PATCH v6 8/23 powerpc/book3s64/pkeys: kill cpu feature key CPU_FTR_PKEY
In reply to Aneesh Kumar K.VWe don't use CPU_FTR_PKEY anymore. Remove the feature bit and mark it free.
--- arch/powerpc/include/asm/cputable.h | 13 ++++++------- arch/powerpc/kernel/dt_cpu_ftrs.c | 6 ------ 2 files changed, 6 insertions(+), 13 deletions(-)
arch/powerpc/include/asm/cputable.h +6 -7
@@ -198,7 +198,7 @@ static inline void cpu_feature_keys_init(void) { }#define CPU_FTR_STCX_CHECKS_ADDRESS LONG_ASM_CONST(0x0000000080000000)#define CPU_FTR_POPCNTB LONG_ASM_CONST(0x0000000100000000)#define CPU_FTR_POPCNTD LONG_ASM_CONST(0x0000000200000000) -#define CPU_FTR_PKEY LONG_ASM_CONST(0x0000000400000000) +/* LONG_ASM_CONST(0x0000000400000000) Free */ #define CPU_FTR_VMX_COPY LONG_ASM_CONST(0x0000000800000000)#define CPU_FTR_TM LONG_ASM_CONST(0x0000001000000000)#define CPU_FTR_CFAR LONG_ASM_CONST(0x0000002000000000)
@@ -438,7 +438,7 @@ static inline void cpu_feature_keys_init(void) { } CPU_FTR_DSCR | CPU_FTR_SAO | CPU_FTR_ASYM_SMT | \ CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \ CPU_FTR_CFAR | CPU_FTR_HVMODE | \ - CPU_FTR_VMX_COPY | CPU_FTR_HAS_PPR | CPU_FTR_DABRX | CPU_FTR_PKEY) + CPU_FTR_VMX_COPY | CPU_FTR_HAS_PPR | CPU_FTR_DABRX )#define CPU_FTRS_POWER8 (CPU_FTR_LWSYNC | \ CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | CPU_FTR_ARCH_206 |\ CPU_FTR_MMCRA | CPU_FTR_SMT | \
@@ -448,7 +448,7 @@ static inline void cpu_feature_keys_init(void) { } CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \ CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \ CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR | \ - CPU_FTR_ARCH_207S | CPU_FTR_TM_COMP | CPU_FTR_PKEY) + CPU_FTR_ARCH_207S | CPU_FTR_TM_COMP )#define CPU_FTRS_POWER8E (CPU_FTRS_POWER8 | CPU_FTR_PMAO_BUG)#define CPU_FTRS_POWER9 (CPU_FTR_LWSYNC | \ CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | CPU_FTR_ARCH_206 |\
@@ -459,8 +459,8 @@ static inline void cpu_feature_keys_init(void) { } CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \ CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \ CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_ARCH_207S | \ - CPU_FTR_TM_COMP | CPU_FTR_ARCH_300 | CPU_FTR_PKEY | \ - CPU_FTR_P9_TLBIE_STQ_BUG | CPU_FTR_P9_TLBIE_ERAT_BUG | CPU_FTR_P9_TIDR) + CPU_FTR_TM_COMP | CPU_FTR_ARCH_300 | CPU_FTR_P9_TLBIE_STQ_BUG | \ + CPU_FTR_P9_TLBIE_ERAT_BUG | CPU_FTR_P9_TIDR)#define CPU_FTRS_POWER9_DD2_0 (CPU_FTRS_POWER9 | CPU_FTR_P9_RADIX_PREFETCH_BUG)#define CPU_FTRS_POWER9_DD2_1 (CPU_FTRS_POWER9 | \ CPU_FTR_P9_RADIX_PREFETCH_BUG | \
@@ -477,8 +477,7 @@ static inline void cpu_feature_keys_init(void) { } CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \ CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \ CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_ARCH_207S | \ - CPU_FTR_TM_COMP | CPU_FTR_ARCH_300 | CPU_FTR_PKEY | \ - CPU_FTR_ARCH_31) + CPU_FTR_TM_COMP | CPU_FTR_ARCH_300 | CPU_FTR_ARCH_31)#define CPU_FTRS_CELL (CPU_FTR_LWSYNC | \ CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \ CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
arch/powerpc/kernel/dt_cpu_ftrs.c +0 -6
@@ -775,12 +775,6 @@ static __init void cpufeatures_cpu_quirks(void) }update_tlbie_feature_flag (version); -/* -* PKEY was not in the initial base or feature node -* specification, but it should become optional in the next -* cpu feature version sequence. -*/ - cur_cpu_spec->cpu_features |=CPU_FTR_PKEY ; }static void __init cpufeatures_setup_finished (void )
Signature · 1 line
2.26.2
PATCH v6 9/23 powerpc/book3s64/pkeys: Simplify pkey disable branch
In reply to Aneesh Kumar K.VMake the default value FALSE (pkey enabled) and set to TRUE when we find the total number of keys supported to be zero.
--- arch/powerpc/include/asm/pkeys.h | 2 +- arch/powerpc/mm/book3s64/pkeys.c | 7 +++---- 2 files changed, 4 insertions(+), 5 deletions(-)
arch/powerpc/include/asm/pkeys.h +1 -1
@@ -11,7 +11,7 @@#include <linux/jump_label.h> #include <asm/firmware.h> -DECLARE_STATIC_KEY_TRUE (pkey_disabled); +DECLARE_STATIC_KEY_FALSE (pkey_disabled);extern int pkeys_total;/* total pkeys as per device tree */ extern u32 initial_allocation_mask;/* bits set for the initially allocated keys */ extern u32 reserved_allocation_mask;/* bits set for reserved keys */
arch/powerpc/mm/book3s64/pkeys.c +3 -4
@@ -12,7 +12,7 @@#include <linux/pkeys.h> #include <linux/of_device.h> -DEFINE_STATIC_KEY_TRUE (pkey_disabled); +DEFINE_STATIC_KEY_FALSE (pkey_disabled);int pkeys_total;/* Total pkeys as per device tree */ u32 initial_allocation_mask;/* Bits set for the initially allocated keys */ /*
@@ -113,9 +113,8 @@ static int pkey_initialize(void)/* scan the device tree for pkey feature */ pkeys_total =scan_pkey_feature (); -if (pkeys_total) -static_branch_disable (&pkey_disabled); -else { +if (!pkeys_total) { +/* No support for pkey. Mark it disabled */ static_branch_enable (&pkey_disabled);return 0 ; }
Signature · 1 line
2.26.2
PATCH v6 10/23 powerpc/book3s64/pkeys: Convert pkey_total to num_pkey
In reply to Aneesh Kumar K.Vnum_pkey now represents max number of keys supported such that we return to userspace 0 - num_pkey - 1.
--- arch/powerpc/include/asm/pkeys.h | 7 +++++-- arch/powerpc/mm/book3s64/pkeys.c | 14 +++++++------- 2 files changed, 12 insertions(+), 9 deletions(-)
arch/powerpc/include/asm/pkeys.h +5 -2
@@ -12,7 +12,7 @@#include <asm/firmware.h> DECLARE_STATIC_KEY_FALSE (pkey_disabled); -extern int pkeys_total;/* total pkeys as per device tree */ +extern int num_pkey;extern u32 initial_allocation_mask;/* bits set for the initially allocated keys */ extern u32 reserved_allocation_mask;/* bits set for reserved keys */
@@ -44,7 +44,10 @@ static inline int vma_pkey(struct vm_area_struct *vma)return (vma->vm_flags &ARCH_VM_PKEY_FLAGS ) >>VM_PKEY_SHIFT ; } -#define arch_max_pkey () pkeys_total +static inline int arch_max_pkey (void ) +{ +return num_pkey; +}#define pkey_alloc_mask (pkey) (0x1 << pkey)
arch/powerpc/mm/book3s64/pkeys.c +7 -7
@@ -13,7 +13,7 @@#include <linux/of_device.h> DEFINE_STATIC_KEY_FALSE (pkey_disabled); -int pkeys_total;/* Total pkeys as per device tree */ +int num_pkey;/* Max number of pkeys supported */ u32 initial_allocation_mask;/* Bits set for the initially allocated keys */ /* * Keys marked in the reservation list cannot be allocated by userspace
@@ -93,7 +93,7 @@ static int scan_pkey_feature(void)static int pkey_initialize (void ) { -int os_reserved, i; +int pkeys_total, i;/* * We define PKEY_DISABLE_EXECUTE in addition to the arch-neutral
@@ -133,12 +133,12 @@ static int pkey_initialize(void)* The OS can manage only 8 pkeys due to its inability to represent them * in the Linux 4K PTE. Mark all other keys reserved. */ - os_reserved = pkeys_total -8 ; + num_pkey =min (8 , pkeys_total);#else - os_reserved =0 ; + num_pkey = pkeys_total;#endif -if (unlikely ((pkeys_total - os_reserved) <= execute_only_key)) { +if (unlikely (num_pkey <= execute_only_key)) {/* * Insufficient number of keys to support * execute only key. Mark it unavailable.
@@ -185,10 +185,10 @@ static int pkey_initialize(void) default_uamor &= ~(0x3ul <<pkeyshift (1 ));/* -* Prevent the usage of OS reserved the keys. Update UAMOR +* Prevent the usage of OS reserved keys. Update UAMOR * for those keys. */ -for (i = (pkeys_total - os_reserved); i < pkeys_total; i++) { +for (i = num_pkey; i < pkeys_total; i++) { reserved_allocation_mask |= (0x1 << i); default_uamor &= ~(0x3ul <<pkeyshift (i)); }
Signature · 1 line
2.26.2
PATCH v6 11/23 powerpc/book3s64/pkeys: Make initial_allocation_mask static
In reply to Aneesh Kumar K.Vinitial_allocation_mask is not used outside this file.
Also mark reserved_allocation_mask and initial_allocation_mask __ro_after_init;
--- arch/powerpc/include/asm/pkeys.h | 1 - arch/powerpc/mm/book3s64/pkeys.c | 7 +++++-- 2 files changed, 5 insertions(+), 3 deletions(-)
arch/powerpc/include/asm/pkeys.h +0 -1
@@ -13,7 +13,6 @@DECLARE_STATIC_KEY_FALSE (pkey_disabled);extern int num_pkey; -extern u32 initial_allocation_mask;/* bits set for the initially allocated keys */ extern u32 reserved_allocation_mask;/* bits set for reserved keys */ #define ARCH_VM_PKEY_FLAGS (VM_PKEY_BIT0 | VM_PKEY_BIT1 | VM_PKEY_BIT2 | \
arch/powerpc/mm/book3s64/pkeys.c +5 -2
@@ -14,11 +14,14 @@DEFINE_STATIC_KEY_FALSE (pkey_disabled);int num_pkey;/* Max number of pkeys supported */ -u32 initial_allocation_mask;/* Bits set for the initially allocated keys */ /* * Keys marked in the reservation list cannot be allocated by userspace */ -u32 reserved_allocation_mask; +u32 reserved_allocation_mask__ro_after_init ; + +/* Bits set for the initially allocated keys */ +static u32 initial_allocation_mask__ro_after_init ; +static bool pkey_execute_disable_supported;/* * Even if we allocate keys with sys_pkey_alloc(), we need to make sure
Signature · 1 line
2.26.2
PATCH v6 12/23 powerpc/book3s64/pkeys: Mark all the pkeys above max pkey as reserved
In reply to Aneesh Kumar K.VThe hypervisor can return less than max allowed pkey (for ex: 31) instead of 32. We should mark all the pkeys above max allowed as reserved so that we avoid the allocation of the wrong pkey(for ex: key 31 in the above case) by userspace.
--- arch/powerpc/mm/book3s64/pkeys.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/powerpc/mm/book3s64/pkeys.c +3 -2
@@ -189,9 +189,10 @@ static int pkey_initialize(void)/* * Prevent the usage of OS reserved keys. Update UAMOR -* for those keys. +* for those keys. Also mark the rest of the bits in the +* 32 bit mask as reserved. */ -for (i = num_pkey; i < pkeys_total; i++) { +for (i = num_pkey; i <32 ; i++) { reserved_allocation_mask |= (0x1 << i); default_uamor &= ~(0x3ul <<pkeyshift (i)); }
Signature · 1 line
2.26.2
PATCH v6 13/23 powerpc/book3s64/pkeys: Add MMU_FTR_PKEY
In reply to Aneesh Kumar K.VParse storage keys related device tree entry in early_init_devtree and enable MMU feature MMU_FTR_PKEY if pkeys are supported.
MMU feature is used instead of CPU feature because this enables us to group MMU_FTR_KUAP and MMU_FTR_PKEY in asm feature fixup code.
--- arch/powerpc/include/asm/book3s/64/mmu.h | 6 +++ arch/powerpc/include/asm/mmu.h | 9 ++++ arch/powerpc/kernel/prom.c | 5 +++ arch/powerpc/mm/book3s64/pkeys.c | 52 ++++++++++++++---------- 4 files changed, 50 insertions(+), 22 deletions(-)
arch/powerpc/include/asm/book3s/64/mmu.h +6 -0
@@ -209,6 +209,12 @@ extern int mmu_io_psize;void mmu_early_init_devtree (void );void hash__early_init_devtree (void );void radix__early_init_devtree (void ); +#ifdef CONFIG_PPC_MEM_KEYS +void pkey_early_init_devtree (void ); +#else +static inline void pkey_early_init_devtree (void ) {} +#endif +extern void hash__early_init_mmu (void );extern void radix__early_init_mmu (void );static inline void __init early_init_mmu (void )
arch/powerpc/include/asm/mmu.h +9 -0
@@ -28,6 +28,10 @@* Individual features below. */ +/* +* Support for memory protection keys. +*/ +#define MMU_FTR_PKEY ASM_CONST(0x00001000)/* * Support for 68 bit VA space. We added that from ISA 2.05 */
@@ -177,6 +181,9 @@ enum { MMU_FTR_RADIX_KUAP |#endif /* CONFIG_PPC_KUAP */ #endif /* CONFIG_PPC_RADIX_MMU */ +#ifdef CONFIG_PPC_MEM_KEYS +MMU_FTR_PKEY | +#endif 0, };
@@ -356,6 +363,8 @@ extern void setup_initial_memory_limit(phys_addr_t first_memblock_base, phys_addr_t first_memblock_size);static inline void mmu_early_init_devtree (void ) { } +static inline void pkey_early_init_devtree (void ) {} +extern void *abatron_pteptrs[2 ];#endif /* __ASSEMBLY__ */ #endif
arch/powerpc/kernel/prom.c +5 -0
@@ -815,6 +815,11 @@ void __init early_init_devtree(void *params)/* Now try to figure out if we are running on LPAR and so on */ pseries_probe_fw_features (); +/* +* Initialize pkey features and default AMR/IAMR values +*/ +pkey_early_init_devtree (); +#ifdef CONFIG_PPC_PS3 /* Identify PS3 firmware */ if (of_flat_dt_is_compatible (of_get_flat_dt_root (),"sony,ps3" ))
arch/powerpc/mm/book3s64/pkeys.c +30 -22
95 lines not on this page · raw message
Signature · 1 line
2.26.2
PATCH v6 14/23 powerpc/book3s64/kuep: Add MMU_FTR_KUEP
In reply to Aneesh Kumar K.VThis will be used to enable/disable Kernel Userspace Execution Prevention (KUEP).
--- arch/powerpc/include/asm/mmu.h | 8 ++++++++ arch/powerpc/mm/book3s64/radix_pgtable.c | 4 +++- 2 files changed, 11 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/mmu.h +8 -0
@@ -28,6 +28,10 @@* Individual features below. */ +/* +* Suppor for KUEP feature. +*/ +#define MMU_FTR_KUEP ASM_CONST(0x00000800)/* * Support for memory protection keys. */
@@ -184,6 +188,10 @@ enum {#ifdef CONFIG_PPC_MEM_KEYS MMU_FTR_PKEY |#endif +#ifdef CONFIG_PPC_KUEP +MMU_FTR_KUEP | +#endif /* CONFIG_PPC_KUAP */ + 0, };
arch/powerpc/mm/book3s64/radix_pgtable.c +3 -1
@@ -519,8 +519,10 @@ void setup_kuep(bool disabled)if (disabled || !early_radix_enabled ())return ; -if (smp_processor_id () == boot_cpuid) +if (smp_processor_id () == boot_cpuid) {pr_info ("Activating Kernel Userspace Execution Prevention\n" ); + cur_cpu_spec->mmu_features |=MMU_FTR_KUEP ; + }/* * Radix always uses key0 of the IAMR to determine if an access is
Signature · 1 line
2.26.2
PATCH v6 15/23 powerpc/book3s64/pkeys: Use pkey_execute_disable_supported
In reply to Aneesh Kumar K.VUse pkey_execute_disable_supported to check for execute key support instead of pkey_disabled.
--- arch/powerpc/include/asm/pkeys.h | 10 +--------- arch/powerpc/mm/book3s64/pkeys.c | 6 +++--- 2 files changed, 4 insertions(+), 12 deletions(-)
arch/powerpc/include/asm/pkeys.h +1 -9
@@ -126,15 +126,7 @@ static inline int mm_pkey_free(struct mm_struct *mm, int pkey)* Try to dedicate one of the protection keys to be used as an * execute-only protection key. */ -extern int __execute_only_pkey (struct mm_struct *mm); -static inline int execute_only_pkey (struct mm_struct *mm) -{ -if (static_branch_likely (&pkey_disabled)) -return -1 ; - -return __execute_only_pkey (mm); -} - +extern int execute_only_pkey (struct mm_struct *mm);extern int __arch_override_mprotect_pkey (struct vm_area_struct *vma,int prot,int pkey); static inline int arch_override_mprotect_pkey(struct vm_area_struct *vma,
arch/powerpc/mm/book3s64/pkeys.c +3 -3
@@ -23,7 +23,6 @@ u32 reserved_allocation_mask __ro_after_init;/* Bits set for the initially allocated keys */ static u32 initial_allocation_mask__ro_after_init ; -static bool pkey_execute_disable_supported;/* * Even if we allocate keys with sys_pkey_alloc(), we need to make sure * other thread still find the access denied using the same keys.
@@ -38,6 +37,7 @@ static u64 default_uamor = ~0x0UL;* We pick key 2 because 0 is special key and 1 is reserved as per ISA. */ static int execute_only_key =2 ; +static bool pkey_execute_disable_supported;#define AMR_BITS_PER_PKEY 2
@@ -151,7 +151,7 @@ void __init pkey_early_init_devtree(void) num_pkey = pkeys_total;#endif -if (unlikely (num_pkey <= execute_only_key)) { +if (unlikely (num_pkey <= execute_only_key) || !pkey_execute_disable_supported) {/* * Insufficient number of keys to support * execute only key. Mark it unavailable.
@@ -359,7 +359,7 @@ void thread_pkey_regs_init(struct thread_struct *thread)write_uamor (default_uamor); } -int __execute_only_pkey (struct mm_struct *mm) +int execute_only_pkey (struct mm_struct *mm) {return mm->context.execute_only_pkey; }
Signature · 1 line
2.26.2
PATCH v6 16/23 powerpc/book3s64/pkeys: Use MMU_FTR_PKEY instead of pkey_disabled static key
In reply to Aneesh Kumar K.VInstead of pkey_disabled static key use mmu feature MMU_FTR_PKEY.
--- arch/powerpc/include/asm/book3s/64/pkeys.h | 2 +- arch/powerpc/include/asm/pkeys.h | 14 ++++++-------- arch/powerpc/mm/book3s64/pkeys.c | 17 +++++++---------- 3 files changed, 14 insertions(+), 19 deletions(-)
arch/powerpc/include/asm/book3s/64/pkeys.h +1 -1
@@ -7,7 +7,7 @@static inline u64 vmflag_to_pte_pkey_bits (u64 vm_flags) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return 0x0UL ;if (radix_enabled ())
arch/powerpc/include/asm/pkeys.h +6 -8
@@ -11,7 +11,6 @@#include <linux/jump_label.h> #include <asm/firmware.h> -DECLARE_STATIC_KEY_FALSE (pkey_disabled);extern int num_pkey;extern u32 reserved_allocation_mask;/* bits set for reserved keys */
@@ -38,7 +37,7 @@ static inline u64 pkey_to_vmflag_bits(u16 pkey)static inline int vma_pkey (struct vm_area_struct *vma) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return 0 ;return (vma->vm_flags &ARCH_VM_PKEY_FLAGS ) >>VM_PKEY_SHIFT ; }
@@ -93,9 +92,8 @@ static inline int mm_pkey_alloc(struct mm_struct *mm)u32 all_pkeys_mask = (u32)(~(0x0 ));int ret; -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return -1 ; -/* * Are we out of pkeys? We must handle this specially because ffz() * behavior is undefined if there are no zeros.
@@ -111,7 +109,7 @@ static inline int mm_pkey_alloc(struct mm_struct *mm)static inline int mm_pkey_free (struct mm_struct *mm,int pkey) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return -1 ;if (!mm_pkey_is_allocated (mm, pkey))
@@ -132,7 +130,7 @@ extern int __arch_override_mprotect_pkey(struct vm_area_struct *vma,static inline int arch_override_mprotect_pkey (struct vm_area_struct *vma,int prot,int pkey) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return 0 ;/*
@@ -150,7 +148,7 @@ extern int __arch_set_user_pkey_access(struct task_struct *tsk, int pkey,static inline int arch_set_user_pkey_access (struct task_struct *tsk,int pkey,unsigned long init_val) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return -EINVAL ;/*
@@ -167,7 +165,7 @@ static inline int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,static inline bool arch_pkeys_enabled (void ) { -return !static_branch_likely (&pkey_disabled); +return mmu_has_feature (MMU_FTR_PKEY ); }extern void pkey_mm_init (struct mm_struct *mm);
arch/powerpc/mm/book3s64/pkeys.c +7 -10
@@ -12,8 +12,6 @@#include <linux/pkeys.h> #include <linux/of_fdt.h> - -DEFINE_STATIC_KEY_FALSE (pkey_disabled);int num_pkey;/* Max number of pkeys supported */ /* * Keys marked in the reservation list cannot be allocated by userspace
@@ -126,7 +124,6 @@ void __init pkey_early_init_devtree(void) pkeys_total =scan_pkey_feature ();if (!pkeys_total) {/* No support for pkey. Mark it disabled */ -static_branch_enable (&pkey_disabled);return ; }
@@ -216,7 +213,7 @@ void __init pkey_early_init_devtree(void)void pkey_mm_init (struct mm_struct *mm) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return ;mm_pkey_allocation_map (mm) = initial_allocation_mask; mm->context.execute_only_pkey = execute_only_key;
@@ -320,7 +317,7 @@ int __arch_set_user_pkey_access(struct task_struct *tsk, int pkey,void thread_pkey_regs_save (struct thread_struct *thread) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return ;/*
@@ -334,7 +331,7 @@ void thread_pkey_regs_save(struct thread_struct *thread)void thread_pkey_regs_restore (struct thread_struct *new_thread,struct thread_struct *old_thread) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return ;if (old_thread->amr != new_thread->amr)
@@ -347,7 +344,7 @@ void thread_pkey_regs_restore(struct thread_struct *new_thread,void thread_pkey_regs_init (struct thread_struct *thread) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return ; thread->amr = default_amr;
@@ -416,7 +413,7 @@ static bool pkey_access_permitted(int pkey, bool write, bool execute)bool arch_pte_access_permitted (u64 pte,bool write,bool execute) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return true;return pkey_access_permitted (pte_to_pkey_bits (pte), write, execute);
@@ -433,7 +430,7 @@ bool arch_pte_access_permitted(u64 pte, bool write, bool execute)bool arch_vma_access_permitted (struct vm_area_struct *vma,bool write,bool execute,bool foreign) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return true;/* * Do not enforce our key-permissions on a foreign vma.
@@ -446,7 +443,7 @@ bool arch_vma_access_permitted(struct vm_area_struct *vma, bool write,void arch_dup_pkeys (struct mm_struct *oldmm,struct mm_struct *mm) { -if (static_branch_likely (&pkey_disabled)) +if (!mmu_has_feature (MMU_FTR_PKEY ))return ;/* Duplicate the oldmm pkey state in mm: */
Signature · 1 line
2.26.2
PATCH v6 17/23 powerpc/book3s64/keys: Print information during boot.
In reply to Aneesh Kumar K.V--- arch/powerpc/mm/book3s64/pkeys.c | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/mm/book3s64/pkeys.c +1 -0
@@ -208,6 +208,7 @@ void __init pkey_early_init_devtree(void)*/ initial_allocation_mask |= reserved_allocation_mask; +pr_info ("Enabling pkeys with max key count %d\n" , num_pkey);return ; }
Signature · 1 line
2.26.2
PATCH v6 20/23 powerpc/selftest/ptrave-pkey: Rename variables to make it easier to follow code
In reply to Aneesh Kumar K.VRename variable to indicate that they are invalid values which we will use to test ptrace update of pkeys.
--- .../selftests/powerpc/ptrace/ptrace-pkey.c | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-)
tools/testing/selftests/powerpc/ptrace/ptrace-pkey.c +13 -13
@@ -44,7 +44,7 @@ struct shared_info {unsigned long amr2;/* AMR value that ptrace should refuse to write to the child. */ -unsigned long amr3; +unsigned long invalid_amr;/* IAMR value the parent expects to read from the child. */ unsigned long expected_iamr;
@@ -57,8 +57,8 @@ struct shared_info {* (even though they're valid ones) because userspace doesn't have * access to those registers. */ -unsigned long new_iamr; -unsigned long new_uamor; +unsigned long invalid_iamr; +unsigned long invalid_uamor; };static int sys_pkey_alloc (unsigned long flags,unsigned long init_access_rights)
@@ -100,7 +100,7 @@ static int child(struct shared_info *info) info->amr1 |=3ul <<pkeyshift (pkey1); info->amr2 |=3ul <<pkeyshift (pkey2); - info->amr3 |= info->amr2 |3ul <<pkeyshift (pkey3); + info->invalid_amr |= info->amr2 |3ul <<pkeyshift (pkey3);if (disable_execute) info->expected_iamr |=1ul <<pkeyshift (pkey1);
@@ -111,8 +111,8 @@ static int child(struct shared_info *info) info->expected_uamor |=3ul <<pkeyshift (pkey1) |3ul <<pkeyshift (pkey2); - info->new_iamr |=1ul <<pkeyshift (pkey1) |1ul <<pkeyshift (pkey2); - info->new_uamor |=3ul <<pkeyshift (pkey1); + info->invalid_iamr |=1ul <<pkeyshift (pkey1) |1ul <<pkeyshift (pkey2); + info->invalid_uamor |=3ul <<pkeyshift (pkey1);/* * We won't use pkey3. We just want a plausible but invalid key to test
@@ -196,9 +196,9 @@ static int parent(struct shared_info *info, pid_t pid)PARENT_SKIP_IF_UNSUPPORTED (ret, &info->child_sync);PARENT_FAIL_IF (ret, &info->child_sync); - info->amr1 = info->amr2 = info->amr3 = regs[0 ]; - info->expected_iamr = info->new_iamr = regs[1 ]; - info->expected_uamor = info->new_uamor = regs[2 ]; + info->amr1 = info->amr2 = info->invalid_amr = regs[0 ]; + info->expected_iamr = info->invalid_iamr = regs[1 ]; + info->expected_uamor = info->invalid_uamor = regs[2 ];/* Wake up child so that it can set itself up. */ ret =prod_child (&info->child_sync);
@@ -234,10 +234,10 @@ static int parent(struct shared_info *info, pid_t pid)return ret;/* Write invalid AMR value in child. */ - ret =ptrace_write_regs (pid,NT_PPC_PKEY , &info->amr3,1 ); + ret =ptrace_write_regs (pid,NT_PPC_PKEY , &info->invalid_amr,1 );PARENT_FAIL_IF (ret, &info->child_sync); -printf ("%-30s AMR: %016lx\n" , ptrace_write_running, info->amr3); +printf ("%-30s AMR: %016lx\n" , ptrace_write_running, info->invalid_amr);/* Wake up child so that it can verify it didn't change. */ ret =prod_child (&info->child_sync);
@@ -249,7 +249,7 @@ static int parent(struct shared_info *info, pid_t pid)/* Try to write to IAMR. */ regs[0 ] = info->amr1; - regs[1 ] = info->new_iamr; + regs[1 ] = info->invalid_iamr; ret =ptrace_write_regs (pid,NT_PPC_PKEY , regs,2 );PARENT_FAIL_IF (!ret, &info->child_sync);
@@ -257,7 +257,7 @@ static int parent(struct shared_info *info, pid_t pid) ptrace_write_running, regs[0], regs[1]);/* Try to write to IAMR and UAMOR. */ - regs[2 ] = info->new_uamor; + regs[2 ] = info->invalid_uamor; ret =ptrace_write_regs (pid,NT_PPC_PKEY , regs,3 );PARENT_FAIL_IF (!ret, &info->child_sync);
Signature · 1 line
2.26.2
PATCH v6 18/23 powerpc/book3s64/keys/kuap: Reset AMR/IAMR values on kexec
In reply to Aneesh Kumar K.VAs we kexec across kernels that use AMR/IAMR for different purposes we need to ensure that new kernels get kexec'd with a reset value of AMR/IAMR. For ex: the new kernel can use key 0 for kernel mapping and the old AMR value prevents access to key 0.
This patch also removes reset if IAMR and AMOR in kexec_sequence. Reset of AMOR is not needed and the IAMR reset is partial (it doesn't do the reset on secondary cpus) and is redundant with this patch.
--- arch/powerpc/include/asm/book3s/64/kexec.h | 23 ++++++++++++++++++++++ arch/powerpc/include/asm/kexec.h | 12 +++++++++++ arch/powerpc/kernel/misc_64.S | 14 ------------- arch/powerpc/kexec/core_64.c | 2 ++ arch/powerpc/mm/book3s64/pgtable.c | 3 +++ 5 files changed, 40 insertions(+), 14 deletions(-) create mode 100644 arch/powerpc/include/asm/book3s/64/kexec.h
arch/powerpc/include/asm/book3s/64/kexec.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_POWERPC_BOOK3S_64_KEXEC_H_ +#define _ASM_POWERPC_BOOK3S_64_KEXEC_H_ + + +#define reset_sprs reset_sprs +static inline void reset_sprs (void ) +{ +if (cpu_has_feature (CPU_FTR_ARCH_206 )) { +mtspr (SPRN_AMR ,0 ); +mtspr (SPRN_UAMOR ,0 ); + } + +if (cpu_has_feature (CPU_FTR_ARCH_207S )) { +mtspr (SPRN_IAMR ,0 ); + } + +/* Do we need isync()? We are going via a kexec reset */ +isync (); +} + +#endif
arch/powerpc/include/asm/kexec.h +12 -0
@@ -150,6 +150,18 @@ static inline void crash_send_ipi(void (*crash_ipi_callback)(struct pt_regs *)) }#endif /* CONFIG_KEXEC_CORE */ + +#ifdef CONFIG_PPC_BOOK3S_64 +#include <asm/book3s/64/kexec.h> +#endif + +#ifndef reset_sprs +#define reset_sprs reset_sprs +static inline void reset_sprs (void ) +{ +} +#endif +#endif /* ! __ASSEMBLY__ */ #endif /* __KERNEL__ */ #endif /* _ASM_POWERPC_KEXEC_H */
arch/powerpc/kernel/misc_64.S +0 -14
@@ -413,20 +413,6 @@ _GLOBAL(kexec_sequence) li r0,0 std r0,16(r1) -BEGIN_FTR_SECTION - /* - * This is the best time to turn AMR/IAMR off. - * key 0 is used in radix for supervisor<->user - * protection, but on hash key 0 is reserved - * ideally we want to enter with a clean state. - * NOTE, we rely on r0 being 0 from above. - */ - mtspr SPRN_IAMR,r0 -BEGIN_FTR_SECTION_NESTED(42) - mtspr SPRN_AMOR,r0 -END_FTR_SECTION_NESTED_IFSET(CPU_FTR_HVMODE, 42) -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) - /* save regs for local vars on new stack. * yes, we won't go back, but ... */
arch/powerpc/kexec/core_64.c +2 -0
@@ -152,6 +152,8 @@ static void kexec_smp_down(void *arg)if (ppc_md.kexec_cpu_down) ppc_md.kexec_cpu_down (0 ,1 ); +reset_sprs (); +kexec_smp_wait ();/* NOTREACHED */ }
arch/powerpc/mm/book3s64/pgtable.c +3 -0
@@ -15,6 +15,7 @@#include <asm/powernv.h> #include <asm/firmware.h> #include <asm/ultravisor.h> +#include <asm/kexec.h> #include <mm/mmu_decl.h> #include <trace/events/thp.h>
@@ -165,6 +166,8 @@ void mmu_cleanup_all(void)radix__mmu_cleanup_all (); else if (mmu_hash_ops.hpte_clear_all) mmu_hash_ops.hpte_clear_all(); + +reset_sprs (); }#ifdef CONFIG_MEMORY_HOTPLUG
Signature · 1 line
2.26.2
PATCH v6 19/23 powerpc/book3s64/kuap: Move UAMOR setup to key init function
In reply to Aneesh Kumar K.VUAMOR values are not application-specific. The kernel initializes its value based on different reserved keys. Remove the thread-specific UAMOR value and don't switch the UAMOR on context switch.
Move UAMOR initialization to key initialization code and remove thread_struct.uamor because it is not used anymore.
Before commit: 4a4a5e5d2aad ("powerpc/pkeys: key allocation/deallocation must not change pkey registers") we used to update uamor based on key allocation and free.
--- arch/powerpc/include/asm/book3s/64/pkeys.h | 2 ++ arch/powerpc/include/asm/processor.h | 1 - arch/powerpc/kernel/ptrace/ptrace-view.c | 27 +++++++++++++++----- arch/powerpc/kernel/smp.c | 1 + arch/powerpc/mm/book3s64/hash_utils.c | 4 +++ arch/powerpc/mm/book3s64/pkeys.c | 29 +++++++++------------- arch/powerpc/mm/book3s64/radix_pgtable.c | 4 +++ 7 files changed, 44 insertions(+), 24 deletions(-)
arch/powerpc/include/asm/book3s/64/pkeys.h +2 -0
@@ -5,6 +5,8 @@#include <asm/book3s/64/hash-pkey.h> +extern u64 __ro_after_init default_uamor; +static inline u64 vmflag_to_pte_pkey_bits (u64 vm_flags) {if (!mmu_has_feature (MMU_FTR_PKEY ))
arch/powerpc/include/asm/processor.h +0 -1
@@ -237,7 +237,6 @@ struct thread_struct {#ifdef CONFIG_PPC_MEM_KEYS unsigned long amr;unsigned long iamr; -unsigned long uamor;#endif #ifdef CONFIG_KVM_BOOK3S_32_HANDLER void * kvm_shadow_vcpu;/* KVM internal data */
arch/powerpc/kernel/ptrace/ptrace-view.c +21 -6
@@ -488,14 +488,21 @@ static int pkey_active(struct task_struct *target, const struct user_regset *regstatic int pkey_get (struct task_struct *target,const struct user_regset *regset,unsigned int pos,unsigned int count,void *kbuf,void __user *ubuf) { +int ret; +BUILD_BUG_ON (TSO (amr) +sizeof (unsigned long ) !=TSO (iamr)); -BUILD_BUG_ON (TSO (iamr) +sizeof (unsigned long ) !=TSO (uamor));if (!arch_pkeys_enabled ())return -ENODEV ; -return user_regset_copyout (&pos, &count, &kbuf, &ubuf, &target->thread.amr, -0 ,ELF_NPKEY *sizeof (unsigned long )); + ret =user_regset_copyout (&pos, &count, &kbuf, &ubuf, &target->thread.amr, +0 ,2 *sizeof (unsigned long )); +if (ret) +return ret; + + ret =user_regset_copyout (&pos, &count, &kbuf, &ubuf, &default_uamor, +2 *sizeof (unsigned long ),3 *sizeof (unsigned long )); +return ret; } static int pkey_set(struct task_struct *target, const struct user_regset *regset,
@@ -517,9 +524,17 @@ static int pkey_set(struct task_struct *target, const struct user_regset *regsetif (ret)return ret; -/* UAMOR determines which bits of the AMR can be set from userspace. */ - target->thread.amr = (new_amr & target->thread.uamor) | - (target->thread.amr & ~target->thread.uamor); +/* +* UAMOR determines which bits of the AMR can be set from userspace. +* UAMOR value 0b11 indicates that the AMR value can be modified +* from userspace. If the kernel is using a specific key, we avoid +* userspace modifying the AMR value for that key by masking them +* via UAMOR 0b00. +* +* Pick the AMR values for the keys that kernel is using. This +* will be indicated by the ~default_uamor bits. +*/ + target->thread.amr = (new_amr & default_uamor) | (target->thread.amr & ~default_uamor);return 0 ; }
arch/powerpc/kernel/smp.c +1 -0
@@ -59,6 +59,7 @@#include <asm/asm-prototypes.h> #include <asm/cpu_has_feature.h> #include <asm/ftrace.h> +#include <asm/kup.h> #ifdef DEBUG #include <asm/udbg.h>
arch/powerpc/mm/book3s64/hash_utils.c +4 -0
@@ -1110,6 +1110,10 @@ void hash__early_init_mmu_secondary(void)if (cpu_has_feature (CPU_FTR_ARCH_206 ) &&cpu_has_feature (CPU_FTR_HVMODE ))tlbiel_all (); + +#ifdef CONFIG_PPC_MEM_KEYS +mtspr (SPRN_UAMOR , default_uamor); +#endif }#endif /* CONFIG_SMP */
arch/powerpc/mm/book3s64/pkeys.c +12 -17
@@ -27,9 +27,7 @@ static u32 initial_allocation_mask __ro_after_init;*/ static u64 default_amr = ~0x0UL ;static u64 default_iamr =0x5555555555555555UL ; - -/* Allow all keys to be modified by default */ -static u64 default_uamor = ~0x0UL ; +u64 default_uamor__ro_after_init ;/* * Key used to implement PROT_EXEC mmap. Denies READ/WRITE * We pick key 2 because 0 is special key and 1 is reserved as per ISA.
@@ -122,10 +120,11 @@ void __init pkey_early_init_devtree(void)/* scan the device tree for pkey feature */ pkeys_total =scan_pkey_feature (); -if (!pkeys_total) { -/* No support for pkey. Mark it disabled */ -return ; - } +if (!pkeys_total) + goto out; + +/* Allow all keys to be modified by default */ + default_uamor = ~0x0UL ; cur_cpu_spec->mmu_features |=MMU_FTR_PKEY ;
@@ -209,6 +208,12 @@ void __init pkey_early_init_devtree(void) initial_allocation_mask |= reserved_allocation_mask;pr_info ("Enabling pkeys with max key count %d\n" , num_pkey); +out: +/* +* Setup uamor on boot cpu +*/ +mtspr (SPRN_UAMOR , default_uamor); +return ; }
@@ -251,11 +256,6 @@ static inline u64 read_uamor(void)return mfspr (SPRN_UAMOR ); } -static inline void write_uamor (u64 value) -{ -mtspr (SPRN_UAMOR , value); -} -static bool is_pkey_enabled (int pkey) {u64 uamor =read_uamor ();
@@ -326,7 +326,6 @@ void thread_pkey_regs_save(struct thread_struct *thread)*/ thread->amr =read_amr (); thread->iamr =read_iamr (); - thread->uamor =read_uamor (); } void thread_pkey_regs_restore(struct thread_struct *new_thread,
@@ -339,8 +338,6 @@ void thread_pkey_regs_restore(struct thread_struct *new_thread,write_amr (new_thread->amr);if (old_thread->iamr != new_thread->iamr)write_iamr (new_thread->iamr); -if (old_thread->uamor != new_thread->uamor) -write_uamor (new_thread->uamor); }void thread_pkey_regs_init (struct thread_struct *thread)
@@ -350,11 +347,9 @@ void thread_pkey_regs_init(struct thread_struct *thread) thread->amr = default_amr; thread->iamr = default_iamr; - thread->uamor = default_uamor;write_amr (default_amr);write_iamr (default_iamr); -write_uamor (default_uamor); }int execute_only_pkey (struct mm_struct *mm)
arch/powerpc/mm/book3s64/radix_pgtable.c +4 -0
@@ -546,6 +546,10 @@ void setup_kuap(bool disabled)/* Make sure userspace can't change the AMR */ mtspr (SPRN_UAMOR ,0 ); + +/* +* Set the default kernel AMR values on all cpus. +*/ mtspr (SPRN_AMR ,AMR_KUAP_BLOCKED );isync (); }
Signature · 1 line
2.26.2
PATCH v6 21/23 powerpc/selftest/ptrace-pkey: Update the test to mark an invalid pkey correctly
In reply to Aneesh Kumar K.V--- .../selftests/powerpc/ptrace/ptrace-pkey.c | 30 ++++++++----------- 1 file changed, 12 insertions(+), 18 deletions(-)
tools/testing/selftests/powerpc/ptrace/ptrace-pkey.c +12 -18
@@ -66,11 +66,6 @@ static int sys_pkey_alloc(unsigned long flags, unsigned long init_access_rights)return syscall (__NR_pkey_alloc, flags, init_access_rights); } -static int sys_pkey_free (int pkey) -{ -return syscall (__NR_pkey_free, pkey); -} -static int child (struct shared_info *info) {unsigned long reg;
@@ -100,7 +95,11 @@ static int child(struct shared_info *info) info->amr1 |=3ul <<pkeyshift (pkey1); info->amr2 |=3ul <<pkeyshift (pkey2); - info->invalid_amr |= info->amr2 |3ul <<pkeyshift (pkey3); +/* +* invalid amr value where we try to force write +* things which are deined by a uamor setting. +*/ + info->invalid_amr = info->amr2 | (~0x0UL & ~info->expected_uamor);if (disable_execute) info->expected_iamr |=1ul <<pkeyshift (pkey1);
@@ -111,17 +110,12 @@ static int child(struct shared_info *info) info->expected_uamor |=3ul <<pkeyshift (pkey1) |3ul <<pkeyshift (pkey2); - info->invalid_iamr |=1ul <<pkeyshift (pkey1) |1ul <<pkeyshift (pkey2); - info->invalid_uamor |=3ul <<pkeyshift (pkey1); -/* -* We won't use pkey3. We just want a plausible but invalid key to test -* whether ptrace will let us write to AMR bits we are not supposed to. -* -* This also tests whether the kernel restores the UAMOR permissions -* after a key is freed. +* Create an IAMR value different from expected value. +* Kernel will reject an IAMR and UAMOR change. */ -sys_pkey_free (pkey3); + info->invalid_iamr = info->expected_iamr | (1ul <<pkeyshift (pkey1) |1ul <<pkeyshift (pkey2)); + info->invalid_uamor = info->expected_uamor & ~(0x3ul <<pkeyshift (pkey1));printf ("%-30s AMR: %016lx pkey1: %d pkey2: %d pkey3: %d\n" , user_write, info->amr1, pkey1, pkey2, pkey3);
@@ -196,9 +190,9 @@ static int parent(struct shared_info *info, pid_t pid)PARENT_SKIP_IF_UNSUPPORTED (ret, &info->child_sync);PARENT_FAIL_IF (ret, &info->child_sync); - info->amr1 = info->amr2 = info->invalid_amr = regs[0 ]; - info->expected_iamr = info->invalid_iamr = regs[1 ]; - info->expected_uamor = info->invalid_uamor = regs[2 ]; + info->amr1 = info->amr2 = regs[0 ]; + info->expected_iamr = regs[1 ]; + info->expected_uamor = regs[2 ];/* Wake up child so that it can set itself up. */ ret =prod_child (&info->child_sync);
Signature · 1 line
2.26.2
PATCH v6 22/23 powerpc/selftest/ptrace-pkey: Don't update expected UAMOR value
In reply to Aneesh Kumar K.Vwith commit: 4a4a5e5d2aad ("powerpc/pkeys: key allocation/deallocation must not change pkey registers") we are not updating UAMOR on key allocation. So don't update the
expected uamor value in the test.
--- tools/testing/selftests/powerpc/ptrace/ptrace-pkey.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
tools/testing/selftests/powerpc/ptrace/ptrace-pkey.c +8 -3
@@ -101,15 +101,20 @@ static int child(struct shared_info *info)*/ info->invalid_amr = info->amr2 | (~0x0UL & ~info->expected_uamor); +/* +* if PKEY_DISABLE_EXECUTE succeeded we should update the expected_iamr +*/ if (disable_execute) info->expected_iamr |=1ul <<pkeyshift (pkey1);else info->expected_iamr &= ~(1ul <<pkeyshift (pkey1)); - info->expected_iamr &= ~(1ul <<pkeyshift (pkey2) |1ul <<pkeyshift (pkey3)); +/* +* We allocated pkey2 and pkey 3 above. Clear the IAMR bits. +*/ + info->expected_iamr &= ~(1ul <<pkeyshift (pkey2)); + info->expected_iamr &= ~(1ul <<pkeyshift (pkey3)); - info->expected_uamor |=3ul <<pkeyshift (pkey1) | -3ul <<pkeyshift (pkey2);/* * Create an IAMR value different from expected value. * Kernel will reject an IAMR and UAMOR change.
Signature · 1 line
2.26.2
PATCH v6 23/23 powerpc/book3s64/pkeys: Remove is_pkey_enabled()
In reply to Aneesh Kumar K.VThere is only one caller to this function and the function is wrongly named. Avoid further confusion w.r.t name and open code this at the only call site. Also remove read_uamor(). There are no users for the same after this.
--- arch/powerpc/mm/book3s64/pkeys.c | 31 +++++++++++-------------------- 1 file changed, 11 insertions(+), 20 deletions(-)
arch/powerpc/mm/book3s64/pkeys.c +11 -20
@@ -251,25 +251,6 @@ static inline void write_iamr(u64 value)mtspr (SPRN_IAMR , value); } -static inline u64 read_uamor (void ) -{ -return mfspr (SPRN_UAMOR ); -} - -static bool is_pkey_enabled (int pkey) -{ -u64 uamor =read_uamor (); -u64 pkey_bits =0x3ul <<pkeyshift (pkey); -u64 uamor_pkey_bits = (uamor & pkey_bits); - -/* -* Both the bits in UAMOR corresponding to the key should be set or -* reset. -*/ -WARN_ON (uamor_pkey_bits && (uamor_pkey_bits != pkey_bits)); -return !!(uamor_pkey_bits); -} -static inline void init_amr (int pkey,u8 init_bits) {u64 new_amr_bits = (((u64 )init_bits &0x3UL ) <<pkeyshift (pkey));
@@ -295,8 +276,18 @@ int __arch_set_user_pkey_access(struct task_struct *tsk, int pkey, {u64 new_amr_bits =0x0ul ;u64 new_iamr_bits =0x0ul ; +u64 pkey_bits, uamor_pkey_bits; -if (!is_pkey_enabled (pkey)) +/* +* Check whether the key is disabled by UAMOR. +*/ + pkey_bits =0x3ul <<pkeyshift (pkey); + uamor_pkey_bits = (default_uamor & pkey_bits); + +/* +* Both the bits in UAMOR corresponding to the key should be set +*/ +if (uamor_pkey_bits != pkey_bits)return -EINVAL ;if (init_val &PKEY_DISABLE_EXECUTE ) {
Signature · 1 line
2.26.2
Michael Ellerman
reviewerRe: [PATCH v6 03/23] powerpc/book3s64/pkeys: pkeys are supported only on hash on book3s.
In reply to Aneesh Kumar K.VQuoted 8 lines · Aneesh Kumar K.V
Move them to hash specific file and add
BUG()for radix path.--- .../powerpc/include/asm/book3s/64/hash-pkey.h | 32 ++++++++++++++++ arch/powerpc/include/asm/book3s/64/pkeys.h | 25 +++++++++++++ arch/powerpc/include/asm/pkeys.h | 37 ++++--------------- 3 files changed, 64 insertions(+), 30 deletions(-) create mode 100644 arch/powerpc/include/asm/book3s/64/hash-pkey.h create mode 100644 arch/powerpc/include/asm/book3s/64/pkeys.h
This isn't signed-off.
I assume you meant to, so I added it.
cheers
Quoted 3 lines
arch/powerpc/include/asm/book3s/64/hash-pkey.h new file +0
-0
Re: [PATCH v6 03/23] powerpc/book3s64/pkeys: pkeys are supported only on hash on book3s.
Quoted 10 lines · Michael Ellerman
Quoted 8 lines · Aneesh Kumar K.V
Move them to hash specific file and add
BUG()for radix path.--- .../powerpc/include/asm/book3s/64/hash-pkey.h | 32 ++++++++++++++++ arch/powerpc/include/asm/book3s/64/pkeys.h | 25 +++++++++++++ arch/powerpc/include/asm/pkeys.h | 37 ++++--------------- 3 files changed, 64 insertions(+), 30 deletions(-)create mode 100644
arch/powerpc/include/asm/book3s/64/hash-pkey.hcreate mode 100644arch/powerpc/include/asm/book3s/64/pkeys.h
Quoted 2 lines
This isn't signed-off.
Not sure how i missed that.
Quoted 2 lines
I assume you meant to, so I added it.
yes. Thanks for taking care of that.
-aneesh
Michael Ellerman
AppliedQuoted 10 lines · Aneesh Kumar K.V
This patch series update the pkey subsystem with more documentation and rename variables so that it is easy to follow the code. We drop the changes to support KUAP/KUEP with hash translation in this update. The changes are adding 200 cycles to null syscalls benchmark and I want to look at that closely before requesting a merge. The rest of the patches are included in this series. This should avoid having to carry a large patchset across the upstream merge. Some of the changes in here make the hash KUEP/KUAP addition simpler.
[...]
Applied to powerpc/next.
[01/23] powerpc/book3s64/pkeys: Use PVR check instead of cpu feature
https://git.kernel.org/powerpc/c/d79e7a5f26f1d179cbb915a8bf2469b6d7431c29 [02/23] powerpc/book3s64/pkeys: Fixup bit numbering https://git.kernel.org/powerpc/c/33699023f51f96ac9be38747e64967ea05e00bab [03/23] powerpc/book3s64/pkeys: pkeys are supported only on hash on book3s. https://git.kernel.org/powerpc/c/b9658f83e721ddfcee3e08b16a6628420de424c3 [04/23] powerpc/book3s64/pkeys: Move pkey related bits in the linux page table https://git.kernel.org/powerpc/c/ee8b39331f89950b0a011c7965db5694f0153166 [05/23] powerpc/book3s64/pkeys: Explain key 1 reservation details https://git.kernel.org/powerpc/c/1f404058e2911afe08417ef82f17aba6adccfc63 [06/23] powerpc/book3s64/pkeys: Simplify the key initialization https://git.kernel.org/powerpc/c/f491fe3fb41eafc7a159874040e032ad41ade210 [07/23] powerpc/book3s64/pkeys: Prevent key 1 modification from userspace. https://git.kernel.org/powerpc/c/718d9b380174eb8fe16d67769395737b79654a02 [08/23] powerpc/book3s64/pkeys: kill cpu feature key CPU_FTR_PKEY https://git.kernel.org/powerpc/c/a24204c307962214996627e3f4caa8772b9b0cf4 [09/23] powerpc/book3s64/pkeys: Simplify pkey disable branch https://git.kernel.org/powerpc/c/a4678d4b477c3d2901f101986ca01406f3b7eaea [10/23] powerpc/book3s64/pkeys: Convert pkey_total to num_pkey https://git.kernel.org/powerpc/c/c529afd7cbc71ae1dc44a31efc7c1c9db3c3a143 [11/23] powerpc/book3s64/pkeys: Make initial_allocation_mask static https://git.kernel.org/powerpc/c/3c8ab47362fe9a74f61b48efe957666a423c55a2 [12/23] powerpc/book3s64/pkeys: Mark all the pkeys above max pkey as reserved https://git.kernel.org/powerpc/c/3e4352aeb8b17eb1040ba288f586620e8294389d [13/23] powerpc/book3s64/pkeys: Add MMU_FTR_PKEY https://git.kernel.org/powerpc/c/d3cd91fb8d2e202cf8ebb6f271898aaf37ecda8f [14/23] powerpc/book3s64/kuep: Add MMU_FTR_KUEP https://git.kernel.org/powerpc/c/e10cc8715d180509a367d3ab25d40e4a1612cb2f [15/23] powerpc/book3s64/pkeys: Use pkey_execute_disable_supported https://git.kernel.org/powerpc/c/2daf298de728dc37f32d0749fa4f59db36fa7d96 [16/23] powerpc/book3s64/pkeys: Use MMU_FTR_PKEY instead of pkey_disabled static key https://git.kernel.org/powerpc/c/f7045a45115b17fe695ea7075f5213706f202edb [17/23] powerpc/book3s64/keys: Print information during boot. https://git.kernel.org/powerpc/c/7cdd3745f2d75aecc2b61368e2563ae54bfac59a [18/23] powerpc/book3s64/keys/kuap: Reset AMR/IAMR values on kexec https://git.kernel.org/powerpc/c/000a42b35a54372597f0657f6b9875b38c641864 [19/23] powerpc/book3s64/kuap: Move UAMOR setup to key init function https://git.kernel.org/powerpc/c/e0d8e991be641ba0034c67785bf86f6c097869d6 [20/23] selftests/powerpc: ptrace-pkey: Rename variables to make it easier to follow code https://git.kernel.org/powerpc/c/9a11f12e0a6c374b3ef1ce81e32ce477d28eb1b8 [21/23] selftests/powerpc: ptrace-pkey: Update the test to mark an invalid pkey correctly https://git.kernel.org/powerpc/c/0eaa3b5ca7b5a76e3783639c828498343be66a01 [22/23] selftests/powerpc: ptrace-pkey: Don't update expected UAMOR value https://git.kernel.org/powerpc/c/3563b9bea0ca7f53e4218b5e268550341a49f333 [23/23] powerpc/book3s64/pkeys: Remove is_pkey_enabled() https://git.kernel.org/powerpc/c/482b9b3948675df60c015b2155011c1f93234992
cheers