powerpc: Memory Protection Keys
Subsystems (11)
v4 cover powerpc: Memory Protection Keys
Memory protection keys enable applications to protect its address space from inadvertent access or corruption from itself.
The overall idea:
A process allocates a key and associates it with
a address range within its address space.
The process than can dynamically set read/write
permissions on the key without involving the
kernel. Any code that violates the permissions
off the address space; as defined by its associated key, will receive a segmentation fault.
This patch series enables the feature on PPC64 HPTE platform.
ISA3.0 section 5.7.13 describes the detailed specifications.
Testing:
This patch series has passed all the protection key tests available in the selftests directory. The tests are updated to work on both x86 and powerpc.
version v4:
(1) patches no more depend on the pte bits to program the hpte -- comment by Balbir
(2) documentation updates
(3) fixed a bug in the selftest.
(4) unlike x86, powerpc lets signal handler change key permission bits; the change will persist across signal handler boundaries. Earlier we allowed the signal handler to modify a field in the siginfo structure which would than be used by the kernel to program the key protection register (AMR) -- resolves a issue raised by Ben. "Calls to sys_swapcontext with a made-up context will end up with a crap AMR if done by code who didn't know about that register".
(5) these changes enable protection keys on 4k-page kernel aswell.
version v3:
(1) split the patches into smaller consumable patches.
(2) added the ability to disable execute permission on a key at creation.
(3) rename calc_pte_to_hpte_pkey_bits() to pte_to_hpte_pkey_bits() -- suggested by Anshuman
(4) some code optimization and clarity in do_page_fault()
(5) A bug fix while invalidating a hpte slot in __hash_page_4K() -- noticed by Aneesh
version v2:
(1) documentation and selftest added
(2) fixed a bug in 4k hpte backed 64k pte where page invalidation was not done correctly, and initialization of second-part-of-the-pte was not done correctly if the pte was not yet Hashed with a hpte. Reported by Aneesh.
(3) Fixed ABI breakage caused in siginfo structure. Reported by Anshuman.
version v1: Initial version
Ram Pai (17):
mm: introduce an additional vma bit for powerpc pkey
mm: ability to disable execute permission on a key at creation
x86: key creation with PKEY_DISABLE_EXECUTE disallowed
powerpc: Implement sys_pkey_alloc and sys_pkey_free system call
powerpc: store and restore the pkey state across context switches
powerpc: Implementation for sys_mprotect_pkey() system call
powerpc: make the hash functions protection-key aware
powerpc: Program HPTE key protection bits
powerpc: call the hash functions with the correct pkey value
powerpc: Macro the mask used for checking DSI exception
powerpc: Handle exceptions caused by pkey violation
powerpc: Deliver SEGV signal on pkey violation
selftest: Move protecton key selftest to arch neutral directory
selftest: PowerPC specific test updates to memory protection keys
Documentation: Move protecton key documentation to arch neutral
directory
Documentation: PowerPC specific updates to memory protection keys
procfs: display the protection-key number associated with a vma
Documentation/filesystems/proc.txt | 3 +- Documentation/vm/protection-keys.txt | 134 +++ Documentation/x86/protection-keys.txt | 85 -- Makefile | 2 +- arch/powerpc/Kconfig | 15 + arch/powerpc/include/asm/book3s/64/hash.h | 2 +- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 19 +- arch/powerpc/include/asm/book3s/64/mmu.h | 10 + arch/powerpc/include/asm/book3s/64/pgtable.h | 62 ++ arch/powerpc/include/asm/mman.h | 8 +- arch/powerpc/include/asm/mmu_context.h | 12 + arch/powerpc/include/asm/paca.h | 1 + arch/powerpc/include/asm/pkeys.h | 159 +++ arch/powerpc/include/asm/processor.h | 5 + arch/powerpc/include/asm/reg.h | 7 +- arch/powerpc/include/asm/systbl.h | 3 + arch/powerpc/include/asm/unistd.h | 6 +- arch/powerpc/include/uapi/asm/ptrace.h | 3 +- arch/powerpc/include/uapi/asm/unistd.h | 3 + arch/powerpc/kernel/asm-offsets.c | 5 + arch/powerpc/kernel/exceptions-64s.S | 18 +- arch/powerpc/kernel/process.c | 18 + arch/powerpc/kernel/signal_32.c | 5 + arch/powerpc/kernel/signal_64.c | 4 + arch/powerpc/kernel/traps.c | 49 + arch/powerpc/mm/Makefile | 1 + arch/powerpc/mm/fault.c | 22 + arch/powerpc/mm/hash64_4k.c | 4 +- arch/powerpc/mm/hash64_64k.c | 8 +- arch/powerpc/mm/hash_utils_64.c | 42 +- arch/powerpc/mm/hugepage-hash64.c | 4 +- arch/powerpc/mm/hugetlbpage-hash64.c | 5 +- arch/powerpc/mm/mem.c | 7 + arch/powerpc/mm/mmu_context_book3s64.c | 5 + arch/powerpc/mm/mmu_decl.h | 5 +- arch/powerpc/mm/pkeys.c | 256 +++++ arch/x86/kernel/fpu/xstate.c | 3 + fs/proc/task_mmu.c | 18 +- include/linux/mm.h | 18 +- include/uapi/asm-generic/mman-common.h | 4 +- tools/testing/selftests/vm/Makefile | 1 + tools/testing/selftests/vm/pkey-helpers.h | 365 ++++++ tools/testing/selftests/vm/protection_keys.c | 1488 +++++++++++++++++++++++++ tools/testing/selftests/x86/Makefile | 2 +- tools/testing/selftests/x86/pkey-helpers.h | 219 ---- tools/testing/selftests/x86/protection_keys.c | 1395 ----------------------- 46 files changed, 2755 insertions(+), 1755 deletions(-) create mode 100644 Documentation/vm/protection-keys.txt delete mode 100644 Documentation/x86/protection-keys.txt create mode 100644 arch/powerpc/include/asm/pkeys.h create mode 100644 arch/powerpc/mm/pkeys.c create mode 100644 tools/testing/selftests/vm/pkey-helpers.h create mode 100644 tools/testing/selftests/vm/protection_keys.c delete mode 100644 tools/testing/selftests/x86/pkey-helpers.h delete mode 100644 tools/testing/selftests/x86/protection_keys.c
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 1/17 mm: introduce an additional vma bit for powerpc pkey
Currently there are only 4bits in the vma flags to support 16 keys on x86. powerpc supports 32 keys, which needs 5bits. This patch introduces an addition bit in the vma flags.
Signed-off-by Ram Pai <[email protected]>
--- fs/proc/task_mmu.c | 6 +++++- include/linux/mm.h | 18 +++++++++++++----- 2 files changed, 18 insertions(+), 6 deletions(-)
fs/proc/task_mmu.c +5 -1
@@ -666,12 +666,16 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_MERGEABLE)] = "mg", [ilog2(VM_UFFD_MISSING)]= "um", [ilog2 (VM_UFFD_WP )] ="uw" , -#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS +#ifdef CONFIG_ARCH_HAS_PKEYS/* These come out via ProtectionKey: */ [ilog2(VM_PKEY_BIT0)] = "", [ilog2(VM_PKEY_BIT1)] = "", [ilog2(VM_PKEY_BIT2)] = "", [ilog2 (VM_PKEY_BIT3 )] ="" , +#endif /* CONFIG_ARCH_HAS_PKEYS */ +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +/* Additional bit in ProtectionKey: */ + [ilog2(VM_PKEY_BIT4)] = "", #endif };size_t i;
include/linux/mm.h +13 -5
@@ -208,21 +208,29 @@ extern int overcommit_kbytes_handler(struct ctl_table *, int, void __user *,#define VM_HIGH_ARCH_BIT_1 33/* bit only usable on 64-bit architectures */ #define VM_HIGH_ARCH_BIT_2 34/* bit only usable on 64-bit architectures */ #define VM_HIGH_ARCH_BIT_3 35/* bit only usable on 64-bit architectures */ +#define VM_HIGH_ARCH_BIT_4 36/* bit only usable on 64-bit arch */ #define VM_HIGH_ARCH_0 BIT(VM_HIGH_ARCH_BIT_0)#define VM_HIGH_ARCH_1 BIT(VM_HIGH_ARCH_BIT_1)#define VM_HIGH_ARCH_2 BIT(VM_HIGH_ARCH_BIT_2)#define VM_HIGH_ARCH_3 BIT(VM_HIGH_ARCH_BIT_3) +#define VM_HIGH_ARCH_4 BIT(VM_HIGH_ARCH_BIT_4)#endif /* CONFIG_ARCH_USES_HIGH_VMA_FLAGS */ -#if defined(CONFIG_X86 ) -# define VM_PAT VM_ARCH_1/* PAT reserves whole VMA at once (x86) */ -#if defined (CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS ) +#ifdef CONFIG_ARCH_HAS_PKEYS # define VM_PKEY_SHIFT VM_HIGH_ARCH_BIT_0 -# define VM_PKEY_BIT0 VM_HIGH_ARCH_0/* A protection key is a 4-bit value */ +# define VM_PKEY_BIT0 VM_HIGH_ARCH_0# define VM_PKEY_BIT1 VM_HIGH_ARCH_1# define VM_PKEY_BIT2 VM_HIGH_ARCH_2# define VM_PKEY_BIT3 VM_HIGH_ARCH_3 -#endif +#endif /* CONFIG_ARCH_HAS_PKEYS */ + +#if defined(CONFIG_PPC64_MEMORY_PROTECTION_KEYS ) +# define VM_PKEY_BIT4 VM_HIGH_ARCH_4/* additional key bit used on ppc64 */ +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ + + +#if defined(CONFIG_X86 ) +# define VM_PAT VM_ARCH_1/* PAT reserves whole VMA at once (x86) */ #elif defined(CONFIG_PPC )# define VM_SAO VM_ARCH_1/* Strong Access Ordering (powerpc) */ #elif defined(CONFIG_PARISC )
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC v4 01/17] mm: introduce an additional vma bit for powerpc pkey
Quoted 20 lines · Ram Pai
Currently there are only 4bits in the vma flags to support 16 keys on x86. powerpc supports 32 keys, which needs 5bits. This patch introduces an addition bit in the vma flags.
Signed-off-by Ram Pai <[email protected]>--- fs/proc/task_mmu.c | 6 +++++- include/linux/mm.h | 18 +++++++++++++----- 2 files changed, 18 insertions(+), 6 deletions(-)fs/proc/task_mmu.c +1
-1@@ -666,12 +666,16 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_MERGEABLE)] = "mg", [ilog2(VM_UFFD_MISSING)]= "um", [ilog2 (VM_UFFD_WP )] ="uw" , -#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS +#ifdef CONFIG_ARCH_HAS_PKEYS
fs/proc/task_mmu.c static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) · line 679
Quoted 10 lines
fs/proc/task_mmu.c +4
-0/* These come out via ProtectionKey: */ [ilog2(VM_PKEY_BIT0)] = "", [ilog2(VM_PKEY_BIT1)] = "", [ilog2(VM_PKEY_BIT2)] = "", [ilog2 (VM_PKEY_BIT3 )] ="" , +#endif /* CONFIG_ARCH_HAS_PKEYS */ +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +/* Additional bit in ProtectionKey: */ + [ilog2(VM_PKEY_BIT4)] = "", #endif
Not sure why these are linked with smap bits, but I guess the keys live in the Supervisor Mode Access Prevention area?
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 2/17 mm: ability to disable execute permission on a key at creation
In reply to Ram PaiCurrently sys_pkey_create() provides the ability to disable read and write permission on the key, at creation. powerpc has the hardware support to disable execute on a pkey as well.This patch enhances the interface to let disable execute at key creation time. x86 does not allow this. Hence the next patch will add ability in x86 to return error if PKEY_DISABLE_EXECUTE is specified.
Signed-off-by Ram Pai <[email protected]>
--- include/uapi/asm-generic/mman-common.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/uapi/asm-generic/mman-common.h +3 -1
@@ -74,7 +74,9 @@#define PKEY_DISABLE_ACCESS 0x1#define PKEY_DISABLE_WRITE 0x2 +#define PKEY_DISABLE_EXECUTE 0x4#define PKEY_ACCESS_MASK (PKEY_DISABLE_ACCESS |\ - PKEY_DISABLE_WRITE) + PKEY_DISABLE_WRITE |\ + PKEY_DISABLE_EXECUTE)#endif /* __ASM_GENERIC_MMAN_COMMON_H */
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC v4 02/17] mm: ability to disable execute permission on a key at creation
Quoted 10 lines · Ram Pai
Currently
sys_pkey_create()provides the ability to disable read and write permission on the key, at creation. powerpc has the hardware support to disable execute on a pkey as well.This patch enhances the interface to let disable execute at key creation time. x86 does not allow this. Hence the next patch will add ability in x86 to return error ifPKEY_DISABLE_EXECUTEis specified.Signed-off-by Ram Pai <[email protected]>---
Acked-by Balbir Singh <[email protected]>
RFC v4 3/17 x86: key creation with PKEY_DISABLE_EXECUTE disallowed
In reply to Ram Paix86 does not support disabling execute permissions on a pkey.
Signed-off-by Ram Pai <[email protected]>
--- arch/x86/kernel/fpu/xstate.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kernel/fpu/xstate.c +3 -0
@@ -900,6 +900,9 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,if (!boot_cpu_has (X86_FEATURE_OSPKE ))return -EINVAL ; +if (init_val &PKEY_DISABLE_EXECUTE ) +return -EINVAL ; +/* Set the bits we need in PKRU: */ if (init_val &PKEY_DISABLE_ACCESS ) new_pkru_bits |=PKRU_AD_BIT ;
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC v4 03/17] x86: key creation with PKEY_DISABLE_EXECUTE disallowed
Quoted 12 lines · Ram Pai
x86 does not support disabling execute permissions on a pkey.
Signed-off-by Ram Pai <[email protected]>--- arch/x86/kernel/fpu/xstate.c | 3 +++ 1 file changed, 3 insertions(+)arch/x86/kernel/fpu/xstate.c +0
-0
arch/x86/kernel/fpu/xstate.c int arch_set_user_pkey_access(struct task_struct *tsk, int pkey, · line 908
Quoted 9 lines
arch/x86/kernel/fpu/xstate.c +3
-0@@ -900,6 +900,9 @@ int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,if (!boot_cpu_has (X86_FEATURE_OSPKE ))return -EINVAL ; +if (init_val &PKEY_DISABLE_EXECUTE ) +return -EINVAL ; +/* Set the bits we need in PKRU: */ if (init_val &PKEY_DISABLE_ACCESS ) new_pkru_bits |=PKRU_AD_BIT ;
I am not an x86 expert. IIUC, execute disable is done via allocating an execute_only_pkey and checking vma_key via AD + vma_flags against VM_EXEC.
Your patch looks good to me
Acked-by Balbir Singh <[email protected]>
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 4/17 powerpc: Implement sys_pkey_alloc and sys_pkey_free system call
In reply to Ram PaiSys_pkey_alloc() allocates and returns available pkey Sys_pkey_free() frees up the pkey.
Total 32 keys are supported on powerpc. However pkey 0,1 and 31 are reserved. So effectively we have 29 pkeys.
Each key can be initialized to disable read, write and execute permissions. On powerpc a key can be initialize to disable execute.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/Kconfig | 15 ++++ arch/powerpc/include/asm/book3s/64/mmu.h | 10 +++ arch/powerpc/include/asm/book3s/64/pgtable.h | 62 ++++++++++++++ arch/powerpc/include/asm/pkeys.h | 124 +++++++++++++++++++++++++++ arch/powerpc/include/asm/systbl.h | 2 + arch/powerpc/include/asm/unistd.h | 4 +- arch/powerpc/include/uapi/asm/unistd.h | 2 + arch/powerpc/mm/Makefile | 1 + arch/powerpc/mm/mmu_context_book3s64.c | 5 ++ arch/powerpc/mm/pkeys.c | 88 +++++++++++++++++++ 10 files changed, 310 insertions(+), 3 deletions(-) create mode 100644 arch/powerpc/include/asm/pkeys.h create mode 100644 arch/powerpc/mm/pkeys.c
arch/powerpc/Kconfig +15 -0
@@ -871,6 +871,21 @@ config SECCOMP If unsure, say Y. Only embedded should say N here. +config PPC64_MEMORY_PROTECTION_KEYS + prompt "PowerPC Memory Protection Keys" + def_bool y + # Note: only available in 64-bit mode + depends on PPC64 + select ARCH_USES_HIGH_VMA_FLAGS + select ARCH_HAS_PKEYS + ---help--- + Memory Protection Keys provides a mechanism for enforcing + page-based protections, but without requiring modification of the + page tables when an application changes protection domains. + + For details, see Documentation/powerpc/protection-keys.txt + + If unsure, say y. endmenu config ISA_DMA_API
arch/powerpc/include/asm/book3s/64/mmu.h +10 -0
@@ -108,6 +108,16 @@ struct patb_entry {#ifdef CONFIG_SPAPR_TCE_IOMMU struct list_head iommu_group_mem_list;#endif + +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +/* +* Each bit represents one protection key. +* bit set -> key allocated +* bit unset -> key available for allocation +*/ +u32 pkey_allocation_map; +s16 execute_only_pkey;/* key holding execute-only protection */ +#endif } mm_context_t;/*
arch/powerpc/include/asm/book3s/64/pgtable.h +62 -0
68 lines not on this page · raw message
arch/powerpc/include/asm/pkeys.h new file +124 -0
124 lines not on this page · raw message
arch/powerpc/include/asm/systbl.h +2 -0
@@ -388,3 +388,5 @@ COMPAT_SYS_SPU(pwritev2) SYSCALL(kexec_file_load) SYSCALL(statx) +SYSCALL(pkey_alloc) +SYSCALL(pkey_free)
arch/powerpc/include/asm/unistd.h +1 -3
@@ -12,13 +12,11 @@#include <uapi/asm/unistd.h> -#define NR_syscalls 384 +#define NR_syscalls 386#define __NR__exit __NR_exit#define __IGNORE_pkey_mprotect -#define __IGNORE_pkey_alloc -#define __IGNORE_pkey_free#ifndef __ASSEMBLY__
arch/powerpc/include/uapi/asm/unistd.h +2 -0
@@ -394,5 +394,7 @@#define __NR_pwritev2 381#define __NR_kexec_file_load 382#define __NR_statx 383 +#define __NR_pkey_alloc 384 +#define __NR_pkey_free 385#endif /* _UAPI_ASM_POWERPC_UNISTD_H_ */
arch/powerpc/mm/Makefile +1 -0
@@ -45,3 +45,4 @@ obj-$(CONFIG_PPC_COPRO_BASE) += copro_fault.o obj-$(CONFIG_SPAPR_TCE_IOMMU) += mmu_context_iommu.o obj-$(CONFIG_PPC_PTDUMP) += dump_linuxpagetables.o obj-$(CONFIG_PPC_HTDUMP) += dump_hashpagetable.o +obj-$(CONFIG_PPC64_MEMORY_PROTECTION_KEYS) += pkeys.o
arch/powerpc/mm/mmu_context_book3s64.c +5 -0
@@ -16,6 +16,7 @@#include <linux/string.h> #include <linux/types.h> #include <linux/mm.h> +#include <linux/pkeys.h> #include <linux/spinlock.h> #include <linux/idr.h> #include <linux/export.h>
@@ -120,6 +121,10 @@ static int hash__init_new_context(struct mm_struct *mm)subpage_prot_init_new_context (mm); +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +pkey_mm_init (mm); +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +return index; }
arch/powerpc/mm/pkeys.c new file +88 -0
88 lines not on this page · raw message
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 5/17 powerpc: store and restore the pkey state across context switches
In reply to Ram PaiStore and restore the AMR, IAMR and UMOR register state of the task before scheduling out and after scheduling in, respectively.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/processor.h | 5 +++++ arch/powerpc/kernel/process.c | 18 ++++++++++++++++++ 2 files changed, 23 insertions(+)
arch/powerpc/include/asm/processor.h +5 -0
@@ -310,6 +310,11 @@ struct thread_struct {struct thread_vr_state ckvr_state;/* Checkpointed VR state */ unsigned long ckvrsave;/* Checkpointed VRSAVE */ #endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +unsigned long amr; +unsigned long iamr; +unsigned long uamor; +#endif #ifdef CONFIG_KVM_BOOK3S_32_HANDLER void * kvm_shadow_vcpu;/* KVM internal data */ #endif /* CONFIG_KVM_BOOK3S_32_HANDLER */
arch/powerpc/kernel/process.c +18 -0
@@ -1096,6 +1096,11 @@ static inline void save_sprs(struct thread_struct *t) t->tar =mfspr (SPRN_TAR ); }#endif +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + t->amr =mfspr (SPRN_AMR ); + t->iamr =mfspr (SPRN_IAMR ); + t->uamor =mfspr (SPRN_UAMOR ); +#endif } static inline void restore_sprs(struct thread_struct *old_thread,
@@ -1131,6 +1136,14 @@ static inline void restore_sprs(struct thread_struct *old_thread,mtspr (SPRN_TAR , new_thread->tar); }#endif +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +if (old_thread->amr != new_thread->amr) +mtspr (SPRN_AMR , new_thread->amr); +if (old_thread->iamr != new_thread->iamr) +mtspr (SPRN_IAMR , new_thread->iamr); +if (old_thread->uamor != new_thread->uamor) +mtspr (SPRN_UAMOR , new_thread->uamor); +#endif } struct task_struct *__switch_to(struct task_struct *prev,
@@ -1686,6 +1699,11 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp) current->thread.tm_texasr =0 ; current->thread.tm_tfiar =0 ;#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */ +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + current->thread.amr =0x0ul ; + current->thread.iamr =0x0ul ; + current->thread.uamor =0x0ul ; +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ }EXPORT_SYMBOL (start_thread);
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 6/17 powerpc: Implementation for sys_mprotect_pkey() system call
In reply to Ram PaiThis system call, associates the pkey with vma corresponding to the given address range.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/mman.h | 8 ++- arch/powerpc/include/asm/pkeys.h | 17 ++++++- arch/powerpc/include/asm/systbl.h | 1 + arch/powerpc/include/asm/unistd.h | 4 +- arch/powerpc/include/uapi/asm/unistd.h | 1 + arch/powerpc/mm/pkeys.c | 93 +++++++++++++++++++++++++++++++++- 6 files changed, 117 insertions(+), 7 deletions(-)
arch/powerpc/include/asm/mman.h +7 -1
@@ -13,6 +13,7 @@#include <asm/cputable.h> #include <linux/mm.h> +#include <linux/pkeys.h> #include <asm/cpu_has_feature.h> /*
@@ -22,7 +23,12 @@static inline unsigned long arch_calc_vm_prot_bits (unsigned long prot,unsigned long pkey) { -return (prot &PROT_SAO ) ?VM_SAO :0 ; +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +return (((prot &PROT_SAO ) ?VM_SAO :0 ) | +pkey_to_vmflag_bits (pkey)); +#else +return ((prot &PROT_SAO ) ?VM_SAO :0 ); +#endif }#define arch_calc_vm_prot_bits (prot, pkey) arch_calc_vm_prot_bits(prot, pkey)
arch/powerpc/include/asm/pkeys.h +16 -1
@@ -14,6 +14,15 @@ VM_PKEY_BIT3 | \ VM_PKEY_BIT4) +static inline unsigned long pkey_to_vmflag_bits (int pkey) +{ +return (((pkey &0x1UL ) ?VM_PKEY_BIT0 :0x0UL ) | + ((pkey &0x2UL ) ?VM_PKEY_BIT1 :0x0UL ) | + ((pkey &0x4UL ) ?VM_PKEY_BIT2 :0x0UL ) | + ((pkey &0x8UL ) ?VM_PKEY_BIT3 :0x0UL ) | + ((pkey &0x10UL ) ?VM_PKEY_BIT4 :0x0UL )); +} +/* * Bits are in BE format. * NOTE: key 31, 1, 0 are not used.
@@ -42,6 +51,12 @@#define mm_set_pkey_is_reserved (mm, pkey) (PKEY_INITIAL_ALLOCAION & \ pkeybit_mask(pkey)) + +static inline int vma_pkey (struct vm_area_struct *vma) +{ +return (vma->vm_flags &ARCH_VM_PKEY_FLAGS ) >>VM_PKEY_SHIFT ; +} +static inline bool mm_pkey_is_allocated (struct mm_struct *mm,int pkey) {/* a reserved key is never considered as 'explicitly allocated' */
@@ -114,7 +129,7 @@ static inline int arch_set_user_pkey_access(struct task_struct *tsk, int pkey,return __arch_set_user_pkey_access (tsk, pkey, init_val); } -static inline pkey_mm_init(struct mm_struct *mm) +static inline void pkey_mm_init (struct mm_struct *mm) {mm_pkey_allocation_map (mm) =PKEY_INITIAL_ALLOCAION ;/* -1 means unallocated or invalid */
arch/powerpc/include/asm/systbl.h +1 -0
@@ -390,3 +390,4 @@ SYSCALL(statx) SYSCALL(pkey_alloc) SYSCALL(pkey_free) +SYSCALL(pkey_mprotect)
arch/powerpc/include/asm/unistd.h +1 -3
@@ -12,12 +12,10 @@#include <uapi/asm/unistd.h> -#define NR_syscalls 386 +#define NR_syscalls 387#define __NR__exit __NR_exit -#define __IGNORE_pkey_mprotect -#ifndef __ASSEMBLY__#include <linux/types.h>
arch/powerpc/include/uapi/asm/unistd.h +1 -0
@@ -396,5 +396,6 @@#define __NR_statx 383#define __NR_pkey_alloc 384#define __NR_pkey_free 385 +#define __NR_pkey_mprotect 386#endif /* _UAPI_ASM_POWERPC_UNISTD_H_ */
arch/powerpc/mm/pkeys.c +91 -2
109 lines not on this page · raw message
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 7/17 powerpc: make the hash functions protection-key aware
In reply to Ram PaiPrepare the hash functions to be aware of protection keys. This key will later be used to program the HPTE.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/book3s/64/hash.h | 2 +- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 14 ++++++----- arch/powerpc/mm/hash64_4k.c | 4 ++-- arch/powerpc/mm/hash64_64k.c | 8 +++---- arch/powerpc/mm/hash_utils_64.c | 34 ++++++++++++++++++--------- arch/powerpc/mm/hugepage-hash64.c | 4 ++-- arch/powerpc/mm/hugetlbpage-hash64.c | 5 ++-- arch/powerpc/mm/mem.c | 1 + arch/powerpc/mm/mmu_decl.h | 5 +++- 9 files changed, 48 insertions(+), 29 deletions(-)
arch/powerpc/include/asm/book3s/64/hash.h +1 -1
@@ -92,7 +92,7 @@ static inline int hash__pgd_bad(pgd_t pgd)extern void hpte_need_flush (struct mm_struct *mm,unsigned long addr,pte_t *ptep,unsigned long pte,int huge); -extern unsigned long htab_convert_pte_flags (unsigned long pteflags); +extern unsigned long htab_convert_pte_flags (unsigned long pteflags,int pkey);/* Atomic PTE updates */ static inline unsigned long hash__pte_update(struct mm_struct *mm, unsigned long addr,
arch/powerpc/include/asm/book3s/64/mmu-hash.h +8 -6
@@ -430,11 +430,11 @@ static inline unsigned long hpt_hash(unsigned long vpn,#define HPTE_NOHPTE_UPDATE 0x2extern int __hash_page_4K (unsigned long ea,unsigned long access, -unsigned long vsid,pte_t *ptep,unsigned long trap, -unsigned long flags,int ssize,int subpage_prot); +unsigned long vsid,pte_t *ptep,unsigned long trap, +unsigned long flags,int ssize,int subpage_prot,int pkey);extern int __hash_page_64K (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap, -unsigned long flags,int ssize); +unsigned long flags,int ssize,int pkey);struct mm_struct ;unsigned int hash_page_do_lazy_icache (unsigned int pp,pte_t pte,int trap); extern int hash_page_mm(struct mm_struct *mm, unsigned long ea,
@@ -444,16 +444,18 @@ extern int hash_page(unsigned long ea, unsigned long access, unsigned long trap, unsigned long dsisr);int __hash_page_huge (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap,unsigned long flags, -int ssize,unsigned int shift,unsigned int mmu_psize); +int ssize,unsigned int shift,unsigned int mmu_psize, +int pkey);#ifdef CONFIG_TRANSPARENT_HUGEPAGE extern int __hash_page_thp (unsigned long ea,unsigned long access,unsigned long vsid,pmd_t *pmdp,unsigned long trap, -unsigned long flags,int ssize,unsigned int psize); +unsigned long flags,int ssize,unsigned int psize, +int pkey);#else static inline int __hash_page_thp (unsigned long ea,unsigned long access,unsigned long vsid,pmd_t *pmdp,unsigned long trap,unsigned long flags, -int ssize,unsigned int psize) +int ssize,unsigned int psize,int pkey) {BUG ();return -1 ;
arch/powerpc/mm/hash64_4k.c +2 -2
@@ -18,7 +18,7 @@int __hash_page_4K (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap,unsigned long flags, -int ssize,int subpg_prot) +int ssize,int subpg_prot,int pkey) {unsigned long hpte_group;unsigned long rflags, pa;
@@ -53,7 +53,7 @@ int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid,* PP bits. _PAGE_USER is already PP bit 0x2, so we only * need to add in 0x1 if it's a read-only user page */ - rflags =htab_convert_pte_flags (new_pte); + rflags =htab_convert_pte_flags (new_pte, pkey);if (cpu_has_feature (CPU_FTR_NOEXECUTE ) && !cpu_has_feature (CPU_FTR_COHERENT_ICACHE ))
arch/powerpc/mm/hash64_64k.c +4 -4
@@ -47,7 +47,7 @@ static unsigned long mark_subptegroup_valid(unsigned long ptev, unsigned long inint __hash_page_4K (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap,unsigned long flags, -int ssize,int subpg_prot) +int ssize,int subpg_prot,int pkey) {real_pte_t rpte;unsigned long *hidxp;
@@ -85,7 +85,7 @@ int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid,* Handle the subpage protection bits */ subpg_pte = new_pte & ~subpg_prot; - rflags =htab_convert_pte_flags (subpg_pte); + rflags =htab_convert_pte_flags (subpg_pte, pkey);if (cpu_has_feature (CPU_FTR_NOEXECUTE ) && !cpu_has_feature (CPU_FTR_COHERENT_ICACHE )) {
@@ -219,7 +219,7 @@ int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid,int __hash_page_64K (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap, -unsigned long flags,int ssize) +unsigned long flags,int ssize,int pkey) {unsigned long hpte_group;unsigned long rflags, pa;
@@ -256,7 +256,7 @@ int __hash_page_64K(unsigned long ea, unsigned long access, new_pte |= _PAGE_DIRTY; }while (!pte_xchg (ptep,__pte (old_pte),__pte (new_pte))); - rflags =htab_convert_pte_flags (new_pte); + rflags =htab_convert_pte_flags (new_pte, pkey);if (cpu_has_feature (CPU_FTR_NOEXECUTE ) && !cpu_has_feature (CPU_FTR_COHERENT_ICACHE ))
arch/powerpc/mm/hash_utils_64.c +23 -11
@@ -35,6 +35,7 @@#include <linux/memblock.h> #include <linux/context_tracking.h> #include <linux/libfdt.h> +#include <linux/pkeys.h> #include <asm/debugfs.h> #include <asm/processor.h>
@@ -176,7 +177,7 @@* - We make sure R is always set and never lost * - C is _PAGE_DIRTY, and *should* always be set for a writeable mapping */ -unsigned long htab_convert_pte_flags (unsigned long pteflags) +unsigned long htab_convert_pte_flags (unsigned long pteflags,int pkey) {unsigned long rflags =0 ;
@@ -244,7 +245,7 @@ int htab_bolt_mapping(unsigned long vstart, unsigned long vend, shift = mmu_psize_defs[psize].shift; step =1 << shift; - prot =htab_convert_pte_flags (prot); + prot =htab_convert_pte_flags (prot,0 );DBG ("htab_bolt_mapping(%lx..%lx -> %lx (%lx,%d,%d)\n" , vstart, vend, pstart, prot, psize, ssize);
@@ -1228,7 +1229,7 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea,unsigned hugeshift;const struct cpumask *tmp;int rc, user_region =0 ; -int psize, ssize; +int psize, ssize, pkey =0 ;DBG_LOW ("hash_page(ea=%016lx, access=%lx, trap=%lx\n" , ea, access, trap);
@@ -1317,11 +1318,13 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea,if (hugeshift) {if (is_thp) rc =__hash_page_thp (ea, access, vsid, (pmd_t *)ptep, - trap, flags, ssize, psize); + trap, flags, ssize, psize, + pkey);#ifdef CONFIG_HUGETLB_PAGE else rc =__hash_page_huge (ea, access, vsid, ptep, trap, - flags, ssize, hugeshift, psize); + flags, ssize, hugeshift, psize, + pkey);#else else {/*
@@ -1381,7 +1384,8 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea,#ifdef CONFIG_PPC_64K_PAGES if (psize ==MMU_PAGE_64K ) rc =__hash_page_64K (ea, access, vsid, ptep, trap, - flags, ssize); + flags, ssize, + pkey); else#endif /* CONFIG_PPC_64K_PAGES */ {
@@ -1390,7 +1394,8 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea, rc =-2 ; else rc =__hash_page_4K (ea, access, vsid, ptep, trap, - flags, ssize, spp); + flags, ssize, spp, + pkey); }/* Dump some info in case of hash insertion failure, they should
@@ -1486,8 +1491,9 @@ static bool should_hash_preload(struct mm_struct *mm, unsigned long ea) }#endif -void hash_preload (struct mm_struct *mm,unsigned long ea, -unsigned long access,unsigned long trap) +void hash_preload_pkey (struct mm_struct *mm,unsigned long ea, +unsigned long access,unsigned long trap, +int pkey) {int hugepage_shift;unsigned long vsid;
@@ -1548,11 +1554,11 @@ void hash_preload(struct mm_struct *mm, unsigned long ea,#ifdef CONFIG_PPC_64K_PAGES if (mm->context.user_psize ==MMU_PAGE_64K ) rc =__hash_page_64K (ea, access, vsid, ptep, trap, - update_flags, ssize); + update_flags, ssize, pkey); else#endif /* CONFIG_PPC_64K_PAGES */ rc =__hash_page_4K (ea, access, vsid, ptep, trap, update_flags, - ssize,subpage_protection (mm, ea)); + ssize,subpage_protection (mm, ea), pkey);/* Dump some info in case of hash insertion failure, they should * never happen so it is really useful to know if/when they do
@@ -1566,6 +1572,12 @@ void hash_preload(struct mm_struct *mm, unsigned long ea,local_irq_restore (flags); } +void hash_preload (struct mm_struct *mm,unsigned long ea, +unsigned long access,unsigned long trap) +{ +hash_preload_pkey (mm, ea, access, trap,0 ); +} +#ifdef CONFIG_PPC_TRANSACTIONAL_MEM static inline void tm_flush_hash_page (int local) {
arch/powerpc/mm/hugepage-hash64.c +2 -2
@@ -20,7 +20,7 @@int __hash_page_thp (unsigned long ea,unsigned long access,unsigned long vsid,pmd_t *pmdp,unsigned long trap,unsigned long flags, -int ssize,unsigned int psize) +int ssize,unsigned int psize,int pkey) {unsigned int index, valid;unsigned char *hpte_slot_array;
@@ -51,7 +51,7 @@ int __hash_page_thp(unsigned long ea, unsigned long access, unsigned long vsid, new_pmd |= _PAGE_DIRTY; }while (!pmd_xchg (pmdp,__pmd (old_pmd),__pmd (new_pmd))); - rflags =htab_convert_pte_flags (new_pmd); + rflags =htab_convert_pte_flags (new_pmd, pkey);#if 0 if (!cpu_has_feature (CPU_FTR_COHERENT_ICACHE )) {
arch/powerpc/mm/hugetlbpage-hash64.c +3 -2
@@ -20,7 +20,8 @@ extern long hpte_insert_repeating(unsigned long hash, unsigned long vpn,int __hash_page_huge (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap,unsigned long flags, -int ssize,unsigned int shift,unsigned int mmu_psize) +int ssize,unsigned int shift,unsigned int mmu_psize, +int pkey) {unsigned long vpn;unsigned long old_pte, new_pte;
@@ -60,7 +61,7 @@ int __hash_page_huge(unsigned long ea, unsigned long access, unsigned long vsid, new_pte |= _PAGE_DIRTY; }while (!pte_xchg (ptep,__pte (old_pte),__pte (new_pte))); - rflags =htab_convert_pte_flags (new_pte); + rflags =htab_convert_pte_flags (new_pte, pkey); sz = ((1UL ) << shift);if (!cpu_has_feature (CPU_FTR_COHERENT_ICACHE ))
arch/powerpc/mm/mem.c +1 -0
@@ -36,6 +36,7 @@#include <linux/hugetlb.h> #include <linux/slab.h> #include <linux/vmalloc.h> +#include <linux/pkeys.h> #include <asm/pgalloc.h> #include <asm/prom.h>
arch/powerpc/mm/mmu_decl.h +4 -1
@@ -82,10 +82,13 @@ static inline void _tlbivax_bcast(unsigned long address, unsigned int pid,#else /* CONFIG_PPC_MMU_NOHASH */ +extern void hash_preload_pkey (struct mm_struct *mm,unsigned long ea, +unsigned long access,unsigned long trap, +int pkey); +extern void hash_preload (struct mm_struct *mm,unsigned long ea,unsigned long access,unsigned long trap); -extern void _tlbie (unsigned long address);extern void _tlbia (void );
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 8/17 powerpc: Program HPTE key protection bits
In reply to Ram PaiMap the PTE protection key bits to the HPTE key protection bits, while creating HPTE entries.
Signed-off-by Ram Pai <[email protected]>
--- Makefile | 2 +- arch/powerpc/include/asm/book3s/64/mmu-hash.h | 5 +++++ arch/powerpc/include/asm/pkeys.h | 9 +++++++++ arch/powerpc/mm/hash_utils_64.c | 4 ++++ 4 files changed, 19 insertions(+), 1 deletion(-)
Makefile +1 -1
@@ -1,7 +1,7 @@ VERSION = 4 PATCHLEVEL = 12 SUBLEVEL = 0 -EXTRAVERSION = -rc3 +EXTRAVERSION = -rc3-64k NAME = Fearless Coyote # *DOCUMENTATION*
arch/powerpc/include/asm/book3s/64/mmu-hash.h +5 -0
@@ -90,6 +90,8 @@#define HPTE_R_PP0 ASM_CONST(0x8000000000000000)#define HPTE_R_TS ASM_CONST(0x4000000000000000)#define HPTE_R_KEY_HI ASM_CONST(0x3000000000000000) +#define HPTE_R_KEY_BIT0 ASM_CONST(0x2000000000000000) +#define HPTE_R_KEY_BIT1 ASM_CONST(0x1000000000000000)#define HPTE_R_RPN_SHIFT 12#define HPTE_R_RPN ASM_CONST(0x0ffffffffffff000)#define HPTE_R_RPN_3_0 ASM_CONST(0x01fffffffffff000)
@@ -104,6 +106,9 @@#define HPTE_R_C ASM_CONST(0x0000000000000080)#define HPTE_R_R ASM_CONST(0x0000000000000100)#define HPTE_R_KEY_LO ASM_CONST(0x0000000000000e00) +#define HPTE_R_KEY_BIT2 ASM_CONST(0x0000000000000800) +#define HPTE_R_KEY_BIT3 ASM_CONST(0x0000000000000400) +#define HPTE_R_KEY_BIT4 ASM_CONST(0x0000000000000200)#define HPTE_V_1TB_SEG ASM_CONST(0x4000000000000000)#define HPTE_V_VRMA_MASK ASM_CONST(0x4001ffffff000000)
arch/powerpc/include/asm/pkeys.h +9 -0
@@ -23,6 +23,15 @@ static inline unsigned long pkey_to_vmflag_bits(int pkey) ((pkey & 0x10UL) ? VM_PKEY_BIT4 : 0x0UL)); } +static inline unsigned long pkey_to_hpte_pkey_bits (int pkey) +{ +return (((pkey &0x10 ) ?HPTE_R_KEY_BIT0 :0x0UL ) | + ((pkey &0x8 ) ?HPTE_R_KEY_BIT1 :0x0UL ) | + ((pkey &0x4 ) ?HPTE_R_KEY_BIT2 :0x0UL ) | + ((pkey &0x2 ) ?HPTE_R_KEY_BIT3 :0x0UL ) | + ((pkey &0x1 ) ?HPTE_R_KEY_BIT4 :0x0UL )); +} +/* * Bits are in BE format. * NOTE: key 31, 1, 0 are not used.
arch/powerpc/mm/hash_utils_64.c +4 -0
@@ -231,6 +231,10 @@ unsigned long htab_convert_pte_flags(unsigned long pteflags, int pkey)*/ rflags |=HPTE_R_M ; +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + rflags |=pkey_to_hpte_pkey_bits (pkey); +#endif +return rflags; }
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 9/17 powerpc: call the hash functions with the correct pkey value
In reply to Ram PaiPass the correct protection key value to the hash functions on page fault.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/pkeys.h | 11 +++++++++++ arch/powerpc/mm/hash_utils_64.c | 4 ++++ arch/powerpc/mm/mem.c | 6 ++++++ 3 files changed, 21 insertions(+)
arch/powerpc/include/asm/pkeys.h +11 -0
@@ -74,6 +74,17 @@ static inline bool mm_pkey_is_allocated(struct mm_struct *mm, int pkey) }/* +* return the protection key of the vma corresponding to the +* given effective address @ea. +*/ +static inline int mm_pkey (struct mm_struct *mm,unsigned long ea) +{ +struct vm_area_struct *vma =find_vma (mm, ea); +int pkey = vma ?vma_pkey (vma) :0 ; +return pkey; +} + +/* * Returns a positive, 5-bit key on success, or -1 on failure. */ static inline int mm_pkey_alloc (struct mm_struct *mm)
arch/powerpc/mm/hash_utils_64.c +4 -0
@@ -1319,6 +1319,10 @@ int hash_page_mm(struct mm_struct *mm, unsigned long ea, goto bail; } +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + pkey =mm_pkey (mm, ea); +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +if (hugeshift) {if (is_thp) rc = __hash_page_thp(ea, access, vsid, (pmd_t *)ptep,
arch/powerpc/mm/mem.c +6 -0
@@ -541,8 +541,14 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long address, return; } +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +hash_preload_pkey (vma->vm_mm, address, access, trap,vma_pkey (vma)); +#else hash_preload (vma->vm_mm, address, access, trap); +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +#endif /* CONFIG_PPC_STD_MMU */ +#if (defined(CONFIG_PPC_BOOK3E_64 ) || defined(CONFIG_PPC_FSL_BOOK3E )) \ && defined(CONFIG_HUGETLB_PAGE )if (is_vm_hugetlb_page (vma))
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC v4 09/17] powerpc: call the hash functions with the correct pkey value
Quoted 20 lines · Ram Pai
Pass the correct protection key value to the hash functions on page fault.
Signed-off-by Ram Pai <[email protected]>--- arch/powerpc/include/asm/pkeys.h | 11 +++++++++++ arch/powerpc/mm/hash_utils_64.c | 4 ++++ arch/powerpc/mm/mem.c | 6 ++++++ 3 files changed, 21 insertions(+)arch/powerpc/include/asm/pkeys.h +2
-0@@ -74,6 +74,17 @@ static inline bool mm_pkey_is_allocated(struct mm_struct *mm, int pkey) }/* +* return the protection key of the vma corresponding to the +* given effective address @ea.
arch/powerpc/include/asm/pkeys.h static inline bool mm_pkey_is_allocated(struct mm_struct *mm, int pkey) · line 88
Quoted 10 lines
arch/powerpc/include/asm/pkeys.h +9
-0+*/ +static inline int mm_pkey (struct mm_struct *mm,unsigned long ea) +{ +struct vm_area_struct *vma =find_vma (mm, ea); +int pkey = vma ?vma_pkey (vma) :0 ; +return pkey; +} + +/*
That is not going to work in hash fault path right ? We can't do a find_vma there without holding the mmap_sem
-aneesh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [RFC v4 09/17] powerpc: call the hash functions with the correct pkey value
Quoted 20 lines · Aneesh Kumar K.V
Quoted 17 lines · Ram Pai
Pass the correct protection key value to the hash functions on page fault.
Signed-off-by Ram Pai <[email protected]>--- arch/powerpc/include/asm/pkeys.h | 11 +++++++++++ arch/powerpc/mm/hash_utils_64.c | 4 ++++ arch/powerpc/mm/mem.c | 6 ++++++ 3 files changed, 21 insertions(+)arch/powerpc/include/asm/pkeys.h +0
-0@@ -74,6 +74,17 @@ static inline bool mm_pkey_is_allocated(struct mm_struct *mm, int pkey) }
Quoted 16 lines
arch/powerpc/include/asm/pkeys.h static inline bool mm_pkey_is_allocated(struct mm_struct *mm, int pkey)
Quoted 13 lines
arch/powerpc/include/asm/pkeys.h +11
-0/* +* return the protection key of the vma corresponding to the +* given effective address @ea. +*/ +static inline int mm_pkey (struct mm_struct *mm,unsigned long ea) +{ +struct vm_area_struct *vma =find_vma (mm, ea); +int pkey = vma ?vma_pkey (vma) :0 ; +return pkey; +} + +/* That is not going to work in hash fault path right ? We can't do a
find_vmathere without holding themmap_sem
There is a fundamental problem with this new design. Looks like we can't hold a lock in that path, without badly hurting the performance.
I am moving back to the old design. Cant by-pass the pte. The keys will be programmed into the pte which will than be used to program the hpte.
RP
RFC v4 10/17 powerpc: Macro the mask used for checking DSI exception
In reply to Ram PaiReplace the magic number used to check for DSI exception with a meaningful value.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/reg.h | 7 ++++++- arch/powerpc/kernel/exceptions-64s.S | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/reg.h +6 -1
@@ -272,16 +272,21 @@#define SPRN_DAR 0x013/* Data Address Register */ #define SPRN_DBCR 0x136/* e300 Data Breakpoint Control Reg */ #define SPRN_DSISR 0x012/* Data Storage Interrupt Status Register */ +#define DSISR_BIT32 0x80000000/* not defined */ #define DSISR_NOHPTE 0x40000000/* no translation found */ +#define DSISR_PAGEATTR_CONFLT 0x20000000/* page attribute conflict */ +#define DSISR_BIT35 0x10000000/* not defined */ #define DSISR_PROTFAULT 0x08000000/* protection fault */ #define DSISR_BADACCESS 0x04000000/* bad access to CI or G */ #define DSISR_ISSTORE 0x02000000/* access was a store */ #define DSISR_DABRMATCH 0x00400000/* hit data breakpoint */ -#define DSISR_NOSEGMENT 0x00200000/* SLB miss */ #define DSISR_KEYFAULT 0x00200000/* Key fault */ +#define DSISR_BIT43 0x00100000/* not defined */ #define DSISR_UNSUPP_MMU 0x00080000/* Unsupported MMU config */ #define DSISR_SET_RC 0x00040000/* Failed setting of R/C bits */ #define DSISR_PGDIRFAULT 0x00020000/* Fault on page directory */ +#define DSISR_PAGE_FAULT_MASK (DSISR_BIT32 | DSISR_PAGEATTR_CONFLT | \ + DSISR_BADACCESS | DSISR_BIT43)#define SPRN_TBRL 0x10C/* Time Base Read Lower Register (user, R/O) */ #define SPRN_TBRU 0x10D/* Time Base Read Upper Register (user, R/O) */ #define SPRN_CIR 0x11B/* Chip Information Register (hyper, R/0) */
arch/powerpc/kernel/exceptions-64s.S +1 -1
@@ -1411,7 +1411,7 @@ USE_TEXT_SECTION() .balign IFETCH_ALIGN_BYTES do_hash_page: #ifdef CONFIG_PPC_STD_MMU_64 - andis. r0,r4,0xa410 /* weird error? */ + andis. r0,r4,DSISR_PAGE_FAULT_MASK@h bne- handle_page_fault /* if not, try to insert a HPTE */ andis. r0,r4,DSISR_DABRMATCH@h bne- handle_dabr_fault
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 11/17 powerpc: Handle exceptions caused by pkey violation
In reply to Ram PaiHandle Data and Instruction exceptions caused by memory protection-key.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/mmu_context.h | 12 ++++++ arch/powerpc/include/asm/reg.h | 2 +- arch/powerpc/mm/fault.c | 20 +++++++++ arch/powerpc/mm/pkeys.c | 79 ++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/mmu_context.h +12 -0
@@ -175,11 +175,23 @@ static inline void arch_bprm_mm_init(struct mm_struct *mm, { } +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +bool arch_pte_access_permitted (pte_t pte,bool write); +bool arch_vma_access_permitted (struct vm_area_struct *vma, +bool write,bool execute,bool foreign); +#else /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +static inline bool arch_pte_access_permitted (pte_t pte,bool write) +{ +/* by default, allow everything */ +return true; +}static inline bool arch_vma_access_permitted (struct vm_area_struct *vma,bool write,bool execute,bool foreign) {/* by default, allow everything */ return true; } +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +#endif /* __KERNEL__ */ #endif /* __ASM_POWERPC_MMU_CONTEXT_H */
arch/powerpc/include/asm/reg.h +1 -1
@@ -286,7 +286,7 @@#define DSISR_SET_RC 0x00040000/* Failed setting of R/C bits */ #define DSISR_PGDIRFAULT 0x00020000/* Fault on page directory */ #define DSISR_PAGE_FAULT_MASK (DSISR_BIT32 | DSISR_PAGEATTR_CONFLT | \ - DSISR_BADACCESS | DSISR_BIT43) + DSISR_BADACCESS | DSISR_KEYFAULT | DSISR_BIT43)#define SPRN_TBRL 0x10C/* Time Base Read Lower Register (user, R/O) */ #define SPRN_TBRU 0x10D/* Time Base Read Upper Register (user, R/O) */ #define SPRN_CIR 0x11B/* Chip Information Register (hyper, R/0) */
arch/powerpc/mm/fault.c +20 -0
@@ -261,6 +261,13 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, }#endif +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +if (error_code &DSISR_KEYFAULT ) { + code =SEGV_PKUERR ; + goto bad_area_nosemaphore; + } +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +/* We restore the interrupt state now */ if (!arch_irq_disabled_regs (regs))local_irq_enable ();
@@ -441,6 +448,19 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,WARN_ON_ONCE (error_code &DSISR_PROTFAULT );#endif /* CONFIG_PPC_STD_MMU */ +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , + is_exec,0 )) { + code =SEGV_PKUERR ; + goto bad_area; + } +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ + +/* handle_mm_fault() needs to know if its a instruction access +* fault. +*/ +if (is_exec) + flags |=FAULT_FLAG_INSTRUCTION ;/* * If for any reason at all we couldn't handle the fault, * make sure we exit gracefully rather than endlessly redo
arch/powerpc/mm/pkeys.c +79 -0
88 lines not on this page · raw message
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 12/17 powerpc: Deliver SEGV signal on pkey violation
In reply to Ram PaiThe value of the AMR register at the time of exception is made available in gp_regs[PT_AMR] of the siginfo.
The value of the pkey, whose protection got violated, is made available in si_pkey field of the siginfo structure.
Signed-off-by Ram Pai <[email protected]>
--- arch/powerpc/include/asm/paca.h | 1 + arch/powerpc/include/uapi/asm/ptrace.h | 3 ++- arch/powerpc/kernel/asm-offsets.c | 5 ++++ arch/powerpc/kernel/exceptions-64s.S | 16 +++++++++-- arch/powerpc/kernel/signal_32.c | 5 ++++ arch/powerpc/kernel/signal_64.c | 4 +++ arch/powerpc/kernel/traps.c | 49 ++++++++++++++++++++++++++++++++++ arch/powerpc/mm/fault.c | 2 ++ 8 files changed, 82 insertions(+), 3 deletions(-)
arch/powerpc/include/asm/paca.h +1 -0
@@ -92,6 +92,7 @@ struct paca_struct {struct dtl_entry *dispatch_log_end;#endif /* CONFIG_PPC_STD_MMU_64 */ u64 dscr_default;/* per-CPU default DSCR */ +u64 paca_amr;/* value of amr at exception */ #ifdef CONFIG_PPC_STD_MMU_64 /*
arch/powerpc/include/uapi/asm/ptrace.h +2 -1
@@ -108,8 +108,9 @@ struct pt_regs {#define PT_DAR 41#define PT_DSISR 42#define PT_RESULT 43 -#define PT_DSCR 44#define PT_REGS_COUNT 44 +#define PT_DSCR 44 +#define PT_AMR 45#define PT_FPR0 48/* each FP reg occupies 2 slots in this space */
arch/powerpc/kernel/asm-offsets.c +5 -0
@@ -241,6 +241,11 @@ int main(void)OFFSET (PACAHWCPUID , paca_struct, hw_cpu_id);OFFSET (PACAKEXECSTATE , paca_struct, kexec_state);OFFSET (PACA_DSCR_DEFAULT , paca_struct, dscr_default); + +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +OFFSET (PACA_AMR , paca_struct, paca_amr); +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +OFFSET (ACCOUNT_STARTTIME , paca_struct, accounting.starttime);OFFSET (ACCOUNT_STARTTIME_USER , paca_struct, accounting.starttime_user);OFFSET (ACCOUNT_USER_TIME , paca_struct, accounting.utime);
arch/powerpc/kernel/exceptions-64s.S +14 -2
@@ -493,9 +493,15 @@ EXC_COMMON_BEGIN(data_access_common) ld r12,_MSR(r1) ld r3,PACA_EXGEN+EX_DAR(r13) lwz r4,PACA_EXGEN+EX_DSISR(r13) - li r5,0x300 std r3,_DAR(r1) std r4,_DSISR(r1) +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + andis. r0,r4,DSISR_KEYFAULT@h /* save AMR only if its a key fault */ + beq+ 1f + mfspr r5,SPRN_AMR + std r5,PACA_AMR(r13) +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +1: li r5,0x300 BEGIN_MMU_FTR_SECTION b do_hash_page /* Try to handle as hpte fault */ MMU_FTR_SECTION_ELSE
@@ -561,9 +567,15 @@ EXC_COMMON_BEGIN(instruction_access_common) ld r12,_MSR(r1) ld r3,_NIP(r1) andis. r4,r12,0x5820 - li r5,0x400 std r3,_DAR(r1) std r4,_DSISR(r1) +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + andis. r0,r4,DSISR_KEYFAULT@h /* save AMR only if its a key fault */ + beq+ 1f + mfspr r5,SPRN_AMR + std r5,PACA_AMR(r13) +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +1: li r5,0x400 BEGIN_MMU_FTR_SECTION b do_hash_page /* Try to handle as hpte fault */ MMU_FTR_SECTION_ELSE
arch/powerpc/kernel/signal_32.c +5 -0
@@ -500,6 +500,11 @@ static int save_user_regs(struct pt_regs *regs, struct mcontext __user *frame, (unsigned long) &frame->tramp[2]); } +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS +if (__put_user (get_paca ()->paca_amr, &frame->mc_gregs[PT_AMR ])) +return 1 ; +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +return 0 ; }
arch/powerpc/kernel/signal_64.c +4 -0
@@ -174,6 +174,10 @@ static long setup_sigcontext(struct sigcontext __user *sc,if (set !=NULL ) err |=__put_user (set->sig[0 ], &sc->oldmask); +#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS + err |=__put_user (get_paca ()->paca_amr, &sc->gp_regs[PT_AMR ]); +#endif /* CONFIG_PPC64_MEMORY_PROTECTION_KEYS */ +return err; }
arch/powerpc/kernel/traps.c +49 -0
67 lines not on this page · raw message
arch/powerpc/mm/fault.c +2 -0
@@ -451,6 +451,8 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , is_exec,0 )) { +/* our caller may not have saved the amr. Lets save it */ +get_paca ()->paca_amr =read_amr (); code =SEGV_PKUERR ; goto bad_area; }
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 13/17 selftest: Move protecton key selftest to arch neutral directory
In reply to Ram PaiSigned-off-by Ram Pai <[email protected]>
--- tools/testing/selftests/vm/Makefile | 1 + tools/testing/selftests/vm/pkey-helpers.h | 219 ++++ tools/testing/selftests/vm/protection_keys.c | 1395 +++++++++++++++++++++++++ tools/testing/selftests/x86/Makefile | 2 +- tools/testing/selftests/x86/pkey-helpers.h | 219 ---- tools/testing/selftests/x86/protection_keys.c | 1395 ------------------------- 6 files changed, 1616 insertions(+), 1615 deletions(-) create mode 100644 tools/testing/selftests/vm/pkey-helpers.h create mode 100644 tools/testing/selftests/vm/protection_keys.c delete mode 100644 tools/testing/selftests/x86/pkey-helpers.h delete mode 100644 tools/testing/selftests/x86/protection_keys.c
tools/testing/selftests/vm/Makefile +1 -0
@@ -17,6 +17,7 @@ TEST_GEN_FILES += transhuge-stress TEST_GEN_FILES += userfaultfd TEST_GEN_FILES += mlock-random-test TEST_GEN_FILES += virtual_address_range +TEST_GEN_FILES += protection_keys TEST_PROGS := run_vmtests
tools/testing/selftests/vm/pkey-helpers.h new file +219 -0
219 lines not on this page · raw message
tools/testing/selftests/vm/protection_keys.c new file +1395 -0
1,395 lines not on this page · raw message
tools/testing/selftests/x86/Makefile +1 -1
@@ -6,7 +6,7 @@ include ../lib.mk TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall test_mremap_vdso \ check_initial_reg_state sigreturn ldt_gdt iopl mpx-mini-test ioperm \ - protection_keys test_vdso + test_vdso TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault test_syscall_vdso unwind_vdso \ test_FCMOV test_FCOMI test_FISTTP \ vdso_restorer
tools/testing/selftests/x86/pkey-helpers.h deleted +0 -219
219 lines not on this page · raw message
tools/testing/selftests/x86/protection_keys.c deleted +0 -1395
1,152 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Part of the diff is not shown here. View the raw message.
RFC v4 14/17 selftest: PowerPC specific test updates to memory protection keys
In reply to Ram PaiAbstracted out the arch specific code into the header file, and added powerpc specific changes.
a) added 4k-backed hpte, memory allocator, powerpc specific.
b) added three test case where the key is associated after the page is accessed/allocated/mapped.
c) cleaned up the code to make checkpatch.pl happy
Signed-off-by Ram Pai <[email protected]>
--- tools/testing/selftests/vm/pkey-helpers.h | 230 +++++++++-- tools/testing/selftests/vm/protection_keys.c | 567 ++++++++++++++++----------- 2 files changed, 518 insertions(+), 279 deletions(-)
tools/testing/selftests/vm/pkey-helpers.h +188 -42
@@ -12,13 +12,72 @@#include <ucontext.h> #include <sys/mman.h> -#define NR_PKEYS 16 -#define PKRU_BITS_PER_PKEY 2 +/* Define some kernel-like types */ +#define u8 uint8_t +#define u16 uint16_t +#define u32 uint32_t +#define u64 uint64_t + +#ifdef __i386__/* arch */ + +#define SYS_mprotect_key 380 +#define SYS_pkey_alloc 381 +#define SYS_pkey_free 382 +#define REG_IP_IDX REG_EIP +#define si_pkey_offset 0x14 + +#define NR_PKEYS 16 +#define NR_RESERVED_PKEYS 1 +#define PKRU_BITS_PER_PKEY 2 +#define PKEY_DISABLE_ACCESS 0x1 +#define PKEY_DISABLE_WRITE 0x2 +#define HPAGE_SIZE (1UL<<21) + +#define INIT_PRKU 0x0UL + +#elif __powerpc64__/* arch */ + +#define SYS_mprotect_key 386 +#define SYS_pkey_alloc 384 +#define SYS_pkey_free 385 +#define si_pkey_offset 0x20 +#define REG_IP_IDX PT_NIP +#define REG_TRAPNO PT_TRAP +#define REG_AMR 45 +#define gregs gp_regs +#define fpregs fp_regs + +#define NR_PKEYS 32 +#define NR_RESERVED_PKEYS 3 +#define PKRU_BITS_PER_PKEY 2 +#define PKEY_DISABLE_ACCESS 0x3/* disable read and write */ +#define PKEY_DISABLE_WRITE 0x2 +#define HPAGE_SIZE (1UL<<24) + +#define INIT_PRKU 0x3UL +#else /* arch */ + + NOT SUPPORTED + +#endif /* arch */ +#ifndef DEBUG_LEVEL #define DEBUG_LEVEL 0#endif #define DPRINT_IN_SIGNAL_BUF_SIZE 4096 + + +static inline u32 pkey_to_shift (int pkey) +{ +#ifdef __i386__/* arch */ +return pkey *PKRU_BITS_PER_PKEY ; +#elif __powerpc64__/* arch */ +return (NR_PKEYS - pkey -1 ) *PKRU_BITS_PER_PKEY ; +#endif /* arch */ +} + +extern int dprint_in_signal;extern char dprint_in_signal_buffer[DPRINT_IN_SIGNAL_BUF_SIZE ];static inline void sigsafe_printf (const char *format, ...)
@@ -53,53 +112,76 @@ static inline void sigsafe_printf(const char *format, ...) #define dprintf3(args...) dprintf_level(3, args) #define dprintf4(args...) dprintf_level(4, args) -extern unsigned int shadow_pkru; -static inline unsigned int __rdpkru (void ) +extern u64 shadow_pkey_reg; + +static inline u64 __rdpkey_reg (void ) { +#ifdef __i386__/* arch */ unsigned int eax, edx;unsigned int ecx =0 ; -unsigned int pkru; +unsigned int pkey_reg; asmvolatile (".byte 0x0f,0x01,0xee\n\t" :"=a" (eax),"=d" (edx) :"c" (ecx)); - pkru = eax; -return pkru; +#elif __powerpc64__/* arch */ +u64 eax; +u64 pkey_reg; + + asmvolatile ("mfspr %0, 0xd" :"=r" ((u64)(eax))); +#endif /* arch */ + pkey_reg = (u64 )eax; +return pkey_reg; } -static inline unsigned int _rdpkru (int line) +static inline u64 _rdpkey_reg (int line) { -unsigned int pkru =__rdpkru (); +u64 pkey_reg =__rdpkey_reg (); -dprintf4 ("rdpkru(line=%d) pkru: %x shadow: %x\n" , - line, pkru, shadow_pkru); -assert (pkru == shadow_pkru); +dprintf4 ("rdpkey_reg(line=%d) pkey_reg: %lx shadow: %lx\n" , + line, pkey_reg, shadow_pkey_reg); +assert (pkey_reg == shadow_pkey_reg); -return pkru; +return pkey_reg; } -#define rdpkru () _rdpkru(__LINE__) +#define rdpkey_reg () _rdpkey_reg(__LINE__) -static inline void __wrpkru (unsigned int pkru) +static inline void __wrpkey_reg (u64 pkey_reg) { -unsigned int eax = pkru; +#ifdef __i386__/* arch */ +unsigned int eax = pkey_reg;unsigned int ecx =0 ;unsigned int edx =0 ; -dprintf4 ("%s() changing %08x to %08x\n" , __func__,__rdpkru (), pkru); +dprintf4 ("%s() changing %lx to %lx\n" , + __func__,__rdpkey_reg (), pkey_reg); asmvolatile (".byte 0x0f,0x01,0xef\n\t" : :"a" (eax),"c" (ecx),"d" (edx)); -assert (pkru ==__rdpkru ()); +dprintf4 ("%s() PKRUP after changing %lx to %lx\n" , + __func__,__rdpkey_reg (), pkey_reg); +#else /* arch */ +u64 eax = pkey_reg; + +dprintf4 ("%s() changing %llx to %llx\n" , + __func__,__rdpkey_reg (), pkey_reg); + asmvolatile ("mtspr 0xd, %0" : :"r" ((unsigned long )(eax)) :"memory" ); +dprintf4 ("%s() PKRUP after changing %llx to %llx\n" , + __func__,__rdpkey_reg (), pkey_reg); +#endif /* arch */ +assert (pkey_reg ==__rdpkey_reg ()); } -static inline void wrpkru (unsigned int pkru) +static inline void wrpkey_reg (u64 pkey_reg) { -dprintf4 ("%s() changing %08x to %08x\n" , __func__,__rdpkru (), pkru); +dprintf4 ("%s() changing %lx to %lx\n" , + __func__,__rdpkey_reg (), pkey_reg);/* will do the shadow check for us: */ -rdpkru (); -__wrpkru (pkru); - shadow_pkru = pkru; -dprintf4 ("%s(%08x) pkru: %08x\n" , __func__, pkru,__rdpkru ()); +rdpkey_reg (); +__wrpkey_reg (pkey_reg); + shadow_pkey_reg = pkey_reg; +dprintf4 ("%s(%lx) pkey_reg: %lx\n" , + __func__, pkey_reg,__rdpkey_reg ()); }/*
@@ -108,40 +190,37 @@ static inline void wrpkru(unsigned int pkru)*/ static inline void __pkey_access_allow (int pkey,int do_allow) { -unsigned int pkru =rdpkru (); +u64 pkey_reg =rdpkey_reg ();int bit = pkey *2 ;if (do_allow) - pkru &= (1 <<bit); + pkey_reg &= (1 <<bit);else - pkru |= (1 <<bit); + pkey_reg |= (1 <<bit); -dprintf4 ("pkru now: %08x\n" ,rdpkru ()); -wrpkru (pkru); +dprintf4 ("pkey_reg now: %lx\n" ,rdpkey_reg ()); +wrpkey_reg (pkey_reg); }static inline void __pkey_write_allow (int pkey,int do_allow_write) { -long pkru =rdpkru (); +u64 pkey_reg =rdpkey_reg ();int bit = pkey *2 +1 ;if (do_allow_write) - pkru &= (1 <<bit); + pkey_reg &= (1 <<bit);else - pkru |= (1 <<bit); + pkey_reg |= (1 <<bit); -wrpkru (pkru); -dprintf4 ("pkru now: %08x\n" ,rdpkru ()); +wrpkey_reg (pkey_reg); +dprintf4 ("pkey_reg now: %lx\n" ,rdpkey_reg ()); } -#define PROT_PKEY0 0x10/* protection key value (bit 0) */ -#define PROT_PKEY1 0x20/* protection key value (bit 1) */ -#define PROT_PKEY2 0x40/* protection key value (bit 2) */ -#define PROT_PKEY3 0x80/* protection key value (bit 3) */ - -#define PAGE_SIZE 4096#define MB (1<<20) +#ifdef __i386__/* arch */ + +#define PAGE_SIZE 4096static inline void __cpuid (unsigned int *eax,unsigned int *ebx,unsigned int *ecx,unsigned int *edx) {
@@ -159,7 +238,7 @@ static inline void __cpuid(unsigned int *eax, unsigned int *ebx,#define X86_FEATURE_PKU (1<<3)/* Protection Keys for Userspace */ #define X86_FEATURE_OSPKE (1<<4)/* OS Protection Keys Enable */ -static inline int cpu_has_pku (void ) +static inline int cpu_has_pkey (void ) {unsigned int eax;unsigned int ebx;
@@ -183,7 +262,6 @@ static inline int cpu_has_pku(void)#define XSTATE_PKRU_BIT (9)#define XSTATE_PKRU 0x200 -int pkru_xstate_offset (void ) {unsigned int eax;
@@ -216,4 +294,72 @@ int pkru_xstate_offset(void)return xstate_offset; } +/* 8-bytes of instruction * 512 bytes = 1 page */ +#define __page_o_noops () asm(".rept 512 ; nopl 0x7eeeeeee(%eax) ; .endr") + +#elif __powerpc64__/* arch */ + +#define PAGE_SIZE (0x1UL << 16) +static inline int cpu_has_pkey (void ) +{ +return 1 ; +} + +/* 8-bytes of instruction * 16384bytes = 1 page */ +#define __page_o_noops () asm(".rept 16384 ; nop; .endr") + +#endif /* arch */ + +#define ARRAY_SIZE (x) (sizeof(x) / sizeof(*(x))) +#define ALIGN_UP (x, align_to) (((x) + ((align_to)-1)) & ~((align_to)-1)) +#define ALIGN_DOWN (x, align_to) ((x) & ~((align_to)-1)) +#define ALIGN_PTR_UP (p, ptr_align_to) \ + ((typeof(p))ALIGN_UP((unsigned long)(p), ptr_align_to)) +#define ALIGN_PTR_DOWN (p, ptr_align_to) \ + ((typeof(p))ALIGN_DOWN((unsigned long)(p), ptr_align_to)) +#define __stringify_1(x...) #x +#define __stringify(x...) __stringify_1(x) + +#define PTR_ERR_ENOTSUP ((void *)-ENOTSUP) + +extern void abort_hooks (void ); +#define pkey_assert (condition) do { \ + if (!(condition)) { \ + dprintf0("assert() at %s::%d test_nr: %d iteration: %d\n", \ + __FILE__, __LINE__, \ + test_nr, iteration_nr); \ + dprintf0("errno at assert: %d", errno); \ + abort_hooks(); \ + assert(condition); \ + } \ +} while (0) +#define raw_assert (cond) assert(cond) + + +static inline int open_hugepage_file (int flag) +{ +int fd; +#ifdef __i386__/* arch */ + fd =open ("/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages" , +O_RDONLY ); +#elif __powerpc64__/* arch */ + fd =open ("/sys/kernel/mm/hugepages/hugepages-16384kB/nr_hugepages" , +O_RDONLY ); +#else /* arch */ + NOT SUPPORTED +#endif /* arch */ +return fd; +} + +static inline int get_start_key (void ) +{ +#ifdef __i386__/* arch */ +return 1 ; +#elif __powerpc64__/* arch */ +return 0 ; +#else /* arch */ + NOT SUPPORTED +#endif /* arch */ +} +#endif /* _PKEYS_HELPER_H */
tools/testing/selftests/vm/protection_keys.c +330 -237
1,023 lines not on this page · raw message
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 15/17 Documentation: Move protecton key documentation to arch neutral directory
In reply to Ram PaiSince PowerPC and Intel both support memory protection keys, moving the documenation to arch-neutral directory.
Signed-off-by Ram Pai <[email protected]>
--- Documentation/vm/protection-keys.txt | 85 +++++++++++++++++++++++++++++++++++ Documentation/x86/protection-keys.txt | 85 ----------------------------------- 2 files changed, 85 insertions(+), 85 deletions(-) create mode 100644 Documentation/vm/protection-keys.txt delete mode 100644 Documentation/x86/protection-keys.txt
Documentation/vm/protection-keys.txt new file +85 -0
@@ -0,0 +1,85 @@ +Memory Protection Keys for Userspace (PKU aka PKEYs) is a CPU feature +which will be found on future Intel CPUs. + +Memory Protection Keys provides a mechanism for enforcing page-based +protections, but without requiring modification of the page tables +when an application changes protection domains. It works by +dedicating 4 previously ignored bits in each page table entry to a +"protection key", giving 16 possible keys. + +There is also a new user-accessible register (PKRU) with two separate +bits (Access Disable and Write Disable) for each key. Being a CPU +register, PKRU is inherently thread-local, potentially giving each +thread a different set of protections from every other thread. + +There are two new instructions (RDPKRU/WRPKRU) for reading and writing +to the new register. The feature is only available in 64-bit mode, +even though there is theoretically space in the PAE PTEs. These +permissions are enforced on data access only and have no effect on +instruction fetches. + +=========================== Syscalls =========================== + +There are 3 system calls which directly interact with pkeys: + + int pkey_alloc(unsigned long flags, unsigned long init_access_rights) + int pkey_free(int pkey); + int pkey_mprotect(unsigned long start, size_t len, + unsigned long prot, int pkey); + +Before a pkey can be used, it must first be allocated with +pkey_alloc(). An application calls the WRPKRU instruction +directly in order to change access permissions to memory covered +with a key. In this example WRPKRU is wrapped by a C function +called pkey_set(). + + int real_prot = PROT_READ|PROT_WRITE; + pkey = pkey_alloc(0, PKEY_DENY_WRITE); + ptr = mmap(NULL, PAGE_SIZE, PROT_NONE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); + ret = pkey_mprotect(ptr, PAGE_SIZE, real_prot, pkey); + ... application runs here + +Now, if the application needs to update the data at 'ptr', it can +gain access, do the update, then remove its write access: + + pkey_set(pkey, 0); // clear PKEY_DENY_WRITE + *ptr = foo; // assign something + pkey_set(pkey, PKEY_DENY_WRITE); // set PKEY_DENY_WRITE again + +Now when it frees the memory, it will also free the pkey since it +is no longer in use: + + munmap(ptr, PAGE_SIZE); + pkey_free(pkey); + +(Note: pkey_set() is a wrapper for the RDPKRU and WRPKRU instructions. + An example implementation can be found in + tools/testing/selftests/x86/protection_keys.c) + +=========================== Behavior =========================== + +The kernel attempts to make protection keys consistent with the +behavior of a plain mprotect(). For instance if you do this: + + mprotect(ptr, size, PROT_NONE); + something(ptr); + +you can expect the same effects with protection keys when doing this: + + pkey = pkey_alloc(0, PKEY_DISABLE_WRITE | PKEY_DISABLE_READ); + pkey_mprotect(ptr, size, PROT_READ|PROT_WRITE, pkey); + something(ptr); + +That should be true whether something() is a direct access to 'ptr' +like: + + *ptr = foo; + +or when the kernel does the access on the application's behalf like +with a read(): + + read(fd, ptr, 1); + +The kernel will send a SIGSEGV in both cases, but si_code will be set +to SEGV_PKERR when violating protection keys versus SEGV_ACCERR when +the plain mprotect() permissions are violated.
Documentation/x86/protection-keys.txt deleted +0 -85
85 lines not on this page · raw message
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
RFC v4 16/17 Documentation: PowerPC specific updates to memory protection keys
In reply to Ram PaiAdd documentation updates that capture PowerPC specific changes.
Signed-off-by Ram Pai <[email protected]>
--- Documentation/vm/protection-keys.txt | 89 ++++++++++++++++++++++++++++-------- 1 file changed, 69 insertions(+), 20 deletions(-)
Documentation/vm/protection-keys.txt +69 -20
@@ -1,21 +1,46 @@ -Memory Protection Keys for Userspace (PKU aka PKEYs) is a CPU feature -which will be found on future Intel CPUs. +Memory Protection Keys for Userspace (PKU aka PKEYs) is a CPU feature found in +new generation of intel CPUs and on PowerPC 7 and higher CPUs. Memory Protection Keys provides a mechanism for enforcing page-based -protections, but without requiring modification of the page tables -when an application changes protection domains. It works by -dedicating 4 previously ignored bits in each page table entry to a -"protection key", giving 16 possible keys. - -There is also a new user-accessible register (PKRU) with two separate -bits (Access Disable and Write Disable) for each key. Being a CPU -register, PKRU is inherently thread-local, potentially giving each -thread a different set of protections from every other thread. - -There are two new instructions (RDPKRU/WRPKRU) for reading and writing -to the new register. The feature is only available in 64-bit mode, -even though there is theoretically space in the PAE PTEs. These -permissions are enforced on data access only and have no effect on +protections, but without requiring modification of the page tables when an +application changes protection domains. + + +On Intel: + + It works by dedicating 4 previously ignored bits in each page table + entry to a "protection key", giving 16 possible keys. + + There is also a new user-accessible register (PKRU) with two separate + bits (Access Disable and Write Disable) for each key. Being a CPU + register, PKRU is inherently thread-local, potentially giving each + thread a different set of protections from every other thread. + + There are two new instructions (RDPKRU/WRPKRU) for reading and writing + to the new register. The feature is only available in 64-bit mode, + even though there is theoretically space in the PAE PTEs. These + permissions are enforced on data access only and have no effect on + instruction fetches. + + +On PowerPC: + + It works by dedicating 5 hash-page table entry bits to a "protection key", + giving 32 possible keys. + + There is a user-accessible register (AMR) with two separate bits; + Access Disable and Write Disable, for each key. Being a CPU + register, AMR is inherently thread-local, potentially giving each + thread a different set of protections from every other thread. NOTE: + Disabling read permission does not disable write and vice-versa. + + The feature is available on 64-bit HPTE mode only. + 'mtspr 0xd, mem' reads the AMR register + 'mfspr mem, 0xd' writes into the AMR register. + + + +Permissions are enforced on data access only and have no effect on instruction fetches. =========================== Syscalls ===========================
@@ -28,9 +53,9 @@ There are 3 system calls which directly interact with pkeys: unsigned long prot, int pkey); Before a pkey can be used, it must first be allocated with -pkey_alloc(). An application calls the WRPKRU instruction +pkey_alloc(). An application calls the WRPKRU/AMR instruction directly in order to change access permissions to memory covered -with a key. In this example WRPKRU is wrapped by a C function +with a key. In this example WRPKRU/AMR is wrapped by a C function called pkey_set(). int real_prot = PROT_READ|PROT_WRITE;
@@ -52,11 +77,11 @@ is no longer in use: munmap(ptr, PAGE_SIZE); pkey_free(pkey); -(Note: pkey_set() is a wrapper for the RDPKRU and WRPKRU instructions. +(Note: pkey_set() is a wrapper for the RDPKRU,WRPKRU or AMR instructions. An example implementation can be found in tools/testing/selftests/x86/protection_keys.c) -=========================== Behavior =========================== +=========================== Behavior ================================= The kernel attempts to make protection keys consistent with the behavior of a plain mprotect(). For instance if you do this:
@@ -83,3 +108,27 @@ with a read(): The kernel will send a SIGSEGV in both cases, but si_code will be set to SEGV_PKERR when violating protection keys versus SEGV_ACCERR when the plain mprotect() permissions are violated. + + +==================================================================== + Semantic differences + +The following semantic differences exist between x86 and power. + +a) powerpc allows creation of a key with execute-disabled. The following + is allowed on powerpc. + pkey = pkey_alloc(0, PKEY_DISABLE_WRITE | PKEY_DISABLE_READ | + PKEY_DISABLE_EXECUTE); + x86 disallows PKEY_DISABLE_EXECUTE during key creation. + +b) x86, PKEY_DISABLE_READ disables read and write on the key. + Powerpc, PKEY_DISABLE_READ just disables read. + PKEY_DISABLE_WRITE has to be specified explicitly to disable write. + +c) changing the permission bits of a key from a signal handler does not + persist on x86. The PKRU specific fpregs entry needs to be modified + for it to persist. On powerpc the permission bits of the key can be + modified by programming the AMR register from the signal handler. + The changes persists across signal boundaries. + +=====================================================================
Signature · 1 line
1.8.3.1
RFC v4 17/17 procfs: display the protection-key number associated with a vma
In reply to Ram PaiDisplay the pkey number associated with the vma in smaps of a task. The key will be seen as below:
VmFlags: rd wr mr mw me dw ac key=0
Signed-off-by Ram Pai <[email protected]>
--- Documentation/filesystems/proc.txt | 3 ++- fs/proc/task_mmu.c | 22 +++++++++++----------- 2 files changed, 13 insertions(+), 12 deletions(-)
Documentation/filesystems/proc.txt +2 -1
@@ -423,7 +423,7 @@ SwapPss: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB -VmFlags: rd ex mr mw me dw +VmFlags: rd ex mr mw me dw key=<num> the first of these lines shows the same information as is displayed for the mapping in /proc/PID/maps. The remaining lines show the size of the mapping
@@ -491,6 +491,7 @@ manner. The codes are the following: hg - huge page advise flag nh - no-huge page advise flag mg - mergable advise flag + key=<num> - the memory protection key number Note that there is no guarantee that every flag and associated mnemonic will be present in all further kernel releases. Things get changed, the flags may
fs/proc/task_mmu.c +11 -11
@@ -1,4 +1,6 @@#include <linux/mm.h> +#include <linux/pkeys.h> +#include <linux/huge_mm.h> #include <linux/vmacache.h> #include <linux/hugetlb.h> #include <linux/huge_mm.h>
@@ -666,22 +668,20 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) [ilog2(VM_MERGEABLE)] = "mg", [ilog2(VM_UFFD_MISSING)]= "um", [ilog2(VM_UFFD_WP)] = "uw", -#ifdef CONFIG_ARCH_HAS_PKEYS -/* These come out via ProtectionKey: */ - [ilog2(VM_PKEY_BIT0)] = "", - [ilog2(VM_PKEY_BIT1)] = "", - [ilog2(VM_PKEY_BIT2)] = "", - [ilog2 (VM_PKEY_BIT3 )] ="" , -#endif /* CONFIG_ARCH_HAS_PKEYS */ -#ifdef CONFIG_PPC64_MEMORY_PROTECTION_KEYS -/* Additional bit in ProtectionKey: */ - [ilog2(VM_PKEY_BIT4)] = "", -#endif };size_t i;seq_puts (m,"VmFlags: " );for (i =0 ; i <BITS_PER_LONG ; i++) { +#ifdef CONFIG_ARCH_HAS_PKEYS +if (i ==ilog2 (VM_PKEY_BIT0 )) { +int keyvalue =vma_pkey (vma); + + i +=ilog2 (arch_max_pkey ())-1 ; +seq_printf (m,"key=%d " , keyvalue); +continue ; + } +#endif /* CONFIG_ARCH_HAS_PKEYS */ if (!mnemonics[i][0 ])continue ;if (vma->vm_flags & (1UL << i)) {
Signature · 7 lines
1.8.3.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Michael Ellerman
reviewerRe: [RFC v4 17/17] procfs: display the protection-key number associated with a vma
Quoted 4 lines · Ram Pai
Display the pkey number associated with the vma in smaps of a task. The key will be seen as below:
VmFlags: rd wr mr mw me dw ac key=0
Why wouldn't we just emit a "ProtectionKey:" line like x86 does?
See their arch_show_smap().
You should probably also do what x86 does, which is to not display the key on CPUs that don't support keys.
cheers
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 34 lines · Ram Pai
Memory protection keys enable applications to protect its address space from inadvertent access or corruption from itself.
The overall idea:
A process allocates a key and associates it with
a address range within its address space.
The process than can dynamically set read/write
permissions on the key without involving the
kernel. Any code that violates the permissions
off the address space; as defined by its associated key, will receive a segmentation fault.This patch series enables the feature on PPC64 HPTE platform.
ISA3.0 section 5.7.13 describes the detailed specifications.
Testing:
This patch series has passed all the protection key tests available in the selftests directory. The tests are updated to work on both x86 and powerpc.version v4:
(1) patches no more depend on the pte bits to program the hpte -- comment by Balbir
(2) documentation updates
(3) fixed a bug in the selftest.
(4) unlike x86, powerpc lets signal handler change key permission bits; the change will persist across signal handler boundaries. Earlier we allowed the signal handler to modify a field in the siginfo
Quoted 8 lines
structure which would than be used by the kernel to program the key protection register (AMR)
-- resolves a issue raised by Ben. "Calls tosys_swapcontextwith a made-up context will end up with a crap AMR if done by code who didn't know about that register".
(5) these changes enable protection keys on 4k-page kernel aswell.
I have not looked at the full series, but it seems cleaner than the original one and the side-effect is that we can support 4k as well. Nice!
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>