Skip to content

PKS/PMEM: Add Stray Write Protection

[email protected] linux-hardening lkml 12 versions – Applied

Status from this message: The following commit has been merged into the x86/mm branch of tip:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v10 Latest44 patches 14 repliesApplied
  2. v945 patches 3 repliesSuperseded
  3. v844 patches 100 repliesSuperseded
Earlier versions · 9
VersionPostedPatchesRepliesStatus
v718 patches23 repliesSuperseded
v610 patches2 repliesSuperseded
v510 patches2 repliesSuperseded
v410 patches3 repliesSuperseded
v3 posting 2/210 patches25 repliesSuperseded
v2 posting 2/210 patches10 repliesApplied
v3 RFC posting 1/29 patches37 repliesSuperseded
v2 RFC posting 1/217 patches55 repliesSuperseded
v1 RFC15 patches14 repliesSuperseded

Changes in v10 from the cover letter

Rebased to 5.18-rc3
Re-aranged the patch series into 3 sections
1-24 PKS core
25-36 PMEM use case
37-44 PKS core testing
Drop the irqentry_exit_cond_resched() fixup patch as that was fixed by
Mark Rutland in:
4624a14f4daa ("sched/preempt: Simplify irqentry_exit_cond_resched() callers")
Adjust irqentry_exit_cond_resched() changes based on Marks fix
Fix test_pks cpu option processing
Move memremap code to memremap.h

Patches in v10 Applied

#Subject and filesSizeRepliesApplied
1/44Documentation/protection-keys: Clean up documentation for User Space pkeysDocumentation/core-api/protection-keys.rst+21 -230 replies
2/44x86/pkeys: Clarify PKRU_AD_KEY macroarch/x86/mm/pkeys.c+9 -60 replies
3/44x86/pkeys: Make PKRU macros genericarch/x86/include/asm/pkeys_common.h arch/x86/include/asm/pkru.h arch/x86/kernel/fpu/xstate.c +1 more+33 -280 replies
4/44x86/fpu: Refactor arch_set_user_pkey_access()arch/x86/include/asm/pkeys.h arch/x86/kernel/fpu/xstate.c arch/x86/mm/pkeys.c+22 -180 replies
5/44mm/pkeys: Add Kconfig options for PKSarch/x86/Kconfig mm/Kconfig+5 -00 replies
6/44x86/pkeys: Add PKS CPU feature bitarch/x86/include/asm/cpufeatures.h arch/x86/include/asm/disabled-features.h+8 -10 replies
7/44x86/fault: Adjust WARN_ON for pkey faultarch/x86/mm/fault.c+4 -40 replies
8/44Documentation/pkeys: Add initial PKS documentationDocumentation/core-api/protection-keys.rst+39 -43 replies
9/44mm/pkeys: Provide for PKS key allocationDocumentation/core-api/protection-keys.rst arch/x86/include/asm/pkeys_common.h include/linux/pks-keys.h+91 -10 replies
10/44x86/pkeys: Enable PKS on cpus which support itarch/x86/include/asm/msr-index.h arch/x86/include/asm/pks.h arch/x86/include/uapi/asm/processor-flags.h +2 more+37 -00 replies
11/44mm/pkeys: Define PKS page table macrosDocumentation/core-api/protection-keys.rst arch/x86/include/asm/pgtable_types.h include/linux/pgtable.h+32 -00 replies
12/44x86/pkeys: Introduce pks_write_pkrs()arch/x86/mm/pkeys.c+41 -00 replies
13/44x86/pkeys: Preserve the PKS MSR on context switcharch/x86/include/asm/pks.h arch/x86/include/asm/processor.h arch/x86/kernel/process_64.c +1 more+27 -10 replies
14/44mm/pkeys: Introduce pks_set_readwrite()Documentation/core-api/protection-keys.rst arch/x86/mm/pkeys.c include/linux/pks.h +1 more+78 -06 replies
15/44mm/pkeys: Introduce pks_set_noaccess()Documentation/core-api/protection-keys.rst include/linux/pks.h+14 -10 replies
16/44mm/pkeys: Introduce PKS fault callbacksDocumentation/core-api/protection-keys.rst arch/x86/include/asm/pks.h arch/x86/mm/fault.c +2 more+116 -60 replies
17/44x86/entry: Add auxiliary pt_regs spacearch/x86/Kconfig arch/x86/entry/calling.h arch/x86/entry/entry_64.S +4 more+91 -00 replies
18/44entry: Pass pt_regs to irqentry_exit_cond_resched()arch/arm64/include/asm/preempt.h arch/arm64/kernel/entry-common.c arch/x86/entry/common.c +3 more+43 -350 replies
19/44entry: Add calls for save/restore auxiliary pt_regsinclude/linux/entry-common.h kernel/entry/common.c+21 -20 replies
20/44x86/entry: Define arch_{save|restore}_auxiliary_pt_regs()arch/x86/include/asm/entry-common.h+12 -00 replies
21/44x86/pkeys: Preserve PKRS MSR across exceptionsarch/x86/Kconfig arch/x86/include/asm/entry-common.h arch/x86/include/asm/pks.h +2 more+44 -10 replies
22/44x86/fault: Print PKS MSR on faultarch/x86/include/asm/pks.h arch/x86/kernel/dumpstack.c arch/x86/kernel/process_64.c +1 more+45 -20 replies
23/44mm/pkeys: Introduce pks_update_exception()Documentation/core-api/protection-keys.rst arch/x86/mm/pkeys.c include/linux/pks.h+58 -80 replies
24/44mm/pkeys: Add pks_available()Documentation/core-api/protection-keys.rst arch/x86/include/asm/pks.h include/linux/pks.h+23 -00 replies
25/44memremap_pages: Add Kconfig for DEVMAP_ACCESS_PROTECTIONmm/Kconfig+18 -00 replies
26/44memremap_pages: Introduce pgmap_protection_available()include/linux/memremap.h+17 -00 replies
27/44memremap_pages: Introduce a PGMAP_PROTECTION flaginclude/linux/memremap.h mm/memremap.c+41 -00 replies
28/44memremap_pages: Introduce devmap_protected()include/linux/memremap.h+17 -00 replies
29/44memremap_pages: Reserve a PKS pkey for eventual use by PMEMinclude/linux/pks-keys.h+7 -20 replies
30/44memremap_pages: Set PKS pkey in PTEs if requestedmm/memremap.c+13 -00 replies
31/44memremap_pages: Define pgmap_set_{readwrite|noaccess}() callsinclude/linux/memremap.h include/linux/sched.h init/init_task.c +1 more+59 -00 replies
32/44memremap_pages: Add memremap.pks_fault_modeDocumentation/admin-guide/kernel-parameters.txt arch/x86/mm/pkeys.c include/linux/memremap.h +1 more+86 -10 replies
33/44kmap: Make kmap work for devmap protected pagesinclude/linux/highmem-internal.h mm/Kconfig+7 -05 replies
34/44dax: Stray access protection for dax_direct_access()drivers/dax/super.c drivers/md/dm-writecache.c fs/dax.c +2 more+82 -10 replies
35/44nvdimm/pmem: Enable stray access protectiondrivers/nvdimm/pmem.c+26 -00 replies
36/44devdax: Enable stray access protectiondrivers/dax/device.c+2 -00 replies
37/44mm/pkeys: PKS testing, add initial test codeDocumentation/core-api/protection-keys.rst include/linux/pks-keys.h lib/Kconfig.debug +3 more+332 -20 replies
38/44x86/selftests: Add test_pksDocumentation/core-api/protection-keys.rst tools/testing/selftests/x86/Makefile tools/testing/selftests/x86/test_pks.c+357 -10 replies
39/44mm/pkeys: PKS testing, add a fault call backarch/x86/mm/pkeys.c include/linux/pks.h lib/pks/pks_test.c+16 -00 replies
40/44mm/pkeys: PKS testing, add pks_set_*() testslib/pks/pks_test.c tools/testing/selftests/x86/test_pks.c+169 -40 replies
41/44mm/pkeys: PKS testing, test context switchinglib/pks/pks_test.c tools/testing/selftests/x86/test_pks.c+208 -50 replies
42/44mm/pkeys: PKS testing, Add exception testarch/x86/mm/pkeys.c include/linux/pks.h lib/pks/pks_test.c +1 more+144 -20 replies
43/44mm/pkeys: PKS testing, test pks_update_exception()lib/pks/pks_test.c tools/testing/selftests/x86/test_pks.c+64 -10 replies
44/44mm/pkeys: PKS testing, add test for all keysDocumentation/core-api/protection-keys.rst arch/x86/mm/pkeys.c include/linux/pks-keys.h +4 more+95 -80 replies

Reviews trailers per patch

3 of 44 patches have a Reviewed-by or Acked-by.

PatchDave HansenDan Williams
1/44
2/44
3/44A-b
4/44A-b
5/44
6/44
7/44
8/44
9/44
10/44
11/44
12/44
13/44
14/44
15/44
16/44
17/44
18/44
19/44
20/44
21/44
22/44
23/44
24/44
25/44
26/44
27/44
28/44
29/44
30/44
31/44
32/44
33/44
34/44
35/44
36/44R-b
37/44
38/44
39/44
40/44
41/44
42/44
43/44
44/44

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v10 thread

Cover letter [PATCH V10 00/44] PKS/PMEM: Add Stray Write Protection

From: Ira Weiny <[email protected]>

I'm looking for Intel acks on the series prior to submitting to maintainers.

Because I did not get a lot of feedback on the previous version I've reworked the order of the patches to lighten the review load.

I'd like to get comments from Peter and Dave on patches 1-24.

Patches 25-36 implement the PMEM use case. I'd like to get Dan to look at those.

Patches 37-44 implement the PKS tests which can be deferred if review time is short.

Code wise there were no significant changes between v9 and v10. But a V10 was required due to upstream changes/conflicts. One of which require dropping a patch because a different fix landed.

This series is now based on 5.18-rc3.

Changes for V10
Rebased to 5.18-rc3
Re-aranged the patch series into 3 sections
1-24 PKS core
25-36 PMEM use case
37-44 PKS core testing
Drop the irqentry_exit_cond_resched() fixup patch as that was fixed by
Mark Rutland in:
4624a14f4daa ("sched/preempt: Simplify irqentry_exit_cond_resched() callers")
Adjust irqentry_exit_cond_resched() changes based on Marks fix
Fix test_pks cpu option processing
Move memremap code to memremap.h

PKS/PMEM Stray write protection
===============================

This series is broken into 2 parts.

1) Introduce Protection Key Supervisor (PKS), testing, and documentation
2) Use PKS to protect PMEM from stray writes

Introduce Protection Key Supervisor (PKS) [Patches 1-24]
--------------------------------------------------------

PKS enables protections on 'domains' of supervisor pages to limit supervisor mode access to pages beyond the normal paging protections. PKS works in a similar fashion to user space pkeys, PKU. As with PKU, supervisor pkeys are checked in addition to normal paging protections. And page mappings are assigned to a domain by setting a 4 bit pkey in the PTE of that mapping.

Unlike PKU, permissions are changed via a MSR update. This update avoids TLB flushes making this an efficient way to alter protections vs PTE updates.

Also, unlike PTE updates PKS permission changes apply only to the current processor. Therefore changing permissions apply only to that thread and not any other cpu/process. This allows protections to remain in place on other cpus for additional protection and isolation.

Even though PKS updates are thread local, XSAVE is not supported for the PKRS MSR. Therefore this implementation saves and restores the MSR across context switches and during exceptions within software. Nested exceptions are supported by each exception getting a new PKS state.

For consistent behavior with current paging protections, pkey 0 is reserved and configured to allow full access via the pkey mechanism, thus preserving the default paging protections because PTEs naturally have a pkey value of 0.

Other keys, (1-15) are statically allocated by kernel consumers when configured. This is done by adding the appropriate PKS_NEW_KEY and PKS_DECLARE_INIT_VALUE macros to pks-keys.h.

Two PKS consumers, PKS_TEST and PMEM stray write protection, are included in this series. When the number of users grows larger the sharing of keys will need to be resolved depending on the needs of the users at that time. Many methods have been contemplated but the number of kernel users and use cases envisioned is still quite small, much less than the 15 available keys.

To summarize, the following are key attributes of PKS.

1) Fast switching of permissions
1a) Prevents access without page table manipulations 1b) No TLB flushes required
2) Works on a per thread basis, thus allowing protections to be preserved on threads which are not actively accessing data through the mapping.

PKS is available with 4 and 5 level paging. For this and simplicity of implementation, the feature is restricted to x86_64.

Use PKS to protect PMEM from stray writes [Patches 25-36]
---------------------------------------------------------

DAX leverages the direct-map to enable 'struct page' services for PMEM. Given that PMEM capacity may be an order of magnitude higher capacity than System RAM it presents a large vulnerability surface to stray writes. Such a stray write becomes a silent data corruption bug.

Stray pointers to System RAM may result in a crash or other undesirable behavior which, while unfortunate, are usually recoverable with a reboot. Stray writes to PMEM are permanent in nature and thus are more likely to result in permanent user data loss. Given that PMEM access from the kernel is limited to a constrained set of locations (PMEM driver, Filesystem-DAX, direct-I/O, and any properly kmap'ed page), it is amenable to PKS protection.

Set up an infrastructure for extra device access protection. Then implement the protection using the new Protection Keys Supervisor (PKS) on architectures which support it.

Because PMEM pages are all associated with a struct dev_pagemap and flags in struct page are valuable the flag of protecting memory can be stored in struct dev_pagemap. All PMEM is protected by the same pkey. So a single flag is all that is needed in each dev_pagemap to indicate protection.

General access in the kernel is supported by modifying the kmap infrastructure which can detect if a page is pks protected and enable access until the corresponding unmap is called.

Because PKS is a thread local mechanism and because kmap was never really intended to create a long term mapping, this implementation does not support the kmap()/kunmap() calls. Calling kmap() on a PMEM protected page is allowed but accessing that mapping will cause a fault.

Originally this series modified many of the kmap call sites to indicate they were thread local.[1] And an attempt to support kmap()[2] was made. But now that kmap_local_page() has been developed[3] and in more wide spread use, kmap() can safely be left unsupported.

How the fault is handled is configurable via a new module parameter memremap.pks_fault_mode. Two modes are supported.

'relaxed' (default) -- WARN_ONCE, disable the protection and allow
access

'strict' -- prevent any unguarded access to a protected dev_pagemap
range

This 'safety valve' feature has already been useful in the development of this feature.

[1] https://lore.kernel.org/lkml/[email protected]/

[2] https://lore.kernel.org/lkml/[email protected]/

[3] https://lore.kernel.org/lkml/[email protected]/

    https://lore.kernel.org/lkml/[email protected]/
    https://lore.kernel.org/lkml/[email protected]/
    https://lore.kernel.org/lkml/[email protected]/

----------------------------------------------------------------------------
Changes for V9

Review and update all commit messages.
Update cover letter below

PKS Core
Separate user and supervisor pkey code in the headers
create linux/pks.h for supervisor calls
This facilitated making the pmem code more efficient
Completely rearchitect the test code
[After Dave Hansen and Rick Edgecombe found issues in the test
code it was easier to rearchitect the code completely rather than attempt to fix it.]
Remove pks_test_callback in favor of using fault hooks
Fault hooks also isolate the fault callbacks from being false positives if non-test consumers are running
Make additional PKS_TEST_RUN_ALL Kconfig option which is
mutually exclusive to any non-test PKS consumer PKS_TEST_RUN_ALL takes over all pkey callbacks
Ensure that each test runs within it's own context and is
mutually exclusive from running while any other test is running.
Ensure test session and context memory is cleaned up on file
close
Use pr_debug() and dynamic debug for in kernel debug messages Enhance test_pks selftest
Add the ability to run all tests not just the context
switch test
Standardize output [PASS][FAIL][SKIP]
Add '-d' option enables dynamic debug to see the kernel
debug messages

Incorporate feedback from Rick Edgecombe
Update all pkey types to u8
Fix up test code barriers
Move patch declaring PKS_INIT_VALUE ahead of the patch which enables
PKS so that PKS_INIT_VALUE can be used when pks_setup() is first created
From Dan Williams
Use macros instead of an enum for a pkey allocation scheme
which is predicated on the config options of consumers This almost worked perfectly. It required a bit of tweeking to be able to allocate all of the keys.

From Dave Hansen
Reposition some code to be near/similar to user pkeys
s/pks_write_current/x86_pkrs_load s/pks_saved_pkrs/pkrs
Update Documentation s/PKR_{RW,AD,WD}_KEY/PKR_{RW,AD,WD}_MASK Consistently use lower case for pkey Update commit messages
Add Acks

PMEM Stray Write
Building on the change to the pks_mk_*() function rename
s/pgmap_mk_*/pgmap_set_*/
s/dax_mk_*/dax_set_*/
From Dan Williams
Avoid adding new dax operations by teaching dax_device about pgmap Remove pgmap_protection_flag_invalid() patch (Just let
kmap'ings fail)

Changes for V8

Feedback from Thomas
* clean up noinstr mess
* Fix static PKEY allocation mess
* Ensure all functions are consistently named.
* Split up patches to do 1 thing per patch
* pkey_update_pkval() implementation
* Streamline the use of pks_write_pkrs() by not disabling preemption
- Leave this to the callers who require it.
- Use documentation and lockdep to prevent errors
* Clean up commit messages to explain in detail _why_ each patch is there.

Feedback from Dave H.
* Leave out pks_mk_readonly() as it is not used by the PMEM use case

Feedback from Peter Anvin
* Replace pks_abandon_pkey() with pks_update_exception() This is an even greater simplification in that it no longer attempts to shield users from faults. As the main use case for abandoning a key was to allow a system to continue running even with an error. This should be a rare event so the performance should not be an issue.

* Simplify ARCH_ENABLE_SUPERVISOR_PKEYS

* Update PKS Test code
- Add default value test
- Split up the test code into patches which follow each feature addition
- simplify test code processing
- ensure consistent reporting of errors.

* Ensure all entry points to the PKS code are protected by cpu_feature_enabled(X86_FEATURE_PKS)
- At the same time make sure non-entry points or sub-functions to the PKS code are not _unnecessarily_ protected by the feature check

* Update documentation
- Use kernel docs to place the docs with the code for easier internal developer use

* Adjust the PMEM use cases for the core changes

* Split the PMEM patches up to be 1 change per patch and help clarify review

* Review all header files and remove those no longer needed

* Review/update/clarify all commit messages

Fenghua Yu (1):
mm/pkeys: Define PKS page table macros

Ira Weiny (42):
Documentation/protection-keys: Clean up documentation for User Space pkeys
x86/pkeys: Clarify PKRU_AD_KEY macro
x86/pkeys: Make PKRU macros generic
x86/fpu: Refactor arch_set_user_pkey_access()
mm/pkeys: Add Kconfig options for PKS
x86/pkeys: Add PKS CPU feature bit
x86/fault: Adjust WARN_ON for pkey fault Documentation/pkeys: Add initial PKS documentation mm/pkeys: Provide for PKS key allocation
x86/pkeys: Enable PKS on cpus which support it
x86/pkeys: Introduce pks_write_pkrs()
x86/pkeys: Preserve the PKS MSR on context switch mm/pkeys: Introduce pks_set_readwrite()
mm/pkeys: Introduce pks_set_noaccess()
x86/entry: Add auxiliary pt_regs space
entry: Pass pt_regs to irqentry_exit_cond_resched()
entry: Add calls for save/restore auxiliary pt_regs x86/entry: Define arch_{save|restore}_auxiliary_pt_regs() x86/pkeys: Preserve PKRS MSR across exceptions
x86/fault: Print PKS MSR on fault
mm/pkeys: Introduce pks_update_exception()
mm/pkeys: Add pks_available()
memremap_pages: Add Kconfig for DEVMAP_ACCESS_PROTECTION
memremap_pages: Introduce pgmap_protection_available()
memremap_pages: Introduce a PGMAP_PROTECTION flag
memremap_pages: Introduce devmap_protected()
memremap_pages: Reserve a PKS pkey for eventual use by PMEM
memremap_pages: Set PKS pkey in PTEs if requested
memremap_pages: Define pgmap_set_{readwrite|noaccess}() calls
memremap_pages: Add memremap.pks_fault_mode
kmap: Make kmap work for devmap protected pages
dax: Stray access protection for dax_direct_access() nvdimm/pmem: Enable stray access protection
devdax: Enable stray access protection
mm/pkeys: PKS testing, add initial test code x86/selftests: Add test_pks
mm/pkeys: PKS testing, add a fault call back
mm/pkeys: PKS testing, add pks_set_*() tests
mm/pkeys: PKS testing, test context switching
mm/pkeys: PKS testing, Add exception test
mm/pkeys: PKS testing, test pks_update_exception() mm/pkeys: PKS testing, add test for all keys

Rick Edgecombe (1):
mm/pkeys: Introduce PKS fault callbacks

.../admin-guide/kernel-parameters.txt | 12 + Documentation/core-api/protection-keys.rst | 130 ++- arch/arm64/include/asm/preempt.h | 2 +- arch/arm64/kernel/entry-common.c | 4 +- arch/x86/Kconfig | 6 +
arch/x86/entry/calling.h | 20 + arch/x86/entry/common.c | 2 +- arch/x86/entry/entry_64.S | 22 + arch/x86/entry/entry_64_compat.S | 6 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/entry-common.h | 15 + arch/x86/include/asm/msr-index.h | 1 + arch/x86/include/asm/pgtable_types.h | 22 + arch/x86/include/asm/pkeys.h | 2 + arch/x86/include/asm/pkeys_common.h | 18 + arch/x86/include/asm/pkru.h | 20 +- arch/x86/include/asm/pks.h | 46 ++ arch/x86/include/asm/processor.h | 15 +- arch/x86/include/asm/ptrace.h | 21 + arch/x86/include/uapi/asm/processor-flags.h | 2 + arch/x86/kernel/asm-offsets_64.c | 15 + arch/x86/kernel/cpu/common.c | 2 + arch/x86/kernel/dumpstack.c | 32 +- arch/x86/kernel/fpu/xstate.c | 22 +- arch/x86/kernel/head_64.S | 6 + arch/x86/kernel/process_64.c | 3 + arch/x86/mm/fault.c | 17 +-
arch/x86/mm/pkeys.c | 320 +++++++- drivers/dax/device.c | 2 +
drivers/dax/super.c | 60 ++ drivers/md/dm-writecache.c | 8 +- drivers/nvdimm/pmem.c | 26 +
fs/dax.c | 8 +
fs/fuse/virtio_fs.c | 2 +
include/linux/dax.h | 5 + include/linux/entry-common.h | 24 +- include/linux/highmem-internal.h | 6 + include/linux/memremap.h | 73 ++ include/linux/pgtable.h | 4 + include/linux/pks-keys.h | 93 +++ include/linux/pks.h | 73 ++ include/linux/sched.h | 7 + include/uapi/asm-generic/mman-common.h | 1 + init/init_task.c | 3 +
kernel/entry/common.c | 29 +- kernel/sched/core.c | 40 +-
lib/Kconfig.debug | 33 +
lib/Makefile | 3 +
lib/pks/Makefile | 3 +
lib/pks/pks_test.c | 755 ++++++++++++++++++ mm/Kconfig | 32 +
mm/memremap.c | 132 +++ tools/testing/selftests/x86/Makefile | 2 +- tools/testing/selftests/x86/test_pks.c | 514 ++++++++++++

55 files changed, 2618 insertions(+), 112 deletions(-)

create mode 100644 arch/x86/include/asm/pkeys_common.h create mode 100644 arch/x86/include/asm/pks.h create mode 100644 include/linux/pks-keys.h create mode 100644 include/linux/pks.h
create mode 100644 lib/pks/Makefile
create mode 100644 lib/pks/pks_test.c
create mode 100644 tools/testing/selftests/x86/test_pks.c

base-commit: b2d229d4ddb17db541098b83524d901257e93845
prerequisite-patch-id: a73f5ec8b3ecec9c95724106ccb5999c4f955b89 --
2.35.1