mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on all architectures with swap PTEs
Subsystems (27)
cover mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on all architectures with swap PTEs
This is the follow-up on [1]:
[PATCH v2 0/8] mm: COW fixes part 3: reliable GUP R/W FOLL_GET of anonymous pages
After we implemented __HAVE_ARCH_PTE_SWP_EXCLUSIVE on most prominent enterprise architectures, implement __HAVE_ARCH_PTE_SWP_EXCLUSIVE on all remaining architectures that support swap PTEs.
This makes sure that exclusive anonymous pages will stay exclusive, even after they were swapped out -- for example, making GUP R/W FOLL_GET of anonymous pages reliable. Details can be found in [1].
This primarily fixes remaining known O_DIRECT memory corruptions that can happen on concurrent swapout, whereby we can lose DMA reads to a page (modifying the user page by writing to it).
To verify, there are two test cases (requiring swap space, obviously):
(1) The O_DIRECT+swapout test case [2] from Andrea. This test case tries triggering a race condition.
(2) My vmsplice() test case [3] that tries to detect if the exclusive marker was lost during swapout, not relying on a race condition.
For example, on 32bit x86 (with and without PAE), my test case fails without these patches:
$ ./test_swp_exclusive FAIL: page was replaced during COW But succeeds with these patches: $ ./test_swp_exclusive PASS: page was not replaced during COW
Why implement __HAVE_ARCH_PTE_SWP_EXCLUSIVE for all architectures, even the ones where swap support might be in a questionable state? This is the first step towards removing "readable_exclusive" migration entries, and instead using pte_swp_exclusive() also with (readable) migration entries instead (as suggested by Peter). The only missing piece for that is supporting pmd_swp_exclusive() on relevant architectures with THP migration support.
As all relevant architectures now implement __HAVE_ARCH_PTE_SWP_EXCLUSIVE,, we can drop __HAVE_ARCH_PTE_SWP_EXCLUSIVE in the last patch.
I tried cross-compiling all relevant setups and tested on x86 and sparc64 so far.
CCing arch maintainers only on this cover letter and on the respective patch(es).
[1] https://lkml.kernel.org/r/[email protected]
[2] https://gitlab.com/aarcange/kernel-testcases-for-v5.11/-/blob/main/page_count_do_wp_page-swap.c
[3] https://gitlab.com/davidhildenbrand/scratchspace/-/blob/main/test_swp_exclusive.c
RFC -> v1:
* Some smaller comment+patch description changes
* "powerpc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit book3s" -> Fixup swap PTE description
David Hildenbrand (26):
mm/debug_vm_pgtable: more pte_swp_exclusive() sanity checks alpha/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE arc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
csky/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE hexagon/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE ia64/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE loongarch/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE m68k/mm: remove dummy __swp definitions for nommu m68k/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE mips/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE nios2/mm: refactor swap PTE layout
nios2/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE openrisc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE parisc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE powerpc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit book3s powerpc/nohash/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
sh/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
sparc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit sparc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 64bit um/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
x86/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE also on 32bit xtensa/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
mm: remove __HAVE_ARCH_PTE_SWP_EXCLUSIVE
arch/alpha/include/asm/pgtable.h | 40 ++++++++- arch/arc/include/asm/pgtable-bits-arcv2.h | 26 +++++- arch/arm/include/asm/pgtable-2level.h | 3 + arch/arm/include/asm/pgtable-3level.h | 3 + arch/arm/include/asm/pgtable.h | 34 +++++-- arch/arm64/include/asm/pgtable.h | 1 - arch/csky/abiv1/inc/abi/pgtable-bits.h | 13 ++- arch/csky/abiv2/inc/abi/pgtable-bits.h | 19 ++-- arch/csky/include/asm/pgtable.h | 17 ++++ arch/hexagon/include/asm/pgtable.h | 36 ++++++-- arch/ia64/include/asm/pgtable.h | 31 ++++++- arch/loongarch/include/asm/pgtable-bits.h | 4 + arch/loongarch/include/asm/pgtable.h | 38 +++++++- arch/m68k/include/asm/mcf_pgtable.h | 35 +++++++- arch/m68k/include/asm/motorola_pgtable.h | 37 +++++++- arch/m68k/include/asm/pgtable_no.h | 6 -- arch/m68k/include/asm/sun3_pgtable.h | 38 +++++++- arch/microblaze/include/asm/pgtable.h | 44 +++++++--- arch/mips/include/asm/pgtable-32.h | 88 ++++++++++++++++--- arch/mips/include/asm/pgtable-64.h | 23 ++++- arch/mips/include/asm/pgtable.h | 35 ++++++++ arch/nios2/include/asm/pgtable-bits.h | 3 + arch/nios2/include/asm/pgtable.h | 37 ++++++-- arch/openrisc/include/asm/pgtable.h | 40 +++++++-- arch/parisc/include/asm/pgtable.h | 40 ++++++++- arch/powerpc/include/asm/book3s/32/pgtable.h | 37 ++++++-- arch/powerpc/include/asm/book3s/64/pgtable.h | 1 - arch/powerpc/include/asm/nohash/32/pgtable.h | 22 +++-- arch/powerpc/include/asm/nohash/32/pte-40x.h | 6 +- arch/powerpc/include/asm/nohash/32/pte-44x.h | 18 +--- arch/powerpc/include/asm/nohash/32/pte-85xx.h | 4 +- arch/powerpc/include/asm/nohash/64/pgtable.h | 24 ++++- arch/powerpc/include/asm/nohash/pgtable.h | 15 ++++ arch/powerpc/include/asm/nohash/pte-e500.h | 1 - arch/riscv/include/asm/pgtable-bits.h | 3 + arch/riscv/include/asm/pgtable.h | 28 ++++-- arch/s390/include/asm/pgtable.h | 1 - arch/sh/include/asm/pgtable_32.h | 53 ++++++++--- arch/sparc/include/asm/pgtable_32.h | 26 +++++- arch/sparc/include/asm/pgtable_64.h | 37 +++++++- arch/sparc/include/asm/pgtsrmmu.h | 14 +-- arch/um/include/asm/pgtable.h | 36 +++++++- arch/x86/include/asm/pgtable-2level.h | 26 ++++-- arch/x86/include/asm/pgtable-3level.h | 26 +++++- arch/x86/include/asm/pgtable.h | 3 - arch/xtensa/include/asm/pgtable.h | 31 +++++-- include/linux/pgtable.h | 29 ------ mm/debug_vm_pgtable.c | 25 +++++- mm/memory.c | 4 - mm/rmap.c | 11 --- 50 files changed, 944 insertions(+), 228 deletions(-)
Signature · 1 line
2.39.0
PATCH 1/26 mm/debug_vm_pgtable: more pte_swp_exclusive() sanity checks
We want to implement __HAVE_ARCH_PTE_SWP_EXCLUSIVE on all architectures. Let's extend our sanity checks, especially testing that our PTE bit does not affect:
* is_swap_pte() -> pte_present() and pte_none()
* the swap entry + type
* pte_swp_soft_dirty()
Especially, the pfn_pte() is dodgy when the swap PTE layout differs heavily from ordinary PTEs. Let's properly construct a swap PTE from swap type+offset.
--- mm/debug_vm_pgtable.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
mm/debug_vm_pgtable.c +22 -1
@@ -811,13 +811,34 @@ static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args) {static void __init pte_swap_exclusive_tests (struct pgtable_debug_args *args) {#ifdef __HAVE_ARCH_PTE_SWP_EXCLUSIVE -pte_t pte =pfn_pte (args->fixed_pte_pfn, args->page_prot); +unsigned long max_swapfile_size =generic_max_swapfile_size (); +swp_entry_t entry, entry2; +pte_t pte;pr_debug ("Validating PTE swap exclusive\n" ); + +/* Create a swp entry with all possible bits set */ + entry =swp_entry ((1 <<MAX_SWAPFILES_SHIFT ) -1 , + max_swapfile_size -1 ); + + pte =swp_entry_to_pte (entry); +WARN_ON (pte_swp_exclusive (pte)); +WARN_ON (!is_swap_pte (pte)); + entry2 =pte_to_swp_entry (pte); +WARN_ON (memcmp (&entry, &entry2,sizeof (entry))); + pte =pte_swp_mkexclusive (pte);WARN_ON (!pte_swp_exclusive (pte)); +WARN_ON (!is_swap_pte (pte)); +WARN_ON (pte_swp_soft_dirty (pte)); + entry2 =pte_to_swp_entry (pte); +WARN_ON (memcmp (&entry, &entry2,sizeof (entry))); + pte =pte_swp_clear_exclusive (pte);WARN_ON (pte_swp_exclusive (pte)); +WARN_ON (!is_swap_pte (pte)); + entry2 =pte_to_swp_entry (pte); +WARN_ON (memcmp (&entry, &entry2,sizeof (entry)));#endif /* __HAVE_ARCH_PTE_SWP_EXCLUSIVE */ }
Signature · 1 line
2.39.0
Re: [PATCH mm-unstable v1 01/26] mm/debug_vm_pgtable: more pte_swp_exclusive() sanity checks
Quoted 7 lines · David Hildenbrand
We want to implement
__HAVE_ARCH_PTE_SWP_EXCLUSIVEon all architectures. Let's extend our sanity checks, especially testing that our PTE bit does not affect:
*is_swap_pte()->pte_present()andpte_none()
* the swap entry + type
*pte_swp_soft_dirty()
Quoted 6 lines
Especially, the
pfn_pte()is dodgy when the swap PTE layout differs heavily from ordinary PTEs. Let's properly construct a swap PTE from swap type+offset.Signed-off-by David Hildenbrand <[email protected]>---
The following fixup for !CONFIG_SWAP on top, which makes it compile for me and passes when booting on x86_64 with CONFIG_DEBUG_VM_PGTABLE:
...
[ 0.347112] Loaded X.509 cert 'Build time autogenerated kernel key: ee6afc0578f6475656fec8a4f9d02832' [ 0.350112] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers [ 0.351217] page_owner is disabled
...
From 6a6162e8af62a4b3f7b9d823fdfae86de3f34a9d Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Sat, 14 Jan 2023 16:47:12 +0100
Subject: [PATCH] fixup: mm/debug_vm_pgtable: more pte_swp_exclusive() sanity
checks
generic_max_swapfile_size() is only available with CONFIG_SWAP -- which makes sense, because without SWAP there are no swap files. Let's simply probe manually which bits we can obtain after storing them in a PTE, and properly call it "max swap offset", which is more generic for a swap entry.
--- mm/debug_vm_pgtable.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
mm/debug_vm_pgtable.c +5 -3
@@ -810,15 +810,17 @@ static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args) {static void __init pte_swap_exclusive_tests (struct pgtable_debug_args *args) { -unsigned long max_swapfile_size =generic_max_swapfile_size (); +unsigned long max_swap_offset;swp_entry_t entry, entry2;pte_t pte;pr_debug ("Validating PTE swap exclusive\n" ); +/* See generic_max_swapfile_size(): probe the maximum offset */ + max_swap_offset =swp_offset (pte_to_swp_entry (swp_entry_to_pte (swp_entry (0 , ~0UL )))); +/* Create a swp entry with all possible bits set */ - entry =swp_entry ((1 <<MAX_SWAPFILES_SHIFT ) -1 , - max_swapfile_size -1 ); + entry =swp_entry ((1 <<MAX_SWAPFILES_SHIFT ) -1 , max_swap_offset); pte =swp_entry_to_pte (entry);WARN_ON (pte_swp_exclusive (pte));
Signature · 8 lines
2.39.0 -- Thanks, David / dhildenb
PATCH 2/26 alpha/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
While at it, mask the type in mk_swap_pte() as well.
--- arch/alpha/include/asm/pgtable.h | 41 ++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-)
arch/alpha/include/asm/pgtable.h +37 -4
@@ -74,6 +74,9 @@ struct vm_area_struct;#define _PAGE_DIRTY 0x20000#define _PAGE_ACCESSED 0x40000 +/* We borrow bit 39 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x8000000000UL +/* * NOTE! The "accessed" bit isn't necessarily exact: it can be kept exactly * by software (use the KRE/URE/KWE/UWE bits appropriately), but I'll fake it.
@@ -301,18 +304,48 @@ extern inline void update_mmu_cache(struct vm_area_struct * vma, }/* -* Non-present pages: high 24 bits are offset, next 8 bits type, -* low 32 bits zero. +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* <------------------- offset ------------------> E <--- type --> +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <--------------------------- zeroes --------------------------> +* +* E is the exclusive marker that is not stored in swap entries. */ extern inline pte_t mk_swap_pte (unsigned long type,unsigned long offset) -{pte_t pte;pte_val (pte) = (type <<32 ) | (offset <<40 );return pte; } +{pte_t pte;pte_val (pte) = ((type &0x7f ) <<32 ) | (offset <<40 );return pte; } -#define __swp_type (x) (((x).val >> 32) & 0xff) +#define __swp_type (x) (((x).val >> 32) & 0x7f)#define __swp_offset (x) ((x).val >> 40)#define __swp_entry (type, off) ((swp_entry_t) { pte_val(mk_swap_pte((type), (off))) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#define pte_ERROR (e) \ printk("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e))#define pmd_ERROR (e) \
Signature · 1 line
2.39.0
PATCH 3/26 arc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by using bit 5, which is yet unused. The only important parts seems to be to not use _PAGE_PRESENT (bit 9).
--- arch/arc/include/asm/pgtable-bits-arcv2.h | 27 ++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-)
arch/arc/include/asm/pgtable-bits-arcv2.h +24 -3
@@ -26,6 +26,9 @@#define _PAGE_GLOBAL (1 << 8)/* ASID agnostic (H) */ #define _PAGE_PRESENT (1 << 9)/* PTE/TLB Valid (H) */ +/* We borrow bit 5 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY +#ifdef CONFIG_ARC_MMU_V4 #define _PAGE_HW_SZ (1 << 10)/* Normal/super (H) */ #else
@@ -106,9 +109,18 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,void update_mmu_cache (struct vm_area_struct *vma,unsigned long address,pte_t *ptep); -/* Encode swap {type,off} tuple into PTE -* We reserve 13 bits for 5-bit @type, keeping bits 12-5 zero, ensuring that -* PAGE_PRESENT is zero in a PTE holding swap "identifier" +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <-------------- offset -------------> <--- zero --> E < type -> +* +* E is the exclusive marker that is not stored in swap entries. +* The zero'ed bits include _PAGE_PRESENT. */ #define __swp_entry (type, off) ((swp_entry_t) \ { ((type) & 0x1f) | ((off) << 13) })
@@ -120,6 +132,15 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long address,#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +PTE_BIT_FUNC(swp_mkexclusive, |= (_PAGE_SWP_EXCLUSIVE)); +PTE_BIT_FUNC(swp_clear_exclusive, &= ~(_PAGE_SWP_EXCLUSIVE)); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE #include <asm/hugepage.h> #endif
Signature · 1 line
2.39.0
PATCH 4/26 arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the offset. This reduces the maximum swap space per file to 64 GiB (was 128 GiB).
While at it drop the PTE_TYPE_FAULT from __swp_entry_to_pte() which is defined to be 0 and is rather confusing because we should be dealing with "Linux PTEs" not "hardware PTEs". Also, properly mask the type in __swp_entry().
--- arch/arm/include/asm/pgtable-2level.h | 3 +++ arch/arm/include/asm/pgtable-3level.h | 3 +++ arch/arm/include/asm/pgtable.h | 35 +++++++++++++++++++++------ 3 files changed, 34 insertions(+), 7 deletions(-)
arch/arm/include/asm/pgtable-2level.h +3 -0
@@ -126,6 +126,9 @@#define L_PTE_SHARED (_AT(pteval_t, 1) << 10)/* shared(v6), coherent(xsc3) */ #define L_PTE_NONE (_AT(pteval_t, 1) << 11) +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define L_PTE_SWP_EXCLUSIVE L_PTE_RDONLY +/* * These are the memory types, defined to be compatible with * pre-ARMv6 CPUs cacheable and bufferable bits: n/a,n/a,C,B
arch/arm/include/asm/pgtable-3level.h +3 -0
@@ -76,6 +76,9 @@#define L_PTE_NONE (_AT(pteval_t, 1) << 57)/* PROT_NONE */ #define L_PTE_RDONLY (_AT(pteval_t, 1) << 58)/* READ ONLY */ +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define L_PTE_SWP_EXCLUSIVE (_AT(pteval_t, 1) << 7) +#define L_PMD_SECT_VALID (_AT(pmdval_t, 1) << 0)#define L_PMD_SECT_DIRTY (_AT(pmdval_t, 1) << 55)#define L_PMD_SECT_NONE (_AT(pmdval_t, 1) << 57)
arch/arm/include/asm/pgtable.h +28 -7
@@ -271,27 +271,48 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) }/* -* Encode and decode a swap entry. Swap entries are stored in the Linux -* page tables as follows: +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: * * 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 -* <--------------- offset ------------------------> < type -> 0 0 +* <------------------- offset ------------------> E < type -> 0 0 +* +* E is the exclusive marker that is not stored in swap entries. * -* This gives us up to 31 swap files and 128GB per swap file. Note that +* This gives us up to 31 swap files and 64GB per swap file. Note that * the offset field is always non-zero. */ #define __SWP_TYPE_SHIFT 2#define __SWP_TYPE_BITS 5#define __SWP_TYPE_MASK ((1 << __SWP_TYPE_BITS) - 1) -#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT) +#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT + 1)#define __swp_type (x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)#define __swp_offset (x) ((x).val >> __SWP_OFFSET_SHIFT) -#define __swp_entry (type,offset) ((swp_entry_t) { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) }) +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & __SWP_TYPE_BITS) << __SWP_TYPE_SHIFT) | \ + ((offset) << __SWP_OFFSET_SHIFT) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) }) -#define __swp_entry_to_pte (swp) __pte((swp).val | PTE_TYPE_FAULT) +#define __swp_entry_to_pte (swp) __pte((swp).val) + +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_isset (pte,L_PTE_SWP_EXCLUSIVE ); +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (L_PTE_SWP_EXCLUSIVE )); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return clear_pte_bit (pte,__pgprot (L_PTE_SWP_EXCLUSIVE )); +}/* * It is an error for the kernel to have more swap files than we can
Signature · 1 line
2.39.0
Re: [PATCH mm-unstable v1 04/26] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 11 lines · David Hildenbrand
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the offset. This reduces the maximum swap space per file to 64 GiB (was 128 GiB).While at it drop the
PTE_TYPE_FAULTfrom__swp_entry_to_pte()which is defined to be 0 and is rather confusing because we should be dealing with "Linux PTEs" not "hardware PTEs". Also, properly mask the type in__swp_entry().Cc Russell King <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Thanks.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Re: [PATCH mm-unstable v1 04/26] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandQuoted 8 lines · David Hildenbrand
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the offset. This reduces the maximum swap space per file to 64 GiB (was 128 GiB).While at it drop the
PTE_TYPE_FAULTfrom__swp_entry_to_pte()which is defined to be 0 and is rather confusing because we should be dealing with "Linux PTEs" not "hardware PTEs". Also, properly mask the type in__swp_entry().
Today's -next (and at least back to Friday, older logs are unclear - I only noticed -next issues today) fails to NFS boot on an AT91SAM9G20-EK (an old ARMv5 platform) with multi_v5_defconfig, a bisect appears to point to this patch (20aae9eff5acd8f5 in today's -next) as the culprit.
The failure happens at some point after starting userspace, the kernel starts spamming the console with messages in the form:
get_swap_device: Bad swap file entry 10120d20
repeating the same entry number, though different numbers per boot. The system is booting a Debian userspace and shouldn't have swap configured (I verfified that successful boots don't), though it only has 64M of RAM so there will be some memory pressure, especially during boot. The exact point things fall over seems to vary a little.
A sample failing job with the full log is here:
https://lava.sirena.org.uk/scheduler/job/262719
Full bisect log:
git bisect start
# bad: [129af770823407ee115a56c69a04b440fd2fbe61] Add linux-next specific files for 20230206
git bisect bad 129af770823407ee115a56c69a04b440fd2fbe61
# good: [4ec5183ec48656cec489c49f989c508b68b518e3] Linux 6.2-rc7
git bisect good 53b3c6467004c627f42d96ef839b223a749bcdd9
# good: [17b9d0b05d4fa79afb7bd00edb1b97397418a57a] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
git bisect good 17b9d0b05d4fa79afb7bd00edb1b97397418a57a
# good: [7044a4e1fab22f437d275b1cf85f5c925741276b] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
git bisect good 7044a4e1fab22f437d275b1cf85f5c925741276b
# good: [bef6844b00f0c24543d60b79c558f353a43709f1] Merge branch 'staging-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git
git bisect good bef6844b00f0c24543d60b79c558f353a43709f1
# good: [f6737c53676f9db99daee069407daf203e75bc0f] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/rppt/memblock.git
git bisect good f6737c53676f9db99daee069407daf203e75bc0f
# bad: [05cda97ecb7046f4192a921741aae33b300dd628] mm: factor out a swap_writepage_bdev helper
git bisect bad 05cda97ecb7046f4192a921741aae33b300dd628
# good: [ee0800c2f6a9e605947ce499d79fb7e2be16d6dd] mm: convert page_add_anon_rmap() to use a folio internally
git bisect good ee0800c2f6a9e605947ce499d79fb7e2be16d6dd
# bad: [590a2b5f0a9b740e415e0d52bd8a0f87fc15b87b] ceph: convert ceph_writepages_start() to use filemap_get_folios_tag()
git bisect bad 590a2b5f0a9b740e415e0d52bd8a0f87fc15b87b
# good: [92644f583d5124b60bc20a3dd21b0bc9142f020c] mm/khugepaged: introduce release_pte_folio() to replace release_pte_page() git bisect good 92644f583d5124b60bc20a3dd21b0bc9142f020c
# bad: [cca10df1029373cda5904887544ca6fcbbd2bac7] sh/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
git bisect bad cca10df1029373cda5904887544ca6fcbbd2bac7
# bad: [ad464ff2c0f91fcacc24167fc435aa45fe0b7d1b] m68k/mm: remove dummy __swp definitions for nommu
git bisect bad ad464ff2c0f91fcacc24167fc435aa45fe0b7d1b
# bad: [20aae9eff5acd8f50f72adca1176f9269a46b827] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
git bisect bad 20aae9eff5acd8f50f72adca1176f9269a46b827
# good: [2321ba3e3733f513e46e29b9c70512ecddbf1085] mm/debug_vm_pgtable: more pte_swp_exclusive() sanity checks
git bisect good 2321ba3e3733f513e46e29b9c70512ecddbf1085
# good: [4a446b3dd335d0bd14a5ca3e563688de3637be0c] arc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
git bisect good 4a446b3dd335d0bd14a5ca3e563688de3637be0c
# first bad commit: [20aae9eff5acd8f50f72adca1176f9269a46b827] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Re: [PATCH mm-unstable v1 04/26] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 10 lines · Mark Brown
Quoted 8 lines · David Hildenbrand
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the offset. This reduces the maximum swap space per file to 64 GiB (was 128 GiB).While at it drop the
PTE_TYPE_FAULTfrom__swp_entry_to_pte()which is defined to be 0 and is rather confusing because we should be dealing with "Linux PTEs" not "hardware PTEs". Also, properly mask the type in__swp_entry().
Quoted 4 lines
Today's -next (and at least back to Friday, older logs are unclear - I only noticed -next issues today) fails to NFS boot on an AT91SAM9G20-EK (an old ARMv5 platform) with
multi_v5_defconfig, a bisect appears to point to this patch (20aae9eff5acd8f5in today's -next) as the culprit.
It's been in -next for quite a while, thanks for the report!
Quoted 6 lines
The failure happens at some point after starting userspace, the kernel starts spamming the console with messages in the form:
get_swap_device: Bad swap file entry10120d20
_swap_info_get() tells us that the swp type seems to be bad.
I assume we're dealing with a migration entry, if swap is disabled, and fail to detect is_migration_entry() correctly because the type is messed up.
Could you give the following a test?
From 8c4bdbd9862f85782d5919d044c172b584063e83 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Wed, 8 Feb 2023 15:08:01 +0100
Subject: [PATCH] arm/mm: Fix swp type masking in __swp_entry()
We're masking with the number of type bits instead of the type mask, which is obviously wrong.
--- arch/arm/include/asm/pgtable.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/include/asm/pgtable.h +1 -1
@@ -292,7 +292,7 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)#define __swp_type (x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)#define __swp_offset (x) ((x).val >> __SWP_OFFSET_SHIFT) -#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & __SWP_TYPE_BITS) << __SWP_TYPE_SHIFT) | \ +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & __SWP_TYPE_MASK) << __SWP_TYPE_SHIFT) | \ ((offset) << __SWP_OFFSET_SHIFT) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })
Signature · 7 lines
2.39.1 -- Thanks, David / dhildenb
Mark Brown
Tested-byRe: [PATCH mm-unstable v1 04/26] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 8 lines · David Hildenbrand
Quoted 4 lines · Mark Brown
Today's -next (and at least back to Friday, older logs are unclear - I only noticed -next issues today) fails to NFS boot on an AT91SAM9G20-EK (an old ARMv5 platform) with
multi_v5_defconfig, a bisect appears to point to this patch (20aae9eff5acd8f5in today's -next) as the culprit.It's been in -next for quite a while, thanks for the report!
Yeah, there's been some other things obscuring the issue.
Quoted 10 lines
Could you give the following a test?
From
8c4bdbd9862f85782d5919d044c172b584063e83Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Wed, 8 Feb 2023 15:08:01 +0100
Subject: [PATCH] arm/mm: Fix swp type masking in__swp_entry()We're masking with the number of type bits instead of the type mask, which is obviously wrong.
but note that I had to manually apply it, though it's pretty trivial so I probably applied the right thing.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH 5/26 csky/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the offset. This reduces the maximum swap space per file to 16 GiB (was 32 GiB).
We might actually be able to reuse one of the other software bits (_PAGE_READ / PAGE_WRITE) instead, because we only have to keep pte_present(), pte_none() and HW happy. For now, let's keep it simple because there might be something non-obvious.
--- arch/csky/abiv1/inc/abi/pgtable-bits.h | 13 +++++++++---- arch/csky/abiv2/inc/abi/pgtable-bits.h | 19 ++++++++++++------- arch/csky/include/asm/pgtable.h | 18 ++++++++++++++++++ 3 files changed, 39 insertions(+), 11 deletions(-)
arch/csky/abiv1/inc/abi/pgtable-bits.h +9 -4
@@ -10,6 +10,9 @@#define _PAGE_ACCESSED (1<<3)#define _PAGE_MODIFIED (1<<4) +/* We borrow bit 9 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1<<9) +/* implemented in hardware */ #define _PAGE_GLOBAL (1<<6)#define _PAGE_VALID (1<<7)
@@ -26,7 +29,8 @@#define _PAGE_PROT_NONE _PAGE_READ/* -* Encode and decode a swap entry +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). * * Format of swap PTE: * bit 0: _PAGE_PRESENT (zero)
@@ -35,15 +39,16 @@* bit 6: _PAGE_GLOBAL (zero) * bit 7: _PAGE_VALID (zero) * bit 8: swap type[4] -* bit 9 - 31: swap offset +* bit 9: exclusive marker +* bit 10 - 31: swap offset */ #define __swp_type (x) ((((x).val >> 2) & 0xf) | \ (((x).val >> 4) & 0x10)) -#define __swp_offset (x) ((x).val >> 9) +#define __swp_offset (x) ((x).val >> 10)#define __swp_entry (type, offset) ((swp_entry_t) { \ ((type & 0xf) << 2) | \ ((type & 0x10) << 4) | \ - ((offset) << 9)}) + ((offset) << 10)})#define HAVE_ARCH_UNMAPPED_AREA
arch/csky/abiv2/inc/abi/pgtable-bits.h +12 -7
@@ -10,6 +10,9 @@#define _PAGE_PRESENT (1<<10)#define _PAGE_MODIFIED (1<<11) +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1<<7) +/* implemented in hardware */ #define _PAGE_GLOBAL (1<<0)#define _PAGE_VALID (1<<1)
@@ -26,23 +29,25 @@#define _PAGE_PROT_NONE _PAGE_WRITE/* -* Encode and decode a swap entry +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). * * Format of swap PTE: * bit 0: _PAGE_GLOBAL (zero) * bit 1: _PAGE_VALID (zero) * bit 2 - 6: swap type -* bit 7 - 8: swap offset[0 - 1] +* bit 7: exclusive marker +* bit 8: swap offset[0] * bit 9: _PAGE_WRITE (zero) * bit 10: _PAGE_PRESENT (zero) -* bit 11 - 31: swap offset[2 - 22] +* bit 11 - 31: swap offset[1 - 21] */ #define __swp_type (x) (((x).val >> 2) & 0x1f) -#define __swp_offset (x) ((((x).val >> 7) & 0x3) | \ - (((x).val >> 9) & 0x7ffffc)) +#define __swp_offset (x) ((((x).val >> 8) & 0x1) | \ + (((x).val >> 10) & 0x3ffffe))#define __swp_entry (type, offset) ((swp_entry_t) { \ ((type & 0x1f) << 2) | \ - ((offset & 0x3) << 7) | \ - ((offset & 0x7ffffc) << 9)}) + ((offset & 0x1) << 8) | \ + ((offset & 0x3ffffe) << 10)})#endif /* __ASM_CSKY_PGTABLE_BITS_H */
arch/csky/include/asm/pgtable.h +18 -0
@@ -200,6 +200,24 @@ static inline pte_t pte_mkyoung(pte_t pte)return pte; } +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#define __HAVE_PHYS_MEM_ACCESS_PROTstruct file ; extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
Signature · 1 line
2.39.0
PATCH 6/26 hexagon/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the offset. This reduces the maximum swap space per file to 16 GiB (was 32 GiB).
While at it, mask the type in __swp_entry().
--- arch/hexagon/include/asm/pgtable.h | 37 +++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 6 deletions(-)
arch/hexagon/include/asm/pgtable.h +31 -6
@@ -61,6 +61,9 @@ extern unsigned long empty_zero_page;* So we'll put up with a bit of inefficiency for now... */ +/* We borrow bit 6 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1<<6) +/* * Top "FOURTH" level (pgd), which for the Hexagon VM is really * only the second from the bottom, pgd and pud both being collapsed.
@@ -359,9 +362,12 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)#define ZERO_PAGE (vaddr) (virt_to_page(&empty_zero_page))/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* * Swap/file PTE definitions. If _PAGE_PRESENT is zero, the rest of the PTE is * interpreted as swap information. The remaining free bits are interpreted as -* swap type/offset tuple. Rather than have the TLB fill handler test +* listed below. Rather than have the TLB fill handler test * _PAGE_PRESENT, we're going to reserve the permissions bits and set them to * all zeros for swap entries, which speeds up the miss handler at the cost of * 3 bits of offset. That trade-off can be revisited if necessary, but Hexagon
@@ -371,9 +377,10 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)* Format of swap PTE: * bit 0: Present (zero) * bits 1-5: swap type (arch independent layer uses 5 bits max) -* bits 6-9: bits 3:0 of offset +* bit 6: exclusive marker +* bits 7-9: bits 2:0 of offset * bits 10-12: effectively _PAGE_PROTNONE (all zero) -* bits 13-31: bits 22:4 of swap offset +* bits 13-31: bits 21:3 of swap offset * * The split offset makes some of the following macros a little gnarly, * but there's plenty of precedent for this sort of thing.
@@ -383,11 +390,29 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)#define __swp_type (swp_pte) (((swp_pte).val >> 1) & 0x1f)#define __swp_offset (swp_pte) \ - ((((swp_pte).val >> 6) & 0xf) | (((swp_pte).val >> 9) & 0x7ffff0)) + ((((swp_pte).val >> 7) & 0x7) | (((swp_pte).val >> 10) & 0x3ffff8))#define __swp_entry (type, offset) \ ((swp_entry_t) { \ - ((type << 1) | \ - ((offset & 0x7ffff0) << 9) | ((offset & 0xf) << 6)) }) + (((type & 0x1f) << 1) | \ + ((offset & 0x3ffff8) << 10) | ((offset & 0x7) << 7)) }) + +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +}#endif
Signature · 1 line
2.39.0
PATCH 7/26 ia64/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
While at it, also mask the type in __swp_entry().
--- arch/ia64/include/asm/pgtable.h | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-)
arch/ia64/include/asm/pgtable.h +29 -3
@@ -58,6 +58,9 @@#define _PAGE_ED (__IA64_UL(1) << 52)/* exception deferral */ #define _PAGE_PROTNONE (__IA64_UL(1) << 63) +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1 << 7) +#define _PFN_MASK _PAGE_PPN_MASK/* Mask of bits which may be changed by pte_modify(); the odd bits are there for _PAGE_PROTNONE */ #define _PAGE_CHG_MASK (_PAGE_P | _PAGE_PROTNONE | _PAGE_PL_MASK | _PAGE_AR_MASK | _PAGE_ED)
@@ -399,6 +402,9 @@ extern pgd_t swapper_pg_dir[PTRS_PER_PGD];extern void paging_init (void );/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* * Note: The macros below rely on the fact that MAX_SWAPFILES_SHIFT <= number of * bits in the swap-type field of the swap pte. It would be nice to * enforce that, but we can't easily include <linux/swap.h> here.
@@ -406,16 +412,36 @@ extern void paging_init (void);* * Format of swap pte: * bit 0 : present bit (must be zero) -* bits 1- 7: swap-type +* bits 1- 6: swap type +* bit 7 : exclusive marker * bits 8-62: swap offset * bit 63 : _PAGE_PROTNONE bit */ -#define __swp_type (entry) (((entry).val >> 1) & 0x7f) +#define __swp_type (entry) (((entry).val >> 1) & 0x3f)#define __swp_offset (entry) (((entry).val << 1) >> 9) -#define __swp_entry (type,offset) ((swp_entry_t) { ((type) << 1) | ((long) (offset) << 8) }) +#define __swp_entry (type, offset) ((swp_entry_t) { ((type & 0x3f) << 1) | \ + ((long) (offset) << 8) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +/* * ZERO_PAGE is a global shared page that is always zero: used * for zero-mapped memory areas etc..
Signature · 1 line
2.39.0
PATCH 8/26 loongarch/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
While at it, also mask the type in mk_swap_pte().
Note that this bit does not conflict with swap PMDs and could also be used in swap PMD context later.
--- arch/loongarch/include/asm/pgtable-bits.h | 4 +++ arch/loongarch/include/asm/pgtable.h | 39 ++++++++++++++++++++--- 2 files changed, 39 insertions(+), 4 deletions(-)
arch/loongarch/include/asm/pgtable-bits.h +4 -0
@@ -20,6 +20,7 @@#define _PAGE_SPECIAL_SHIFT 11#define _PAGE_HGLOBAL_SHIFT 12/* HGlobal is a PMD bit */ #define _PAGE_PFN_SHIFT 12 +#define _PAGE_SWP_EXCLUSIVE_SHIFT 23#define _PAGE_PFN_END_SHIFT 48#define _PAGE_NO_READ_SHIFT 61#define _PAGE_NO_EXEC_SHIFT 62
@@ -33,6 +34,9 @@#define _PAGE_PROTNONE (_ULCAST_(1) << _PAGE_PROTNONE_SHIFT)#define _PAGE_SPECIAL (_ULCAST_(1) << _PAGE_SPECIAL_SHIFT) +/* We borrow bit 23 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (_ULCAST_(1) << _PAGE_SWP_EXCLUSIVE_SHIFT) +/* Used by TLB hardware (placed in EntryLo*) */ #define _PAGE_VALID (_ULCAST_(1) << _PAGE_VALID_SHIFT)#define _PAGE_DIRTY (_ULCAST_(1) << _PAGE_DIRTY_SHIFT)
arch/loongarch/include/asm/pgtable.h +35 -4
@@ -249,13 +249,26 @@ extern void pud_init(void *addr);extern void pmd_init (void *addr);/* -* Non-present pages: high 40 bits are offset, next 8 bits type, -* low 16 bits zero. +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* <--------------------------- offset --------------------------- +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* --------------> E <--- type ---> <---------- zeroes ----------> +* +* E is the exclusive marker that is not stored in swap entries. +* The zero'ed bits include _PAGE_PRESENT and _PAGE_PROTNONE. */ static inline pte_t mk_swap_pte (unsigned long type,unsigned long offset) -{pte_t pte;pte_val (pte) = (type <<16 ) | (offset <<24 );return pte; } +{pte_t pte;pte_val (pte) = ((type &0x7f ) <<16 ) | (offset <<24 );return pte; } -#define __swp_type (x) (((x).val >> 16) & 0xff) +#define __swp_type (x) (((x).val >> 16) & 0x7f)#define __swp_offset (x) ((x).val >> 24)#define __swp_entry (type, offset) ((swp_entry_t) { pte_val(mk_swap_pte((type), (offset))) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })
@@ -263,6 +276,24 @@ static inline pte_t mk_swap_pte(unsigned long type, unsigned long offset)#define __pmd_to_swp_entry (pmd) ((swp_entry_t) { pmd_val(pmd) })#define __swp_entry_to_pmd (x) ((pmd_t) { (x).val | _PAGE_HUGE }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +extern void paging_init (void );#define pte_none (pte) (!(pte_val(pte) & ~_PAGE_GLOBAL))
Signature · 1 line
2.39.0
PATCH 9/26 m68k/mm: remove dummy __swp definitions for nommu
In reply to David HildenbrandThe definitions are not required, let's remove them.
--- arch/m68k/include/asm/pgtable_no.h | 6 ------ 1 file changed, 6 deletions(-)
arch/m68k/include/asm/pgtable_no.h +0 -6
@@ -31,12 +31,6 @@extern void paging_init (void );#define swapper_pg_dir ((pgd_t *) 0) -#define __swp_type (x) (0) -#define __swp_offset (x) (0) -#define __swp_entry (typ,off) ((swp_entry_t) { ((typ) | ((off) << 7)) }) -#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) }) -#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -/* * ZERO_PAGE is a global shared page that is always zero: used * for zero-mapped memory areas etc..
Signature · 1 line
2.39.0
PATCH 10/26 m68k/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
While at it, make sure for sun3 that the valid bit never gets set by properly masking it off and mask the type in __swp_entry().
--- arch/m68k/include/asm/mcf_pgtable.h | 36 ++++++++++++++++++++-- arch/m68k/include/asm/motorola_pgtable.h | 38 +++++++++++++++++++++-- arch/m68k/include/asm/sun3_pgtable.h | 39 ++++++++++++++++++++++-- 3 files changed, 104 insertions(+), 9 deletions(-)
arch/m68k/include/asm/mcf_pgtable.h +33 -3
@@ -46,6 +46,9 @@#define _CACHEMASK040 (~0x060)#define _PAGE_GLOBAL040 0x400/* 68040 global bit, used for kva descs */ +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x080 +/* * Externally used page protection values. */
@@ -254,15 +257,42 @@ static inline pte_t pte_mkcache(pte_t pte)extern pgd_t kernel_pg_dir[PTRS_PER_PGD ];/* -* Encode and de-code a swap entry (must be !pte_none(e) && !pte_present(e)) +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <------------------ offset -------------> 0 0 0 E <-- type ---> +* +* E is the exclusive marker that is not stored in swap entries. */ -#define __swp_type (x) ((x).val & 0xFF) +#define __swp_type (x) ((x).val & 0x7f)#define __swp_offset (x) ((x).val >> 11) -#define __swp_entry (typ, off) ((swp_entry_t) { (typ) | \ +#define __swp_entry (typ, off) ((swp_entry_t) { ((typ) & 0x7f) | \ (off << 11) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) (__pte((x).val)) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#define pmd_pfn (pmd) (pmd_val(pmd) >> PAGE_SHIFT)#define pmd_page (pmd) (pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT))
arch/m68k/include/asm/motorola_pgtable.h +35 -3
@@ -41,6 +41,9 @@#define _PAGE_PROTNONE 0x004 +/* We borrow bit 11 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x800 +#ifndef __ASSEMBLY__/* This is the cache mode to be used for pages containing page descriptors for
@@ -169,12 +172,41 @@ static inline pte_t pte_mkcache(pte_t pte)#define swapper_pg_dir kernel_pg_dirextern pgd_t kernel_pg_dir[128 ]; -/* Encode and de-code a swap entry (must be !pte_none(e) && !pte_present(e)) */ -#define __swp_type (x) (((x).val >> 4) & 0xff) +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <----------------- offset ------------> E <-- type ---> 0 0 0 0 +* +* E is the exclusive marker that is not stored in swap entries. +*/ +#define __swp_type (x) (((x).val >> 4) & 0x7f)#define __swp_offset (x) ((x).val >> 12) -#define __swp_entry (type, offset) ((swp_entry_t) { ((type) << 4) | ((offset) << 12) }) +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & 0x7f) << 4) | ((offset) << 12) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#endif /* !__ASSEMBLY__ */ #endif /* _MOTOROLA_PGTABLE_H */
arch/m68k/include/asm/sun3_pgtable.h +36 -3
@@ -71,6 +71,9 @@#define SUN3_PMD_MASK (0x0000003F)#define SUN3_PMD_MAGIC (0x0000002B) +/* We borrow bit 6 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x040 +#ifndef __ASSEMBLY__/*
@@ -152,12 +155,42 @@ static inline pte_t pte_mkcache(pte_t pte) { return pte; }extern pgd_t swapper_pg_dir[PTRS_PER_PGD ];extern pgd_t kernel_pg_dir[PTRS_PER_PGD ]; -/* Macros to (de)construct the fake PTEs representing swap pages. */ -#define __swp_type (x) ((x).val & 0x7F) +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* 0 <--------------------- offset ----------------> E <- type --> +* +* E is the exclusive marker that is not stored in swap entries. +*/ +#define __swp_type (x) ((x).val & 0x3f)#define __swp_offset (x) (((x).val) >> 7) -#define __swp_entry (type,offset) ((swp_entry_t) { ((type) | ((offset) << 7)) }) +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & 0x3f) | \ + (((offset) << 7) & ~SUN3_PAGE_VALID)) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#endif /* !__ASSEMBLY__ */ #endif /* !_SUN3_PGTABLE_H */
Signature · 1 line
2.39.0
PATCH 11/26 microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
The shift by 2 when converting between PTE and arch-specific swap entry makes the swap PTE layout a little bit harder to decipher.
While at it, drop the comment from paulus---copy-and-paste leftover from powerpc where we actually have _PAGE_HASHPTE---and mask the type in __swp_entry_to_pte() as well.
--- arch/m68k/include/asm/mcf_pgtable.h | 4 +-- arch/microblaze/include/asm/pgtable.h | 45 +++++++++++++++++++++------ 2 files changed, 37 insertions(+), 12 deletions(-)
arch/m68k/include/asm/mcf_pgtable.h +2 -2
@@ -46,8 +46,8 @@#define _CACHEMASK040 (~0x060)#define _PAGE_GLOBAL040 0x400/* 68040 global bit, used for kva descs */ -/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ -#define _PAGE_SWP_EXCLUSIVE 0x080 +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE CF_PAGE_NOCACHE/* * Externally used page protection values.
arch/microblaze/include/asm/pgtable.h +35 -10
69 lines not on this page · raw message
Signature · 1 line
2.39.0
Geert Uytterhoeven
maintainerRe: [PATCH mm-unstable v1 11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Hi David,
Quoted 7 lines · David Hildenbrand
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.The shift by 2 when converting between PTE and arch-specific swap entry makes the swap PTE layout a little bit harder to decipher.
Quoted 6 lines
While at it, drop the comment from paulus---copy-and-paste leftover from powerpc where we actually have _PAGE_HASHPTE---and mask the type in
__swp_entry_to_pte()as well.Cc Michal Simek <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Thanks for your patch, which is now commit b5c88f21531c3457 ("microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE") in
Quoted 1 line
arch/m68k/include/asm/mcf_pgtable.h | 4 +--
What is this m68k change doing here?
Sorry for not noticing this earlier.
Furthermore, several things below look strange to me...
Quoted 8 lines
arch/microblaze/include/asm/pgtable.h | 45 +++++++++++++++++++++------ 2 files changed, 37 insertions(+), 12 deletions(-)arch/m68k/include/asm/mcf_pgtable.h +0
-0
arch/m68k/include/asm/mcf_pgtable.h line 50
Quoted 7 lines
arch/m68k/include/asm/mcf_pgtable.h +2
-2@@ -46,8 +46,8 @@#define _CACHEMASK040 (~0x060)#define _PAGE_GLOBAL040 0x400/* 68040 global bit, used for kva descs */ -/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ -#define _PAGE_SWP_EXCLUSIVE 0x080 +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE CF_PAGE_NOCACHE
CF_PAGE_NOCACHE is 0x80, so this is still bit 7, thus the new comment is wrong?
Quoted 9 lines
arch/m68k/include/asm/mcf_pgtable.h +0
-0/* * Externally used page protection values. arch/microblaze/include/asm/pgtable.h +0
-0@@ -131,10 +131,10 @@ extern pte_t *va_to_pte(unsigned long address);* of the 16 available. Bit 24-26 of the TLB are cleared in the TLB
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 137
Quoted 10 lines
arch/microblaze/include/asm/pgtable.h +4
-4* miss handler. Bit 27 is PAGE_USER, thus selecting the correct * zone. -* - PRESENT *must* be in the bottom two bits because swap cache -* entries use the top 30 bits. Because 4xx doesn't support SMP -* anyway, M is irrelevant so we borrow it for PAGE_PRESENT. Bit 30 -* is cleared in the TLB miss handler before the TLB entry is loaded. +* - PRESENT *must* be in the bottom two bits because swap PTEs use the top +* 30 bits. Because 4xx doesn't support SMP anyway, M is irrelevant so we +* borrow it for PAGE_PRESENT. Bit 30 is cleared in the TLB miss handler +* before the TLB entry is loaded.
So the PowerPC 4xx comment is still here?
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 159
Quoted 9 lines
arch/microblaze/include/asm/pgtable.h +2
-0* - All other bits of the PTE are loaded into TLBLO without * * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for * software PTE bits. We actually use bits 21, 24, 25, and @@ -155,6 +155,9 @@ extern pte_t *va_to_pte(unsigned long address);#define _PAGE_ACCESSED 0x400/* software: R: page referenced */ #define _PMD_PRESENT PAGE_MASK +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY
_PAGE_DIRTY is 0x80, so this is also bit 7, thus the new comment is wrong?
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH mm-unstable v1 11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 1 line · Geert Uytterhoeven
Hi David,
Hi Geert,
Quoted 13 lines · David Hildenbrand
Quoted 11 lines
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.The shift by 2 when converting between PTE and arch-specific swap entry makes the swap PTE layout a little bit harder to decipher.
While at it, drop the comment from paulus---copy-and-paste leftover from powerpc where we actually have _PAGE_HASHPTE---and mask the type in
__swp_entry_to_pte()as well.
Quoted 6 lines
Quoted 2 lines
Cc Michal Simek <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Thanks for your patch, which is now commit
b5c88f21531c3457("microblaze/mm: support__HAVE_ARCH_PTE_SWP_EXCLUSIVE") in
Right, it went upstream, so we can only fixup.
Quoted 4 lines
Quoted 1 line
arch/m68k/include/asm/mcf_pgtable.h | 4 +--What is this m68k change doing here?
Sorry for not noticing this earlier.
Thanks for the late review, still valuable :)
That hunk should have gone into the previous patch, looks like I messed that up when reworking.
Quoted 14 lines
Furthermore, several things below look strange to me...
Quoted 11 lines
arch/microblaze/include/asm/pgtable.h | 45 +++++++++++++++++++++------ 2 files changed, 37 insertions(+), 12 deletions(-)arch/m68k/include/asm/mcf_pgtable.h +0
-0@@ -46,8 +46,8 @@#define _CACHEMASK040 (~0x060)#define _PAGE_GLOBAL040 0x400/* 68040 global bit, used for kva descs */
Quoted 7 lines
arch/m68k/include/asm/mcf_pgtable.h line 50
Quoted 4 lines
arch/m68k/include/asm/mcf_pgtable.h +2
-2-/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ -#define _PAGE_SWP_EXCLUSIVE 0x080 +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE CF_PAGE_NOCACHE
CF_PAGE_NOCACHEis0x80, so this is still bit 7, thus the new comment is wrong?
You're right, it's still bit 7 (and we use LSB-0 bit numbering in that file). I'll send a fixup.
Quoted 14 lines
Quoted 14 lines
arch/m68k/include/asm/mcf_pgtable.h +0
-0/* * Externally used page protection values. arch/microblaze/include/asm/pgtable.h +0
-2@@ -131,10 +131,10 @@ extern pte_t *va_to_pte(unsigned long address);* of the 16 available. Bit 24-26 of the TLB are cleared in the TLB * miss handler. Bit 27 is PAGE_USER, thus selecting the correct * zone. -* - PRESENT *must* be in the bottom two bits because swap cache -* entries use the top 30 bits. Because 4xx doesn't support SMP
Quoted 8 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 137
Quoted 6 lines
arch/microblaze/include/asm/pgtable.h +4
-2-* anyway, M is irrelevant so we borrow it for PAGE_PRESENT. Bit 30 -* is cleared in the TLB miss handler before the TLB entry is loaded. +* - PRESENT *must* be in the bottom two bits because swap PTEs use the top +* 30 bits. Because 4xx doesn't support SMP anyway, M is irrelevant so we +* borrow it for PAGE_PRESENT. Bit 30 is cleared in the TLB miss handler +* before the TLB entry is loaded. So the PowerPC 4xx comment is still here?
I only dropped the comment above __swp_type(). I guess you mean that we could also drop the "Because 4xx doesn't support SMP anyway, M is irrelevant so we borrow it for PAGE_PRESENT." sentence, correct? Not sure about the "Bit 30 is cleared in the TLB miss handler" comment, if that can similarly be dropped.
Quoted 8 lines
Quoted 8 lines
arch/microblaze/include/asm/pgtable.h +0
-0* - All other bits of the PTE are loaded into TLBLO without * * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for * software PTE bits. We actually use bits 21, 24, 25, and @@ -155,6 +155,9 @@ extern pte_t *va_to_pte(unsigned long address);#define _PAGE_ACCESSED 0x400/* software: R: page referenced */ #define _PMD_PRESENT PAGE_MASK
Quoted 5 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 159
Quoted 2 lines
arch/microblaze/include/asm/pgtable.h +2
-0+/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY
_PAGE_DIRTYis0x80, so this is also bit 7, thus the new comment is wrong?
In the example, I use MSB-0 bit numbering (which I determined to be correct in microblaze context eventually, but I got confused a couple a times because it's very inconsistent). That should be MSB-0 bit 24.
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
Geert Uytterhoeven
maintainerRe: [PATCH mm-unstable v1 11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Hi David,
Quoted 54 lines · David Hildenbrand
Quoted 12 lines · Geert Uytterhoeven
Quoted 11 lines · David Hildenbrand
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.The shift by 2 when converting between PTE and arch-specific swap entry makes the swap PTE layout a little bit harder to decipher.
While at it, drop the comment from paulus---copy-and-paste leftover from powerpc where we actually have _PAGE_HASHPTE---and mask the type in
__swp_entry_to_pte()as well.Quoted 6 lines
Quoted 2 lines
Cc Michal Simek <[email protected]>Signed-off-by David Hildenbrand <[email protected]>Thanks for your patch, which is now commit
b5c88f21531c3457("microblaze/mm: support__HAVE_ARCH_PTE_SWP_EXCLUSIVE") inRight, it went upstream, so we can only fixup.
Quoted 4 lines
Quoted 1 line
arch/m68k/include/asm/mcf_pgtable.h | 4 +--What is this m68k change doing here?
Sorry for not noticing this earlier.Thanks for the late review, still valuable :)
That hunk should have gone into the previous patch, looks like I messed that up when reworking.
Quoted 14 lines
Furthermore, several things below look strange to me...
Quoted 11 lines
arch/microblaze/include/asm/pgtable.h | 45 +++++++++++++++++++++------ 2 files changed, 37 insertions(+), 12 deletions(-)arch/m68k/include/asm/mcf_pgtable.h +0
-0@@ -46,8 +46,8 @@#define _CACHEMASK040 (~0x060)#define _PAGE_GLOBAL040 0x400/* 68040 global bit, used for kva descs */ Quoted 7 lines
arch/m68k/include/asm/mcf_pgtable.h line 50
Quoted 4 lines
arch/m68k/include/asm/mcf_pgtable.h +2
-2-/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ -#define _PAGE_SWP_EXCLUSIVE 0x080 +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE CF_PAGE_NOCACHE
CF_PAGE_NOCACHEis0x80, so this is still bit 7, thus the new comment is wrong?
Quoted 2 lines
You're right, it's still bit 7 (and we use LSB-0 bit numbering in that file). I'll send a fixup.
OK.
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
arch/m68k/include/asm/mcf_pgtable.h +0
-0/* * Externally used page protection values. arch/microblaze/include/asm/pgtable.h +0
-2@@ -131,10 +131,10 @@ extern pte_t *va_to_pte(unsigned long address);* of the 16 available. Bit 24-26 of the TLB are cleared in the TLB * miss handler. Bit 27 is PAGE_USER, thus selecting the correct * zone. -* - PRESENT *must* be in the bottom two bits because swap cache -* entries use the top 30 bits. Because 4xx doesn't support SMP Quoted 8 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 137
Quoted 6 lines
arch/microblaze/include/asm/pgtable.h +4
-2-* anyway, M is irrelevant so we borrow it for PAGE_PRESENT. Bit 30 -* is cleared in the TLB miss handler before the TLB entry is loaded. +* - PRESENT *must* be in the bottom two bits because swap PTEs use the top +* 30 bits. Because 4xx doesn't support SMP anyway, M is irrelevant so we +* borrow it for PAGE_PRESENT. Bit 30 is cleared in the TLB miss handler +* before the TLB entry is loaded. So the PowerPC 4xx comment is still here?
Quoted 3 lines
I only dropped the comment above
__swp_type(). I guess you mean that we could also drop the "Because 4xx doesn't support SMP anyway, M is irrelevant so we borrow it forPAGE_PRESENT." sentence, correct? Not
Yes, that's what I meant.
Quoted 2 lines
sure about the "Bit 30 is cleared in the TLB miss handler" comment, if that can similarly be dropped.
No idea, didn't check. But if it was copied from PPC, chances are high it's no longer true....
Quoted 13 lines
Quoted 7 lines
Quoted 7 lines
arch/microblaze/include/asm/pgtable.h +0
-0* - All other bits of the PTE are loaded into TLBLO without * * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for * software PTE bits. We actually use bits 21, 24, 25, and @@ -155,6 +155,9 @@ extern pte_t *va_to_pte(unsigned long address);#define _PAGE_ACCESSED 0x400/* software: R: page referenced */ #define _PMD_PRESENT PAGE_MASKQuoted 5 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 159
Quoted 2 lines
arch/microblaze/include/asm/pgtable.h +2
-0+/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY
_PAGE_DIRTYis0x80, so this is also bit 7, thus the new comment is wrong?
Quoted 3 lines
In the example, I use MSB-0 bit numbering (which I determined to be correct in microblaze context eventually, but I got confused a couple a times because it's very inconsistent). That should be MSB-0 bit 24.
Thanks, TIL microblaze uses IBM bit numbering...
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH mm-unstable v1 11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 25 lines
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
/*
* Externally used page protection values.arch/microblaze/include/asm/pgtable.h +0
-2@@ -131,10 +131,10 @@ extern pte_t *va_to_pte(unsigned long address);* of the 16 available. Bit 24-26 of the TLB are cleared in the TLB * miss handler. Bit 27 is PAGE_USER, thus selecting the correct * zone. -* - PRESENT *must* be in the bottom two bits because swap cache -* entries use the top 30 bits. Because 4xx doesn't support SMP Quoted 8 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 137
Quoted 6 lines
arch/microblaze/include/asm/pgtable.h +4
-2-* anyway, M is irrelevant so we borrow it for PAGE_PRESENT. Bit 30 -* is cleared in the TLB miss handler before the TLB entry is loaded. +* - PRESENT *must* be in the bottom two bits because swap PTEs use the top +* 30 bits. Because 4xx doesn't support SMP anyway, M is irrelevant so we +* borrow it for PAGE_PRESENT. Bit 30 is cleared in the TLB miss handler +* before the TLB entry is loaded. So the PowerPC 4xx comment is still here?
Quoted 3 lines
I only dropped the comment above
__swp_type(). I guess you mean that we could also drop the "Because 4xx doesn't support SMP anyway, M is irrelevant so we borrow it forPAGE_PRESENT." sentence, correct? Not
Quoted 7 lines
Yes, that's what I meant.
Quoted 2 lines
sure about the "Bit 30 is cleared in the TLB miss handler" comment, if that can similarly be dropped.
No idea, didn't check. But if it was copied from PPC, chances are high it's no longer true....
I'll have a look.
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 159
Quoted 10 lines
arch/microblaze/include/asm/pgtable.h +2
-0* - All other bits of the PTE are loaded into TLBLO without * * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for * software PTE bits. We actually use bits 21, 24, 25, and @@ -155,6 +155,9 @@ extern pte_t *va_to_pte(unsigned long address);#define _PAGE_ACCESSED 0x400/* software: R: page referenced */ #define _PMD_PRESENT PAGE_MASK +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY
_PAGE_DIRTYis0x80, so this is also bit 7, thus the new comment is wrong?
Quoted 5 lines
Quoted 3 lines
In the example, I use MSB-0 bit numbering (which I determined to be correct in microblaze context eventually, but I got confused a couple a times because it's very inconsistent). That should be MSB-0 bit 24.
Thanks, TIL microblaze uses IBM bit numbering...
I assume IBM bit numbering corresponds to MSB-0 bit numbering, correct?
I recall that I used the comment above "/* Definitions for MicroBlaze. */" as an orientation.
0 1 2 3 4 ... 18 19 20 21 22 23 24 25 26 27 28 29 30 31 RPN..................... 0 0 EX WR ZSEL....... W I M G
So ... either we adjust both or we leave it as is. (again, depends on what the right thing to to is -- which I don't know :) )
Signature · 3 lines
Thanks, David / dhildenb
Geert Uytterhoeven
maintainerRe: [PATCH mm-unstable v1 11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Hi David,
Quoted 20 lines · David Hildenbrand
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 11 lines
/*
* Externally used page protection values.arch/microblaze/include/asm/pgtable.h +0
-0* - All other bits of the PTE are loaded into TLBLO without * * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for * software PTE bits. We actually use bits 21, 24, 25, and arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 159
Quoted 5 lines
arch/microblaze/include/asm/pgtable.h +2
-0@@ -155,6 +155,9 @@ extern pte_t *va_to_pte(unsigned long address);#define _PAGE_ACCESSED 0x400/* software: R: page referenced */ #define _PMD_PRESENT PAGE_MASK +/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY
_PAGE_DIRTYis0x80, so this is also bit 7, thus the new comment is wrong?
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
In the example, I use MSB-0 bit numbering (which I determined to be correct in microblaze context eventually, but I got confused a couple a times because it's very inconsistent). That should be MSB-0 bit 24.
Thanks, TIL microblaze uses IBM bit numbering...
I assume IBM bit numbering corresponds to MSB-0 bit numbering, correct?
Correct, as seen in s370 and PowerPC manuals...
Quoted 5 lines
I recall that I used the comment above "/* Definitions for MicroBlaze. */" as an orientation.
0 1 2 3 4 ... 18 19 20 21 22 23 24 25 26 27 28 29 30 31 RPN..................... 0 0 EX WR ZSEL....... W I M G
Indeed, that's where I noticed the "unconventional" numbering...
Quoted 2 lines
So ... either we adjust both or we leave it as is. (again, depends on what the right thing to to is -- which I don't know :) )
It depends whether you want to match the hardware documentation, or the Linux BIT() macro and friends...
Gr{oetje,eeting}s,
Geert
Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected] In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds
Re: [PATCH mm-unstable v1 11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 27 lines · Geert Uytterhoeven
Hi David,
Quoted 24 lines · David Hildenbrand
Quoted 24 lines
Quoted 23 lines
Quoted 14 lines
Quoted 14 lines
/*
* Externally used page protection values.arch/microblaze/include/asm/pgtable.h +0
-0* - All other bits of the PTE are loaded into TLBLO without * * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for * software PTE bits. We actually use bits 21, 24, 25, and @@ -155,6 +155,9 @@ extern pte_t *va_to_pte(unsigned long address);#define _PAGE_ACCESSED 0x400/* software: R: page referenced */ #define _PMD_PRESENT PAGE_MASKQuoted 5 lines
arch/microblaze/include/asm/pgtable.h extern pte_t *va_to_pte(unsigned long address); · line 159
Quoted 2 lines
arch/microblaze/include/asm/pgtable.h +2
-0+/* We borrow bit 24 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_DIRTY
_PAGE_DIRTYis0x80, so this is also bit 7, thus the new comment is wrong?In the example, I use MSB-0 bit numbering (which I determined to be correct in microblaze context eventually, but I got confused a couple a times because it's very inconsistent). That should be MSB-0 bit 24.
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Thanks, TIL microblaze uses IBM bit numbering...
I assume IBM bit numbering corresponds to MSB-0 bit numbering, correct?
Correct, as seen in s370 and PowerPC manuals...
Good, I have some solid s390x background, but thinking about the term "IBM PC" made me double-check that we're talking about the same thing ;)
Quoted 7 lines
Quoted 6 lines
I recall that I used the comment above "/* Definitions for MicroBlaze. */" as an orientation.
0 1 2 3 4 ... 18 19 20 21 22 23 24 25 26 27 28 29 30 31 RPN..................... 0 0 EX WR ZSEL....... W I M G
Quoted 7 lines
Indeed, that's where I noticed the "unconventional" numbering...
Quoted 2 lines
So ... either we adjust both or we leave it as is. (again, depends on what the right thing to to is -- which I don't know :) )
It depends whether you want to match the hardware documentation, or the Linux
BIT()macro and friends...
The hardware documentation, so we should be good.
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
PATCH 12/26 mips/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE.
On 64bit, steal one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
On 32bit we're able to locate unused bits. As the PTE layout for 32 bit is very confusing, document it a bit better.
While at it, mask the type in __swp_entry()/mk_swap_pte().
--- arch/mips/include/asm/pgtable-32.h | 88 ++++++++++++++++++++++++++---- arch/mips/include/asm/pgtable-64.h | 23 ++++++-- arch/mips/include/asm/pgtable.h | 36 ++++++++++++ 3 files changed, 131 insertions(+), 16 deletions(-)
arch/mips/include/asm/pgtable-32.h +76 -12
@@ -191,49 +191,113 @@ static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot)#define pte_page (x) pfn_to_page(pte_pfn(x)) +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +*/ #if defined(CONFIG_CPU_R3K_TLB ) -/* Swap entries must have VALID bit cleared. */ +/* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <----------- offset ------------> < type -> V G E 0 0 0 0 0 0 P +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P), _PAGE_VALID (V) and_PAGE_GLOBAL (G) have to remain +* unused. +*/ #define __swp_type (x) (((x).val >> 10) & 0x1f)#define __swp_offset (x) ((x).val >> 15) -#define __swp_entry (type,offset) ((swp_entry_t) { ((type) << 10) | ((offset) << 15) }) +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & 0x1f) << 10) | ((offset) << 15) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1 << 7) +#else #if defined(CONFIG_XPA ) -/* Swap entries must have VALID and GLOBAL bits cleared. */ +/* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* 0 0 0 0 0 0 E P <------------------ zeroes -------------------> +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <----------------- offset ------------------> < type -> V G 0 0 +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P), _PAGE_VALID (V) and_PAGE_GLOBAL (G) have to remain +* unused. +*/ #define __swp_type (x) (((x).val >> 4) & 0x1f)#define __swp_offset (x) ((x).val >> 9) -#define __swp_entry (type,offset) ((swp_entry_t) { ((type) << 4) | ((offset) << 9) }) +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & 0x1f) << 4) | ((offset) << 9) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { (pte).pte_high })#define __swp_entry_to_pte (x) ((pte_t) { 0, (x).val }) +/* +* We borrow bit 57 (bit 25 in the low PTE) to store the exclusive marker in +* swap PTEs. +*/ +#define _PAGE_SWP_EXCLUSIVE (1 << 25) +#elif defined(CONFIG_PHYS_ADDR_T_64BIT ) && defined(CONFIG_CPU_MIPS32 ) -/* Swap entries must have VALID and GLOBAL bits cleared. */ +/* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* <------------------ zeroes -------------------> E P 0 0 0 0 0 0 +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <------------------- offset --------------------> < type -> V G +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P), _PAGE_VALID (V) and_PAGE_GLOBAL (G) have to remain +* unused. +*/ #define __swp_type (x) (((x).val >> 2) & 0x1f)#define __swp_offset (x) ((x).val >> 7) -#define __swp_entry (type, offset) ((swp_entry_t) { ((type) << 2) | ((offset) << 7) }) +#define __swp_entry (type, offset) ((swp_entry_t) { (((type) & 0x1f) << 2) | ((offset) << 7) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { (pte).pte_high })#define __swp_entry_to_pte (x) ((pte_t) { 0, (x).val }) +/* +* We borrow bit 39 (bit 7 in the low PTE) to store the exclusive marker in swap +* PTEs. +*/ +#define _PAGE_SWP_EXCLUSIVE (1 << 7) +#else /* -* Constraints: -* _PAGE_PRESENT at bit 0 -* _PAGE_MODIFIED at bit 4 -* _PAGE_GLOBAL at bit 6 -* _PAGE_VALID at bit 7 +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <------------- offset --------------> < type -> 0 0 0 0 0 0 E P +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P), _PAGE_VALID (V) and_PAGE_GLOBAL (G) have to remain +* unused. The location of V and G varies. */ #define __swp_type (x) (((x).val >> 8) & 0x1f)#define __swp_offset (x) ((x).val >> 13) -#define __swp_entry (type,offset) ((swp_entry_t) { ((type) << 8) | ((offset) << 13) }) +#define __swp_entry (type, offset) ((swp_entry_t) { ((type) << 8) | ((offset) << 13) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +/* We borrow bit 1 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1 << 1) +#endif /* defined(CONFIG_PHYS_ADDR_T_64BIT) && defined(CONFIG_CPU_MIPS32) */ #endif /* defined(CONFIG_CPU_R3K_TLB) */
arch/mips/include/asm/pgtable-64.h +19 -4
@@ -320,16 +320,31 @@ extern void pud_init(void *addr);extern void pmd_init (void *addr);/* -* Non-present pages: high 40 bits are offset, next 8 bits type, -* low 16 bits zero. +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* <--------------------------- offset --------------------------- +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* --------------> E <-- type ---> <---------- zeroes -----------> +* +* E is the exclusive marker that is not stored in swap entries. */ static inline pte_t mk_swap_pte (unsigned long type,unsigned long offset) -{pte_t pte;pte_val (pte) = (type <<16 ) | (offset <<24 );return pte; } +{pte_t pte;pte_val (pte) = ((type &0x7f ) <<16 ) | (offset <<24 );return pte; } -#define __swp_type (x) (((x).val >> 16) & 0xff) +#define __swp_type (x) (((x).val >> 16) & 0x7f)#define __swp_offset (x) ((x).val >> 24)#define __swp_entry (type, offset) ((swp_entry_t) { pte_val(mk_swap_pte((type), (offset))) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +/* We borrow bit 23 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1 << 23) +#endif /* _ASM_PGTABLE_64_H */
arch/mips/include/asm/pgtable.h +36 -0
@@ -528,6 +528,42 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) }#endif +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +#if defined(CONFIG_PHYS_ADDR_T_64BIT ) && defined(CONFIG_CPU_MIPS32 ) +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte.pte_low & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ + pte.pte_low |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ + pte.pte_low &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#else +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#endif extern void __update_tlb (struct vm_area_struct *vma,unsigned long address,pte_t pte);
Signature · 1 line
2.39.0
PATCH 13/26 nios2/mm: refactor swap PTE layout
In reply to David Hildenbrandnios2 disables swap for a good reason: it doesn't even provide sufficient type bits as required by core MM. However, swap entries are nowadays also used for other purposes (migration entries, PTE markers, HWPoison, ...), and accidential use could be problematic.
Let's properly use 5 bits for the swap type and document the layout. Bits 26--31 should get ignored by hardware completely, so they can be used.
--- arch/nios2/include/asm/pgtable.h | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
arch/nios2/include/asm/pgtable.h +10 -8
@@ -232,19 +232,21 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) __FILE__, __LINE__, pgd_val(e))/* -* Encode and decode a swap entry (must be !pte_none(pte) && !pte_present(pte): +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). * -* 31 30 29 28 27 26 25 24 23 22 21 20 19 18 ... 1 0 -* 0 0 0 0 type. 0 0 0 0 0 0 offset......... +* Format of swap PTEs: * -* This gives us up to 2**2 = 4 swap files and 2**20 * 4K = 4G per swap file. +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* 0 < type -> 0 0 0 0 0 0 <-------------- offset ---------------> * -* Note that the offset field is always non-zero, thus !pte_none(pte) is always -* true. +* Note that the offset field is always non-zero if the swap type is 0, thus +* !pte_none() is always true. */ -#define __swp_type (swp) (((swp).val >> 26) & 0x3) +#define __swp_type (swp) (((swp).val >> 26) & 0x1f)#define __swp_offset (swp) ((swp).val & 0xfffff) -#define __swp_entry (type, off) ((swp_entry_t) { (((type) & 0x3) << 26) \ +#define __swp_entry (type, off) ((swp_entry_t) { (((type) & 0x1f) << 26) \ | ((off) & 0xfffff) })#define __swp_entry_to_pte (swp) ((pte_t) { (swp).val })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })
Signature · 1 line
2.39.0
PATCH 14/26 nios2/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by using the yet-unused bit 31.
--- arch/nios2/include/asm/pgtable-bits.h | 3 +++ arch/nios2/include/asm/pgtable.h | 22 +++++++++++++++++++++- 2 files changed, 24 insertions(+), 1 deletion(-)
arch/nios2/include/asm/pgtable-bits.h +3 -0
@@ -31,4 +31,7 @@#define _PAGE_ACCESSED (1<<26)/* page referenced */ #define _PAGE_DIRTY (1<<27)/* dirty page */ +/* We borrow bit 31 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1<<31) +#endif /* _ASM_NIOS2_PGTABLE_BITS_H */
arch/nios2/include/asm/pgtable.h +21 -1
@@ -239,7 +239,9 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)* * 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 -* 0 < type -> 0 0 0 0 0 0 <-------------- offset ---------------> +* E < type -> 0 0 0 0 0 0 <-------------- offset ---------------> +* +* E is the exclusive marker that is not stored in swap entries. * * Note that the offset field is always non-zero if the swap type is 0, thus * !pte_none() is always true.
@@ -251,6 +253,24 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)#define __swp_entry_to_pte (swp) ((pte_t) { (swp).val })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +extern void __init paging_init (void );extern void __init mmu_init (void );
Signature · 1 line
2.39.0
PATCH 15/26 openrisc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
While at it, mask the type in __swp_entry().
--- arch/openrisc/include/asm/pgtable.h | 41 +++++++++++++++++++++++++---- 1 file changed, 36 insertions(+), 5 deletions(-)
arch/openrisc/include/asm/pgtable.h +36 -5
@@ -154,6 +154,9 @@ extern void paging_init(void);#define _KERNPG_TABLE \ (_PAGE_BASE | _PAGE_SRE | _PAGE_SWE | _PAGE_ACCESSED | _PAGE_DIRTY) +/* We borrow bit 11 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_U_SHARED +#define PAGE_NONE __pgprot(_PAGE_ALL)#define PAGE_READONLY __pgprot(_PAGE_ALL | _PAGE_URE | _PAGE_SRE)#define PAGE_READONLY_X __pgprot(_PAGE_ALL | _PAGE_URE | _PAGE_SRE | _PAGE_EXEC)
@@ -385,16 +388,44 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,/* __PHX__ FIXME, SWAP, this probably doesn't work */ -/* Encode and de-code a swap entry (must be !pte_none(e) && !pte_present(e)) */ -/* Since the PAGE_PRESENT bit is bit 4, we can use the bits above */ - -#define __swp_type (x) (((x).val >> 5) & 0x7f) +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <-------------- offset ---------------> E <- type --> 0 0 0 0 0 +* +* E is the exclusive marker that is not stored in swap entries. +* The zero'ed bits include _PAGE_PRESENT. +*/ +#define __swp_type (x) (((x).val >> 5) & 0x3f)#define __swp_offset (x) ((x).val >> 12)#define __swp_entry (type, offset) \ - ((swp_entry_t) { ((type) << 5) | ((offset) << 12) }) + ((swp_entry_t) { (((type) & 0x3f) << 5) | ((offset) << 12) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +typedef pte_t *pte_addr_t ;#endif /* __ASSEMBLY__ */
Signature · 1 line
2.39.0
PATCH 16/26 parisc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by using the yet-unused _PAGE_ACCESSED location in the swap PTE. Looking at pte_present() and pte_none() checks, there seems to be no actual reason why we cannot use it: we only have to make sure we're not using _PAGE_PRESENT.
Reusing this bit avoids having to steal one bit from the swap offset.
--- arch/parisc/include/asm/pgtable.h | 41 ++++++++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 3 deletions(-)
arch/parisc/include/asm/pgtable.h +38 -3
@@ -218,6 +218,9 @@ extern void __update_cache(pte_t pte);#define _PAGE_KERNEL_RWX (_PAGE_KERNEL_EXEC | _PAGE_WRITE)#define _PAGE_KERNEL (_PAGE_KERNEL_RO | _PAGE_WRITE) +/* We borrow bit 23 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_ACCESSED +/* The pgd/pmd contains a ptr (in phys addr space); since all pgds/pmds * are page-aligned, we don't care about the PAGE_OFFSET bits, except * for a few meta-information bits, so we shift the address to be
@@ -394,17 +397,49 @@ extern void paging_init (void);#define update_mmu_cache (vms,addr,ptep) __update_cache(*ptep) -/* Encode and de-code a swap entry */ - +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs (32bit): +* +* 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 +* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +* <---------------- offset -----------------> P E <ofs> < type -> +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P) must be 0. +* +* For the 64bit version, the offset is extended by 32bit. +*/ #define __swp_type (x) ((x).val & 0x1f)#define __swp_offset (x) ( (((x).val >> 6) & 0x7) | \ (((x).val >> 8) & ~0x7) ) -#define __swp_entry (type, offset) ((swp_entry_t) { (type) | \ +#define __swp_entry (type, offset) ((swp_entry_t) { \ + ((type) & 0x1f) | \ ((offset & 0x7) << 6) | \ ((offset & ~0x7) << 8) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +static inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {pte_t pte;
Signature · 1 line
2.39.0
PATCH 17/26 powerpc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit book3s
In reply to David HildenbrandWe already implemented support for 64bit book3s in commit bff9beaa2e80 ("powerpc/pgtable: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE for book3s")
Let's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE also in 32bit by reusing yet unused LSB 2 / MSB 29. There seems to be no real reason why that bit cannot be used, and reusing it avoids having to steal one bit from the swap offset.
While at it, mask the type in __swp_entry().
--- arch/powerpc/include/asm/book3s/32/pgtable.h | 38 +++++++++++++++++--- 1 file changed, 33 insertions(+), 5 deletions(-)
arch/powerpc/include/asm/book3s/32/pgtable.h +33 -5
@@ -42,6 +42,9 @@#define _PMD_PRESENT_MASK (PAGE_MASK)#define _PMD_BAD (~PAGE_MASK) +/* We borrow the _PAGE_USER bit to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_USER +/* And here we include common definitions */ #define _PAGE_KERNEL_RO 0
@@ -363,17 +366,42 @@ static inline void __ptep_set_access_flags(struct vm_area_struct *vma,#define pmd_page (pmd) pfn_to_page(pmd_pfn(pmd))/* -* Encode and decode a swap entry. -* Note that the bits we use in a PTE for representing a swap entry -* must not include the _PAGE_PRESENT bit or the _PAGE_HASHPTE bit (if used). -* -- paulus +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs (32bit PTEs): +* +* 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 +* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +* <----------------- offset --------------------> < type -> E H P +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P) and __PAGE_HASHPTE (H) must be 0. +* +* For 64bit PTEs, the offset is extended by 32bit. */ #define __swp_type (entry) ((entry).val & 0x1f)#define __swp_offset (entry) ((entry).val >> 5) -#define __swp_entry (type, offset) ((swp_entry_t) { (type) | ((offset) << 5) }) +#define __swp_entry (type, offset) ((swp_entry_t) { ((type) & 0x1f) | ((offset) << 5) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) >> 3 })#define __swp_entry_to_pte (x) ((pte_t) { (x).val << 3 }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) | _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~_PAGE_SWP_EXCLUSIVE); +} +/* Generic accessors to PTE bits */ static inline int pte_write (pte_t pte) {return !!(pte_val (pte) & _PAGE_RW);}static inline int pte_read (pte_t pte) {return 1 ; }
Signature · 1 line
2.39.0
Re: [PATCH mm-unstable v1 17/26] powerpc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit book3s
Quoted 10 lines · David Hildenbrand
We already implemented support for 64bit book3s in commit
bff9beaa2e80("powerpc/pgtable: support__HAVE_ARCH_PTE_SWP_EXCLUSIVEfor book3s")Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEalso in 32bit by reusing yet unused LSB 2 / MSB 29. There seems to be no real reason why that bit cannot be used, and reusing it avoids having to steal one bit from the swap offset.While at it, mask the type in
__swp_entry().
Quoted 7 lines
Cc Michael Ellerman <[email protected]>Cc Nicholas Piggin <[email protected]>Cc Christophe Leroy <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/powerpc/include/asm/book3s/32/pgtable.h | 38 +++++++++++++++++--- 1 file changed, 33 insertions(+), 5 deletions(-)
I gave this a quick test on a ppc32 machine, everything seems fine.
Your test_swp_exclusive.c passes, and an LTP run looks normal.
cheers
Quoted 70 lines
arch/powerpc/include/asm/book3s/32/pgtable.h +33
-5@@ -42,6 +42,9 @@#define _PMD_PRESENT_MASK (PAGE_MASK)#define _PMD_BAD (~PAGE_MASK) +/* We borrow the _PAGE_USER bit to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_USER +/* And here we include common definitions */ #define _PAGE_KERNEL_RO 0@@ -363,17 +366,42 @@ static inline void __ptep_set_access_flags(struct vm_area_struct *vma,#define pmd_page (pmd) pfn_to_page(pmd_pfn(pmd))/* -* Encode and decode a swap entry. -* Note that the bits we use in a PTE for representing a swap entry -* must not include the _PAGE_PRESENT bit or the _PAGE_HASHPTE bit (if used). -* -- paulus +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs (32bit PTEs): +* +* 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 +* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +* <----------------- offset --------------------> < type -> E H P +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_PRESENT (P) and __PAGE_HASHPTE (H) must be 0. +* +* For 64bit PTEs, the offset is extended by 32bit. */ #define __swp_type (entry) ((entry).val & 0x1f)#define __swp_offset (entry) ((entry).val >> 5) -#define __swp_entry (type, offset) ((swp_entry_t) { (type) | ((offset) << 5) }) +#define __swp_entry (type, offset) ((swp_entry_t) { ((type) & 0x1f) | ((offset) << 5) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) >> 3 })#define __swp_entry_to_pte (x) ((pte_t) { (x).val << 3 }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) | _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~_PAGE_SWP_EXCLUSIVE); +} +/* Generic accessors to PTE bits */ static inline int pte_write (pte_t pte) {return !!(pte_val (pte) & _PAGE_RW);}static inline int pte_read (pte_t pte) {return 1 ; }Signature · 7 lines
2.39.0 _______________________________________________ linux-riscv mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-riscv
PATCH 18/26 powerpc/nohash/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit and 64bit.
On 64bit, let's use MSB 56 (LSB 7), located right next to the page type. On 32bit, let's use LSB 2 to avoid stealing one bit from the swap offset.
There seems to be no real reason why these bits cannot be used for swap PTEs. The important part is that _PAGE_PRESENT and _PAGE_HASHPTE remain 0.
While at it, mask the type in __swp_entry() and remove _PAGE_BIT_SWAP_TYPE from pte-e500.h: while it was used in 64bit code it was ignored in 32bit code.
--- arch/powerpc/include/asm/nohash/32/pgtable.h | 22 +++++++++++++---- arch/powerpc/include/asm/nohash/32/pte-40x.h | 6 ++--- arch/powerpc/include/asm/nohash/32/pte-44x.h | 18 ++++---------- arch/powerpc/include/asm/nohash/32/pte-85xx.h | 4 ++-- arch/powerpc/include/asm/nohash/64/pgtable.h | 24 ++++++++++++++++--- arch/powerpc/include/asm/nohash/pgtable.h | 16 +++++++++++++ arch/powerpc/include/asm/nohash/pte-e500.h | 1 - 7 files changed, 63 insertions(+), 28 deletions(-)
arch/powerpc/include/asm/nohash/32/pgtable.h +17 -5
@@ -360,18 +360,30 @@ static inline int pte_young(pte_t pte)#endif #define pmd_page (pmd) pfn_to_page(pmd_pfn(pmd)) +/* -* Encode and decode a swap entry. -* Note that the bits we use in a PTE for representing a swap entry -* must not include the _PAGE_PRESENT bit. -* -- paulus +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs (32bit PTEs): +* +* 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 +* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +* <------------------ offset -------------------> < type -> E 0 0 +* +* E is the exclusive marker that is not stored in swap entries. +* +* For 64bit PTEs, the offset is extended by 32bit. */ #define __swp_type (entry) ((entry).val & 0x1f)#define __swp_offset (entry) ((entry).val >> 5) -#define __swp_entry (type, offset) ((swp_entry_t) { (type) | ((offset) << 5) }) +#define __swp_entry (type, offset) ((swp_entry_t) { ((type) & 0x1f) | ((offset) << 5) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) >> 3 })#define __swp_entry_to_pte (x) ((pte_t) { (x).val << 3 }) +/* We borrow LSB 2 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x000004 +#endif /* !__ASSEMBLY__ */ #endif /* __ASM_POWERPC_NOHASH_32_PGTABLE_H */
arch/powerpc/include/asm/nohash/32/pte-40x.h +3 -3
@@ -27,9 +27,9 @@* of the 16 available. Bit 24-26 of the TLB are cleared in the TLB * miss handler. Bit 27 is PAGE_USER, thus selecting the correct * zone. -* - PRESENT *must* be in the bottom two bits because swap cache -* entries use the top 30 bits. Because 40x doesn't support SMP -* anyway, M is irrelevant so we borrow it for PAGE_PRESENT. Bit 30 +* - PRESENT *must* be in the bottom two bits because swap PTEs +* use the top 30 bits. Because 40x doesn't support SMP anyway, M is +* irrelevant so we borrow it for PAGE_PRESENT. Bit 30 * is cleared in the TLB miss handler before the TLB entry is loaded. * - All other bits of the PTE are loaded into TLBLO without * modification, leaving us only the bits 20, 21, 24, 25, 26, 30 for
arch/powerpc/include/asm/nohash/32/pte-44x.h +4 -14
@@ -56,20 +56,10 @@* above bits. Note that the bit values are CPU specific, not architecture * specific. * -* The kernel PTE entry holds an arch-dependent swp_entry structure under -* certain situations. In other words, in such situations some portion of -* the PTE bits are used as a swp_entry. In the PPC implementation, the -* 3-24th LSB are shared with swp_entry, however the 0-2nd three LSB still -* hold protection values. That means the three protection bits are -* reserved for both PTE and SWAP entry at the most significant three -* LSBs. -* -* There are three protection bits available for SWAP entry: -* _PAGE_PRESENT -* _PAGE_HASHPTE (if HW has) -* -* So those three bits have to be inside of 0-2nd LSB of PTE. -* +* The kernel PTE entry can be an ordinary PTE mapping a page or a special swap +* PTE. In case of a swap PTE, LSB 2-24 are used to store information regarding +* the swap entry. However LSB 0-1 still hold protection values, for example, +* to distinguish swap PTEs from ordinary PTEs, and must be used with care. */ #define _PAGE_PRESENT 0x00000001/* S: PTE valid */
arch/powerpc/include/asm/nohash/32/pte-85xx.h +2 -2
@@ -11,8 +11,8 @@32 33 34 35 36 ... 50 51 52 53 54 55 56 57 58 59 60 61 62 63 RPN...................... 0 0 U0 U1 U2 U3 UX SX UW SW UR SR -- PRESENT *must* be in the bottom three bits because swap cache -entries use the top 29 bits. +- PRESENT *must* be in the bottom two bits because swap PTEs use +the top 30 bits. */
arch/powerpc/include/asm/nohash/64/pgtable.h +21 -3
@@ -276,22 +276,40 @@ static inline void __ptep_set_access_flags(struct vm_area_struct *vma,#define pgd_ERROR (e) \ pr_err("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, pgd_val(e)) -/* Encode and de-code a swap entry */ +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 +* 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +* <-------------------------- offset ---------------------------- +* +* 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 +* 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 +* --------------> <----------- zero ------------> E < type -> 0 0 +* +* E is the exclusive marker that is not stored in swap entries. +*/ #define MAX_SWAPFILES_CHECK () do { \ BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS); \ } while (0)#define SWP_TYPE_BITS 5 -#define __swp_type (x) (((x).val >> _PAGE_BIT_SWAP_TYPE) \ +#define __swp_type (x) (((x).val >> 2) \ & ((1UL << SWP_TYPE_BITS) - 1))#define __swp_offset (x) ((x).val >> PTE_RPN_SHIFT)#define __swp_entry (type, offset) ((swp_entry_t) { \ - ((type) << _PAGE_BIT_SWAP_TYPE) \ + (((type) & 0x1f) << 2) \ | ((offset) << PTE_RPN_SHIFT) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val((pte)) })#define __swp_entry_to_pte (x) __pte((x).val) +/* We borrow MSB 56 (LSB 7) to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x80 +int map_kernel_page (unsigned long ea,unsigned long pa,pgprot_t prot);void unmap_kernel_page (unsigned long va); extern int __meminit vmemmap_create_mapping(unsigned long start,
arch/powerpc/include/asm/nohash/pgtable.h +16 -0
@@ -151,6 +151,22 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)return __pte ((pte_val (pte) & _PAGE_CHG_MASK) |pgprot_val (newprot)); } +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) | _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~_PAGE_SWP_EXCLUSIVE); +} +/* Insert a PTE, top-level function is out of line. It uses an inline * low level function in the respective pgtable-* files */
arch/powerpc/include/asm/nohash/pte-e500.h +0 -1
@@ -12,7 +12,6 @@/* Architected bits */ #define _PAGE_PRESENT 0x000001/* software: pte contains a translation */ #define _PAGE_SW1 0x000002 -#define _PAGE_BIT_SWAP_TYPE 2#define _PAGE_BAP_SR 0x000004#define _PAGE_BAP_UR 0x000008#define _PAGE_BAP_SW 0x000010
Signature · 1 line
2.39.0
PATCH 19/26 riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the offset. This reduces the maximum swap space per file: on 32bit to 16 GiB (was 32 GiB).
Note that this bit does not conflict with swap PMDs and could also be used in swap PMD context later.
While at it, mask the type in __swp_entry().
--- arch/riscv/include/asm/pgtable-bits.h | 3 +++ arch/riscv/include/asm/pgtable.h | 29 ++++++++++++++++++++++----- 2 files changed, 27 insertions(+), 5 deletions(-)
arch/riscv/include/asm/pgtable-bits.h +3 -0
@@ -27,6 +27,9 @@*/ #define _PAGE_PROT_NONE _PAGE_GLOBAL +/* Used for swap PTEs only. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_ACCESSED +#define _PAGE_PFN_SHIFT 10/*
arch/riscv/include/asm/pgtable.h +24 -5
@@ -724,16 +724,18 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /* -* Encode and decode a swap entry +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). * * Format of swap PTE: * bit 0: _PAGE_PRESENT (zero) * bit 1 to 3: _PAGE_LEAF (zero) * bit 5: _PAGE_PROT_NONE (zero) -* bits 6 to 10: swap type -* bits 10 to XLEN-1: swap offset +* bit 6: exclusive marker +* bits 7 to 11: swap type +* bits 11 to XLEN-1: swap offset */ -#define __SWP_TYPE_SHIFT 6 +#define __SWP_TYPE_SHIFT 7#define __SWP_TYPE_BITS 5#define __SWP_TYPE_MASK ((1UL << __SWP_TYPE_BITS) - 1)#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT)
@@ -744,11 +746,28 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#define __swp_type (x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)#define __swp_offset (x) ((x).val >> __SWP_OFFSET_SHIFT)#define __swp_entry (type, offset) ((swp_entry_t) \ - { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) }) + { (((type) & __SWP_TYPE_MASK) << __SWP_TYPE_SHIFT) | \ + ((offset) << __SWP_OFFSET_SHIFT) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) | _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~_PAGE_SWP_EXCLUSIVE); +} +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION #define __pmd_to_swp_entry (pmd) ((swp_entry_t) { pmd_val(pmd) })#define __swp_entry_to_pmd (swp) __pmd((swp).val)
Signature · 1 line
2.39.0
Re: [PATCH mm-unstable v1 19/26] riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 3 lines · On Fri, 13 Jan 2023 09:10:19 PST (-0800), [email protected] wrote:
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the offset. This reduces the maximum swap space per file: on 32bit to 16 GiB (was 32 GiB).
Seems fine to me, I doubt anyone wants a huge pile of swap on rv32.
Quoted 76 lines
Note that this bit does not conflict with swap PMDs and could also be used in swap PMD context later.
While at it, mask the type in
__swp_entry().Cc Paul Walmsley <[email protected]>Cc Palmer Dabbelt <[email protected]>Cc Albert Ou <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- arch/riscv/include/asm/pgtable-bits.h | 3 +++ arch/riscv/include/asm/pgtable.h | 29 ++++++++++++++++++++++----- 2 files changed, 27 insertions(+), 5 deletions(-)arch/riscv/include/asm/pgtable-bits.h +3
-0@@ -27,6 +27,9 @@*/ #define _PAGE_PROT_NONE _PAGE_GLOBAL +/* Used for swap PTEs only. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_ACCESSED +#define _PAGE_PFN_SHIFT 10/* arch/riscv/include/asm/pgtable.h +17
-5@@ -724,16 +724,18 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ /* -* Encode and decode a swap entry +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). * * Format of swap PTE: * bit 0: _PAGE_PRESENT (zero) * bit 1 to 3: _PAGE_LEAF (zero) * bit 5: _PAGE_PROT_NONE (zero) -* bits 6 to 10: swap type -* bits 10 to XLEN-1: swap offset +* bit 6: exclusive marker +* bits 7 to 11: swap type +* bits 11 to XLEN-1: swap offset */ -#define __SWP_TYPE_SHIFT 6 +#define __SWP_TYPE_SHIFT 7#define __SWP_TYPE_BITS 5#define __SWP_TYPE_MASK ((1UL << __SWP_TYPE_BITS) - 1)#define __SWP_OFFSET_SHIFT (__SWP_TYPE_BITS + __SWP_TYPE_SHIFT)@@ -744,11 +746,28 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#define __swp_type (x) (((x).val >> __SWP_TYPE_SHIFT) & __SWP_TYPE_MASK)#define __swp_offset (x) ((x).val >> __SWP_OFFSET_SHIFT)#define __swp_entry (type, offset) ((swp_entry_t) \ - { ((type) << __SWP_TYPE_SHIFT) | ((offset) << __SWP_OFFSET_SHIFT) }) + { (((type) & __SWP_TYPE_MASK) << __SWP_TYPE_SHIFT) | \ + ((offset) << __SWP_OFFSET_SHIFT) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) | _PAGE_SWP_EXCLUSIVE);
arch/riscv/include/asm/pgtable.h static inline pmd_t pmdp_establish(struct vm_area_struct *vma, · line 773
Quoted 10 lines
arch/riscv/include/asm/pgtable.h +7
-0+} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~_PAGE_SWP_EXCLUSIVE); +} +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION #define __pmd_to_swp_entry (pmd) ((swp_entry_t) { pmd_val(pmd) })#define __swp_entry_to_pmd (swp) __pmd((swp).val)
Re: [PATCH mm-unstable v1 19/26] riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
Quoted 6 lines · Palmer Dabbelt
Quoted 3 lines · On Fri, 13 Jan 2023 09:10:19 PST (-0800), [email protected] wrote:
Let's support
__HAVE_ARCH_PTE_SWP_EXCLUSIVEby stealing one bit from the offset. This reduces the maximum swap space per file: on 32bit to 16 GiB (was 32 GiB).Seems fine to me, I doubt anyone wants a huge pile of swap on rv32.
Patch is already upstream, so we can't add tags unfortunately. Thanks for the review!
Signature · 3 lines
Thanks, David / dhildenb
PATCH 20/26 sh/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by using bit 6 in the PTE, reducing the swap type in the !CONFIG_X2TLB case to 5 bits. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit is effectively unused.
Interrestingly, the swap type in the !CONFIG_X2TLB case could currently overlap with the _PAGE_PRESENT bit, because there is a sneaky shift by 1 in __pte_to_swp_entry() and __swp_entry_to_pte(). Bit 0-7 in the architecture specific swap PTE would get shifted to bit 1-8 in the PTE. As generic MM uses 5 bits only, this didn't matter so far.
While at it, mask the type in __swp_entry().
--- arch/sh/include/asm/pgtable_32.h | 54 +++++++++++++++++++++++++------- 1 file changed, 42 insertions(+), 12 deletions(-)
arch/sh/include/asm/pgtable_32.h +42 -12
@@ -423,40 +423,70 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)#endif /* -* Encode and de-code a swap entry +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). * * Constraints: * _PAGE_PRESENT at bit 8 * _PAGE_PROTNONE at bit 9 * -* For the normal case, we encode the swap type into bits 0:7 and the -* swap offset into bits 10:30. For the 64-bit PTE case, we keep the -* preserved bits in the low 32-bits and use the upper 32 as the swap -* offset (along with a 5-bit type), following the same approach as x86 -* PAE. This keeps the logic quite simple. +* For the normal case, we encode the swap type and offset into the swap PTE +* such that bits 8 and 9 stay zero. For the 64-bit PTE case, we use the +* upper 32 for the swap offset and swap type, following the same approach as +* x86 PAE. This keeps the logic quite simple. * * As is evident by the Alpha code, if we ever get a 64-bit unsigned * long (swp_entry_t) to match up with the 64-bit PTEs, this all becomes * much cleaner.. -* -* NOTE: We should set ZEROs at the position of _PAGE_PRESENT -* and _PAGE_PROTNONE bits */ +#ifdef CONFIG_X2TLB +/* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* <--------------------- offset ----------------------> < type -> +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <------------------- zeroes --------------------> E 0 0 0 0 0 0 +*/ #define __swp_type (x) ((x).val & 0x1f)#define __swp_offset (x) ((x).val >> 5) -#define __swp_entry (type, offset) ((swp_entry_t){ (type) | (offset) << 5}) +#define __swp_entry (type, offset) ((swp_entry_t){ ((type) & 0x1f) | (offset) << 5})#define __pte_to_swp_entry (pte) ((swp_entry_t){ (pte).pte_high })#define __swp_entry_to_pte (x) ((pte_t){ 0, (x).val })#else -#define __swp_type (x) ((x).val & 0xff) +/* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <--------------- offset ----------------> 0 0 0 0 E < type -> 0 +* +* E is the exclusive marker that is not stored in swap entries. +*/ +#define __swp_type (x) ((x).val & 0x1f)#define __swp_offset (x) ((x).val >> 10) -#define __swp_entry (type, offset) ((swp_entry_t){(type) | (offset) <<10}) +#define __swp_entry (type, offset) ((swp_entry_t){((type) & 0x1f) | (offset) << 10})#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) >> 1 })#define __swp_entry_to_pte (x) ((pte_t) { (x).val << 1 })#endif +/* In both cases, we borrow bit 6 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_USER + +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte.pte_low & _PAGE_SWP_EXCLUSIVE; +} + +PTE_BIT_FUNC(low, swp_mkexclusive, |= _PAGE_SWP_EXCLUSIVE); +PTE_BIT_FUNC(low, swp_clear_exclusive, &= ~_PAGE_SWP_EXCLUSIVE); +#endif /* __ASSEMBLY__ */ #endif /* __ASM_SH_PGTABLE_32_H */
Signature · 1 line
2.39.0
PATCH 21/26 sparc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by reusing the SRMMU_DIRTY bit as that seems to be safe to reuse inside a swap PTE. This avoids having to steal one bit from the swap offset.
While at it, relocate the swap PTE layout documentation and use the same style now used for most other archs. Note that the old documentation was
wrong: we use 20 bit for the offset and the reserved bits were 8 instead of 7 bits in the ascii art.
--- arch/sparc/include/asm/pgtable_32.h | 27 ++++++++++++++++++++++++++- arch/sparc/include/asm/pgtsrmmu.h | 14 +++----------- 2 files changed, 29 insertions(+), 12 deletions(-)
arch/sparc/include/asm/pgtable_32.h +26 -1
@@ -323,7 +323,16 @@ void srmmu_mapiorange(unsigned int bus, unsigned long xpa, unsigned long xva, unsigned int len);void srmmu_unmapiorange (unsigned long virt_addr,unsigned int len); -/* Encode and de-code a swap entry */ +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <-------------- offset ---------------> < type -> E 0 0 0 0 0 0 +*/ static inline unsigned long __swp_type (swp_entry_t entry) {return (entry.val >>SRMMU_SWP_TYPE_SHIFT ) &SRMMU_SWP_TYPE_MASK ;
@@ -344,6 +353,22 @@ static inline swp_entry_t __swp_entry(unsigned long type, unsigned long offset)#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) &SRMMU_SWP_EXCLUSIVE ; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) |SRMMU_SWP_EXCLUSIVE ); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~SRMMU_SWP_EXCLUSIVE ); +} +static inline unsigned long __get_phys (unsigned long addr) {
arch/sparc/include/asm/pgtsrmmu.h +3 -11
@@ -53,21 +53,13 @@#define SRMMU_CHG_MASK (0xffffff00 | SRMMU_REF | SRMMU_DIRTY) -/* SRMMU swap entry encoding -* -* We use 5 bits for the type and 19 for the offset. This gives us -* 32 swapfiles of 4GB each. Encoding looks like: -* -* oooooooooooooooooootttttRRRRRRRR -* fedcba9876543210fedcba9876543210 -* -* The bottom 7 bits are reserved for protection and status bits, especially -* PRESENT. -*/ +/* SRMMU swap entry encoding */ #define SRMMU_SWP_TYPE_MASK 0x1f#define SRMMU_SWP_TYPE_SHIFT 7#define SRMMU_SWP_OFF_MASK 0xfffff#define SRMMU_SWP_OFF_SHIFT (SRMMU_SWP_TYPE_SHIFT + 5) +/* We borrow bit 6 to store the exclusive marker in swap PTEs. */ +#define SRMMU_SWP_EXCLUSIVE SRMMU_DIRTY/* Some day I will implement true fine grained access bits for * user pages because the SRMMU gives us the capabilities to
Signature · 1 line
2.39.0
PATCH 22/26 sparc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 64bit
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by stealing one bit from the type. Generic MM currently only uses 5 bits for the type (MAX_SWAPFILES_SHIFT), so the stolen bit was effectively unused.
While at it, mask the type in __swp_entry().
--- arch/sparc/include/asm/pgtable_64.h | 38 ++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-)
arch/sparc/include/asm/pgtable_64.h +35 -3
@@ -187,6 +187,9 @@ bool kern_addr_valid(unsigned long addr);#define _PAGE_SZHUGE_4U _PAGE_SZ4MB_4U#define _PAGE_SZHUGE_4V _PAGE_SZ4MB_4V +/* We borrow bit 20 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _AC(0x0000000000100000, UL) +#ifndef __ASSEMBLY__pte_t mk_pte_io (unsigned long ,pgprot_t ,int ,unsigned long );
@@ -961,18 +964,47 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);#endif -/* Encode and de-code a swap entry */ -#define __swp_type (entry) (((entry).val >> PAGE_SHIFT) & 0xffUL) +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* <--------------------------- offset --------------------------- +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* --------------------> E <-- type ---> <------- zeroes --------> +*/ +#define __swp_type (entry) (((entry).val >> PAGE_SHIFT) & 0x7fUL)#define __swp_offset (entry) ((entry).val >> (PAGE_SHIFT + 8UL))#define __swp_entry (type, offset) \ ( (swp_entry_t) \ { \ - (((long)(type) << PAGE_SHIFT) | \ + ((((long)(type) & 0x7fUL) << PAGE_SHIFT) | \ ((long)(offset) << (PAGE_SHIFT + 8UL))) \ } )#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +return __pte (pte_val (pte) | _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +return __pte (pte_val (pte) & ~_PAGE_SWP_EXCLUSIVE); +} +int page_in_phys_avail (unsigned long paddr);/*
Signature · 1 line
2.39.0
PATCH 23/26 um/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by using bit 10, which is yet unused for swap PTEs.
The pte_mkuptodate() is a bit weird in __pte_to_swp_entry() for a swap PTE ... but it only messes with bit 1 and 2 and there is a comment in set_pte(), so leave these bits alone.
While at it, mask the type in __swp_entry().
--- arch/um/include/asm/pgtable.h | 37 +++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-)
arch/um/include/asm/pgtable.h +35 -2
@@ -21,6 +21,9 @@#define _PAGE_PROTNONE 0x010/* if the user mapped it with PROT_NONE; pte_present gives true */ +/* We borrow bit 10 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE 0x400 +#ifdef CONFIG_3_LEVEL_PGTABLES #include <asm/pgtable-3level.h> #else
@@ -288,16 +291,46 @@ extern pte_t *virt_to_pte(struct mm_struct *mm, unsigned long addr);#define update_mmu_cache (vma,address,ptep) do {} while (0) -/* Encode and de-code a swap entry */ +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <--------------- offset ----------------> E < type -> 0 0 0 1 0 +* +* E is the exclusive marker that is not stored in swap entries. +* _PAGE_NEWPAGE (bit 1) is always set to 1 in set_pte(). +*/ #define __swp_type (x) (((x).val >> 5) & 0x1f)#define __swp_offset (x) ((x).val >> 11)#define __swp_entry (type, offset) \ - ((swp_entry_t) { ((type) << 5) | ((offset) << 11) }) + ((swp_entry_t) { (((type) & 0x1f) << 5) | ((offset) << 11) })#define __pte_to_swp_entry (pte) \ ((swp_entry_t) { pte_val(pte_mkuptodate(pte)) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_get_bits (pte, _PAGE_SWP_EXCLUSIVE); +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_set_bits (pte, _PAGE_SWP_EXCLUSIVE); +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_clear_bits (pte, _PAGE_SWP_EXCLUSIVE); +return pte; +} +/* Clear a kernel PTE and flush it from the TLB */ #define kpte_clear_flush (ptep, vaddr) \ do { \
Signature · 1 line
2.39.0
PATCH 24/26 x86/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE also on 32bit
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE just like we already do on x86-64. After deciphering the PTE layout it becomes clear that there are still unused bits for 2-level and 3-level page tables that we should be able to use. Reusing a bit avoids stealing one bit from the swap offset.
While at it, mask the type in __swp_entry(); use some helper definitions to make the macros easier to grasp.
--- arch/x86/include/asm/pgtable-2level.h | 26 +++++++++++++++++++++----- arch/x86/include/asm/pgtable-3level.h | 26 +++++++++++++++++++++++--- arch/x86/include/asm/pgtable.h | 2 -- 3 files changed, 44 insertions(+), 10 deletions(-)
arch/x86/include/asm/pgtable-2level.h +21 -5
@@ -80,21 +80,37 @@ static inline unsigned long pte_bitop(unsigned long value, unsigned int rightshireturn ((value >> rightshift) & mask) << leftshift; } -/* Encode and de-code a swap entry */ +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* <----------------- offset ------------------> 0 E <- type --> 0 +* +* E is the exclusive marker that is not stored in swap entries. +*/ #define SWP_TYPE_BITS 5 +#define _SWP_TYPE_MASK ((1U << SWP_TYPE_BITS) - 1) +#define _SWP_TYPE_SHIFT (_PAGE_BIT_PRESENT + 1)#define SWP_OFFSET_SHIFT (_PAGE_BIT_PROTNONE + 1) -#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS) +#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > 5) -#define __swp_type (x) (((x).val >> (_PAGE_BIT_PRESENT + 1)) \ - & ((1U << SWP_TYPE_BITS) - 1)) +#define __swp_type (x) (((x).val >> _SWP_TYPE_SHIFT) \ + & _SWP_TYPE_MASK)#define __swp_offset (x) ((x).val >> SWP_OFFSET_SHIFT)#define __swp_entry (type, offset) ((swp_entry_t) { \ - ((type) << (_PAGE_BIT_PRESENT + 1)) \ + (((type) & _SWP_TYPE_MASK) << _SWP_TYPE_SHIFT) \ | ((offset) << SWP_OFFSET_SHIFT) })#define __pte_to_swp_entry (pte) ((swp_entry_t) { (pte).pte_low })#define __swp_entry_to_pte (x) ((pte_t) { .pte = (x).val }) +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_PSE +/* No inverted PFNs on 2 level page tables */ static inline u64 protnone_mask (u64 val)
arch/x86/include/asm/pgtable-3level.h +23 -3
@@ -145,8 +145,24 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, }#endif -/* Encode and de-code a swap entry */ +/* +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). +* +* Format of swap PTEs: +* +* 6 6 6 6 5 5 5 5 5 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 +* 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 +* < type -> <---------------------- offset ---------------------- +* +* 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 +* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 +* --------------------------------------------> 0 E 0 0 0 0 0 0 0 +* +* E is the exclusive marker that is not stored in swap entries. +*/ #define SWP_TYPE_BITS 5 +#define _SWP_TYPE_MASK ((1U << SWP_TYPE_BITS) - 1)#define SWP_OFFSET_FIRST_BIT (_PAGE_BIT_PROTNONE + 1)
@@ -154,9 +170,10 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#define SWP_OFFSET_SHIFT (SWP_OFFSET_FIRST_BIT + SWP_TYPE_BITS)#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS) -#define __swp_type (x) (((x).val) & ((1UL << SWP_TYPE_BITS) - 1)) +#define __swp_type (x) (((x).val) & _SWP_TYPE_MASK)#define __swp_offset (x) ((x).val >> SWP_TYPE_BITS) -#define __swp_entry (type, offset) ((swp_entry_t){(type) | (offset) << SWP_TYPE_BITS}) +#define __swp_entry (type, offset) ((swp_entry_t){((type) & _SWP_TYPE_MASK) \ + | (offset) << SWP_TYPE_BITS})/* * Normally, __swp_entry() converts from arch-independent swp_entry_t to
@@ -184,6 +201,9 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#define __pte_to_swp_entry (pte) (__swp_entry(__pteval_swp_type(pte), \ __pteval_swp_offset(pte))) +/* We borrow bit 7 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE _PAGE_PSE +#include <asm/pgtable-invert.h> #endif /* _ASM_X86_PGTABLE_3LEVEL_H */
arch/x86/include/asm/pgtable.h +0 -2
@@ -1299,7 +1299,6 @@ static inline void update_mmu_cache_pud(struct vm_area_struct *vma, unsigned long addr, pud_t *pud) { } -#ifdef _PAGE_SWP_EXCLUSIVE#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline pte_t pte_swp_mkexclusive (pte_t pte) {
@@ -1315,7 +1314,6 @@ static inline pte_t pte_swp_clear_exclusive(pte_t pte) {return pte_clear_flags (pte, _PAGE_SWP_EXCLUSIVE); } -#endif /* _PAGE_SWP_EXCLUSIVE */ #ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY static inline pte_t pte_swp_mksoft_dirty (pte_t pte)
Signature · 1 line
2.39.0
PATCH 25/26 xtensa/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David HildenbrandLet's support __HAVE_ARCH_PTE_SWP_EXCLUSIVE by using bit 1. This bit should be safe to use for our usecase.
Most importantly, we can still distinguish swap PTEs from PAGE_NONE PTEs (see pte_present()) and don't use one of the two reserved attribute masks (1101 and 1111). Attribute mask 1100 and 1110 now identify swap PTEs.
While at it, remove SWP_TYPE_BITS (not really helpful as it's not used in the actual swap macros) and mask the type in __swp_entry().
--- arch/xtensa/include/asm/pgtable.h | 32 ++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-)
arch/xtensa/include/asm/pgtable.h +27 -5
@@ -96,7 +96,7 @@* +- - - - - - - - - - - - - - - - - - - - -+ * (PAGE_NONE)| PPN | 0 | 00 | ADW | 01 | 11 | 11 | * +-----------------------------------------+ -* swap | index | type | 01 | 11 | 00 | +* swap | index | type | 01 | 11 | e0 | * +-----------------------------------------+ * * For T1050 hardware and earlier the layout differs for present and (PAGE_NONE)
@@ -112,6 +112,7 @@* RI ring (0=privileged, 1=user, 2 and 3 are unused) * CA cache attribute: 00 bypass, 01 writeback, 10 writethrough * (11 is invalid and used to mark pages that are not present) +* e exclusive marker in swap PTEs * w page is writable (hw) * x page is executable (hw) * index swap offset / PAGE_SIZE (bit 11-31: 21 bits -> 8 GB)
@@ -158,6 +159,9 @@#define _PAGE_DIRTY (1<<7)/* software: page dirty */ #define _PAGE_ACCESSED (1<<8)/* software: page accessed (read) */ +/* We borrow bit 1 to store the exclusive marker in swap PTEs. */ +#define _PAGE_SWP_EXCLUSIVE (1<<1) +#ifdef CONFIG_MMU #define _PAGE_CHG_MASK (PAGE_MASK | _PAGE_ACCESSED | _PAGE_DIRTY)
@@ -343,19 +347,37 @@ ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) }/* -* Encode and decode a swap and file entry. +* Encode/decode swap entries and swap PTEs. Swap PTEs are all PTEs that +* are !pte_none() && !pte_present(). */ -#define SWP_TYPE_BITS 5 -#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS) +#define MAX_SWAPFILES_CHECK () BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > 5)#define __swp_type (entry) (((entry).val >> 6) & 0x1f)#define __swp_offset (entry) ((entry).val >> 11)#define __swp_entry (type,offs) \ - ((swp_entry_t){((type) << 6) | ((offs) << 11) | \ + ((swp_entry_t){(((type) & 0x1f) << 6) | ((offs) << 11) | \ _PAGE_CA_INVALID | _PAGE_USER})#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) +#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE +static inline int pte_swp_exclusive (pte_t pte) +{ +return pte_val (pte) & _PAGE_SWP_EXCLUSIVE; +} + +static inline pte_t pte_swp_mkexclusive (pte_t pte) +{ +pte_val (pte) |= _PAGE_SWP_EXCLUSIVE; +return pte; +} + +static inline pte_t pte_swp_clear_exclusive (pte_t pte) +{ +pte_val (pte) &= ~_PAGE_SWP_EXCLUSIVE; +return pte; +} +#endif /* !defined (__ASSEMBLY__) */
Signature · 1 line
2.39.0
PATCH 26/26 mm: remove __HAVE_ARCH_PTE_SWP_EXCLUSIVE
In reply to David Hildenbrand__HAVE_ARCH_PTE_SWP_EXCLUSIVE is now supported by all architectures that support swp PTEs, so let's drop it.
--- arch/alpha/include/asm/pgtable.h | 1 - arch/arc/include/asm/pgtable-bits-arcv2.h | 1 - arch/arm/include/asm/pgtable.h | 1 - arch/arm64/include/asm/pgtable.h | 1 - arch/csky/include/asm/pgtable.h | 1 - arch/hexagon/include/asm/pgtable.h | 1 - arch/ia64/include/asm/pgtable.h | 1 - arch/loongarch/include/asm/pgtable.h | 1 - arch/m68k/include/asm/mcf_pgtable.h | 1 - arch/m68k/include/asm/motorola_pgtable.h | 1 - arch/m68k/include/asm/sun3_pgtable.h | 1 - arch/microblaze/include/asm/pgtable.h | 1 - arch/mips/include/asm/pgtable.h | 1 - arch/nios2/include/asm/pgtable.h | 1 - arch/openrisc/include/asm/pgtable.h | 1 - arch/parisc/include/asm/pgtable.h | 1 - arch/powerpc/include/asm/book3s/32/pgtable.h | 1 - arch/powerpc/include/asm/book3s/64/pgtable.h | 1 - arch/powerpc/include/asm/nohash/pgtable.h | 1 - arch/riscv/include/asm/pgtable.h | 1 - arch/s390/include/asm/pgtable.h | 1 - arch/sh/include/asm/pgtable_32.h | 1 - arch/sparc/include/asm/pgtable_32.h | 1 - arch/sparc/include/asm/pgtable_64.h | 1 - arch/um/include/asm/pgtable.h | 1 - arch/x86/include/asm/pgtable.h | 1 - arch/xtensa/include/asm/pgtable.h | 1 - include/linux/pgtable.h | 29 -------------------- mm/debug_vm_pgtable.c | 2 -- mm/memory.c | 4 --- mm/rmap.c | 11 -------- 31 files changed, 73 deletions(-)
arch/alpha/include/asm/pgtable.h +0 -1
@@ -328,7 +328,6 @@ extern inline pte_t mk_swap_pte(unsigned long type, unsigned long offset)#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/arc/include/asm/pgtable-bits-arcv2.h +0 -1
@@ -132,7 +132,6 @@ void update_mmu_cache(struct vm_area_struct *vma, unsigned long address,#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/arm/include/asm/pgtable.h +0 -1
@@ -298,7 +298,6 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (swp) __pte((swp).val) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_isset (pte,L_PTE_SWP_EXCLUSIVE );
arch/arm64/include/asm/pgtable.h +0 -1
@@ -417,7 +417,6 @@ static inline pgprot_t mk_pmd_sect_prot(pgprot_t prot)return __pgprot ((pgprot_val (prot) & ~PMD_TABLE_BIT ) |PMD_TYPE_SECT ); } -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline pte_t pte_swp_mkexclusive (pte_t pte) {return set_pte_bit (pte,__pgprot (PTE_SWP_EXCLUSIVE ));
arch/csky/include/asm/pgtable.h +0 -1
@@ -200,7 +200,6 @@ static inline pte_t pte_mkyoung(pte_t pte)return pte; } -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/hexagon/include/asm/pgtable.h +0 -1
@@ -397,7 +397,6 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) (((type & 0x1f) << 1) | \ ((offset & 0x3ffff8) << 10) | ((offset & 0x7) << 7)) }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/ia64/include/asm/pgtable.h +0 -1
@@ -424,7 +424,6 @@ extern void paging_init (void);#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/loongarch/include/asm/pgtable.h +0 -1
@@ -276,7 +276,6 @@ static inline pte_t mk_swap_pte(unsigned long type, unsigned long offset)#define __pmd_to_swp_entry (pmd) ((swp_entry_t) { pmd_val(pmd) })#define __swp_entry_to_pmd (x) ((pmd_t) { (x).val | _PAGE_HUGE }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/m68k/include/asm/mcf_pgtable.h +0 -1
@@ -275,7 +275,6 @@ extern pgd_t kernel_pg_dir[PTRS_PER_PGD];#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) (__pte((x).val)) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/m68k/include/asm/motorola_pgtable.h +0 -1
@@ -190,7 +190,6 @@ extern pgd_t kernel_pg_dir[128];#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/m68k/include/asm/sun3_pgtable.h +0 -1
@@ -174,7 +174,6 @@ extern pgd_t kernel_pg_dir[PTRS_PER_PGD];#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/microblaze/include/asm/pgtable.h +0 -1
@@ -412,7 +412,6 @@ extern pgd_t swapper_pg_dir[PTRS_PER_PGD];#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) >> 2 })#define __swp_entry_to_pte (x) ((pte_t) { (x).val << 2 }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/mips/include/asm/pgtable.h +0 -1
@@ -528,7 +528,6 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) }#endif -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVE#if defined(CONFIG_PHYS_ADDR_T_64BIT ) && defined(CONFIG_CPU_MIPS32 )static inline int pte_swp_exclusive (pte_t pte) {
arch/nios2/include/asm/pgtable.h +0 -1
@@ -253,7 +253,6 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)#define __swp_entry_to_pte (swp) ((pte_t) { (swp).val })#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/openrisc/include/asm/pgtable.h +0 -1
@@ -408,7 +408,6 @@ static inline void update_mmu_cache(struct vm_area_struct *vma,#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/parisc/include/asm/pgtable.h +0 -1
@@ -422,7 +422,6 @@ extern void paging_init (void);#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/powerpc/include/asm/book3s/32/pgtable.h +0 -1
@@ -386,7 +386,6 @@ static inline void __ptep_set_access_flags(struct vm_area_struct *vma,#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) >> 3 })#define __swp_entry_to_pte (x) ((pte_t) { (x).val << 3 }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/powerpc/include/asm/book3s/64/pgtable.h +0 -1
@@ -717,7 +717,6 @@ static inline pte_t pte_swp_clear_soft_dirty(pte_t pte) }#endif /* CONFIG_HAVE_ARCH_SOFT_DIRTY */ -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline pte_t pte_swp_mkexclusive (pte_t pte) {return __pte_raw (pte_raw (pte) |cpu_to_be64 (_PAGE_SWP_EXCLUSIVE));
arch/powerpc/include/asm/nohash/pgtable.h +0 -1
@@ -151,7 +151,6 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)return __pte ((pte_val (pte) & _PAGE_CHG_MASK) |pgprot_val (newprot)); } -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/riscv/include/asm/pgtable.h +0 -1
@@ -752,7 +752,6 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma,#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/s390/include/asm/pgtable.h +0 -1
@@ -812,7 +812,6 @@ static inline int pmd_protnone(pmd_t pmd) }#endif -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/sh/include/asm/pgtable_32.h +0 -1
@@ -479,7 +479,6 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)/* In both cases, we borrow bit 6 to store the exclusive marker in swap PTEs. */ #define _PAGE_SWP_EXCLUSIVE _PAGE_USER -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte.pte_low & _PAGE_SWP_EXCLUSIVE;
arch/sparc/include/asm/pgtable_32.h +0 -1
@@ -353,7 +353,6 @@ static inline swp_entry_t __swp_entry(unsigned long type, unsigned long offset)#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) &SRMMU_SWP_EXCLUSIVE ;
arch/sparc/include/asm/pgtable_64.h +0 -1
@@ -989,7 +989,6 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
arch/um/include/asm/pgtable.h +0 -1
@@ -313,7 +313,6 @@ extern pte_t *virt_to_pte(struct mm_struct *mm, unsigned long addr); ((swp_entry_t) { pte_val(pte_mkuptodate(pte)) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_get_bits (pte, _PAGE_SWP_EXCLUSIVE);
arch/x86/include/asm/pgtable.h +0 -1
@@ -1299,7 +1299,6 @@ static inline void update_mmu_cache_pud(struct vm_area_struct *vma, unsigned long addr, pud_t *pud) { } -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline pte_t pte_swp_mkexclusive (pte_t pte) {return pte_set_flags (pte, _PAGE_SWP_EXCLUSIVE);
arch/xtensa/include/asm/pgtable.h +0 -1
@@ -360,7 +360,6 @@ ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep)#define __pte_to_swp_entry (pte) ((swp_entry_t) { pte_val(pte) })#define __swp_entry_to_pte (x) ((pte_t) { (x).val }) -#define __HAVE_ARCH_PTE_SWP_EXCLUSIVEstatic inline int pte_swp_exclusive (pte_t pte) {return pte_val (pte) & _PAGE_SWP_EXCLUSIVE;
include/linux/pgtable.h +0 -29
@@ -1064,35 +1064,6 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)#define arch_start_context_switch (prev) do {} while (0)#endif -/* -* When replacing an anonymous page by a real (!non) swap entry, we clear -* PG_anon_exclusive from the page and instead remember whether the flag was -* set in the swp pte. During fork(), we have to mark the entry as !exclusive -* (possibly shared). On swapin, we use that information to restore -* PG_anon_exclusive, which is very helpful in cases where we might have -* additional (e.g., FOLL_GET) references on a page and wouldn't be able to -* detect exclusivity. -* -* These functions don't apply to non-swap entries (e.g., migration, hwpoison, -* ...). -*/ -#ifndef __HAVE_ARCH_PTE_SWP_EXCLUSIVE -static inline pte_t pte_swp_mkexclusive (pte_t pte) -{ -return pte; -} - -static inline int pte_swp_exclusive (pte_t pte) -{ -return false; -} - -static inline pte_t pte_swp_clear_exclusive (pte_t pte) -{ -return pte; -} -#endif -#ifdef CONFIG_HAVE_ARCH_SOFT_DIRTY #ifndef CONFIG_ARCH_ENABLE_THP_MIGRATION static inline pmd_t pmd_swp_mksoft_dirty (pmd_t pmd)
mm/debug_vm_pgtable.c +0 -2
@@ -810,7 +810,6 @@ static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *args) {static void __init pte_swap_exclusive_tests (struct pgtable_debug_args *args) { -#ifdef __HAVE_ARCH_PTE_SWP_EXCLUSIVEunsigned long max_swapfile_size =generic_max_swapfile_size ();swp_entry_t entry, entry2;pte_t pte;
@@ -839,7 +838,6 @@ static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *args)WARN_ON (!is_swap_pte (pte)); entry2 =pte_to_swp_entry (pte);WARN_ON (memcmp (&entry, &entry2,sizeof (entry))); -#endif /* __HAVE_ARCH_PTE_SWP_EXCLUSIVE */ }static void __init pte_swap_tests (struct pgtable_debug_args *args)
mm/memory.c +0 -4
@@ -3870,10 +3870,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)* the swap entry concurrently) for certainly exclusive pages. */ if (!folio_test_ksm (folio)) { -/* -* Note that pte_swp_exclusive() == false for architectures -* without __HAVE_ARCH_PTE_SWP_EXCLUSIVE. -*/ exclusive =pte_swp_exclusive (vmf->orig_pte);if (folio != swapcache) {/*
mm/rmap.c +0 -11
@@ -1710,17 +1710,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,page_vma_mapped_walk_done (&pvmw);break ; } -/* -* Note: We *don't* remember if the page was mapped -* exclusively in the swap pte if the architecture -* doesn't support __HAVE_ARCH_PTE_SWP_EXCLUSIVE. In -* that case, swapin code has to re-determine that -* manually and might detect the page as possibly -* shared, for example, if there are other references on -* the page or if the page is under writeback. We made -* sure that there are no GUP pins on the page that -* would rely on it, so for GUP pins this is fine. -*/ if (list_empty (&mm->mmlist)) {spin_lock (&mmlist_lock);if (list_empty (&mm->mmlist))
Signature · 1 line
2.39.0
This series was applied to riscv/linux.git (for-next) by Andrew Morton <[email protected]>:
Hello:
This series was applied to riscv/linux.git (for-next) by Andrew Morton <[email protected]>:
Quoted 9 lines · On Fri, 13 Jan 2023 18:10:00 +0100 you wrote:
This is the follow-up on [1]:
[PATCH v2 0/8] mm: COW fixes part 3: reliable GUP R/WFOLL_GETof anonymous pagesAfter we implemented
__HAVE_ARCH_PTE_SWP_EXCLUSIVEon most prominent enterprise architectures, implement__HAVE_ARCH_PTE_SWP_EXCLUSIVEon all remaining architectures that support swap PTEs.[...]
Here is the summary with links:
- [mm-unstable,v1,01/26] mm/debug_vm_pgtable: more pte_swp_exclusive() sanity checks (no matching commit)
- [mm-unstable,v1,02/26] alpha/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,03/26] arc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
- [mm-unstable,v1,04/26] arm/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
- [mm-unstable,v1,05/26] csky/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
- [mm-unstable,v1,06/26] hexagon/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,07/26] ia64/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
- [mm-unstable,v1,08/26] loongarch/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,09/26] m68k/mm: remove dummy __swp definitions for nommu (no matching commit)
- [mm-unstable,v1,10/26] m68k/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
- [mm-unstable,v1,11/26] microblaze/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,12/26] mips/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
- [mm-unstable,v1,13/26] nios2/mm: refactor swap PTE layout
(no matching commit)
- [mm-unstable,v1,14/26] nios2/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,15/26] openrisc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,16/26] parisc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,17/26] powerpc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit book3s (no matching commit)
- [mm-unstable,v1,18/26] powerpc/nohash/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,19/26] riscv/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE
https://git.kernel.org/riscv/c/51a1007d4113
- [mm-unstable,v1,20/26] sh/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,21/26] sparc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 32bit (no matching commit)
- [mm-unstable,v1,22/26] sparc/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE on 64bit (no matching commit)
- [mm-unstable,v1,23/26] um/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,24/26] x86/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE also on 32bit (no matching commit)
- [mm-unstable,v1,25/26] xtensa/mm: support __HAVE_ARCH_PTE_SWP_EXCLUSIVE (no matching commit)
- [mm-unstable,v1,26/26] mm: remove __HAVE_ARCH_PTE_SWP_EXCLUSIVE
(no matching commit)
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html