Fix swap and page migration for LPAE

PATCH New Will Deacon 2 participants 5 messages Jul 26, 2012 Arm (32-bit)

lore

Share & tools

lore

Will Deacon

author

2012-07-26 16:01 UTC #0 lore raw

cover Fix swap and page migration for LPAE

Hello,

These two patches fix swapping and page migration for LPAE-enabled kernels. Without these changes, we corrupt the offset stored in the file pte when updating the page tables and may truncate the pfn when storing it into a swp_entry_t.

I'll CC stable on the final patches.

All feedback welcome,

Will

Will Deacon (2):
ARM: mm: avoid setting nG bit for user mappings that aren't present
ARM: mm: use 5 bits for swapfile type encoding

 arch/arm/include/asm/pgtable.h |   40 +++++++++++++++++++++-------------------
 arch/arm/mm/flush.c            |    2 --
 2 files changed, 21 insertions(+), 21 deletions(-)

Signature · 1 line

1.7.4.1

Will Deacon

author

2012-07-26 16:01 UTC #1 lore raw

PATCH 1/2 ARM: mm: avoid setting nG bit for user mappings that aren't present

Swap entries are encoding in ptes such that !pte_present(pte) and pte_file(pte). The remaining bits of the descriptor are used to identify the swapfile and offset within it to the swap entry.

When writing such a pte for a user virtual address, set_pte_at unconditionally sets the nG bit, which (in the case of LPAE) will corrupt the swapfile offset and lead to a BUG:

[  140.494067] swap_free: Unused swap offset entry 000763b4
[  140.509989] BUG: Bad page map in process rs:main Q:Reg  pte:0ec76800 pmd:8f92e003

This patch fixes the problem by only setting the nG bit for user mappings that are actually present.

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm/include/asm/pgtable.h |   34 ++++++++++++++++++----------------
 arch/arm/mm/flush.c            |    2 --
 2 files changed, 18 insertions(+), 18 deletions(-)

arch/arm/include/asm/pgtable.h +18 ~~-16~~

index f66626d..d88f9f0 100644
@@ -195,6 +195,18 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd)
 
 #define pte_clear(mm,addr,ptep)	set_pte_ext(ptep, __pte(0), 0)
 
+#define pte_none(pte)		(!pte_val(pte))
+#define pte_present(pte)	(pte_val(pte) & L_PTE_PRESENT)
+#define pte_write(pte)		(!(pte_val(pte) & L_PTE_RDONLY))
+#define pte_dirty(pte)		(pte_val(pte) & L_PTE_DIRTY)
+#define pte_young(pte)		(pte_val(pte) & L_PTE_YOUNG)
+#define pte_exec(pte)		(!(pte_val(pte) & L_PTE_XN))
+#define pte_special(pte)	(0)
+
+#define pte_present_user(pte) \
+	((pte_val(pte) & (L_PTE_PRESENT | L_PTE_USER)) == \
+	 (L_PTE_PRESENT | L_PTE_USER))
+
 #if __LINUX_ARM_ARCH__ < 6
 static inline void __sync_icache_dcache(pte_t pteval)
 {

@@ -206,25 +218,15 @@ extern void __sync_icache_dcache(pte_t pteval);
 static inline void set_pte_at(struct mm_struct *mm, unsigned long addr,
 			      pte_t *ptep, pte_t pteval)
 {
-	if (addr >= TASK_SIZE)
-		set_pte_ext(ptep, pteval, 0);
-	else {
+	unsigned long ext = 0;
+
+	if (addr < TASK_SIZE && pte_present_user(pteval)) {
 		__sync_icache_dcache(pteval);
-		set_pte_ext(ptep, pteval, PTE_EXT_NG);
+		ext |= PTE_EXT_NG;
 	}
-}
 
-#define pte_none(pte)		(!pte_val(pte))
-#define pte_present(pte)	(pte_val(pte) & L_PTE_PRESENT)
-#define pte_write(pte)		(!(pte_val(pte) & L_PTE_RDONLY))
-#define pte_dirty(pte)		(pte_val(pte) & L_PTE_DIRTY)
-#define pte_young(pte)		(pte_val(pte) & L_PTE_YOUNG)
-#define pte_exec(pte)		(!(pte_val(pte) & L_PTE_XN))
-#define pte_special(pte)	(0)
-
-#define pte_present_user(pte) \
-	((pte_val(pte) & (L_PTE_PRESENT | L_PTE_USER)) == \
-	 (L_PTE_PRESENT | L_PTE_USER))
+	set_pte_ext(ptep, pteval, ext);
+}
 
 #define PTE_BIT_FUNC(fn,op) \
 static inline pte_t pte_##fn(pte_t pte) { pte_val(pte) op; return pte; }

arch/arm/mm/flush.c +0 -2

index 7745854..40ca11e 100644
@@ -231,8 +231,6 @@ void __sync_icache_dcache(pte_t pteval)
 	struct page *page;
 	struct address_space *mapping;
 
-	if (!pte_present_user(pteval))
-		return;
 	if (cache_is_vipt_nonaliasing() && !pte_exec(pteval))
 		/* only flush non-aliasing VIPT caches for exec mappings */
 		return;

Signature · 1 line

1.7.4.1

Catalin Marinas

2012-07-26 16:49 UTC #2 lore raw

PATCH 1/2 ARM: mm: avoid setting nG bit for user mappings that aren't present

Quoted 8 lines · Will Deacon

Swap entries are encoding in ptes such that !pte_present(pte) and pte_file(pte). The remaining bits of the descriptor are used to identify the swapfile and offset within it to the swap entry.
When writing such a pte for a user virtual address, set_pte_at unconditionally sets the nG bit, which (in the case of LPAE) will corrupt the swapfile offset and lead to a BUG:

Quoted 7 lines

[  140.494067] swap_free: Unused swap offset entry 000763b4
[  140.509989] BUG: Bad page map in process rs:main Q:Reg  pte:0ec76800 pmd:8f92e003
This patch fixes the problem by only setting the nG bit for user mappings that are actually present.
Signed-off-by Will Deacon <[email protected]>

Reviewed-by Catalin Marinas <[email protected]>

Will Deacon

author

2012-07-26 16:01 UTC #3 lore raw

PATCH 2/2 ARM: mm: use 5 bits for swapfile type encoding

In reply to Will Deacon

Page migration encodes the pfn in the offset field of a swp_entry_t. For LPAE, we support physical addresses of up to 36 bits (due to sparsemem limitations with the size of page flags), requiring 24 bits to represent a pfn. A further 3 bits are used to encode a swp_entry into a pte, leaving 5 bits for the type field. Furthermore, the core code defines MAX_SWAPFILES_SHIFT as 5, so the additional type bit does not get used.

This patch reduces the width of the type field to 5 bits, allowing us to create up to 31 swapfiles of 64GB each.

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm/include/asm/pgtable.h |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

arch/arm/include/asm/pgtable.h +3 -3

index d88f9f0..41dc31f 100644
@@ -253,13 +253,13 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
  *
  *   3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1
  *   1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
- *   <--------------- offset --------------------> <- type --> 0 0 0
+ *   <--------------- offset ----------------------> < type -> 0 0 0
  *
- * This gives us up to 63 swap files and 32GB per swap file.  Note that
+ * This gives us up to 31 swap files and 64GB per swap file.  Note that
  * the offset field is always non-zero.
  */
 #define __SWP_TYPE_SHIFT	3
-#define __SWP_TYPE_BITS		6
+#define __SWP_TYPE_BITS		5
 #define __SWP_TYPE_MASK		((1 << __SWP_TYPE_BITS) - 1)
 #define __SWP_OFFSET_SHIFT	(__SWP_TYPE_BITS + __SWP_TYPE_SHIFT)

Signature · 1 line

1.7.4.1

Catalin Marinas

2012-07-26 16:49 UTC #4 lore raw

PATCH 2/2 ARM: mm: use 5 bits for swapfile type encoding

Quoted 8 lines · Will Deacon

Page migration encodes the pfn in the offset field of a swp_entry_t. For LPAE, we support physical addresses of up to 36 bits (due to sparsemem limitations with the size of page flags), requiring 24 bits to represent a pfn. A further 3 bits are used to encode a swp_entry into a pte, leaving 5 bits for the type field. Furthermore, the core code defines MAX_SWAPFILES_SHIFT as 5, so the additional type bit does not get used.

Quoted 4 lines

This patch reduces the width of the type field to 5 bits, allowing us to create up to 31 swapfiles of 64GB each.
Signed-off-by Will Deacon <[email protected]>

Reviewed-by Catalin Marinas <[email protected]>