arm64: mm: fix pmd_write CoW brokenness

PATCH New Will Deacon 2 participants 2 messages May 28, 2014 arm64

Share & tools

Will Deacon

author

2014-05-28 17:07 UTC #0 lore raw

PATCH 1/1 arm64: mm: fix pmd_write CoW brokenness

Commit 9c7e535fcc17 ("arm64: mm: Route pmd thp functions through pte equivalents") changed the pmd manipulator and accessor functions to convert the target pmd to a pte, process it with the pte functions, then convert it back. Along the way, we gained support for PTE_WRITE, however this is completely ignored by set_pmd_at, and so we fail to set the PMD_SECT_RDONLY for PMDs, resulting in all sorts of lovely failures (like CoW not working).

Partially reverting the offending commit (by making use of PMD_SECT_RDONLY explicitly for pmd_{write,wrprotect,mkwrite} functions) leads to further issues because pmd_write can then return potentially incorrect values for page table entries marked as RDONLY, leading to BUG_ON(pmd_write(entry)) tripping under some THP workloads.

This patch fixes the issue by routing set_pmd_at through set_pte_at, which correctly takes the PTE_WRITE flag into account. Given that THP mappings are always anonymous, the additional cache-flushing code in __sync_icache_dcache won't impose any significant overhead as the flush will be skipped.

Cc Steve Capper <[email protected]>

Cc Catalin Marinas <[email protected]>

Signed-off-by Will Deacon <[email protected]>

---

Whilst this is a fairly scary change at this point in the cycle, I can't get through an LTP run without it. Furthermore, spurious CoW failures for tasks that happen to get transparent hugepages are a pretty significant regression.

 arch/arm64/include/asm/pgtable.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/arm64/include/asm/pgtable.h +1 -1

index 90c811f05a2e..20785f9da95c 100644
@@ -266,7 +266,7 @@ static inline pmd_t pte_pmd(pte_t pte)
 
 #define pmd_page(pmd)           pfn_to_page(__phys_to_pfn(pmd_val(pmd) & PHYS_MASK))
 
-#define set_pmd_at(mm, addr, pmdp, pmd)	set_pmd(pmdp, pmd)
+#define set_pmd_at(mm, addr, pmdp, pmd)	set_pte_at(mm, addr, pmdp, pmd_pte(pmd))
 
 static inline int has_transparent_hugepage(void)
 {

Signature · 1 line

1.9.2

Marc Zyngier

2014-05-28 17:49 UTC #1 lore raw

PATCH 1/1 arm64: mm: fix pmd_write CoW brokenness

Hi Will,

Quoted 38 lines · Will Deacon

Commit 9c7e535fcc17 ("arm64: mm: Route pmd thp functions through pte equivalents") changed the pmd manipulator and accessor functions to convert the target pmd to a pte, process it with the pte functions, then convert it back. Along the way, we gained support for PTE_WRITE, however this is completely ignored by set_pmd_at, and so we fail to set the PMD_SECT_RDONLY for PMDs, resulting in all sorts of lovely failures (like CoW not working).
Partially reverting the offending commit (by making use of PMD_SECT_RDONLY explicitly for pmd_{write,wrprotect,mkwrite} functions) leads to further issues because pmd_write can then return potentially incorrect values for page table entries marked as RDONLY, leading to BUG_ON(pmd_write(entry)) tripping under some THP workloads.
This patch fixes the issue by routing set_pmd_at through set_pte_at, which correctly takes the PTE_WRITE flag into account. Given that THP mappings are always anonymous, the additional cache-flushing code in __sync_icache_dcache won't impose any significant overhead as the flush will be skipped.
Cc Steve Capper <[email protected]>
Cc Catalin Marinas <[email protected]>
Signed-off-by Will Deacon <[email protected]>
---
Whilst this is a fairly scary change at this point in the cycle, I can't get through an LTP run without it. Furthermore, spurious CoW failures for tasks that happen to get transparent hugepages are a pretty significant regression.
 arch/arm64/include/asm/pgtable.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/include/asm/pgtable.h +0 -0
index 90c811f05a2e..20785f9da95c 100644

arch/arm64/include/asm/pgtable.h static inline pmd_t pte_pmd(pte_t pte) · line 272

Quoted 8 lines

arch/arm64/include/asm/pgtable.h +1 -1

@@ -266,7 +266,7 @@ static inline pmd_t pte_pmd(pte_t pte)
 
 #define pmd_page(pmd)           pfn_to_page(__phys_to_pfn(pmd_val(pmd) & PHYS_MASK))
 
-#define set_pmd_at(mm, addr, pmdp, pmd)	set_pmd(pmdp, pmd)
+#define set_pmd_at(mm, addr, pmdp, pmd)	set_pte_at(mm, addr, pmdp, pmd_pte(pmd))
 
 static inline int has_transparent_hugepage(void)
 {

I managed to reliably reproduce the failure on my favourite arm64 board running a KVM guest with THP enabled. Adding this patch to the guest kernel made it behave correctly.

So FWIW:

Tested-by Marc Zyngier <[email protected]>

M.

Signature · 1 line

Jazz is not dead. It just smells funny...