arm64: Enforce BBM for huge IO/VMAP mappings
PATCH 1/1 arm64: Enforce BBM for huge IO/VMAP mappings
ioremap_page_range doesn't honour break-before-make and attempts to put down huge mappings (using p*d_set_huge) over the top of pre-existing table entries. This leads to us leaking page table memory and also gives rise to TLB conflicts and spurious aborts, which have been seen in practice on Cortex-A75.
Until this has been resolved, refuse to put block mappings when the existing entry is found to be present.
--- arch/arm64/mm/mmu.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/arm64/mm/mmu.c +10 -0
@@ -933,6 +933,11 @@ int pud_set_huge(pud_t *pudp, phys_addr_t phys, pgprot_t prot) {pgprot_t sect_prot =__pgprot (PUD_TYPE_SECT |pgprot_val (mk_sect_prot (prot))); + +/* ioremap_page_range doesn't honour BBM */ +if (pud_present (READ_ONCE (*pudp))) +return 0 ; +BUG_ON (phys & ~PUD_MASK );set_pud (pudp,pfn_pud (__phys_to_pfn (phys), sect_prot));return 1 ;
@@ -942,6 +947,11 @@ int pmd_set_huge(pmd_t *pmdp, phys_addr_t phys, pgprot_t prot) {pgprot_t sect_prot =__pgprot (PMD_TYPE_SECT |pgprot_val (mk_sect_prot (prot))); + +/* ioremap_page_range doesn't honour BBM */ +if (pmd_present (READ_ONCE (*pmdp))) +return 0 ; +BUG_ON (phys & ~PMD_MASK );set_pmd (pmdp,pfn_pmd (__phys_to_pfn (phys), sect_prot));return 1 ;
Signature · 1 line
2.1.4
PATCH 1/1 arm64: Enforce BBM for huge IO/VMAP mappings
Quoted 35 lines · Will Deacon
ioremap_page_rangedoesn't honour break-before-make and attempts to put down huge mappings (using p*d_set_huge) over the top of pre-existing table entries. This leads to us leaking page table memory and also gives rise to TLB conflicts and spurious aborts, which have been seen in practice on Cortex-A75.Until this has been resolved, refuse to put block mappings when the existing entry is found to be present.
Cc Ard Biesheuvel <[email protected]>Reported-by Hanjun Guo <[email protected]>Reported-by Lei Li <[email protected]>Signed-off-by Will Deacon <[email protected]>--- arch/arm64/mm/mmu.c | 10 ++++++++++ 1 file changed, 10 insertions(+)arch/arm64/mm/mmu.c +5
-0@@ -933,6 +933,11 @@ int pud_set_huge(pud_t *pudp, phys_addr_t phys, pgprot_t prot) {pgprot_t sect_prot =__pgprot (PUD_TYPE_SECT |pgprot_val (mk_sect_prot (prot))); + +/* ioremap_page_range doesn't honour BBM */ +if (pud_present (READ_ONCE (*pudp))) +return 0 ; +BUG_ON (phys & ~PUD_MASK );set_pud (pudp,pfn_pud (__phys_to_pfn (phys), sect_prot));return 1 ;@@ -942,6 +947,11 @@ int pmd_set_huge(pmd_t *pmdp, phys_addr_t phys, pgprot_t prot) {
arch/arm64/mm/mmu.c int pmd_set_huge(pmd_t *pmdp, phys_addr_t phys, pgprot_t prot) · line 957
Quoted 10 lines
arch/arm64/mm/mmu.c +5
-0pgprot_t sect_prot =__pgprot (PMD_TYPE_SECT |pgprot_val (mk_sect_prot (prot))); + +/* ioremap_page_range doesn't honour BBM */ +if (pmd_present (READ_ONCE (*pmdp))) +return 0 ; +BUG_ON (phys & ~PMD_MASK );set_pmd (pmdp,pfn_pmd (__phys_to_pfn (phys), sect_prot));return 1 ;
This is a nice way of dealing with this IMO, as large mappings that are only created at boot (e.g., for the ECAM space or gfx BARs) will still use huge pages.
Catalin Marinas
maintainerPATCH 1/1 arm64: Enforce BBM for huge IO/VMAP mappings
In reply to Will DeaconQuoted 6 lines · Will Deacon
ioremap_page_rangedoesn't honour break-before-make and attempts to put down huge mappings (using p*d_set_huge) over the top of pre-existing table entries. This leads to us leaking page table memory and also gives rise to TLB conflicts and spurious aborts, which have been seen in practice on Cortex-A75.
Quoted 7 lines
Until this has been resolved, refuse to put block mappings when the existing entry is found to be present.
Cc Ard Biesheuvel <[email protected]>Reported-by Hanjun Guo <[email protected]>Reported-by Lei Li <[email protected]>Signed-off-by Will Deacon <[email protected]>
Queued for 4.16. Shall we put a cc stable?
Signature · 1 line
Catalin
PATCH 1/1 arm64: Enforce BBM for huge IO/VMAP mappings
Quoted 10 lines · Catalin Marinas
Quoted 9 lines · Will Deacon
ioremap_page_rangedoesn't honour break-before-make and attempts to put down huge mappings (using p*d_set_huge) over the top of pre-existing table entries. This leads to us leaking page table memory and also gives rise to TLB conflicts and spurious aborts, which have been seen in practice on Cortex-A75.Until this has been resolved, refuse to put block mappings when the existing entry is found to be present.
Quoted 6 lines
Quoted 4 lines
Cc Ard Biesheuvel <[email protected]>Reported-by Hanjun Guo <[email protected]>Reported-by Lei Li <[email protected]>Signed-off-by Will Deacon <[email protected]>Queued for 4.16. Shall we put a cc stable?
Yes, good idea:
Will