Fix issues with huge mapping in ioremap for ARM64

This patch ("mm/vmalloc: Add interfaces to free unmapped page table") adds following 2 interfaces to free the page table in case we implement huge mapping.

pud_free_pmd_page() and pmd_free_pte_page()

Some architectures (like arm64) needs to do proper TLB maintanance after updating pagetable entry even in map. Why ? Read this,
https://patchwork.kernel.org/patch/10134581/

Pass 'addr' in these interfaces so that proper TLB ops can be performed.

Signed-off-by Chintan Pandya <[email protected]>

Quoted 11 lines · Chintan Pandya

This patch ("mm/vmalloc: Add interfaces to free unmapped page table") adds following 2 interfaces to free the page table in case we implement huge mapping.

pud_free_pmd_page() and pmd_free_pte_page()

Some architectures (like arm64) needs to do proper TLB maintanance after updating pagetable entry even in map. Why ? Read this,
https://patchwork.kernel.org/patch/10134581/

Quoted 4 lines

Pass 'addr' in these interfaces so that proper TLB ops can be performed.

Signed-off-by Chintan Pandya <[email protected]>

:

arch/x86/mm/pgtable.c int pmd_clear_huge(pmd_t *pmd) · line 713

Quoted 10 lines

arch/x86/mm/pgtable.c +1 -1

index 34cda7e..20b0322 100644
@@ -710,7 +710,7 @@ int pmd_clear_huge(pmd_t *pmd)
   * Context: The pud range has been unmaped and TLB purged.
   * Return: 1 if clearing the entry succeeded. 0 otherwise.
   */
-int pud_free_pmd_page(pud_t *pud)
+int pud_free_pmd_page(pud_t *pud, unsigned long addr)

Please update the function header as well. Same for pmd.

+ * @addr: Virtual address associated with pud.

--- a/include/asm-generic/pgtable.h

arch/x86/mm/pgtable.c static inline int p4d_clear_huge(p4d_t *p4d) · line 989

Quoted 11 lines

arch/x86/mm/pgtable.c +3 -2

+++ b/include/asm-generic/pgtable.h

@@ -983,8 +983,8 @@ static inline int p4d_clear_huge(p4d_t *p4d)
  int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot);
  int pud_clear_huge(pud_t *pud);
  int pmd_clear_huge(pmd_t *pmd);
-int pud_free_pmd_page(pud_t *pud);
-int pmd_free_pte_page(pmd_t *pmd);
+int pud_free_pmd_page(pud_t *pud, unsigned long addr);
+int pmd_free_pte_page(pmd_t *pmd, unsigned long addr);
  #else	/* !CONFIG_HAVE_ARCH_HUGE_VMAP */
  static inline int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t

prot)

arch/x86/mm/pgtable.c static inline int pmd_clear_huge(pmd_t *pmd) · line 1017

Quoted 12 lines

arch/x86/mm/pgtable.c +2 -2

  {

@@ -1010,11 +1010,11 @@ static inline int pmd_clear_huge(pmd_t *pmd)
  {
  	return 0;
  }
-static inline int pud_free_pmd_page(pud_t *pud)
+static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr)
  {
  	return 0;
  }
-static inline int pmd_free_pte_page(pud_t *pmd)
+static inline int pmd_free_pte_page(pud_t *pmd, unsigned long addr)

Please base your patches on top of the mm tree or at least pick up 8cab93de994f6 in the mm tree. Andrew fixed the typo; the 'pud_t' above should have been 'pmd_t'.

Thanks,
-Toshi

Implement pud_free_pmd_page() and pmd_free_pte_page().

Implementation requires,
1) Freeing of the un-used next level page tables
2) Clearing off the current pud/pmd entry
3) Invalidate TLB which could have previously
valid but not stale entry

Signed-off-by Chintan Pandya <[email protected]>

---
 arch/arm64/mm/mmu.c | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

arch/arm64/mm/mmu.c +1 -0

index da98828..c70f139 100644
@@ -45,6 +45,7 @@
 #include <asm/memblock.h>
 #include <asm/mmu_context.h>
 #include <asm/ptdump.h>
+#include <asm/tlbflush.h>
 
 #define NO_BLOCK_MAPPINGS	BIT(0)
 #define NO_CONT_MAPPINGS	BIT(1)

@@ -975,10 +976,35 @@ int pmd_clear_huge(pmd_t *pmdp)
 
 int pud_free_pmd_page(pud_t *pud, unsigned long addr)
 {

Quoted 36 lines · Chintan Pandya

Implement pud_free_pmd_page() and pmd_free_pte_page().

Implementation requires,
1) Freeing of the un-used next level page tables
2) Clearing off the current pud/pmd entry
3) Invalidate TLB which could have previously valid but not stale entry

Signed-off-by Chintan Pandya <[email protected]>

---
  arch/arm64/mm/mmu.c | 30 ++++++++++++++++++++++++++++--
  1 file changed, 28 insertions(+), 2 deletions(-)

arch/arm64/mm/mmu.c +7 -1

index da98828..c70f139 100644
@@ -45,6 +45,7 @@
  #include <asm/memblock.h>
  #include <asm/mmu_context.h>
  #include <asm/ptdump.h>
+#include <asm/tlbflush.h>
 
  #define NO_BLOCK_MAPPINGS	BIT(0)
  #define NO_CONT_MAPPINGS	BIT(1)

@@ -975,10 +976,35 @@ int pmd_clear_huge(pmd_t *pmdp)
 
  int pud_free_pmd_page(pud_t *pud, unsigned long addr)
  {
-	return pud_none(*pud);
+	pmd_t *pmd;
+	int i;
+
+	pmd = __va(pud_val(*pud));
+	if (pud_val(*pud)) {
+		for (i = 0; i < PTRS_PER_PMD; i++)

arch/arm64/mm/mmu.c int pmd_clear_huge(pmd_t *pmdp) · line 987

Quoted 3 lines

arch/arm64/mm/mmu.c +3 -0

+			pmd_free_pte_page(&pmd[i], addr + (i * PMD_SIZE));
+
+		free_page((unsigned long) pmd);

Why do you want to free this pmd page before clearing the pud entry on this arm64 version (as it seems you intentionally changed it from the x86 version)? It can be reused while being pointed by the pud. Same for pmd.

arch/arm64/mm/mmu.c int pmd_clear_huge(pmd_t *pmdp) · line 989

Quoted 3 lines

arch/arm64/mm/mmu.c +2 -0

+		pud_clear(pud);
+		flush_tlb_kernel_range(addr, addr + PUD_SIZE);

Since you purge the entire pud range here, do you still need to call pmd_free_pte_page() to purge each pmd range? This looks very expensive. You may want to consider if calling internal __pmd_free_pte_page() without the purge operation works.

-Toshi