arm64: switch to generic version of pte allocation
A message that is not in the archive (sent off-list or not yet received): 20190712035802.eeH5anzpz%[email protected]
Actually, this patch is slightly off. There is one delta need to apply (ignore the part in pgtable.h which has already in mainline via the commit 615c48ad8f42 "arm64/mm: don't initialize pgd_cache twice") in.
https://lore.kernel.org/linux-mm/20190617151252.GF16810@rapoport-lnx/
Quoted 186 lines · On Thu, 2019-07-11 at 20:58 -0700, [email protected] wrote:
From: Mike Rapoport <[email protected]>
Subject: arm64: switch to generic version of pte allocationThe PTE allocations in arm64 are identical to the generic ones modulo the GFP flags.
Using the generic
pte_alloc_one()functions ensures that the user page tables are allocated with__GFP_ACCOUNTset.The arm64 definition of
PGALLOC_GFPis removed and replaced withGFP_PGTABLE_USERfor p[gum]d_alloc_one() for the user page tablesandpgtable.h
GFP_PGTABLE_KERNELfor the kernel page tables. The KVM memory cache is now usingGFP_PGTABLE_USER.The mappings created with
create_pgd_mapping()are now usingGFP_PGTABLE_KERNEL.The conversion to the generic version of
pte_free_kernel()removes the NULL check for pte.The
pte_free()version on arm64 is identical to the generic one and can be simply dropped.[[email protected]: fix a bogus GFP flag in pgd_alloc()]m.com
Signed-off-by Mike Rapoport <[email protected]>Cc Albert Ou <[email protected]>Cc Anshuman Khandual <[email protected]>Cc Anton Ivanov <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Catalin Marinas <[email protected]>Cc Geert Uytterhoeven <[email protected]>Cc Greentime Hu <[email protected]>Cc Guan Xuetao <[email protected]>Cc Guo Ren <[email protected]>Cc Guo Ren <[email protected]>Cc Helge Deller <[email protected]>Cc Ley Foon Tan <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Matt Turner <[email protected]>Cc Michael Ellerman <[email protected]>Cc Michal Hocko <[email protected]>Cc Palmer Dabbelt <[email protected]>Cc Paul Burton <[email protected]>Cc Ralf Baechle <[email protected]>Cc Richard Kuo <[email protected]>Cc Richard Weinberger <[email protected]>Cc Russell King <[email protected]>Cc Sam Creasey <[email protected]>Cc Vincent Chen <[email protected]>Signed-off-by Andrew Morton <[email protected]>---
arch/arm64/include/asm/pgalloc.h | 47 ++++------------------------- arch/arm64/mm/mmu.c | 2 - arch/arm64/mm/pgd.c | 9 ++++- virt/kvm/arm/mmu.c | 2 -
4 files changed, 17 insertions(+), 43 deletions(-)--- a/arch/arm64/include/asm/pgalloc.h~arm64-switch-to-generic-version-of-pte- allocation
+++a/arch/arm64/include/asm/pgalloc.h(quoted diff) +0
-0@@ -13,18 +13,23 @@ #include <asm/cacheflush.h> #include <asm/tlbflush.h>(quoted diff) +2
-0+#include <asm-generic/pgalloc.h> /* for pte_{alloc,free}_one */ +#define
check_pgt_cache()do { } while (0)
-#definePGALLOC_GFP(GFP_KERNEL|__GFP_ZERO)
#definePGD_SIZE(PTRS_PER_PGD*sizeof(pgd_t))
#ifCONFIG_PGTABLE_LEVELS> 2
static inlinepmd_t*pmd_alloc_one(structmm_struct*mm, unsigned long addr) {(quoted diff) +1
-0+ gfp_t gfp = GFP_PGTABLE_USER;struct page *page;
(quoted diff) +4
-1- page = alloc_page(PGALLOC_GFP); + if (mm == &init_mm) + gfp = GFP_PGTABLE_KERNEL; + + page = alloc_page(gfp);if (!page)
return NULL;
if (!pgtable_pmd_page_ctor(page)) {(quoted diff) +0
-0@@ -61,7 +66,7 @@ static inline void __pud_populate(pud_t
static inlinepud_t*pud_alloc_one(structmm_struct*mm, unsigned long addr) {(quoted diff) +1
-1- return (pud_t *)__get_free_page(PGALLOC_GFP); + return (pud_t *)__get_free_page(GFP_PGTABLE_USER);}
static inline voidpud_free(struct mm_struct *mm, pud_t *pudp)(quoted diff) +0
-0@@ -89,42 +94,6 @@ static inline void __pgd_populate(pgd_t extern pgd_t *pgd_alloc(struct mm_struct *mm); extern void pgd_free(struct mm_struct *mm, pgd_t *pgdp);(quoted diff) +0
-36-static inline pte_t * -pte_alloc_one_kernel(struct mm_struct *mm) -{ - return (pte_t *)__get_free_page(PGALLOC_GFP); -} - -static inline pgtable_t -pte_alloc_one(struct mm_struct *mm) -{ - struct page *pte; - - pte = alloc_pages(PGALLOC_GFP, 0); - if (!pte) - return NULL; - if (!pgtable_page_ctor(pte)) { - __free_page(pte); - return NULL; - } - return pte; -} - -/* - * Free a PTE table. - */ -static inline void pte_free_kernel(struct mm_struct *mm, pte_t *ptep) -{ - if (ptep) - free_page((unsigned long)ptep); -} - -static inline void pte_free(struct mm_struct *mm, pgtable_t pte) -{ - pgtable_page_dtor(pte); - __free_page(pte); -} -static inline void
__pmd_populate(pmd_t *pmdp, phys_addr_t ptep, pmdval_t prot)
{arch/arm64/mm/mmu.c renamed from arch/arm64/mm/mmu.c~arm64-switch-to-generic-version-of-pte-allocation +0
-0@@ -362,7 +362,7 @@ static void __create_pgd_mapping(pgd_t *
staticphys_addr_t__pgd_pgtable_alloc(int shift){arch/arm64/mm/mmu.c renamed from arch/arm64/mm/mmu.c~arm64-switch-to-generic-version-of-pte-allocation +1
-1-void *ptr = (void *)__get_free_page (PGALLOC_GFP ); +void *ptr = (void *)__get_free_page (GFP_PGTABLE_KERNEL );
BUG_ON(!ptr);
/* Ensure the zeroed page is visible to the page table walker */arch/arm64/mm/pgd.c renamed from arch/arm64/mm/pgd.c~arm64-switch-to-generic-version-of-pte-allocation +0
-0@@ -19,10 +19,15 @@ static struct kmem_cache *pgd_cache __ro
pgd_t *pgd_alloc(structmm_struct*mm)
{arch/arm64/mm/pgd.c renamed from arch/arm64/mm/pgd.c~arm64-switch-to-generic-version-of-pte-allocation +5
-0+gfp_t gfp =GFP_PGTABLE_USER ; + +if (unlikely (mm == &init_mm)) + gfp =GFP_PGTABLE_KERNEL ; +if (
PGD_SIZE==PAGE_SIZE)arch/arm64/mm/pgd.c renamed from arch/arm64/mm/pgd.c~arm64-switch-to-generic-version-of-pte-allocation +1
-1-return (pgd_t *)__get_free_page (PGALLOC_GFP ); +return (pgd_t *)__get_free_page (gfp);else
arch/arm64/mm/pgd.c renamed from arch/arm64/mm/pgd.c~arm64-switch-to-generic-version-of-pte-allocation +1
-1-return kmem_cache_alloc (pgd_cache,PGALLOC_GFP ); +return kmem_cache_alloc (pgd_cache,GFP_PGTABLE_KERNEL );}
voidpgd_free(struct mm_struct *mm, pgd_t *pgd)virt/kvm/arm/mmu.c renamed from virt/kvm/arm/mmu.c~arm64-switch-to-generic-version-of-pte-allocation +1
-1@@ -129,7 +129,7 @@ static int mmu_topup_memory_cache(struct if (cache->nobjs >= min) return 0; while (cache->nobjs < max) { - page = (void *)__get_free_page (PGALLOC_GFP ); + page = (void *)__get_free_page (GFP_PGTABLE_USER ); if (!page) return -ENOMEM; cache->objects[cache->nobjs++] = page;_
Quoted 6 lines · Qian Cai
Actually, this patch is slightly off. There is one delta need to apply (ignore the part in
pgtable.hwhich has already in mainline via the commit615c48ad8f42"arm64/mm: don't initializepgd_cachetwice") in.https://lore.kernel.org/linux-mm/20190617151252.GF16810@rapoport-lnx/
I fixed it up, hopefully correctly.
Linus
Quoted 2 lines · Linus Torvalds
I fixed it up, hopefully correctly.
.. and it's pushed out now so you can all point and laugh at me.
Linus
Quoted 5 lines · On Fri, 12 Jul 2019 08:49:07 -0400 Qian Cai <[email protected]> wrote:
Actually, this patch is slightly off. There is one delta need to apply (ignore the part in
pgtable.hwhich has already in mainline via the commit615c48ad8f42"arm64/mm: don't initializepgd_cachetwice") in.https://lore.kernel.org/linux-mm/20190617151252.GF16810@rapoport-lnx/
That's already merged - it went in via the arm64 tree I think.
Quoted 6 lines · Andrew Morton
Quoted 2 lines · On Fri, 12 Jul 2019 08:49:07 -0400 Qian Cai <[email protected]> wrote:
https://lore.kernel.org/linux-mm/20190617151252.GF16810@rapoport-lnx/
That's already merged - it went in via the arm64 tree I think.
No. Only the arch/arm64/include/asm/pgtable.h part got in through the arm64 tree (commit 615c48ad8f42: "arm64/mm: don't initialize pgd_cache twice").
The arch/arm64/mm/pgd.c part was missing.
I think I fixed it all up correctly.
Linus
Quoted 6 lines · Linus Torvalds
Quoted 5 lines · Andrew Morton
Quoted 2 lines · On Fri, 12 Jul 2019 08:49:07 -0400 Qian Cai <[email protected]> wrote:
https://lore.kernel.org/linux-mm/20190617151252.GF16810@rapoport-lnx/
Quoted 7 lines
Quoted 1 line
That's already merged - it went in via the arm64 tree I think.
No. Only the
arch/arm64/include/asm/pgtable.hpart got in through the arm64 tree (commit615c48ad8f42: "arm64/mm: don't initializepgd_cachetwice").The
arch/arm64/mm/pgd.cpart was missing.
That's right; we split the patch because part of it was a fix for an existing issue.
Quoted 1 line
I think I fixed it all up correctly.
Looks good to me, thanks.
Will