use struct ptdesc to replace pgtable_t
Subsystems (33)
cover use struct ptdesc to replace pgtable_t
From: Alex Shi <[email protected]>
We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while pgtable_t is used widely.
The pgtable_t is typedefed as 'pte_t *' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'.
These blocks the conception and code update for page table descriptor to struct ptdesc.
So, the simple idea to push the ptdesc conception forward is to update all pgtable_t by ptdesc or pte_t pointer. But this needs widely knowledges for most all of different archs. Common code change is easy for include/ and mm/ directory, but it's hard in all archs.
Thanks for intel LKP framework, I fixed most all of build issues except a bug on powerpc which reports a "struct ptdesc *" incompatible with struct ptdesc *' pointer issue...
Another trouble is pmd_pgtable() conversion in the last patch. Maybe some of arch need define theirself own pmd_ptdesc()?
This patchset is immature, even except above 2 issues, I just tested virutal machine booting and kselftest mm on x86 and arm64.
Anyway any input are appreciated!
Thanks
Alex
Alex Shi (18):
mm/pgtable: use ptdesc in pte_free_now/pte_free_defer mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer fs/dax: use ptdesc in dax_pmd_load_hole
mm/thp: use ptdesc pointer in __do_huge_pmd_anonymous_page mm/thp: use ptdesc in do_huge_pmd_anonymous_page
mm/thp: convert insert_pfn_pmd and its caller to use ptdesc mm/thp: use ptdesc in copy_huge_pmd
mm/memory: use ptdesc in __pte_alloc
mm/pgtable: fully use ptdesc in pte_alloc_one series functions mm/pgtable: pass ptdesc to pte_free()
mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range() mm/thp: pass ptdesc to set_huge_zero_folio function mm/pgtable: return ptdesc pointer in pgtable_trans_huge_withdraw mm/pgtable: use ptdesc in pgtable_trans_huge_deposit mm/pgtable: pass ptdesc to pmd_populate
mm/pgtable: pass ptdesc to pmd_install
mm: convert vmf.prealloc_pte to struct ptdesc pointer mm/pgtable: pass ptdesc in pte_free_defer
arch/alpha/include/asm/pgalloc.h | 4 +- arch/arc/include/asm/pgalloc.h | 4 +- arch/arm/include/asm/pgalloc.h | 13 +-- arch/arm/include/asm/tlb.h | 4 +- arch/arm/mm/pgd.c | 2 +- arch/arm64/include/asm/pgalloc.h | 4 +- arch/arm64/include/asm/tlb.h | 4 +- arch/csky/include/asm/pgalloc.h | 4 +- arch/hexagon/include/asm/pgalloc.h | 8 +- arch/loongarch/include/asm/pgalloc.h | 8 +- arch/m68k/include/asm/motorola_pgalloc.h | 12 +- arch/m68k/include/asm/sun3_pgalloc.h | 4 +- arch/microblaze/include/asm/pgalloc.h | 2 +- arch/mips/include/asm/pgalloc.h | 4 +- arch/nios2/include/asm/pgalloc.h | 4 +- arch/openrisc/include/asm/pgalloc.h | 8 +- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/32/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 8 +- arch/powerpc/include/asm/book3s/64/radix.h | 4 +- arch/powerpc/include/asm/pgalloc.h | 8 +- arch/powerpc/mm/book3s64/hash_pgtable.c | 10 +- arch/powerpc/mm/book3s64/radix_pgtable.c | 10 +- arch/riscv/include/asm/pgalloc.h | 8 +- arch/s390/include/asm/pgalloc.h | 4 +- arch/s390/include/asm/pgtable.h | 4 +- arch/s390/mm/pgalloc.c | 2 +- arch/s390/mm/pgtable.c | 14 +-- arch/sh/include/asm/pgalloc.h | 4 +- arch/sparc/include/asm/pgalloc_32.h | 6 +- arch/sparc/include/asm/pgalloc_64.h | 2 +- arch/sparc/include/asm/pgtable_64.h | 4 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 6 +- arch/sparc/mm/tlb.c | 14 +-- arch/x86/include/asm/pgalloc.h | 10 +- arch/x86/mm/pgtable.c | 8 +- arch/xtensa/include/asm/pgalloc.h | 12 +- fs/dax.c | 14 +-- include/asm-generic/pgalloc.h | 10 +- include/linux/mm.h | 16 ++- include/linux/mm_types.h | 4 +- include/linux/pgtable.h | 6 +- mm/debug_vm_pgtable.c | 6 +- mm/huge_memory.c | 103 +++++++++--------- mm/internal.h | 2 +- mm/khugepaged.c | 14 +-- mm/memory.c | 15 +-- mm/mremap.c | 2 +- mm/pgtable-generic.c | 37 +++---- 53 files changed, 240 insertions(+), 236 deletions(-)
Signature · 1 line
2.43.0
RFC PATCH 1/18 mm/pgtable: use ptdesc in pte_free_now/pte_free_defer
From: Alex Shi <[email protected]>
page table descriptor is splited from struct page, use it to replace struct page in right place.
--- mm/pgtable-generic.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
mm/pgtable-generic.c +6 -7
@@ -238,18 +238,17 @@ pmd_t pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long address,#ifndef pte_free_deferstatic void pte_free_now (struct rcu_head *head) { -struct page *page; +struct ptdesc *ptdesc; - page = container_of(head, struct page, rcu_head); -pte_free (NULL /* mm not passed and not used */ , (pgtable_t )page); + ptdesc = container_of(head, struct ptdesc, pt_rcu_head); +pte_free (NULL /* mm not passed and not used */ , (pgtable_t )ptdesc); }void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable) { -struct page *page; +struct ptdesc *ptdesc =page_ptdesc (pgtable); - page = pgtable; -call_rcu (&page->rcu_head, pte_free_now); +call_rcu (&ptdesc->pt_rcu_head, pte_free_now); }#endif /* pte_free_defer */ #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
@@ -330,7 +329,7 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd,* kmapped if necessary (when CONFIG_HIGHPTE), and locked against concurrent * modification by software, with a pointer to that spinlock in ptlp (in some * configs mm->page_table_lock, in SPLIT_PTLOCK configs a spinlock in table's -* struct page). pte_unmap_unlock(pte, ptl) to unlock and unmap afterwards. +* struct ptdesc). pte_unmap_unlock(pte, ptl) to unlock and unmap afterwards. * * But it is unsuccessful, returning NULL with *ptlp unchanged, if there is no * page table at *pmd: if, for example, the page table has just been removed,
Signature · 1 line
2.43.0
RFC PATCH 2/18 mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer
In reply to [email protected]From: Alex Shi <[email protected]>
folio/page.pmd_huge_pte is a pointer to pagetable descriptor: pgtable_t. In most arch, it is a typedef of 'struct page *'. But we have ptdesc now, tt's better to convert it to right one: struct ptdesc pointer.
Different from others, s390/sparc use typedef 'pte_t *' as pgtable_t, so they need different cost in their arch.
Thanks for lkp found the build issue for s390/sparc, it fixed now.
--- arch/s390/mm/pgtable.c | 6 +++--- arch/sparc/mm/tlb.c | 6 +++--- include/linux/mm_types.h | 4 ++-- mm/pgtable-generic.c | 16 ++++++++-------- 4 files changed, 16 insertions(+), 16 deletions(-)
arch/s390/mm/pgtable.c +3 -3
@@ -574,7 +574,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,INIT_LIST_HEAD (lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); -pmd_huge_pte (mm, pmdp) = pgtable; +pmd_huge_pte (mm, pmdp) = (struct ptdesc *)pgtable; }pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp)
@@ -586,12 +586,12 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)assert_spin_locked (pmd_lockptr (mm, pmdp));/* FIFO */ - pgtable =pmd_huge_pte (mm, pmdp); + pgtable = (pte_t *)pmd_huge_pte (mm, pmdp); lh = (struct list_head *) pgtable;if (list_empty (lh))pmd_huge_pte (mm, pmdp) =NULL ;else { -pmd_huge_pte (mm, pmdp) = (pgtable_t ) lh->next; +pmd_huge_pte (mm, pmdp) = (struct ptdesc *) lh->next;list_del (lh); } ptep = (pte_t *) pgtable;
arch/sparc/mm/tlb.c +3 -3
@@ -278,7 +278,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,INIT_LIST_HEAD (lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); -pmd_huge_pte (mm, pmdp) = pgtable; +pmd_huge_pte (mm, pmdp) = (struct ptdesc *)pgtable; }pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp)
@@ -289,12 +289,12 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)assert_spin_locked (&mm->page_table_lock);/* FIFO */ - pgtable =pmd_huge_pte (mm, pmdp); + pgtable = (pte_t *)pmd_huge_pte (mm, pmdp); lh = (struct list_head *) pgtable;if (list_empty (lh))pmd_huge_pte (mm, pmdp) =NULL ;else { -pmd_huge_pte (mm, pmdp) = (pgtable_t ) lh->next; +pmd_huge_pte (mm, pmdp) = (struct ptdesc *) lh->next;list_del (lh); }pte_val (pgtable[0 ]) =0 ;
include/linux/mm_types.h +2 -2
@@ -462,7 +462,7 @@ struct ptdesc {struct list_head pt_list;struct {unsigned long _pt_pad_1; -pgtable_t pmd_huge_pte; +struct ptdesc *pmd_huge_pte; }; };unsigned long __page_mapping;
@@ -948,7 +948,7 @@ struct mm_struct {struct mmu_notifier_subscriptions *notifier_subscriptions;#endif #if defined(CONFIG_TRANSPARENT_HUGEPAGE ) && !USE_SPLIT_PMD_PTLOCKS -pgtable_t pmd_huge_pte;/* protected by page_table_lock */ +struct ptdesc *pmd_huge_pte;/* protected by page_table_lock */ #endif #ifdef CONFIG_NUMA_BALANCING /*
mm/pgtable-generic.c +8 -8
@@ -171,8 +171,8 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,if (!pmd_huge_pte (mm, pmdp))INIT_LIST_HEAD (&pgtable->lru);else -list_add (&pgtable->lru, &pmd_huge_pte (mm, pmdp)->lru); - pmd_huge_pte(mm, pmdp) = pgtable; +list_add (&pgtable->lru, &pmd_huge_pte (mm, pmdp)->pt_list); + pmd_huge_pte(mm, pmdp) = page_ptdesc(pgtable); }#endif
@@ -180,17 +180,17 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,/* no "address" argument so destroys page coloring of some arch */ pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) { -pgtable_t pgtable; +struct ptdesc *ptdesc;assert_spin_locked (pmd_lockptr (mm, pmdp));/* FIFO */ - pgtable =pmd_huge_pte (mm, pmdp); -pmd_huge_pte (mm, pmdp) =list_first_entry_or_null (&pgtable->lru, - struct page, lru); + ptdesc =pmd_huge_pte (mm, pmdp); +pmd_huge_pte (mm, pmdp) =list_first_entry_or_null (&ptdesc->pt_list, + struct ptdesc, pt_list);if (pmd_huge_pte (mm, pmdp)) -list_del (&pgtable->lru); -return pgtable; +list_del (&ptdesc->pt_list); +return ptdesc_page (ptdesc); }#endif
Signature · 1 line
2.43.0
RFC PATCH 3/18 fs/dax: use ptdesc in dax_pmd_load_hole
In reply to [email protected]From: Alex Shi <[email protected]>
Since we have ptdesc struct now, better to use replace pgtable_t, aka 'struct page *'.
It's a prepare for return ptdesc pointer in pte_alloc_one series function.
--- fs/dax.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
fs/dax.c +7 -7
@@ -1206,7 +1206,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf,unsigned long pmd_addr = vmf->address &PMD_MASK ;struct vm_area_struct *vma = vmf->vma;struct inode *inode = mapping->host; -pgtable_t pgtable =NULL ; +struct ptdesc *ptdesc =NULL ;struct folio *zero_folio;spinlock_t *ptl;pmd_t pmd_entry;
@@ -1222,8 +1222,8 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, DAX_PMD | DAX_ZERO_PAGE);if (arch_needs_pgtable_deposit ()) { - pgtable =pte_alloc_one (vma->vm_mm); -if (!pgtable) + ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); +if (!ptdesc)return VM_FAULT_OOM ; }
@@ -1233,8 +1233,8 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, goto fallback; } -if (pgtable) { -pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd, pgtable); +if (ptdesc) { +pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd,ptdesc_page (ptdesc));mm_inc_nr_ptes (vma->vm_mm); } pmd_entry =mk_pmd (&zero_folio->page, vmf->vma->vm_page_prot);
@@ -1245,8 +1245,8 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf,return VM_FAULT_NOPAGE ; fallback: -if (pgtable) -pte_free (vma->vm_mm, pgtable); +if (ptdesc) +pte_free (vma->vm_mm,ptdesc_page (ptdesc));trace_dax_pmd_load_hole_fallback (inode, vmf, zero_folio, *entry);return VM_FAULT_FALLBACK ; }
Signature · 1 line
2.43.0
RFC PATCH 4/18 mm/thp: use ptdesc pointer in __do_huge_pmd_anonymous_page
In reply to [email protected]From: Alex Shi <[email protected]>
Since we have ptdesc struct now, better to use replace pgtable_t, aka 'struct page *'. It's alaos a preparation for return ptdesc pointer in pte_alloc_one series function.
--- mm/huge_memory.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
mm/huge_memory.c +7 -7
@@ -943,7 +943,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, {struct vm_area_struct *vma = vmf->vma;struct folio *folio =page_folio (page); -pgtable_t pgtable; +struct ptdesc *ptdesc;unsigned long haddr = vmf->address &HPAGE_PMD_MASK ;vm_fault_t ret =0 ;
@@ -959,8 +959,8 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, }folio_throttle_swaprate (folio, gfp); - pgtable =pte_alloc_one (vma->vm_mm); -if (unlikely (!pgtable)) { + ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); +if (unlikely (!ptdesc)) { ret =VM_FAULT_OOM ; goto release; }
@@ -987,7 +987,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,if (userfaultfd_missing (vma)) {spin_unlock (vmf->ptl);folio_put (folio); -pte_free (vma->vm_mm, pgtable); +pte_free (vma->vm_mm,ptdesc_page (ptdesc)); ret =handle_userfault (vmf,VM_UFFD_MISSING );VM_BUG_ON (ret &VM_FAULT_FALLBACK );return ret;
@@ -997,7 +997,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, entry =maybe_pmd_mkwrite (pmd_mkdirty (entry), vma);folio_add_new_anon_rmap (folio, vma, haddr,RMAP_EXCLUSIVE );folio_add_lru_vma (folio, vma); -pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd, pgtable); +pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd,ptdesc_page (ptdesc));set_pmd_at (vma->vm_mm, haddr, vmf->pmd, entry);update_mmu_cache_pmd (vma, vmf->address, vmf->pmd);add_mm_counter (vma->vm_mm,MM_ANONPAGES ,HPAGE_PMD_NR );
@@ -1012,8 +1012,8 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, unlock_release:spin_unlock (vmf->ptl); release: -if (pgtable) -pte_free (vma->vm_mm, pgtable); +if (ptdesc) +pte_free (vma->vm_mm,ptdesc_page (ptdesc));folio_put (folio);return ret;
Signature · 1 line
2.43.0
RFC PATCH 5/18 mm/thp: use ptdesc in do_huge_pmd_anonymous_page
In reply to [email protected]From: Alex Shi <[email protected]>
ince we have ptdesc struct now, better to use replace pgtable_t, aka 'struct page *'. It's alaos a preparation for return ptdesc pointer in pte_alloc_one series function.
--- mm/huge_memory.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
mm/huge_memory.c +8 -8
@@ -1087,16 +1087,16 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf)if (!(vmf->flags &FAULT_FLAG_WRITE ) && !mm_forbids_zeropage (vma->vm_mm) &&transparent_hugepage_use_zero_page ()) { -pgtable_t pgtable; +struct ptdesc *ptdesc;struct folio *zero_folio;vm_fault_t ret; - pgtable =pte_alloc_one (vma->vm_mm); -if (unlikely (!pgtable)) + ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); +if (unlikely (!ptdesc))return VM_FAULT_OOM ; zero_folio =mm_get_huge_zero_folio (vma->vm_mm);if (unlikely (!zero_folio)) { -pte_free (vma->vm_mm, pgtable); +pte_free (vma->vm_mm,ptdesc_page (ptdesc));count_vm_event (THP_FAULT_FALLBACK );return VM_FAULT_FALLBACK ; }
@@ -1106,21 +1106,21 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) ret =check_stable_address_space (vma->vm_mm);if (ret) {spin_unlock (vmf->ptl); -pte_free (vma->vm_mm, pgtable); +pte_free (vma->vm_mm,ptdesc_page (ptdesc)); }else if (userfaultfd_missing (vma)) {spin_unlock (vmf->ptl); -pte_free (vma->vm_mm, pgtable); +pte_free (vma->vm_mm,ptdesc_page (ptdesc)); ret =handle_userfault (vmf,VM_UFFD_MISSING );VM_BUG_ON (ret &VM_FAULT_FALLBACK ); }else { -set_huge_zero_folio (pgtable, vma->vm_mm, vma, +set_huge_zero_folio (ptdesc_page (ptdesc), vma->vm_mm, vma, haddr, vmf->pmd, zero_folio);update_mmu_cache_pmd (vma, vmf->address, vmf->pmd);spin_unlock (vmf->ptl); } }else {spin_unlock (vmf->ptl); -pte_free (vma->vm_mm, pgtable); +pte_free (vma->vm_mm,ptdesc_page (ptdesc)); }return ret; }
Signature · 1 line
2.43.0
RFC PATCH 6/18 mm/thp: convert insert_pfn_pmd and its caller to use ptdesc
In reply to [email protected]From: Alex Shi <[email protected]>
Since we have ptdesc struct now, better to use replace pgtable_t, aka 'struct page *'. It's alaos a preparation for return ptdesc pointer in pte_alloc_one series function.
--- mm/huge_memory.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
mm/huge_memory.c +10 -10
@@ -1136,7 +1136,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf)static void insert_pfn_pmd (struct vm_area_struct *vma,unsigned long addr,pmd_t *pmd,pfn_t pfn,pgprot_t prot,bool write, -pgtable_t pgtable) +struct ptdesc *ptdesc) {struct mm_struct *mm = vma->vm_mm;pmd_t entry;
@@ -1166,10 +1166,10 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, entry =maybe_pmd_mkwrite (entry, vma); } -if (pgtable) { -pgtable_trans_huge_deposit (mm, pmd, pgtable); +if (ptdesc) { +pgtable_trans_huge_deposit (mm, pmd,ptdesc_page (ptdesc));mm_inc_nr_ptes (mm); - pgtable =NULL ; + ptdesc =NULL ; }set_pmd_at (mm, addr, pmd, entry);
@@ -1177,8 +1177,8 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, out_unlock:spin_unlock (ptl); -if (pgtable) -pte_free (mm, pgtable); +if (ptdesc) +pte_free (mm,ptdesc_page (ptdesc)); }/**
@@ -1196,7 +1196,7 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write)unsigned long addr = vmf->address &PMD_MASK ;struct vm_area_struct *vma = vmf->vma;pgprot_t pgprot = vma->vm_page_prot; -pgtable_t pgtable =NULL ; +struct ptdesc *ptdesc =NULL ;/* * If we had pmd_special, we could avoid all these restrictions,
@@ -1213,14 +1213,14 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write)return VM_FAULT_SIGBUS ;if (arch_needs_pgtable_deposit ()) { - pgtable =pte_alloc_one (vma->vm_mm); -if (!pgtable) + ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); +if (!ptdesc)return VM_FAULT_OOM ; }track_pfn_insert (vma, &pgprot, pfn); -insert_pfn_pmd (vma, addr, vmf->pmd, pfn, pgprot, write, pgtable); +insert_pfn_pmd (vma, addr, vmf->pmd, pfn, pgprot, write, ptdesc);return VM_FAULT_NOPAGE ; }EXPORT_SYMBOL_GPL (vmf_insert_pfn_pmd);
Signature · 1 line
2.43.0
RFC PATCH 7/18 mm/thp: use ptdesc in copy_huge_pmd
In reply to [email protected]From: Alex Shi <[email protected]>
Since we have ptdesc struct now, better to use replace pgtable_t, aka 'struct page *'. It's alaos a preparation for return ptdesc pointer in pte_alloc_one series function.
--- mm/huge_memory.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
mm/huge_memory.c +7 -7
@@ -1369,15 +1369,15 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,struct page *src_page;struct folio *src_folio;pmd_t pmd; -pgtable_t pgtable =NULL ; +struct ptdesc *ptdesc =NULL ;int ret = -ENOMEM ;/* Skip if can be re-fill on fault */ if (!vma_is_anonymous (dst_vma))return 0 ; - pgtable =pte_alloc_one (dst_mm); -if (unlikely (!pgtable)) + ptdesc =page_ptdesc (pte_alloc_one (dst_mm)); +if (unlikely (!ptdesc)) goto out; dst_ptl =pmd_lock (dst_mm, dst_pmd);
@@ -1404,7 +1404,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, }add_mm_counter (dst_mm,MM_ANONPAGES ,HPAGE_PMD_NR );mm_inc_nr_ptes (dst_mm); -pgtable_trans_huge_deposit (dst_mm, dst_pmd, pgtable); +pgtable_trans_huge_deposit (dst_mm, dst_pmd,ptdesc_page (ptdesc));if (!userfaultfd_wp (dst_vma)) pmd =pmd_swp_clear_uffd_wp (pmd);set_pmd_at (dst_mm, addr, dst_pmd, pmd);
@@ -1414,7 +1414,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,#endif if (unlikely (!pmd_trans_huge (pmd))) { -pte_free (dst_mm, pgtable); +pte_free (dst_mm,ptdesc_page (ptdesc)); goto out_unlock; }/*
@@ -1440,7 +1440,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,if (unlikely (folio_try_dup_anon_rmap_pmd (src_folio, src_page, src_vma))) {/* Page maybe pinned: split and retry the fault on PTEs. */ folio_put (src_folio); -pte_free (dst_mm, pgtable); +pte_free (dst_mm,ptdesc_page (ptdesc));spin_unlock (src_ptl);spin_unlock (dst_ptl);__split_huge_pmd (src_vma, src_pmd, addr, false,NULL );
@@ -1449,7 +1449,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,add_mm_counter (dst_mm,MM_ANONPAGES ,HPAGE_PMD_NR ); out_zero_page:mm_inc_nr_ptes (dst_mm); -pgtable_trans_huge_deposit (dst_mm, dst_pmd, pgtable); +pgtable_trans_huge_deposit (dst_mm, dst_pmd,ptdesc_page (ptdesc));pmdp_set_wrprotect (src_mm, addr, src_pmd);if (!userfaultfd_wp (dst_vma)) pmd =pmd_clear_uffd_wp (pmd);
Signature · 1 line
2.43.0
RFC PATCH 8/18 mm/memory: use ptdesc in __pte_alloc
In reply to [email protected]From: Alex Shi <[email protected]>
Replace pgtable_t by ptdesc in function __pte_alloc. We will remove pgtable_t from all place.
--- mm/memory.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
mm/memory.c +5 -5
@@ -445,13 +445,13 @@ void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte)int __pte_alloc (struct mm_struct *mm,pmd_t *pmd) { -pgtable_t new =pte_alloc_one (mm); -if (!new) +struct ptdesc *ptdesc =page_ptdesc (pte_alloc_one (mm)); +if (!ptdesc)return -ENOMEM ; -pmd_install (mm, pmd, &new); -if (new) -pte_free (mm, new); +pmd_install (mm, pmd, (pgtable_t *)&ptdesc); +if (ptdesc) +pte_free (mm,ptdesc_page (ptdesc));return 0 ; }
Signature · 1 line
2.43.0
RFC PATCH 9/18 mm/pgtable: fully use ptdesc in pte_alloc_one series functions
In reply to [email protected]From: Alex Shi <[email protected]>
Replace pgtable_t and struct page by ptdesc in pte_alloc_one series functions.
--- arch/arm/include/asm/pgalloc.h | 9 ++++----- arch/powerpc/include/asm/pgalloc.h | 4 ++-- arch/s390/include/asm/pgalloc.h | 2 +- arch/sparc/include/asm/pgalloc_32.h | 2 +- arch/sparc/include/asm/pgalloc_64.h | 2 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 4 ++-- arch/x86/include/asm/pgalloc.h | 2 +- arch/x86/mm/pgtable.c | 2 +- arch/xtensa/include/asm/pgalloc.h | 12 ++++++------ fs/dax.c | 2 +- include/asm-generic/pgalloc.h | 6 +++--- mm/huge_memory.c | 8 ++++---- mm/memory.c | 8 ++++---- 14 files changed, 32 insertions(+), 33 deletions(-)
arch/arm/include/asm/pgalloc.h +4 -5
@@ -91,16 +91,15 @@ pte_alloc_one_kernel(struct mm_struct *mm)#define PGTABLE_HIGHMEM 0#endif -static inline pgtable_t -pte_alloc_one (struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one (struct mm_struct *mm) { -struct page *pte; +struct ptdesc *pte; pte =__pte_alloc_one (mm,GFP_PGTABLE_USER |PGTABLE_HIGHMEM );if (!pte)return NULL ; -if (!PageHighMem (pte)) -clean_pte_table (page_address (pte)); +if (!PageHighMem (ptdesc_page (pte))) +clean_pte_table (ptdesc_address (pte));return pte; }
arch/powerpc/include/asm/pgalloc.h +2 -2
@@ -27,9 +27,9 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm)return (pte_t *)pte_fragment_alloc (mm,1 ); } -static inline pgtable_t pte_alloc_one (struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one (struct mm_struct *mm) { -return (pgtable_t )pte_fragment_alloc (mm,0 ); +return (struct ptdesc *)pte_fragment_alloc (mm,0 ); }void pte_frag_destroy (void *pte_frag);
arch/s390/include/asm/pgalloc.h +1 -1
@@ -137,7 +137,7 @@ static inline void pmd_populate(struct mm_struct *mm,* page table entry allocation/free routines. */ #define pte_alloc_one_kernel (mm) ((pte_t *)page_table_alloc(mm)) -#define pte_alloc_one (mm) ((pte_t *)page_table_alloc(mm)) +#define pte_alloc_one (mm) ((struct ptdesc *)page_table_alloc(mm))#define pte_free_kernel (mm, pte) page_table_free(mm, (unsigned long *) pte)#define pte_free (mm, pte) page_table_free(mm, (unsigned long *) pte)
arch/sparc/include/asm/pgalloc_32.h +1 -1
@@ -55,7 +55,7 @@ static inline void free_pmd_fast(pmd_t * pmd)void pmd_set (pmd_t *pmdp,pte_t *ptep);#define pmd_populate_kernel pmd_populate -pgtable_t pte_alloc_one (struct mm_struct *mm); +struct ptdesc *pte_alloc_one (struct mm_struct *mm);static inline pte_t *pte_alloc_one_kernel (struct mm_struct *mm) {
arch/sparc/include/asm/pgalloc_64.h +1 -1
@@ -61,7 +61,7 @@ static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) }pte_t *pte_alloc_one_kernel (struct mm_struct *mm); -pgtable_t pte_alloc_one (struct mm_struct *mm); +struct ptdesc *pte_alloc_one (struct mm_struct *mm);void pte_free_kernel (struct mm_struct *mm,pte_t *pte);void pte_free (struct mm_struct *mm,pgtable_t ptepage);
arch/sparc/mm/init_64.c +1 -1
@@ -2900,7 +2900,7 @@ pte_t *pte_alloc_one_kernel(struct mm_struct *mm)return pte; } -pgtable_t pte_alloc_one (struct mm_struct *mm) +struct ptdesc *pte_alloc_one (struct mm_struct *mm) {struct ptdesc *ptdesc =pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 );
arch/sparc/mm/srmmu.c +2 -2
@@ -346,7 +346,7 @@ pgd_t *get_pgd_fast(void)* Alignments up to the page size are the same for physical and virtual * addresses of the nocache area. */ -pgtable_t pte_alloc_one (struct mm_struct *mm) +struct ptdesc *pte_alloc_one (struct mm_struct *mm) {pte_t *ptep;struct page *page;
@@ -362,7 +362,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) }spin_unlock (&mm->page_table_lock); -return ptep; +return (struct ptdesc *)ptep; }void pte_free (struct mm_struct *mm,pgtable_t ptep)
arch/x86/include/asm/pgalloc.h +1 -1
@@ -51,7 +51,7 @@ extern gfp_t __userpte_alloc_gfp;extern pgd_t *pgd_alloc (struct mm_struct *);extern void pgd_free (struct mm_struct *mm,pgd_t *pgd); -extern pgtable_t pte_alloc_one (struct mm_struct *); +extern struct ptdesc *pte_alloc_one (struct mm_struct *);extern void ___pte_free_tlb (struct mmu_gather *tlb,struct page *pte);
arch/x86/mm/pgtable.c +1 -1
@@ -28,7 +28,7 @@ void paravirt_tlb_remove_table(struct mmu_gather *tlb, void *table)gfp_t __userpte_alloc_gfp =GFP_PGTABLE_USER |PGTABLE_HIGHMEM ; -pgtable_t pte_alloc_one (struct mm_struct *mm) +struct ptdesc *pte_alloc_one (struct mm_struct *mm) {return __pte_alloc_one (mm, __userpte_alloc_gfp); }
arch/xtensa/include/asm/pgalloc.h +6 -6
@@ -51,15 +51,15 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) return ptep; } -static inline pgtable_t pte_alloc_one (struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one (struct mm_struct *mm) { -struct page *page; +struct ptdesc *ptdesc; - page =__pte_alloc_one (mm,GFP_PGTABLE_USER ); -if (!page) + ptdesc =__pte_alloc_one (mm,GFP_PGTABLE_USER ); +if (!ptdesc)return NULL ; -ptes_clear (page_address (page)); -return page; +ptes_clear (ptdesc_address (ptdesc)); +return ptdesc; }#endif /* CONFIG_MMU */
fs/dax.c +1 -1
@@ -1222,7 +1222,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, DAX_PMD | DAX_ZERO_PAGE);if (arch_needs_pgtable_deposit ()) { - ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); + ptdesc =pte_alloc_one (vma->vm_mm);if (!ptdesc)return VM_FAULT_OOM ; }
include/asm-generic/pgalloc.h +3 -3
@@ -63,7 +63,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)* * Return: `struct page` referencing the ptdesc or %NULL on error */ -static inline pgtable_t __pte_alloc_one_noprof (struct mm_struct *mm,gfp_t gfp) +static inline struct ptdesc *__pte_alloc_one_noprof (struct mm_struct *mm,gfp_t gfp) {struct ptdesc *ptdesc;
@@ -75,7 +75,7 @@ static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp)return NULL ; } -return ptdesc_page (ptdesc); +return ptdesc; }#define __pte_alloc_one (...) alloc_hooks(__pte_alloc_one_noprof(__VA_ARGS__))
@@ -88,7 +88,7 @@ static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp)* * Return: `struct page` referencing the ptdesc or %NULL on error */ -static inline pgtable_t pte_alloc_one_noprof (struct mm_struct *mm) +static inline struct ptdesc *pte_alloc_one_noprof (struct mm_struct *mm) {return __pte_alloc_one_noprof (mm,GFP_PGTABLE_USER ); }
mm/huge_memory.c +4 -4
@@ -959,7 +959,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, }folio_throttle_swaprate (folio, gfp); - ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); + ptdesc =pte_alloc_one (vma->vm_mm);if (unlikely (!ptdesc)) { ret =VM_FAULT_OOM ; goto release;
@@ -1091,7 +1091,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf)struct folio *zero_folio;vm_fault_t ret; - ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); + ptdesc =pte_alloc_one (vma->vm_mm);if (unlikely (!ptdesc))return VM_FAULT_OOM ; zero_folio =mm_get_huge_zero_folio (vma->vm_mm);
@@ -1213,7 +1213,7 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write)return VM_FAULT_SIGBUS ;if (arch_needs_pgtable_deposit ()) { - ptdesc =page_ptdesc (pte_alloc_one (vma->vm_mm)); + ptdesc =pte_alloc_one (vma->vm_mm);if (!ptdesc)return VM_FAULT_OOM ; }
@@ -1376,7 +1376,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,if (!vma_is_anonymous (dst_vma))return 0 ; - ptdesc =page_ptdesc (pte_alloc_one (dst_mm)); + ptdesc =pte_alloc_one (dst_mm);if (unlikely (!ptdesc)) goto out;
mm/memory.c +4 -4
@@ -445,7 +445,7 @@ void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte)int __pte_alloc (struct mm_struct *mm,pmd_t *pmd) { -struct ptdesc *ptdesc =page_ptdesc (pte_alloc_one (mm)); +struct ptdesc *ptdesc =pte_alloc_one (mm);if (!ptdesc)return -ENOMEM ;
@@ -4647,7 +4647,7 @@ static vm_fault_t __do_fault(struct vm_fault *vmf)* # flush A, B to clear the writeback */ if (pmd_none (*vmf->pmd) && !vmf->prealloc_pte) { - vmf->prealloc_pte =pte_alloc_one (vma->vm_mm); + vmf->prealloc_pte =ptdesc_page (pte_alloc_one (vma->vm_mm));if (!vmf->prealloc_pte)return VM_FAULT_OOM ; }
@@ -4725,7 +4725,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page)* related to pte entry. Use the preallocated table for that. */ if (arch_needs_pgtable_deposit () && !vmf->prealloc_pte) { - vmf->prealloc_pte =pte_alloc_one (vma->vm_mm); + vmf->prealloc_pte =ptdesc_page (pte_alloc_one (vma->vm_mm));if (!vmf->prealloc_pte)return VM_FAULT_OOM ; }
@@ -5010,7 +5010,7 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf) pte_off + vma_pages(vmf->vma) - vma_off) - 1;if (pmd_none (*vmf->pmd)) { - vmf->prealloc_pte =pte_alloc_one (vmf->vma->vm_mm); + vmf->prealloc_pte =ptdesc_page (pte_alloc_one (vmf->vma->vm_mm));if (!vmf->prealloc_pte)return VM_FAULT_OOM ; }
Signature · 1 line
2.43.0
RFC PATCH 10/18 mm/pgtable: pass ptdesc to pte_free()
In reply to [email protected]From: Alex Shi <[email protected]>
Now we could remove couple of page<->ptdesc converters now.
--- arch/arm/mm/pgd.c | 2 +- arch/m68k/include/asm/motorola_pgalloc.h | 4 ++-- arch/powerpc/include/asm/book3s/64/pgalloc.h | 2 +- arch/powerpc/include/asm/pgalloc.h | 2 +- arch/sparc/include/asm/pgalloc_32.h | 2 +- arch/sparc/mm/srmmu.c | 2 +- fs/dax.c | 2 +- include/asm-generic/pgalloc.h | 4 +--- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 20 ++++++++++---------- mm/memory.c | 4 ++-- mm/pgtable-generic.c | 2 +- 12 files changed, 23 insertions(+), 25 deletions(-)
arch/arm/mm/pgd.c +1 -1
@@ -168,7 +168,7 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd_base) pte =pmd_pgtable (*pmd);pmd_clear (pmd); -pte_free (mm, pte); +pte_free (mm,page_ptdesc (pte));mm_dec_nr_ptes (mm); no_pmd:pud_clear (pud);
arch/m68k/include/asm/motorola_pgalloc.h +2 -2
@@ -39,9 +39,9 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm)return get_pointer_table (TABLE_PTE ); } -static inline void pte_free (struct mm_struct *mm,pgtable_t pgtable) +static inline void pte_free (struct mm_struct *mm,struct ptdesc *ptdesc) { -free_pointer_table (pgtable,TABLE_PTE ); +free_pointer_table (ptdesc_page (ptdesc),TABLE_PTE ); } static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable,
arch/powerpc/include/asm/book3s/64/pgalloc.h +1 -1
@@ -162,7 +162,7 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -pgtable_t pte_page) +struct ptdesc *pte_page) { *pmd =__pmd (__pgtable_ptr_val (pte_page) |PMD_VAL_BITS ); }
arch/powerpc/include/asm/pgalloc.h +1 -1
@@ -40,7 +40,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)pte_fragment_free ((unsigned long *)pte,1 ); } -static inline void pte_free (struct mm_struct *mm,pgtable_t ptepage) +static inline void pte_free (struct mm_struct *mm,struct ptdesc *ptepage) {pte_fragment_free ((unsigned long *)ptepage,0 ); }
arch/sparc/include/asm/pgalloc_32.h +1 -1
@@ -71,7 +71,7 @@ static inline void free_pte_fast(pte_t *pte)#define pte_free_kernel (mm, pte) free_pte_fast(pte) -void pte_free (struct mm_struct * mm,pgtable_t pte); +void pte_free (struct mm_struct *mm,struct ptdesc *pte);#define __pte_free_tlb (tlb, pte, addr) pte_free((tlb)->mm, pte)#endif /* _SPARC_PGALLOC_H */
arch/sparc/mm/srmmu.c +1 -1
@@ -365,7 +365,7 @@ struct ptdesc *pte_alloc_one(struct mm_struct *mm)return (struct ptdesc *)ptep; } -void pte_free (struct mm_struct *mm,pgtable_t ptep) +void pte_free (struct mm_struct *mm,struct ptdesc *ptep) {struct page *page;
fs/dax.c +1 -1
@@ -1246,7 +1246,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, fallback:if (ptdesc) -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc);trace_dax_pmd_load_hole_fallback (inode, vmf, zero_folio, *entry);return VM_FAULT_FALLBACK ; }
include/asm-generic/pgalloc.h +1 -3
@@ -105,10 +105,8 @@ static inline struct ptdesc *pte_alloc_one_noprof(struct mm_struct *mm)* @mm: the mm_struct of the current context * @pte_page: the `struct page` referencing the ptdesc */ -static inline void pte_free (struct mm_struct *mm,struct page *pte_page) +static inline void pte_free (struct mm_struct *mm,struct ptdesc *ptdesc) { -struct ptdesc *ptdesc =page_ptdesc (pte_page); -pagetable_pte_dtor (ptdesc);pagetable_free (ptdesc); }
mm/debug_vm_pgtable.c +1 -1
@@ -1049,7 +1049,7 @@ static void __init destroy_args(struct pgtable_debug_args *args)/* Free page table entries */ if (args->start_ptep) { -pte_free (args->mm, args->start_ptep); +pte_free (args->mm,page_ptdesc (args->start_ptep));mm_dec_nr_ptes (args->mm); }
mm/huge_memory.c +10 -10
@@ -987,7 +987,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,if (userfaultfd_missing (vma)) {spin_unlock (vmf->ptl);folio_put (folio); -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc); ret =handle_userfault (vmf,VM_UFFD_MISSING );VM_BUG_ON (ret &VM_FAULT_FALLBACK );return ret;
@@ -1013,7 +1013,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,spin_unlock (vmf->ptl); release:if (ptdesc) -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc);folio_put (folio);return ret;
@@ -1096,7 +1096,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf)return VM_FAULT_OOM ; zero_folio =mm_get_huge_zero_folio (vma->vm_mm);if (unlikely (!zero_folio)) { -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc);count_vm_event (THP_FAULT_FALLBACK );return VM_FAULT_FALLBACK ; }
@@ -1106,10 +1106,10 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) ret =check_stable_address_space (vma->vm_mm);if (ret) {spin_unlock (vmf->ptl); -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc); }else if (userfaultfd_missing (vma)) {spin_unlock (vmf->ptl); -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc); ret =handle_userfault (vmf,VM_UFFD_MISSING );VM_BUG_ON (ret &VM_FAULT_FALLBACK ); }else {
@@ -1120,7 +1120,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) } }else {spin_unlock (vmf->ptl); -pte_free (vma->vm_mm,ptdesc_page (ptdesc)); +pte_free (vma->vm_mm, ptdesc); }return ret; }
@@ -1178,7 +1178,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, out_unlock:spin_unlock (ptl);if (ptdesc) -pte_free (mm,ptdesc_page (ptdesc)); +pte_free (mm, ptdesc); }/**
@@ -1414,7 +1414,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,#endif if (unlikely (!pmd_trans_huge (pmd))) { -pte_free (dst_mm,ptdesc_page (ptdesc)); +pte_free (dst_mm, ptdesc); goto out_unlock; }/*
@@ -1440,7 +1440,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,if (unlikely (folio_try_dup_anon_rmap_pmd (src_folio, src_page, src_vma))) {/* Page maybe pinned: split and retry the fault on PTEs. */ folio_put (src_folio); -pte_free (dst_mm,ptdesc_page (ptdesc)); +pte_free (dst_mm, ptdesc);spin_unlock (src_ptl);spin_unlock (dst_ptl);__split_huge_pmd (src_vma, src_pmd, addr, false,NULL );
@@ -1830,7 +1830,7 @@ static inline void zap_deposited_table(struct mm_struct *mm, pmd_t *pmd)pgtable_t pgtable; pgtable =pgtable_trans_huge_withdraw (mm, pmd); -pte_free (mm, pgtable); +pte_free (mm,page_ptdesc (pgtable));mm_dec_nr_ptes (mm); }
mm/memory.c +2 -2
@@ -451,7 +451,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd)pmd_install (mm, pmd, (pgtable_t *)&ptdesc);if (ptdesc) -pte_free (mm,ptdesc_page (ptdesc)); +pte_free (mm, ptdesc);return 0 ; }
@@ -5196,7 +5196,7 @@ static vm_fault_t do_fault(struct vm_fault *vmf)/* preallocated pagetable is unused: free it */ if (vmf->prealloc_pte) { -pte_free (vm_mm, vmf->prealloc_pte); +pte_free (vm_mm,page_ptdesc (vmf->prealloc_pte)); vmf->prealloc_pte =NULL ; }return ret;
mm/pgtable-generic.c +1 -1
@@ -241,7 +241,7 @@ static void pte_free_now(struct rcu_head *head)struct ptdesc *ptdesc; ptdesc = container_of(head, struct ptdesc, pt_rcu_head); -pte_free (NULL /* mm not passed and not used */ , (pgtable_t )ptdesc); +pte_free (NULL /* mm not passed and not used */ , ptdesc); }void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable)
Signature · 1 line
2.43.0
RFC PATCH 11/18 mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range()
In reply to [email protected]From: Alex Shi <[email protected]>
Replace pgtable_t by ptdesc in free_pte_range and it's callee pte_free_tlb series functions. And save some converters now. We have to use type casting for pmd_pgtable() instead of page_ptdesc() helper since different arch has different type of pgtable_t.
btw, we can not simplify pmd_ptdesc() via replace pmd_pgtable_page by pmd_page, since some arch may have no pmd_page yet.
--- arch/arm/include/asm/tlb.h | 4 +--- arch/arm64/include/asm/tlb.h | 4 +--- arch/csky/include/asm/pgalloc.h | 4 ++-- arch/hexagon/include/asm/pgalloc.h | 4 ++-- arch/loongarch/include/asm/pgalloc.h | 4 ++-- arch/m68k/include/asm/motorola_pgalloc.h | 4 ++-- arch/openrisc/include/asm/pgalloc.h | 4 ++-- arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 2 +- arch/riscv/include/asm/pgalloc.h | 8 +++----- arch/x86/include/asm/pgalloc.h | 4 ++-- arch/x86/mm/pgtable.c | 6 +++--- include/linux/mm.h | 14 ++++++++++++++ mm/memory.c | 3 ++- 14 files changed, 38 insertions(+), 29 deletions(-)
arch/arm/include/asm/tlb.h +1 -3
@@ -37,10 +37,8 @@ static inline void __tlb_remove_table(void *_table)#include <asm-generic/tlb.h> static inline void -__pte_free_tlb (struct mmu_gather *tlb,pgtable_t pte,unsigned long addr) +__pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *ptdesc,unsigned long addr) { -struct ptdesc *ptdesc =page_ptdesc (pte); -pagetable_pte_dtor (ptdesc);#ifndef CONFIG_ARM_LPAE
arch/arm64/include/asm/tlb.h +1 -3
@@ -77,11 +77,9 @@ static inline void tlb_flush(struct mmu_gather *tlb) last_level, tlb_level); } -static inline void __pte_free_tlb (struct mmu_gather *tlb,pgtable_t pte, +static inline void __pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *ptdesc,unsigned long addr) { -struct ptdesc *ptdesc =page_ptdesc (pte); -pagetable_pte_dtor (ptdesc);tlb_remove_ptdesc (tlb, ptdesc); }
arch/csky/include/asm/pgalloc.h +2 -2
@@ -63,8 +63,8 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm)#define __pte_free_tlb (tlb, pte, address) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc(tlb, page_ptdesc(pte)); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc(tlb, pte); \ } while (0)extern void pagetable_init (void );
arch/hexagon/include/asm/pgalloc.h +2 -2
@@ -89,8 +89,8 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd,#define __pte_free_tlb (tlb, pte, addr) \ do { \ - pagetable_pte_dtor((page_ptdesc(pte))); \ - tlb_remove_page_ptdesc((tlb), (page_ptdesc(pte))); \ + pagetable_pte_dtor((pte)); \ + tlb_remove_page_ptdesc((tlb), (pte)); \ } while (0)#endif
arch/loongarch/include/asm/pgalloc.h +2 -2
@@ -46,8 +46,8 @@ extern pgd_t *pgd_alloc(struct mm_struct *mm);#define __pte_free_tlb (tlb, pte, address) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc((tlb), page_ptdesc(pte)); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc((tlb), pte); \ } while (0)#ifndef __PAGETABLE_PMD_FOLDED
arch/m68k/include/asm/motorola_pgalloc.h +2 -2
@@ -44,10 +44,10 @@ static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptdesc)free_pointer_table (ptdesc_page (ptdesc),TABLE_PTE ); } -static inline void __pte_free_tlb (struct mmu_gather *tlb,pgtable_t pgtable, +static inline void __pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *ptdesc,unsigned long address) { -free_pointer_table (pgtable,TABLE_PTE ); +free_pointer_table (ptdesc_page (ptdesc),TABLE_PTE ); }
arch/openrisc/include/asm/pgalloc.h +2 -2
@@ -68,8 +68,8 @@ extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm);#define __pte_free_tlb (tlb, pte, addr) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc((tlb), (page_ptdesc(pte))); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc((tlb), (pte)); \ } while (0)#endif
arch/powerpc/include/asm/book3s/32/pgalloc.h +1 -1
@@ -64,7 +64,7 @@ static inline void __tlb_remove_table(void *_table)pgtable_free (table, shift); } -static inline void __pte_free_tlb (struct mmu_gather *tlb,pgtable_t table, +static inline void __pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *table,unsigned long address) {pgtable_free_tlb (tlb, table,0 );
arch/powerpc/include/asm/book3s/64/pgalloc.h +1 -1
@@ -167,7 +167,7 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, *pmd =__pmd (__pgtable_ptr_val (pte_page) |PMD_VAL_BITS ); } -static inline void __pte_free_tlb (struct mmu_gather *tlb,pgtable_t table, +static inline void __pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *table,unsigned long address) {pgtable_free_tlb (tlb, table,PTE_INDEX );
arch/riscv/include/asm/pgalloc.h +3 -5
@@ -183,13 +183,11 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd,#endif /* __PAGETABLE_PMD_FOLDED */ -static inline void __pte_free_tlb (struct mmu_gather *tlb,pgtable_t pte, +static inline void __pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *pte,unsigned long addr) { -struct ptdesc *ptdesc =page_ptdesc (pte); - -pagetable_pte_dtor (ptdesc); -riscv_tlb_remove_ptdesc (tlb, ptdesc); +pagetable_pte_dtor (pte); +riscv_tlb_remove_ptdesc (tlb, pte); }#endif /* CONFIG_MMU */
arch/x86/include/asm/pgalloc.h +2 -2
@@ -53,9 +53,9 @@ extern void pgd_free(struct mm_struct *mm, pgd_t *pgd);extern struct ptdesc *pte_alloc_one (struct mm_struct *); -extern void ___pte_free_tlb (struct mmu_gather *tlb,struct page *pte); +extern void ___pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *pte); -static inline void __pte_free_tlb (struct mmu_gather *tlb,struct page *pte, +static inline void __pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *pte,unsigned long address) {___pte_free_tlb (tlb, pte);
arch/x86/mm/pgtable.c +3 -3
@@ -50,10 +50,10 @@ static int __init setup_userpte(char *arg) }early_param ("userpte" , setup_userpte); -void ___pte_free_tlb (struct mmu_gather *tlb,struct page *pte) +void ___pte_free_tlb (struct mmu_gather *tlb,struct ptdesc *pte) { -pagetable_pte_dtor (page_ptdesc (pte)); -paravirt_release_pte (page_to_pfn (pte)); +pagetable_pte_dtor (pte); +paravirt_release_pte (ptdesc_pfn (pte));paravirt_tlb_remove_table (tlb, pte); }
include/linux/mm.h +14 -0
@@ -2859,6 +2859,20 @@ static inline bool pagetable_is_reserved(struct ptdesc *pt)return folio_test_reserved (ptdesc_folio (pt)); } +/** +* ptdesc_pfn - Return the Page Frame Number of a ptdesc. +* @ptdesc: The ptdesc. +* +* A ptdesc may contain multiple pages. The pages have consecutive +* Page Frame Numbers. +* +* Return: The Page Frame Number of the first page in the ptdesc. +*/ +static inline unsigned long ptdesc_pfn (struct ptdesc *ptdesc) +{ +return page_to_pfn (ptdesc_page (ptdesc)); +} +/** * pagetable_alloc - Allocate pagetables * @gfp: GFP flags
mm/memory.c +2 -1
@@ -189,7 +189,8 @@ void mm_trace_rss_stat(struct mm_struct *mm, int member)static void free_pte_range (struct mmu_gather *tlb,pmd_t *pmd,unsigned long addr) { -pgtable_t token =pmd_pgtable (*pmd); +struct ptdesc *token = (struct ptdesc *)pmd_pgtable (*pmd); +pmd_clear (pmd);pte_free_tlb (tlb, token, addr);mm_dec_nr_ptes (tlb->mm);
Signature · 1 line
2.43.0
RFC PATCH 12/18 mm/thp: pass ptdesc to set_huge_zero_folio function
From: Alex Shi <[email protected]>
Aim is still replace struct page to ptdesc.
--- mm/huge_memory.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
mm/huge_memory.c +3 -3
@@ -1055,7 +1055,7 @@ gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma) }/* Caller must hold page table lock. */ -static void set_huge_zero_folio (pgtable_t pgtable,struct mm_struct *mm, +static void set_huge_zero_folio (struct ptdesc *ptdesc,struct mm_struct *mm,struct vm_area_struct *vma,unsigned long haddr,pmd_t *pmd,struct folio *zero_folio) {
@@ -1064,7 +1064,7 @@ static void set_huge_zero_folio(pgtable_t pgtable, struct mm_struct *mm,return ; entry =mk_pmd (&zero_folio->page, vma->vm_page_prot); entry =pmd_mkhuge (entry); -pgtable_trans_huge_deposit (mm, pmd, pgtable); +pgtable_trans_huge_deposit (mm, pmd,ptdesc_page (ptdesc));set_pmd_at (mm, haddr, pmd, entry);mm_inc_nr_ptes (mm); }
@@ -1113,7 +1113,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) ret =handle_userfault (vmf,VM_UFFD_MISSING );VM_BUG_ON (ret &VM_FAULT_FALLBACK ); }else { -set_huge_zero_folio (ptdesc_page (ptdesc), vma->vm_mm, vma, +set_huge_zero_folio (ptdesc, vma->vm_mm, vma, haddr, vmf->pmd, zero_folio);update_mmu_cache_pmd (vma, vmf->address, vmf->pmd);spin_unlock (vmf->ptl);
Signature · 1 line
2.43.0
RFC PATCH 13/18 mm/pgtable: return ptdesc pointer in pgtable_trans_huge_withdraw
In reply to [email protected]From: Alex Shi <[email protected]>
Way to replace pgtable_t aka struct page in most of archs.
--- arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +-- arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +-- arch/powerpc/include/asm/book3s/64/pgtable.h | 2 +- arch/powerpc/include/asm/book3s/64/radix.h | 4 +-- arch/powerpc/mm/book3s64/hash_pgtable.c | 4 +-- arch/powerpc/mm/book3s64/radix_pgtable.c | 4 +-- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/mm/pgtable.c | 4 +-- arch/sparc/include/asm/pgtable_64.h | 2 +- arch/sparc/mm/tlb.c | 4 +-- include/linux/pgtable.h | 2 +- mm/huge_memory.c | 35 ++++++++++--------- mm/pgtable-generic.c | 4 +-- 13 files changed, 38 insertions(+), 37 deletions(-)
arch/powerpc/include/asm/book3s/64/hash-4k.h +2 -2
@@ -133,8 +133,8 @@ extern unsigned long hash__pmd_hugepage_update(struct mm_struct *mm,extern pmd_t hash__pmdp_collapse_flush (struct vm_area_struct *vma,unsigned long address,pmd_t *pmdp);extern void hash__pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable); -extern pgtable_t hash__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp); +struct ptdesc *ptdesc); +extern struct ptdesc *hash__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);extern pmd_t hash__pmdp_huge_get_and_clear (struct mm_struct *mm,unsigned long addr,pmd_t *pmdp);extern int hash__has_transparent_hugepage (void );
arch/powerpc/include/asm/book3s/64/hash-64k.h +2 -2
@@ -274,8 +274,8 @@ extern unsigned long hash__pmd_hugepage_update(struct mm_struct *mm,extern pmd_t hash__pmdp_collapse_flush (struct vm_area_struct *vma,unsigned long address,pmd_t *pmdp);extern void hash__pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable); -extern pgtable_t hash__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp); +struct ptdesc *ptdesc); +extern struct ptdesc *hash__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);extern pmd_t hash__pmdp_huge_get_and_clear (struct mm_struct *mm,unsigned long addr,pmd_t *pmdp);extern int hash__has_transparent_hugepage (void );
arch/powerpc/include/asm/book3s/64/pgtable.h +1 -1
@@ -1373,7 +1373,7 @@ static inline void pgtable_trans_huge_deposit(struct mm_struct *mm, }#define __HAVE_ARCH_PGTABLE_WITHDRAW -static inline pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm, +static inline struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) {if (radix_enabled ())
arch/powerpc/include/asm/book3s/64/radix.h +2 -2
@@ -291,8 +291,8 @@ extern unsigned long radix__pud_hugepage_update(struct mm_struct *mm, unsigned lextern pmd_t radix__pmdp_collapse_flush (struct vm_area_struct *vma,unsigned long address,pmd_t *pmdp);extern void radix__pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable); -extern pgtable_t radix__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp); +struct ptdesc *ptdesc); +extern struct ptdesc *radix__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);extern pmd_t radix__pmdp_huge_get_and_clear (struct mm_struct *mm,unsigned long addr,pmd_t *pmdp); pud_t radix__pudp_huge_get_and_clear(struct mm_struct *mm,
arch/powerpc/mm/book3s64/hash_pgtable.c +2 -2
@@ -284,7 +284,7 @@ void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,smp_wmb (); } -pgtable_t hash__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) +struct ptdesc *hash__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) {pgtable_t pgtable;pgtable_t *pgtable_slot;
@@ -302,7 +302,7 @@ pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)* zero out the content on withdraw. */ memset (pgtable,0 ,PTE_FRAG_SIZE ); -return pgtable; +return (struct ptdesc *)pgtable; }/*
arch/powerpc/mm/book3s64/radix_pgtable.c +2 -2
@@ -1492,7 +1492,7 @@ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,pmd_huge_pte (mm, pmdp) = pgtable; } -pgtable_t radix__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) +struct ptdesc *radix__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) {pte_t *ptep;pgtable_t pgtable;
@@ -1513,7 +1513,7 @@ pgtable_t radix__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) *ptep =__pte (0 ); ptep++; *ptep =__pte (0 ); -return pgtable; +return (struct ptdesc *)pgtable; } pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm,
arch/s390/include/asm/pgtable.h +1 -1
@@ -1738,7 +1738,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pgtable_t pgtable);#define __HAVE_ARCH_PGTABLE_WITHDRAW -pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp); +struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);#define __HAVE_ARCH_PMDP_SET_ACCESS_FLAGS static inline int pmdp_set_access_flags(struct vm_area_struct *vma,
arch/s390/mm/pgtable.c +2 -2
@@ -577,7 +577,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,pmd_huge_pte (mm, pmdp) = (struct ptdesc *)pgtable; } -pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) +struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) {struct list_head *lh;pgtable_t pgtable;
@@ -598,7 +598,7 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)set_pte (ptep,__pte (_PAGE_INVALID)); ptep++;set_pte (ptep,__pte (_PAGE_INVALID)); - return pgtable; + return (struct ptdesc *)pgtable; }#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
arch/sparc/include/asm/pgtable_64.h +1 -1
@@ -998,7 +998,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, pgtable_t pgtable);#define __HAVE_ARCH_PGTABLE_WITHDRAW -pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp); +struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);#endif /*
arch/sparc/mm/tlb.c +2 -2
@@ -281,7 +281,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,pmd_huge_pte (mm, pmdp) = (struct ptdesc *)pgtable; } -pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) +struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) {struct list_head *lh;pgtable_t pgtable;
@@ -300,6 +300,6 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp)pte_val (pgtable[0 ]) =0 ;pte_val (pgtable[1 ]) =0 ; - return pgtable; + return (struct ptdesc *)pgtable; }#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
include/linux/pgtable.h +1 -1
@@ -929,7 +929,7 @@ extern void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,#endif #ifndef __HAVE_ARCH_PGTABLE_WITHDRAW -extern pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp); +extern struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);#endif #ifndef arch_needs_pgtable_deposit
mm/huge_memory.c +18 -17
@@ -1827,10 +1827,10 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma,static inline void zap_deposited_table (struct mm_struct *mm,pmd_t *pmd) { -pgtable_t pgtable; +struct ptdesc *ptdesc; - pgtable =pgtable_trans_huge_withdraw (mm, pmd); -pte_free (mm,page_ptdesc (pgtable)); + ptdesc =pgtable_trans_huge_withdraw (mm, pmd); +pte_free (mm, ptdesc);mm_dec_nr_ptes (mm); }
@@ -1959,9 +1959,10 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,VM_BUG_ON (!pmd_none (*new_pmd));if (pmd_move_must_withdraw (new_ptl, old_ptl, vma)) { -pgtable_t pgtable; - pgtable =pgtable_trans_huge_withdraw (mm, old_pmd); -pgtable_trans_huge_deposit (mm, new_pmd, pgtable); +struct ptdesc *ptdesc; + + ptdesc =pgtable_trans_huge_withdraw (mm, old_pmd); +pgtable_trans_huge_deposit (mm, new_pmd,ptdesc_page (ptdesc)); } pmd =move_soft_dirty_pmd (pmd);set_pmd_at (mm, new_addr, new_pmd, pmd);
@@ -2130,7 +2131,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pmstruct folio *src_folio;struct anon_vma *src_anon_vma;spinlock_t *src_ptl, *dst_ptl; -pgtable_t src_pgtable; +struct ptdesc *src_ptdesc;struct mmu_notifier_range range;int err =0 ;
@@ -2234,8 +2235,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm }set_pmd_at (mm, dst_addr, dst_pmd, _dst_pmd); - src_pgtable =pgtable_trans_huge_withdraw (mm, src_pmd); -pgtable_trans_huge_deposit (mm, dst_pmd, src_pgtable); + src_ptdesc =pgtable_trans_huge_withdraw (mm, src_pmd); +pgtable_trans_huge_deposit (mm, dst_pmd,ptdesc_page (src_ptdesc)); unlock_ptls:double_pt_unlock (src_ptl, dst_ptl);if (src_anon_vma) {
@@ -2347,7 +2348,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, unsigned long haddr, pmd_t *pmd) { struct mm_struct *mm = vma->vm_mm; -pgtable_t pgtable; +struct ptdesc *ptdesc;pmd_t _pmd, old_pmd;unsigned long addr;pte_t *pte;
@@ -2363,8 +2364,8 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma,*/ old_pmd =pmdp_huge_clear_flush (vma, haddr, pmd); - pgtable =pgtable_trans_huge_withdraw (mm, pmd); -pmd_populate (mm, &_pmd, pgtable); + ptdesc =pgtable_trans_huge_withdraw (mm, pmd); +pmd_populate (mm, &_pmd,ptdesc_page (ptdesc)); pte =pte_offset_map (&_pmd, haddr);VM_BUG_ON (!pte);
@@ -2381,7 +2382,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, }pte_unmap (pte -1 );smp_wmb ();/* make pte visible before pmd */ -pmd_populate (mm, pmd, pgtable); +pmd_populate (mm, pmd,ptdesc_page (ptdesc)); } static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
@@ -2390,7 +2391,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,struct mm_struct *mm = vma->vm_mm;struct folio *folio;struct page *page; -pgtable_t pgtable; +struct ptdesc *ptdesc;pmd_t old_pmd, _pmd;bool young, write, soft_dirty, pmd_migration = false, uffd_wp = false;bool anon_exclusive = false, dirty = false;
@@ -2535,8 +2536,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,* Withdraw the table only after we mark the pmd entry invalid. * This's critical for some architectures (Power). */ - pgtable =pgtable_trans_huge_withdraw (mm, pmd); -pmd_populate (mm, &_pmd, pgtable); + ptdesc =pgtable_trans_huge_withdraw (mm, pmd); +pmd_populate (mm, &_pmd,ptdesc_page (ptdesc)); pte =pte_offset_map (&_pmd, haddr);VM_BUG_ON (!pte);
@@ -2601,7 +2602,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,put_page (page);smp_wmb ();/* make pte visible before pmd */ -pmd_populate (mm, pmd, pgtable); +pmd_populate (mm, pmd,ptdesc_page (ptdesc)); } void split_huge_pmd_locked(struct vm_area_struct *vma, unsigned long address,
mm/pgtable-generic.c +2 -2
@@ -178,7 +178,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,#ifndef __HAVE_ARCH_PGTABLE_WITHDRAW/* no "address" argument so destroys page coloring of some arch */ -pgtable_t pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) +struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp) {struct ptdesc *ptdesc;
@@ -190,7 +190,7 @@ pgtable_t pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp) struct ptdesc, pt_list);if (pmd_huge_pte (mm, pmdp))list_del (&ptdesc->pt_list); - return ptdesc_page(ptdesc); + return ptdesc; }#endif
Signature · 1 line
2.43.0
RFC PATCH 14/18 mm/pgtable: use ptdesc in pgtable_trans_huge_deposit
In reply to [email protected]From: Alex Shi <[email protected]>
A step to replace pgtable_t to struct ptdesc.
--- arch/powerpc/include/asm/book3s/64/pgtable.h | 6 +++--- arch/powerpc/mm/book3s64/hash_pgtable.c | 6 +++--- arch/powerpc/mm/book3s64/radix_pgtable.c | 6 +++--- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/mm/pgtable.c | 6 +++--- arch/sparc/include/asm/pgtable_64.h | 2 +- arch/sparc/mm/tlb.c | 6 +++--- fs/dax.c | 2 +- include/linux/pgtable.h | 2 +- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 14 +++++++------- mm/khugepaged.c | 2 +- mm/memory.c | 2 +- mm/pgtable-generic.c | 8 ++++---- 14 files changed, 33 insertions(+), 33 deletions(-)
arch/powerpc/include/asm/book3s/64/pgtable.h +3 -3
@@ -1365,11 +1365,11 @@ pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma,#define __HAVE_ARCH_PGTABLE_DEPOSITstatic inline void pgtable_trans_huge_deposit (struct mm_struct *mm, -pmd_t *pmdp,pgtable_t pgtable) +pmd_t *pmdp,struct ptdesc *ptdesc) {if (radix_enabled ()) -return radix__pgtable_trans_huge_deposit (mm, pmdp, pgtable); -return hash__pgtable_trans_huge_deposit (mm, pmdp, pgtable); +return radix__pgtable_trans_huge_deposit (mm, pmdp, ptdesc); +return hash__pgtable_trans_huge_deposit (mm, pmdp, ptdesc); }#define __HAVE_ARCH_PGTABLE_WITHDRAW
arch/powerpc/mm/book3s64/hash_pgtable.c +3 -3
@@ -265,16 +265,16 @@ pmd_t hash__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addres* the base page size hptes */ void hash__pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable) +struct ptdesc *ptdesc) { -pgtable_t *pgtable_slot; +pte_t **pgtable_slot;assert_spin_locked (pmd_lockptr (mm, pmdp));/* * we store the pgtable in the second half of PMD */ pgtable_slot = (pgtable_t *)pmdp +PTRS_PER_PMD ; - *pgtable_slot = pgtable; + *pgtable_slot = (pte_t )ptdesc;/* * expose the deposited pgtable to other cpus. * before we set the hugepage PTE at pmd level
arch/powerpc/mm/book3s64/radix_pgtable.c +3 -3
@@ -1478,9 +1478,9 @@ pmd_t radix__pmdp_collapse_flush(struct vm_area_struct *vma, unsigned long addre* list_head memory area. */ void radix__pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable) +struct ptdesc *ptdesc) { -struct list_head *lh = (struct list_head *) pgtable; +struct list_head *lh = (struct list_head *)ptdesc;assert_spin_locked (pmd_lockptr (mm, pmdp));
@@ -1489,7 +1489,7 @@ void radix__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,INIT_LIST_HEAD (lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); -pmd_huge_pte (mm, pmdp) = pgtable; +pmd_huge_pte (mm, pmdp) =ptdesc_page (ptdesc); }struct ptdesc *radix__pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp)
arch/s390/include/asm/pgtable.h +1 -1
@@ -1735,7 +1735,7 @@ pud_t pudp_xchg_direct(struct mm_struct *, unsigned long, pud_t *, pud_t);#define __HAVE_ARCH_PGTABLE_DEPOSITvoid pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable); +struct ptdesc *ptdesc);#define __HAVE_ARCH_PGTABLE_WITHDRAWstruct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);
arch/s390/mm/pgtable.c +3 -3
@@ -563,9 +563,9 @@ EXPORT_SYMBOL(pudp_xchg_direct);#ifdef CONFIG_TRANSPARENT_HUGEPAGE void pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable) +struct ptdesc *ptdesc) { -struct list_head *lh = (struct list_head *) pgtable; +struct list_head *lh = (struct list_head *)ptdesc;assert_spin_locked (pmd_lockptr (mm, pmdp));
@@ -574,7 +574,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,INIT_LIST_HEAD (lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); -pmd_huge_pte (mm, pmdp) = (struct ptdesc *)pgtable; +pmd_huge_pte (mm, pmdp) = ptdesc; }struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp)
arch/sparc/include/asm/pgtable_64.h +1 -1
@@ -995,7 +995,7 @@ extern pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,#define __HAVE_ARCH_PGTABLE_DEPOSITvoid pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable); +struct ptdesc *ptdesc);#define __HAVE_ARCH_PGTABLE_WITHDRAWstruct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp);
arch/sparc/mm/tlb.c +3 -3
@@ -267,9 +267,9 @@ pmd_t pmdp_invalidate(struct vm_area_struct *vma, unsigned long address, }void pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable) +struct ptdesc *ptdesc) { -struct list_head *lh = (struct list_head *) pgtable; +struct list_head *lh = (struct list_head *)ptdesc;assert_spin_locked (&mm->page_table_lock);
@@ -278,7 +278,7 @@ void pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,INIT_LIST_HEAD (lh); else list_add(lh, (struct list_head *) pmd_huge_pte(mm, pmdp)); -pmd_huge_pte (mm, pmdp) = (struct ptdesc *)pgtable; +pmd_huge_pte (mm, pmdp) = ptdesc; }struct ptdesc *pgtable_trans_huge_withdraw (struct mm_struct *mm,pmd_t *pmdp)
fs/dax.c +1 -1
@@ -1234,7 +1234,7 @@ static vm_fault_t dax_pmd_load_hole(struct xa_state *xas, struct vm_fault *vmf, }if (ptdesc) { -pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd, ptdesc);mm_inc_nr_ptes (vma->vm_mm); } pmd_entry =mk_pmd (&zero_folio->page, vmf->vma->vm_page_prot);
include/linux/pgtable.h +1 -1
@@ -925,7 +925,7 @@ static inline pmd_t pmdp_collapse_flush(struct vm_area_struct *vma,#ifndef __HAVE_ARCH_PGTABLE_DEPOSITextern void pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable); +struct ptdesc *ptdesc);#endif #ifndef __HAVE_ARCH_PGTABLE_WITHDRAW
mm/debug_vm_pgtable.c +1 -1
@@ -225,7 +225,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args)/* Align the address wrt HPAGE_PMD_SIZE */ vaddr &=HPAGE_PMD_MASK ; -pgtable_trans_huge_deposit (args->mm, args->pmdp, args->start_ptep); +pgtable_trans_huge_deposit (args->mm, args->pmdp,page_ptdesc (args->start_ptep)); pmd =pfn_pmd (args->pmd_pfn, args->page_prot);set_pmd_at (args->mm, vaddr, args->pmdp, pmd);
mm/huge_memory.c +7 -7
@@ -997,7 +997,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, entry =maybe_pmd_mkwrite (pmd_mkdirty (entry), vma);folio_add_new_anon_rmap (folio, vma, haddr,RMAP_EXCLUSIVE );folio_add_lru_vma (folio, vma); -pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd, ptdesc);set_pmd_at (vma->vm_mm, haddr, vmf->pmd, entry);update_mmu_cache_pmd (vma, vmf->address, vmf->pmd);add_mm_counter (vma->vm_mm,MM_ANONPAGES ,HPAGE_PMD_NR );
@@ -1064,7 +1064,7 @@ static void set_huge_zero_folio(struct ptdesc *ptdesc, struct mm_struct *mm,return ; entry =mk_pmd (&zero_folio->page, vma->vm_page_prot); entry =pmd_mkhuge (entry); -pgtable_trans_huge_deposit (mm, pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (mm, pmd, ptdesc);set_pmd_at (mm, haddr, pmd, entry);mm_inc_nr_ptes (mm); }
@@ -1167,7 +1167,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, }if (ptdesc) { -pgtable_trans_huge_deposit (mm, pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (mm, pmd, ptdesc);mm_inc_nr_ptes (mm); ptdesc =NULL ; }
@@ -1404,7 +1404,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, }add_mm_counter (dst_mm,MM_ANONPAGES ,HPAGE_PMD_NR );mm_inc_nr_ptes (dst_mm); -pgtable_trans_huge_deposit (dst_mm, dst_pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (dst_mm, dst_pmd, ptdesc);if (!userfaultfd_wp (dst_vma)) pmd =pmd_swp_clear_uffd_wp (pmd);set_pmd_at (dst_mm, addr, dst_pmd, pmd);
@@ -1449,7 +1449,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,add_mm_counter (dst_mm,MM_ANONPAGES ,HPAGE_PMD_NR ); out_zero_page:mm_inc_nr_ptes (dst_mm); -pgtable_trans_huge_deposit (dst_mm, dst_pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (dst_mm, dst_pmd, ptdesc);pmdp_set_wrprotect (src_mm, addr, src_pmd);if (!userfaultfd_wp (dst_vma)) pmd =pmd_clear_uffd_wp (pmd);
@@ -1962,7 +1962,7 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr,struct ptdesc *ptdesc; ptdesc =pgtable_trans_huge_withdraw (mm, old_pmd); -pgtable_trans_huge_deposit (mm, new_pmd,ptdesc_page (ptdesc)); +pgtable_trans_huge_deposit (mm, new_pmd, ptdesc); } pmd =move_soft_dirty_pmd (pmd);set_pmd_at (mm, new_addr, new_pmd, pmd);
@@ -2236,7 +2236,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pmset_pmd_at (mm, dst_addr, dst_pmd, _dst_pmd); src_ptdesc =pgtable_trans_huge_withdraw (mm, src_pmd); -pgtable_trans_huge_deposit (mm, dst_pmd,ptdesc_page (src_ptdesc)); +pgtable_trans_huge_deposit (mm, dst_pmd, src_ptdesc); unlock_ptls:double_pt_unlock (src_ptl, dst_ptl);if (src_anon_vma) {
mm/khugepaged.c +1 -1
@@ -1232,7 +1232,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,BUG_ON (!pmd_none (*pmd));folio_add_new_anon_rmap (folio, vma, address,RMAP_EXCLUSIVE );folio_add_lru_vma (folio, vma); -pgtable_trans_huge_deposit (mm, pmd, pgtable); +pgtable_trans_huge_deposit (mm, pmd,page_ptdesc (pgtable));set_pmd_at (mm, address, pmd, _pmd);update_mmu_cache_pmd (vma, address, pmd);spin_unlock (pmd_ptl);
mm/memory.c +1 -1
@@ -4687,7 +4687,7 @@ static void deposit_prealloc_pte(struct vm_fault *vmf) {struct vm_area_struct *vma = vmf->vma; -pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd, vmf->prealloc_pte); +pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd,page_ptdesc (vmf->prealloc_pte));/* * We are going to consume the prealloc table, * count that as nr_ptes.
mm/pgtable-generic.c +4 -4
@@ -163,16 +163,16 @@ pud_t pudp_huge_clear_flush(struct vm_area_struct *vma, unsigned long address,#ifndef __HAVE_ARCH_PGTABLE_DEPOSITvoid pgtable_trans_huge_deposit (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pgtable) +struct ptdesc *ptdesc) {assert_spin_locked (pmd_lockptr (mm, pmdp));/* FIFO */ if (!pmd_huge_pte (mm, pmdp)) -INIT_LIST_HEAD (&pgtable->lru); +INIT_LIST_HEAD (&ptdesc->pt_list);else -list_add (&pgtable->lru, &pmd_huge_pte (mm, pmdp)->pt_list); -pmd_huge_pte (mm, pmdp) =page_ptdesc (pgtable); +list_add (&ptdesc->pt_list, &pmd_huge_pte (mm, pmdp)->pt_list); +pmd_huge_pte (mm, pmdp) = ptdesc; }#endif
Signature · 1 line
2.43.0
RFC PATCH 15/18 mm/pgtable: pass ptdesc to pmd_populate
In reply to [email protected]From: Alex Shi <[email protected]>
Pass struct ptdesc to pmd_populate to further replace pgtable_t. We use type casting instead of page_ptdesc() helper since different arch has different type of pgtable_t.
Helper ptdesc_pfn used for arch openrisc and hexagon.
--- arch/alpha/include/asm/pgalloc.h | 4 ++-- arch/arc/include/asm/pgalloc.h | 4 ++-- arch/arm/include/asm/pgalloc.h | 4 ++-- arch/arm64/include/asm/pgalloc.h | 4 ++-- arch/hexagon/include/asm/pgalloc.h | 4 ++-- arch/loongarch/include/asm/pgalloc.h | 4 ++-- arch/m68k/include/asm/motorola_pgalloc.h | 4 ++-- arch/m68k/include/asm/sun3_pgalloc.h | 4 ++-- arch/microblaze/include/asm/pgalloc.h | 2 +- arch/mips/include/asm/pgalloc.h | 4 ++-- arch/nios2/include/asm/pgalloc.h | 4 ++-- arch/openrisc/include/asm/pgalloc.h | 4 ++-- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 +- arch/sh/include/asm/pgalloc.h | 4 ++-- arch/sparc/include/asm/pgalloc_32.h | 2 +- arch/x86/include/asm/pgalloc.h | 4 ++-- mm/debug_vm_pgtable.c | 2 +- mm/huge_memory.c | 8 ++++---- mm/khugepaged.c | 4 ++-- mm/memory.c | 2 +- mm/mremap.c | 2 +- 22 files changed, 39 insertions(+), 39 deletions(-)
arch/alpha/include/asm/pgalloc.h +2 -2
@@ -14,9 +14,9 @@*/ static inline void -pmd_populate (struct mm_struct *mm,pmd_t *pmd,pgtable_t pte) +pmd_populate (struct mm_struct *mm,pmd_t *pmd,struct ptdesc *pte) { -pmd_set (pmd, (pte_t *)(page_to_pa (pte) +PAGE_OFFSET )); +pmd_set (pmd, (pte_t *)(page_to_pa (ptdesc_page (pte)) +PAGE_OFFSET )); } static inline void
arch/arc/include/asm/pgalloc.h +2 -2
@@ -46,9 +46,9 @@ pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte)set_pmd (pmd,__pmd ((unsigned long )pte)); } -static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,pgtable_t pte_page) +static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,struct ptdesc *pte) { -set_pmd (pmd,__pmd ((unsigned long )page_address (pte_page))); +set_pmd (pmd,__pmd ((unsigned long )ptdesc_address (pte))); }static inline pgd_t *pgd_alloc (struct mm_struct *mm)
arch/arm/include/asm/pgalloc.h +2 -2
@@ -130,7 +130,7 @@ pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep) }static inline void -pmd_populate (struct mm_struct *mm,pmd_t *pmdp,pgtable_t ptep) +pmd_populate (struct mm_struct *mm,pmd_t *pmdp,struct ptdesc *ptep) {extern pmdval_t user_pmd_table;pmdval_t prot;
@@ -140,7 +140,7 @@ pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep) else prot = _PAGE_USER_TABLE; - __pmd_populate(pmdp, page_to_phys(ptep), prot); + __pmd_populate(pmdp, page_to_phys(ptdesc_page(ptep)), prot); }#endif /* CONFIG_MMU */
arch/arm64/include/asm/pgalloc.h +2 -2
@@ -131,10 +131,10 @@ pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep) }static inline void -pmd_populate (struct mm_struct *mm,pmd_t *pmdp,pgtable_t ptep) +pmd_populate (struct mm_struct *mm,pmd_t *pmdp,struct ptdesc *ptep) {VM_BUG_ON (mm == &init_mm); -__pmd_populate (pmdp,page_to_phys (ptep),PMD_TYPE_TABLE |PMD_TABLE_PXN ); +__pmd_populate (pmdp,page_to_phys (ptdesc_page (ptep)),PMD_TYPE_TABLE |PMD_TABLE_PXN ); }#endif
arch/hexagon/include/asm/pgalloc.h +2 -2
@@ -42,13 +42,13 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -pgtable_t pte) +struct ptdesc *pte) {/* * Conveniently, zero in 3 LSB means indirect 4K page table. * Not so convenient when you're trying to vary the page size. */ -set_pmd (pmd,__pmd (((unsigned long )page_to_pfn (pte) <<PAGE_SHIFT ) | +set_pmd (pmd,__pmd (((unsigned long )ptdesc_pfn (pte) <<PAGE_SHIFT ) |HEXAGON_L1_PTE_SIZE )); }
arch/loongarch/include/asm/pgalloc.h +2 -2
@@ -18,9 +18,9 @@ static inline void pmd_populate_kernel(struct mm_struct *mm,set_pmd (pmd,__pmd ((unsigned long )pte)); } -static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,pgtable_t pte) +static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,struct ptdesc *pte) { -set_pmd (pmd,__pmd ((unsigned long )page_address (pte))); +set_pmd (pmd,__pmd ((unsigned long )ptdesc_address (pte))); }#ifndef __PAGETABLE_PMD_FOLDED
arch/m68k/include/asm/motorola_pgalloc.h +2 -2
@@ -84,9 +84,9 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pmd_set (pmd, pte); } -static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,pgtable_t page) +static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,struct ptdesc *page) { -pmd_set (pmd, page); +pmd_set (pmd,ptdesc_page (page)); }static inline void pud_populate (struct mm_struct *mm,pud_t *pud,pmd_t *pmd)
arch/m68k/include/asm/sun3_pgalloc.h +2 -2
@@ -28,9 +28,9 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pmd_val (*pmd) =__pa ((unsigned long )pte); } -static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,pgtable_t page) +static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd,struct ptdesc *ptdesc) { -pmd_val (*pmd) =__pa ((unsigned long )page_address (page)); +pmd_val (*pmd) =__pa ((unsigned long )ptdesc_address (ptdesc)); }/*
arch/microblaze/include/asm/pgalloc.h +1 -1
@@ -33,7 +33,7 @@ extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm);#define __pte_free_tlb (tlb, pte, addr) pte_free((tlb)->mm, (pte))#define pmd_populate (mm, pmd, pte) \ - (pmd_val(*(pmd)) = (unsigned long)page_address(pte)) + (pmd_val(*(pmd)) = (unsigned long)page_address(ptdesc_page(pte)))#define pmd_populate_kernel (mm, pmd, pte) \ (pmd_val(*(pmd)) = (unsigned long) (pte))
arch/mips/include/asm/pgalloc.h +2 -2
@@ -25,9 +25,9 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -pgtable_t pte) +struct ptdesc *pte) { -set_pmd (pmd,__pmd ((unsigned long )page_address (pte))); +set_pmd (pmd,__pmd ((unsigned long )ptdesc_address (pte))); }/*
arch/nios2/include/asm/pgalloc.h +2 -2
@@ -21,9 +21,9 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -pgtable_t pte) +struct ptdesc *pte) { -set_pmd (pmd,__pmd ((unsigned long )page_address (pte))); +set_pmd (pmd,__pmd ((unsigned long )ptdesc_address (pte))); }extern pgd_t *pgd_alloc (struct mm_struct *mm);
arch/openrisc/include/asm/pgalloc.h +2 -2
@@ -29,10 +29,10 @@ extern int mem_init_done; set_pmd(pmd, __pmd(_KERNPG_TABLE + __pa(pte)))static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -struct page *pte) +struct ptdesc *pte) {set_pmd (pmd,__pmd (_KERNPG_TABLE + - ((unsigned long )page_to_pfn (pte) << + ((unsigned long )ptdesc_pfn (pte) << (unsigned long )PAGE_SHIFT ))); }
arch/parisc/include/asm/pgalloc.h +1 -1
@@ -68,6 +68,6 @@ pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte) }#define pmd_populate (mm, pmd, pte_page) \ - pmd_populate_kernel(mm, pmd, page_address(pte_page)) + pmd_populate_kernel(mm, pmd, page_address(ptdesc_page(pte_page)))#endif
arch/powerpc/include/asm/book3s/32/pgalloc.h +1 -1
@@ -32,7 +32,7 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmdp, -pgtable_t pte_page) +struct ptdesc *pte_page) { *pmdp =__pmd (__pa (pte_page) | _PMD_PRESENT); }
arch/sh/include/asm/pgalloc.h +2 -2
@@ -27,9 +27,9 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -pgtable_t pte) +struct ptdesc *pte) { -set_pmd (pmd,__pmd ((unsigned long )page_address (pte))); +set_pmd (pmd,__pmd ((unsigned long )ptdesc_address (pte))); }#define __pte_free_tlb (tlb, pte, addr) \
arch/sparc/include/asm/pgalloc_32.h +1 -1
@@ -50,7 +50,7 @@ static inline void free_pmd_fast(pmd_t * pmd)#define pmd_free (mm, pmd) free_pmd_fast(pmd)#define __pmd_free_tlb (tlb, pmd, addr) pmd_free((tlb)->mm, pmd) -#define pmd_populate (mm, pmd, pte) pmd_set(pmd, pte) +#define pmd_populate (mm, pmd, pte) pmd_set(pmd, (pte_t *)pte)void pmd_set (pmd_t *pmdp,pte_t *ptep);#define pmd_populate_kernel pmd_populate
arch/x86/include/asm/pgalloc.h +2 -2
@@ -76,9 +76,9 @@ static inline void pmd_populate_kernel_safe(struct mm_struct *mm, }static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmd, -struct page *pte) +struct ptdesc *pte) { -unsigned long pfn =page_to_pfn (pte); +unsigned long pfn =page_to_pfn (ptdesc_page (pte));paravirt_alloc_pte (mm, pfn);set_pmd (pmd,__pmd (((pteval_t )pfn <<PAGE_SHIFT ) | _PAGE_TABLE));
mm/debug_vm_pgtable.c +1 -1
@@ -645,7 +645,7 @@ static void __init pmd_populate_tests(struct pgtable_debug_args *args)* This entry points to next level page table page. * Hence this must not qualify as pmd_bad(). */ -pmd_populate (args->mm, args->pmdp, args->start_ptep); +pmd_populate (args->mm, args->pmdp,page_ptdesc (args->start_ptep)); pmd =READ_ONCE (*args->pmdp);WARN_ON (pmd_bad (pmd)); }
mm/huge_memory.c +4 -4
@@ -2365,7 +2365,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, old_pmd =pmdp_huge_clear_flush (vma, haddr, pmd); ptdesc =pgtable_trans_huge_withdraw (mm, pmd); -pmd_populate (mm, &_pmd,ptdesc_page (ptdesc)); +pmd_populate (mm, &_pmd, ptdesc); pte =pte_offset_map (&_pmd, haddr);VM_BUG_ON (!pte);
@@ -2382,7 +2382,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma, }pte_unmap (pte -1 );smp_wmb ();/* make pte visible before pmd */ -pmd_populate (mm, pmd,ptdesc_page (ptdesc)); +pmd_populate (mm, pmd, ptdesc); } static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
@@ -2537,7 +2537,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,* This's critical for some architectures (Power). */ ptdesc =pgtable_trans_huge_withdraw (mm, pmd); -pmd_populate (mm, &_pmd,ptdesc_page (ptdesc)); +pmd_populate (mm, &_pmd, ptdesc); pte =pte_offset_map (&_pmd, haddr);VM_BUG_ON (!pte);
@@ -2602,7 +2602,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,put_page (page);smp_wmb ();/* make pte visible before pmd */ -pmd_populate (mm, pmd,ptdesc_page (ptdesc)); +pmd_populate (mm, pmd, ptdesc); } void split_huge_pmd_locked(struct vm_area_struct *vma, unsigned long address,
mm/khugepaged.c +2 -2
@@ -769,7 +769,7 @@ static void __collapse_huge_page_copy_failed(pte_t *pte,* acquiring anon_vma_lock_write is unnecessary. */ pmd_ptl =pmd_lock (vma->vm_mm, pmd); -pmd_populate (vma->vm_mm, pmd,pmd_pgtable (orig_pmd)); +pmd_populate (vma->vm_mm, pmd, (struct ptdesc *)pmd_pgtable (orig_pmd));spin_unlock (pmd_ptl);/* * Release both raw and compound pages isolated
@@ -1198,7 +1198,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,* hugepmds and never for establishing regular pmds that * points to regular pagetables. Use pmd_populate for that */ -pmd_populate (mm, pmd,pmd_pgtable (_pmd)); +pmd_populate (mm, pmd, (struct ptdesc *)pmd_pgtable (_pmd));spin_unlock (pmd_ptl);anon_vma_unlock_write (vma->anon_vma); goto out_up_write;
mm/memory.c +1 -1
@@ -438,7 +438,7 @@ void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte)* smp_rmb() barriers in page table walking code. */ smp_wmb ();/* Could be smp_wmb__xxx(before|after)_spin_lock */ -pmd_populate (mm, pmd, *pte); +pmd_populate (mm, pmd, (struct ptdesc *)(*pte)); *pte =NULL ; }spin_unlock (ptl);
mm/mremap.c +1 -1
@@ -283,7 +283,7 @@ static bool move_normal_pmd(struct vm_area_struct *vma, unsigned long old_addr,VM_BUG_ON (!pmd_none (*new_pmd)); -pmd_populate (mm, new_pmd,pmd_pgtable (pmd)); +pmd_populate (mm, new_pmd, (struct ptdesc *)pmd_pgtable (pmd));flush_tlb_range (vma, old_addr, old_addr +PMD_SIZE );if (new_ptl != old_ptl)spin_unlock (new_ptl);
Signature · 1 line
2.43.0
RFC PATCH 16/18 mm/pgtable: pass ptdesc to pmd_install
In reply to [email protected]From: Alex Shi <[email protected]>
A new step to replace pgtable_t by ptdesc, also a preparation to change vmf.prealloc_pte to ptdesc too.
--- mm/filemap.c | 2 +- mm/internal.h | 2 +- mm/memory.c | 8 ++++---- 3 files changed, 6 insertions(+), 6 deletions(-)
mm/filemap.c +1 -1
@@ -3453,7 +3453,7 @@ static bool filemap_map_pmd(struct vm_fault *vmf, struct folio *folio, }if (pmd_none (*vmf->pmd) && vmf->prealloc_pte) -pmd_install (mm, vmf->pmd, &vmf->prealloc_pte); +pmd_install (mm, vmf->pmd, (struct ptdesc **)&vmf->prealloc_pte);return false; }
mm/internal.h +1 -1
@@ -320,7 +320,7 @@ void folio_activate(struct folio *folio);void free_pgtables (struct mmu_gather *tlb,struct ma_state *mas,struct vm_area_struct *start_vma,unsigned long floor,unsigned long ceiling,bool mm_wr_locked); -void pmd_install (struct mm_struct *mm,pmd_t *pmd,pgtable_t *pte); +void pmd_install (struct mm_struct *mm,pmd_t *pmd,struct ptdesc **pte);struct zap_details ; void unmap_page_range(struct mmu_gather *tlb,
mm/memory.c +4 -4
@@ -418,7 +418,7 @@ void free_pgtables(struct mmu_gather *tlb, struct ma_state *mas, }while (vma); } -void pmd_install (struct mm_struct *mm,pmd_t *pmd,pgtable_t *pte) +void pmd_install (struct mm_struct *mm,pmd_t *pmd,struct ptdesc **pte) {spinlock_t *ptl =pmd_lock (mm, pmd);
@@ -438,7 +438,7 @@ void pmd_install(struct mm_struct *mm, pmd_t *pmd, pgtable_t *pte)* smp_rmb() barriers in page table walking code. */ smp_wmb ();/* Could be smp_wmb__xxx(before|after)_spin_lock */ -pmd_populate (mm, pmd, (struct ptdesc *)(*pte)); +pmd_populate (mm, pmd, *pte); *pte =NULL ; }spin_unlock (ptl);
@@ -450,7 +450,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd)if (!ptdesc)return -ENOMEM ; -pmd_install (mm, pmd, (pgtable_t *)&ptdesc); +pmd_install (mm, pmd, &ptdesc);if (ptdesc)pte_free (mm, ptdesc);return 0 ;
@@ -4868,7 +4868,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf) }if (vmf->prealloc_pte) -pmd_install (vma->vm_mm, vmf->pmd, &vmf->prealloc_pte); +pmd_install (vma->vm_mm, vmf->pmd, (struct ptdesc **)&vmf->prealloc_pte);else if (unlikely (pte_alloc (vma->vm_mm, vmf->pmd)))return VM_FAULT_OOM ; }
Signature · 1 line
2.43.0
RFC PATCH 17/18 mm: convert vmf.prealloc_pte to struct ptdesc pointer
In reply to [email protected]From: Alex Shi <[email protected]>
vmfs.prealloc_pte is a pointer to page table memory, so converter it to struct ptdesc pointer.
--- include/linux/mm.h | 2 +- mm/filemap.c | 2 +- mm/memory.c | 12 ++++++------ 3 files changed, 8 insertions(+), 8 deletions(-)
include/linux/mm.h +1 -1
@@ -567,7 +567,7 @@ struct vm_fault {* Protects pte page table if 'pte' * is not NULL, otherwise pmd. */ -pgtable_t prealloc_pte;/* Pre-allocated pte page table. +struct ptdesc *prealloc_pte;/* Pre-allocated pte page table. * vm_ops->map_pages() sets up a page * table from atomic context. * do_fault_around() pre-allocates
mm/filemap.c +1 -1
@@ -3453,7 +3453,7 @@ static bool filemap_map_pmd(struct vm_fault *vmf, struct folio *folio, }if (pmd_none (*vmf->pmd) && vmf->prealloc_pte) -pmd_install (mm, vmf->pmd, (struct ptdesc **)&vmf->prealloc_pte); +pmd_install (mm, vmf->pmd, &vmf->prealloc_pte);return false; }
mm/memory.c +6 -6
@@ -4648,7 +4648,7 @@ static vm_fault_t __do_fault(struct vm_fault *vmf)* # flush A, B to clear the writeback */ if (pmd_none (*vmf->pmd) && !vmf->prealloc_pte) { - vmf->prealloc_pte =ptdesc_page (pte_alloc_one (vma->vm_mm)); + vmf->prealloc_pte =pte_alloc_one (vma->vm_mm);if (!vmf->prealloc_pte)return VM_FAULT_OOM ; }
@@ -4687,7 +4687,7 @@ static void deposit_prealloc_pte(struct vm_fault *vmf) {struct vm_area_struct *vma = vmf->vma; -pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd,page_ptdesc (vmf->prealloc_pte)); +pgtable_trans_huge_deposit (vma->vm_mm, vmf->pmd, vmf->prealloc_pte);/* * We are going to consume the prealloc table, * count that as nr_ptes.
@@ -4726,7 +4726,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page)* related to pte entry. Use the preallocated table for that. */ if (arch_needs_pgtable_deposit () && !vmf->prealloc_pte) { - vmf->prealloc_pte =ptdesc_page (pte_alloc_one (vma->vm_mm)); + vmf->prealloc_pte =pte_alloc_one (vma->vm_mm);if (!vmf->prealloc_pte)return VM_FAULT_OOM ; }
@@ -4868,7 +4868,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf) }if (vmf->prealloc_pte) -pmd_install (vma->vm_mm, vmf->pmd, (struct ptdesc **)&vmf->prealloc_pte); +pmd_install (vma->vm_mm, vmf->pmd, &vmf->prealloc_pte);else if (unlikely (pte_alloc (vma->vm_mm, vmf->pmd)))return VM_FAULT_OOM ; }
@@ -5011,7 +5011,7 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf) pte_off + vma_pages(vmf->vma) - vma_off) - 1;if (pmd_none (*vmf->pmd)) { - vmf->prealloc_pte =ptdesc_page (pte_alloc_one (vmf->vma->vm_mm)); + vmf->prealloc_pte =pte_alloc_one (vmf->vma->vm_mm);if (!vmf->prealloc_pte)return VM_FAULT_OOM ; }
@@ -5197,7 +5197,7 @@ static vm_fault_t do_fault(struct vm_fault *vmf)/* preallocated pagetable is unused: free it */ if (vmf->prealloc_pte) { -pte_free (vm_mm,page_ptdesc (vmf->prealloc_pte)); +pte_free (vm_mm, vmf->prealloc_pte); vmf->prealloc_pte =NULL ; }return ret;
Signature · 1 line
2.43.0
RFC PATCH 18/18 mm/pgtable: pass ptdesc in pte_free_defer
In reply to [email protected]From: Alex Shi <[email protected]>
pass ptdesc in pte_free_defer() and use ptdesc in collapse_huge_page().
This patch is immature, there is a issue from pmd_pgtable() conversion in few archs. The problem need a fix.
--- arch/powerpc/include/asm/pgalloc.h | 2 +- arch/s390/include/asm/pgalloc.h | 2 +- arch/s390/mm/pgalloc.c | 2 +- include/linux/pgtable.h | 2 +- mm/khugepaged.c | 10 +++++----- mm/pgtable-generic.c | 4 +--- 6 files changed, 10 insertions(+), 12 deletions(-)
arch/powerpc/include/asm/pgalloc.h +1 -1
@@ -47,7 +47,7 @@ static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptepage)/* arch use pte_free_defer() implementation in arch/powerpc/mm/pgtable-frag.c */ #define pte_free_defer pte_free_defer -void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable); +void pte_free_defer (struct mm_struct *mm,struct ptdesc *pgtable);/* * Functions that deal with pagetables that could be at any level of
arch/s390/include/asm/pgalloc.h +1 -1
@@ -144,7 +144,7 @@ static inline void pmd_populate(struct mm_struct *mm,/* arch use pte_free_defer() implementation in arch/s390/mm/pgalloc.c */ #define pte_free_defer pte_free_defer -void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable); +void pte_free_defer (struct mm_struct *mm,struct ptdesc *pgtable);void vmem_map_init (void );void *vmem_crst_alloc (unsigned long val);
arch/s390/mm/pgalloc.c +1 -1
@@ -214,7 +214,7 @@ static void pte_free_now(struct rcu_head *head)pagetable_pte_dtor_free (ptdesc); } -void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable) +void pte_free_defer (struct mm_struct *mm,struct ptdesc *pgtable) {struct ptdesc *ptdesc =virt_to_ptdesc (pgtable);
include/linux/pgtable.h +1 -1
@@ -116,7 +116,7 @@ static inline void pte_unmap(pte_t *pte) }#endif -void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable); +void pte_free_defer (struct mm_struct *mm,struct ptdesc *ptdesc);/* Find an entry in the second-level page table.. */ #ifndef pmd_offset
mm/khugepaged.c +5 -5
@@ -1094,7 +1094,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,LIST_HEAD (compound_pagelist);pmd_t *pmd, _pmd;pte_t *pte; -pgtable_t pgtable; +struct ptdesc *ptdesc;struct folio *folio;spinlock_t *pmd_ptl, *pte_ptl;int result =SCAN_FAIL ;
@@ -1223,7 +1223,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,* write. */ __folio_mark_uptodate (folio); - pgtable =pmd_pgtable (_pmd); + ptdesc =pmd_ptdesc (&_pmd); _pmd =mk_huge_pmd (&folio->page, vma->vm_page_prot); _pmd =maybe_pmd_mkwrite (pmd_mkdirty (_pmd), vma);
@@ -1232,7 +1232,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address,BUG_ON (!pmd_none (*pmd));folio_add_new_anon_rmap (folio, vma, address,RMAP_EXCLUSIVE );folio_add_lru_vma (folio, vma); -pgtable_trans_huge_deposit (mm, pmd,page_ptdesc (pgtable)); +pgtable_trans_huge_deposit (mm, pmd, ptdesc);set_pmd_at (mm, address, pmd, _pmd);update_mmu_cache_pmd (vma, address, pmd);spin_unlock (pmd_ptl);
@@ -1664,7 +1664,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,mm_dec_nr_ptes (mm);page_table_check_pte_clear_range (mm, haddr, pgt_pmd); -pte_free_defer (mm,pmd_pgtable (pgt_pmd)); +pte_free_defer (mm,pmd_ptdesc (&pgt_pmd)); maybe_install_pmd:/* step 5: install pmd entry */
@@ -1777,7 +1777,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)if (retracted) {mm_dec_nr_ptes (mm);page_table_check_pte_clear_range (mm, addr, pgt_pmd); -pte_free_defer (mm,pmd_pgtable (pgt_pmd)); +pte_free_defer (mm,pmd_ptdesc (&pgt_pmd)); } }i_mmap_unlock_read (mapping);
mm/pgtable-generic.c +1 -3
@@ -244,10 +244,8 @@ static void pte_free_now(struct rcu_head *head) pte_free(NULL /* mm not passed and not used */, ptdesc); } -void pte_free_defer (struct mm_struct *mm,pgtable_t pgtable) +void pte_free_defer (struct mm_struct *mm,struct ptdesc *ptdesc) { -struct ptdesc *ptdesc =page_ptdesc (pgtable); -call_rcu (&ptdesc->pt_rcu_head, pte_free_now); }#endif /* pte_free_defer */
Signature · 1 line
2.43.0
BTW, the patchset based on the latest git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-unstable branch.
Quoted 106 lines · On 7/30/24 2:46 PM, [email protected] wrote:
From: Alex Shi <[email protected]>
We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while
pgtable_tis used widely.The
pgtable_tis typedefed as 'pte_t*' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'.These blocks the conception and code update for page table descriptor to struct ptdesc.
So, the simple idea to push the ptdesc conception forward is to update all
pgtable_tby ptdesc orpte_tpointer. But this needs widely knowledges for most all of different archs. Common code change is easy forinclude/andmm/directory, but it's hard in all archs.Thanks for intel LKP framework, I fixed most all of build issues except a bug on powerpc which reports a "struct ptdesc *" incompatible with struct ptdesc *' pointer issue...
Another trouble is
pmd_pgtable()conversion in the last patch. Maybe some of arch need define theirself ownpmd_ptdesc()?This patchset is immature, even except above 2 issues, I just tested virutal machine booting and kselftest mm on x86 and arm64.
Anyway any input are appreciated!
Thanks
AlexAlex Shi (18):
mm/pgtable: use ptdesc in pte_free_now/pte_free_defer mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer fs/dax: use ptdesc indax_pmd_load_hole
mm/thp: use ptdesc pointer in__do_huge_pmd_anonymous_pagemm/thp: use ptdesc indo_huge_pmd_anonymous_page
mm/thp: convertinsert_pfn_pmdand its caller to use ptdesc mm/thp: use ptdesc incopy_huge_pmd
mm/memory: use ptdesc in__pte_alloc
mm/pgtable: fully use ptdesc inpte_alloc_oneseries functions mm/pgtable: pass ptdesc topte_free()
mm/pgtable: introduceptdesc_pfnand use ptdesc infree_pte_range()mm/thp: pass ptdesc toset_huge_zero_foliofunction mm/pgtable: return ptdesc pointer inpgtable_trans_huge_withdrawmm/pgtable: use ptdesc inpgtable_trans_huge_depositmm/pgtable: pass ptdesc topmd_populate
mm/pgtable: pass ptdesc topmd_install
mm: convert vmf.prealloc_pte to struct ptdesc pointer mm/pgtable: pass ptdesc inpte_free_deferarch/alpha/include/asm/pgalloc.h | 4 +- arch/arc/include/asm/pgalloc.h | 4 +- arch/arm/include/asm/pgalloc.h | 13 +-- arch/arm/include/asm/tlb.h | 4 +- arch/arm/mm/pgd.c | 2 +- arch/arm64/include/asm/pgalloc.h | 4 +- arch/arm64/include/asm/tlb.h | 4 +- arch/csky/include/asm/pgalloc.h | 4 +- arch/hexagon/include/asm/pgalloc.h | 8 +- arch/loongarch/include/asm/pgalloc.h | 8 +- arch/m68k/include/asm/motorola_pgalloc.h | 12 +- arch/m68k/include/asm/sun3_pgalloc.h | 4 +- arch/microblaze/include/asm/pgalloc.h | 2 +- arch/mips/include/asm/pgalloc.h | 4 +- arch/nios2/include/asm/pgalloc.h | 4 +- arch/openrisc/include/asm/pgalloc.h | 8 +- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/32/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 8 +- arch/powerpc/include/asm/book3s/64/radix.h | 4 +- arch/powerpc/include/asm/pgalloc.h | 8 +- arch/powerpc/mm/book3s64/hash_pgtable.c | 10 +- arch/powerpc/mm/book3s64/radix_pgtable.c | 10 +- arch/riscv/include/asm/pgalloc.h | 8 +- arch/s390/include/asm/pgalloc.h | 4 +- arch/s390/include/asm/pgtable.h | 4 +- arch/s390/mm/pgalloc.c | 2 +- arch/s390/mm/pgtable.c | 14 +-- arch/sh/include/asm/pgalloc.h | 4 +- arch/sparc/include/asm/pgalloc_32.h | 6 +- arch/sparc/include/asm/pgalloc_64.h | 2 +- arch/sparc/include/asm/pgtable_64.h | 4 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 6 +- arch/sparc/mm/tlb.c | 14 +-- arch/x86/include/asm/pgalloc.h | 10 +- arch/x86/mm/pgtable.c | 8 +- arch/xtensa/include/asm/pgalloc.h | 12 +- fs/dax.c | 14 +-- include/asm-generic/pgalloc.h | 10 +- include/linux/mm.h | 16 ++- include/linux/mm_types.h | 4 +- include/linux/pgtable.h | 6 +- mm/debug_vm_pgtable.c | 6 +- mm/huge_memory.c | 103 +++++++++--------- mm/internal.h | 2 +- mm/khugepaged.c | 14 +-- mm/memory.c | 15 +-- mm/mremap.c | 2 +- mm/pgtable-generic.c | 37 +++---- 53 files changed, 240 insertions(+), 236 deletions(-)
Hi,
Le 30/07/2024 à 08:46, [email protected] a écrit :
Quoted 11 lines
From: Alex Shi <[email protected]>
We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while
pgtable_tis used widely.The
pgtable_tis typedefed as 'pte_t*' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'.These blocks the conception and code update for page table descriptor to struct ptdesc.
Quoted 8 lines
So, the simple idea to push the ptdesc conception forward is to update all
pgtable_tby ptdesc orpte_tpointer. But this needs widely knowledges for most all of different archs. Common code change is easy forinclude/andmm/directory, but it's hard in all archs.Thanks for intel LKP framework, I fixed most all of build issues except a bug on powerpc which reports a "struct ptdesc *" incompatible with struct ptdesc *' pointer issue...
Can you tell more about that problem on powerpc ? Which defconfig for instance ?
Quoted 8 lines
Another trouble is
pmd_pgtable()conversion in the last patch. Maybe some of arch need define theirself ownpmd_ptdesc()?This patchset is immature, even except above 2 issues, I just tested virutal machine booting and kselftest mm on x86 and arm64.
Anyway any input are appreciated!
Can you tell on which tree you based this series ? Last patch doesn't apply on 6.11-rc1:
Applying: mm/pgtable: pass ptdesc in pte_free_defer
error: sha1 information is lacking or useless (mm/khugepaged.c).
error: could not build fake ancestor
Patch failed at 0019 mm/pgtable: pass ptdesc in pte_free_defer
Following hunk is the problem:
diff a/mm/khugepaged.c b/mm/khugepaged.c (rejected hunks)
@@ -1777,7 +1777,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)
if (retracted) {
mm_dec_nr_ptes(mm); page_table_check_pte_clear_range(mm, addr, pgt_pmd);
- pte_free_defer(mm, pmd_pgtable(pgt_pmd));
+ pte_free_defer(mm, pmd_ptdesc(&pgt_pmd));
}
}
i_mmap_unlock_read(mapping);
Christophe
Quoted 79 lines
Thanks
AlexAlex Shi (18):
mm/pgtable: use ptdesc in pte_free_now/pte_free_defer mm/pgtable: convert ptdesc.pmd_huge_pte to ptdesc pointer fs/dax: use ptdesc indax_pmd_load_hole
mm/thp: use ptdesc pointer in__do_huge_pmd_anonymous_pagemm/thp: use ptdesc indo_huge_pmd_anonymous_page
mm/thp: convertinsert_pfn_pmdand its caller to use ptdesc mm/thp: use ptdesc incopy_huge_pmd
mm/memory: use ptdesc in__pte_alloc
mm/pgtable: fully use ptdesc inpte_alloc_oneseries functions mm/pgtable: pass ptdesc topte_free()
mm/pgtable: introduceptdesc_pfnand use ptdesc infree_pte_range()mm/thp: pass ptdesc toset_huge_zero_foliofunction mm/pgtable: return ptdesc pointer inpgtable_trans_huge_withdrawmm/pgtable: use ptdesc inpgtable_trans_huge_depositmm/pgtable: pass ptdesc topmd_populate
mm/pgtable: pass ptdesc topmd_install
mm: convert vmf.prealloc_pte to struct ptdesc pointer mm/pgtable: pass ptdesc inpte_free_deferarch/alpha/include/asm/pgalloc.h | 4 +- arch/arc/include/asm/pgalloc.h | 4 +- arch/arm/include/asm/pgalloc.h | 13 +-- arch/arm/include/asm/tlb.h | 4 +- arch/arm/mm/pgd.c | 2 +- arch/arm64/include/asm/pgalloc.h | 4 +- arch/arm64/include/asm/tlb.h | 4 +- arch/csky/include/asm/pgalloc.h | 4 +- arch/hexagon/include/asm/pgalloc.h | 8 +- arch/loongarch/include/asm/pgalloc.h | 8 +- arch/m68k/include/asm/motorola_pgalloc.h | 12 +- arch/m68k/include/asm/sun3_pgalloc.h | 4 +- arch/microblaze/include/asm/pgalloc.h | 2 +- arch/mips/include/asm/pgalloc.h | 4 +- arch/nios2/include/asm/pgalloc.h | 4 +- arch/openrisc/include/asm/pgalloc.h | 8 +- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/32/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-4k.h | 4 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 4 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 4 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 8 +- arch/powerpc/include/asm/book3s/64/radix.h | 4 +- arch/powerpc/include/asm/pgalloc.h | 8 +- arch/powerpc/mm/book3s64/hash_pgtable.c | 10 +- arch/powerpc/mm/book3s64/radix_pgtable.c | 10 +- arch/riscv/include/asm/pgalloc.h | 8 +- arch/s390/include/asm/pgalloc.h | 4 +- arch/s390/include/asm/pgtable.h | 4 +- arch/s390/mm/pgalloc.c | 2 +- arch/s390/mm/pgtable.c | 14 +-- arch/sh/include/asm/pgalloc.h | 4 +- arch/sparc/include/asm/pgalloc_32.h | 6 +- arch/sparc/include/asm/pgalloc_64.h | 2 +- arch/sparc/include/asm/pgtable_64.h | 4 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 6 +- arch/sparc/mm/tlb.c | 14 +-- arch/x86/include/asm/pgalloc.h | 10 +- arch/x86/mm/pgtable.c | 8 +- arch/xtensa/include/asm/pgalloc.h | 12 +- fs/dax.c | 14 +-- include/asm-generic/pgalloc.h | 10 +- include/linux/mm.h | 16 ++- include/linux/mm_types.h | 4 +- include/linux/pgtable.h | 6 +- mm/debug_vm_pgtable.c | 6 +- mm/huge_memory.c | 103 +++++++++--------- mm/internal.h | 2 +- mm/khugepaged.c | 14 +-- mm/memory.c | 15 +-- mm/mremap.c | 2 +- mm/pgtable-generic.c | 37 +++---- 53 files changed, 240 insertions(+), 236 deletions(-)
Quoted 19 lines · LEROY Christophe
Hi,
Le 30/07/2024 à 08:46, [email protected] a écrit :
Quoted 16 lines
From: Alex Shi <[email protected]>
We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while
pgtable_tis used widely.The
pgtable_tis typedefed as 'pte_t*' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'.These blocks the conception and code update for page table descriptor to struct ptdesc.
So, the simple idea to push the ptdesc conception forward is to update all
pgtable_tby ptdesc orpte_tpointer. But this needs widely knowledges for most all of different archs. Common code change is easy forinclude/andmm/directory, but it's hard in all archs.
Quoted 7 lines
Quoted 3 lines
Thanks for intel LKP framework, I fixed most all of build issues except a bug on powerpc which reports a "struct ptdesc *" incompatible with struct ptdesc *' pointer issue...
Can you tell more about that problem on powerpc ? Which defconfig for instance ?
Hi Leroy,
Thanks a lot for response. The detailed error and code tree is from LKP:
Quoted 14 lines · kernel test robot
tree: https://github.com/alexshi/linux.git ptdesc head: 3735e634f8feb67c941d17222b53906ace2fd2b8 commit: 8ac2ea5f299444d065b292043aed0a1e94996159 [14/17] mm/pgtable: use ptdesc in pgtable_trans_huge_deposit config: powerpc-allyesconfig (https://download.01.org/0day-ci/archive/20240803/[email protected]/config) compiler: clang version 20.0.0git (https://github.com/llvm/llvm-project 423aec6573df4424f90555468128e17073ddc69e) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240803/[email protected]/reproduce)In file included from
arch/powerpc/include/asm/book3s/64/mmu-hash.h:20:Quoted 6 lines
Quoted 1 line
arch/powerpc/include/asm/book3s/64/pgtable.h:1372:52: error: incompatible pointer types passing 'struct ptdesc *' to parameter of type 'struct ptdesc *' [-Werror,-Wincompatible-pointer-types]Another trouble is
pmd_pgtable()conversion in the last patch. Maybe some of arch need define theirself ownpmd_ptdesc()?
Quoted 7 lines
Quoted 4 lines
This patchset is immature, even except above 2 issues, I just tested virutal machine booting and kselftest mm on x86 and arm64.
Anyway any input are appreciated!
Can you tell on which tree you based this series ? Last patch doesn't apply on 6.11-rc1:
The code based on akpm/mm-unstable tree, not upstream, you can check the code on:
https://github.com/alexshi/linux.git ptdesc
Thanks a lot!
Alex
Matthew Wilcox
maintainerQuoted 2 lines · On Tue, Jul 30, 2024 at 02:46:54PM +0800, [email protected] wrote:
We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while
pgtable_tis used widely.
Hum, I thought I responded to this to point out the problem, but I don't see the response anywhere, so I'll try again.
Quoted 2 lines
The
pgtable_tis typedefed as 'pte_t*' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'.
And there's a very good reason for that. On s390 and powerpc (I cannot speak to the sparc/m68k), each page table is (potentially) smaller than PAGE_SIZE. So we cannot do what your patch purports to do, as we would not know whether we're referring to the first or subsequent page tables contained within a page.
Maybe at some point in the distant future we'll be able to allocate a ptdesc per page table instead of per page allocated for use by page tables. But we cannot do that yet.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
Quoted 10 lines · Matthew Wilcox
Quoted 2 lines · On Tue, Jul 30, 2024 at 02:46:54PM +0800, [email protected] wrote:
We have struct ptdesc for page table descriptor a year ago, but it has no much usages in kernel, while
pgtable_tis used widely.Hum, I thought I responded to this to point out the problem, but I don't see the response anywhere, so I'll try again.
Quoted 2 lines
The
pgtable_tis typedefed as 'pte_t*' in sparc, s390, powerpc and m68k except SUN3, others archs are all same as 'struct page *'.
Quoted 8 lines
And there's a very good reason for that. On s390 and powerpc (I cannot speak to the sparc/m68k), each page table is (potentially) smaller than
PAGE_SIZE. So we cannot do what your patch purports to do, as we would not know whether we're referring to the first or subsequent page tables contained within a page.Quoted 1 line
Maybe at some point in the distant future we'll be able to allocate
a ptdesc per page table instead of per page allocated for use by page tables. But we cannot do that yet.
Got it. Thanks a lot for the info!