Convert 64-bit x86/mm/pat to ptdescs
Subsystems (3)
v2 cover Convert 64-bit x86/mm/pat to ptdescs
x86/mm/pat should be using ptdescs. One line has already been converted to pagetable_free(), while the allocation sites use get_free_pages(). This causes issues separately allocating ptdescs from struct page.
These patches convert the allocation/free sites to use ptdescs. In the short term, this helps enable Matthew's work to allocate frozen pagetables[1]. And in the long term, this will help us cleanly split ptdesc allocations from struct page.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/T/#u
------
I've also tested this on a tree that separately allocates ptdescs. That didn't find any lingering alloc/free issues.
I've realized that the pgd_list should also be using ptdescs (for 32bit in this file). This can be done in a different patchset since there's other users of pgd_list that still need to be converted.
Based on current mm-new.
v2:
- Use pagetable_alloc() in populate_pgd() - in patch 2
- Rename subject line to specify 64-bit (i.e. 32-bit wasn't converted)
- Added reference links to the projects mentioned in the cover letter
Vishal Moola (Oracle) (3):
x86/mm/pat: Convert pte code to use ptdescs x86/mm/pat: Convert pmd code to use ptdescs x86/mm/pat: Convert split_large_page() to use ptdescs
arch/x86/mm/pat/set_memory.c | 43 ++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 19 deletions(-)
Signature · 1 line
2.52.0
PATCH v2 1/3 x86/mm/pat: Convert pte code to use ptdescs
In order to separately allocate ptdescs from pages, we need all allocation and free sites to use the appropriate functions. Convert these pte allocation/free sites to use ptdescs.
--- arch/x86/mm/pat/set_memory.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/mm/pat/set_memory.c +6 -5
@@ -1408,7 +1408,7 @@ static bool try_to_free_pte_page(pte_t *pte)if (!pte_none (pte[i]))return false; -free_page ((unsigned long )pte); +pagetable_free (virt_to_ptdesc ((void *)pte));return true; }
@@ -1537,9 +1537,10 @@ static void unmap_pud_range(p4d_t *p4d, unsigned long start, unsigned long end)*/ } -static int alloc_pte_page (pmd_t *pmd) +static int alloc_pte_ptdesc (pmd_t *pmd) { -pte_t *pte = (pte_t *)get_zeroed_page (GFP_KERNEL ); +pte_t *pte = (pte_t *)ptdesc_address ( +pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 ));if (!pte)return -1 ;
@@ -1600,7 +1601,7 @@ static long populate_pmd(struct cpa_data *cpa,*/ pmd =pmd_offset (pud, start);if (pmd_none (*pmd)) -if (alloc_pte_page (pmd)) +if (alloc_pte_ptdesc (pmd))return -1 ;populate_pte (cpa, start, pre_end, cur_pages, pmd, pgprot);
@@ -1641,7 +1642,7 @@ static long populate_pmd(struct cpa_data *cpa,if (start < end) { pmd =pmd_offset (pud, start);if (pmd_none (*pmd)) -if (alloc_pte_page (pmd)) +if (alloc_pte_ptdesc (pmd))return -1 ; populate_pte(cpa, start, end, num_pages - cur_pages,
Signature · 1 line
2.52.0
Re: [PATCH v2 1/3] x86/mm/pat: Convert pte code to use ptdescs
Quoted 23 lines · Vishal Moola (Oracle)
In order to separately allocate ptdescs from pages, we need all allocation and free sites to use the appropriate functions. Convert these pte allocation/free sites to use ptdescs.
Signed-off-by Vishal Moola (Oracle) <[email protected]>--- arch/x86/mm/pat/set_memory.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)arch/x86/mm/pat/set_memory.c +1
-1@@ -1408,7 +1408,7 @@ static bool try_to_free_pte_page(pte_t *pte)if (!pte_none (pte[i]))return false; -free_page ((unsigned long )pte); +pagetable_free (virt_to_ptdesc ((void *)pte));return true; }
arch/x86/mm/pat/set_memory.c static void unmap_pud_range(p4d_t *p4d, unsigned long start, unsigned long end) · line 1543
Quoted 9 lines
arch/x86/mm/pat/set_memory.c +3
-2@@ -1537,9 +1537,10 @@ static void unmap_pud_range(p4d_t *p4d, unsigned long start, unsigned long end)*/ } -static int alloc_pte_page (pmd_t *pmd) +static int alloc_pte_ptdesc (pmd_t *pmd) { -pte_t *pte = (pte_t *)get_zeroed_page (GFP_KERNEL ); +pte_t *pte = (pte_t *)ptdesc_address ( +pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 ));
Sorry I missed this last time, but ptdesc_address(NULL) does not return NULL.
The allocation and conversion should be split IMHO.
This applies to all instances in all the patches.
Quoted 24 lines
arch/x86/mm/pat/set_memory.c +2
-2if (!pte)return -1 ;@@ -1600,7 +1601,7 @@ static long populate_pmd(struct cpa_data *cpa,*/ pmd =pmd_offset (pud, start);if (pmd_none (*pmd)) -if (alloc_pte_page (pmd)) +if (alloc_pte_ptdesc (pmd))return -1 ;populate_pte (cpa, start, pre_end, cur_pages, pmd, pgprot);@@ -1641,7 +1642,7 @@ static long populate_pmd(struct cpa_data *cpa,if (start < end) { pmd =pmd_offset (pud, start);if (pmd_none (*pmd)) -if (alloc_pte_page (pmd)) +if (alloc_pte_ptdesc (pmd))return -1 ; populate_pte(cpa, start, end, num_pages - cur_pages,Signature · 1 line
2.52.0
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v2 1/3] x86/mm/pat: Convert pte code to use ptdescs
Quoted 27 lines · Mike Rapoport
Quoted 26 lines · Vishal Moola (Oracle)
In order to separately allocate ptdescs from pages, we need all allocation and free sites to use the appropriate functions. Convert these pte allocation/free sites to use ptdescs.
Signed-off-by Vishal Moola (Oracle) <[email protected]>--- arch/x86/mm/pat/set_memory.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)arch/x86/mm/pat/set_memory.c +1
-1@@ -1408,7 +1408,7 @@ static bool try_to_free_pte_page(pte_t *pte)if (!pte_none (pte[i]))return false; -free_page ((unsigned long )pte); +pagetable_free (virt_to_ptdesc ((void *)pte));return true; }@@ -1537,9 +1537,10 @@ static void unmap_pud_range(p4d_t *p4d, unsigned long start, unsigned long end)*/ }
Quoted 10 lines
arch/x86/mm/pat/set_memory.c static void unmap_pud_range(p4d_t *p4d, unsigned long start, unsigned long end) · line 1543
Quoted 6 lines
arch/x86/mm/pat/set_memory.c +3
-2-static int alloc_pte_page (pmd_t *pmd) +static int alloc_pte_ptdesc (pmd_t *pmd) { -pte_t *pte = (pte_t *)get_zeroed_page (GFP_KERNEL ); +pte_t *pte = (pte_t *)ptdesc_address ( +pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 ));Sorry I missed this last time, but
ptdesc_address(NULL)does not return NULL.
The allocation and conversion should be split IMHO.
Good catch.
Quoted 1 line
This applies to all instances in all the patches.
Thanks for reviewing, I'll send v3 with your feedback included next week.
Quoted 28 lines
Quoted 24 lines
arch/x86/mm/pat/set_memory.c +2
-2if (!pte)return -1 ;@@ -1600,7 +1601,7 @@ static long populate_pmd(struct cpa_data *cpa,*/ pmd =pmd_offset (pud, start);if (pmd_none (*pmd)) -if (alloc_pte_page (pmd)) +if (alloc_pte_ptdesc (pmd))return -1 ;populate_pte (cpa, start, pre_end, cur_pages, pmd, pgprot);@@ -1641,7 +1642,7 @@ static long populate_pmd(struct cpa_data *cpa,if (start < end) { pmd =pmd_offset (pud, start);if (pmd_none (*pmd)) -if (alloc_pte_page (pmd)) +if (alloc_pte_ptdesc (pmd))return -1 ; populate_pte(cpa, start, end, num_pages - cur_pages,Signature · 1 line
2.52.0Signature · 2 lines
Sincerely yours, Mike.
PATCH v2 2/3 x86/mm/pat: Convert pmd code to use ptdescs
In reply to Vishal Moola (Oracle)In order to separately allocate ptdescs from pages, we need all allocation and free sites to use the appropriate functions. Convert these pmd allocation/free sites to use ptdescs.
populate_pgd() also allocates pagetables that may later be freed by try_to_free_pmd_page(), so allocate ptdescs there as well.
--- arch/x86/mm/pat/set_memory.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -8
@@ -1420,7 +1420,7 @@ static bool try_to_free_pmd_page(pmd_t *pmd)if (!pmd_none (pmd[i]))return false; -free_page ((unsigned long )pmd); +pagetable_free (virt_to_ptdesc ((void *)pmd));return true; }
@@ -1548,9 +1548,10 @@ static int alloc_pte_ptdesc(pmd_t *pmd)return 0 ; } -static int alloc_pmd_page (pud_t *pud) +static int alloc_pmd_ptdesc (pud_t *pud) { -pmd_t *pmd = (pmd_t *)get_zeroed_page (GFP_KERNEL ); +pmd_t *pmd = (pmd_t *)ptdesc_address ( +pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 ));if (!pmd)return -1 ;
@@ -1623,7 +1624,7 @@ static long populate_pmd(struct cpa_data *cpa,* We cannot use a 1G page so allocate a PMD page if needed. */ if (pud_none (*pud)) -if (alloc_pmd_page (pud)) +if (alloc_pmd_ptdesc (pud))return -1 ; pmd =pmd_offset (pud, start);
@@ -1679,7 +1680,7 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, p4d_t *p4d,* Need a PMD page? */ if (pud_none (*pud)) -if (alloc_pmd_page (pud)) +if (alloc_pmd_ptdesc (pud))return -1 ; cur_pages = populate_pmd(cpa, start, pre_end, cur_pages,
@@ -1716,7 +1717,7 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, p4d_t *p4d, pud =pud_offset (p4d, start);if (pud_none (*pud)) -if (alloc_pmd_page (pud)) +if (alloc_pmd_ptdesc (pud))return -1 ; tmp = populate_pmd(cpa, start, end, cpa->numpages - cur_pages,
@@ -1744,7 +1745,8 @@ static int populate_pgd(struct cpa_data *cpa, unsigned long addr) pgd_entry = cpa->pgd +pgd_index (addr);if (pgd_none (*pgd_entry)) { - p4d = (p4d_t *)get_zeroed_page (GFP_KERNEL ); + p4d = (p4d_t *)ptdesc_address ( +pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 ));if (!p4d)return -1 ;
@@ -1756,7 +1758,8 @@ static int populate_pgd(struct cpa_data *cpa, unsigned long addr)*/ p4d =p4d_offset (pgd_entry, addr);if (p4d_none (*p4d)) { - pud = (pud_t *)get_zeroed_page (GFP_KERNEL ); + pud = (pud_t *)ptdesc_address ( +pagetable_alloc (GFP_KERNEL | __GFP_ZERO,0 ));if (!pud)return -1 ;
Signature · 1 line
2.52.0
PATCH v2 3/3 x86/mm/pat: Convert split_large_page() to use ptdescs
In reply to Vishal Moola (Oracle)In order to separately allocate ptdescs from pages, we need all allocation and free sites to use the appropriate functions.
split_large_page() allocates a page to be used as a page table. This should be allocating a ptdesc, so convert it.
--- arch/x86/mm/pat/set_memory.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
arch/x86/mm/pat/set_memory.c +7 -6
@@ -1119,9 +1119,10 @@ static void split_set_pte(struct cpa_data *cpa, pte_t *pte, unsigned long pfn,static int __split_large_page (struct cpa_data *cpa,pte_t *kpte,unsigned long address, -struct page *base) +struct ptdesc *ptdesc) {unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc =1 ; +struct page *base =ptdesc_page (ptdesc);pte_t *pbase = (pte_t *)page_address (base);unsigned int i, level;pgprot_t ref_prot;
@@ -1226,18 +1227,18 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,static int split_large_page (struct cpa_data *cpa,pte_t *kpte,unsigned long address) { -struct page *base; +struct ptdesc *ptdesc;if (!debug_pagealloc_enabled ())spin_unlock (&cpa_lock); - base =alloc_pages (GFP_KERNEL ,0 ); + ptdesc =pagetable_alloc (GFP_KERNEL ,0 );if (!debug_pagealloc_enabled ())spin_lock (&cpa_lock); -if (!base) +if (!ptdesc)return -ENOMEM ; -if (__split_large_page (cpa, kpte, address, base)) -__free_page (base); +if (__split_large_page (cpa, kpte, address, ptdesc)) +pagetable_free (ptdesc);return 0 ; }
Signature · 1 line
2.52.0
Hi Vishal,
Quoted 13 lines · Vishal Moola (Oracle)
x86/mm/pat should be using ptdescs. One line has already been converted to
pagetable_free(), while the allocation sites useget_free_pages(). This causes issues separately allocating ptdescs from struct page.These patches convert the allocation/free sites to use ptdescs. In the short term, this helps enable Matthew's work to allocate frozen pagetables[1]. And in the long term, this will help us cleanly split ptdesc allocations from struct page.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/T/#u
Quoted 8 lines
------
I've also tested this on a tree that separately allocates ptdescs. That didn't find any lingering alloc/free issues.
I've realized that the
pgd_listshould also be using ptdescs (for 32bit in this file). This can be done in a different patchset since there's other users ofpgd_listthat still need to be converted.
Since Andrew merges cover-letter text into the first commit changelog, some explanation about pgd_list should be a part of that combined changelog.
Quoted 18 lines
Based on current mm-new.
v2:
- Usepagetable_alloc()inpopulate_pgd()- in patch 2
- Rename subject line to specify 64-bit (i.e. 32-bit wasn't converted)
- Added reference links to the projects mentioned in the cover letterVishal Moola (Oracle) (3):
x86/mm/pat: Convert pte code to use ptdescs x86/mm/pat: Convert pmd code to use ptdescs x86/mm/pat: Convertsplit_large_page()to use ptdescsarch/x86/mm/pat/set_memory.c | 43 ++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 19 deletions(-)Signature · 1 line
2.52.0
Signature · 2 lines
Sincerely yours, Mike.