iommu/amd: re-mapping fixes
cover iommu/amd: re-mapping fixes
This patch series tries to address a few issues encountered when replacing existing mappings:
.> pages leak in free_pagetable()
.> allow downgrading default page-sizes in alloc_pte()
.> tear-down all the replicated PTEs of a large mapping when downgrading to smaller mappings
Andrei Dulea (4):
iommu/amd: Fix pages leak in free_pagetable() iommu/amd: Fix downgrading default page-sizes in alloc_pte() iommu/amd: Introduce first_pte_l7() helper iommu/amd: Unmap all L7 PTEs when downgrading page-sizes
drivers/iommu/amd_iommu.c | 73 +++++++++++++++++++++++++++++++-------- 1 file changed, 58 insertions(+), 15 deletions(-)
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 1/4 iommu/amd: Fix pages leak in free_pagetable()
Take into account the gathered freelist in free_sub_pt(), otherwise we end up leaking all that pages.
--- drivers/iommu/amd_iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/amd_iommu.c +1 -1
@@ -1425,7 +1425,7 @@ static void free_pagetable(struct protection_domain *domain)BUG_ON (domain->mode <PAGE_MODE_NONE || domain->mode >PAGE_MODE_6_LEVEL ); -free_sub_pt (root, domain->mode, freelist); + freelist =free_sub_pt (root, domain->mode, freelist);free_page_list (freelist); }
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Joerg Roedel
maintainerRe: [PATCH 1/4] iommu/amd: Fix pages leak in free_pagetable()
Hi Andrei,
Quoted 14 lines · Andrei Dulea
Take into account the gathered freelist in
free_sub_pt(), otherwise we end up leaking all that pages.Fixes 409afa44f9ba ("iommu/amd: Introduce free_sub_pt() function")Signed-off-by Andrei Dulea <[email protected]>--- drivers/iommu/amd_iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/iommu/amd_iommu.c +0
-0
drivers/iommu/amd_iommu.c static void free_pagetable(struct protection_domain *domain) · line 1428
Quoted 5 lines
drivers/iommu/amd_iommu.c +1
-1@@ -1425,7 +1425,7 @@ static void free_pagetable(struct protection_domain *domain)BUG_ON (domain->mode <PAGE_MODE_NONE || domain->mode >PAGE_MODE_6_LEVEL ); -free_sub_pt (root, domain->mode, freelist); + freelist =free_sub_pt (root, domain->mode, freelist);
What a stupid mistake, thanks for fixing this. That is a clear example which shows that we need more reviewers. Applied the whole series.
Thanks again,
Joerg
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 2/4 iommu/amd: Fix downgrading default page-sizes in alloc_pte()
In reply to Andrei Dulea via iommuDowngrading an existing large mapping to a mapping using smaller page-sizes works only for the mappings created with page-mode 7 (i.e. non-default page size).
Treat large mappings created with page-mode 0 (i.e. default page size) like a non-present mapping and allow to overwrite it in alloc_pte().
While around, make sure that we flush the TLB only if we change an existing mapping, otherwise we might end up acting on garbage PTEs.
--- drivers/iommu/amd_iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu.c +2 -1
@@ -1490,6 +1490,7 @@ static u64 *alloc_pte(struct protection_domain *domain, pte_level =PM_PTE_LEVEL (__pte);if (!IOMMU_PTE_PRESENT (__pte) || + pte_level ==PAGE_MODE_NONE || pte_level ==PAGE_MODE_7_LEVEL ) { page = (u64 *)get_zeroed_page (gfp);if (!page)
@@ -1500,7 +1501,7 @@ static u64 *alloc_pte(struct protection_domain *domain,/* pte could have been changed somewhere. */ if (cmpxchg64 (pte, __pte, __npte) != __pte)free_page ((unsigned long )page); -else if (pte_level ==PAGE_MODE_7_LEVEL ) +else if (IOMMU_PTE_PRESENT (__pte)) domain->updated = true;continue ;
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 3/4 iommu/amd: Introduce first_pte_l7() helper
In reply to Andrei Dulea via iommuGiven an arbitrary pte that is part of a large mapping, this function returns the first pte of the series (and optionally the mapped size and number of PTEs)
It will be re-used in a subsequent patch to replace an existing L7 mapping.
--- drivers/iommu/amd_iommu.c | 40 ++++++++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 11 deletions(-)
drivers/iommu/amd_iommu.c +29 -11
@@ -501,6 +501,29 @@ static void iommu_uninit_device(struct device *dev)*/ } +/* +* Helper function to get the first pte of a large mapping +*/ +static u64 *first_pte_l7 (u64 *pte,unsigned long *page_size, +unsigned long *count) +{ +unsigned long pte_mask, pg_size, cnt; +u64 *fpte; + + pg_size =PTE_PAGE_SIZE (*pte); + cnt =PAGE_SIZE_PTE_COUNT (pg_size); + pte_mask = ~((cnt <<3 ) -1 ); + fpte = (u64 *)(((unsigned long )pte) & pte_mask); + +if (page_size) + *page_size = pg_size; + +if (count) + *count = cnt; + +return fpte; +} +/**************************************************************************** * * Interrupt handling functions
@@ -1567,17 +1590,12 @@ static u64 *fetch_pte(struct protection_domain *domain, *page_size =PTE_LEVEL_PAGE_SIZE (level); } -if (PM_PTE_LEVEL (*pte) ==0x07 ) { -unsigned long pte_mask; - -/* -* If we have a series of large PTEs, make -* sure to return a pointer to the first one. -*/ - *page_size = pte_mask =PTE_PAGE_SIZE (*pte); - pte_mask = ~((PAGE_SIZE_PTE_COUNT (pte_mask) <<3 ) -1 ); - pte = (u64 *)(((unsigned long )pte) & pte_mask); - } +/* +* If we have a series of large PTEs, make +* sure to return a pointer to the first one. +*/ +if (PM_PTE_LEVEL (*pte) ==PAGE_MODE_7_LEVEL ) + pte =first_pte_l7 (pte, page_size,NULL );return pte; }
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH 4/4 iommu/amd: Unmap all L7 PTEs when downgrading page-sizes
In reply to Andrei Dulea via iommuWhen replacing a large mapping created with page-mode 7 (i.e. non-default page size), tear down the entire series of replicated PTEs. Besides providing access to the old mapping, another thing that might go wrong with this issue is on the fetch_pte() code path that can return a PDE entry of the newly re-mapped range.
While at it, make sure that we flush the TLB in case alloc_pte() fails and returns NULL at a lower level.
--- drivers/iommu/amd_iommu.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-)
drivers/iommu/amd_iommu.c +27 -3
@@ -1512,10 +1512,32 @@ static u64 *alloc_pte(struct protection_domain *domain, __pte = *pte; pte_level =PM_PTE_LEVEL (__pte); -if (!IOMMU_PTE_PRESENT (__pte) || - pte_level ==PAGE_MODE_NONE || +/* +* If we replace a series of large PTEs, we need +* to tear down all of them. +*/ +if (IOMMU_PTE_PRESENT (__pte) && pte_level ==PAGE_MODE_7_LEVEL ) { +unsigned long count, i; +u64 *lpte; + + lpte =first_pte_l7 (pte,NULL , &count); + +/* +* Unmap the replicated PTEs that still match the +* original large mapping +*/ +for (i =0 ; i < count; ++i) +cmpxchg64 (&lpte[i], __pte,0ULL ); + + domain->updated = true; +continue ; + } + +if (!IOMMU_PTE_PRESENT (__pte) || + pte_level ==PAGE_MODE_NONE ) { page = (u64 *)get_zeroed_page (gfp); +if (!page)return NULL ;
@@ -1646,8 +1668,10 @@ static int iommu_map_page(struct protection_domain *dom, count =PAGE_SIZE_PTE_COUNT (page_size); pte =alloc_pte (dom, bus_addr, page_size,NULL , gfp); -if (!pte) +if (!pte) { +update_domain (dom);return -ENOMEM ; + }for (i =0 ; i < count; ++i) freelist =free_clear_pte (&pte[i], pte[i], freelist);
Signature · 6 lines
2.17.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu