mm/thp: fix __split_huge_pmd_locked() for migration PMD
Subsystems (3)
PATCH 1/1 mm/thp: fix __split_huge_pmd_locked() for migration PMD
From: Zi Yan <[email protected]>
For 4.19.
[Upstream commitid ec0abae6dcdf7ef88607c869bf35a4b63ce1b370]
From: Ralph Campbell <[email protected]>
Date: Fri, 18 Sep 2020 21:20:24 -0700
Subject: [PATCH] mm/thp: fix __split_huge_pmd_locked() for migration PMD
A migrating transparent huge page has to already be unmapped. Otherwise, the page could be modified while it is being copied to a new page and data could be lost. The function __split_huge_pmd() checks for a PMD migration entry before calling __split_huge_pmd_locked() leading one to think that __split_huge_pmd_locked() can handle splitting a migrating PMD.
However, the code always increments the page->_mapcount and adjusts the memory control group accounting assuming the page is mapped.
Also, if the PMD entry is a migration PMD entry, the call to is_huge_zero_pmd(*pmd) is incorrect because it calls pmd_pfn(pmd) instead of migration_entry_to_pfn(pmd_to_swp_entry(pmd)). Fix these problems by checking for a PMD migration entry.
--- mm/huge_memory.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-)
mm/huge_memory.c +20 -17
@@ -2145,7 +2145,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,put_page (page);add_mm_counter (mm,mm_counter_file (page), -HPAGE_PMD_NR );return ; - }else if (is_huge_zero_pmd (*pmd)) { + }else if (pmd_trans_huge (*pmd) &&is_huge_zero_pmd (*pmd)) {/* * FIXME: Do we want to invalidate secondary mmu by calling * mmu_notifier_invalidate_range() see comments below inside
@@ -2233,26 +2233,29 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, pte =pte_offset_map (&_pmd, addr);BUG_ON (!pte_none (*pte));set_pte_at (mm, addr, pte, entry); -atomic_inc (&page[i]._mapcount); -pte_unmap (pte); - } - -/* -* Set PG_double_map before dropping compound_mapcount to avoid -* false-negative page_mapped(). -*/ -if (compound_mapcount (page) >1 && !TestSetPageDoubleMap (page)) { -for (i =0 ; i <HPAGE_PMD_NR ; i++) +if (!pmd_migration)atomic_inc (&page[i]._mapcount); +pte_unmap (pte); } -if (atomic_add_negative (-1 ,compound_mapcount_ptr (page))) { -/* Last compound_mapcount is gone. */ -__dec_node_page_state (page,NR_ANON_THPS ); -if (TestClearPageDoubleMap (page)) { -/* No need in mapcount reference anymore */ +if (!pmd_migration) { +/* +* Set PG_double_map before dropping compound_mapcount to avoid +* false-negative page_mapped(). +*/ +if (compound_mapcount (page) >1 && !TestSetPageDoubleMap (page)) {for (i =0 ; i <HPAGE_PMD_NR ; i++) -atomic_dec (&page[i]._mapcount); +atomic_inc (&page[i]._mapcount); + } + +if (atomic_add_negative (-1 ,compound_mapcount_ptr (page))) { +/* Last compound_mapcount is gone. */ +__dec_node_page_state (page,NR_ANON_THPS ); +if (TestClearPageDoubleMap (page)) { +/* No need in mapcount reference anymore */ +for (i =0 ; i <HPAGE_PMD_NR ; i++) +atomic_dec (&page[i]._mapcount); + } } }
Signature · 1 line
2.28.0
Zi Yan
reviewerQuoted 3 lines · Zi Yan
From: Zi Yan <[email protected]>
For 4.19.
It applies to v5.4.y too.
Quoted 102 lines
[Upstream commitid
ec0abae6dcdf7ef88607c869bf35a4b63ce1b370]From: Ralph Campbell <[email protected]>
Date: Fri, 18 Sep 2020 21:20:24 -0700
Subject: [PATCH] mm/thp: fix__split_huge_pmd_locked()for migration PMDA migrating transparent huge page has to already be unmapped. Otherwise, the page could be modified while it is being copied to a new page and data could be lost. The function
__split_huge_pmd()checks for a PMD migration entry before calling__split_huge_pmd_locked()leading one to think that__split_huge_pmd_locked()can handle splitting a migrating PMD.However, the code always increments the
page->_mapcountand adjusts the memory control group accounting assuming the page is mapped.Also, if the PMD entry is a migration PMD entry, the call to
is_huge_zero_pmd(*pmd)is incorrect because it callspmd_pfn(pmd)instead of migration_entry_to_pfn(pmd_to_swp_entry(pmd)). Fix these problems by checking for a PMD migration entry.Fixes 84c3fc4e9c56 ("mm: thp: check pmd migration entry in common path")Signed-off-by Ralph Campbell <[email protected]>Signed-off-by Andrew Morton <[email protected]>Reviewed-by Yang Shi <[email protected]>Reviewed-by Zi Yan <[email protected]>Cc Jerome Glisse <[email protected]>Cc John Hubbard <[email protected]>Cc Alistair Popple <[email protected]>Cc Christoph Hellwig <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Bharata B Rao <[email protected]>Cc Ben Skeggs <[email protected]>Cc Shuah Khan <[email protected]>Cc <[email protected]> [4.14+]Signed-off-by Linus Torvalds <[email protected]>--- mm/huge_memory.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-)mm/huge_memory.c +20
-17@@ -2145,7 +2145,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,put_page (page);add_mm_counter (mm,mm_counter_file (page), -HPAGE_PMD_NR );return ; - }else if (is_huge_zero_pmd (*pmd)) { + }else if (pmd_trans_huge (*pmd) &&is_huge_zero_pmd (*pmd)) {/* * FIXME: Do we want to invalidate secondary mmu by calling * mmu_notifier_invalidate_range() see comments below inside @@ -2233,26 +2233,29 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, pte =pte_offset_map (&_pmd, addr);BUG_ON (!pte_none (*pte));set_pte_at (mm, addr, pte, entry); -atomic_inc (&page[i]._mapcount); -pte_unmap (pte); - } - -/* -* Set PG_double_map before dropping compound_mapcount to avoid -* false-negative page_mapped(). -*/ -if (compound_mapcount (page) >1 && !TestSetPageDoubleMap (page)) { -for (i =0 ; i <HPAGE_PMD_NR ; i++) +if (!pmd_migration)atomic_inc (&page[i]._mapcount); +pte_unmap (pte); } -if (atomic_add_negative (-1 ,compound_mapcount_ptr (page))) { -/* Last compound_mapcount is gone. */ -__dec_node_page_state (page,NR_ANON_THPS ); -if (TestClearPageDoubleMap (page)) { -/* No need in mapcount reference anymore */ +if (!pmd_migration) { +/* +* Set PG_double_map before dropping compound_mapcount to avoid +* false-negative page_mapped(). +*/ +if (compound_mapcount (page) >1 && !TestSetPageDoubleMap (page)) {for (i =0 ; i <HPAGE_PMD_NR ; i++) -atomic_dec (&page[i]._mapcount); +atomic_inc (&page[i]._mapcount); + } + +if (atomic_add_negative (-1 ,compound_mapcount_ptr (page))) { +/* Last compound_mapcount is gone. */ +__dec_node_page_state (page,NR_ANON_THPS ); +if (TestClearPageDoubleMap (page)) { +/* No need in mapcount reference anymore */ +for (i =0 ; i <HPAGE_PMD_NR ; i++) +atomic_dec (&page[i]._mapcount); + } } }Signature · 1 line
2.28.0
—
Best Regards,
Yan Zi