mm/khugepaged: clean up refcount check using folio_expected_ref_count()
Subsystems (2)
PATCH v2 1/2 mm/khugepaged: clean up refcount check using folio_expected_ref_count()
Use folio_expected_ref_count() instead of open-coded logic in is_refcount_suitable(). This avoids code duplication and improves clarity.
Drop is_refcount_suitable() as it is no longer needed.
--- mm/khugepaged.c | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-)
mm/khugepaged.c +3 -16
@@ -548,19 +548,6 @@ static void release_pte_pages(pte_t *pte, pte_t *_pte, } } -static bool is_refcount_suitable (struct folio *folio) -{ -int expected_refcount =folio_mapcount (folio); - -if (!folio_test_anon (folio) ||folio_test_swapcache (folio)) - expected_refcount +=folio_nr_pages (folio); - -if (folio_test_private (folio)) - expected_refcount++; - -return folio_ref_count (folio) == expected_refcount; -} - static int __collapse_huge_page_isolate(struct vm_area_struct *vma, unsigned long address, pte_t *pte,
@@ -652,7 +639,7 @@ static int __collapse_huge_page_isolate(struct vm_area_struct *vma,* but not from this process. The other process cannot write to * the page, only trigger CoW. */ -if (!is_refcount_suitable (folio)) { +if (folio_expected_ref_count (folio) !=folio_ref_count (folio)) {folio_unlock (folio); result =SCAN_PAGE_COUNT ; goto out;
@@ -1402,7 +1389,7 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm,* has excessive GUP pins (i.e. 512). Anyway the same check * will be done again later the risk seems low. */ -if (!is_refcount_suitable (folio)) { +if (folio_expected_ref_count (folio) !=folio_ref_count (folio)) { result =SCAN_PAGE_COUNT ; goto out_unmap; }
@@ -2320,7 +2307,7 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,break ; } -if (!is_refcount_suitable (folio)) { +if (folio_expected_ref_count (folio) !=folio_ref_count (folio)) { result =SCAN_PAGE_COUNT ;break ; }
Signature · 1 line
2.34.1
PATCH v2 2/2 mm/khugepaged: fix race with folio split/free using temporary reference
hpage_collapse_scan_file() calls folio_expected_ref_count(), which in turn calls folio_mapcount(). folio_mapcount() checks folio_test_large() before proceeding to folio_large_mapcount(), but there is a race window where the folio may get split/freed between these checks, triggering:
VM_WARN_ON_FOLIO(!folio_test_large(folio), folio)
Take a temporary reference to the folio in hpage_collapse_scan_file(). This stabilizes the folio during refcount check and prevents incorrect large folio detection due to concurrent split/free.
---
V1: https://lore.kernel.org/linux-mm/[email protected]
--- mm/khugepaged.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)
mm/khugepaged.c +17 -1
@@ -2282,6 +2282,17 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,continue ; } +if (!folio_try_get (folio)) { +xas_reset (&xas); +continue ; + } + +if (unlikely (folio !=xas_reload (&xas))) { +folio_put (folio); +xas_reset (&xas); +continue ; + } +if (folio_order (folio) ==HPAGE_PMD_ORDER && folio->index == start) {/* Maybe PMD-mapped */
@@ -2292,23 +2303,27 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,* it's safe to skip LRU and refcount checks before * returning. */ +folio_put (folio);break ; } node =folio_nid (folio);if (hpage_collapse_scan_abort (node, cc)) { result =SCAN_SCAN_ABORT ; +folio_put (folio);break ; } cc->node_load[node]++;if (!folio_test_lru (folio)) { result =SCAN_PAGE_LRU ; +folio_put (folio);break ; } -if (folio_expected_ref_count (folio) !=folio_ref_count (folio)) { +if (folio_expected_ref_count (folio) +1 !=folio_ref_count (folio)) { result =SCAN_PAGE_COUNT ; +folio_put (folio);break ; }
@@ -2320,6 +2335,7 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,*/ present +=folio_nr_pages (folio); +folio_put (folio);if (need_resched ()) {xas_pause (&xas);
Signature · 1 line
2.34.1
Re: [PATCH V2 2/2] mm/khugepaged: fix race with folio split/free using temporary reference
Quoted 11 lines · Shivank Garg
hpage_collapse_scan_file()callsfolio_expected_ref_count(), which in turn callsfolio_mapcount().folio_mapcount()checksfolio_test_large()before proceeding tofolio_large_mapcount(), but there is a race window where the folio may get split/freed between these checks, triggering:
VM_WARN_ON_FOLIO(!folio_test_large(folio), folio)Take a temporary reference to the folio in
hpage_collapse_scan_file(). This stabilizes the folio during refcount check and prevents incorrect large folio detection due to concurrent split/free.
Quoted 8 lines
Fixes 05c5323b2a34 ("mm: track mapcount of large folios in single value")Reported-by [email protected]Suggested-by David Hildenbrand <[email protected]>Signed-off-by Shivank Garg <[email protected]>---
V1: https://lore.kernel.org/linux-mm/[email protected]
---
Assuming we have this as patch #1:
Signature · 3 lines
Cheers, David / dhildenb
Quoted 9 lines · Shivank Garg
Use
folio_expected_ref_count()instead of open-coded logic inis_refcount_suitable(). This avoids code duplication and improves clarity.Drop
is_refcount_suitable()as it is no longer needed.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Shivank Garg <[email protected]>---
Likely we should revert the patches, so we can have the fix in first.
So in this patch here, we would only convert the remaining 2 instances.
Signature · 3 lines
Cheers, David / dhildenb