Skip to contentSkip to conversation

khugepaged: Reduce race probability between migration and khugepaged

PATCH Applied Dev Jain 3 participants 3 messages Jul 4, 2025 – Jul 23, 2025

Subsystems (2)
lore
Share & tools
lore

Dev Jain

author reviewer
#0 lore raw

PATCH v3 1/1 khugepaged: Reduce race probability between migration and khugepaged

Suppose a folio is under migration, and khugepaged is also trying to collapse it. collapse_pte_mapped_thp() will retrieve the folio from the page cache via filemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in
__folio_migrate_mapping -> folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.

This fixes the migration-shared-anon-thp testcase failure on Apple M3.

Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".

---

v2->v3:
- Improve comment (David)

v1->v2:
- Remove SCAN_PMD_MIGRATION, merge into SCAN_PMD_MAPPED (David, Anshuman)
- Add a comment (Lorenzo)

v1:
- https://lore.kernel.org/all/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -0
index 1aa7ca67c756..a55fb1dcd224 100644
@@ -941,6 +941,14 @@ static inline int check_pmd_state(pmd_t *pmd)
 
 	if (pmd_none(pmde))
 		return SCAN_PMD_NONE;
+
+	/*
+	 * The folio may be under migration when khugepaged is trying to
+	 * collapse it. Migration success or failure will eventually end
+	 * up with a present PMD mapping a folio again.
+	 */
+	if (is_pmd_migration_entry(pmde))
+		return SCAN_PMD_MAPPED;
 	if (!pmd_present(pmde))
 		return SCAN_PMD_NULL;
 	if (pmd_trans_huge(pmde))
Signature · 1 line
2.30.2
Reviewed-by on v3 1/1
#1 lore raw
Quoted 15 lines · Dev Jain

Suppose a folio is under migration, and khugepaged is also trying to collapse it. collapse_pte_mapped_thp() will retrieve the folio from the page cache via filemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in
__folio_migrate_mapping -> folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.

This fixes the migration-shared-anon-thp testcase failure on Apple M3.

Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".

Quoted 6 lines

LGTM, so:

Quoted 37 lines

---

v2->v3:
- Improve comment (David)

v1->v2:
- Remove SCAN_PMD_MIGRATION, merge into SCAN_PMD_MAPPED (David, Anshuman)
- Add a comment (Lorenzo)

v1:
- https://lore.kernel.org/all/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -1
index 1aa7ca67c756..a55fb1dcd224 100644
@@ -941,6 +941,14 @@ static inline int check_pmd_state(pmd_t *pmd)
 
 	if (pmd_none(pmde))
 		return SCAN_PMD_NONE;
+
+	/*
+	 * The folio may be under migration when khugepaged is trying to
+	 * collapse it. Migration success or failure will eventually end
+	 * up with a present PMD mapping a folio again.
+	 */
+	if (is_pmd_migration_entry(pmde))
+		return SCAN_PMD_MAPPED;
 	if (!pmd_present(pmde))
 		return SCAN_PMD_NULL;
 	if (pmd_trans_huge(pmde))
--

2.30.2

Reviewed-by on v3 1/1
#2 lore raw
In reply to Dev Jain
Quoted 16 lines · Dev Jain

Suppose a folio is under migration, and khugepaged is also trying to collapse it. collapse_pte_mapped_thp() will retrieve the folio from the page cache via filemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in
__folio_migrate_mapping -> folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.

This fixes the migration-shared-anon-thp testcase failure on Apple M3.

Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".

Quoted 6 lines

LGTM! This is a nice check to have here :)

Quoted 37 lines

---

v2->v3:
- Improve comment (David)

v1->v2:
- Remove SCAN_PMD_MIGRATION, merge into SCAN_PMD_MAPPED (David, Anshuman)
- Add a comment (Lorenzo)

v1:
- https://lore.kernel.org/all/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -1
index 1aa7ca67c756..a55fb1dcd224 100644
@@ -941,6 +941,14 @@ static inline int check_pmd_state(pmd_t *pmd)
 
        if (pmd_none(pmde))
                return SCAN_PMD_NONE;
+
+       /*
+        * The folio may be under migration when khugepaged is trying to
+        * collapse it. Migration success or failure will eventually end
+        * up with a present PMD mapping a folio again.
+        */
+       if (is_pmd_migration_entry(pmde))
+               return SCAN_PMD_MAPPED;
        if (!pmd_present(pmde))
                return SCAN_PMD_NULL;
        if (pmd_trans_huge(pmde))
--

2.30.2