khugepaged: Reduce race probability between migration and khugepaged
Subsystems (2)
Dev Jain
reviewerPATCH v2 1/1 khugepaged: Reduce race probability between migration and khugepaged
Suppose a folio is under migration, and khugepaged is also trying to collapse it. collapse_pte_mapped_thp() will retrieve the folio from the page cache via filemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping -> folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.
This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
---
v1->v2:
- Remove SCAN_PMD_MIGRATION, merge into SCAN_PMD_MAPPED (David, Anshuman)
- Add a comment (Lorenzo)
v1:
- https://lore.kernel.org/all/[email protected]/
mm/khugepaged.c | 9 +++++++++ 1 file changed, 9 insertions(+)
mm/khugepaged.c +9 -0
@@ -941,6 +941,15 @@ static inline int check_pmd_state(pmd_t *pmd)if (pmd_none (pmde))return SCAN_PMD_NONE ; + +/* +* The folio may be under migration when khugepaged is trying to +* collapse it. Migration success or failure will eventually end +* up with the PMD still pointing to a PMD-order folio, so return +* SCAN_PMD_MAPPED. +*/ +if (is_pmd_migration_entry (pmde)) +return SCAN_PMD_MAPPED ;if (!pmd_present (pmde))return SCAN_PMD_NULL ;if (pmd_trans_huge (pmde))
Signature · 1 line
2.30.2
Quoted 33 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>---
v1->v2:
- RemoveSCAN_PMD_MIGRATION, merge intoSCAN_PMD_MAPPED(David, Anshuman)
- Add a comment (Lorenzo)v1:
- https://lore.kernel.org/all/[email protected]/mm/khugepaged.c | 9 +++++++++ 1 file changed, 9 insertions(+)mm/khugepaged.c +0
-0
mm/khugepaged.c static inline int check_pmd_state(pmd_t *pmd) · line 949
Quoted 9 lines
mm/khugepaged.c +6
-0@@ -941,6 +941,15 @@ static inline int check_pmd_state(pmd_t *pmd)if (pmd_none (pmde))return SCAN_PMD_NONE ; + +/* +* The folio may be under migration when khugepaged is trying to +* collapse it. Migration success or failure will eventually end +* up with the PMD still pointing to a PMD-order folio, so return +* SCAN_PMD_MAPPED.
Nit: the last part (, so return ..) is obvious from the code.
I would have written
/*
* The folio may be under migration when khugepaged is trying to
* collapse it. Migration success or failure will eventually end
* up with a present PMD entry again.
*/
Signature · 3 lines
Cheers, David / dhildenb
Anshuman Khandual
Reviewed-byQuoted 45 lines · David Hildenbrand
Quoted 33 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>---
v1->v2:
- RemoveSCAN_PMD_MIGRATION, merge intoSCAN_PMD_MAPPED(David, Anshuman)
- Add a comment (Lorenzo)v1:
- https://lore.kernel.org/all/[email protected]/
mm/khugepaged.c| 9 +++++++++1 file changed, 9 insertions(+)mm/khugepaged.c +0
-0mm/khugepaged.c static inline int check_pmd_state(pmd_t *pmd) · line 948
Quoted 8 lines
mm/khugepaged.c +6
-0@@ -941,6 +941,15 @@ static inline int check_pmd_state(pmd_t *pmd) if (pmd_none(pmde)) return SCAN_PMD_NONE; + + /* +* The folio may be under migration when khugepaged is trying to +* collapse it. Migration success or failure will eventually end +* up with the PMD still pointing to a PMD-order folio, so return +* SCAN_PMD_MAPPED. Nit: the last part (, so return ..) is obvious from the code.
Quoted 7 lines
I would have written
/*
* The folio may be under migration when khugepaged is trying to
* collapse it. Migration success or failure will eventually end
* up with a present PMD entry again.
*/
+1
Quoted 4 lines
Dev Jain
reviewerQuoted 45 lines · David Hildenbrand
Quoted 33 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>---
v1->v2:
- RemoveSCAN_PMD_MIGRATION, merge intoSCAN_PMD_MAPPED(David, Anshuman)
- Add a comment (Lorenzo)v1:
- https://lore.kernel.org/all/[email protected]/
mm/khugepaged.c| 9 +++++++++1 file changed, 9 insertions(+)mm/khugepaged.c +0
-0mm/khugepaged.c static inline int check_pmd_state(pmd_t *pmd) · line 948
Quoted 8 lines
mm/khugepaged.c +6
-0@@ -941,6 +941,15 @@ static inline int check_pmd_state(pmd_t *pmd) if (pmd_none(pmde)) return SCAN_PMD_NONE; + + /* +* The folio may be under migration when khugepaged is trying to +* collapse it. Migration success or failure will eventually end +* up with the PMD still pointing to a PMD-order folio, so return +* SCAN_PMD_MAPPED. Nit: the last part (, so return ..) is obvious from the code.
Quoted 8 lines
I would have written
/*
* The folio may be under migration when khugepaged is trying to
* collapse it. Migration success or failure will eventually end
* up with a present PMD entry again.
*/
Thanks for the suggestion, but
PMD pointing to PMD-order folio necessarily implies present PMD entry,
but the converse is not true? For example it may point to a PTE table.
So I wanted to make it explicitly clear that the transient entry will be
converted to a PMD-THP in the end.
Quoted 2 lines
Thanks!
Quoted 57 lines · Dev Jain
Quoted 47 lines · David Hildenbrand
Quoted 35 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio
and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>---
v1->v2:
- RemoveSCAN_PMD_MIGRATION, merge intoSCAN_PMD_MAPPED(David,
Anshuman)
- Add a comment (Lorenzo)v1:
- https://lore.kernel.org/all/[email protected]/mm/khugepaged.c | 9 +++++++++ 1 file changed, 9 insertions(+)mm/khugepaged.c +0
-0mm/khugepaged.c static inline int check_pmd_state(pmd_t *pmd) · line 948
Quoted 8 lines
mm/khugepaged.c +6
-0@@ -941,6 +941,15 @@ static inline int check_pmd_state(pmd_t *pmd) if (pmd_none(pmde)) return SCAN_PMD_NONE; + + /* +* The folio may be under migration when khugepaged is trying to +* collapse it. Migration success or failure will eventually end +* up with the PMD still pointing to a PMD-order folio, so return +* SCAN_PMD_MAPPED. Nit: the last part (, so return ..) is obvious from the code.
Quoted 8 lines
I would have written
/*
* The folio may be under migration when khugepaged is trying to
* collapse it. Migration success or failure will eventually end
* up with a present PMD entry again.
*/
Quoted 4 lines
Thanks for the suggestion, but
PMD pointing to PMD-order folio necessarily implies present PMD entry,
Quoted 1 line
but the converse is not true? For example it may point to a PTE table.
I see, talking about orders is confusing though.
"with a present PMD mapping a folio again."
Signature · 3 lines
Cheers, David / dhildenb
Dev Jain
reviewerQuoted 60 lines · David Hildenbrand
Quoted 59 lines · Dev Jain
Quoted 47 lines · David Hildenbrand
Quoted 35 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio
and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>---
v1->v2:
- RemoveSCAN_PMD_MIGRATION, merge intoSCAN_PMD_MAPPED(David, Anshuman)
- Add a comment (Lorenzo)v1:
- https://lore.kernel.org/all/[email protected]/
mm/khugepaged.c| 9 +++++++++1 file changed, 9 insertions(+)mm/khugepaged.c +0
-0mm/khugepaged.c static inline int check_pmd_state(pmd_t *pmd) · line 948
Quoted 8 lines
mm/khugepaged.c +6
-0@@ -941,6 +941,15 @@ static inline int check_pmd_state(pmd_t *pmd) if (pmd_none(pmde)) return SCAN_PMD_NONE; + + /* +* The folio may be under migration when khugepaged is trying to +* collapse it. Migration success or failure will eventually end +* up with the PMD still pointing to a PMD-order folio, so return +* SCAN_PMD_MAPPED. Nit: the last part (, so return ..) is obvious from the code.
Quoted 8 lines
I would have written
/*
* The folio may be under migration when khugepaged is trying to
* collapse it. Migration success or failure will eventually end
* up with a present PMD entry again.
*/Thanks for the suggestion, but
Quoted 7 lines
Quoted 2 lines
PMD pointing to PMD-order folio necessarily implies present PMD entry,
Quoted 1 line
but the converse is not true? For example it may point to a PTE
table.
I see, talking about orders is confusing though.
"with a present PMD mapping a folio again."
Sure. Shall respin.
Quoted 9 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.
Quoted 7 lines
This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>
David's comment refering to a 'present PMD entry' seems more clear to me, but
Signature · 2 lines
Oscar Salvador SUSE Labs
Quoted 22 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>---
v1->v2:
- RemoveSCAN_PMD_MIGRATION, merge intoSCAN_PMD_MAPPED(David, Anshuman)
- Add a comment (Lorenzo)
Quoted 6 lines
v1:
- https://lore.kernel.org/all/[email protected]/mm/khugepaged.c | 9 +++++++++ 1 file changed, 9 insertions(+)
Best Regards,
Yan, Zi
Quoted 9 lines · Dev Jain
Suppose a folio is under migration, and khugepaged is also trying to collapse it.
collapse_pte_mapped_thp()will retrieve the folio from the page cache viafilemap_lock_folio(), thus taking a reference on the folio and sleeping on the folio lock, since the lock is held by the migration path. Migration will then fail in__folio_migrate_mapping->folio_ref_freeze. Reduce the probability of such a race happening (leading to migration failure) by bailing out if we detect a PMD is marked with a migration entry.
Quoted 7 lines
This fixes the migration-shared-anon-thp testcase failure on Apple M3.
Note that, this is not a "fix" since it only reduces the chance of interference of khugepaged with migration, wherein both the kernel functionalities are deemed "best-effort".
Signed-off-by Dev Jain <[email protected]>
With David's comments addressed, LGTM.