mm/mapping_dirty_helpers: Update huge page-table entry callbacks
Subsystems (2)
PATCH 1/1 mm/mapping_dirty_helpers: Update huge page-table entry callbacks
From: Thomas Hellstrom <[email protected]>
Following the update of pagewalk code
commit a07984d48146 ("mm: pagewalk: add p4d_entry() and pgd_entry()") we can modify the mapping_dirty_helpers' huge page-table entry callbacks to avoid splitting when a huge pud or -pmd is encountered.
Cc Andrew Morton <[email protected]>
Signed-off-by Thomas Hellstrom <[email protected]>
Reviewed-by Steven Price <[email protected]>
---
v2: Added R-B from Steven.
--- mm/mapping_dirty_helpers.c | 42 ++++++++++++++++++++++++++++++++++---- 1 file changed, 38 insertions(+), 4 deletions(-)
mm/mapping_dirty_helpers.c +38 -4
@@ -111,26 +111,60 @@ static int clean_record_pte(pte_t *pte, unsigned long addr,return 0 ; } -/* wp_clean_pmd_entry - The pagewalk pmd callback. */ +/* +* wp_clean_pmd_entry - The pagewalk pmd callback. +* +* Dirty-tracking should take place on the PTE level, so +* WARN() if encountering a dirty huge pmd. +* Furthermore, never split huge pmds, since that currently +* causes dirty info loss. The pagefault handler should do +* that if needed. +*/ static int wp_clean_pmd_entry (pmd_t *pmd,unsigned long addr,unsigned long end,struct mm_walk *walk) { -/* Dirty-tracking should be handled on the pte level */ pmd_t pmdval =pmd_read_atomic (pmd); +if (!pmd_trans_unstable (&pmdval)) +return 0 ; + +if (pmd_none (pmdval)) { + walk->action =ACTION_AGAIN ; +return 0 ; + } + +/* Huge pmd, present or migrated */ + walk->action =ACTION_CONTINUE ;if (pmd_trans_huge (pmdval) ||pmd_devmap (pmdval))WARN_ON (pmd_write (pmdval) ||pmd_dirty (pmdval));return 0 ; } -/* wp_clean_pud_entry - The pagewalk pud callback. */ +/* +* wp_clean_pud_entry - The pagewalk pud callback. +* +* Dirty-tracking should take place on the PTE level, so +* WARN() if encountering a dirty huge puds. +* Furthermore, never split huge puds, since that currently +* causes dirty info loss. The pagefault handler should do +* that if needed. +*/ static int wp_clean_pud_entry (pud_t *pud,unsigned long addr,unsigned long end,struct mm_walk *walk) { -/* Dirty-tracking should be handled on the pte level */ pud_t pudval =READ_ONCE (*pud); +if (!pud_trans_unstable (&pudval)) +return 0 ; + +if (pud_none (pudval)) { + walk->action =ACTION_AGAIN ; +return 0 ; + } + +/* Huge pud */ + walk->action =ACTION_CONTINUE ;if (pud_trans_huge (pudval) ||pud_devmap (pudval))WARN_ON (pud_write (pudval) ||pud_dirty (pudval));
Signature · 1 line
2.21.1