mm: fixes for THP migration, swap-in and device-private smaps

This series contains three fixes for existing THP migration, PTE swap-in and smaps paths. They were extracted from the work on PMD swap entries for anonymous THPs [1] and can be merged independently. Sending this separately as requested by Andrew (should have sent them separately anyways, my bad).

1. Avoid a folio reference leak when concurrent MADV_DONTNEED clears or splits a migration PMD before the device migration split helper acquires the PMD lock.

2. Prevent PTE swap-in batching from mapping poisoned subpages of a large swap-cache folio. Include the subpage scan helper with this fix so it can be applied on its own.

3. Account device-private PMDs using mapcounts in smaps, matching the PTE path. Shared mappings get proportional PSS, and exclusive mappings are counted as private.

[1] PMD-level swap entries for anonymous THPs:
https://lore.kernel.org/r/[email protected]/

Usama Arif (3):
mm: don't PTE-batch a swap-in over a hardware-poisoned subpage fs/proc/task_mmu: don't read young/dirty from a non-present PMD in
smaps
fs/proc/task_mmu: account device-private PMDs as present in smaps

 fs/proc/task_mmu.c | 12 ++++++++----
 mm/internal.h      | 15 +++++++++++++++
 mm/memory.c        |  4 ++++
 3 files changed, 27 insertions(+), 4 deletions(-)

2.53.0-Meta

do_swap_page() checks PageHWPoison() on the faulting page, but the large-folio batching path then maps the rest of the folio without looking at the other subpages. A poisoned subpage that is not the one being faulted gets mapped anyway, and the fault that would have reported it never happens.

Skip batching for such a folio and let each page take its own fault, where the existing check catches it.

Scan the subpages rather than testing the folio-level PG_has_hwpoisoned:
memory_failure() sets PageHWPoison() on the subpage first and only sets the folio flag much later, and the bail-outs in between - for example get_hwpoison_page() losing a page lifecycle race - return with the subpage marked and the folio flag never set.

---
 mm/internal.h | 15 +++++++++++++++
 mm/memory.c   |  4 ++++
 2 files changed, 19 insertions(+)

index cd3675c044e0a..e61c02f4019b0 100644
@@ -28,6 +28,21 @@ struct folio_batch;
 int __copy_remote_mm_str(struct mm_struct *mm, unsigned long addr,
 		void *buf, int len, unsigned int gup_flags);
 
+/*
+ * Check every page of the folio for HWPoison, rather than relying on the
+ * folio-level PG_has_hwpoisoned, which memory_failure() can leave clear.
+ */
+static inline bool folio_has_any_hwpoisoned_page(const struct folio *folio)
+{
+	long nr = folio_nr_pages(folio);
+	long i;
+
+	for (i = 0; i < nr; i++)
+		if (PageHWPoison(folio_page(folio, i)))
+			return true;
+	return false;
+}
+
 /* mm/workingset.c */
 bool workingset_test_recent(void *shadow, bool file, bool *workingset,
 			    bool flush);

index d477a4f986bc8..4b7d456332200 100644
@@ -5158,6 +5158,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 		    swap_pte_batch(folio_ptep, nr, folio_pte) != nr)
 			goto check_folio;
 
+		/* Let a poisoned subpage take its own fault. */
+		if (folio_has_any_hwpoisoned_page(folio))
+			goto check_folio;
+
 		page_idx = idx;
 		address = folio_start;
 		ptep = folio_ptep;

2.53.0-Meta

smaps_pmd_entry() passes pmd_young(*pmd) and pmd_dirty(*pmd) to smaps_account() for every PMD, including migration and device-private entries, where those bits hold part of the swap offset rather than accessed/dirty state. smaps_pte_entry() only reads them when the entry is present.

On x86_64 the offset is stored inverted across bits 9-58, which overlap _PAGE_SAVED_DIRTY, so pmd_dirty() returns garbage: a clean 2MB device-private mapping is accounted as 2048 kB dirty, and lazyfree accounting is suppressed. pgtable_64.h also warns that the hardware can set the accessed and dirty bits on non-present entries by mistake.

Only read the bits when the PMD is present.

---
 fs/proc/task_mmu.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

index 22361607020d1..c0d391186c2b8 100644
@@ -1019,13 +1019,15 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
 	struct vm_area_struct *vma = walk->vma;
 	bool locked = !!(vma->vm_flags & VM_LOCKED);
 	struct page *page = NULL;
-	bool present = false;
+	bool present = false, young = false, dirty = false;
 	struct folio *folio;
 
 	if (pmd_none(*pmd))
 		return;
 	if (pmd_present(*pmd)) {
 		page = vm_normal_page_pmd(vma, addr, *pmd);
+		young = pmd_young(*pmd);
+		dirty = pmd_dirty(*pmd);
 		present = true;
 	} else if (unlikely(thp_migration_supported())) {
 		const softleaf_t entry = softleaf_from_pmd(*pmd);

@@ -1045,8 +1047,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
 	else
 		mss->file_thp += HPAGE_PMD_SIZE;
 
-	smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd),
-		      locked, present);
+	smaps_account(mss, page, true, young, dirty, locked, present);
 }
 #else
 static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,

2.53.0-Meta

Device-private PMDs represent mapped pages, but smaps_pmd_entry() passes present=false to smaps_account(). That treats them like migration
entries: mapcounts are ignored, PSS is charged in full, and exclusive mappings are accounted as shared.

Match smaps_pte_entry() and mark device-private PMDs as present so smaps_account() uses mapcounts for PSS and private/shared accounting.

---
 fs/proc/task_mmu.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

index c0d391186c2b8..ad97532fc1184 100644
@@ -1032,8 +1032,11 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr,
 	} else if (unlikely(thp_migration_supported())) {
 		const softleaf_t entry = softleaf_from_pmd(*pmd);
 
-		if (softleaf_has_pfn(entry))
+		if (softleaf_has_pfn(entry)) {
+			if (softleaf_is_device_private(entry))
+				present = true;
 			page = softleaf_to_page(entry);
+		}
 	}
 	if (IS_ERR_OR_NULL(page))
 		return;

2.53.0-Meta