mm: handle device-private PMDs in walk callbacks

Since commit 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations") a PMD may hold a device-private swap entry whenever an HMM-based GPU driver migrates an anonymous THP folio to device memory via migrate_vma_pages().

pmd_trans_huge_lock() succeeds for such PMDs (pmd_is_huge() returns true for any non-present, non-none huge PMD), so several MM walk callbacks that used to assume present THP or migration entry are now reachable with a device-private PMD. The results range from a VM_BUG_ON() firing on debug kernels, to an oops on a bogus vmemmap dereference, to silently isolating an unrelated live folio from LRU in the aliasing case.

---
v2 -> v3: https://lore.kernel.org/all/[email protected]/
- Patch 1: use pmdp_get() instead of *pmd. (Zi)
- Patches 2 and 3: drop the thp_migration_supported() guard and downgrade VM_BUG_ON to VM_WARN_ON_ONCE(). thp_migration_supported() expands to IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_SOFTLEAF), and both pmd_is_migration_entry() and pmd_is_device_private_entry() already return false without that config, so the guard suppresses only the case where the warning would already be silent. (David)

v1 -> v2: https://lore.kernel.org/all/[email protected]/
- Patch 1 now gates queue_folios_pmd() on !pmd_present() instead of checking only pmd_is_device_private_entry(). This matches the PTE path, keeps migration entries counted in qp->nr_failed, and skips other non-present PMDs such as device-private entries before calling pmd_folio(). (Joshua)
- Patches 2 and 3 now fold device-private PMD handling into the existing !pmd_present() VM_BUG_ON() condition instead of using a separate early pmd_is_device_private_entry() check. (Joshua)
- cc stable (Zi)

Usama Arif (3):
mm/mempolicy: skip non-present PMDs when queueing folios mm/madvise: skip device-private PMDs in cold and pageout walks mm/huge_memory: skip device-private PMDs in madvise_free_huge_pmd

 mm/huge_memory.c | 4 ++--
 mm/madvise.c     | 4 ++--
 mm/mempolicy.c   | 8 +++++---
 3 files changed, 9 insertions(+), 7 deletions(-)

2.53.0-Meta