Skip to contentSkip to conversation

fixup: mm/rmap: extend rmap and migration support device-private entries

PATCH In review Balbir Singh 3 participants 5 messages Nov 15, 2025 – Nov 17, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fixup: mm/rmap: extend rmap and migration support device-private entries

Follow the pattern used in remove_migration_pte() in remove_migration_pmd(). Process the migration entries and if the entry type is device private, override the pmde with a device private entry and set the soft dirty and uffd_wp bits with the pmd_swp_mksoft_dirty and pmd_swp_mkuffd_wp

Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Zi Yan <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Lyude Paul <[email protected]>
Cc Danilo Krummrich <[email protected]>
Cc David Airlie <[email protected]>
Cc Simona Vetter <[email protected]>
Cc Ralph Campbell <[email protected]>
Cc Mika Penttilä <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Francois Dugast <[email protected]>

---
This fixup should be squashed into the patch "mm/rmap: extend rmap and migration support" of mm/mm-unstable

 mm/huge_memory.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)
mm/huge_memory.c +17 -10
index 9dda8c48daca..50ba458efcab 100644
@@ -4698,16 +4698,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
 	folio_get(folio);
 	pmde = folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot));
 
-	if (folio_is_device_private(folio)) {
-		if (pmd_write(pmde))
-			entry = make_writable_device_private_entry(
-							page_to_pfn(new));
-		else
-			entry = make_readable_device_private_entry(
-							page_to_pfn(new));
-		pmde = swp_entry_to_pmd(entry);
-	}
-
 	if (pmd_swp_soft_dirty(*pvmw->pmd))
 		pmde = pmd_mksoft_dirty(pmde);
 	if (is_writable_migration_entry(entry))
@@ -4720,6 +4710,23 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
 	if (folio_test_dirty(folio) && is_migration_entry_dirty(entry))
 		pmde = pmd_mkdirty(pmde);
 
+	if (folio_is_device_private(folio)) {
+		swp_entry_t entry;
+
+		if (pmd_write(pmde))
+			entry = make_writable_device_private_entry(
+							page_to_pfn(new));
+		else
+			entry = make_readable_device_private_entry(
+							page_to_pfn(new));
+		pmde = swp_entry_to_pmd(entry);
+
+		if (pmd_swp_soft_dirty(*pvmw->pmd))
+			pmde = pmd_swp_mksoft_dirty(pmde);
+		if (pmd_swp_uffd_wp(*pvmw->pmd))
+			pmde = pmd_swp_mkuffd_wp(pmde);
+	}
+
 	if (folio_test_anon(folio)) {
 		rmap_t rmap_flags = RMAP_NONE;
 
Signature · 1 line
2.51.1
maintainer
#1 lore raw
Quoted 11 lines · On Sat, 15 Nov 2025 11:28:35 +1100 Balbir Singh <[email protected]> wrote:

Follow the pattern used in remove_migration_pte() in remove_migration_pmd(). Process the migration entries and if the entry type is device private, override the pmde with a device private entry and set the soft dirty and uffd_wp bits with the pmd_swp_mksoft_dirty and pmd_swp_mkuffd_wp

...

This fixup should be squashed into the patch "mm/rmap: extend rmap and migration support" of mm/mm-unstable

OK. After fixing up mm-replace-pmd_to_swp_entry-with-softleaf_from_pmd.patch, mm.git's mm/huge_memory.c has the below. Please double-check.

void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)

{
	struct folio *folio = page_folio(new);
	struct vm_area_struct *vma = pvmw->vma;
	struct mm_struct *mm = vma->vm_mm;
	unsigned long address = pvmw->address;
	unsigned long haddr = address & HPAGE_PMD_MASK;
	pmd_t pmde;
	softleaf_t entry;

	if (!(pvmw->pmd && !pvmw->pte))
		return;

	entry = softleaf_from_pmd(*pvmw->pmd);
	folio_get(folio);
	pmde = folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot));

	if (pmd_swp_soft_dirty(*pvmw->pmd))
		pmde = pmd_mksoft_dirty(pmde);
	if (softleaf_is_migration_write(entry))
		pmde = pmd_mkwrite(pmde, vma);
	if (pmd_swp_uffd_wp(*pvmw->pmd))
		pmde = pmd_mkuffd_wp(pmde);
	if (!softleaf_is_migration_young(entry))
		pmde = pmd_mkold(pmde);
	/* NOTE: this may contain setting soft-dirty on some archs */
	if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry))
		pmde = pmd_mkdirty(pmde);

	if (folio_is_device_private(folio)) {
		swp_entry_t entry;

		if (pmd_write(pmde))
			entry = make_writable_device_private_entry(
							page_to_pfn(new));
		else
			entry = make_readable_device_private_entry(
							page_to_pfn(new));
		pmde = swp_entry_to_pmd(entry);

		if (pmd_swp_soft_dirty(*pvmw->pmd))
			pmde = pmd_swp_mksoft_dirty(pmde);
		if (pmd_swp_uffd_wp(*pvmw->pmd))
			pmde = pmd_swp_mkuffd_wp(pmde);
	}

	if (folio_test_anon(folio)) {
		rmap_t rmap_flags = RMAP_NONE;

		if (!softleaf_is_migration_read(entry))
			rmap_flags |= RMAP_EXCLUSIVE;

		folio_add_anon_rmap_pmd(folio, new, vma, haddr, rmap_flags);
	} else {
		folio_add_file_rmap_pmd(folio, new, vma);
	}
	VM_BUG_ON(pmd_write(pmde) && folio_test_anon(folio) && !PageAnonExclusive(new));
	set_pmd_at(mm, haddr, pvmw->pmd, pmde);

	/* No need to invalidate - it was non-present before */
	update_mmu_cache_pmd(vma, address, pvmw->pmd);
	trace_remove_migration_pmd(address, pmd_val(pmde));
}
#2 lore raw
Quoted 74 lines · Andrew Morton
Quoted 11 lines · On Sat, 15 Nov 2025 11:28:35 +1100 Balbir Singh <[email protected]> wrote:

Follow the pattern used in remove_migration_pte() in remove_migration_pmd(). Process the migration entries and if the entry type is device private, override the pmde with a device private entry and set the soft dirty and uffd_wp bits with the pmd_swp_mksoft_dirty and pmd_swp_mkuffd_wp

...

This fixup should be squashed into the patch "mm/rmap: extend rmap and migration support" of mm/mm-unstable

OK. After fixing up mm-replace-pmd_to_swp_entry-with-softleaf_from_pmd.patch, mm.git's mm/huge_memory.c has the below. Please double-check.

void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)

{
	struct folio *folio = page_folio(new);
	struct vm_area_struct *vma = pvmw->vma;
	struct mm_struct *mm = vma->vm_mm;
	unsigned long address = pvmw->address;
	unsigned long haddr = address & HPAGE_PMD_MASK;
	pmd_t pmde;
	softleaf_t entry;

	if (!(pvmw->pmd && !pvmw->pte))
		return;

	entry = softleaf_from_pmd(*pvmw->pmd);
	folio_get(folio);
	pmde = folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot));

	if (pmd_swp_soft_dirty(*pvmw->pmd))
		pmde = pmd_mksoft_dirty(pmde);
	if (softleaf_is_migration_write(entry))
		pmde = pmd_mkwrite(pmde, vma);
	if (pmd_swp_uffd_wp(*pvmw->pmd))
		pmde = pmd_mkuffd_wp(pmde);
	if (!softleaf_is_migration_young(entry))
		pmde = pmd_mkold(pmde);
	/* NOTE: this may contain setting soft-dirty on some archs */
	if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry))
		pmde = pmd_mkdirty(pmde);

	if (folio_is_device_private(folio)) {
		swp_entry_t entry;

		if (pmd_write(pmde))
			entry = make_writable_device_private_entry(
							page_to_pfn(new));
		else
			entry = make_readable_device_private_entry(
							page_to_pfn(new));
		pmde = swp_entry_to_pmd(entry);

		if (pmd_swp_soft_dirty(*pvmw->pmd))
			pmde = pmd_swp_mksoft_dirty(pmde);
		if (pmd_swp_uffd_wp(*pvmw->pmd))
			pmde = pmd_swp_mkuffd_wp(pmde);
	}

	if (folio_test_anon(folio)) {
		rmap_t rmap_flags = RMAP_NONE;

		if (!softleaf_is_migration_read(entry))
			rmap_flags |= RMAP_EXCLUSIVE;

		folio_add_anon_rmap_pmd(folio, new, vma, haddr, rmap_flags);
	} else {
		folio_add_file_rmap_pmd(folio, new, vma);
Quoted 8 lines
	}
	VM_BUG_ON(pmd_write(pmde) && folio_test_anon(folio) && !PageAnonExclusive(new));
	set_pmd_at(mm, haddr, pvmw->pmd, pmde);

	/* No need to invalidate - it was non-present before */
	update_mmu_cache_pmd(vma, address, pvmw->pmd);
	trace_remove_migration_pmd(address, pmd_val(pmde));
}

Thanks, Andrew! Looks good!

Balbir

#3 lore raw
In reply to Balbir Singh
Quoted 62 lines · Balbir Singh

Follow the pattern used in remove_migration_pte() in remove_migration_pmd(). Process the migration entries and if the entry type is device private, override the pmde with a device private entry and set the soft dirty and uffd_wp bits with the pmd_swp_mksoft_dirty and pmd_swp_mkuffd_wp

Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Zi Yan <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Lyude Paul <[email protected]>
Cc Danilo Krummrich <[email protected]>
Cc David Airlie <[email protected]>
Cc Simona Vetter <[email protected]>
Cc Ralph Campbell <[email protected]>
Cc Mika Penttilä <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Francois Dugast <[email protected]>

---
This fixup should be squashed into the patch "mm/rmap: extend rmap and migration support" of mm/mm-unstable

  mm/huge_memory.c | 27 +++++++++++++++++----------
  1 file changed, 17 insertions(+), 10 deletions(-)
mm/huge_memory.c +0 -10
index 9dda8c48daca..50ba458efcab 100644
@@ -4698,16 +4698,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
  	folio_get(folio);
  	pmde = folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot));
 
-	if (folio_is_device_private(folio)) {
-		if (pmd_write(pmde))
-			entry = make_writable_device_private_entry(
-							page_to_pfn(new));
-		else
-			entry = make_readable_device_private_entry(
-							page_to_pfn(new));
-		pmde = swp_entry_to_pmd(entry);
-	}
-
  	if (pmd_swp_soft_dirty(*pvmw->pmd))
  		pmde = pmd_mksoft_dirty(pmde);
  	if (is_writable_migration_entry(entry))

mm/huge_memory.c void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) · line 4714

Quoted 5 lines
mm/huge_memory.c +2 -0
@@ -4720,6 +4710,23 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
  	if (folio_test_dirty(folio) && is_migration_entry_dirty(entry))
  		pmde = pmd_mkdirty(pmde);
 
+	if (folio_is_device_private(folio)) {
+		swp_entry_t entry;

It's a bit nasty to have the same variable shadowed here.

We could reuse the existing entry by handling the code more similar to remove_migration_pte(): determine RMAP_EXCLUSIVE earlier.

Quoted 8 lines
mm/huge_memory.c +8 -0
+
+		if (pmd_write(pmde))
+			entry = make_writable_device_private_entry(
+							page_to_pfn(new));
+		else
+			entry = make_readable_device_private_entry(
+							page_to_pfn(new));
+		pmde = swp_entry_to_pmd(entry);

mm/huge_memory.c void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) · line 4732

Quoted 10 lines
mm/huge_memory.c +7 -0
+
+		if (pmd_swp_soft_dirty(*pvmw->pmd))
+			pmde = pmd_swp_mksoft_dirty(pmde);
+		if (pmd_swp_uffd_wp(*pvmw->pmd))
+			pmde = pmd_swp_mkuffd_wp(pmde);
+	}
+
  	if (folio_test_anon(folio)) {
  		rmap_t rmap_flags = RMAP_NONE;
 

I guess at some point we could separate both parts completely (no need to do all this work on pmdb before the folio_is_device_private(folio) check, so this could be

if (folio_is_device_private(folio)) {
	...
} else {
	entry = pmd_to_swp_entry(*pvmw->pmd);
	folio_get(folio);
	...
}

That is something for another day though, and remove_migration_pte() should be cleaned up then as well.

Signature · 3 lines
Cheers

David
#4 lore raw
Quoted 100 lines · David Hildenbrand (Red Hat)
Quoted 61 lines · Balbir Singh

Follow the pattern used in remove_migration_pte() in remove_migration_pmd(). Process the migration entries and if the entry type is device private, override the pmde with a device private entry and set the soft dirty and uffd_wp bits with the pmd_swp_mksoft_dirty and pmd_swp_mkuffd_wp

Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Zi Yan <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Lyude Paul <[email protected]>
Cc Danilo Krummrich <[email protected]>
Cc David Airlie <[email protected]>
Cc Simona Vetter <[email protected]>
Cc Ralph Campbell <[email protected]>
Cc Mika Penttilä <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Francois Dugast <[email protected]>

---
This fixup should be squashed into the patch "mm/rmap: extend rmap and migration support" of mm/mm-unstable

  mm/huge_memory.c | 27 +++++++++++++++++----------

  1 file changed, 17 insertions(+), 10 deletions(-)
mm/huge_memory.c +0 -9
index 9dda8c48daca..50ba458efcab 100644
@@ -4698,16 +4698,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
       folio_get(folio);
       pmde = folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot));
   -    if (folio_is_device_private(folio)) {
-        if (pmd_write(pmde))
-            entry = make_writable_device_private_entry(
-                            page_to_pfn(new));
-        else
-            entry = make_readable_device_private_entry(
-                            page_to_pfn(new));
-        pmde = swp_entry_to_pmd(entry);
-    }
-
       if (pmd_swp_soft_dirty(*pvmw->pmd))
           pmde = pmd_mksoft_dirty(pmde);
       if (is_writable_migration_entry(entry))

mm/huge_memory.c void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) · line 4713

Quoted 4 lines
mm/huge_memory.c +1 -0
@@ -4720,6 +4710,23 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
       if (folio_test_dirty(folio) && is_migration_entry_dirty(entry))
           pmde = pmd_mkdirty(pmde);
   +    if (folio_is_device_private(folio)) {
+        swp_entry_t entry;

It's a bit nasty to have the same variable shadowed here.

We could reuse the existing entry by handling the code more similar to remove_migration_pte(): determine RMAP_EXCLUSIVE earlier.

Quoted 17 lines
mm/huge_memory.c +15 -0
+
+        if (pmd_write(pmde))
+            entry = make_writable_device_private_entry(
+                            page_to_pfn(new));
+        else
+            entry = make_readable_device_private_entry(
+                            page_to_pfn(new));
+        pmde = swp_entry_to_pmd(entry);
+
+        if (pmd_swp_soft_dirty(*pvmw->pmd))
+            pmde = pmd_swp_mksoft_dirty(pmde);
+        if (pmd_swp_uffd_wp(*pvmw->pmd))
+            pmde = pmd_swp_mkuffd_wp(pmde);
+    }
+
       if (folio_test_anon(folio)) {
           rmap_t rmap_flags = RMAP_NONE;
Quoted 1 line

I guess at some point we could separate both parts completely (no need to do all this work on pmdb before the folio_is_device_private(folio) check, so this could be

if (folio_is_device_private(folio)) {
    ...
} else {
    entry = pmd_to_swp_entry(*pvmw->pmd);
    folio_get(folio);
    ...
}

Quoted 2 lines

That is something for another day though, and remove_migration_pte() should be cleaned up then as well.

Agreed and Thanks for the review!

Balbir