Skip to contentSkip to conversation

mm/huge_memory: introduce folio_split_unmapped

PATCH In review Balbir Singh 2 participants 4 messages Nov 15, 2025 – Nov 19, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 mm/huge_memory: introduce folio_split_unmapped

Unmapped was added as a parameter to __folio_split() and related call sites to support splitting of folios already in the midst of a migration. This special case arose for device private folio migration since during migration there could be a disconnect between source and destination on the folio size.

Introduce folio_split_unmapped() to handle this special case. Also refactor code and add __folio_freeze_and_split_unmapped() helper that is common to both __folio_split() and folio_split_unmapped().

This in turn removes the special casing introduced by the unmapped parameter in __folio_split().

Cc Andrew Morton <[email protected]>
Cc David Hildenbrand <[email protected]>
Cc Zi Yan <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Gregory Price <[email protected]>
Cc Ying Huang <[email protected]>
Cc Alistair Popple <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Lyude Paul <[email protected]>
Cc Danilo Krummrich <[email protected]>
Cc David Airlie <[email protected]>
Cc Simona Vetter <[email protected]>
Cc Ralph Campbell <[email protected]>
Cc Mika Penttilä <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Francois Dugast <[email protected]>

---
Changelog v2
- Fix subject prefix to be mm/huge_memory
- Fix a bug reported by kernel-test-robot, nr_shmem_dropped needs to be updated and returned from the helper function
- Added a debug warning folio_split_unmapped() should be called only on anonymous folios

This patch should replace "mm-huge_memoryc-introduce-folio_split_unmapped.patch" in mm/mm-new

 include/linux/huge_mm.h |   5 +-
 mm/huge_memory.c        | 334 +++++++++++++++++++++++-----------------
 mm/migrate_device.c     |   3 +-
 3 files changed, 193 insertions(+), 149 deletions(-)
include/linux/huge_mm.h +3 -2
index e2e91aa1a042..1d439de1ca2c 100644
@@ -371,7 +371,8 @@ enum split_type {
 
 bool can_split_folio(struct folio *folio, int caller_pins, int *pextra_pins);
 int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
-		unsigned int new_order, bool unmapped);
+		unsigned int new_order);
+int folio_split_unmapped(struct folio *folio, unsigned int new_order);
 int min_order_for_split(struct folio *folio);
 int split_folio_to_list(struct folio *folio, struct list_head *list);
 bool folio_split_supported(struct folio *folio, unsigned int new_order,
@@ -382,7 +383,7 @@ int folio_split(struct folio *folio, unsigned int new_order, struct page *page,
 static inline int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
 		unsigned int new_order)
 {
-	return __split_huge_page_to_list_to_order(page, list, new_order, false);
+	return __split_huge_page_to_list_to_order(page, list, new_order);
 }
 static inline int split_huge_page_to_order(struct page *page, unsigned int new_order)
 {
mm/huge_memory.c +189 -145

400 lines not on this page · raw message

mm/migrate_device.c +1 -2
index 46dd68cfc503..05ce95f43ab9 100644
@@ -909,8 +909,7 @@ static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate,
 
 	folio_get(folio);
 	split_huge_pmd_address(migrate->vma, addr, true);
-	ret = __split_huge_page_to_list_to_order(folio_page(folio, 0), NULL,
-							0, true);
+	ret = folio_split_unmapped(folio, 0);
 	if (ret)
 		return ret;
 	migrate->src[idx] &= ~MIGRATE_PFN_COMPOUND;
Signature · 1 line
2.51.1
#1 lore raw
Quoted 6 lines · Balbir Singh

Unmapped was added as a parameter to __folio_split() and related call sites to support splitting of folios already in the midst of a migration. This special case arose for device private folio migration since during migration there could be a disconnect between source and destination on the folio size.

Quoted 7 lines

Introduce folio_split_unmapped() to handle this special case. Also refactor code and add __folio_freeze_and_split_unmapped() helper that is common to both __folio_split() and folio_split_unmapped().

This in turn removes the special casing introduced by the unmapped parameter in __folio_split().

I was briefly wondering: why can't we just detect at the beginning of the __folio_split() that the folio is unmapped (!folio_mapped()) and then continue assuming the folio is unmapped?

The folio is locked, so it shouldn't just become mapped again.

But then I looked into the details and figured that we will also not try to remap (replace migration entries) and focus on anon folios only.

I think we really have to document this properly. See below.

[...]

Quoted 1 line

+/*

Can we have proper kerneldoc?

Quoted 7 lines

+ * This function is a helper for splitting folios that have already been unmapped.
+ * The use case is that the device or the CPU can refuse to migrate THP pages in
+ * the middle of migration, due to allocation issues on either side
+ *
+ * The high level code is copied from __folio_split, since the pages are anonymous
+ * and are already isolated from the LRU, the code has been simplified to not
+ * burden __folio_split with unmapped sprinkled into the code.

Please drop the history of how this code was obtained :) Focus on ducmenting what the function does, what it expects from the caller, and what the result of the operation will be.

Quoted 2 lines

+ *
+ * None of the split folios are unlocked

Looking into the details, I think this function also does not

(a) remap the folio

(b) call things like free_folio_and_swap_cache()

Which locks do have to be held by the caller? I'd assume the anon vma lock and the folio lock?

Would this function currently work for anon folios that are in the swapcache?

And I assume this function works for ZONE_DEVICE and !ZONE_DEVICE?

Please carefully document all that (locks held, folio isolated, folio unmapped, will not remap the folio, anon folios only, etc).

Quoted 53 lines

+ */

mm/huge_memory.c +23 -3
+int folio_split_unmapped(struct folio *folio, unsigned int new_order)
+{
+	int extra_pins, ret = 0;
+
+	VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
+	VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+	VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio);
+	VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio);
+
+	if (!can_split_folio(folio, 1, &extra_pins))
+		return -EAGAIN;
+
+	local_irq_disable();
+	ret = __folio_freeze_and_split_unmapped(folio, new_order, &folio->page, NULL,
+						NULL, false, NULL, SPLIT_TYPE_UNIFORM,
+						0, NULL, extra_pins);
+	local_irq_enable();
+	return ret;
+}
+
  /*
   * This function splits a large folio into smaller folios of order @new_order.
   * @page can point to any page of the large folio to split. The split operation
@@ -4127,12 +4171,12 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
   * with the folio. Splitting to order 0 is compatible with all folios.
   */
  int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
-				     unsigned int new_order, bool unmapped)
+				     unsigned int new_order)
  {
  	struct folio *folio = page_folio(page);
 
  	return __folio_split(folio, new_order, &folio->page, page, list,
-			     SPLIT_TYPE_UNIFORM, unmapped);
+			     SPLIT_TYPE_UNIFORM);
  }
 
  /**
@@ -4163,7 +4207,7 @@ int folio_split(struct folio *folio, unsigned int new_order,
  		struct page *split_at, struct list_head *list)
  {
  	return __folio_split(folio, new_order, split_at, &folio->page, list,
-			     SPLIT_TYPE_NON_UNIFORM, false);
+			     SPLIT_TYPE_NON_UNIFORM);
  }
 
  int min_order_for_split(struct folio *folio)
mm/migrate_device.c +0 -0
index 46dd68cfc503..05ce95f43ab9 100644

mm/migrate_device.c static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate, · line 915

Quoted 9 lines
mm/migrate_device.c +1 -2
@@ -909,8 +909,7 @@ static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate,
 
  	folio_get(folio);
  	split_huge_pmd_address(migrate->vma, addr, true);
-	ret = __split_huge_page_to_list_to_order(folio_page(folio, 0), NULL,
-							0, true);
+	ret = folio_split_unmapped(folio, 0);
  	if (ret)
  		return ret;
  	migrate->src[idx] &= ~MIGRATE_PFN_COMPOUND;

This is clearly a win.

Signature · 3 lines
Cheers

David
#2 lore raw
Quoted 17 lines · David Hildenbrand (Red Hat)
Quoted 6 lines · Balbir Singh

Unmapped was added as a parameter to __folio_split() and related call sites to support splitting of folios already in the midst of a migration. This special case arose for device private folio migration since during migration there could be a disconnect between source and destination on the folio size.

Quoted 7 lines

Introduce folio_split_unmapped() to handle this special case. Also refactor code and add __folio_freeze_and_split_unmapped() helper that is common to both __folio_split() and folio_split_unmapped().

This in turn removes the special casing introduced by the unmapped parameter in __folio_split().

I was briefly wondering: why can't we just detect at the beginning of the __folio_split() that the folio is unmapped (!folio_mapped()) and then continue assuming the folio is unmapped?

Quoted 8 lines

The folio is locked, so it shouldn't just become mapped again.

But then I looked into the details and figured that we will also not try to remap (replace migration entries) and focus on anon folios only.

I think we really have to document this properly. See below.

[...]

Your observations are correct.

Quoted 24 lines
Quoted 1 line

+/*

Can we have proper kerneldoc?

Quoted 7 lines

+ * This function is a helper for splitting folios that have already been unmapped.
+ * The use case is that the device or the CPU can refuse to migrate THP pages in
+ * the middle of migration, due to allocation issues on either side
+ *
+ * The high level code is copied from __folio_split, since the pages are anonymous
+ * and are already isolated from the LRU, the code has been simplified to not
+ * burden __folio_split with unmapped sprinkled into the code.

Please drop the history of how this code was obtained :) Focus on ducmenting what the function does, what it expects from the caller, and what the result of the operation will be.

Quoted 2 lines

+ *
+ * None of the split folios are unlocked

Looking into the details, I think this function also does not

(a) remap the folio

(b) call things like free_folio_and_swap_cache()

Quoted 8 lines

Which locks do have to be held by the caller? I'd assume the anon vma lock and the folio lock?

Would this function currently work for anon folios that are in the swapcache?

And I assume this function works for ZONE_DEVICE and !ZONE_DEVICE?

Please carefully document all that (locks held, folio isolated, folio unmapped, will not remap the folio, anon folios only, etc).

Something like this below?

/**
* folio_split_unmapped() - split a large anon folio that is already unmapped
* @folio: folio to split
* @new_order: the order of folios after split
*
* This function is a helper for splitting folios that have already been
* unmapped. The use case is that the device or the CPU can refuse to migrate
* THP pages in the middle of migration, due to allocation issues on either
* side.
*
* anon_vma_lock is not required to be held, mmap_read_lock() or
* mmap_write_lock() should be held. @folio is expected to be locked by the
* caller. device-private and non device-private folios are supported along
* with folios that are in the swapcache. @folio should also be unmapped and
* isolated from LRU (if applicable)
*
* Upon return, the folio is not remapped, split folios are not added to LRU,
* free_folio_and_swap_cache() is not called, and new folios remain locked. *
* Return: 0 on success, -EAGAIN if the folio cannot be split (e.g., due to
* insufficient reference count or extra pins).
*/

Quoted 53 lines
Quoted 53 lines

+ */

mm/huge_memory.c +20 -0
+int folio_split_unmapped(struct folio *folio, unsigned int new_order)
+{
+    int extra_pins, ret = 0;
+
+    VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
+    VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+    VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio);
+    VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio);
+
+    if (!can_split_folio(folio, 1, &extra_pins))
+        return -EAGAIN;
+
+    local_irq_disable();
+    ret = __folio_freeze_and_split_unmapped(folio, new_order, &folio->page, NULL,
+                        NULL, false, NULL, SPLIT_TYPE_UNIFORM,
+                        0, NULL, extra_pins);
+    local_irq_enable();
+    return ret;
+}
+

  /*
   * This function splits a large folio into smaller folios of order @new_order.
   * @page can point to any page of the large folio to split. The split operation

mm/huge_memory.c +3 -3
@@ -4127,12 +4171,12 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
    * with the folio. Splitting to order 0 is compatible with all folios.
    */
   int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
-                     unsigned int new_order, bool unmapped)
+                     unsigned int new_order)
   {
       struct folio *folio = page_folio(page);
         return __folio_split(folio, new_order, &folio->page, page, list,
-                 SPLIT_TYPE_UNIFORM, unmapped);
+                 SPLIT_TYPE_UNIFORM);
   }
     /**
@@ -4163,7 +4207,7 @@ int folio_split(struct folio *folio, unsigned int new_order,
           struct page *split_at, struct list_head *list)
   {
       return __folio_split(folio, new_order, split_at, &folio->page, list,
-                 SPLIT_TYPE_NON_UNIFORM, false);
+                 SPLIT_TYPE_NON_UNIFORM);
   }
     int min_order_for_split(struct folio *folio)
mm/migrate_device.c +0 -0
index 46dd68cfc503..05ce95f43ab9 100644
@@ -909,8 +909,7 @@ static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate,
         folio_get(folio);
       split_huge_pmd_address(migrate->vma, addr, true);
Quoted 9 lines

mm/migrate_device.c static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate, · line 915

Quoted 6 lines
mm/migrate_device.c +1 -2
-    ret = __split_huge_page_to_list_to_order(folio_page(folio, 0), NULL,
-                            0, true);
+    ret = folio_split_unmapped(folio, 0);
       if (ret)
           return ret;
       migrate->src[idx] &= ~MIGRATE_PFN_COMPOUND;

This is clearly a win.

Thanks for the review!
Balbir

#3 lore raw
In reply to Balbir Singh

[Reforwarding - not sure if the original response made it back]

Quoted 17 lines · David Hildenbrand (Red Hat)
Quoted 6 lines · Balbir Singh

Unmapped was added as a parameter to __folio_split() and related call sites to support splitting of folios already in the midst of a migration. This special case arose for device private folio migration since during migration there could be a disconnect between source and destination on the folio size.

Quoted 7 lines

Introduce folio_split_unmapped() to handle this special case. Also refactor code and add __folio_freeze_and_split_unmapped() helper that is common to both __folio_split() and folio_split_unmapped().

This in turn removes the special casing introduced by the unmapped parameter in __folio_split().

I was briefly wondering: why can't we just detect at the beginning of the __folio_split() that the folio is unmapped (!folio_mapped()) and then continue assuming the folio is unmapped?

Quoted 8 lines

The folio is locked, so it shouldn't just become mapped again.

But then I looked into the details and figured that we will also not try to remap (replace migration entries) and focus on anon folios only.

I think we really have to document this properly. See below.

[...]

Your observations are correct.

Quoted 24 lines
Quoted 1 line

+/*

Can we have proper kerneldoc?

Quoted 7 lines

+ * This function is a helper for splitting folios that have already been unmapped.
+ * The use case is that the device or the CPU can refuse to migrate THP pages in
+ * the middle of migration, due to allocation issues on either side
+ *
+ * The high level code is copied from __folio_split, since the pages are anonymous
+ * and are already isolated from the LRU, the code has been simplified to not
+ * burden __folio_split with unmapped sprinkled into the code.

Please drop the history of how this code was obtained 🙂 Focus on ducmenting what the function does, what it expects from the caller, and what the result of the operation will be.

Quoted 2 lines

+ *
+ * None of the split folios are unlocked

Looking into the details, I think this function also does not

(a) remap the folio

(b) call things like free_folio_and_swap_cache()

Quoted 8 lines

Which locks do have to be held by the caller? I'd assume the anon vma lock and the folio lock?

Would this function currently work for anon folios that are in the swapcache?

And I assume this function works for ZONE_DEVICE and !ZONE_DEVICE?

Please carefully document all that (locks held, folio isolated, folio unmapped, will not remap the folio, anon folios only, etc).

Something like this below?

/**
* folio_split_unmapped() - split a large anon folio that is already unmapped
* @folio: folio to split
* @new_order: the order of folios after split
*
* This function is a helper for splitting folios that have already been
* unmapped. The use case is that the device or the CPU can refuse to migrate
* THP pages in the middle of migration, due to allocation issues on either
* side.
*
* anon_vma_lock is not required to be held, mmap_read_lock() or
* mmap_write_lock() should be held. @folio is expected to be locked by the
* caller. device-private and non device-private folios are supported along
* with folios that are in the swapcache. @folio should also be unmapped and
* isolated from LRU (if applicable)
*
* Upon return, the folio is not remapped, split folios are not added to LRU,
* free_folio_and_swap_cache() is not called, and new folios remain locked. *
* Return: 0 on success, -EAGAIN if the folio cannot be split (e.g., due to
* insufficient reference count or extra pins).
*/

Quoted 53 lines
Quoted 53 lines

+ */

mm/huge_memory.c +23 -3
+int folio_split_unmapped(struct folio *folio, unsigned int new_order)
+{
+    int extra_pins, ret = 0;
+
+    VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
+    VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+    VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio);
+    VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio);
+
+    if (!can_split_folio(folio, 1, &extra_pins))
+        return -EAGAIN;
+
+    local_irq_disable();
+    ret = __folio_freeze_and_split_unmapped(folio, new_order, &folio->page, NULL,
+                        NULL, false, NULL, SPLIT_TYPE_UNIFORM,
+                        0, NULL, extra_pins);
+    local_irq_enable();
+    return ret;
+}
+
  /*
   * This function splits a large folio into smaller folios of order @new_order.
   * @page can point to any page of the large folio to split. The split operation
@@ -4127,12 +4171,12 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
   * with the folio. Splitting to order 0 is compatible with all folios.
   */
  int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
-                     unsigned int new_order, bool unmapped)
+                     unsigned int new_order)
  {
      struct folio *folio = page_folio(page);
        return __folio_split(folio, new_order, &folio->page, page, list,
-                 SPLIT_TYPE_UNIFORM, unmapped);
+                 SPLIT_TYPE_UNIFORM);
  }
    /**
@@ -4163,7 +4207,7 @@ int folio_split(struct folio *folio, unsigned int new_order,
          struct page *split_at, struct list_head *list)
  {
      return __folio_split(folio, new_order, split_at, &folio->page, list,
-                 SPLIT_TYPE_NON_UNIFORM, false);
+                 SPLIT_TYPE_NON_UNIFORM);
  }
    int min_order_for_split(struct folio *folio)
mm/migrate_device.c +0 -0
index 46dd68cfc503..05ce95f43ab9 100644
@@ -909,8 +909,7 @@ static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate,
        folio_get(folio);
      split_huge_pmd_address(migrate->vma, addr, true);
Quoted 9 lines

mm/migrate_device.c static int migrate_vma_split_unmapped_folio(struct migrate_vma *migrate, · line 915

Quoted 6 lines
mm/migrate_device.c +1 -2
-    ret = __split_huge_page_to_list_to_order(folio_page(folio, 0), NULL,
-                            0, true);
+    ret = folio_split_unmapped(folio, 0);
      if (ret)
          return ret;
      migrate->src[idx] &= ~MIGRATE_PFN_COMPOUND;

This is clearly a win.

Thanks for the review!
Balbir