Skip to contentSkip to conversation

mm: migrate: requeue destination folio on deferred split queue

PATCH Superseded Usama Arif 4 participants 10 messages Mar 10 – Jun 22

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Usama Arif

author
#0 lore raw

PATCH v2 1/1 mm: migrate: requeue destination folio on deferred split queue

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)

---
 mm/migrate.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
mm/migrate.c +17 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {
+		src_deferred_split = true;
+		src_partially_mapped = folio_test_partially_mapped(src);
+	}
+
 	rc = move_to_new_folio(dst, src, mode);
 	if (rc)
 		goto out;
@@ -1393,6 +1401,15 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	if (old_page_state & PAGE_WAS_MAPPED)
 		remove_migration_ptes(src, dst, 0);
 
+	/*
+	 * Requeue the destination folio on the deferred split queue if
+	 * the source was on the queue.  The source is unqueued in
+	 * __folio_migrate_mapping(), so we recorded the state from
+	 * before move_to_new_folio().
+	 */
+	if (src_deferred_split)
+		deferred_split_folio(dst, src_partially_mapped);
+
 out_unlock_both:
 	folio_unlock(dst);
 	folio_set_owner_migrate_reason(dst, reason);
Signature · 1 line
2.47.3
reviewer Acked-by on v2 1/1
#1 lore raw
Quoted 21 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")
Quoted 9 lines

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)

---
 mm/migrate.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

LGTM.

Best Regards,
Yan, Zi

maintainer Acked-by on v2 1/1
#2 lore raw
In reply to Usama Arif
Quoted 44 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)

---
 mm/migrate.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1378

Quoted 4 lines
mm/migrate.c +1 -0
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&

I don't think the folio_test_large_rmappable() check is required. Other folios we migrate here would always have _deferred_list initialized but unused.

Signature · 3 lines
Cheers,

David
author
#3 lore raw
Quoted 48 lines · David Hildenbrand (Arm)
Quoted 47 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)

---
 mm/migrate.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
Quoted 8 lines

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1378

Quoted 1 line
mm/migrate.c +1 -0
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&

I don't think the folio_test_large_rmappable() check is required. Other folios we migrate here would always have _deferred_list initialized but unused.

I have been auditing the THP shrinker code when it comes to NUMA migration and I think we need another fix for this. I have sent it here https://lore.kernel.org/all/[email protected]/

#4 lore raw
In reply to Usama Arif
Quoted 44 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)
---
mm/migrate.c | 17 +++++++++++++++++

1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1379

Quoted 5 lines
mm/migrate.c +2 -0
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {

We usually check order > 1, before accessing _deferred_list, because it is in subpage 2.

I am not sure why we don't do it here. Do I miss something?

Quoted 26 lines
mm/migrate.c +13 -0
+		src_deferred_split = true;
+		src_partially_mapped = folio_test_partially_mapped(src);
+	}
+
 	rc = move_to_new_folio(dst, src, mode);
 	if (rc)
 		goto out;
@@ -1393,6 +1401,15 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	if (old_page_state & PAGE_WAS_MAPPED)
 		remove_migration_ptes(src, dst, 0);
 
+	/*
+	 * Requeue the destination folio on the deferred split queue if
+	 * the source was on the queue.  The source is unqueued in
+	 * __folio_migrate_mapping(), so we recorded the state from
+	 * before move_to_new_folio().
+	 */
+	if (src_deferred_split)
+		deferred_split_folio(dst, src_partially_mapped);
+

out_unlock_both:

	folio_unlock(dst);
	folio_set_owner_migrate_reason(dst, reason);
Signature · 1 line
2.47.3
Signature · 2 lines
Wei Yang
Help you, Help me
#5 lore raw
Quoted 48 lines · Wei Yang
Quoted 47 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)
---
mm/migrate.c | 17 +++++++++++++++++

1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
Quoted 7 lines

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1379

Quoted 2 lines
mm/migrate.c +2 -0
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {

We usually check order > 1, before accessing _deferred_list, because it is in subpage 2.

I am not sure why we don't do it here. Do I miss something?

Valid point! non-anon folios could trigger that.

Signature · 3 lines
Cheers,

David
#6 lore raw
In reply to Usama Arif
Quoted 44 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)
---
mm/migrate.c | 17 +++++++++++++++++

1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1382

Quoted 8 lines
mm/migrate.c +5 -0
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {
+		src_deferred_split = true;
+		src_partially_mapped = folio_test_partially_mapped(src);
+	}

Hi, Usama

I am afraid there maybe a race between migration and defer_split.

A B
migrate_pages_batch deferred_split_scan

    migrate_folio_unmap                   list_del_init(&folio->_deferred_list)
      folio_lock/folio_trylock

    migrate_folios_move
      migrate_folio_move
        list_empty(&src->_deferred_list)
                                          folio_trylock()
                                          requeue:

In case list_empty() check happens after folio removed from defer_list but before requeued, we will miss this folio.

And the behavior is the same before commit fafaeceb89a5, IIUC.

Do you thinks this will happen?

Quoted 23 lines
mm/migrate.c +10 -0
+
 	rc = move_to_new_folio(dst, src, mode);
 	if (rc)
 		goto out;
@@ -1393,6 +1401,15 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	if (old_page_state & PAGE_WAS_MAPPED)
 		remove_migration_ptes(src, dst, 0);
 
+	/*
+	 * Requeue the destination folio on the deferred split queue if
+	 * the source was on the queue.  The source is unqueued in
+	 * __folio_migrate_mapping(), so we recorded the state from
+	 * before move_to_new_folio().
+	 */
+	if (src_deferred_split)
+		deferred_split_folio(dst, src_partially_mapped);
+

out_unlock_both:

	folio_unlock(dst);
	folio_set_owner_migrate_reason(dst, reason);
Signature · 1 line
2.47.3
Signature · 2 lines
Wei Yang
Help you, Help me
#7 lore raw
Quoted 69 lines · Wei Yang
Quoted 44 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)
---
mm/migrate.c | 17 +++++++++++++++++

1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1382

Quoted 8 lines
mm/migrate.c +5 -0
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {
+		src_deferred_split = true;
+		src_partially_mapped = folio_test_partially_mapped(src);
+	}

Hi, Usama

I am afraid there maybe a race between migration and defer_split.

A B
migrate_pages_batch deferred_split_scan

    migrate_folio_unmap                   list_del_init(&folio->_deferred_list)
      folio_lock/folio_trylock

    migrate_folios_move
      migrate_folio_move
        list_empty(&src->_deferred_list)
                                          folio_trylock()
                                          requeue:
Quoted 2 lines

In case list_empty() check happens after folio removed from defer_list but before requeued, we will miss this folio.

deferred_split_isolate() would grab a reference through folio_try_get().

How can we migrate a folio with a raised refcount?

Signature · 3 lines
Cheers,

David
#8 lore raw
Quoted 70 lines · David Hildenbrand (Arm)
Quoted 69 lines · Wei Yang
Quoted 44 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)
---
mm/migrate.c | 17 +++++++++++++++++

1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1382

Quoted 8 lines
mm/migrate.c +5 -0
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {
+		src_deferred_split = true;
+		src_partially_mapped = folio_test_partially_mapped(src);
+	}

Hi, Usama

I am afraid there maybe a race between migration and defer_split.

A B
migrate_pages_batch deferred_split_scan

    migrate_folio_unmap                   list_del_init(&folio->_deferred_list)
      folio_lock/folio_trylock

    migrate_folios_move
      migrate_folio_move
        list_empty(&src->_deferred_list)
                                          folio_trylock()
                                          requeue:
Quoted 7 lines
Quoted 2 lines

In case list_empty() check happens after folio removed from defer_list but before requeued, we will miss this folio.

deferred_split_isolate() would grab a reference through folio_try_get().

How can we migrate a folio with a raised refcount?

Thanks, I missed expected_refcount check in __migrate_folio().

Quoted 4 lines
Signature · 3 lines
Cheers,

David
Signature · 2 lines
Wei Yang
Help you, Help me
author
#9 lore raw
Quoted 74 lines · Wei Yang
Quoted 73 lines · David Hildenbrand (Arm)
Quoted 71 lines · Wei Yang
Quoted 44 lines · Usama Arif

During folio migration, __folio_migrate_mapping() removes the source folio from the deferred split queue, but the destination folio is never re-queued. This causes underutilized THPs to escape the shrinker after NUMA migration, since they silently drop off the deferred split list.

Fix this by recording whether the source folio was on the deferred split queue and its partially mapped state before move_to_new_folio() unqueues it, and re-queuing the destination folio after a successful migration if it was.

By the time migrate_folio_move() runs, partially mapped folios without a pin have already been split by migrate_pages_batch(). So only two cases remain on the deferred list at this point:
1. Partially mapped folios with a pin (split failed).
2. Fully mapped but potentially underused folios.
The recorded partially_mapped state is forwarded to deferred_split_folio() so that the destination folio is correctly re-queued in both cases.

Fixes dafff3f4c850 ("mm: split underused THPs")

---
v1 -> v2:
- record whether source folio was on the deferred split queue before move_to_folio() (David)
- record partially mapped state and update commit message (Zi)
---
mm/migrate.c | 17 +++++++++++++++++

1 file changed, 17 insertions(+)
mm/migrate.c +2 -0
index ece77ccb2ec0..61013d258eb4 100644
@@ -1360,6 +1360,8 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 	int rc;
 	int old_page_state = 0;
 	struct anon_vma *anon_vma = NULL;
+	bool src_deferred_split = false;
+	bool src_partially_mapped = false;
 	struct list_head *prev;
 
 	__migrate_folio_extract(dst, &old_page_state, &anon_vma);

mm/migrate.c static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private, · line 1382

Quoted 8 lines
mm/migrate.c +5 -0
@@ -1373,6 +1375,12 @@ static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,
 		goto out_unlock_both;
 	}
 
+	if (folio_test_large(src) && folio_test_large_rmappable(src) &&
+	    !data_race(list_empty(&src->_deferred_list))) {
+		src_deferred_split = true;
+		src_partially_mapped = folio_test_partially_mapped(src);
+	}

Hi, Usama

I am afraid there maybe a race between migration and defer_split.

A B
migrate_pages_batch deferred_split_scan

    migrate_folio_unmap                   list_del_init(&folio->_deferred_list)
      folio_lock/folio_trylock

    migrate_folios_move
      migrate_folio_move
        list_empty(&src->_deferred_list)
                                          folio_trylock()
                                          requeue:

In case list_empty() check happens after folio removed from defer_list but before requeued, we will miss this folio.

Quoted 7 lines
Quoted 4 lines

deferred_split_isolate() would grab a reference through folio_try_get().

How can we migrate a folio with a raised refcount?

Thanks, I missed expected_refcount check in __migrate_folio().

Thanks David for pointing it out! I have just started looking at the mailing list for today :)

Quoted 5 lines
Quoted 4 lines
Signature · 3 lines
Cheers,

David