Skip to contentSkip to conversation

Extend migrate_misplaced_page() to support batch migration

PATCH Superseded Baolin Wang 2 participants 11 messages Aug 19, 2023 – Aug 21, 2023

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Baolin Wang

author reviewer
#0 lore raw

cover Extend migrate_misplaced_page() to support batch migration

Hi,

Currently, on our ARM servers with NUMA enabled, we found the cross-die latency is a little larger that will significantly impact the workload's performance. So on ARM servers we will rely on the NUMA balancing to avoid the cross-die accessing. And I posted a patchset[1] to support speculative numa fault to improve the NUMA balancing's performance according to the principle of data locality. Moreover, thanks to Huang Ying's patchset[2], which introduced batch migration as a way to reduce the cost of TLB flush, and it will also benefit the migration of multiple pages all at once during NUMA balancing.

So we plan to continue to support batch migration in do_numa_page() to improve the NUMA balancing's performance, but before adding complicated batch migration algorithm for NUMA balancing, some cleanup and preparation work need to do firstly, which are done in this patch set. In short, this patchset extends the migrate_misplaced_page() interface to support batch migration, and no functional changes intended.

[1] https://lore.kernel.org/lkml/[email protected]/t/#mc45929849b5d0e29b5fdd9d50425f8e95b8f2563
[2] https://lore.kernel.org/all/[email protected]/T/#u

Baolin Wang (4):
mm: migrate: move migration validation into numa_migrate_prep()
mm: migrate: move the numamigrate_isolate_page() into do_numa_page()
mm: migrate: change migrate_misplaced_page() to support multiple pages
migration
mm: migrate: change to return the number of pages migrated
successfully

 include/linux/migrate.h | 15 ++++++++---
 mm/huge_memory.c        | 19 +++++++++++---
 mm/memory.c             | 34 +++++++++++++++++++++++-
 mm/migrate.c            | 58 ++++++++---------------------------------
 4 files changed, 71 insertions(+), 55 deletions(-)
Signature · 1 line
2.39.3
author reviewer
#1 lore raw

PATCH 1/4 mm: migrate: move migration validation into numa_migrate_prep()

Now there are 3 places will validate if a page can mirate or not, and some validations are performed later, which will waste some CPU to call numa_migrate_prep().

Thus we can move all the migration validation into numa_migrate_prep(), which is more maintainable as well as saving some CPU resources. Another benefit is that it can serve as a preparation for supporting batch migration in do_numa_page() in future.

---
 mm/memory.c  | 19 +++++++++++++++++++
 mm/migrate.c | 19 -------------------
 2 files changed, 19 insertions(+), 19 deletions(-)
mm/memory.c +19 -0
index d003076b218d..bee9b1e86ef0 100644
@@ -4747,6 +4747,25 @@ int numa_migrate_prep(struct page *page, struct vm_area_struct *vma,
 		*flags |= TNF_FAULT_LOCAL;
 	}
 
+	/*
+	 * Don't migrate file pages that are mapped in multiple processes
+	 * with execute permissions as they are probably shared libraries.
+	 */
+	if (page_mapcount(page) != 1 && page_is_file_lru(page) &&
+	    (vma->vm_flags & VM_EXEC))
+		return NUMA_NO_NODE;
+
+	/*
+	 * Also do not migrate dirty pages as not all filesystems can move
+	 * dirty pages in MIGRATE_ASYNC mode which is a waste of cycles.
+	 */
+	if (page_is_file_lru(page) && PageDirty(page))
+		return NUMA_NO_NODE;
+
+	/* Do not migrate THP mapped by multiple processes */
+	if (PageTransHuge(page) && total_mapcount(page) > 1)
+		return NUMA_NO_NODE;
+
 	return mpol_misplaced(page, vma, addr);
 }
 
mm/migrate.c +0 -19
index e21d5a7e7447..9cc98fb1d6ec 100644
@@ -2485,10 +2485,6 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
 
 	VM_BUG_ON_PAGE(order && !PageTransHuge(page), page);
 
-	/* Do not migrate THP mapped by multiple processes */
-	if (PageTransHuge(page) && total_mapcount(page) > 1)
-		return 0;
-
 	/* Avoid migrating to a node that is nearly full */
 	if (!migrate_balanced_pgdat(pgdat, nr_pages)) {
 		int z;
@@ -2533,21 +2529,6 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
 	LIST_HEAD(migratepages);
 	int nr_pages = thp_nr_pages(page);
 
-	/*
-	 * Don't migrate file pages that are mapped in multiple processes
-	 * with execute permissions as they are probably shared libraries.
-	 */
-	if (page_mapcount(page) != 1 && page_is_file_lru(page) &&
-	    (vma->vm_flags & VM_EXEC))
-		goto out;
-
-	/*
-	 * Also do not migrate dirty pages as not all filesystems can move
-	 * dirty pages in MIGRATE_ASYNC mode which is a waste of cycles.
-	 */
-	if (page_is_file_lru(page) && PageDirty(page))
-		goto out;
-
 	isolated = numamigrate_isolate_page(pgdat, page);
 	if (!isolated)
 		goto out;
Signature · 1 line
2.39.3
#2 lore raw

Re: [PATCH 1/4] mm: migrate: move migration validation into numa_migrate_prep()

Quoted 33 lines · Baolin Wang

Now there are 3 places will validate if a page can mirate or not, and some validations are performed later, which will waste some CPU to call numa_migrate_prep().

Thus we can move all the migration validation into numa_migrate_prep(), which is more maintainable as well as saving some CPU resources. Another benefit is that it can serve as a preparation for supporting batch migration in do_numa_page() in future.

---
 mm/memory.c  | 19 +++++++++++++++++++
 mm/migrate.c | 19 -------------------
 2 files changed, 19 insertions(+), 19 deletions(-)
mm/memory.c +10 -0
index d003076b218d..bee9b1e86ef0 100644
@@ -4747,6 +4747,25 @@ int numa_migrate_prep(struct page *page, struct vm_area_struct *vma,
 		*flags |= TNF_FAULT_LOCAL;
 	}
 
+	/*
+	 * Don't migrate file pages that are mapped in multiple processes
+	 * with execute permissions as they are probably shared libraries.
+	 */
+	if (page_mapcount(page) != 1 && page_is_file_lru(page) &&
+	    (vma->vm_flags & VM_EXEC))
+		return NUMA_NO_NODE;
+
+	/*
+	 * Also do not migrate dirty pages as not all filesystems can move

mm/memory.c int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, · line 4769

Quoted 10 lines
mm/memory.c +9 -0
+	 * dirty pages in MIGRATE_ASYNC mode which is a waste of cycles.
+	 */
+	if (page_is_file_lru(page) && PageDirty(page))
+		return NUMA_NO_NODE;
+
+	/* Do not migrate THP mapped by multiple processes */
+	if (PageTransHuge(page) && total_mapcount(page) > 1)
+		return NUMA_NO_NODE;
+
 	return mpol_misplaced(page, vma, addr);

In mpol_misplaced()->should_numa_migrate_memory(), accessing CPU and PID will be recorded. So the code change above will introduce some behavior change.

How about move these checks into a separate function which is called between numa_migrate_prep() and migrate_misplaced_page() after unlocking PTL?

--
Best Regards,
Huang, Ying

Quoted 39 lines
mm/memory.c +0 -0
 }
 
mm/migrate.c +0 -19
index e21d5a7e7447..9cc98fb1d6ec 100644
@@ -2485,10 +2485,6 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
 
 	VM_BUG_ON_PAGE(order && !PageTransHuge(page), page);
 
-	/* Do not migrate THP mapped by multiple processes */
-	if (PageTransHuge(page) && total_mapcount(page) > 1)
-		return 0;
-
 	/* Avoid migrating to a node that is nearly full */
 	if (!migrate_balanced_pgdat(pgdat, nr_pages)) {
 		int z;
@@ -2533,21 +2529,6 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
 	LIST_HEAD(migratepages);
 	int nr_pages = thp_nr_pages(page);
 
-	/*
-	 * Don't migrate file pages that are mapped in multiple processes
-	 * with execute permissions as they are probably shared libraries.
-	 */
-	if (page_mapcount(page) != 1 && page_is_file_lru(page) &&
-	    (vma->vm_flags & VM_EXEC))
-		goto out;
-
-	/*
-	 * Also do not migrate dirty pages as not all filesystems can move
-	 * dirty pages in MIGRATE_ASYNC mode which is a waste of cycles.
-	 */
-	if (page_is_file_lru(page) && PageDirty(page))
-		goto out;
-
 	isolated = numamigrate_isolate_page(pgdat, page);
 	if (!isolated)
 		goto out;
author reviewer
#3 lore raw

Re: [PATCH 1/4] mm: migrate: move migration validation into numa_migrate_prep()

Quoted 41 lines · Ying
Quoted 39 lines · Baolin Wang

Now there are 3 places will validate if a page can mirate or not, and some validations are performed later, which will waste some CPU to call numa_migrate_prep().

Thus we can move all the migration validation into numa_migrate_prep(), which is more maintainable as well as saving some CPU resources. Another benefit is that it can serve as a preparation for supporting batch migration in do_numa_page() in future.

---
  mm/memory.c  | 19 +++++++++++++++++++
  mm/migrate.c | 19 -------------------
  2 files changed, 19 insertions(+), 19 deletions(-)
mm/memory.c +16 -0
index d003076b218d..bee9b1e86ef0 100644
@@ -4747,6 +4747,25 @@ int numa_migrate_prep(struct page *page, struct vm_area_struct *vma,
  		*flags |= TNF_FAULT_LOCAL;
  	}
 
+	/*
+	 * Don't migrate file pages that are mapped in multiple processes
+	 * with execute permissions as they are probably shared libraries.
+	 */
+	if (page_mapcount(page) != 1 && page_is_file_lru(page) &&
+	    (vma->vm_flags & VM_EXEC))
+		return NUMA_NO_NODE;
+
+	/*
+	 * Also do not migrate dirty pages as not all filesystems can move
+	 * dirty pages in MIGRATE_ASYNC mode which is a waste of cycles.
+	 */
+	if (page_is_file_lru(page) && PageDirty(page))
+		return NUMA_NO_NODE;
+
+	/* Do not migrate THP mapped by multiple processes */
Quoted 8 lines

mm/memory.c int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, · line 4769

Quoted 4 lines
mm/memory.c +3 -0
+	if (PageTransHuge(page) && total_mapcount(page) > 1)
+		return NUMA_NO_NODE;
+
  	return mpol_misplaced(page, vma, addr);

In mpol_misplaced()->should_numa_migrate_memory(), accessing CPU and PID will be recorded. So the code change above will introduce some behavior change.

Indeed.

Quoted 4 lines

How about move these checks into a separate function which is called between numa_migrate_prep() and migrate_misplaced_page() after unlocking PTL?

Sounds reasonable to me. Thanks for your input.

Quoted 44 lines

--
Best Regards,
Huang, Ying

Quoted 39 lines
mm/memory.c +0 -0
  }
 
mm/migrate.c +0 -19
index e21d5a7e7447..9cc98fb1d6ec 100644
@@ -2485,10 +2485,6 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
 
  	VM_BUG_ON_PAGE(order && !PageTransHuge(page), page);
 
-	/* Do not migrate THP mapped by multiple processes */
-	if (PageTransHuge(page) && total_mapcount(page) > 1)
-		return 0;
-
  	/* Avoid migrating to a node that is nearly full */
  	if (!migrate_balanced_pgdat(pgdat, nr_pages)) {
  		int z;
@@ -2533,21 +2529,6 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
  	LIST_HEAD(migratepages);
  	int nr_pages = thp_nr_pages(page);
 
-	/*
-	 * Don't migrate file pages that are mapped in multiple processes
-	 * with execute permissions as they are probably shared libraries.
-	 */
-	if (page_mapcount(page) != 1 && page_is_file_lru(page) &&
-	    (vma->vm_flags & VM_EXEC))
-		goto out;
-
-	/*
-	 * Also do not migrate dirty pages as not all filesystems can move
-	 * dirty pages in MIGRATE_ASYNC mode which is a waste of cycles.
-	 */
-	if (page_is_file_lru(page) && PageDirty(page))
-		goto out;
-
  	isolated = numamigrate_isolate_page(pgdat, page);
  	if (!isolated)
  		goto out;
author reviewer
#4 lore raw

PATCH 2/4 mm: migrate: move the numamigrate_isolate_page() into do_numa_page()

In reply to Baolin Wang

Move the numamigrate_isolate_page() into do_numa_page() to simplify the migrate_misplaced_page(), which now only focuses on page migration, and it also serves as a preparation for supporting batch migration for NUMA balancing.

---
 include/linux/migrate.h |  6 ++++++
 mm/huge_memory.c        | 10 ++++++++++
 mm/memory.c             | 10 ++++++++++
 mm/migrate.c            | 16 ++++------------
 4 files changed, 30 insertions(+), 12 deletions(-)
include/linux/migrate.h +6 -0
index 711dd9412561..7c5189043707 100644
@@ -144,12 +144,18 @@ const struct movable_operations *page_movable_ops(struct page *page)
 #ifdef CONFIG_NUMA_BALANCING
 int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
 			   int node);
+int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page);
 #else
 static inline int migrate_misplaced_page(struct page *page,
 					 struct vm_area_struct *vma, int node)
 {
 	return -EAGAIN; /* can't migrate now */
 }
+
+static inline int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
+{
+	return -EAGAIN;
+}
 #endif /* CONFIG_NUMA_BALANCING */
 
 #ifdef CONFIG_MIGRATION
mm/huge_memory.c +10 -0
index cb4432792b88..b7cc6828ce9e 100644
@@ -1496,6 +1496,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 	int target_nid, last_cpupid = (-1 & LAST_CPUPID_MASK);
 	bool migrated = false, writable = false;
 	int flags = 0;
+	pg_data_t *pgdat;
+	int isolated;
 
 	vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
 	if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
@@ -1540,11 +1542,19 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 	spin_unlock(vmf->ptl);
 	writable = false;
 
+	pgdat = NODE_DATA(target_nid);
+	isolated = numamigrate_isolate_page(pgdat, page);
+	if (!isolated) {
+		put_page(page);
+		goto isolate_fail;
+	}
+
 	migrated = migrate_misplaced_page(page, vma, target_nid);
 	if (migrated) {
 		flags |= TNF_MIGRATED;
 		page_nid = target_nid;
 	} else {
+isolate_fail:
 		flags |= TNF_MIGRATE_FAIL;
 		vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
 		if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
mm/memory.c +10 -0
index bee9b1e86ef0..01b1980d4fb7 100644
@@ -4779,6 +4779,8 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 	int target_nid;
 	pte_t pte, old_pte;
 	int flags = 0;
+	pg_data_t *pgdat;
+	int isolated;
 
 	/*
 	 * The "pte" at this point cannot be used safely without
@@ -4849,11 +4851,19 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 	pte_unmap_unlock(vmf->pte, vmf->ptl);
 	writable = false;
 
+	pgdat = NODE_DATA(target_nid);
+	isolated = numamigrate_isolate_page(pgdat, page);
+	if (!isolated) {
+		put_page(page);
+		goto isolate_fail;
+	}
+
 	/* Migrate to the requested node */
 	if (migrate_misplaced_page(page, vma, target_nid)) {
 		page_nid = target_nid;
 		flags |= TNF_MIGRATED;
 	} else {
+isolate_fail:
 		flags |= TNF_MIGRATE_FAIL;
 		vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd,
 					       vmf->address, &vmf->ptl);
mm/migrate.c +4 -12
index 9cc98fb1d6ec..5eeeb2cda21c 100644
@@ -2478,7 +2478,7 @@ static struct folio *alloc_misplaced_dst_folio(struct folio *src,
 	return __folio_alloc_node(gfp, order, nid);
 }
 
-static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
+int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
 {
 	int nr_pages = thp_nr_pages(page);
 	int order = compound_order(page);
@@ -2523,16 +2523,12 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
 			   int node)
 {
 	pg_data_t *pgdat = NODE_DATA(node);
-	int isolated;
+	int migrated = 1;
 	int nr_remaining;
 	unsigned int nr_succeeded;
 	LIST_HEAD(migratepages);
 	int nr_pages = thp_nr_pages(page);
 
-	isolated = numamigrate_isolate_page(pgdat, page);
-	if (!isolated)
-		goto out;
-
 	list_add(&page->lru, &migratepages);
 	nr_remaining = migrate_pages(&migratepages, alloc_misplaced_dst_folio,
 				     NULL, node, MIGRATE_ASYNC,
@@ -2544,7 +2540,7 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
 					page_is_file_lru(page), -nr_pages);
 			putback_lru_page(page);
 		}
-		isolated = 0;
+		migrated = 0;
 	}
 	if (nr_succeeded) {
 		count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
@@ -2553,11 +2549,7 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
 					    nr_succeeded);
 	}
 	BUG_ON(!list_empty(&migratepages));
-	return isolated;
-
-out:
-	put_page(page);
-	return 0;
+	return migrated;
 }
 #endif /* CONFIG_NUMA_BALANCING */
 #endif /* CONFIG_NUMA */
Signature · 1 line
2.39.3
author reviewer
#5 lore raw

PATCH 3/4 mm: migrate: change migrate_misplaced_page() to support multiple pages migration

In reply to Baolin Wang

Expanding the migrate_misplaced_page() function to allow passing in a list to support multiple pages migration as a preparation to support batch migration for NUMA balancing.

---
 include/linux/migrate.h |  9 +++++----
 mm/huge_memory.c        |  4 +++-
 mm/memory.c             |  4 +++-
 mm/migrate.c            | 26 ++++++++++----------------
 4 files changed, 21 insertions(+), 22 deletions(-)
include/linux/migrate.h +5 -4
index 7c5189043707..2599f95d6c9e 100644
@@ -142,12 +142,13 @@ const struct movable_operations *page_movable_ops(struct page *page)
 }
 
 #ifdef CONFIG_NUMA_BALANCING
-int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
-			   int node);
+int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct *vma,
+			   int source_nid, int target_nid);
 int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page);
 #else
-static inline int migrate_misplaced_page(struct page *page,
-					 struct vm_area_struct *vma, int node)
+static inline int migrate_misplaced_page(struct list_head *migratepages,
+					 struct vm_area_struct *vma,
+					 int source_nid, int target_nid)
 {
 	return -EAGAIN; /* can't migrate now */
 }
mm/huge_memory.c +3 -1
index b7cc6828ce9e..53a9d63cfb1e 100644
@@ -1498,6 +1498,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 	int flags = 0;
 	pg_data_t *pgdat;
 	int isolated;
+	LIST_HEAD(migratepages);
 
 	vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
 	if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
@@ -1549,7 +1550,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 		goto isolate_fail;
 	}
 
-	migrated = migrate_misplaced_page(page, vma, target_nid);
+	list_add(&page->lru, &migratepages);
+	migrated = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid);
 	if (migrated) {
 		flags |= TNF_MIGRATED;
 		page_nid = target_nid;
mm/memory.c +3 -1
index 01b1980d4fb7..973403a83797 100644
@@ -4781,6 +4781,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 	int flags = 0;
 	pg_data_t *pgdat;
 	int isolated;
+	LIST_HEAD(migratepages);
 
 	/*
 	 * The "pte" at this point cannot be used safely without
@@ -4858,8 +4859,9 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 		goto isolate_fail;
 	}
 
+	list_add(&page->lru, &migratepages);
 	/* Migrate to the requested node */
-	if (migrate_misplaced_page(page, vma, target_nid)) {
+	if (migrate_misplaced_page(&migratepages, vma, page_nid, target_nid)) {
 		page_nid = target_nid;
 		flags |= TNF_MIGRATED;
 	} else {
mm/migrate.c +10 -16
index 5eeeb2cda21c..93d359471b95 100644
@@ -2519,36 +2519,30 @@ int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page)
  * node. Caller is expected to have an elevated reference count on
  * the page that will be dropped by this function before returning.
  */
-int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma,
-			   int node)
+int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct *vma,
+			   int source_nid, int target_nid)
 {
-	pg_data_t *pgdat = NODE_DATA(node);
+	pg_data_t *pgdat = NODE_DATA(target_nid);
 	int migrated = 1;
 	int nr_remaining;
 	unsigned int nr_succeeded;
-	LIST_HEAD(migratepages);
-	int nr_pages = thp_nr_pages(page);
 
-	list_add(&page->lru, &migratepages);
-	nr_remaining = migrate_pages(&migratepages, alloc_misplaced_dst_folio,
-				     NULL, node, MIGRATE_ASYNC,
+	nr_remaining = migrate_pages(migratepages, alloc_misplaced_dst_folio,
+				     NULL, target_nid, MIGRATE_ASYNC,
 				     MR_NUMA_MISPLACED, &nr_succeeded);
 	if (nr_remaining) {
-		if (!list_empty(&migratepages)) {
-			list_del(&page->lru);
-			mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON +
-					page_is_file_lru(page), -nr_pages);
-			putback_lru_page(page);
-		}
+		if (!list_empty(migratepages))
+			putback_movable_pages(migratepages);
+
 		migrated = 0;
 	}
 	if (nr_succeeded) {
 		count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
-		if (!node_is_toptier(page_to_nid(page)) && node_is_toptier(node))
+		if (!node_is_toptier(source_nid) && node_is_toptier(target_nid))
 			mod_node_page_state(pgdat, PGPROMOTE_SUCCESS,
 					    nr_succeeded);
 	}
-	BUG_ON(!list_empty(&migratepages));
+	BUG_ON(!list_empty(migratepages));
 	return migrated;
 }
 #endif /* CONFIG_NUMA_BALANCING */
Signature · 1 line
2.39.3
author reviewer
#6 lore raw

PATCH 4/4 mm: migrate: change to return the number of pages migrated successfully

In reply to Baolin Wang

Change the migrate_misplaced_page() to return the number of pages migrated successfully, which is used to calculate how many pages are failed to migrate for batch migration.

---
 mm/huge_memory.c | 7 ++++---
 mm/memory.c      | 5 +++--
 mm/migrate.c     | 5 +----
 3 files changed, 8 insertions(+), 9 deletions(-)
mm/huge_memory.c +4 -3
index 53a9d63cfb1e..a9c454160984 100644
@@ -1494,11 +1494,12 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 	unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
 	int page_nid = NUMA_NO_NODE;
 	int target_nid, last_cpupid = (-1 & LAST_CPUPID_MASK);
-	bool migrated = false, writable = false;
+	bool writable = false;
 	int flags = 0;
 	pg_data_t *pgdat;
 	int isolated;
 	LIST_HEAD(migratepages);
+	int nr_successed = 0;
 
 	vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
 	if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) {
@@ -1551,8 +1552,8 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf)
 	}
 
 	list_add(&page->lru, &migratepages);
-	migrated = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid);
-	if (migrated) {
+	nr_successed = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid);
+	if (nr_successed) {
 		flags |= TNF_MIGRATED;
 		page_nid = target_nid;
 	} else {
mm/memory.c +3 -2
index 973403a83797..edfd2d528e7e 100644
@@ -4780,7 +4780,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 	pte_t pte, old_pte;
 	int flags = 0;
 	pg_data_t *pgdat;
-	int isolated;
+	int isolated, nr_succeeded;
 	LIST_HEAD(migratepages);
 
 	/*
@@ -4861,7 +4861,8 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)
 
 	list_add(&page->lru, &migratepages);
 	/* Migrate to the requested node */
-	if (migrate_misplaced_page(&migratepages, vma, page_nid, target_nid)) {
+	nr_succeeded = migrate_misplaced_page(&migratepages, vma, page_nid, target_nid);
+	if (nr_succeeded) {
 		page_nid = target_nid;
 		flags |= TNF_MIGRATED;
 	} else {
mm/migrate.c +1 -4
index 93d359471b95..45f92376ba6f 100644
@@ -2523,7 +2523,6 @@ int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct
 			   int source_nid, int target_nid)
 {
 	pg_data_t *pgdat = NODE_DATA(target_nid);
-	int migrated = 1;
 	int nr_remaining;
 	unsigned int nr_succeeded;
 
@@ -2533,8 +2532,6 @@ int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct
 	if (nr_remaining) {
 		if (!list_empty(migratepages))
 			putback_movable_pages(migratepages);
-
-		migrated = 0;
 	}
 	if (nr_succeeded) {
 		count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_succeeded);
@@ -2543,7 +2540,7 @@ int migrate_misplaced_page(struct list_head *migratepages, struct vm_area_struct
 					    nr_succeeded);
 	}
 	BUG_ON(!list_empty(migratepages));
-	return migrated;
+	return nr_succeeded;
 }
 #endif /* CONFIG_NUMA_BALANCING */
 #endif /* CONFIG_NUMA */
Signature · 1 line
2.39.3
#7 lore raw
In reply to Baolin Wang
Quoted 11 lines · Baolin Wang

Hi,

Currently, on our ARM servers with NUMA enabled, we found the cross-die latency is a little larger that will significantly impact the workload's performance. So on ARM servers we will rely on the NUMA balancing to avoid the cross-die accessing. And I posted a patchset[1] to support speculative numa fault to improve the NUMA balancing's performance according to the principle of data locality. Moreover, thanks to Huang Ying's patchset[2], which introduced batch migration as a way to reduce the cost of TLB flush, and it will also benefit the migration of multiple pages all at once during NUMA balancing.

Quoted 6 lines

So we plan to continue to support batch migration in do_numa_page() to improve the NUMA balancing's performance, but before adding complicated batch migration algorithm for NUMA balancing, some cleanup and preparation work need to do firstly, which are done in this patch set. In short, this patchset extends the migrate_misplaced_page() interface to support batch migration, and no functional changes intended.

Will these cleanup benefit anything except batching migration? If not, I suggest you to post the whole series. In this way, people will be more clear about why we need these cleanup.

--
Best Regards,
Huang, Ying

Quoted 16 lines

[1] https://lore.kernel.org/lkml/[email protected]/t/#mc45929849b5d0e29b5fdd9d50425f8e95b8f2563
[2] https://lore.kernel.org/all/[email protected]/T/#u

Baolin Wang (4):
mm: migrate: move migration validation into numa_migrate_prep()
mm: migrate: move the numamigrate_isolate_page() into do_numa_page()
mm: migrate: change migrate_misplaced_page() to support multiple pages
migration
mm: migrate: change to return the number of pages migrated
successfully

 include/linux/migrate.h | 15 ++++++++---
 mm/huge_memory.c        | 19 +++++++++++---
 mm/memory.c             | 34 +++++++++++++++++++++++-
 mm/migrate.c            | 58 ++++++++---------------------------------
 4 files changed, 71 insertions(+), 55 deletions(-)
author reviewer
#8 lore raw
Quoted 13 lines · Ying
Quoted 11 lines · Baolin Wang

Hi,

Currently, on our ARM servers with NUMA enabled, we found the cross-die latency is a little larger that will significantly impact the workload's performance. So on ARM servers we will rely on the NUMA balancing to avoid the cross-die accessing. And I posted a patchset[1] to support speculative numa fault to improve the NUMA balancing's performance according to the principle of data locality. Moreover, thanks to Huang Ying's patchset[2], which introduced batch migration as a way to reduce the cost of TLB flush, and it will also benefit the migration of multiple pages all at once during NUMA balancing.

Quoted 8 lines
Quoted 6 lines

So we plan to continue to support batch migration in do_numa_page() to improve the NUMA balancing's performance, but before adding complicated batch migration algorithm for NUMA balancing, some cleanup and preparation work need to do firstly, which are done in this patch set. In short, this patchset extends the migrate_misplaced_page() interface to support batch migration, and no functional changes intended.

Will these cleanup benefit anything except batching migration? If not,

I hope these cleanup can also benefit the compound page's NUMA balancing, which was discussed in the thread[1]. IIUC, for the compound page's NUMA balancing, it is possible that partial pages were successfully migrated, so it is necessary to return the number of pages that were successfully migrated from migrate_misplaced_page(). (But I did not look this in detail yet, please correct me if I missed something, and I will find some time to look this in detail). That is why I think these cleanups are straightforward.

Yes, I will post the batch migration patches after more polish and testing, but I think these cleanups are separate and straightforward, so I plan to submit the patches separately.

[1] https://lore.kernel.org/all/[email protected]/

Quoted 23 lines

I suggest you to post the whole series. In this way, people will be more clear about why we need these cleanup.

--
Best Regards,
Huang, Ying

Quoted 16 lines

[1] https://lore.kernel.org/lkml/[email protected]/t/#mc45929849b5d0e29b5fdd9d50425f8e95b8f2563
[2] https://lore.kernel.org/all/[email protected]/T/#u

Baolin Wang (4):
mm: migrate: move migration validation into numa_migrate_prep()
mm: migrate: move the numamigrate_isolate_page() into do_numa_page()
mm: migrate: change migrate_misplaced_page() to support multiple pages
migration
mm: migrate: change to return the number of pages migrated
successfully

  include/linux/migrate.h | 15 ++++++++---
  mm/huge_memory.c        | 19 +++++++++++---
  mm/memory.c             | 34 +++++++++++++++++++++++-
  mm/migrate.c            | 58 ++++++++---------------------------------
  4 files changed, 71 insertions(+), 55 deletions(-)
#9 lore raw
Quoted 33 lines · Baolin Wang
Quoted 13 lines · Ying
Quoted 11 lines · Baolin Wang

Hi,

Currently, on our ARM servers with NUMA enabled, we found the cross-die latency is a little larger that will significantly impact the workload's performance. So on ARM servers we will rely on the NUMA balancing to avoid the cross-die accessing. And I posted a patchset[1] to support speculative numa fault to improve the NUMA balancing's performance according to the principle of data locality. Moreover, thanks to Huang Ying's patchset[2], which introduced batch migration as a way to reduce the cost of TLB flush, and it will also benefit the migration of multiple pages all at once during NUMA balancing.

Quoted 8 lines
Quoted 6 lines

So we plan to continue to support batch migration in do_numa_page() to improve the NUMA balancing's performance, but before adding complicated batch migration algorithm for NUMA balancing, some cleanup and preparation work need to do firstly, which are done in this patch set. In short, this patchset extends the migrate_misplaced_page() interface to support batch migration, and no functional changes intended.

Will these cleanup benefit anything except batching migration? If not,

I hope these cleanup can also benefit the compound page's NUMA balancing, which was discussed in the thread[1]. IIUC, for the compound page's NUMA balancing, it is possible that partial pages were successfully migrated, so it is necessary to return the number of pages that were successfully migrated from migrate_misplaced_page(). (But I did not look this in detail yet, please correct me if I missed something, and I will find some time to look this in detail). That is why I think these cleanups are straightforward.

Quoted 3 lines

Yes, I will post the batch migration patches after more polish and testing, but I think these cleanups are separate and straightforward, so I plan to submit the patches separately.

Then, please state the benefit explicitly in the patch description instead of just preparation for batching migration.

--
Best Regards,
Huang, Ying

Quoted 25 lines

[1]
https://lore.kernel.org/all/[email protected]/

Quoted 22 lines

I suggest you to post the whole series. In this way, people will be more clear about why we need these cleanup.
--
Best Regards,
Huang, Ying

Quoted 16 lines

[1] https://lore.kernel.org/lkml/[email protected]/t/#mc45929849b5d0e29b5fdd9d50425f8e95b8f2563
[2] https://lore.kernel.org/all/[email protected]/T/#u

Baolin Wang (4):
mm: migrate: move migration validation into numa_migrate_prep()
mm: migrate: move the numamigrate_isolate_page() into do_numa_page()
mm: migrate: change migrate_misplaced_page() to support multiple pages
migration
mm: migrate: change to return the number of pages migrated
successfully

  include/linux/migrate.h | 15 ++++++++---
  mm/huge_memory.c        | 19 +++++++++++---
  mm/memory.c             | 34 +++++++++++++++++++++++-
  mm/migrate.c            | 58 ++++++++---------------------------------
  4 files changed, 71 insertions(+), 55 deletions(-)
author reviewer
#10 lore raw
Quoted 35 lines · Ying
Quoted 33 lines · Baolin Wang
Quoted 13 lines · Ying
Quoted 11 lines · Baolin Wang

Hi,

Currently, on our ARM servers with NUMA enabled, we found the cross-die latency is a little larger that will significantly impact the workload's performance. So on ARM servers we will rely on the NUMA balancing to avoid the cross-die accessing. And I posted a patchset[1] to support speculative numa fault to improve the NUMA balancing's performance according to the principle of data locality. Moreover, thanks to Huang Ying's patchset[2], which introduced batch migration as a way to reduce the cost of TLB flush, and it will also benefit the migration of multiple pages all at once during NUMA balancing.

Quoted 8 lines
Quoted 6 lines

So we plan to continue to support batch migration in do_numa_page() to improve the NUMA balancing's performance, but before adding complicated batch migration algorithm for NUMA balancing, some cleanup and preparation work need to do firstly, which are done in this patch set. In short, this patchset extends the migrate_misplaced_page() interface to support batch migration, and no functional changes intended.

Will these cleanup benefit anything except batching migration? If not,

I hope these cleanup can also benefit the compound page's NUMA balancing, which was discussed in the thread[1]. IIUC, for the compound page's NUMA balancing, it is possible that partial pages were successfully migrated, so it is necessary to return the number of pages that were successfully migrated from migrate_misplaced_page(). (But I did not look this in detail yet, please correct me if I missed something, and I will find some time to look this in detail). That is why I think these cleanups are straightforward.

Quoted 6 lines
Quoted 3 lines

Yes, I will post the batch migration patches after more polish and testing, but I think these cleanups are separate and straightforward, so I plan to submit the patches separately.

Then, please state the benefit explicitly in the patch description instead of just preparation for batching migration.

Sure, will do. Thanks.