mm/damon/ops-common: prevent migration fallback to non-target nodes

From: Jiahui Zhang <[email protected]>

DAMOS_MIGRATE_{HOT,COLD} passes a target NUMA node to migrate_pages(). But alloc_migration_target() only treats mtc->nid as a preferred node unless __GFP_THISNODE is set. Hence target allocation can fall back to another node, and migrate_pages() can report success without placing the folio on the requested target node.

Consider a two-node tiered system where node 0 is a fast tier and node 1 is a CPU-less slow tier such as CXL memory, and the user wants to promote hot regions from node 1 to node 0 with a command like:

  sudo damo start --ops vaddr --target_pid ${workload_pid} \
      --damos_action migrate_hot 0 \
      --damos_access_rate 70% max

Without the __GFP_THISNODE flag, when the memory allocator finds that node 0 is nearly full, it can fall back to node 1 without waking up kswapd. Then the pages allocated for migrate_pages() are still on node 1, and the regions that are expected to be promoted to node 0 are only moved to different physical pages on node 1.

Meanwhile, both the mm_migrate_pages tracepoint and DAMOS's own sz_applied statistics (reported via the damos_stat_after_apply_interval tracepoint) show the migrations as successful, which makes the failure practically invisible and hard to investigate.

Running a demotion-purpose DAMOS scheme alongside the promotion scheme does not fully avoid this either. If demotion cannot keep up with the promotion rate, allocation can still fall back to node 1 during promotion, and the same misleading statistics show up.

Make DAMON's migration target allocation strict by setting __GFP_THISNODE, so that a failed allocation on the target node is reported as a failure instead of silently landing on a different node. This is consistent with alloc_misplaced_dst_folio(), alloc_demote_folio(), and with do_move_pages_to_node(), which all use __GFP_THISNODE for migrations to an explicit destination node.

---
Changes from v2
- v2: https://lore.kernel.org/ <MN2PR02MB665479119535A1235D69F125B5F92@MN2PR02MB6654.namprd02.prod.outlook.com>
- Collect R-b: from SJ.
- Rebase to the latest mm-new.
Changes since v1:
- Use the mm/damon/ops-common subject prefix.
- Describe the observed misleading migration accounting and how strict
target-node allocation resolves it.
- Use the formal author name.
- No functional/code change from v1
- Link to v1: https://lore.kernel.org/damon/[email protected]

 mm/damon/ops-common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index d7d7f100389b0..e59f77eca83b2 100644
@@ -312,7 +312,7 @@ static unsigned int __damon_migrate_folio_list(
 		 * instead of migrated.
 		 */
 		.gfp_mask = (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) |
-			__GFP_NOMEMALLOC | GFP_NOWAIT,
+			__GFP_NOMEMALLOC | GFP_NOWAIT | __GFP_THISNODE,
 		.nid = target_nid,
 	};
 

base-commit: 86e9ddaf85a3e872e81e1946ed410ef7f5341e16

2.47.3

The new behavior looks fine for now. DAMON-based memory tiering is usually for proactive use, so no much memory pressure is expected. We migth need to revisit this later for making it more cleaner, though. Maybe it should be more similar to that of do_move_pages_to_node().

Thanks,
SJ

[...]