mm/damon: add node_eligible_mem_bp goal metric

Sashiko AI review found 4 potential issue(s):
- [High] Missing cond_resched() inside the inner PFN-scanning loop causes CPU soft lockups.
- [High] Casting 64-bit phys_addr_t to 32-bit unsigned long causes massive quota calculation errors on 32-bit PAE systems.
- [High] Physical address wraparound of folio_end causes an infinite loop and system hang.
- [High] Pre-existing VM_BUG_ON in speculative folio access within damon_get_folio().

--

commit 377bf0664129c87741fa640c563456b379c8a816
Author: Ravi Jonnalagadda <[email protected]>

mm/damon: add node_eligible_mem_bp goal metric

This patch enables system-wide page distribution with target-state goals using PA-mode DAMON schemes. It tracks the ratio of scheme-eligible memory on a target node to adjust migration aggressiveness. The implementation calculates the current memory distribution across regions that match the scheme's access pattern.

Quoted 4 lines

mm/damon/core.c +0 -0

index 3dbbbfdeff71..9a4220174a61 100644

[ ... ]

Quoted 1 line

mm/damon/core.c +0 -0

@@ -2287,7 +2306,115 @@ static unsigned long damos_get_node_memcg_used_bp(

[ ... ]

mm/damon/core.c static unsigned long damos_get_node_memcg_used_bp( · line 2351

Quoted 7 lines

mm/damon/core.c +7 -0

+			while (addr < end_addr) {
+				struct folio *folio;
+				phys_addr_t folio_start, folio_end;
+				phys_addr_t overlap_start, overlap_end;
+				phys_addr_t counted;
+
+				folio = damon_get_folio(PHYS_PFN(addr));

Since this code heavily invokes damon_get_folio() in a new PFN scanner, could the speculative folio access inside damon_get_folio() trigger a VM_BUG_ON? If damon_get_folio() tests folio_test_lru() before acquiring a reference with folio_try_get(), the page could be concurrently split or reallocated as a tail page.

Quoted 26 lines

mm/damon/core.c +25 -0

+				if (!folio) {
+					addr = PAGE_ALIGN_DOWN(addr +
+							PAGE_SIZE);
+					if (!addr)
+						break;
+					continue;
+				}
+
+				/*
+				 * Calculate exact overlap between the region
+				 * [addr, end_addr) and the folio range.
+				 * The folio may start before addr if addr is
+				 * in the middle of a large folio.
+				 */
+				folio_start = PFN_PHYS(folio_pfn(folio));
+				folio_end = folio_start + folio_size(folio);
+
+				overlap_start = max(addr, folio_start);
+				overlap_end = min(end_addr, folio_end);
+
+				if (overlap_end > overlap_start) {
+					counted = overlap_end - overlap_start;
+					total_eligible += counted;
+					if (folio_nid(folio) == nid)
+						node_eligible += counted;

mm/damon/core.c static unsigned long damos_get_node_memcg_used_bp( · line 2380

Quoted 4 lines

mm/damon/core.c +4 -0

+				}
+
+				/* Advance past the entire folio */
+				addr = folio_end;

If a folio is located at the very end of the physical address space, could folio_end wrap around to 0? If addr becomes 0, could it cause an infinite loop since addr < end_addr would remain true?

mm/damon/core.c static unsigned long damos_get_node_memcg_used_bp( · line 2383

Quoted 4 lines

mm/damon/core.c +3 -0

+				folio_put(folio);
+			}
+			cond_resched();

Could scanning a large DAMON region without yielding the CPU in the inner loop cause soft lockups? Moving the cond_resched() inside the while loop would prevent it from monopolizing the processor.

Quoted 25 lines

mm/damon/core.c +24 -0

+		}
+	}
+
+	*total = total_eligible;
+	return node_eligible;
+}
+
+static unsigned long damos_get_node_eligible_mem_bp(struct damon_ctx *c,
+		struct damos *s, int nid)
+{
+	phys_addr_t total_eligible = 0;
+	phys_addr_t node_eligible;
+
+	if (c->ops.id != DAMON_OPS_PADDR)
+		return 0;
+
+	if (nid < 0 || nid >= MAX_NUMNODES || !node_online(nid))
+		return 0;
+
+	node_eligible = damos_calc_eligible_bytes(c, s, nid, &total_eligible);
+
+	if (!(unsigned long)total_eligible)
+		return 0;
+

mm/damon/core.c static unsigned long damos_get_node_memcg_used_bp( · line 2409

Quoted 2 lines

mm/damon/core.c +2 -0

+	return mult_frac((unsigned long)node_eligible, 10000,
+			(unsigned long)total_eligible);

Does casting these phys_addr_t variables to unsigned long cause truncation errors on 32-bit PAE systems where physical addresses are 64-bit? For example, if the eligible memory size exceeds 4GB, the truncation could corrupt the ratio metrics and break the quota autotuner. Would converting byte counts to page counts before casting avoid this?

Background and Motivation
=========================

In heterogeneous memory systems, controlling memory distribution across NUMA nodes is essential for performance optimization. This patch enables system-wide page distribution with target-state goals such as "maintain 60% of scheme-eligible memory on DRAM" using PA-mode DAMON schemes.

Rather than using absolute thresholds, this metric tracks the ratio of memory that matches each scheme's access pattern filters on a target node, enabling the quota system to automatically adjust migration aggressiveness to maintain the desired distribution.

What This Metric Measures
=========================

node_eligible_mem_bp:
scheme_eligible_bytes_on_node / total_scheme_eligible_bytes * 10000

Two-Scheme Setup for Hot Page Distribution
==========================================

For maintaining 60% of hot memory on DRAM (node 0) and 40% on CXL (node 1):

PULL scheme: migrate_hot to node 0
goal: node_eligible_mem_bp, nid=0, target=6000
addr filter: node 1 address range (only migrate FROM CXL) "Move hot pages to DRAM if less than 60% of hot data is in DRAM"

PUSH scheme: migrate_hot to node 1
goal: node_eligible_mem_bp, nid=1, target=4000
addr filter: node 0 address range (only migrate FROM DRAM) "Move hot pages to CXL if less than 40% of hot data is in CXL"

Each scheme independently measures its own eligible memory and adjusts its quota to achieve its target ratio. The schemes work in concert through DAMON's unified monitoring context, with the quota autotuner balancing their relative aggressiveness.

Implementation Details
======================

The implementation adds a new quota goal metric type DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP to the existing DAMOS quota goal framework. When this metric is configured for a scheme:

1. During each quota adjustment cycle, damos_get_node_eligible_mem_bp() is called to calculate the current memory distribution.

2. The function iterates through all regions that match the scheme's access pattern (via __damos_valid_target()) and calculates:
- Total eligible bytes across all nodes
- Eligible bytes specifically on the target node (goal->nid)

3. For each eligible region, damos_calc_eligible_bytes() walks through the physical address range, using damon_get_folio() to look up each folio and determine its NUMA node via folio_nid().

4. Large folios are handled by calculating the exact overlap between the region boundaries and folio boundaries, ensuring accurate byte counts even when regions partially span folios.

5. The ratio (node_eligible / total_eligible * 10000) is returned as basis points, which the quota autotuner uses to adjust the scheme's effective quota size (esz).

The implementation requires CONFIG_DAMON_PADDR since damon_get_folio() is only available for physical address space monitoring.

Testing Results
===============

Functionally tested on a two-node heterogeneous memory system with DRAM (node 0) and CXL memory (node 1). A PUSH+PULL scheme configuration using migrate_hot actions was used to reach a target hot memory ratio between the two tiers.

With the TEMPORAL tuner, the system converges quickly to the target distribution. The tuner drives esz to maximum when under goal and to zero once the goal is met, forming a simple on/off feedback loop that stabilizes at the desired ratio.

With the CONSIST tuner, the scheme still converges but more slowly, as it migrates and then throttles itself based on quota feedback. The time to reach the goal varies depending on workload intensity.

Note: This metric works with both TEMPORAL and CONSIST goal tuners.

Suggested-by SeongJae Park <[email protected]>

Signed-off-by Ravi Jonnalagadda <[email protected]>

Reviewed-by SeongJae Park <[email protected]>