mm/damon: Introduce node_target_mem_bp Quota Goal Metric

This series introduces a new DAMON quota goal metric, node_target_mem_bp, designed for controlling memory migration in heterogeneous memory systems (e.g., DRAM and CXL memory tiering).

v1: https://lore.kernel.org/linux-mm/[email protected]/T/#u

Changes since v1:
=================

- Renamed metric from node_sys_bp to node_target_mem_bp for consistency with existing node-related quota goal metrics (node_mem_used_bp, node_mem_free_bp) as suggested by SJ.

- Fixed the metric calculation:
* Numerator: Now correctly counts only scheme-eligible bytes (regions matching the scheme's access pattern criteria).
* Denominator: Now uses node capacity instead of total system memory.

- Removed the get_goal_metric() ops callback. The implementation now resides in core.c, following the existing pattern for other metrics that have ops-layer dependencies.

- Removed the early-exit optimization patch. As SJ noted, this would introduce a behavioral change for existing users and should be an opt-in feature with a properly designed interface. This can be addressed in a separate follow-up series.

- Removed capacity clamping logic (was tied to early-exit behavior).

Background and Motivation
=========================

A previous patch series [1] added weighted interleave support for DAMON migrate_{hot,cold} actions for vaddr schemes. That approach requires VMA offset information to determine target nodes, which for paddr schemes would require costly rmap walks.

This series takes a different approach for PA-based migration control using basis points (bp) target-state goals instead of weight-based action rates, avoiding the need for rmap walks entirely.

What This Metric Does
=====================

The node_target_mem_bp metric measures:

scheme_eligible_bytes_on_node / node_capacity

expressed in basis points (bp, 1/10000).

"Scheme-eligible bytes" are regions that match the scheme's access pattern criteria (size, nr_accesses, age). This allows users to specify goals like:

"Migrate hot pages until node N contains X% hot memory"

Unlike weight-based approaches that specify ACTION RATES, this metric specifies a TARGET STATE, which naturally prevents oscillation issues that would occur with weight-based PA migration without rmap.

Two-Context Setup for Hot Page Distribution
===========================================

For distributing hot pages between two NUMA nodes (e.g., DRAM node 0 and CXL node 1), two DAMON contexts work together:

    Context 0: monitors node 0, migrate_hot -> node 1
      goal: node_target_mem_bp, nid=0, target=6000
      "Migrate hot pages out when node 0 exceeds 60% hot"

    Context 1: monitors node 1, migrate_hot -> node 0
      goal: node_target_mem_bp, nid=1, target=4000
      "Migrate hot pages out when node 1 exceeds 40% hot"

Each context migrates excess hot pages to the other node. The system converges when both nodes reach their target hot memory ratios.

Complementary to Existing vaddr Migration
=========================================

This series complements rather than replaces the vaddr weighted interleave migration:

vaddr migration (weight-based):
- Per-process control
- Fine-grained interleave patterns via VMA offset
- Deterministic placement based on weights

paddr migration (bp-based, this series):
- System-wide control
- Target-state goals for node capacity management
- No rmap overhead

Patch Organization
==================

1. mm/damon/core: add DAMOS_QUOTA_NODE_TARGET_MEM_BP metric
- Adds new enum value and documentation

2. mm/damon/core: implement NODE_TARGET_MEM_BP metric calculation
- Adds damos_get_node_target_mem_bp() function
- Updates function signatures to pass ctx and scheme through call chain

3. mm/damon/sysfs-schemes: expose NODE_TARGET_MEM_BP metric
- Exposes metric as 'node_target_mem_bp' in sysfs

Status
======

These patches have been compile-tested but have NOT been tested on actual hardware. Feedback on the design and approach is appreciated.

References
==========

[1] mm/damon/vaddr: Allow interleaving in migrate_{hot,cold} actions

    https://lore.kernel.org/linux-mm/[email protected]/

Ravi Jonnalagadda (3):
mm/damon/core: add DAMOS_QUOTA_NODE_TARGET_MEM_BP metric mm/damon/core: implement NODE_TARGET_MEM_BP metric calculation mm/damon/sysfs-schemes: expose NODE_TARGET_MEM_BP metric

 include/linux/damon.h    |  5 +++
 mm/damon/core.c          | 66 +++++++++++++++++++++++++++++++++++-----
 mm/damon/sysfs-schemes.c |  5 +++
 3 files changed, 69 insertions(+), 7 deletions(-)

2.43.0

Add a new quota goal metric DAMOS_QUOTA_NODE_TARGET_MEM_BP for physical address space-based DAMON operation schemes. This metric represents the ratio of scheme-eligible memory on a specific NUMA node to that node's total capacity, expressed in basis points (1/10000).

The metric enables auto-tuning of DAMOS quotas based on how much memory on a node matches the scheme's access pattern criteria, which is essential for controlling memory migration in heterogeneous memory systems (e.g., DRAM and CXL memory tiering).

---
 include/linux/damon.h | 5 +++++
 1 file changed, 5 insertions(+)

index 3813373a9200..5eebb8aaab85 100644
@@ -155,6 +155,7 @@ enum damos_action {
  * @DAMOS_QUOTA_NODE_MEM_FREE_BP:	MemFree ratio of a node.
  * @DAMOS_QUOTA_NODE_MEMCG_USED_BP:	MemUsed ratio of a node for a cgroup.
  * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP:	MemFree ratio of a node for a cgroup.
+ * @DAMOS_QUOTA_NODE_TARGET_MEM_BP:	Scheme-eligible memory ratio of a node.
  * @NR_DAMOS_QUOTA_GOAL_METRICS:	Number of DAMOS quota goal metrics.
  *
  * Metrics equal to larger than @NR_DAMOS_QUOTA_GOAL_METRICS are unsupported.

@@ -166,6 +167,7 @@ enum damos_quota_goal_metric {
 	DAMOS_QUOTA_NODE_MEM_FREE_BP,
 	DAMOS_QUOTA_NODE_MEMCG_USED_BP,
 	DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
+	DAMOS_QUOTA_NODE_TARGET_MEM_BP,
 	NR_DAMOS_QUOTA_GOAL_METRICS,
 };
 

@@ -193,6 +195,9 @@ enum damos_quota_goal_metric {
  *
  * If @metric is DAMOS_QUOTA_NODE_MEMCG_{USED,FREE}_BP, @nid and @memcg_id
  * represents the node id and the cgroup to account the used memory for.
+ *
+ * If @metric is DAMOS_QUOTA_NODE_TARGET_MEM_BP, @nid represents the node
+ * to measure scheme-eligible memory ratio against its capacity.
  */
 struct damos_quota_goal {
 	enum damos_quota_goal_metric metric;

2.43.0

Please keep "RFC" on the subject consistently.

As I mentioned on the cover letter, other parts including the design and impelmentation of all patches look good to me. Thank you for doing this.

Thanks,
SJ

[...]

Add damos_get_node_target_mem_bp() function that calculates the ratio of scheme-eligible memory on a specific NUMA node to the total node capacity, expressed in basis points (bp, 1/10000).

The function iterates through all regions that match the scheme's access pattern criteria (checked via __damos_valid_target) and counts how many pages from those regions reside on the specified node. This enables quota auto-tuning based on the actual distribution of hot/cold pages across NUMA nodes.

To support this new metric which requires access to both the DAMON context and scheme:
- Update damos_set_quota_goal_current_value() signature
- Update damos_quota_score() to pass ctx and scheme through
- Update damos_set_effective_quota() and its callers

This metric is particularly useful for heterogeneous memory systems (e.g., DRAM + CXL) where controlling the distribution of hot pages across nodes can optimize memory bandwidth utilization.

---
 mm/damon/core.c | 66 +++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 59 insertions(+), 7 deletions(-)

index 84f80a20f233..1482c97828e8 100644
@@ -12,6 +12,7 @@
 #include <linux/kthread.h>
 #include <linux/memcontrol.h>
 #include <linux/mm.h>
+#include <linux/mmzone.h>
 #include <linux/psi.h>
 #include <linux/slab.h>
 #include <linux/string.h>

@@ -2119,8 +2120,52 @@ static unsigned long damos_get_node_memcg_used_bp(
 }
 #endif
 
+static unsigned long damos_get_node_target_mem_bp(
+		struct damon_ctx *ctx, struct damos *scheme,
+		struct damos_quota_goal *goal)
+{
+	int nid = goal->nid;
+	unsigned long node_capacity, scheme_node_bytes = 0;
+	unsigned long addr_unit = ctx->addr_unit;
+	struct damon_target *t;
+	struct damon_region *r;
+	unsigned long start_pfn, end_pfn, pfn;
+
+	/* Only supported for physical address space monitoring */
+	if (ctx->ops.id != DAMON_OPS_PADDR)
+		return 0;
+
+	if (nid < 0 || nid >= MAX_NUMNODES || !node_online(nid))
+		return 0;
+
+	node_capacity = NODE_DATA(nid)->node_spanned_pages << PAGE_SHIFT;
+	if (!node_capacity)
+		return 0;
+
+	damon_for_each_target(t, ctx) {
+		damon_for_each_region(r, t) {
+			if (!__damos_valid_target(r, scheme))
+				continue;
+
+			start_pfn = (phys_addr_t)r->ar.start *
+					addr_unit >> PAGE_SHIFT;
+			end_pfn = (phys_addr_t)r->ar.end *
+					addr_unit >> PAGE_SHIFT;
+
+			for (pfn = start_pfn; pfn < end_pfn; pfn++) {
+				if (pfn_valid(pfn) &&
+				    page_to_nid(pfn_to_page(pfn)) == nid)
+					scheme_node_bytes += PAGE_SIZE;
+			}
+		}
+	}
+
+	return mult_frac(scheme_node_bytes, 10000, node_capacity);
+}
 
-static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
+static void damos_set_quota_goal_current_value(
+		struct damon_ctx *ctx, struct damos *scheme,
+		struct damos_quota_goal *goal)
 {
 	u64 now_psi_total;
 

@@ -2141,19 +2186,25 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
 	case DAMOS_QUOTA_NODE_MEMCG_FREE_BP:
 		goal->current_value = damos_get_node_memcg_used_bp(goal);
 		break;
+	case DAMOS_QUOTA_NODE_TARGET_MEM_BP:
+		goal->current_value = damos_get_node_target_mem_bp(
+				ctx, scheme, goal);
+		break;
 	default:
 		break;
 	}
 }
 
 /* Return the highest score since it makes schemes least aggressive */
-static unsigned long damos_quota_score(struct damos_quota *quota)
+static unsigned long damos_quota_score(
+		struct damon_ctx *ctx, struct damos *scheme,
+		struct damos_quota *quota)
 {
 	struct damos_quota_goal *goal;
 	unsigned long highest_score = 0;
 
 	damos_for_each_quota_goal(goal, quota) {
-		damos_set_quota_goal_current_value(goal);
+		damos_set_quota_goal_current_value(ctx, scheme, goal);
 		highest_score = max(highest_score,
 				goal->current_value * 10000 /
 				goal->target_value);

@@ -2165,7 +2216,8 @@ static unsigned long damos_quota_score(struct damos_quota *quota)
 /*
  * Called only if quota->ms, or quota->sz are set, or quota->goals is not empty
  */
-static void damos_set_effective_quota(struct damos_quota *quota)
+static void damos_set_effective_quota(struct damon_ctx *ctx,
+		struct damos *scheme, struct damos_quota *quota)
 {
 	unsigned long throughput;
 	unsigned long esz = ULONG_MAX;

@@ -2176,7 +2228,7 @@ static void damos_set_effective_quota(struct damos_quota *quota)
 	}
 
 	if (!list_empty(&quota->goals)) {
-		unsigned long score = damos_quota_score(quota);
+		unsigned long score = damos_quota_score(ctx, scheme, quota);
 
 		quota->esz_bp = damon_feed_loop_next_input(
 				max(quota->esz_bp, 10000UL),

@@ -2227,7 +2279,7 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
 	/* First charge window */
 	if (!quota->total_charged_sz && !quota->charged_from) {
 		quota->charged_from = jiffies;
-		damos_set_effective_quota(quota);
+		damos_set_effective_quota(c, s, quota);
 	}
 
 	/* New charge window starts */

@@ -2240,7 +2292,7 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
 		quota->charged_sz = 0;
 		if (trace_damos_esz_enabled())
 			cached_esz = quota->esz;
-		damos_set_effective_quota(quota);
+		damos_set_effective_quota(c, s, quota);
 		if (trace_damos_esz_enabled() && quota->esz != cached_esz)
 			damos_trace_esz(c, s, quota);
 	}

2.43.0

Add sysfs support for the new NODE_TARGET_MEM_BP quota goal metric. This exposes the metric as 'node_target_mem_bp' under the quota goal's target_metric attribute.

The metric measures the ratio of scheme-eligible memory (regions matching the scheme's access pattern) on a specified NUMA node to the node's total capacity, expressed in basis points (bp, 1/10000).

Users can configure this metric by:
1. Setting target_metric to 'node_target_mem_bp'
2. Setting nid to the target NUMA node
3. Setting target_value to the desired ratio in basis points

The current_value attribute shows the measured ratio, which can be used by userspace to compute the actual bytes of scheme-eligible memory on the node: bytes = current_value * node_capacity / 10000.

This is particularly useful for tiered memory systems to monitor and control the distribution of hot pages across NUMA nodes.

---
 mm/damon/sysfs-schemes.c | 5 +++++
 1 file changed, 5 insertions(+)

index 3a699dcd5a7f..50133263c592 100644
@@ -1038,6 +1038,10 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
 		.metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
 		.name = "node_memcg_free_bp",
 	},
+	{
+		.metric = DAMOS_QUOTA_NODE_TARGET_MEM_BP,
+		.name = "node_target_mem_bp",
+	},
 };
 
 static ssize_t target_metric_show(struct kobject *kobj,

@@ -2554,6 +2558,7 @@ static int damos_sysfs_add_quota_score(
 			break;
 		case DAMOS_QUOTA_NODE_MEM_USED_BP:
 		case DAMOS_QUOTA_NODE_MEM_FREE_BP:
+		case DAMOS_QUOTA_NODE_TARGET_MEM_BP:
 			goal->nid = sysfs_goal->nid;
 			break;
 		case DAMOS_QUOTA_NODE_MEMCG_USED_BP:

2.43.0

[...]

Thank you for adding this example use case! This is very helpful for understanding how people can use this feature, and if there is a wrong assumption.

I think the use case idea is nice and making sense to me. Nonetheless, I find a DAMON's devil in the detail.

DAMOS quota autotuning assumes applying the given scheme action more aggressively (increasing quota) will help increasing the quota goal metric. In other words, it believes the aggressiveness (tuned quota size) and the metric value are proportional. Hence, for the first context, DAMON will migrate hot pages of node 0 to node 1, when the hot pages in node 0 is less than 60%, and start gradually decreasing and eventually stop the migration after hot memory portion on node 0 reaches and exceeds 60%. A human readable interpretation of it would be, "Migrate hot pages out when node 0 not exceeds 60% hot", which makes no sense for your use case.

To make it work as you described, you may implement another metric representing the ratio of scheme-uneligible memory on the given node. Say, 'node_ineligible_mem_bp'? To borrow your above nice notation, it could be calculated as below:

(node_capacity - scheme_eligible_bytes_on_node) / node_capacity

Using this, your above use case could implemented like below:

    Context 0: monitors node 0, migrate_hot -> node 1
      goal: node_ineligible_mem_bp, nid=0, target=4000

    Context 1: monitors node 1, migrate_hot -> node 0
      goal: node_ineligible_mem_bp, nid=1, target=6000

And I'm not very sure if that is really what you want. For example, if node 0 has 30% hot memory and node 1 has 20% hot memory, no migration will happen.

I think you might want node 0 to have more hot memory, but no more than 60% of the node. DAMON-based auto-tuned memory tiering [1], for example, use this kind of approach. If that's what you want, you could use node_target_mem_bp together, like below.

    Context 0: monitors node 0, migrate_hot -> node 1
      goal: node_ineligible_mem_bp, nid=0, target=4000

    Context 1: monitors node 1, migrate_hot -> node 0
      goal: node_target_mem_bp, nid=0, target=6000

I'm not still very confident if I understand what you want, because you mentioned dynamic weighted interleaving was the major motivation of this project. In the case, you might want only hot memory be distributed across NUMA nodes in a specific ratio. In the case, you may want the denominator be "scheme-eligible memory of the system" instead of "node capacity". To borrow your notation again,

scheme_eligible_bytes_on_node / scheme_eligible_bytes_on_system

Let's call this just node_target_mem_bp2. Then, if you want node 0 and 1 to have 60% and 40% of hot memory, you could setup DAMOS as below:

    Context 0: monitors node 0, migrate_hot -> node 1
      goal: node_target_mem_bp2, nid=1, target=4000

    Context 1: monitors node 1, migrate_hot -> node 0
      goal: node_target_mem_bp2, nid=0, target=6000

[...]

It will be very helpful!

So you might need to change the definition and name of the metric, and/or add new metrics. But the basic theory of the requirements, the design and the implementation approach of this patch series looks good to me!

[1] https://github.com/damonitor/damo/blob/next/scripts/mem_tier.sh

Thanks,
SJ

[...]

[...]

In offline, Ravi enlightened me that using a single context with two schemes instead of the above two contexts setup can be more efficienct and useful. I agree that. It will be able to only single kdamond, and there could be more flexible use cases that can use the whole-memory access pattern.

That is, we can use single context with the two schemes, but adding a core layer DAMOS filters for applying the schemes to only memory of node 0 and node 1, respectively. Similar for memory tiering use cases.

But I was recommending the multi contexts approach to people because the current implementation of DAMOS is not efficient when both quota and core layer filters are used. I was actually working on making it improved, and just posted an RFC patch series [1]. After the patches are merged, hopefully the single context approach will be useful and effcient enough for varying use cases including the memory tiering.

[1] https://lore.kernel.org/[email protected]

Thanks,
SJ

[...]

Thanks for providing the DAMOS_FILTER patch update SJ.

For v3, I plan to introduce two complementary metrics:
DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP
and DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP.

This will support the following approaches for hot memory migration:
1. Single context with two schemes using both metrics. (along with DAMOS_FILTER_TYPE_ADDR)
2. Two DAMON contexts each using DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP.

Will provide more details on the implementation and usage in the v3 series.

Thanks,
Ravi.

Sounds good!

Looking forward to it!

Thanks,
SJ

[...]