mm/damon: Introduce node_eligible_mem_bp and node_ineligible_mem_bp Quota Goal Metrics

RFC PATCH Superseded Ravi Jonnalagadda 2 participants 21 messages Feb 23 – Feb 27

Subsystems (3)

Memory management Documentation DAMON

lore

Share & tools

lore

Ravi Jonnalagadda

author

2026-02-23 12:32 UTC #0 lore raw

v3 cover mm/damon: Introduce node_eligible_mem_bp and node_ineligible_mem_bp Quota Goal Metrics

MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This series introduces two new DAMON quota goal metrics for controlling memory migration in heterogeneous memory systems (e.g., DRAM and CXL memory tiering) using physical address (PA) mode monitoring.

v2: https://lore.kernel.org/linux-mm/[email protected]/

Changes since v2:
=================

- Split single metric into two complementary metrics:
* node_eligible_mem_bp: hot memory present ON the specified node
* node_ineligible_mem_bp: hot memory NOT on the specified node. This enables both PUSH and PULL schemes to work together.

- Added PA-mode detection lag compensation cache (see dedicated section below for design details).

- Added fix for esz=0 quota bypass that allowed unlimited migration when goal was achieved.

- Added fix for goal_tuner sysfs setting being ignored due to damon_new_scheme() always defaulting to CONSIST.

- Rebased on SJ's damon/next branch which includes the TEMPORAL goal tuner required for these metrics.

Background and Motivation
=========================

In heterogeneous memory systems, controlling hot memory distribution across NUMA nodes is essential for performance optimization. This series enables system wide hot page distribution with target-state goals like "maintain 30% of hot memory on CXL" using PA-mode DAMON schemes.

Two-Scheme Setup for Hot Page Distribution
==========================================

For maintaining 30% of hot memory on CXL (node 1):

    PUSH scheme (DRAM->CXL): migrate_hot from node 0 -> node 1
      goal: node_eligible_mem_bp, nid=1, target=3000
      Activates when node 1 has less than 30% hot memory

    PULL scheme (CXL->DRAM): migrate_hot from node 1 -> node 0
      goal: node_ineligible_mem_bp, nid=1, target=7000
      Activates when node 1 has more than 30% hot memory

Both schemes use the TEMPORAL goal tuner which sets esz to maximum when under goal and zero when achieved. Together they converge to equilibrium at the target distribution.

What These Metrics Do
=====================

node_eligible_mem_bp measures:
effective_hot_bytes_on_node / total_hot_bytes * 10000

node_ineligible_mem_bp measures:
(total_hot_bytes - effective_hot_bytes_on_node) / total_hot_bytes * 10000

The metrics are complementary: eligible_bp + ineligible_bp = 10000 bp.

PA-Mode Detection Lag and Cache Design
======================================

In PA-mode, when pages are migrated:
1. Source node detection drops immediately (pages are gone)
2. Target node detection increases slowly (new addresses need sampling)

This asymmetry causes temporary underestimation of hot memory on the target node. Without compensation, the system keeps migrating even after reaching the goal.

The cache addresses this by remembering how much was recently migrated. When calculating effective hot memory:
- Source node: reduce detected amount by recent migrations out
- Target node: boost detected amount by recent migrations in

The cache uses a rolling window to track migrations over time, and expires after a configurable timeout (default 10s) when no migration activity occurs. It also detects when its baseline becomes stale due to new hot memory appearing in the workload.

Dependencies
============

This series is based on SJ's damon/next branch which includes:

- mm/damon/core: introduce damos_quota_goal_tuner [1]
- mm/damon/core: set quota-score histogram with core filters [2]
- mm/damon: always respect min_nr_regions from the beginning [3]
- mm/damon/core: disallow non-power of two min_region_sz [4]

[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/

Patch Organization
==================

1. mm/damon/sysfs: set goal_tuner after scheme creation
- Fixes goal_tuner initialization order in sysfs scheme creation

2. mm/damon: fix esz=0 quota bypass allowing unlimited migration
- Ensures esz=0 stops migration rather than bypassing quota entirely

3. mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics
- Adds the two complementary metrics for hot memory distribution control

4. mm/damon: add PA-mode cache for eligible memory detection lag
- Implements rolling window cache to compensate for PA-mode detection lag
- Adds configurable cache timeout via sysfs

Testing Status
==============

Functionally tested on a two-node heterogeneous memory system (DRAM + CXL) with PUSH+PULL scheme configuration.

This is an RFC and feedback on the design is appreciated.

Ravi Jonnalagadda (4):
mm/damon/sysfs: set goal_tuner after scheme creation mm/damon: fix esz=0 quota bypass allowing unlimited migration mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal
metrics
mm/damon: add PA-mode cache for eligible memory detection lag

 include/linux/damon.h    |  51 ++++
 mm/damon/core.c          | 496 +++++++++++++++++++++++++++++++++++++--
 mm/damon/sysfs-schemes.c |  43 ++++
 3 files changed, 576 insertions(+), 14 deletions(-)

Signature · 1 line

2.43.0

Ravi Jonnalagadda

author

2026-02-23 12:32 UTC #1 lore raw

RFC PATCH v3 1/4 mm/damon/sysfs: set goal_tuner after scheme creation

damon_new_scheme() always sets quota.goal_tuner to CONSIST (the default) regardless of what was passed in the quota struct. This caused the sysfs goal_tuner setting to be ignored.

The comment in damon_new_scheme() says "quota.goals and .goal_tuner should be separately set by caller", but the sysfs code wasn't doing this. Add explicit assignment of goal_tuner after damon_new_scheme() returns to properly apply the user's setting.

Without this fix, setting goal_tuner to "temporal" via sysfs has no effect - the scheme always uses the CONSIST (feed loop) tuner, causing overshoot when the goal is reached instead of immediate stop.

Signed-off-by Ravi Jonnalagadda <[email protected]>

---
 mm/damon/sysfs-schemes.c | 3 +++
 1 file changed, 3 insertions(+)

mm/damon/sysfs-schemes.c +3 -0

index bbea908074bb..fe2e3b2db9e1 100644
@@ -2809,6 +2809,9 @@ static struct damos *damon_sysfs_mk_scheme(
 	if (!scheme)
 		return NULL;
 
+	/* Set goal_tuner after damon_new_scheme() as it defaults to CONSIST */
+	scheme->quota.goal_tuner = sysfs_quotas->goal_tuner;
+
 	err = damos_sysfs_add_quota_score(sysfs_quotas->goals, &scheme->quota);
 	if (err) {
 		damon_destroy_scheme(scheme);

Signature · 1 line

2.43.0

SeongJae Park

maintainer Reviewed-by on v3 1/4

2026-02-24 01:40 UTC #2 lore raw

Re: [RFC PATCH v3 1/4] mm/damon/sysfs: set goal_tuner after scheme creation

Quoted 4 lines · On Mon, 23 Feb 2026 12:32:29 +0000 Ravi Jonnalagadda <[email protected]>…

damon_new_scheme() always sets quota.goal_tuner to CONSIST (the default) regardless of what was passed in the quota struct. This caused the sysfs goal_tuner setting to be ignored.

Quoted 8 lines

The comment in damon_new_scheme() says "quota.goals and .goal_tuner should be separately set by caller", but the sysfs code wasn't doing this. Add explicit assignment of goal_tuner after damon_new_scheme() returns to properly apply the user's setting.
Without this fix, setting goal_tuner to "temporal" via sysfs has no effect - the scheme always uses the CONSIST (feed loop) tuner, causing overshoot when the goal is reached instead of immediate stop.

Thank you for catching this, Ravi! So, this is a fix for the RFC patch series
[1] that not yet merged, right? I think this fix is better to be carried with the series, and squashed into the broken commit to not introduce unnecessary regression.

So, if you don't mind, I will squash this into the sysfs-schemes part change on my tree, with your Co-developed-by: tag.

[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]

Quoted 2 lines

Signed-off-by Ravi Jonnalagadda <[email protected]>

Reviewed-by SeongJae Park <[email protected]>

Quoted 9 lines

---
 mm/damon/sysfs-schemes.c | 3 +++
 1 file changed, 3 insertions(+)

mm/damon/sysfs-schemes.c +0 -0

index bbea908074bb..fe2e3b2db9e1 100644

mm/damon/sysfs-schemes.c static struct damos *damon_sysfs_mk_scheme( · line 2817

Quoted 9 lines

mm/damon/sysfs-schemes.c +3 -0

@@ -2809,6 +2809,9 @@ static struct damos *damon_sysfs_mk_scheme(
 	if (!scheme)
 		return NULL;
 
+	/* Set goal_tuner after damon_new_scheme() as it defaults to CONSIST */
+	scheme->quota.goal_tuner = sysfs_quotas->goal_tuner;
+
 	err = damos_sysfs_add_quota_score(sysfs_quotas->goals, &scheme->quota);
 	if (err) {
 		damon_destroy_scheme(scheme);

To follow the order on the comment ("quota.goals and .goal_tuner should be separately set by caller"), I'd prefer setting the goal_tuner after damos_sysfs_add_quota_Score() call here, if you don't mind. Let me know if you prefer keeping the current order. If not, I will just make the change when I apply this to damon/next.

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-25 18:23 UTC #3 lore raw

Re: [RFC PATCH v3 1/4] mm/damon/sysfs: set goal_tuner after scheme creation

Quoted 16 lines · SeongJae Park

Quoted 4 lines · On Mon, 23 Feb 2026 12:32:29 +0000 Ravi Jonnalagadda <[email protected]>…
damon_new_scheme() always sets quota.goal_tuner to CONSIST (the default) regardless of what was passed in the quota struct. This caused the sysfs goal_tuner setting to be ignored.
Quoted 8 lines
The comment in damon_new_scheme() says "quota.goals and .goal_tuner should be separately set by caller", but the sysfs code wasn't doing this. Add explicit assignment of goal_tuner after damon_new_scheme() returns to properly apply the user's setting.
Without this fix, setting goal_tuner to "temporal" via sysfs has no effect - the scheme always uses the CONSIST (feed loop) tuner, causing overshoot when the goal is reached instead of immediate stop.

Quoted 7 lines

Thank you for catching this, Ravi! So, this is a fix for the RFC patch series
[1] that not yet merged, right? I think this fix is better to be carried with the series, and squashed into the broken commit to not introduce unnecessary regression.
So, if you don't mind, I will squash this into the sysfs-schemes part change on my tree, with your Co-developed-by: tag.

Yes. Please go ahead and squash it into your series.

Quoted 21 lines

[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]
Quoted 2 lines
Signed-off-by Ravi Jonnalagadda <[email protected]>
Reviewed-by SeongJae Park <[email protected]>
Quoted 12 lines
---
 mm/damon/sysfs-schemes.c | 3 +++
 1 file changed, 3 insertions(+)
mm/damon/sysfs-schemes.c +0 -0
index bbea908074bb..fe2e3b2db9e1 100644
@@ -2809,6 +2809,9 @@ static struct damos *damon_sysfs_mk_scheme(
      if (!scheme)
              return NULL;
 

Quoted 13 lines

mm/damon/sysfs-schemes.c static struct damos *damon_sysfs_mk_scheme( · line 2817
Quoted 6 lines
mm/damon/sysfs-schemes.c +3 -0
+     /* Set goal_tuner after damon_new_scheme() as it defaults to CONSIST */
+     scheme->quota.goal_tuner = sysfs_quotas->goal_tuner;
+
      err = damos_sysfs_add_quota_score(sysfs_quotas->goals, &scheme->quota);
      if (err) {
              damon_destroy_scheme(scheme);
To follow the order on the comment ("quota.goals and .goal_tuner should be separately set by caller"), I'd prefer setting the goal_tuner after damos_sysfs_add_quota_Score() call here, if you don't mind. Let me know if you prefer keeping the current order. If not, I will just make the change when I apply this to damon/next.

Agreed. Please make that change when you apply it.

Quoted 5 lines

Thanks,
SJ
[...]

SeongJae Park

maintainer

2026-02-26 00:53 UTC #4 lore raw

Re: [RFC PATCH v3 1/4] mm/damon/sysfs: set goal_tuner after scheme creation

Quoted 62 lines · On Wed, 25 Feb 2026 10:23:09 -0800 Ravi Jonnalagadda <[email protected]>…

Quoted 16 lines · SeongJae Park
Quoted 4 lines · On Mon, 23 Feb 2026 12:32:29 +0000 Ravi Jonnalagadda <[email protected]>…
damon_new_scheme() always sets quota.goal_tuner to CONSIST (the default) regardless of what was passed in the quota struct. This caused the sysfs goal_tuner setting to be ignored.
Quoted 8 lines
The comment in damon_new_scheme() says "quota.goals and .goal_tuner should be separately set by caller", but the sysfs code wasn't doing this. Add explicit assignment of goal_tuner after damon_new_scheme() returns to properly apply the user's setting.
Without this fix, setting goal_tuner to "temporal" via sysfs has no effect - the scheme always uses the CONSIST (feed loop) tuner, causing overshoot when the goal is reached instead of immediate stop.
Quoted 7 lines
Thank you for catching this, Ravi! So, this is a fix for the RFC patch series
[1] that not yet merged, right? I think this fix is better to be carried with the series, and squashed into the broken commit to not introduce unnecessary regression.
So, if you don't mind, I will squash this into the sysfs-schemes part change on my tree, with your Co-developed-by: tag.
Yes. Please go ahead and squash it into your series.
Quoted 21 lines
[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]
Quoted 2 lines
Signed-off-by Ravi Jonnalagadda <[email protected]>
Reviewed-by SeongJae Park <[email protected]>
Quoted 12 lines
---
 mm/damon/sysfs-schemes.c | 3 +++
 1 file changed, 3 insertions(+)
mm/damon/sysfs-schemes.c +0 -0
index bbea908074bb..fe2e3b2db9e1 100644
@@ -2809,6 +2809,9 @@ static struct damos *damon_sysfs_mk_scheme(
      if (!scheme)
              return NULL;
 
Quoted 13 lines
mm/damon/sysfs-schemes.c static struct damos *damon_sysfs_mk_scheme( · line 2817
Quoted 6 lines
mm/damon/sysfs-schemes.c +3 -0
+     /* Set goal_tuner after damon_new_scheme() as it defaults to CONSIST */
+     scheme->quota.goal_tuner = sysfs_quotas->goal_tuner;
+
      err = damos_sysfs_add_quota_score(sysfs_quotas->goals, &scheme->quota);
      if (err) {
              damon_destroy_scheme(scheme);
To follow the order on the comment ("quota.goals and .goal_tuner should be separately set by caller"), I'd prefer setting the goal_tuner after damos_sysfs_add_quota_Score() call here, if you don't mind. Let me know if you prefer keeping the current order. If not, I will just make the change when I apply this to damon/next.

Quoted 1 line

Agreed. Please make that change when you apply it.

Thank you, I will do!

Thanks,
SJ

[...]

SeongJae Park

maintainer

2026-02-27 02:04 UTC #5 lore raw

Re: [RFC PATCH v3 1/4] mm/damon/sysfs: set goal_tuner after scheme creation

Quoted 64 lines · On Wed, 25 Feb 2026 16:53:48 -0800 SeongJae Park <[email protected]> wrote:

Quoted 62 lines · On Wed, 25 Feb 2026 10:23:09 -0800 Ravi Jonnalagadda <[email protected]>…
Quoted 16 lines · SeongJae Park
Quoted 4 lines · On Mon, 23 Feb 2026 12:32:29 +0000 Ravi Jonnalagadda <[email protected]>…
damon_new_scheme() always sets quota.goal_tuner to CONSIST (the default) regardless of what was passed in the quota struct. This caused the sysfs goal_tuner setting to be ignored.
Quoted 8 lines
The comment in damon_new_scheme() says "quota.goals and .goal_tuner should be separately set by caller", but the sysfs code wasn't doing this. Add explicit assignment of goal_tuner after damon_new_scheme() returns to properly apply the user's setting.
Without this fix, setting goal_tuner to "temporal" via sysfs has no effect - the scheme always uses the CONSIST (feed loop) tuner, causing overshoot when the goal is reached instead of immediate stop.
Quoted 7 lines
Thank you for catching this, Ravi! So, this is a fix for the RFC patch series
[1] that not yet merged, right? I think this fix is better to be carried with the series, and squashed into the broken commit to not introduce unnecessary regression.
So, if you don't mind, I will squash this into the sysfs-schemes part change on my tree, with your Co-developed-by: tag.
Yes. Please go ahead and squash it into your series.
Quoted 34 lines
[1] https://lore.kernel.org/[email protected]
[2] https://lore.kernel.org/[email protected]
Quoted 2 lines
Signed-off-by Ravi Jonnalagadda <[email protected]>
Reviewed-by SeongJae Park <[email protected]>
Quoted 9 lines
---
 mm/damon/sysfs-schemes.c | 3 +++
 1 file changed, 3 insertions(+)
mm/damon/sysfs-schemes.c +0 -0
index bbea908074bb..fe2e3b2db9e1 100644
mm/damon/sysfs-schemes.c static struct damos *damon_sysfs_mk_scheme( · line 2817
Quoted 9 lines
mm/damon/sysfs-schemes.c +3 -0
@@ -2809,6 +2809,9 @@ static struct damos *damon_sysfs_mk_scheme(
      if (!scheme)
              return NULL;
 
+     /* Set goal_tuner after damon_new_scheme() as it defaults to CONSIST */
+     scheme->quota.goal_tuner = sysfs_quotas->goal_tuner;
+
      err = damos_sysfs_add_quota_score(sysfs_quotas->goals, &scheme->quota);
      if (err) {
              damon_destroy_scheme(scheme);
To follow the order on the comment ("quota.goals and .goal_tuner should be separately set by caller"), I'd prefer setting the goal_tuner after damos_sysfs_add_quota_Score() call here, if you don't mind. Let me know if you prefer keeping the current order. If not, I will just make the change when I apply this to damon/next.

Quoted 3 lines

Quoted 1 line
Agreed. Please make that change when you apply it.
Thank you, I will do!

I now think it is better to just remove the damon_new_scheme() internal initialization. I added the detail [1] to the original thread. I'll keep your
Co-developed-by: tag with the new fix though, unless you mind. Let's keep discussing there if you have more opinions.

[1] https://lore.kernel.org/[email protected]

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-23 12:32 UTC #6 lore raw

RFC PATCH v3 2/4 mm/damon: fix esz=0 quota bypass allowing unlimited migration

In reply to Ravi Jonnalagadda

When the TEMPORAL goal tuner sets esz_bp=0 to signal that a goal has been achieved, the quota check was not actually stopping migration.

The condition:

    if (quota->esz && quota->charged_sz >= quota->esz)

When esz=0, this evaluates to (false && ...) = false, so the continue is never executed and migration proceeds without limit.

Change the logic to:

    if (!quota->esz || quota->charged_sz >= quota->esz)

Now when esz=0, (!0 = true) causes the continue to execute, properly stopping migration when the goal is achieved.

This is critical for the TEMPORAL tuner to work correctly - without this fix, setting esz=0 has no effect and migration continues until all hot memory is moved, overshooting the target goal.

Signed-off-by Ravi Jonnalagadda <[email protected]>

---
 mm/damon/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

mm/damon/core.c +2 -2

index 614f1f08eee9..b438355ab54a 100644
@@ -2394,8 +2394,8 @@ static void damon_do_apply_schemes(struct damon_ctx *c,
 		if (!s->wmarks.activated)
 			continue;
 
-		/* Check the quota */
-		if (quota->esz && quota->charged_sz >= quota->esz)
+		/* Check the quota: skip if esz=0 (goal achieved) or exhausted */
+		if (!quota->esz || quota->charged_sz >= quota->esz)
 			continue;
 
 		if (damos_skip_charged_region(t, r, s, c->min_region_sz))

Signature · 1 line

2.43.0

SeongJae Park

maintainer

2026-02-24 01:54 UTC #7 lore raw

Re: [RFC PATCH v3 2/4] mm/damon: fix esz=0 quota bypass allowing unlimited migration

Quoted 8 lines · On Mon, 23 Feb 2026 12:32:30 +0000 Ravi Jonnalagadda <[email protected]>…

When the TEMPORAL goal tuner sets esz_bp=0 to signal that a goal has been achieved, the quota check was not actually stopping migration.
The condition:
    if (quota->esz && quota->charged_sz >= quota->esz)
When esz=0, this evaluates to (false && ...) = false, so the continue is never executed and migration proceeds without limit.

Nice finding, thank you for sharing this!

Quoted 6 lines

Change the logic to:
    if (!quota->esz || quota->charged_sz >= quota->esz)
Now when esz=0, (!0 = true) causes the continue to execute, properly stopping migration when the goal is achieved.

But this code is written in the way because the current code assumes zero 'esz' means it is not being used and therefore be ignored. This change should be ok for your use case, but could introduce an unexpected behavioral change for other users.

One easy workaround would be setting esz_bp with a value smaller than 40960000 instead of 0. That is, esz_bp is the bytes in bp, so setting it smaller than 40960000 will result in making it effectively zero, e.g., like below.

'''

mm/damon/core.c +1 -1

@@ -2623,7 +2623,7 @@ static void damos_goal_tune_esz_bp_temporal(struct damos_quota *quota)
        unsigned long score = damos_quota_score(quota);
 
        if (score >= 10000)
-               quota->esz_bp = 0;
+               quota->esz_bp = 10000;
        else if (quota->sz)
                quota->esz_bp = quota->sz * 10000;
        else

'''

But maybe there is a better way to cleanly fix this. Let me take a time to think more...

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-25 18:28 UTC #8 lore raw

Re: [RFC PATCH v3 2/4] mm/damon: fix esz=0 quota bypass allowing unlimited migration

Quoted 6 lines · SeongJae Park

Quoted 3 lines · On Mon, 23 Feb 2026 12:32:30 +0000 Ravi Jonnalagadda <[email protected]>…
When the TEMPORAL goal tuner sets esz_bp=0 to signal that a goal has been achieved, the quota check was not actually stopping migration.

Quoted 7 lines

Quoted 5 lines
The condition:
    if (quota->esz && quota->charged_sz >= quota->esz)
When esz=0, this evaluates to (false && ...) = false, so the continue is never executed and migration proceeds without limit.
Nice finding, thank you for sharing this!

Thanks!

Quoted 31 lines

Quoted 7 lines
Change the logic to:
    if (!quota->esz || quota->charged_sz >= quota->esz)
Now when esz=0, (!0 = true) causes the continue to execute, properly stopping migration when the goal is achieved.
But this code is written in the way because the current code assumes zero 'esz' means it is not being used and therefore be ignored. This change should be ok for your use case, but could introduce an unexpected behavioral change for other users.
One easy workaround would be setting esz_bp with a value smaller than 40960000 instead of 0. That is, esz_bp is the bytes in bp, so setting it smaller than 40960000 will result in making it effectively zero, e.g., like below.
'''
mm/damon/core.c +1 -1
@@ -2623,7 +2623,7 @@ static void damos_goal_tune_esz_bp_temporal(struct damos_quota *quota)
        unsigned long score = damos_quota_score(quota);
 
        if (score >= 10000)
-               quota->esz_bp = 0;
+               quota->esz_bp = 10000;
        else if (quota->sz)
                quota->esz_bp = quota->sz * 10000;
        else
'''

Quoted 3 lines

But maybe there is a better way to cleanly fix this. Let me take a time to think more...

I understand. Please take your time. I'm happy to test whatever approach you come up with.

Quoted 5 lines

Thanks,
SJ
[...]

SeongJae Park

maintainer

2026-02-26 00:54 UTC #9 lore raw

Re: [RFC PATCH v3 2/4] mm/damon: fix esz=0 quota bypass allowing unlimited migration

Quoted 48 lines · On Wed, 25 Feb 2026 10:28:30 -0800 Ravi Jonnalagadda <[email protected]>…

Quoted 6 lines · SeongJae Park
Quoted 3 lines · On Mon, 23 Feb 2026 12:32:30 +0000 Ravi Jonnalagadda <[email protected]>…
When the TEMPORAL goal tuner sets esz_bp=0 to signal that a goal has been achieved, the quota check was not actually stopping migration.
Quoted 7 lines
Quoted 5 lines
The condition:
    if (quota->esz && quota->charged_sz >= quota->esz)
When esz=0, this evaluates to (false && ...) = false, so the continue is never executed and migration proceeds without limit.
Nice finding, thank you for sharing this!
Thanks!
Quoted 31 lines
Quoted 7 lines
Change the logic to:
    if (!quota->esz || quota->charged_sz >= quota->esz)
Now when esz=0, (!0 = true) causes the continue to execute, properly stopping migration when the goal is achieved.
But this code is written in the way because the current code assumes zero 'esz' means it is not being used and therefore be ignored. This change should be ok for your use case, but could introduce an unexpected behavioral change for other users.
One easy workaround would be setting esz_bp with a value smaller than 40960000 instead of 0. That is, esz_bp is the bytes in bp, so setting it smaller than 40960000 will result in making it effectively zero, e.g., like below.
'''
mm/damon/core.c +1 -1
@@ -2623,7 +2623,7 @@ static void damos_goal_tune_esz_bp_temporal(struct damos_quota *quota)
        unsigned long score = damos_quota_score(quota);
 
        if (score >= 10000)
-               quota->esz_bp = 0;
+               quota->esz_bp = 10000;
        else if (quota->sz)
                quota->esz_bp = quota->sz * 10000;
        else
'''

Quoted 6 lines

Quoted 3 lines
But maybe there is a better way to cleanly fix this. Let me take a time to think more...
I understand. Please take your time. I'm happy to test whatever approach you come up with.

Thank you for being flexible, I will do!

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-23 12:33 UTC #10 lore raw

RFC PATCH v3 3/4 mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics

In reply to Ravi Jonnalagadda

Add new quota goal metrics for memory tiering that track scheme-eligible (hot) memory distribution across NUMA nodes:

- DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: ratio of hot memory on a node
- DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: ratio of hot memory NOT on a node

These complementary metrics enable push-pull migration schemes that maintain a target hot memory distribution. For example, to keep 30% of hot memory on CXL node 1:

- PUSH scheme (DRAM→CXL): node_eligible_mem_bp, nid=1, target=3000 Activates when node 1 has less than 30% hot memory
- PULL scheme (CXL→DRAM): node_ineligible_mem_bp, nid=1, target=7000 Activates when node 1 has more than 30% hot memory

Together with the TEMPORAL goal tuner, the schemes converge to equilibrium at the target distribution.

The metrics use detected eligible bytes per node, calculated by summing the size of regions that match the scheme's access pattern (size, nr_accesses, age) on each NUMA node.

Signed-off-by Ravi Jonnalagadda <[email protected]>

---
 include/linux/damon.h    |   6 ++
 mm/damon/core.c          | 123 ++++++++++++++++++++++++++++++++++++++-
 mm/damon/sysfs-schemes.c |  10 ++++
 3 files changed, 137 insertions(+), 2 deletions(-)

include/linux/damon.h +6 -0

index ee2d0879c292..6df716533fbf 100644
@@ -191,6 +191,8 @@ enum damos_action {
  * @DAMOS_QUOTA_NODE_MEM_FREE_BP:	MemFree ratio of a node.
  * @DAMOS_QUOTA_NODE_MEMCG_USED_BP:	MemUsed ratio of a node for a cgroup.
  * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP:	MemFree ratio of a node for a cgroup.
+ * @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:	Scheme-eligible memory ratio of a node.
+ * @DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:	Scheme-ineligible memory ratio of a node.
  * @DAMOS_QUOTA_ACTIVE_MEM_BP:		Active to total LRU memory ratio.
  * @DAMOS_QUOTA_INACTIVE_MEM_BP:	Inactive to total LRU memory ratio.
  * @NR_DAMOS_QUOTA_GOAL_METRICS:	Number of DAMOS quota goal metrics.

@@ -204,6 +206,8 @@ enum damos_quota_goal_metric {
 	DAMOS_QUOTA_NODE_MEM_FREE_BP,
 	DAMOS_QUOTA_NODE_MEMCG_USED_BP,
 	DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
+	DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+	DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP,
 	DAMOS_QUOTA_ACTIVE_MEM_BP,
 	DAMOS_QUOTA_INACTIVE_MEM_BP,
 	NR_DAMOS_QUOTA_GOAL_METRICS,

@@ -555,6 +559,7 @@ struct damos_migrate_dests {
  * @ops_filters:	ops layer handling &struct damos_filter objects list.
  * @last_applied:	Last @action applied ops-managing entity.
  * @stat:		Statistics of this scheme.
+ * @eligible_bytes_per_node: Scheme-eligible bytes per NUMA node.
  * @max_nr_snapshots:	Upper limit of nr_snapshots stat.
  * @list:		List head for siblings.
  *

@@ -644,6 +649,7 @@ struct damos {
 	struct list_head ops_filters;
 	void *last_applied;
 	struct damos_stat stat;
+	unsigned long eligible_bytes_per_node[MAX_NUMNODES];
 	unsigned long max_nr_snapshots;
 	struct list_head list;
 };

mm/damon/core.c +121 -2

157 lines not on this page · raw message

mm/damon/sysfs-schemes.c +10 -0

index fe2e3b2db9e1..232b33f5cbfb 100644
@@ -1079,6 +1079,14 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
 		.metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
 		.name = "node_memcg_free_bp",
 	},
+	{
+		.metric = DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+		.name = "node_eligible_mem_bp",
+	},
+	{
+		.metric = DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP,
+		.name = "node_ineligible_mem_bp",
+	},
 	{
 		.metric = DAMOS_QUOTA_ACTIVE_MEM_BP,
 		.name = "active_mem_bp",

@@ -2669,6 +2677,8 @@ static int damos_sysfs_add_quota_score(
 			break;
 		case DAMOS_QUOTA_NODE_MEM_USED_BP:
 		case DAMOS_QUOTA_NODE_MEM_FREE_BP:
+		case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:
+		case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:
 			goal->nid = sysfs_goal->nid;
 			break;
 		case DAMOS_QUOTA_NODE_MEMCG_USED_BP:

Signature · 1 line

2.43.0

SeongJae Park

maintainer

2026-02-24 04:27 UTC #11 lore raw

Re: [RFC PATCH v3 3/4] mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics

Quoted 15 lines · On Mon, 23 Feb 2026 12:32:31 +0000 Ravi Jonnalagadda <[email protected]>…

Add new quota goal metrics for memory tiering that track scheme-eligible (hot) memory distribution across NUMA nodes:
- DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: ratio of hot memory on a node
- DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: ratio of hot memory NOT on a node
These complementary metrics enable push-pull migration schemes that maintain a target hot memory distribution. For example, to keep 30% of hot memory on CXL node 1:
- PUSH scheme (DRAM→CXL): node_eligible_mem_bp, nid=1, target=3000 Activates when node 1 has less than 30% hot memory
- PULL scheme (CXL→DRAM): node_ineligible_mem_bp, nid=1, target=7000 Activates when node 1 has more than 30% hot memory

Quoted 6 lines

Together with the TEMPORAL goal tuner, the schemes converge to equilibrium at the target distribution.
The metrics use detected eligible bytes per node, calculated by summing the size of regions that match the scheme's access pattern (size, nr_accesses, age) on each NUMA node.

Looks good in general! I have some comments about trivials and the design below, though.

Quoted 14 lines

Suggested-by SeongJae Park <[email protected]>
Signed-off-by Ravi Jonnalagadda <[email protected]>
---
 include/linux/damon.h    |   6 ++
 mm/damon/core.c          | 123 ++++++++++++++++++++++++++++++++++++++-
 mm/damon/sysfs-schemes.c |  10 ++++
 3 files changed, 137 insertions(+), 2 deletions(-)
include/linux/damon.h +0 -0
index ee2d0879c292..6df716533fbf 100644

include/linux/damon.h enum damos_action { · line 195

Quoted 5 lines

include/linux/damon.h +2 -0

@@ -191,6 +191,8 @@ enum damos_action {
  * @DAMOS_QUOTA_NODE_MEM_FREE_BP:	MemFree ratio of a node.
  * @DAMOS_QUOTA_NODE_MEMCG_USED_BP:	MemUsed ratio of a node for a cgroup.
  * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP:	MemFree ratio of a node for a cgroup.
+ * @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:	Scheme-eligible memory ratio of a node.
+ * @DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:	Scheme-ineligible memory ratio of a node.

Nit. Let's wrap the line for 80 columns limit.

Quoted 21 lines

include/linux/damon.h +3 -0

  * @DAMOS_QUOTA_ACTIVE_MEM_BP:		Active to total LRU memory ratio.
  * @DAMOS_QUOTA_INACTIVE_MEM_BP:	Inactive to total LRU memory ratio.
  * @NR_DAMOS_QUOTA_GOAL_METRICS:	Number of DAMOS quota goal metrics.

@@ -204,6 +206,8 @@ enum damos_quota_goal_metric {
 	DAMOS_QUOTA_NODE_MEM_FREE_BP,
 	DAMOS_QUOTA_NODE_MEMCG_USED_BP,
 	DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
+	DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+	DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP,
 	DAMOS_QUOTA_ACTIVE_MEM_BP,
 	DAMOS_QUOTA_INACTIVE_MEM_BP,
 	NR_DAMOS_QUOTA_GOAL_METRICS,

@@ -555,6 +559,7 @@ struct damos_migrate_dests {
  * @ops_filters:	ops layer handling &struct damos_filter objects list.
  * @last_applied:	Last @action applied ops-managing entity.
  * @stat:		Statistics of this scheme.
+ * @eligible_bytes_per_node: Scheme-eligible bytes per NUMA node.
  * @max_nr_snapshots:	Upper limit of nr_snapshots stat.
  * @list:		List head for siblings.
  *

include/linux/damon.h struct damos { · line 652

Quoted 4 lines

include/linux/damon.h +1 -0

@@ -644,6 +649,7 @@ struct damos {
 	struct list_head ops_filters;
 	void *last_applied;
 	struct damos_stat stat;
+	unsigned long eligible_bytes_per_node[MAX_NUMNODES];

I understand this could make it time-efficient. That is, without this, you will need to iterate the regions for number of node_[in]eligible_mem_bp goals per scheme. By having this you need to iterate regions only once per scheme. I'm bit worried about the increased size of 'struct damos', though.

Do you think the overhead is really significant? If not, what about making it simply iterates the regions per goal, and add optimization later if it turns out really needs?

If this optimization is really needed right now, I'd like it to at least be dynamically allocated, for only num_online_nodes() or num_possible_nodes() at least.

Quoted 53 lines

include/linux/damon.h +0 -0

 	unsigned long max_nr_snapshots;
 	struct list_head list;
 };

mm/damon/core.c +42 -0

index b438355ab54a..3e1cb850f067 100644
@@ -2544,6 +2544,111 @@ static unsigned long damos_get_node_memcg_used_bp(
 }
 #endif
 
+#ifdef CONFIG_NUMA
+/*
+ * damos_scheme_uses_eligible_metrics() - Check if scheme uses eligible metrics.
+ * @s: The scheme
+ *
+ * Returns true if any quota goal uses node_eligible_mem_bp or
+ * node_ineligible_mem_bp metrics, which require eligible bytes calculation.
+ */
+static bool damos_scheme_uses_eligible_metrics(struct damos *s)
+{
+	struct damos_quota_goal *goal;
+	struct damos_quota *quota = &s->quota;
+
+	damos_for_each_quota_goal(goal, quota) {
+		if (goal->metric == DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP ||
+		    goal->metric == DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP)
+			return true;
+	}
+	return false;
+}
+
+/*
+ * damos_calc_eligible_bytes_per_node() - Calculate eligible bytes per node.
+ * @c: The DAMON context
+ * @s: The scheme
+ *
+ * Calculates scheme-eligible bytes per NUMA node based on access pattern
+ * matching. A region is eligible if it matches the scheme's access pattern
+ * (size, nr_accesses, age).
+ */
+static void damos_calc_eligible_bytes_per_node(struct damon_ctx *c,
+		struct damos *s)
+{
+	struct damon_target *t;
+	struct damon_region *r;
+	phys_addr_t paddr;
+	int nid;
+
+	memset(s->eligible_bytes_per_node, 0,
+			sizeof(s->eligible_bytes_per_node));
+
+	damon_for_each_target(t, c) {

mm/damon/core.c static unsigned long damos_get_node_memcg_used_bp( · line 2598

Quoted 10 lines

mm/damon/core.c +10 -0

+		damon_for_each_region(r, t) {
+			if (!__damos_valid_target(r, s))
+				continue;
+			paddr = (phys_addr_t)r->ar.start * c->addr_unit;
+			nid = pfn_to_nid(PHYS_PFN(paddr));
+			if (nid >= 0 && nid < MAX_NUMNODES)
+				s->eligible_bytes_per_node[nid] +=
+					damon_sz_region(r) * c->addr_unit;
+		}
+	}

Seems the above code assumes entire region will belong in the same node. But the region might be laying over a nodes boundary. In the case, miscalculations could happen.

What about getting start/end addresses of the node, and checking the crossing boundary case?

Quoted 82 lines

mm/damon/core.c +63 -1

+}
+
+static unsigned long damos_get_node_eligible_mem_bp(struct damos *s, int nid)
+{
+	unsigned long total_eligible = 0;
+	unsigned long node_eligible;
+	int n;
+
+	if (nid < 0 || nid >= MAX_NUMNODES)
+		return 0;
+
+	for_each_online_node(n)
+		total_eligible += s->eligible_bytes_per_node[n];
+
+	if (!total_eligible)
+		return 0;
+
+	node_eligible = s->eligible_bytes_per_node[nid];
+
+	return mult_frac(node_eligible, 10000, total_eligible);
+}
+
+static unsigned long damos_get_node_ineligible_mem_bp(struct damos *s, int nid)
+{
+	unsigned long eligible_bp = damos_get_node_eligible_mem_bp(s, nid);
+
+	if (eligible_bp == 0)
+		return 10000;
+
+	return 10000 - eligible_bp;
+}
+#else
+static bool damos_scheme_uses_eligible_metrics(struct damos *s)
+{
+	return false;
+}
+
+static void damos_calc_eligible_bytes_per_node(struct damon_ctx *c,
+		struct damos *s)
+{
+}
+
+static unsigned long damos_get_node_eligible_mem_bp(struct damos *s, int nid)
+{
+	return 0;
+}
+
+static unsigned long damos_get_node_ineligible_mem_bp(struct damos *s, int nid)
+{
+	return 0;
+}
+#endif
+
 /*
  * Returns LRU-active or inactive memory to total LRU memory size ratio.
  */

@@ -2562,7 +2667,8 @@ static unsigned int damos_get_in_active_mem_bp(bool active_ratio)
 	return mult_frac(inactive, 10000, total);
 }
 
-static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
+static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal,
+		struct damos *s)
 {
 	u64 now_psi_total;

@@ -2584,6 +2690,14 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
 	case DAMOS_QUOTA_NODE_MEMCG_FREE_BP:
 		goal->current_value = damos_get_node_memcg_used_bp(goal);
 		break;
+	case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:
+		goal->current_value = damos_get_node_eligible_mem_bp(s,
+				goal->nid);
+		break;
+	case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:
+		goal->current_value = damos_get_node_ineligible_mem_bp(s,
+				goal->nid);
+		break;
 	case DAMOS_QUOTA_ACTIVE_MEM_BP:
 	case DAMOS_QUOTA_INACTIVE_MEM_BP:
 		goal->current_value = damos_get_in_active_mem_bp(

mm/damon/core.c static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal) · line 2714

Quoted 4 lines

mm/damon/core.c +1 -0

@@ -2597,11 +2711,12 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
 /* Return the highest score since it makes schemes least aggressive */
 static unsigned long damos_quota_score(struct damos_quota *quota)
 {
+	struct damos *s = container_of(quota, struct damos, quota);

I'd prefer passing 's' from the caller.

Quoted 50 lines

mm/damon/core.c +5 -1

 	struct damos_quota_goal *goal;
 	unsigned long highest_score = 0;
 
 	damos_for_each_quota_goal(goal, quota) {
-		damos_set_quota_goal_current_value(goal);
+		damos_set_quota_goal_current_value(goal, s);
 		highest_score = max(highest_score,
 				mult_frac(goal->current_value, 10000,
 					goal->target_value));

@@ -2693,6 +2808,10 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
 	if (!quota->ms && !quota->sz && list_empty(&quota->goals))
 		return;
 
+	/* Calculate eligible bytes per node for quota goal metrics */
+	if (damos_scheme_uses_eligible_metrics(s))
+		damos_calc_eligible_bytes_per_node(c, s);
+
 	/* First charge window */
 	if (!quota->total_charged_sz && !quota->charged_from) {
 		quota->charged_from = jiffies;

mm/damon/sysfs-schemes.c +10 -0

index fe2e3b2db9e1..232b33f5cbfb 100644
@@ -1079,6 +1079,14 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
 		.metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
 		.name = "node_memcg_free_bp",
 	},
+	{
+		.metric = DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+		.name = "node_eligible_mem_bp",
+	},
+	{
+		.metric = DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP,
+		.name = "node_ineligible_mem_bp",
+	},
 	{
 		.metric = DAMOS_QUOTA_ACTIVE_MEM_BP,
 		.name = "active_mem_bp",

@@ -2669,6 +2677,8 @@ static int damos_sysfs_add_quota_score(
 			break;
 		case DAMOS_QUOTA_NODE_MEM_USED_BP:
 		case DAMOS_QUOTA_NODE_MEM_FREE_BP:
+		case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:
+		case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:
 			goal->nid = sysfs_goal->nid;
 			break;
 		case DAMOS_QUOTA_NODE_MEMCG_USED_BP:

Signature · 1 line

2.43.0

So, the overall concept and definition of the new goal metrics sound good to me. But I'd prefer having less optimized but simpler code, and nodes boundary crossing regions handling.

Thanks,
SJ

Ravi Jonnalagadda

author

2026-02-25 18:47 UTC #12 lore raw

Re: [RFC PATCH v3 3/4] mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics

Quoted 21 lines · SeongJae Park

Quoted 18 lines · On Mon, 23 Feb 2026 12:32:31 +0000 Ravi Jonnalagadda <[email protected]>…
Add new quota goal metrics for memory tiering that track scheme-eligible (hot) memory distribution across NUMA nodes:
- DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP: ratio of hot memory on a node
- DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP: ratio of hot memory NOT on a node
These complementary metrics enable push-pull migration schemes that maintain a target hot memory distribution. For example, to keep 30% of hot memory on CXL node 1:
- PUSH scheme (DRAM→CXL): node_eligible_mem_bp, nid=1, target=3000 Activates when node 1 has less than 30% hot memory
- PULL scheme (CXL→DRAM): node_ineligible_mem_bp, nid=1, target=7000 Activates when node 1 has more than 30% hot memory
Together with the TEMPORAL goal tuner, the schemes converge to equilibrium at the target distribution.

Quoted 7 lines

Quoted 3 lines
The metrics use detected eligible bytes per node, calculated by summing the size of regions that match the scheme's access pattern (size, nr_accesses, age) on each NUMA node.
Looks good in general! I have some comments about trivials and the design below, though.

Thank you for the detailed review!

Quoted 9 lines

Quoted 9 lines
Suggested-by SeongJae Park <[email protected]>
Signed-off-by Ravi Jonnalagadda <[email protected]>
---
 include/linux/damon.h    |   6 ++
 mm/damon/core.c          | 123 ++++++++++++++++++++++++++++++++++++++-
 mm/damon/sysfs-schemes.c |  10 ++++
 3 files changed, 137 insertions(+), 2 deletions(-)

Quoted 12 lines

include/linux/damon.h enum damos_action { · line 195

Quoted 10 lines

include/linux/damon.h +2 -0

index ee2d0879c292..6df716533fbf 100644
@@ -191,6 +191,8 @@ enum damos_action {
  * @DAMOS_QUOTA_NODE_MEM_FREE_BP:    MemFree ratio of a node.
  * @DAMOS_QUOTA_NODE_MEMCG_USED_BP:  MemUsed ratio of a node for a cgroup.
  * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP:  MemFree ratio of a node for a cgroup.
+ * @DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:        Scheme-eligible memory ratio of a node.
+ * @DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:      Scheme-ineligible memory ratio of a node.

Nit. Let's wrap the line for 80 columns limit.

Will fix it.

Quoted 32 lines

Quoted 22 lines

include/linux/damon.h +3 -0

  * @DAMOS_QUOTA_ACTIVE_MEM_BP:               Active to total LRU memory ratio.
  * @DAMOS_QUOTA_INACTIVE_MEM_BP:     Inactive to total LRU memory ratio.
  * @NR_DAMOS_QUOTA_GOAL_METRICS:     Number of DAMOS quota goal metrics.

@@ -204,6 +206,8 @@ enum damos_quota_goal_metric {
      DAMOS_QUOTA_NODE_MEM_FREE_BP,
      DAMOS_QUOTA_NODE_MEMCG_USED_BP,
      DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
+     DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+     DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP,
      DAMOS_QUOTA_ACTIVE_MEM_BP,
      DAMOS_QUOTA_INACTIVE_MEM_BP,
      NR_DAMOS_QUOTA_GOAL_METRICS,

@@ -555,6 +559,7 @@ struct damos_migrate_dests {
  * @ops_filters:     ops layer handling &struct damos_filter objects list.
  * @last_applied:    Last @action applied ops-managing entity.
  * @stat:            Statistics of this scheme.
+ * @eligible_bytes_per_node: Scheme-eligible bytes per NUMA node.
  * @max_nr_snapshots:        Upper limit of nr_snapshots stat.
  * @list:            List head for siblings.
  *

include/linux/damon.h struct damos { · line 652

Quoted 4 lines

include/linux/damon.h +1 -0

@@ -644,6 +649,7 @@ struct damos {
      struct list_head ops_filters;
      void *last_applied;
      struct damos_stat stat;
+     unsigned long eligible_bytes_per_node[MAX_NUMNODES];

Quoted 4 lines

Do you think the overhead is really significant? If not, what about making it simply iterates the regions per goal, and add optimization later if it turns out really needs?

Got it. I'll change it to simply iterate regions per goal.

Quoted 3 lines

If this optimization is really needed right now, I'd like it to at least be dynamically allocated, for only num_online_nodes() or num_possible_nodes() at least.

Got it. If we ever need to implement the optimization, will follow the dynamic allocation path.

Quoted 69 lines

Quoted 54 lines

include/linux/damon.h +0 -0

      unsigned long max_nr_snapshots;
      struct list_head list;
 };

mm/damon/core.c +42 -0

index b438355ab54a..3e1cb850f067 100644
@@ -2544,6 +2544,111 @@ static unsigned long damos_get_node_memcg_used_bp(
 }
 #endif
 
+#ifdef CONFIG_NUMA
+/*
+ * damos_scheme_uses_eligible_metrics() - Check if scheme uses eligible metrics.
+ * @s: The scheme
+ *
+ * Returns true if any quota goal uses node_eligible_mem_bp or
+ * node_ineligible_mem_bp metrics, which require eligible bytes calculation.
+ */
+static bool damos_scheme_uses_eligible_metrics(struct damos *s)
+{
+     struct damos_quota_goal *goal;
+     struct damos_quota *quota = &s->quota;
+
+     damos_for_each_quota_goal(goal, quota) {
+             if (goal->metric == DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP ||
+                 goal->metric == DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP)
+                     return true;
+     }
+     return false;
+}
+
+/*
+ * damos_calc_eligible_bytes_per_node() - Calculate eligible bytes per node.
+ * @c: The DAMON context
+ * @s: The scheme
+ *
+ * Calculates scheme-eligible bytes per NUMA node based on access pattern
+ * matching. A region is eligible if it matches the scheme's access pattern
+ * (size, nr_accesses, age).
+ */
+static void damos_calc_eligible_bytes_per_node(struct damon_ctx *c,
+             struct damos *s)
+{
+     struct damon_target *t;
+     struct damon_region *r;
+     phys_addr_t paddr;
+     int nid;
+
+     memset(s->eligible_bytes_per_node, 0,
+                     sizeof(s->eligible_bytes_per_node));
+
+     damon_for_each_target(t, c) {

mm/damon/core.c static unsigned long damos_get_node_memcg_used_bp( · line 2598

Quoted 10 lines

mm/damon/core.c +10 -0

+             damon_for_each_region(r, t) {
+                     if (!__damos_valid_target(r, s))
+                             continue;
+                     paddr = (phys_addr_t)r->ar.start * c->addr_unit;
+                     nid = pfn_to_nid(PHYS_PFN(paddr));
+                     if (nid >= 0 && nid < MAX_NUMNODES)
+                             s->eligible_bytes_per_node[nid] +=
+                                     damon_sz_region(r) * c->addr_unit;
+             }
+     }

Seems the above code assumes entire region will belong in the same node. But the region might be laying over a nodes boundary. In the case, miscalculations could happen.

Quoted 2 lines

What about getting start/end addresses of the node, and checking the crossing boundary case?

Got it. I'll add handling for regions that span node boundaries by checking the node's address range and splitting the calculation accordingly.

Quoted 81 lines

mm/damon/core.c +63 -1

+}
+
+static unsigned long damos_get_node_eligible_mem_bp(struct damos *s, int nid)
+{
+     unsigned long total_eligible = 0;
+     unsigned long node_eligible;
+     int n;
+
+     if (nid < 0 || nid >= MAX_NUMNODES)
+             return 0;
+
+     for_each_online_node(n)
+             total_eligible += s->eligible_bytes_per_node[n];
+
+     if (!total_eligible)
+             return 0;
+
+     node_eligible = s->eligible_bytes_per_node[nid];
+
+     return mult_frac(node_eligible, 10000, total_eligible);
+}
+
+static unsigned long damos_get_node_ineligible_mem_bp(struct damos *s, int nid)
+{
+     unsigned long eligible_bp = damos_get_node_eligible_mem_bp(s, nid);
+
+     if (eligible_bp == 0)
+             return 10000;
+
+     return 10000 - eligible_bp;
+}
+#else
+static bool damos_scheme_uses_eligible_metrics(struct damos *s)
+{
+     return false;
+}
+
+static void damos_calc_eligible_bytes_per_node(struct damon_ctx *c,
+             struct damos *s)
+{
+}
+
+static unsigned long damos_get_node_eligible_mem_bp(struct damos *s, int nid)
+{
+     return 0;
+}
+
+static unsigned long damos_get_node_ineligible_mem_bp(struct damos *s, int nid)
+{
+     return 0;
+}
+#endif
+
 /*
  * Returns LRU-active or inactive memory to total LRU memory size ratio.
  */

@@ -2562,7 +2667,8 @@ static unsigned int damos_get_in_active_mem_bp(bool active_ratio)
      return mult_frac(inactive, 10000, total);
 }
 
-static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
+static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal,
+             struct damos *s)
 {
      u64 now_psi_total;

@@ -2584,6 +2690,14 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
      case DAMOS_QUOTA_NODE_MEMCG_FREE_BP:
              goal->current_value = damos_get_node_memcg_used_bp(goal);
              break;
+     case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:
+             goal->current_value = damos_get_node_eligible_mem_bp(s,
+                             goal->nid);
+             break;
+     case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:
+             goal->current_value = damos_get_node_ineligible_mem_bp(s,
+                             goal->nid);
+             break;
      case DAMOS_QUOTA_ACTIVE_MEM_BP:
      case DAMOS_QUOTA_INACTIVE_MEM_BP:

Quoted 8 lines

mm/damon/core.c static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal) · line 2714

Quoted 6 lines

mm/damon/core.c +1 -0

              goal->current_value = damos_get_in_active_mem_bp(

@@ -2597,11 +2711,12 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
 /* Return the highest score since it makes schemes least aggressive */
 static unsigned long damos_quota_score(struct damos_quota *quota)
 {
+     struct damos *s = container_of(quota, struct damos, quota);

I'd prefer passing 's' from the caller.

Will change to pass 's' from the caller instead of using container_of().

Quoted 47 lines

mm/damon/core.c +5 -1

      struct damos_quota_goal *goal;
      unsigned long highest_score = 0;
 
      damos_for_each_quota_goal(goal, quota) {
-             damos_set_quota_goal_current_value(goal);
+             damos_set_quota_goal_current_value(goal, s);
              highest_score = max(highest_score,
                              mult_frac(goal->current_value, 10000,
                                      goal->target_value));

@@ -2693,6 +2808,10 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s)
      if (!quota->ms && !quota->sz && list_empty(&quota->goals))
              return;
 
+     /* Calculate eligible bytes per node for quota goal metrics */
+     if (damos_scheme_uses_eligible_metrics(s))
+             damos_calc_eligible_bytes_per_node(c, s);
+
      /* First charge window */
      if (!quota->total_charged_sz && !quota->charged_from) {
              quota->charged_from = jiffies;

mm/damon/sysfs-schemes.c +10 -0

index fe2e3b2db9e1..232b33f5cbfb 100644
@@ -1079,6 +1079,14 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
              .metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
              .name = "node_memcg_free_bp",
      },
+     {
+             .metric = DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP,
+             .name = "node_eligible_mem_bp",
+     },
+     {
+             .metric = DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP,
+             .name = "node_ineligible_mem_bp",
+     },
      {
              .metric = DAMOS_QUOTA_ACTIVE_MEM_BP,
              .name = "active_mem_bp",

@@ -2669,6 +2677,8 @@ static int damos_sysfs_add_quota_score(
                      break;
              case DAMOS_QUOTA_NODE_MEM_USED_BP:
              case DAMOS_QUOTA_NODE_MEM_FREE_BP:
+             case DAMOS_QUOTA_NODE_ELIGIBLE_MEM_BP:
+             case DAMOS_QUOTA_NODE_INELIGIBLE_MEM_BP:
                      goal->nid = sysfs_goal->nid;

Quoted 9 lines

Quoted 4 lines
mm/damon/sysfs-schemes.c +0 -1
                      break;
              case DAMOS_QUOTA_NODE_MEMCG_USED_BP:
--
2.43.0
So, the overall concept and definition of the new goal metrics sound good to me. But I'd prefer having less optimized but simpler code, and nodes boundary crossing regions handling.

I'll prepare a v4 addressing all these points:
1. Fix 80-column wrapping
2. Remove eligible_bytes_per_node[] array, iterate regions per goal
instead
3. Handle regions crossing node boundaries
4. Pass scheme pointer from caller

I'm currently on a break and will send the updated patch after March 10th.

Thanks,
Ravi.

Quoted 3 lines

Thanks,
SJ

SeongJae Park

maintainer

2026-02-26 00:57 UTC #13 lore raw

Re: [RFC PATCH v3 3/4] mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics

Quoted 4 lines · On Wed, 25 Feb 2026 10:46:56 -0800 Ravi Jonnalagadda <[email protected]>…

Quoted 3 lines · SeongJae Park
On Mon, 23 Feb 2026 12:32:31 +0000 Ravi Jonnalagadda <[email protected]> wrote:

[...]

Quoted 6 lines

Quoted 5 lines
So, the overall concept and definition of the new goal metrics sound good to me. But I'd prefer having less optimized but simpler code, and nodes boundary crossing regions handling.

Quoted 6 lines

I'll prepare a v4 addressing all these points:
1. Fix 80-column wrapping
2. Remove eligible_bytes_per_node[] array, iterate regions per goal
instead
3. Handle regions crossing node boundaries
4. Pass scheme pointer from caller

Thank you for flexibly accepting my suggestions and listing those again here. All looks good and catching all the points!

Quoted 3 lines

I'm currently on a break and will send the updated patch after March 10th.

I hope you a great break!

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-23 12:33 UTC #14 lore raw

RFC PATCH v4 4/4 mm/damon: add PA-mode cache for eligible memory detection lag

In reply to Ravi Jonnalagadda

In PA-mode, DAMON needs time to re-detect hot memory at new physical addresses after migration. This causes the goal metrics to temporarily show incorrect values until detection catches up.

Add an eligible cache mechanism to compensate for this detection lag:

- Track migration deltas per node using a rolling window that automatically expires old data
- Use direction-aware adjustment: for target nodes (receiving memory), use max(detected, predicted) to ensure migrated memory is counted even before detection catches up; for source nodes (losing memory), use predicted values when detection shows unreliable low values
- Maintain the zero-sum property across nodes to preserve total eligible memory
- Include cooldown mechanism to keep cache active while detection stabilizes after migration stops
- Add time-based expiry to clear stale cache data when no migration occurs for a configured period

The cache uses max_eligible tracking to handle detection oscillation, prioritizing peak observed values over potentially stale snapshots. A threshold check prevents quota oscillation when detection swings between zero and small values.

Signed-off-by Ravi Jonnalagadda <[email protected]>

---
 include/linux/damon.h    |  45 +++++
 mm/damon/core.c          | 421 +++++++++++++++++++++++++++++++++++----
 mm/damon/sysfs-schemes.c |  30 +++
 3 files changed, 460 insertions(+), 36 deletions(-)

include/linux/damon.h +45 -0

index 6df716533fbf..230f95910aab 100644
@@ -541,6 +541,49 @@ struct damos_migrate_dests {
 	size_t nr_dests;
 };
 
+#define DAMOS_ELIGIBLE_CACHE_SLOTS	4
+#define DAMOS_ELIGIBLE_CACHE_COOLDOWN	3
+#define DAMOS_ELIGIBLE_CACHE_TIMEOUT_MS	10000	/* 10 seconds */
+
+/**
+ * struct damos_eligible_cache - Cache for bridging detection lag after migration.
+ * @base_eligible:	Snapshot of eligible_bytes_per_node at cache creation.
+ * @max_eligible:	Maximum detected eligible seen while cache active.
+ * @migration_delta:	Net bytes migrated TO each node per slot (negative = FROM).
+ * @current_slot:	Current slot index in the rolling window.
+ * @cooldown:		Intervals remaining before cache can deactivate.
+ * @active:		True if cache has recent migration data.
+ * @last_migration_jiffies: Timestamp of last migration for time-based expiry.
+ *
+ * For PA-mode migration, DAMON needs time to re-detect hot memory at new
+ * physical addresses after migration. This cache tracks recent migrations
+ * using a rolling window, allowing goal metric calculation to account for
+ * detection lag. The rolling window automatically expires old migrations
+ * after DAMOS_ELIGIBLE_CACHE_SLOTS intervals.
+ *
+ * The cache maintains zero-sum property: bytes subtracted from source node
+ * equal bytes added to target node, preserving total eligible memory.
+ *
+ * max_eligible tracks the highest detected eligible value seen while the cache
+ * is active. This provides a fallback when both base_eligible and current
+ * detection are 0 due to detection oscillation timing.
+ *
+ * Time-based expiry: The cache clears all slots and deactivates if no migration
+ * occurs for DAMOS_ELIGIBLE_CACHE_TIMEOUT_MS milliseconds. This prevents stale
+ * delta data from persisting indefinitely across test runs or after migration
+ * completes.
+ */
+struct damos_eligible_cache {
+	unsigned long base_eligible[MAX_NUMNODES];
+	unsigned long max_eligible[MAX_NUMNODES];
+	long migration_delta[DAMOS_ELIGIBLE_CACHE_SLOTS][MAX_NUMNODES];
+	unsigned int current_slot;
+	unsigned int cooldown;
+	bool active;
+	unsigned long last_migration_jiffies;
+	unsigned long timeout_ms;	/* Configurable timeout, 0 = use default */
+};
+
 /**
  * struct damos - Represents a Data Access Monitoring-based Operation Scheme.
  * @pattern:		Access pattern of target regions.

@@ -560,6 +603,7 @@ struct damos_migrate_dests {
  * @last_applied:	Last @action applied ops-managing entity.
  * @stat:		Statistics of this scheme.
  * @eligible_bytes_per_node: Scheme-eligible bytes per NUMA node.
+ * @eligible_cache:	Cache for detection lag compensation.
  * @max_nr_snapshots:	Upper limit of nr_snapshots stat.
  * @list:		List head for siblings.
  *

@@ -650,6 +694,7 @@ struct damos {
 	void *last_applied;
 	struct damos_stat stat;
 	unsigned long eligible_bytes_per_node[MAX_NUMNODES];
+	struct damos_eligible_cache eligible_cache;
 	unsigned long max_nr_snapshots;
 	struct list_head list;
 };

mm/damon/core.c +385 ~~-36~~

555 lines not on this page · raw message

mm/damon/sysfs-schemes.c +30 -0

index 232b33f5cbfb..bf68c3157c19 100644
@@ -1501,6 +1501,7 @@ struct damon_sysfs_quotas {
 	unsigned long reset_interval_ms;
 	unsigned long effective_sz;	/* Effective size quota in bytes */
 	enum damos_quota_goal_tuner goal_tuner;
+	unsigned long effective_bytes_cache_timeout_ms;
 };
 
 static struct damon_sysfs_quotas *damon_sysfs_quotas_alloc(void)

@@ -1675,6 +1676,27 @@ static ssize_t goal_tuner_store(struct kobject *kobj,
 	return -EINVAL;
 }
 
+static ssize_t effective_bytes_cache_timeout_ms_show(struct kobject *kobj,
+		struct kobj_attribute *attr, char *buf)
+{
+	struct damon_sysfs_quotas *quotas = container_of(kobj,
+			struct damon_sysfs_quotas, kobj);
+
+	return sysfs_emit(buf, "%lu\n", quotas->effective_bytes_cache_timeout_ms);
+}
+
+static ssize_t effective_bytes_cache_timeout_ms_store(struct kobject *kobj,
+		struct kobj_attribute *attr, const char *buf, size_t count)
+{
+	struct damon_sysfs_quotas *quotas = container_of(kobj,
+			struct damon_sysfs_quotas, kobj);
+	int err = kstrtoul(buf, 0, &quotas->effective_bytes_cache_timeout_ms);
+
+	if (err)
+		return -EINVAL;
+	return count;
+}
+
 static void damon_sysfs_quotas_release(struct kobject *kobj)
 {
 	kfree(container_of(kobj, struct damon_sysfs_quotas, kobj));

@@ -1695,12 +1717,16 @@ static struct kobj_attribute damon_sysfs_quotas_effective_bytes_attr =
 static struct kobj_attribute damon_sysfs_quotas_goal_tuner_attr =
 		__ATTR_RW_MODE(goal_tuner, 0600);
 
+static struct kobj_attribute damon_sysfs_quotas_cache_timeout_ms_attr =
+		__ATTR_RW_MODE(effective_bytes_cache_timeout_ms, 0600);
+
 static struct attribute *damon_sysfs_quotas_attrs[] = {
 	&damon_sysfs_quotas_ms_attr.attr,
 	&damon_sysfs_quotas_sz_attr.attr,
 	&damon_sysfs_quotas_reset_interval_ms_attr.attr,
 	&damon_sysfs_quotas_effective_bytes_attr.attr,
 	&damon_sysfs_quotas_goal_tuner_attr.attr,
+	&damon_sysfs_quotas_cache_timeout_ms_attr.attr,
 	NULL,
 };
 ATTRIBUTE_GROUPS(damon_sysfs_quotas);

@@ -2822,6 +2848,10 @@ static struct damos *damon_sysfs_mk_scheme(
 	/* Set goal_tuner after damon_new_scheme() as it defaults to CONSIST */
 	scheme->quota.goal_tuner = sysfs_quotas->goal_tuner;
 
+	/* Set cache timeout, use default if 0 */
+	scheme->eligible_cache.timeout_ms =
+		sysfs_quotas->effective_bytes_cache_timeout_ms;
+
 	err = damos_sysfs_add_quota_score(sysfs_quotas->goals, &scheme->quota);
 	if (err) {
 		damon_destroy_scheme(scheme);

Signature · 1 line

2.43.0

SeongJae Park

maintainer

2026-02-24 05:54 UTC #15 lore raw

Re: [RFC PATCH v4 4/4] mm/damon: add PA-mode cache for eligible memory detection lag

Quoted 3 lines · On Mon, 23 Feb 2026 12:32:32 +0000 Ravi Jonnalagadda <[email protected]>…

In PA-mode, DAMON needs time to re-detect hot memory at new physical addresses after migration. This causes the goal metrics to temporarily show incorrect values until detection catches up.

I agree this can happen, and could be problematic on some setup.

Quoted 16 lines

Add an eligible cache mechanism to compensate for this detection lag:
- Track migration deltas per node using a rolling window that automatically expires old data
- Use direction-aware adjustment: for target nodes (receiving memory), use max(detected, predicted) to ensure migrated memory is counted even before detection catches up; for source nodes (losing memory), use predicted values when detection shows unreliable low values
- Maintain the zero-sum property across nodes to preserve total eligible memory
- Include cooldown mechanism to keep cache active while detection stabilizes after migration stops
- Add time-based expiry to clear stale cache data when no migration occurs for a configured period

Quoted 4 lines

The cache uses max_eligible tracking to handle detection oscillation, prioritizing peak observed values over potentially stale snapshots. A threshold check prevents quota oscillation when detection swings between zero and small values.

But, I feel this might be too overfit solution for a specific setup.

Quoted 7 lines

Signed-off-by Ravi Jonnalagadda <[email protected]>

---
 include/linux/damon.h    |  45 +++++
 mm/damon/core.c          | 421 +++++++++++++++++++++++++++++++++++----
 mm/damon/sysfs-schemes.c |  30 +++
 3 files changed, 460 insertions(+), 36 deletions(-)

The size of the change is quite big. I'm now curious if the problem is significant enough for this size of change, and if this solution is only the single and the best one.

First of all, I'm curious if the problem is that significant. I assume you may seen the issue from your test setup that you shared with the cover letter. From my understanding of the cover letter of this patch series, however, you are testing this on a setup having two complementary schemes. And you use TEMPORAL tuner. The motivation of TEMPORAL tuner was for setup that not having a factor to move the quota goal value without additional intervention. In complementary schemes setup, the schemes becomes such factors for each other. In the case, TEMPORAL tuner might be worse in terms of the size of temporal oscillations. I don't know details of your test setup, but I suspect the use of TEMPORAL tuner might made the issue bigger than real.

I also assume the real world people may use DAMON with auto-tuning mostly because they don't know the access pattern of the system and assume it will be dynamic. In the case, even if we perfectly solve the issue, some of oscillation will happen. So, I think the issue in the real world might be smaller than that we can find on some specific test setups.

Meanwhile, the node_[in]eligible_mem_bp concept makes sense to me. I'm worried if this patch is unnecessarily delaying the progress of the main change.

So, unless we have clear evidence of the significance of this issue, I'd prefer dropping this for now. After that, if the issue turns out to be significant or this solution is proven to be significantly beneficial, from your next more realistic test setup, or from real world usage after upstreaming of the main change, we can revisit. What do you think?

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-25 18:58 UTC #16 lore raw

Re: [RFC PATCH v4 4/4] mm/damon: add PA-mode cache for eligible memory detection lag

Quoted 9 lines · SeongJae Park

Quoted 3 lines · On Mon, 23 Feb 2026 12:32:32 +0000 Ravi Jonnalagadda <[email protected]>…
In PA-mode, DAMON needs time to re-detect hot memory at new physical addresses after migration. This causes the goal metrics to temporarily show incorrect values until detection catches up.
I agree this can happen, and could be problematic on some setup.

Thank you for acknowledging the issue.

Quoted 31 lines

Quoted 16 lines
Add an eligible cache mechanism to compensate for this detection lag:
- Track migration deltas per node using a rolling window that automatically expires old data
- Use direction-aware adjustment: for target nodes (receiving memory), use max(detected, predicted) to ensure migrated memory is counted even before detection catches up; for source nodes (losing memory), use predicted values when detection shows unreliable low values
- Maintain the zero-sum property across nodes to preserve total eligible memory
- Include cooldown mechanism to keep cache active while detection stabilizes after migration stops
- Add time-based expiry to clear stale cache data when no migration occurs for a configured period
Quoted 4 lines
The cache uses max_eligible tracking to handle detection oscillation, prioritizing peak observed values over potentially stale snapshots. A threshold check prevents quota oscillation when detection swings between zero and small values.
But, I feel this might be too overfit solution for a specific setup.
Quoted 7 lines
Signed-off-by Ravi Jonnalagadda <[email protected]>
---
 include/linux/damon.h    |  45 +++++
 mm/damon/core.c          | 421 +++++++++++++++++++++++++++++++++++----
 mm/damon/sysfs-schemes.c |  30 +++
 3 files changed, 460 insertions(+), 36 deletions(-)

Quoted 3 lines

The size of the change is quite big. I'm now curious if the problem is significant enough for this size of change, and if this solution is only the single and the best one.

I understand. The cache was consciously separated as patch 4 because it represents ONE possible approach to handle detection lag - not necessarily THE approach.
My goal was to share what was needed to achieve equilibrium with my synthetic benchmark workload,
while making it clear this could be dropped or replaced with alternatives.

Quoted 11 lines

First of all, I'm curious if the problem is that significant. I assume you may seen the issue from your test setup that you shared with the cover letter. From my understanding of the cover letter of this patch series, however, you are testing this on a setup having two complementary schemes. And you use TEMPORAL tuner. The motivation of TEMPORAL tuner was for setup that not having a factor to move the quota goal value without additional intervention. In complementary schemes setup, the schemes becomes such factors for each other. In the case, TEMPORAL tuner might be worse in terms of the size of temporal oscillations. I don't know details of your test setup, but I suspect the use of TEMPORAL tuner might made the issue bigger than real.

That's a fair point. I chose TEMPORAL because I wanted to move the required amount of pages as quickly as possible to reach equilibrium - essentially "migrate at full speed until target is reached, then stop." For my multiload benchmark with uniformly hot memory, this seemed like the most direct approach.

You're right that with complementary schemes, the schemes act as factors for each other, and CONSIST tuner with its feedback loop might make the detection lag problem more manageable through gradual adjustment.

Quoted 6 lines

I also assume the real world people may use DAMON with auto-tuning mostly because they don't know the access pattern of the system and assume it will be dynamic. In the case, even if we perfectly solve the issue, some of oscillation will happen. So, I think the issue in the real world might be smaller than that we can find on some specific test setups.

Agreed. Real-world workloads with mixed hot/cold memory and dynamic access patterns might behave differently from my synthetic benchmark where all memory is uniformly hot. The uniform-hot case is essentially a worst-case scenario that forces continuous oscillation regardless of detection lag compensation.

Quoted 9 lines

Meanwhile, the node_[in]eligible_mem_bp concept makes sense to me. I'm worried if this patch is unnecessarily delaying the progress of the main change.
So, unless we have clear evidence of the significance of this issue, I'd prefer dropping this for now. After that, if the issue turns out to be significant or this solution is proven to be significantly beneficial, from your next more realistic test setup, or from real world usage after upstreaming of the main change, we can revisit. What do you think?

I agree with dropping this patch for now. Let's focus on getting the core metrics merged first.
The cache mechanism can be revisited later if real-world usage shows it's needed.

Thanks,
Ravi.

Quoted 6 lines

Thanks,
SJ
[...]

SeongJae Park

maintainer

2026-02-26 00:59 UTC #17 lore raw

Re: [RFC PATCH v4 4/4] mm/damon: add PA-mode cache for eligible memory detection lag

Quoted 3 lines · On Wed, 25 Feb 2026 10:58:15 -0800 Ravi Jonnalagadda <[email protected]>…

Quoted 2 lines · SeongJae Park
On Mon, 23 Feb 2026 12:32:32 +0000 Ravi Jonnalagadda <[email protected]> wrote:

[...]

Quoted 4 lines

I agree with dropping this patch for now. Let's focus on getting the core metrics merged first.
The cache mechanism can be revisited later if real-world usage shows it's needed.

Thank you for flexibly accepting my suggestion, Ravi. I hope to continue more discussions on the issue later.

Thanks,
SJ

[...]

SeongJae Park

maintainer

2026-02-24 05:36 UTC #18 lore raw

In reply to Ravi Jonnalagadda

Hello Ravi,

Quoted 7 lines · On Mon, 23 Feb 2026 12:32:28 +0000 Ravi Jonnalagadda <[email protected]>…

MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This series introduces two new DAMON quota goal metrics for controlling memory migration in heterogeneous memory systems (e.g., DRAM and CXL memory tiering) using physical address (PA) mode monitoring.

Thank you for keep working on and sharing this :)

Quoted 10 lines

v2: https://lore.kernel.org/linux-mm/[email protected]/
Changes since v2:
=================
- Split single metric into two complementary metrics:
* node_eligible_mem_bp: hot memory present ON the specified node
* node_ineligible_mem_bp: hot memory NOT on the specified node. This enables both PUSH and PULL schemes to work together.

This perfectly aligns with the direction we agreed on the previous discussion. Sounds good and reasonable to me.

Quoted 3 lines

- Added PA-mode detection lag compensation cache (see dedicated section below for design details).

I'm not very sure if this is really needed, though. I'll leave comment on the dedicated section below.

Quoted 6 lines

- Added fix for esz=0 quota bypass that allowed unlimited migration when goal was achieved.
- Added fix for goal_tuner sysfs setting being ignored due to damon_new_scheme() always defaulting to CONSIST.

Thank you for finding and fixing these issues in my previously shared RFC patch series! I left a few comments to the patches. In short, the second fix looks good and I will add that to the next revision of my RFC patch series, if you don't mind. For the first fix, I'd like to take more time on thinking more cleaner solution.

Quoted 3 lines

- Rebased on SJ's damon/next branch which includes the TEMPORAL goal tuner required for these metrics.

Thank you for clarifying this! This kind of context is very helpful at revidewing patches.

Quoted 22 lines

Background and Motivation
=========================
In heterogeneous memory systems, controlling hot memory distribution across NUMA nodes is essential for performance optimization. This series enables system wide hot page distribution with target-state goals like "maintain 30% of hot memory on CXL" using PA-mode DAMON schemes.
Two-Scheme Setup for Hot Page Distribution
==========================================
For maintaining 30% of hot memory on CXL (node 1):
    PUSH scheme (DRAM->CXL): migrate_hot from node 0 -> node 1
      goal: node_eligible_mem_bp, nid=1, target=3000
      Activates when node 1 has less than 30% hot memory

    PULL scheme (CXL->DRAM): migrate_hot from node 1 -> node 0
      goal: node_ineligible_mem_bp, nid=1, target=7000
      Activates when node 1 has more than 30% hot memory

Quoted 3 lines

Both schemes use the TEMPORAL goal tuner which sets esz to maximum when under goal and zero when achieved. Together they converge to equilibrium at the target distribution.

When this kind of complementary setup is being used, in my opinion, CONSIST tuner might be better, especially when the access pattern is dynamic. But it is up to user's choice.

Quoted 11 lines

What These Metrics Do
=====================
node_eligible_mem_bp measures:
effective_hot_bytes_on_node / total_hot_bytes * 10000
node_ineligible_mem_bp measures:
(total_hot_bytes - effective_hot_bytes_on_node) / total_hot_bytes * 10000
The metrics are complementary: eligible_bp + ineligible_bp = 10000 bp.

All make sense to me, so far.

Quoted 7 lines

PA-Mode Detection Lag and Cache Design
======================================
In PA-mode, when pages are migrated:
1. Source node detection drops immediately (pages are gone)
2. Target node detection increases slowly (new addresses need sampling)

I agree. And this is not what I clearly expected during the previous discussion. Thank you for sharing this issue.

Quoted 4 lines

This asymmetry causes temporary underestimation of hot memory on the target node. Without compensation, the system keeps migrating even after reaching the goal.

But, is this really significant? I believe people may use complementary auto-tune setup especially when they expect dynamic access pattern. In the case, even if we can perfectly compensate this kind of gap, some of oscillation will happen. You also mentioned "eventual convergence" could be acceptable.

Quoted 10 lines

The cache addresses this by remembering how much was recently migrated. When calculating effective hot memory:
- Source node: reduce detected amount by recent migrations out
- Target node: boost detected amount by recent migrations in
The cache uses a rolling window to track migrations over time, and expires after a configurable timeout (default 10s) when no migration activity occurs. It also detects when its baseline becomes stale due to new hot memory appearing in the workload.

I will leave more comments to the patch implementing this. But this seems too much at the current stage, unless there are clear test results showing its needs. I'd recommend proceeding without this, and later revisit if the problem becomes clearly significant.

Quoted 32 lines

Dependencies
============
This series is based on SJ's damon/next branch which includes:
- mm/damon/core: introduce damos_quota_goal_tuner [1]
- mm/damon/core: set quota-score histogram with core filters [2]
- mm/damon: always respect min_nr_regions from the beginning [3]
- mm/damon/core: disallow non-power of two min_region_sz [4]
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/
Patch Organization
==================
1. mm/damon/sysfs: set goal_tuner after scheme creation
- Fixes goal_tuner initialization order in sysfs scheme creation
2. mm/damon: fix esz=0 quota bypass allowing unlimited migration
- Ensures esz=0 stops migration rather than bypassing quota entirely
3. mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics
- Adds the two complementary metrics for hot memory distribution control
4. mm/damon: add PA-mode cache for eligible memory detection lag
- Implements rolling window cache to compensate for PA-mode detection lag
- Adds configurable cache timeout via sysfs

Quoted 5 lines

Testing Status
==============
Functionally tested on a two-node heterogeneous memory system (DRAM + CXL) with PUSH+PULL scheme configuration.

Glad to hear the functionality is tested. Looking forward to the next results!

Quoted 2 lines

This is an RFC and feedback on the design is appreciated.

I'm yet to further reply to the fourth patch, but I hope my comments be worthy :)

Thanks,
SJ

[...]

Ravi Jonnalagadda

author

2026-02-25 18:19 UTC #19 lore raw

Quoted 9 lines · SeongJae Park

Hello Ravi,
Quoted 4 lines · On Mon, 23 Feb 2026 12:32:28 +0000 Ravi Jonnalagadda <[email protected]>…
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Quoted 5 lines

Quoted 3 lines
This series introduces two new DAMON quota goal metrics for controlling memory migration in heterogeneous memory systems (e.g., DRAM and CXL memory tiering) using physical address (PA) mode monitoring.
Thank you for keep working on and sharing this :)

Thank you for the detailed review!

Quoted 7 lines

Quoted 7 lines
v2: https://lore.kernel.org/linux-mm/[email protected]/
Changes since v2:
=================

Quoted 7 lines

Quoted 4 lines
- Split single metric into two complementary metrics:
* node_eligible_mem_bp: hot memory present ON the specified node
* node_ineligible_mem_bp: hot memory NOT on the specified node. This enables both PUSH and PULL schemes to work together.
This perfectly aligns with the direction we agreed on the previous discussion. Sounds good and reasonable to me.

Great, glad the approach makes sense.

Quoted 7 lines

Quoted 4 lines
- Added PA-mode detection lag compensation cache (see dedicated section below for design details).
I'm not very sure if this is really needed, though. I'll leave comment on the dedicated section below.

Understood. I consciously separated the cache implementation (patch 4) from the core metrics (patch 3) because the cache is ONE possible approach to handle detection lag - not necessarily THE approach. My goal was to share what was needed to achieve equilibrium with my synthetic benchmark workload (multiload),
while making it clear that the cache mechanism could be dropped or replaced with alternatives.

Quoted 5 lines

Quoted 5 lines
- Added fix for esz=0 quota bypass that allowed unlimited migration when goal was achieved.

Quoted 8 lines

Quoted 2 lines
- Added fix for goal_tuner sysfs setting being ignored due to damon_new_scheme() always defaulting to CONSIST.
Thank you for finding and fixing these issues in my previously shared RFC patch series! I left a few comments to the patches. In short, the second fix looks good and I will add that to the next revision of my RFC patch series, if you don't mind. For the first fix, I'd like to take more time on thinking more cleaner solution.

Sounds good. Please go ahead and incorporate the goal_tuner fix into your series.
Happy to test whatever approach you come up with for the esz=0 issue.

Quoted 30 lines

Quoted 4 lines
- Rebased on SJ's damon/next branch which includes the TEMPORAL goal tuner required for these metrics.
Thank you for clarifying this! This kind of context is very helpful at revidewing patches.
Quoted 22 lines
Background and Motivation
=========================
In heterogeneous memory systems, controlling hot memory distribution across NUMA nodes is essential for performance optimization. This series enables system wide hot page distribution with target-state goals like "maintain 30% of hot memory on CXL" using PA-mode DAMON schemes.
Two-Scheme Setup for Hot Page Distribution
==========================================
For maintaining 30% of hot memory on CXL (node 1):
    PUSH scheme (DRAM->CXL): migrate_hot from node 0 -> node 1
      goal: node_eligible_mem_bp, nid=1, target=3000
      Activates when node 1 has less than 30% hot memory

    PULL scheme (CXL->DRAM): migrate_hot from node 1 -> node 0
      goal: node_ineligible_mem_bp, nid=1, target=7000
      Activates when node 1 has more than 30% hot memory

Quoted 8 lines

Quoted 3 lines
Both schemes use the TEMPORAL goal tuner which sets esz to maximum when under goal and zero when achieved. Together they converge to equilibrium at the target distribution.
When this kind of complementary setup is being used, in my opinion, CONSIST tuner might be better, especially when the access pattern is dynamic. But it is up to user's choice.

That's a fair point. I chose TEMPORAL because I wanted to move the required amount of pages as quickly as possible to reach equilibrium. For my multiload benchmark with uniformly hot memory, this seemed like the most direct approach to test the metrics.

Quoted 18 lines

Quoted 11 lines
What These Metrics Do
=====================
node_eligible_mem_bp measures:
effective_hot_bytes_on_node / total_hot_bytes * 10000
node_ineligible_mem_bp measures:
(total_hot_bytes - effective_hot_bytes_on_node) / total_hot_bytes * 10000
The metrics are complementary: eligible_bp + ineligible_bp = 10000 bp.
All make sense to me, so far.
Quoted 4 lines
PA-Mode Detection Lag and Cache Design
======================================

Quoted 6 lines

Quoted 3 lines
In PA-mode, when pages are migrated:
1. Source node detection drops immediately (pages are gone)
2. Target node detection increases slowly (new addresses need sampling)
I agree. And this is not what I clearly expected during the previous discussion. Thank you for sharing this issue.

I'm glad this observation is useful. It was something I discovered during testing that wasn't obvious until I looked at the trace data closely.

Quoted 11 lines

Quoted 5 lines
This asymmetry causes temporary underestimation of hot memory on the target node. Without compensation, the system keeps migrating even after reaching the goal.
But, is this really significant? I believe people may use complementary auto-tune setup especially when they expect dynamic access pattern. In the case, even if we can perfectly compensate this kind of gap, some of oscillation will happen. You also mentioned "eventual convergence" could be acceptable.

This is a valid point. Real-world workloads with mixed hot/cold memory and dynamic access patterns might behave differently from my synthetic benchmark where all memory is uniformly hot. The uniform-hot case is essentially a worst-case scenario that forces continuous oscillation regardless of detection lag compensation.

For my multiload workload without patch 4, there was significant unnecessary page movement due to the detection lag. However, as you note, that may be an artifact of the synthetic benchmark + TEMPORAL tuner combination rather than a fundamental problem that affects all use cases.

Quoted 11 lines

Quoted 10 lines
The cache addresses this by remembering how much was recently migrated. When calculating effective hot memory:
- Source node: reduce detected amount by recent migrations out
- Target node: boost detected amount by recent migrations in
The cache uses a rolling window to track migrations over time, and expires after a configurable timeout (default 10s) when no migration activity occurs. It also detects when its baseline becomes stale due to new hot memory appearing in the workload.

Quoted 4 lines

I will leave more comments to the patch implementing this. But this seems too much at the current stage, unless there are clear test results showing its needs. I'd recommend proceeding without this, and later revisit if the problem becomes clearly significant.

I agree. Let's drop patch 4 for now and focus on getting the core metrics merged.
The cache mechanism can be revisited later if real-world usage shows it's needed.

Quoted 39 lines

Quoted 33 lines
Dependencies
============
This series is based on SJ's damon/next branch which includes:
- mm/damon/core: introduce damos_quota_goal_tuner [1]
- mm/damon/core: set quota-score histogram with core filters [2]
- mm/damon: always respect min_nr_regions from the beginning [3]
- mm/damon/core: disallow non-power of two min_region_sz [4]
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/
Patch Organization
==================
1. mm/damon/sysfs: set goal_tuner after scheme creation
- Fixes goal_tuner initialization order in sysfs scheme creation
2. mm/damon: fix esz=0 quota bypass allowing unlimited migration
- Ensures esz=0 stops migration rather than bypassing quota entirely
3. mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics
- Adds the two complementary metrics for hot memory distribution control
4. mm/damon: add PA-mode cache for eligible memory detection lag
- Implements rolling window cache to compensate for PA-mode detection lag
- Adds configurable cache timeout via sysfs
Quoted 5 lines
Testing Status
==============
Functionally tested on a two-node heterogeneous memory system (DRAM + CXL) with PUSH+PULL scheme configuration.

Quoted 8 lines

Glad to hear the functionality is tested. Looking forward to the next results!
Quoted 2 lines
This is an RFC and feedback on the design is appreciated.
I'm yet to further reply to the fourth patch, but I hope my comments be worthy :)

Very much so! Your feedback has been invaluable in shaping this work. :-)

I'm currently on a break and will be back after March 10th. Once I return, I'll send the updated patch 3 and share test results with CONSIST tuner.

Thanks again for the thorough review!

Ravi

Quoted 5 lines

Thanks,
SJ
[...]

SeongJae Park

maintainer

2026-02-26 00:52 UTC #20 lore raw

Quoted 10 lines · On Wed, 25 Feb 2026 10:19:11 -0800 Ravi Jonnalagadda <[email protected]>…

Quoted 9 lines · SeongJae Park
Hello Ravi,
Quoted 4 lines · On Mon, 23 Feb 2026 12:32:28 +0000 Ravi Jonnalagadda <[email protected]>…
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Quoted 7 lines

Quoted 5 lines
Quoted 3 lines
This series introduces two new DAMON quota goal metrics for controlling memory migration in heterogeneous memory systems (e.g., DRAM and CXL memory tiering) using physical address (PA) mode monitoring.
Thank you for keep working on and sharing this :)
Thank you for the detailed review!

My pleasure!

[...]

Quoted 6 lines

Quoted 5 lines
Quoted 2 lines
- Added PA-mode detection lag compensation cache (see dedicated section below for design details).
I'm not very sure if this is really needed, though. I'll leave comment on the dedicated section below.

Quoted 7 lines

Understood. I consciously separated the cache implementation (patch 4) from the core metrics (patch 3) because the cache is ONE possible approach to handle detection lag - not necessarily THE approach. My goal was to share what was needed to achieve equilibrium with my synthetic benchmark workload (multiload),
while making it clear that the cache mechanism could be dropped or replaced with alternatives.

That was indeed helpful for reviewing, thank you!

Quoted 15 lines

Quoted 6 lines
Quoted 6 lines
- Added fix for esz=0 quota bypass that allowed unlimited migration when goal was achieved.
Quoted 8 lines
Quoted 2 lines
- Added fix for goal_tuner sysfs setting being ignored due to damon_new_scheme() always defaulting to CONSIST.
Thank you for finding and fixing these issues in my previously shared RFC patch series! I left a few comments to the patches. In short, the second fix looks good and I will add that to the next revision of my RFC patch series, if you don't mind. For the first fix, I'd like to take more time on thinking more cleaner solution.

Quoted 3 lines

Sounds good. Please go ahead and incorporate the goal_tuner fix into your series.
Happy to test whatever approach you come up with for the esz=0 issue.

Thank you, I will do!

[...]

Quoted 9 lines

Quoted 6 lines
Quoted 3 lines
In PA-mode, when pages are migrated:
1. Source node detection drops immediately (pages are gone)
2. Target node detection increases slowly (new addresses need sampling)
I agree. And this is not what I clearly expected during the previous discussion. Thank you for sharing this issue.
I'm glad this observation is useful. It was something I discovered during testing that wasn't obvious until I looked at the trace data closely.

Thank you for sharing the pain point. I recently added a few more DAMOS tracepoints motivated by our offline discussion. I'm planning to add better supports of those in DAMON user-space tool. Knowing this kind of pain points is essential and useful at improving DAMON, thank you!

[...]

Quoted 9 lines

Quoted 4 lines
I will leave more comments to the patch implementing this. But this seems too much at the current stage, unless there are clear test results showing its needs. I'd recommend proceeding without this, and later revisit if the problem becomes clearly significant.
I agree. Let's drop patch 4 for now and focus on getting the core metrics merged.
The cache mechanism can be revisited later if real-world usage shows it's needed.

Thank you for flexibly accepting my suggestion!

[...]

Quoted 5 lines

Quoted 3 lines
I'm yet to further reply to the fourth patch, but I hope my comments be worthy :)
Very much so! Your feedback has been invaluable in shaping this work. :-)

More than exciting to hear that :D

Quoted 4 lines

I'm currently on a break and will be back after March 10th. Once I return, I'll send the updated patch 3 and share test results with CONSIST tuner.

Sounds perfect, I hope you to have great break!

Thanks,
SJ

[...]