mm/damon/core: detect internal variation above max_nr_regions/2
Subsystems (2)
v2 cover mm/damon/core: detect internal variation above max_nr_regions/2
Sorry for the late.
kdamond_split_regions() bails out early when nr_regions is already above max_nr_regions / 2. A large region that picks up new internal variation after that point never gets split, so we lose visibility into its hot/cold structure.
We hit this with damon-paddr on hugepage workloads and damon-vaddr on processes that mmap a large anonymous range.
Example with max_nr_regions == 1500. A target ends up with 799 small hot/cold regions plus one big region (an earlier merge collapsed a uniformly-accessed range into a single piece):
H:hot
C:cold
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH..........................|
nr_regions = 800 > max_nr_regions / 2 = 750Now a cold subarea shows up inside r800:
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH........CCCCCC.............|
The small regions can't merge with each other (their access counts differ), so budget never frees up. r800 can't be split because nr_regions > max_nr_regions / 2 returns early. The cold subarea stays invisible.
Patch 1 keeps refining on this path: when nr_regions is above max_nr_regions / 2 but still under the maximum, it splits a fraction of the regions instead of returning. The fraction shrinks as the remaining budget shrinks, so the count approaches max_nr_regions smoothly. A useless split is undone by the next merge cycle.
Patch 2 adds a KUnit test for the case where nr_regions is already above max_nr_regions / 2.
Thanks to SeongJae for the suggestion to drive the split fraction from the remaining budget rather than an age-based filter.
v1 -> v2: Some feedback from SJ.
v1: https://lore.kernel.org/damon/[email protected]/
Jiayuan Chen (2):
mm/damon/core: split a fraction of regions when nr_regions exceeds
max/2
mm/damon/tests/core-kunit: test split above max_nr_regions/2
mm/damon/core.c | 49 +++++++++++++++++++++++++--- mm/damon/tests/core-kunit.h | 64 +++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+), 5 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/2 mm/damon/core: split a fraction of regions when nr_regions exceeds max/2
From: Jiayuan Chen <[email protected]>
kdamond_split_regions() returns early when nr_regions is above max_nr_regions / 2, leaving internal access variation inside a large region undetected.
Such a layout is common with damon-paddr on hugepage workloads or damon-vaddr on processes with a large anonymous mmap.
For example, with max_nr_regions == 1500, a target may end up with 799 small alternating-temperature regions plus one large region that absorbed a uniformly-accessed range during an earlier merge:
H:hot
C:cold
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH..........................|
nr_regions = 800 > max_nr_regions / 2 = 750If a cold subarea later emerges inside r800:
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH........CCCCCC.............|
The small regions cannot merge with each other (different access counts), so the budget stays full. r800 cannot be split because nr_regions > max_nr_regions / 2 causes an early return. The cold subarea is never discovered.
When nr_regions is above max_nr_regions / 2 but still under the maximum, split only a fraction of the regions instead of returning. One region in every 'max_nr_regions / budget' regions is split, where budget is the remaining room (max_nr_regions - nr_regions), starting from a rotating offset so different regions get picked over time. The fraction shrinks as the budget shrinks, so the region count keeps refining while approaching max_nr_regions smoothly rather than overshooting it. An unnecessary split is reverted by the next kdamond_merge_regions().
--- mm/damon/core.c | 49 ++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 44 insertions(+), 5 deletions(-)
mm/damon/core.c +44 -5
@@ -3712,6 +3712,37 @@ static void damon_split_regions_of(struct damon_ctx *ctx, } } +/* Split one in every @split_step regions into two, from a rotating offset */ +static void damon_split_some_regions (struct damon_ctx *ctx, +unsigned long split_step) +{ +static unsigned long rotation; +struct damon_target *t; +struct damon_region *r, *next; +unsigned long offset = rotation++ % split_step; +unsigned long idx =0 ; + +damon_for_each_target (t, ctx) { +damon_for_each_region_safe (r, next, t) { +unsigned long sz_region, sz_sub; + +if (idx++ % split_step != offset) +continue ; + sz_region =damon_sz_region (r); +if (sz_region <2 * ctx->min_region_sz) +continue ; + + sz_sub =ALIGN_DOWN (damon_rand (ctx,1 ,10 ) * + sz_region /10 , ctx->min_region_sz); +/* Do not allow blank region */ +if (sz_sub ==0 || sz_sub >= sz_region) +continue ; + +damon_split_region_at (t, r, sz_sub); + } + } +} +/* * Split every target region into randomly-sized small regions *
@@ -3725,25 +3756,33 @@ static void damon_split_regions_of(struct damon_ctx *ctx,static void kdamond_split_regions (struct damon_ctx *ctx) {struct damon_target *t; -unsigned int nr_regions =0 ; -static unsigned int last_nr_regions; +unsigned long nr_regions =0 ; +unsigned long max_nr_regions = ctx->attrs.max_nr_regions; +static unsigned long last_nr_regions;int nr_subregions =2 ;damon_for_each_target (t, ctx) nr_regions +=damon_nr_regions (t); -if (nr_regions > ctx->attrs.max_nr_regions /2 ) -return ; +if (nr_regions >= max_nr_regions) + goto done; + +if (nr_regions > max_nr_regions /2 ) { +damon_split_some_regions (ctx, + max_nr_regions / (max_nr_regions - nr_regions)); + goto done; + }/* Maybe the middle of the region has different access frequency */ if (last_nr_regions == nr_regions && - nr_regions < ctx->attrs.max_nr_regions /3 ) + nr_regions < max_nr_regions /3 ) nr_subregions =3 ;damon_for_each_target (t, ctx)damon_split_regions_of (ctx, t, nr_subregions, ctx->min_region_sz); +done: last_nr_regions = nr_regions; }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v2 1/2] mm/damon/core: split a fraction of regions when nr_regions exceeds max/2
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Data race on the globally shared static variable `rotation` causes undefined behavior and…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Data race on the globally shared static variable rotation causes undefined behavior and inter-context interference. --
commit 2895b546218b4ccc570b82e50e62f788bac4157b
Author: Jiayuan Chen <[email protected]>
mm/damon/core: split a fraction of regions when nr_regions exceeds max/2
This commit modifies kdamond_split_regions() to split only a fraction of regions rather than returning early when nr_regions is above max_nr_regions / 2. It uses a rotating offset to ensure different regions are picked for splitting over time as the region count smoothly approaches max_nr_regions.
Quoted 3 lines
mm/damon/core.c +0
-0
[ ... ]
mm/damon/core.c static void damon_split_regions_of(struct damon_ctx *ctx, · line 3722
Quoted 8 lines
mm/damon/core.c +8
-0+/* Split one in every @split_step regions into two, from a rotating offset */ +static void damon_split_some_regions (struct damon_ctx *ctx, +unsigned long split_step) +{ +static unsigned long rotation; +struct damon_target *t; +struct damon_region *r, *next; +unsigned long offset = rotation++ % split_step;
[Severity: Medium]
Will this cause a data race if multiple DAMON contexts are running concurrently?
Since rotation is declared static and locklessly incremented, independent kdamond worker threads could read and mutate it simultaneously.
This unsynchronized read-modify-write on shared state might trigger automated data race reports. Additionally, does this mean different contexts will step on each other's rotation offsets, breaking the intended round-robin behavior for each individual context?
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
SeongJae Park
maintainerRe: [PATCH v2 1/2] mm/damon/core: split a fraction of regions when nr_regions exceeds max/2
Quoted 32 lines · On Fri, 26 Jun 2026 09:09:15 +0000 [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Data race on the globally shared static variablerotationcauses undefined behavior and inter-context interference. --commit
2895b546218b4ccc570b82e50e62f788bac4157b
Author: Jiayuan Chen <[email protected]>
mm/damon/core: split a fraction of regions whennr_regionsexceeds max/2This commit modifies
kdamond_split_regions()to split only a fraction of regions rather than returning early whennr_regionsis abovemax_nr_regions/ 2. It uses a rotating offset to ensure different regions are picked for splitting over time as the region count smoothly approachesmax_nr_regions.Quoted 3 lines
mm/damon/core.c +0
-0[ ... ]
mm/damon/core.c static void damon_split_regions_of(struct damon_ctx *ctx, · line 3722
Quoted 8 lines
mm/damon/core.c +8
-0+/* Split one in every @split_step regions into two, from a rotating offset */ +static void damon_split_some_regions (struct damon_ctx *ctx, +unsigned long split_step) +{ +static unsigned long rotation; +struct damon_target *t; +struct damon_region *r, *next; +unsigned long offset = rotation++ % split_step;[Severity: Medium]
Will this cause a data race if multiple DAMON contexts are running concurrently?
Quoted 7 lines
Since rotation is declared static and locklessly incremented, independent kdamond worker threads could read and mutate it simultaneously.
This unsynchronized read-modify-write on shared state might trigger automated data race reports. Additionally, does this mean different contexts will step on each other's rotation offsets, breaking the intended round-robin behavior for each individual context?
I don't think this matters. We don't really need strict round-robin but a level of fairness. Let me know if I'm wrong.
Thanks,
SJ
[...]
Re: [PATCH v2 1/2] mm/damon/core: split a fraction of regions when nr_regions exceeds max/2
In reply to Jiayuan ChenQuoted 31 lines · On Fri, 26 Jun 2026 16:58:37 +0800 Jiayuan Chen <[email protected]> wrote:
From: Jiayuan Chen <[email protected]>
kdamond_split_regions()returns early whennr_regionsis abovemax_nr_regions/ 2, leaving internal access variation inside a large region undetected.Such a layout is common with damon-paddr on hugepage workloads or damon-vaddr on processes with a large anonymous mmap.
For example, with
max_nr_regions== 1500, a target may end up with 799 small alternating-temperature regions plus one large region that absorbed a uniformly-accessed range during an earlier merge:H:hot
C:coldr1 r2 r3 r800 HHHHHH|CCCCCC|HHHHHH|...|HHHHHH..........................| nr_regions = 800 > max_nr_regions / 2 = 750If a cold subarea later emerges inside r800:
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH........CCCCCC.............|The small regions cannot merge with each other (different access counts), so the budget stays full. r800 cannot be split because
nr_regions>max_nr_regions/ 2 causes an early return. The cold subarea is never discovered.
Quoted 9 lines
When
nr_regionsis abovemax_nr_regions/ 2 but still under the maximum, split only a fraction of the regions instead of returning. One region in every 'max_nr_regions/ budget' regions is split, where budget is the remaining room (max_nr_regions-nr_regions), starting from a rotating offset so different regions get picked over time. The fraction shrinks as the budget shrinks, so the region count keeps refining while approachingmax_nr_regionssmoothly rather than overshooting it. An unnecessary split is reverted by the nextkdamond_merge_regions().
Looks good!
Thinking loud. After this change, some users might surprise because the number of regions became higher than before. But it is still under the contracted maximum number, and they will show better accuracy. So I believe this is a reasonable improvement rather than a breaking change.
Thank you for this patch, Jiayuan.
Quoted 3 lines
Cc Jiayuan Chen <[email protected]>Signed-off-by Jiayuan Chen <[email protected]>
Thanks,
SJ
[...]
PATCH v2 2/2 mm/damon/tests/core-kunit: test split above max_nr_regions/2
In reply to Jiayuan ChenFrom: Jiayuan Chen <[email protected]>
Add a test that exercises kdamond_split_regions() when the total region count is already above max_nr_regions / 2, asserting that the function still splits a fraction of the regions (makes progress) and does not overshoot max_nr_regions.
The region size and min_region_sz are picked so the split arithmetic does not depend on the page size.
All tests pass:
damon: pass:31 fail:0 skip:0 total:31
Totals: pass:31 fail:0 skip:0 total:31
--- mm/damon/tests/core-kunit.h | 64 +++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)
mm/damon/tests/core-kunit.h +64 -0
@@ -341,6 +341,69 @@ static void damon_test_split_regions_of(struct kunit *test)damon_destroy_ctx (c); } +/* +* When the total region count is already above max_nr_regions / 2, +* kdamond_split_regions() must keep refining the resolution by splitting a +* fraction of the regions (making progress), without exceeding +* max_nr_regions. +*/ +static void damon_test_split_above_half_progresses (struct kunit *test) +{ +struct damon_ctx *c; +struct damon_target *t; +struct damon_region *r; +unsigned long start; +unsigned int nr_before, nr_after, i; +const unsigned int nr_init =760 ; +const unsigned long region_sz =100 ; + + c =damon_new_ctx (); +if (!c) +kunit_skip (test,"ctx alloc fail" ); + +/* Keep the split arithmetic independent of the page size */ + c->min_region_sz =1 ; + c->attrs.min_nr_regions =10 ; + c->attrs.max_nr_regions =1500 ; + + t =damon_new_target (); +if (!t) { +damon_destroy_ctx (c); +kunit_skip (test,"target alloc fail" ); + } + +for (i =0 ; i < nr_init; i++) { + start = i * region_sz; + r =damon_new_region (start, start + region_sz); +if (!r) { +damon_free_target (t); +damon_destroy_ctx (c); +kunit_skip (test,"region alloc fail" ); + } + r->nr_accesses = (i &1 ) ?0 :100 ; + r->age =5 ; +damon_add_region (r, t); + } + +damon_add_target (c, t); + + nr_before =damon_nr_regions (t); +/* Above max_nr_regions / 2, so the blanket-split path is skipped */ +KUNIT_EXPECT_GT (test, (unsigned long )nr_before, + c->attrs.max_nr_regions /2 ); + +kdamond_split_regions (c); + + nr_after =damon_nr_regions (t); +/* Still made progress ... */ +KUNIT_EXPECT_GT (test, nr_after, nr_before); +/* ... but did not overshoot the configured maximum */ +KUNIT_EXPECT_LE (test, (unsigned long )nr_after, + c->attrs.max_nr_regions); + +damon_destroy_ctx (c); +} +static void damon_test_ops_registration (struct kunit *test) {struct damon_ctx *c =damon_new_ctx ();
@@ -1521,6 +1584,7 @@ static struct kunit_case damon_test_cases[] = {KUNIT_CASE (damon_test_merge_two),KUNIT_CASE (damon_test_merge_regions_of),KUNIT_CASE (damon_test_split_regions_of), +KUNIT_CASE (damon_test_split_above_half_progresses),KUNIT_CASE (damon_test_ops_registration),KUNIT_CASE (damon_test_set_regions), KUNIT_CASE(damon_test_nr_accesses_to_accesses_bp),
Signature · 1 line
2.43.0
Re: [PATCH v2 2/2] mm/damon/tests/core-kunit: test split above max_nr_regions/2
Quoted 7 lines · On Fri, 26 Jun 2026 16:58:38 +0800 Jiayuan Chen <[email protected]> wrote:
From: Jiayuan Chen <[email protected]>
Add a test that exercises
kdamond_split_regions()when the total region count is already abovemax_nr_regions/ 2, asserting that the function still splits a fraction of the regions (makes progress) and does not overshootmax_nr_regions.
Quoted 6 lines
The region size and
min_region_szare picked so the split arithmetic does not depend on the page size.All tests pass:
damon: pass:31 fail:0 skip:0 total:31
Totals: pass:31 fail:0 skip:0 total:31
Nice!
Quoted 3 lines
Cc Jiayuan Chen <[email protected]>Signed-off-by Jiayuan Chen <[email protected]>
I have trivial comments below, though.
Quoted 52 lines
--- mm/damon/tests/core-kunit.h | 64 +++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)mm/damon/tests/core-kunit.h +40
-0@@ -341,6 +341,69 @@ static void damon_test_split_regions_of(struct kunit *test)damon_destroy_ctx (c); } +/* +* When the total region count is already above max_nr_regions / 2, +* kdamond_split_regions() must keep refining the resolution by splitting a +* fraction of the regions (making progress), without exceeding +* max_nr_regions. +*/ +static void damon_test_split_above_half_progresses (struct kunit *test) +{ +struct damon_ctx *c; +struct damon_target *t; +struct damon_region *r; +unsigned long start; +unsigned int nr_before, nr_after, i; +const unsigned int nr_init =760 ; +const unsigned long region_sz =100 ; + + c =damon_new_ctx (); +if (!c) +kunit_skip (test,"ctx alloc fail" ); + +/* Keep the split arithmetic independent of the page size */ + c->min_region_sz =1 ; + c->attrs.min_nr_regions =10 ; + c->attrs.max_nr_regions =1500 ; + + t =damon_new_target (); +if (!t) { +damon_destroy_ctx (c); +kunit_skip (test,"target alloc fail" ); + } + +for (i =0 ; i < nr_init; i++) { + start = i * region_sz; + r =damon_new_region (start, start + region_sz); +if (!r) { +damon_free_target (t); +damon_destroy_ctx (c); +kunit_skip (test,"region alloc fail" ); + } + r->nr_accesses = (i &1 ) ?0 :100 ;
mm/damon/tests/core-kunit.h static void damon_test_split_regions_of(struct kunit *test) · line 393
Quoted 10 lines
mm/damon/tests/core-kunit.h +10
-0+ r->age =5 ; +damon_add_region (r, t); + } + +damon_add_target (c, t); + + nr_before =damon_nr_regions (t); +/* Above max_nr_regions / 2, so the blanket-split path is skipped */ +KUNIT_EXPECT_GT (test, (unsigned long )nr_before, + c->attrs.max_nr_regions /2 );
Above KUNIT_EXPECT_GT() feels unnecessary.
mm/damon/tests/core-kunit.h static void damon_test_split_regions_of(struct kunit *test) · line 402
Quoted 9 lines
mm/damon/tests/core-kunit.h +9
-0+ +kdamond_split_regions (c); + + nr_after =damon_nr_regions (t); +/* Still made progress ... */ +KUNIT_EXPECT_GT (test, nr_after, nr_before); +/* ... but did not overshoot the configured maximum */ +KUNIT_EXPECT_LE (test, (unsigned long )nr_after, + c->attrs.max_nr_regions);
In future, we may do this multiple times.
mm/damon/tests/core-kunit.h static void damon_test_split_regions_of(struct kunit *test) · line 374
Quoted 4 lines
mm/damon/tests/core-kunit.h +4
-0+ +damon_destroy_ctx (c); +} +
Thanks,
SJ
[...]
SeongJae Park
maintainerQuoted 1 line · On Fri, 26 Jun 2026 16:58:36 +0800 Jiayuan Chen <[email protected]> wrote:
Sorry for the late.
No worry!
Quoted 31 lines
kdamond_split_regions()bails out early whennr_regionsis already abovemax_nr_regions/ 2. A large region that picks up new internal variation after that point never gets split, so we lose visibility into its hot/cold structure.We hit this with damon-paddr on hugepage workloads and damon-vaddr on processes that mmap a large anonymous range.
Example with
max_nr_regions== 1500. A target ends up with 799 small hot/cold regions plus one big region (an earlier merge collapsed a uniformly-accessed range into a single piece):H:hot
C:coldr1 r2 r3 r800 HHHHHH|CCCCCC|HHHHHH|...|HHHHHH..........................| nr_regions = 800 > max_nr_regions / 2 = 750Now a cold subarea shows up inside r800:
r1 r2 r3 r800
HHHHHH|CCCCCC|HHHHHH|...|HHHHHH........CCCCCC.............|The small regions can't merge with each other (their access counts differ), so budget never frees up. r800 can't be split because
nr_regions>max_nr_regions/ 2 returns early. The cold subarea stays invisible.
Quoted 5 lines
Patch 1 keeps refining on this path: when
nr_regionsis abovemax_nr_regions/ 2 but still under the maximum, it splits a fraction of the regions instead of returning. The fraction shrinks as the remaining budget shrinks, so the count approachesmax_nr_regionssmoothly. A useless split is undone by the next merge cycle.
I assume you confirmed this change is making somee progress in a test or on your real use case? It would be great if you can confirm that and/or even share your measurements.
Quoted 6 lines
Patch 2 adds a KUnit test for the case where
nr_regionsis already abovemax_nr_regions/ 2.Thanks to SeongJae for the suggestion to drive the split fraction from the remaining budget rather than an age-based filter.
I'm glad to help this grateful improvement!
Quoted 9 lines
v1 -> v2: Some feedback from SJ.
v1: https://lore.kernel.org/damon/[email protected]/Jiayuan Chen (2):
mm/damon/core: split a fraction of regions whennr_regionsexceedsmax/2
mm/damon/tests/core-kunit: test split above max_nr_regions/2
I have trivial comments to the second patch, but all looks good enough.
I will apply this series to damon/next [1] tree. If this patch is not added to mm.git in short term (~1 week?), I will ask mm.git maintainer (Andrew Morton) to pick this. So, no action from your side is needed for now. If it seems I also forgot doing that or you cannot wait for my action, please feel free to directly ask that to Andrew.
[1] https://origin.kernel.org/doc/html/latest/mm/damon/maintainer-profile.html#scm-trees
Thanks,
SJ
[...]