Skip to contentSkip to conversation

mm/damon: advance DAMOS-based LRU sorting

PATCH In review SeongJae Park 2 participants 21 messages Jan 13 – Jan 14

Subsystems (4)
lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

cover mm/damon: advance DAMOS-based LRU sorting

DAMOS_LRU_[DE]PRIO actions were added to DAMOS for more access-aware LRU lists sorting. For simple usage, a specialized kernel module, namely DAMON_LRU_SORT, has also been introduced. After the introduction of the module, DAMON got a few important new features, including the aim-based quota auto-tuning, age tracking, young page filter, and monitoring intervals auto-tuning. Meanwhile, DAMOS-based LRU sorting had no direct updates. Now we show some rooms to advance for DAMOS-based LRU sorting.

Firstly, the aim-oriented quota auto-tuning can simplify the LRU sorting parameters tuning. But there is no good auto-tuning target metric for LRU sorting use case. Secondly, the behavior of DAMOS_LRU_[DE]PRIO are not very symmetric. DAMOS_LRU_DEPRIO directly moves the pages to inactive LRU list, while DAMOS_LRU_PRIO only marks the page as accessed, so that the page can not directly but only eventually moved to the active LRU list. Finally, DAMON_LRU_SORT users cannot utilize the modern features that can be useful for them, too.

Improve the situation with the following changes. First, introduce a new DAMOS quota auto-tuning target metric for active:inactive memory size ratio. Since LRU sorting is a kind of balancing of active and inactive pages, the active:inactive memory size ratio can be intuitively set. Second, update DAMOS_LRU_[DE]PRIO behaviors to be more intuitive and symmetric, by letting them directly move the pages to [in]active LRU list. Third, update the DAMON_LRU_SORT module user interface to be able to fully utilize the modern features including the [in]active memory size ratio-based quota auto-tuning, young page filter, and monitoring intervals auto-tuning.

With these changes, for example, users can now ask DAMON to "find hot/cold memory regions with auto-tuned monitoring intervals, do one more page level access check for found hot/cold memory, and move pages of those to active or inactive LRU lists accordingly, aiming X:Y active to inactive memory ratio." For example, if they know 30% of the memory is better to be protected from reclamation, 30:70 can be set as the target ratio.

Test Results
------------

I ran DAMON_LRU_SORT with the features introduced by this series, on a real world server workload. For the active:inactive ratio goal, I set 50:50. I confirmed it achieves the target active:inactive ratio, without manual tuning of the monitoring intervals and the hot/coldness thresholds. The baseline system that was not running the DAMON_LRU_SORT was keeping active:inactive ratio of about 1:10.

Note that the test didn't show a clear performance difference, though. I believe that was mainly because the workload was not very memory intensive. Also, whether the 50:50 target ratio was optimum is unclear. Nonetheless, the positive performance impact of the basic LRU sorting idea is already confirmed with the initial DAMON_LRU_SORT introduction patch series. The goal of this patch series is simplifying the parameters tuning of DAMOS-based LRU sorting, and the test confirmed the aimed goals are achieved.

Patches Sequence
----------------

First three patches extend DAMOS quota auto-tuning to support [in]active memory ratio target metric type. Those (patches 1-3) introduce new metrics, implement DAMON sysfs support, and update the documentation, respectively.

Following patch (patch 4) makes DAMOS_LRU_PRIO action to directly move target pages to active LRU list, instead of only marking them accessed.

Following seven patches (patches 5-11) updates DAMON_LRU_SORT to support modern DAMON features. Patch 5 makes it uses not only access frequency but also age at under-quota regions prioritization. Patches 6-11 add the support for young page filtering, active:inactive memory ratio based quota auto-tuning, and monitoring intervals auto-tuning, with appropriate document updates.

Changes from RFC
(https://lore.kernel.org/[email protected])
- rebase to latest mm-new
- add test results on the cover letter
- minor wordsmithing and typo fixes

SeongJae Park (11):
mm/damon/core: introduce [in]active memory ratio damos quota goal
metric
mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP Docs/mm/damon/design: document DAMOS_QUOTA_[IN]ACTIVE_MEM_BP mm/damon/paddr: activate DAMOS_LRU_PRIO targets instead of marking
accessed
mm/damon/lru_sort: consider age for quota prioritization mm/damon/lru_sort: support young page filters Docs/admin-guide/mm/damon/lru_sort: document filter_young_pages mm/damon/lru_sort: support active:inactive memory ratio based
auto-tuning
Docs/admin-guide/mm/damon/lru_sort: document active_mem_bp parameter mm/damon/lru_sort: add monitoring intervals auto-tuning parameter Docs/admin-guide/mm/damon/lru_sort: document intervals autotuning

 .../admin-guide/mm/damon/lru_sort.rst         |  37 ++++++
 Documentation/mm/damon/design.rst             |   4 +
 include/linux/damon.h                         |   4 +
 mm/damon/core.c                               |  22 ++++
 mm/damon/lru_sort.c                           | 110 +++++++++++++++++-
 mm/damon/paddr.c                              |  18 ++-
 mm/damon/sysfs-schemes.c                      |   8 ++
 7 files changed, 189 insertions(+), 14 deletions(-)

base-commit: 7a1491b1b0a461872951e8fdf885fc243f67f5e4

Signature · 1 line
2.47.3
author maintainer
#1 lore raw

PATCH 1/11 mm/damon/core: introduce [in]active memory ratio damos quota goal metric

DAMOS_LRU_[DE]PRIO are DAMOS actions for making balance of active and inactive memory size. There is no appropriate DAMOS quota auto-tuning target metric for the use case. Add two new DAMOS quota goal metrics for the purpose, namely DAMOS_QUOTA_[IN]ACTIVE_MEM_BP. Those will represent the ratio of [in]active memory to total (inactive + active) memory. Hence, users will be able to ask DAMON to, for example, "find hot and cold memory, and move pages of those to active and inactive LRU lists, adjusting the hot/cold thresholds aiming 50:50 active:inactive memory ratio."

---
 include/linux/damon.h |  4 ++++
 mm/damon/core.c       | 22 ++++++++++++++++++++++
 2 files changed, 26 insertions(+)
include/linux/damon.h +4 -0
index 650e7ecfa32b..26fb8e90dff6 100644
@@ -155,6 +155,8 @@ enum damos_action {
  * @DAMOS_QUOTA_NODE_MEM_FREE_BP:	MemFree ratio of a node.
  * @DAMOS_QUOTA_NODE_MEMCG_USED_BP:	MemUsed ratio of a node for a cgroup.
  * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP:	MemFree ratio of a node for a cgroup.
+ * @DAMOS_QUOTA_ACTIVE_MEM_BP:		Active to total LRU memory ratio.
+ * @DAMOS_QUOTA_INACTIVE_MEM_BP:	Inactive to total LRU memory ratio.
  * @NR_DAMOS_QUOTA_GOAL_METRICS:	Number of DAMOS quota goal metrics.
  *
  * Metrics equal to larger than @NR_DAMOS_QUOTA_GOAL_METRICS are unsupported.
@@ -166,6 +168,8 @@ enum damos_quota_goal_metric {
 	DAMOS_QUOTA_NODE_MEM_FREE_BP,
 	DAMOS_QUOTA_NODE_MEMCG_USED_BP,
 	DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
+	DAMOS_QUOTA_ACTIVE_MEM_BP,
+	DAMOS_QUOTA_INACTIVE_MEM_BP,
 	NR_DAMOS_QUOTA_GOAL_METRICS,
 };
 
mm/damon/core.c +22 -0
index abda850c28c0..d2ed75e521bd 100644
@@ -2132,6 +2132,23 @@ static unsigned long damos_get_node_memcg_used_bp(
 }
 #endif
 
+/*
+ * Returns LRU-active or inactive memory to total LRU memory size ratio.
+ */
+static unsigned int damos_get_in_active_mem_bp(bool active_ratio)
+{
+	unsigned long active, inactive, total;
+
+	/* This should align with /proc/meminfo output */
+	active = global_node_page_state(NR_LRU_BASE + LRU_ACTIVE_ANON) +
+		global_node_page_state(NR_LRU_BASE + LRU_ACTIVE_FILE);
+	inactive = global_node_page_state(NR_LRU_BASE + LRU_INACTIVE_ANON) +
+		global_node_page_state(NR_LRU_BASE + LRU_INACTIVE_FILE);
+	total = active + inactive;
+	if (active_ratio)
+		return active * 10000 / total;
+	return inactive * 10000 / total;
+}
 
 static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
 {
@@ -2154,6 +2171,11 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)
 	case DAMOS_QUOTA_NODE_MEMCG_FREE_BP:
 		goal->current_value = damos_get_node_memcg_used_bp(goal);
 		break;
+	case DAMOS_QUOTA_ACTIVE_MEM_BP:
+	case DAMOS_QUOTA_INACTIVE_MEM_BP:
+		goal->current_value = damos_get_in_active_mem_bp(
+				goal->metric == DAMOS_QUOTA_ACTIVE_MEM_BP);
+		break;
 	default:
 		break;
 	}
Signature · 1 line
2.47.3
author maintainer
#2 lore raw

PATCH 2/11 mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP

In reply to SeongJae Park

Add support of DAMOS_QUOTA_[IN]ACTIVE_MEM_BP on DAMON sysfs interface. Users can use [in]active_mem_bp keyword input to the target_metric sysfs file to use the new DAMOS quota auto-tune target metrics.

---
 mm/damon/sysfs-schemes.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/damon/sysfs-schemes.c +8 -0
index 419d6e7ee945..2b05a6477188 100644
@@ -1079,6 +1079,14 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = {
 		.metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP,
 		.name = "node_memcg_free_bp",
 	},
+	{
+		.metric = DAMOS_QUOTA_ACTIVE_MEM_BP,
+		.name = "active_mem_bp",
+	},
+	{
+		.metric = DAMOS_QUOTA_INACTIVE_MEM_BP,
+		.name = "inactive_mem_bp",
+	},
 };
 
 static ssize_t target_metric_show(struct kobject *kobj,
Signature · 1 line
2.47.3
Acked-by on v1 2/11
#3 lore raw

Re: [PATCH 02/11] mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP

Quoted 6 lines

Add support of DAMOS_QUOTA_[IN]ACTIVE_MEM_BP on DAMON sysfs interface. Users can use [in]active_mem_bp keyword input to the target_metric sysfs file to use the new DAMOS quota auto-tune target metrics.

Hi SeongJae,

The sysfs interface updates look correct

--
Best Regards,
wang lian

Acked-by on v1 2/11
#4 lore raw

Re: [PATCH 02/11] mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP

In reply to SeongJae Park
Quoted 6 lines

Add support of DAMOS_QUOTA_[IN]ACTIVE_MEM_BP on DAMON sysfs interface. Users can use [in]active_mem_bp keyword input to the target_metric sysfs file to use the new DAMOS quota auto-tune target metrics.

Hi SeongJae,

The sysfs interface updates look correct.

--
Best Regards,
wang lian

author maintainer
#5 lore raw

PATCH 3/11 Docs/mm/damon/design: document DAMOS_QUOTA_[IN]ACTIVE_MEM_BP

In reply to SeongJae Park

Update design document for newly added DAMOS_QUOTA_[IN]ACTIVE_MEM_BP metrics. Note that API document is automatically updated by kernel-doc comment, and the usage document points to the design document which uses keywords same to that for sysfs inputs. Hence updating only design document is sufficient.

---
 Documentation/mm/damon/design.rst | 4 ++++
 1 file changed, 4 insertions(+)
Documentation/mm/damon/design.rst +4 -0
index 7fd819b8bbf7..0cfd4c25e92d 100644
@@ -585,6 +585,10 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to
   specific NUMA node, in bp (1/10,000).
 - ``node_memcg_free_bp``: Specific cgroup's node unused memory ratio for a
   specific NUMA node, in bp (1/10,000).
+- ``active_mem_bp``: Active to active + inactive (LRU) memory size ratio in bp
+  (1/10,000).
+- ``inactive_mem_bp``: Inactive to active + inactive (LRU) memory size ratio in
+  bp (1/10,000).
 
 ``nid`` is optionally required for only ``node_mem_used_bp``,
 ``node_mem_free_bp``, ``node_memcg_used_bp`` and ``node_memcg_free_bp`` to
Signature · 1 line
2.47.3
Acked-by on v1 3/11
#6 lore raw

Re: [PATCH 03/11] Docs/mm/damon/design: document DAMOS_QUOTA_[IN]ACTIVE_MEM_BP

Quoted 8 lines

Update design document for newly added DAMOS_QUOTA_[IN]ACTIVE_MEM_BP metrics. Note that API document is automatically updated by kernel-doc comment, and the usage document points to the design document which uses keywords same to that for sysfs inputs. Hence updating only design document is sufficient.

Hi SeongJae,

The documentation clearly reflects the definition implemented in patch 1.

--
Best Regards,
wang lian

author maintainer
#7 lore raw

PATCH 4/11 mm/damon/paddr: activate DAMOS_LRU_PRIO targets instead of marking accessed

In reply to SeongJae Park

DAMOS_LRU_DEPRIOD directly deactivates the pages, while DAMOS_LRU_PRIO calls folio_mark_accessed(), which does incremental activation. The incremental activation was assumed to be useful for making sure the pages of the hot memory region are really hot. After the introduction of DAMOS_LRU_PRIO, the young page filter has added. Users can use the young page filter to make sure the page is eligible to be activated. Meanwhile, the asymmetric behavior of DAMOS_LRU_[DE]PRIO can confuse users.

Directly activate given pages for DAMOS_LRU_PRIO, to eliminate the unnecessary incremental activation steps, and be symmetric with DAMOS_LRU_DEPRIO for easier usages.

---
 mm/damon/paddr.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)
mm/damon/paddr.c +8 -10
index 7d887a3c0866..4c2c935d82d6 100644
@@ -206,9 +206,9 @@ static unsigned long damon_pa_pageout(struct damon_region *r,
 	return damon_pa_core_addr(applied * PAGE_SIZE, addr_unit);
 }
 
-static inline unsigned long damon_pa_mark_accessed_or_deactivate(
+static inline unsigned long damon_pa_de_activate(
 		struct damon_region *r, unsigned long addr_unit,
-		struct damos *s, bool mark_accessed,
+		struct damos *s, bool activate,
 		unsigned long *sz_filter_passed)
 {
 	phys_addr_t addr, applied = 0;
@@ -227,8 +227,8 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate(
 		else
 			*sz_filter_passed += folio_size(folio) / addr_unit;
 
-		if (mark_accessed)
-			folio_mark_accessed(folio);
+		if (activate)
+			folio_activate(folio);
 		else
 			folio_deactivate(folio);
 		applied += folio_nr_pages(folio);
@@ -240,20 +240,18 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate(
 	return damon_pa_core_addr(applied * PAGE_SIZE, addr_unit);
 }
 
-static unsigned long damon_pa_mark_accessed(struct damon_region *r,
+static unsigned long damon_pa_activate_pages(struct damon_region *r,
 		unsigned long addr_unit, struct damos *s,
 		unsigned long *sz_filter_passed)
 {
-	return damon_pa_mark_accessed_or_deactivate(r, addr_unit, s, true,
-			sz_filter_passed);
+	return damon_pa_de_activate(r, addr_unit, s, true, sz_filter_passed);
 }
 
 static unsigned long damon_pa_deactivate_pages(struct damon_region *r,
 		unsigned long addr_unit, struct damos *s,
 		unsigned long *sz_filter_passed)
 {
-	return damon_pa_mark_accessed_or_deactivate(r, addr_unit, s, false,
-			sz_filter_passed);
+	return damon_pa_de_activate(r, addr_unit, s, false, sz_filter_passed);
 }
 
 static unsigned long damon_pa_migrate(struct damon_region *r,
@@ -327,7 +325,7 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx,
 	case DAMOS_PAGEOUT:
 		return damon_pa_pageout(r, aunit, scheme, sz_filter_passed);
 	case DAMOS_LRU_PRIO:
-		return damon_pa_mark_accessed(r, aunit, scheme,
+		return damon_pa_activate_pages(r, aunit, scheme,
 				sz_filter_passed);
 	case DAMOS_LRU_DEPRIO:
 		return damon_pa_deactivate_pages(r, aunit, scheme,
Signature · 1 line
2.47.3
author maintainer
#8 lore raw

PATCH 5/11 mm/damon/lru_sort: consider age for quota prioritization

In reply to SeongJae Park

DAMON_LRU_SORT is doing under-quota access pattern based regions prioritization using only access frequency. Age of regions is another useful information for distinguishing hot and cold regions. Use it for prioritization, too.

---
 mm/damon/lru_sort.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/damon/lru_sort.c +1 -1
index 9388b091deb7..a74c4ec170a9 100644
@@ -71,7 +71,7 @@ static struct damos_quota damon_lru_sort_quota = {
 	/* Within the quota, mark hotter regions accessed first. */
 	.weight_sz = 0,
 	.weight_nr_accesses = 1,
-	.weight_age = 0,
+	.weight_age = 1,
 };
 DEFINE_DAMON_MODULES_DAMOS_TIME_QUOTA(damon_lru_sort_quota);
 
Signature · 1 line
2.47.3
author maintainer
#9 lore raw

PATCH 6/11 mm/damon/lru_sort: support young page filters

In reply to SeongJae Park

DAMON monitors access patterns at the region level, and hence there could be some page level mismatches. A few hot pages could be located in cold regions, and vice versa. Young page filters can be useful for doing additional page level access checks before applying some DAMOS action.

DAMON_LRU_SORT is not using young page filters, though. Add a parameter for using it.

---
 mm/damon/lru_sort.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
mm/damon/lru_sort.c +41 -0
index a74c4ec170a9..f1fdb37b9b47 100644
@@ -41,6 +41,21 @@ static bool enabled __read_mostly;
 static bool commit_inputs __read_mostly;
 module_param(commit_inputs, bool, 0600);
 
+/*
+ * Filter [non-]young pages accordingly for LRU [de]prioritizations.
+ *
+ * If this is set, check page level access (youngness) once again before each
+ * LRU [de]prioritization operation.  LRU prioritization operation is skipped
+ * if the page has not accessed since the last check (not young).  LRU
+ * deprioritization operation is skipped if the page has accessed since the
+ * last check (young).  The feature is enabled or disabled if this parameter is
+ * set as ``Y`` or ``N``, respectively.
+ *
+ * Disabled by default.
+ */
+static bool filter_young_pages __read_mostly;
+module_param(filter_young_pages, bool, 0600);
+
 /*
  * Access frequency threshold for hot memory regions identification in permil.
  *
@@ -193,6 +208,28 @@ static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thres)
 	return damon_lru_sort_new_scheme(&pattern, DAMOS_LRU_DEPRIO);
 }
 
+static int damon_lru_sort_add_filters(struct damos *hot_scheme,
+		struct damos *cold_scheme)
+{
+	struct damos_filter *filter;
+
+	if (!filter_young_pages)
+		return 0;
+
+	/* disallow prioritizing not-young pages */
+	filter = damos_new_filter(DAMOS_FILTER_TYPE_YOUNG, false, false);
+	if (!filter)
+		return -ENOMEM;
+	damos_add_filter(hot_scheme, filter);
+
+	/* disabllow de-prioritizing young pages */
+	filter = damos_new_filter(DAMOS_FILTER_TYPE_YOUNG, true, false);
+	if (!filter)
+		return -ENOMEM;
+	damos_add_filter(cold_scheme, filter);
+	return 0;
+}
+
 static int damon_lru_sort_apply_parameters(void)
 {
 	struct damon_ctx *param_ctx;
@@ -240,6 +277,10 @@ static int damon_lru_sort_apply_parameters(void)
 	damon_set_schemes(param_ctx, &hot_scheme, 1);
 	damon_add_scheme(param_ctx, cold_scheme);
 
+	err = damon_lru_sort_add_filters(hot_scheme, cold_scheme);
+	if (err)
+		goto out;
+
 	err = damon_set_region_biggest_system_ram_default(param_target,
 					&monitor_region_start,
 					&monitor_region_end,
Signature · 1 line
2.47.3
author maintainer
#10 lore raw

PATCH 7/11 Docs/admin-guide/mm/damon/lru_sort: document filter_young_pages

In reply to SeongJae Park

Document the new DAMON_LRU_SORT parameter, filter_young_pages. It can be used to use page level access re-check for the LRU sorting.

---
 Documentation/admin-guide/mm/damon/lru_sort.rst | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
Documentation/admin-guide/mm/damon/lru_sort.rst +14 -0
index 72a943202676..bb222a32aefd 100644
@@ -79,6 +79,20 @@ of parametrs except ``enabled`` again.  Once the re-reading is done, this
 parameter is set as ``N``.  If invalid parameters are found while the
 re-reading, DAMON_LRU_SORT will be disabled.
 
+filter_young_pages
+------------------
+
+Filter [non-]young pages accordingly for LRU [de]prioritizations.
+
+If this is set, check page level access (youngness) once again before each
+LRU [de]prioritization operation.  LRU prioritization operation is skipped
+if the page has not accessed since the last check (not young).  LRU
+deprioritization operation is skipped if the page has accessed since the
+last check (young).  The feature is enabled or disabled if this parameter is
+set as ``Y`` or ``N``, respectively.
+
+Disabled by default.
+
 hot_thres_access_freq
 ---------------------
 
Signature · 1 line
2.47.3
author maintainer
#12 lore raw

PATCH 8/11 mm/damon/lru_sort: support active:inactive memory ratio based auto-tuning

In reply to SeongJae Park

Doing DAMOS_LRU_[DE]PRIO with DAMOS_QUOTA_[IN]ACTIVE_MEM_BP based quota auto-tuning can be easy and intuitive, compared to the manual [de]prioritization target access pattern thresholds tuning. For example, users can ask DAMON to "find hot/cold pages and activate/deactivate those aiming 50:50 active:inactive memory size." But DAMON_LRU_SORT has no interface to do that. Add a module parameter for setting the target ratio.

---
 mm/damon/lru_sort.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
mm/damon/lru_sort.c +37 -0
index f1fdb37b9b47..f3a9dfc246b6 100644
@@ -41,6 +41,20 @@ static bool enabled __read_mostly;
 static bool commit_inputs __read_mostly;
 module_param(commit_inputs, bool, 0600);
 
+/*
+ * Desired active to [in]active memory ratio in bp (1/10,000).
+ *
+ * While keeping the caps that set by other quotas, DAMON_LRU_SORT
+ * automatically increases and decreases the effective level of the quota
+ * aiming the LRU [de]prioritizations of the hot and cold memory resulting in
+ * this active to [in]active memory ratio.  Value zero means disabling this
+ * auto-tuning feature.
+ *
+ * Disabled by default.
+ */
+static unsigned long active_mem_bp __read_mostly;
+module_param(active_mem_bp, ulong, 0600);
+
 /*
  * Filter [non-]young pages accordingly for LRU [de]prioritizations.
  *
@@ -208,6 +222,26 @@ static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thres)
 	return damon_lru_sort_new_scheme(&pattern, DAMOS_LRU_DEPRIO);
 }
 
+static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme,
+		struct damos *cold_scheme)
+{
+	struct damos_quota_goal *goal;
+
+	if (!active_mem_bp)
+		return 0;
+	goal = damos_new_quota_goal(DAMOS_QUOTA_ACTIVE_MEM_BP, active_mem_bp);
+	if (!goal)
+		return -ENOMEM;
+	damos_add_quota_goal(&hot_scheme->quota, goal);
+	/* aim 0.2 % goal conflict, to keep little ping pong */
+	goal = damos_new_quota_goal(DAMOS_QUOTA_INACTIVE_MEM_BP,
+			10000 - active_mem_bp + 2);
+	if (!goal)
+		return -ENOMEM;
+	damos_add_quota_goal(&hot_scheme->quota, goal);
+	return 0;
+}
+
 static int damon_lru_sort_add_filters(struct damos *hot_scheme,
 		struct damos *cold_scheme)
 {
@@ -277,6 +311,9 @@ static int damon_lru_sort_apply_parameters(void)
 	damon_set_schemes(param_ctx, &hot_scheme, 1);
 	damon_add_scheme(param_ctx, cold_scheme);
 
+	err = damon_lru_sort_add_quota_goals(hot_scheme, cold_scheme);
+	if (err)
+		goto out;
 	err = damon_lru_sort_add_filters(hot_scheme, cold_scheme);
 	if (err)
 		goto out;
Signature · 1 line
2.47.3
#13 lore raw

Re: [PATCH 08/11] mm/damon/lru_sort: support active:inactive memory ratio based auto-tuning

Quoted 10 lines

Doing DAMOS_LRU_[DE]PRIO with DAMOS_QUOTA_[IN]ACTIVE_MEM_BP based quota auto-tuning can be easy and intuitive, compared to the manual [de]prioritization target access pattern thresholds tuning. For example, users can ask DAMON to "find hot/cold pages and activate/deactivate those aiming 50:50 active:inactive memory size." But DAMON_LRU_SORT has no interface to do that. Add a module parameter for setting the target ratio.

```

Quoted 9 lines
mm/damon/lru_sort.c +3 -0
+static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme,
+		struct damos *cold_scheme)
+{
	struct damos_quota_goal *goal;
mm/damon/lru_sort.c +5 -0
+
+	if (!active_mem_bp)
+		return 0;
+	goal = damos_new_quota_goal(DAMOS_QUOTA_ACTIVE_MEM_BP, active_mem_bp);
+	if (!goal)

mm/damon/lru_sort.c static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thres) · line 243

Quoted 10 lines
mm/damon/lru_sort.c +10 -0
+		return -ENOMEM;
+	damos_add_quota_goal(&hot_scheme->quota, goal);
+	/* aim 0.2 % goal conflict, to keep little ping pong */
+	goal = damos_new_quota_goal(DAMOS_QUOTA_INACTIVE_MEM_BP,
+			10000 - active_mem_bp + 2);
+	if (!goal)
+		return -ENOMEM;
+	damos_add_quota_goal(&hot_scheme->quota, goal);
+	return 0;
+}

The second goal (Inactive ratio target) is added to hot_scheme again. Should this be cold_scheme? Currently the cold_scheme parameter appears to be unused.

--
Best Regards,
wang lian

author maintainer
#14 lore raw

Re: [PATCH 08/11] mm/damon/lru_sort: support active:inactive memory ratio based auto-tuning

Quoted 26 lines · On Wed, 14 Jan 2026 13:36:28 +0800 wang lian <[email protected]> wrote:
Quoted 10 lines

Doing DAMOS_LRU_[DE]PRIO with DAMOS_QUOTA_[IN]ACTIVE_MEM_BP based quota auto-tuning can be easy and intuitive, compared to the manual [de]prioritization target access pattern thresholds tuning. For example, users can ask DAMON to "find hot/cold pages and activate/deactivate those aiming 50:50 active:inactive memory size." But DAMON_LRU_SORT has no interface to do that. Add a module parameter for setting the target ratio.

```

Quoted 14 lines
(quoted diff) +3 -0
+static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme,
+		struct damos *cold_scheme)
+{
	struct damos_quota_goal *goal;
(quoted diff) +10 -0
+
+	if (!active_mem_bp)
+		return 0;
+	goal = damos_new_quota_goal(DAMOS_QUOTA_ACTIVE_MEM_BP, active_mem_bp);
+	if (!goal)
+		return -ENOMEM;
+	damos_add_quota_goal(&hot_scheme->quota, goal);
+	/* aim 0.2 % goal conflict, to keep little ping pong */
+	goal = damos_new_quota_goal(DAMOS_QUOTA_INACTIVE_MEM_BP,
+			10000 - active_mem_bp + 2);
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+	if (!goal)
+		return -ENOMEM;
+	damos_add_quota_goal(&hot_scheme->quota, goal);
+	return 0;
+}

The second goal (Inactive ratio target) is added to hot_scheme again. Should this be cold_scheme? Currently the cold_scheme parameter appears to be unused.

You are right, thank you for catching this, Wang!

Andrew, could you please add below attaching fixup?

Thanks,
SJ

[...]
=== >8 ===
From c8dbbf0cfb01de606d47f455bccc2d7b2d6d07ef Mon Sep 17 00:00:00 2001
From: SeongJae Park <[email protected]>
Date: Tue, 13 Jan 2026 21:48:53 -0800
Subject: [PATCH] mm/damon/lru_sort: add inactive mem ratio quota goal to
cold_scheme

damon_lru_sort_add_quota_goals() is adding inactive memory ratio quota goal, which is for cold pages scheme, to hot pages scheme. Fix it.

---
 mm/damon/lru_sort.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/damon/lru_sort.c +1 -1
index f3a9dfc246b6..8af97642912a 100644
@@ -238,7 +238,7 @@ static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme,
 			10000 - active_mem_bp + 2);
 	if (!goal)
 		return -ENOMEM;
-	damos_add_quota_goal(&hot_scheme->quota, goal);
+	damos_add_quota_goal(&cold_scheme->quota, goal);
 	return 0;
 }
 
Signature · 1 line
2.47.3
author maintainer
#15 lore raw

PATCH 9/11 Docs/admin-guide/mm/damon/lru_sort: document active_mem_bp parameter

In reply to SeongJae Park

Document a newly added DAMON_LRU_SORT parameter for doing auto-tuning aiming an active to inactive memory size ratio.

---
 Documentation/admin-guide/mm/damon/lru_sort.rst | 12 ++++++++++++
 1 file changed, 12 insertions(+)
Documentation/admin-guide/mm/damon/lru_sort.rst +12 -0
index bb222a32aefd..6af3ab5579a3 100644
@@ -79,6 +79,18 @@ of parametrs except ``enabled`` again.  Once the re-reading is done, this
 parameter is set as ``N``.  If invalid parameters are found while the
 re-reading, DAMON_LRU_SORT will be disabled.
 
+active_mem_bp
+-------------
+
+Desired active to [in]active memory ratio in bp (1/10,000).
+
+While keeping the caps that set by other quotas, DAMON_LRU_SORT automatically
+increases and decreases the effective level of the quota aiming the LRU
+[de]prioritizations of the hot and cold memory resulting in this active to
+[in]active memory ratio.  Value zero means disabling this auto-tuning feature.
+
+Disabled by default.
+
 filter_young_pages
 ------------------
 
Signature · 1 line
2.47.3
author maintainer
#17 lore raw

PATCH 10/11 mm/damon/lru_sort: add monitoring intervals auto-tuning parameter

In reply to SeongJae Park

DAMON monitoring intervals tuning was crucial for every DAMON use case. Now there are a tuning guideline and an automated intervals tuning feature. DAMON_LRU_SORT is still using manual control of intervals. Add a module parameter for utilizing the auto-tuning feature with a suggested auto-tuning parameters.

---
 mm/damon/lru_sort.c | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)
mm/damon/lru_sort.c +27 -3
index f3a9dfc246b6..717cc585d207 100644
@@ -55,6 +55,20 @@ module_param(commit_inputs, bool, 0600);
 static unsigned long active_mem_bp __read_mostly;
 module_param(active_mem_bp, ulong, 0600);
 
+/*
+ * Auto-tune monitoring intervals.
+ *
+ * If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes
+ * DAMON's sampling and aggregation intervals.  The auto-tuning aims to capture
+ * meaningful amount of access events in each DAMON-snapshot, while keeping the
+ * sampling interval 5 milliseconds in minimum, and 10 seconds in maximum.
+ * Setting this as ``N`` disables the auto-tuning.
+ *
+ * Disabled by default.
+ */
+static bool autotune_monitoring_intervals __read_mostly;
+module_param(autotune_monitoring_intervals, bool, 0600);
+
 /*
  * Filter [non-]young pages accordingly for LRU [de]prioritizations.
  *
@@ -268,6 +282,7 @@ static int damon_lru_sort_apply_parameters(void)
 {
 	struct damon_ctx *param_ctx;
 	struct damon_target *param_target;
+	struct damon_attrs attrs;
 	struct damos *hot_scheme, *cold_scheme;
 	unsigned int hot_thres, cold_thres;
 	int err;
@@ -290,18 +305,27 @@ static int damon_lru_sort_apply_parameters(void)
 		goto out;
 	}
 
-	err = damon_set_attrs(param_ctx, &damon_lru_sort_mon_attrs);
+	attrs = damon_lru_sort_mon_attrs;
+	if (autotune_monitoring_intervals) {
+		attrs.sample_interval = 5000;
+		attrs.aggr_interval = 100000;
+		attrs.intervals_goal.access_bp = 40;
+		attrs.intervals_goal.aggrs = 3;
+		attrs.intervals_goal.min_sample_us = 5000;
+		attrs.intervals_goal.max_sample_us = 10 * 1000 * 1000;
+	}
+	err = damon_set_attrs(param_ctx, &attrs);
 	if (err)
 		goto out;
 
 	err = -ENOMEM;
-	hot_thres = damon_max_nr_accesses(&damon_lru_sort_mon_attrs) *
+	hot_thres = damon_max_nr_accesses(&attrs) *
 		hot_thres_access_freq / 1000;
 	hot_scheme = damon_lru_sort_new_hot_scheme(hot_thres);
 	if (!hot_scheme)
 		goto out;
 
-	cold_thres = cold_min_age / damon_lru_sort_mon_attrs.aggr_interval;
+	cold_thres = cold_min_age / attrs.aggr_interval;
 	cold_scheme = damon_lru_sort_new_cold_scheme(cold_thres);
 	if (!cold_scheme) {
 		damon_destroy_scheme(hot_scheme);
Signature · 1 line
2.47.3
Acked-by on v1 10/11
#18 lore raw

Re: [PATCH 10/11] mm/damon/lru_sort: add monitoring intervals auto-tuning parameter

Quoted 8 lines

DAMON monitoring intervals tuning was crucial for every DAMON use case. Now there are a tuning guideline and an automated intervals tuning feature. DAMON_LRU_SORT is still using manual control of intervals. Add a module parameter for utilizing the auto-tuning feature with a suggested auto-tuning parameters.

--
Best Regards,
wang lian

author maintainer
#19 lore raw

PATCH 11/11 Docs/admin-guide/mm/damon/lru_sort: document intervals autotuning

In reply to SeongJae Park

Document a newly added DAMON_LRU_SORT module parameter for using monitoring intervals auto-tuning feature of DAMON.

---
 Documentation/admin-guide/mm/damon/lru_sort.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/mm/damon/lru_sort.rst +11 -0
index 6af3ab5579a3..20a8378d5a94 100644
@@ -91,6 +91,17 @@ increases and decreases the effective level of the quota aiming the LRU
 
 Disabled by default.
 
+Auto-tune monitoring intervals
+------------------------------
+
+If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes DAMON's
+sampling and aggregation intervals.  The auto-tuning aims to capture meaningful
+amount of access events in each DAMON-snapshot, while keeping the sampling
+interval 5 milliseconds in minimum, and 10 seconds in maximum.  Setting this as
+``N`` disables the auto-tuning.
+
+Disabled by default.
+
 filter_young_pages
 ------------------
 
Signature · 1 line
2.47.3