mm/damon: advance DAMOS-based LRU sorting
Subsystems (4)
SeongJae Park
maintainercover mm/damon: advance DAMOS-based LRU sorting
DAMOS_LRU_[DE]PRIO actions were added to DAMOS for more access-aware LRU lists sorting. For simple usage, a specialized kernel module, namely DAMON_LRU_SORT, has also been introduced. After the introduction of the module, DAMON got a few important new features, including the aim-based quota auto-tuning, age tracking, young page filter, and monitoring intervals auto-tuning. Meanwhile, DAMOS-based LRU sorting had no direct updates. Now we show some rooms to advance for DAMOS-based LRU sorting.
Firstly, the aim-oriented quota auto-tuning can simplify the LRU sorting parameters tuning. But there is no good auto-tuning target metric for LRU sorting use case. Secondly, the behavior of DAMOS_LRU_[DE]PRIO are not very symmetric. DAMOS_LRU_DEPRIO directly moves the pages to inactive LRU list, while DAMOS_LRU_PRIO only marks the page as accessed, so that the page can not directly but only eventually moved to the active LRU list. Finally, DAMON_LRU_SORT users cannot utilize the modern features that can be useful for them, too.
Improve the situation with the following changes. First, introduce a new DAMOS quota auto-tuning target metric for active:inactive memory size ratio. Since LRU sorting is a kind of balancing of active and inactive pages, the active:inactive memory size ratio can be intuitively set. Second, update DAMOS_LRU_[DE]PRIO behaviors to be more intuitive and symmetric, by letting them directly move the pages to [in]active LRU list. Third, update the DAMON_LRU_SORT module user interface to be able to fully utilize the modern features including the [in]active memory size ratio-based quota auto-tuning, young page filter, and monitoring intervals auto-tuning.
With these changes, for example, users can now ask DAMON to "find hot/cold memory regions with auto-tuned monitoring intervals, do one more page level access check for found hot/cold memory, and move pages of those to active or inactive LRU lists accordingly, aiming X:Y active to inactive memory ratio." For example, if they know 30% of the memory is better to be protected from reclamation, 30:70 can be set as the target ratio.
Test Results
------------
I ran DAMON_LRU_SORT with the features introduced by this series, on a real world server workload. For the active:inactive ratio goal, I set 50:50. I confirmed it achieves the target active:inactive ratio, without manual tuning of the monitoring intervals and the hot/coldness thresholds. The baseline system that was not running the DAMON_LRU_SORT was keeping active:inactive ratio of about 1:10.
Note that the test didn't show a clear performance difference, though. I believe that was mainly because the workload was not very memory intensive. Also, whether the 50:50 target ratio was optimum is unclear. Nonetheless, the positive performance impact of the basic LRU sorting idea is already confirmed with the initial DAMON_LRU_SORT introduction patch series. The goal of this patch series is simplifying the parameters tuning of DAMOS-based LRU sorting, and the test confirmed the aimed goals are achieved.
Patches Sequence
----------------
First three patches extend DAMOS quota auto-tuning to support [in]active memory ratio target metric type. Those (patches 1-3) introduce new metrics, implement DAMON sysfs support, and update the documentation, respectively.
Following patch (patch 4) makes DAMOS_LRU_PRIO action to directly move target pages to active LRU list, instead of only marking them accessed.
Following seven patches (patches 5-11) updates DAMON_LRU_SORT to support modern DAMON features. Patch 5 makes it uses not only access frequency but also age at under-quota regions prioritization. Patches 6-11 add the support for young page filtering, active:inactive memory ratio based quota auto-tuning, and monitoring intervals auto-tuning, with appropriate document updates.
Changes from RFC
(https://lore.kernel.org/[email protected])
- rebase to latest mm-new
- add test results on the cover letter
- minor wordsmithing and typo fixes
SeongJae Park (11):
mm/damon/core: introduce [in]active memory ratio damos quota goal
metric
mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP Docs/mm/damon/design: document DAMOS_QUOTA_[IN]ACTIVE_MEM_BP mm/damon/paddr: activate DAMOS_LRU_PRIO targets instead of marking
accessed
mm/damon/lru_sort: consider age for quota prioritization mm/damon/lru_sort: support young page filters Docs/admin-guide/mm/damon/lru_sort: document filter_young_pages mm/damon/lru_sort: support active:inactive memory ratio based
auto-tuning
Docs/admin-guide/mm/damon/lru_sort: document active_mem_bp parameter mm/damon/lru_sort: add monitoring intervals auto-tuning parameter Docs/admin-guide/mm/damon/lru_sort: document intervals autotuning
.../admin-guide/mm/damon/lru_sort.rst | 37 ++++++ Documentation/mm/damon/design.rst | 4 + include/linux/damon.h | 4 + mm/damon/core.c | 22 ++++ mm/damon/lru_sort.c | 110 +++++++++++++++++- mm/damon/paddr.c | 18 ++- mm/damon/sysfs-schemes.c | 8 ++ 7 files changed, 189 insertions(+), 14 deletions(-)
base-commit: 7a1491b1b0a461872951e8fdf885fc243f67f5e4
Signature · 1 line
2.47.3
SeongJae Park
maintainerPATCH 1/11 mm/damon/core: introduce [in]active memory ratio damos quota goal metric
DAMOS_LRU_[DE]PRIO are DAMOS actions for making balance of active and inactive memory size. There is no appropriate DAMOS quota auto-tuning target metric for the use case. Add two new DAMOS quota goal metrics for the purpose, namely DAMOS_QUOTA_[IN]ACTIVE_MEM_BP. Those will represent the ratio of [in]active memory to total (inactive + active) memory. Hence, users will be able to ask DAMON to, for example, "find hot and cold memory, and move pages of those to active and inactive LRU lists, adjusting the hot/cold thresholds aiming 50:50 active:inactive memory ratio."
--- include/linux/damon.h | 4 ++++ mm/damon/core.c | 22 ++++++++++++++++++++++ 2 files changed, 26 insertions(+)
include/linux/damon.h +4 -0
@@ -155,6 +155,8 @@ enum damos_action {* @DAMOS_QUOTA_NODE_MEM_FREE_BP: MemFree ratio of a node. * @DAMOS_QUOTA_NODE_MEMCG_USED_BP: MemUsed ratio of a node for a cgroup. * @DAMOS_QUOTA_NODE_MEMCG_FREE_BP: MemFree ratio of a node for a cgroup. +* @DAMOS_QUOTA_ACTIVE_MEM_BP: Active to total LRU memory ratio. +* @DAMOS_QUOTA_INACTIVE_MEM_BP: Inactive to total LRU memory ratio. * @NR_DAMOS_QUOTA_GOAL_METRICS: Number of DAMOS quota goal metrics. * * Metrics equal to larger than @NR_DAMOS_QUOTA_GOAL_METRICS are unsupported.
@@ -166,6 +168,8 @@ enum damos_quota_goal_metric {DAMOS_QUOTA_NODE_MEM_FREE_BP ,DAMOS_QUOTA_NODE_MEMCG_USED_BP ,DAMOS_QUOTA_NODE_MEMCG_FREE_BP , +DAMOS_QUOTA_ACTIVE_MEM_BP , +DAMOS_QUOTA_INACTIVE_MEM_BP , NR_DAMOS_QUOTA_GOAL_METRICS, };
mm/damon/core.c +22 -0
@@ -2132,6 +2132,23 @@ static unsigned long damos_get_node_memcg_used_bp( }#endif +/* +* Returns LRU-active or inactive memory to total LRU memory size ratio. +*/ +static unsigned int damos_get_in_active_mem_bp (bool active_ratio) +{ +unsigned long active, inactive, total; + +/* This should align with /proc/meminfo output */ + active =global_node_page_state (NR_LRU_BASE +LRU_ACTIVE_ANON ) + +global_node_page_state (NR_LRU_BASE +LRU_ACTIVE_FILE ); + inactive =global_node_page_state (NR_LRU_BASE +LRU_INACTIVE_ANON ) + +global_node_page_state (NR_LRU_BASE +LRU_INACTIVE_FILE ); + total = active + inactive; +if (active_ratio) +return active *10000 / total; +return inactive *10000 / total; +}static void damos_set_quota_goal_current_value (struct damos_quota_goal *goal) {
@@ -2154,6 +2171,11 @@ static void damos_set_quota_goal_current_value(struct damos_quota_goal *goal)case DAMOS_QUOTA_NODE_MEMCG_FREE_BP : goal->current_value =damos_get_node_memcg_used_bp (goal);break ; +case DAMOS_QUOTA_ACTIVE_MEM_BP : +case DAMOS_QUOTA_INACTIVE_MEM_BP : + goal->current_value =damos_get_in_active_mem_bp ( + goal->metric ==DAMOS_QUOTA_ACTIVE_MEM_BP ); +break ;default :break ; }
Signature · 1 line
2.47.3
SeongJae Park
maintainerPATCH 2/11 mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP
In reply to SeongJae ParkAdd support of DAMOS_QUOTA_[IN]ACTIVE_MEM_BP on DAMON sysfs interface. Users can use [in]active_mem_bp keyword input to the target_metric sysfs file to use the new DAMOS quota auto-tune target metrics.
--- mm/damon/sysfs-schemes.c | 8 ++++++++ 1 file changed, 8 insertions(+)
mm/damon/sysfs-schemes.c +8 -0
@@ -1079,6 +1079,14 @@ struct damos_sysfs_qgoal_metric_name damos_sysfs_qgoal_metric_names[] = { .metric = DAMOS_QUOTA_NODE_MEMCG_FREE_BP, .name = "node_memcg_free_bp", }, + { + .metric = DAMOS_QUOTA_ACTIVE_MEM_BP, + .name = "active_mem_bp", + }, + { + .metric = DAMOS_QUOTA_INACTIVE_MEM_BP, + .name = "inactive_mem_bp", + }, }; static ssize_t target_metric_show(struct kobject *kobj,
Signature · 1 line
2.47.3
Re: [PATCH 02/11] mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP
Quoted 6 lines
Add support of DAMOS_QUOTA_[IN]ACTIVE_MEM_BP on DAMON sysfs interface. Users can use [in]active_mem_bp keyword input to the
target_metricsysfs file to use the new DAMOS quota auto-tune target metrics.Signed-off-by SeongJae Park <[email protected]>
Hi SeongJae,
The sysfs interface updates look correct
--
Best Regards,
wang lian
Re: [PATCH 02/11] mm/damon/sysfs-schemes: support DAMOS_QUOTA_[IN]ACTIVE_MEM_BP
In reply to SeongJae ParkQuoted 6 lines
Add support of DAMOS_QUOTA_[IN]ACTIVE_MEM_BP on DAMON sysfs interface. Users can use [in]active_mem_bp keyword input to the
target_metricsysfs file to use the new DAMOS quota auto-tune target metrics.Signed-off-by SeongJae Park <[email protected]>
Hi SeongJae,
The sysfs interface updates look correct.
--
Best Regards,
wang lian
SeongJae Park
maintainerPATCH 3/11 Docs/mm/damon/design: document DAMOS_QUOTA_[IN]ACTIVE_MEM_BP
In reply to SeongJae ParkUpdate design document for newly added DAMOS_QUOTA_[IN]ACTIVE_MEM_BP metrics. Note that API document is automatically updated by kernel-doc comment, and the usage document points to the design document which uses keywords same to that for sysfs inputs. Hence updating only design document is sufficient.
--- Documentation/mm/damon/design.rst | 4 ++++ 1 file changed, 4 insertions(+)
Documentation/mm/damon/design.rst +4 -0
@@ -585,6 +585,10 @@ mechanism tries to make ``current_value`` of ``target_metric`` be same to specific NUMA node, in bp (1/10,000). - ``node_memcg_free_bp``: Specific cgroup's node unused memory ratio for a specific NUMA node, in bp (1/10,000). +- ``active_mem_bp``: Active to active + inactive (LRU) memory size ratio in bp + (1/10,000). +- ``inactive_mem_bp``: Inactive to active + inactive (LRU) memory size ratio in + bp (1/10,000). ``nid`` is optionally required for only ``node_mem_used_bp``, ``node_mem_free_bp``, ``node_memcg_used_bp`` and ``node_memcg_free_bp`` to
Signature · 1 line
2.47.3
Re: [PATCH 03/11] Docs/mm/damon/design: document DAMOS_QUOTA_[IN]ACTIVE_MEM_BP
Quoted 8 lines
Update design document for newly added DAMOS_QUOTA_[IN]ACTIVE_MEM_BP metrics. Note that API document is automatically updated by kernel-doc comment, and the usage document points to the design document which uses keywords same to that for sysfs inputs. Hence updating only design document is sufficient.
Signed-off-by SeongJae Park <[email protected]>
Hi SeongJae,
The documentation clearly reflects the definition implemented in patch 1.
--
Best Regards,
wang lian
SeongJae Park
maintainerPATCH 4/11 mm/damon/paddr: activate DAMOS_LRU_PRIO targets instead of marking accessed
In reply to SeongJae ParkDAMOS_LRU_DEPRIOD directly deactivates the pages, while DAMOS_LRU_PRIO calls folio_mark_accessed(), which does incremental activation. The incremental activation was assumed to be useful for making sure the pages of the hot memory region are really hot. After the introduction of DAMOS_LRU_PRIO, the young page filter has added. Users can use the young page filter to make sure the page is eligible to be activated. Meanwhile, the asymmetric behavior of DAMOS_LRU_[DE]PRIO can confuse users.
Directly activate given pages for DAMOS_LRU_PRIO, to eliminate the unnecessary incremental activation steps, and be symmetric with DAMOS_LRU_DEPRIO for easier usages.
--- mm/damon/paddr.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
mm/damon/paddr.c +8 -10
@@ -206,9 +206,9 @@ static unsigned long damon_pa_pageout(struct damon_region *r,return damon_pa_core_addr (applied *PAGE_SIZE , addr_unit); } -static inline unsigned long damon_pa_mark_accessed_or_deactivate ( +static inline unsigned long damon_pa_de_activate (struct damon_region *r,unsigned long addr_unit, -struct damos *s,bool mark_accessed, +struct damos *s,bool activate,unsigned long *sz_filter_passed) {phys_addr_t addr, applied =0 ;
@@ -227,8 +227,8 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate(else *sz_filter_passed +=folio_size (folio) / addr_unit; -if (mark_accessed) -folio_mark_accessed (folio); +if (activate) +folio_activate (folio);else folio_deactivate (folio); applied +=folio_nr_pages (folio);
@@ -240,20 +240,18 @@ static inline unsigned long damon_pa_mark_accessed_or_deactivate(return damon_pa_core_addr (applied *PAGE_SIZE , addr_unit); } -static unsigned long damon_pa_mark_accessed (struct damon_region *r, +static unsigned long damon_pa_activate_pages (struct damon_region *r,unsigned long addr_unit,struct damos *s,unsigned long *sz_filter_passed) { -return damon_pa_mark_accessed_or_deactivate (r, addr_unit, s, true, - sz_filter_passed); +return damon_pa_de_activate (r, addr_unit, s, true, sz_filter_passed); }static unsigned long damon_pa_deactivate_pages (struct damon_region *r,unsigned long addr_unit,struct damos *s,unsigned long *sz_filter_passed) { -return damon_pa_mark_accessed_or_deactivate (r, addr_unit, s, false, - sz_filter_passed); +return damon_pa_de_activate (r, addr_unit, s, false, sz_filter_passed); } static unsigned long damon_pa_migrate(struct damon_region *r,
@@ -327,7 +325,7 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx,case DAMOS_PAGEOUT :return damon_pa_pageout (r, aunit, scheme, sz_filter_passed);case DAMOS_LRU_PRIO : -return damon_pa_mark_accessed (r, aunit, scheme, +return damon_pa_activate_pages (r, aunit, scheme, sz_filter_passed);case DAMOS_LRU_DEPRIO : return damon_pa_deactivate_pages(r, aunit, scheme,
Signature · 1 line
2.47.3
SeongJae Park
maintainerPATCH 5/11 mm/damon/lru_sort: consider age for quota prioritization
In reply to SeongJae ParkDAMON_LRU_SORT is doing under-quota access pattern based regions prioritization using only access frequency. Age of regions is another useful information for distinguishing hot and cold regions. Use it for prioritization, too.
--- mm/damon/lru_sort.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/damon/lru_sort.c +1 -1
@@ -71,7 +71,7 @@ static struct damos_quota damon_lru_sort_quota = {/* Within the quota, mark hotter regions accessed first. */ .weight_sz = 0, .weight_nr_accesses = 1, - .weight_age = 0, + .weight_age = 1, };DEFINE_DAMON_MODULES_DAMOS_TIME_QUOTA (damon_lru_sort_quota);
Signature · 1 line
2.47.3
SeongJae Park
maintainerPATCH 6/11 mm/damon/lru_sort: support young page filters
In reply to SeongJae ParkDAMON monitors access patterns at the region level, and hence there could be some page level mismatches. A few hot pages could be located in cold regions, and vice versa. Young page filters can be useful for doing additional page level access checks before applying some DAMOS action.
DAMON_LRU_SORT is not using young page filters, though. Add a parameter for using it.
--- mm/damon/lru_sort.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
mm/damon/lru_sort.c +41 -0
@@ -41,6 +41,21 @@ static bool enabled __read_mostly;static bool commit_inputs__read_mostly ;module_param (commit_inputs, bool,0600 ); +/* +* Filter [non-]young pages accordingly for LRU [de]prioritizations. +* +* If this is set, check page level access (youngness) once again before each +* LRU [de]prioritization operation. LRU prioritization operation is skipped +* if the page has not accessed since the last check (not young). LRU +* deprioritization operation is skipped if the page has accessed since the +* last check (young). The feature is enabled or disabled if this parameter is +* set as ``Y`` or ``N``, respectively. +* +* Disabled by default. +*/ +static bool filter_young_pages__read_mostly ; +module_param (filter_young_pages, bool,0600 ); +/* * Access frequency threshold for hot memory regions identification in permil. *
@@ -193,6 +208,28 @@ static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thres)return damon_lru_sort_new_scheme (&pattern,DAMOS_LRU_DEPRIO ); } +static int damon_lru_sort_add_filters (struct damos *hot_scheme, +struct damos *cold_scheme) +{ +struct damos_filter *filter; + +if (!filter_young_pages) +return 0 ; + +/* disallow prioritizing not-young pages */ + filter =damos_new_filter (DAMOS_FILTER_TYPE_YOUNG , false, false); +if (!filter) +return -ENOMEM ; +damos_add_filter (hot_scheme, filter); + +/* disabllow de-prioritizing young pages */ + filter =damos_new_filter (DAMOS_FILTER_TYPE_YOUNG , true, false); +if (!filter) +return -ENOMEM ; +damos_add_filter (cold_scheme, filter); +return 0 ; +} +static int damon_lru_sort_apply_parameters (void ) {struct damon_ctx *param_ctx;
@@ -240,6 +277,10 @@ static int damon_lru_sort_apply_parameters(void)damon_set_schemes (param_ctx, &hot_scheme,1 );damon_add_scheme (param_ctx, cold_scheme); + err =damon_lru_sort_add_filters (hot_scheme, cold_scheme); +if (err) + goto out; + err = damon_set_region_biggest_system_ram_default(param_target, &monitor_region_start, &monitor_region_end,
Signature · 1 line
2.47.3
SeongJae Park
maintainerPATCH 7/11 Docs/admin-guide/mm/damon/lru_sort: document filter_young_pages
In reply to SeongJae ParkDocument the new DAMON_LRU_SORT parameter, filter_young_pages. It can be used to use page level access re-check for the LRU sorting.
--- Documentation/admin-guide/mm/damon/lru_sort.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
Documentation/admin-guide/mm/damon/lru_sort.rst +14 -0
@@ -79,6 +79,20 @@ of parametrs except ``enabled`` again. Once the re-reading is done, this parameter is set as ``N``. If invalid parameters are found while the re-reading, DAMON_LRU_SORT will be disabled. +filter_young_pages +------------------ + +Filter [non-]young pages accordingly for LRU [de]prioritizations. + +If this is set, check page level access (youngness) once again before each +LRU [de]prioritization operation. LRU prioritization operation is skipped +if the page has not accessed since the last check (not young). LRU +deprioritization operation is skipped if the page has accessed since the +last check (young). The feature is enabled or disabled if this parameter is +set as ``Y`` or ``N``, respectively. + +Disabled by default. + hot_thres_access_freq ---------------------
Signature · 1 line
2.47.3
Re: [PATCH 07/11] Docs/admin-guide/mm/damon/lru_sort: document filter_young_pages
Quoted 5 lines
Document the new
DAMON_LRU_SORTparameter,filter_young_pages. It can be used to use page level access re-check for the LRU sorting.Signed-off-by SeongJae Park <[email protected]>
--
Best Regards,
wang lian
SeongJae Park
maintainerPATCH 8/11 mm/damon/lru_sort: support active:inactive memory ratio based auto-tuning
In reply to SeongJae ParkDoing DAMOS_LRU_[DE]PRIO with DAMOS_QUOTA_[IN]ACTIVE_MEM_BP based quota auto-tuning can be easy and intuitive, compared to the manual [de]prioritization target access pattern thresholds tuning. For example, users can ask DAMON to "find hot/cold pages and activate/deactivate those aiming 50:50 active:inactive memory size." But DAMON_LRU_SORT has no interface to do that. Add a module parameter for setting the target ratio.
--- mm/damon/lru_sort.c | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
mm/damon/lru_sort.c +37 -0
@@ -41,6 +41,20 @@ static bool enabled __read_mostly;static bool commit_inputs__read_mostly ;module_param (commit_inputs, bool,0600 ); +/* +* Desired active to [in]active memory ratio in bp (1/10,000). +* +* While keeping the caps that set by other quotas, DAMON_LRU_SORT +* automatically increases and decreases the effective level of the quota +* aiming the LRU [de]prioritizations of the hot and cold memory resulting in +* this active to [in]active memory ratio. Value zero means disabling this +* auto-tuning feature. +* +* Disabled by default. +*/ +static unsigned long active_mem_bp__read_mostly ; +module_param (active_mem_bp, ulong,0600 ); +/* * Filter [non-]young pages accordingly for LRU [de]prioritizations. *
@@ -208,6 +222,26 @@ static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thres)return damon_lru_sort_new_scheme (&pattern,DAMOS_LRU_DEPRIO ); } +static int damon_lru_sort_add_quota_goals (struct damos *hot_scheme, +struct damos *cold_scheme) +{ +struct damos_quota_goal *goal; + +if (!active_mem_bp) +return 0 ; + goal =damos_new_quota_goal (DAMOS_QUOTA_ACTIVE_MEM_BP , active_mem_bp); +if (!goal) +return -ENOMEM ; +damos_add_quota_goal (&hot_scheme->quota, goal); +/* aim 0.2 % goal conflict, to keep little ping pong */ + goal =damos_new_quota_goal (DAMOS_QUOTA_INACTIVE_MEM_BP , +10000 - active_mem_bp +2 ); +if (!goal) +return -ENOMEM ; +damos_add_quota_goal (&hot_scheme->quota, goal); +return 0 ; +} +static int damon_lru_sort_add_filters (struct damos *hot_scheme,struct damos *cold_scheme) {
@@ -277,6 +311,9 @@ static int damon_lru_sort_apply_parameters(void)damon_set_schemes (param_ctx, &hot_scheme,1 );damon_add_scheme (param_ctx, cold_scheme); + err =damon_lru_sort_add_quota_goals (hot_scheme, cold_scheme); +if (err) + goto out; err =damon_lru_sort_add_filters (hot_scheme, cold_scheme);if (err) goto out;
Signature · 1 line
2.47.3
Re: [PATCH 08/11] mm/damon/lru_sort: support active:inactive memory ratio based auto-tuning
Quoted 10 lines
Doing DAMOS_LRU_[DE]PRIO with DAMOS_QUOTA_[IN]ACTIVE_MEM_BP based quota auto-tuning can be easy and intuitive, compared to the manual [de]prioritization target access pattern thresholds tuning. For example, users can ask DAMON to "find hot/cold pages and activate/deactivate those aiming 50:50 active:inactive memory size." But
DAMON_LRU_SORThas no interface to do that. Add a module parameter for setting the target ratio.Signed-off-by SeongJae Park <[email protected]>
```
Quoted 9 lines
mm/damon/lru_sort.c +3
-0+static int damon_lru_sort_add_quota_goals (struct damos *hot_scheme, +struct damos *cold_scheme) +{struct damos_quota_goal *goal;mm/damon/lru_sort.c +5
-0+ +if (!active_mem_bp) +return 0 ; + goal =damos_new_quota_goal (DAMOS_QUOTA_ACTIVE_MEM_BP , active_mem_bp); +if (!goal)
mm/damon/lru_sort.c static struct damos *damon_lru_sort_new_cold_scheme(unsigned int cold_thres) · line 243
Quoted 10 lines
mm/damon/lru_sort.c +10
-0+return -ENOMEM ; +damos_add_quota_goal (&hot_scheme->quota, goal); +/* aim 0.2 % goal conflict, to keep little ping pong */ + goal =damos_new_quota_goal (DAMOS_QUOTA_INACTIVE_MEM_BP , +10000 - active_mem_bp +2 ); +if (!goal) +return -ENOMEM ; +damos_add_quota_goal (&hot_scheme->quota, goal); +return 0 ; +}
The second goal (Inactive ratio target) is added to hot_scheme again. Should this be cold_scheme? Currently the cold_scheme parameter appears to be unused.
--
Best Regards,
wang lian
SeongJae Park
maintainerRe: [PATCH 08/11] mm/damon/lru_sort: support active:inactive memory ratio based auto-tuning
Quoted 26 lines · On Wed, 14 Jan 2026 13:36:28 +0800 wang lian <[email protected]> wrote:
Quoted 10 lines
Doing DAMOS_LRU_[DE]PRIO with DAMOS_QUOTA_[IN]ACTIVE_MEM_BP based quota auto-tuning can be easy and intuitive, compared to the manual [de]prioritization target access pattern thresholds tuning. For example, users can ask DAMON to "find hot/cold pages and activate/deactivate those aiming 50:50 active:inactive memory size." But
DAMON_LRU_SORThas no interface to do that. Add a module parameter for setting the target ratio.Signed-off-by SeongJae Park <[email protected]>```
Quoted 14 lines
(quoted diff) +3
-0+static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme, + struct damos *cold_scheme) +{struct damos_quota_goal *goal;(quoted diff) +10
-0+ + if (!active_mem_bp) + return 0; + goal = damos_new_quota_goal(DAMOS_QUOTA_ACTIVE_MEM_BP, active_mem_bp); + if (!goal) + return -ENOMEM; + damos_add_quota_goal(&hot_scheme->quota, goal); + /* aim 0.2 % goal conflict, to keep little ping pong */ + goal = damos_new_quota_goal(DAMOS_QUOTA_INACTIVE_MEM_BP, + 10000 - active_mem_bp + 2);
Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ if (!goal) + return -ENOMEM; + damos_add_quota_goal(&hot_scheme->quota, goal); + return 0; +}The second goal (Inactive ratio target) is added to
hot_schemeagain. Should this becold_scheme? Currently thecold_schemeparameter appears to be unused.
You are right, thank you for catching this, Wang!
Andrew, could you please add below attaching fixup?
Thanks,
SJ
[...]
=== >8 ===
From c8dbbf0cfb01de606d47f455bccc2d7b2d6d07ef Mon Sep 17 00:00:00 2001
From: SeongJae Park <[email protected]>
Date: Tue, 13 Jan 2026 21:48:53 -0800
Subject: [PATCH] mm/damon/lru_sort: add inactive mem ratio quota goal to
cold_scheme
damon_lru_sort_add_quota_goals() is adding inactive memory ratio quota goal, which is for cold pages scheme, to hot pages scheme. Fix it.
--- mm/damon/lru_sort.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/damon/lru_sort.c +1 -1
@@ -238,7 +238,7 @@ static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme, 10000 - active_mem_bp + 2);if (!goal)return -ENOMEM ; -damos_add_quota_goal (&hot_scheme->quota, goal); +damos_add_quota_goal (&cold_scheme->quota, goal);return 0 ; }
Signature · 1 line
2.47.3
SeongJae Park
maintainerPATCH 9/11 Docs/admin-guide/mm/damon/lru_sort: document active_mem_bp parameter
In reply to SeongJae ParkDocument a newly added DAMON_LRU_SORT parameter for doing auto-tuning aiming an active to inactive memory size ratio.
--- Documentation/admin-guide/mm/damon/lru_sort.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+)
Documentation/admin-guide/mm/damon/lru_sort.rst +12 -0
@@ -79,6 +79,18 @@ of parametrs except ``enabled`` again. Once the re-reading is done, this parameter is set as ``N``. If invalid parameters are found while the re-reading, DAMON_LRU_SORT will be disabled. +active_mem_bp +------------- + +Desired active to [in]active memory ratio in bp (1/10,000). + +While keeping the caps that set by other quotas, DAMON_LRU_SORT automatically +increases and decreases the effective level of the quota aiming the LRU +[de]prioritizations of the hot and cold memory resulting in this active to +[in]active memory ratio. Value zero means disabling this auto-tuning feature. + +Disabled by default. + filter_young_pages ------------------
Signature · 1 line
2.47.3
Re: [PATCH 09/11] Docs/admin-guide/mm/damon/lru_sort: document active_mem_bp parameter
Quoted 5 lines
Document a newly added
DAMON_LRU_SORTparameter for doing auto-tuning aiming an active to inactive memory size ratio.Signed-off-by SeongJae Park <[email protected]>
--
Best Regards,
wang lian
SeongJae Park
maintainerPATCH 10/11 mm/damon/lru_sort: add monitoring intervals auto-tuning parameter
In reply to SeongJae ParkDAMON monitoring intervals tuning was crucial for every DAMON use case. Now there are a tuning guideline and an automated intervals tuning feature. DAMON_LRU_SORT is still using manual control of intervals. Add a module parameter for utilizing the auto-tuning feature with a suggested auto-tuning parameters.
--- mm/damon/lru_sort.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-)
mm/damon/lru_sort.c +27 -3
@@ -55,6 +55,20 @@ module_param(commit_inputs, bool, 0600);static unsigned long active_mem_bp__read_mostly ;module_param (active_mem_bp, ulong,0600 ); +/* +* Auto-tune monitoring intervals. +* +* If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes +* DAMON's sampling and aggregation intervals. The auto-tuning aims to capture +* meaningful amount of access events in each DAMON-snapshot, while keeping the +* sampling interval 5 milliseconds in minimum, and 10 seconds in maximum. +* Setting this as ``N`` disables the auto-tuning. +* +* Disabled by default. +*/ +static bool autotune_monitoring_intervals__read_mostly ; +module_param (autotune_monitoring_intervals, bool,0600 ); +/* * Filter [non-]young pages accordingly for LRU [de]prioritizations. *
@@ -268,6 +282,7 @@ static int damon_lru_sort_apply_parameters(void) {struct damon_ctx *param_ctx;struct damon_target *param_target; +struct damon_attrs attrs;struct damos *hot_scheme, *cold_scheme;unsigned int hot_thres, cold_thres;int err;
@@ -290,18 +305,27 @@ static int damon_lru_sort_apply_parameters(void) goto out; } - err =damon_set_attrs (param_ctx, &damon_lru_sort_mon_attrs); + attrs = damon_lru_sort_mon_attrs; +if (autotune_monitoring_intervals) { + attrs.sample_interval =5000 ; + attrs.aggr_interval =100000 ; + attrs.intervals_goal.access_bp =40 ; + attrs.intervals_goal.aggrs =3 ; + attrs.intervals_goal.min_sample_us =5000 ; + attrs.intervals_goal.max_sample_us =10 *1000 *1000 ; + } + err =damon_set_attrs (param_ctx, &attrs);if (err) goto out; err = -ENOMEM ; - hot_thres =damon_max_nr_accesses (&damon_lru_sort_mon_attrs) * + hot_thres =damon_max_nr_accesses (&attrs) * hot_thres_access_freq /1000 ; hot_scheme =damon_lru_sort_new_hot_scheme (hot_thres);if (!hot_scheme) goto out; - cold_thres = cold_min_age / damon_lru_sort_mon_attrs.aggr_interval; + cold_thres = cold_min_age / attrs.aggr_interval; cold_scheme =damon_lru_sort_new_cold_scheme (cold_thres);if (!cold_scheme) {damon_destroy_scheme (hot_scheme);
Signature · 1 line
2.47.3
Re: [PATCH 10/11] mm/damon/lru_sort: add monitoring intervals auto-tuning parameter
Quoted 8 lines
DAMON monitoring intervals tuning was crucial for every DAMON use case. Now there are a tuning guideline and an automated intervals tuning feature.
DAMON_LRU_SORTis still using manual control of intervals. Add a module parameter for utilizing the auto-tuning feature with a suggested auto-tuning parameters.Signed-off-by SeongJae Park <[email protected]>
--
Best Regards,
wang lian
SeongJae Park
maintainerPATCH 11/11 Docs/admin-guide/mm/damon/lru_sort: document intervals autotuning
In reply to SeongJae ParkDocument a newly added DAMON_LRU_SORT module parameter for using monitoring intervals auto-tuning feature of DAMON.
--- Documentation/admin-guide/mm/damon/lru_sort.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)
Documentation/admin-guide/mm/damon/lru_sort.rst +11 -0
@@ -91,6 +91,17 @@ increases and decreases the effective level of the quota aiming the LRU Disabled by default. +Auto-tune monitoring intervals +------------------------------ + +If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes DAMON's +sampling and aggregation intervals. The auto-tuning aims to capture meaningful +amount of access events in each DAMON-snapshot, while keeping the sampling +interval 5 milliseconds in minimum, and 10 seconds in maximum. Setting this as +``N`` disables the auto-tuning. + +Disabled by default. + filter_young_pages ------------------
Signature · 1 line
2.47.3
Re: [PATCH 11/11] Docs/admin-guide/mm/damon/lru_sort: document intervals autotuning
Quoted 5 lines
Document a newly added
DAMON_LRU_SORTmodule parameter for using monitoring intervals auto-tuning feature of DAMON.Signed-off-by SeongJae Park <[email protected]>
--
Best Regards,
wang lian