mm/damon: Support online tuning
Subsystems (4)
[email protected]
maintainercover mm/damon: Support online tuning
From: SeongJae Park <[email protected]>
Effects of DAMON and DAMON-based Operation Schemes highly depends on the configurations. Wrong configurations could even result in unexpected efficiency degradations. For finding a best configuration, repeating incremental configuration changes and results measurements, in other words, online tuning, could be helpful.
Nevertheless, DAMON kernel API supports only restrictive online tuning. Worse yet, the sysfs-based DAMON user interface doesn't support online tuning at all. DAMON_RECLAIM also doesn't support online tuning.
This patchset makes the DAMON kernel API, DAMON sysfs interface, and DAMON_RECLAIM supports online tuning.
Sequence of patches
-------------------
First two patches enhance DAMON online tuning for kernel API users. Specifically, patch 1 let kernel API users to be able to do DAMON online tuning without a restriction, and patch 2 makes error handling easier.
Following seven patches (patches 3-9) refactor code for better readability and easier reuse of code fragments that will be useful for online tuning support.
Patch 10 introduces DAMON callback based user request handling structure for DAMON sysfs interface, and patch 11 enables DAMON online tuning via DAMON sysfs interface. Documentation patch (patch 12) for usage of it follows.
Patch 13 enables online tuning of DAMON_RECLAIM and finally patch 14 documents the DAMON_RECLAIM online tuning usage.
SeongJae Park (14):
mm/damon/core: add a new callback for watermarks checks mm/damon/core: finish kdamond as soon as any callback returns an error mm/damon/vaddr: generalize damon_va_apply_three_regions() mm/damon/vaddr: move 'damon_set_regions()' to core mm/damon/vaddr: remove damon_va_apply_three_regions() mm/damon/sysfs: prohibit multiple physical address space monitoring
targets
mm/damon/sysfs: move targets setup code to a separated function mm/damon/sysfs: reuse damon_set_regions() for regions setting mm/damon/sysfs: use enum for 'state' input handling mm/damon/sysfs: update schemes stat in the kdamond context mm/damon/sysfs: support online inputs update Docs/{ABI,admin-guide}/damon: Update for 'state' sysfs file input
keyword, 'commit'
mm/damon/reclaim: support online inputs update Docs/admin-guide/mm/damon/reclaim: document 'commit_inputs' parameter
.../ABI/testing/sysfs-kernel-mm-damon | 7 +- .../admin-guide/mm/damon/reclaim.rst | 11 + Documentation/admin-guide/mm/damon/usage.rst | 9 +- include/linux/damon.h | 9 + mm/damon/core.c | 89 +++- mm/damon/reclaim.c | 95 +++-- mm/damon/sysfs.c | 382 +++++++++++++++--- mm/damon/vaddr-test.h | 6 +- mm/damon/vaddr.c | 73 +--- 9 files changed, 500 insertions(+), 181 deletions(-)
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 1/14 mm/damon/core: add a new callback for watermarks checks
From: SeongJae Park <[email protected]>
For updating input parameters for running DAMON contexts, DAMON kernel API users can use the contexts' callbacks, as it is the safe place for context internal data accesses. When the context has DAMON-based operation schemes and all schemes are deactivated due to their watermarks, however, DAMON does nothing but only watermarks checks. As a result, no callbacks will be called back, and therefore the kernel API users cannot update the input parameters including monitoring attributes, DAMON-based operation schemes, and watermarks.
To let users easily update such DAMON input parameters in such a case, this commit adds a new callback, 'after_wmarks_check()'. It will be called after each watermarks check. Users can do the online input parameters update in the callback even under the schemes deactivated case.
--- include/linux/damon.h | 7 +++++++ mm/damon/core.c | 8 +++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
include/linux/damon.h +7 -0
@@ -343,6 +343,7 @@ struct damon_operations {* struct damon_callback - Monitoring events notification callbacks. * * @before_start: Called before starting the monitoring. +* @after_wmarks_check: Called after each schemes' watermarks check. * @after_sampling: Called after each sampling. * @after_aggregation: Called after each aggregation. * @before_terminate: Called before terminating the monitoring.
@@ -353,6 +354,11 @@ struct damon_operations {* respectively. Therefore, those are good places for installing and cleaning * @private. * +* The monitoring thread calls @after_wmarks_check after each DAMON-based +* operation schemes' watermarks check. If users need to make changes to the +* attributes of the monitoring context while it's deactivated due to the +* watermarks, this is the good place to do. +* * The monitoring thread calls @after_sampling and @after_aggregation for each * of the sampling intervals and aggregation intervals, respectively. * Therefore, users can safely access the monitoring results without additional
@@ -365,6 +371,7 @@ struct damon_callback {void *private;int (*before_start)(struct damon_ctx *context); +int (*after_wmarks_check)(struct damon_ctx *context);int (*after_sampling)(struct damon_ctx *context);int (*after_aggregation)(struct damon_ctx *context);void (*before_terminate)(struct damon_ctx *context);
mm/damon/core.c +7 -1
@@ -1103,6 +1103,10 @@ static int kdamond_wait_activation(struct damon_ctx *ctx)return 0 ;kdamond_usleep (min_wait_time); + +if (ctx->callback.after_wmarks_check && + ctx->callback.after_wmarks_check (ctx)) +break ; }return -EBUSY ; }
@@ -1129,8 +1133,10 @@ static int kdamond_fn(void *data) sz_limit =damon_region_sz_limit (ctx);while (!kdamond_need_stop (ctx) && !done) { -if (kdamond_wait_activation (ctx)) +if (kdamond_wait_activation (ctx)) { + done = true;continue ; + }if (ctx->ops.prepare_access_checks) ctx->ops.prepare_access_checks (ctx);
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 2/14 mm/damon/core: finish kdamond as soon as any callback returns an error
In reply to [email protected]From: SeongJae Park <[email protected]>
When 'after_sampling()' or 'after_aggregation()' DAMON callbacks return an error, kdamond continues the remaining loop once. It makes no much sense to run the remaining part while something wrong already happened. The context might be corrupted or having invalid data. This commit therefore makes kdamond skips the remaining works and immediately finish in the cases.
--- mm/damon/core.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
mm/damon/core.c +6 -2
@@ -1141,8 +1141,10 @@ static int kdamond_fn(void *data)if (ctx->ops.prepare_access_checks) ctx->ops.prepare_access_checks (ctx);if (ctx->callback.after_sampling && - ctx->callback.after_sampling (ctx)) + ctx->callback.after_sampling (ctx)) { done = true; +continue ; + }kdamond_usleep (ctx->sample_interval);
@@ -1154,8 +1156,10 @@ static int kdamond_fn(void *data) max_nr_accesses /10 , sz_limit);if (ctx->callback.after_aggregation && - ctx->callback.after_aggregation (ctx)) + ctx->callback.after_aggregation (ctx)) { done = true; +continue ; + }kdamond_apply_schemes (ctx);kdamond_reset_aggregated (ctx);kdamond_split_regions (ctx);
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 3/14 mm/damon/vaddr: generalize damon_va_apply_three_regions()
In reply to [email protected]From: SeongJae Park <[email protected]>
'damon_va_apply_three_regions()' is for adjusting address ranges to fit in three discontiguous ranges. The function can be generalized for arbitrary number of discontiguous ranges and reused for future usage, such as arbitrary online regions update. For such future usage, this commit introduces a generalized version of the function called 'damon_set_regions()'.
--- mm/damon/vaddr.c | 66 ++++++++++++++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 24 deletions(-)
mm/damon/vaddr.c +42 -24
@@ -298,59 +298,77 @@ static bool damon_intersect(struct damon_region *r, }/* -* Update damon regions for the three big regions of the given target +* damon_set_regions() - Set regions of a target for given address ranges. +* @t: the given target. +* @ranges: array of new monitoring target ranges. +* @nr_ranges: length of @ranges. * -* t the given target -* bregions the three big regions of the target +* This function adds new regions to, or modify existing regions of a +* monitoring target to fit in specific ranges. +* +* Return: 0 if success, or negative error code otherwise. */ -static void damon_va_apply_three_regions (struct damon_target *t, -struct damon_addr_range bregions[3 ]) +static int damon_set_regions (struct damon_target *t, +struct damon_addr_range *ranges,unsigned int nr_ranges) {struct damon_region *r, *next;unsigned int i; -/* Remove regions which are not in the three big regions now */ +/* Remove regions which are not in the new ranges */ damon_for_each_region_safe (r, next, t) { -for (i =0 ; i <3 ; i++) { -if (damon_intersect (r, &bregions[i])) +for (i =0 ; i < nr_ranges; i++) { +if (damon_intersect (r, &ranges[i]))break ; } -if (i ==3 ) +if (i == nr_ranges)damon_destroy_region (r, t); } -/* Adjust intersecting regions to fit with the three big regions */ -for (i =0 ; i <3 ; i++) { -struct damon_region *first =NULL , *last; -struct damon_region *newr; -struct damon_addr_range *br; +/* Add new regions or resize existing regions to fit in the ranges */ +for (i =0 ; i < nr_ranges; i++) { +struct damon_region *first =NULL , *last, *newr; +struct damon_addr_range *range; - br = &bregions[i]; -/* Get the first and last regions which intersects with br */ + range = &ranges[i]; +/* Get the first/last regions intersecting with the range */ damon_for_each_region (r, t) { -if (damon_intersect (r, br)) { +if (damon_intersect (r, range)) {if (!first) first = r; last = r; } -if (r->ar.start >= br->end) +if (r->ar.start >= range->end)break ; }if (!first) { -/* no damon_region intersects with this big region */ +/* no region intersects with this range */ newr =damon_new_region ( -ALIGN_DOWN (br->start, +ALIGN_DOWN (range->start,DAMON_MIN_REGION ), -ALIGN (br->end,DAMON_MIN_REGION )); +ALIGN (range->end,DAMON_MIN_REGION ));if (!newr) -continue ; +return -ENOMEM ;damon_insert_region (newr,damon_prev_region (r), r, t); }else { - first->ar.start =ALIGN_DOWN (br->start, +/* resize intersecting regions to fit in this range */ + first->ar.start =ALIGN_DOWN (range->start,DAMON_MIN_REGION ); - last->ar.end =ALIGN (br->end,DAMON_MIN_REGION ); + last->ar.end =ALIGN (range->end,DAMON_MIN_REGION ); } } +return 0 ; +} + +/* +* Update damon regions for the three big regions of the given target +* +* t the given target +* bregions the three big regions of the target +*/ +static void damon_va_apply_three_regions (struct damon_target *t, +struct damon_addr_range bregions[3 ]) +{ +damon_set_regions (t, bregions,3 ); }/*
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 4/14 mm/damon/vaddr: move 'damon_set_regions()' to core
In reply to [email protected]From: SeongJae Park <[email protected]>
This commit moves 'damon_set_regions()' from vaddr to core, as it is aimed to be used by not only 'vaddr' but also other parts of DAMON.
--- include/linux/damon.h | 2 ++ mm/damon/core.c | 73 +++++++++++++++++++++++++++++++++++++++++++ mm/damon/vaddr.c | 73 ------------------------------------------- 3 files changed, 75 insertions(+), 73 deletions(-)
include/linux/damon.h +2 -0
@@ -494,6 +494,8 @@ static inline void damon_insert_region(struct damon_region *r,void damon_add_region (struct damon_region *r,struct damon_target *t);void damon_destroy_region (struct damon_region *r,struct damon_target *t); +int damon_set_regions (struct damon_target *t,struct damon_addr_range *ranges, +unsigned int nr_ranges); struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region,
mm/damon/core.c +73 -0
79 lines not on this page · raw message
mm/damon/vaddr.c +0 -73
79 lines not on this page · raw message
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 5/14 mm/damon/vaddr: remove damon_va_apply_three_regions()
In reply to [email protected]From: SeongJae Park <[email protected]>
'damon_va_apply_three_regions()' is just a wrapper of its general version, 'damon_set_regions()'. This commit replaces the wrapper calls to directly call the general version.
--- mm/damon/vaddr-test.h | 6 +++--- mm/damon/vaddr.c | 18 +----------------- 2 files changed, 4 insertions(+), 20 deletions(-)
mm/damon/vaddr-test.h +3 -3
@@ -98,7 +98,7 @@ static struct damon_region *__nth_region_of(struct damon_target *t, int idx) }/* -* Test 'damon_va_apply_three_regions()' +* Test 'damon_set_regions()' * * test kunit object * regions an array containing start/end addresses of current
@@ -113,7 +113,7 @@ static struct damon_region *__nth_region_of(struct damon_target *t, int idx)* the change, DAMON periodically reads the mappings, simplifies it to the * three regions, and updates the monitoring target regions to fit in the three * regions. The update of current target regions is the role of -* 'damon_va_apply_three_regions()'. +* 'damon_set_regions()'. * * This test passes the given target regions and the new three regions that * need to be applied to the function and check whether it updates the regions
@@ -134,7 +134,7 @@ static void damon_do_test_apply_three_regions(struct kunit *test,damon_add_region (r, t); } -damon_va_apply_three_regions (t, three_regions); +damon_set_regions (t, three_regions,3 );for (i =0 ; i < nr_expected /2 ; i++) { r =__nth_region_of (t, i);
mm/damon/vaddr.c +1 -17
@@ -282,22 +282,6 @@ static void damon_va_init(struct damon_ctx *ctx) } } -/* -* Functions for the dynamic monitoring target regions update -*/ - -/* -* Update damon regions for the three big regions of the given target -* -* t the given target -* bregions the three big regions of the target -*/ -static void damon_va_apply_three_regions (struct damon_target *t, -struct damon_addr_range bregions[3 ]) -{ -damon_set_regions (t, bregions,3 ); -} -/* * Update regions for current memory mappings */
@@ -309,7 +293,7 @@ static void damon_va_update(struct damon_ctx *ctx)damon_for_each_target (t, ctx) {if (damon_va_three_regions (t, three_regions))continue ; -damon_va_apply_three_regions (t, three_regions); +damon_set_regions (t, three_regions,3 ); } }
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 6/14 mm/damon/sysfs: prohibit multiple physical address space monitoring targets
In reply to [email protected]From: SeongJae Park <[email protected]>
Having multiple targets for physical address space monitoring makes no sense. This commit prohibits such a ridiculous DAMON context setup my making the DAMON context build function to check and return an error for the case.
--- mm/damon/sysfs.c | 4 ++++ 1 file changed, 4 insertions(+)
mm/damon/sysfs.c +4 -0
@@ -2124,6 +2124,10 @@ static int damon_sysfs_set_targets(struct damon_ctx *ctx, {int i, err; +/* Multiple physical address space monitoring targets makes no sense */ +if (ctx->ops.id ==DAMON_OPS_PADDR && sysfs_targets->nr >1 ) +return -EINVAL ; +for (i =0 ; i < sysfs_targets->nr; i++) {struct damon_sysfs_target *sys_target = sysfs_targets->targets_arr[i];
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 7/14 mm/damon/sysfs: move targets setup code to a separated function
In reply to [email protected]From: SeongJae Park <[email protected]>
This commit separates DAMON sysfs interface's monitoring context targets setup code to a new function for better readability.
--- mm/damon/sysfs.c | 49 +++++++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 21 deletions(-)
mm/damon/sysfs.c +28 -21
@@ -2119,6 +2119,31 @@ static int damon_sysfs_set_regions(struct damon_target *t,return 0 ; } +static int damon_sysfs_add_target (struct damon_sysfs_target *sys_target, +struct damon_ctx *ctx) +{ +struct damon_target *t =damon_new_target (); +int err = -EINVAL ; + +if (!t) +return -ENOMEM ; +if (ctx->ops.id ==DAMON_OPS_VADDR || + ctx->ops.id ==DAMON_OPS_FVADDR ) { + t->pid =find_get_pid (sys_target->pid); +if (!t->pid) + goto destroy_targets_out; + } +damon_add_target (ctx, t); + err =damon_sysfs_set_regions (t, sys_target->regions); +if (err) + goto destroy_targets_out; +return 0 ; + +destroy_targets_out: +damon_sysfs_destroy_targets (ctx); +return err; +} +static int damon_sysfs_set_targets (struct damon_ctx *ctx,struct damon_sysfs_targets *sysfs_targets) {
@@ -2129,28 +2154,10 @@ static int damon_sysfs_set_targets(struct damon_ctx *ctx,return -EINVAL ;for (i =0 ; i < sysfs_targets->nr; i++) { -struct damon_sysfs_target *sys_target = - sysfs_targets->targets_arr[i]; -struct damon_target *t =damon_new_target (); - -if (!t) { -damon_sysfs_destroy_targets (ctx); -return -ENOMEM ; - } -if (ctx->ops.id ==DAMON_OPS_VADDR || - ctx->ops.id ==DAMON_OPS_FVADDR ) { - t->pid =find_get_pid (sys_target->pid); -if (!t->pid) { -damon_sysfs_destroy_targets (ctx); -return -EINVAL ; - } - } -damon_add_target (ctx, t); - err =damon_sysfs_set_regions (t, sys_target->regions); -if (err) { -damon_sysfs_destroy_targets (ctx); + err =damon_sysfs_add_target ( + sysfs_targets->targets_arr[i], ctx); +if (err)return err; - } }return 0 ; }
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 8/14 mm/damon/sysfs: reuse damon_set_regions() for regions setting
In reply to [email protected]From: SeongJae Park <[email protected]>
'damon_set_regions()' is general enough so that it can also be used for only creating regions. This commit makes DAMON sysfs interface to reuse the function rather keeping two implementations for a same purpose.
--- mm/damon/sysfs.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-)
mm/damon/sysfs.c +18 -15
@@ -2095,28 +2095,31 @@ static void damon_sysfs_destroy_targets(struct damon_ctx *ctx)static int damon_sysfs_set_regions (struct damon_target *t,struct damon_sysfs_regions *sysfs_regions) { -int i; +struct damon_addr_range *ranges =kmalloc_array (sysfs_regions->nr, +sizeof (*ranges),GFP_KERNEL | __GFP_NOWARN); +int i, err = -EINVAL ; +if (!ranges) +return -ENOMEM ;for (i =0 ; i < sysfs_regions->nr; i++) {struct damon_sysfs_region *sys_region = sysfs_regions->regions_arr[i]; -struct damon_region *prev, *r;if (sys_region->start > sys_region->end) -return -EINVAL ; - r =damon_new_region (sys_region->start, sys_region->end); -if (!r) -return -ENOMEM ; -damon_add_region (r, t); -if (damon_nr_regions (t) >1 ) { - prev =damon_prev_region (r); -if (prev->ar.end > r->ar.start) { -damon_destroy_region (r, t); -return -EINVAL ; - } - } + goto out; + + ranges[i].start = sys_region->start; + ranges[i].end = sys_region->end; +if (i ==0 ) +continue ; +if (ranges[i -1 ].end > ranges[i].start) + goto out; } -return 0 ; + err =damon_set_regions (t, ranges, sysfs_regions->nr); +out: +kfree (ranges); +return err; + } static int damon_sysfs_add_target(struct damon_sysfs_target *sys_target,
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 9/14 mm/damon/sysfs: use enum for 'state' input handling
In reply to [email protected]From: SeongJae Park <[email protected]>
DAMON sysfs 'state' file handling code is using string literals in both 'state_show()' and 'state_store()'. This makes the code error prone and inflexible for future extensions.
To improve the sitaution, this commit defines possible input strings and 'enum' for identifying each input keyword only once, and refactors the code to reuse those.
--- mm/damon/sysfs.c | 72 +++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 62 insertions(+), 10 deletions(-)
mm/damon/sysfs.c +62 -10
@@ -2053,6 +2053,32 @@ static bool damon_sysfs_ctx_running(struct damon_ctx *ctx)return running; } +/* +* enum damon_sysfs_cmd - Commands for a specific kdamond. +*/ +enum damon_sysfs_cmd { +/* @DAMON_SYSFS_CMD_ON: Turn the kdamond on. */ +DAMON_SYSFS_CMD_ON , +/* @DAMON_SYSFS_CMD_OFF: Turn the kdamond off. */ +DAMON_SYSFS_CMD_OFF , +/* +* @DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS: Update scheme stats sysfs +* files. +*/ +DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS , +/* +* @NR_DAMON_SYSFS_CMDS: Total number of DAMON sysfs commands. +*/ +NR_DAMON_SYSFS_CMDS , +}; + +/* Should match with enum damon_sysfs_cmd */ +static const char *const damon_sysfs_cmd_strs[] = { +"on" , +"off" , +"update_schemes_stats" , +}; +static ssize_t state_show (struct kobject *kobj,struct kobj_attribute *attr,char *buf) {
@@ -2066,7 +2092,9 @@ static ssize_t state_show(struct kobject *kobj, struct kobj_attribute *attr,else running =damon_sysfs_ctx_running (ctx); -return sysfs_emit (buf,"%s\n" , running ?"on" :"off" ); +return sysfs_emit (buf,"%s\n" , running ? + damon_sysfs_cmd_strs[DAMON_SYSFS_CMD_ON ] : + damon_sysfs_cmd_strs[DAMON_SYSFS_CMD_OFF ]); } static int damon_sysfs_set_attrs(struct damon_ctx *ctx,
@@ -2324,23 +2352,47 @@ static int damon_sysfs_update_schemes_stats(struct damon_sysfs_kdamond *kdamond)return 0 ; } +/* +* damon_sysfs_handle_cmd() - Handle a command for a specific kdamond. +* @cmd: The command to handle. +* @kdamond: The kobject wrapper for the associated kdamond. +* +* This function handles a DAMON sysfs command for a kdamond. +* +* Return: 0 on success, negative error code otherwise. +*/ +static int damon_sysfs_handle_cmd (enum damon_sysfs_cmd cmd, +struct damon_sysfs_kdamond *kdamond) +{ +switch (cmd) { +case DAMON_SYSFS_CMD_ON : +return damon_sysfs_turn_damon_on (kdamond); +case DAMON_SYSFS_CMD_OFF : +return damon_sysfs_turn_damon_off (kdamond); +case DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS : +return damon_sysfs_update_schemes_stats (kdamond); +default : +break ; + } +return -EINVAL ; +} +static ssize_t state_store (struct kobject *kobj,struct kobj_attribute *attr,const char *buf,size_t count) {struct damon_sysfs_kdamond *kdamond =container_of (kobj, struct damon_sysfs_kdamond, kobj); -ssize_t ret; +enum damon_sysfs_cmd cmd; +ssize_t ret = -EINVAL ;if (!mutex_trylock (&damon_sysfs_lock))return -EBUSY ; -if (sysfs_streq (buf,"on" )) - ret =damon_sysfs_turn_damon_on (kdamond); -else if (sysfs_streq (buf,"off" )) - ret =damon_sysfs_turn_damon_off (kdamond); -else if (sysfs_streq (buf,"update_schemes_stats" )) - ret =damon_sysfs_update_schemes_stats (kdamond); -else - ret = -EINVAL ; +for (cmd =0 ; cmd <NR_DAMON_SYSFS_CMDS ; cmd++) { +if (sysfs_streq (buf, damon_sysfs_cmd_strs[cmd])) { + ret =damon_sysfs_handle_cmd (cmd, kdamond); +break ; + } + }mutex_unlock (&damon_sysfs_lock);if (!ret) ret = count;
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 10/14 mm/damon/sysfs: update schemes stat in the kdamond context
In reply to [email protected]From: SeongJae Park <[email protected]>
Only '->kdamond' and '->kdamond_stop' are protected by 'kdamond_lock' of 'struct damon_ctx'. All other DAMON context internal data items are recommended to be accessed in DAMON callbacks, or under some additional synchronizations. But, DAMON sysfs is accessing the schemes stat under 'kdamond_lock'.
It makes no big issue as the read values are not used anywhere inside kernel, but would better to be fixed. This commit moves the reads to DAMON callback context, as supposed to be used for the purpose.
--- mm/damon/sysfs.c | 161 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 135 insertions(+), 26 deletions(-)
mm/damon/sysfs.c +135 -26
@@ -2079,6 +2079,25 @@ static const char * const damon_sysfs_cmd_strs[] = { "update_schemes_stats", }; +/* +* struct damon_sysfs_cmd_request - A request to the DAMON callback. +* @cmd: The command that needs to be handled by the callback. +* @kdamond: The kobject wrapper that associated to the kdamond thread. +* +* This structure represents a sysfs command request that need to access some +* DAMON context-internal data. Because DAMON context-internal data can be +* safely accessed from DAMON callbacks without additional synchronization, the +* request will be handled by the DAMON callback. None-``NULL`` @kdamond means +* the request is valid. +*/ +struct damon_sysfs_cmd_request { +enum damon_sysfs_cmd cmd; +struct damon_sysfs_kdamond *kdamond; +}; + +/* Current DAMON callback request. Protected by damon_sysfs_lock. */ +static struct damon_sysfs_cmd_request damon_sysfs_cmd_request; +static ssize_t state_show (struct kobject *kobj,struct kobj_attribute *attr,char *buf) {
@@ -2257,6 +2276,70 @@ static void damon_sysfs_before_terminate(struct damon_ctx *ctx)mutex_unlock (&ctx->kdamond_lock); } +/* +* damon_sysfs_upd_schemes_stats() - Update schemes stats sysfs files. +* @kdamond: The kobject wrapper that associated to the kdamond thread. +* +* This function reads the schemes stats of specific kdamond and update the +* related values for sysfs files. This function should be called from DAMON +* callbacks while holding ``damon_syfs_lock``, to safely access the DAMON +* contexts-internal data and DAMON sysfs variables. +*/ +static int damon_sysfs_upd_schemes_stats (struct damon_sysfs_kdamond *kdamond) +{ +struct damon_ctx *ctx = kdamond->damon_ctx; +struct damon_sysfs_schemes *sysfs_schemes; +struct damos *scheme; +int schemes_idx =0 ; + +if (!ctx) +return -EINVAL ; + sysfs_schemes = kdamond->contexts->contexts_arr[0 ]->schemes; +damon_for_each_scheme (scheme, ctx) { +struct damon_sysfs_stats *sysfs_stats; + + sysfs_stats = sysfs_schemes->schemes_arr[schemes_idx++]->stats; + sysfs_stats->nr_tried = scheme->stat.nr_tried; + sysfs_stats->sz_tried = scheme->stat.sz_tried; + sysfs_stats->nr_applied = scheme->stat.nr_applied; + sysfs_stats->sz_applied = scheme->stat.sz_applied; + sysfs_stats->qt_exceeds = scheme->stat.qt_exceeds; + } +return 0 ; +} + +/* +* damon_sysfs_cmd_request_callback() - DAMON callback for handling requests. +* @c: The DAMON context of the callback. +* +* This function is periodically called back from the kdamond thread for @c. +* Then, it checks if there is a waiting DAMON sysfs request and handles it. +*/ +static int damon_sysfs_cmd_request_callback (struct damon_ctx *c) +{ +struct damon_sysfs_kdamond *kdamond; +int err =0 ; + +/* avoid deadlock due to concurrent state_store('off') */ +if (!mutex_trylock (&damon_sysfs_lock)) +return 0 ; + kdamond = damon_sysfs_cmd_request.kdamond; +if (!kdamond || kdamond->damon_ctx != c) + goto out; +switch (damon_sysfs_cmd_request.cmd) { +case DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS : + err =damon_sysfs_upd_schemes_stats (kdamond); +break ; +default : +break ; + } +/* Mark the request as invalid now. */ + damon_sysfs_cmd_request.kdamond =NULL ; +out: +mutex_unlock (&damon_sysfs_lock); +return err; +} +static struct damon_ctx *damon_sysfs_build_ctx (struct damon_sysfs_context *sys_ctx) {
@@ -2279,6 +2362,8 @@ static struct damon_ctx *damon_sysfs_build_ctx(if (err) goto out; + ctx->callback.after_wmarks_check = damon_sysfs_cmd_request_callback; + ctx->callback.after_aggregation = damon_sysfs_cmd_request_callback; ctx->callback.before_terminate = damon_sysfs_before_terminate;return ctx;
@@ -2295,6 +2380,8 @@ static int damon_sysfs_turn_damon_on(struct damon_sysfs_kdamond *kdamond)if (kdamond->damon_ctx &&damon_sysfs_ctx_running (kdamond->damon_ctx))return -EBUSY ; +if (damon_sysfs_cmd_request.kdamond == kdamond) +return -EBUSY ;/* TODO: support multiple contexts per kdamond */ if (kdamond->contexts->nr !=1 )return -EINVAL ;
@@ -2327,29 +2414,11 @@ static int damon_sysfs_turn_damon_off(struct damon_sysfs_kdamond *kdamond)*/ } -static int damon_sysfs_update_schemes_stats (struct damon_sysfs_kdamond *kdamond) +static inline bool damon_sysfs_kdamond_running ( +struct damon_sysfs_kdamond *kdamond) { -struct damon_ctx *ctx = kdamond->damon_ctx; -struct damos *scheme; -int schemes_idx =0 ; - -if (!ctx) -return -EINVAL ; -mutex_lock (&ctx->kdamond_lock); -damon_for_each_scheme (scheme, ctx) { -struct damon_sysfs_schemes *sysfs_schemes; -struct damon_sysfs_stats *sysfs_stats; - - sysfs_schemes = kdamond->contexts->contexts_arr[0 ]->schemes; - sysfs_stats = sysfs_schemes->schemes_arr[schemes_idx++]->stats; - sysfs_stats->nr_tried = scheme->stat.nr_tried; - sysfs_stats->sz_tried = scheme->stat.sz_tried; - sysfs_stats->nr_applied = scheme->stat.nr_applied; - sysfs_stats->sz_applied = scheme->stat.sz_applied; - sysfs_stats->qt_exceeds = scheme->stat.qt_exceeds; - } -mutex_unlock (&ctx->kdamond_lock); -return 0 ; +return kdamond->damon_ctx && +damon_sysfs_ctx_running (kdamond->damon_ctx); }/*
@@ -2357,24 +2426,58 @@ static int damon_sysfs_update_schemes_stats(struct damon_sysfs_kdamond *kdamond)* @cmd: The command to handle. * @kdamond: The kobject wrapper for the associated kdamond. * -* This function handles a DAMON sysfs command for a kdamond. +* This function handles a DAMON sysfs command for a kdamond. For commands +* that need to access running DAMON context-internal data, it requests +* handling of the command to the DAMON callback +* (@damon_sysfs_cmd_request_callback()) and wait until it is properly handled, +* or the context is completed. * * Return: 0 on success, negative error code otherwise. */ static int damon_sysfs_handle_cmd (enum damon_sysfs_cmd cmd,struct damon_sysfs_kdamond *kdamond) { +bool need_wait = true; + +/* Handle commands that doesn't access DAMON context-internal data */ switch (cmd) {case DAMON_SYSFS_CMD_ON :return damon_sysfs_turn_damon_on (kdamond);case DAMON_SYSFS_CMD_OFF :return damon_sysfs_turn_damon_off (kdamond); -case DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS : -return damon_sysfs_update_schemes_stats (kdamond);default :break ; } -return -EINVAL ; + +/* Pass the command to DAMON callback for safe DAMON context access */ +if (damon_sysfs_cmd_request.kdamond) +return -EBUSY ; +if (!damon_sysfs_kdamond_running (kdamond)) +return -EINVAL ; + damon_sysfs_cmd_request.cmd = cmd; + damon_sysfs_cmd_request.kdamond = kdamond; + +/* +* wait until damon_sysfs_cmd_request_callback() handles the request +* from kdamond context +*/ +mutex_unlock (&damon_sysfs_lock); +while (need_wait) { +schedule_timeout_idle (msecs_to_jiffies (100 )); +if (!mutex_trylock (&damon_sysfs_lock)) +continue ; +if (!damon_sysfs_cmd_request.kdamond) { +/* damon_sysfs_cmd_request_callback() handled */ + need_wait = false; + }else if (!damon_sysfs_kdamond_running (kdamond)) { +/* kdamond has already finished */ + need_wait = false; + damon_sysfs_cmd_request.kdamond =NULL ; + } +mutex_unlock (&damon_sysfs_lock); + } +mutex_lock (&damon_sysfs_lock); +return 0 ; } static ssize_t state_store(struct kobject *kobj, struct kobj_attribute *attr,
@@ -2511,6 +2614,12 @@ static int damon_sysfs_kdamonds_add_dirs(struct damon_sysfs_kdamonds *kdamonds,if (damon_sysfs_nr_running_ctxs (kdamonds->kdamonds_arr, kdamonds->nr))return -EBUSY ; +for (i =0 ; i < kdamonds->nr; i++) { +if (damon_sysfs_cmd_request.kdamond == + kdamonds->kdamonds_arr[i]) +return -EBUSY ; + } +damon_sysfs_kdamonds_rm_dirs (kdamonds);if (!nr_kdamonds)return 0 ;
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 11/14 mm/damon/sysfs: support online inputs update
In reply to [email protected]From: SeongJae Park <[email protected]>
Currently, DAMON sysfs interface doesn't provide a way for adjusting DAMON input parameters while it is turned on. Therefore, users who want to reconfigure DAMON need to stop DAMON and restart. This means all the monitoring results that accumulated so far, which could be useful, should be flushed. This would be inefficient for many cases.
For an example, let's suppose a sysadmin was running a DAMON-based Operation Scheme to find memory regions not accessed for more than 5 mins and page out the regions. If it turns out the 5 mins threshold was too long and therefore the sysadmin wants to reduce it to 4 mins, the sysadmin should turn off DAMON, restart it, and wait for at least 4 more minutes so that DAMON can find the cold memory regions, even though DAMON was knowing there are regions that not accessed for 4 mins at the time of shutdown.
This commit makes DAMON sysfs interface to support online DAMON input parameters updates by adding a new input keyword for the 'state' DAMON sysfs file, 'commit'. Writing the keyword to the 'state' file while the corresponding kdamond is running makes the kdamond to read the sysfs file values again and update the DAMON context.
--- mm/damon/sysfs.c | 99 +++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 90 insertions(+), 9 deletions(-)
mm/damon/sysfs.c +90 -9
@@ -2061,6 +2061,8 @@ enum damon_sysfs_cmd {DAMON_SYSFS_CMD_ON ,/* @DAMON_SYSFS_CMD_OFF: Turn the kdamond off. */ DAMON_SYSFS_CMD_OFF , +/* @DAMON_SYSFS_CMD_COMMIT: Update kdamond inputs. */ + DAMON_SYSFS_CMD_COMMIT,/* * @DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS: Update scheme stats sysfs * files.
@@ -2076,6 +2078,7 @@ enum damon_sysfs_cmd {static const char *const damon_sysfs_cmd_strs[] = {"on" ,"off" , +"commit" ,"update_schemes_stats" , };
@@ -2194,6 +2197,39 @@ static int damon_sysfs_add_target(struct damon_sysfs_target *sys_target,return err; } +/* +* Search a target in a context that corresponds to the sysfs target input. +* +* Return: pointer to the target if found, NULL if not found, or negative +* error code if the search failed. +*/ +static struct damon_target *damon_sysfs_existing_target ( +struct damon_sysfs_target *sys_target,struct damon_ctx *ctx) +{ +struct pid *pid; +struct damon_target *t; + +if (ctx->ops.id ==DAMON_OPS_PADDR ) { +/* Up to only one target for paddr could exist */ +damon_for_each_target (t, ctx) +return t; +return NULL ; + } + +/* ops.id should be DAMON_OPS_VADDR or DAMON_OPS_FVADDR */ + pid =find_get_pid (sys_target->pid); +if (!pid) +return ERR_PTR (-EINVAL ); +damon_for_each_target (t, ctx) { +if (t->pid == pid) { +put_pid (pid); +return t; + } + } +put_pid (pid); +return NULL ; +} +static int damon_sysfs_set_targets (struct damon_ctx *ctx,struct damon_sysfs_targets *sysfs_targets) {
@@ -2204,8 +2240,15 @@ static int damon_sysfs_set_targets(struct damon_ctx *ctx,return -EINVAL ;for (i =0 ; i < sysfs_targets->nr; i++) { - err =damon_sysfs_add_target ( - sysfs_targets->targets_arr[i], ctx); +struct damon_sysfs_target *st = sysfs_targets->targets_arr[i]; +struct damon_target *t =damon_sysfs_existing_target (st, ctx); + +if (IS_ERR (t)) +return PTR_ERR (t); +if (!t) + err =damon_sysfs_add_target (st, ctx); +else + err =damon_sysfs_set_regions (t, st->regions);if (err)return err; }
@@ -2308,6 +2351,48 @@ static int damon_sysfs_upd_schemes_stats(struct damon_sysfs_kdamond *kdamond)return 0 ; } +static inline bool damon_sysfs_kdamond_running ( +struct damon_sysfs_kdamond *kdamond) +{ +return kdamond->damon_ctx && +damon_sysfs_ctx_running (kdamond->damon_ctx); +} + +/* +* damon_sysfs_commit_input() - Commit user inputs to a running kdamond. +* @kdamond: The kobject wrapper for the associated kdamond. +* +* If the sysfs input is wrong, the kdamond will be terminated. +*/ +static int damon_sysfs_commit_input (struct damon_sysfs_kdamond *kdamond) +{ +struct damon_ctx *ctx = kdamond->damon_ctx; +struct damon_sysfs_context *sys_ctx; +int err =0 ; + +if (!damon_sysfs_kdamond_running (kdamond)) +return -EINVAL ; +/* TODO: Support multiple contexts per kdamond */ +if (kdamond->contexts->nr !=1 ) +return -EINVAL ; + + sys_ctx = kdamond->contexts->contexts_arr[0 ]; + + err =damon_select_ops (ctx, sys_ctx->ops_id); +if (err) +return err; + err =damon_sysfs_set_attrs (ctx, sys_ctx->attrs); +if (err) +return err; + err =damon_sysfs_set_targets (ctx, sys_ctx->targets); +if (err) +return err; + err =damon_sysfs_set_schemes (ctx, sys_ctx->schemes); +if (err) +return err; +return err; +} +/* * damon_sysfs_cmd_request_callback() - DAMON callback for handling requests. * @c: The DAMON context of the callback.
@@ -2330,6 +2415,9 @@ static int damon_sysfs_cmd_request_callback(struct damon_ctx *c)case DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS : err =damon_sysfs_upd_schemes_stats (kdamond);break ; +case DAMON_SYSFS_CMD_COMMIT : + err =damon_sysfs_commit_input (kdamond); +break ;default :break ; }
@@ -2414,13 +2502,6 @@ static int damon_sysfs_turn_damon_off(struct damon_sysfs_kdamond *kdamond)*/ } -static inline bool damon_sysfs_kdamond_running ( -struct damon_sysfs_kdamond *kdamond) -{ -return kdamond->damon_ctx && -damon_sysfs_ctx_running (kdamond->damon_ctx); -} -/* * damon_sysfs_handle_cmd() - Handle a command for a specific kdamond. * @cmd: The command to handle.
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 12/14 Docs/{ABI,admin-guide}/damon: Update for 'state' sysfs file input keyword, 'commit'
In reply to [email protected]From: SeongJae Park <[email protected]>
This commit documents the newly added 'state' sysfs file input keyword, 'commit', which allows online tuning of DAMON contexts.
--- Documentation/ABI/testing/sysfs-kernel-mm-damon | 7 ++++--- Documentation/admin-guide/mm/damon/usage.rst | 9 +++++---- 2 files changed, 9 insertions(+), 7 deletions(-)
Documentation/ABI/testing/sysfs-kernel-mm-damon +4 -3
@@ -23,9 +23,10 @@ Date: Mar 2022 Contact: SeongJae Park <[email protected]> Description: Writing 'on' or 'off' to this file makes the kdamond starts or stops, respectively. Reading the file returns the keywords - based on the current status. Writing 'update_schemes_stats' to - the file updates contents of schemes stats files of the - kdamond. + based on the current status. Writing 'commit' to this file + makes the kdamond reads the user inputs in the sysfs files + except 'state' again. Writing 'update_schemes_stats' to the + file updates contents of schemes stats files of the kdamond. What: /sys/kernel/mm/damon/admin/kdamonds/<K>/pid Date: Mar 2022
Documentation/admin-guide/mm/damon/usage.rst +5 -4
@@ -121,10 +121,11 @@ In each kdamond directory, two files (``state`` and ``pid``) and one directory Reading ``state`` returns ``on`` if the kdamond is currently running, or ``off`` if it is not running. Writing ``on`` or ``off`` makes the kdamond be -in the state. Writing ``update_schemes_stats`` to ``state`` file updates the -contents of stats files for each DAMON-based operation scheme of the kdamond. -For details of the stats, please refer to :ref:`stats section -<sysfs_schemes_stats>`. +in the state. Writing ``commit`` to the ``state`` file makes kdamond reads the +user inputs in the sysfs files except ``state`` file again. Writing +``update_schemes_stats`` to ``state`` file updates the contents of stats files +for each DAMON-based operation scheme of the kdamond. For details of the +stats, please refer to :ref:`stats section <sysfs_schemes_stats>`. If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 13/14 mm/damon/reclaim: support online inputs update
In reply to [email protected]From: SeongJae Park <[email protected]>
DAMON_RECLAIM reads the user input parameters only when it starts. To allow more efficient online tuning, this commit implements a new input parameter called 'commit_inputs'. Writing true to the parameter makes DAMON_RECLAIM reads the input parameters again.
--- mm/damon/reclaim.c | 95 ++++++++++++++++++++++++++++++---------------- 1 file changed, 62 insertions(+), 33 deletions(-)
mm/damon/reclaim.c +62 -33
@@ -29,6 +29,18 @@*/ static bool enabled__read_mostly ; +/* +* Make DAMON_RECLAIM reads the input parameters again, except ``enabled``. +* +* Input parameters that updated while DAMON_RECLAIM is running are not applied +* by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values +* of parametrs except ``enabled`` again. Once the re-reading is done, this +* parameter is set as ``N``. If invalid parameters are found while the +* re-reading, DAMON_RECLAIM will be disabled. +*/ +static bool commit_inputs__read_mostly ; +module_param (commit_inputs, bool,0600 ); +/* * Time threshold for cold memory regions identification in microseconds. *
@@ -289,57 +301,56 @@ static struct damos *damon_reclaim_new_scheme(void)return scheme; } -static int damon_reclaim_turn (bool on) +static int damon_reclaim_apply_parameters (void ) { -struct damon_region *region;struct damos *scheme; -int err; - -if (!on) { - err =damon_stop (&ctx,1 ); -if (!err) - kdamond_pid =-1 ; -return err; - } +struct damon_addr_range addr_range; +int err =0 ; err =damon_set_attrs (ctx, sample_interval, aggr_interval,0 , min_nr_regions, max_nr_regions);if (err)return err; +/* Will be freed by next 'damon_set_schemes()' below */ + scheme =damon_reclaim_new_scheme (); +if (!scheme) +return -ENOMEM ; + err =damon_set_schemes (ctx, &scheme,1 ); +if (err) +return err; +if (monitor_region_start > monitor_region_end)return -EINVAL ;if (!monitor_region_start && !monitor_region_end && !get_monitoring_region (&monitor_region_start, &monitor_region_end))return -EINVAL ; -/* DAMON will free this on its own when finish monitoring */ - region =damon_new_region (monitor_region_start, monitor_region_end); -if (!region) -return -ENOMEM ; -damon_add_region (region, target); + addr_range.start = monitor_region_start; + addr_range.end = monitor_region_end; +return damon_set_regions (target, &addr_range,1 ); +} -/* Will be freed by 'damon_set_schemes()' below */ - scheme =damon_reclaim_new_scheme (); -if (!scheme) { - err = -ENOMEM ; - goto free_region_out; +static int damon_reclaim_turn (bool on) +{ +int err; + +if (!on) { + err =damon_stop (&ctx,1 ); +if (!err) + kdamond_pid =-1 ; +return err; } - err =damon_set_schemes (ctx, &scheme,1 ); + + err =damon_reclaim_apply_parameters ();if (err) - goto free_scheme_out; +return err; err =damon_start (&ctx,1 , true); -if (!err) { - kdamond_pid = ctx->kdamond->pid; -return 0 ; - } - -free_scheme_out: -damon_destroy_scheme (scheme); -free_region_out: -damon_destroy_region (region, target); -return err; +if (err) +return err; + kdamond_pid = ctx->kdamond->pid; +return 0 ; }#define ENABLE_CHECK_INTERVAL_MS 1000
@@ -389,6 +400,7 @@ MODULE_PARM_DESC(enabled,static int damon_reclaim_after_aggregation (struct damon_ctx *c) {struct damos *s; +int err =0 ;/* update the stats parameter */ damon_for_each_scheme (s, c) {
@@ -398,7 +410,23 @@ static int damon_reclaim_after_aggregation(struct damon_ctx *c) bytes_reclaimed_regions = s->stat.sz_applied; nr_quota_exceeds = s->stat.qt_exceeds; } -return 0 ; + +if (commit_inputs) { + err =damon_reclaim_apply_parameters (); + commit_inputs = false; + } +return err; +} + +static int damon_reclaim_after_wmarks_check (struct damon_ctx *c) +{ +int err =0 ; + +if (commit_inputs) { + err =damon_reclaim_apply_parameters (); + commit_inputs = false; + } +return err; }static int __init damon_reclaim_init (void )
@@ -410,6 +438,7 @@ static int __init damon_reclaim_init(void)if (damon_select_ops (ctx,DAMON_OPS_PADDR ))return -EINVAL ; + ctx->callback.after_wmarks_check = damon_reclaim_after_wmarks_check; ctx->callback.after_aggregation = damon_reclaim_after_aggregation; target =damon_new_target ();
Signature · 1 line
2.25.1
[email protected]
maintainerPATCH 14/14 Docs/admin-guide/mm/damon/reclaim: document 'commit_inputs' parameter
In reply to [email protected]From: SeongJae Park <[email protected]>
This commit documents the new DAMON_RECLAIM parameter, 'commit_inputs' in its usage document.
--- Documentation/admin-guide/mm/damon/reclaim.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)
Documentation/admin-guide/mm/damon/reclaim.rst +11 -0
@@ -66,6 +66,17 @@ Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do no real monitoring and reclamation due to the watermarks-based activation condition. Refer to below descriptions for the watermarks parameter for this. +commit_inputs +------------- + +Make DAMON_RECLAIM reads the input parameters again, except ``enabled``. + +Input parameters that updated while DAMON_RECLAIM is running are not applied +by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values +of parametrs except ``enabled`` again. Once the re-reading is done, this +parameter is set as ``N``. If invalid parameters are found while the +re-reading, DAMON_RECLAIM will be disabled. + min_age -------
Signature · 1 line
2.25.1