mm/damon: implement DAMOS apply intervals
Subsystems (5)
SeongJae Park
maintainercover mm/damon: implement DAMOS apply intervals
DAMON-based operation schemes are applied for every aggregation interval. That is mainly because schemes are using nr_accesses, which be complete to be used for every aggregation interval. However, DAMON provides nr_accesses_bp, which is updated for each sampling interval in a way that reasonable to be used. Therefore, there is no reason to not use nr_accesses_bp instead and apply schemes for their own time interval instead of the aggregation interval.
Actually, the alignment with the aggregation interval is also making some use case of DAMOS tricky. Quota setting under long aggregation interval is one such example. Suppose the aggregation interval is ten seconds, and there is a scheme having CPU quota 100ms per 1s. The scheme will actually uses 100ms per ten seconds, since it cannobe be applied before next aggregation interval. The feature is working as intended, but the results might not that intuitive for some users. This could be fixed by updating the quota to 1s per 10s. But, in the case, the CPU usage of DAMOS could look like spikes, and actually make a bad effect to other CPU-sensitive workloads.
This patchset makes DAMOS schemes to use nr_accesses_bp instead of nr_accesses, and have their own timing intervals. Note that the interval is 0 by default, and it is interpreted to use the aggregation interval instead. This is for avoid making behavioral changes to the old users.
Patches Seuqeunce
-----------------
The first patch (patch 1/8) makes DAMOS uses nr_accesses_bp instead of nr_accesses, and following two patches (patches 2/8 and 3/8) updates DAMON sysfs interface for DAMOS tried regions and the DAMOS before_apply tracespoint to expose nr_accesses_bp instead of nr_accesses, respectively.
The following two patches (patches 4/8 and 5/8) implements the scheme-specific apply interval for DAMON kernel API users and update the design document for the new feature. Finally, the following three patches (patches 6/8, 7/8, and 8/8) add support of the feature in DAMON sysfs interface and documents it on usage and ABI documents, repsectively.
SeongJae Park (8):
mm/damon/core: make DAMOS uses nr_accesses_bp instead of nr_accesses mm/damon/sysfs-schemes: expose nr_accesses_bp via
tried_regions/<N>/nr_accesses
mm/damon/core: expose nr_accesses_bp from damos_before_apply
tracepoint
mm/damon/core: implement scheme-specific apply interval Docs/mm/damon/design: document DAMOS apply intervals mm/damon/sysfs-schemes: support DAMOS apply interval Docs/admin-guide/mm/damon/usage: update for DAMOS apply intervals Docs/ABI/damon: update for DAMOS apply intervals
.../ABI/testing/sysfs-kernel-mm-damon | 7 ++ Documentation/admin-guide/mm/damon/usage.rst | 9 ++- Documentation/mm/damon/design.rst | 3 +- include/linux/damon.h | 17 +++- include/trace/events/damon.h | 2 +- mm/damon/core.c | 80 +++++++++++++++++-- mm/damon/dbgfs.c | 3 +- mm/damon/lru_sort.c | 2 + mm/damon/reclaim.c | 2 + mm/damon/sysfs-schemes.c | 40 ++++++++-- 10 files changed, 144 insertions(+), 21 deletions(-)
base-commit: 2a43f312aed581fa5044c4a0c0d20cfd4e632aa6
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 1/8 mm/damon/core: make DAMOS uses nr_accesses_bp instead of nr_accesses
DAMON provides nr_accesses_bp, which becomes same to nr_accesses * 10000 for every aggregation interval, but updated every sampling interval with a reasonable accuracy. Since DAMON-based operation schemes are applied in every aggregation interval using nr_accesses, using nr_accesses_bp instead will make no difference. This also makes future DAMOS changes for applying the scheme in a time interval that less than the aggregation interval possible.
--- mm/damon/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
mm/damon/core.c +3 -2
@@ -799,12 +799,13 @@ static void damon_split_region_at(struct damon_target *t,static bool __damos_valid_target (struct damon_region *r,struct damos *s) {unsigned long sz; +unsigned int nr_accesses = r->nr_accesses_bp /10000 ; sz =damon_sz_region (r);return s->pattern.min_sz_region <= sz && sz <= s->pattern.max_sz_region && - s->pattern.min_nr_accesses <= r->nr_accesses && - r->nr_accesses <= s->pattern.max_nr_accesses && + s->pattern.min_nr_accesses <= nr_accesses && + nr_accesses <= s->pattern.max_nr_accesses && s->pattern.min_age_region <= r->age && r->age <= s->pattern.max_age_region; }
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 2/8 mm/damon/sysfs-schemes: expose nr_accesses_bp via tried_regions/<N>/nr_accesses
In reply to SeongJae ParkDAMON sysfs interface exposes access rate of each region via DAMOS tried regions directory. For this, the nr_accesses field of the region is being used. DAMOS was using nr_accesses in the past, but it uses nr_accesses_bp now. Expose the value that it is really using. Note that it doesn't expose nr_accesses_bp as is (in basis point), but after converting it to the natural number by dividing the value by 10,000. That's for not making unnecessary confusion to the old users.
--- mm/damon/sysfs-schemes.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/damon/sysfs-schemes.c +1 -1
@@ -31,7 +31,7 @@ static struct damon_sysfs_scheme_region *damon_sysfs_scheme_region_alloc(return NULL ; sysfs_region->kobj = (struct kobject ){}; sysfs_region->ar = region->ar; - sysfs_region->nr_accesses = region->nr_accesses; + sysfs_region->nr_accesses = region->nr_accesses_bp /10000 ; sysfs_region->age = region->age;INIT_LIST_HEAD (&sysfs_region->list);return sysfs_region;
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 3/8 mm/damon/core: expose nr_accesses_bp from damos_before_apply tracepoint
In reply to SeongJae Parkdamos_before_apply tracepoint is exposing access rate of DAMON regions using nr_accesses, which was actually used by DAMOS in the past. However, it has changed to use nr_accesses_bp instead. Update the tracepoint to expose the value that DAMOS is really using. Note that it doesn't expose the value as is in the basis point, but after converting it to the natural number by dividing it by 10,000. That's for avoiding confuses for old users.
--- include/trace/events/damon.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/trace/events/damon.h +1 -1
@@ -34,7 +34,7 @@ TRACE_EVENT(damos_before_apply, __entry->target_idx = target_idx; __entry->start = r->ar.start; __entry->end = r->ar.end; - __entry->nr_accesses = r->nr_accesses; + __entry->nr_accesses = r->nr_accesses_bp /10000 ; __entry->age = r->age; __entry->nr_regions = nr_regions; ),
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 4/8 mm/damon/core: implement scheme-specific apply interval
In reply to SeongJae ParkDAMON-based operation schemes are applied for every aggregation interval. That was mainly because schemes were using nr_accesses, which be complete to be used for every aggregation interval. However, the schemes are now using nr_accesses_bp, which is updated for each sampling interval in a way that reasonable to be used. Therefore, there is no reason to apply schemes for each aggregation interval.
The unnecessary alignment with aggregation interval was also making some use case of DAMOS tricky. Quota setting under long aggregation interval is one such example. Suppose the aggregation interval is ten seconds, and there is a scheme having CPU quota 100ms per 1s. The scheme will actually uses 100ms per ten seconds, since it cannobe be applied before next aggregation interval. The feature is working as intended, but the results might not that intuitive for some users. This could be fixed by updating the quota to 1s per 10s. But, in the case, the CPU usage of DAMOS could look like spikes, and actually make a bad effect to other CPU-sensitive workloads.
Implement a dedicated timing interval for each DAMON-based operation scheme, namely apply_interval. The interval will be sampling interval aligned, and each scheme will be applied for its apply_interval. The interval is set to 0 by default, and it means the scheme should use the aggregation interval instead. This avoids old users getting any behavioral difference.
--- include/linux/damon.h | 17 +++++++-- mm/damon/core.c | 75 ++++++++++++++++++++++++++++++++++++---- mm/damon/dbgfs.c | 3 +- mm/damon/lru_sort.c | 2 ++ mm/damon/reclaim.c | 2 ++ mm/damon/sysfs-schemes.c | 2 +- 6 files changed, 91 insertions(+), 10 deletions(-)
include/linux/damon.h +15 -2
@@ -314,16 +314,19 @@ struct damos_access_pattern {* struct damos - Represents a Data Access Monitoring-based Operation Scheme. * @pattern: Access pattern of target regions. * @action: &damo_action to be applied to the target regions. +* @apply_interval_us: The time between applying the @action. * @quota: Control the aggressiveness of this scheme. * @wmarks: Watermarks for automated (in)activation of this scheme. * @filters: Additional set of &struct damos_filter for &action. * @stat: Statistics of this scheme. * @list: List head for siblings. * -* For each aggregation interval, DAMON finds regions which fit in the +* For each @apply_interval_us, DAMON finds regions which fit in the * &pattern and applies &action to those. To avoid consuming too much * CPU time or IO resources for the &action, "a is used. * +* If @apply_interval_us is zero, &damon_attrs->aggr_interval is used instead. +* * To do the work only when needed, schemes can be activated for specific * system situations using &wmarks. If all schemes that registered to the * monitoring context are inactive, DAMON stops monitoring either, and just
@@ -340,6 +343,14 @@ struct damos_access_pattern {struct damos {struct damos_access_pattern pattern;enum damos_action action; +unsigned long apply_interval_us; +/* private: internal use only */ +/* +* number of sample intervals that should be passed before applying +* @action +*/ +unsigned long next_apply_sis; +/* public: */ struct damos_quota quota;struct damos_watermarks wmarks;struct list_head filters;
@@ -641,7 +652,9 @@ void damos_add_filter(struct damos *s, struct damos_filter *f);void damos_destroy_filter (struct damos_filter *f);struct damos *damon_new_scheme (struct damos_access_pattern *pattern, -enum damos_action action,struct damos_quota *quota, +enum damos_action action, +unsigned long apply_interval_us, +struct damos_quota *quota,struct damos_watermarks *wmarks);void damon_add_scheme (struct damon_ctx *ctx,struct damos *s);void damon_destroy_scheme (struct damos *s);
mm/damon/core.c +69 -6
144 lines not on this page · raw message
mm/damon/dbgfs.c +2 -1
@@ -278,7 +278,8 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, goto fail; pos += parsed; - scheme =damon_new_scheme (&pattern, action, "a, &wmarks); + scheme =damon_new_scheme (&pattern, action,0 , "a, + &wmarks);if (!scheme) goto fail;
mm/damon/lru_sort.c +2 -0
@@ -158,6 +158,8 @@ static struct damos *damon_lru_sort_new_scheme( pattern,/* (de)prioritize on LRU-lists */ action, +/* for each aggregation interval */ +0 ,/* under the quota. */ "a,/* (De)activate this according to the watermarks. */
mm/damon/reclaim.c +2 -0
@@ -142,6 +142,8 @@ static struct damos *damon_reclaim_new_scheme(void) &pattern,/* page out those, as soon as found */ DAMOS_PAGEOUT , +/* for each aggregation interval */ +0 ,/* under the quota. */ &damon_reclaim_quota,/* (De)activate this according to the watermarks. */
mm/damon/sysfs-schemes.c +1 -1
@@ -1610,7 +1610,7 @@ static struct damos *damon_sysfs_mk_scheme( .low = sysfs_wmarks->low, }; - scheme =damon_new_scheme (&pattern, sysfs_scheme->action, "a, + scheme =damon_new_scheme (&pattern, sysfs_scheme->action,0 , "a, &wmarks);if (!scheme)return NULL ;
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 5/8 Docs/mm/damon/design: document DAMOS apply intervals
In reply to SeongJae ParkUpdate DAMON design doc to explain about DAMOS apply intervals.
--- Documentation/mm/damon/design.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Documentation/mm/damon/design.rst +2 -1
@@ -259,7 +259,8 @@ works, DAMON provides a feature called Data Access Monitoring-based Operation Schemes (DAMOS). It lets users specify their desired schemes at a high level. For such specifications, DAMON starts monitoring, finds regions having the access pattern of interest, and applies the user-desired operation actions -to the regions as soon as found. +to the regions, for every user-specified time interval called +``apply_interval``. .. _damon_design_damos_action:
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 6/8 mm/damon/sysfs-schemes: support DAMOS apply interval
In reply to SeongJae ParkUpdate DAMON sysfs interval to support DAMOS apply intervals. Users can set and get the interval for each scheme in microseconds by writing to and reading from the new sysfs file, 'apply_interval_us', in each scheme sysfs directory.
--- mm/damon/sysfs-schemes.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-)
mm/damon/sysfs-schemes.c +34 -4
@@ -1121,6 +1121,7 @@ struct damon_sysfs_scheme {struct kobject kobj;enum damos_action action;struct damon_sysfs_access_pattern *access_pattern; +unsigned long apply_interval_us;struct damon_sysfs_quotas *quotas;struct damon_sysfs_watermarks *watermarks;struct damon_sysfs_scheme_filters *filters;
@@ -1141,7 +1142,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { };static struct damon_sysfs_scheme *damon_sysfs_scheme_alloc ( -enum damos_action action) +enum damos_action action,unsigned long apply_interval_us) {struct damon_sysfs_scheme *scheme =kmalloc (sizeof (*scheme),GFP_KERNEL );
@@ -1150,6 +1151,7 @@ static struct damon_sysfs_scheme *damon_sysfs_scheme_alloc(return NULL ; scheme->kobj = (struct kobject ){}; scheme->action = action; + scheme->apply_interval_us = apply_interval_us;return scheme; }
@@ -1353,6 +1355,25 @@ static ssize_t action_store(struct kobject *kobj, struct kobj_attribute *attr,return -EINVAL ; } +static ssize_t apply_interval_us_show (struct kobject *kobj, +struct kobj_attribute *attr,char *buf) +{ +struct damon_sysfs_scheme *scheme =container_of (kobj, + struct damon_sysfs_scheme, kobj); + +return sysfs_emit (buf,"%lu\n" , scheme->apply_interval_us); +} + +static ssize_t apply_interval_us_store (struct kobject *kobj, +struct kobj_attribute *attr,const char *buf,size_t count) +{ +struct damon_sysfs_scheme *scheme =container_of (kobj, + struct damon_sysfs_scheme, kobj); +int err =kstrtoul (buf,0 , &scheme->apply_interval_us); + +return err ? err : count; +} +static void damon_sysfs_scheme_release (struct kobject *kobj) {kfree (container_of(kobj ,struct damon_sysfs_scheme ,kobj ));
@@ -1361,8 +1382,12 @@ static void damon_sysfs_scheme_release(struct kobject *kobj)static struct kobj_attribute damon_sysfs_scheme_action_attr =__ATTR_RW_MODE (action,0600 ); +static struct kobj_attribute damon_sysfs_scheme_apply_interval_us_attr = +__ATTR_RW_MODE (apply_interval_us,0600 ); +static struct attribute *damon_sysfs_scheme_attrs[] = { &damon_sysfs_scheme_action_attr.attr, + &damon_sysfs_scheme_apply_interval_us_attr.attr,NULL , };ATTRIBUTE_GROUPS (damon_sysfs_scheme);
@@ -1413,7 +1438,11 @@ static int damon_sysfs_schemes_add_dirs(struct damon_sysfs_schemes *schemes, schemes->schemes_arr = schemes_arr;for (i =0 ; i < nr_schemes; i++) { - scheme =damon_sysfs_scheme_alloc (DAMOS_STAT ); +/* +* apply_interval_us as 0 means same to aggregation interval +* (same to before-apply_interval behavior) +*/ + scheme =damon_sysfs_scheme_alloc (DAMOS_STAT ,0 );if (!scheme) {damon_sysfs_schemes_rm_dirs (schemes);return -ENOMEM ;
@@ -1610,8 +1639,8 @@ static struct damos *damon_sysfs_mk_scheme( .low = sysfs_wmarks->low, }; - scheme =damon_new_scheme (&pattern, sysfs_scheme->action,0 , "a, - &wmarks); + scheme =damon_new_scheme (&pattern, sysfs_scheme->action, + sysfs_scheme->apply_interval_us, "a, &wmarks);if (!scheme)return NULL ;
@@ -1641,6 +1670,7 @@ static void damon_sysfs_update_scheme(struct damos *scheme, scheme->pattern.max_age_region = access_pattern->age->max; scheme->action = sysfs_scheme->action; + scheme->apply_interval_us = sysfs_scheme->apply_interval_us; scheme->quota.ms = sysfs_quotas->ms; scheme->quota.sz = sysfs_quotas->sz;
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 7/8 Docs/admin-guide/mm/damon/usage: update for DAMOS apply intervals
In reply to SeongJae ParkUpdate DAMON usage document's DAMON sysfs interface section for the newly added DAMOS apply intervals support (apply_interval_us file).
--- Documentation/admin-guide/mm/damon/usage.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
Documentation/admin-guide/mm/damon/usage.rst +6 -3
@@ -76,7 +76,7 @@ comma (","). :: │ │ │ │ │ │ │ │ ... │ │ │ │ │ │ ... │ │ │ │ │ schemes/nr_schemes - │ │ │ │ │ │ 0/action + │ │ │ │ │ │ 0/action,apply_interval_us │ │ │ │ │ │ │ access_pattern/ │ │ │ │ │ │ │ │ sz/min,max │ │ │ │ │ │ │ │ nr_accesses/min,max
@@ -269,8 +269,8 @@ schemes/<N>/ ------------ In each scheme directory, five directories (``access_pattern``, ``quotas``, -``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and one file -(``action``) exist. +``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and two files +(``action`` and ``apply_interval``) exist. The ``action`` file is for setting and getting the scheme's :ref:`action <damon_design_damos_action>`. The keywords that can be written to and read
@@ -296,6 +296,9 @@ Note that support of each action depends on the running DAMON operations set - ``stat``: Do nothing but count the statistics. Supported by all operations sets. +The ``apply_interval_us`` file is for setting and getting the scheme's +:ref:`apply_interval <damon_design_damos>` in microseconds. + schemes/<N>/access_pattern/ ---------------------------
Signature · 1 line
2.25.1
SeongJae Park
maintainerRFC 8/8 Docs/ABI/damon: update for DAMOS apply intervals
In reply to SeongJae ParkUpdate DAMON ABI document for the newly added DAMON sysfs file for DAMOS apply intervals (apply_interval_us file).
--- Documentation/ABI/testing/sysfs-kernel-mm-damon | 7 +++++++ 1 file changed, 7 insertions(+)
Documentation/ABI/testing/sysfs-kernel-mm-damon +7 -0
@@ -151,6 +151,13 @@ Contact: SeongJae Park <[email protected]> Description: Writing to and reading from this file sets and gets the action of the scheme. +What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/apply_interval_us +Date: Sep 2023 +Contact: SeongJae Park <[email protected]> +Description: Writing a value to this file sets the action apply interval of + the scheme in microseconds. Reading this file returns the + value. + What: /sys/kernel/mm/damon/admin/kdamonds/<K>/contexts/<C>/schemes/<S>/access_pattern/sz/min Date: Mar 2022 Contact: SeongJae Park <[email protected]>
Signature · 1 line
2.25.1