Skip to contentSkip to conversation

Docs/mm/damon: add tuning guide and misc updates

RFC PATCH New SeongJae Park 1 participant 6 messages Jan 2, 2025

Subsystems (4)
lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

cover Docs/mm/damon: add tuning guide and misc updates

Add DAMON monitoring parameters tuning guide, with misc documentation fixes, clarifications and updates.

SeongJae Park (5):
Docs/mm/damon/design: add monitoring parameters tuning guide Docs/mm/damon: add an example monitoring intervals tuning Docs/admin-guide/mm/damon/usage: fix and add missing DAMOS filter
sysfs files on files hierarchy
Docs/admin-guide/mm/damon/start: update snapshot example mm/damon: explain "effective quota" on kernel-doc comment

 Documentation/admin-guide/mm/damon/start.rst  |  67 +++--
 Documentation/admin-guide/mm/damon/usage.rst  |   2 +-
 Documentation/mm/damon/design.rst             |  57 ++++
 .../monitoring_intervals_tuning_example.rst   | 247 ++++++++++++++++++
 include/linux/damon.h                         |  13 +-
 5 files changed, 354 insertions(+), 32 deletions(-)
 create mode 100644 Documentation/mm/damon/monitoring_intervals_tuning_example.rst
Signature · 1 line
2.39.5
author maintainer
#1 lore raw

RFC PATCH 1/5 Docs/mm/damon/design: add monitoring parameters tuning guide

DAMON monitoring parameters including sampling and aggregation intervals require tunings for given workloads. However, the fact is not explicitly documented. Also there is no official guide to help the tuning. This apparently confused a number of people[1]. Add a guide on the design document.

[1] https://lore.kernel.org/[email protected]

---
 Documentation/mm/damon/design.rst | 48 +++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
Documentation/mm/damon/design.rst +48 -0
index 0265aaef2544..cc405e15d1b2 100644
@@ -203,6 +203,8 @@ This scheme, however, cannot preserve the quality of the output if the
 assumption is not guaranteed.
 
 
+.. _damon_design_adaptive_regions_adjustment:
+
 Adaptive Regions Adjustment
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -264,6 +266,52 @@ tracepoints.  For more details, please refer to the documentations for
 respectively.
 
 
+Monitoring Parameters Tuning Guide
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In short, set ``aggregation interval`` to capture meaningful amount of accesses
+for the purpose.  The amount of accesses can be measured using ``nr_accesses``
+and ``age`` of regions in the aggregated monitoring results snapshot.  The
+default value of the interval, ``100ms``, turns out to be too short in many
+cases.  Set ``sampling interval`` proportional to ``aggregation interval``.  By
+default, ``1/20`` is recommended as the ratio.
+
+``Aggregation interval`` should be set as the time interval that the workload
+can make an amount of accesses for the monitoring purpose, within the interval.
+If the interval is too short, only small number of accesses are captured.  As a
+result, the monitoring results look everything is samely accessed only rarely.
+For many purposes, that would be useless.  If it is too long, however, the time
+to converge regions with the :ref:`regions adjustment mechanism
+<damon_design_adaptive_regions_adjustment>` can be too long, depending on the
+time scale of the given purpose.  This could happen if the workload is actually
+making only rare accesses but the user thinks the amount of accesses for the
+monitoring purpose too high.  For such cases, the target amount of access to
+capture per ``aggregation interval`` should carefully reconsidered.  Also, note
+that the captured amount of accesses is represented with not only
+``nr_accesses``, but also ``age``.  For example, even if every region on the
+monitoring results show zero ``nr_accesses``, regions could still be
+distinguished using ``age`` values as the recency information.
+
+Hence the optimum value of ``aggregation interval`` depends on the access
+intensiveness of the workload.  The user should tune the interval based on the
+amount of access that captured on each aggregated snapshot of the monitoring
+results.
+
+Note that the default value of the interval is 100 milliseconds, which is too
+short in many cases, especially on large systems.
+
+``Sampling interval`` defines the resolution of each aggregation.  If it is set
+too large, monitoring results will look like every region was samely rarely
+accessed, or samely frequently accessed.  That is, regions become
+undistinguishable based on access pattern, and therefore the results will be
+useless in many use cases.  If ``sampling interval`` is too small, it will not
+degrade the resolution, but will increase the monitoring overhead.  If it is
+appropriate enough to provide a resolution of the monitoring results that
+sufficient for the given purpose, it shouldn't be unnecessarily further
+lowered.  It is recommended to be set proportional to ``aggregation interval``.
+By default, the ratio is set as ``1/20``, and it is still recommended.
+
+
 .. _damon_design_damos:
 
 Operation Schemes
Signature · 1 line
2.39.5
author maintainer
#2 lore raw

RFC PATCH 2/5 Docs/mm/damon: add an example monitoring intervals tuning

In reply to SeongJae Park

Add a DAMON monitoring intervals tuning example that follows the guide. The example includes outputs of the tuning steps which are collected from a real server workloads.

The example will help users better understand the guide and what outputs they can expect in what way. The easier understanding and verification of the expectation will also help finding rooms to improvement on the guide.

---
 Documentation/mm/damon/design.rst             |   9 +
 .../monitoring_intervals_tuning_example.rst   | 247 ++++++++++++++++++
 2 files changed, 256 insertions(+)
 create mode 100644 Documentation/mm/damon/monitoring_intervals_tuning_example.rst
Documentation/mm/damon/design.rst +9 -0
index cc405e15d1b2..dd654720c433 100644
@@ -266,6 +266,8 @@ tracepoints.  For more details, please refer to the documentations for
 respectively.
 
 
+.. _damon_design_monitoring_params_tuning_guide:
+
 Monitoring Parameters Tuning Guide
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -311,6 +313,13 @@ sufficient for the given purpose, it shouldn't be unnecessarily further
 lowered.  It is recommended to be set proportional to ``aggregation interval``.
 By default, the ratio is set as ``1/20``, and it is still recommended.
 
+Refer to below documents for an example tuning based on the above guide.
+
+.. toctree::
+   :maxdepth: 1
+
+   monitoring_intervals_tuning_example
+
 
 .. _damon_design_damos:
 
Documentation/mm/damon/monitoring_intervals_tuning_example.rst new file +247 -0

247 lines not on this page · raw message

Signature · 1 line
2.39.5
author maintainer
#3 lore raw

RFC PATCH 3/5 Docs/admin-guide/mm/damon/usage: fix and add missing DAMOS filter sysfs files on files hierarchy

In reply to SeongJae Park

DAMON sysfs files hierarchy on DAMON usage document is wrong about DAMOS filter directory. The directory has 'memcg_path' file, but it wrongly describes the name as 'memcg_id'. Also the directory has 'addr_start', 'addr_end', and 'target_idx' files, but missing those. Fix the wrong name and add missed files.

---
 Documentation/admin-guide/mm/damon/usage.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/admin-guide/mm/damon/usage.rst +1 -1
index ab343d6ac771..022f72cf5e14 100644
@@ -89,7 +89,7 @@ comma (",").
     │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value
     │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
     │ │ │ │ │ │ │ :ref:`filters <sysfs_filters>`/nr_filters
-    │ │ │ │ │ │ │ │ 0/type,matching,memcg_id,pass
+    │ │ │ │ │ │ │ │ 0/type,matching,pass,memcg_path,addr_start,addr_end,target_idx
     │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds
     │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
     │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
Signature · 1 line
2.39.5
author maintainer
#4 lore raw

RFC PATCH 4/5 Docs/admin-guide/mm/damon/start: update snapshot example

In reply to SeongJae Park

Two of DAMON user-space tool (damo) commands that are used for examples on DAMON getting started document, namely 'damo show' and 'damo report heats' are deprecated[1,2], and replaced by new commands that provides same functions with a simplified user interface. Also the example output of 'damo show' is outdasted. 'damo schemes' command is not deprecated, but users are recommended to use 'damo start' instead.

Update the examples to use the replacements, recommdnations, and up-to-date output formats.

[1] https://git.kernel.org/sj/damo/c/3272e0ac94ecc5e1
[2] https://git.kernel.org/sj/damo/c/da3ec66bbdd9e87d

---
 Documentation/admin-guide/mm/damon/start.rst | 67 ++++++++++++--------
 1 file changed, 40 insertions(+), 27 deletions(-)
Documentation/admin-guide/mm/damon/start.rst +40 -27
index c4dddf6733cd..ede14b679d02 100644
@@ -42,32 +42,45 @@ the execution. ::
 
     $ git clone https://github.com/sjp38/masim; cd masim; make
     $ sudo damo start "./masim ./configs/stairs.cfg --quiet"
-    $ sudo ./damo show
-    0   addr [85.541 TiB  , 85.541 TiB ) (57.707 MiB ) access 0 %   age 10.400 s
-    1   addr [85.541 TiB  , 85.542 TiB ) (413.285 MiB) access 0 %   age 11.400 s
-    2   addr [127.649 TiB , 127.649 TiB) (57.500 MiB ) access 0 %   age 1.600 s
-    3   addr [127.649 TiB , 127.649 TiB) (32.500 MiB ) access 0 %   age 500 ms
-    4   addr [127.649 TiB , 127.649 TiB) (9.535 MiB  ) access 100 % age 300 ms
-    5   addr [127.649 TiB , 127.649 TiB) (8.000 KiB  ) access 60 %  age 0 ns
-    6   addr [127.649 TiB , 127.649 TiB) (6.926 MiB  ) access 0 %   age 1 s
-    7   addr [127.998 TiB , 127.998 TiB) (120.000 KiB) access 0 %   age 11.100 s
-    8   addr [127.998 TiB , 127.998 TiB) (8.000 KiB  ) access 40 %  age 100 ms
-    9   addr [127.998 TiB , 127.998 TiB) (4.000 KiB  ) access 0 %   age 11 s
-    total size: 577.590 MiB
-    $ sudo ./damo stop
+    $ sudo damo report access
+    heatmap: 641111111000000000000000000000000000000000000000000000[...]33333333333333335557984444[...]7
+    # min/max temperatures: -1,840,000,000, 370,010,000, column size: 3.925 MiB
+    0   addr 86.182 TiB   size 8.000 KiB   access 0 %   age 14.900 s
+    1   addr 86.182 TiB   size 8.000 KiB   access 60 %  age 0 ns
+    2   addr 86.182 TiB   size 3.422 MiB   access 0 %   age 4.100 s
+    3   addr 86.182 TiB   size 2.004 MiB   access 95 %  age 2.200 s
+    4   addr 86.182 TiB   size 29.688 MiB  access 0 %   age 14.100 s
+    5   addr 86.182 TiB   size 29.516 MiB  access 0 %   age 16.700 s
+    6   addr 86.182 TiB   size 29.633 MiB  access 0 %   age 17.900 s
+    7   addr 86.182 TiB   size 117.652 MiB access 0 %   age 18.400 s
+    8   addr 126.990 TiB  size 62.332 MiB  access 0 %   age 9.500 s
+    9   addr 126.990 TiB  size 13.980 MiB  access 0 %   age 5.200 s
+    10  addr 126.990 TiB  size 9.539 MiB   access 100 % age 3.700 s
+    11  addr 126.990 TiB  size 16.098 MiB  access 0 %   age 6.400 s
+    12  addr 127.987 TiB  size 132.000 KiB access 0 %   age 2.900 s
+    total size: 314.008 MiB
+    $ sudo damo stop
 
 The first command of the above example downloads and builds an artificial
 memory access generator program called ``masim``.  The second command asks DAMO
-to execute the artificial generator process start via the given command and
-make DAMON monitors the generator process.  The third command retrieves the
-current snapshot of the monitored access pattern of the process from DAMON and
-shows the pattern in a human readable format.
-
-Each line of the output shows which virtual address range (``addr [XX, XX)``)
-of the process is how frequently (``access XX %``) accessed for how long time
-(``age XX``).  For example, the fifth region of ~9 MiB size is being most
-frequently accessed for last 300 milliseconds.  Finally, the fourth command
-stops DAMON.
+to start the program via the given command and make DAMON monitors the newly
+started process.  The third command retrieves the current snapshot of the
+monitored access pattern of the process from DAMON and shows the pattern in a
+human readable format.
+
+The first line of the output shows the relative access temperature (hotness) of
+the regions in a single row hetmap format.  Each column on the heatmap
+represents regions of same size on the monitored virtual address space.  The
+position of the colun on the row and the number on the column represents the
+relative location and access temperature of the region.  ``[...]`` means
+unmapped huge regions on the virtual address spaces.  The second line shows
+additional information for better understanding the heatmap.
+
+Each line of the output from the third line shows which virtual address range
+(``addr XX size XX``) of the process is how frequently (``access XX %``)
+accessed for how long time (``age XX``).  For example, the evelenth region of
+~9.5 MiB size is being most frequently accessed for last 3.7 seconds.  Finally,
+the fourth command stops DAMON.
 
 Note that DAMON can monitor not only virtual address spaces but multiple types
 of address spaces including the physical address space.
@@ -95,7 +108,7 @@ Visualizing Recorded Patterns
 You can visualize the pattern in a heatmap, showing which memory region
 (x-axis) got accessed when (y-axis) and how frequently (number).::
 
-    $ sudo damo report heats --heatmap stdout
+    $ sudo damo report heatmap
     22222222222222222222222222222222222222211111111111111111111111111111111111111100
     44444444444444444444444444444444444444434444444444444444444444444444444444443200
     44444444444444444444444444444444444444433444444444444444444444444444444444444200
@@ -160,6 +173,6 @@ Data Access Pattern Aware Memory Management
 Below command makes every memory region of size >=4K that has not accessed for
 >=60 seconds in your workload to be swapped out. ::
 
-    $ sudo damo schemes --damos_access_rate 0 0 --damos_sz_region 4K max \
-                        --damos_age 60s max --damos_action pageout \
-                        <pid of your workload>
+    $ sudo damo start --damos_access_rate 0 0 --damos_sz_region 4K max \
+                      --damos_age 60s max --damos_action pageout \
+                      <pid of your workload>
Signature · 1 line
2.39.5
author maintainer
#5 lore raw

RFC PATCH 5/5 mm/damon: explain "effective quota" on kernel-doc comment

In reply to SeongJae Park

The kernel-doc comment for 'struct damos_quota' describes how "effective quota" is calculated, but does not explain what it is. Actually there was an input[1] about it. Add the explanation on the comment.

Also, fix a trivial typo on the comment block: s/empt/empty/

[1] https://github.com/damonitor/damo/issues/17#issuecomment-2497525043

---

Changes from single-RFC
(https://lore.kernel.org/[email protected])
- fix a typo and grammartical error (Honggyu Kim)
- Integrate into this patch series

 include/linux/damon.h | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
include/linux/damon.h +9 -4
index ad0360711aca..ce477a0ed6a2 100644
@@ -193,11 +193,16 @@ struct damos_quota_goal {
  * size quota is set, DAMON tries to apply the action only up to &sz bytes
  * within &reset_interval.
  *
- * Internally, the time quota is transformed to a size quota using estimated
- * throughput of the scheme's action.  DAMON then compares it against &sz and
- * uses smaller one as the effective quota.
+ * To convince the different types of quotas and goals, DAMON internally
+ * converts those into one single size quota called "effective quota".  DAMON
+ * internally uses it as the only one real quota.  The conversion is made as
+ * follows.
  *
- * If @goals is not empt, DAMON calculates yet another size quota based on the
+ * The time quota is transformed to a size quota using estimated throughput of
+ * the scheme's action.  DAMON then compares it against &sz and uses smaller
+ * one as the effective quota.
+ *
+ * If @goals is not empty, DAMON calculates yet another size quota based on the
  * goals using its internal feedback loop algorithm, for every @reset_interval.
  * Then, if the new size quota is smaller than the effective quota, it uses the
  * new size quota as the effective quota.
Signature · 1 line
2.39.5