Docs/{mm,admin-guide}damon: update design and usage docs
Subsystems (4)
SeongJae Park
maintainercover Docs/{mm,admin-guide}damon: update design and usage docs
Update DAMON design and usage documents for outdated and unnecessarily duplicated parts.
SeongJae Park (7):
Docs/mm/damon/design: document 'age' of region Docs/admin-guide/mm/damon/start: update DAMOS example command Docs/admin-guide/mm/damon/usage: fix typos in references and commas Docs/admin-guide/mm/damon/usage: remove unnecessary sentences about
supported address spaces
Docs/admin-guide/mm/damon/usage: link design document for DAMOS Docs/admin-guide/mm/damon/usage: clarify quotas and watermarks sysfs
interface
Docs/admin-guide/mm/damon/usage: update the ways for getting
monitoring results
Documentation/admin-guide/mm/damon/start.rst | 10 +- Documentation/admin-guide/mm/damon/usage.rst | 146 ++++++++----------- Documentation/mm/damon/design.rst | 28 ++++ 3 files changed, 95 insertions(+), 89 deletions(-)
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 1/7 Docs/mm/damon/design: document 'age' of region
The 'age' of each region in DAMON monitoring results is an important concept for both monitoring part and DAMOS. And DAMOS section of the design document is mentioning it. However, the age itself is not explained in the document. Add a section for that.
--- Documentation/mm/damon/design.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
Documentation/mm/damon/design.rst +14 -0
@@ -190,6 +190,20 @@ In this way, DAMON provides its best-effort quality and minimal overhead while keeping the bounds users set for their trade-off. +Age Tracking +~~~~~~~~~~~~ + +By analyzing the monitoring results, users can also find how long the current +access pattern of a region has maintained. That could be used for good +understanding of the access pattern. For example, page placement algorithm +utilizing both the frequency and the recency could be implemented using that. +To make such access pattern maintained period analysis easier, DAMON maintains +yet another counter called ``age`` in each region. For each ``aggregation +interval``, DAMON checks if the region's size and access frequency +(``nr_accesses``) has significantly changed. If so, the counter is reset to +zero. Otherwise, the counter is increased. + + Dynamic Target Space Updates Handling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 2/7 Docs/admin-guide/mm/damon/start: update DAMOS example command
In reply to SeongJae ParkDAMON user-space tool, damo, has deprecated[1] its old DAMOS schemes specification format. However, an example of DAMON documentation is still using it. Update the example to use one of the alternative options.
[1] https://github.com/awslabs/damo/commit/e9950ae68f6c
--- Documentation/admin-guide/mm/damon/start.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Documentation/admin-guide/mm/damon/start.rst +5 -5
@@ -119,9 +119,9 @@ set size has chronologically changed.:: Data Access Pattern Aware Memory Management =========================================== -Below three commands make every memory region of size >=4K that doesn't -accessed for >=60 seconds in your workload to be swapped out. :: +Below command makes every memory region of size >=4K that has not accessed for +>=60 seconds in your workload to be swapped out. :: - $ echo "#min-size max-size min-acc max-acc min-age max-age action" > test_scheme - $ echo "4K max 0 0 60s max pageout" >> test_scheme - $ damo schemes -c test_scheme <pid of your workload> + $ sudo damo schemes --damos_access_rate 0 0 --damos_sz_region 4K max \ + --damos_age 60s max --damos_action pageout \ + <pid of your workload>
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 3/7 Docs/admin-guide/mm/damon/usage: fix typos in references and commas
In reply to SeongJae ParkFix typos including a unnecessary comma and incomplete ':ref:' keywords.
--- Documentation/admin-guide/mm/damon/usage.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
Documentation/admin-guide/mm/damon/usage.rst +3 -4
@@ -139,7 +139,7 @@ scheme of the kdamond. Writing ``clear_schemes_tried_regions`` to ``state`` file clears the DAMON-based operating scheme action tried regions directory for each DAMON-based operation scheme of the kdamond. For details of the DAMON-based operation scheme action tried regions directory, please refer to -:ref:tried_regions section <sysfs_schemes_tried_regions>`. +:ref:`tried_regions section <sysfs_schemes_tried_regions>`. If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
@@ -282,7 +282,7 @@ memory regions having specific access pattern of the interest. The keywords that can be written to and read from the file and their meaning are as below. Note that support of each action depends on the running DAMON operations set -`implementation <sysfs_contexts>`. +:ref:`implementation <sysfs_contexts>`. - ``willneed``: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Supported by ``vaddr`` and ``fvaddr`` operations set.
@@ -432,8 +432,7 @@ starting from ``0`` under this directory. Each directory contains files exposing detailed information about each of the memory region that the corresponding scheme's ``action`` has tried to be applied under this directory, during next :ref:`aggregation interval <sysfs_monitoring_attrs>`. The -information includes address range, ``nr_accesses``, , and ``age`` of the -region. +information includes address range, ``nr_accesses``, and ``age`` of the region. The directories will be removed when another special keyword, ``clear_schemes_tried_regions``, is written to the relevant
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 4/7 Docs/admin-guide/mm/damon/usage: remove unnecessary sentences about supported address spaces
In reply to SeongJae ParkBrief explanation of DAMON user space tool and sysfs interface are unnecessarily and repeatedly mentioning the list of address spaces that DAMON is supporting. Remove those.
--- Documentation/admin-guide/mm/damon/usage.rst | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
Documentation/admin-guide/mm/damon/usage.rst +5 -7
@@ -10,9 +10,8 @@ DAMON provides below interfaces for different users. `This <https://github.com/awslabs/damo>`_ is for privileged people such as system administrators who want a just-working human-friendly interface. Using this, users can use the DAMON’s major features in a human-friendly way. - It may not be highly tuned for special cases, though. It supports both - virtual and physical address spaces monitoring. For more detail, please - refer to its `usage document + It may not be highly tuned for special cases, though. For more detail, + please refer to its `usage document <https://github.com/awslabs/damo/blob/next/USAGE.md>`_. - *sysfs interface.* :ref:`This <sysfs_interface>` is for privileged user space programmers who
@@ -20,10 +19,9 @@ DAMON provides below interfaces for different users. features by reading from and writing to special sysfs files. Therefore, you can write and use your personalized DAMON sysfs wrapper programs that reads/writes the sysfs files instead of you. The `DAMON user space tool - <https://github.com/awslabs/damo>`_ is one example of such programs. It - supports both virtual and physical address spaces monitoring. Note that this - interface provides only simple :ref:`statistics <damos_stats>` for the - monitoring results. For detailed monitoring results, DAMON provides a + <https://github.com/awslabs/damo>`_ is one example of such programs. Note + that this interface provides only simple :ref:`statistics <damos_stats>` for + the monitoring results. For detailed monitoring results, DAMON provides a :ref:`tracepoint <tracepoint>`. - *debugfs interface. (DEPRECATED!)* :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 5/7 Docs/admin-guide/mm/damon/usage: link design document for DAMOS
In reply to SeongJae ParkThe background and concept of DAMOS is redundantly documented, in the design document and the usage document. Replace the duplicated ones in usage document with links to the design document.
--- Documentation/admin-guide/mm/damon/usage.rst | 104 +++++++------------ Documentation/mm/damon/design.rst | 14 +++ 2 files changed, 51 insertions(+), 67 deletions(-)
Documentation/admin-guide/mm/damon/usage.rst +37 -67
@@ -257,12 +257,9 @@ be equal or smaller than ``start`` of directory ``N+1``. contexts/<N>/schemes/ --------------------- -For usual DAMON-based data access aware memory management optimizations, users -would normally want the system to apply a memory management action to a memory -region of a specific access pattern. DAMON receives such formalized operation -schemes from the user and applies those to the target memory regions. Users -can get and set the schemes by reading from and writing to files under this -directory. +The directory for DAMON-based Operation Schemes (:ref:`DAMOS +<damon_design_damos>`). Users can get and set the schemes by reading from and +writing to files under this directory. In the beginning, this directory has only one file, ``nr_schemes``. Writing a number (``N``) to the file creates the number of child directories named ``0``
@@ -275,9 +272,9 @@ In each scheme directory, five directories (``access_pattern``, ``quotas``, ``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and one file (``action``) exist. -The ``action`` file is for setting and getting what action you want to apply to -memory regions having specific access pattern of the interest. The keywords -that can be written to and read from the file and their meaning are as below. +The ``action`` file is for setting and getting the scheme's :ref:`action +<damon_design_damos_action>`. The keywords that can be written to and read +from the file and their meaning are as below. Note that support of each action depends on the running DAMON operations set :ref:`implementation <sysfs_contexts>`.
@@ -302,10 +299,8 @@ Note that support of each action depends on the running DAMON operations set schemes/<N>/access_pattern/ --------------------------- -The target access pattern of each DAMON-based operation scheme is constructed -with three ranges including the size of the region in bytes, number of -monitored accesses per aggregate interval, and number of aggregated intervals -for the age of the region. +The directory for the target access :ref:`pattern +<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme. Under the ``access_pattern`` directory, three directories (``sz``, ``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
@@ -316,18 +311,8 @@ to and reading from the ``min`` and ``max`` files under ``sz``, schemes/<N>/quotas/ ------------------- -Optimal ``target access pattern`` for each ``action`` is workload dependent, so -not easy to find. Worse yet, setting a scheme of some action too aggressive -can cause severe overhead. To avoid such overhead, users can limit time and -size quota for each scheme. In detail, users can ask DAMON to try to use only -up to specific time (``time quota``) for applying the action, and to apply the -action to only up to specific amount (``size quota``) of memory regions having -the target access pattern within a given time interval (``reset interval``). - -When the quota limit is expected to be exceeded, DAMON prioritizes found memory -regions of the ``target access pattern`` based on their size, access frequency, -and age. For personalized prioritization, users can set the weights for the -three properties. +The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given +DAMON-based operation scheme. Under ``quotas`` directory, three files (``ms``, ``bytes``, ``reset_interval_ms``) and one directory (``weights``) having three files
@@ -335,23 +320,20 @@ Under ``quotas`` directory, three files (``ms``, ``bytes``, You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and ``reset interval`` in milliseconds by writing the values to the three files, -respectively. You can also set the prioritization weights for size, access -frequency, and age in per-thousand unit by writing the values to the three -files under the ``weights`` directory. +respectively. You can also set the :ref:`prioritization weights +<damon_design_damos_quotas_prioritization>` for size, access frequency, and age +in per-thousand unit by writing the values to the three files under the +``weights`` directory. schemes/<N>/watermarks/ ----------------------- -To allow easy activation and deactivation of each scheme based on system -status, DAMON provides a feature called watermarks. The feature receives five -values called ``metric``, ``interval``, ``high``, ``mid``, and ``low``. The -``metric`` is the system metric such as free memory ratio that can be measured. -If the metric value of the system is higher than the value in ``high`` or lower -than ``low`` at the memoent, the scheme is deactivated. If the value is lower -than ``mid``, the scheme is activated. +The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the +given DAMON-based operation scheme. Under the watermarks directory, five files (``metric``, ``interval_us``, -``high``, ``mid``, and ``low``) for setting each value exist. You can set and +``high``, ``mid``, and ``low``) for setting the metric, the time interval +between check of the metric, and the three watermarks exist. You can set and get the five values by writing to the files, respectively. Keywords and meanings of those that can be written to the ``metric`` file are
@@ -365,12 +347,8 @@ The ``interval`` should written in microseconds unit. schemes/<N>/filters/ -------------------- -Users could know something more than the kernel for specific types of memory. -In the case, users could do their own management for the memory and hence -doesn't want DAMOS bothers that. Users could limit DAMOS by setting the access -pattern of the scheme and/or the monitoring regions for the purpose, but that -can be inefficient in some cases. In such cases, users could set non-access -pattern driven filters using files in this directory. +The directory for the :ref:`filters <damon_design_damos_filters>` of the given +DAMON-based operation scheme. In the beginning, this directory has only one file, ``nr_filters``. Writing a number (``N``) to the file creates the number of child directories named ``0``
@@ -597,15 +575,10 @@ update. Schemes ------- -For usual DAMON-based data access aware memory management optimizations, users -would simply want the system to apply a memory management action to a memory -region of a specific access pattern. DAMON receives such formalized operation -schemes from the user and applies those to the target processes. - -Users can get and set the schemes by reading from and writing to ``schemes`` -debugfs file. Reading the file also shows the statistics of each scheme. To -the file, each of the schemes should be represented in each line in below -form:: +Users can get and set the DAMON-based operation :ref:`schemes +<damon_design_damos>` by reading from and writing to ``schemes`` debugfs file. +Reading the file also shows the statistics of each scheme. To the file, each +of the schemes should be represented in each line in below form:: <target access pattern> <action> <quota> <watermarks>
@@ -614,8 +587,9 @@ You can disable schemes by simply writing an empty string to the file. Target Access Pattern ~~~~~~~~~~~~~~~~~~~~~ -The ``<target access pattern>`` is constructed with three ranges in below -form:: +The target access :ref:`pattern <damon_design_damos_access_pattern>` of the +scheme. The ``<target access pattern>`` is constructed with three ranges in +below form:: min-size max-size min-acc max-acc min-age max-age
@@ -628,9 +602,9 @@ closed interval. Action ~~~~~~ -The ``<action>`` is a predefined integer for memory management actions, which -DAMON will apply to the regions having the target access pattern. The -supported numbers and their meanings are as below. +The ``<action>`` is a predefined integer for memory management :ref:`actions +<damon_design_damos_action>`. The supported numbers and their meanings are as +below. - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Ignored if ``target`` is ``paddr``.
@@ -646,10 +620,8 @@ supported numbers and their meanings are as below. Quota ~~~~~ -Optimal ``target access pattern`` for each ``action`` is workload dependent, so -not easy to find. Worse yet, setting a scheme of some action too aggressive -can cause severe overhead. To avoid such overhead, users can limit time and -size quota for the scheme via the ``<quota>`` in below form:: +Users can set the :ref:`quotas <damon_design_damos_quotas>` of the given scheme +via the ``<quota>`` in below form:: <ms> <sz> <reset interval> <priority weights>
@@ -659,19 +631,17 @@ the action to memory regions of the ``target access pattern`` within the ``<sz>`` bytes of memory regions within the ``<reset interval>``. Setting both ``<ms>`` and ``<sz>`` zero disables the quota limits. -When the quota limit is expected to be exceeded, DAMON prioritizes found memory -regions of the ``target access pattern`` based on their size, access frequency, -and age. For personalized prioritization, users can set the weights for the -three properties in ``<priority weights>`` in below form:: +For the :ref:`prioritization <damon_design_damos_quotas_prioritization>`, users +can set the weights for the three properties in ``<priority weights>`` in below +form:: <size weight> <access frequency weight> <age weight> Watermarks ~~~~~~~~~~ -Some schemes would need to run based on current value of the system's specific -metrics like free memory ratio. For such cases, users can specify watermarks -for the condition.:: +Users can specify :ref:`watermarks <damon_design_damos_watermarks>` of the +given scheme via ``<watermarks>`` in below form:: <metric> <check interval> <high mark> <middle mark> <low mark>
Documentation/mm/damon/design.rst +14 -0
@@ -218,6 +218,8 @@ abstracted monitoring target memory area only for each of a user-specified time interval (``update interval``). +.. _damon_design_damos: + Operation Schemes -----------------
@@ -255,6 +257,8 @@ the access pattern of interest, and applies the user-desired operation actions to the regions as soon as found. +.. _damon_design_damos_action: + Operation Action ~~~~~~~~~~~~~~~~
@@ -277,6 +281,8 @@ characteristics. Hence, DAMOS resets the age of regions when an action is applied to those. +.. _damon_design_damos_access_pattern: + Target Access Pattern ~~~~~~~~~~~~~~~~~~~~~
@@ -288,6 +294,8 @@ region's three properties are in the ranges, DAMOS classifies it as one of the regions that the scheme is having an interest in. +.. _damon_design_damos_quotas: + Quotas ~~~~~~
@@ -305,6 +313,8 @@ can use for applying the action, and/or a maximum bytes of memory regions that the action can be applied within a user-specified time duration. +.. _damon_design_damos_quotas_prioritization: + Prioritization ^^^^^^^^^^^^^^
@@ -330,6 +340,8 @@ the weight will be respected are up to the underlying prioritization mechanism implementation. +.. _damon_design_damos_watermarks: + Watermarks ~~~~~~~~~~
@@ -350,6 +362,8 @@ is also deactivated. In this case, the DAMON worker thread only periodically checks the watermarks and therefore incurs nearly zero overhead. +.. _damon_design_damos_filters: + Filters ~~~~~~~
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 6/7 Docs/admin-guide/mm/damon/usage: clarify quotas and watermarks sysfs interface
In reply to SeongJae ParkExplanation of DAMOS quotas and watermarks are not clearly explaining the meaning and expectation of each file. Add more clarification for those.
--- Documentation/admin-guide/mm/damon/usage.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
Documentation/admin-guide/mm/damon/usage.rst +9 -2
@@ -306,7 +306,8 @@ Under the ``access_pattern`` directory, three directories (``sz``, ``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) exist. You can set and get the access pattern for the given scheme by writing to and reading from the ``min`` and ``max`` files under ``sz``, -``nr_accesses``, and ``age`` directories, respectively. +``nr_accesses``, and ``age`` directories, respectively. Note that the ``min`` +and the ``max`` form a closed interval. schemes/<N>/quotas/ -------------------
@@ -320,7 +321,13 @@ Under ``quotas`` directory, three files (``ms``, ``bytes``, You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and ``reset interval`` in milliseconds by writing the values to the three files, -respectively. You can also set the :ref:`prioritization weights +respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds +for applying the ``action`` to memory regions of the ``access_pattern``, and to +apply the action to only up to ``bytes`` bytes of memory regions within the +``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the +quota limits. + +You can also set the :ref:`prioritization weights <damon_design_damos_quotas_prioritization>` for size, access frequency, and age in per-thousand unit by writing the values to the three files under the ``weights`` directory.
Signature · 1 line
2.25.1
SeongJae Park
maintainerPATCH 7/7 Docs/admin-guide/mm/damon/usage: update the ways for getting monitoring results
In reply to SeongJae ParkThe recommended ways for getting DAMON monitoring results are using tried_regions sysfs directory for partial snapshot of the results, and DAMON tracepoint for full record of the results. However, the tried_regions sysfs directory usage has not sufficiently updated on some sections of the DAMON usage document. Update those.
--- Documentation/admin-guide/mm/damon/usage.rst | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
Documentation/admin-guide/mm/damon/usage.rst +12 -8
@@ -19,10 +19,7 @@ DAMON provides below interfaces for different users. features by reading from and writing to special sysfs files. Therefore, you can write and use your personalized DAMON sysfs wrapper programs that reads/writes the sysfs files instead of you. The `DAMON user space tool - <https://github.com/awslabs/damo>`_ is one example of such programs. Note - that this interface provides only simple :ref:`statistics <damos_stats>` for - the monitoring results. For detailed monitoring results, DAMON provides a - :ref:`tracepoint <tracepoint>`. + <https://github.com/awslabs/damo>`_ is one example of such programs. - *debugfs interface. (DEPRECATED!)* :ref:`This <debugfs_interface>` is almost identical to :ref:`sysfs interface <sysfs_interface>`. This is deprecated, so users should move to the
@@ -421,6 +418,11 @@ The directories will be removed when another special keyword, ``clear_schemes_tried_regions``, is written to the relevant ``kdamonds/<N>/state`` file. +The expected usage of this directory is investigations of schemes' behaviors, +and query-like efficient data access monitoring results retrievals. For the +latter use case, in particular, users can set the ``action`` as ``stat`` and +set the ``access pattern`` as their interested pattern that they want to query. + tried_regions/<N>/ ------------------
@@ -771,10 +773,12 @@ root directory only. Tracepoint for Monitoring Results ================================= -DAMON provides the monitoring results via a tracepoint, -``damon:damon_aggregated``. While the monitoring is turned on, you could -record the tracepoint events and show results using tracepoint supporting tools -like ``perf``. For example:: +Users can get the monitoring results via the :ref:`tried_regions +<sysfs_schemes_tried_regions>` or a tracepoint, ``damon:damon_aggregated``. +While the tried regions directory is useful for getting a snapshot, the +tracepoint is useful for getting a full record of the results. While the +monitoring is turned on, you could record the tracepoint events and show +results using tracepoint supporting tools like ``perf``. For example:: # echo on > monitor_on # perf record -e damon:damon_aggregated &
Signature · 1 line
2.25.1