Docs/damon: minor fixups and improvements
Subsystems (4)
SeongJae Park
maintainercover Docs/damon: minor fixups and improvements
Fixup typos, clarify regions merging operation design with recent change, add access pattern snapshot example use case, and improve readability of the design document and subsystem documents index by reorganizing/wordsmithing and adding links to other sections and/or documents for easy browsing.
SeongJae Park (9):
Docs/mm/damon/design: fix two typos
Docs/mm/damon/design: clarify regions merging operation Docs/admin-guide/mm/damon/start: add access pattern snapshot example Docs/mm/damon/design: add links from overall architecture to sections
of details
Docs/mm/damon/design: move 'Configurable Operations Set' section into
'Operations Set Layer' section
Docs/mm/damon/design: Remove 'Programmable Modules' section in favor
of 'Modules' section
Docs/mm/damon/design: add links to sections of DAMON sysfs interface
usage doc
Docs/mm/damon/index: add links to design Docs/mm/damon/index: add links to admin-guide doc
Documentation/admin-guide/mm/damon/start.rst | 46 +++++- Documentation/mm/damon/design.rst | 145 +++++++++++++------ Documentation/mm/damon/index.rst | 22 +-- 3 files changed, 152 insertions(+), 61 deletions(-)
base-commit: 32b57f0583f399fc92419cb9db525d7b174d6d77
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 1/9 Docs/mm/damon/design: fix two typos
Fix two typos. The first one is just a simple typo:
s/accurach/accuracy/
The second one is made by the author being out of their mind. 'Region Based Sampling' section of the doc is mistakenly calling the access frequency counter of region as 'nr_regions'. Fix it with the correct name, 'nr_accesses'.
--- Documentation/mm/damon/design.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/mm/damon/design.rst +2 -2
@@ -25,7 +25,7 @@ DAMON subsystem is configured with three layers including - Operations Set: Implements fundamental operations for DAMON that depends on the given monitoring target address-space and available set of software/hardware primitives, -- Core: Implements core logics including monitoring overhead/accurach control +- Core: Implements core logics including monitoring overhead/accuracy control and access-aware system operations on top of the operations set layer, and - Modules: Implements kernel modules for various purposes that provides interfaces for the user space, on top of the core layer.
@@ -192,7 +192,7 @@ one page in the region is required to be checked. Thus, for each ``sampling interval``, DAMON randomly picks one page in each region, waits for one ``sampling interval``, checks whether the page is accessed meanwhile, and increases the access frequency counter of the region if so. The counter is -called ``nr_regions`` of the region. Therefore, the monitoring overhead is +called ``nr_accesses`` of the region. Therefore, the monitoring overhead is controllable by setting the number of regions. DAMON allows users to set the minimum and the maximum number of regions for the trade-off.
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 2/9 Docs/mm/damon/design: clarify regions merging operation
In reply to SeongJae ParkDAMON design document is not explaining how min_nr_regions limit is kept, and what happens if the number of regions exceeds max_nr_regions. Add more clarification for those.
--- Documentation/mm/damon/design.rst | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-)
Documentation/mm/damon/design.rst +12 -5
@@ -209,11 +209,18 @@ the data access pattern can be dynamically changed. This will result in low monitoring quality. To keep the assumption as much as possible, DAMON adaptively merges and splits each region based on their access frequency. -For each ``aggregation interval``, it compares the access frequencies of -adjacent regions and merges those if the frequency difference is small. Then, -after it reports and clears the aggregated access frequency of each region, it -splits each region into two or three regions if the total number of regions -will not exceed the user-specified maximum number of regions after the split. +For each ``aggregation interval``, it compares the access frequencies +(``nr_accesses``) of adjacent regions. If the difference is small, and if the +sum of the two regions' sizes is smaller than the size of total regions divided +by the ``minimum number of regions``, DAMON merges the two regions. If the +resulting number of total regions is still higher than ``maximum number of +regions``, it repeats the merging with increasing access frequenceis difference +threshold until the upper-limit of the number of regions is met, or the +threshold becomes higher than possible maximum value (``aggregation interval`` +divided by ``sampling interval``). Then, after it reports and clears the +aggregated access frequency of each region, it splits each region into two or +three regions if the total number of regions will not exceed the user-specified +maximum number of regions after the split. In this way, DAMON provides its best-effort quality and minimal overhead while keeping the bounds users set for their trade-off.
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 3/9 Docs/admin-guide/mm/damon/start: add access pattern snapshot example
In reply to SeongJae ParkDAMON user-space tool (damo) provides access pattern snapshot feature, which is expected to be frequently used for real time access pattern analysis. The snapshot output is also showing what DAMON provides on its own, including the 'age' information.
In contrast, the recorded access patterns, which is shown as an example usage on the quick start section, shows what users can make from what DAMON provided. It includes information that generated outside of DAMON and makes the 'age' concept bit unclear. Hence snapshot output is easier at understanding the raw realtime output of DAMON. Add the snapshot usage example on the quick start section.
--- Documentation/admin-guide/mm/damon/start.rst | 46 ++++++++++++++++++-- 1 file changed, 42 insertions(+), 4 deletions(-)
Documentation/admin-guide/mm/damon/start.rst +42 -4
@@ -34,18 +34,56 @@ detail) of DAMON, you should ensure :doc:`sysfs </filesystems/sysfs>` is mounted. +Snapshot Data Access Patterns +============================= + +The commands below show the memory access pattern of a program at the moment of +the execution. :: + + $ git clone https://github.com/sjp38/masim; cd masim; make + $ sudo damo start "./masim ./configs/stairs.cfg --quiet" + $ sudo ./damo show + 0 addr [85.541 TiB , 85.541 TiB ) (57.707 MiB ) access 0 % age 10.400 s + 1 addr [85.541 TiB , 85.542 TiB ) (413.285 MiB) access 0 % age 11.400 s + 2 addr [127.649 TiB , 127.649 TiB) (57.500 MiB ) access 0 % age 1.600 s + 3 addr [127.649 TiB , 127.649 TiB) (32.500 MiB ) access 0 % age 500 ms + 4 addr [127.649 TiB , 127.649 TiB) (9.535 MiB ) access 100 % age 300 ms + 5 addr [127.649 TiB , 127.649 TiB) (8.000 KiB ) access 60 % age 0 ns + 6 addr [127.649 TiB , 127.649 TiB) (6.926 MiB ) access 0 % age 1 s + 7 addr [127.998 TiB , 127.998 TiB) (120.000 KiB) access 0 % age 11.100 s + 8 addr [127.998 TiB , 127.998 TiB) (8.000 KiB ) access 40 % age 100 ms + 9 addr [127.998 TiB , 127.998 TiB) (4.000 KiB ) access 0 % age 11 s + total size: 577.590 MiB + $ sudo ./damo stop + +The first command of the above example downloads and builds an artificial +memory access generator program called ``masim``. The second command asks DAMO +to execute the artificial generator process start via the given command and +make DAMON monitors the generator process. The third command retrieves the +current snapshot of the monitored access pattern of the process from DAMON and +shows the pattern in a human readable format. + +Each line of the output shows which virtual address range (``addr [XX, XX)``) +of the process is how frequently (``access XX %``) accessed for how long time +(``age XX``). For example, the fifth region of ~9 MiB size is being most +frequently accessed for last 300 milliseconds. Finally, the fourth command +stops DAMON. + +Note that DAMON can monitor not only virtual address spaces but multiple types +of address spaces including the physical address space. + + Recording Data Access Patterns ============================== The commands below record the memory access patterns of a program and save the monitoring results to a file. :: - $ git clone https://github.com/sjp38/masim - $ cd masim; make; ./masim ./configs/zigzag.cfg & + $ ./masim ./configs/zigzag.cfg & $ sudo damo record -o damon.data $(pidof masim) -The first two lines of the commands download an artificial memory access -generator program and run it in the background. The generator will repeatedly +The line of the commands run the artificial memory access +generator program again. The generator will repeatedly access two 100 MiB sized memory regions one by one. You can substitute this with your real workload. The last line asks ``damo`` to record the access pattern in the ``damon.data`` file.
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 4/9 Docs/mm/damon/design: add links from overall architecture to sections of details
In reply to SeongJae ParkDAMON design document briefly explains the overall layers architecture first, and then provides detailed explanations of each layer with dedicated sections. Letting readers go directly to the detailed sections for specific layers could help easy browsing of the not-very-short document. Add links from the overall summary to the sections of details.
--- Documentation/mm/damon/design.rst | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-)
Documentation/mm/damon/design.rst +13 -7
@@ -22,13 +22,15 @@ Overall Architecture DAMON subsystem is configured with three layers including -- Operations Set: Implements fundamental operations for DAMON that depends on - the given monitoring target address-space and available set of - software/hardware primitives, -- Core: Implements core logics including monitoring overhead/accuracy control - and access-aware system operations on top of the operations set layer, and -- Modules: Implements kernel modules for various purposes that provides - interfaces for the user space, on top of the core layer. +- :ref:`Operations Set <damon_operations_set>`: Implements fundamental + operations for DAMON that depends on the given monitoring target + address-space and available set of software/hardware primitives, +- :ref:`Core <damon_core_logic>`: Implements core logics including monitoring + overhead/accuracy control and access-aware system operations on top of the + operations set layer, and +- :ref:`Modules <damon_modules>`: Implements kernel modules for various + purposes that provides interfaces for the user space, on top of the core + layer. .. _damon_design_configurable_operations_set:
@@ -140,6 +142,8 @@ conflict with the reclaim logic using ``PG_idle`` and ``PG_young`` page flags, as Idle page tracking does. +.. _damon_core_logic: + Core Logics ===========
@@ -512,6 +516,8 @@ interface, namely ``include/linux/damon.h``. Please refer to the API :doc:`document </mm/damon/api>` for details of the interface. +.. _damon_modules: + Modules =======
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 5/9 Docs/mm/damon/design: move 'Configurable Operations Set' section into 'Operations Set Layer' section
In reply to SeongJae Park'Configurable Operations Set' section is for providing a description of the pluggability of the operations set layer. Just after that, 'Operations Set Layer' section, which is dedicated for the entire things of the layer, follows. The layout is odd, and some descriptions are duplicated. Move 'Configurable Operations Set' section into 'Operations Set Layer' and re-write some of the detailed descriptions.
--- Documentation/mm/damon/design.rst | 47 +++++++++++++++---------------- 1 file changed, 22 insertions(+), 25 deletions(-)
Documentation/mm/damon/design.rst +22 -25
@@ -33,30 +33,6 @@ DAMON subsystem is configured with three layers including layer. -.. _damon_design_configurable_operations_set: - -Configurable Operations Set ---------------------------- - -For data access monitoring and additional low level work, DAMON needs a set of -implementations for specific operations that are dependent on and optimized for -the given target address space. On the other hand, the accuracy and overhead -tradeoff mechanism, which is the core logic of DAMON, is in the pure logic -space. DAMON separates the two parts in different layers, namely DAMON -Operations Set and DAMON Core Logics Layers, respectively. It further defines -the interface between the layers to allow various operations sets to be -configured with the core logic. - -Due to this design, users can extend DAMON for any address space by configuring -the core logic to use the appropriate operations set. If any appropriate set -is unavailable, users can implement one on their own. - -For example, physical memory, virtual memory, swap space, those for specific -processes, NUMA nodes, files, and backing memory devices would be supportable. -Also, if some architectures or devices supporting special optimized access -check primitives, those will be easily configurable. - - Programmable Modules --------------------
@@ -72,11 +48,32 @@ used by the user space end users. Operations Set Layer ==================== -The monitoring operations are defined in two parts: +.. _damon_design_configurable_operations_set: + +For data access monitoring and additional low level work, DAMON needs a set of +implementations for specific operations that are dependent on and optimized for +the given target address space. For example, below two operations for access +monitoring are address-space dependent. 1. Identification of the monitoring target address range for the address space. 2. Access check of specific address range in the target space. +DAMON consolidates these implementations in a layer called DAMON Operations +Set, and defines the interface between it and the upper layer. The upper layer +is dedicated for DAMON's core logics including the mechanism for control of the +monitoring accruracy and the overhead. + +Hence, DAMON can easily be extended for any address space and/or available +hardware features by configuring the core logic to use the appropriate +operations set. If there is no available operations set for a given purpose, a +new operations set can be implemented following the interface between the +layers. + +For example, physical memory, virtual memory, swap space, those for specific +processes, NUMA nodes, files, and backing memory devices would be supportable. +Also, if some architectures or devices support special optimized access check +features, those will be easily configurable. + DAMON currently provides below three operation sets. Below two subsections describe how those work.
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 6/9 Docs/mm/damon/design: Remove 'Programmable Modules' section in favor of 'Modules' section
In reply to SeongJae Park'Programmable Modules' section provides high level descriptions of the DAMON API-based kernel modules layer. But 'Modules' section, which is at the end of the document, provides every detail about the layer including that of 'Programmable Modules' section.
Since the brief summary of the layers at the beginning of the document has a link to the 'Modules' section, browsing to the section is not that difficult. Remove 'Programmable Modules' section in favor of 'Modules' section and reducing duplicates.
--- Documentation/mm/damon/design.rst | 10 ---------- 1 file changed, 10 deletions(-)
Documentation/mm/damon/design.rst +0 -10
@@ -33,16 +33,6 @@ DAMON subsystem is configured with three layers including layer. -Programmable Modules --------------------- - -Core layer of DAMON is implemented as a framework, and exposes its application -programming interface to all kernel space components such as subsystems and -modules. For common use cases of DAMON, DAMON subsystem provides kernel -modules that built on top of the core layer using the API, which can be easily -used by the user space end users. - - .. _damon_operations_set: Operations Set Layer
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 7/9 Docs/mm/damon/design: add links to sections of DAMON sysfs interface usage doc
In reply to SeongJae ParkReaders of the design document would wonder how they can configure and use specific DAMON features. Add links to sections of DAMON sysfs interface usage document that provides the answers for easier browsing.
--- Documentation/mm/damon/design.rst | 48 +++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)
Documentation/mm/damon/design.rst +48 -0
@@ -16,6 +16,9 @@ called DAMON ``context``. DAMON executes each context with a kernel thread called ``kdamond``. Multiple kdamonds could run in parallel, for different types of monitoring. +To know how user-space can do the configurations and start/stop DAMON, refer to +:ref:`DAMON sysfs interface <sysfs_interface>` documentation. + Overall Architecture ====================
@@ -71,6 +74,10 @@ describe how those work. - fvaddr: Monitor fixed virtual address ranges - paddr: Monitor the physical address space of the system +To know how user-space can do the configuration via :ref:`DAMON sysfs interface +<sysfs_interface>`, refer to :ref:`operations <sysfs_context>` file part of the +documentation. + .. _damon_design_vaddr_target_regions_construction:
@@ -143,6 +150,10 @@ monitoring attributes, ``sampling interval``, ``aggregation interval``, ``update interval``, ``minimum number of regions``, and ``maximum number of regions``. +To know how user-space can set the attributes via :ref:`DAMON sysfs interface +<sysfs_interface>`, refer to :ref:`monitoring_attrs <sysfs_monitoring_attrs>` +part of the documentation. + Access Frequency Monitoring ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -246,6 +257,11 @@ and applies it to monitoring operations-related data structures such as the abstracted monitoring target memory area only for each of a user-specified time interval (``update interval``). +User-space can get the monitoring results via DAMON sysfs interface and/or +tracepoints. For more details, please refer to the documentations for +:ref:`DAMOS tried regions <sysfs_schemes_tried_regions>` and :ref:`tracepoint`, +respectively. + .. _damon_design_damos:
@@ -286,6 +302,10 @@ the access pattern of interest, and applies the user-desired operation actions to the regions, for every user-specified time interval called ``apply_interval``. +To know how user-space can set ``apply_interval`` via :ref:`DAMON sysfs +interface <sysfs_interface>`, refer to :ref:`apply_interval_us <sysfs_scheme>` +part of the documentation. + .. _damon_design_damos_action:
@@ -334,6 +354,10 @@ Applying the actions except ``stat`` to a region is considered as changing the region's characteristics. Hence, DAMOS resets the age of regions when any such actions are applied to those. +To know how user-space can set the action via :ref:`DAMON sysfs interface +<sysfs_interface>`, refer to :ref:`action <sysfs_scheme>` part of the +documentation. + .. _damon_design_damos_access_pattern:
@@ -347,6 +371,10 @@ interest by setting minimum and maximum values of the three properties. If a region's three properties are in the ranges, DAMOS classifies it as one of the regions that the scheme is having an interest in. +To know how user-space can set the access pattern via :ref:`DAMON sysfs +interface <sysfs_interface>`, refer to :ref:`access_pattern +<sysfs_access_pattern>` part of the documentation. + .. _damon_design_damos_quotas:
@@ -366,6 +394,10 @@ feature called quotas. It lets users specify an upper limit of time that DAMOS can use for applying the action, and/or a maximum bytes of memory regions that the action can be applied within a user-specified time duration. +To know how user-space can set the basic quotas via :ref:`DAMON sysfs interface +<sysfs_interface>`, refer to :ref:`quotas <sysfs_quotas>` part of the +documentation. + .. _damon_design_damos_quotas_prioritization:
@@ -393,6 +425,10 @@ information to the underlying mechanism. Nevertheless, how and even whether the weight will be respected are up to the underlying prioritization mechanism implementation. +To know how user-space can set the prioritization weights via :ref:`DAMON sysfs +interface <sysfs_interface>`, refer to :ref:`weights <sysfs_quotas>` part of +the documentation. + .. _damon_design_damos_quotas_auto_tuning:
@@ -422,6 +458,10 @@ Currently, two ``target_metric`` are provided. DAMOS does the measurement on its own, so only ``target_value`` need to be set by users at the initial time. In other words, DAMOS does self-feedback. +To know how user-space can set the tuning goal metric, the target value, and/or +the current value via :ref:`DAMON sysfs interface <sysfs_interface>`, refer to +:ref:`quota goals <sysfs_schemes_quota_goals>` part of the documentation. + .. _damon_design_damos_watermarks:
@@ -444,6 +484,10 @@ is activated. If all schemes are deactivated by the watermarks, the monitoring is also deactivated. In this case, the DAMON worker thread only periodically checks the watermarks and therefore incurs nearly zero overhead. +To know how user-space can set the watermarks via :ref:`DAMON sysfs interface +<sysfs_interface>`, refer to :ref:`watermarks <sysfs_watermarks>` part of the +documentation. + .. _damon_design_damos_filters:
@@ -490,6 +534,10 @@ Below types of filters are currently supported. - Applied to pages that belonging to a given DAMON monitoring target. - Handled by the core logic. +To know how user-space can set the watermarks via :ref:`DAMON sysfs interface +<sysfs_interface>`, refer to :ref:`filters <sysfs_filters>` part of the +documentation. + Application Programming Interface ---------------------------------
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 8/9 Docs/mm/damon/index: add links to design
In reply to SeongJae ParkDAMON subsystem documents index page provides a short intro of DAMON core concepts. Add links to sections of the design document to let users easily browse to the details.
--- Documentation/mm/damon/design.rst | 1 + Documentation/mm/damon/index.rst | 11 ++++++----- 2 files changed, 7 insertions(+), 5 deletions(-)
Documentation/mm/damon/design.rst +1 -0
@@ -141,6 +141,7 @@ as Idle page tracking does. Core Logics =========== +.. _damon_design_monitoring: Monitoring ----------
Documentation/mm/damon/index.rst +6 -5
@@ -6,7 +6,7 @@ DAMON: Data Access MONitor DAMON is a Linux kernel subsystem that provides a framework for data access monitoring and the monitoring results based system operations. The core -monitoring mechanisms of DAMON (refer to :doc:`design` for the detail) make it +monitoring :ref:`mechanisms <damon_design_monitoring>` of DAMON make it - *accurate* (the monitoring output is useful enough for DRAM level memory management; It might not appropriate for CPU Cache levels, though),
@@ -21,10 +21,11 @@ users who have special information about their workloads can write personalized applications for better understanding and optimizations of their workloads and systems. -For easier development of such systems, DAMON provides a feature called DAMOS -(DAMon-based Operation Schemes) in addition to the monitoring. Using the -feature, DAMON users in both kernel and user spaces can do access-aware system -operations with no code but simple configurations. +For easier development of such systems, DAMON provides a feature called +:ref:`DAMOS <damon_design_damos>` (DAMon-based Operation Schemes) in addition +to the monitoring. Using the feature, DAMON users in both kernel and user +spaces can do access-aware system operations with no code but simple +configurations. .. toctree:: :maxdepth: 2
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 9/9 Docs/mm/damon/index: add links to admin-guide doc
In reply to SeongJae ParkReaders of DAMON subsystem documents index would want to further learn how they can use DAMON from the user-space. Add the link to the admin guide.
--- Documentation/mm/damon/index.rst | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
Documentation/mm/damon/index.rst +10 -7
@@ -16,16 +16,16 @@ monitoring :ref:`mechanisms <damon_design_monitoring>` of DAMON make it of the size of target workloads). Using this framework, therefore, the kernel can operate system in an -access-aware fashion. Because the features are also exposed to the user space, -users who have special information about their workloads can write personalized -applications for better understanding and optimizations of their workloads and -systems. +access-aware fashion. Because the features are also exposed to the :doc:`user +space </admin-guide/mm/damon/index>`, users who have special information about +their workloads can write personalized applications for better understanding +and optimizations of their workloads and systems. For easier development of such systems, DAMON provides a feature called :ref:`DAMOS <damon_design_damos>` (DAMon-based Operation Schemes) in addition -to the monitoring. Using the feature, DAMON users in both kernel and user -spaces can do access-aware system operations with no code but simple -configurations. +to the monitoring. Using the feature, DAMON users in both kernel and :doc:`user +spaces </admin-guide/mm/damon/index>` can do access-aware system operations +with no code but simple configurations. .. toctree:: :maxdepth: 2
@@ -34,3 +34,6 @@ configurations. design api maintainer-profile + +To utilize and control DAMON from the user-space, please refer to the +administration :doc:`guide </admin-guide/mm/damon/index>`.
Signature · 1 line
2.39.2