Skip to contentSkip to conversation

selftests/damon: add python and drgn based DAMON sysfs functionality tests

PATCH Applied SeongJae Park 1 participant 7 messages Jun 28, 2025

Subsystems (3)
lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

cover selftests/damon: add python and drgn based DAMON sysfs functionality tests

DAMON sysfs interface is the bridge between the user space and the kernel space for DAMON parameters. There is no good and simple test to see if the parameters are set as expected. Existing DAMON selftests therefore test end-to-end features. For example, damos_quota_goal.py runs a DAMOS scheme with quota goal set against a test program running an artificial access pattern, and see if the result is as expected. Such tests cover only a few part of DAMON. Adding more tests is also complicated. Finally, the reliability of the test itself on different systems is bad.

'drgn' is a tool that can extract kernel internal data structures like DAMON parameters. Add a test that passes specific DAMON parameters via DAMON sysfs reusing _damon_sysfs.py, extract resulting DAMON parameters via 'drgn', and compare those. Note that this test is not adding exhaustive tests of all DAMON parameters and input combinations but very basic things. Advancing the test infrastructure and adding more tests are future works.

Changes from RFC
(https://lore.kernel.org/[email protected])
- Rebase on latest mm-new

SeongJae Park (6):
selftests/damon: add drgn script for extracting damon status selftests/damon/_damon_sysfs: set Kdamond.pid in start() selftests/damon: add python and drgn-based DAMON sysfs test selftests/damon/sysfs.py: test monitoring attribute parameters selftests/damon/sysfs.py: test adaptive targets parameter selftests/damon/sysfs.py: test DAMOS schemes parameters setup

 tools/testing/selftests/damon/Makefile        |   1 +
 tools/testing/selftests/damon/_damon_sysfs.py |   3 +
 .../selftests/damon/drgn_dump_damon_status.py | 161 ++++++++++++++++++
 tools/testing/selftests/damon/sysfs.py        | 115 +++++++++++++
 4 files changed, 280 insertions(+)
 create mode 100755 tools/testing/selftests/damon/drgn_dump_damon_status.py
 create mode 100755 tools/testing/selftests/damon/sysfs.py

base-commit: 5ab6feac2d83ebbf0d0d2eedf0505878ba677dcb

Signature · 1 line
2.39.5
author maintainer
#1 lore raw

PATCH 1/6 selftests/damon: add drgn script for extracting damon status

'drgn' is a useful tool for extracting kernel internal data structures such as DAMON's parameter and running status. Add a 'drgn' script that extracts such DAMON internal data at runtime, for using it as a tool for seeing if a test input has made expected results in the kernel.

The script saves or prints out the DAMON internal data as a json file or string. This is for making use of it not very depends on 'drgn'. If 'drgn' is not available on a test setup and we find alternative tools for doing that, the json-based tests can be updated to use an alternative tool in future.

Note that the script is tested with 'drgn v0.0.22'.

---
 .../selftests/damon/drgn_dump_damon_status.py | 161 ++++++++++++++++++
 1 file changed, 161 insertions(+)
 create mode 100755 tools/testing/selftests/damon/drgn_dump_damon_status.py
tools/testing/selftests/damon/drgn_dump_damon_status.py new file +161 -0
new file mode 100755
index 000000000000..333a0d0c4bff
@@ -0,0 +1,161 @@
+#!/usr/bin/env drgn
+# SPDX-License-Identifier: GPL-2.0
+
+'''
+Read DAMON context data and dump as a json string.
+'''
+import drgn
+from drgn import FaultError, NULL, Object, cast, container_of, execscript, offsetof, reinterpret, sizeof
+from drgn.helpers.common import *
+from drgn.helpers.linux import *
+
+import json
+import sys
+
+if "prog" not in globals():
+    try:
+        prog = drgn.get_default_prog()
+    except drgn.NoDefaultProgramError:
+        prog = drgn.program_from_kernel()
+        drgn.set_default_prog(prog)
+
+def to_dict(object, attr_name_converter):
+    d = {}
+    for attr_name, converter in attr_name_converter:
+        d[attr_name] = converter(getattr(object, attr_name))
+    return d
+
+def intervals_goal_to_dict(goal):
+    return to_dict(goal, [
+        ['access_bp', int],
+        ['aggrs', int],
+        ['min_sample_us', int],
+        ['max_sample_us', int],
+        ])
+
+def attrs_to_dict(attrs):
+    return to_dict(attrs, [
+        ['sample_interval', int],
+        ['aggr_interval', int],
+        ['ops_update_interval', int],
+        ['intervals_goal', intervals_goal_to_dict],
+        ['min_nr_regions', int],
+        ['max_nr_regions', int],
+        ])
+
+def addr_range_to_dict(addr_range):
+    return to_dict(addr_range, [
+        ['start', int],
+        ['end', int],
+        ])
+
+def region_to_dict(region):
+    return to_dict(region, [
+        ['ar', addr_range_to_dict],
+        ['sampling_addr', int],
+        ['nr_accesses', int],
+        ['nr_accesses_bp', int],
+        ['age', int],
+        ])
+
+def regions_to_list(regions):
+    return [region_to_dict(r)
+            for r in list_for_each_entry(
+                'struct damon_region', regions.address_of_(), 'list')]
+
+def target_to_dict(target):
+    return to_dict(target, [
+        ['pid', int],
+        ['nr_regions', int],
+        ['regions_list', regions_to_list],
+        ])
+
+def targets_to_list(targets):
+    return [target_to_dict(t)
+            for t in list_for_each_entry(
+                'struct damon_target', targets.address_of_(), 'list')]
+
+def damos_access_pattern_to_dict(pattern):
+    return to_dict(pattern, [
+        ['min_sz_region', int],
+        ['max_sz_region', int],
+        ['min_nr_accesses', int],
+        ['max_nr_accesses', int],
+        ['min_age_region', int],
+        ['max_age_region', int],
+        ])
+
+def damos_quota_goal_to_dict(goal):
+    return to_dict(goal, [
+        ['metric', int],
+        ['target_value', int],
+        ['current_value', int],
+        ['last_psi_total', int],
+        ['nid', int],
+        ])
+
+def damos_quota_goals_to_list(goals):
+    return [damos_quota_goal_to_dict(g)
+            for g in list_for_each_entry(
+                'struct damos_quota_goal', goals.address_of_(), 'list')]
+
+def damos_quota_to_dict(quota):
+    return to_dict(quota, [
+        ['reset_interval', int],
+        ['ms', int], ['sz', int],
+        ['goals', damos_quota_goals_to_list],
+        ['esz', int],
+        ['weight_sz', int],
+        ['weight_nr_accesses', int],
+        ['weight_age', int],
+        ])
+
+def damos_watermarks_to_dict(watermarks):
+    return to_dict(watermarks, [
+        ['metric', int],
+        ['interval', int],
+        ['high', int], ['mid', int], ['low', int],
+        ])
+
+def scheme_to_dict(scheme):
+    return to_dict(scheme, [
+        ['pattern', damos_access_pattern_to_dict],
+        ['action', int],
+        ['apply_interval_us', int],
+        ['quota', damos_quota_to_dict],
+        ['wmarks', damos_watermarks_to_dict],
+        ['target_nid', int],
+        ])
+
+def schemes_to_list(schemes):
+    return [scheme_to_dict(s)
+            for s in list_for_each_entry(
+                'struct damos', schemes.address_of_(), 'list')]
+
+def damon_ctx_to_dict(ctx):
+    return to_dict(ctx, [
+        ['attrs', attrs_to_dict],
+        ['adaptive_targets', targets_to_list],
+        ['schemes', schemes_to_list],
+        ])
+
+def main():
+    if len(sys.argv) < 3:
+        print('Usage: %s <kdamond pid> <file>' % sys.argv[0])
+        exit(1)
+
+    pid = int(sys.argv[1])
+    file_to_store = sys.argv[2]
+
+    kthread_data = cast('struct kthread *',
+                        find_task(prog, pid).worker_private).data
+    ctx = cast('struct damon_ctx *', kthread_data)
+    status = {'contexts': [damon_ctx_to_dict(ctx)]}
+    if file_to_store == 'stdout':
+        print(json.dumps(status, indent=4))
+    else:
+        with open(file_to_store, 'w') as f:
+            json.dump(status, f, indent=4)
+
+if __name__ == '__main__':
+    main()
Signature · 1 line
2.39.5
author maintainer
#2 lore raw

PATCH 2/6 selftests/damon/_damon_sysfs: set Kdamond.pid in start()

In reply to SeongJae Park

_damon_sysfs.py is a Python module for reading and writing DAMON sysfs for testing. It is not reading resulting kdamond pids. Read and update those when starting kdamonds.

---
 tools/testing/selftests/damon/_damon_sysfs.py | 3 +++
 1 file changed, 3 insertions(+)
tools/testing/selftests/damon/_damon_sysfs.py +3 -0
index 5b1cb6b3ce4e..f587e117472e 100644
@@ -408,6 +408,9 @@ class Kdamond:
             if err is not None:
                 return err
         err = write_file(os.path.join(self.sysfs_dir(), 'state'), 'on')
+        if err is not None:
+            return err
+        self.pid, err = read_file(os.path.join(self.sysfs_dir(), 'pid'))
         return err
 
     def stop(self):
Signature · 1 line
2.39.5
author maintainer
#3 lore raw

PATCH 3/6 selftests/damon: add python and drgn-based DAMON sysfs test

In reply to SeongJae Park

Add a python-written DAMON sysfs functionality selftest. It sets DAMON parameters using Python module _damon_sysfs, reads updated kernel internal DAMON status and parameters using a 'drgn' script, namely drgn_dump_damon_status.py, and compare if the resulted DAMON internal status is as expected. The test is very minimum at the moment.

---
 tools/testing/selftests/damon/Makefile |  1 +
 tools/testing/selftests/damon/sysfs.py | 42 ++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)
 create mode 100755 tools/testing/selftests/damon/sysfs.py
tools/testing/selftests/damon/Makefile +1 -0
index e888455e3cf8..5b230deb19e8 100644
@@ -7,6 +7,7 @@ TEST_FILES = _damon_sysfs.py
 
 # functionality tests
 TEST_PROGS += sysfs.sh
+TEST_PROGS += sysfs.py
 TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
 TEST_PROGS += damos_quota.py damos_quota_goal.py damos_apply_interval.py
 TEST_PROGS += damos_tried_regions.py damon_nr_regions.py
tools/testing/selftests/damon/sysfs.py new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.39.5
author maintainer
#4 lore raw

PATCH 4/6 selftests/damon/sysfs.py: test monitoring attribute parameters

In reply to SeongJae Park

Add DAMON sysfs interface functionality tests for DAMON monitoring attribute parameters, including intervals, intervals tuning goals, and min/max number of regions.

---
 tools/testing/selftests/damon/sysfs.py | 34 ++++++++++++++++++++++++--
 1 file changed, 32 insertions(+), 2 deletions(-)
tools/testing/selftests/damon/sysfs.py +32 -2
index 4ff99db0d247..a721901a880d 100755
@@ -20,6 +20,11 @@ def dump_damon_status_dict(pid):
     except Exception as e:
         return None, 'json.load fail (%s)' % e
 
+def fail(expectation, status):
+    print('unexpected %s' % expectation)
+    print(json.dumps(status, indent=4))
+    exit(1)
+
 def main():
     kdamonds = _damon_sysfs.Kdamonds(
             [_damon_sysfs.Kdamond(contexts=[_damon_sysfs.DamonCtx()])])
@@ -34,8 +39,33 @@ def main():
         exit(1)
 
     if len(status['contexts']) != 1:
-        print('number of contexts: %d' % len(status['contexts']))
-        exit(1)
+        fail('number of contexts', status)
+
+    ctx = status['contexts'][0]
+    attrs = ctx['attrs']
+    if attrs['sample_interval'] != 5000:
+        fail('sample interval', status)
+    if attrs['aggr_interval'] != 100000:
+        fail('aggr interval', status)
+    if attrs['ops_update_interval'] != 1000000:
+        fail('ops updte interval', status)
+
+    if attrs['intervals_goal'] != {
+            'access_bp': 0, 'aggrs': 0,
+            'min_sample_us': 0, 'max_sample_us': 0}:
+        fail('intervals goal')
+
+    if attrs['min_nr_regions'] != 10:
+        fail('min_nr_regions')
+    if attrs['max_nr_regions'] != 1000:
+        fail('max_nr_regions')
+
+    if ctx['adaptive_targets'] != []:
+        fail('adaptive_targets')
+
+    if ctx['schemes'] != []:
+        fail('schemes')
+
     kdamonds.stop()
 
 if __name__ == '__main__':
Signature · 1 line
2.39.5
author maintainer
#5 lore raw

PATCH 5/6 selftests/damon/sysfs.py: test adaptive targets parameter

In reply to SeongJae Park

Add DAMON sysfs interface functionality tests for setup of basic adaptive targets parameters.

---
 tools/testing/selftests/damon/sysfs.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/testing/selftests/damon/sysfs.py +6 -3
index a721901a880d..3b085268f342 100755
@@ -27,7 +27,9 @@ def fail(expectation, status):
 
 def main():
     kdamonds = _damon_sysfs.Kdamonds(
-            [_damon_sysfs.Kdamond(contexts=[_damon_sysfs.DamonCtx()])])
+            [_damon_sysfs.Kdamond(
+                contexts=[_damon_sysfs.DamonCtx(
+                    targets=[_damon_sysfs.DamonTarget(pid=-1)])])])
     err = kdamonds.start()
     if err is not None:
         print('kdamond start failed: %s' % err)
@@ -60,8 +62,9 @@ def main():
     if attrs['max_nr_regions'] != 1000:
         fail('max_nr_regions')
 
-    if ctx['adaptive_targets'] != []:
-        fail('adaptive_targets')
+    if ctx['adaptive_targets'] != [
+            { 'pid': 0, 'nr_regions': 0, 'regions_list': []}]:
+        fail('adaptive targets', status)
 
     if ctx['schemes'] != []:
         fail('schemes')
Signature · 1 line
2.39.5
author maintainer
#6 lore raw

PATCH 6/6 selftests/damon/sysfs.py: test DAMOS schemes parameters setup

In reply to SeongJae Park

Add DAMON sysfs interface functionality tests for basic DAMOS schemes parameters setup.

---
 tools/testing/selftests/damon/sysfs.py | 46 ++++++++++++++++++++++++--
 1 file changed, 43 insertions(+), 3 deletions(-)
tools/testing/selftests/damon/sysfs.py +43 -3
index 3b085268f342..e67008fd055d 100755
@@ -29,7 +29,9 @@ def main():
     kdamonds = _damon_sysfs.Kdamonds(
             [_damon_sysfs.Kdamond(
                 contexts=[_damon_sysfs.DamonCtx(
-                    targets=[_damon_sysfs.DamonTarget(pid=-1)])])])
+                    targets=[_damon_sysfs.DamonTarget(pid=-1)],
+                    schemes=[_damon_sysfs.Damos()],
+                    )])])
     err = kdamonds.start()
     if err is not None:
         print('kdamond start failed: %s' % err)
@@ -66,8 +68,46 @@ def main():
             { 'pid': 0, 'nr_regions': 0, 'regions_list': []}]:
         fail('adaptive targets', status)
 
-    if ctx['schemes'] != []:
-        fail('schemes')
+    if len(ctx['schemes']) != 1:
+        fail('number of schemes', status)
+
+    scheme = ctx['schemes'][0]
+    if scheme['pattern'] != {
+            'min_sz_region': 0,
+            'max_sz_region': 2**64 - 1,
+            'min_nr_accesses': 0,
+            'max_nr_accesses': 2**32 - 1,
+            'min_age_region': 0,
+            'max_age_region': 2**32 - 1,
+            }:
+        fail('damos pattern', status)
+    if scheme['action'] != 9:   # stat
+        fail('damos action', status)
+    if scheme['apply_interval_us'] != 0:
+        fail('damos apply interval', status)
+    if scheme['target_nid'] != -1:
+        fail('damos target nid', status)
+
+    if scheme['quota'] != {
+            'reset_interval': 0,
+            'ms': 0,
+            'sz': 0,
+            'goals': [],
+            'esz': 0,
+            'weight_sz': 0,
+            'weight_nr_accesses': 0,
+            'weight_age': 0,
+            }:
+        fail('damos quota', status)
+
+    if scheme['wmarks'] != {
+            'metric': 0,
+            'interval': 0,
+            'high': 0,
+            'mid': 0,
+            'low': 0,
+            }:
+        fail('damos wmarks', status)
 
     kdamonds.stop()
 
Signature · 1 line
2.39.5