selftests/damon: add more tests for core functionalities and corner cases
Subsystems (3)
SeongJae Park
maintainercover selftests/damon: add more tests for core functionalities and corner cases
Continue DAMON selftests' test coverage improvement works with a trivial improvement of the test code itself. The sequence of the patches in patchset is as follows.
The first five patches add two DAMON core functionalities tests. Those begins with three patches (patches 1-3) that update the test-purpose DAMON sysfs interface wrapper to support DAMOS quota, stats, and apply interval features, respectively. The fourth patch implements and adds a selftest for DAMOS quota feature, using the DAMON sysfs interface wrapper's newly added support of the quota and the stats feature. The fifth patch further implements and adds a selftest for DAMOS apply interval using the DAMON sysfs interface wrapper's newly added support of the apply interval and the stats feature.
Two patches (patches 6 and 7) for implementing and adding two corner cases handling selftests follow. Those try to avoid two previously fixed bugs from recurring.
Finally, a patch for making DAMON debugfs selftests dependency checker to use /proc/mounts instead of the hard-coded mount point assumption follows.
SeongJae Park (8):
selftests/damon/_damon_sysfs: support DAMOS quota selftests/damon/_damon_sysfs: support DAMOS stats selftests/damon/_damon_sysfs: support DAMOS apply interval selftests/damon: add a test for DAMOS quota selftests/damon: add a test for DAMOS apply intervals selftests/damon: add a test for a race between target_ids_read() and
dbgfs_before_terminate()
selftests/damon: add a test for the pid leak of
dbgfs_target_ids_write()
selftests/damon/_chk_dependency: get debugfs mount point from
/proc/mounts
tools/testing/selftests/damon/.gitignore | 2 + tools/testing/selftests/damon/Makefile | 5 ++ .../selftests/damon/_chk_dependency.sh | 9 ++- tools/testing/selftests/damon/_damon_sysfs.py | 77 ++++++++++++++++-- .../selftests/damon/damos_apply_interval.py | 67 ++++++++++++++++ tools/testing/selftests/damon/damos_quota.py | 67 ++++++++++++++++ .../damon/debugfs_target_ids_pid_leak.c | 68 ++++++++++++++++ .../damon/debugfs_target_ids_pid_leak.sh | 22 +++++ ...fs_target_ids_read_before_terminate_race.c | 80 +++++++++++++++++++ ...s_target_ids_read_before_terminate_race.sh | 14 ++++ 10 files changed, 403 insertions(+), 8 deletions(-) create mode 100755 tools/testing/selftests/damon/damos_apply_interval.py create mode 100755 tools/testing/selftests/damon/damos_quota.py create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh
base-commit: f51e629727d8cc526a3156a2c80489b8f050410f
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 1/8 selftests/damon/_damon_sysfs: support DAMOS quota
Update the test-purpose DAMON sysfs control Python module to support DAMOS quota.
--- tools/testing/selftests/damon/_damon_sysfs.py | 42 +++++++++++++++---- 1 file changed, 33 insertions(+), 9 deletions(-)
tools/testing/selftests/damon/_damon_sysfs.py +33 -9
@@ -70,18 +70,48 @@ class DamosAccessPattern: if err != None: return err +class DamosQuota: + sz = None # size quota, in bytes + ms = None # time quota + reset_interval_ms = None # quota reset interval + scheme = None # owner scheme + + def __init__(self, sz=0, ms=0, reset_interval_ms=0): + self.sz = sz + self.ms = ms + self.reset_interval_ms = reset_interval_ms + + def sysfs_dir(self): + return os.path.join(self.scheme.sysfs_dir(), 'quotas') + + def stage(self): + err = write_file(os.path.join(self.sysfs_dir(), 'bytes'), self.sz) + if err != None: + return err + err = write_file(os.path.join(self.sysfs_dir(), 'ms'), self.ms) + if err != None: + return err + err = write_file(os.path.join(self.sysfs_dir(), 'reset_interval_ms'), + self.reset_interval_ms) + if err != None: + return err + class Damos: action = None access_pattern = None - # todo: Support quotas, watermarks, stats, tried_regions + quota = None + # todo: Support watermarks, stats, tried_regions idx = None context = None tried_bytes = None - def __init__(self, action='stat', access_pattern=DamosAccessPattern()): + def __init__(self, action='stat', access_pattern=DamosAccessPattern(), + quota=DamosQuota()): self.action = action self.access_pattern = access_pattern self.access_pattern.scheme = self + self.quota = quota + self.quota.scheme = self def sysfs_dir(self): return os.path.join(
@@ -94,13 +124,7 @@ class Damos: err = self.access_pattern.stage() if err != None: return err - - # disable quotas - err = write_file(os.path.join(self.sysfs_dir(), 'quotas', 'ms'), '0') - if err != None: - return err - err = write_file( - os.path.join(self.sysfs_dir(), 'quotas', 'bytes'), '0') + err = self.quota.stage() if err != None: return err
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 2/8 selftests/damon/_damon_sysfs: support DAMOS stats
In reply to SeongJae ParkUpdate the test-purpose DAMON sysfs control Python module to support DAMOS stats.
--- tools/testing/selftests/damon/_damon_sysfs.py | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+)
tools/testing/selftests/damon/_damon_sysfs.py +32 -0
@@ -96,6 +96,20 @@ class DamosQuota: if err != None: return err +class DamosStats: + nr_tried = None + sz_tried = None + nr_applied = None + sz_applied = None + qt_exceeds = None + + def __init__(self, nr_tried, sz_tried, nr_applied, sz_applied, qt_exceeds): + self.nr_tried = nr_tried + self.sz_tried = sz_tried + self.nr_applied = nr_applied + self.sz_applied = sz_applied + self.qt_exceeds = qt_exceeds + class Damos: action = None access_pattern = None
@@ -104,6 +118,7 @@ class Damos: idx = None context = None tried_bytes = None + stats = None def __init__(self, action='stat', access_pattern=DamosAccessPattern(), quota=DamosQuota()):
@@ -322,6 +337,23 @@ class Kdamond: return err scheme.tried_bytes = int(content) + def update_schemes_stats(self): + err = write_file(os.path.join(self.sysfs_dir(), 'state'), + 'update_schemes_stats') + if err != None: + return err + for context in self.contexts: + for scheme in context.schemes: + stat_values = [] + for stat in ['nr_tried', 'sz_tried', 'nr_applied', + 'sz_applied', 'qt_exceeds']: + content, err = read_file( + os.path.join(scheme.sysfs_dir(), 'stats', stat)) + if err != None: + return err + stat_values.append(int(content)) + scheme.stats = DamosStats(*stat_values) + class Kdamonds: kdamonds = []
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 3/8 selftests/damon/_damon_sysfs: support DAMOS apply interval
In reply to SeongJae ParkUpdate the test-purpose DAMON sysfs control Python module to support DAMOS apply interval.
--- tools/testing/selftests/damon/_damon_sysfs.py | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
tools/testing/selftests/damon/_damon_sysfs.py +8 -1
@@ -114,6 +114,7 @@ class Damos: action = None access_pattern = None quota = None + apply_interval_us = None # todo: Support watermarks, stats, tried_regions idx = None context = None
@@ -121,12 +122,13 @@ class Damos: stats = None def __init__(self, action='stat', access_pattern=DamosAccessPattern(), - quota=DamosQuota()): + quota=DamosQuota(), apply_interval_us=0): self.action = action self.access_pattern = access_pattern self.access_pattern.scheme = self self.quota = quota self.quota.scheme = self + self.apply_interval_us = apply_interval_us def sysfs_dir(self): return os.path.join(
@@ -139,6 +141,11 @@ class Damos: err = self.access_pattern.stage() if err != None: return err + err = write_file(os.path.join(self.sysfs_dir(), 'apply_interval_us'), + '%d' % self.apply_interval_us) + if err != None: + return err + err = self.quota.stage() if err != None: return err
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 4/8 selftests/damon: add a test for DAMOS quota
In reply to SeongJae ParkAdd a selftest for verifying the DAMOS quota feature. The test is very similar to sysfs_update_schemes_tried_regions_wss_estimation.py. It starts an artificial workload of 20 MiB working set, run DAMON to find the working set size, but with 1 MiB/100 ms size quota. Then, it collect the DAMON-found working set size every 100 ms and check if the quota was always applied as expected. For the confirmation, the tests shows the stat-applied region size and the qt_exceeds stat.
--- tools/testing/selftests/damon/Makefile | 1 + tools/testing/selftests/damon/damos_quota.py | 67 ++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100755 tools/testing/selftests/damon/damos_quota.py
tools/testing/selftests/damon/Makefile +1 -0
@@ -12,6 +12,7 @@ TEST_PROGS += debugfs_rm_non_contexts.sh TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py +TEST_PROGS += damos_quota.py TEST_PROGS += reclaim.sh lru_sort.sh include ../lib.mk
tools/testing/selftests/damon/damos_quota.py new file +67 -0
67 lines not on this page · raw message
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 5/8 selftests/damon: add a test for DAMOS apply intervals
In reply to SeongJae ParkAdd a selftest for DAMOS apply intervals. It runs two schemes having different apply interval agains an artificial memory access workload, and check if the scheme with smaller apply interval was applied more frequently.
--- tools/testing/selftests/damon/Makefile | 2 +- .../selftests/damon/damos_apply_interval.py | 67 +++++++++++++++++++ 2 files changed, 68 insertions(+), 1 deletion(-) create mode 100755 tools/testing/selftests/damon/damos_apply_interval.py
tools/testing/selftests/damon/Makefile +1 -1
@@ -12,7 +12,7 @@ TEST_PROGS += debugfs_rm_non_contexts.sh TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py -TEST_PROGS += damos_quota.py +TEST_PROGS += damos_quota.py damos_apply_interval.py TEST_PROGS += reclaim.sh lru_sort.sh include ../lib.mk
tools/testing/selftests/damon/damos_apply_interval.py new file +67 -0
67 lines not on this page · raw message
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 6/8 selftests/damon: add a test for a race between target_ids_read() and dbgfs_before_terminate()
In reply to SeongJae Parkcommit 34796417964b ("mm/damon/dbgfs: protect targets destructions with kdamond_lock") fixed a race of DAMON debugfs interface. Specifically, the race was happening between target_ids_read() and dbgfs_before_terminate(). Add a test for the issue to prevent the problem from accidentally recurring.
--- tools/testing/selftests/damon/.gitignore | 1 + tools/testing/selftests/damon/Makefile | 2 + ...fs_target_ids_read_before_terminate_race.c | 80 +++++++++++++++++++ ...s_target_ids_read_before_terminate_race.sh | 14 ++++ 4 files changed, 97 insertions(+) create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh
tools/testing/selftests/damon/.gitignore +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only huge_count_read_write +debugfs_target_ids_read_before_terminate_race
tools/testing/selftests/damon/Makefile +2 -0
@@ -2,6 +2,7 @@ # Makefile for damon selftests TEST_GEN_FILES += huge_count_read_write +TEST_GEN_FILES += debugfs_target_ids_read_before_terminate_race TEST_GEN_FILES += access_memory TEST_FILES = _chk_dependency.sh _debugfs_common.sh
@@ -9,6 +10,7 @@ TEST_PROGS = debugfs_attrs.sh debugfs_schemes.sh debugfs_target_ids.sh TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh TEST_PROGS += debugfs_duplicate_context_creation.sh TEST_PROGS += debugfs_rm_non_contexts.sh +TEST_PROGS += debugfs_target_ids_read_before_terminate_race.sh TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.c new file +80 -0
80 lines not on this page · raw message
tools/testing/selftests/damon/debugfs_target_ids_read_before_terminate_race.sh new file +14 -0
@@ -0,0 +1,14 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +dmesg -C + +./debugfs_target_ids_read_before_terminate_race 5000 + +if dmesg | grep -q dbgfs_target_ids_read +then + dmesg + exit 1 +else + exit 0 +fi
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 7/8 selftests/damon: add a test for the pid leak of dbgfs_target_ids_write()
In reply to SeongJae ParkCommit ebb3f994dd92 ("mm/damon/dbgfs: fix 'struct pid' leaks in 'dbgfs_target_ids_write()'") fixes a pid leak bug in DAMON debugfs interface, namely dbgfs_target_ids_write() function. Add a selftest for the issue to prevent the problem from mistakenly recurring.
--- tools/testing/selftests/damon/.gitignore | 1 + tools/testing/selftests/damon/Makefile | 2 + .../damon/debugfs_target_ids_pid_leak.c | 68 +++++++++++++++++++ .../damon/debugfs_target_ids_pid_leak.sh | 22 ++++++ 4 files changed, 93 insertions(+) create mode 100644 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c create mode 100755 tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh
tools/testing/selftests/damon/.gitignore +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only huge_count_read_write debugfs_target_ids_read_before_terminate_race +debugfs_target_ids_pid_leak
tools/testing/selftests/damon/Makefile +2 -0
@@ -3,6 +3,7 @@ TEST_GEN_FILES += huge_count_read_write TEST_GEN_FILES += debugfs_target_ids_read_before_terminate_race +TEST_GEN_FILES += debugfs_target_ids_pid_leak TEST_GEN_FILES += access_memory TEST_FILES = _chk_dependency.sh _debugfs_common.sh
@@ -11,6 +12,7 @@ TEST_PROGS += debugfs_empty_targets.sh debugfs_huge_count_read_write.sh TEST_PROGS += debugfs_duplicate_context_creation.sh TEST_PROGS += debugfs_rm_non_contexts.sh TEST_PROGS += debugfs_target_ids_read_before_terminate_race.sh +TEST_PROGS += debugfs_target_ids_pid_leak.sh TEST_PROGS += sysfs.sh sysfs_update_removed_scheme_dir.sh TEST_PROGS += sysfs_update_schemes_tried_regions_hang.py TEST_PROGS += sysfs_update_schemes_tried_regions_wss_estimation.py
tools/testing/selftests/damon/debugfs_target_ids_pid_leak.c new file +68 -0
68 lines not on this page · raw message
tools/testing/selftests/damon/debugfs_target_ids_pid_leak.sh new file +22 -0
@@ -0,0 +1,22 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +before=$(grep "^pid " /proc/slabinfo | awk '{print $2}') + +nr_leaks=$(./debugfs_target_ids_pid_leak 1000) +expected_after_max=$((before + nr_leaks / 2)) + +after=$(grep "^pid " /proc/slabinfo | awk '{print $2}') + +echo > /sys/kernel/debug/damon/target_ids + +echo "tried $nr_leaks pid leak" +echo "number of active pid slabs: $before -> $after" +echo "(up to $expected_after_max expected)" +if [ $after -gt $expected_after_max ] +then + echo "maybe pids are leaking" + exit 1 +else + exit 0 +fi
Signature · 1 line
2.39.2
SeongJae Park
maintainerPATCH 8/8 selftests/damon/_chk_dependency: get debugfs mount point from /proc/mounts
In reply to SeongJae ParkDAMON debugfs selftests dependency checker assumes debugfs would be mounted at /sys/kernel/debug. That would be ok for many cases, but some systems might mounted the file system on some different places. Parse the real mount point using /proc/mounts file.
--- tools/testing/selftests/damon/_chk_dependency.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
tools/testing/selftests/damon/_chk_dependency.sh +8 -1
@@ -4,7 +4,14 @@ # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 -DBGFS=/sys/kernel/debug/damon +DBGFS=$(grep debugfs /proc/mounts --max-count 1 | awk '{print $2}') +if [ "$DBGFS" = "" ] +then + echo "debugfs not mounted" + exit $ksft_skip +fi + +DBGFS+="/damon" if [ $EUID -ne 0 ]; then
Signature · 1 line
2.39.2