DAMON: Make coexistable with Idle Page Tracking
Subsystems (5)
cover DAMON: Make coexistable with Idle Page Tracking
From: SeongJae Park <[email protected]>
NOTE: This is an RFC for future change of DAMON patchsets[1,2,3], which is not merged in the mainline yet. The aim of this RFC is to show how the patchset would be changed in the next version. So, if you have some interest in this RFC, please consider reviewing the DAMON patchset, either.
Currently, DAMON is configured to be exclusive with Idle Page Tracking because both of the subsystems use PG_Idle flag and there is no way to synchronize with Idle Page Tracking. Though there are many use cases DAMON could do better than Idle Page Tracking, DAMON cannot fully replace Idle Page Tracking, since
- DAMON doesn't support all features of Idle Page Tracking from the beginning (e.g., physical address space is supported from the third DAMON patchset[3]), and
- there are some use cases Idle Page Tracking could be more efficient (e.g., page size granularity working set size calculation).
Therefore, this patchset makes DAMON coexistable with Idle Page Tracking. As the first decision of making DAMON exclusive was not a good idea, this change will be merged in the next versions of the original patchsets[1,2,3]. Therefore, you could skip detail of the changes but wait for postings of the next versions of the patchsets, except the 4th patch.
The changes significantly refactor the code, especially 'damon.c' and 'damon-test.c'. Though the refactoring changes are only straightforward, if you gave 'Reviewed-by' before and you want to drop it due to the changes, please let me know.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
Sequence of Patches
===================
The 1st patch separates DAMON components that unnecessarily implemented in one source file and depend on one config option (CONFIG_DAMON)
to multiple files and apply fine-grained dependency. As a result, the core framework part of DAMON becomes coexistable with Idle Page Tracking.
Following two patches further refactor the code for cleaner bound between the components.
The 4th patch implements a synchronization infrastructure for PG_idle flag users. We implement it to eventually used for DAMON, but the change is independent with DAMON and the also required for Idle Page Tracking itself. This could be picked before DAMON patchsets merged.
Finally, the 5th patch updates DAMON to use the PG_idle synchronization infrastructure and fully coexistable with Page Idle Tracking.
Baseline and Complete Git Trees
===============================
The patches are based on the v5.8 plus DAMON v20 patchset[1], RFC v14 of DAMOS patchset, RFC v8 of physical address space support patchset, RFC v1 of user space improvement[4], and some more trivial fixes (s/snprintf/scnprintf). You can also clone the complete git tree:
$ git clone git://github.com/sjp38/linux -b damon-usi/rfc/v1
The web is also available:
https://github.com/sjp38/linux/releases/tag/damon-usi/rfc/v1
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/
SeongJae Park (5):
mm/damon: Separate components and apply fine-grained dependencies mm/damon: Separate DAMON schemes application to primitives mm/damon: Move recording feature from core to dbgfs mm/page_idle: Avoid interferences from concurrent users mm/damon/primitives: Make coexistable with Idle Page Tracking
.../admin-guide/mm/idle_page_tracking.rst | 22 +- MAINTAINERS | 3 +- include/linux/damon.h | 109 +- include/linux/page_idle.h | 2 + mm/Kconfig | 25 +- mm/Makefile | 2 +- mm/damon-test.h | 724 ----- mm/damon.c | 2754 ----------------- mm/damon/Kconfig | 68 + mm/damon/Makefile | 5 + mm/damon/core-test.h | 253 ++ mm/damon/core.c | 860 +++++ mm/damon/damon.h | 7 + mm/damon/dbgfs-test.h | 264 ++ mm/damon/dbgfs.c | 1158 +++++++ mm/damon/primitives-test.h | 328 ++ mm/damon/primitives.c | 896 ++++++ mm/page_idle.c | 40 + 18 files changed, 3982 insertions(+), 3538 deletions(-) delete mode 100644 mm/damon-test.h delete mode 100644 mm/damon.c create mode 100644 mm/damon/Kconfig create mode 100644 mm/damon/Makefile create mode 100644 mm/damon/core-test.h create mode 100644 mm/damon/core.c create mode 100644 mm/damon/damon.h create mode 100644 mm/damon/dbgfs-test.h create mode 100644 mm/damon/dbgfs.c create mode 100644 mm/damon/primitives-test.h create mode 100644 mm/damon/primitives.c
Signature · 1 line
2.17.1
RFC PATCH 1/5 mm/damon: Separate components and apply fine-grained dependencies
From: SeongJae Park <[email protected]>
DAMON depends on one kernel configuration, 'CONFIG_DAMON' and implemented in one source file, 'damon.c'. There are three independent components in the file, though:
- the core logic for the overhead-accuracy tradeoff,
- the reference implementations of low level monitoring primitives for the virtual and physical address spaces, and
- debugfs interface for the user space.
Only the core logic is the essence of DAMON, which is an extensible framework. The other two components are default extensions built on top of the framework interface that implemented to let users use DAMON for usual use cases without their own extensions. Also, those are intended to be used as reference applications of the DAMON framework.
Putting these independent components under one configuration option is not only unnecessary but makes their boundary unclear and even make false dependencies. For example, because the 'primitives' depends on 'CONFIG_PAGE_IDLE_FLAG', even the 'core' also depedns on it and as a result cannot co-exist with the 'Idle Page Tracking'.
Though the default 'primitives' could cover lots of 'Idle Page Tracking' use cases more efficiently, some usecases that 'Idle Page Tracking' can do better (e.g., page size granularity working set size calculation) also exists. In some cases, someone could want to extend DAMON with thir own primitives implementation that can co-exist with 'Idle Page Tracking'. Therefore, making the DAMON 'core' exclusive with it makes no sense.
For the reason, this commit separates the parts to independent files and apply fine-grained config dependency. After this commit, the 'core' depedns on nothing and therefore could co-exist with the 'Idle Page Tracking'.
--- MAINTAINERS | 3 +- include/linux/damon.h | 87 +- mm/Kconfig | 25 +- mm/Makefile | 2 +- mm/damon-test.h | 724 ---------- mm/damon.c | 2754 ------------------------------------ mm/damon/Kconfig | 68 + mm/damon/Makefile | 5 + mm/damon/core-test.h | 288 ++++ mm/damon/core.c | 1065 ++++++++++++++ mm/damon/damon.h | 35 + mm/damon/dbgfs-test.h | 179 +++ mm/damon/dbgfs.c | 882 ++++++++++++ mm/damon/primitives-test.h | 328 +++++ mm/damon/primitives.c | 811 +++++++++++ 15 files changed, 3738 insertions(+), 3518 deletions(-) delete mode 100644 mm/damon-test.h delete mode 100644 mm/damon.c create mode 100644 mm/damon/Kconfig create mode 100644 mm/damon/Makefile create mode 100644 mm/damon/core-test.h create mode 100644 mm/damon/core.c create mode 100644 mm/damon/damon.h create mode 100644 mm/damon/dbgfs-test.h create mode 100644 mm/damon/dbgfs.c create mode 100644 mm/damon/primitives-test.h create mode 100644 mm/damon/primitives.c
MAINTAINERS +1 -2
@@ -4759,8 +4759,7 @@ F: Documentation/admin-guide/mm/damon/* F: Documentation/vm/damon/* F: include/linux/damon.h F: include/trace/events/damon.h -F: mm/damon-test.h -F: mm/damon.c +F: mm/damon/* F: tools/damon/* F: tools/testing/selftests/damon/*
include/linux/damon.h +74 -13
110 lines not on this page · raw message
mm/Kconfig +1 -24
@@ -880,29 +880,6 @@ config ARCH_HAS_HUGEPD config MAPPING_DIRTY_HELPERS bool -config DAMON - bool "Data Access Monitor" - depends on MMU && !IDLE_PAGE_TRACKING - select PAGE_EXTENSION if !64BIT - select PAGE_IDLE_FLAG - help - This feature allows to monitor access frequency of each memory - region. The information can be useful for performance-centric DRAM - level memory management. - - See https://damonitor.github.io/doc/html/latest-damon/index.html for - more information. - If unsure, say N. - -config DAMON_KUNIT_TEST - bool "Test for damon" - depends on DAMON=y && KUNIT - help - This builds the DAMON Kunit test suite. - - For more information on KUnit and unit tests in general, please refer - to the KUnit documentation. - - If unsure, say N. +source "mm/damon/Kconfig" endmenu
mm/Makefile +1 -1
@@ -121,4 +121,4 @@ obj-$(CONFIG_MEMFD_CREATE) += memfd.o obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += mapping_dirty_helpers.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o -obj-$(CONFIG_DAMON) += damon.o +obj-$(CONFIG_DAMON) += damon/
mm/damon-test.h deleted +0 -724
724 lines not on this page · raw message
mm/damon.c deleted +0 -2754
2,122 lines not on this page · raw message
mm/damon/Kconfig new file +68 -0
68 lines not on this page · raw message
mm/damon/core-test.h new file +288 -0
288 lines not on this page · raw message
mm/damon/core.c new file +1065 -0
1,065 lines not on this page · raw message
mm/damon/dbgfs-test.h new file +179 -0
179 lines not on this page · raw message
mm/damon/dbgfs.c new file +882 -0
882 lines not on this page · raw message
mm/damon/primitives-test.h new file +328 -0
328 lines not on this page · raw message
mm/damon/primitives.c new file +811 -0
811 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
RFC PATCH 2/5 mm/damon: Separate DAMON schemes application to primitives
In reply to SeongJae ParkFrom: SeongJae Park <[email protected]>
DAMON-based operation schemes feature is implemented inside DAMON 'core'. Though the access pattern based schemes target region tracking part makes sense to reside in the 'core', applying the scheme action would better to be reside in the 'primitives', as the work highly depends on the type of the target region.
For the reason, this commit moves the part to 'primitives' by adding one more context callback, 'apply_scheme' and implementing it in the reference primitives implementation for the virtual address spaces. Note that this doesn't add the implementation for the physical address space, as it didn't exist before. Nonetheless, the extension for physical space would be easily done in this way in future.
--- include/linux/damon.h | 8 +++++ mm/damon/core.c | 65 ++------------------------------------ mm/damon/damon.h | 28 ----------------- mm/damon/primitives.c | 73 ++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 82 insertions(+), 92 deletions(-)
include/linux/damon.h +8 -0
@@ -170,6 +170,7 @@ struct damos {* @check_accesses: Checks the access of target regions. * @target_valid: Determine if the target is valid. * @cleanup: Cleans up the context. +* @apply_scheme: Apply a DAMON-based operation scheme. * @sample_cb: Called for each sampling interval. * @aggregate_cb: Called for each aggregation interval. *
@@ -193,6 +194,9 @@ struct damos {* monitoring. * @cleanup is called from @kdamond just before its termination. After this * call, only @kdamond_lock and @kdamond will be touched. +* @apply_scheme is called from @kdamond when a region for user provided +* DAMON-based operation scheme is found. It should apply the scheme's action +* to the region. * * @sample_cb and @aggregate_cb are called from @kdamond for each of the * sampling intervals and aggregation intervals, respectively. Therefore,
@@ -229,6 +233,8 @@ struct damon_ctx {unsigned int (*check_accesses)(struct damon_ctx *context);bool (*target_valid)(struct damon_target *target);void (*cleanup)(struct damon_ctx *context); +int (*apply_scheme)(struct damon_ctx *context,struct damon_target *t, +struct damon_region *r,struct damos *scheme);void (*sample_cb)(struct damon_ctx *context);void (*aggregate_cb)(struct damon_ctx *context); };
@@ -312,6 +318,8 @@ void kdamond_prepare_vm_access_checks(struct damon_ctx *ctx);unsigned int kdamond_check_vm_accesses (struct damon_ctx *ctx);bool kdamond_vm_target_valid (struct damon_target *t);void kdamond_vm_cleanup (struct damon_ctx *ctx); +int kdamond_vm_apply_scheme (struct damon_ctx *context,struct damon_target *t, +struct damon_region *r,struct damos *scheme);void damon_set_vaddr_primitives (struct damon_ctx *ctx);/* Reference callback implementations for physical memory */
mm/damon/core.c +2 -63
77 lines not on this page · raw message
mm/damon/damon.h +0 -28
@@ -5,31 +5,3 @@/* Get a random number in [l, r) */ #define damon_rand (l, r) (l + prandom_u32() % (r - l)) - -/* -* 't->id' should be the pointer to the relevant 'struct pid' having reference -* count. Caller must put the returned task, unless it is NULL. -*/ -#define damon_get_task_struct (t) \ - (get_pid_task((struct pid *)t->id, PIDTYPE_PID)) - -/* -* Get the mm_struct of the given target -* -* Caller _must_ put the mm_struct after use, unless it is NULL. -* -* Returns the mm_struct of the target on success, NULL on failure -*/ -static inline struct mm_struct *damon_get_mm (struct damon_target *t) -{ -struct task_struct *task; -struct mm_struct *mm; - - task =damon_get_task_struct (t); -if (!task) -return NULL ; - - mm =get_task_mm (task); -put_task_struct (task); -return mm; -}
mm/damon/primitives.c +72 -1
104 lines not on this page · raw message
Signature · 1 line
2.17.1
RFC PATCH 3/5 mm/damon: Move recording feature from core to dbgfs
In reply to SeongJae ParkFrom: SeongJae Park <[email protected]>
DAMON passes the monitoring results to user space via two ways: 1) a tracepoint and 2) it's recording feature. The recording feature is for the users who want simplest use.
However, as the feature is for the user space only while the core is fundamentally a framework for the kernel space, keeping the feature in the core would make no sense. Therefore, this commit moves the feature to the debugfs interface of DAMON.
--- include/linux/damon.h | 23 +--- mm/damon/core-test.h | 57 ++------- mm/damon/core.c | 150 +----------------------- mm/damon/dbgfs-test.h | 87 +++++++++++++- mm/damon/dbgfs.c | 264 ++++++++++++++++++++++++++++++++++++++++-- 5 files changed, 359 insertions(+), 222 deletions(-)
include/linux/damon.h +4 -19
@@ -134,14 +134,6 @@ struct damos {* in case of virtual memory monitoring) and applies the changes for each * @regions_update_interval. All time intervals are in micro-seconds. * -* @rbuf: In-memory buffer for monitoring result recording. -* @rbuf_len: The length of @rbuf. -* @rbuf_offset: The offset for next write to @rbuf. -* @rfile_path: Record file path. -* -* If @rbuf, @rbuf_len, and @rfile_path are set, the monitored results are -* automatically stored in @rfile_path file. -* * @kdamond: Kernel thread who does the monitoring. * @kdamond_stop: Notifies whether kdamond should stop. * @kdamond_lock: Mutex for the synchronizations with @kdamond.
@@ -164,6 +156,8 @@ struct damos {* @targets_list: Head of monitoring targets (&damon_target) list. * @schemes_list: Head of schemes (&damos) list. * +* @private Private user data. +* * @init_target_regions: Constructs initial monitoring target regions. * @update_target_regions: Updates monitoring target regions. * @prepare_access_checks: Prepares next access check of target regions.
@@ -214,11 +208,6 @@ struct damon_ctx {struct timespec64 last_aggregation;struct timespec64 last_regions_update; -unsigned char *rbuf; -unsigned int rbuf_len; -unsigned int rbuf_offset; -char *rfile_path; -struct task_struct *kdamond;bool kdamond_stop;struct mutex kdamond_lock;
@@ -226,6 +215,8 @@ struct damon_ctx {struct list_head targets_list;/* 'damon_target' objects */ struct list_head schemes_list;/* 'damos' objects */ +void *private; +/* callbacks */ void (*init_target_regions)(struct damon_ctx *context);void (*update_target_regions)(struct damon_ctx *context);
@@ -241,10 +232,6 @@ struct damon_ctx {#ifdef CONFIG_DAMON -#define MIN_RECORD_BUFFER_LEN 1024 -#define MAX_RECORD_BUFFER_LEN (4 * 1024 * 1024) -#define MAX_RFILE_PATH_LEN 256 -#define damon_next_region (r) \ (container_of(r->list.next, struct damon_region, list))
@@ -298,8 +285,6 @@ int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int, unsigned long min_nr_reg, unsigned long max_nr_reg);int damon_set_schemes (struct damon_ctx *ctx,struct damos **schemes,ssize_t nr_schemes); -int damon_set_recording (struct damon_ctx *ctx, -unsigned int rbuf_len,char *rfile_path);int damon_nr_running_ctxs (void );int damon_start (struct damon_ctx *ctxs,int nr_ctxs);
mm/damon/core-test.h +11 -46
89 lines not on this page · raw message
mm/damon/core.c +3 -147
211 lines not on this page · raw message
mm/damon/dbgfs-test.h +86 -1
108 lines not on this page · raw message
mm/damon/dbgfs.c +255 -9
347 lines not on this page · raw message
Signature · 1 line
2.17.1
RFC PATCH 4/5 mm/page_idle: Avoid interferences from concurrent users
In reply to SeongJae ParkFrom: SeongJae Park <[email protected]>
Concurrent Idle Page Tracking users can interfere each other because the interface doesn't provide a central rule for synchronization between the users. Users could implement their own synchronization rule, but even in that case, applications developed by different users would not know how to synchronize with others. To help this situation, this commit introduces a centralized synchronization infrastructure of Idle Page Tracking.
In detail, this commit introduces a mutex lock for Idle Page Tracking, called 'page_idle_lock'. It is exposed to user space via a new bool sysfs file, '/sys/kernel/mm/page_idle/lock'. By writing to and reading from the file, users can hold/release and read status of the mutex. Writes to the Idle Page Tracking 'bitmap' file fails if the lock is not held, while reads of the file can be done regardless of the lock status.
Note that users could still interfere each other if they abuse this locking rule. Nevertheless, this change will let them notice the rule.
--- .../admin-guide/mm/idle_page_tracking.rst | 22 +++++++--- mm/page_idle.c | 40 +++++++++++++++++++ 2 files changed, 56 insertions(+), 6 deletions(-)
Documentation/admin-guide/mm/idle_page_tracking.rst +16 -6
@@ -21,13 +21,13 @@ User API ======== The idle page tracking API is located at ``/sys/kernel/mm/page_idle``. -Currently, it consists of the only read-write file, -``/sys/kernel/mm/page_idle/bitmap``. +Currently, it consists of two read-write file, +``/sys/kernel/mm/page_idle/bitmap`` and ``/sys/kernel/mm/page_idle/lock``. -The file implements a bitmap where each bit corresponds to a memory page. The -bitmap is represented by an array of 8-byte integers, and the page at PFN #i is -mapped to bit #i%64 of array element #i/64, byte order is native. When a bit is -set, the corresponding page is idle. +The ``bitmap`` file implements a bitmap where each bit corresponds to a memory +page. The bitmap is represented by an array of 8-byte integers, and the page at +PFN #i is mapped to bit #i%64 of array element #i/64, byte order is native. +When a bit is set, the corresponding page is idle. A page is considered idle if it has not been accessed since it was marked idle (for more details on what "accessed" actually means see the :ref:`Implementation
@@ -74,6 +74,16 @@ See :ref:`Documentation/admin-guide/mm/pagemap.rst <pagemap>` for more information about ``/proc/pid/pagemap``, ``/proc/kpageflags``, and ``/proc/kpagecgroup``. +The ``lock`` file is for avoidance of interference from concurrent users. If +the content of the ``lock`` file is ``1``, it means the ``bitmap`` file is +currently being used by someone. While the content of the ``lock`` file is +``1``, writing ``1`` to the file fails. Therefore, users should first +successfully write ``1`` to the ``lock`` file before starting use of ``bitmap`` +file and write ``0`` to the ``lock`` file after they finished use of the +``bitmap`` file. If a user writes the ``bitmap`` file while the ``lock`` is +``0``, the write fails. Meanwhile, reads of the ``bitmap`` file success +regardless of the ``lock`` status. + .. _impl_details: Implementation Details
mm/page_idle.c +40 -0
@@ -16,6 +16,8 @@#define BITMAP_CHUNK_SIZE sizeof(u64)#define BITMAP_CHUNK_BITS (BITMAP_CHUNK_SIZE * BITS_PER_BYTE) +static DEFINE_MUTEX (page_idle_lock); +/* * Idle page tracking only considers user memory pages, for other types of * pages the idle flag is always unset and an attempt to set it is silently
@@ -169,6 +171,9 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,unsigned long pfn, end_pfn;int bit; +if (!mutex_is_locked (&page_idle_lock)) +return -EPERM ; +if (pos %BITMAP_CHUNK_SIZE || count %BITMAP_CHUNK_SIZE )return -EINVAL ;
@@ -197,17 +202,52 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,return (char *)in - buf; } +static ssize_t page_idle_lock_show (struct kobject *kobj, +struct kobj_attribute *attr,char *buf) +{ +return sprintf (buf,"%d\n" ,mutex_is_locked (&page_idle_lock)); +} + +static ssize_t page_idle_lock_store (struct kobject *kobj, +struct kobj_attribute *attr,const char *buf,size_t count) +{ +bool do_lock; +int ret; + + ret =kstrtobool (buf, &do_lock); +if (ret <0 ) +return ret; + +if (do_lock) { +if (!mutex_trylock (&page_idle_lock)) +return -EBUSY ; + }else { +mutex_unlock (&page_idle_lock); + } + +return count; +} +static struct bin_attribute page_idle_bitmap_attr =__BIN_ATTR (bitmap,0600 , page_idle_bitmap_read, page_idle_bitmap_write,0 ); +static struct kobj_attribute page_idle_lock_attr = +__ATTR (lock,0600 , page_idle_lock_show, page_idle_lock_store); +static struct bin_attribute *page_idle_bin_attrs[] = { &page_idle_bitmap_attr,NULL , }; +static struct attribute *page_idle_lock_attrs[] = { + &page_idle_lock_attr.attr, +NULL , +}; +static const struct attribute_group page_idle_attr_group = { .bin_attrs = page_idle_bin_attrs, + .attrs = page_idle_lock_attrs, .name ="page_idle" , };
Signature · 1 line
2.17.1
RFC PATCH 5/5 mm/damon/primitives: Make coexistable with Idle Page Tracking
In reply to SeongJae ParkFrom: SeongJae Park <[email protected]>
DAMON's reference 'primitives' internally use 'PG_Idle' flag. Because the flag is also used by Idle Page Tracking but there was no way to synchronize with it, the 'primitives' were configured to be exclusive with Idle Page Tracking before. However, as we can now synchronize with Idle Page Tracking using 'idle_page_lock', this commit makes the primitives to do the synchronization and coexistable with Idle Page Tracking.
In more detail, the 'primitives' only require the users to do the synchronization by themselves. Real synchronization is done by the DAMON debugfs interface, who is the only one user of the 'primitives' as of now.
--- include/linux/damon.h | 1 + include/linux/page_idle.h | 2 ++ mm/damon/Kconfig | 2 +- mm/damon/dbgfs.c | 32 +++++++++++++++++++++++++++++++- mm/damon/primitives.c | 16 +++++++++++++++- mm/page_idle.c | 2 +- 6 files changed, 51 insertions(+), 4 deletions(-)
include/linux/damon.h +1 -0
@@ -312,6 +312,7 @@ void kdamond_init_phys_regions(struct damon_ctx *ctx);void kdamond_update_phys_regions (struct damon_ctx *ctx);void kdamond_prepare_phys_access_checks (struct damon_ctx *ctx);unsigned int kdamond_check_phys_accesses (struct damon_ctx *ctx); +bool kdamond_phys_target_valid (struct damon_target *t);void damon_set_paddr_primitives (struct damon_ctx *ctx);#endif /* CONFIG_DAMON_PRIMITIVES */
include/linux/page_idle.h +2 -0
@@ -8,6 +8,8 @@ #ifdef CONFIG_PAGE_IDLE_FLAG +extern struct mutex page_idle_lock; + #ifdef CONFIG_64BIT static inline bool page_is_young(struct page *page) {
mm/damon/Kconfig +1 -1
@@ -26,7 +26,7 @@ config DAMON_KUNIT_TEST config DAMON_PRIMITIVES bool "DAMON primitives for virtual/physical address spaces monitoring" - depends on DAMON && MMU && !IDLE_PAGE_TRACKING + depends on DAMON && MMU select PAGE_EXTENSION if !64BIT select PAGE_IDLE_FLAG help
mm/damon/dbgfs.c +31 -1
@@ -12,6 +12,7 @@#include <linux/file.h> #include <linux/mm.h> #include <linux/module.h> +#include <linux/page_idle.h> #include <linux/slab.h> #define MIN_RECORD_BUFFER_LEN 1024
@@ -28,6 +29,7 @@ struct debugfs_recorder {/* Monitoring contexts for debugfs interface users. */ static struct damon_ctx **debugfs_ctxs;static int debugfs_nr_ctxs =1 ; +static int debugfs_nr_terminated_ctxs;static DEFINE_MUTEX (damon_dbgfs_lock);
@@ -106,9 +108,20 @@ static void debugfs_init_vm_regions(struct damon_ctx *ctx)kdamond_init_vm_regions (ctx); } +static void debugfs_unlock_page_idle_lock (void ) +{ +mutex_lock (&damon_dbgfs_lock); +if (++debugfs_nr_terminated_ctxs == debugfs_nr_ctxs) { + debugfs_nr_terminated_ctxs =0 ; +mutex_unlock (&page_idle_lock); + } +mutex_unlock (&damon_dbgfs_lock); +} +static void debugfs_vm_cleanup (struct damon_ctx *ctx) {debugfs_flush_rbuffer (ctx->private); +debugfs_unlock_page_idle_lock ();kdamond_vm_cleanup (ctx); }
@@ -120,6 +133,8 @@ static void debugfs_init_phys_regions(struct damon_ctx *ctx)static void debugfs_phys_cleanup (struct damon_ctx *ctx) {debugfs_flush_rbuffer (ctx->private); +debugfs_unlock_page_idle_lock (); + }/*
@@ -197,6 +212,21 @@ static char *user_input_str(const char __user *buf, size_t count, loff_t *ppos)return kbuf; } +static int debugfs_start_ctx_ptrs (struct damon_ctx **ctxs,int nr_ctxs) +{ +int rc; + +if (!mutex_trylock (&page_idle_lock)) +return -EBUSY ; + + rc =damon_start_ctx_ptrs (ctxs, nr_ctxs); +if (rc) +mutex_unlock (&page_idle_lock); + +return rc; +} + +static ssize_t debugfs_monitor_on_write (struct file *file,const char __user *buf,size_t count,loff_t *ppos) {
@@ -212,7 +242,7 @@ static ssize_t debugfs_monitor_on_write(struct file *file,if (sscanf (kbuf,"%s" , kbuf) !=1 )return -EINVAL ;if (!strncmp (kbuf,"on" , count)) - err =damon_start_ctx_ptrs (debugfs_ctxs, debugfs_nr_ctxs); + err =debugfs_start_ctx_ptrs (debugfs_ctxs, debugfs_nr_ctxs);else if (!strncmp (kbuf,"off" , count)) err =damon_stop_ctx_ptrs (debugfs_ctxs, debugfs_nr_ctxs);else
mm/damon/primitives.c +15 -1
@@ -30,6 +30,10 @@#include "damon.h" +#ifndef CONFIG_IDLE_PAGE_TRACKING +DEFINE_MUTEX (page_idle_lock); +#endif +/* Minimal region size. Every damon_region is aligned by this. */ #ifndef CONFIG_DAMON_KUNIT_TEST #define MIN_REGION PAGE_SIZE
@@ -776,6 +780,9 @@ bool kdamond_vm_target_valid(struct damon_target *t) {struct task_struct *task; +if (!mutex_is_locked (&page_idle_lock)) +return false; + task =damon_get_task_struct (t);if (task) {put_task_struct (task);
@@ -795,6 +802,13 @@ void kdamond_vm_cleanup(struct damon_ctx *ctx) } } +bool kdamond_phys_target_valid (struct damon_target *t) +{ +if (!mutex_is_locked (&page_idle_lock)) +return false; +return true; +} +#ifndef CONFIG_ADVISE_SYSCALLS static int damos_madvise (struct damon_target *target,struct damon_region *r,int behavior)
@@ -874,7 +888,7 @@ void damon_set_paddr_primitives(struct damon_ctx *ctx) ctx->update_target_regions = kdamond_update_phys_regions; ctx->prepare_access_checks = kdamond_prepare_phys_access_checks; ctx->check_accesses = kdamond_check_phys_accesses; - ctx->target_valid =NULL ; + ctx->target_valid = kdamond_phys_target_valid; ctx->cleanup =NULL ; ctx->apply_scheme =NULL ; }
mm/page_idle.c +1 -1
@@ -16,7 +16,7 @@#define BITMAP_CHUNK_SIZE sizeof(u64)#define BITMAP_CHUNK_BITS (BITMAP_CHUNK_SIZE * BITS_PER_BYTE) -static DEFINE_MUTEX (page_idle_lock); +DEFINE_MUTEX (page_idle_lock);/* * Idle page tracking only considers user memory pages, for other types of
Signature · 1 line
2.17.1