mm/damon/core: support multi-source reports-based access monitoring
Subsystems (2)
SeongJae Park
maintainercover mm/damon/core: support multi-source reports-based access monitoring
TL; DR: Extend DAMON core-operations set layers interface for operations set driven report-based monitoring such as per-CPU and write-only access monitoring.
Background
----------
Existing DAMON operations set implementations, namely paddr, vaddr, and fvaddr, use Accessed bits of page tables as the main source of the access information. Accessed bits has some restrictions, though. For example, it cannot tell which CPU or CPU made the access, whether the access was read or write, and which part of the mapped entity was really accessed. Also, it cannot capture accesses done without page table walks, e.g., TLB hit case, or unmapped pages. Exisiting DAMON operations set can capture the accesses to unmapped pages utilizing PG_Idle, though.
Depending on the use case, the limitations can be problematic. Because the issue stems from the nature of page table Accessed bit, utilizing access information from different sources can mitigate the issue. Page faults, memory access instructions sampling interrupts, system calls, or any information from other kernel space friends such as subsystems or device drivers of CXL or GPUs could be examples of the different sources.
DAMON separates its core and operations set layer. Operations set layer handles the low level access information handling, and the core layer handles more high level works such as the region-based overhead/accuracy control and access-aware system operations. Hence we can extend DAMON to use the different sources by implementing and using another DAMON operations set. The core layer features will still be available with the new sources, without additional changes.
Nevertheless, the current interface between the core and the operations set layers is optimized for the Accessed bits case. Specifically, the interface asks the operations set if a give part of memory has accessed or not in a given time period (last sampling interval). It is easy for Accessed bit use case, since operations set can simply read the current value of the Accessed bit, which is already well manageed for each mapping entities by the memory management subsystems. For some sources other than Accessed bits, the operations set may need to keep the access information on its own, and answer to the core layer's question by reading its own access information collection. Similar implementations of such operations set internal access information may required for multiple operations set implementations.
Core Layer Changes for Reporting-based Monitoring
-------------------------------------------------
Optimize such possible duplicated efforts, by updating DAMON core layer to support real time access reporting. The updated interface allows operations set implementations to report (or, push) their information to the core layer, on their preferred schedule. DAMON core layer will handle the reports by managing meta data and updating the final monitoring results (DAMON regions) accordingly.
Also add another operations set callback to determine if a given access report is eligible to be used for given operations set. For example, if the operations set implementation is for monitoring only specific CPU or writes, the operations set could ask core layer to ignore reported accesses that made by other CPUs, or was for reads.
How Per-CPU or Write-only Monitoring Can Be Implemented
-------------------------------------------------------
With these core layer changes, we can implement new DAMON operations set that utilizes different information source for page table Accessed bits restricted use cases. For example, to utilize page faults information, we could implement an operations set that installs the fake page fault protections to access sampling pages per DAMON region for every sampling interval. For that, damon_operations->preapre_access_checks() callback coudl be a good place to implement. Then the developer would need to further update PROT_NONE page fault handlers to report the access events using the new report API, namely damon_report_access(). The report could contain information about which CPU, process, or thread made the access, and whether the access is for read or write. And finally implementing the report-ignore logic to ignore accesses made by CPUs of no interest or reads, we can do per-CPU or write-only access monitoring.
Patches Sequence
----------------
The first patch introduces damon_report_access() that any kernel code that can sleep can use, to report their access information on their schedule. The second patch adds DAMON core-operations set interface for ignoring specific types of data access reports for the given operations set configuration. The third patch further update core layer to really use the reported access information for making the monitoring results (DAMON regions) for API callers and ABI users. The fourth patch updates virtual address operations set to ignore access reports for none-target virtual spaces.
The fifth and sixth patches are for only showing possible future extensions of the report struct for per-CPU and write-only monitoring.
Plan for Dropping RFC
---------------------
This patch series is an RFC for early sharing of the idea that also shared on the last LSFMMBPF[1], as 'damon_report_access()' API plan. I actually wanted to post this later, but recently I received a few questiosn about this, so sharing this very early version. The implementation is pretty simple and unoptimized. No real use case of the changed interface (new operations set implementation) exists. We will further optimize the core layer implementation and add one or more real operations set implementations that utilizing the report-based interface, by the final version of this patch series.
[1] https://lwn.net/Articles/1016525/
SeongJae Park (6):
mm/damon/core: introduce damon_report_access() mm/damon/core: add eliglble_report() ops callback mm/damon/core: check received access reports mm/damon/vaddr: impleement eligible_report() callback mm/damon: add node_id to damon_access_report mm/damon: add write field to damon_access_report
include/linux/damon.h | 30 +++++++++++++++++ mm/damon/core.c | 76 +++++++++++++++++++++++++++++++++++++++++++ mm/damon/vaddr.c | 7 ++++ 3 files changed, 113 insertions(+)
base-commit: 78af14fb3cf19eea2f204650a0090aa2c7022257
Signature · 1 line
2.39.5
SeongJae Park
maintainerRFC PATCH 1/6 mm/damon/core: introduce damon_report_access()
DAMON core layer asks operations set layer about past access information, on core layer's schedule. In other words, core layer "pulls" the information from the operations set layer. This is problematic for a case that the operations set layer have no time and space to save the information until the core layer queries.
Add a new DAMON API function for reporting identified data accesses to DAMON, on the identifiers' schedule. In other words, it lets the operations set layer to "push" the information to the core layer. The function internally uses mutex, so reporting kernel code should be safe to sleep.
--- include/linux/damon.h | 21 +++++++++++++++++++++ mm/damon/core.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+)
include/linux/damon.h +21 -0
@@ -104,6 +104,25 @@ struct damon_target {struct list_head list; }; +/** +* struct damon_access_report - Represent single acces report information. +* @pid: The PID of the virtual address space of the address. +* NULL if it is of the physical address. +* @addr: The start address of the reporting region. +* @size: The size of the reporting region. +* @nr_accesses: Number of detected accesses to the region. +* +* @pid could be stale, and hence shouldn't be de-referenced. +*/ +struct damon_access_report { +struct pid *pid; +unsigned long addr; +unsigned long size; +int nr_accesses; +/* private: */ +unsigned long report_jiffies;/* when this report is made */ +}; +/** * enum damos_action - Represents an action of a Data Access Monitoring-based * Operation Scheme.
@@ -961,6 +980,8 @@ int damon_stop(struct damon_ctx **ctxs, int nr_ctxs);int damon_call (struct damon_ctx *ctx,struct damon_call_control *control);int damos_walk (struct damon_ctx *ctx,struct damos_walk_control *control); +void damon_report_access (struct damon_access_report *report); +int damon_set_region_biggest_system_ram_default (struct damon_target *t,unsigned long *start,unsigned long *end);
mm/damon/core.c +35 -0
@@ -24,6 +24,8 @@#define DAMON_MIN_REGION 1#endif +#define DAMON_ACCESS_REPORTS_CAP 1000 +static DEFINE_MUTEX (damon_lock);static int nr_running_ctxs;static bool running_exclusive_ctxs;
@@ -33,6 +35,11 @@ static struct damon_operations damon_registered_ops[NR_DAMON_OPS];static struct kmem_cache *damon_region_cache__ro_after_init ; +static DEFINE_MUTEX (damon_access_reports_lock); +static struct damon_access_report damon_access_reports[ +DAMON_ACCESS_REPORTS_CAP ]; +static int damon_access_reports_len; +/* Should be called under damon_ops_lock with id smaller than NR_DAMON_OPS */ static bool __damon_is_registered_ops (enum damon_ops_id id) {
@@ -1403,6 +1410,34 @@ int damos_walk(struct damon_ctx *ctx, struct damos_walk_control *control)return 0 ; } +/** +* damon_report_access() - Report identified access events to DAMON. +* @report: The reporting access information. +* +* Report access events to DAMON. +* +* Context: May sleep. +* +* NOTE: we may be able to implement this as a lockless queue, and allow any +* context. As the overhead is unknown, and region-based DAMON logics would +* guarantee the reports would be not made that frequently, let's start with +* this simple implementation, though. +*/ +void damon_report_access (struct damon_access_report *report) +{ +struct damon_access_report *dst; + +/* silently fail for races */ +if (!mutex_trylock (&damon_access_reports_lock)) +return ; + dst = &damon_access_reports[damon_access_reports_len++]; +if (damon_access_reports_len ==DAMON_ACCESS_REPORTS_CAP ) + damon_access_reports_len =0 ; + *dst = *report; + dst->report_jiffies = jiffies; +mutex_unlock (&damon_access_reports_lock); +} +/* * Warn and fix corrupted ->nr_accesses[_bp] for investigations and preventing * the problem being propagated.
Signature · 1 line
2.39.5
SeongJae Park
maintainerRFC PATCH 2/6 mm/damon/core: add eliglble_report() ops callback
In reply to SeongJae ParkNot every reported access information will be eligible for all DAMON ops and target. For example, virtual address access report will be not eligible for 'padr' ops, or monitoring targets for a process that different from the process for the repor. If it is for monitoring accesses from specific CPU or write, reports from other CPUs or reads should be ignored. Add operations set callback for this report eligibility checking.
--- include/linux/damon.h | 5 +++++ 1 file changed, 5 insertions(+)
include/linux/damon.h +5 -0
@@ -592,6 +592,7 @@ enum damon_ops_id {* @update: Update operations-related data structures. * @prepare_access_checks: Prepare next access check of target regions. * @check_accesses: Check the accesses to target regions. +* @eligible_report: Verify an access report for a target. * @get_scheme_score: Get the score of a region for a scheme. * @apply_scheme: Apply a DAMON-based operation scheme. * @target_valid: Determine if the target is valid.
@@ -618,6 +619,8 @@ enum damon_ops_id {* last preparation and update the number of observed accesses of each region. * It should also return max number of observed accesses that made as a result * of its update. The value will be used for regions adjustment threshold. +* @eligible_report should check if the given access report is eligible to be +* used by this operations set for the given target. * @get_scheme_score should return the priority score of a region for a scheme * as an integer in [0, &DAMOS_MAX_SCORE]. * @apply_scheme is called from @kdamond when a region for user provided
@@ -635,6 +638,8 @@ struct damon_operations {void (*update)(struct damon_ctx *context);void (*prepare_access_checks)(struct damon_ctx *context);unsigned int (*check_accesses)(struct damon_ctx *context); +bool (*eligible_report)(struct damon_access_report *report, +struct damon_target *t);int (*get_scheme_score)(struct damon_ctx *context,struct damon_target *t,struct damon_region *r,struct damos *scheme);
Signature · 1 line
2.39.5
SeongJae Park
maintainerRFC PATCH 3/6 mm/damon/core: check received access reports
In reply to SeongJae ParkReported access information is only saved in the core layer's internal data structure. Those are not really being used for final monitoring results. Update core layer access information (DAMON regions) using the reported access information.
--- mm/damon/core.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
mm/damon/core.c +41 -0
@@ -2472,6 +2472,46 @@ static void kdamond_init_ctx(struct damon_ctx *ctx) } } +static void kdamond_apply_access_report (struct damon_access_report *report, +struct damon_target *t,struct damon_ctx *ctx) +{ +struct damon_region *r; + +if (ctx->ops.eligible_report && !ctx->ops.eligible_report (report, t)) +return ; + +/* todo: make search faster, e.g., binary search? */ +damon_for_each_region (r, t) { +if (report->addr < r->ar.start) +continue ; +if (r->ar.end < report->addr + report->size) +continue ; + r->nr_accesses += report->nr_accesses; + } +} + +static void kdamond_check_reported_accesses (struct damon_ctx *ctx) +{ +int i; +struct damon_access_report *report; +struct damon_target *t; + +mutex_lock (&damon_access_reports_lock); +for (i =0 ; i < damon_access_reports_len; i++) { + report = &damon_access_reports[i]; +if (time_before (report->report_jiffies, + jiffies - +usecs_to_jiffies ( + ctx->attrs.sample_interval))) +continue ; +if (report->pid && !damon_target_has_pid (ctx)) +continue ; +damon_for_each_target (t, ctx) +kdamond_apply_access_report (report, t, ctx); + } +mutex_unlock (&damon_access_reports_lock); +} +/* * The monitoring daemon that runs as a kernel thread */
@@ -2518,6 +2558,7 @@ static int kdamond_fn(void *data)kdamond_usleep (sample_interval); ctx->passed_sample_intervals++; +kdamond_check_reported_accesses (ctx);if (ctx->ops.check_accesses) max_nr_accesses = ctx->ops.check_accesses (ctx);
Signature · 1 line
2.39.5
SeongJae Park
maintainerRFC PATCH 4/6 mm/damon/vaddr: impleement eligible_report() callback
In reply to SeongJae ParkFor [f]vaddr DAMON ops, access reports for virtual address space of a process that different from that for a given DAMON target shouldn't be used. Implement an eligible_report() callback for [f]vaddr that does the check.
--- mm/damon/vaddr.c | 7 +++++++ 1 file changed, 7 insertions(+)
mm/damon/vaddr.c +7 -0
@@ -610,6 +610,12 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)return max_nr_accesses; } +static bool damon_va_eligible_report (struct damon_access_report *report, +struct damon_target *t) +{ +return report->pid == t->pid; +} +/* * Functions for the target validity check and cleanup */
@@ -710,6 +716,7 @@ static int __init damon_va_initcall(void) .update = damon_va_update, .prepare_access_checks = damon_va_prepare_access_checks, .check_accesses = damon_va_check_accesses, + .eligible_report = damon_va_eligible_report, .target_valid = damon_va_target_valid, .cleanup = NULL, .apply_scheme = damon_va_apply_scheme,
Signature · 1 line
2.39.5
SeongJae Park
maintainerRFC PATCH 5/6 mm/damon: add node_id to damon_access_report
In reply to SeongJae ParkIn future, per-CPU or per-NUMA node monitoring might be required. We can add the information in damon_access_report and implement the damon_report_access() callers filling the information. And the new operations set implementation for the per-CPU or per-NUMA node access monitoring can show and ignore unnecessary reports via damon_operations->eligible_report() callback.
This commit is for showing the first part of the idea.
--- include/linux/damon.h | 2 ++ 1 file changed, 2 insertions(+)
include/linux/damon.h +2 -0
@@ -111,6 +111,7 @@ struct damon_target {* @addr: The start address of the reporting region. * @size: The size of the reporting region. * @nr_accesses: Number of detected accesses to the region. +* @node_id: NUMA node that made the access. * * @pid could be stale, and hence shouldn't be de-referenced. */
@@ -119,6 +120,7 @@ struct damon_access_report {unsigned long addr;unsigned long size;int nr_accesses; +int node_id;/* private: */ unsigned long report_jiffies;/* when this report is made */ };
Signature · 1 line
2.39.5
SeongJae Park
maintainerRFC PATCH 6/6 mm/damon: add write field to damon_access_report
In reply to SeongJae ParkThis commit is for showing the first part of the idea for implementing write-only access monitoring.
--- include/linux/damon.h | 2 ++ 1 file changed, 2 insertions(+)
include/linux/damon.h +2 -0
@@ -112,6 +112,7 @@ struct damon_target {* @size: The size of the reporting region. * @nr_accesses: Number of detected accesses to the region. * @node_id: NUMA node that made the access. +* @write: Whether the access is write. * * @pid could be stale, and hence shouldn't be de-referenced. */
@@ -121,6 +122,7 @@ struct damon_access_report {unsigned long size;int nr_accesses;int node_id; +bool write;/* private: */ unsigned long report_jiffies;/* when this report is made */ };
Signature · 1 line
2.39.5