mm/damon: hardware-sampled access reports + AMD IBS Op example
Subsystems (3)
cover mm/damon: hardware-sampled access reports + AMD IBS Op example
Hi all,
This is an RFC, not for merge. The series exercises and validates damon_report_access() -- the consumer API SeongJae introduced in [1] -- as a substrate for ingesting access reports from hardware-sampling sources. The series includes one worked-example backend, an AMD IBS Op module (damon_ibs.ko), that runs on Zen 3+ silicon via the existing perf event subsystem.
Combined with node_eligible_mem_bp [2], the recently-merged DAMOS goal metric, the same DAMON interface composes naturally for two operational regimes from one set of primitives:
1. Traditional tiering -- promote hot pages to DRAM up to a target cap.
2. System-wide bandwidth interleaving -- split hot pages between DRAM and CXL at an operator-chosen ratio, for workloads where placing some hot pages on CXL improves aggregate throughput.
Either regime composes with a separately-configured migrate_cold scheme to pair bandwidth shaping with capacity expansion: the hot-page schemes drive placement to meet the bandwidth target while migrate_cold reclaims DRAM by demoting cold pages.
The demonstration in this RFC exercises different
target ratios of the same PULL+PUSH setup.
Why a hardware-source primitive complements existing primitives
===============================================================
DAMON's existing access-check primitives observe access through software paths:
- PTE-Accessed bit scanning samples Accessed bits and clears them periodically. The hardware sets PTE-A on TLB miss, so already- resident TLB entries do not re-set the bit until they're evicted. For pages whose translations stay TLB-resident across DAMON's aggregation interval, nr_accesses reflects fewer accesses than the page actually serviced. This is correct behaviour for the primitive -- it observes what the TLB-miss path observes.
- Page-fault sampling (NUMA hint faults) requires unmapping pages to provoke the fault, then samples access on the fault path. For closed-loop schemes that drive migrate_hot from the same observations, the unmap and the migrate action interact.
Both primitives produce a view of hotness that converges to the true distribution over the aggregation interval. For systems where the address space is small relative to the aggregation rate, this is the right tool. On large heterogeneous-memory systems with goal- driven schemes asking the closed-loop tuner to converge on a target distribution, a complementary lower-latency view of accesses can tighten the loop -- reducing the time DAMON's nr_accesses takes to reflect the workload's actual access distribution, which in turn reduces ramp duration and oscillation amplitude during convergence of goal-driven schemes.
A hardware-sampling primitive provides this complementary view:
hardware retirement records each access at its natural event rate, with a physical address per sample, independent of TLB state and independent of the unmap/fault path.
This RFC adds the substrate (damon_report_access) so any hardware sampler -- IBS, PEBS, future CXL hotness monitoring units -- can feed access reports into the kdamond drain path and existing DAMOS schemes. The substrate is the contribution; the IBS backend is one worked example proving it on broadly-available silicon today.
Demonstration
=============
The two-scheme PULL+PUSH setup from the node_eligible_mem_bp introduction holds a target hot-memory ratio across DRAM and CXL. With damon_ibs.ko feeding damon_report_access, we observe two operational regimes:
Cold-start convergence -- workload starts at an even DRAM/CXL distribution (numactl --interleave=DRAM,CXL), DAMON context starts with the target ratio set at kdamond launch, schemes converge from the initial distribution to the target distribution.
+-----------+--------+----------+---------+ | Target | Mean | Offset | Stddev | +-----------+--------+----------+---------+ | 70% DRAM | 69.73% | -0.27pp | 0.70pp | | 30% DRAM | 31.00% | +1.00pp | 1.28pp | +-----------+--------+----------+---------+
Live target changes from a converged state -- kdamond context runs continuously, target ratio updated via DAMOS commit_schemes_quota_goals without kdamond teardown.
+-----------+--------+----------+---------+ | Target | Mean | Offset | Stddev | +-----------+--------+----------+---------+ | 90% DRAM | 89.74% | -0.26pp | 0.64pp | | 85% DRAM | 84.61% | -0.39pp | 0.60pp | +-----------+--------+----------+---------+
In both regimes, convergence to target is quick, and the workload's measured DRAM share then holds within 1.3 percentage points of target with standard deviation under 1.3 percentage points, sustained over runs of 15-30 minutes per target.
Hardware envelope: AMD EPYC dual-socket, CXL.mem on a separate NUMA node, 32GB hot working set, two migrate_hot schemes with complementary address filters, temporal quota tuner, 256-entry per-CPU report ring, 512 MiB per-scheme quota, 1s reset interval.
What's in this series
=====================
Patch 1. mm/damon/core: refcount ops owner module to prevent
rmmod UAF
Patch 2. mm/damon/paddr: export damon_pa_* ops for IBS module Patch 3. mm/damon/core: replace mutex-protected report buffer
with per-CPU lockless ring
Patch 4. mm/damon/core: flat-array snapshot + bsearch in ring-
drain loop
Patch 5. mm/damon: add sysfs binding and dispatch hookup for
paddr_ibs operations
Patch 6. mm/damon/core: accept paddr_ibs in node_eligible_mem_bp
ops check
Patch 7. mm/damon/damon_ibs: add AMD IBS-based access sampling
backend
Patches 1, 3, and 4 are general infrastructure that benefits any consumer of damon_report_access(). Patches 2, 5, 6, and 7 are the worked-example backend (paddr_ibs ops, sysfs binding, IBS module).
Patches worth folding into damon/next
=====================================
Patches 1, 3, and 4 are not specific to IBS or to this RFC's backend. Each is preparatory infrastructure that any consumer of damon_report_access() will need:
- Patch 1 (refcount ops owner) -- any modular ops set, including out-of-tree backends, needs clean module unload to avoid UAF on damon_unregister_ops.
- Patch 3 (per-CPU lockless ring) -- damon_report_access() cannot be called from NMI context with the current mutex-protected buffer. Hardware samplers all need NMI-safe submission.
- Patch 4 (flat-array snapshot + bsearch drain) -- the linear- scan drain is O(reports x regions) and exceeds the sample interval at high-CPU x large-region products. Bsearch brings it to O(reports x log regions).
If these belong directly on damon/next as preparatory patches for damon_report_access() rather than living inside an IBS-specific track, we are happy to rebase and resend them that way.
Relation to prior and ongoing work
==================================
The IBS sampling pattern in patch 7 -- attr.config=0 to use IBS Op default config, dc_phy_addr_valid filter, NMI-safe sample submission -- is derived from concepts in Bharata B Rao's pghot RFC v5 [3]. The attribution header is in mm/damon/damon_ibs.c and the patch carries a Suggested-by: trailer.
Bharata's pghot v7 [4] introduces a different IBS driver targeting the new IBS Memory Profiler (IBS-MProf) facility, which Bharata describes as a facility "that will be present in future AMD processors" -- a separate IBS instance from the one this RFC's backend uses. This version of driver based out of v5 [3] is an example of how DAMON can be benefited from AMD IBS Hardware source and validates importance of IBS information indepedently. It is not meant to be merged in the current form. @Bharata if you see a path where IBS samples can be consumed by DAMON at some point, will be happy to collaborate.
Akinobu Mita's perf-event-based access-check RFC [5] explores a configurable perf-event-driven access source for DAMON. IBS has vendor-specific MSR setup beyond what perf_event_attr alone expresses (e.g. dc_phy_addr_valid filtering on the produced sample, not on the perf attr), so the IBS path here appears complementary to [5] -- operators choose based on whether their hardware sampler fits stock perf or needs additional kernel-side setup.
Specific asks
=============
To SeongJae:
1. Patches 1, 3, and 4 are infrastructure that benefits any consumer of damon_report_access(), not just the IBS backend in this RFC. Would these belong directly on damon/next as preparatory patches for damon_report_access(), rather than living inside an IBS-specific track? Happy to rebase and resend them that way if you'd prefer that shape. Tested-by: tags can come along.
Future work
===========
- Longer-duration stability and broader workload coverage.
Test branch
===========
A single fetch reproduces the cover-letter measurements on top of both this RFC and the companion DAMOS quota controller and paddr migration walk fixes posted separately at [6]:
git fetch https://github.com/ravis-opensrc/linux.git \ damon/hw-hotness-rfc-v1-testing
The companion fixes are not required for this RFC to function, but the closed-loop measurements above were collected on the testing branch which has both applied. The standalone series-only branches are also available:
git fetch https://github.com/ravis-opensrc/linux.git \ damon/hw-hotness-rfc-v1 git fetch https://github.com/ravis-opensrc/linux.git \ damon/closed-loop-fixes-v1
Links
=====
[1] [RFC PATCH v3 00/37] mm/damon: introduce per-CPUs/threads/ write/read monitoring (SeongJae Park)
https://lore.kernel.org/linux-mm/[email protected]/
Patch 01 introduces damon_report_access(), the consumer API this RFC builds on.
[2] mm/damon: add node_eligible_mem_bp goal metric
https://lore.kernel.org/linux-mm/[email protected]/
[3] [RFC PATCH v5 00/10] mm: Hot page tracking and promotion infrastructure (Bharata B Rao)
https://lore.kernel.org/linux-mm/[email protected]/
[4] [PATCH v7 0/7] mm: Hot page tracking and promotion infrastructure (Bharata B Rao)
https://lore.kernel.org/linux-mm/[email protected]/
[5] [RFC PATCH v3 0/4] mm/damon: introduce perf event based access check (Akinobu Mita)
https://lore.kernel.org/linux-mm/[email protected]/
[6] [PATCH 0/5] mm/damon: DAMOS quota controller and paddr migration walk fixes (Ravi Jonnalagadda)
https://lore.kernel.org/linux-mm/[email protected]/
Ravi Jonnalagadda (7):
mm/damon/core: refcount ops owner module to prevent rmmod UAF mm/damon/paddr: export damon_pa_* ops for IBS module mm/damon/core: replace mutex-protected report buffer with per-CPU
lockless ring
mm/damon/core: flat-array snapshot + bsearch in ring-drain loop mm/damon: add sysfs binding and dispatch hookup for paddr_ibs
operations
mm/damon/core: accept paddr_ibs in node_eligible_mem_bp ops check mm/damon/damon_ibs: add AMD IBS-based access sampling backend
include/linux/damon.h | 13 ++ mm/damon/Kconfig | 10 + mm/damon/Makefile | 1 + mm/damon/core.c | 341 +++++++++++++++++++++++++++------ mm/damon/damon_ibs.c | 369 ++++++++++++++++++++++++++++++++++++ mm/damon/ops-common.h | 13 ++ mm/damon/paddr.c | 15 +- mm/damon/sysfs.c | 12 +- mm/damon/tests/core-kunit.h | 2 +- 9 files changed, 707 insertions(+), 69 deletions(-) create mode 100644 mm/damon/damon_ibs.c
base-commit: 606bfbf72120df4f406ef46971d48053706f6f75
Signature · 1 line
2.43.0
RFC PATCH 1/7 mm/damon/core: refcount ops owner module to prevent rmmod UAF
damon_select_ops() copies the registered damon_operations struct into ctx->ops by value. After damon_unregister_ops() is called from a backend module's exit path, the registry slot is cleared but any surviving ctx still holds function pointers that resolve into the unloaded module's text. Restarting kdamond on such a ctx, or invoking any ops callback, jumps into freed code.
Add a struct module *owner field to damon_operations. In damon_select_ops(), take a reference to ops->owner via try_module_get() after locating the registry entry; on failure return -EBUSY without binding the ctx. If the ctx already had an ops bound (re-select case), drop the previous owner's reference before installing the new one to keep the refcount balanced. In damon_destroy_ctx(), release the reference via module_put(ctx->ops.owner).
In damon_commit_ctx(), the live ops field is overwritten by a value copy from src. Balance the refcount when the owner changes: take a ref on the new owner (return -EBUSY on failure) and put the ref on the old owner before the assignment.
Built-in ops sets (vaddr, paddr) leave owner = NULL; try_module_get(NULL) returns true and module_put(NULL) is a no-op. Loadable backends set owner = THIS_MODULE in their registration.
Also add damon_unregister_ops() so loadable backends have a clean exit path.
--- include/linux/damon.h | 4 ++++ mm/damon/core.c | 46 ++++++++++++++++++++++++++++++++++--- mm/damon/tests/core-kunit.h | 2 +- 3 files changed, 48 insertions(+), 4 deletions(-)
include/linux/damon.h +4 -0
@@ -682,6 +682,8 @@ enum damon_ops_id {* struct damon_operations - Monitoring operations for given use cases. * * @id: Identifier of this operations set. +* @owner: Module that provides this operations set, or NULL +* for built-in ops. * @init: Initialize operations-related data structures. * @update: Update operations-related data structures. * @prepare_access_checks: Prepare next access check of target regions.
@@ -728,6 +730,7 @@ enum damon_ops_id {*/ struct damon_operations {enum damon_ops_id id; +struct module *owner;void (*init)(struct damon_ctx *context);void (*update)(struct damon_ctx *context);void (*prepare_access_checks)(struct damon_ctx *context);
@@ -1206,6 +1209,7 @@ int damon_commit_ctx(struct damon_ctx *old_ctx, struct damon_ctx *new_ctx);int damon_nr_running_ctxs (void );bool damon_is_registered_ops (enum damon_ops_id id);int damon_register_ops (struct damon_operations *ops); +int damon_unregister_ops (enum damon_ops_id id);int damon_select_ops (struct damon_ctx *ctx,enum damon_ops_id id);static inline bool damon_target_has_pid (const struct damon_ctx *ctx)
mm/damon/core.c +43 -3
77 lines not on this page · raw message
mm/damon/tests/core-kunit.h +1 -1
@@ -342,7 +342,7 @@ static void damon_test_split_regions_of(struct kunit *test)static void damon_test_ops_registration (struct kunit *test) {struct damon_ctx *c =damon_new_ctx (); -struct damon_operations ops = {.id =DAMON_OPS_VADDR }, bak; +struct damon_operations ops = {.id =DAMON_OPS_VADDR }, bak = {};bool need_cleanup = false;if (!c)
Signature · 1 line
2.43.0
RFC PATCH 2/7 mm/damon/paddr: export damon_pa_* ops for IBS module
In reply to Ravi JonnalagaddaRemove static qualifier from damon_pa_prepare_access_checks, damon_pa_check_accesses, damon_pa_apply_probes, damon_pa_apply_scheme, and damon_pa_scheme_score. Add EXPORT_SYMBOL_GPL for each.
These functions are used as ops callbacks by the IBS backend module (damon_ibs.ko) which registers paddr_ibs operations.
--- mm/damon/ops-common.h | 13 +++++++++++++ mm/damon/paddr.c | 15 ++++++++++----- 2 files changed, 23 insertions(+), 5 deletions(-)
mm/damon/ops-common.h +13 -0
@@ -23,3 +23,16 @@ bool damos_folio_filter_match(struct damos_filter *filter, struct folio *folio);unsigned long damon_migrate_pages (struct list_head *folio_list,int target_nid);bool damos_ops_has_filter (struct damos *s); + +/* +* paddr ops callbacks, declared here so paddr-family backends +* (e.g. paddr_ibs) can reuse the paddr operation implementations. +*/ +void damon_pa_prepare_access_checks (struct damon_ctx *ctx); +unsigned int damon_pa_check_accesses (struct damon_ctx *ctx); +void damon_pa_apply_probes (struct damon_ctx *ctx); +unsigned long damon_pa_apply_scheme (struct damon_ctx *ctx, +struct damon_target *t,struct damon_region *r, +struct damos *scheme,unsigned long *sz_filter_passed); +int damon_pa_scheme_score (struct damon_ctx *context, +struct damon_region *r,struct damos *scheme);
mm/damon/paddr.c +10 -5
@@ -124,13 +124,14 @@ static void damon_pa_prepare_access_checks_faults(struct damon_ctx *ctx) } } -static void damon_pa_prepare_access_checks (struct damon_ctx *ctx) +void damon_pa_prepare_access_checks (struct damon_ctx *ctx) {if (ctx->sample_control.primitives_enabled.page_table)damon_pa_prepare_access_checks_abit (ctx);if (ctx->sample_control.primitives_enabled.page_fault)damon_pa_prepare_access_checks_faults (ctx); } +EXPORT_SYMBOL_GPL (damon_pa_prepare_access_checks);static bool damon_pa_young (phys_addr_t paddr,unsigned long *folio_sz) {
@@ -168,7 +169,7 @@ static void __damon_pa_check_access(struct damon_region *r, last_addr = sampling_addr; } -static unsigned int damon_pa_check_accesses (struct damon_ctx *ctx) +unsigned int damon_pa_check_accesses (struct damon_ctx *ctx) {struct damon_target *t;struct damon_region *r;
@@ -184,6 +185,7 @@ static unsigned int damon_pa_check_accesses(struct damon_ctx *ctx)return max_nr_accesses; } +EXPORT_SYMBOL_GPL (damon_pa_check_accesses);static bool damon_pa_filter_match (struct damon_filter *filter,struct folio *folio)
@@ -234,7 +236,7 @@ static bool damon_pa_filter_pass(phys_addr_t pa, struct folio *folio,return pass; } -static void damon_pa_apply_probes (struct damon_ctx *ctx) +void damon_pa_apply_probes (struct damon_ctx *ctx) {struct damon_target *t;struct damon_region *r;
@@ -259,6 +261,7 @@ static void damon_pa_apply_probes(struct damon_ctx *ctx) } } } +EXPORT_SYMBOL_GPL (damon_pa_apply_probes);/* * damos_pa_filter_out - Return true if the page should be filtered out.
@@ -542,7 +545,7 @@ static unsigned long damon_pa_alloc_or_free(#endif -static unsigned long damon_pa_apply_scheme (struct damon_ctx *ctx, +unsigned long damon_pa_apply_scheme (struct damon_ctx *ctx,struct damon_target *t,struct damon_region *r,struct damos *scheme,unsigned long *sz_filter_passed) {
@@ -574,8 +577,9 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, }return 0 ; } +EXPORT_SYMBOL_GPL (damon_pa_apply_scheme); -static int damon_pa_scheme_score (struct damon_ctx *context, +int damon_pa_scheme_score (struct damon_ctx *context,struct damon_region *r,struct damos *scheme) {switch (scheme->action) {
@@ -595,6 +599,7 @@ static int damon_pa_scheme_score(struct damon_ctx *context,return DAMOS_MAX_SCORE ; } +EXPORT_SYMBOL_GPL (damon_pa_scheme_score);static int __init damon_pa_initcall (void ) {
Signature · 1 line
2.43.0
RFC PATCH 3/7 mm/damon/core: replace mutex-protected report buffer with per-CPU lockless ring
In reply to Ravi JonnalagaddaReplace the mutex-protected fixed-size array (DAMON_ACCESS_REPORTS_CAP=1000) with a per-CPU lockless ring buffer. This enables damon_report_access() to be called from NMI context.
Ring design:
- Producer is serialized per CPU: only one in-flight producer per CPU is allowed. A per-CPU damon_report_ring_busy counter detects NMI-on-process nesting and drops the nested attempt, preserving the single-writer invariant on the slot.
- head is advanced by the producer with smp_wmb() before publish.
- tail is advanced by the consumer (kdamond) after the entries[] reads.
- Overflow: sample silently dropped. NMI context is allocation-free and access reports are best-effort.
To keep the producer/consumer pattern scalable on systems with many CPUs and a high NMI rate, the ring layout follows three rules:
- head, tail and entries[] live on separate cache lines via ____cacheline_aligned_in_smp, so producer and consumer do not invalidate each other's working set on every advance.
- DAMON_REPORT_RING_SIZE is bounded so the per-CPU footprint stays small (256 entries x sizeof(struct damon_access_report) plus head and tail cache lines), keeping draining all rings during one kdamond tick from evicting unrelated data on contemporary server parts.
- A cpumask, damon_rings_pending, is set by the producer after publishing and cleared by the consumer per ring drained, so the consumer iterates only CPUs with pending entries instead of walking every online CPU. An smp_mb__before_atomic() between the head publish and the cpumask_set_cpu() ensures observers of the pending bit also observe the published head; without it, weakly- ordered architectures could let the consumer drain stale head and delay the report. The consumer pairs this with an smp_mb__after_atomic() between cpumask_clear_cpu() and reading head, so a producer that publishes between the consumer's clear and head-read is observed via the bit it re-sets rather than silently stranded.
Consumer (kdamond_check_reported_accesses) drains the rings of CPUs in damon_rings_pending, applying reports to targets.
--- mm/damon/core.c | 143 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 101 insertions(+), 42 deletions(-)
mm/damon/core.c +101 -42
@@ -25,7 +25,26 @@#define CREATE_TRACE_POINTS #include <trace/events/damon.h> -#define DAMON_ACCESS_REPORTS_CAP 1000 +/* Sized so the per-CPU ring set fits in L3 on typical multi-socket boxes. */ +#define DAMON_REPORT_RING_SIZE 256 +#define DAMON_REPORT_RING_MASK (DAMON_REPORT_RING_SIZE - 1) + +struct damon_report_ring { +unsigned int head;/* written by producer (NMI) */ +unsigned int tail/* written by consumer (kdamond) */ +____cacheline_aligned_in_smp ; +struct damon_access_report entries[DAMON_REPORT_RING_SIZE ] +____cacheline_aligned_in_smp ; +}; + +static DEFINE_PER_CPU(struct damon_report_ring, damon_report_rings); +static DEFINE_PER_CPU(int, damon_report_ring_busy); +/* +* Per-CPU bitmap: producer (NMI) sets after publishing a report; +* consumer (kdamond) clears before draining the corresponding ring. +* Hot-write under sampling load - do NOT mark __read_mostly. +*/ +static cpumask_t damon_rings_pending;static DEFINE_MUTEX (damon_lock);static int nr_running_ctxs;
@@ -36,10 +55,6 @@ static struct damon_operations damon_registered_ops[NR_DAMON_OPS];static struct kmem_cache *damon_region_cache__ro_after_init ; -static DEFINE_MUTEX (damon_access_reports_lock); -static struct damon_access_report damon_access_reports[ -DAMON_ACCESS_REPORTS_CAP ]; -static int damon_access_reports_len;/* Should be called under damon_ops_lock with id smaller than NR_DAMON_OPS */ static bool __damon_is_registered_ops (enum damon_ops_id id)
@@ -2127,33 +2142,56 @@ int damos_walk(struct damon_ctx *ctx, struct damos_walk_control *control) }/** -* damon_report_access() - Report identified access events to DAMON. -* @report: The reporting access information. +* damon_report_access() - Report a hardware-observed memory access. +* @report: pointer to a filled damon_access_report struct. * -* Report access events to DAMON. -* -* Context: May sleep. -* -* NOTE: we may be able to implement this as a lockless queue, and allow any -* context. As the overhead is unknown, and region-based DAMON logics would -* guarantee the reports would be not made that frequently, let's start with -* this simple implementation. +* Context: NMI-safe. No sleeping, no allocation, no locks. */ void damon_report_access (struct damon_access_report *report) { -struct damon_access_report *dst; +struct damon_report_ring *ring; +unsigned int head, next; -/* silently fail for races */ -if (!mutex_trylock (&damon_access_reports_lock)) -return ; - dst = &damon_access_reports[damon_access_reports_len++]; -/* just drop all existing reports in favor of simplicity. */ -if (damon_access_reports_len ==DAMON_ACCESS_REPORTS_CAP ) - damon_access_reports_len =0 ; - *dst = *report; - dst->report_jiffies = jiffies; -mutex_unlock (&damon_access_reports_lock); +/* Pin to a CPU so the SPSC invariant holds for preemptible callers. */ +preempt_disable (); +/* +* NMI nesting on the same CPU as a process-context producer would +* stomp the same entries[head] slot. Detect and drop instead. +*/ +if (this_cpu_inc_return (damon_report_ring_busy) !=1 ) { +/* NMI nested on a process-context producer; drop. */ + goto out; + } + + ring =this_cpu_ptr (&damon_report_rings); + head = ring->head; + next = (head +1 ) &DAMON_REPORT_RING_MASK ; + +if (next ==READ_ONCE (ring->tail)) { +/* Ring full; consumer is behind, drop the report. */ + goto out; + } + + ring->entries[head] = *report; + ring->entries[head].report_jiffies = jiffies; +smp_wmb ();/* ensure entry visible before head advance */ +WRITE_ONCE (ring->head, next); +/* +* Order the head advance before publishing the pending bit +* so that the consumer, on observing the bit, is also +* guaranteed to observe the new head. set_bit/cpumask_set_cpu +* are documented as unordered RMW (atomic_bitops.txt), hence +* the explicit barrier; without it, a weakly-ordered arch +* could let the consumer drain stale head, clear the bit, and +* delay the report until the next producer sets the bit again. +*/ +smp_mb__before_atomic (); +cpumask_set_cpu (smp_processor_id (), &damon_rings_pending); +out: +this_cpu_dec (damon_report_ring_busy); +preempt_enable (); } +EXPORT_SYMBOL_GPL (damon_report_access);#ifdef CONFIG_MMU void damon_report_page_fault (struct vm_fault *vmf,bool huge_pmd)
@@ -3814,26 +3852,47 @@ static unsigned int kdamond_apply_zero_access_report(struct damon_ctx *ctx)static unsigned int kdamond_check_reported_accesses (struct damon_ctx *ctx) { -int i; -struct damon_access_report *report; +int cpu;struct damon_target *t; -/* currently damon_access_report supports only physical address */ -if (damon_target_has_pid (ctx)) -return 0 ; +for_each_cpu (cpu, &damon_rings_pending) { +struct damon_report_ring *ring = +per_cpu_ptr (&damon_report_rings, cpu); +unsigned int head, tail; -mutex_lock (&damon_access_reports_lock); -for (i =0 ; i < damon_access_reports_len; i++) { - report = &damon_access_reports[i]; -if (time_before (report->report_jiffies, - jiffies - -usecs_to_jiffies ( - ctx->attrs.sample_interval))) -continue ; -damon_for_each_target (t, ctx) -kdamond_apply_access_report (report, t, ctx); +cpumask_clear_cpu (cpu, &damon_rings_pending); +/* +* Pair with the producer's smp_mb__before_atomic() between +* the head publish and cpumask_set_cpu(): order the bit +* clear before the head read so that a producer publishing +* between our clear and our READ_ONCE(head) is observed via +* the bit it re-sets, not lost as a stale-head drain. +*/ +smp_mb__after_atomic (); + head =READ_ONCE (ring->head); +/* +* Pair with smp_wmb in damon_report_access(): the entry +* data published before the producer advanced head must be +* visible to the entries[] reads inside the loop below. +*/ +smp_rmb (); + tail = ring->tail; + +while (tail != head) { +struct damon_access_report *report = + &ring->entries[tail]; + +if (!time_before (report->report_jiffies, + jiffies -usecs_to_jiffies ( + ctx->attrs.sample_interval))) { +damon_for_each_target (t, ctx) +kdamond_apply_access_report ( + report, t, ctx); + } + tail = (tail +1 ) &DAMON_REPORT_RING_MASK ; + } +WRITE_ONCE (ring->tail, tail); } -mutex_unlock (&damon_access_reports_lock);/* For nr_accesses_bp, absence of access should also be reported. */ return kdamond_apply_zero_access_report (ctx); }
Signature · 1 line
2.43.0
RFC PATCH 4/7 mm/damon/core: flat-array snapshot + bsearch in ring-drain loop
In reply to Ravi JonnalagaddaThe drain loop is O(reports x regions) when matching each ring entry back to a region. At sufficiently large CPU x region products the linear scan exceeds the sample interval, producing unbounded backlog.
At drain start, snapshot each target's regions into a flat array (struct damon_target_lookup), already sorted by ascending r->ar.start. Replace the linear lookup with binary search over the snapshot, reducing the drain to O(reports x log2(regions)).
Reject reports that straddle a region boundary
(addr + report->size > r->ar.end) so partial-region accesses are not credited to the lower region.
If the snapshot allocation fails under memory pressure, log ratelimited and fall through to zero-access reporting; the next tick retries.
Hoist the per-report damon_sample_filter_out() check out of the per-target loop so it runs once per ring entry rather than N times.
--- include/linux/damon.h | 7 +++ mm/damon/core.c | 137 ++++++++++++++++++++++++++++++++++++------ 2 files changed, 126 insertions(+), 18 deletions(-)
include/linux/damon.h +7 -0
@@ -1045,6 +1045,13 @@ struct damon_ctx {/* Per-ctx PRNG state for damon_rand(); kdamond is the sole consumer. */ struct rnd_state rnd_state; +/* Reusable drain-loop snapshot buffer (avoids per-tick kmalloc) */ +struct { +struct damon_target_lookup *lookups; +unsigned int nr_lookups; +struct damon_region **region_buf; +unsigned int region_buf_cap; + } drain_snapshot; };/* Get a random number in [@l, @r) using @ctx's lockless PRNG. */
mm/damon/core.c +119 -18
188 lines not on this page · raw message
Signature · 1 line
2.43.0
RFC PATCH 5/7 mm/damon: add sysfs binding and dispatch hookup for paddr_ibs operations
In reply to Ravi JonnalagaddaExtend damon_ops_id enum to include DAMON_OPS_PADDR_IBS and add the corresponding 'paddr_ibs' name to the sysfs ops_names array so users can select AMD IBS-based PA-mode monitoring via /sys/kernel/mm/damon/admin/kdamonds/<N>/contexts/<N>/operations.
Route ops that report accesses through the hardware-sampling ring (currently only DAMON_OPS_PADDR_IBS) through the existing kdamond_check_reported_accesses() drain path used for page-fault reports. A small helper damon_ops_is_hw_hotness() centralises the classification so any future paddr-family backend that also reports through the ring just adds a case here.
This routing is bound to ops.id rather than to a separate per-context flag. A flag in damon_sample_control would have to be set by the ops .init callback after damon_select_ops() and would then need to be preserved by damon_commit_sample_control() across sysfs commits; deriving from ops.id avoids both pitfalls.
--- include/linux/damon.h | 2 ++ mm/damon/core.c | 13 ++++++++++++- mm/damon/sysfs.c | 12 +++++++++--- 3 files changed, 23 insertions(+), 4 deletions(-)
include/linux/damon.h +2 -0
@@ -669,12 +669,14 @@ struct damos {* @DAMON_OPS_FVADDR: Monitoring operations for only fixed ranges of virtual * address spaces * @DAMON_OPS_PADDR: Monitoring operations for the physical address space +* @DAMON_OPS_PADDR_IBS: AMD IBS-based PA-mode monitoring * @NR_DAMON_OPS: Number of monitoring operations implementations */ enum damon_ops_id {DAMON_OPS_VADDR ,DAMON_OPS_FVADDR ,DAMON_OPS_PADDR , +DAMON_OPS_PADDR_IBS ,NR_DAMON_OPS , };
mm/damon/core.c +12 -1
@@ -73,6 +73,16 @@ static bool __damon_is_registered_ops(enum damon_ops_id id)return true; } +/* +* Returns true if the given ops id reports access samples through the +* hardware-sampling ring-buffer drain path (rather than its own +* .check_accesses callback). +*/ +static bool damon_ops_is_hw_hotness (enum damon_ops_id id) +{ +return id ==DAMON_OPS_PADDR_IBS ; +} +/** * damon_is_registered_ops() - Check if a given damon_operations is registered. * @id: Id of the damon_operations to check if registered.
@@ -4048,7 +4058,8 @@ static int kdamond_fn(void *data) ctx->passed_sample_intervals++;/* todo: make these non-exclusive */ -if (ctx->sample_control.primitives_enabled.page_fault) +if (ctx->sample_control.primitives_enabled.page_fault || +damon_ops_is_hw_hotness (ctx->ops.id)) max_nr_accesses =kdamond_check_reported_accesses (ctx);else if (ctx->ops.check_accesses) max_nr_accesses = ctx->ops.check_accesses (ctx);
mm/damon/sysfs.c +9 -3
@@ -1388,6 +1388,10 @@ static const struct damon_sysfs_ops_name damon_sysfs_ops_names[] = { .ops_id = DAMON_OPS_PADDR, .name = "paddr", }, + { + .ops_id = DAMON_OPS_PADDR_IBS, + .name = "paddr_ibs", + }, };struct damon_sysfs_context {
@@ -2023,7 +2027,8 @@ static int damon_sysfs_add_targets(struct damon_ctx *ctx,int i, err;/* Multiple physical address space monitoring targets makes no sense */ -if (ctx->ops.id ==DAMON_OPS_PADDR && sysfs_targets->nr >1 ) +if ((ctx->ops.id ==DAMON_OPS_PADDR || + ctx->ops.id ==DAMON_OPS_PADDR_IBS ) && sysfs_targets->nr >1 )return -EINVAL ;for (i =0 ; i < sysfs_targets->nr; i++) {
@@ -2072,8 +2077,9 @@ static int damon_sysfs_apply_inputs(struct damon_ctx *ctx,if (err)return err; ctx->addr_unit = sys_ctx->addr_unit; -/* addr_unit is respected by only DAMON_OPS_PADDR */ -if (sys_ctx->ops_id ==DAMON_OPS_PADDR ) +/* addr_unit is respected by only paddr-family ops */ +if (sys_ctx->ops_id ==DAMON_OPS_PADDR || + sys_ctx->ops_id ==DAMON_OPS_PADDR_IBS ) ctx->min_region_sz =max (DAMON_MIN_REGION_SZ / sys_ctx->addr_unit,1 ); ctx->pause = sys_ctx->pause;
Signature · 1 line
2.43.0
RFC PATCH 6/7 mm/damon/core: accept paddr_ibs in node_eligible_mem_bp ops check
In reply to Ravi Jonnalagaddadamos_get_node_eligible_mem_bp() and the damon_commit_ctx() validation path reject any ops.id != DAMON_OPS_PADDR, which caused paddr_ibs to always get 0 from the node-eligible helper. This caused the quota control loop to run open-loop (esz doubles every tick) when using the paddr_ibs backend with a node_eligible_mem_bp goal.
Introduce damon_ops_id_is_paddr_family() and use it at both sites so DAMON_OPS_PADDR_IBS is accepted alongside DAMON_OPS_PADDR. The helper also gives any future paddr-family backend a single line to extend.
--- mm/damon/core.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
mm/damon/core.c +13 -3
@@ -83,6 +83,16 @@ static bool damon_ops_is_hw_hotness(enum damon_ops_id id)return id ==DAMON_OPS_PADDR_IBS ; } +/* +* Returns true if the ops id treats the monitoring target as a +* physical-address region (no per-task PID). Used by paddr-only +* gates such as node_eligible_mem_bp. +*/ +static bool damon_ops_id_is_paddr_family (enum damon_ops_id id) +{ +return id ==DAMON_OPS_PADDR || id ==DAMON_OPS_PADDR_IBS ; +} +/** * damon_is_registered_ops() - Check if a given damon_operations is registered. * @id: Id of the damon_operations to check if registered.
@@ -1787,8 +1797,8 @@ int damon_commit_ctx(struct damon_ctx *dst, struct damon_ctx *src)if (!is_power_of_2 (src->min_region_sz))return -EINVAL ; -/* node_eligible_mem_bp metric requires PADDR ops */ -if (src->ops.id !=DAMON_OPS_PADDR ) { +/* node_eligible_mem_bp metric requires PADDR-family ops */ +if (!damon_ops_id_is_paddr_family (src->ops.id)) {damon_for_each_scheme (scheme, src) {struct damos_quota *quota = &scheme->quota;
@@ -3041,7 +3051,7 @@ static unsigned long damos_get_node_eligible_mem_bp(struct damon_ctx *c,phys_addr_t total_eligible =0 ;phys_addr_t node_eligible; -if (c->ops.id !=DAMON_OPS_PADDR ) +if (!damon_ops_id_is_paddr_family (c->ops.id))return 0 ;if (nid <0 || nid >=MAX_NUMNODES || !node_online (nid))
Signature · 1 line
2.43.0
RFC PATCH 7/7 mm/damon/damon_ibs: add AMD IBS-based access sampling backend
In reply to Ravi JonnalagaddaAdd paddr_ibs operations using AMD IBS Op sampling via perf_event_create_kernel_counter(). IBS delivers physical-address- keyed access reports to DAMON's shared-layer ring buffer (damon_report_access()), without dependency on PTE Accessed-bit scanning or page faults.
Per-CPU IBS events are created and torn down via cpuhp notifiers (CPUHP_AP_ONLINE_DYN). Routing of access reports through the ring- drain path is bound to ops.id == DAMON_OPS_PADDR_IBS at the dispatch site (see "mm/damon: add sysfs binding and dispatch hookup for paddr_ibs operations"), so .init does not need to flip a per-context flag.
Sample-time discipline:
- PERF_SAMPLE_PHYS_ADDR is requested in attr.sample_type, but the IBS perf driver only fills data->phys_addr when IBS_OP_DATA3.dc_phy_addr_valid is set. Skip stale-PA samples by inspecting data->sample_flags rather than testing phys_addr for zero (which would also drop legitimate page 0).
- PERF_SAMPLE_DATA_SRC is requested so the perf driver decodes IBS_OP_DATA3.{ld_op,st_op} into data->data_src.mem_op; the backend reports load vs store accordingly via damon_access_report.is_write.
Module parameters:
- max_cnt: IBS Op MaxCnt (writable; writes call damon_ibs_set_sample_rate() to restart sampling at the new rate)
- samples_total / samples_filtered: per-CPU-aggregated counters
(read-only)
Source file is mm/damon/damon_ibs.c (renamed from mm/damon/ibs.c) so the resulting module is loaded as damon_ibs.ko, avoiding the generic "ibs" namespace.
The IBS sampling approach is derived from Bharata B Rao's pghot RFC v5 series; the attribution header is in damon_ibs.c.
--- mm/damon/Kconfig | 10 ++ mm/damon/Makefile | 1 + mm/damon/damon_ibs.c | 369 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 380 insertions(+) create mode 100644 mm/damon/damon_ibs.c
mm/damon/Kconfig +10 -0
@@ -131,4 +131,14 @@ config DAMON_ACMA min/max memory for the system and maximum memory pressure stall time ratio. +config DAMON_IBS + tristate "AMD IBS-based access sampling for DAMON" + depends on DAMON_PADDR && CPU_SUP_AMD && X86_64 && PERF_EVENTS + help + Uses AMD IBS (Instruction-Based Sampling) hardware to deliver + physical-address-keyed access reports to DAMON's shared-layer + ring buffer, without relying on PTE Accessed-bit scanning or + page faults. Registers as the "paddr_ibs" operations set. + Requires AMD processors with IBS Op support. + endmenu
mm/damon/Makefile +1 -0
@@ -9,3 +9,4 @@ obj-$(CONFIG_DAMON_RECLAIM) += modules-common.o reclaim.o obj-$(CONFIG_DAMON_LRU_SORT) += modules-common.o lru_sort.o obj-$(CONFIG_DAMON_STAT) += modules-common.o stat.o obj-$(CONFIG_DAMON_ACMA) += modules-common.o acma.o +obj-$(CONFIG_DAMON_IBS) += damon_ibs.o
mm/damon/damon_ibs.c new file +369 -0
369 lines not on this page · raw message
Signature · 1 line
2.43.0
SeongJae Park
maintainer+ Akinobu
Hello Ravi,
Quoted 8 lines · On Sat, 16 May 2026 15:34:25 -0700 Ravi Jonnalagadda <[email protected]>…
Hi all,
This is an RFC, not for merge. The series exercises and validates
damon_report_access()-- the consumer API SeongJae introduced in [1] -- as a substrate for ingesting access reports from hardware-sampling sources. The series includes one worked-example backend, an AMD IBS Op module (damon_ibs.ko), that runs on Zen 3+ silicon via the existing perf event subsystem.
Thank you for sharing this great RFC series!
[...]
Quoted 2 lines
Why a hardware-source primitive complements existing primitives
===============================================================
[...]
Quoted 11 lines
Both primitives produce a view of hotness that converges to the true distribution over the aggregation interval. For systems where the address space is small relative to the aggregation rate, this is the right tool. On large heterogeneous-memory systems with goal- driven schemes asking the closed-loop tuner to converge on a target distribution, a complementary lower-latency view of accesses can tighten the loop -- reducing the time DAMON's
nr_accessestakes to reflect the workload's actual access distribution, which in turn reduces ramp duration and oscillation amplitude during convergence of goal-driven schemes.
Quoted 4 lines
A hardware-sampling primitive provides this complementary view:
hardware retirement records each access at its natural event rate, with a physical address per sample, independent of TLB state and independent of the unmap/fault path.
Yes, I fully agree. Different multiple access check primitives have different characteristics.
[...]
Quoted 2 lines
Demonstration
=============
[...]
Quoted 4 lines
In both regimes, convergence to target is quick, and the workload's measured DRAM share then holds within 1.3 percentage points of target with standard deviation under 1.3 percentage points, sustained over runs of 15-30 minutes per target.
I understand this demonstration shows your AMD IBS-based version of DAMON is functioning as expected. Thank you for sharing this!
[...]
Quoted 17 lines
What's in this series
=====================Patch 1.
mm/damon/core: refcount ops owner module to prevent
rmmod UAF
Patch 2.mm/damon/paddr: export damon_pa_* ops for IBS module Patch 3.mm/damon/core: replace mutex-protected report buffer
with per-CPU lockless ring
Patch 4.mm/damon/core: flat-array snapshot + bsearch in ring-
drain loop
Patch 5. mm/damon: add sysfs binding and dispatch hookup for
paddr_ibsoperations
Patch 6.mm/damon/core: acceptpaddr_ibsinnode_eligible_mem_bp
ops check
Patch 7.mm/damon/damon_ibs: add AMD IBS-based access sampling
backend
Quoted 3 lines
Patches 1, 3, and 4 are general infrastructure that benefits any consumer of
damon_report_access(). Patches 2, 5, 6, and 7 are the worked-example backend (paddr_ibsops, sysfs binding, IBS module).
I didn't read the detailed code of each patch. But my high level understanding is as below.
Patches 1 and 2 are needed for supporting loadable module-based DAMON operation sets (access sampling backend).
Patch 3 is needed for supporting access check primitives that can provide the access information in only nmi context. It can also speedup the access reporting in general, though.
Patch 4 makes DAMON's internal reported access information retrieval faster, so will help any reporting-based DAMON operation set use case.
Patches 5-7 are required for only the IBS-based DAMON operations set (paddr_ibs).
So I agree patch 4 is a general infrastructure improvement that benefits multiple use cases.
Patch 3 is also arguably general infrastructure improvement, as it will make the reporting faster in general.
Patch 1 is not technically coupled with paddr_ibs, and will be needed for general loadable module based access check primitives. But, should we support lodable modules? If so, why?
Patch 2 is also not technically coupled with paddr_ibs, to my understanding, so should be categorized together with patch 1? In other words, if we agree we should support lodable modules based DAMON operation sets, this should be useful for not only paddr_ibs but more general cases.
Correct me if I'm wrong.
Quoted 20 lines
Patches worth folding into damon/next
=====================================Patches 1, 3, and 4 are not specific to IBS or to this RFC's backend. Each is preparatory infrastructure that any consumer of
damon_report_access()will need:- Patch 1 (refcount ops owner) -- any modular ops set, including out-of-tree backends, needs clean module unload to avoid UAF on
damon_unregister_ops.
- Patch 3 (per-CPU lockless ring) --damon_report_access()cannot be called from NMI context with the current mutex-protected buffer. Hardware samplers all need NMI-safe submission.
- Patch 4 (flat-array snapshot + bsearch drain) -- the linear- scan drain is O(reports x regions) and exceeds the sample interval at high-CPU x large-region products. Bsearch brings it to O(reports x log regions).
Quoted 3 lines
If these belong directly on damon/next as preparatory patches for
damon_report_access()rather than living inside an IBS-specific track, we are happy to rebase and resend them that way.
So I'm bit unsure about patch 1. If we don't have a plan to support lodable modules based DAMON operations set, we might not need it for now.
For patches 3 and 4, I agree those will be useful in general. Nonetheless, I'd slightly prefer to do that optimizations at the later part of the long term project.
Quoted 22 lines
Relation to prior and ongoing work
==================================The IBS sampling pattern in patch 7 -- attr.config=0 to use IBS Op default config,
dc_phy_addr_validfilter, NMI-safe sample submission -- is derived from concepts in Bharata B Rao's pghot RFC v5 [3]. The attribution header is inmm/damon/damon_ibs.cand the patch carries a Suggested-by: trailer.Bharata's pghot v7 [4] introduces a different IBS driver targeting the new IBS Memory Profiler (IBS-MProf) facility, which Bharata describes as a facility "that will be present in future AMD processors" -- a separate IBS instance from the one this RFC's backend uses. This version of driver based out of v5 [3] is an example of how DAMON can be benefited from AMD IBS Hardware source and validates importance of IBS information indepedently. It is not meant to be merged in the current form. @Bharata if you see a path where IBS samples can be consumed by DAMON at some point, will be happy to collaborate.
Quoted 7 lines
Akinobu Mita's perf-event-based access-check RFC [5] explores a configurable perf-event-driven access source for DAMON. IBS has vendor-specific MSR setup beyond what
perf_event_attralone expresses (e.g.dc_phy_addr_validfiltering on the produced sample, not on the perf attr), so the IBS path here appears complementary to [5] -- operators choose based on whether their hardware sampler fits stock perf or needs additional kernel-side setup.
So apparently there are multiple approaches to develop and use h/w-based access monitoring. Akinobu and you are trying to do that using DAMON as the frontend, and already made the working prototypes. There were more people who showed interest and will to contribute to this project other than you, too. I 100% agree h/w-based access monitoring can be useful, and I of course thinking using DAMON as the fronend is the right approach. I'm all for making this upstreamed.
I was therefore spending time on thinking about in what long-term maintainable shape this capability can successfully be upstreamed. I suggested damon_report_access() as the internal interface between DAMON and the h/w-based access check primitives, and apparently we all (I, Ravi and Akinobu in this context) agreed. Akinobu thankfully revisioned his implementation based on damon_report_access() interface. Ravi also implemented this RFC based on the interface.
After making the consensus with Akinobu, I was taking time on the user space interface. When I was discussing with Akinobu, my idea was extending the user interface for the page faults based monitoring v3 [1]. But, recently I decided to make this more general, so proposed data attributes monitoring extension [2] at LSFMMBPF. The patch series for the initial change [3] is merged into mm-new for more testing, today. The cover letter of the patch series is also sharing how it will be extended for h/w based access monitoring in long term.
I of course want us to go in this direction. I believe you already had chances to take a look on the long term plan and didn't make some voice because you don't strongly disagree about the plan. If not, please make a voice.
Assuming you don't have concern on the long term plan yet, I will take time to write down more formal and detailed plan. It will explain the overall roadmap, timeline and how we could collaborate. On top of that, we could further discuss.
Quoted 5 lines
Specific asks
=============
Quoted 8 lines
To SeongJae:
1. Patches 1, 3, and 4 are infrastructure that benefits any consumer of
damon_report_access(), not just the IBS backend in this RFC. Would these belong directly on damon/next as preparatory patches fordamon_report_access(), rather than living inside an IBS-specific track? Happy to rebase and resend them that way if you'd prefer that shape. Tested-by: tags can come along.
I'm still thinking about how we can collaborate well. The answer for the above question would be a part of that. In other words, I have no good answer right now, sorry. Could you please give me more time to think more and share the plan? I will share the plan as another mail. On the thread, we could further discuss. Of course, we could have DAMON beer/coffee/tea chats [4] like additional discussions before/after/during the plan discussion.
So, long story short, we agreed this project (h/w-based data access monitoring) should be upstreamed. But give me little more time on thinking about how we will do it and collaborate. It will take some time. Please bear in mind. Sorry for making you wait, but I pretty sure and promise that we will eventually make it.
[1] https://lore.kernel.org/[email protected]
[2] https://lwn.net/Articles/1071256/
[3] https://lore.kernel.org/[email protected]
[4] https://docs.google.com/document/d/1v43Kcj3ly4CYqmAkMaZzLiM2GEnWfgdGbZAH3mi2vpM/edit?usp=sharing
Thanks,
SJ
[...]
Quoted 194 lines · SeongJae Park
+ Akinobu
Hello Ravi,
Quoted 8 lines · On Sat, 16 May 2026 15:34:25 -0700 Ravi Jonnalagadda <[email protected]>…
Hi all,
This is an RFC, not for merge. The series exercises and validates
damon_report_access()-- the consumer API SeongJae introduced in [1] -- as a substrate for ingesting access reports from hardware-sampling sources. The series includes one worked-example backend, an AMD IBS Op module (damon_ibs.ko), that runs on Zen 3+ silicon via the existing perf event subsystem.Thank you for sharing this great RFC series!
[...]
Quoted 2 lines
Why a hardware-source primitive complements existing primitives
===============================================================[...]
Quoted 11 lines
Both primitives produce a view of hotness that converges to the true distribution over the aggregation interval. For systems where the address space is small relative to the aggregation rate, this is the right tool. On large heterogeneous-memory systems with goal- driven schemes asking the closed-loop tuner to converge on a target distribution, a complementary lower-latency view of accesses can tighten the loop -- reducing the time DAMON's
nr_accessestakes to reflect the workload's actual access distribution, which in turn reduces ramp duration and oscillation amplitude during convergence of goal-driven schemes.Quoted 4 lines
A hardware-sampling primitive provides this complementary view:
hardware retirement records each access at its natural event rate, with a physical address per sample, independent of TLB state and independent of the unmap/fault path.Yes, I fully agree. Different multiple access check primitives have different characteristics.
[...]
Quoted 2 lines
Demonstration
=============[...]
Quoted 4 lines
In both regimes, convergence to target is quick, and the workload's measured DRAM share then holds within 1.3 percentage points of target with standard deviation under 1.3 percentage points, sustained over runs of 15-30 minutes per target.
I understand this demonstration shows your AMD IBS-based version of DAMON is functioning as expected. Thank you for sharing this!
[...]
Quoted 17 lines
What's in this series
=====================Patch 1.
mm/damon/core: refcount ops owner module to prevent
rmmod UAF
Patch 2.mm/damon/paddr: export damon_pa_* ops for IBS module Patch 3.mm/damon/core: replace mutex-protected report buffer
with per-CPU lockless ring
Patch 4.mm/damon/core: flat-array snapshot + bsearch in ring-
drain loop
Patch 5. mm/damon: add sysfs binding and dispatch hookup for
paddr_ibsoperations
Patch 6.mm/damon/core: acceptpaddr_ibsinnode_eligible_mem_bp
ops check
Patch 7.mm/damon/damon_ibs: add AMD IBS-based access sampling
backendQuoted 3 lines
Patches 1, 3, and 4 are general infrastructure that benefits any consumer of
damon_report_access(). Patches 2, 5, 6, and 7 are the worked-example backend (paddr_ibsops, sysfs binding, IBS module).I didn't read the detailed code of each patch. But my high level understanding is as below.
Patches 1 and 2 are needed for supporting loadable module-based DAMON operation sets (access sampling backend).
Patch 3 is needed for supporting access check primitives that can provide the access information in only nmi context. It can also speedup the access reporting in general, though.
Patch 4 makes DAMON's internal reported access information retrieval faster, so will help any reporting-based DAMON operation set use case.
Patches 5-7 are required for only the IBS-based DAMON operations set (
paddr_ibs).So I agree patch 4 is a general infrastructure improvement that benefits multiple use cases.
Patch 3 is also arguably general infrastructure improvement, as it will make the reporting faster in general.
Patch 1 is not technically coupled with
paddr_ibs, and will be needed for general loadable module based access check primitives. But, should we support lodable modules? If so, why?Patch 2 is also not technically coupled with
paddr_ibs, to my understanding, so should be categorized together with patch 1? In other words, if we agree we should support lodable modules based DAMON operation sets, this should be useful for not onlypaddr_ibsbut more general cases.Correct me if I'm wrong.
Quoted 20 lines
Patches worth folding into damon/next
=====================================Patches 1, 3, and 4 are not specific to IBS or to this RFC's backend. Each is preparatory infrastructure that any consumer of
damon_report_access()will need:- Patch 1 (refcount ops owner) -- any modular ops set, including out-of-tree backends, needs clean module unload to avoid UAF on
damon_unregister_ops.
- Patch 3 (per-CPU lockless ring) --damon_report_access()cannot be called from NMI context with the current mutex-protected buffer. Hardware samplers all need NMI-safe submission.
- Patch 4 (flat-array snapshot + bsearch drain) -- the linear- scan drain is O(reports x regions) and exceeds the sample interval at high-CPU x large-region products. Bsearch brings it to O(reports x log regions).Quoted 3 lines
If these belong directly on damon/next as preparatory patches for
damon_report_access()rather than living inside an IBS-specific track, we are happy to rebase and resend them that way.So I'm bit unsure about patch 1. If we don't have a plan to support lodable modules based DAMON operations set, we might not need it for now.
For patches 3 and 4, I agree those will be useful in general. Nonetheless, I'd slightly prefer to do that optimizations at the later part of the long term project.
Quoted 22 lines
Relation to prior and ongoing work
==================================The IBS sampling pattern in patch 7 -- attr.config=0 to use IBS Op default config,
dc_phy_addr_validfilter, NMI-safe sample submission -- is derived from concepts in Bharata B Rao's pghot RFC v5 [3]. The attribution header is inmm/damon/damon_ibs.cand the patch carries a Suggested-by: trailer.Bharata's pghot v7 [4] introduces a different IBS driver targeting the new IBS Memory Profiler (IBS-MProf) facility, which Bharata describes as a facility "that will be present in future AMD processors" -- a separate IBS instance from the one this RFC's backend uses. This version of driver based out of v5 [3] is an example of how DAMON can be benefited from AMD IBS Hardware source and validates importance of IBS information indepedently. It is not meant to be merged in the current form. @Bharata if you see a path where IBS samples can be consumed by DAMON at some point, will be happy to collaborate.
Quoted 7 lines
Akinobu Mita's perf-event-based access-check RFC [5] explores a configurable perf-event-driven access source for DAMON. IBS has vendor-specific MSR setup beyond what
perf_event_attralone expresses (e.g.dc_phy_addr_validfiltering on the produced sample, not on the perf attr), so the IBS path here appears complementary to [5] -- operators choose based on whether their hardware sampler fits stock perf or needs additional kernel-side setup.So apparently there are multiple approaches to develop and use h/w-based access monitoring. Akinobu and you are trying to do that using DAMON as the frontend, and already made the working prototypes. There were more people who showed interest and will to contribute to this project other than you, too. I 100% agree h/w-based access monitoring can be useful, and I of course thinking using DAMON as the fronend is the right approach. I'm all for making this upstreamed.
I was therefore spending time on thinking about in what long-term maintainable shape this capability can successfully be upstreamed. I suggested
damon_report_access()as the internal interface between DAMON and the h/w-based access check primitives, and apparently we all (I, Ravi and Akinobu in this context) agreed. Akinobu thankfully revisioned his implementation based ondamon_report_access()interface. Ravi also implemented this RFC based on the interface.After making the consensus with Akinobu, I was taking time on the user space interface. When I was discussing with Akinobu, my idea was extending the user interface for the page faults based monitoring v3 [1]. But, recently I decided to make this more general, so proposed data attributes monitoring extension [2] at LSFMMBPF. The patch series for the initial change [3] is merged into mm-new for more testing, today. The cover letter of the patch series is also sharing how it will be extended for h/w based access monitoring in long term.
Quoted 4 lines
I of course want us to go in this direction. I believe you already had chances to take a look on the long term plan and didn't make some voice because you don't strongly disagree about the plan. If not, please make a voice.
Hi SJ,
One layering question I'd like to flag before the plan is written, since it affects how this RFC's substrate slots in:
In [3], .apply_probes is a periodic per-region classifier driven from kdamond_fn after .check_accesses, in process context, that applies a (folio -> bool) predicate to each region's sampling_addr and accounts the results in r->probe_hits[]. damon_report_access() on the other hand is a per-event delivery callback into a per-CPU buffer, called from the access source (NMI for IBS / PEBS / SPE, process context for page-fault-based sources). These appear to me to sit at different layers - delivery vs. classification.
The reason I want to confirm this: NMI context for HW samplers precludes the operations .apply_probes can do today (no mutex, no kmalloc, no sleep, no folio lookup that touches pte_lock). And the data shape is inverted - .apply_probes asks "does region R's sampling_addr have attribute A?", evaluated on the kdamond-chosen address; an HW sample announces "PA Y was accessed at retirement time T", arriving asynchronously and needing to find the region it falls into. If access events end up routed through .apply_probes in the long-term plan, the IBS / PEBS / SPE backends would each need a deferral path under it (per-CPU ring for NMI-safe submission, region mapping at drain time).
Happy to be wrong here if you see a unified shape that handles both - just want to surface the constraint before the plan is written.
On the loadable-module question for patches 1 and 2: agreed it's a genuinely open architectural call, not just a paddr_ibs convenience.
- paddr_ibs (this RFC) targets the existing IBS Op facility on Zen 3+ silicon via the perf event subsystem and uses a
vendor-specific
overflow-handler filter that perf_event_attr cannot express (dc_phy_addr_valid in IBS_OP_DATA3). Bharata's pghot v7 [pghot-v7] introduces a separate IBS driver targeting the new
IBS-MProf
facility on future AMD silicon via direct MSR programming - not perf at all. These are two AMD-specific HW samplers with non-overlapping silicon coverage and non-overlapping kernel paths. A distro shipping a single kernel image to a fleet with mixed silicon needs runtime-selectable backends, which obj=y can't do across exclusive depends on chains.
- Akinobu's perf-event RFC v3 [akinobu-v3] is a useful contrast:
it stays builtin because it's a generic configurable perf_event_attr passthrough, no vendor-specific code in the overflow handler. The tristate case is specifically for the backends that need vendor logic outside perf_event_attr (IBS dc_phy_addr_valid, future ARM SPE record-format handling, future Intel PEBS DLA quirks if they need kernel-side filtering beyond what perf delivers).
Bharata, would value your perspective on two related questions: in your long-term plan for pghot, do you see the legacy IBS Op path (this RFC) staying as a DAMON-side backend, while the new IBS-MProf path lands under pghot? Or do you envision both IBS facilities eventually feeding through a common HW-sampler primitive (pghot or DAMON), with frontend selectable by user config? And on existing Zen 3+ silicon: is the legacy IBS Op driver in this RFC the right home for those processors going forward.
Thanks,
Ravi
Quoted 42 lines
Assuming you don't have concern on the long term plan yet, I will take time to write down more formal and detailed plan. It will explain the overall roadmap, timeline and how we could collaborate. On top of that, we could further discuss.
Quoted 5 lines
Specific asks
=============Quoted 8 lines
To SeongJae:
1. Patches 1, 3, and 4 are infrastructure that benefits any consumer of
damon_report_access(), not just the IBS backend in this RFC. Would these belong directly on damon/next as preparatory patches fordamon_report_access(), rather than living inside an IBS-specific track? Happy to rebase and resend them that way if you'd prefer that shape. Tested-by: tags can come along.I'm still thinking about how we can collaborate well. The answer for the above question would be a part of that. In other words, I have no good answer right now, sorry. Could you please give me more time to think more and share the plan? I will share the plan as another mail. On the thread, we could further discuss. Of course, we could have DAMON beer/coffee/tea chats [4] like additional discussions before/after/during the plan discussion.
So, long story short, we agreed this project (h/w-based data access monitoring) should be upstreamed. But give me little more time on thinking about how we will do it and collaborate. It will take some time. Please bear in mind. Sorry for making you wait, but I pretty sure and promise that we will eventually make it.
[1] https://lore.kernel.org/[email protected]
[2] https://lwn.net/Articles/1071256/
[3] https://lore.kernel.org/[email protected]
[4] https://docs.google.com/document/d/1v43Kcj3ly4CYqmAkMaZzLiM2GEnWfgdGbZAH3mi2vpM/edit?usp=sharingThanks,
SJ[...]
SeongJae Park
maintainerQuoted 195 lines · On Wed, 20 May 2026 12:01:43 -0700 Ravi Jonnalagadda <[email protected]>…
Quoted 194 lines · SeongJae Park
+ Akinobu
Hello Ravi,
Quoted 8 lines · On Sat, 16 May 2026 15:34:25 -0700 Ravi Jonnalagadda <[email protected]>…
Hi all,
This is an RFC, not for merge. The series exercises and validates
damon_report_access()-- the consumer API SeongJae introduced in [1] -- as a substrate for ingesting access reports from hardware-sampling sources. The series includes one worked-example backend, an AMD IBS Op module (damon_ibs.ko), that runs on Zen 3+ silicon via the existing perf event subsystem.Thank you for sharing this great RFC series!
[...]
Quoted 2 lines
Why a hardware-source primitive complements existing primitives
===============================================================[...]
Quoted 11 lines
Both primitives produce a view of hotness that converges to the true distribution over the aggregation interval. For systems where the address space is small relative to the aggregation rate, this is the right tool. On large heterogeneous-memory systems with goal- driven schemes asking the closed-loop tuner to converge on a target distribution, a complementary lower-latency view of accesses can tighten the loop -- reducing the time DAMON's
nr_accessestakes to reflect the workload's actual access distribution, which in turn reduces ramp duration and oscillation amplitude during convergence of goal-driven schemes.Quoted 4 lines
A hardware-sampling primitive provides this complementary view:
hardware retirement records each access at its natural event rate, with a physical address per sample, independent of TLB state and independent of the unmap/fault path.Yes, I fully agree. Different multiple access check primitives have different characteristics.
[...]
Quoted 2 lines
Demonstration
=============[...]
Quoted 4 lines
In both regimes, convergence to target is quick, and the workload's measured DRAM share then holds within 1.3 percentage points of target with standard deviation under 1.3 percentage points, sustained over runs of 15-30 minutes per target.
I understand this demonstration shows your AMD IBS-based version of DAMON is functioning as expected. Thank you for sharing this!
[...]
Quoted 17 lines
What's in this series
=====================Patch 1.
mm/damon/core: refcount ops owner module to prevent
rmmod UAF
Patch 2.mm/damon/paddr: export damon_pa_* ops for IBS module Patch 3.mm/damon/core: replace mutex-protected report buffer
with per-CPU lockless ring
Patch 4.mm/damon/core: flat-array snapshot + bsearch in ring-
drain loop
Patch 5. mm/damon: add sysfs binding and dispatch hookup for
paddr_ibsoperations
Patch 6.mm/damon/core: acceptpaddr_ibsinnode_eligible_mem_bp
ops check
Patch 7.mm/damon/damon_ibs: add AMD IBS-based access sampling
backendQuoted 3 lines
Patches 1, 3, and 4 are general infrastructure that benefits any consumer of
damon_report_access(). Patches 2, 5, 6, and 7 are the worked-example backend (paddr_ibsops, sysfs binding, IBS module).I didn't read the detailed code of each patch. But my high level understanding is as below.
Patches 1 and 2 are needed for supporting loadable module-based DAMON operation sets (access sampling backend).
Patch 3 is needed for supporting access check primitives that can provide the access information in only nmi context. It can also speedup the access reporting in general, though.
Patch 4 makes DAMON's internal reported access information retrieval faster, so will help any reporting-based DAMON operation set use case.
Patches 5-7 are required for only the IBS-based DAMON operations set (
paddr_ibs).So I agree patch 4 is a general infrastructure improvement that benefits multiple use cases.
Patch 3 is also arguably general infrastructure improvement, as it will make the reporting faster in general.
Patch 1 is not technically coupled with
paddr_ibs, and will be needed for general loadable module based access check primitives. But, should we support lodable modules? If so, why?Patch 2 is also not technically coupled with
paddr_ibs, to my understanding, so should be categorized together with patch 1? In other words, if we agree we should support lodable modules based DAMON operation sets, this should be useful for not onlypaddr_ibsbut more general cases.Correct me if I'm wrong.
Quoted 20 lines
Patches worth folding into damon/next
=====================================Patches 1, 3, and 4 are not specific to IBS or to this RFC's backend. Each is preparatory infrastructure that any consumer of
damon_report_access()will need:- Patch 1 (refcount ops owner) -- any modular ops set, including out-of-tree backends, needs clean module unload to avoid UAF on
damon_unregister_ops.
- Patch 3 (per-CPU lockless ring) --damon_report_access()cannot be called from NMI context with the current mutex-protected buffer. Hardware samplers all need NMI-safe submission.
- Patch 4 (flat-array snapshot + bsearch drain) -- the linear- scan drain is O(reports x regions) and exceeds the sample interval at high-CPU x large-region products. Bsearch brings it to O(reports x log regions).Quoted 3 lines
If these belong directly on damon/next as preparatory patches for
damon_report_access()rather than living inside an IBS-specific track, we are happy to rebase and resend them that way.So I'm bit unsure about patch 1. If we don't have a plan to support lodable modules based DAMON operations set, we might not need it for now.
For patches 3 and 4, I agree those will be useful in general. Nonetheless, I'd slightly prefer to do that optimizations at the later part of the long term project.
Quoted 22 lines
Relation to prior and ongoing work
==================================The IBS sampling pattern in patch 7 -- attr.config=0 to use IBS Op default config,
dc_phy_addr_validfilter, NMI-safe sample submission -- is derived from concepts in Bharata B Rao's pghot RFC v5 [3]. The attribution header is inmm/damon/damon_ibs.cand the patch carries a Suggested-by: trailer.Bharata's pghot v7 [4] introduces a different IBS driver targeting the new IBS Memory Profiler (IBS-MProf) facility, which Bharata describes as a facility "that will be present in future AMD processors" -- a separate IBS instance from the one this RFC's backend uses. This version of driver based out of v5 [3] is an example of how DAMON can be benefited from AMD IBS Hardware source and validates importance of IBS information indepedently. It is not meant to be merged in the current form. @Bharata if you see a path where IBS samples can be consumed by DAMON at some point, will be happy to collaborate.
Quoted 7 lines
Akinobu Mita's perf-event-based access-check RFC [5] explores a configurable perf-event-driven access source for DAMON. IBS has vendor-specific MSR setup beyond what
perf_event_attralone expresses (e.g.dc_phy_addr_validfiltering on the produced sample, not on the perf attr), so the IBS path here appears complementary to [5] -- operators choose based on whether their hardware sampler fits stock perf or needs additional kernel-side setup.So apparently there are multiple approaches to develop and use h/w-based access monitoring. Akinobu and you are trying to do that using DAMON as the frontend, and already made the working prototypes. There were more people who showed interest and will to contribute to this project other than you, too. I 100% agree h/w-based access monitoring can be useful, and I of course thinking using DAMON as the fronend is the right approach. I'm all for making this upstreamed.
I was therefore spending time on thinking about in what long-term maintainable shape this capability can successfully be upstreamed. I suggested
damon_report_access()as the internal interface between DAMON and the h/w-based access check primitives, and apparently we all (I, Ravi and Akinobu in this context) agreed. Akinobu thankfully revisioned his implementation based ondamon_report_access()interface. Ravi also implemented this RFC based on the interface.After making the consensus with Akinobu, I was taking time on the user space interface. When I was discussing with Akinobu, my idea was extending the user interface for the page faults based monitoring v3 [1]. But, recently I decided to make this more general, so proposed data attributes monitoring extension [2] at LSFMMBPF. The patch series for the initial change [3] is merged into mm-new for more testing, today. The cover letter of the patch series is also sharing how it will be extended for h/w based access monitoring in long term.
Quoted 8 lines
Quoted 4 lines
I of course want us to go in this direction. I believe you already had chances to take a look on the long term plan and didn't make some voice because you don't strongly disagree about the plan. If not, please make a voice.
Hi SJ,
One layering question I'd like to flag before the plan is written, since it affects how this RFC's substrate slots in:
To my understanding, this RFC reuses the damon_report_access() infrastructure that shared with the per-CPUs/threds/writes/reads monitoring series [1]. My plan at the moment is to keep using it. So from high level view, I think the final picture would be not really different from this RFC.
Quoted 10 lines
In [3], .apply_probes is a periodic per-region classifier driven from
kdamond_fnafter .check_accesses, in process context, that applies a (folio -> bool) predicate to each region'ssampling_addrand accounts the results in r->probe_hits[].damon_report_access()on the other hand is a per-event delivery callback into a per-CPU buffer, called from the access source (NMI for IBS / PEBS / SPE, process context for page-fault-based sources). These appear to me to sit at different layers - delivery vs. classification.
Quoted 11 lines
The reason I want to confirm this: NMI context for HW samplers precludes the operations .apply_probes can do today (no mutex, no kmalloc, no sleep, no folio lookup that touches
pte_lock). And the data shape is inverted - .apply_probes asks "does region R'ssampling_addrhave attribute A?", evaluated on the kdamond-chosen address; an HW sample announces "PA Y was accessed at retirement time T", arriving asynchronously and needing to find the region it falls into. If access events end up routed through .apply_probes in the long-term plan, the IBS / PEBS / SPE backends would each need a deferral path under it (per-CPU ring for NMI-safe submission, region mapping at drain time).
It will not routed through .apply_probes, but work in a way similar to the damon_report_access() based design. That is, each (sampled) access event will syncronously call damon_report_access() with the access information. The information is stored in DAMON's internal data structure. The information will contain the access destination address, the accessor CPU/thread, whether it was reads or writes etc, if available.
Then kdamond will read the reports in the data structure once per sampling interval and assess if each region got accessed or not since the last sampling interval.
So my plan is not to reuse .apply_probes, but in terms of who consumes the information, it is not very different. Accessor will produce the information (report), and kdamond will consume those. But this is how damon_report_access() based structure is working on, so my understanding is that your RFC is also not very different. Am I missing something, or do you have any concern on this structure?
Quoted 21 lines
Happy to be wrong here if you see a unified shape that handles both - just want to surface the constraint before the plan is written.
On the loadable-module question for patches 1 and 2: agreed it's a genuinely open architectural call, not just a
paddr_ibsconvenience.-
paddr_ibs(this RFC) targets the existing IBS Op facility on Zen 3+ silicon via the perf event subsystem and uses a
vendor-specific
overflow-handler filter thatperf_event_attrcannot express (dc_phy_addr_validinIBS_OP_DATA3). Bharata's pghot v7 [pghot-v7] introduces a separate IBS driver targeting the new
IBS-MProf
facility on future AMD silicon via direct MSR programming - not perf at all. These are two AMD-specific HW samplers with non-overlapping silicon coverage and non-overlapping kernel paths. A distro shipping a single kernel image to a fleet with mixed silicon needs runtime-selectable backends, which obj=y can't do across exclusivedepends onchains.
Quoted 8 lines
- Akinobu's perf-event RFC v3 [akinobu-v3] is a useful contrast:
it stays builtin because it's a generic configurableperf_event_attrpassthrough, no vendor-specific code in the overflow handler. The tristate case is specifically for the backends that need vendor logic outsideperf_event_attr(IBSdc_phy_addr_valid, future ARM SPE record-format handling, future Intel PEBS DLA quirks if they need kernel-side filtering beyond what perf delivers).
I'm still not familiar with IBS and perf events. Please bear in mind with me. My understanding is that there are vendor-specific knobs for IBS that perf event is not supporting. So far, that makes sense. And are you saying that you have to write paddr_ibs as a loadable module if you want to support the vendor-specific knobs? If I'm understanding you correctly could you further share why it cannot be done as a builtin module?
[1] https://lore.kernel.org/[email protected]
Thanks,
SJ
[...]
Quoted 204 lines · SeongJae Park
Quoted 201 lines · On Wed, 20 May 2026 12:01:43 -0700 Ravi Jonnalagadda <[email protected]>…
Quoted 194 lines · SeongJae Park
+ Akinobu
Hello Ravi,
Quoted 8 lines · On Sat, 16 May 2026 15:34:25 -0700 Ravi Jonnalagadda <[email protected]>…
Hi all,
This is an RFC, not for merge. The series exercises and validates
damon_report_access()-- the consumer API SeongJae introduced in [1] -- as a substrate for ingesting access reports from hardware-sampling sources. The series includes one worked-example backend, an AMD IBS Op module (damon_ibs.ko), that runs on Zen 3+ silicon via the existing perf event subsystem.Thank you for sharing this great RFC series!
[...]
Quoted 2 lines
Why a hardware-source primitive complements existing primitives
===============================================================[...]
Quoted 11 lines
Both primitives produce a view of hotness that converges to the true distribution over the aggregation interval. For systems where the address space is small relative to the aggregation rate, this is the right tool. On large heterogeneous-memory systems with goal- driven schemes asking the closed-loop tuner to converge on a target distribution, a complementary lower-latency view of accesses can tighten the loop -- reducing the time DAMON's
nr_accessestakes to reflect the workload's actual access distribution, which in turn reduces ramp duration and oscillation amplitude during convergence of goal-driven schemes.Quoted 4 lines
A hardware-sampling primitive provides this complementary view:
hardware retirement records each access at its natural event rate, with a physical address per sample, independent of TLB state and independent of the unmap/fault path.Yes, I fully agree. Different multiple access check primitives have different characteristics.
[...]
Quoted 2 lines
Demonstration
=============[...]
Quoted 4 lines
In both regimes, convergence to target is quick, and the workload's measured DRAM share then holds within 1.3 percentage points of target with standard deviation under 1.3 percentage points, sustained over runs of 15-30 minutes per target.
I understand this demonstration shows your AMD IBS-based version of DAMON is functioning as expected. Thank you for sharing this!
[...]
Quoted 17 lines
What's in this series
=====================Patch 1.
mm/damon/core: refcount ops owner module to prevent
rmmod UAF
Patch 2.mm/damon/paddr: export damon_pa_* ops for IBS module Patch 3.mm/damon/core: replace mutex-protected report buffer
with per-CPU lockless ring
Patch 4.mm/damon/core: flat-array snapshot + bsearch in ring-
drain loop
Patch 5. mm/damon: add sysfs binding and dispatch hookup for
paddr_ibsoperations
Patch 6.mm/damon/core: acceptpaddr_ibsinnode_eligible_mem_bp
ops check
Patch 7.mm/damon/damon_ibs: add AMD IBS-based access sampling
backendQuoted 3 lines
Patches 1, 3, and 4 are general infrastructure that benefits any consumer of
damon_report_access(). Patches 2, 5, 6, and 7 are the worked-example backend (paddr_ibsops, sysfs binding, IBS module).I didn't read the detailed code of each patch. But my high level understanding is as below.
Patches 1 and 2 are needed for supporting loadable module-based DAMON operation sets (access sampling backend).
Patch 3 is needed for supporting access check primitives that can provide the access information in only nmi context. It can also speedup the access reporting in general, though.
Patch 4 makes DAMON's internal reported access information retrieval faster, so will help any reporting-based DAMON operation set use case.
Patches 5-7 are required for only the IBS-based DAMON operations set (
paddr_ibs).So I agree patch 4 is a general infrastructure improvement that benefits multiple use cases.
Patch 3 is also arguably general infrastructure improvement, as it will make the reporting faster in general.
Patch 1 is not technically coupled with
paddr_ibs, and will be needed for general loadable module based access check primitives. But, should we support lodable modules? If so, why?Patch 2 is also not technically coupled with
paddr_ibs, to my understanding, so should be categorized together with patch 1? In other words, if we agree we should support lodable modules based DAMON operation sets, this should be useful for not onlypaddr_ibsbut more general cases.Correct me if I'm wrong.
Quoted 20 lines
Patches worth folding into damon/next
=====================================Patches 1, 3, and 4 are not specific to IBS or to this RFC's backend. Each is preparatory infrastructure that any consumer of
damon_report_access()will need:- Patch 1 (refcount ops owner) -- any modular ops set, including out-of-tree backends, needs clean module unload to avoid UAF on
damon_unregister_ops.
- Patch 3 (per-CPU lockless ring) --damon_report_access()cannot be called from NMI context with the current mutex-protected buffer. Hardware samplers all need NMI-safe submission.
- Patch 4 (flat-array snapshot + bsearch drain) -- the linear- scan drain is O(reports x regions) and exceeds the sample interval at high-CPU x large-region products. Bsearch brings it to O(reports x log regions).Quoted 3 lines
If these belong directly on damon/next as preparatory patches for
damon_report_access()rather than living inside an IBS-specific track, we are happy to rebase and resend them that way.So I'm bit unsure about patch 1. If we don't have a plan to support lodable modules based DAMON operations set, we might not need it for now.
For patches 3 and 4, I agree those will be useful in general. Nonetheless, I'd slightly prefer to do that optimizations at the later part of the long term project.
Quoted 22 lines
Relation to prior and ongoing work
==================================The IBS sampling pattern in patch 7 -- attr.config=0 to use IBS Op default config,
dc_phy_addr_validfilter, NMI-safe sample submission -- is derived from concepts in Bharata B Rao's pghot RFC v5 [3]. The attribution header is inmm/damon/damon_ibs.cand the patch carries a Suggested-by: trailer.Bharata's pghot v7 [4] introduces a different IBS driver targeting the new IBS Memory Profiler (IBS-MProf) facility, which Bharata describes as a facility "that will be present in future AMD processors" -- a separate IBS instance from the one this RFC's backend uses. This version of driver based out of v5 [3] is an example of how DAMON can be benefited from AMD IBS Hardware source and validates importance of IBS information indepedently. It is not meant to be merged in the current form. @Bharata if you see a path where IBS samples can be consumed by DAMON at some point, will be happy to collaborate.
Quoted 7 lines
Akinobu Mita's perf-event-based access-check RFC [5] explores a configurable perf-event-driven access source for DAMON. IBS has vendor-specific MSR setup beyond what
perf_event_attralone expresses (e.g.dc_phy_addr_validfiltering on the produced sample, not on the perf attr), so the IBS path here appears complementary to [5] -- operators choose based on whether their hardware sampler fits stock perf or needs additional kernel-side setup.So apparently there are multiple approaches to develop and use h/w-based access monitoring. Akinobu and you are trying to do that using DAMON as the frontend, and already made the working prototypes. There were more people who showed interest and will to contribute to this project other than you, too. I 100% agree h/w-based access monitoring can be useful, and I of course thinking using DAMON as the fronend is the right approach. I'm all for making this upstreamed.
I was therefore spending time on thinking about in what long-term maintainable shape this capability can successfully be upstreamed. I suggested
damon_report_access()as the internal interface between DAMON and the h/w-based access check primitives, and apparently we all (I, Ravi and Akinobu in this context) agreed. Akinobu thankfully revisioned his implementation based ondamon_report_access()interface. Ravi also implemented this RFC based on the interface.After making the consensus with Akinobu, I was taking time on the user space interface. When I was discussing with Akinobu, my idea was extending the user interface for the page faults based monitoring v3 [1]. But, recently I decided to make this more general, so proposed data attributes monitoring extension [2] at LSFMMBPF. The patch series for the initial change [3] is merged into mm-new for more testing, today. The cover letter of the patch series is also sharing how it will be extended for h/w based access monitoring in long term.
Quoted 4 lines
I of course want us to go in this direction. I believe you already had chances to take a look on the long term plan and didn't make some voice because you don't strongly disagree about the plan. If not, please make a voice.
Hi SJ,
Quoted 8 lines
Quoted 2 lines
One layering question I'd like to flag before the plan is written, since it affects how this RFC's substrate slots in:
To my understanding, this RFC reuses the
damon_report_access()infrastructure that shared with the per-CPUs/threds/writes/reads monitoring series [1]. My plan at the moment is to keep using it. So from high level view, I think the final picture would be not really different from this RFC.
Hi SJ,
Sorry for the delayed reply. Was away for a couple of days.
This resolves the layering question for me.
Quoted 70 lines
Quoted 10 lines
In [3], .apply_probes is a periodic per-region classifier driven from
kdamond_fnafter .check_accesses, in process context, that applies a (folio -> bool) predicate to each region'ssampling_addrand accounts the results in r->probe_hits[].damon_report_access()on the other hand is a per-event delivery callback into a per-CPU buffer, called from the access source (NMI for IBS / PEBS / SPE, process context for page-fault-based sources). These appear to me to sit at different layers - delivery vs. classification.Quoted 11 lines
The reason I want to confirm this: NMI context for HW samplers precludes the operations .apply_probes can do today (no mutex, no kmalloc, no sleep, no folio lookup that touches
pte_lock). And the data shape is inverted - .apply_probes asks "does region R'ssampling_addrhave attribute A?", evaluated on the kdamond-chosen address; an HW sample announces "PA Y was accessed at retirement time T", arriving asynchronously and needing to find the region it falls into. If access events end up routed through .apply_probes in the long-term plan, the IBS / PEBS / SPE backends would each need a deferral path under it (per-CPU ring for NMI-safe submission, region mapping at drain time).It will not routed through .apply_probes, but work in a way similar to the
damon_report_access()based design. That is, each (sampled) access event will syncronously calldamon_report_access()with the access information. The information is stored in DAMON's internal data structure. The information will contain the access destination address, the accessor CPU/thread, whether it was reads or writes etc, if available.Then kdamond will read the reports in the data structure once per sampling interval and assess if each region got accessed or not since the last sampling interval.
So my plan is not to reuse .apply_probes, but in terms of who consumes the information, it is not very different. Accessor will produce the information (report), and kdamond will consume those. But this is how
damon_report_access()based structure is working on, so my understanding is that your RFC is also not very different. Am I missing something, or do you have any concern on this structure?Quoted 21 lines
Happy to be wrong here if you see a unified shape that handles both - just want to surface the constraint before the plan is written.
On the loadable-module question for patches 1 and 2: agreed it's a genuinely open architectural call, not just a
paddr_ibsconvenience.-
paddr_ibs(this RFC) targets the existing IBS Op facility on Zen 3+ silicon via the perf event subsystem and uses a
vendor-specific
overflow-handler filter thatperf_event_attrcannot express (dc_phy_addr_validinIBS_OP_DATA3). Bharata's pghot v7 [pghot-v7] introduces a separate IBS driver targeting the new
IBS-MProf
facility on future AMD silicon via direct MSR programming - not perf at all. These are two AMD-specific HW samplers with non-overlapping silicon coverage and non-overlapping kernel paths. A distro shipping a single kernel image to a fleet with mixed silicon needs runtime-selectable backends, which obj=y can't do across exclusivedepends onchains.Quoted 8 lines
- Akinobu's perf-event RFC v3 [akinobu-v3] is a useful contrast:
it stays builtin because it's a generic configurableperf_event_attrpassthrough, no vendor-specific code in the overflow handler. The tristate case is specifically for the backends that need vendor logic outsideperf_event_attr(IBSdc_phy_addr_valid, future ARM SPE record-format handling, future Intel PEBS DLA quirks if they need kernel-side filtering beyond what perf delivers).
Quoted 7 lines
I'm still not familiar with IBS and perf events. Please bear in mind with me. My understanding is that there are vendor-specific knobs for IBS that perf event is not supporting. So far, that makes sense. And are you saying that you have to write
paddr_ibsas a loadable module if you want to support the vendor-specific knobs? If I'm understanding you correctly could you further share why it cannot be done as a builtin module?
This RFC's IBS backend does not claim exclusive use of IBS.
The reason patch 7 ships as tristate is precedent and reuse: Bharata's pghot v5 posted its IBS driver as a module, and I matched that shape during development so the two consumers could potentially share IBS plumbing instead of duplicating it. Patches 1 and 2 exist to support loadable ops modules generally.
More clarity on the right shape may emerge once Bharata comments on the questions below. If pghot ends up being the sole in-tree consumer of legacy IBS Op, the loadable-ops question can be deferred to whenever a second primitive actually needs it - patches 1 and 2 dropped, patch 7 made obj=y or folded. If both pghot and DAMON consume IBS, the modular shape earns its keep.
Bharata - can you please weigh in on the long-term home for legacy IBS Op (under pghot alongside IBS-MProf [pghot-v7] vs DAMON-side backend).
Related: any thoughts on the operator-facing knob to pick between consumers when both are present? Happy to align.
Thanks,
Ravi
Quoted 7 lines
[1] https://lore.kernel.org/[email protected]
Thanks,
SJ[...]
SeongJae Park
maintainerQuoted 11 lines · On Mon, 25 May 2026 17:05:06 -0700 Ravi Jonnalagadda <[email protected]>…
Quoted 10 lines · SeongJae Park
Quoted 7 lines · On Wed, 20 May 2026 12:01:43 -0700 Ravi Jonnalagadda <[email protected]>…
Quoted 6 lines · SeongJae Park
+ Akinobu
Hello Ravi,
On Sat, 16 May 2026 15:34:25 -0700 Ravi Jonnalagadda <[email protected]> wrote:
[...]
Quoted 9 lines
Quoted 5 lines
To my understanding, this RFC reuses the
damon_report_access()infrastructure that shared with the per-CPUs/threds/writes/reads monitoring series [1]. My plan at the moment is to keep using it. So from high level view, I think the final picture would be not really different from this RFC.Hi SJ,
Sorry for the delayed reply. Was away for a couple of days.
No worry!
Quoted 1 line
This resolves the layering question for me.
Glad to hear it helps!
[...]
Quoted 8 lines
Quoted 7 lines
I'm still not familiar with IBS and perf events. Please bear in mind with me. My understanding is that there are vendor-specific knobs for IBS that perf event is not supporting. So far, that makes sense. And are you saying that you have to write
paddr_ibsas a loadable module if you want to support the vendor-specific knobs? If I'm understanding you correctly could you further share why it cannot be done as a builtin module?
Quoted 7 lines
This RFC's IBS backend does not claim exclusive use of IBS.
The reason patch 7 ships as tristate is precedent and reuse: Bharata's pghot v5 posted its IBS driver as a module, and I matched that shape during development so the two consumers could potentially share IBS plumbing instead of duplicating it. Patches 1 and 2 exist to support loadable ops modules generally.
Got it, thank you for clarifying!
Thanks,
SJ
[...]