x86/resctrl: Introduce MMIO-based CMT access for Enhanced RDT

RFC PATCH Superseded Chen Yu 2 participants 9 messages May 27 – May 29

Subsystems (2)

x86 RDT - resource allocation

lore

Share & tools

lore

Chen Yu

author

2026-05-27 09:34 UTC #0 lore raw

cover x86/resctrl: Introduce MMIO-based CMT access for Enhanced RDT

Intel Enhanced Resource Director Technology (ERDT) extends the existing RDT framework with two major capabilities:

1. MMIO-based access to monitoring and allocation registers, replacing the legacy MSR-based interface.
2. Region-aware RDT for fine-grained control over different tiers of

     memory (e.g., CXL.mem, DDR).

This is described in the Intel RDT Architecture Specification:
https://cdrdv2-public.intel.com/789566/356688-intel-rdt-arch-spec.pdf

This patch set focuses on the first part: enabling MMIO-based access for Cache Monitoring Technology (CMT), while CAT/MBM/MBA are still using MSR. The platform advertises the MMIO register layout through the ACPI ERDT (Enhanced Resource Director Technology) table, which contains sub-tables describing per-domain register regions for monitoring and allocation.

With ERDT, L3 cache occupancy counters are read via MMIO rather than MSR, allowing the reads to be performed from any CPU without requiring cross-CPU IPIs. This series parses the relevant ACPI sub-tables (RMDD, CMRC), prepares the resctrl monitor infrastructure for MMIO-based reads, and adds initial support for reading L3 occupancy via the CMRC interface.

kselftest of CMT and L3_CAT has passed with minor adjustment at
https://lore.kernel.org/lkml/[email protected]/.

Anil S Keshavamurthy (1):
x86/resctrl: Parse ACPI ERDT table and map RMDD domains by L3 cache ID

Chen Yu (4):
x86/resctrl: Parse ACPI CMRC table
x86/resctrl: Rename prev_msr to prev_mon_val x86/resctrl: Refactor the monitor read function x86/resctrl: Add support for L3 occupancy monitoring via RMID MMIO
read

Tony Luck (1):
fs/resctrl: Do not invoke smp_processor_id() in preemptible context

 arch/x86/Kconfig                       |   4 +-
 arch/x86/include/asm/resctrl.h         |   4 +
 arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
 arch/x86/kernel/cpu/resctrl/core.c     |   5 +-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 451 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  11 +-
 arch/x86/kernel/cpu/resctrl/monitor.c  |  64 ++--
 fs/resctrl/monitor.c                   |  38 ++-
 8 files changed, 538 insertions(+), 40 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c

Signature · 1 line

2.25.1

Chen Yu

author

2026-05-27 09:36 UTC #1 lore raw

RFC PATCH 1/6 x86/resctrl: Parse ACPI ERDT table and map RMDD domains by L3 cache ID

From: Anil S Keshavamurthy <[email protected]>

ERDT(Enhanced RDT) introduces a new top-level ACPI structure (the ERDT) that the kernel must parse before any enhanced RDT feature can be used. The ERDT improves the existing RDT by switching low-level register access from MSR-based to MMIO-based, which is more efficient.

The ERDT structure may include several sub ACPI tables:

- Resource Management Domain Description Structure (RMDD)
- CPU Agent Collection Description Structure (CACD)
- Cache Monitoring Registers for CPU Agents Description Structure

    (CMRC)

There is one ERDT table per platform.
Each RMDD substructure in ERDT represents one resource management domain (RMD), also known as an L3 domain. Thus, the total number of RMDDs equals the number of L3 domains on the platform. Each RMDD contains information such as MMIO addresses. This address is used to retrieve RDT metrics like L3 occupancy.

Add basic ERDT ACPI table and sub-table parsing, and store the relevant tables for later processing.

Among these sub-tables, RMDD requires special handling. There is one RMDD per domain, and the domain ID reuses the L3 cache ID. Many code paths need to retrieve an RMDD efficiently by domain ID (L3 cache ID). Because L3 cache IDs are derived from x2APIC IDs and are not contiguous, using a plain array indexed by domain ID would waste memory. As a trade-off, an xarray is used to store these tables, with the L3 cache ID as the key.

Suggested-by Tony Luck <[email protected]>

Co-developed-by Chen Yu <[email protected]>

Signed-off-by Chen Yu <[email protected]>

Signed-off-by Anil S Keshavamurthy <[email protected]>

---
 arch/x86/Kconfig                       |   4 +-
 arch/x86/include/asm/resctrl.h         |   2 +
 arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
 arch/x86/kernel/cpu/resctrl/core.c     |   3 +
 arch/x86/kernel/cpu/resctrl/erdt.c     | 321 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |   3 +
 6 files changed, 332 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c

arch/x86/Kconfig +2 -2

index f3f7cb01d69d..97d210bd9bb5 100644
@@ -515,7 +515,7 @@ config X86_MPPARSE
 
 config X86_CPU_RESCTRL
 	bool "x86 CPU resource control support"
-	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
+	depends on X86_64 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	depends on MISC_FILESYSTEMS
 	select ARCH_HAS_CPU_RESCTRL
 	select RESCTRL_FS

@@ -538,7 +538,7 @@ config X86_CPU_RESCTRL
 
 config X86_CPU_RESCTRL_INTEL_AET
 	bool "Intel Application Energy Telemetry"
-	depends on X86_64 && X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y
+	depends on X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y
 	help
 	  Enable per-RMID telemetry events in resctrl.

arch/x86/include/asm/resctrl.h +2 -0

index 575f8408a9e7..97c2f6bc7a5f 100644
@@ -40,6 +40,8 @@ struct resctrl_pqr_state {
 	u32			default_closid;
 };
 
+bool erdt_enabled(void);
+
 DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state);
 
 extern bool rdt_alloc_capable;

arch/x86/kernel/cpu/resctrl/Makefile +1 -0

index 273ddfa30836..2216ee084832 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_X86_CPU_RESCTRL_INTEL_AET)	+= intel_aet.o
+obj-$(CONFIG_X86_CPU_RESCTRL)		+= erdt.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
 
 # To allow define_trace.h's recursive include:

arch/x86/kernel/cpu/resctrl/core.c +3 -0

index 7667cf7c4e94..893d2efe76f0 100644
@@ -1012,6 +1012,7 @@ static __init void check_quirks(void)
 
 static __init bool get_rdt_resources(void)
 {
+	erdt_init();
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();

@@ -1165,6 +1166,8 @@ static void __exit resctrl_arch_exit(void)
 	cpuhp_remove_state(rdt_online);
 
 	resctrl_exit();
+
+	erdt_exit();
 }
 
 __exitcall(resctrl_arch_exit);

arch/x86/kernel/cpu/resctrl/erdt.c new file +321 -0

321 lines not on this page · raw message

arch/x86/kernel/cpu/resctrl/internal.h +3 -0

index e3cfa0c10e92..9c59bd5e028e 100644
@@ -253,4 +253,7 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
 static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
 #endif
 
+int erdt_init(void);
+void erdt_exit(void);
+
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */

Signature · 1 line

2.25.1

Luck, Tony

maintainer

2026-05-28 20:21 UTC #2 lore raw

Re: [RFC PATCH 1/6] x86/resctrl: Parse ACPI ERDT table and map RMDD domains by L3 cache ID

arch/x86/kernel/cpu/resctrl/erdt.c line 94

Quoted 13 lines · Chen Yu

arch/x86/kernel/cpu/resctrl/erdt.c new file +13 -0

+static __init int get_l3_cache_id_from_cacd(struct acpi_erdt_cacd *cacd)
+{
+	int num_ids, cpu, online_cpu = -1, cache_id = -1, tmp;
+	struct cacheinfo *ci;
+
+	if (cacd->header.length < sizeof(*cacd) + sizeof(cacd->X2APICIDS[0])) {
+		pr_warn(FW_BUG "Invalid x2apicid CACD table\n");
+		return -1;
+	}
+
+	num_ids = (cacd->header.length - sizeof(*cacd)) / sizeof(cacd->X2APICIDS[0]);
+
+	guard(cpus_read_lock)();

This file should #include <linux/cleanup.h> (maybe some other headers too) to follow the "include-what-you-use" best practice (which is only lightly enforced in the kernel - but saves pain when some refactor breaks your code because the implicit chain of includes is changed).

-Tony

Chen, Yu C

author

2026-05-29 02:16 UTC #3 lore raw

Re: [RFC PATCH 1/6] x86/resctrl: Parse ACPI ERDT table and map RMDD domains by L3 cache ID

Quoted 11 lines · Tony

Quoted 10 lines · Chen Yu

arch/x86/kernel/cpu/resctrl/erdt.c new file +10 -0

+static __init int get_l3_cache_id_from_cacd(struct acpi_erdt_cacd *cacd)
+{
+	int num_ids, cpu, online_cpu = -1, cache_id = -1, tmp;
+	struct cacheinfo *ci;
+
+	if (cacd->header.length < sizeof(*cacd) + sizeof(cacd->X2APICIDS[0])) {
+		pr_warn(FW_BUG "Invalid x2apicid CACD table\n");
+		return -1;
+	}
+

Quoted 9 lines

arch/x86/kernel/cpu/resctrl/erdt.c line 94
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/erdt.c new file +3 -0
+	num_ids = (cacd->header.length - sizeof(*cacd)) / sizeof(cacd->X2APICIDS[0]);
+
+	guard(cpus_read_lock)();
This file should #include <linux/cleanup.h> (maybe some other headers too) to follow the "include-what-you-use" best practice (which is only lightly enforced in the kernel - but saves pain when some refactor breaks your code because the implicit chain of includes is changed).

Thanks Tony, will fix this. Appreciate the guidance on IWYU.

thanks,
Chenyu

Chen Yu

author

2026-05-27 09:36 UTC #4 lore raw

RFC PATCH 2/6 x86/resctrl: Parse ACPI CMRC table

In reply to Chen Yu

The CMRC (Cache Monitoring Registers for CPU Agents Description) sub-table of ERDT describes the MMIO registers used to read cache monitoring counters (e.g. LLC occupancy) for an RMD.

Parse each CMRC sub-table, ioremap its register window, and save the CMRC pointer in the corresponding ERDT domain entry so that later monitoring code can read the counters via MMIO.

Suggested-by Tony Luck <[email protected]>

Signed-off-by Chen Yu <[email protected]>

---
 arch/x86/kernel/cpu/resctrl/erdt.c | 40 ++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

arch/x86/kernel/cpu/resctrl/erdt.c +40 -0

index 2813f48f1411..30b7dfd99086 100644
@@ -38,6 +38,7 @@ static bool erdt_available;
 static DEFINE_XARRAY(erdt_domain_xa); /* Indexed by L3 cache ID */
 
 #define ERDT_VALID_VERSION 1
+#define CMRC_VALID_INDEX_FUNC_VERSION 1
 
 static u32 valid_subtbl_mask;

@@ -159,6 +160,7 @@ static void erdt_iounmap_domain(struct erdt_domain_info *domain)
 static void cleanup_one_domain(struct erdt_domain_info *d)
 {
 	erdt_iounmap_domain(d);
+	kfree(d->cmrc);
 	kfree(d);
 }

@@ -171,6 +173,40 @@ static __init bool cacd_init(struct erdt_domain_info *d,
 	return *l3_cache_id != -1;
 }
 
+static __init bool cmrc_init(struct erdt_domain_info *d, struct acpi_subtbl_hdr_16 *subtbl)
+{
+	struct acpi_erdt_cmrc *cmrc = (struct acpi_erdt_cmrc *)subtbl;
+
+	if (subtbl->length < sizeof(*cmrc)) {
+		pr_warn(FW_BUG "Truncated CMRC subtable\n");
+		return false;
+	}
+
+	if (cmrc->index_fn != CMRC_VALID_INDEX_FUNC_VERSION) {
+		pr_info("Unknown CMRC index function %d\n", cmrc->index_fn);
+		return false;
+	}
+
+	if (!cmrc->clump_size) {
+		pr_warn(FW_BUG "CMRC clump_size is zero\n");
+		return false;
+	}
+
+	d->base[ERDT_MMIO_CMRC_BASE] = erdt_ioremap_checked(cmrc->cmt_reg_base,
+							    cmrc->cmt_reg_size, "CMRC base");
+	if (!d->base[ERDT_MMIO_CMRC_BASE])
+		return false;
+
+	d->cmrc = kmemdup(cmrc, subtbl->length, GFP_KERNEL);
+	if (!d->cmrc) {
+		iounmap(d->base[ERDT_MMIO_CMRC_BASE]);
+		d->base[ERDT_MMIO_CMRC_BASE] = NULL;
+		return false;
+	}
+
+	return true;
+}
+
 static __init bool parse_rmdd_entry(struct acpi_subtbl_hdr_16 *rmdd_hdr)
 {
 	struct acpi_erdt_rmdd *rmdd;

@@ -219,6 +255,10 @@ static __init bool parse_rmdd_entry(struct acpi_subtbl_hdr_16 *rmdd_hdr)
 			if (cacd_init(domain_info, subtbl, &l3_cache_id))
 				subtbl_mask |= BIT(ACPI_ERDT_TYPE_CACD);
 			break;
+		case ACPI_ERDT_TYPE_CMRC:
+			if (cmrc_init(domain_info, subtbl))
+				subtbl_mask |= BIT(ACPI_ERDT_TYPE_CMRC);
+			break;
 		default:
 			break;
 		}

Signature · 1 line

2.25.1

Chen Yu

author

2026-05-27 09:36 UTC #5 lore raw

RFC PATCH 3/6 x86/resctrl: Rename prev_msr to prev_mon_val

In reply to Chen Yu

Rename the prev_msr field in struct arch_mbm_state to prev_mon_val. With ERDT, the previous monitor value may come from an MMIO register rather than from an MSR, so the "msr" suffix is no longer accurate. The new name describes the field by its meaning (the previous monitor value) instead of by the access method.

This is preparation for ERDT support, which reads monitoring counters via MMIO.

No functional change.

Co-developed-by Anil S Keshavamurthy <[email protected]>

Signed-off-by Anil S Keshavamurthy <[email protected]>

Signed-off-by Chen Yu <[email protected]>

---
 arch/x86/kernel/cpu/resctrl/internal.h |  8 +++----
 arch/x86/kernel/cpu/resctrl/monitor.c  | 30 +++++++++++++-------------
 2 files changed, 19 insertions(+), 19 deletions(-)

arch/x86/kernel/cpu/resctrl/internal.h +4 -4

index 9c59bd5e028e..97065dc6e14f 100644
@@ -31,13 +31,13 @@
 /**
  * struct arch_mbm_state - values used to compute resctrl_arch_rmid_read()s
  *			   return value.
- * @chunks:	Total data moved (multiply by rdt_group.mon_scale to get bytes)
- * @prev_msr:	Value of IA32_QM_CTR last time it was read for the RMID used to
- *		find this struct.
+ * @chunks:		Total data moved (multiply by rdt_group.mon_scale to get bytes)
+ * @prev_mon_val:	Previous monitor counter value for the RMID used to
+ *			find this struct.
  */
 struct arch_mbm_state {
 	u64	chunks;
-	u64	prev_msr;
+	u64	prev_mon_val;
 };
 
 /* Setting bit 0 in L3_QOS_EXT_CFG enables the ABMC feature. */

arch/x86/kernel/cpu/resctrl/monitor.c +15 ~~-15~~

index 9bd87bae4983..991f0a796551 100644
@@ -186,7 +186,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 
 		prmid = logical_rmid_to_physical_rmid(cpu, rmid);
 		/* Record any initial, non-zero count value. */
-		__rmid_read_phys(prmid, eventid, &am->prev_msr);
+		__rmid_read_phys(prmid, eventid, &am->prev_mon_val);
 	}
 }

@@ -209,16 +209,16 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_l3_mon_domai
 	}
 }
 
-static u64 mbm_overflow_count(u64 prev_msr, u64 cur_msr, unsigned int width)
+static u64 mbm_overflow_count(u64 prev_val, u64 cur_val, unsigned int width)
 {
 	u64 shift = 64 - width, chunks;
 
-	chunks = (cur_msr << shift) - (prev_msr << shift);
+	chunks = (cur_val << shift) - (prev_val << shift);
 	return chunks >> shift;
 }
 
 static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
-			     u32 rmid, enum resctrl_event_id eventid, u64 msr_val)
+			     u32 rmid, enum resctrl_event_id eventid, u64 mon_val)
 {
 	struct rdt_hw_l3_mon_domain *hw_dom = resctrl_to_arch_mon_dom(d);
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);

@@ -227,12 +227,12 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 
 	am = get_arch_mbm_state(hw_dom, rmid, eventid);
 	if (am) {
-		am->chunks += mbm_overflow_count(am->prev_msr, msr_val,
+		am->chunks += mbm_overflow_count(am->prev_mon_val, mon_val,
 						 hw_res->mbm_width);
 		chunks = get_corrected_mbm_count(rmid, am->chunks);
-		am->prev_msr = msr_val;
+		am->prev_mon_val = mon_val;
 	} else {
-		chunks = msr_val;
+		chunks = mon_val;
 	}
 
 	return chunks * hw_res->mon_scale;

@@ -245,7 +245,7 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	struct rdt_hw_l3_mon_domain *hw_dom;
 	struct rdt_l3_mon_domain *d;
 	struct arch_mbm_state *am;
-	u64 msr_val;
+	u64 mon_val;
 	u32 prmid;
 	int cpu;
 	int ret;

@@ -262,14 +262,14 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	hw_dom = resctrl_to_arch_mon_dom(d);
 	cpu = cpumask_any(&hdr->cpu_mask);
 	prmid = logical_rmid_to_physical_rmid(cpu, rmid);
-	ret = __rmid_read_phys(prmid, eventid, &msr_val);
+	ret = __rmid_read_phys(prmid, eventid, &mon_val);
 
 	if (!ret) {
-		*val = get_corrected_val(r, d, rmid, eventid, msr_val);
+		*val = get_corrected_val(r, d, rmid, eventid, mon_val);
 	} else if (ret == -EINVAL) {
 		am = get_arch_mbm_state(hw_dom, rmid, eventid);
 		if (am)
-			am->prev_msr = 0;
+			am->prev_mon_val = 0;
 	}
 
 	return ret;

@@ -324,7 +324,7 @@ void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 		memset(am, 0, sizeof(*am));
 
 		/* Record any initial, non-zero count value. */
-		__cntr_id_read(cntr_id, &am->prev_msr);
+		__cntr_id_read(cntr_id, &am->prev_mon_val);
 	}
 }

@@ -332,14 +332,14 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d,
 			   u32 unused, u32 rmid, int cntr_id,
 			   enum resctrl_event_id eventid, u64 *val)
 {
-	u64 msr_val;
+	u64 mon_val;
 	int ret;
 
-	ret = __cntr_id_read(cntr_id, &msr_val);
+	ret = __cntr_id_read(cntr_id, &mon_val);
 	if (ret)
 		return ret;
 
-	*val = get_corrected_val(r, d, rmid, eventid, msr_val);
+	*val = get_corrected_val(r, d, rmid, eventid, mon_val);
 
 	return 0;
 }

Signature · 1 line

2.25.1

Chen Yu

author

2026-05-27 09:36 UTC #6 lore raw

RFC PATCH 4/6 x86/resctrl: Refactor the monitor read function

In reply to Chen Yu

Split the monitor read helper into an L3 read path and an AET (Intel Application Energy Telemetry) read path. This makes the two distinct monitoring sources easier to extend independently and prepares the L3 path for ERDT-based MMIO reads added in a later patch.

No functional change.

Signed-off-by Chen Yu <[email protected]>

---
 arch/x86/kernel/cpu/resctrl/monitor.c | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

arch/x86/kernel/cpu/resctrl/monitor.c +19 -8

index 991f0a796551..1e81b3c33843 100644
@@ -238,9 +238,9 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 	return chunks * hw_res->mon_scale;
 }
 
-int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
-			   u32 unused, u32 rmid, enum resctrl_event_id eventid,
-			   void *arch_priv, u64 *val, void *ignored)
+static int arch_l3_read_event(struct rdt_domain_hdr *hdr, u32 rmid,
+			      enum resctrl_event_id eventid, u64 *val,
+			      struct rdt_resource *r)
 {
 	struct rdt_hw_l3_mon_domain *hw_dom;
 	struct rdt_l3_mon_domain *d;

@@ -250,11 +250,6 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	int cpu;
 	int ret;
 
-	resctrl_arch_rmid_read_context_check();
-
-	if (r->rid == RDT_RESOURCE_PERF_PKG)
-		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
-
 	if (!domain_header_is_valid(hdr, RESCTRL_MON_DOMAIN, RDT_RESOURCE_L3))
 		return -EINVAL;

@@ -275,6 +270,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	return ret;
 }
 
+int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
+			   u32 unused, u32 rmid, enum resctrl_event_id eventid,
+			   void *arch_priv, u64 *val, void *ignored)
+{
+	resctrl_arch_rmid_read_context_check();
+
+	switch (r->rid) {
+	case RDT_RESOURCE_L3:
+		return arch_l3_read_event(hdr, rmid, eventid, val, r);
+	case RDT_RESOURCE_PERF_PKG:
+		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
+	default:
+		return -EINVAL;
+	}
+}
+
 static int __cntr_id_read(u32 cntr_id, u64 *val)
 {
 	u64 msr_val;

Signature · 1 line

2.25.1

Chen Yu

author

2026-05-27 09:37 UTC #7 lore raw

RFC PATCH 5/6 fs/resctrl: Do not invoke smp_processor_id() in preemptible context

In reply to Chen Yu

From: Tony Luck <[email protected]>

__l3_mon_event_count() and __l3_mon_event_count_sum() call smp_processor_id() to obtain the current CPU. However, some monitor events can be read from any CPU in task context via mon_event_count(); in that case the calling context is preemptible and smp_processor_id() triggers a debug warning.

Fix this by skipping the current-CPU lookup when the event's any_cpu flag is set, since such events do not need to run on a specific CPU.

Signed-off-by Tony Luck <[email protected]>

Signed-off-by Chen Yu <[email protected]>

---
 fs/resctrl/monitor.c | 38 ++++++++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 10 deletions(-)

fs/resctrl/monitor.c +28 ~~-10~~

index 9fd901c78dc6..371ccae04892 100644
@@ -417,9 +417,33 @@ static void mbm_cntr_free(struct rdt_l3_mon_domain *d, int cntr_id)
 	memset(&d->cntr_cfg[cntr_id], 0, sizeof(*d->cntr_cfg));
 }
 
+/*
+ * Called from preemptible context via a direct call of mon_event_count() for
+ * events that can be read on any CPU.
+ * Called from preemptible but non-migratable process context (mon_event_count()
+ * via smp_call_on_cpu()) OR non-preemptible context (mon_event_count() via
+ * smp_call_function_any()) for events that need to be read on a specific CPU.
+ */
+static bool cpu_on_correct_domain(struct rmid_read *rr)
+{
+	int cpu;
+
+	/* Any CPU is OK for this event */
+	if (rr->evt->any_cpu)
+		return true;
+
+	cpu = smp_processor_id();
+
+	/* Single domain. Must be on a CPU in that domain. */
+	if (rr->hdr)
+		return cpumask_test_cpu(cpu, &rr->hdr->cpu_mask);
+
+	/* Summing domains that share a cache, must be on a CPU for that cache. */
+	return cpumask_test_cpu(cpu, &rr->ci->shared_cpu_map);
+}
+
 static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
-	int cpu = smp_processor_id();
 	u32 closid = rdtgrp->closid;
 	u32 rmid = rdtgrp->mon.rmid;
 	struct rdt_l3_mon_domain *d;

@@ -452,9 +476,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 		return 0;
 	}
 
-	/* Reading a single domain, must be on a CPU in that domain. */
-	if (!cpumask_test_cpu(cpu, &d->hdr.cpu_mask))
-		return -EINVAL;
 	if (rr->is_mbm_cntr)
 		rr->err = resctrl_arch_cntr_read(rr->r, d, closid, rmid, cntr_id,
 						 rr->evt->evtid, &tval);

@@ -472,7 +493,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 
 static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
-	int cpu = smp_processor_id();
 	u32 closid = rdtgrp->closid;
 	u32 rmid = rdtgrp->mon.rmid;
 	struct rdt_l3_mon_domain *d;

@@ -490,10 +510,6 @@ static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *r
 		return -EINVAL;
 	}
 
-	/* Summing domains that share a cache, must be on a CPU for that cache. */
-	if (!cpumask_test_cpu(cpu, &rr->ci->shared_cpu_map))
-		return -EINVAL;
-
 	/*
 	 * Legacy files must report the sum of an event across all
 	 * domains that share the same L3 cache instance.

@@ -524,7 +540,9 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
 	switch (rr->r->rid) {
 	case RDT_RESOURCE_L3:
-		WARN_ON_ONCE(rr->evt->any_cpu);
+		if (!cpu_on_correct_domain(rr))
+			return -EINVAL;
+
 		if (rr->hdr)
 			return __l3_mon_event_count(rdtgrp, rr);
 		else

Signature · 1 line

2.25.1

Chen Yu

author

2026-05-27 09:37 UTC #8 lore raw

RFC PATCH 6/6 x86/resctrl: Add support for L3 occupancy monitoring via RMID MMIO read

In reply to Chen Yu

The CMRC (Cache Monitoring Registers for CPU Agents Description) ACPI sub-table provides the MMIO address used to read the LLC occupancy counter for each RMID. When ERDT is enabled on the platform, use this MMIO interface instead of the legacy MSR read to obtain the L3 occupancy value.

Introduce erdt_mon_read(), a helper that retrieves monitoring data for a given RMID and event ID from an ERDT domain. Initial support is added for the L3 occupancy monitoring event (QOS_L3_OCCUP_EVENT_ID).

If the platform supports ERDT, CMRC-based MMIO access is used by default. If ERDT is unavailable, the implementation is to use MSR-based operations.

Suggested-by Tony Luck <[email protected]>

Co-developed-by Anil S Keshavamurthy <[email protected]>

Signed-off-by Anil S Keshavamurthy <[email protected]>

Signed-off-by Chen Yu <[email protected]>

---
 arch/x86/include/asm/resctrl.h        |  2 +
 arch/x86/kernel/cpu/resctrl/core.c    |  2 +-
 arch/x86/kernel/cpu/resctrl/erdt.c    | 90 +++++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/monitor.c |  7 +++
 4 files changed, 100 insertions(+), 1 deletion(-)

arch/x86/include/asm/resctrl.h +2 -0

index 97c2f6bc7a5f..9b3b03279dd8 100644
@@ -41,6 +41,8 @@ struct resctrl_pqr_state {
 };
 
 bool erdt_enabled(void);
+struct rdt_domain_hdr;
+int erdt_mon_read(struct rdt_domain_hdr *hdr, int ev_id, int rmid, u64 *val);
 
 DECLARE_PER_CPU(struct resctrl_pqr_state, pqr_state);

arch/x86/kernel/cpu/resctrl/core.c +1 -1

index 893d2efe76f0..6e0feae2595c 100644
@@ -965,7 +965,7 @@ static __init bool get_rdt_mon_resources(void)
 	bool ret = false;
 
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
-		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false, 0, NULL);
+		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, erdt_enabled(), 0, NULL);
 		ret = true;
 	}
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {

arch/x86/kernel/cpu/resctrl/erdt.c +90 -0

102 lines not on this page · raw message

arch/x86/kernel/cpu/resctrl/monitor.c +7 -0

index 1e81b3c33843..12b4014e47f3 100644
@@ -278,6 +278,13 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 
 	switch (r->rid) {
 	case RDT_RESOURCE_L3:
+		/*
+		 * No SNC for mmio based L3 occupancy, so there is no need
+		 * to convert logical RMID to a physical RMID via
+		 * logical_rmid_to_physical_rmid().
+		 */
+		if (erdt_enabled() && eventid == QOS_L3_OCCUP_EVENT_ID)
+			return erdt_mon_read(hdr, eventid, rmid, val);
 		return arch_l3_read_event(hdr, rmid, eventid, val, r);
 	case RDT_RESOURCE_PERF_PKG:
 		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);

Signature · 1 line

2.25.1