Introduce MMIO-based CMT access for Enhanced RDT

v6: https://lore.kernel.org/lkml/[email protected]/
v5: https://lore.kernel.org/lkml/[email protected]/
v4: https://lore.kernel.org/lkml/[email protected]/
v3: https://lore.kernel.org/lkml/[email protected]/
v2: https://lore.kernel.org/lkml/[email protected]/
v1: https://lore.kernel.org/lkml/[email protected]/

Intel Enhanced Resource Director Technology (ERDT) extends the existing RDT framework with two major capabilities:

1. MMIO-based access to monitoring and allocation registers, replacing the legacy MSR-based interface.
2. Region-aware RDT for fine-grained control over different tiers of

     memory (e.g., CXL.mem, DDR).

This is described in the Intel RDT Architecture Specification:
https://cdrdv2-public.intel.com/789566/356688-intel-rdt-arch-spec.pdf

This patch set focuses on the first part: enabling MMIO-based access for Cache Monitoring Technology (CMT), while CAT/MBM/MBA are still using MSR. The platform advertises the MMIO register layout through the ACPI ERDT (Enhanced Resource Director Technology) table, which contains sub-tables describing per-domain register regions for monitoring and allocation.

With ERDT, L3 cache occupancy counters are read via MMIO rather than MSR, allowing the reads to be performed from any CPU without requiring cross-CPU IPIs. This series parses the relevant ACPI sub-tables (RMDD, CMRC), prepares the resctrl monitor infrastructure for MMIO-based reads, and adds initial support for reading L3 occupancy via the CMRC interface.

kselftest of CMT and L3_CAT has passed with minor adjustment at
https://lore.kernel.org/lkml/[email protected]/.

Thanks Tony, Reinette, Thomas, Dave, Boris, Peter, Hongyu for your time to look at this patch set.

V7 has gone through local sashiko review, with false positives left unaddressed.

Changes from V6 to V7:
- Let get_rdt_resources() clean up its own state on failure and drop the asymmetric __resctrl_arch_late_init() wrapper. (Reinette Chatre)
- Make erdt_max_rmid and num_ids unsigned and use a plain min(), match erdt_ioremap() argument types to ioremap(), and reset erdt_max_rmid in erdt_exit(). (Reinette Chatre)
- Run erdt_cpu_valid() once in resctrl_arch_online_cpu(), and match an ERDT domain to a resctrl monitoring domain by domain id rather than by cpumask. Warn on an invalid dom_id and on a duplicate ERDT-to-resctrl domain mapping. (Reinette Chatre)
- Fix the silent truncation of the 64-bit CMRC up_scale: reject an out-of-range value with a FW_BUG warning and fail closed instead of using max_t(int, ...). (Reinette Chatre, Tony Luck)
- Keep the original resctrl_arch_rmid_read() parameter order (resource first) and drop the unused closid/arch_priv arguments. (Reinette Chatre)
- Keep a single erdt_cpu_has() declaration in asm/resctrl.h, required by the inline resctrl_arch_round_mon_val(). (Reinette Chatre)
- Use switch() in erdt_support(), clamp max_rmid with min_t() so MSR-based MBM events are not handed the ERDT RMID, and divide the ERDT scale by snc_nodes_per_l3_cache under SNC. (Reinette Chatre)

Changes from V5 to V6:
- Reorder the series so that the x86/topology change, which touches a different subsystem, comes first. (Reinette Chatre)
- Drop the v5 "x86/resctrl: Replace 'msr' in monitoring data identifiers" patch. None of the renamed identifiers are used by the MMIO code. (Reinette Chatre)
- Split the domain setup into erdt_cpu_valid() and erdt_l3_mon_domain_setup(). Validation now happens in domain_add_cpu_mon() before the CPU is added to the domain cpumask, and attaching ERDT data to a freshly created domain (Reinette Chatre)
- Drop the SNC special case and the new resctrl_disable_mon_event(). (Reinette Chatre)
- resctrl_arch_round_mon_val() now rounds to the ERDT scale instead of returning the value unchanged (Reinette Chatre)
- Make erdt_get_max_rmid() a global value instead of a per-CPU lookup. (Reinette Chatre)
- Use struct cpumask instead of cpumask_var_t, list_for_each_entry() instead of open coded container_of(), and document every member of struct erdt_domain_info. (Reinette Chatre)

Thanks Tony, Reinette, Thomas, Hongyu for your time to look at this patch set.

Changes from V4 to V5:
There are some major changes since v4:
- (biggest change) Eliminate the xarray for runtime lookups; embed struct erdt_domain_info directly in rdt_hw_l3_mon_domain and assign during l3_mon_domain_setup().
- Separate CPUID and ACPI enumeration cleanly. Do not use CPUID feature flags to gate MMIO-based monitoring. Use ACPI table presence (e.g., CMRC table) to determine event enablement.
- Use ACPI RMDD's own "Max RMID" field for MMIO access instead of relying on CPUID's max RMID (which applies to MSR).
- Enforce the SNC constraint in code rather than burying it behind a comment WARN. Disable mon_capable in rdt_get_l3_mon_config() when ERDT is enabled and snc_nodes_per_l3_cache > 1.
- Split non-resctrl changes (topology.c, apic.h) into a separate preparatory patch prefixed with x86/topology.
- Move the "depends on X86" to "X86_64" adjustment to a separate patch with explicit justification in its changelog.

Changes from V3 to V4:
- Remove the redundant table length check in subtbl_valid() (Thomas Gleixner)
- Reuse subtbl_valid() for all the table iteration (Thomas Gleixner)
- Refine the commit log of [PATCH 5/6] to state that this change is a preparation for [PATCH 6/6] rather than fixing an existing issue (Thomas Gleixner, Reinette Chatre, Tony Luck)
- Fix if CACD lists all CPUs in the LLC domain (sashiko)
- Deal with a corner case that if there is no valid RMDD tables, the erdt_enabled_flag should remain false.(sashiko)
- Add Thomas's Reviewed-by and Hongyu's Tested-by.

Changes from V2 to V3:
- Wrap __resctrl_arch_late_init() to avoid the goto logic. (Thomas Gleixner)
- Make the variables in struct erdt_domain_info tabular format (Thomas Gleixner)
- Remove tail comments (Thomas Gleixner)
- Make the name of erdt_enabled() and variable in it consistent and comprehensible. (Thomas Gleixner)
- Use topo_lookup_cpuid() to search the CPU id according to the x2apic id (Thomas Gleixner)
- Fix kernel doc comment format (Thomas Gleixner)
- Use brackets for multiple lines "if" case. (Thomas Gleixner)
- Let the parameter for cacd_init() to fully utilize 100 characters. (Thomas Gleixner)
- Variables are reordered in reverse fir-tree.(Thomas Gleixner)
- Added a named constant and use it in the rmdd->flags check. (Thomas Gleixner)
- Introduce helper functions to make the code readable when iterating the RMDD tables. (Thomas Gleixner)
- Make the macros tabular format. (Thomas Gleixner)

Changes from V1 to V2:
- Add #include <linux/cleanup.h> to follow the "include-what-you-use" best practice (Tony Luck)
- Fix 3 issues reported by:
https://sashiko.dev/#/patchset/cover.1779872016.git.yu.c.chen%40intel.com
Remove the variable of cacd in struct erdt_domain_info as it will never be used after initialization.
Invoke erdt_exit() to avoid resource leak if rdt_alloc_capable and rdt_mon_capable are both false.
Adjust the comments suggested by sashiko.

Chen Yu (8):
x86/topology: Export topo_lookup_cpuid() for resctrl use x86/resctrl: Require 64-bit x86 for resctrl support x86/resctrl: Parse ACPI ERDT table and save CACD cpumask for RMDD
domains
x86/resctrl: Attach ACPI ERDT information to L3 mon domain on CPU
online
x86/resctrl: Parse ACPI CMRC table
x86/resctrl: Refactor the monitor read function x86/resctrl: Introduce erdt_cpu_has() and erdt_support() x86/resctrl: Add MMIO-based LLC occupancy monitoring support

Tony Luck (1):
fs/resctrl: Do not invoke smp_processor_id() in preemptible context

 arch/x86/Kconfig                       |   4 +-
 arch/x86/include/asm/apic.h            |   1 +
 arch/x86/include/asm/resctrl.h         |  11 +-
 arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
 arch/x86/kernel/cpu/resctrl/core.c     |  61 ++-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 518 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  50 ++-
 arch/x86/kernel/cpu/resctrl/monitor.c  |  38 +-
 arch/x86/kernel/cpu/topology.c         |   2 +-
 fs/resctrl/monitor.c                   |  44 ++-
 10 files changed, 702 insertions(+), 28 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c

2.25.1

Enhanced RDT (ERDT) enumerates its monitoring and control resources through ACPI tables that identify each CPU by its x2APIC ID. To associate this firmware-provided data with the kernel's per-CPU structures, ERDT must translate those x2APIC IDs into logical CPU numbers.

topo_lookup_cpuid() already performs this translation, but it is static to topology.c and therefore not reachable from resctrl.

Export topo_lookup_cpuid() and add its declaration to asm/apic.h that already exposes the inverse mapping via cpuid_to_apicid[], so that ERDT can resolve x2APIC IDs from ACPI tables to logical CPU numbers.

No functional change intended.

---
v6->v7:
Add motivation for choosing asm/apic.h, which already exposes the inverse mapping via cpuid_to_apicid[]. (Reinette Chatre)

---
 arch/x86/include/asm/apic.h    | 1 +
 arch/x86/kernel/cpu/topology.c | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

index 9cd493d467d4..bb84651b14bd 100644
@@ -54,6 +54,7 @@ static inline void x86_32_probe_apic(void) { }
 #endif
 
 extern u32 cpuid_to_apicid[];
+int topo_lookup_cpuid(u32 apic_id);
 
 #define CPU_ACPIID_INVALID	U32_MAX
 

index 4913b64ec592..bcee70fb9277 100644
@@ -92,7 +92,7 @@ static inline u32 topo_apicid(u32 apicid, enum x86_topology_domains dom)
 	return apicid & (UINT_MAX << x86_topo_system.dom_shifts[dom - 1]);
 }
 
-static int topo_lookup_cpuid(u32 apic_id)
+int topo_lookup_cpuid(u32 apic_id)
 {
 	int i;
 

2.25.1

All known hardware that supports Intel RDT or AMD QoS is 64-bit. MMIO-based Enhanced RDT (ERDT) monitoring requires readq()/writeq(), which are only available on 64-bit.

32-bit x86 is maintained for legacy reasons. New CPU features are generally not enabled for 32-bit systems, its test coverage keeps shrinking, and modern 32-bit distributions have become rare. Discouraging the use of a 32-bit kernel is discussed at:
https://lore.kernel.org/all/[email protected]/

Drop 32-bit support by changing the X86_CPU_RESCTRL dependency from X86 to X86_64.

Since X86_CPU_RESCTRL now implies X86_64, remove the redundant X86_64 dependency from X86_CPU_RESCTRL_INTEL_AET.

---
v6->v7:
Replace the inaccurate "no realistic server use case" rationale with the legacy, shrinking test coverage and rare distro reasoning. (Reinette Chatre, Tony Luck, Dave Hansen, Borislav Petkov, Peter Zijlstra)

---
 arch/x86/Kconfig | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index bdad90f210e4..6685e120cff3 100644
@@ -514,7 +514,7 @@ config X86_MPPARSE
 
 config X86_CPU_RESCTRL
 	bool "x86 CPU resource control support"
-	depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD)
+	depends on X86_64 && (CPU_SUP_INTEL || CPU_SUP_AMD)
 	depends on MISC_FILESYSTEMS
 	select ARCH_HAS_CPU_RESCTRL
 	select RESCTRL_FS

@@ -537,7 +537,7 @@ config X86_CPU_RESCTRL
 
 config X86_CPU_RESCTRL_INTEL_AET
 	bool "Intel Application Energy Telemetry"
-	depends on X86_64 && X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y
+	depends on X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y
 	help
 	  Enable per-RMID telemetry events in resctrl.
 

2.25.1

There is one Enhanced RDT (ERDT) ACPI table per platform. Each Resource Management Domain Description (RMDD) sub-table within it describes one resource management domain (RMD), also known as an L3 domain, and carries MMIO base information for monitoring support. The CPU agents within the scope of an RMDD are enumerated by their x2APIC IDs in a nested CPU Agent Collection Description (CACD) sub-table.

Parse the RMDD sub-tables within the ERDT ACPI table and their nested CACD entries to construct per-domain CPU masks.

For each RMDD, parse the associated CACD, map its x2APIC IDs to logical CPUs, and save the resulting CPU mask. Associate every ERDT domain with the CPUs that belong to it to prepare for attaching ERDT data to resctrl monitoring domains.

Based on original work from Anil S Keshavamurthy.

---
v6->v7:
Let get_rdt_resources() clean up its own state on failure and drop the asymmetric __resctrl_arch_late_init() wrapper. (Reinette Chatre) Make erdt_max_rmid and num_ids unsigned, add an explanatory comment and use a plain min(). (Reinette Chatre)
Reset erdt_max_rmid in erdt_exit(). (Reinette Chatre) Match erdt_ioremap() argument types to ioremap() by using resource_size_t. (Reinette Chatre)
Initialise rmdd at declaration and check its length via header.length. (Reinette Chatre)
Use a for() loop instead of while() for the table parsing, for consistency. (Reinette Chatre)
Use sizeof(*erdt) instead of sizeof(struct ...) and align pr_warn() to the open parenthesis. (Reinette Chatre)

---
 arch/x86/kernel/cpu/resctrl/Makefile   |   1 +
 arch/x86/kernel/cpu/resctrl/core.c     |  16 +-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 266 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  31 +++
 4 files changed, 312 insertions(+), 2 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c

index 273ddfa30836..2216ee084832 100644
@@ -2,6 +2,7 @@
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= core.o rdtgroup.o monitor.o
 obj-$(CONFIG_X86_CPU_RESCTRL)		+= ctrlmondata.o
 obj-$(CONFIG_X86_CPU_RESCTRL_INTEL_AET)	+= intel_aet.o
+obj-$(CONFIG_X86_CPU_RESCTRL)		+= erdt.o
 obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK)	+= pseudo_lock.o
 
 # To allow define_trace.h's recursive include:

index 9c01d2562b7a..30a0bc1c8d8d 100644
@@ -1013,10 +1013,17 @@ static __init void check_quirks(void)
 
 static __init bool get_rdt_resources(void)
 {
+	bool succeed;
+
+	erdt_init();
 	rdt_alloc_capable = get_rdt_alloc_resources();
 	rdt_mon_capable = get_rdt_mon_resources();
 
-	return (rdt_mon_capable || rdt_alloc_capable);
+	succeed = (rdt_mon_capable || rdt_alloc_capable);
+	if (!succeed)
+		erdt_exit();
+
+	return succeed;
 }
 
 static __init void rdt_init_res_defs_intel(void)

@@ -1138,12 +1145,15 @@ static int __init resctrl_arch_late_init(void)
 				  "x86/resctrl/cat:online:",
 				  resctrl_arch_online_cpu,
 				  resctrl_arch_offline_cpu);
-	if (state < 0)
+	if (state < 0) {
+		erdt_exit();
 		return state;
+	}
 
 	ret = resctrl_init();
 	if (ret) {
 		cpuhp_remove_state(state);
+		erdt_exit();
 		return ret;
 	}
 	rdt_online = state;

@@ -1166,6 +1176,8 @@ static void __exit resctrl_arch_exit(void)
 	cpuhp_remove_state(rdt_online);
 
 	resctrl_exit();
+
+	erdt_exit();
 }
 
 __exitcall(resctrl_arch_exit);

index e3cfa0c10e92..d5b05b5e1517 100644
@@ -21,6 +21,33 @@
 
 #define RMID_VAL_UNAVAIL		BIT_ULL(62)
 
+/*
+ * Index into erdt_domain_info::base[] for each MMIO region.
+ * @ERDT_MMIO_RMDD_CREG: RMDD control register base address
+ */
+enum erdt_mmio_type {
+	ERDT_MMIO_RMDD_CREG,
+	ERDT_MMIO_LAST = ERDT_MMIO_RMDD_CREG
+};
+
+#define ERDT_MMIO_NUM_TYPES	(ERDT_MMIO_LAST + 1)
+
+/**
+ * struct erdt_domain_info - Per-domain ERDT information
+ * @base:	Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_*
+ * @cpu_mask:	CPUs belonging to this resource management domain
+ * @max_rmid:	Maximum RMID supported by this domain
+ * @dom_id:	L3 cache ID shared by all CPUs in this domain (-1 if unset)
+ * @entry:	Links into the global domain_info_list
+ */
+struct erdt_domain_info {
+	void __iomem		*base[ERDT_MMIO_NUM_TYPES];
+	struct cpumask		cpu_mask;
+	u32			max_rmid;
+	int			dom_id;
+	struct list_head	entry;
+};
+
 /*
  * With the above fields in use 62 bits remain in MSR_IA32_QM_CTR for
  * data to be returned. The counter width is discovered from the hardware

@@ -253,4 +280,8 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
 static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
 #endif
 
+unsigned int erdt_get_max_rmid(void);
+int erdt_init(void);
+void erdt_exit(void);
+
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */

2.25.1

Reading LLC occupancy counters via MMIO requires the per-domain ERDT information, parsed from the ACPI ERDT table, to be reachable from the resctrl L3 monitoring domain. Nothing links the two yet, so the monitoring code cannot locate the MMIO registers of a domain.

ERDT and CPUID enumerate CPU-to-L3-domain membership independently: CPUID leaf 4 describes the L3 cache topology, while the firmware CACD sub-table lists the CPUs of each ERDT domain. Both views must agree on a CPU's L3 domain for that CPU to be monitored safely.

When a CPU comes online, validate that firmware and CPUID agree on its L3 domain before adding it to any resctrl domain. Exclude the CPU from all resctrl domains on a mismatch because a topology inconsistency between ERDT and CPUID indicates a firmware defect that makes the CPU's domain placement unreliable for any resource. Otherwise attach the matching ERDT domain information to the L3 monitoring domain so that monitoring data can be read via ERDT and its sub-tables.

---
v6->v7:
Run erdt_cpu_valid() once in resctrl_arch_online_cpu(). (Reinette Chatre)
Call erdt_l3_mon_domain_setup() directly from l3_mon_domain_setup(). (Reinette Chatre)
Match an ERDT domain to a resctrl domain by domain id instead of by cpumask. (Reinette Chatre)
Return false and emit a FW_BUG warning from erdt_cpu_valid() when dom_id < 0. (Reinette Chatre)
Warn on a duplicate ERDT-to-resctrl domain mapping. (Reinette Chatre) Document that domain_list_lock also protects domain_info_list (Reinette Chatre)

---
 arch/x86/kernel/cpu/resctrl/core.c     |  17 ++++
 arch/x86/kernel/cpu/resctrl/erdt.c     | 105 +++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |   5 ++
 3 files changed, 127 insertions(+)

index 30a0bc1c8d8d..51185a8770bf 100644
@@ -34,6 +34,9 @@
  * the domain list must either take cpus_read_lock(), or rely on an RCU
  * read-side critical section, to avoid observing concurrent modification.
  * All writers take this mutex:
+ *
+ * This mutex also protects the ERDT domain_info_list, which is modified when a
+ * CPU comes online.
  */
 static DEFINE_MUTEX(domain_list_lock);
 

@@ -563,7 +566,10 @@ static void l3_mon_domain_setup(int cpu, int id, struct rdt_resource *r, struct
 		list_del_rcu(&d->hdr.list);
 		synchronize_rcu();
 		l3_mon_domain_free(hw_dom);
+		return;
 	}
+
+	erdt_l3_mon_domain_setup(id, &d->hdr);
 }
 
 static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)

@@ -739,6 +745,17 @@ static int resctrl_arch_online_cpu(unsigned int cpu)
 	struct rdt_resource *r;
 
 	mutex_lock(&domain_list_lock);
+	/*
+	 * A CPU whose ERDT and CPUID L3 domain views disagree is not added to
+	 * any domain. resctrl_arch_offline_cpu() still tries to remove it when
+	 * it goes offline and warns that no domain contains it. That warning is
+	 * expected.
+	 */
+	if (!erdt_cpu_valid(cpu)) {
+		mutex_unlock(&domain_list_lock);
+		return 0;
+	}
+
 	for_each_capable_rdt_resource(r)
 		domain_add_cpu(cpu, r);
 	mutex_unlock(&domain_list_lock);

index d5b05b5e1517..dac1d7e71c0d 100644
@@ -99,14 +99,19 @@ struct rdt_hw_ctrl_domain {
  * @arch_mbm_states:	Per-event pointer to the MBM event's saved state.
  *			An MBM event's state is an array of struct arch_mbm_state
  *			indexed by RMID on x86.
+ * @d_info:		ERDT table information of this domain
  *
  * Members of this structure are accessed via helpers that provide abstraction.
  */
 struct rdt_hw_l3_mon_domain {
 	struct rdt_l3_mon_domain	d_resctrl;
 	struct arch_mbm_state		*arch_mbm_states[QOS_NUM_L3_MBM_EVENTS];
+	const struct erdt_domain_info	*d_info;
 };
 
+bool erdt_cpu_valid(int cpu);
+void erdt_l3_mon_domain_setup(int id, struct rdt_domain_hdr *hdr);
+
 static inline struct rdt_hw_ctrl_domain *resctrl_to_arch_ctrl_dom(struct rdt_ctrl_domain *r)
 {
 	return container_of(r, struct rdt_hw_ctrl_domain, d_resctrl);

2.25.1

The CMRC (Cache Monitoring Registers for CPU Agents Description) sub-table of ERDT describes the MMIO registers used to read cache monitoring counters (e.g. LLC occupancy) for a monitoring domain.

Parse each CMRC sub-table, ioremap its register window, and save a copy of the CMRC table in the corresponding ERDT domain entry so that monitoring code can read the counters via MMIO.

---
v6->v7:
Make erdt_scale and erdt_max_rmid unsigned int. (Reinette Chatre) Reject an out-of-range 64-bit up_scale with a FW_BUG warning and fail closed instead of using max_t(int, ...). (Reinette Chatre, Tony Luck) Make cmrc_init() failure fail closed. (Reinette Chatre)

---
 arch/x86/include/asm/resctrl.h         |  2 +
 arch/x86/kernel/cpu/resctrl/erdt.c     | 73 ++++++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/internal.h |  6 ++-
 3 files changed, 80 insertions(+), 1 deletion(-)

index 575f8408a9e7..4225a518fada 100644
@@ -49,6 +49,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);
 DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
 DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);
 
+unsigned int erdt_get_scale(void);
+
 static inline bool resctrl_arch_alloc_capable(void)
 {
 	return rdt_alloc_capable;

index dac1d7e71c0d..160174c25792 100644
@@ -24,10 +24,12 @@
 /*
  * Index into erdt_domain_info::base[] for each MMIO region.
  * @ERDT_MMIO_RMDD_CREG: RMDD control register base address
+ * @ERDT_MMIO_CMRC_BASE: CMRC monitoring register base address
  */
 enum erdt_mmio_type {
 	ERDT_MMIO_RMDD_CREG,
-	ERDT_MMIO_LAST = ERDT_MMIO_RMDD_CREG
+	ERDT_MMIO_CMRC_BASE,
+	ERDT_MMIO_LAST = ERDT_MMIO_CMRC_BASE
 };
 
 #define ERDT_MMIO_NUM_TYPES	(ERDT_MMIO_LAST + 1)

@@ -35,6 +37,7 @@ enum erdt_mmio_type {
 /**
  * struct erdt_domain_info - Per-domain ERDT information
  * @base:	Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_*
+ * @cmrc:	Copy of the ACPI CMRC sub-table for this domain
  * @cpu_mask:	CPUs belonging to this resource management domain
  * @max_rmid:	Maximum RMID supported by this domain
  * @dom_id:	L3 cache ID shared by all CPUs in this domain (-1 if unset)

@@ -42,6 +45,7 @@ enum erdt_mmio_type {
  */
 struct erdt_domain_info {
 	void __iomem		*base[ERDT_MMIO_NUM_TYPES];
+	struct acpi_erdt_cmrc	*cmrc;
 	struct cpumask		cpu_mask;
 	u32			max_rmid;
 	int			dom_id;

2.25.1

Split the monitor read helper into an L3 read path and an AET (Intel Application Energy Telemetry) read path. This makes the two distinct monitoring sources easier to extend independently and prepares the L3 path for ERDT-based MMIO reads.

No functional change.

---
v6->v7:
Keep the original resctrl_arch_rmid_read() parameter order with the resource first, and drop the unused closid/arch_priv arguments. (Reinette Chatre)

---
 arch/x86/kernel/cpu/resctrl/monitor.c | 26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)

index 569894d6e5c8..8057f8403e56 100644
@@ -239,9 +239,8 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d
 	return chunks * hw_res->mon_scale;
 }
 
-int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
-			   u32 unused, u32 rmid, enum resctrl_event_id eventid,
-			   void *arch_priv, u64 *val, void *ignored)
+static int arch_l3_read_event(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
+			      u32 rmid, enum resctrl_event_id eventid, u64 *val)
 {
 	struct rdt_hw_l3_mon_domain *hw_dom;
 	struct rdt_l3_mon_domain *d;

@@ -251,11 +250,6 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	int cpu;
 	int ret;
 
-	resctrl_arch_rmid_read_context_check();
-
-	if (r->rid == RDT_RESOURCE_PERF_PKG)
-		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
-
 	if (!domain_header_is_valid(hdr, RESCTRL_MON_DOMAIN, RDT_RESOURCE_L3))
 		return -EINVAL;
 

@@ -281,6 +275,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 	return ret;
 }
 
+int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
+			   u32 unused, u32 rmid, enum resctrl_event_id eventid,
+			   void *arch_priv, u64 *val, void *ignored)
+{
+	resctrl_arch_rmid_read_context_check();
+
+	switch (r->rid) {
+	case RDT_RESOURCE_L3:
+		return arch_l3_read_event(r, hdr, rmid, eventid, val);
+	case RDT_RESOURCE_PERF_PKG:
+		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);
+	default:
+		return -EINVAL;
+	}
+}
+
 static int __cntr_id_read(u32 cntr_id, u64 *val)
 {
 	u64 msr_val;

2.25.1

From: Tony Luck <[email protected]>

An event flagged with mon_evt::any_cpu is read directly in preemptible task context instead of being dispatched to a CPU of the monitoring domain. Only Intel AET events set this flag, so __mon_event_count() rejects an any_cpu L3 event with WARN_ON_ONCE(). The L3 read paths can therefore call smp_processor_id() unconditionally to check that the current CPU belongs to the domain being read.

MMIO-based LLC occupancy counters can be read from any CPU, so the L3 read paths have to tolerate an any_cpu event. Move that check into a helper which returns early for any_cpu events, avoiding smp_processor_id() in preemptible context, and report a wrong CPU via rmid_read::err.

---
v6->v7:
Describe the current code instead of framing the changelog as a bug fix. (Reinette Chatre)
Drop the now-merged lore.kernel.org link. (Reinette Chatre)

---
 fs/resctrl/monitor.c | 44 ++++++++++++++++++++++++++++++++++----------
 1 file changed, 34 insertions(+), 10 deletions(-)

index a932a1fea818..c08d2f90d721 100644
@@ -422,9 +422,37 @@ static void mbm_cntr_free(struct rdt_l3_mon_domain *d, int cntr_id)
 	memset(&d->cntr_cfg[cntr_id], 0, sizeof(*d->cntr_cfg));
 }
 
+/**
+ * cpu_on_correct_domain() - Check if current CPU is in the correct
+ *			      domain for the event.
+ * @rr: The rmid_read structure containing event and domain information.
+ *
+ * Context: Preemptible process context when @rr->evt->any_cpu is set.
+ *          Non-migratable process context (via smp_call_on_cpu()) or
+ *          non-preemptible context (via smp_call_function_any()) when
+ *          the event must be read on a specific CPU.
+ * Return: true if the current CPU can read this event, false otherwise.
+ */
+static bool cpu_on_correct_domain(struct rmid_read *rr)
+{
+	int cpu;
+
+	/* Any CPU is OK for this event */
+	if (rr->evt->any_cpu)
+		return true;
+
+	cpu = smp_processor_id();
+
+	/* Single domain. Must be on a CPU in that domain. */
+	if (rr->hdr)
+		return cpumask_test_cpu(cpu, &rr->hdr->cpu_mask);
+
+	/* Summing domains that share a cache, must be on a CPU for that cache. */
+	return cpumask_test_cpu(cpu, &rr->ci->shared_cpu_map);
+}
+
 static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
-	int cpu = smp_processor_id();
 	u32 closid = rdtgrp->closid;
 	u32 rmid = rdtgrp->mon.rmid;
 	struct rdt_l3_mon_domain *d;

@@ -457,9 +485,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 		return 0;
 	}
 
-	/* Reading a single domain, must be on a CPU in that domain. */
-	if (!cpumask_test_cpu(cpu, &d->hdr.cpu_mask))
-		return -EINVAL;
 	if (rr->is_mbm_cntr)
 		rr->err = resctrl_arch_cntr_read(rr->r, d, closid, rmid, cntr_id,
 						 rr->evt->evtid, &tval);

@@ -477,7 +502,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 
 static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
-	int cpu = smp_processor_id();
 	u32 closid = rdtgrp->closid;
 	u32 rmid = rdtgrp->mon.rmid;
 	struct rdt_l3_mon_domain *d;

@@ -495,10 +519,6 @@ static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *r
 		return -EINVAL;
 	}
 
-	/* Summing domains that share a cache, must be on a CPU for that cache. */
-	if (!cpumask_test_cpu(cpu, &rr->ci->shared_cpu_map))
-		return -EINVAL;
-
 	/*
 	 * Legacy files must report the sum of an event across all
 	 * domains that share the same L3 cache instance.

@@ -529,7 +549,11 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
 	switch (rr->r->rid) {
 	case RDT_RESOURCE_L3:
-		WARN_ON_ONCE(rr->evt->any_cpu);
+		if (!cpu_on_correct_domain(rr)) {
+			rr->err = -EIO;
+			return -EINVAL;
+		}
+
 		if (rr->hdr)
 			return __l3_mon_event_count(rdtgrp, rr);
 		else

2.25.1

Introduce erdt_cpu_has() to verify if a specific RDT feature is backed by an ERDT table. erdt_cpu_has() is derived from rdt_cpu_has(), which not only considers firmware (ERDT table and its sub-tables) support for a feature, but also considers userspace input like "rdt=!cmt". Similar to rdt_cpu_has(), erdt_cpu_has() expects an x86 feature flag as parameter.

Introduce erdt_support() as a helper called by erdt_cpu_has() to check whether the ERDT firmware supports a given feature.

erdt_support() deliberately claims no feature yet and returns false. A feature may only be reported as ERDT backed once the low level accessors that read its monitoring data via MMIO exist, otherwise callers would take the ERDT path and find no implementation behind it. Features should be added to erdt_support() as their MMIO implementations are introduced.

---
v6->v7:
Reword "an event" to "a feature", since the check is not monitoring-only. (Reinette Chatre)
Reword "are added" to "should be added". (Reinette Chatre) Remove the duplicate erdt_cpu_has() declaration, keeping only the one in asm/resctrl.h required by the inline resctrl_arch_round_mon_val(). (Reinette Chatre)

---
 arch/x86/include/asm/resctrl.h         |  1 +
 arch/x86/kernel/cpu/resctrl/core.c     | 23 +++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/erdt.c     |  5 +++++
 arch/x86/kernel/cpu/resctrl/internal.h |  1 +
 4 files changed, 30 insertions(+)

index 4225a518fada..a8dfaca5c6f8 100644
@@ -49,6 +49,7 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);
 DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
 DECLARE_STATIC_KEY_FALSE(rdt_mon_enable_key);
 
+bool erdt_cpu_has(int flag);
 unsigned int erdt_get_scale(void);
 
 static inline bool resctrl_arch_alloc_capable(void)

index 51185a8770bf..2db9c71bf1f0 100644
@@ -895,6 +895,29 @@ bool rdt_cpu_has(int flag)
 	return ret;
 }
 
+bool erdt_cpu_has(int flag)
+{
+	struct rdt_options *o;
+	bool ret;
+
+	ret = erdt_support(flag);
+
+	if (!ret)
+		return ret;
+
+	for (o = rdt_options; o < &rdt_options[NUM_RDT_OPTIONS]; o++) {
+		if (flag == o->flag) {
+			if (o->force_off)
+				ret = false;
+			if (o->force_on)
+				ret = true;
+			break;
+		}
+	}
+
+	return ret;
+}
+
 bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt)
 {
 	if (!rdt_cpu_has(X86_FEATURE_BMEC))

index 1c477e35d044..e6eba4067348 100644
@@ -48,6 +48,11 @@ static unsigned int erdt_max_rmid;
  */
 static unsigned int erdt_scale;
 
+bool erdt_support(int flag)
+{
+	return false;
+}
+
 unsigned int erdt_get_max_rmid(void)
 {
 	return erdt_max_rmid;

index 160174c25792..bb7a902b5b9d 100644
@@ -289,6 +289,7 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resour
 static inline bool intel_handle_aet_option(bool force_off, char *tok) { return false; }
 #endif
 
+bool erdt_support(int flag);
 unsigned int erdt_get_max_rmid(void);
 int erdt_init(void);
 void erdt_exit(void);

2.25.1

Add erdt_mon_read() to read LLC occupancy via MMIO and use it when the platform supports ERDT. Register the L3 occupancy event with ERDT enabled when available, falling back to the MSR-based path otherwise.

Use the CMRC (Cache Monitoring Registers for CPU Agents Description) ACPI sub-table to read LLC occupancy counters for each RMID via MMIO when ERDT is enabled. Store the CMRC information in the rdt_hw_l3_mon_domain, which could be accessed directly.

Although the occupancy counters can now be read from any CPU via MMIO, the per-domain limbo handler is kept. A global handler would still have to iterate over every domain and would only save one worker thread, which does not justify maintaining two limbo handler models.

---
v6->v7:
Use switch() instead of an if-chain in erdt_support(). (Reinette Chatre) Clamp max_rmid with min_t() so MSR-based MBM events are not handed the ERDT RMID. (Reinette Chatre)
Divide the ERDT scale by snc_nodes_per_l3_cache under SNC. (Reinette Chatre)

---
 arch/x86/include/asm/resctrl.h         |  8 ++-
 arch/x86/kernel/cpu/resctrl/core.c     |  5 +-
 arch/x86/kernel/cpu/resctrl/erdt.c     | 73 +++++++++++++++++++++++++-
 arch/x86/kernel/cpu/resctrl/internal.h |  9 +++-
 arch/x86/kernel/cpu/resctrl/monitor.c  | 12 ++++-
 5 files changed, 100 insertions(+), 7 deletions(-)

index a8dfaca5c6f8..14d69d1bb200 100644
@@ -132,7 +132,13 @@ static inline void __resctrl_sched_in(struct task_struct *tsk)
 
 static inline unsigned int resctrl_arch_round_mon_val(unsigned int val)
 {
-	unsigned int scale = boot_cpu_data.x86_cache_occ_scale;
+	unsigned int scale = boot_cpu_data.x86_cache_occ_scale, escale;
+
+	if (erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
+		escale = erdt_get_scale();
+		if (escale)
+			scale = escale;
+	}
 
 	/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */
 	val /= scale;

index 2db9c71bf1f0..d6ea005b0858 100644
@@ -1005,7 +1005,10 @@ static __init bool get_rdt_mon_resources(void)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
 	bool ret = false;
 
-	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
+	if (erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
+		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, true, 0, NULL);
+		ret = true;
+	} else if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
 		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false, 0, NULL);
 		ret = true;
 	}

index bb7a902b5b9d..df1707105eb9 100644
@@ -158,7 +158,11 @@ struct msr_param {
  *			which has been corrected for features like CDP.
  * @msr_base:		Base MSR address for CBMs
  * @msr_update:		Function pointer to update QOS MSRs
- * @mon_scale:		cqm counter * mon_scale = occupancy in bytes
+ * @mon_scale:		Scale factor applied to a raw counter value on the
+ *			MSR-based read path: CMT occupancy counter * mon_scale =
+ *			occupancy in bytes, and MBM chunk count * mon_scale = bytes
+ *			transferred. ERDT reads occupancy via MMIO and applies its
+ *			own firmware-provided scale instead.
  * @mbm_width:		Monitor width, to detect and correct for overflow.
  * @cdp_enabled:	CDP state of this resource
  * @mbm_cntr_assign_enabled:	ABMC feature is enabled

@@ -187,6 +191,8 @@ static inline struct rdt_hw_resource *resctrl_to_arch_res(struct rdt_resource *r
 
 extern struct rdt_hw_resource rdt_resources_all[];
 
+extern int snc_nodes_per_l3_cache;
+
 void arch_mon_domain_online(struct rdt_resource *r, struct rdt_l3_mon_domain *d);
 
 /* CPUID.(EAX=10H, ECX=ResID=1).EAX */

@@ -291,6 +297,7 @@ static inline bool intel_handle_aet_option(bool force_off, char *tok) { return f
 
 bool erdt_support(int flag);
 unsigned int erdt_get_max_rmid(void);
+int erdt_mon_read(struct rdt_domain_hdr *hdr, enum resctrl_event_id evtid, u32 rmid, u64 *val);
 int erdt_init(void);
 void erdt_exit(void);
 

index 8057f8403e56..ac822e305afa 100644
@@ -34,7 +34,7 @@ bool rdt_mon_capable;
 
 #define CF(cf)	((unsigned long)(1048576 * (cf) + 0.5))
 
-static int snc_nodes_per_l3_cache = 1;
+int snc_nodes_per_l3_cache = 1;
 
 /*
  * The correction factor table is documented in Documentation/filesystems/resctrl.rst.

@@ -283,6 +283,10 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,
 
 	switch (r->rid) {
 	case RDT_RESOURCE_L3:
+		if (eventid == QOS_L3_OCCUP_EVENT_ID &&
+		    erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC))
+			return erdt_mon_read(hdr, eventid, rmid, val);
+
 		return arch_l3_read_event(r, hdr, rmid, eventid, val);
 	case RDT_RESOURCE_PERF_PKG:
 		return intel_aet_read_event(hdr->id, rmid, arch_priv, val);

@@ -429,12 +433,16 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	unsigned int threshold;
 	u32 eax, ebx, ecx, edx;
+	int max_rmid;
 
 	snc_nodes_per_l3_cache = snc_get_config();
 
+	max_rmid = erdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC) ?
+			min_t(int, erdt_get_max_rmid(), boot_cpu_data.x86_cache_max_rmid) :
+			boot_cpu_data.x86_cache_max_rmid;
 	resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size * 1024;
 	hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache;
-	r->mon.num_rmid = (boot_cpu_data.x86_cache_max_rmid + 1) / snc_nodes_per_l3_cache;
+	r->mon.num_rmid = (max_rmid + 1) / snc_nodes_per_l3_cache;
 	hw_res->mbm_width = MBM_CNTR_WIDTH_BASE;
 
 	if (mbm_offset > 0 && mbm_offset <= MBM_CNTR_WIDTH_OFFSET_MAX)

2.25.1

Hi Chenyu,

What is this series based on? It does not apply on top of resctrl changes that have been queued for more than a month before you sent this version.

You can find latest staged resctrl changes on branch x86/cache on
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git

Could you please rebase this series on top of x86/cache and resubmit?

The baseline may need to change to master branch of tip.git when considering this series also touches other x86 code but for now it can start with x86/cache since it does not change as much as master.

Thank you

Reinette

ps Looks like Sashiko found a working base of x86/core of tip.git which was/is
at v7.2-rc5 and without any of the staged resctrl changes that have since been merged into v7.3-rc1.

Hi Reinette,

This patch set was based on v7.2. Let me rebase this on top of x86/cache. Thanks for the notice.

thanks,
Chenyu