Introduce region-aware RDT support
Subsystems (4)
cover Introduce region-aware RDT support
Hi everyone,
This is a PoC patch set posted to support region-aware RDT on Intel platform. Its purpose is to demonstrate what region-aware RDT is and to seek for suggestion from the community on how Linux could support this feature properly. It also allows people to have an early try with this preliminary version if they would like to explore it in advance. This version is not intended for inclusion for upstream in its current shape.
This patch set is based on the generic schemata PoC patch set, which introduced multiple controllers per rdt_resource. The multi-controller support is the essential framework for region-aware RDT, as each memory region needs its own set of controls on the same resource. The generic schemata PoC is at:
git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git
branch resctrl/controls_rfc_v1
Since many details are still under discussion above, current patch set is evolving as well, and will be adjusted as the discussion and the decisions move forward.
What is region-aware RDT
========================
Intel hardware supports Region-Aware Memory Bandwidth Allocation (MBA) and Region-Aware Memory Bandwidth Monitoring (MBM). With Region-Aware MBA, independent bandwidth control (throttling) of L3 domain bandwidth to multiple regions is supported, enabling users to dynamically rebalance bandwidth control limits across different memory regions, each of which may have distinct bandwidth, latency, and capacity characteristics. Region-Aware MBM includes the capability to independently track multiple domains that are simultaneously accessing several memory regions. These memory regions correspond to different levels of memory tiers, such as directly attached memory (Tier 1), CXL-attached memory (Tier 2), and CXL accelerator devices with attached memory. Note, the region ID is per socket scope. Intel platform supports up to 4 regions for now.
Suppose there are 2 regions in each socket:
+------------------------+------------------------+ | +--------+ +--------+ | +--------+ +--------+ | | | L3_00 | | L3_01 | | | L3_02 | | L3_03 | | | +--------+ +--------+ | +--------+ +--------+ | | +-------------------+ | +-------------------+ | | | IMC1 | | | IMC2 | | | +-------------------+ | +-------------------+ | | +-------+ +-------+ | +-------+ +-------+ | | | DDR1 | | CXL1 | | | DDR2 | | CXL2 | | | +-------+ +-------+ | +-------+ +-------+ | | socket0 | socket1 | +------------------------+------------------------+
In above graph, from the perspective of a CPU associated with the L3_00 domain, Region 0 typically represents the local memory region (DDR1), while Region 2 typically represents the remote memory region (DDR2). Similarly, for CPUs attached to L3_00, Region 1 represents the local memory region (CXL1), and Region 3 represents the remote memory region (CXL2). Here the terms "local" and "remote" are defined at the socket level.
Take the region aware MBM for example. For the L3_00 domain, the memory bandwidth of Region 0 refers to the data transferred when the L3 miss occurs in L3_00 and the data is refilled from DDR1 - note, data refilled from L3_01 to L3_00 is not counted in. The bandwidth of Region 2, by contrast, refers to the data refilled from DDR2. Similarly, the same calculation logic applies to Region 1 (CXL1) and Region 3 (CXL2).
For Region-Aware MBA, when setting values for Region 0 on CPUs attached to L3_00, this configuration controls the traffic generated when data is transferred between cores and L3_00 targeting DDR1.
The resctrl interface
=====================
Region-aware MBM adds one file per region to each mon_data domain directory. For example, with 4 regions the following files are created:
mbm_region0_bytes, mbm_region1_bytes, mbm_region2_bytes and mbm_region3_bytes
For region-aware MBA, every region has its own set of controls, and each of them appears as a separate schemata line named "MB_REGION<n>_<type>", where <n> is the region number and <type> is one of OPT, MIN or MAX. Only the control types that the hardware reports in the MARC ACPI sub-table are present. The legacy "MB" control is emulated by the MAX control of every region, so writing "MB" throttles all regions at once, while writing an individual "MB_REGION<n>_MAX" line throttles only that region.
Below is an example on a platform with 2 memory regions, adjusting the tier1 local memory bandwidth (usually the DDR) of the second domain:
# cat schemata
MB:0=100;1=100;2=100;3=100
MB_REGION0_OPT:0=511;1=511;2=511;3=511
MB_REGION0_MIN:0=511;1=511;2=511;3=511
MB_REGION0_MAX:0=511;1=511;2=511;3=511
MB_REGION1_OPT:0=511;1=511;2=511;3=511
MB_REGION1_MIN:0=511;1=511;2=511;3=511
MB_REGION1_MAX:0=511;1=511;2=511;3=511
L3:0=3ff;1=3ff;2=3ff;3=3ff # echo "MB_REGION0_MAX:1=200" > schemata
# cat schemata
MB:0=100;1=100;2=100;3=100
MB_REGION0_OPT:0=511;1=511;2=511;3=511
MB_REGION0_MIN:0=511;1=511;2=511;3=511
MB_REGION0_MAX:0=511;1=200;2=511;3=511
MB_REGION1_OPT:0=511;1=511;2=511;3=511
MB_REGION1_MIN:0=511;1=511;2=511;3=511
MB_REGION1_MAX:0=511;1=511;2=511;3=511
L3:0=3ff;1=3ff;2=3ff;3=3ffHow this patch set is composed
==============================
Patches 1 to 9, from "x86/topology: Export topo_lookup_cpuid() for resctrl use" to "x86/resctrl: Add MMIO-based LLC occupancy monitoring support", add the MMIO-based CMT support, which has already been posted at:
https://lore.kernel.org/all/[email protected]/
They are included here because they have been slightly modified to apply on top of Reinette's generic schemata PoC.
Patches 10 to 14, from Revert "x86/resctrl: NOT_FOR_INCLUSION: Example support for multiple controls" to "x86/resctrl: Add emulation controller list to resctrl_ctrl", prepare for region-aware RDT.
Patches 15 to 19, from "x86/resctrl: Parse ACPI MMRC table" to "x86/resctrl: Enable the region based events by adding them into the event", add the region-aware MBM support.
Patches 20 to 30, from "x86/resctrl: Rename msr_update to hw_update" to the "fs/resctrl: Fix excessive padding in schemata output", add the region-aware MBA support.
Patch 31 is the documentation for region-aware RDT.
Any comments would be much appreciated.
thanks,
Chenyu
Anil S Keshavamurthy (1):
x86/resctrl: Parse ACPI ERDT table and save CACD cpumask for RMDD
domains
Chen Yu (29):
x86/topology: Export topo_lookup_cpuid() for resctrl use x86/resctrl: Require 64-bit x86 for resctrl support x86/resctrl: Attach ACPI ERDT information to L3 mon domain on CPU
online
x86/resctrl: Parse ACPI CMRC table
x86/resctrl: Refactor the monitor read function x86/resctrl: Introduce erdt_cpu_has() and erdt_support() x86/resctrl: Add MMIO-based LLC occupancy monitoring support Revert "x86/resctrl: NOT_FOR_INCLUSION: Example support for multiple
controls"
x86/resctrl: Rename struct resctrl_membw to struct resctrl_ctrl_scalar x86/resctrl: Rename struct resctrl_cache to struct resctrl_ctrl_bitmap x86/resctrl: Add per-control and per-resource flags x86/resctrl: Add emulation controller list to resctrl_ctrl x86/resctrl: Parse ACPI MMRC table
x86/resctrl: Replace "msr" in monitoring data identifiers x86/resctrl: Introduce region aware MBM event definitions x86/resctrl: Introduce memory region based MBM read callback on MMIO
space
x86/resctrl: Enable the region based events by adding them into the
event
x86/resctrl: Rename msr_update to hw_update
x86/resctrl: Parse ACPI MARC table
fs/resctrl: Add region-based control names and
resctrl_ctrl_name_region()
x86/resctrl: Add region aware MBA controllers
x86/resctrl: Attach ACPI ERDT information to ctrl domain on CPU online x86/resctrl: Introduce region-based MBA write implementation on MMIO
space
x86/resctrl: Allow control writes from any CPU for MMIO controllers x86/resctrl: Enable region-aware MBM/MBA via the RDT_CTRL register x86/resctrl: Emulate the legacy MBA controller via the region MAX
controls
fs/resctrl: Expose emulation controllers in a resource_schemata subdir fs/resctrl: Fix excessive padding in schemata output x86,fs/resctrl: Update Documentation for region aware RDT
Tony Luck (1):
fs/resctrl: Do not invoke smp_processor_id() in preemptible context
Documentation/filesystems/resctrl.rst | 110 +++ arch/x86/Kconfig | 4 +- arch/x86/include/asm/apic.h | 1 + arch/x86/include/asm/resctrl.h | 11 +- arch/x86/kernel/cpu/resctrl/Makefile | 1 + arch/x86/kernel/cpu/resctrl/core.c | 258 +++---- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 106 ++- arch/x86/kernel/cpu/resctrl/erdt.c | 886 ++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 95 ++- arch/x86/kernel/cpu/resctrl/monitor.c | 82 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 14 +- arch/x86/kernel/cpu/topology.c | 2 +- drivers/resctrl/mpam_resctrl.c | 18 +- fs/resctrl/ctrlmondata.c | 49 +- fs/resctrl/monitor.c | 73 +- fs/resctrl/pseudo_lock.c | 2 +- fs/resctrl/rdtgroup.c | 111 ++- include/linux/resctrl.h | 82 +- include/linux/resctrl_types.h | 21 +- 19 files changed, 1634 insertions(+), 292 deletions(-) create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c
Signature · 1 line
2.43.0
RFC PATCH 1/31 x86/topology: Export topo_lookup_cpuid() for resctrl use
Enhanced RDT (ERDT) enumerates its monitoring and control resources through ACPI tables that identify each CPU by its x2APIC ID. To associate this firmware-provided data with the kernel's per-CPU structures, ERDT must translate those x2APIC IDs into logical CPU numbers.
topo_lookup_cpuid() already performs this translation, but it is static to topology.c and therefore not reachable from resctrl.
Export topo_lookup_cpuid() and add its declaration to asm/apic.h so that ERDT can resolve x2APIC IDs from ACPI tables to logical CPU numbers.
No functional change intended.
--- arch/x86/include/asm/apic.h | 1 + arch/x86/kernel/cpu/topology.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
arch/x86/include/asm/apic.h +1 -0
@@ -54,6 +54,7 @@ static inline void x86_32_probe_apic(void) { }#endif extern u32 cpuid_to_apicid[]; +int topo_lookup_cpuid (u32 apic_id);#define CPU_ACPIID_INVALID U32_MAX
arch/x86/kernel/cpu/topology.c +1 -1
@@ -92,7 +92,7 @@ static inline u32 topo_apicid(u32 apicid, enum x86_topology_domains dom)return apicid & (UINT_MAX << x86_topo_system.dom_shifts[dom -1 ]); } -static int topo_lookup_cpuid (u32 apic_id) +int topo_lookup_cpuid (u32 apic_id) {int i;
Signature · 1 line
2.43.0
RFC PATCH 2/31 x86/resctrl: Require 64-bit x86 for resctrl support
In reply to Chen YuAll known hardware that supports Intel RDT or AMD QoS is 64-bit. Future enhancements like MMIO-based Enhanced RDT (ERDT) monitoring require readq()/writeq() which are only available on 64-bit. There is also no realistic use case for users to build a 32-bit kernel on a server and enable resctrl.
Drop 32-bit support by changing the X86_CPU_RESCTRL dependency from X86 to X86_64.
Since X86_CPU_RESCTRL now implies X86_64, remove the redundant X86_64 dependency from X86_CPU_RESCTRL_INTEL_AET.
--- arch/x86/Kconfig | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/Kconfig +2 -2
@@ -515,7 +515,7 @@ config X86_MPPARSE config X86_CPU_RESCTRL bool "x86 CPU resource control support" - depends on X86 && (CPU_SUP_INTEL || CPU_SUP_AMD) + depends on X86_64 && (CPU_SUP_INTEL || CPU_SUP_AMD) depends on MISC_FILESYSTEMS select ARCH_HAS_CPU_RESCTRL select RESCTRL_FS
@@ -538,7 +538,7 @@ config X86_CPU_RESCTRL config X86_CPU_RESCTRL_INTEL_AET bool "Intel Application Energy Telemetry" - depends on X86_64 && X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y + depends on X86_CPU_RESCTRL && CPU_SUP_INTEL && INTEL_PMT_TELEMETRY=y && INTEL_TPMI=y help Enable per-RMID telemetry events in resctrl.
Signature · 1 line
2.43.0
RFC PATCH 3/31 x86/resctrl: Parse ACPI ERDT table and save CACD cpumask for RMDD domains
In reply to Chen YuFrom: Anil S Keshavamurthy <[email protected]>
Parse the RMDD subtables within the ERDT ACPI table and their nested CACD entries to construct per-domain CPU masks.
There is one ERDT table per platform. Each RMDD describes one resource management domain (RMD), also known as an L3 domain, and carries MMIO base information for later monitoring support.
For each RMDD, parse the associated CACD, map its x2APIC IDs to logical CPUs, and save the resulting CPU mask. This mask associates each ERDT domain with the CPUs that belong to it and is used later when attaching ERDT data to resctrl monitoring domains.
--- arch/x86/kernel/cpu/resctrl/Makefile | 1 + arch/x86/kernel/cpu/resctrl/core.c | 14 +- arch/x86/kernel/cpu/resctrl/erdt.c | 266 +++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 31 +++ 4 files changed, 311 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kernel/cpu/resctrl/erdt.c
arch/x86/kernel/cpu/resctrl/Makefile +1 -0
@@ -2,6 +2,7 @@ obj-$(CONFIG_X86_CPU_RESCTRL) += core.o rdtgroup.o monitor.o obj-$(CONFIG_X86_CPU_RESCTRL) += ctrlmondata.o obj-$(CONFIG_X86_CPU_RESCTRL_INTEL_AET) += intel_aet.o +obj-$(CONFIG_X86_CPU_RESCTRL) += erdt.o obj-$(CONFIG_RESCTRL_FS_PSEUDO_LOCK) += pseudo_lock.o # To allow define_trace.h's recursive include:
arch/x86/kernel/cpu/resctrl/core.c +13 -1
@@ -1147,6 +1147,7 @@ static __init void check_quirks(void)static __init bool get_rdt_resources (void ) { +erdt_init (); rdt_alloc_capable =get_rdt_alloc_resources (); rdt_mon_capable =get_rdt_mon_resources ();
@@ -1232,7 +1233,7 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c) } } -static int __init resctrl_arch_late_init (void ) +static int __init __resctrl_arch_late_init (void ) {struct rdt_resource *r;int state, ret, i;
@@ -1275,6 +1276,15 @@ static int __init resctrl_arch_late_init(void)return 0 ; } +static int __init resctrl_arch_late_init (void ) +{ +int ret =__resctrl_arch_late_init (); + +if (ret) +erdt_exit (); +return ret; +} +late_initcall (resctrl_arch_late_init); static void __exit resctrl_arch_exit(void)
@@ -1288,6 +1298,8 @@ static void __exit resctrl_arch_exit(void)* removed here. */ resctrl_exit (); + +erdt_exit (); }__exitcall (resctrl_arch_exit);
arch/x86/kernel/cpu/resctrl/erdt.c new file +266 -0
266 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +31 -0
@@ -21,6 +21,33 @@#define RMID_VAL_UNAVAIL BIT_ULL(62) +/* +* Index into erdt_domain_info::base[] for each MMIO region. +* @ERDT_MMIO_RMDD_CREG: RMDD control register base address +*/ +enum erdt_mmio_type { +ERDT_MMIO_RMDD_CREG , +ERDT_MMIO_LAST =ERDT_MMIO_RMDD_CREG +}; + +#define ERDT_MMIO_NUM_TYPES (ERDT_MMIO_LAST + 1) + +/** +* struct erdt_domain_info - Per-domain ERDT information +* @base: Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_* type +* @cpu_mask: CPUs belonging to this resource management domain +* @max_rmid: Maximum RMID supported by this domain +* @dom_id: L3 cache ID shared by all CPUs in this domain (-1 if unset) +* @entry: Links into the global domain_info_list +*/ +struct erdt_domain_info { +void __iomem *base[ERDT_MMIO_NUM_TYPES ]; +struct cpumask cpu_mask; +u32 max_rmid; +int dom_id; +struct list_head entry; +}; +/* * With the above fields in use 62 bits remain in MSR_IA32_QM_CTR for * data to be returned. The counter width is discovered from the hardware
@@ -269,4 +296,8 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resourstatic inline bool intel_handle_aet_option (bool force_off,char *tok) {return false; }#endif +int erdt_get_max_rmid (void ); +int erdt_init (void ); +void erdt_exit (void ); +#endif /* _ASM_X86_RESCTRL_INTERNAL_H */
Signature · 1 line
2.43.0
RFC PATCH 4/31 x86/resctrl: Attach ACPI ERDT information to L3 mon domain on CPU online
In reply to Chen YuReading LLC occupancy counters via MMIO requires the per-domain ERDT information, parsed earlier from the ACPI ERDT table, to be reachable from the resctrl L3 monitoring domain. Nothing links the two yet, so the monitoring code cannot locate the MMIO registers of a domain.
ERDT and CPUID enumerate CPU-to-L3-domain membership independently:
CPUID leaf 4 describes the L3 cache topology, while the firmware CACD sub-table lists the CPUs of each ERDT domain. Both views must agree on a CPU's L3 domain for that CPU to be monitored safely.
When a CPU comes online, validate that firmware and CPUID agree on its L3 domain before adding it to a resctrl monitoring domain. Exclude the CPU from all monitoring domains on a mismatch because a topology inconsistency between ERDT and CPUID indicates a firmware defect that makes the CPU's domain placement unreliable for any resource. Otherwise attach the matching ERDT domain information to the L3 monitoring domain so that later code can read monitoring data via ERDT and its sub-tables.
--- arch/x86/kernel/cpu/resctrl/core.c | 11 ++++- arch/x86/kernel/cpu/resctrl/erdt.c | 66 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ++ 3 files changed, 81 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/core.c +10 -1
@@ -705,6 +705,9 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)return ; } +if (!erdt_cpu_valid (cpu)) +return ; + hdr =resctrl_find_domain (&r->mon_domains, id, &add_pos);if (hdr)cpumask_set_cpu (cpu, &hdr->cpu_mask);
@@ -714,8 +717,14 @@ static void domain_add_cpu_mon(int cpu, struct rdt_resource *r)/* Update the mbm_assign_mode state for the CPU if supported */ if (r->mon.mbm_cntr_assignable)resctrl_arch_mbm_cntr_assign_set_one (r); -if (!hdr) +if (!hdr) {l3_mon_domain_setup (cpu, id, r, add_pos); + hdr =resctrl_find_domain (&r->mon_domains, id,NULL ); + } + +if (hdr) +erdt_l3_mon_domain_setup (cpu, hdr); +break ;case RDT_RESOURCE_PERF_PKG :if (!hdr)
arch/x86/kernel/cpu/resctrl/erdt.c +66 -0
72 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +5 -0
@@ -132,14 +132,19 @@ struct rdt_hw_ctrl_domain {* @arch_mbm_states: Per-event pointer to the MBM event's saved state. * An MBM event's state is an array of struct arch_mbm_state * indexed by RMID on x86. +* @d_info: ERDT table information of this domain * * Members of this structure are accessed via helpers that provide abstraction. */ struct rdt_hw_l3_mon_domain {struct rdt_l3_mon_domain d_resctrl;struct arch_mbm_state *arch_mbm_states[QOS_NUM_L3_MBM_EVENTS ]; +const struct erdt_domain_info *d_info; }; +bool erdt_cpu_valid (int cpu); +void erdt_l3_mon_domain_setup (int cpu,struct rdt_domain_hdr *hdr); +static inline struct rdt_hw_ctrl_domain *resctrl_to_arch_ctrl_dom (struct rdt_ctrl_domain *r) { return container_of(r, struct rdt_hw_ctrl_domain, d_resctrl);
Signature · 1 line
2.43.0
RFC PATCH 5/31 x86/resctrl: Parse ACPI CMRC table
In reply to Chen YuThe CMRC (Cache Monitoring Registers for CPU Agents Description) sub-table of ERDT describes the MMIO registers used to read cache monitoring counters (e.g. LLC occupancy) for an RMD.
Parse each CMRC sub-table, ioremap its register window, and save a copy of the CMRC table in the corresponding ERDT domain entry so that later monitoring code can read the counters via MMIO.
--- arch/x86/include/asm/resctrl.h | 2 + arch/x86/kernel/cpu/resctrl/erdt.c | 57 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 6 ++- 3 files changed, 64 insertions(+), 1 deletion(-)
arch/x86/include/asm/resctrl.h +2 -0
@@ -49,6 +49,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +int erdt_get_scale (void ); +static inline bool resctrl_arch_alloc_capable (void ) {return rdt_alloc_capable;
arch/x86/kernel/cpu/resctrl/erdt.c +57 -0
92 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +5 -1
@@ -24,10 +24,12 @@/* * Index into erdt_domain_info::base[] for each MMIO region. * @ERDT_MMIO_RMDD_CREG: RMDD control register base address +* @ERDT_MMIO_CMRC_BASE: CMRC monitoring register base address */ enum erdt_mmio_type {ERDT_MMIO_RMDD_CREG , -ERDT_MMIO_LAST =ERDT_MMIO_RMDD_CREG +ERDT_MMIO_CMRC_BASE , +ERDT_MMIO_LAST =ERDT_MMIO_CMRC_BASE };#define ERDT_MMIO_NUM_TYPES (ERDT_MMIO_LAST + 1)
@@ -35,6 +37,7 @@ enum erdt_mmio_type {/** * struct erdt_domain_info - Per-domain ERDT information * @base: Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_* type +* @cmrc: Copy of the ACPI CMRC sub-table for this domain * @cpu_mask: CPUs belonging to this resource management domain * @max_rmid: Maximum RMID supported by this domain * @dom_id: L3 cache ID shared by all CPUs in this domain (-1 if unset)
@@ -42,6 +45,7 @@ enum erdt_mmio_type {*/ struct erdt_domain_info {void __iomem *base[ERDT_MMIO_NUM_TYPES ]; +struct acpi_erdt_cmrc *cmrc;struct cpumask cpu_mask;u32 max_rmid;int dom_id;
Signature · 1 line
2.43.0
RFC PATCH 6/31 x86/resctrl: Refactor the monitor read function
In reply to Chen YuSplit the monitor read helper into an L3 read path and an AET (Intel Application Energy Telemetry) read path. This makes the two distinct monitoring sources easier to extend independently and prepares the L3 path for ERDT-based MMIO reads.
No functional change.
--- arch/x86/kernel/cpu/resctrl/monitor.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/resctrl/monitor.c +19 -8
@@ -238,9 +238,9 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *dreturn chunks * hw_res->mon_scale; } -int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain_hdr *hdr, -u32 unused,u32 rmid,enum resctrl_event_id eventid, -void *arch_priv,u64 *val,void *ignored) +static int arch_l3_read_event (struct rdt_domain_hdr *hdr,u32 rmid, +enum resctrl_event_id eventid,u64 *val, +struct rdt_resource *r) {struct rdt_hw_l3_mon_domain *hw_dom;struct rdt_l3_mon_domain *d;
@@ -250,11 +250,6 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,int cpu;int ret; -resctrl_arch_rmid_read_context_check (); - -if (r->rid ==RDT_RESOURCE_PERF_PKG ) -return intel_aet_read_event (hdr->id, rmid, arch_priv, val); -if (!domain_header_is_valid (hdr,RESCTRL_MON_DOMAIN ,RDT_RESOURCE_L3 ))return -EINVAL ;
@@ -275,6 +270,22 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,return ret; } +int resctrl_arch_rmid_read (struct rdt_resource *r,struct rdt_domain_hdr *hdr, +u32 unused,u32 rmid,enum resctrl_event_id eventid, +void *arch_priv,u64 *val,void *ignored) +{ +resctrl_arch_rmid_read_context_check (); + +switch (r->rid) { +case RDT_RESOURCE_L3 : +return arch_l3_read_event (hdr, rmid, eventid, val, r); +case RDT_RESOURCE_PERF_PKG : +return intel_aet_read_event (hdr->id, rmid, arch_priv, val); +default : +return -EINVAL ; + } +} +static int __cntr_id_read (u32 cntr_id,u64 *val) {u64 msr_val;
Signature · 1 line
2.43.0
RFC PATCH 7/31 fs/resctrl: Do not invoke smp_processor_id() in preemptible context
In reply to Chen YuFrom: Tony Luck <[email protected]>
LLC occupancy can be read on any CPU when the counter is accessed via MMIO, so such an event is read from task context on whatever CPU the caller happens to be running on rather than being bounced to a CPU in the monitoring domain. mon_evt::any_cpu marks these CPU-agnostic events.
__l3_mon_event_count() calls smp_processor_id() to find the CPU to read from. For an any_cpu event that lookup is unsafe:
the code runs in preemptible task context, so smp_processor_id() emits a debug warning.
Skip the current-CPU lookup when an event's any_cpu flag is set, events with this flag do not require execution on a specific CPU. For legacy MSR-based access, update rmid_read::err if the reading of the event was dispatched to a wrong CPU, according to the change at:
https://lore.kernel.org/lkml/6b3c66a49788828bd8c04a6911bd74c91ccd56f3.1782857711.git.reinette.chatre@intel.com/
--- fs/resctrl/monitor.c | 48 +++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 14 deletions(-)
fs/resctrl/monitor.c +34 -14
@@ -417,9 +417,37 @@ static void mbm_cntr_free(struct rdt_l3_mon_domain *d, int cntr_id)memset (&d->cntr_cfg[cntr_id],0 ,sizeof (*d->cntr_cfg)); } +/** +* cpu_on_correct_domain() - Check if current CPU is in the correct +* domain for the event. +* @rr: The rmid_read structure containing event and domain information. +* +* Context: Preemptible process context when @rr->evt->any_cpu is set. +* Non-migratable process context (via smp_call_on_cpu()) or +* non-preemptible context (via smp_call_function_any()) when +* the event must be read on a specific CPU. +* Return: true if the current CPU can read this event, false otherwise. +*/ +static bool cpu_on_correct_domain (struct rmid_read *rr) +{ +int cpu; + +/* Any CPU is OK for this event */ +if (rr->evt->any_cpu) +return true; + + cpu =smp_processor_id (); + +/* Single domain. Must be on a CPU in that domain. */ +if (rr->hdr) +return cpumask_test_cpu (cpu, &rr->hdr->cpu_mask); + +/* Summing domains that share a cache, must be on a CPU for that cache. */ +return cpumask_test_cpu (cpu, &rr->ci->shared_cpu_map); +} +static int __l3_mon_event_count (struct rdtgroup *rdtgrp,struct rmid_read *rr) { -int cpu =smp_processor_id ();u32 closid = rdtgrp->closid;u32 rmid = rdtgrp->mon.rmid;struct rdt_l3_mon_domain *d;
@@ -452,11 +480,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)return 0 ; } -/* Reading a single domain, must be on a CPU in that domain. */ -if (!cpumask_test_cpu (cpu, &d->hdr.cpu_mask)) { - rr->err = -EIO ; -return -EINVAL ; - }if (rr->is_mbm_cntr) rr->err =resctrl_arch_cntr_read (rr->r, d, closid, rmid, cntr_id, rr->evt->evtid, &tval);
@@ -474,7 +497,6 @@ static int __l3_mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)static int __l3_mon_event_count_sum (struct rdtgroup *rdtgrp,struct rmid_read *rr) { -int cpu =smp_processor_id ();u32 closid = rdtgrp->closid;u32 rmid = rdtgrp->mon.rmid;struct rdt_l3_mon_domain *d;
@@ -492,12 +514,6 @@ static int __l3_mon_event_count_sum(struct rdtgroup *rdtgrp, struct rmid_read *rreturn -EINVAL ; } -/* Summing domains that share a cache, must be on a CPU for that cache. */ -if (!cpumask_test_cpu (cpu, &rr->ci->shared_cpu_map)) { - rr->err = -EIO ; -return -EINVAL ; - } -/* * Legacy files must report the sum of an event across all * domains that share the same L3 cache instance.
@@ -528,7 +544,11 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr) {switch (rr->r->rid) {case RDT_RESOURCE_L3 : -WARN_ON_ONCE (rr->evt->any_cpu); +if (!cpu_on_correct_domain (rr)) { + rr->err = -EIO ; +return -EINVAL ; + } +if (rr->hdr)return __l3_mon_event_count (rdtgrp, rr);else
Signature · 1 line
2.43.0
RFC PATCH 8/31 x86/resctrl: Introduce erdt_cpu_has() and erdt_support()
In reply to Chen YuIntroduce erdt_cpu_has() to verify if a specific RDT feature is backed by an ERDT table. erdt_cpu_has() is derived from rdt_cpu_has(), which not only considers firmware (ERDT table and its sub-tables) support for an event, but also considers userspace input like "rdt=!cmt". Similar to rdt_cpu_has(), erdt_cpu_has() expects an x86 feature flag as parameter.
Introduce erdt_support() as a helper called by erdt_cpu_has() to check whether the ERDT firmware supports a given feature.
erdt_support() deliberately claims no feature yet and returns false. A feature may only be reported as ERDT backed once the low level accessors that read its monitoring data via MMIO exist, otherwise callers would take the ERDT path and find no implementation behind it. Features are added to erdt_support() as their MMIO implementations are introduced.
--- arch/x86/include/asm/resctrl.h | 1 + arch/x86/kernel/cpu/resctrl/core.c | 23 +++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/erdt.c | 5 +++++ arch/x86/kernel/cpu/resctrl/internal.h | 2 ++ 4 files changed, 31 insertions(+)
arch/x86/include/asm/resctrl.h +1 -0
@@ -49,6 +49,7 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +bool erdt_cpu_has (int flag);int erdt_get_scale (void );static inline bool resctrl_arch_alloc_capable (void )
arch/x86/kernel/cpu/resctrl/core.c +23 -0
@@ -1021,6 +1021,29 @@ bool rdt_cpu_has(int flag)return ret; } +bool erdt_cpu_has (int flag) +{ +struct rdt_options *o; +bool ret; + + ret =erdt_support (flag); + +if (!ret) +return ret; + +for (o = rdt_options; o < &rdt_options[NUM_RDT_OPTIONS ]; o++) { +if (flag == o->flag) { +if (o->force_off) + ret = false; +if (o->force_on) + ret = true; +break ; + } + } + +return ret; +} +bool resctrl_arch_is_evt_configurable (enum resctrl_event_id evt) {if (!rdt_cpu_has (X86_FEATURE_BMEC ))
arch/x86/kernel/cpu/resctrl/erdt.c +5 -0
@@ -37,6 +37,11 @@ static int erdt_max_rmid;/* Scale to bytes for the monitoring counters when ERDT is enabled. */ static int erdt_scale; +bool erdt_support (int flag) +{ +return false; +} +int erdt_get_max_rmid (void ) {return erdt_max_rmid;
arch/x86/kernel/cpu/resctrl/internal.h +2 -0
@@ -305,6 +305,8 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resourstatic inline bool intel_handle_aet_option (bool force_off,char *tok) {return false; }#endif +bool erdt_support (int flag); +bool erdt_cpu_has (int flag);int erdt_get_max_rmid (void );int erdt_init (void );void erdt_exit (void );
Signature · 1 line
2.43.0
RFC PATCH 9/31 x86/resctrl: Add MMIO-based LLC occupancy monitoring support
In reply to Chen YuAdd erdt_mon_read() to read LLC occupancy via MMIO and use it when the platform supports ERDT. Register the L3 occupancy event with ERDT enabled when available, falling back to the MSR-based path otherwise.
Use the CMRC (Cache Monitoring Registers for CPU Agents Description) ACPI sub-table to read LLC occupancy counters for each RMID via MMIO when ERDT is enabled. This CMRC information is stored in the rdt_hw_l3_mon_domain, which could be accessed directly.
Currently, the per-domain limbo handler is still in use. There is no need to switch to a global limbo handler, because even after such a switch, the worker thread would still have to iterate through all domains one by one. The per-domain handler already accomplishes this using a worker thread rather than costly IPIs, so there is no clear benefit to switching to a global handler.
--- arch/x86/include/asm/resctrl.h | 8 +++- arch/x86/kernel/cpu/resctrl/core.c | 5 ++- arch/x86/kernel/cpu/resctrl/erdt.c | 59 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 7 ++- arch/x86/kernel/cpu/resctrl/monitor.c | 9 +++- 5 files changed, 84 insertions(+), 4 deletions(-)
arch/x86/include/asm/resctrl.h +7 -1
@@ -132,7 +132,13 @@ static inline void __resctrl_sched_in(struct task_struct *tsk)static inline unsigned int resctrl_arch_round_mon_val (unsigned int val) { -unsigned int scale = boot_cpu_data.x86_cache_occ_scale; +unsigned int scale = boot_cpu_data.x86_cache_occ_scale, escale; + +if (erdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) { + escale =erdt_get_scale (); +if (escale) + scale = escale; + }/* h/w works in units of "boot_cpu_data.x86_cache_occ_scale" */ val /= scale;
arch/x86/kernel/cpu/resctrl/core.c +4 -1
@@ -1131,7 +1131,10 @@ static __init bool get_rdt_mon_resources(void)struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3 ].r_resctrl;bool ret = false; -if (rdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) { +if (erdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) { +resctrl_enable_mon_event (QOS_L3_OCCUP_EVENT_ID , true,0 ,NULL ); + ret = true; + }else if (rdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) {resctrl_enable_mon_event (QOS_L3_OCCUP_EVENT_ID , false,0 ,NULL ); ret = true; }
arch/x86/kernel/cpu/resctrl/erdt.c +59 -0
77 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +6 -1
@@ -175,7 +175,11 @@ struct rdt_perf_pkg_mon_domain {* resctrl_arch_get_num_closid() to avoid confusion * with struct rdt_resource_final's property of the same * name, which has been corrected for features like CDP. -* @mon_scale: cqm counter * mon_scale = occupancy in bytes +* @mon_scale: Scale factor applied to a raw counter value on the +* MSR-based read path: CMT occupancy counter * mon_scale = +* occupancy in bytes, and MBM chunk count * mon_scale = bytes +* transferred. ERDT reads occupancy via MMIO and applies its +* own firmware-provided scale instead. * @mbm_width: Monitor width, to detect and correct for overflow. * @cdp_enabled: CDP state of this resource * @has_per_cpu_cache_cfg: True if QOS_CFG register for this cache resource
@@ -308,6 +312,7 @@ static inline bool intel_handle_aet_option(bool force_off, char *tok) { return fbool erdt_support (int flag);bool erdt_cpu_has (int flag);int erdt_get_max_rmid (void ); +int erdt_mon_read (struct rdt_domain_hdr *hdr,enum resctrl_event_id evtid,u32 rmid,u64 *val);int erdt_init (void );void erdt_exit (void );
arch/x86/kernel/cpu/resctrl/monitor.c +8 -1
@@ -278,6 +278,10 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,switch (r->rid) {case RDT_RESOURCE_L3 : +if (eventid ==QOS_L3_OCCUP_EVENT_ID && +erdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) +return erdt_mon_read (hdr, eventid, rmid, val); +return arch_l3_read_event (hdr, rmid, eventid, val, r);case RDT_RESOURCE_PERF_PKG :return intel_aet_read_event (hdr->id, rmid, arch_priv, val);
@@ -419,12 +423,15 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);unsigned int threshold;u32 eax, ebx, ecx, edx; +int max_rmid; snc_nodes_per_l3_cache =snc_get_config (); + max_rmid =erdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC ) ? +erdt_get_max_rmid () : boot_cpu_data.x86_cache_max_rmid; resctrl_rmid_realloc_limit = boot_cpu_data.x86_cache_size *1024 ; hw_res->mon_scale = boot_cpu_data.x86_cache_occ_scale / snc_nodes_per_l3_cache; - r->mon.num_rmid = (boot_cpu_data.x86_cache_max_rmid +1 ) / snc_nodes_per_l3_cache; + r->mon.num_rmid = (max_rmid +1 ) / snc_nodes_per_l3_cache; hw_res->mbm_width =MBM_CNTR_WIDTH_BASE ;if (mbm_offset >0 && mbm_offset <=MBM_CNTR_WIDTH_OFFSET_MAX )
Signature · 1 line
2.43.0
RFC PATCH 10/31 Revert "x86/resctrl: NOT_FOR_INCLUSION: Example support for multiple controls"
In reply to Chen YuThis reverts commit 37557cac804539e802e887a320e85c7a916a0bb9.
--- arch/x86/kernel/cpu/resctrl/core.c | 115 ++++++----------------------- 1 file changed, 21 insertions(+), 94 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +21 -94
@@ -53,8 +53,6 @@ bool rdt_alloc_capable;static void mba_wrmsr_intel (struct msr_param *m);static void cat_wrmsr (struct msr_param *m);static void mba_wrmsr_amd (struct msr_param *m); -static void update_temporary_max (struct msr_param *m); -static void update_temporary_min (struct msr_param *m);#define ctrl_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.controls)#define mon_domain_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.mon_domains)
@@ -182,47 +180,10 @@ static inline void cache_alloc_hsw_probe(void) rdt_alloc_capable = true; } -static __init bool __temporary_multiple_mba_intel_controls (struct rdt_resource *r, -struct resctrl_hw_ctrl *hw_ctrl, -u32 ecx,u32 max_delay, -enum resctrl_ctrl_name name) -{ - hw_ctrl->r_ctrl.scope =RESCTRL_L3_CACHE ; - hw_ctrl->r_ctrl.type =RESCTRL_CTRL_SCALAR ; - hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ; -INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); - - hw_ctrl->r_ctrl.membw.max_bw =MAX_MBA_BW ; - hw_ctrl->r_ctrl.membw.min_bw =MAX_MBA_BW - max_delay; - hw_ctrl->r_ctrl.membw.bw_gran =MAX_MBA_BW - max_delay; - - hw_ctrl->r_ctrl.membw.resolution =100 ; - hw_ctrl->r_ctrl.membw.tolerance =5 ; - hw_ctrl->r_ctrl.membw.scale =1 ; - hw_ctrl->r_ctrl.membw.unit =RESCTRL_CTRL_UNIT_ALL ; - - hw_ctrl->r_ctrl.name = name; - -switch (name) { -case RESCTRL_CTRL_NAME_DEF : - hw_ctrl->msr_base =MSR_IA32_MBA_THRTL_BASE ; - hw_ctrl->msr_update = mba_wrmsr_intel; -break ; -case RESCTRL_CTRL_NAME_MIN : - hw_ctrl->msr_update = update_temporary_min; -break ; -case RESCTRL_CTRL_NAME_MAX : - hw_ctrl->msr_update = update_temporary_max; -break ; - } - -return true; -} -static __init bool __get_mem_config_intel (struct rdt_resource *r) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct resctrl_hw_ctrl *hw_ctrl0, *hw_ctrl1, *hw_ctrl2; +struct resctrl_hw_ctrl *hw_ctrl;union cpuid_0x10_3_eax eax;union cpuid_0x10_x_edx edx;u32 ebx, ecx, max_delay;
@@ -234,50 +195,34 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r)if (!(ecx &MBA_IS_LINEAR ))return false; + hw_ctrl =kzalloc_obj (*hw_ctrl); +if (!hw_ctrl) +return false; + + hw_ctrl->r_ctrl.scope =RESCTRL_L3_CACHE ; + hw_ctrl->r_ctrl.type =RESCTRL_CTRL_SCALAR ; + hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ; +INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); + + hw_ctrl->r_ctrl.membw.max_bw =MAX_MBA_BW ; + hw_ctrl->r_ctrl.membw.min_bw =MAX_MBA_BW - max_delay; + hw_ctrl->r_ctrl.membw.bw_gran =MAX_MBA_BW - max_delay; + r->bw_delay_linear = true;if (boot_cpu_has (X86_FEATURE_PER_THREAD_MBA )) r->bw_throttle_mode =THREAD_THROTTLE_PER_THREAD ;else r->bw_throttle_mode =THREAD_THROTTLE_MAX ; - hw_ctrl0 =kzalloc_obj (*hw_ctrl0); -if (!hw_ctrl0) -return false; - -if (!__temporary_multiple_mba_intel_controls (r, hw_ctrl0, ecx, max_delay,RESCTRL_CTRL_NAME_DEF )) { -kfree (hw_ctrl0); -return false; - } - - hw_ctrl1 =kzalloc_obj (*hw_ctrl1); -if (!hw_ctrl1) { -kfree (hw_ctrl0); -return false; - } - -if (!__temporary_multiple_mba_intel_controls (r, hw_ctrl1, ecx, max_delay,RESCTRL_CTRL_NAME_MIN )) { -kfree (hw_ctrl0); -kfree (hw_ctrl1); -return false; - } - - hw_ctrl2 =kzalloc_obj (*hw_ctrl2); -if (!hw_ctrl2) { -kfree (hw_ctrl0); -kfree (hw_ctrl1); -return false; - } + hw_ctrl->r_ctrl.membw.resolution =100 ; + hw_ctrl->r_ctrl.membw.tolerance =5 ; + hw_ctrl->r_ctrl.membw.scale =1 ; + hw_ctrl->r_ctrl.membw.unit =RESCTRL_CTRL_UNIT_ALL ; -if (!__temporary_multiple_mba_intel_controls (r, hw_ctrl2, ecx, max_delay,RESCTRL_CTRL_NAME_MAX )) { -kfree (hw_ctrl0); -kfree (hw_ctrl1); -kfree (hw_ctrl2); -return false; - } + hw_ctrl->msr_base =MSR_IA32_MBA_THRTL_BASE ; + hw_ctrl->msr_update = mba_wrmsr_intel; +list_add (&hw_ctrl->r_ctrl.entry, &r->controls); -list_add (&hw_ctrl0->r_ctrl.entry, &r->controls); -list_add (&hw_ctrl1->r_ctrl.entry, &r->controls); -list_add (&hw_ctrl2->r_ctrl.entry, &r->controls); r->alloc_capable = true;return true;
@@ -434,24 +379,6 @@ static u32 delay_bw_map(unsigned long bw, struct rdt_resource *r)return MAX_MBA_BW ; } -static void update_temporary_max (struct msr_param *m) -{ -struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom); - -/* Any control properties available via m->ctrl */ -pr_info ("Updata temporary MAX control on domain %d with user value %i\n" , - m->dom->hdr.id, hw_dom->ctrl_val[m->low]); -} - -static void update_temporary_min (struct msr_param *m) -{ -struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom); - -/* Any control properties available via m->ctrl */ -pr_info ("Updata temporary MIN control on domain %d with user value %i\n" , - m->dom->hdr.id, hw_dom->ctrl_val[m->low]); -} -static void mba_wrmsr_intel (struct msr_param *m) {struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom);
Signature · 1 line
2.43.0
RFC PATCH 11/31 x86/resctrl: Rename struct resctrl_membw to struct resctrl_ctrl_scalar
In reply to Chen YuRename struct resctrl_membw to struct resctrl_ctrl_scalar to better reflect its role as the property container for scalar-type controls. Also rename the union member from 'membw' to 'scalar' for consistency.
This follows the design discussed in the generic schema RFC where controls are classified by type (scalar vs bitmap) rather than by the specific resource they manage (memory bandwidth).
--- arch/x86/kernel/cpu/resctrl/core.c | 30 +++++++++++++++--------------- drivers/resctrl/mpam_resctrl.c | 6 +++--- fs/resctrl/ctrlmondata.c | 6 +++--- fs/resctrl/monitor.c | 8 ++++---- fs/resctrl/rdtgroup.c | 20 ++++++++++---------- include/linux/resctrl.h | 16 ++++++++-------- 6 files changed, 43 insertions(+), 43 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +15 -15
@@ -204,9 +204,9 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r) hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); - hw_ctrl->r_ctrl.membw.max_bw =MAX_MBA_BW ; - hw_ctrl->r_ctrl.membw.min_bw =MAX_MBA_BW - max_delay; - hw_ctrl->r_ctrl.membw.bw_gran =MAX_MBA_BW - max_delay; + hw_ctrl->r_ctrl.scalar.max_bw =MAX_MBA_BW ; + hw_ctrl->r_ctrl.scalar.min_bw =MAX_MBA_BW - max_delay; + hw_ctrl->r_ctrl.scalar.bw_gran =MAX_MBA_BW - max_delay; r->bw_delay_linear = true;if (boot_cpu_has (X86_FEATURE_PER_THREAD_MBA ))
@@ -214,10 +214,10 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r)else r->bw_throttle_mode =THREAD_THROTTLE_MAX ; - hw_ctrl->r_ctrl.membw.resolution =100 ; - hw_ctrl->r_ctrl.membw.tolerance =5 ; - hw_ctrl->r_ctrl.membw.scale =1 ; - hw_ctrl->r_ctrl.membw.unit =RESCTRL_CTRL_UNIT_ALL ; + hw_ctrl->r_ctrl.scalar.resolution =100 ; + hw_ctrl->r_ctrl.scalar.tolerance =5 ; + hw_ctrl->r_ctrl.scalar.scale =1 ; + hw_ctrl->r_ctrl.scalar.unit =RESCTRL_CTRL_UNIT_ALL ; hw_ctrl->msr_base =MSR_IA32_MBA_THRTL_BASE ; hw_ctrl->msr_update = mba_wrmsr_intel;
@@ -246,7 +246,7 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)if (!hw_ctrl)return false; -if (BITS_PER_TYPE (hw_ctrl->r_ctrl.membw.max_bw) <= eax) { +if (BITS_PER_TYPE (hw_ctrl->r_ctrl.scalar.max_bw) <= eax) {pr_warn ("Unable to support hardware's maximum bandwidth\n" );kfree (hw_ctrl);return false;
@@ -257,7 +257,7 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r) hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); - hw_ctrl->r_ctrl.membw.max_bw =BIT (eax); + hw_ctrl->r_ctrl.scalar.max_bw =BIT (eax);/* AMD does not use delay */ r->bw_delay_linear = false;
@@ -267,12 +267,12 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)* the allocation like Intel does. */ r->bw_throttle_mode =THREAD_THROTTLE_UNDEFINED ; - hw_ctrl->r_ctrl.membw.min_bw =0 ; - hw_ctrl->r_ctrl.membw.bw_gran =1 ; - hw_ctrl->r_ctrl.membw.resolution =8 ; - hw_ctrl->r_ctrl.membw.tolerance =0 ; - hw_ctrl->r_ctrl.membw.scale =1 ; - hw_ctrl->r_ctrl.membw.unit =RESCTRL_CTRL_UNIT_GBPS ; + hw_ctrl->r_ctrl.scalar.min_bw =0 ; + hw_ctrl->r_ctrl.scalar.bw_gran =1 ; + hw_ctrl->r_ctrl.scalar.resolution =8 ; + hw_ctrl->r_ctrl.scalar.tolerance =0 ; + hw_ctrl->r_ctrl.scalar.scale =1 ; + hw_ctrl->r_ctrl.scalar.unit =RESCTRL_CTRL_UNIT_GBPS ;if (r->rid ==RDT_RESOURCE_MBA ) { hw_ctrl->msr_base =MSR_IA32_MBA_BW_BASE ;
drivers/resctrl/mpam_resctrl.c +3 -3
@@ -1043,9 +1043,9 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) r->bw_delay_linear = true; r->bw_throttle_mode =THREAD_THROTTLE_UNDEFINED ; - mpam_ctrl->r_ctrl.membw.min_bw =get_mba_min (cprops); - mpam_ctrl->r_ctrl.membw.max_bw =MAX_MBA_BW ; - mpam_ctrl->r_ctrl.membw.bw_gran =get_mba_granularity (cprops); + mpam_ctrl->r_ctrl.scalar.min_bw =get_mba_min (cprops); + mpam_ctrl->r_ctrl.scalar.max_bw =MAX_MBA_BW ; + mpam_ctrl->r_ctrl.scalar.bw_gran =get_mba_granularity (cprops);list_add (&mpam_ctrl->r_ctrl.entry, &r->controls); r->name ="MB" ;
fs/resctrl/ctrlmondata.c +3 -3
@@ -86,13 +86,13 @@ static bool bw_validate(char *buf, u32 *data, struct rdt_resource *r,return true; } -if (bw < ctrl->membw.min_bw || bw > ctrl->membw.max_bw) { +if (bw < ctrl->scalar.min_bw || bw > ctrl->scalar.max_bw) {rdt_last_cmd_printf ("MB value %u out of range [%d,%d]\n" , - bw, ctrl->membw.min_bw, ctrl->membw.max_bw); + bw, ctrl->scalar.min_bw, ctrl->scalar.max_bw);return false; } - *data =roundup (bw, (unsigned long )ctrl->membw.bw_gran); + *data =roundup (bw, (unsigned long )ctrl->scalar.bw_gran);return true; }
fs/resctrl/monitor.c +4 -4
@@ -758,11 +758,11 @@ static void update_mba_bw(struct rdtgroup *rgrp, struct rdt_l3_mon_domain *dom_m* 40% would go past the limit by multiplying current bandwidth by * "(30 + 10) / 30". */ -if (cur_msr_val > ctrl->membw.min_bw && user_bw < cur_bw) { - new_msr_val = cur_msr_val - ctrl->membw.bw_gran; +if (cur_msr_val > ctrl->scalar.min_bw && user_bw < cur_bw) { + new_msr_val = cur_msr_val - ctrl->scalar.bw_gran; }else if (cur_msr_val <MAX_MBA_BW && - (user_bw > (cur_bw * (cur_msr_val + ctrl->membw.min_bw) / cur_msr_val))) { - new_msr_val = cur_msr_val + ctrl->membw.bw_gran; + (user_bw > (cur_bw * (cur_msr_val + ctrl->scalar.min_bw) / cur_msr_val))) { + new_msr_val = cur_msr_val + ctrl->scalar.bw_gran; }else {return ; }
fs/resctrl/rdtgroup.c +10 -10
@@ -1179,7 +1179,7 @@ static int rdt_min_bw_show(struct kernfs_open_file *of,if (!ctrl || ctrl->type !=RESCTRL_CTRL_SCALAR )return 0 ; -seq_printf (seq,"%u\n" , ctrl->membw.min_bw); +seq_printf (seq,"%u\n" , ctrl->scalar.min_bw);return 0 ; }
@@ -1222,7 +1222,7 @@ static int rdt_bw_gran_show(struct kernfs_open_file *of,if (!ctrl || ctrl->type !=RESCTRL_CTRL_SCALAR )return 0 ; -seq_printf (seq,"%u\n" , ctrl->membw.bw_gran); +seq_printf (seq,"%u\n" , ctrl->scalar.bw_gran);return 0 ; }
@@ -1622,7 +1622,7 @@ bool is_mba_sc(struct rdt_resource *r, struct resctrl_ctrl *ctrl)return false; } -return ctrl->membw.mba_sc; +return ctrl->scalar.mba_sc; }/*
@@ -2256,7 +2256,7 @@ static int resctrl_ctrl_min_show(struct kernfs_open_file *of, {struct resctrl_ctrl *ctrl =rdt_kn_parent_priv (of->kn); -seq_printf (seq,"%u\n" , ctrl->membw.min_bw); +seq_printf (seq,"%u\n" , ctrl->scalar.min_bw);return 0 ; }
@@ -2266,7 +2266,7 @@ static int resctrl_ctrl_max_show(struct kernfs_open_file *of, {struct resctrl_ctrl *ctrl =rdt_kn_parent_priv (of->kn); -seq_printf (seq,"%u\n" , ctrl->membw.max_bw); +seq_printf (seq,"%u\n" , ctrl->scalar.max_bw);return 0 ; }
@@ -2276,7 +2276,7 @@ static int resctrl_ctrl_resolution_show(struct kernfs_open_file *of, {struct resctrl_ctrl *ctrl =rdt_kn_parent_priv (of->kn); -seq_printf (seq,"%u\n" , ctrl->membw.resolution); +seq_printf (seq,"%u\n" , ctrl->scalar.resolution);return 0 ; }
@@ -2286,7 +2286,7 @@ static int resctrl_ctrl_tolerance_show(struct kernfs_open_file *of, {struct resctrl_ctrl *ctrl =rdt_kn_parent_priv (of->kn); -seq_printf (seq,"%d\n" , ctrl->membw.tolerance); +seq_printf (seq,"%d\n" , ctrl->scalar.tolerance);return 0 ; }
@@ -2296,7 +2296,7 @@ static int resctrl_ctrl_scale_show(struct kernfs_open_file *of, {struct resctrl_ctrl *ctrl =rdt_kn_parent_priv (of->kn); -seq_printf (seq,"%u\n" , ctrl->membw.scale); +seq_printf (seq,"%u\n" , ctrl->scalar.scale);return 0 ; }
@@ -2306,7 +2306,7 @@ static int resctrl_ctrl_unit_show(struct kernfs_open_file *of, {struct resctrl_ctrl *ctrl =rdt_kn_parent_priv (of->kn); -switch (ctrl->membw.unit) { +switch (ctrl->scalar.unit) {case RESCTRL_CTRL_UNIT_ALL :seq_puts (seq,"all\n" );return 0 ;
@@ -2901,7 +2901,7 @@ static int set_mba_sc(bool mba_sc)if (!supports_mba_mbps (ctrl) || mba_sc ==is_mba_sc (r, ctrl))return -EINVAL ; - ctrl->membw.mba_sc = mba_sc; + ctrl->scalar.mba_sc = mba_sc; rdtgroup_default.mba_mbps_event = mba_mbps_default_event;
include/linux/resctrl.h +8 -8
@@ -258,10 +258,10 @@ enum resctrl_ctrl_unit { };/** -* struct resctrl_membw - Memory bandwidth allocation related data -* @min_bw: Minimum memory bandwidth percentage user can request -* @max_bw: Maximum memory bandwidth value, used as the reset value -* @bw_gran: Granularity at which the memory bandwidth is allocated +* struct resctrl_ctrl_scalar - Scalar control type properties +* @min_bw: Minimum scalar control value user can request +* @max_bw: Maximum scalar control value, used as the reset value +* @bw_gran: Granularity at which the scalar control is allocated * @resolution: If the control is proportional (for example, a * percentage) this is the number of divisions that * resource can be divided into. Otherwise, since this
@@ -284,7 +284,7 @@ enum resctrl_ctrl_unit {* the amount of resource allocated by this control is: * C * scale / resolution * unit */ -struct resctrl_membw { +struct resctrl_ctrl_scalar {u32 min_bw;u32 max_bw;u32 bw_gran;
@@ -370,7 +370,7 @@ enum resctrl_ctrl_name {* For example, with resource name "MB" and control name "MAX" the * schema entry will be "MB_MAX". * @cache: Cache allocation control properties. -* @membw: Bandwidth control properties. +* @scalar: Scalar control properties. */ struct resctrl_ctrl {struct list_head entry;
@@ -380,7 +380,7 @@ struct resctrl_ctrl {enum resctrl_ctrl_name name;union {struct resctrl_cache cache; -struct resctrl_membw membw; +struct resctrl_ctrl_scalar scalar; }; };
@@ -483,7 +483,7 @@ static inline u32 resctrl_get_default_ctrlval(struct resctrl_ctrl *ctrl)case RESCTRL_CTRL_BITMAP :return BIT_MASK (ctrl->cache.cbm_len) -1 ;case RESCTRL_CTRL_SCALAR : -return ctrl->membw.max_bw; +return ctrl->scalar.max_bw; }return WARN_ON_ONCE (1 );
Signature · 1 line
2.43.0
RFC PATCH 12/31 x86/resctrl: Rename struct resctrl_cache to struct resctrl_ctrl_bitmap
In reply to Chen YuRename struct resctrl_cache to struct resctrl_ctrl_bitmap to better reflect its role as the property container for bitmap-type controls. Also rename the union member from 'cache' to 'bitmap' for consistency.
This follows the design discussed in the generic schema RFC where controls are classified by type (scalar vs bitmap) rather than by the specific resource they manage (cache allocation).
--- arch/x86/kernel/cpu/resctrl/core.c | 20 +++++++++--------- drivers/resctrl/mpam_resctrl.c | 8 +++---- fs/resctrl/ctrlmondata.c | 12 +++++------ fs/resctrl/pseudo_lock.c | 2 +- fs/resctrl/rdtgroup.c | 34 +++++++++++++++--------------- include/linux/resctrl.h | 10 ++++----- 6 files changed, 43 insertions(+), 43 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +10 -10
@@ -167,10 +167,10 @@ static inline void cache_alloc_hsw_probe(void) hw_ctrl->r_ctrl.type =RESCTRL_CTRL_BITMAP ; hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); - hw_ctrl->r_ctrl.cache.cbm_len =20 ; - hw_ctrl->r_ctrl.cache.shareable_bits =0xc0000 ; - hw_ctrl->r_ctrl.cache.min_cbm_bits =2 ; - hw_ctrl->r_ctrl.cache.arch_has_sparse_bitmasks = false; + hw_ctrl->r_ctrl.bitmap.cbm_len =20 ; + hw_ctrl->r_ctrl.bitmap.shareable_bits =0xc0000 ; + hw_ctrl->r_ctrl.bitmap.min_cbm_bits =2 ; + hw_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = false;list_add (&hw_ctrl->r_ctrl.entry, &r->controls); hw_res->num_closid =4 ;
@@ -308,16 +308,16 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r) hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); - hw_ctrl->r_ctrl.cache.cbm_len = eax.split.cbm_len +1 ; + hw_ctrl->r_ctrl.bitmap.cbm_len = eax.split.cbm_len +1 ; default_ctrl =BIT_MASK (eax.split.cbm_len +1 ) -1 ; - hw_ctrl->r_ctrl.cache.shareable_bits = ebx & default_ctrl; + hw_ctrl->r_ctrl.bitmap.shareable_bits = ebx & default_ctrl;if (boot_cpu_data.x86_vendor ==X86_VENDOR_INTEL ) { - hw_ctrl->r_ctrl.cache.arch_has_sparse_bitmasks = ecx.split.noncont; - hw_ctrl->r_ctrl.cache.min_cbm_bits =1 ; + hw_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = ecx.split.noncont; + hw_ctrl->r_ctrl.bitmap.min_cbm_bits =1 ; }else if (boot_cpu_data.x86_vendor ==X86_VENDOR_AMD || boot_cpu_data.x86_vendor ==X86_VENDOR_HYGON ) { - hw_ctrl->r_ctrl.cache.arch_has_sparse_bitmasks = true; - hw_ctrl->r_ctrl.cache.min_cbm_bits =0 ; + hw_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = true; + hw_ctrl->r_ctrl.bitmap.min_cbm_bits =0 ; }else {return ; }
drivers/resctrl/mpam_resctrl.c +4 -4
@@ -1009,11 +1009,11 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) mpam_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD_RCU (&mpam_ctrl->r_ctrl.domains); - mpam_ctrl->r_ctrl.cache.arch_has_sparse_bitmasks = true; + mpam_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = true; - mpam_ctrl->r_ctrl.cache.cbm_len = class->props.cpbm_wd; + mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;/* mpam_devices will reject empty bitmaps */ - mpam_ctrl->r_ctrl.cache.min_cbm_bits =1 ; + mpam_ctrl->r_ctrl.bitmap.min_cbm_bits =1 ;if (r->rid ==RDT_RESOURCE_L2 ) { r->name ="L2" ;
@@ -1031,7 +1031,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)* we have configured the SMMU and GIC not to do this 'all the * bits' is the correct answer here. */ - mpam_ctrl->r_ctrl.cache.shareable_bits =resctrl_get_default_ctrlval (&mpam_ctrl->r_ctrl); + mpam_ctrl->r_ctrl.bitmap.shareable_bits =resctrl_get_default_ctrlval (&mpam_ctrl->r_ctrl);list_add (&mpam_ctrl->r_ctrl.entry, &r->controls); r->alloc_capable = true;break ;
fs/resctrl/ctrlmondata.c +6 -6
@@ -136,8 +136,8 @@ static int parse_bw(struct rdt_parse_data *data, struct rdt_resource_final *f,*/ static bool cbm_validate (char *buf,u32 *data,struct resctrl_ctrl *ctrl) { -u32 supported_bits =BIT_MASK (ctrl->cache.cbm_len) -1 ; -unsigned int cbm_len = ctrl->cache.cbm_len; +u32 supported_bits =BIT_MASK (ctrl->bitmap.cbm_len) -1 ; +unsigned int cbm_len = ctrl->bitmap.cbm_len;unsigned long first_bit, zero_bit, val;int ret;
@@ -147,7 +147,7 @@ static bool cbm_validate(char *buf, u32 *data, struct resctrl_ctrl *ctrl)return false; } -if ((ctrl->cache.min_cbm_bits >0 && val ==0 ) || val > supported_bits) { +if ((ctrl->bitmap.min_cbm_bits >0 && val ==0 ) || val > supported_bits) {rdt_last_cmd_puts ("Mask out of range\n" );return false; }
@@ -156,15 +156,15 @@ static bool cbm_validate(char *buf, u32 *data, struct resctrl_ctrl *ctrl) zero_bit =find_next_zero_bit (&val, cbm_len, first_bit);/* Are non-contiguous bitmasks allowed? */ -if (!ctrl->cache.arch_has_sparse_bitmasks && +if (!ctrl->bitmap.arch_has_sparse_bitmasks && (find_next_bit (&val, cbm_len, zero_bit) < cbm_len)) {rdt_last_cmd_printf ("The mask %lx has non-consecutive 1-bits\n" , val);return false; } -if ((zero_bit - first_bit) < ctrl->cache.min_cbm_bits) { +if ((zero_bit - first_bit) < ctrl->bitmap.min_cbm_bits) {rdt_last_cmd_printf ("Need at least %d bits in the mask\n" , - ctrl->cache.min_cbm_bits); + ctrl->bitmap.min_cbm_bits);return false; }
fs/resctrl/pseudo_lock.c +1 -1
@@ -639,7 +639,7 @@ bool rdtgroup_cbm_overlaps_pseudo_locked(struct rdt_ctrl_domain *d, unsigned lonreturn false; } - cbm_len = ctrl->cache.cbm_len; + cbm_len = ctrl->bitmap.cbm_len; cbm_b = d->plr->cbm;if (bitmap_intersects (&cbm, &cbm_b, cbm_len))return true;
fs/resctrl/rdtgroup.c +17 -17
@@ -1030,7 +1030,7 @@ static int rdt_min_cbm_bits_show(struct kernfs_open_file *of,if (!ctrl)return 0 ; -seq_printf (seq,"%u\n" , ctrl->cache.min_cbm_bits); +seq_printf (seq,"%u\n" , ctrl->bitmap.min_cbm_bits);return 0 ; }
@@ -1045,7 +1045,7 @@ static int rdt_shareable_bits_show(struct kernfs_open_file *of,if (!ctrl)return 0 ; -seq_printf (seq,"%x\n" , ctrl->cache.shareable_bits); +seq_printf (seq,"%x\n" , ctrl->bitmap.shareable_bits);return 0 ; }
@@ -1091,7 +1091,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of,list_for_each_entry (dom, &ctrl->domains, hdr.list) {if (sep)seq_putc (seq,';' ); - hw_shareable = ctrl->cache.shareable_bits; + hw_shareable = ctrl->bitmap.shareable_bits; sw_shareable =0 ; exclusive =0 ;seq_printf (seq,"%d=" , dom->hdr.id);
@@ -1141,7 +1141,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, hw_shareable |= ctrl_val; } -for (i = ctrl->cache.cbm_len -1 ; i >=0 ; i--) { +for (i = ctrl->bitmap.cbm_len -1 ; i >=0 ; i--) { pseudo_locked = dom->plr ? dom->plr->cbm :0 ; hwb =test_bit (i, &hw_shareable); swb =test_bit (i, &sw_shareable);
@@ -1337,7 +1337,7 @@ static int rdt_has_sparse_bitmasks_show(struct kernfs_open_file *of,if (!ctrl)return 0 ; -seq_printf (seq,"%u\n" , ctrl->cache.arch_has_sparse_bitmasks); +seq_printf (seq,"%u\n" , ctrl->bitmap.arch_has_sparse_bitmasks);return 0 ; }
@@ -1375,8 +1375,8 @@ static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_ctrl_doma/* Check for any overlap with regions used by hardware directly */ if (!exclusive) { - ctrl_b = ctrl->cache.shareable_bits; -if (bitmap_intersects (&cbm, &ctrl_b, ctrl->cache.cbm_len)) + ctrl_b = ctrl->bitmap.shareable_bits; +if (bitmap_intersects (&cbm, &ctrl_b, ctrl->bitmap.cbm_len))return true; }
@@ -1386,7 +1386,7 @@ static bool __rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_ctrl_doma mode =rdtgroup_mode_by_closid (i);if (closid_allocated (i) && i != closid && mode !=RDT_MODE_PSEUDO_LOCKSETUP ) { -if (bitmap_intersects (&cbm, &ctrl_b, ctrl->cache.cbm_len)) { +if (bitmap_intersects (&cbm, &ctrl_b, ctrl->bitmap.cbm_len)) {if (exclusive) {if (mode ==RDT_MODE_EXCLUSIVE )return true;
@@ -1591,10 +1591,10 @@ unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct resctrl_ctrl *cif (WARN_ON_ONCE (ctrl->scope !=RESCTRL_L2_CACHE && ctrl->scope !=RESCTRL_L3_CACHE ))return size; - num_b =bitmap_weight (&cbm, ctrl->cache.cbm_len); + num_b =bitmap_weight (&cbm, ctrl->bitmap.cbm_len); ci =get_cpu_cacheinfo_level (cpumask_any (&d->hdr.cpu_mask), ctrl->scope);if (ci) - size = ci->size / ctrl->cache.cbm_len * num_b; + size = ci->size / ctrl->bitmap.cbm_len * num_b;return size; }
@@ -3883,11 +3883,11 @@ static int mkdir_mondata_all(struct kernfs_node *parent_kn,static u32 cbm_ensure_valid (u32 _val,struct rdt_resource *r,struct resctrl_ctrl *ctrl) { -unsigned int cbm_len = ctrl->cache.cbm_len; +unsigned int cbm_len = ctrl->bitmap.cbm_len;unsigned long first_bit, zero_bit;unsigned long val; -if (!_val || ctrl->cache.arch_has_sparse_bitmasks) +if (!_val || ctrl->bitmap.arch_has_sparse_bitmasks)return _val; val = _val;
@@ -3920,8 +3920,8 @@ static int __init_one_rdt_domain(struct rdt_ctrl_domain *d, struct rdt_resource_ cfg = &d->staged_config[t]; cfg->have_new_ctrl = false; - cfg->new_ctrl = ctrl->cache.shareable_bits; - used_b = ctrl->cache.shareable_bits; + cfg->new_ctrl = ctrl->bitmap.shareable_bits; + used_b = ctrl->bitmap.shareable_bits;for (i =0 ; i <closids_supported (); i++) {if (closid_allocated (i) && i != closid) { mode =rdtgroup_mode_by_closid (i);
@@ -3951,8 +3951,8 @@ static int __init_one_rdt_domain(struct rdt_ctrl_domain *d, struct rdt_resource_ }if (d->plr && d->plr->cbm >0 ) used_b |= d->plr->cbm; - unused_b = used_b ^ (BIT_MASK (ctrl->cache.cbm_len) -1 ); - unused_b &=BIT_MASK (ctrl->cache.cbm_len) -1 ; + unused_b = used_b ^ (BIT_MASK (ctrl->bitmap.cbm_len) -1 ); + unused_b &=BIT_MASK (ctrl->bitmap.cbm_len) -1 ; cfg->new_ctrl |= unused_b;/* * Force the initial CBM to be valid, user can
@@ -3964,7 +3964,7 @@ static int __init_one_rdt_domain(struct rdt_ctrl_domain *d, struct rdt_resource_* bitmap_weight() does not access out-of-bound memory. */ tmp_cbm = cfg->new_ctrl; -if (bitmap_weight (&tmp_cbm, ctrl->cache.cbm_len) < ctrl->cache.min_cbm_bits) { +if (bitmap_weight (&tmp_cbm, ctrl->bitmap.cbm_len) < ctrl->bitmap.min_cbm_bits) {rdt_last_cmd_printf ("No space on %s:%d\n" , f->name, d->hdr.id);return -ENOSPC ; }
include/linux/resctrl.h +5 -5
@@ -211,7 +211,7 @@ struct rdt_l3_mon_domain { };/** -* struct resctrl_cache - Cache allocation related data +* struct resctrl_ctrl_bitmap - Bitmap control type properties * @cbm_len: Length of the cache bit mask * @min_cbm_bits: Minimum number of consecutive bits to be set. * The value 0 means the architecture can support
@@ -220,7 +220,7 @@ struct rdt_l3_mon_domain {* executing entities * @arch_has_sparse_bitmasks: True if a bitmask like f00f is valid. */ -struct resctrl_cache { +struct resctrl_ctrl_bitmap {unsigned int cbm_len;unsigned int min_cbm_bits;unsigned int shareable_bits;
@@ -369,7 +369,7 @@ enum resctrl_ctrl_name {* Specifically, "rdt_resource_final::name"_"resctrl_ctrl::name". * For example, with resource name "MB" and control name "MAX" the * schema entry will be "MB_MAX". -* @cache: Cache allocation control properties. +* @bitmap: Bitmap control properties. * @scalar: Scalar control properties. */ struct resctrl_ctrl {
@@ -379,7 +379,7 @@ struct resctrl_ctrl {enum resctrl_ctrl_type type;enum resctrl_ctrl_name name;union { -struct resctrl_cache cache; +struct resctrl_ctrl_bitmap bitmap;struct resctrl_ctrl_scalar scalar; }; };
@@ -481,7 +481,7 @@ static inline u32 resctrl_get_default_ctrlval(struct resctrl_ctrl *ctrl) {switch (ctrl->type) {case RESCTRL_CTRL_BITMAP : -return BIT_MASK (ctrl->cache.cbm_len) -1 ; +return BIT_MASK (ctrl->bitmap.cbm_len) -1 ;case RESCTRL_CTRL_SCALAR :return ctrl->scalar.max_bw; }
Signature · 1 line
2.43.0
RFC PATCH 13/31 x86/resctrl: Add per-control and per-resource flags
In reply to Chen YuIntroduce a 'flags' field in both struct rdt_resource and struct resctrl_ctrl to expose control properties via the resource_schemata info directory. Define two flags:
RESCTRL_CTRL_FLAG_LINEAR - scalar control uses linear delay
RESCTRL_CTRL_FLAG_SPARSE_BITMASKS - bitmap control supports sparse masks
The resource-level flags (rdt_resource::flags) serve as the source of truth for properties shared by all controls of a resource. Each control duplicates these flags into its own resctrl_ctrl::flags field for per-control visibility in the info/<resource>/resource_schemata/<ctrl>/ directory.
The existing fields (bw_delay_linear, arch_has_sparse_bitmasks) are preserved for backward compatibility with the legacy info directory files (delay_linear, sparse_masks).
--- arch/x86/kernel/cpu/resctrl/core.c | 12 ++++++++++++ drivers/resctrl/mpam_resctrl.c | 4 ++++ include/linux/resctrl.h | 11 +++++++++++ 3 files changed, 27 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +12 -0
@@ -171,6 +171,8 @@ static inline void cache_alloc_hsw_probe(void) hw_ctrl->r_ctrl.bitmap.shareable_bits =0xc0000 ; hw_ctrl->r_ctrl.bitmap.min_cbm_bits =2 ; hw_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = false; + r->flags =0 ; + hw_ctrl->r_ctrl.flags =0 ;list_add (&hw_ctrl->r_ctrl.entry, &r->controls); hw_res->num_closid =4 ;
@@ -209,6 +211,8 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r) hw_ctrl->r_ctrl.scalar.bw_gran =MAX_MBA_BW - max_delay; r->bw_delay_linear = true; + r->flags =RESCTRL_CTRL_FLAG_LINEAR ; + hw_ctrl->r_ctrl.flags =RESCTRL_CTRL_FLAG_LINEAR ;if (boot_cpu_has (X86_FEATURE_PER_THREAD_MBA )) r->bw_throttle_mode =THREAD_THROTTLE_PER_THREAD ;else
@@ -261,6 +265,8 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)/* AMD does not use delay */ r->bw_delay_linear = false; + r->flags =0 ; + hw_ctrl->r_ctrl.flags =0 ;/* * AMD does not use memory delay throttle model to control
@@ -313,10 +319,16 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r) hw_ctrl->r_ctrl.bitmap.shareable_bits = ebx & default_ctrl;if (boot_cpu_data.x86_vendor ==X86_VENDOR_INTEL ) { hw_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = ecx.split.noncont; +if (ecx.split.noncont) { + r->flags =RESCTRL_CTRL_FLAG_SPARSE_BITMASKS ; + hw_ctrl->r_ctrl.flags =RESCTRL_CTRL_FLAG_SPARSE_BITMASKS ; + } hw_ctrl->r_ctrl.bitmap.min_cbm_bits =1 ; }else if (boot_cpu_data.x86_vendor ==X86_VENDOR_AMD || boot_cpu_data.x86_vendor ==X86_VENDOR_HYGON ) { hw_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = true; + r->flags =RESCTRL_CTRL_FLAG_SPARSE_BITMASKS ; + hw_ctrl->r_ctrl.flags =RESCTRL_CTRL_FLAG_SPARSE_BITMASKS ; hw_ctrl->r_ctrl.bitmap.min_cbm_bits =0 ; }else {return ;
drivers/resctrl/mpam_resctrl.c +4 -0
@@ -1010,6 +1010,8 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)INIT_LIST_HEAD_RCU (&mpam_ctrl->r_ctrl.domains); mpam_ctrl->r_ctrl.bitmap.arch_has_sparse_bitmasks = true; + r->flags =RESCTRL_CTRL_FLAG_SPARSE_BITMASKS ; + mpam_ctrl->r_ctrl.flags =RESCTRL_CTRL_FLAG_SPARSE_BITMASKS ; mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;/* mpam_devices will reject empty bitmaps */
@@ -1042,6 +1044,8 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)INIT_LIST_HEAD_RCU (&mpam_ctrl->r_ctrl.domains); r->bw_delay_linear = true; + r->flags =RESCTRL_CTRL_FLAG_LINEAR ; + mpam_ctrl->r_ctrl.flags =RESCTRL_CTRL_FLAG_LINEAR ; r->bw_throttle_mode =THREAD_THROTTLE_UNDEFINED ; mpam_ctrl->r_ctrl.scalar.min_bw =get_mba_min (cprops); mpam_ctrl->r_ctrl.scalar.max_bw =MAX_MBA_BW ;
include/linux/resctrl.h +11 -0
@@ -356,6 +356,10 @@ enum resctrl_ctrl_name { RESCTRL_CTRL_NAME_LAST = RESCTRL_CTRL_NAME_MAX }; +/* Flags for rdt_resource::flags and resctrl_ctrl::flags */ +#define RESCTRL_CTRL_FLAG_LINEAR BIT(0) +#define RESCTRL_CTRL_FLAG_SPARSE_BITMASKS BIT(1) +/** * struct resctrl_ctrl - A resource control * @entry: List entry of rdt_resource::controls
@@ -369,6 +373,9 @@ enum resctrl_ctrl_name {* Specifically, "rdt_resource_final::name"_"resctrl_ctrl::name". * For example, with resource name "MB" and control name "MAX" the * schema entry will be "MB_MAX". +* @flags: Optional flags describing properties of the control. +* Duplicated from resource-level properties for per-control +* visibility via the resource_schemata info directory. * @bitmap: Bitmap control properties. * @scalar: Scalar control properties. */
@@ -378,6 +385,7 @@ struct resctrl_ctrl {struct list_head domains;enum resctrl_ctrl_type type;enum resctrl_ctrl_name name; +unsigned long flags;union {struct resctrl_ctrl_bitmap bitmap;struct resctrl_ctrl_scalar scalar;
@@ -399,6 +407,8 @@ struct resctrl_ctrl {* different memory bandwidths * @cache_io_alloc_capable:True if portion of the cache can be configured * for I/O traffic. +* @flags: Resource-level flags (RESCTRL_CTRL_FLAG_*). Duplicated +* into each control's flags field for per-control visibility. * @controls: List of controls of an alloc_capable resource */ struct rdt_resource {
@@ -413,6 +423,7 @@ struct rdt_resource {bool bw_delay_linear;enum membw_throttle_mode bw_throttle_mode;bool cache_io_alloc_capable; +unsigned long flags;struct list_head controls; };
Signature · 1 line
2.43.0
RFC PATCH 14/31 x86/resctrl: Add emulation controller list to resctrl_ctrl
In reply to Chen YuThere is a requirement to emulate Controller A using a set of other controllers. The purpose is to either supply hierarchy information in the schema info directory or emulate legacy controllers via the new controllers for backward compatibility.
A legacy controller can be emulated by more than one controller. For example, on region aware platforms the legacy MBA control is emulated by the MAX control of every available memory region. Track the emulating controllers on a list anchored at resctrl_ctrl::emul, linked via resctrl_ctrl::emul_entry.
--- arch/x86/kernel/cpu/resctrl/core.c | 4 ++++ include/linux/resctrl.h | 4 ++++ 2 files changed, 8 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +4 -0
@@ -167,6 +167,7 @@ static inline void cache_alloc_hsw_probe(void) hw_ctrl->r_ctrl.type =RESCTRL_CTRL_BITMAP ; hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); +INIT_LIST_HEAD (&hw_ctrl->r_ctrl.emul); hw_ctrl->r_ctrl.bitmap.cbm_len =20 ; hw_ctrl->r_ctrl.bitmap.shareable_bits =0xc0000 ; hw_ctrl->r_ctrl.bitmap.min_cbm_bits =2 ;
@@ -205,6 +206,7 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r) hw_ctrl->r_ctrl.type =RESCTRL_CTRL_SCALAR ; hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); +INIT_LIST_HEAD (&hw_ctrl->r_ctrl.emul); hw_ctrl->r_ctrl.scalar.max_bw =MAX_MBA_BW ; hw_ctrl->r_ctrl.scalar.min_bw =MAX_MBA_BW - max_delay;
@@ -260,6 +262,7 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r) hw_ctrl->r_ctrl.type =RESCTRL_CTRL_SCALAR ; hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); +INIT_LIST_HEAD (&hw_ctrl->r_ctrl.emul); hw_ctrl->r_ctrl.scalar.max_bw =BIT (eax);
@@ -313,6 +316,7 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r) hw_ctrl->r_ctrl.type =RESCTRL_CTRL_BITMAP ; hw_ctrl->r_ctrl.name =RESCTRL_CTRL_NAME_DEF ;INIT_LIST_HEAD (&hw_ctrl->r_ctrl.domains); +INIT_LIST_HEAD (&hw_ctrl->r_ctrl.emul); hw_ctrl->r_ctrl.bitmap.cbm_len = eax.split.cbm_len +1 ; default_ctrl =BIT_MASK (eax.split.cbm_len +1 ) -1 ;
include/linux/resctrl.h +4 -0
@@ -378,6 +378,8 @@ enum resctrl_ctrl_name {* visibility via the resource_schemata info directory. * @bitmap: Bitmap control properties. * @scalar: Scalar control properties. +* @emul: List of controls that emulate the current control. +* @emul_entry: List entry of the emulated control's resctrl_ctrl::emul. */ struct resctrl_ctrl {struct list_head entry;
@@ -390,6 +392,8 @@ struct resctrl_ctrl {struct resctrl_ctrl_bitmap bitmap;struct resctrl_ctrl_scalar scalar; }; +struct list_head emul; +struct list_head emul_entry; };/**
Signature · 1 line
2.43.0
RFC PATCH 15/31 x86/resctrl: Parse ACPI MMRC table
In reply to Chen YuThe MMRC table is used to provide Memory Bandwidth Monitor Register information. Its register addresses are mapped, and the MMRC pointer is saved in the corresponding domain information entry.
--- arch/x86/kernel/cpu/resctrl/erdt.c | 51 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 6 ++- 2 files changed, 56 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/erdt.c +51 -0
@@ -24,6 +24,7 @@ static bool erdt_enabled;#define ERDT_VALID_VERSION 1#define CMRC_SUPPORTED_INDEX_FN 1 +#define MMRC_SUPPORTED_INDEX_FN 1#define RMDD_FLAG_CPU_L3_DOMAIN BIT(0)/* Set in a monitoring counter when it holds no valid data to report. */
@@ -141,6 +142,7 @@ static void cleanup_one_domain(struct erdt_domain_info *d) {erdt_iounmap_domain (d);kfree (d->cmrc); +kfree (d->mmrc);kfree (d); }
@@ -211,6 +213,49 @@ static __init int cmrc_init(struct acpi_subtbl_hdr_16 *subtbl,return 0 ; } +static __init int mmrc_init (struct acpi_subtbl_hdr_16 *subtbl, +struct erdt_domain_info *domain_info) +{ +struct acpi_erdt_mmrc *mmrc = (struct acpi_erdt_mmrc *)subtbl; + +if (subtbl->length <sizeof (*mmrc)) { +pr_warn (FW_BUG "Truncated MMRC subtable\n" ); +return -EIO ; + } + +if (mmrc->index_fn !=MMRC_SUPPORTED_INDEX_FN ) { +pr_info ("Unsupported MMRC index function %d\n" , mmrc->index_fn); +return -EIO ; + } + +/* +* The correction factor list is a trailing flexible array, so it is +* not covered by the sizeof(*mmrc) check above. Firmware could claim +* more entries than the sub-table actually carries, so only trust the +* length the sub-table can back. +*/ +if (subtbl->length <struct_size (mmrc, corr_factor_list, + mmrc->corr_factor_list_len)) { +pr_warn (FW_BUG "MMRC correction factor list of %u entries exceeds sub-table\n" , + mmrc->corr_factor_list_len); +return -EIO ; + } + + domain_info->base[ERDT_MMIO_MMRC_BASE ] = +erdt_ioremap (mmrc->reg_base, mmrc->reg_size,"MMRC base" ); +if (!domain_info->base[ERDT_MMIO_MMRC_BASE ]) +return -EIO ; + + domain_info->mmrc =kmemdup (mmrc, subtbl->length,GFP_KERNEL ); +if (!domain_info->mmrc) { +iounmap (domain_info->base[ERDT_MMIO_MMRC_BASE ]); + domain_info->base[ERDT_MMIO_MMRC_BASE ] =NULL ; +return -ENOMEM ; + } + +return 0 ; +} +static inline struct acpi_subtbl_hdr_16 *rmdd_subtbl (struct acpi_erdt_rmdd *rmdd) {return (void *)rmdd +sizeof (*rmdd);
@@ -287,6 +332,12 @@ static __init bool parse_rmdd_table(struct acpi_subtbl_hdr_16 *rmdd_hdr) !cmrc_init(subtbl, domain_info)) subtbl_mask |=BIT (ACPI_ERDT_TYPE_CMRC ); +break ; +case ACPI_ERDT_TYPE_MMRC : +if (!(subtbl_mask &BIT (ACPI_ERDT_TYPE_MMRC )) && + !mmrc_init (subtbl, domain_info)) + subtbl_mask |=BIT (ACPI_ERDT_TYPE_MMRC ); +break ;default :break ;
arch/x86/kernel/cpu/resctrl/internal.h +5 -1
@@ -25,11 +25,13 @@* Index into erdt_domain_info::base[] for each MMIO region. * @ERDT_MMIO_RMDD_CREG: RMDD control register base address * @ERDT_MMIO_CMRC_BASE: CMRC monitoring register base address +* @ERDT_MMIO_MMRC_BASE: MMRC monitoring register base address */ enum erdt_mmio_type {ERDT_MMIO_RMDD_CREG ,ERDT_MMIO_CMRC_BASE , -ERDT_MMIO_LAST =ERDT_MMIO_CMRC_BASE +ERDT_MMIO_MMRC_BASE , +ERDT_MMIO_LAST =ERDT_MMIO_MMRC_BASE };#define ERDT_MMIO_NUM_TYPES (ERDT_MMIO_LAST + 1)
@@ -38,6 +40,7 @@ enum erdt_mmio_type {* struct erdt_domain_info - Per-domain ERDT information * @base: Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_* type * @cmrc: Copy of the ACPI CMRC sub-table for this domain +* @mmrc: Copy of the ACPI MMRC sub-table for this domain * @cpu_mask: CPUs belonging to this resource management domain * @max_rmid: Maximum RMID supported by this domain * @dom_id: L3 cache ID shared by all CPUs in this domain (-1 if unset)
@@ -46,6 +49,7 @@ enum erdt_mmio_type {struct erdt_domain_info {void __iomem *base[ERDT_MMIO_NUM_TYPES ];struct acpi_erdt_cmrc *cmrc; +struct acpi_erdt_mmrc *mmrc;struct cpumask cpu_mask;u32 max_rmid;int dom_id;
Signature · 1 line
2.43.0
RFC PATCH 16/31 x86/resctrl: Replace "msr" in monitoring data identifiers
In reply to Chen YuMonitoring counter values are currently obtained via MSR and many identifiers involved in processing these values contain "msr" as part of their names, for example "prev_msr" and "msr_val".
The ERDT ACPI table describes MMIO registers for monitoring data access. Rename "msr"-based identifiers to be interface-agnostic to support their upcoming use for MMIO-read values.
No functional change.
--- arch/x86/kernel/cpu/resctrl/internal.h | 8 +++---- arch/x86/kernel/cpu/resctrl/monitor.c | 30 +++++++++++++------------- 2 files changed, 19 insertions(+), 19 deletions(-)
arch/x86/kernel/cpu/resctrl/internal.h +4 -4
@@ -66,13 +66,13 @@ struct erdt_domain_info {/** * struct arch_mbm_state - values used to compute resctrl_arch_rmid_read()s * return value. -* @chunks: Total data moved (multiply by rdt_group.mon_scale to get bytes) -* @prev_msr: Value of IA32_QM_CTR last time it was read for the RMID used to -* find this struct. +* @chunks: Total data moved (multiply by rdt_group.mon_scale to get bytes) +* @prev_mon_val: Previous monitor counter value for the RMID used to +* find this struct. */ struct arch_mbm_state {u64 chunks; -u64 prev_msr; +u64 prev_mon_val; };/* Setting bit 0 in L3_QOS_EXT_CFG enables the ABMC feature. */
arch/x86/kernel/cpu/resctrl/monitor.c +15 -15
@@ -186,7 +186,7 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_l3_mon_domain *d prmid =logical_rmid_to_physical_rmid (cpu, rmid);/* Record any initial, non-zero count value. */ -__rmid_read_phys (prmid, eventid, &am->prev_msr); +__rmid_read_phys (prmid, eventid, &am->prev_mon_val); } }
@@ -209,16 +209,16 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_l3_mon_domai } } -static u64 mbm_overflow_count (u64 prev_msr,u64 cur_msr,unsigned int width) +static u64 mbm_overflow_count (u64 prev_val,u64 cur_val,unsigned int width) {u64 shift =64 - width, chunks; - chunks = (cur_msr << shift) - (prev_msr << shift); + chunks = (cur_val << shift) - (prev_val << shift);return chunks >> shift; }static u64 get_corrected_val (struct rdt_resource *r,struct rdt_l3_mon_domain *d, -u32 rmid,enum resctrl_event_id eventid,u64 msr_val) +u32 rmid,enum resctrl_event_id eventid,u64 mon_val) {struct rdt_hw_l3_mon_domain *hw_dom =resctrl_to_arch_mon_dom (d);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);
@@ -227,12 +227,12 @@ static u64 get_corrected_val(struct rdt_resource *r, struct rdt_l3_mon_domain *d am =get_arch_mbm_state (hw_dom, rmid, eventid);if (am) { - am->chunks +=mbm_overflow_count (am->prev_msr, msr_val, + am->chunks +=mbm_overflow_count (am->prev_mon_val, mon_val, hw_res->mbm_width); chunks =get_corrected_mbm_count (rmid, am->chunks); - am->prev_msr = msr_val; + am->prev_mon_val = mon_val; }else { - chunks = msr_val; + chunks = mon_val; }return chunks * hw_res->mon_scale;
@@ -245,7 +245,7 @@ static int arch_l3_read_event(struct rdt_domain_hdr *hdr, u32 rmid,struct rdt_hw_l3_mon_domain *hw_dom;struct rdt_l3_mon_domain *d;struct arch_mbm_state *am; -u64 msr_val; +u64 mon_val;u32 prmid;int cpu;int ret;
@@ -257,14 +257,14 @@ static int arch_l3_read_event(struct rdt_domain_hdr *hdr, u32 rmid, hw_dom =resctrl_to_arch_mon_dom (d); cpu =cpumask_any (&hdr->cpu_mask); prmid =logical_rmid_to_physical_rmid (cpu, rmid); - ret =__rmid_read_phys (prmid, eventid, &msr_val); + ret =__rmid_read_phys (prmid, eventid, &mon_val);if (!ret) { - *val =get_corrected_val (r, d, rmid, eventid, msr_val); + *val =get_corrected_val (r, d, rmid, eventid, mon_val); }else if (ret == -EINVAL ) { am =get_arch_mbm_state (hw_dom, rmid, eventid);if (am) - am->prev_msr =0 ; + am->prev_mon_val =0 ; }return ret;
@@ -339,7 +339,7 @@ void resctrl_arch_reset_cntr(struct rdt_resource *r, struct rdt_l3_mon_domain *dmemset (am,0 ,sizeof (*am));/* Record any initial, non-zero count value. */ -__cntr_id_read (cntr_id, &am->prev_msr); +__cntr_id_read (cntr_id, &am->prev_mon_val); } }
@@ -347,14 +347,14 @@ int resctrl_arch_cntr_read(struct rdt_resource *r, struct rdt_l3_mon_domain *d, u32 unused, u32 rmid, int cntr_id, enum resctrl_event_id eventid, u64 *val) { - u64 msr_val; + u64 mon_val;int ret; - ret =__cntr_id_read (cntr_id, &msr_val); + ret =__cntr_id_read (cntr_id, &mon_val);if (ret)return ret; - *val =get_corrected_val (r, d, rmid, eventid, msr_val); + *val =get_corrected_val (r, d, rmid, eventid, mon_val);return 0 ; }
Signature · 1 line
2.43.0
RFC PATCH 17/31 x86/resctrl: Introduce region aware MBM event definitions
In reply to Chen YuRegion-Aware RDT supports per-region MBM. Per the specification, there are a maximum of 4 memory regions currently: DDR, CXL link, UPI link, and Persistent Memory (including both local and remote regions).
Based on this information, introduce 4 region-based MBM events accordingly.
--- fs/resctrl/monitor.c | 13 +++++++++++++ fs/resctrl/rdtgroup.c | 15 ++++++++++++++- include/linux/resctrl.h | 8 ++++++-- include/linux/resctrl_types.h | 14 +++++++++++++- 4 files changed, 46 insertions(+), 4 deletions(-)
fs/resctrl/monitor.c +13 -0
@@ -802,6 +802,17 @@ static void mbm_update_one_event(struct rdt_resource *r, struct rdt_l3_mon_domairesctrl_arch_mon_ctx_free (rr.r, evtid, rr.arch_mon_ctx); } +static void rmbm_update (struct rdt_resource *r,struct rdt_l3_mon_domain *d, +struct rdtgroup *rdtgrp) +{ +enum resctrl_event_id evt; + +for_each_rmbm_event_id (evt) { +if (resctrl_is_mon_event_enabled (evt)) +mbm_update_one_event (r, d, rdtgrp, evt); + } +} +static void mbm_update (struct rdt_resource *r,struct rdt_l3_mon_domain *d,struct rdtgroup *rdtgrp) {
@@ -814,6 +825,8 @@ static void mbm_update(struct rdt_resource *r, struct rdt_l3_mon_domain *d,if (resctrl_is_mon_event_enabled (QOS_L3_MBM_LOCAL_EVENT_ID ))mbm_update_one_event (r, d, rdtgrp,QOS_L3_MBM_LOCAL_EVENT_ID ); + +rmbm_update (r, d, rdtgrp); }/*
fs/resctrl/rdtgroup.c +14 -1
@@ -126,10 +126,23 @@ void rdt_staged_configs_clear(void) } } +static bool resctrl_is_rmbm_enabled (void ) +{ +enum resctrl_event_id evt; + +for_each_rmbm_event_id (evt) { +if (resctrl_is_mon_event_enabled (evt)) +return true; + } + +return false; +} +static bool resctrl_is_mbm_enabled (void ) {return (resctrl_is_mon_event_enabled (QOS_L3_MBM_TOTAL_EVENT_ID ) || -resctrl_is_mon_event_enabled (QOS_L3_MBM_LOCAL_EVENT_ID )); +resctrl_is_mon_event_enabled (QOS_L3_MBM_LOCAL_EVENT_ID ) || +resctrl_is_rmbm_enabled ()); }/*
include/linux/resctrl.h +6 -2
@@ -519,7 +519,7 @@ bool resctrl_arch_is_evt_configurable(enum resctrl_event_id evt);static inline bool resctrl_is_mbm_event (enum resctrl_event_id eventid) {return (eventid >=QOS_L3_MBM_TOTAL_EVENT_ID && - eventid <=QOS_L3_MBM_LOCAL_EVENT_ID ); + eventid <=QOS_L3_MBM_R3_EVENT_ID ); }u32 resctrl_get_mon_evt_cfg (enum resctrl_event_id eventid);
@@ -527,7 +527,11 @@ u32 resctrl_get_mon_evt_cfg(enum resctrl_event_id eventid);/* Iterate over all memory bandwidth events */ #define for_each_mbm_event_id (eventid) \ for (eventid = QOS_L3_MBM_TOTAL_EVENT_ID; \ - eventid <= QOS_L3_MBM_LOCAL_EVENT_ID; eventid++) + eventid <= QOS_L3_MBM_R3_EVENT_ID; eventid++) + +#define for_each_rmbm_event_id (eventid) \ + for (eventid = QOS_L3_MBM_R0_EVENT_ID; \ + eventid <= QOS_L3_MBM_R3_EVENT_ID; eventid++)/* Iterate over memory bandwidth arrays in domain structures */ #define for_each_mbm_idx (idx) \
include/linux/resctrl_types.h +13 -1
@@ -50,6 +50,18 @@ enum resctrl_event_id {QOS_L3_MBM_TOTAL_EVENT_ID =0x02 ,QOS_L3_MBM_LOCAL_EVENT_ID =0x03 , +/* +* Region based MBM events. +* at most 4 regions for now, these +* 4 regions could be mapped to +* different memory regions, including +* local and remote regions. +*/ +QOS_L3_MBM_R0_EVENT_ID , +QOS_L3_MBM_R1_EVENT_ID , +QOS_L3_MBM_R2_EVENT_ID , +QOS_L3_MBM_R3_EVENT_ID , +/* Intel Telemetry Events */ PMT_EVENT_ENERGY , PMT_EVENT_ACTIVITY,
@@ -65,7 +77,7 @@ enum resctrl_event_id { QOS_NUM_EVENTS, }; -#define QOS_NUM_L3_MBM_EVENTS (QOS_L3_MBM_LOCAL_EVENT_ID - QOS_L3_MBM_TOTAL_EVENT_ID + 1) +#define QOS_NUM_L3_MBM_EVENTS (QOS_L3_MBM_R3_EVENT_ID - QOS_L3_MBM_TOTAL_EVENT_ID + 1)#define MBM_STATE_IDX (evt) ((evt) - QOS_L3_MBM_TOTAL_EVENT_ID)#endif /* __LINUX_RESCTRL_TYPES_H */
Signature · 1 line
2.43.0
RFC PATCH 18/31 x86/resctrl: Introduce memory region based MBM read callback on MMIO space
In reply to Chen YuEnhance erdt_mon_read() to support region-based memory bandwidth reads, and implement the low-level MMIO-based region-aware erdt_read_region_mbm() to deal with region-aware MBM.
--- arch/x86/kernel/cpu/resctrl/erdt.c | 102 +++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 4 + arch/x86/kernel/cpu/resctrl/monitor.c | 8 +- include/linux/resctrl_types.h | 5 ++ 4 files changed, 115 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/resctrl/erdt.c +102 -0
@@ -30,6 +30,8 @@ static bool erdt_enabled;/* Set in a monitoring counter when it holds no valid data to report. */ #define UNAVAILABLE_COUNTER BIT_ULL(63)#define CMRC_FLAG_UNAVAILABLE_BIT BIT(0) +#define MMRC_FLAG_UNAVAILABLE_BIT BIT(0) +#define FIXPOINT_LOW_BITS 16/* Bitmask of valid sub-tables found in the first RMDD, used to ensure all RMDDs match. */ static u32 valid_subtbl_mask;
@@ -96,6 +98,103 @@ static int erdt_read_l3_occupancy(const struct erdt_domain_info *d, u32 rmid, u6return 0 ; } +static void __iomem *mmrc_index_function_1 (const struct erdt_domain_info *d, +struct acpi_erdt_mmrc *mmrc, +int rmid,int region_idx) +{ +u64 blk_rmid, blk_offset; +void __iomem *vaddr; + +/* +* Block_to_locate_RMID# = floor((RMID# % 32) / 8) x 4 x 4096B; +* Offset_within_this_Block = (floor(((RMID#/32)x8)+RMID#%8) x +* 8B)+(Region# x 2048B); +* MMIO_ADDRESS_for_RMID#_Region# = +* MBM Register Block Base Address + Block_to_locate_RMID# + +* Offset_within_this_Block; +*/ + blk_rmid = ((rmid %32 ) /8 ) *4 *4096 ; + blk_offset = ((rmid /32 ) *8 + (rmid %8 )) *8 + region_idx *2048 ; + vaddr = d->base[ERDT_MMIO_MMRC_BASE ] + blk_rmid + blk_offset; + +return vaddr; +} + +static u64 apply_correction_factor (u64 val,u32 factor) +{ +if (!factor) +return val; + +return ((val * factor) >>FIXPOINT_LOW_BITS ); +} + +static int erdt_read_region_mbm (struct rdt_domain_hdr *hdr, +const struct erdt_domain_info *d,int rmid, +int eventid,u64 *val) +{ +int region_idx = eventid -QOS_L3_MBM_R0_EVENT_ID ; +int corr_factor_len, corr_factor =0 ; +struct rdt_hw_l3_mon_domain *hw_dom; +u64 mbm_rmid_count =0 , chunks =0 ; +struct acpi_erdt_mmrc *mmrc =NULL ; +struct rdt_l3_mon_domain *mon_dom; +struct arch_mbm_state *am; +void __iomem *vaddr; + + mmrc = d->mmrc; +if (!mmrc) +return -EIO ; + +if (!domain_header_is_valid (hdr,RESCTRL_MON_DOMAIN ,RDT_RESOURCE_L3 )) +return -EIO ; + + mon_dom = container_of(hdr, struct rdt_l3_mon_domain, hdr); + hw_dom =resctrl_to_arch_mon_dom (mon_dom); + + vaddr =mmrc_index_function_1 (d, mmrc, rmid, region_idx); + mbm_rmid_count =readq (vaddr); + +/* +* Bit 63 only reports whether the data is unavailable on domains that +* advertise support for it. Where it is not supported the bit is part +* of the counter and must not be interpreted as a status flag. +*/ +if ((mmrc->flags &MMRC_FLAG_UNAVAILABLE_BIT ) && + (mbm_rmid_count &UNAVAILABLE_COUNTER )) +return -EINVAL ; + + corr_factor_len = mmrc->corr_factor_list_len; +if (corr_factor_len) { +/* +* 0: Do not apply a correction factor to +* the MBM values. +* 1: Apply a single correction factor. +* Max RMID+1: Apply the indicated indexed +* correction factor to the corresponding +* RMID value for MBM counter. +*/ +if (corr_factor_len ==1 ) + corr_factor = mmrc->corr_factor_list[0 ]; +else if (rmid < corr_factor_len) + corr_factor = mmrc->corr_factor_list[rmid]; +else +return -EINVAL ; + } + + am =get_arch_mbm_state (hw_dom, rmid, eventid); +if (am) { + am->chunks +=mbm_overflow_count (am->prev_mon_val, mbm_rmid_count, + mmrc->counter_width); + + chunks =apply_correction_factor (am->chunks, corr_factor); + am->prev_mon_val = mbm_rmid_count; + } + + *val = chunks * mmrc->up_scale; + +return 0 ; +} +int erdt_mon_read (struct rdt_domain_hdr *hdr,enum resctrl_event_id evtid,u32 rmid,u64 *val) {struct rdt_hw_l3_mon_domain *hw_dom;
@@ -109,6 +208,9 @@ int erdt_mon_read(struct rdt_domain_hdr *hdr, enum resctrl_event_id evtid, u32 rif (evtid ==QOS_L3_OCCUP_EVENT_ID )return erdt_read_l3_occupancy (d, rmid, val); +if (rmbm_event (evtid)) +return erdt_read_region_mbm (hdr, d, rmid, evtid, val); +return -EIO ; }
arch/x86/kernel/cpu/resctrl/internal.h +4 -0
@@ -320,4 +320,8 @@ int erdt_mon_read(struct rdt_domain_hdr *hdr, enum resctrl_event_id evtid, u32 rint erdt_init (void );void erdt_exit (void ); +u64 mbm_overflow_count (u64 prev_val,u64 cur_val,unsigned int width); +struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_l3_mon_domain *hw_dom, +u32 rmid, +enum resctrl_event_id eventid);#endif /* _ASM_X86_RESCTRL_INTERNAL_H */
arch/x86/kernel/cpu/resctrl/monitor.c +4 -4
@@ -157,9 +157,9 @@ static int __rmid_read_phys(u32 prmid, enum resctrl_event_id eventid, u64 *val)return 0 ; } -static struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_l3_mon_domain *hw_dom, -u32 rmid, -enum resctrl_event_id eventid) +struct arch_mbm_state *get_arch_mbm_state (struct rdt_hw_l3_mon_domain *hw_dom, +u32 rmid, +enum resctrl_event_id eventid) {struct arch_mbm_state *state;
@@ -209,7 +209,7 @@ void resctrl_arch_reset_rmid_all(struct rdt_resource *r, struct rdt_l3_mon_domai } } -static u64 mbm_overflow_count (u64 prev_val,u64 cur_val,unsigned int width) +u64 mbm_overflow_count (u64 prev_val,u64 cur_val,unsigned int width) {u64 shift =64 - width, chunks;
include/linux/resctrl_types.h +5 -0
@@ -77,6 +77,11 @@ enum resctrl_event_id { QOS_NUM_EVENTS, }; +static inline bool rmbm_event (unsigned int e) +{ +return (e >=QOS_L3_MBM_R0_EVENT_ID ) && (e <=QOS_L3_MBM_R3_EVENT_ID ); +} +#define QOS_NUM_L3_MBM_EVENTS (QOS_L3_MBM_R3_EVENT_ID - QOS_L3_MBM_TOTAL_EVENT_ID + 1)#define MBM_STATE_IDX (evt) ((evt) - QOS_L3_MBM_TOTAL_EVENT_ID)
Signature · 1 line
2.43.0
RFC PATCH 19/31 x86/resctrl: Enable the region based events by adding them into the event
In reply to Chen YuThe following MBM events will be displayed under each domain:
[root@ mon_data]# tree
.
├── mon_L3_00
│ ├── llc_occupancy
│ ├── mbm_region0_bytes
│ └── mbm_region1_bytes
├── mon_L3_01
│ ├── llc_occupancy
│ ├── mbm_region0_bytes
│ └── mbm_region1_bytes
├── mon_L3_02
│ ├── llc_occupancy
│ ├── mbm_region0_bytes
│ └── mbm_region1_bytes
└── mon_L3_03
├── llc_occupancy
├── mbm_region0_bytes
└── mbm_region1_bytesNote, the actual maximum number of available memory regions is provided by the MRRM table. Suppose the maximum number of available memory regions exposed by MRRM is 2; then each domain can access at most 2 regions.
Region-aware MBM counters live in MMIO rather than in the QM_CTR MSR, so resctrl_arch_reset_rmid() must read the initial counter value through erdt_mon_read() instead of __rmid_read_phys(). Add a "first" parameter to tell erdt_read_region_mbm() which kind of read this is: when set, only record the raw counter value in am->prev_mon_val so that later reads can compute a delta from it, without accumulating that absolute value into am->chunks as if it had been consumed.
--- arch/x86/kernel/cpu/resctrl/core.c | 8 +++++ arch/x86/kernel/cpu/resctrl/erdt.c | 48 +++++++++++++++++++++++--- arch/x86/kernel/cpu/resctrl/internal.h | 4 ++- arch/x86/kernel/cpu/resctrl/monitor.c | 14 ++++++-- fs/resctrl/monitor.c | 4 +++ include/linux/resctrl_types.h | 2 ++ 6 files changed, 71 insertions(+), 9 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +8 -0
@@ -1081,6 +1081,13 @@ static __init bool get_rdt_mon_resources(void)resctrl_enable_mon_event (QOS_L3_OCCUP_EVENT_ID , false,0 ,NULL ); ret = true; } + +if (erdt_enable_mon ()) { + ret = true; + + goto done; + } +if (rdt_cpu_has (X86_FEATURE_CQM_MBM_TOTAL )) {resctrl_enable_mon_event (QOS_L3_MBM_TOTAL_EVENT_ID , false,0 ,NULL ); ret = true;
@@ -1092,6 +1099,7 @@ static __init bool get_rdt_mon_resources(void)if (rdt_cpu_has (X86_FEATURE_ABMC )) ret = true; +done:if (!ret)return false;
arch/x86/kernel/cpu/resctrl/erdt.c +43 -5
88 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +3 -1
@@ -315,8 +315,10 @@ static inline bool intel_handle_aet_option(bool force_off, char *tok) { return fbool erdt_support (int flag);bool erdt_cpu_has (int flag); +bool erdt_enable_mon (void );int erdt_get_max_rmid (void ); -int erdt_mon_read (struct rdt_domain_hdr *hdr,enum resctrl_event_id evtid,u32 rmid,u64 *val); +int erdt_mon_read (struct rdt_domain_hdr *hdr,enum resctrl_event_id evtid,u32 rmid, +u64 *val,bool first);int erdt_init (void );void erdt_exit (void );
arch/x86/kernel/cpu/resctrl/monitor.c +11 -3
@@ -179,14 +179,19 @@ void resctrl_arch_reset_rmid(struct rdt_resource *r, struct rdt_l3_mon_domain *dint cpu =cpumask_any (&d->hdr.cpu_mask);struct arch_mbm_state *am;u32 prmid; +u64 tmp; am =get_arch_mbm_state (hw_dom, rmid, eventid);if (am) {memset (am,0 ,sizeof (*am)); - prmid =logical_rmid_to_physical_rmid (cpu, rmid);/* Record any initial, non-zero count value. */ -__rmid_read_phys (prmid, eventid, &am->prev_mon_val); +if (rmbm_event (eventid) &&erdt_cpu_has (X86_FEATURE_CQM_MBM_TOTAL )) { +erdt_mon_read (&d->hdr, eventid, rmid, &tmp, true); + }else { + prmid =logical_rmid_to_physical_rmid (cpu, rmid); +__rmid_read_phys (prmid, eventid, &am->prev_mon_val); + } } }
@@ -280,7 +285,10 @@ int resctrl_arch_rmid_read(struct rdt_resource *r, struct rdt_domain_hdr *hdr,case RDT_RESOURCE_L3 :if (eventid ==QOS_L3_OCCUP_EVENT_ID &&erdt_cpu_has (X86_FEATURE_CQM_OCCUP_LLC )) -return erdt_mon_read (hdr, eventid, rmid, val); +return erdt_mon_read (hdr, eventid, rmid, val, false); + +if (rmbm_event (eventid) &&erdt_cpu_has (X86_FEATURE_CQM_MBM_TOTAL )) +return erdt_mon_read (hdr, eventid, rmid, val, false);return arch_l3_read_event (hdr, rmid, eventid, val, r);case RDT_RESOURCE_PERF_PKG :
fs/resctrl/monitor.c +4 -0
@@ -1031,6 +1031,10 @@ struct mon_evt mon_event_all[QOS_NUM_EVENTS] = {MON_EVENT (PMT_EVENT_AUTO_C6_RES ,"c6_res" ,RDT_RESOURCE_PERF_PKG , false),MON_EVENT (PMT_EVENT_UNHALTED_REF_CYCLES ,"unhalted_ref_cycles" ,RDT_RESOURCE_PERF_PKG , false),MON_EVENT (PMT_EVENT_UOPS_RETIRED ,"uops_retired" ,RDT_RESOURCE_PERF_PKG , false), +MON_EVENT (QOS_L3_MBM_R0_EVENT_ID ,"mbm_region0_bytes" ,RDT_RESOURCE_L3 , false), +MON_EVENT (QOS_L3_MBM_R1_EVENT_ID ,"mbm_region1_bytes" ,RDT_RESOURCE_L3 , false), +MON_EVENT (QOS_L3_MBM_R2_EVENT_ID ,"mbm_region2_bytes" ,RDT_RESOURCE_L3 , false), + MON_EVENT(QOS_L3_MBM_R3_EVENT_ID, "mbm_region3_bytes", RDT_RESOURCE_L3, false), }; bool resctrl_enable_mon_event(enum resctrl_event_id eventid, bool any_cpu,
include/linux/resctrl_types.h +2 -0
@@ -82,6 +82,8 @@ static inline bool rmbm_event(unsigned int e)return (e >=QOS_L3_MBM_R0_EVENT_ID ) && (e <=QOS_L3_MBM_R3_EVENT_ID ); } +#define RMBM_STATE_IDX (evt) ((evt) - QOS_L3_MBM_R0_EVENT_ID) +#define QOS_NUM_L3_MBM_EVENTS (QOS_L3_MBM_R3_EVENT_ID - QOS_L3_MBM_TOTAL_EVENT_ID + 1)#define MBM_STATE_IDX (evt) ((evt) - QOS_L3_MBM_TOTAL_EVENT_ID)
Signature · 1 line
2.43.0
RFC PATCH 20/31 x86/resctrl: Rename msr_update to hw_update
In reply to Chen YuPrepare for the region aware RDT hardware update, and the region aware RDT touches the MMIO address rather than MSR registers.
No functional change expected.
--- arch/x86/kernel/cpu/resctrl/core.c | 28 ++++++++--------- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 38 +++++++++++------------ arch/x86/kernel/cpu/resctrl/internal.h | 8 ++--- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 14 ++++----- 4 files changed, 44 insertions(+), 44 deletions(-)
arch/x86/kernel/cpu/resctrl/core.c +14 -14
@@ -50,9 +50,9 @@ DEFINE_PER_CPU(struct resctrl_pqr_state, pqr_state);*/ bool rdt_alloc_capable; -static void mba_wrmsr_intel (struct msr_param *m); -static void cat_wrmsr (struct msr_param *m); -static void mba_wrmsr_amd (struct msr_param *m); +static void mba_wrmsr_intel (struct hw_param *m); +static void cat_wrmsr (struct hw_param *m); +static void mba_wrmsr_amd (struct hw_param *m);#define ctrl_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.controls)#define mon_domain_init (id) LIST_HEAD_INIT(rdt_resources_all[id].r_resctrl.mon_domains)
@@ -226,7 +226,7 @@ static __init bool __get_mem_config_intel(struct rdt_resource *r) hw_ctrl->r_ctrl.scalar.unit =RESCTRL_CTRL_UNIT_ALL ; hw_ctrl->msr_base =MSR_IA32_MBA_THRTL_BASE ; - hw_ctrl->msr_update = mba_wrmsr_intel; + hw_ctrl->hw_update = mba_wrmsr_intel;list_add (&hw_ctrl->r_ctrl.entry, &r->controls); r->alloc_capable = true;
@@ -285,10 +285,10 @@ static __init bool __rdt_get_mem_config_amd(struct rdt_resource *r)if (r->rid ==RDT_RESOURCE_MBA ) { hw_ctrl->msr_base =MSR_IA32_MBA_BW_BASE ; - hw_ctrl->msr_update = mba_wrmsr_amd; + hw_ctrl->hw_update = mba_wrmsr_amd; }else {/* r->rid == RDT_RESOURCE_SMBA */ hw_ctrl->msr_base =MSR_IA32_SMBA_BW_BASE ; - hw_ctrl->msr_update = mba_wrmsr_amd; + hw_ctrl->hw_update = mba_wrmsr_amd; }list_add (&hw_ctrl->r_ctrl.entry, &r->controls);
@@ -339,7 +339,7 @@ static void rdt_get_cache_alloc_cfg(int idx, struct rdt_resource *r) } hw_ctrl->msr_base = idx ==1 ?MSR_IA32_L3_CBM_BASE :MSR_IA32_L2_CBM_BASE ; - hw_ctrl->msr_update = cat_wrmsr; + hw_ctrl->hw_update = cat_wrmsr;list_add (&hw_ctrl->r_ctrl.entry, &r->controls);
@@ -371,7 +371,7 @@ static void rdt_get_cdp_l2_config(void)rdt_get_cdp_config (RDT_RESOURCE_L2 ); } -static void mba_wrmsr_amd (struct msr_param *m) +static void mba_wrmsr_amd (struct hw_param *m) {struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom);struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (m->ctrl);
@@ -395,7 +395,7 @@ static u32 delay_bw_map(unsigned long bw, struct rdt_resource *r)return MAX_MBA_BW ; } -static void mba_wrmsr_intel (struct msr_param *m) +static void mba_wrmsr_intel (struct hw_param *m) {struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom);struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (m->ctrl);
@@ -406,7 +406,7 @@ static void mba_wrmsr_intel(struct msr_param *m)wrmsrq (hw_ctrl->msr_base + i,delay_bw_map (hw_dom->ctrl_val[i], m->res)); } -static void cat_wrmsr (struct msr_param *m) +static void cat_wrmsr (struct hw_param *m) {struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom);struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (m->ctrl);
@@ -424,10 +424,10 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *r)void rdt_ctrl_update (void *arg) {struct resctrl_hw_ctrl *hw_ctrl; -struct msr_param *m = arg; +struct hw_param *m = arg; hw_ctrl =resctrl_to_arch_ctrl (m->ctrl); - hw_ctrl->msr_update (m); + hw_ctrl->hw_update (m); } static void setup_default_ctrlval(struct rdt_resource *r, struct resctrl_ctrl *ctrl,
@@ -466,7 +466,7 @@ static int domain_setup_ctrlval(struct rdt_resource *r, struct resctrl_ctrl *ctrstruct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (ctrl);struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); -struct msr_param m; +struct hw_param m;u32 *dc; dc = kmalloc_array(hw_res->num_closid, sizeof(*hw_dom->ctrl_val),
@@ -482,7 +482,7 @@ static int domain_setup_ctrlval(struct rdt_resource *r, struct resctrl_ctrl *ctr m.dom = d; m.low =0 ; m.high = hw_res->num_closid; - hw_ctrl->msr_update (&m); + hw_ctrl->hw_update (&m);return 0 ; }
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +19 -19
@@ -26,19 +26,19 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct resctrl_ctrl *ctrl,struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (d);struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (ctrl);u32 idx =resctrl_get_config_index (closid, t); -struct msr_param msr_param; +struct hw_param hw_param;if (!cpumask_test_cpu (smp_processor_id (), &d->hdr.cpu_mask))return -EINVAL ; hw_dom->ctrl_val[idx] = cfg_val; - msr_param.res = r; - msr_param.ctrl = ctrl; - msr_param.dom = d; - msr_param.low = idx; - msr_param.high = idx +1 ; - hw_ctrl->msr_update (&msr_param); + hw_param.res = r; + hw_param.ctrl = ctrl; + hw_param.dom = d; + hw_param.low = idx; + hw_param.high = idx +1 ; + hw_ctrl->hw_update (&hw_param);return 0 ; }
@@ -48,7 +48,7 @@ static void _resctrl_arch_update_domains(struct rdt_resource *r, {struct resctrl_staged_config *cfg;struct rdt_hw_ctrl_domain *hw_dom; -struct msr_param msr_param; +struct hw_param hw_param;struct rdt_ctrl_domain *d;enum resctrl_conf_type t;u32 idx;
@@ -56,10 +56,10 @@ static void _resctrl_arch_update_domains(struct rdt_resource *r,/* Walking ctrl->domains, ensure it can't race with cpuhp */ lockdep_assert_cpus_held (); - msr_param.ctrl = ctrl; + hw_param.ctrl = ctrl;list_for_each_entry (d, &ctrl->domains, hdr.list) { hw_dom =resctrl_to_arch_ctrl_dom (d); - msr_param.res =NULL ; + hw_param.res =NULL ;for (t =0 ; t <CDP_NUM_TYPES ; t++) { cfg = &hw_dom->d_resctrl.staged_config[t];if (!cfg->have_new_ctrl)
@@ -70,18 +70,18 @@ static void _resctrl_arch_update_domains(struct rdt_resource *r,continue ; hw_dom->ctrl_val[idx] = cfg->new_ctrl; -if (!msr_param.res) { - msr_param.low = idx; - msr_param.high = msr_param.low +1 ; - msr_param.res = r; - msr_param.dom = d; +if (!hw_param.res) { + hw_param.low = idx; + hw_param.high = hw_param.low +1 ; + hw_param.res = r; + hw_param.dom = d; }else { - msr_param.low =min (msr_param.low, idx); - msr_param.high =max (msr_param.high, idx +1 ); + hw_param.low =min (hw_param.low, idx); + hw_param.high =max (hw_param.high, idx +1 ); } } -if (msr_param.res) -smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, &msr_param,1 ); +if (hw_param.res) +smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, &hw_param,1 ); } }
arch/x86/kernel/cpu/resctrl/internal.h +4 -4
@@ -88,14 +88,14 @@ struct arch_mbm_state {#define SDCIAE_ENABLE_BIT 1/** -* struct msr_param - set a range of MSRs from a domain +* struct hw_param - set a range of registers (MSRs, etc) from a domain * @res: The resource to use * @ctrl: Properties of the control being updated * @dom: The domain to update * @low: Beginning index from base MSR * @high: End index */ -struct msr_param { +struct hw_param {struct rdt_resource *res;struct resctrl_ctrl *ctrl;struct rdt_ctrl_domain *dom;
@@ -107,12 +107,12 @@ struct msr_param {* struct resctrl_hw_ctrl - Arch private properties of a resource control * @r_ctrl: Control properties exposed to resctrl file system * @msr_base: Base MSR address where control values should be programmed -* @msr_update: Function pointer to update control values +* @hw_update: Function pointer to update QOS hardware, MSRs, etc */ struct resctrl_hw_ctrl {struct resctrl_ctrl r_ctrl;unsigned int msr_base; -void (*msr_update)(struct msr_param *m); +void (*hw_update)(struct hw_param *m); };static inline struct resctrl_hw_ctrl *resctrl_to_arch_ctrl (struct resctrl_ctrl *c)
arch/x86/kernel/cpu/resctrl/rdtgroup.c +7 -7
@@ -234,7 +234,7 @@ void resctrl_arch_reset_all_ctrls(struct rdt_resource *r) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r);struct rdt_hw_ctrl_domain *hw_dom; -struct msr_param msr_param; +struct hw_param hw_param;struct resctrl_ctrl *ctrl;struct rdt_ctrl_domain *d;int i;
@@ -242,9 +242,9 @@ void resctrl_arch_reset_all_ctrls(struct rdt_resource *r)/* Walking ctrl->domains, ensure it can't race with cpuhp */ lockdep_assert_cpus_held (); - msr_param.res = r; - msr_param.low =0 ; - msr_param.high = hw_res->num_closid; + hw_param.res = r; + hw_param.low =0 ; + hw_param.high = hw_res->num_closid;/* * Disable resource control for this resource by setting all
@@ -252,14 +252,14 @@ void resctrl_arch_reset_all_ctrls(struct rdt_resource *r)* Pick one CPU from each domain to update the MSRs below. */ for_each_resource_ctrl (ctrl, r) { - msr_param.ctrl = ctrl; + hw_param.ctrl = ctrl;list_for_each_entry (d, &ctrl->domains, hdr.list) { hw_dom =resctrl_to_arch_ctrl_dom (d);for (i =0 ; i < hw_res->num_closid; i++) hw_dom->ctrl_val[i] =resctrl_get_default_ctrlval (ctrl); - msr_param.dom = d; -smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, &msr_param,1 ); + hw_param.dom = d; +smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, &hw_param,1 ); } }
Signature · 1 line
2.43.0
RFC PATCH 21/31 x86/resctrl: Parse ACPI MARC table
In reply to Chen YuThe MARC table is used to provide Memory Bandwidth Allocation Register information. Its register addresses are mapped, and the MARC pointer is saved in the corresponding domain information entry.
--- arch/x86/kernel/cpu/resctrl/erdt.c | 70 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 10 +++- 2 files changed, 79 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/erdt.c +70 -0
@@ -25,8 +25,13 @@ static bool erdt_enabled;#define ERDT_VALID_VERSION 1#define CMRC_SUPPORTED_INDEX_FN 1#define MMRC_SUPPORTED_INDEX_FN 1 +#define MARC_SUPPORTED_INDEX_FN 1#define RMDD_FLAG_CPU_L3_DOMAIN BIT(0) +#define MARC_FLAG_OPT BIT(0) +#define MARC_FLAG_MIN BIT(1) +#define MARC_FLAG_MAX BIT(2) +/* Set in a monitoring counter when it holds no valid data to report. */ #define UNAVAILABLE_COUNTER BIT_ULL(63)#define CMRC_FLAG_UNAVAILABLE_BIT BIT(0)
@@ -283,6 +288,7 @@ static void cleanup_one_domain(struct erdt_domain_info *d)erdt_iounmap_domain (d);kfree (d->cmrc);kfree (d->mmrc); +kfree (d->marc);kfree (d); }
@@ -396,6 +402,64 @@ static __init int mmrc_init(struct acpi_subtbl_hdr_16 *subtbl,return 0 ; } +static __init int marc_init (struct acpi_subtbl_hdr_16 *subtbl, +struct erdt_domain_info *domain_info) +{ +struct acpi_erdt_marc *marc = (struct acpi_erdt_marc *)subtbl; + +if (subtbl->length <sizeof (*marc)) { +pr_warn (FW_BUG "Truncated MARC subtable\n" ); +return -EIO ; + } + +if (marc->index_fn !=MARC_SUPPORTED_INDEX_FN ) { +pr_info ("Unsupported MARC index function %d\n" , marc->index_fn); +return -EIO ; + } + +if (marc->flags &MARC_FLAG_OPT ) { + domain_info->base[ERDT_MMIO_MARC_OPT ] = +erdt_ioremap (marc->reg_base_opt, marc->mba_reg_size,"MARC OPT base" ); +if (!domain_info->base[ERDT_MMIO_MARC_OPT ]) +return -EIO ; + } + +if (marc->flags &MARC_FLAG_MIN ) { + domain_info->base[ERDT_MMIO_MARC_MIN ] = +erdt_ioremap (marc->reg_base_min, marc->mba_reg_size,"MARC MIN base" ); +if (!domain_info->base[ERDT_MMIO_MARC_MIN ]) + goto unmap; + } + +if (marc->flags &MARC_FLAG_MAX ) { + domain_info->base[ERDT_MMIO_MARC_MAX ] = +erdt_ioremap (marc->reg_base_max, marc->mba_reg_size,"MARC MAX base" ); +if (!domain_info->base[ERDT_MMIO_MARC_MAX ]) + goto unmap; + } + + domain_info->marc =kmemdup (marc, subtbl->length,GFP_KERNEL ); +if (!domain_info->marc) + goto unmap; + +return 0 ; + +unmap: +if (domain_info->base[ERDT_MMIO_MARC_OPT ]) { +iounmap (domain_info->base[ERDT_MMIO_MARC_OPT ]); + domain_info->base[ERDT_MMIO_MARC_OPT ] =NULL ; + } +if (domain_info->base[ERDT_MMIO_MARC_MIN ]) { +iounmap (domain_info->base[ERDT_MMIO_MARC_MIN ]); + domain_info->base[ERDT_MMIO_MARC_MIN ] =NULL ; + } +if (domain_info->base[ERDT_MMIO_MARC_MAX ]) { +iounmap (domain_info->base[ERDT_MMIO_MARC_MAX ]); + domain_info->base[ERDT_MMIO_MARC_MAX ] =NULL ; + } +return -EIO ; +} +static inline struct acpi_subtbl_hdr_16 *rmdd_subtbl (struct acpi_erdt_rmdd *rmdd) {return (void *)rmdd +sizeof (*rmdd);
@@ -478,6 +542,12 @@ static __init bool parse_rmdd_table(struct acpi_subtbl_hdr_16 *rmdd_hdr) !mmrc_init(subtbl, domain_info)) subtbl_mask |=BIT (ACPI_ERDT_TYPE_MMRC ); +break ; +case ACPI_ERDT_TYPE_MARC : +if (!(subtbl_mask &BIT (ACPI_ERDT_TYPE_MARC )) && + !marc_init (subtbl, domain_info)) + subtbl_mask |=BIT (ACPI_ERDT_TYPE_MARC ); +break ;default :break ;
arch/x86/kernel/cpu/resctrl/internal.h +9 -1
@@ -26,12 +26,18 @@* @ERDT_MMIO_RMDD_CREG: RMDD control register base address * @ERDT_MMIO_CMRC_BASE: CMRC monitoring register base address * @ERDT_MMIO_MMRC_BASE: MMRC monitoring register base address +* @ERDT_MMIO_MARC_OPT: MARC optimal BW register base address +* @ERDT_MMIO_MARC_MIN: MARC minimum BW register base address +* @ERDT_MMIO_MARC_MAX: MARC maximum BW register base address */ enum erdt_mmio_type {ERDT_MMIO_RMDD_CREG ,ERDT_MMIO_CMRC_BASE ,ERDT_MMIO_MMRC_BASE , -ERDT_MMIO_LAST =ERDT_MMIO_MMRC_BASE +ERDT_MMIO_MARC_OPT , +ERDT_MMIO_MARC_MIN , +ERDT_MMIO_MARC_MAX , +ERDT_MMIO_LAST =ERDT_MMIO_MARC_MAX };#define ERDT_MMIO_NUM_TYPES (ERDT_MMIO_LAST + 1)
@@ -41,6 +47,7 @@ enum erdt_mmio_type {* @base: Array of ioremapped MMIO region base addresses, indexed by ERDT_MMIO_* type * @cmrc: Copy of the ACPI CMRC sub-table for this domain * @mmrc: Copy of the ACPI MMRC sub-table for this domain +* @marc: Copy of the ACPI MARC sub-table for this domain * @cpu_mask: CPUs belonging to this resource management domain * @max_rmid: Maximum RMID supported by this domain * @dom_id: L3 cache ID shared by all CPUs in this domain (-1 if unset)
@@ -50,6 +57,7 @@ struct erdt_domain_info {void __iomem *base[ERDT_MMIO_NUM_TYPES ];struct acpi_erdt_cmrc *cmrc;struct acpi_erdt_mmrc *mmrc; +struct acpi_erdt_marc *marc;struct cpumask cpu_mask;u32 max_rmid;int dom_id;
Signature · 1 line
2.43.0
RFC PATCH 22/31 fs/resctrl: Add region-based control names and resctrl_ctrl_name_region()
In reply to Chen YuERDT exposes per-memory-region bandwidth allocation controls, where each region has an optimal (OPT), minimum (MIN) and maximum (MAX) control.
Extend enum resctrl_ctrl_name with the REGION<n>_{OPT,MIN,MAX} names (and their string forms). These enumerators are only assigned to a control when arch code adds a region control to rdt_resource::controls, which happens only when the hardware supports ERDT (erdt_cpu_has()); on other platforms they are simply unused.
Add resctrl_ctrl_name_region() so that the architecture code can retrieve the control name for a given region and control type when creating the corresponding controllers.
--- fs/resctrl/ctrlmondata.c | 12 ++++++++++++ include/linux/resctrl.h | 29 ++++++++++++++++++++++++++++- 2 files changed, 40 insertions(+), 1 deletion(-)
fs/resctrl/ctrlmondata.c +12 -0
@@ -304,6 +304,18 @@ static const char * const resctrl_ctrl_name[] = { [RESCTRL_CTRL_NAME_DEF] = "", [RESCTRL_CTRL_NAME_MIN] = "MIN", [RESCTRL_CTRL_NAME_MAX] = "MAX", + [RESCTRL_CTRL_NAME_REGION0_OPT] = "REGION0_OPT", + [RESCTRL_CTRL_NAME_REGION0_MIN] = "REGION0_MIN", + [RESCTRL_CTRL_NAME_REGION0_MAX] = "REGION0_MAX", + [RESCTRL_CTRL_NAME_REGION1_OPT] = "REGION1_OPT", + [RESCTRL_CTRL_NAME_REGION1_MIN] = "REGION1_MIN", + [RESCTRL_CTRL_NAME_REGION1_MAX] = "REGION1_MAX", + [RESCTRL_CTRL_NAME_REGION2_OPT] = "REGION2_OPT", + [RESCTRL_CTRL_NAME_REGION2_MIN] = "REGION2_MIN", + [RESCTRL_CTRL_NAME_REGION2_MAX] = "REGION2_MAX", + [RESCTRL_CTRL_NAME_REGION3_OPT] = "REGION3_OPT", + [RESCTRL_CTRL_NAME_REGION3_MIN] = "REGION3_MIN", + [RESCTRL_CTRL_NAME_REGION3_MAX] = "REGION3_MAX", };const char *resctrl_ctrl_name_str (enum resctrl_ctrl_name name)
include/linux/resctrl.h +28 -1
@@ -348,14 +348,41 @@ struct resctrl_mon {* has the same name as the resource. * @RESCTRL_CTRL_NAME_MIN: "MIN" * @RESCTRL_CTRL_NAME_MAX: "MAX" +* +* The REGION* names describe the region based bandwidth allocation controls of +* each memory region has an optimal (OPT), minimum (MIN) andcmaximum (MAX) control. +* They are only ever assigned to a control when arch code adds a region control +* to rdt_resource::controls, which happens only when the hardware supports ERDT +* (erdt_cpu_has()); on other platforms these enumerators are simply unused. */ enum resctrl_ctrl_name {RESCTRL_CTRL_NAME_DEF ,RESCTRL_CTRL_NAME_MIN ,RESCTRL_CTRL_NAME_MAX , -RESCTRL_CTRL_NAME_LAST =RESCTRL_CTRL_NAME_MAX +RESCTRL_CTRL_NAME_REGION0_OPT , +RESCTRL_CTRL_NAME_REGION0_MIN , +RESCTRL_CTRL_NAME_REGION0_MAX , +RESCTRL_CTRL_NAME_REGION1_OPT , +RESCTRL_CTRL_NAME_REGION1_MIN , +RESCTRL_CTRL_NAME_REGION1_MAX , +RESCTRL_CTRL_NAME_REGION2_OPT , +RESCTRL_CTRL_NAME_REGION2_MIN , +RESCTRL_CTRL_NAME_REGION2_MAX , +RESCTRL_CTRL_NAME_REGION3_OPT , +RESCTRL_CTRL_NAME_REGION3_MIN , +RESCTRL_CTRL_NAME_REGION3_MAX , +RESCTRL_CTRL_NAME_LAST =RESCTRL_CTRL_NAME_REGION3_MAX }; +#define RESCTRL_CTRL_REGION_NR_CTRLS 3 + +static inline enum resctrl_ctrl_name +resctrl_ctrl_name_region (unsigned int region,unsigned int type) +{ +return RESCTRL_CTRL_NAME_REGION0_OPT + + region *RESCTRL_CTRL_REGION_NR_CTRLS + type; +} +/* Flags for rdt_resource::flags and resctrl_ctrl::flags */ #define RESCTRL_CTRL_FLAG_LINEAR BIT(0)#define RESCTRL_CTRL_FLAG_SPARSE_BITMASKS BIT(1)
Signature · 1 line
2.43.0
RFC PATCH 23/31 x86/resctrl: Add region aware MBA controllers
In reply to Chen YuAdd region aware MBA controllers for ERDT platforms that have the MARC (Memory-bandwidth Allocation Registers for CPU Agents) ACPI subtable.
For each memory region reported by MRRM, create OPT, MIN, and MAX controllers (as indicated by MARC flags) using resctrl_ctrl_name_region(). The controller properties (bandwidth range, granularity, unit) are derived from the MARC subtable fields.
The hw_update callback is a stub for now; the actual MMIO write implementation will follow in a subsequent patch.
--- arch/x86/kernel/cpu/resctrl/core.c | 3 ++ arch/x86/kernel/cpu/resctrl/erdt.c | 67 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 5 ++ 3 files changed, 75 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +3 -0
@@ -1063,6 +1063,9 @@ static __init bool get_rdt_alloc_resources(void)if (get_mem_config ()) ret = true; +if (erdt_get_mem_config (&rdt_resources_all[RDT_RESOURCE_MBA ].r_resctrl)) + ret = true; +if (get_slow_mem_config ()) ret = true;
arch/x86/kernel/cpu/resctrl/erdt.c +67 -0
91 lines not on this page · raw message
arch/x86/kernel/cpu/resctrl/internal.h +5 -0
@@ -30,6 +30,10 @@* @ERDT_MMIO_MARC_MIN: MARC minimum BW register base address * @ERDT_MMIO_MARC_MAX: MARC maximum BW register base address */ +#define RESCTRL_CTRL_REGION_TYPE_OPT 0 +#define RESCTRL_CTRL_REGION_TYPE_MIN 1 +#define RESCTRL_CTRL_REGION_TYPE_MAX 2 +enum erdt_mmio_type {ERDT_MMIO_RMDD_CREG ,ERDT_MMIO_CMRC_BASE ,
@@ -324,6 +328,7 @@ static inline bool intel_handle_aet_option(bool force_off, char *tok) { return fbool erdt_support (int flag);bool erdt_cpu_has (int flag);bool erdt_enable_mon (void ); +bool __init erdt_get_mem_config (struct rdt_resource *r);int erdt_get_max_rmid (void );int erdt_mon_read (struct rdt_domain_hdr *hdr,enum resctrl_event_id evtid,u32 rmid,u64 *val,bool first);
Signature · 1 line
2.43.0
RFC PATCH 24/31 x86/resctrl: Attach ACPI ERDT information to ctrl domain on CPU online
In reply to Chen YuAfter the rdt_hw_ctrl_domain has been created during CPU online, attach the pre-parsed ACPI ERDT table information to the rdt_hw_ctrl_domain to facilitate MBA control programming via the MARC MMIO registers.
This mirrors the approach used by the monitoring domain (rdt_hw_l3_mon_domain::d_info) where ERDT domain information is attached during domain creation.
Add erdt_find_domain_info() to look up the erdt_domain_info for a given CPU by matching against the CACD-derived cpu_mask in each domain.
--- arch/x86/kernel/cpu/resctrl/core.c | 1 + arch/x86/kernel/cpu/resctrl/erdt.c | 12 ++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 3 +++ 3 files changed, 16 insertions(+)
arch/x86/kernel/cpu/resctrl/core.c +1 -0
@@ -575,6 +575,7 @@ static void domain_add_cpu_ctrl(int cpu, struct rdt_resource *r, d->hdr.type =RESCTRL_CTRL_DOMAIN ; d->hdr.rid = r->rid;cpumask_set_cpu (cpu, &d->hdr.cpu_mask); + hw_dom->d_info =erdt_find_domain_info (cpu);rdt_domain_reconfigure_cdp (r);
arch/x86/kernel/cpu/resctrl/erdt.c +12 -0
@@ -261,6 +261,18 @@ int erdt_mon_read(struct rdt_domain_hdr *hdr, enum resctrl_event_id evtid, u32 rreturn -EIO ; } +struct erdt_domain_info *erdt_find_domain_info (int cpu) +{ +struct erdt_domain_info *d; + +list_for_each_entry (d, &domain_info_list, entry) { +if (cpumask_test_cpu (cpu, &d->cpu_mask)) +return d; + } + +return NULL ; +} +static void marc_hw_update (struct hw_param *m) { }
arch/x86/kernel/cpu/resctrl/internal.h +3 -0
@@ -137,12 +137,14 @@ static inline struct resctrl_hw_ctrl *resctrl_to_arch_ctrl(struct resctrl_ctrl ** a resource for a control function * @d_resctrl: Properties exposed to the resctrl file system * @ctrl_val: array of cache or mem ctrl values (indexed by CLOSID) +* @d_info: ERDT table information of this domain (read-only) * * Members of this structure are accessed via helpers that provide abstraction. */ struct rdt_hw_ctrl_domain {struct rdt_ctrl_domain d_resctrl;u32 *ctrl_val; +struct erdt_domain_info *d_info; };/**
@@ -325,6 +327,7 @@ static inline void intel_aet_mon_domain_setup(int cpu, int id, struct rdt_resourstatic inline bool intel_handle_aet_option (bool force_off,char *tok) {return false; }#endif +struct erdt_domain_info *erdt_find_domain_info (int cpu);bool erdt_support (int flag);bool erdt_cpu_has (int flag);bool erdt_enable_mon (void );
Signature · 1 line
2.43.0
RFC PATCH 25/31 x86/resctrl: Introduce region-based MBA write implementation on MMIO space
In reply to Chen YuImplement the marc_hw_update() callback to program per-region MBA bandwidth values via MMIO.
The MARC register layout packs 4 regions into each 64-bit register, with a 9-bit bandwidth field per region at (region % 4) * 16 bits. The MMIO address for a given CLOSID is calculated using the MARC index function 1:
addr = base + Floor(Region / 4) * 512 + CLOS * 8
Each controller handles a single (region, type) pair, but a write always touches the whole 64-bit register. The callback therefore does a
read-modify-write: it changes only its own region's field and writes every other bit back exactly as it was read, so the reserved bits keep their original values. A shadow copy of the register (marc_buf) holds the value last written, which spares the read on later updates.
--- arch/x86/kernel/cpu/resctrl/erdt.c | 52 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 4 ++ 2 files changed, 56 insertions(+)
arch/x86/kernel/cpu/resctrl/erdt.c +52 -0
@@ -275,6 +275,49 @@ struct erdt_domain_info *erdt_find_domain_info(int cpu)static void marc_hw_update (struct hw_param *m) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom); +struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (m->ctrl); +enum resctrl_ctrl_name name = hw_ctrl->r_ctrl.name; +unsigned int offset, region, type, region_offset_bits; +struct erdt_domain_info *d = hw_dom->d_info; +enum erdt_mmio_type mmio_type; +void __iomem *addr; +int closid_idx; +unsigned int i; +u64 val; + +if (!d || !d->marc) +return ; + + offset = name -RESCTRL_CTRL_NAME_REGION0_OPT ; + region = offset /RESCTRL_CTRL_REGION_NR_CTRLS ; + type = offset %RESCTRL_CTRL_REGION_NR_CTRLS ; + mmio_type =ERDT_MMIO_MARC_OPT + type; + region_offset_bits = (region %4 ) *16 ; + +if (d->marc_buf_type != mmio_type) { +memset (d->marc_buf,0 , + d->marc->mba_reg_size *512 *sizeof (u64)); + d->marc_buf_type = mmio_type; + } + +for (i = m->low; i < m->high; i++) { + closid_idx = (region /4 ) *64 + i; + addr = d->base[mmio_type] + closid_idx *8 ; + +/* The cached value retains the reserved bits to be preserved. */ + val = d->marc_buf[closid_idx]; +if (!val) + val =readq (addr); + +if (WARN_ON_ONCE (!val)) +return ; + + val &= ~(0x1ffULL << region_offset_bits); + val |= (u64)(hw_dom->ctrl_val[i] &0x1ff ) << region_offset_bits; + d->marc_buf[closid_idx] = val; +writeq (val, addr); + } }__init bool erdt_get_mem_config (struct rdt_resource *r)
@@ -364,6 +407,7 @@ static void erdt_iounmap_domain(struct erdt_domain_info *domain)static void cleanup_one_domain (struct erdt_domain_info *d) {erdt_iounmap_domain (d); +kfree (d->marc_buf);kfree (d->cmrc);kfree (d->mmrc);kfree (d->marc);
@@ -520,6 +564,14 @@ static __init int marc_init(struct acpi_subtbl_hdr_16 *subtbl,if (!domain_info->marc) goto unmap; + domain_info->marc_buf =kcalloc (marc->mba_reg_size *512 ,sizeof (u64), +GFP_KERNEL ); +if (!domain_info->marc_buf) { +kfree (domain_info->marc); + domain_info->marc =NULL ; + goto unmap; + } +return 0 ; unmap:
arch/x86/kernel/cpu/resctrl/internal.h +4 -0
@@ -52,6 +52,8 @@ enum erdt_mmio_type {* @cmrc: Copy of the ACPI CMRC sub-table for this domain * @mmrc: Copy of the ACPI MMRC sub-table for this domain * @marc: Copy of the ACPI MARC sub-table for this domain +* @marc_buf: Shadow copy of the MARC MMIO registers for read-modify-write +* @marc_buf_type: MARC MMIO region currently cached in @marc_buf * @cpu_mask: CPUs belonging to this resource management domain * @max_rmid: Maximum RMID supported by this domain * @dom_id: L3 cache ID shared by all CPUs in this domain (-1 if unset)
@@ -62,6 +64,8 @@ struct erdt_domain_info {struct acpi_erdt_cmrc *cmrc;struct acpi_erdt_mmrc *mmrc;struct acpi_erdt_marc *marc; +u64 *marc_buf; +enum erdt_mmio_type marc_buf_type;struct cpumask cpu_mask;u32 max_rmid;int dom_id;
Signature · 1 line
2.43.0
RFC PATCH 26/31 x86/resctrl: Allow control writes from any CPU for MMIO controllers
In reply to Chen YuProgramming a control that lives in domain-local MSR space must run on a CPU that belongs to the target domain, so resctrl_arch_update_domains() issues the hw_update() via an IPI. MMIO based controls (e.g. the ERDT MARC registers) have no such affinity requirement and can be programmed from the current CPU, making the IPI pure overhead.
Introduce RESCTRL_CTRL_FLAG_ANY_CPU to describe a control whose hardware can be programmed from any CPU, mirroring the mon_evt::any_cpu flag used on the monitoring read path. Set it on the MARC controllers at creation time.
--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 11 ++++++++++- arch/x86/kernel/cpu/resctrl/erdt.c | 2 ++ include/linux/resctrl.h | 1 + 3 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +10 -1
@@ -19,6 +19,15 @@#include "internal.h" +static void ctrl_hw_update (struct resctrl_ctrl *ctrl,struct rdt_ctrl_domain *d, +struct hw_param *m) +{ +if (ctrl->flags &RESCTRL_CTRL_FLAG_ANY_CPU ) +rdt_ctrl_update (m); +else +smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, m,1 ); +} +int resctrl_arch_update_one (struct rdt_resource *r,struct resctrl_ctrl *ctrl,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val)
@@ -81,7 +90,7 @@ static void _resctrl_arch_update_domains(struct rdt_resource *r, } }if (hw_param.res) -smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, &hw_param,1 ); +ctrl_hw_update (ctrl, d, &hw_param); } }
arch/x86/kernel/cpu/resctrl/erdt.c +2 -0
@@ -369,6 +369,8 @@ __init bool erdt_get_mem_config(struct rdt_resource *r) hw_ctrl->r_ctrl.scalar.unit =RESCTRL_CTRL_UNIT_ALL ; hw_ctrl->hw_update = marc_hw_update; +/* MARC controls live in MMIO, program from any CPU. */ + hw_ctrl->r_ctrl.flags |=RESCTRL_CTRL_FLAG_ANY_CPU ;list_add_tail (&hw_ctrl->r_ctrl.entry, &r->controls); } }
include/linux/resctrl.h +1 -0
@@ -386,6 +386,7 @@ resctrl_ctrl_name_region(unsigned int region, unsigned int type)/* Flags for rdt_resource::flags and resctrl_ctrl::flags */ #define RESCTRL_CTRL_FLAG_LINEAR BIT(0)#define RESCTRL_CTRL_FLAG_SPARSE_BITMASKS BIT(1) +#define RESCTRL_CTRL_FLAG_ANY_CPU BIT(2)/** * struct resctrl_ctrl - A resource control
Signature · 1 line
2.43.0
RFC PATCH 27/31 x86/resctrl: Enable region-aware MBM/MBA via the RDT_CTRL register
In reply to Chen YuRegion-aware MBM and MBA are exposed through per-RMDD MMIO register blocks. These MMIO interfaces are only honored by hardware once the domain is taken out of legacy "Total" mode.
Clear the legacy-mode bit once an RMDD's region-aware ACPI sub-tables have parsed successfully, so the switch to region-aware mode only happens for fully validated domains. The specification requires that region-aware MBM and MBA be enabled together and consistently across all RMDDs.
--- arch/x86/kernel/cpu/resctrl/erdt.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
arch/x86/kernel/cpu/resctrl/erdt.c +21 -0
@@ -29,6 +29,8 @@ static u32 erdt_max_clos;#define MARC_SUPPORTED_INDEX_FN 1#define RMDD_FLAG_CPU_L3_DOMAIN BIT(0) +#define RDT_CTRL_LEGACY_MODE BIT(2) +#define MARC_FLAG_OPT BIT(0)#define MARC_FLAG_MIN BIT(1)#define MARC_FLAG_MAX BIT(2)
@@ -406,8 +408,23 @@ static void erdt_iounmap_domain(struct erdt_domain_info *domain) } } +static void region_aware_enable (void __iomem *addr,bool enable) +{ +u64 rdt_ctrl =readq (addr); + +if (enable) + rdt_ctrl &= ~RDT_CTRL_LEGACY_MODE ; +else + rdt_ctrl |=RDT_CTRL_LEGACY_MODE ; + +writeq (rdt_ctrl, addr); +} +static void cleanup_one_domain (struct erdt_domain_info *d) { +if (d->base[ERDT_MMIO_RMDD_CREG ]) +region_aware_enable (d->base[ERDT_MMIO_RMDD_CREG ], false); +erdt_iounmap_domain (d);kfree (d->marc_buf);kfree (d->cmrc);
@@ -804,6 +821,7 @@ static __init int enumerate_erdt_table(struct acpi_table_header *table_hdr) {struct acpi_table_erdt *erdt = (struct acpi_table_erdt *)table_hdr;struct acpi_subtbl_hdr_16 *subtbl; +struct erdt_domain_info *d;void *table_end;if (erdt->header.revision !=ERDT_VALID_VERSION ) {
@@ -831,6 +849,9 @@ static __init int enumerate_erdt_table(struct acpi_table_header *table_hdr)if (list_empty (&domain_info_list)) goto cleanup; +list_for_each_entry (d, &domain_info_list, entry) +region_aware_enable (d->base[ERDT_MMIO_RMDD_CREG ], true); + erdt_max_clos = erdt->max_clos; erdt_enabled = true;
Signature · 1 line
2.43.0
RFC PATCH 28/31 x86/resctrl: Emulate the legacy MBA controller via the region MAX controls
In reply to Chen YuOn ERDT platforms the legacy MBA "default" controller does have hardware of its own, but the region-based MAX controllers are the preferred way to throttle memory bandwidth. Rather than programming the legacy MSR, realize the default controller's effect through the MAX controller of every memory region, which keeps a single consistent throttling mechanism in use. Link the default controller to the MAX controller of every available region via resctrl_ctrl::emul in erdt_get_mem_config() so they are associated once the controllers are created.
Route the arch control writes accordingly. When a control is emulated (ctrl->emul is not empty), do not program the legacy MSR; instead translate the configured value into each emulation controller's units and program that controller's hardware.
--- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 75 ++++++++++++++++++++--- arch/x86/kernel/cpu/resctrl/erdt.c | 18 ++++++ include/linux/resctrl.h | 3 + 3 files changed, 88 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/resctrl/ctrlmondata.c +67 -8
@@ -28,6 +28,48 @@ static void ctrl_hw_update(struct resctrl_ctrl *ctrl, struct rdt_ctrl_domain *d,smp_call_function_any (&d->hdr.cpu_mask, rdt_ctrl_update, m,1 ); } +static u32 emul_translate_val (struct resctrl_ctrl *src,struct resctrl_ctrl *dst, +u32 val) +{ +return val * dst->scalar.max_bw / src->scalar.max_bw; +} + +static void _resctrl_arch_update_emul (struct rdt_resource *r, +struct resctrl_ctrl *ctrl, +struct rdt_ctrl_domain *d,u32 low,u32 high) +{ +struct rdt_hw_ctrl_domain *emul_hw_dom; +struct rdt_hw_ctrl_domain *hw_dom; +struct rdt_ctrl_domain *emul_d; +struct resctrl_ctrl *emul; +struct rdt_domain_hdr *hdr; +struct hw_param emul_param; +unsigned int i; + + hw_dom =resctrl_to_arch_ctrl_dom (d); + +for_each_emul_ctrl (emul, ctrl) { + hdr =resctrl_find_domain (&emul->domains, d->hdr.id,NULL ); +if (!hdr) +continue ; + + emul_d = container_of(hdr, struct rdt_ctrl_domain, hdr); + emul_hw_dom =resctrl_to_arch_ctrl_dom (emul_d); + +for (i = low; i < high; i++) + emul_hw_dom->ctrl_val[i] =emul_translate_val (ctrl, emul, + hw_dom->ctrl_val[i]); + + emul_param.res = r; + emul_param.ctrl = emul; + emul_param.dom = emul_d; + emul_param.low = low; + emul_param.high = high; + +ctrl_hw_update (emul, emul_d, &emul_param); + } +} +int resctrl_arch_update_one (struct rdt_resource *r,struct resctrl_ctrl *ctrl,struct rdt_ctrl_domain *d,u32 closid,enum resctrl_conf_type t,u32 cfg_val)
@@ -42,12 +84,20 @@ int resctrl_arch_update_one(struct rdt_resource *r, struct resctrl_ctrl *ctrl, hw_dom->ctrl_val[idx] = cfg_val; - hw_param.res = r; - hw_param.ctrl = ctrl; - hw_param.dom = d; - hw_param.low = idx; - hw_param.high = idx +1 ; - hw_ctrl->hw_update (&hw_param); +/* +* When this control is emulated, program the emulation controllers' +* hardware instead. +*/ +if (!list_empty (&ctrl->emul)) { +_resctrl_arch_update_emul (r, ctrl, d, idx, idx +1 ); + }else { + hw_param.res = r; + hw_param.ctrl = ctrl; + hw_param.dom = d; + hw_param.low = idx; + hw_param.high = idx +1 ; + hw_ctrl->hw_update (&hw_param); + }return 0 ; }
@@ -89,8 +139,17 @@ static void _resctrl_arch_update_domains(struct rdt_resource *r, hw_param.high =max (hw_param.high, idx +1 ); } } -if (hw_param.res) -ctrl_hw_update (ctrl, d, &hw_param); +if (hw_param.res) { +/* +* When this control is emulated, program the emulation +* controllers' hardware instead. +*/ +if (!list_empty (&ctrl->emul)) +_resctrl_arch_update_emul (r, ctrl, d, + hw_param.low, hw_param.high); +else +ctrl_hw_update (ctrl, d, &hw_param); + } } }
arch/x86/kernel/cpu/resctrl/erdt.c +18 -0
@@ -325,6 +325,7 @@ static void marc_hw_update(struct hw_param *m)__init bool erdt_get_mem_config (struct rdt_resource *r) {struct rdt_hw_resource *hw_res =resctrl_to_arch_res (r); +struct resctrl_ctrl *def_ctrl =NULL , *ctrl;struct resctrl_hw_ctrl *hw_ctrl;struct erdt_domain_info *d;struct acpi_erdt_marc *marc;
@@ -343,6 +344,13 @@ __init bool erdt_get_mem_config(struct rdt_resource *r) max_regions =acpi_mrrm_max_mem_region (); hw_res->num_closid =max (hw_res->num_closid, erdt_max_clos +1 ); +for_each_resource_ctrl (ctrl, r) { +if (ctrl->name ==RESCTRL_CTRL_NAME_DEF ) { + def_ctrl = ctrl; +break ; + } + } +for (region =0 ; region < max_regions; region++) {for (type =RESCTRL_CTRL_REGION_TYPE_OPT ; type <RESCTRL_CTRL_REGION_NR_CTRLS ; type++) {if (type ==RESCTRL_CTRL_REGION_TYPE_OPT && !(marc->flags &MARC_FLAG_OPT ))
@@ -374,6 +382,16 @@ __init bool erdt_get_mem_config(struct rdt_resource *r)/* MARC controls live in MMIO, program from any CPU. */ hw_ctrl->r_ctrl.flags |=RESCTRL_CTRL_FLAG_ANY_CPU ;list_add_tail (&hw_ctrl->r_ctrl.entry, &r->controls); + +/* +* Throttle memory bandwidth through the region MAX +* controls rather than the legacy MBA MSR, so let the +* MAX control of every region emulate the legacy MBA +* control. +*/ +if (def_ctrl && type ==RESCTRL_CTRL_REGION_TYPE_MAX ) +list_add_tail (&hw_ctrl->r_ctrl.emul_entry, + &def_ctrl->emul); } }
include/linux/resctrl.h +3 -0
@@ -51,6 +51,9 @@ int proc_resctrl_show(struct seq_file *m,#define for_each_resource_ctrl (ctrl, r) \ list_for_each_entry(ctrl, &r->controls, entry) +#define for_each_emul_ctrl (emul, ctrl) \ + list_for_each_entry(emul, &ctrl->emul, emul_entry) +enum resctrl_res_level {RDT_RESOURCE_L3 ,RDT_RESOURCE_L2 ,
Signature · 1 line
2.43.0
RFC PATCH 29/31 fs/resctrl: Expose emulation controllers in a resource_schemata subdir
In reply to Chen YuThe legacy MBA default controller is emulated by other controls (for ERDT, "MB" is emulated by the MAX control of every memory region).
When a control is emulated (resctrl_ctrl::emul is not empty), do not expose the control's own properties. Keep the control's own resource_schemata directory and, underneath it, create a sub-directory named after each emulation controller that holds that emulation controller's properties. For example:
[root@ MB]# tree resource_schemata/
resource_schemata/
├── MB │ ├── MB_REGION0_MAX │ │ ├── max │ │ ├── min │ │ ├── resolution │ │ ├── scale │ │ ├── scope │ │ ├── tolerance │ │ ├── type │ │ └── unit │ └── MB_REGION1_MAX │ ├── max │ ├── min │ ├── resolution │ ├── scale │ ├── scope │ ├── tolerance │ ├── type │ └── unit
Controls without an emulation controller keep exposing their own properties directly.
--- fs/resctrl/rdtgroup.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-)
fs/resctrl/rdtgroup.c +27 -4
@@ -2741,13 +2741,36 @@ static int resctrl_mkdir_schemata_dir(struct kernfs_node *kn,if (IS_ERR (kn_ctrl))return PTR_ERR (kn_ctrl); - ret =rdtgroup_kn_set_ugid (kn_subdir); + ret =rdtgroup_kn_set_ugid (kn_ctrl);if (ret)return ret; - ret =resctrl_add_ctrl_files (kn_ctrl, ctrl); -if (ret) -return ret; +if (!list_empty (&ctrl->emul)) { +struct kernfs_node *kn_emul; +struct resctrl_ctrl *emul; + +for_each_emul_ctrl (emul, ctrl) { +snprintf (ctrl_full_name,sizeof (ctrl_full_name),"%s_%s" , + f->name,resctrl_ctrl_name_str (emul->name)); + + kn_emul =kernfs_create_dir (kn_ctrl, ctrl_full_name, + kn_ctrl->mode, emul); +if (IS_ERR (kn_emul)) +return PTR_ERR (kn_emul); + + ret =rdtgroup_kn_set_ugid (kn_emul); +if (ret) +return ret; + + ret =resctrl_add_ctrl_files (kn_emul, emul); +if (ret) +return ret; + } + }else { + ret =resctrl_add_ctrl_files (kn_ctrl, ctrl); +if (ret) +return ret; + } }kernfs_activate (kn_subdir);
Signature · 1 line
2.43.0
RFC PATCH 30/31 fs/resctrl: Fix excessive padding in schemata output
In reply to Chen YuWhen a resource has several controls (such as the region-aware MBA controls), the schemata and size files pad every line with a large amount of leading whitespace and the control names no longer line up into a table.
Compute the column width from the longest single control name and format the full control name within that width so the output is aligned again.
--- fs/resctrl/ctrlmondata.c | 19 +++++++++++-------- fs/resctrl/rdtgroup.c | 11 ++++++----- 2 files changed, 17 insertions(+), 13 deletions(-)
fs/resctrl/ctrlmondata.c +11 -8
@@ -405,16 +405,16 @@ static struct resctrl_ctrl *resctrl_resource_ctrl_get(struct rdt_resource *r,size_t resctrl_resource_ctrl_max_len (struct rdt_resource *r) {struct resctrl_ctrl *ctrl; -size_t total =0 ; +size_t max =0 ;size_t len;for_each_resource_ctrl (ctrl,r) { len =strlen (resctrl_ctrl_name_str (ctrl->name));if (len) - total +=1 + len; + max =max_t (size_t, max,1 + len); } -return total; +return max; } static int rdtgroup_parse_ctrl(char *ctrlname, char *tok,
@@ -522,6 +522,7 @@ static void show_doms(struct seq_file *s, struct rdt_resource_final *f, bool print_ctrl, int closid, struct resctrl_ctrl *ctrl) { struct rdt_resource *r = f->res; +char ctrl_full_name[20 ];struct rdt_ctrl_domain *dom;bool sep = false;u32 ctrl_val;
@@ -529,11 +530,13 @@ static void show_doms(struct seq_file *s, struct rdt_resource_final *f,/* Walking r->domains, ensure it can't race with cpuhp */ lockdep_assert_cpus_held (); -if (print_ctrl) -seq_printf (s,"%*s%s%s:" , max_name_width, f->name, -resctrl_ctrl_is_default (ctrl) ?"" :"_" , -resctrl_ctrl_is_default (ctrl) ? -"" :resctrl_ctrl_name_str (ctrl->name)); +if (print_ctrl) { +snprintf (ctrl_full_name,sizeof (ctrl_full_name),"%s%s%s" , f->name, +resctrl_ctrl_is_default (ctrl) ?"" :"_" , +resctrl_ctrl_is_default (ctrl) ? +"" :resctrl_ctrl_name_str (ctrl->name)); +seq_printf (s,"%*s:" , max_name_width, ctrl_full_name); + }list_for_each_entry (dom, &ctrl->domains, hdr.list) {if (sep)seq_puts (s,";" );
fs/resctrl/rdtgroup.c +6 -5
@@ -1653,6 +1653,7 @@ static int rdtgroup_size_show(struct kernfs_open_file *of,struct resctrl_ctrl *ctrl;struct rdtgroup *rdtgrp;struct rdt_resource *r; +char ctrl_full_name[20 ];unsigned int size;u32 ctrl_val;int ret =0 ;
@@ -1693,11 +1694,11 @@ static int rdtgroup_size_show(struct kernfs_open_file *of, type = f->conf_type;for_each_resource_ctrl (ctrl, r) { sep = false; -seq_printf (s,"%*s" , max_name_width, f->name); -if (!resctrl_ctrl_is_default (ctrl)) -seq_printf (s,"_%s:" ,resctrl_ctrl_name_str (ctrl->name)); -else -seq_putc (s,':' ); +snprintf (ctrl_full_name,sizeof (ctrl_full_name),"%s%s%s" , f->name, +resctrl_ctrl_is_default (ctrl) ?"" :"_" , +resctrl_ctrl_is_default (ctrl) ? +"" :resctrl_ctrl_name_str (ctrl->name)); +seq_printf (s,"%*s:" , max_name_width, ctrl_full_name);list_for_each_entry (d, &ctrl->domains, hdr.list) {if (sep)seq_putc (s,';' );
Signature · 1 line
2.43.0
RFC PATCH 31/31 x86,fs/resctrl: Update Documentation for region aware RDT
In reply to Chen YuSupport region based MBM display and MBA schemata, update the document accordingly.
--- Documentation/filesystems/resctrl.rst | 110 ++++++++++++++++++++++++++ 1 file changed, 110 insertions(+)
Documentation/filesystems/resctrl.rst +110 -0
@@ -28,6 +28,7 @@ SMBA (Slow Memory Bandwidth Allocation) "" BMEC (Bandwidth Monitoring Event Configuration) "" ABMC (Assignable Bandwidth Monitoring Counters) "" SDCIAE (Smart Data Cache Injection Allocation Enforcement) "" +RMBA (Region Aware Memory Bandwidth Allocation) "" =============================================================== ================================ Historically, new features were made visible by default in /proc/cpuinfo. This
@@ -651,6 +652,11 @@ When monitoring is enabled all MON groups will also contain: "core" of the CPU (arithmetic units, TLB, L1 and L2 caches, etc.). They do not include L3 cache, memory, I/O devices etc. + If region aware RDT is enabled, several region related files are created. + For example, if there are 4 regions, the corresponding files are + mbm_region0_bytes, mbm_region1_bytes, mbm_region2_bytes and + mbm_region3_bytes. + All other events report decimal integer values. In a MON group these files provide a read out of the current value of
@@ -993,6 +999,56 @@ is formatted as: SMBA:<cache_id0>=bandwidth0;<cache_id1>=bandwidth1;... +Region Aware Memory Bandwidth Allocation and Monitor +---------------------------------------------------- +Intel hardware supports Region-Aware Memory Bandwidth Allocation (MBA) +and Region-Aware Memory Bandwidth Monitoring (MBM). With Region-Aware +MBA, independent bandwidth control (throttling) of L3 domain bandwidth +to multiple regions is supported, enabling users to dynamically rebalance +bandwidth control limits across different memory regions, each of which +may have distinct bandwidth, latency, and capacity characteristics. +Region-Aware MBM includes the capability to independently track multiple +domains that are simultaneously accessing several memory regions. These +memory regions correspond to different levels of memory tiers, such as +directly attached memory (Tier 1), CXL-attached memory (Tier 2), and +CXL accelerator devices with attached memory. Note, the region ID is +per socket scope. Intel platform supports up to 4 regions for now. + +Suppose there are 2 regions in each socket: + + +------------------------+------------------------+ + | +--------+ +--------+ | +--------+ +--------+ | + | | L3_00 | | L3_01 | | | L3_02 | | L3_03 | | + | +--------+ +--------+ | +--------+ +--------+ | + | +-------------------+ | +-------------------+ | + | | IMC1 | | | IMC2 | | + | +-------------------+ | +-------------------+ | + | +-------+ +-------+ | +-------+ +-------+ | + | | DDR1 | | CXL1 | | | DDR2 | | CXL2 | | + | +-------+ +-------+ | +-------+ +-------+ | + | socket0 | socket1 | + +------------------------+------------------------+ + +In above graph, from the perspective of a CPU associated with +the L3_00 domain, Region 0 typically represents the local memory +region (DDR1), while Region 2 typically represents the remote +memory region (DDR2). Similarly, for CPUs attached to L3_00, +Region 1 represents the local memory region (CXL1), and Region 3 +represents the remote memory region (CXL2). Here the terms "local" +and "remote" here are defined at the socket level. + +Take the region aware MBM for example. For the L3_00 domain, the +memory bandwidth of Region 0 refers to the data transferred when +the L3 miss occurs in L3_00 and the data is refilled from DDR1 - +note, data refilled from L3_01 to L3_00 is not counted in. The +bandwidth of Region 2, by contrast, refers to the data refilled +from DDR2. Similarly, the same calculation logic applies to +Region 1 (CXL1) and Region 3 (CXL2). + +For Region-Aware MBA, when setting values for Region 0 on CPUs +attached to L3_00, this configuration controls the traffic generated +when data is transferred between cores and L3_00 targeting DDR1. + Reading/writing the schemata file --------------------------------- Reading the schemata file will show the state of all resources
@@ -1048,6 +1104,60 @@ For example, to allocate 8GB/s limit on the first cache id: MB:0=2048;1=2048;2=2048;3=2048 L3:0=ffff;1=ffff;2=ffff;3=ffff +Reading/writing the schemata file (on Intel systems) with region MBA feature +---------------------------------------------------------------------------- +The schemata file provides fine grained control over each memory region. +Every region has its own set of controls, and each of them appears as a +separate line named "MB_REGION<n>_<type>", where <n> is the region number +and <type> is one of OPT, MIN or MAX. Only the control types the hardware +reports in the MARC ACPI sub-table are present. + +The legacy "MB" control is emulated by the MAX control of every region, so +writing "MB" throttles all regions at once, while writing an individual +"MB_REGION<n>_MAX" line throttles just that region. + +Reading and writing the schemata file is illustrated below for a platform +with 2 memory regions. For example, to adjust the tier1 local memory +bandwidth (usually the DDR): + +:: + + # cat schemata + MB:0=100;1=100;2=100;3=100 + MB_REGION0_OPT:0=511;1=511;2=511;3=511 + MB_REGION0_MIN:0=511;1=511;2=511;3=511 + MB_REGION0_MAX:0=511;1=511;2=511;3=511 + MB_REGION1_OPT:0=511;1=511;2=511;3=511 + MB_REGION1_MIN:0=511;1=511;2=511;3=511 + MB_REGION1_MAX:0=511;1=511;2=511;3=511 + L3:0=3ff;1=3ff;2=3ff;3=3ff + + # echo "MB_REGION0_MAX:1=200" > schemata + # cat schemata + MB:0=100;1=100;2=100;3=100 + MB_REGION0_OPT:0=511;1=511;2=511;3=511 + MB_REGION0_MIN:0=511;1=511;2=511;3=511 + MB_REGION0_MAX:0=511;1=200;2=511;3=511 + MB_REGION1_OPT:0=511;1=511;2=511;3=511 + MB_REGION1_MIN:0=511;1=511;2=511;3=511 + MB_REGION1_MAX:0=511;1=511;2=511;3=511 + L3:0=3ff;1=3ff;2=3ff;3=3ff + +The value range of a region control is reported by the hardware and is +available under the resource_schemata directory of that control, together +with the rest of its properties. + +Users can obtain tiering information via sysfs, specifically +through the path /sys/bus/memory_tier/devices/memory_tier.<id>/nodelist. +The user can query the /sys/firmware/acpi/memory_ranges to find +the corresponding node id for a specific region number. + +Once users have the corresponding node numbers, they can use the command +numactl -H to retrieve additional node-related information if they need +to further understand what each node represents. This information includes +details such as the CPUs associated with the node (if any), the memory size +of each node, and the distance between nodes. + Cache Pseudo-Locking ==================== CAT enables a user to specify the amount of cache space that an
Signature · 1 line
2.43.0
Luck, Tony
maintainerRe: [RFC PATCH 05/31] x86/resctrl: Parse ACPI CMRC table
In reply to Chen YuQuoted 4 lines · Chen Yu
The CMRC (Cache Monitoring Registers for CPU Agents Description) sub-table of ERDT describes the MMIO registers used to read cache monitoring counters (e.g. LLC occupancy) for an RMD.
Quoted 8 lines
Parse each CMRC sub-table, ioremap its register window, and save a copy of the CMRC table in the corresponding ERDT domain entry so that later monitoring code can read the counters via MMIO.
Suggested-by Tony Luck <[email protected]>Tested-by Hongyu Ning <[email protected]>Reviewed-by Thomas Gleixner <[email protected]>Signed-off-by Chen Yu <[email protected]>
See the "Ordering of commit tags" section in Documentation/process/maintainer-tip.rst
Quoted 32 lines
--- arch/x86/include/asm/resctrl.h | 2 + arch/x86/kernel/cpu/resctrl/erdt.c | 57 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 6 ++- 3 files changed, 64 insertions(+), 1 deletion(-)arch/x86/include/asm/resctrl.h +2
-0@@ -49,6 +49,8 @@ DECLARE_STATIC_KEY_FALSE(rdt_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_alloc_enable_key);DECLARE_STATIC_KEY_FALSE (rdt_mon_enable_key); +int erdt_get_scale (void ); +static inline bool resctrl_arch_alloc_capable (void ) {return rdt_alloc_capable;arch/x86/kernel/cpu/resctrl/erdt.c +1
-0@@ -23,6 +23,7 @@ static LIST_HEAD(domain_info_list);static bool erdt_enabled;#define ERDT_VALID_VERSION 1 +#define CMRC_SUPPORTED_INDEX_FN 1#define RMDD_FLAG_CPU_L3_DOMAIN BIT(0)/* Bitmask of valid sub-tables found in the first RMDD, used to ensure all RMDDs match. */
arch/x86/kernel/cpu/resctrl/erdt.c static u16 first_rmdd_domain_id; · line 38
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/erdt.c +2
-0@@ -33,11 +34,19 @@ static u16 first_rmdd_domain_id;static int erdt_max_rmid; +/* Scale to bytes for the monitoring counters when ERDT is enabled. */ +static int erdt_scale;
Single value for scale? See below.
Quoted 49 lines
arch/x86/kernel/cpu/resctrl/erdt.c +30
-0+int erdt_get_max_rmid (void ) {return erdt_max_rmid; } +int erdt_get_scale (void ) +{ +return erdt_scale; +} +static void __iomem *erdt_ioremap (phys_addr_t base,u32 num_pages,const char *desc) {void __iomem *addr;@@ -67,6 +76,7 @@ static void erdt_iounmap_domain(struct erdt_domain_info *domain)static void cleanup_one_domain (struct erdt_domain_info *d) {erdt_iounmap_domain (d); +kfree (d->cmrc);kfree (d); }@@ -100,6 +110,43 @@ static __init int cacd_init(struct acpi_subtbl_hdr_16 *subtbl,return 0 ; } +static __init int cmrc_init (struct acpi_subtbl_hdr_16 *subtbl, +struct erdt_domain_info *domain_info) +{ +struct acpi_erdt_cmrc *cmrc = (struct acpi_erdt_cmrc *)subtbl; + +if (cmrc->header.length <sizeof (*cmrc)) { +pr_warn (FW_BUG "Truncated CMRC subtable\n" ); +return -EIO ; + } + +if (cmrc->index_fn !=CMRC_SUPPORTED_INDEX_FN ) { +pr_info ("Unsupported CMRC index function %u\n" , cmrc->index_fn); +return -EIO ; + } + +if (!cmrc->clump_size) { +pr_warn (FW_BUG "CMRC clump_size is zero\n" ); +return -EIO ; + } + + domain_info->base[ERDT_MMIO_CMRC_BASE ] = +erdt_ioremap (cmrc->cmt_reg_base, cmrc->cmt_reg_size,"CMRC base" ); +if (!domain_info->base[ERDT_MMIO_CMRC_BASE ])
arch/x86/kernel/cpu/resctrl/erdt.c static __init int cacd_init(struct acpi_subtbl_hdr_16 *subtbl, · line 145
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/erdt.c +10
-0+return -EIO ; + + domain_info->cmrc =kmemdup (cmrc, cmrc->header.length,GFP_KERNEL ); +if (!domain_info->cmrc) { +iounmap (domain_info->base[ERDT_MMIO_CMRC_BASE ]); + domain_info->base[ERDT_MMIO_CMRC_BASE ] =NULL ; +return -ENOMEM ; + } + + erdt_scale =max_t (int, erdt_scale, cmrc->up_scale);
I'd expect that on a machine all CMRC tables would report the same up_scale factor. But RDT architecture allows them to be different. Two ways to handle this:
1) Check that they are all the same. Complain if they are different and don't enable cache occupancy events.
2) Save the value for each domain and use that value to upscale when reporting to user.
Picking the max value doesn't feel like a good answer.
arch/x86/kernel/cpu/resctrl/erdt.c static __init int cacd_init(struct acpi_subtbl_hdr_16 *subtbl, · line 148
Quoted 3 lines
arch/x86/kernel/cpu/resctrl/erdt.c +3
-0+ +return 0 ; +}
-Tony
Luck, Tony
maintainerRe: [RFC PATCH 05/31] x86/resctrl: Parse ACPI CMRC table
Quoted 7 lines · Tony
arch/x86/kernel/cpu/resctrl/erdt.c static u16 first_rmdd_domain_id; · line 38
Quoted 3 lines · Chen Yu
arch/x86/kernel/cpu/resctrl/erdt.c +2
-0+/* Scale to bytes for the monitoring counters when ERDT is enabled. */ +static int erdt_scale;Single value for scale? See below.
Now I've read ahead to patch 9 and see how this is used to set resctrl_rmid_realloc_threshold. Needs better comments here and for above erdt_get_scale()
Quoted 18 lines
arch/x86/kernel/cpu/resctrl/erdt.c static u16 first_rmdd_domain_id;
Quoted 13 lines
arch/x86/kernel/cpu/resctrl/erdt.c +7
-0+int erdt_get_max_rmid (void ) {return erdt_max_rmid; } +int erdt_get_scale (void ) +{ +return erdt_scale; +} + + erdt_scale =max_t (int, erdt_scale, cmrc->up_scale);I'd expect that on a machine all CMRC tables would report the same
up_scalefactor. But RDT architecture allows them to be different. Two ways to handle this:
Quoted 5 lines
1) Check that they are all the same. Complain if they are different and don't enable cache occupancy events.
2) Save the value for each domain and use that value to upscale when reporting to user.
I see in patch 9 that you do use this option "2" with "cmrc->up_scale".
Quoted 1 line
Picking the max value doesn't feel like a good answer.
Given how this is actually used, seems like the best option.
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/erdt.c static __init int cacd_init(struct acpi_subtbl_hdr_16 *subtbl, · line 148
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/erdt.c +3
-0+ +return 0 ; +}
-Tony
Luck, Tony
maintainerRe: [RFC PATCH 25/31] x86/resctrl: Introduce region-based MBA write implementation on MMIO space
In reply to Chen YuQuoted 65 lines · Chen Yu
Implement the
marc_hw_update()callback to program per-region MBA bandwidth values via MMIO.The MARC register layout packs 4 regions into each 64-bit register, with a 9-bit bandwidth field per region at (region % 4) * 16 bits. The MMIO address for a given CLOSID is calculated using the MARC index function 1:
addr = base + Floor(Region / 4) * 512 + CLOS * 8
Each controller handles a single (region, type) pair, but a write always touches the whole 64-bit register. The callback therefore does a
read-modify-write: it changes only its own region's field and writes every other bit back exactly as it was read, so the reserved bits keep their original values. A shadow copy of the register (marc_buf) holds the value last written, which spares the read on later updates.Signed-off-by Chen Yu <[email protected]>--- arch/x86/kernel/cpu/resctrl/erdt.c | 52 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/resctrl/internal.h | 4 ++ 2 files changed, 56 insertions(+)arch/x86/kernel/cpu/resctrl/erdt.c +34
-0@@ -275,6 +275,49 @@ struct erdt_domain_info *erdt_find_domain_info(int cpu)static void marc_hw_update (struct hw_param *m) { +struct rdt_hw_ctrl_domain *hw_dom =resctrl_to_arch_ctrl_dom (m->dom); +struct resctrl_hw_ctrl *hw_ctrl =resctrl_to_arch_ctrl (m->ctrl); +enum resctrl_ctrl_name name = hw_ctrl->r_ctrl.name; +unsigned int offset, region, type, region_offset_bits; +struct erdt_domain_info *d = hw_dom->d_info; +enum erdt_mmio_type mmio_type; +void __iomem *addr; +int closid_idx; +unsigned int i; +u64 val; + +if (!d || !d->marc) +return ; + + offset = name -RESCTRL_CTRL_NAME_REGION0_OPT ; + region = offset /RESCTRL_CTRL_REGION_NR_CTRLS ; + type = offset %RESCTRL_CTRL_REGION_NR_CTRLS ; + mmio_type =ERDT_MMIO_MARC_OPT + type; + region_offset_bits = (region %4 ) *16 ; + +if (d->marc_buf_type != mmio_type) { +memset (d->marc_buf,0 , + d->marc->mba_reg_size *512 *sizeof (u64)); + d->marc_buf_type = mmio_type; + } + +for (i = m->low; i < m->high; i++) { + closid_idx = (region /4 ) *64 + i; + addr = d->base[mmio_type] + closid_idx *8 ; + +/* The cached value retains the reserved bits to be preserved. */ + val = d->marc_buf[closid_idx]; +if (!val) + val =readq (addr);
arch/x86/kernel/cpu/resctrl/erdt.c struct erdt_domain_info *erdt_find_domain_info(int cpu) · line 321
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/erdt.c +9
-0+ +if (WARN_ON_ONCE (!val)) +return ; + + val &= ~(0x1ffULL << region_offset_bits); + val |= (u64)(hw_dom->ctrl_val[i] &0x1ff ) << region_offset_bits; + d->marc_buf[closid_idx] = val; +writeq (val, addr); + } }
There are lots of opinions about bit fields. I'm a fan for h/w registers accessed in architecture specific code.
I think the version below is easier to read than all the inline masks and shifts.
-Tony
union bw_ctrl {
u64 reg;
struct {
u16 val : 9;
u16 rsvd : 7;
} regions[4];
};static void marc_hw_update(struct hw_param *m)
{
struct rdt_hw_ctrl_domain *hw_dom = resctrl_to_arch_ctrl_dom(m->dom);
struct resctrl_hw_ctrl *hw_ctrl = resctrl_to_arch_ctrl(m->ctrl);
enum resctrl_ctrl_name name = hw_ctrl->r_ctrl.name;
struct erdt_domain_info *d = hw_dom->d_info;
unsigned int offset, region, type;
enum erdt_mmio_type mmio_type;
union bw_ctrl mmio_ctrl;
void __iomem *addr;
int closid_idx;
unsigned int i;
if (!d || !d->marc)
return;
offset = name - RESCTRL_CTRL_NAME_REGION0_OPT;
region = offset / RESCTRL_CTRL_REGION_NR_CTRLS;
type = offset % RESCTRL_CTRL_REGION_NR_CTRLS;
mmio_type = ERDT_MMIO_MARC_OPT + type;
if (d->marc_buf_type != mmio_type) {
memset(d->marc_buf, 0,
d->marc->mba_reg_size * 512 * sizeof(u64));
d->marc_buf_type = mmio_type;
}
for (i = m->low; i < m->high; i++) {
closid_idx = (region / 4) * 64 + i;
addr = d->base[mmio_type] + closid_idx * 8;
/* The cached value retains the reserved bits to be preserved. */
mmio_ctrl.reg = d->marc_buf[closid_idx];
if (!mmio_ctrl.reg)
mmio_ctrl.reg = readq(addr);
if (WARN_ON_ONCE(!mmio_ctrl.reg))
return;
mmio_ctrl.regions[region].val = hw_dom->ctrl_val[i];
d->marc_buf[closid_idx] = mmio_ctrl.reg;
writeq(mmio_ctrl.reg, addr);
}
}Re: [RFC PATCH 05/31] x86/resctrl: Parse ACPI CMRC table
In reply to Luck, TonyHi Tony,
Quoted 4 lines · Tony
Quoted 3 lines · Tony
Quoted 1 line · Chen Yu
Quoted 8 lines
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/erdt.c static u16 first_rmdd_domain_id; · line 38
Quoted 2 lines
arch/x86/kernel/cpu/resctrl/erdt.c +2
-0+/* Scale to bytes for the monitoring counters when ERDT is enabled. */ +static int erdt_scale;Single value for scale? See below.
Now I've read ahead to patch 9 and see how this is used to set
resctrl_rmid_realloc_threshold. Needs better comments here and for aboveerdt_get_scale()
OK, will add comment here.
resctrl_rmid_realloc_threshold is a single global value, and resctrl_arch_round_mon_val() takes no domain argument, so a single scale has to be derived from the per-domain cmrc->up_scale. max() was chosen because the rounding is a floor: a larger scale yields a slightly lower threshold, i.e. an RMID has to drop to a slightly lower occupancy before it is reused.
Quoted 21 lines
Quoted 21 lines
arch/x86/kernel/cpu/resctrl/erdt.c static u16 first_rmdd_domain_id;
Quoted 13 lines
arch/x86/kernel/cpu/resctrl/erdt.c +7
-0+int erdt_get_max_rmid (void ) {return erdt_max_rmid; } +int erdt_get_scale (void ) +{ +return erdt_scale; +} + + erdt_scale =max_t (int, erdt_scale, cmrc->up_scale);I'd expect that on a machine all CMRC tables would report the same
up_scalefactor. But RDT architecture allows them to be different. Two ways to handle this:1) Check that they are all the same. Complain if they are different and don't enable cache occupancy events.
Quoted 8 lines
Quoted 2 lines
2) Save the value for each domain and use that value to upscale when reporting to user.
I see in patch 9 that you do use this option "2" with "
cmrc->up_scale".Quoted 1 line
Picking the max value doesn't feel like a good answer.
Given how this is actually used, seems like the best option.
OK, the erdt_mon_read() reads via the per domain cmrc->up_scale.
thanks,
Chenyu
Quoted 6 lines
Quoted 5 lines
arch/x86/kernel/cpu/resctrl/erdt.c static __init int cacd_init(struct acpi_subtbl_hdr_16 *subtbl, · line 148
Quoted 4 lines
arch/x86/kernel/cpu/resctrl/erdt.c +3
-0+ +return 0 ; +}-Tony
Re: [RFC PATCH 25/31] x86/resctrl: Introduce region-based MBA write implementation on MMIO space
In reply to Luck, TonyQuoted 22 lines · Tony
Quoted 8 lines
arch/x86/kernel/cpu/resctrl/erdt.c +8
-0+for (i = m->low; i < m->high; i++) { + closid_idx = (region /4 ) *64 + i; + addr = d->base[mmio_type] + closid_idx *8 ; + +/* The cached value retains the reserved bits to be preserved. */ + val = d->marc_buf[closid_idx]; +if (!val) + val =readq (addr);arch/x86/kernel/cpu/resctrl/erdt.c struct erdt_domain_info *erdt_find_domain_info(int cpu) · line 321
Quoted 10 lines
arch/x86/kernel/cpu/resctrl/erdt.c +9
-0+ +if (WARN_ON_ONCE (!val)) +return ; + + val &= ~(0x1ffULL << region_offset_bits); + val |= (u64)(hw_dom->ctrl_val[i] &0x1ff ) << region_offset_bits; + d->marc_buf[closid_idx] = val; +writeq (val, addr); + } }There are lots of opinions about bit fields. I'm a fan for h/w registers accessed in architecture specific code.
Quoted 3 lines
I think the version below is easier to read than all the inline masks and shifts.
Got it, learned a lot, I will revise it in this direction.
thanks,
Chenyu